Results 1 to 17 of 17

Thread: What causes servers to lock up/max out/stop functioning

  1. #1
    Ace
    Join Date
    May 2015
    Location
    Kansas City, Missouri area
    Posts
    515
    Post Thanks / Like
    Total Downloaded
    130.02 MB

    What causes servers to lock up/max out/stop functioning

    Some of you may know that we have had issues over the past couple of years with the TWC servers where the server just gets sort of maxed out and stops functioning. Suddenly it will take 10-15 minutes to enter the server, any communication to the server might be quite slow, and so on.

    The strange thing is, it usually happens when I make some rather small changes that don't seem like they would cause a dramatic effect at all.

    I've been gradually getting my head around what must be the root cause of this, and another incident this weekend has completely confirmed it in my mind.

    So, briefly, here are the conclusions:

    • It's not the number of aircraft per se - neither breathers nor ai nor the combination of the two. It is certainly possible to get overloaded with aircraft but the symptoms are different and the failure is more graceful and gradual than sudden. FYI a powerful server can handle 150-200 aircraft in-game quite well, 200-250 maybe but not optimal, and then 250+ is where things get a lot more dicey. But if you run into the problem I outline below you can remove 50 aircraft with no noticeable effect and say 100 with barely noticeable effect.
    • Ground stationaries in general don't seem to have much effect - particularly if they're "just scenery" that doesn't interact with players, for example by shooting at them. It may take a little longer to get these loaded on entering the server but the penalty for say tens of thousands of statics vs just a couple of hundred is remarkably small. When I've encountered this problem I've tried removing many thousands of ground stationaries with no noticeable effect at all.
    • When you have the problem I'm describing, you can monitor it on the server by simply watching server CPU load. The problem occurs when the server CPU is maxed out. With multi-core CPUs of course you won't see 100% CPU but you'll see one core maxed out with the main server thread and that is when you'll find yourself getting into trouble. The server can do some limited multi-tasking but like if you have a quad-core CPU, you start your server, and you find launcher64.exe is continually taking 25%+ CPU (let's say before any players have even entered) then you're likely in trouble. If you were even at 18% rather than 25-30% you might have enough breathing room but if it's more like 25%+ at "idle" then you're likely in trouble. More to the point, you can monitor things rather well just by keeping an eye on that CPU percentage on the server.

      [[Note here that when I say "25%" what I actually mean is that one core of the CPU is at 100%. This is what is important for the server, as it essentially runs in a single core. It's just that our typical CPU monitoring software like Windows Task Manager or Performance Monitor, defines 100% CPU as full usage of all cores. So if you have 4 cores, you'll see full usage of one core show up as 25% usage, if you have 8 cores it will be 12% usage, 12 or 16 cores will be different/lower percentages yet, etc. That is just a quirk of the reporting software - the issue here is whether that one core is maxed out or not.]]

      Just for example, one symptom of this problem is very long server load times for players joining the server. You'll see for example, CPU doesn't look maxed out and players join the server in 2-3 minutes. When the server is maxed out or a bit beyond, suddenly the players take 10-15-20 minutes to join. The "join" bar gets stuck at 99% for a long, long, long time.

      Players complain that "you have too many things in your server". But that is not the root cause at all. You can add 20 or 30K ground stationaries and then remove them without making a noticeable difference. You can add 150 aircraft and remove them and again, no noticeable difference.

      It's not "things in the server" per se but one particular type of thing that causes this particular problem.
    • Related, I noticed there were clear thresholds where a certain version of the server wouldn't run at all on Computer A, would barely run on Computer B, and would run just fine on Computer C. The difference in every case was simply raw CPU power. Computer C had enough CPU cycles to handle this particular type of load (see below), which seems to be fairly steady-state regardless of how many players are in the server etc, with just enough cycles left over to run everything else, too. (All the other thing that go on in the server, like AI aircraft, player aircraft, etc etc, clearly take a far lower percentage of the CPU, or perhaps they scale more gracefully somehow as max load is approached.) The CPUs on computers A & B couldn't even keep up with this load and had nothing left over for the rest, thus what ran just fine on Computer C would barely run or not run at all on the less capable CPUs.
    • The lock-up situation seems to have a rather distinct threshold effect. Like if you keep adding more aircraft to the server you'll find a sort of gradual drop-off of responsiveness. But if you add more of this problem element (see what it is below) you'll have some hundreds to thousands of them in-game and all is well, and you add just a few more (10? 20? maybe 100? whatever it is, a small amount added in proportion to the amount you already have) and suddenly you go from "perfectly functional working server" to "completely disastrous non-responsive server"
    • I have long suspected that AA/artillery is the ultimate cause of this problem. You can test this by gradually adding more & more AA/Artillery to a server and sure enough you find behavior like I outline above - up to a point everything works fine and then you reach a definite threshold and suddenly your server is completely non-functional. You can get to a point where you can't even join the server even though there is nothing going on in it.
    • So AA & artillery is culprit #1.
    • But what I hadn't thought through is a large number of other game elements act just like AA/artillery (and must have the same underlying code to a great degree) and so they add to the problem exactly as through they are AA/artillery. These include:
      • Ships - anything that has a gun or shoots (some kinds of barges & transport ships etc) but even more so, MILITARY ships. Those things are like 10 batteries of artillery. Some of them can fire at a tremendous rate and have a bunch of different guns all on one single platform. So, one military ship counts for maybe a dozen or a few dozen individual AA/artillery - depending on the exact ship and armament.
      • Ground vehicles. Again - anything that fires or has a gun. So for example, if you have a ground convoy of six or eight vehicles. Each vehicle can pack a lot of firepower (depending on which exact vehicle you choose) and some fire at a high rate of speed. So again, half a dozen convoy vehicles might be worth a dozen or two artillery or AA pieces.
      • Finally, trains seem to punch above their weight in terms of CPU load. I don't know that they are exactly in the same class as the artillery/AA and similar shoot-ey things, but they are more problematic than you would think. At one point we had like 3 trains running on each side (so 6 total) and found that you could either have those 6 trains or like 100 AI aircraft - that was about the tradeoff.



    There might be a few more things on the list, but the point is, anything that aims, shoots, and fires is going to use a percentage of the available CPU power. AA/artillery, ships whether stationary or moving (particularly military ships), and convoy type vehicles that shoot and defend themselves - all will add to the problem and at the point you have "too many" of these your server will go from working nicely to broken in about 1 second flat.

    You have a clearly limited amount of "artillery power" available on the server, depending on your server's CPU capability, and if you exceed the workable amount you'll see a rather quick transition from "functions well" to "doesn't function at all".

    At any rate, that's my observation. If you have a server that was running fine, you made a few changes and suddenly it's laggy and takes forever for players to log in, look at these "shooter" type elements first.

    Just for example, I added what I thought of as "just a few" ships and suddenly our server went from functional to dysfunctional. (Of course, some of the ships were military, so count them as like 4 dozen AA/artillery pieces.)

    Another time I added "a few" convoys, with similar effect. This was about 4 convoys with 6 or 8 vehicles in them. That seems like a small amount to me. (Also if you added 10 dozen of these items as static "stationaries" you wouldn't see any noticeable change in server functionality. But the live moving/shooting version looks similar but has a far, far different requirement as far as CPU power.) And once again it's not just that these have to be thought of as "aha, another artillery/AA but mobile." On the contrary, these items seem to punch somewhat above their weight as far as CPU usage, when compared with AA or artillery pieces. One of these is worth several AA/artillery pieces - whether 2 or 5 or 10 or 20 I couldn't say, but definitely more than just one.

    FYI!
    Last edited by TWC_Flug; Dec-13-2022 at 23:00.
    System: Microsoft Windows 10 Pro 64 bit, 10.0.18362 N/A Build 18362, 20,437 MB |
    ASUS GeForce GTX 1060 3GB | Intel Core i5-2500 Quad-Core Processor 3.3 GHz 6 MB Cache LGA 1155 | Intel DB65AL motherboard | ARCTIC Freezer i11 CPU Cooler | SVGA 500 watt power supply | Microsoft Sidewinder 2 Force Feedback joystick

  2. #2
    ATAG_Colander's Avatar
    Join Date
    Nov 2011
    Location
    Bir Tawil
    Posts
    11,128
    Post Thanks / Like
    Total Downloaded
    255.73 MB

    Re: What causes servers to lock up/max out/stop functioning

    My two cents...

    The server needs to update O objects. This means that as O increases, the CPU needs for each game tick also increase hence, at a certain high O, the server will not be able to keep up (depending on the CPU).
    Then there are P players. The status of the updated objects O and the status of each player P need to be sent to the other players. Because of this, the network, after a reaching a high enough P will also have issues.

    Now, each P will send updates to the server which, in turn, will send updates to the other P's. This means that you have an exponential function in the number of updates the server must handle with increasing P.


    In short, every object, player, ship, gun, bullet etc that moves, fires, dies, etc, will have to be processed by the CPU and then sent to all the players P. This means that is not as simple as "do not add more than 5 guns" or "do not add ships". There is no rule as any increase anywhere can potentially make the server hit the wall.

    Now, to make things more complicated, the players have different internet speeds and qualities. This means that the server might have to retry sending updates and whatnot.



    PS: I'm hungover so this might not make any sense

  3. #3
    Team Fusion Artist's Avatar
    Join Date
    Mar 2010
    Posts
    2,866
    Post Thanks / Like
    Total Downloaded
    319.97 MB

    Re: What causes servers to lock up/max out/stop functioning

    What Colander wrote.

    And also: All these active objects constantly update their "awareness bubble" independent of there being any player on the server or not: Vehicles and trains compute their paths, check for obstacles, collisions, etc.. AA and artillery scan their 'bubble' for something to attack. There's a whole world of activity. Hence the 25% base load that you've observed.
    Last edited by Artist; Nov-01-2021 at 15:44.

  4. #4
    ATAG Member ATAG_Torian's Avatar
    Join Date
    Sep 2011
    Location
    Newcastle, Australia
    Posts
    1,592
    Post Thanks / Like
    Total Downloaded
    108.53 MB

    Re: What causes servers to lock up/max out/stop functioning

    Quote Originally Posted by TWC_Flug View Post
    You can add 150 aircraft and remove them and again, no noticeable difference.

    But surely if they are bombers they have shooty things bristling all over them. We've all had the joy of a hundred laser guided guns lighting up while trying to do slash attacks on a large bomber formation. In my own experience with mission building there are definite limits as to the number of AI aircraft, particularly bombers, that I can host on my pc. Now I realise my pc is nothing comparable to what u likely have but proportionally wouldn't the same apply ? Certainly your observations about AA/ships/vehicles/trains and cpu load are absolutely valid but AI aircraft not having much impact seems counter intuitive. Perhaps they don't add much load until they go into action and suddenly a hundred gunners wake up ?
    Anyway, a good post and a worthwhile discussion.
    System specs:
    Gaming laptop: Metabox P750ZM, CPU Intel i7-4790K @ 4GHz, GTX 980m with 8Gb DDR5 VRAM, 16Gb Dual channel DDR3 RAM, 2 x Samsung EVO 840 500Gb SSDs, Win10 Home 64bit.

    Gaming PC: MSI PRO Z690 WIFI DDR5 mobo, Intel i5 12600KF cpu, 32Gb Corsair DDR5 5200Mhz RAM, Samsung 980 Pro 1 TB M2 SSD, Corsair 850w psu, MSI RTX 3070 8GB vram vid card, Windows 10 Home 64bit.

  5. Likes ATAG_Marlow, BlacKnight liked this post
  6. #5
    Supporting Member Tibsun's Avatar
    Join Date
    Feb 2016
    Location
    Austria
    Posts
    915
    Post Thanks / Like
    Total Downloaded
    220.96 MB

    Re: What causes servers to lock up/max out/stop functioning

    Quote Originally Posted by Artist View Post
    What Colander wrote.

    And also: All these active objects constantly update their "awareness bubble" independent of there being any player on the server or not: Vehicles and trains compute their paths, check for obstacles, collisions, etc.. AA and artillery scan their 'bubble' for something to attack. There's a whole world of activity. Hence the 25% base load that you've observed.
    Sad to see that also servers cannot really take advantage of mutiple cores, I always thought that those things like computing paths and checking for obstacles and so on could be computed on a 2nd core/thread.
    Atleast we can take away, that if a single core is utilized to almost 100%, things could improve using cpus with faster single core performance.

  7. Likes Baffin liked this post
  8. #6
    Supporting Member Baffin's Avatar
    Join Date
    Apr 2013
    Location
    Northwestern Virginia, USA
    Posts
    1,980
    Post Thanks / Like
    Total Downloaded
    127.26 MB

    Re: What causes servers to lock up/max out/stop functioning

    Quote Originally Posted by Tibsun View Post
    Sad to see that also servers cannot really take advantage of mutiple cores, I always thought that those things like computing paths and checking for obstacles and so on could be computed on a 2nd core/thread.
    Atleast we can take away, that if a single core is utilized to almost 100%, things could improve using cpus with faster single core performance.
    Do most of you feel that performance may increase if we select no Hyperthreading? I'm not sure if my system will allow single core only, but I can stop this Intel multithreading in the BIOS.

    Would doing so impact other players or just myself?
    Windows 11 Pro, ASUS ROG Maximus Z790 Dark Hero, 2 TB Samsung M.2 SSD 990PRO. Intel Core i9 14900KF using TPUII BIOS feature. Air Cooling with Thermalright Peerless Assassin 120 SE CPU Cooler w/ 2 fans. Crucial 96GB DDR 5 RAM at 5600 MT/s. LG 55" 4K OLEDC7P TV, NVIDIA GeForce RTX 4090 Gaming X Trio 24G. Realtek High Definition Audio, Sony Surround amp w/ optical cable for 5.1 speakers, Ear Buds from Motherboard for Discord/TeamSpeak3. TrackIR5, Buttkicker Gamer 2, Thrustmaster Warthog, 2x Saitek X-52 (Buttons & Gear), Gear-Falcon Trim Box, Thrustmaster TPR Pendular Rudder Pedals. Voice Activated Controls.

  9. #7
    Supporting Member Tibsun's Avatar
    Join Date
    Feb 2016
    Location
    Austria
    Posts
    915
    Post Thanks / Like
    Total Downloaded
    220.96 MB

    Re: What causes servers to lock up/max out/stop functioning

    Quote Originally Posted by Baffin View Post
    Do most of you feel that performance may increase if we select no Hyperthreading? I'm not sure if my system will allow single core only, but I can stop this Intel multithreading in the BIOS.

    Would doing so impact other players or just myself?
    Never go single core, have you even checked how many threads of your cpu are being used in the OS and how many in the game?
    If each has some work, even just 20%, it would mean this 20% will be shifted to the single core being already overwhelmed and running at 80-100%.
    It means it's already not keeping up and you want to shove more work on it, you're better off not messing around with cores and threading if you don't know what you're doing.

    If you need game performance as a CloD player buy a graphicscard and if the fastest graphicscard isn't enough for other games, you can afterwards get a new cpu (with all it's necessary parts).
    Very fitting, Intel just released new cpus with a new architecture, big power cores with smaller economic cores with high single core performance for the power cores.
    The i5 12600k is roughly 80% faster than your 7800x in single core. Quite an new astonishing milestone for Intel. Current GPUs also have an incredible increase in performance, but are very expensive - still duo to "shortages"

    Pretty sure servers could make good use of the new Intel 12000s cpus

  10. #8
    Ace
    Join Date
    May 2015
    Location
    Kansas City, Missouri area
    Posts
    515
    Post Thanks / Like
    Total Downloaded
    130.02 MB

    Re: What causes servers to lock up/max out/stop functioning

    Quote Originally Posted by ATAG_Torian View Post
    But surely if they are bombers they have shooty things bristling all over them. We've all had the joy of a hundred laser guided guns lighting up while trying to do slash attacks on a large bomber formation. In my own experience with mission building there are definite limits as to the number of AI aircraft, particularly bombers, that I can host on my pc.
    It's not so much that AI aircraft (and live pilots, too) don't affect the server load, but rather I think we all pretty much understand that both AI and live aircraft do in fact affect server load and we also understand about how many aircraft the server can support - 100 to 150-ish, more or less, depending on your server CPU, internet connection, etc.

    The point is, these other shoot-ey things are an almost independent factor to the number of live/AI aircraft, that is easy to overlook. Especially it is easy to overlook that there are numbers of shoot-ey things beyond AA guns - and that on bring the server to a complete stop with a fairly strong threshold effect. That is to say, you have 2000 shoot-ey things and you add say 20 more - suddenly you've gone from nicely functional server to a completely dysfunctional server that no one can even join.

    This, to me, is somewhat unexpected.

    Also the fact that you can keep an eye on CPU load as a decent approximation of server load or overload is a useful thing to know. Obviously the number of live/AI aircraft figures into the server's CPU load.
    Last edited by TWC_Flug; Nov-29-2021 at 16:57.
    System: Microsoft Windows 10 Pro 64 bit, 10.0.18362 N/A Build 18362, 20,437 MB |
    ASUS GeForce GTX 1060 3GB | Intel Core i5-2500 Quad-Core Processor 3.3 GHz 6 MB Cache LGA 1155 | Intel DB65AL motherboard | ARCTIC Freezer i11 CPU Cooler | SVGA 500 watt power supply | Microsoft Sidewinder 2 Force Feedback joystick

  11. #9
    Ace
    Join Date
    May 2015
    Location
    Kansas City, Missouri area
    Posts
    515
    Post Thanks / Like
    Total Downloaded
    130.02 MB

    Re: What causes servers to lock up/max out/stop functioning

    By the way, another possible cause of "unexpected" server failure is if the AI aircraft ever get into giant battles with each other.

    It's one thing to have 75 AI on one side and another 75 on the other side, all flying various patrols and routes widely dispersed around the map.

    And it's quite another thing if a bunch of those all meet each other and start fighting. If even say 20 or 30 or 40 of these get into a giant furball you'll likely see the server brought to its knees - that's one of the things I've noticed, anyway, though I'll confess I haven't tested it with extreme thoroughness. But it's the sort of thing that makes a good degree of sense - the server is having to fly all of those planes against each other with rather fine control, aim and shoot, handle multiple weapons per aircraft shooting, etc, compared with what it takes to fly a straight line at 7000 meters with no enemy in sight.

    (Just from a server and campaign design standpoint there is little point in having AI on AI battles in some far corner of the map, even if they do nothing more than add 0.01% server load. They are still pointless and don't add anything of value to the campaign, if there are literally no breathers around to even observe or notice what is happening. So when we noticed this problem happening we just set out to avoid it as best we could, for all those reasons, including maybe it helps the server remain more responsive.)
    System: Microsoft Windows 10 Pro 64 bit, 10.0.18362 N/A Build 18362, 20,437 MB |
    ASUS GeForce GTX 1060 3GB | Intel Core i5-2500 Quad-Core Processor 3.3 GHz 6 MB Cache LGA 1155 | Intel DB65AL motherboard | ARCTIC Freezer i11 CPU Cooler | SVGA 500 watt power supply | Microsoft Sidewinder 2 Force Feedback joystick

  12. Likes Artist liked this post
  13. #10
    Ace
    Join Date
    May 2015
    Location
    Kansas City, Missouri area
    Posts
    515
    Post Thanks / Like
    Total Downloaded
    130.02 MB

    Re: What causes servers to lock up/max out/stop functioning

    > Do most of you feel that performance may increase if we select no Hyperthreading?
    I did a little reading on this and my understanding is that, as Tibsun said, allowing hyperthreading is almost always advantageous.

    The CPU percentages you see if you have "logical cores" are somewhat misleading. As in, let's say you have a CPU with four real cores and 8 total cores including the logical cores.

    So if you turn hyperthreading off and run a single core-type process you will see that it takes up 25% CPU.

    Now turn hyperthreading on and run the same process - you will see it now takes 12.5% CPU.

    But . . . in reality those are both the same amount of CPU. Your process would run the same speed, reach the same benchmark speeds, whatever, in both situations.

    So even though 12.5% CPU looks like have as much, and you would imagine it would be half as fast, it just isn't. It's actually using one core to the full extent possible, so the 12.5% really is equivalent to 25% with hyperthreading turned off.

    The improvement with hyperthreading, though, is that the CPU can handle multiple demands better. It has literally 2X as many places to put threads, whether they are threads from the main program (launcher -server in this case) or just the various random things the pop up in the windows environment. It can even run threads, especially brief ones, on the other side of the core where the main program is running, because hyperthreading sort of fits the extra logical core into CPU cycles that aren't being used anyway.

    Anway, long & short is that hyperthreading improves performance in pretty much every conceivable situation but the way CPU usage is reported in percentages makes you think that maybe that isn't the case.
    System: Microsoft Windows 10 Pro 64 bit, 10.0.18362 N/A Build 18362, 20,437 MB |
    ASUS GeForce GTX 1060 3GB | Intel Core i5-2500 Quad-Core Processor 3.3 GHz 6 MB Cache LGA 1155 | Intel DB65AL motherboard | ARCTIC Freezer i11 CPU Cooler | SVGA 500 watt power supply | Microsoft Sidewinder 2 Force Feedback joystick

  14. #11
    ATAG Member ATAG_Oskar's Avatar
    Join Date
    Nov 2017
    Location
    Canada
    Posts
    986
    Post Thanks / Like
    Total Downloaded
    908.31 MB

    Re: What causes servers to lock up/max out/stop functioning

    If you use the Resource Monitor you can see the CPU load broken out by core.

  15. #12
    Ace
    Join Date
    May 2015
    Location
    Kansas City, Missouri area
    Posts
    515
    Post Thanks / Like
    Total Downloaded
    130.02 MB

    Re: What causes servers to lock up/max out/stop functioning

    > And also: All these active objects constantly update their "awareness bubble" independent of there being any player on the server or not: Vehicles and trains compute their paths, check for obstacles, collisions, etc.. AA and artillery scan their 'bubble' for something to attack. There's a whole world of activity. Hence the 25% base load that you've observed.

    Thanks - these are all helpful observation.

    Along those lines, one way we've found to include more mobile objects in missions, without killing the CPU quite as much, is to include a lot of "Time Out" in their waypoints. (In FMB this is shown as "Time Out" - you enter the number of hours/minutes you want that actor to wait without moving.) So maybe they move for 5 minutes, then "time out" for 5 minutes. Or . . . maybe for **20** minutes or even ***40*** minutes.

    The objects are still 'active' when they are in time out - if something attacks them, they will respond. But they are not *moving* and that seems to reduce the CPU load a lot.

    Just for example, instead of having one vehicle convoy in your mission that is moving 100% of the time, you could probably have 5, where each is moving 20% of the time and standing still 80% of the time.

    Similarly for ships (the moving type, not the stationary type) and trains.

    As a mission designer, it gives you some more options for having more objects in the server that are moving vs static, and having more than just one or two of them in your mission, yet also not killing the CPU.
    Last edited by TWC_Flug; Dec-30-2022 at 16:40.
    System: Microsoft Windows 10 Pro 64 bit, 10.0.18362 N/A Build 18362, 20,437 MB |
    ASUS GeForce GTX 1060 3GB | Intel Core i5-2500 Quad-Core Processor 3.3 GHz 6 MB Cache LGA 1155 | Intel DB65AL motherboard | ARCTIC Freezer i11 CPU Cooler | SVGA 500 watt power supply | Microsoft Sidewinder 2 Force Feedback joystick

  16. #13
    Ace
    Join Date
    May 2015
    Location
    Kansas City, Missouri area
    Posts
    515
    Post Thanks / Like
    Total Downloaded
    130.02 MB

    Re: What causes servers to lock up/max out/stop functioning

    The other reason I wanted to point this out, is it is a possible area for TF to look at to improve the function of SP and multi-player servers even more.

    (Not that it is terrible right now - by far the best I've seen!)

    But most everything else I've looked at degrades in a pretty gradual way as the server load increases. This is by far the biggest "threshold effect" type thing I've experienced in running a server.

    So maybe there is a way to make this effect degrade more gracefully, too? Like, if average CPU load gets about X then AA & ships reduce their scanning bubble by 10%, and their firing rate by 10%. Maybe mobile actors slow their speed by 10%. Or maybe AI actors that are out of site of any live pilot slow or stop what they are doing.

    It seems like a few rounds of this, while perhaps not idea would be preferable to having the server completely lock up at some threshold.

    I realize this might not be the top priority right now, and perhaps not easily done or possible at all. But I thought I'd put it out there, just because this is the biggest threshold effect I've seen on server performance - so perhaps worth looking at, at some point.
    System: Microsoft Windows 10 Pro 64 bit, 10.0.18362 N/A Build 18362, 20,437 MB |
    ASUS GeForce GTX 1060 3GB | Intel Core i5-2500 Quad-Core Processor 3.3 GHz 6 MB Cache LGA 1155 | Intel DB65AL motherboard | ARCTIC Freezer i11 CPU Cooler | SVGA 500 watt power supply | Microsoft Sidewinder 2 Force Feedback joystick

  17. #14
    Student Pilot
    Join Date
    Apr 2023
    Posts
    13
    Post Thanks / Like
    Total Downloaded
    0

    Re: What causes servers to lock up/max out/stop functioning

    I know this is a very old thread so apologies for the necro post but as I'm reading the forum properly for the first time in a LONG time I just wanted to ask;

    Does anyone running a server ever get the "CHANNEL IS DESTROYED" message? We occasionally used to get this on SoW back in the day, but could never pin point what it was, but once we got it, the server ground to a halt immediately and had to be hard stopped and rebooted. Usually we'd see the message spammed into the log (one of the things I wondered was if the log writing was taking all the cpu cycles!).

    It did lead inevitably to the classic Dietrich joke about the mighty LW having destroyed the entire channel so it wasn't completely annoying.

  18. #15
    ATAG Member ATAG_Oskar's Avatar
    Join Date
    Nov 2017
    Location
    Canada
    Posts
    986
    Post Thanks / Like
    Total Downloaded
    908.31 MB

    Re: What causes servers to lock up/max out/stop functioning

    I'm wondering if past crashes were related to the server bandwidth. We did a test a few years ago where we had 90 human players and a similar number of AI aircraft. This worked fine on the ATAG server. ATAG servers are hosted at a site with very good internet connectivity, and are fairly fast servers. If you were running a server at home the limited bandwidth, especially on upload, might lead to network problems. More players on the server would make the problem worse.

  19. #16
    Student Pilot
    Join Date
    Apr 2023
    Posts
    13
    Post Thanks / Like
    Total Downloaded
    0

    Re: What causes servers to lock up/max out/stop functioning

    Quote Originally Posted by ATAG_Oskar View Post
    I'm wondering if past crashes were related to the server bandwidth. We did a test a few years ago where we had 90 human players and a similar number of AI aircraft. This worked fine on the ATAG server. ATAG servers are hosted at a site with very good internet connectivity, and are fairly fast servers. If you were running a server at home the limited bandwidth, especially on upload, might lead to network problems. More players on the server would make the problem worse.
    That's entirely possible but I don't know if that is the answer. We regularly had a full (86 players) server, multiple bomber raids/mini raids (probably not to the level of number of players but 40-50), and a mission file that had over 8k objects including something in the region of several thousand AAA units. We only got these issues on rare occasions, and they always seemed to be associated with some sort of client drop out. Restarting the server entirely and replaying the exact same mission generally worked so I don't think it's as simple as a home based server not having decent enough bandwidth. If it were, we'd have run into the problem a lot more than we did IMO.

  20. #17
    ATAG Member ATAG_Oskar's Avatar
    Join Date
    Nov 2017
    Location
    Canada
    Posts
    986
    Post Thanks / Like
    Total Downloaded
    908.31 MB

    Re: What causes servers to lock up/max out/stop functioning

    Quote Originally Posted by No.54 Reddog View Post
    That's entirely possible but I don't know if that is the answer. We regularly had a full (86 players) server, multiple bomber raids/mini raids (probably not to the level of number of players but 40-50), and a mission file that had over 8k objects including something in the region of several thousand AAA units. We only got these issues on rare occasions, and they always seemed to be associated with some sort of client drop out. Restarting the server entirely and replaying the exact same mission generally worked so I don't think it's as simple as a home based server not having decent enough bandwidth. If it were, we'd have run into the problem a lot more than we did IMO.
    If you can run that mission it sounds like hardware/bandwidth is not the problem. Log files would be very helpful here.

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •