Network desynchronisation

[Gibsonm]

3 hours ago, mrivers said:

At this point, I don't know if there's a connection to the ping times logged.

 

 

I'm glad we discovered a bug, but I'm just curious.

 

Wouldn't this be a global effect - impacting all users since they all played the scenario with the issue, not just one?

 

Edited June 1, 2023 by Gibsonm

[Ssnake]

Invalid combatant IDs are a bit of a lurking horror just beyond your peripheral vision. Things can go about normally for a good while until there's some form of an interaction with them that changes their status (like, shooting them, or a collision damage). Once that this happens, the wrench is thrown into the gearbox with the associated "funny effects".

[Abraxas]

I would like to point out the following:

1. The problems occurred with scenarios from different creators.

2. As far as I know, the problems only occurred with one user.

3. The problems occurred with a license greater than 8 participants.

4. The problems occurred since version 4.379.

However, I have now revised the scenario "Roter Herbst - Red Autumn" regarding the templates.

[Ssnake]

3 hours ago, Abraxas said:

1. The problems occurred with scenarios from different creators.

We cannot rule out (at this point) that they have a similar problem with unit IDs.

 

3 hours ago, Abraxas said:

2. As far as I know, the problems only occurred with one user.

If that user happened to be the only one to interact with a unit with invalid ID, it may be coincidence. Or we're dealing with more than one issue, that certainly is also a possibility.

[Abraxas]

The other creator (Gladiator) did not use any civilian actors, but this does not rule out the possibility of ID being assigned twice or more, because these are already specified in SB!

Here's an example:

Test same ID.sce

[Falcon]

3 hours ago, Ssnake said:

If that user happened to be the only one to interact with a unit with invalid ID, it may be coincidence. Or we're dealing with more than one issue, that certainly is also a possibility.

Is it possible that invalid ID unit was a Redfor unit? I've noticed my issues at ca. T+01:29:50, I was engaging remaining vehicles of Red BMP platoon. The same group was already engaged by Gladiator and Gibsonm, who didn't report any issue.

I was re-checking AAR. At T+01:29:46 my tank is shooting (dust is up and gun is elevated in order to reload). But both mine and Gibsonm's AARs doesn't register any hit. Gibsonm's AAR register only a BMP kill at T+01:30:01, which is not showing on my AAR (as I've written here previously).


I didn't notice nor recall any interaction with Civilian units. I remember that I saw a few of them on the map at the beginning of the scenario. Later on, AAR shows that there was one bus driving in the same town as were tanks under my control (A and A3), but the bus was not even in the same street so there shouldn't be any collisions/interactions with that unit.


From my limited experience with software, even a rather small error can cause instability in the system. Although I don't observe any instability issues of my PC/OS, this error could potentially be the trigger causing this. It is also interesting the specificy of issue - 90minutes, Leo2A4 and Abraxas's server as a session host. So far we didn't manage to recreate the problem at other time or in different vehicle.

If there are any ideas what to test next-time, let me know. I am happy to help as much as I can, as this is quite limiting issue for me.

Edited June 2, 2023 by Falcon

[Ssnake]

2 hours ago, Abraxas said:

The other creator (Gladiator) did not use any civilian actors, but this does not rule out the possibility of ID being assigned twice or more, because these are already specified in SB!

Here's an example:

We need to differentiate between Callsigns (that's what's exposed in the UI and in the dialog mentioned by Mike further above), and CombatantIDs (which is what Steel Beasts internally uses). Dupicate Callsigns are bad for users, but as long as the CombatantIDs are unique, no problem for Steel Beasts.

[mrivers]

23 hours ago, Gibsonm said:

 

I'm glad we discovered a bug, but I'm just curious.

 

Wouldn't this be a global effect - impacting all users since they all played the scenario with the issue, not just one?

 

I'd expect there are some issues appearing on all clients. 

 

The problem is that some network data arrives in a packet that contains data for multiple combatants.  When the invalid callsign/combatantId is encountered, decoding the packet is aborted and any subsequent combatant data in the packet is lost.   So some combatants might be operating on stale data... but they are likely to get an update eventually, masking the problem.

 

FWIW, the callsign and combatantId are the same thing (the callsign is just user friendly, while the combatantId is coder hostile).

Edited June 2, 2023 by mrivers

[mrivers]

13 hours ago, Abraxas said:

I would like to point out the following:

1. The problems occurred with scenarios from different creators.

2. As far as I know, the problems only occurred with one user.

3. The problems occurred with a license greater than 8 participants.

4. The problems occurred since version 4.379.

However, I have now revised the scenario "Roter Herbst - Red Autumn" regarding the templates.

I'll continue to look into this next week.

 

Questions regarding 3:

* When you use the license for >8 players, are you using the PE Server exe? (I assume you must, in order to have >8 players).

* How many people are actually playing during each session?

* What exact versions of the exes are in use host/clients?

 

If you want to post the scenario again, I can verify if all the callsign/combatantID problems are resolved.

[Falcon]

The problem repeated today.

Leo2A6
120min (issues from T+70min)
Same host
Kaspersky Free Antivirus + VPN OFF (checked multiple times via Task Manager before, during and after the Online Session)

If I will have time tomorrow (I have busy 4 days ahead), I will do a few things (play the scenarios in Offline Session, check AARs, go through Debugs, ...).


However, the problem appears to be connected with CPU usage. There are moments, when all my 8 logical processors went into 100 % for a brief moment (2s). I was checking it live with Debug and it started to cause the High Ping Errors.

The high CPU usage appeared on these occasions:

Switching positions:
TC <=> Gunner
Map => 3D View

and when working in Gunner's seat:
F1 <=> F2 (WHOT, I didn't test other regimes in GPS)

I didn't noticed this issue in TC's position.
GPU usage spike to max 50 %


This time, the session was better at re-synchronizing. The High-Ping Errors in Debug were successfully "extinghuished", however it had a permanent effect on synchronisation on the battlefield (Red wrecks were not visible on my PC). I will investigate AARs later and report back (or let me know and I can send them to you).



Perhaps it is my subjective bias, but I can't remember these issues prior the "Thermal sights update", which was released a few months back. I wish I could give you better feedback about this, but I started to play regularly SB since last November (I've played 2x30 days in last 2 years).

Anyway, I will report back later during the following week.


note1:
Here is the visible CPU spike after switching from Map into 3D view. Unfortunately I didn't have the opportunity to measure this when inside my own tank. My HW specifications are visible there as well.

 

Edited June 2, 2023 by Falcon

[Abraxas]

10 hours ago, mrivers said:

Questions regarding 3:

* When you use the license for >8 players, are you using the PE Server exe? (I assume you must, in order to have >8 players).

* How many people are actually playing during each session?

* What exact versions of the exes are in use host/clients?

 

If you want to post the scenario again, I can verify if all the callsign/combatantID problems are resolved.

Hi Mike!

Answers:

1. Of course, otherwise it's not possible!

2. 10-14 participants

3. all partipants use SB 4.379 or do you mean another "exe"?

Here is the updated scenario (if you need the passwords contact me by email or PM!):

 

ROTER-HERBST_PzKp_ohne.rar

Edited June 3, 2023 by Abraxas
passwords

[mrivers]

Just a heads up that we're still investigating.  

 

"Perhaps it is my subjective bias, but I can't remember these issues prior the "Thermal sights update", which was released a few months back"

 

This is just a simple rendering trick, it may consume a little more GPU/CPU time, but it's a single threaded feature and can't result in all CPU threads being maxed out.

 

I'm sorry, I have yet another request...

Please have the host do an 'In Progress Save' when the problem appears, and upload/link it here.  I'm hoping it might capture the issue.

 

 

 

The most recent ROTER-HERBST sce still has problems on the Zivil side, some units still have invalid Ids.  One possible indicator is that the unit icons won't show a name(though I'm not 100% certain).

[Abraxas]

Thanks Mike!

 I've checked the civilian units and found 3 vehicles with the same ID. This was only in the scenario ROTER-HERBST_PzKp and is now revised!

ROTER-HERBST_PzKp.rar

[Abraxas]

11 hours ago, mrivers said:

Please have the host do an 'In Progress Save' when the problem appears, and upload/link it here.  I'm hoping it might capture the issue.

"In Progress Save", as a host: how do I do that?

[Ssnake]

Pause the (network) session in Map view (Pause key), then open the File Menu, "Save as..."

[Gibsonm]

1 hour ago, Abraxas said:

"In Progress Save", as a host: how do I do that?

 

I can talk you through it on the "night" if you like.

 

Look for "In-progress save ..." (in German I guess), 5th choice in the File option, just below the horizontal line:

 

 

Edited June 8, 2023 by Gibsonm

[Abraxas]

Thanks, Nils and Mike!

[mrivers]

9 hours ago, Abraxas said:

Thanks Mike!

 I've checked the civilian units and found 3 vehicles with the same ID. This was only in the scenario ROTER-HERBST_PzKp and is now revised!

ROTER-HERBST_PzKp.rar 175.44 kB · 0 downloads

This scenario looks good now.

[Falcon]

Today, no visible issues.
Debug reports only 2x Error regarding High Ping (3-4s) and 2x Error regarding artillery Smart ammunition (note 1). This error was not previously noted, however it doesn't look as a cause for these issues.
If needed, let me know and I will send a link for Debug, AAR and Scenario file.

Leo2A5
Same host
ca. 110min scenario


note1:
Error line from Debug:
[21:32:15,536] CGame ERROR: A nonimpotent CHEArtySplash was created for ammo: HEAT-EFP-SF 'SMArt DM702'^

note2:
Scenario had lower amount of units in comparison with those with networking issue. Tomorrow, I will test the larger scenario in Offline in order to observe any unusually high CPU usage or spikes of it.

[Gibsonm]

12 hours ago, Falcon said:

Tomorrow, I will test the larger scenario in Offline in order to observe any unusually high CPU usage or spikes of it.

 

That wont help with network traffic issues, but may help in terms of your PC's limitations (if any).

 

Edited June 10, 2023 by Gibsonm

[Falcon]

On 6/10/2023 at 1:19 AM, Gibsonm said:

 

That wont help with network traffic issues, but may help in terms of your PC's limitations (if any).

 

No issue when playing the same scenario Offline, my PC hardware is thus not the issue.
CPU avg. usage was ca. 30 % (no difference between Online session).
I was switching seats and tanks a lot, no observeable spike unless I went in GPS and was looking at burning MechInf Coy. (which is expected, a lot of effects has to load, and even then it was a spike to 40 %).

I don't think that Host's Server hardware is the issue as well, as then everyone in the sessions would experience this issue.


I am out of ideas. If there is anything I can do/test, let me know.

Edited June 11, 2023 by Falcon

[Lumituisku]

9 hours ago, Falcon said:

I am out of ideas. If there is anything I can do/test, let me know.

 

Try to get server host to do in progress save when problems occur. 

 

If you have scenario in mind where problem would reliably occur. You could get some of us play it with you just to try to get it happen again. 

 

If nothing else I could probably at next weekend. 

 

 

Also look / try to remember for commonalities

 

Was there lot of indirect fires (especially smoke) in the mission when problem happened?

 

Perhaps lots of trees / objects such as buildings on the map area you were observing?  

 

Was there lots of infantry or other units in near vicinity of your vehicle. Like few hundred meters?

 

 

Edited June 12, 2023 by Lumituisku