SB Pro PE 4.0 Hardware Recommendations

[Ssnake]

All right, this is going to be a difficult topic for a variety of reasons. But the results of our internal benchmarking are trickling in and a few trends can already be seen.

 

• Difficulty #1: Compared with Steel Beasts Pro 3.0 the frame rates have taken a significant hit. We're basically dying from a thousand paper cuts, and have already dealt with some of them. But there isn't enough time left to make significant progress right now. Just because you know what needs to be fixed doesn't mean that it's easy to do, or quick to accomplish.
• Difficulty #2: There is no clear winner. My eight-core i7 @3.5GHz with a GTX980 doesn't fare much better than my four-core i5 @2.9GHz with GTX980M. In fact, the same notebook with the integrated graphics chip HD4600 can still run 4.0, albeit on minimal graphics settings rather than the default ones.

 

So far, everything we tested (with a single exception) falls into the "low performance" category (orange, in the 3.0 color code) which we define as "the lowest frame rate must still exceed 24 FPS"; the next higher category (yellow) we defined at "average frame rate >36 FPS AND lowest frame rate >30 FPS). In our benchmark scenario, NO hardware setup managed to accomplish this, and I don't think we'll see that anytime soon. The exception was an older AMD FX-6100 machine with a Radeon HD 6700 where the battalion sized battle in the end made the frame rate dip just below 10 FPS, presumably with default graphics settings.

 

Now, what does that mean?

Surprisingly, this is less disastrous than it may sound. At least on both of my machines the "feel" of the benchmark scenario wasn't that bad. The particle system doesn't kill performance per se, the new terrain (once made available to you) won't either. But of course nobody likes low frame rates. Here are some ideas:

 

1. Cutting back on graphics settings CAN help to stabilize the frame rates, particularly on older/weaker machines. Disable the shadows, reduce the ground clutter to a minimum, disable antialiasing if you must. These three steps alone will usually bring a useless 10 FPS stutter to 20...25 frames where you can still aim at moving targets from the gunner's position.
2. Reducing the screen resolution from 1920x1080 to 1280x720 can help a bit too and may not be such a terrible compromise to make.
3. Above all, enjoy the bad weather!  Now, this may sound terrible, but hear me out: First of all, rain and snow are new and you want to try out these options anyway. Second, on my fast machine (i7, GTX980) reducing the visibility from 6km daysight to 3km helped to boost a certain scene from 40 to over 60 frames per second, a 55% increase. And for a European autumn day 3km visibility is actually somewhat above average, so it's not even unrealistic. Not every machine will profit from this strategy, but most of the machines that we tested do. One machine even almost tripled its daysight frame rate. Thermals profit less from this option, and the i5/intel HD4600 didn't change its frame rate at all - but still.

 

In the meantime we will apply more band-aid. Lots of it. Like I wrote, we know where it's coming from and we know what must be done. Not everything can be done quickly, but SOME of it can and will be done in the coming months. To that extent there's hope that over time the efficiency of our render process can be improved enough to increase the visibility again to distances that you're accustomed to.

[Ssnake]

1 hour ago, Apocalypse 31 said:

Can you please clarify?

What does 'wasn't that bad' mean, in terms of a FPS #?

 

I stand by our version 3.0 categories. I'm not redefining them:

 

For an "unusable" rating the average frame rate in a test would be under 20 frames per second. Some people say they can still work with 15 FPS, but our threshold is 20. Anything less than that, don't bother, period.

 

For a "not recommended" rating the average frame rate in a benchmark scenario would be under 24 FPS.

A "low" performance rating is defined as an average frame rate above 24.

To earn a "medium" performance rating the average frame rate must exceed 36 FPS, and the lowest recorded frame rate would be above 30.

For "good", it must be 48 FPS average, and 40 FPS minimum.

"Great" performance is defined as an average frame rate avove 55 FPS.

 

 

Like I said, we stand by these categories. That there currently is no known hardware configuration that would perform better than "medium" (an eight-core Xeaon processor, a GTX 1080, and 32 GByte system RAM) only says one thing, that we, eSim Games, cannot be happy with the performance of our engine. I won't sugarcoat this. So, we must do more to optimize things. The question is, shall we not release anything until we improved the performance, or should we at least give everybody the choice to decide for himself.

 

Personally, the low performance (between 22 (low), 63 (top), and 39 FPS (average)) appears "fully usable". But someone else - you! - may have a different opinion about it. So, while I wouldn't be exactly "happy" about this kind of a performance from an "almost top of the line" hardware setup, at least _I_, personally, wouldn't feel cheated either if I had just bought a 4.0 upgrade license. But I would also understand if some other guy said that he would rather wait for better performance.

 

 

 

However, in the light of all this, keep in mind that roughly the same performance can be had from rather mediocre hardware setups as well. Just look at the i5 processor (2.9 GHz) with integrated graphics chipset HD4600. For heaven's sake, it's an Intel graphics processor, and you can still squeeze an average of nearly 34 FPS from it in this benchmark, albeit at minimum settings. So that's not too terrible IMO, and it shows that even weaker systems can still get a somewhat useful performance out of SB Pro PE 4.0 in its current form.

At the same time it is imperative for our development team to improve the overall performance in 3D scenes (from the map screen, everything runs just fine at 60 FPS). We will invest time and effort to improve here.

 

 

A few words about our benchmark scenario. It starts with a "maximum overdraw" situation where you look in high magnification into a deep forest, both in daysight and in thermal view. Since theremal views generally have a 3x longer render distance, it's no surprise that the day view generally performs better. The same scene then changes from 6,000m visibility to just 3km, and usually yields an about 40% frame rate increase.

 

The next scene shows a frame rate for a vehicle commander in the unbuttoned eye view in a rural environment.

 

Then follows a busy street scene inside a larger town with hundreds of pedestrians around.

This is then subject to some intense artillery shelling which, contrary to my expectations, actually doesn't change the frame rate much. So, this counts as our "urban combat" test.

 

Finally there's a big battle with one company team defending against a reinforced mech battalion's assault. Here we look at the frame rate when there are a lot of line of sight calculations are involved. Finally we look at the frame rate at the end of the battle when there's a lot of burning vehicles around that emit a lot of smoke and fire particles.

 

In other words, I tried to create a benchmark that reflects the reality of Steel Beasts scenarios. It's demanding, as are our performance categories. Even with SB Pro PE 3.0 only few hardware combinations will earn the "great" performance seal. This, by the way, says more about the (lack of) efficiency of the Steel Beasts engine. It's a constant reminder that we need to get better. 

[Gibsonm]

12 hours ago, Ssnake said:

A comparison between 4.010 and 4.017(+) would also be appreciated.

 

Do we have a standardised "benchmark" scenario for testing this?

 

Or do I just play a scenario of my choice (presumably with lots of Artillery, rain, etc.) on my:

 

Win 7 box using 4.010

 

Win 7 box using 4.017(+)?

 

New Win 10 box using 4.010

 

New Win 10 box using 4.017(+)?

[Ssnake]

I don't think a comparison between Win7 and Win10 performance is needed. Win10 is just as fast (in some cases a tad faster) than Win7. So, that's two test runs eliminated right away.

 

Yes, there is a somewhat standardised benchmark scenario:

http://www.steelbeasts.com/files/file/2518-sb-pro-pe-40-benchmark-scenario/

 

 

[tankenator]

On 3/27/2017 at 6:14 PM, Ssnake said:

I don't think a comparison between Win7 and Win10 performance is needed. Win10 is just as fast (in some cases a tad faster) than Win7. So, that's two test runs eliminated right away.

 

Yes, there is a somewhat standardised benchmark scenario:

http://www.steelbeasts.com/files/file/2518-sb-pro-pe-40-benchmark-scenario/

 

 

Just did the benchmark, running 4.010 on Intel i5 6600K on asus z170a mb with 16gig of 2666 ddr4 and samsung 850 evo 256gig ssd with gigabyte r7 360 video card (the weak point of this rig, havent had budget to improve it yet)  Numbers as follows: 

Daylight woods: 30

Thermal woods: 60

Weather woods: 60

Weather Thermal woods: 44

Open Rural: 60

City Observation of Traffic: 30

City Observation of Arty: 27

Coy vs BTL battle midpoint: 40

Coy vs BTL endpoint: 25

 

According to the guidelines its low performance, but doesnt seem slow at all.....the dips really arent evident unless there is a ton of stuff on map.   Very very playable..

 

 

 

[Ssnake]

OK, as mentioned: We created a benchmark scenario for SB Pro PE 4.0 which has been uploaded to the public Downloads section on this site, which tests scenes as mentioned in the description of the scenario file:

1. Daysight, deep forest - 6,000m visibility

2. Thermal sight, deep forest - 6,000m visibility

3. Daysight, deep forest - 3,000m visibility

4. Thermal sight, deep forest - 3,000m visibility

5. External view, rural/open - 6,000m visibility

6. Commander's binocular, urban street life - 6,000m visibility

7. Commander's eye sight, urban/artillery - 6,000m visibility

8. Battalion level battle (max LOS) - 6,000m visibility

9. Battalion level battle, aftermath - 6,000m visibility

 

 

Tests are supposed to be made with screen resolutions "Full HD" (=1920 x 1080) and "HD" (=1280 x 720).

We use six categories to grade performance,

 

1. Unusable:
   Frame rates at any point of the benchmark are too low to operate the simulation effectively (<15 fps).
2. Not recommended:
   Either the average frame rate, or just a single one of the three scenes yields a frame rate of under 24 fps (no rounding up allowed - for this, or all other categories)
3. Low performance:
   Both the average and the minimum frame of all three scenes rate must be above 24 fps
4. Medium performance:
   The average frame rate must be above 36, the lowest frame rate above 30 fps
5. Good performance:
   The average frame rate must be above 48, the lowest frame rate above 40 fps
6. Great performance:
   Both minimum and average frame rate must exceed 55 frames per second

[Ssnake]

Test results version 4.000—4.010

Note that version 4.017+ requires new assessment to a large number of changes to the render engine.

• Every tested system performed "low" per initial thread description (that's why there is no tabular overview, the results are all the same), except:
• Unusable: Intel E8600 (@3.33GHz), Radeon HD5970, even at low detail settings

[Ssnake]

 

I'm happy to report that the same version 4.0 benchmark scenario performs much better on my main machine (i7-4770K with GTX980) with the latest 4.1 beta version, passing muster to be rated "good". None of the scenes yielded less than 45 frames per second, with an average frame rate of 54. A lot of hard work by artists, beta testers, and programmers alike has paid off big time. 6,000m+ visibility settings will no longer be prohibitive for reasonably powered machines (case in point, my machine is about four or five years old now).

 

The beta team will start a more thorough and systematical test with the benchmark scenario so that we can provide ratings for a bigger spectrum of CPU and GPU combinations, so please don't treat this as the final result. But, I'm almoste elated. It also confirms what I wrote three years ago, that there wouldn't be the one easy fix to boost the frame rates. Rather, a lot of different optimizations had to be made, but they paid off as far as I can tell.

[Ssnake]

On 4/16/2019 at 2:01 AM, Ssnake said:

I'm happy to report that the same version 4.0 benchmark scenario performs much better on my main machine (i7-4770K with GTX980) with the latest 4.1 beta version, passing muster to be rated "good". None of the scenes yielded less than 45 frames per second, with an average frame rate of 54.

The more thorough evaluation comes back slightly less positive with above 40 fps as the lowest, and above 49 frames on average, on default settings, in Full HD resolution.

 

With minimal settings (no shadows, no antialiasing, no nothing) I could even get an Intel HD4600 graphics chipset to work in HD resolution, with an overall "not recommended" verdict (but still above "unusable"), where the minimum framerate was above 21, and the average framerate being a somewhat respectable 32. An hour into the test the graphics developed strange artifacts.. Pretty much anything above one notch in the graphics settings made Steel Beasts stop responding, however, so definitely "not recommended".

 

Overall, and this is still preliminary, we can at least see that different cards/video RAM/CPU combinations deliver different results again (that is, other than "poor", which was the universal result with 4.0). While that isn't where I'd like it to see, it's still a marked improvement over 4.0 results. In all fairness towards the programming team I have to point out that if the settings are dialed back to where they were when we started with version 3.0, the framerates are quite comparable.