"Great spirits have always encountered violent opposition from mediocre minds."
- Albert Einstein
More pages: 1 ... 9 10 11 12 13 14 15 16 17 18 19 ... 21 ... 31 ... 41 ... 47
Performance
Wednesday, July 22, 2009 | Permalink

The key to getting good performance is properly measuring it. This is something that every game developer should know. Far from every academic figure understands this though. When a paper is published on a particular rendering techniques it's important to me as a game developer to get a good picture of what the cost of using this technique is. It bothers me that the performance part of the majority of techical papers are poorly done. Take for instance this paper on SSDO. This paper isn't necessarily worse than the rest, but just an example of common bad practice.

First of all, FPS is useless. Gamers like them, they often show up in benchmarks and so on, but game developers generally talk in milliseconds. Why? Because frames per second by definition includes everything in the frame, even things that are not core components of the rendering technique. For SSDO, which is a post-process technique, any time the GPU needs for buffer clears and main scene rendering etc is irrelevant. But it's part of the fps number, and you don't know how much of the fps number it is. Therefore a claimed overhead of 2.4% of SSDO vs. SSAO, for instance, is meaningless. The SSDO part may actually be twice as costly as SSAO, but large overhead in other parts of the scene rendering could hide this fact. For instance if the GPU needs 12 milliseconds to complete main scene rendering, and 0.5ms to complete SSAO, that'll give you 12.5ms in total for a framerate of 80 fps. If SSDO increases the cost to 1ms, which would then be twice that of SSAO, the total frame runs in 13ms for a framerate of 77 fps. 4% higher cost doesn't seem bad, but it's 0.5ms vs. 1ms that's relevant to me, which would be twice as expensive and thus paint a totally different picture of the cost/quality of it versus SSAO. Not that these numbers are likely to be true for SSDO though. This paper was kind enough to mention a "pure rendering" number (124fps at 1600x1200), which is rare to see, so at least in this can we can reverse engineer some actual useful numbers from the useless fps table, although only for the 1600x1200 resolution.

So main scene rendering is 1000/124 = 8.1ms, which is definitely not an insignificant number.
With SSAO it's 1000/24.8ms = 40.3ms
With SSDO it's 1000/24.2ms = 41.3ms

So the cost of SSAO is 40.3ms - 8.1ms = 33.2ms, and for SSDO it's 41.3ms - 8.1ms = 34.2ms
Therefore, SSDO is actually 3% more costly than SSAO, rather than 2.4%. Still pretty decent, if you don't consider that their SSAO implementation obviously is much slower than any SSAO implementation ever used in any game. 33.2ms means it doesn't matter how fast the rest of the scene is, you'll still never go above 30fps. Clearly not useful in games.

Another reason that FPS is a poor measure is that it skews the perception of what's expensive and not. If you have a game running at 20fps and you add an additional 1ms worth of work, you're now running at 19.6fps, which may not look like you really affected performance all that much. If you have a game running at 100fps and add the exact same 1ms workload you're now running at 90.9fps, a substantially larger decrease in performance both in absolute fps numbers and as a percentage.

Percentage numbers btw are also useless generally speaking. When viewed in isolation to compare two techniques and you compare the actual time cost of that technique only, it can be a valid comparison. Like how many percent costlier SSDO is vs. SSAO (when eliminating the rest of the rendering cost from the equation). But when comparing optimizations or the cost of adding new features it's misleading. Assume you have a game running at 50fps, which is a frame time of 20ms. You find an optimization that eliminates 4ms. You're now running at 16ms (62.5fps). The framerate increased 25%, and the frame time went down 20%. Already there it's a bit unclear what number to quote. Gamers would probably quote 25% and game developers -20% (unless the game developers wants to brag, which is not entirely an unlikely scenario ). Now lets say the next day another developer finds another optimization that shaves off another 4ms. The game now runs at 12ms (83.3fps). The framerate went up 33% and the frame time was reduced by 25%. By looking at the numbers it would appear as the second developer did a better optimization. However, they are both equally good since they shaved off the same amount of work, and had the second developer done his optimization first he would instead have gotten the lower score.

Another thing to note is that it's never valid to average framerate numbers. If you render three frames at 4ms, 4ms and 100ms you get an average framerate of (1000/4 + 1000/4 + 1000/100) / 3 = 170fps. If you render at 10ms, 10ms and 10ms you get an average framerate of 1000/10 = 100fps. Not only does the latter complete in just 30ms versus 108ms, but it also did so hitch free, yet gets a lower average framerate.

The only really useful number is time taken. Framerate can be mentioned as a convenience, but any performance chapter in technical papers should write their results in milliseconds (or microseconds if you so prefer). There are really no excuses for using fps. These days there are excellent tools for measuring performance, so if the researcher would just run his little app through PIX he could separate the relevant cost from everything else and provide the reader with accurate numbers.

[ 10 comments | Last comment by Rob L. (2009-07-26 21:35:10) ]

IGN: The 10 Best Game Engines of This Generation
Wednesday, July 15, 2009 | Permalink

Well, Avalanche Engine is one of them.

[ 6 comments | Last comment by Agadoul (2009-07-23 10:10:19) ]

Bullshit!
Monday, July 13, 2009 | Permalink

I'm a big fan of the Penn & Teller Bullshit! show. They usually do a great job at exposing all kinds of bullshit for what it really is. The latest episode takes a jab at everything that video games are blamed for, despite lack of scientic evidence backing up such claims. Well worth watching.

Part 1 Part 2 Part 3

[ 10 comments | Last comment by Rob L. (2009-07-25 11:58:06) ]

Yet another particle trimmer update
Wednesday, July 8, 2009 | Permalink

In the last release I messed up atlases again, so now that's fixed.

I also added an option to optimize vertex ordering in the spirit of these findings. You provide your index buffer and it outputs the vertices in the order that minimizes the internal edges. This option will probably only have a measurable effect on small particles though. But it's a trivial thing to compute, so it doesn't hurt to do it.

Finally I've changed the input options to a more standard and practical format as the number of options have increased, so now you don't need to provide all parameters just because you wanted to change the last one, but instead you can for instance do this to only change maximum hull size:
> ParticleTrimmer imagefile 4 8 -h 40

With these changes I guess the tool is pretty much "final", unless I come up with a solution to the performance problem other than by reducing the number of edges in the original convex hull to a managable number.

[ 0 comments ]

Google to release an OS
Wednesday, July 8, 2009 | Permalink

This is an interesting piece of news. According to the official Google blog they are working on a new OS called "Google Chrome OS". This is excellent news. We seriously lack competition in the OS market where Windows dominates. I have always wanted Apple to release Mac OS X for PC. I'm pretty sure it would be quite successful and be able to give Microsoft a run for the money. But so far they haven't been willing to do so, because they still want to sell their hardware at premium prices. I know it's possible to run Mac OS X on a PC if you're willing to hack around and have the right hardware, but the people doing that are of course a very tiny fraction of the market.

It'll be interesting to see what Google's OS will be like. From the description so far it sounds mostly like a Linux kernel and a browser, which to me doesn't sound like an actual OS, but more like a customized Linux distribution slimmed down for netbooks. But we'll see what it really is when it's released. I for one hope that we'll see a real competitor on this market.

[ 6 comments | Last comment by Shaun (2009-11-02 05:09:54) ]

Particle trimmer updated
Tuesday, July 7, 2009 | Permalink

I have made another update to the particle trimmer. Previous revisions produced somewhat inaccurate results since it didn't take texture filtering into account. For most textures it doesn't matter much, but for low-res textures it can be a big deal. In this revision I've added a subpixel precision mode to deal with this problem (default is a 16x16 split of the area between four texels). As a result a lot more points are added to the convex hull and it now pretty much always has to apply the reduction to 50 vertices. If anyone has an idea of how to do the convex hull reduction other than by brute force I'd like to know. I haven't been able to google up any algorithms for that.

I have also made it handle edge and corner cases correctly.

[ 0 comments ]

Photos of photographers
Thursday, June 25, 2009 | Permalink

I saw this awesome gallery of photographers doing their thing. Fun stuff.

[ 0 comments ]

New gallery
Saturday, June 20, 2009 | Permalink

As usual I'm lagging behind with updating my galleries. The last one was from a year ago. So in an attempt to catch up I've now added another gallery. It's from last fall. So now I'm only about 7-8 months behind.

[ 3 comments | Last comment by Nadja (2009-06-24 11:29:39) ]

More pages: 1 ... 9 10 11 12 13 14 15 16 17 18 19 ... 21 ... 31 ... 41 ... 47