The state of ethics in 3D graphics
Friday, May 23, 2003 | Permalink
We live in sad times. Around the time of the release of Radeon 8500 news spread on the web the
ATi applied cheats to gain performance in Quake3. SiS has been
busted occasionally for lowering IQ for performance in benchmarks. Not too long ago Trident was accused of
cheating too. Recently nVidia officially joined the crew when
their cheats was exposed. This time it went as far as to Futuremark having to issue a
statement and provide a
patch to disable the cheats and a
detailed report on the cheats applied. No less than 8 different cheats has been applied by the driver which added no less than 24% more performance. This puts the NV35 into a less pleasing light. Instead of being the chip that brings nVidia's floating point performance up on par with ATi the painful truth again after disabling the cheats looks to be that the floating point performance still is crap. At this time I also have no hope that any NV3x well ever perform well in that area, the hardware is just that slow, as patched 3dmarks proves and other applications has proven for some time. It should also be said that Futuremark found oddities in one test where ATi seems to get a small but suspicious performance drop with the patch applied. The jury is still out on that one though, but in all likelyhood the truth may be less beautiful than one might hope.
It's sad. It's really sad.
But as usual, there's something to learn. The lesson is NOT that 3dmarks is useless as a benchmark that some sites, which I refuse to name and link to, has implied. And it's absolutely not that one's cheating justifies another one's as endless fanboys are ranting. One may have have objective opinions on 3dmarks, but the real lesson is that reviewers need to spend more time looking at Image Quality and look for rendering anomalities. Reviewers need to break away from traditional benchmarks and look for alternatives. Benchmark games that are less known. Write your own benchmarking tools. I have little hope on the last one though given the enourmous amount of ignorance out there. Most review sites can hardly discern a pipeline from a pixel. The last, and maybe the most important point though is that we as consumers need to be aware of this. We need to demand that this does not happend. We all vote with our wallets. We much make it so that it's not worth it to cheat.
[
9 comments |
Last comment by Doomtrooper (2003-05-28 01:22:56) ]
