I ran GTA3 in 1024x768x32 resolution and using a program called "fraps" I was able to monitor the frame rate and even take screen shots. In both drivers sets, using the original file name the game ran quite admirably slowing down only on very complicated scenes. However, when I renamed the file the frame rate dropped almost in half and image quality was considerably sharper. The pattern was pretty much the same as it was for 3dmark.

Unfortunately renaming the exe caused some sort of problem with GTA3 and I was getting some weird transparent image overlay on the top of the game screen so screen grabs were useless - but it was still obvious to me that it was cleaner underneath. The sheer fact renaming it caused problems with the game shows there must be a considerable difference in the way the card works without these "optimizations" on.

So, Trident are definitely "optimizing" both games and benchmarks for better performance and at the expense of image quality. Our initial reaction might be that of disgust and that they are cheating. But I don't think this is quite the appropriate stance to take.

Firstly, if I was given the option I would take the slightly worse image quality and 5400 3d marks over the 2600 3d marks and good image quality any day. But I think the emphasis here should be if I was given the option. Hiding the fact these optimizations are present does their reputation no good at all. All it would take is a simple control panel option with a slider offering varying quality or performance settings and I think 99% of people would probably be quite happy to put up with the lower image quality anyway - but being given the choice is the important bit.

As far as optimizing specifically for 3dmark, an optimization should increase speed with a lossless effect in image quality. I.e, get the same work done with less effort. If there is an image quality loss, this is cutting corners instead of actually doing the job and this is cheating.

But looking globally across the 3D market, I'm sure we can all remember the incident a while back with ATI when renaming the Quake3 executable gave a much lower score due to "optimizations". And of course, the more recent situation with Nvidia's optimizations for 3dMark2k3 and the GeforceFX cards (press release here). And as Xbitlabs have shown so well, even SIS aren't innocent when it comes to their Xabre line of cards - see here.

What worries me the most is how "optimizations" like this can slip through the net so easily when benchmarking. We recently gave the Xabre 200 quite a rave review - perhaps if we had known about the image quality loss that Xbitlabs uncovered our opinions would have changed.

If Trident had managed to cover up the 3dmark optimizations as well as they have in the newest drivers, would we even be having this article now? Without making unfounded accusations, it is quite possible that many games and benchmarks are already being optimized for without our knowledge.

I think the industry needs a big shake up when it comes benchmarks. There NEEDS to be a fair comparison.

<< Back | 1 | | 2 | | 3 | Next >>