Faking Benchmarks, The Good Benchmark List, How To Spot Cheats and More...

[Kunaak]

sounds great to me, the idea is to keep the actual benchmark outta peoples hands so they can't cheat on it.

unfortunatly, I just no of no way to make a cheat free, windows based pi program.
they all work in the same basic way, and are vulernerable to the same basic cheats, and when done in small moderation, like only a second or two faster, IE-should be 40 seconds, but they cheat to make it 38 seconds.... then it's impossible to spot.

both super Pi and Pi fast, are extremely easy to cheat on, in literally any degree. a tiny cheat is impossible to spot, which makes it really hard to be excited about records done with these programs anymore for me

[eshbach]

well there will still be a downloaded program (~50k right now) that runs the math locally...

[racoontje]

Remind me, why don't we just use a normalized linux boot cd? It's really stable, you can use a ram disk w/ MD5'ed tarball (the MD5 is downloaded at runtime) and it sends data tor a server somewhere.

[Slickthellama]

you know, CS is having speed hack problems right now but in the opposite direction. People dont slow down windows they speed it up O_O. This makes them move extremely fast and reload very fast, shoot very fast. But cheating in benchmarks? why? It's rediculous. Good post Kunaak.

[Kunaak]

I am aware of cheats, that will allow you to reverse the speed of windows to super speeds. these cheats are meant to enable online cheating.
unfortunatly, I know of no way to disable, or stop these cheats.

I added a few new things to the list of good and useless programs.

for a PI program, I would highly suggest people look into "Cpu Mathmark"

I found only 1 method to cheat on this program, and it's not an easy one.
it's a fairly good solid program.
the details can be found in my list on the first page of good, or atleast decent programs you can use to verify OC's.

[racoontje]

Well, running programs off of remote shells (I'm assuming you're not going to be a dumbass and run telnet over the internet) will cause it to actually run remotely, not locally. What I would do:

• Make your own Gentoo Linux livecd using Catalyst (poke me if you need someone inaugurated)
• Make sure it has a very recent kernel, preferably with racoontje's closed source patches of joy (I've been replacing lots of low level kernel functions w/ ASM)
• Make sure it has all the modules you need, think ethernet cards
• Run a python script nohup (ie "disable all user intervention until program is done or user pulls power wire :P)

This script would:

• Check the MD5sum of the tarball that's on the CD
• Create a small (<1 MB) RAM disk where the tarball is extracted
• Runs a series of tests

Remember that 1 MB will get you a LOT of tests... ASM people can put a lot of processor torture in a few bytes

[IvanAndreevich]

If the quarz on the motherboard gets modded then how do we detect it?

[masterofpuppets]

Someone needs to write a decent superpi equivalent for unix.

[Kunaak]

NEWS

I am helping to beta test a pi program right now from a author here.
so far...

his work is very encouraging as I have not been able to cheat on it with any method I know of.
neither windows speed cheats.
timer cheats. clock cheats. pause cheats or manual cheats have effected this little program...

it's not perfect.
does have a few little bugs to fix.
but so far, the author has been very very fast about fixing this stuff.

if he were to release it, even in the buggy phase...
It would still be the only PI benchmark I have heard of, that I cannot cheat on I look forward to testing the next build, and will keep any news of the program, listed here, and hopefully in a few days have a version he's willing to release.

it's kinda like Hexus Pifast.
but has 2 versions, one for SSE, and one for SSE2.
nothing mentiond about a SSE3 version of prescotts.
but this is definatly, the best news I know of, for a benchmark, thats cheat free.

[bh2k]

NEWS

I am helping to beta test a pi program right now from a author here.
so far...

his work is very encouraging as I have not been able to cheat on it with any method I know of.
neither windows speed cheats.
timer cheats. clock cheats. pause cheats or manual cheats have effected this little program...

it's not perfect.
does have a few little bugs to fix.
but so far, the author has been very very fast about fixing this stuff.

if he were to release it, even in the buggy phase...
It would still be the only PI benchmark I have heard of, that I cannot cheat on I look forward to testing the next build, and will keep any news of the program, listed here, and hopefully in a few days have a version he's willing to release.

it's kinda like Hexus Pifast.
but has 2 versions, one for SSE, and one for SSE2.
nothing mentiond about a SSE3 version of prescotts.
but this is definatly, the best news I know of, for a benchmark, thats cheat free.

Sounds promising, but I think it should be kept a secret until all the bugs are worked out. Who knows someone could use one of those bugs to exploit the program.

[IvanAndreevich]

Kunaak
Excellent. Does it require internet connection?

[Kunaak]

no internet connection.

less then 100k download.
takes about 1 minute to run.

all big bugs, have been worked out.
only the little ones remain.
nit picking stuff.
like the time reported isn't accurate.
for me it reports 110 seconds.
but it really takes 40 seconds.
easy fix I think, since the first version had the proper time down.

just the guys off in never never land dreaming away till tommorow...
time for patience to kick in.

[Killeroy]

Here's a classic 3d2001 fake:



Obtained by yet another method not mentioned here.
Let's just say it involves changing the reference clock oscillators on the mainboard. This would be a hack that is very hard to detect if done in a less obvious way then shown above. The only method to prevent that would be to validate the system clock with a verified time server on the internet.

Killeroy.

[ImmortalNoob]

can someone check this screenshot

[jjcom]

whats wrong? I few of these latest Winchesters can do things like that. Besides theres a thread for this...something about post here to verify a CPU-Z shot or something

jjcom

EDIT: I spelled "jjcom" wrong lol

[ImmortalNoob]

serious. 2500 at nearly 1.4v? ok

[Karnivore]

serious. 2500 at nearly 1.4v? ok

checksum does matches, however, older CPUZ is to easy to cheat with, hence XS Staff' Official "we won't check pre CPUZ 1.26 screenies" statement. But anyway, Yes there is a thread for checksum verification, sticky in XOC.

Chip isn't really all that spectacular, I've actually never even bothered to see what my NC which does 2650 air, and 2.5 at default voltage will run undervolted, but it wouldn't surprise me to see similar results.

[Tom Holck]

checksum does matches, however, older CPUZ is to easy to cheat with, hence XS Staff' Official "we won't check pre CPUZ 1.26 screenies" statement. But anyway, Yes there is a thread for checksum verification, sticky in XOC.

Chip isn't really all that spectacular, I've actually never even bothered to see what my NC which does 2650 air, and 2.5 at default voltage will run undervolted, but it wouldn't surprise me to see similar results.

Well, tried to lover volt at 2500 Mhz

-And the temp? -132 C

[2fink]

Well, tried to lover volt at 2500 Mhz

-And the temp? -132 C

wow very nice... undervolting is very good for barebone user like me! my cpu is much colder with 1,1 than with 1,525.

[IvanAndreevich]

Tom Holck
Killer! Is it stable at those speeds with those voltages?

[Tom Holck]

Tom Holck
Killer! Is it stable at those speeds with those voltages?

Yes, Pifast and M1 tried. It's not the CPU that is special. It's because of the supercooling. No big dial

[SAE]

Great thread here, Kunaak... :thumbsup:

Didn't know people would bother to use apps like this to fake their results (only knew of Photoshoppers). Didn't even know of apps like this

And again, I learnt something new.

Ah, have to add something here:

Prime95. A friend of mine accidently found out, there is an easy method to cheat. If you are running the torture test and you put your system in suspend mode, Prime will count the time on and on. Restarting the rig you will get (after stopping the torture test of course) a result as if you were running it for several hours without problems and w/out any error. And your system did not even sweat doing it

Dunno if it still applies to the actual version though, I think it's been 6months ago...

[unregistered]

Just a little FYI, there are other ways of cheating:

http://www.lostcircuits.com/cpu/P4-373e/4.shtml and following pages.

In this case, it actually turned out that the main thing causing this was the termination voltage setting on the mainboard, increasing it countered the heat effect on the clock gen and completely negated the "cheating".

Apparently, the same issues we found were observed by others as well and as so often, more eyes see more and, therefore, it is not surprising that we got some feedback on the issue as well along with a number of suggestions. Among the more interesting suggestions was the one to set the FSB termination voltage to a fixed value of 1.3V, supposedly, this would completely take care of the problem. Needless to say that we were rather curious ourselves.

( http://www.lostcircuits.com/motherboard/asus_p5ad2e/ )

As mentioned in this thread before, 3DMark2001 was one of the few benchmarks completely immune

[Kunaak]

that was a very interesting article, and a very strange way to cheat a benchmark, but I am glad to know it, and how easy it can be to manipulate a 915 or 925 board.

[JNav89GT]

Edit! LOL seems I was a bit too late with LC linkage

If this has been discussed please forgive me, that is 925XE chipset PLL issue. This screenie is of a test system I assembled for DDR2 testing purposes. I have no cheat programs at all, I think the whole idea of cheating is horrible. Anyway, the following screenie is of an Abit AA8XE 925XE chipset board, I have fluctuating PLL after 270fsb where under stress system/PLL downclocks bus. There is NO CPU throttle at all, this is the bus causing lowering of clocks. This in effect causes the pause or stutter cheat. Michael Schutte covered this in a Lost Circuits article people should read http://www.lostcircuits.com/cpu/P4-373e/8.shtml

Anywho, take a gander! All P4 775 results at high bus speed in non verified benches are suspect to me at this point

Also, during benching I can see cpu/bus speed fluctuate wildly while no throttle is recorded by throttlewatch.