A few more browser speed tests

A bit more than 4 months after my previous browser speed test, I thought it was time for an update with the latest versions. Before going further I shall remind you that browser speed benchmarks are meaningless – really, they are. Yet it seems that people love them, so… I do run them sometimes, just for the show…
NB: if you want the results for Opera 11.50 preview and Iron 10 (should be like Chrome 10), see this addendum.

Materials used

I used the same hardware, except for the RAM, which I upgraded recently, so: AMD Phenom 2 X4 Mobile N930, 8 GiB DDR3 RAM, ATI Mobility Radeon HD 5650 1 GiB VRAM.
On the software side, I used today’s Futuremark Peacekeeper and SunSpider (version 0.9.1).
The computer was running on Windows 7 x64 Enterprise with ATI Catalyst 11.2.
The exact browser versions were: SRWare Iron 9.0.600.2, Opera 11.01, Firefox 4.0 RC1 (RC candidate built on 4 March 2011), MSIE 9.0 beta (9.0.7930.16406), Safari 5.0.3

Results

SunSpider

SunSpider is a JavaScript benchmark. Only JavaScript, just JavaScript. Also, note that it was originally developed by Apple’s WebKit team… so maybe WebKit-based browsers (here, Safari and Iron/Chrome) are a bit privileged. The result is the time taken to run all tests, so obviously lower is better.

1. Iron 9: 388.2 ms (100%)
2. Opera 11.01: 400.4 ms (103.14%)
3. Safari 5.0.3: 413 ms (106.39%)
4. Firefox 4 RC: 527 ms (135.75%)
5. MSIE 9 beta: 588.7 ms (151.65%)

Peacekeeper

Peacekeeper was developed by Futuremark, well-known for their 3DMark series of PC graphics benchmarking programs. Like SunSpider, Peacekeeper benchmarks JavaScript, but it tests a wider range of features, notably DOM operations.
The result is a score pretty much like 3DMark/PCMark scores, so higher is better.

1. Iron 9: 6405 (100%)
2. Opera 11.01: 5363 (83.73%)
3. Safari 5.0.3: 3330 (51.99%)
4. Firefox 4 RC: 3186 (49.74%)
5. MSIE 9 beta: 1998 (31.19%)

Interestingly, compared to the previous test I ran in October 2010, all browsers but Iron saw their score decrease… So now Iron and Opera aren’t tied anymore: in Peacekeeper, Opera 11.01 is 16% slower than Iron 9. And the other browsers are really behind (Fx and Safari are 50% slower, MSIE 70%).

Addendum: Microsoft Mr Potato Gun

Often, a few hours after I posted I new entry, I like to do a few related searches in Google to see how well it’s placed. At this occasion, I came across very weird results on Conveivablytech, with MSIE 9 RC beating Firefox 4 RC by 10% at SunSpider, and by 200% at a Microsoft-specific test, Mr Potato Gun (from Microsoft Test Drive Site). At first I thought why not, maybe MSIE 9 did huge progress between version 9 beta and 9 RC, but still, that 200% difference seemed a lot, so I decided to run this test too. And here are the results (higher is better):

1. Firefox 4 RC: 15051 (100%)
2. MSIE 9 beta: 9242 (61.4%)
3. Safari 5.0.3: 2851 (18.94%)
4. Iron 9: 1755 (11.66%)
5. Opera 11.01: 1023 (6.8%)

So… MSIE doesn’t exactly beat Firefox on this one. Now, let’s accept the hypothesis that MSIE 9 did a huge improvement between beta and RC (climbing from 40% slower than Firefox to 200% faster), there’s still something weird about Conceivablytech’s results: why does my Firefox get the same score as theirs on a clearly weaker machine? They get 15703 on a good desktop (Phenom 2 X6 1055T (3.3GHz with Turbo) + Radeon HD 5870), I get 15051 on a cheap laptop (Phenom 2 X4 Mobile N930 (2 Ghz no Turbo) + Mobility Radeon HD 5650)… If their results were correct, I should get maybe 7000 to 9000, certainly not 15000. So, most likely their results are flawed.
On a side note, the reason why I didn’t upgrade to MSIE 9 RC is because I read somewhere it might create issues on my Windows version. It sure would be easier to just run the test myself on MSIE 9 RC but I really don’t want to take the risk of screwing my PC right now. Of course I’ll make sure to run those tests again when MSIE 9 gets final, so we’ll see then how false (or true?) Conceivablytech’s results were. 🙂
I don’t think the other browsers’ results need commenting. Obviously Mozilla and Microsoft worked to optimize their browser for that benchmark, while the other vendors didn’t. Simple as that. So, once again, browser benchmarks are meaningless.

Quick Internet Explorer 9 final update

I’ll release more detailed benchmarks “later” (ie, when I find the time…), but I have some preliminary results with MSIE 9 final:

• massive improvements on SunSpider (some else’s been doing their homework too :D): 394.6ms (beta was 588.7 ms)
• large improvements on Peacekeeper too: 3362 (beta was 1998)
• confirmed absence of improvement on Mr Potato Gun: 9350 (beta was 9242). Don’t get me wrong: this is a good result. But it confirms that Conceivablytech’s results aren’t very… accurate, to say the least

Update (2011-03-10): added Mr Potato Gun
Update (2011-03-21): added a few lines about MSIE 9 final