Microsoft's own speed tests show IE beating Chrome, Firefox

[CvP]

Microsoft has released its own tests that show IE8 can load many websites faster than two open source browsers: Firefox and Chrome


Just before announcing that Internet Explorer 8 has been finalized, Microsoft has released a new report titled "Measuring Browser Performance: Understanding issues in benchmarking and performance analysis." The document explains the various browser and network components and how each piece can impact performance when benchmarking, capabilities and limitations of various benchmarking tools, as well as ways to design tests to avoid these issues.


What makes this report extremely dubious is the chart that is buried at the bottom. It shows IE8 outperforming Firefox 3.05 and Chrome 1.0, something that no other test on the Internet has ever shown before.



Before I start to rant and rave, I want to explain that I am an IE8 user (in fact this post was written using IE8 on Windows 7), though I do use other browsers from time to time. IE8 is a vast improvement over IE7, especially in performance, but I will be the first to admit that it still does not compete with third-party browsers when it comes to speed. However, the table Microsoft provides shows the load times for the top 25 websites according to comScore, and IE8 does quite well.


The report notes that Microsoft "used the browser 'Done' indicator for timing when the page when the page is completely loaded at that point. For pages which continue to load and change after the 'one' indication we have used common visual cues to generate the timings. Timing is started when the Go button is pressed. These timings were captured in January 2009; because Internet content is always changing you may get different timings when you run these tests."






As you can see, IE8 outperforms Firefox 3.05 and Chrome 1.0 in loading 12 websites, Chrome 1.0 places second by loading nine sites first, and Firefox brings up the rear by loading four sites faster than the other two browsers. Also, in case you missed it, IE loads mozilla.com faster than Firefox, and Firefox loads microsoft.com faster than IE, just for kicks. This report is bound to stir up a lot of controversy, especially since Microsoft outlines its exact methodology. The report is available publicly at the Microsoft Download Center in PDF and XPS formats, so check it out if you have the time for reading through 14 pages. Microsoft chooses approximately 25 websites for daily testing, and tens of thousands on a monthly basis. If you're going to do your own tests, Microsoft emphasizes that "any list of websites to be used for benchmarking must contain a variety of websites, including international websites, to help ensure a complete picture of performance as users would experience on the Internet."



Microsoft obviously wants to get users who are using alternative browser to come and try out IE8, which is why this report is quietly being released just before the new browser becomes available this month. Is this the best way to go about it? The fact that this is the first test (at least that I'm aware of) to show IE8 loading multiple websites faster than Firefox and Chrome, and the top 25 websites no less, is very supicious. What do you think?




Microsoft has a promotional video up on microsoft.com that shows off these speed tests. Looks like the report may have been released quietly, but Microsoft is promoting the data from it quite strongly.

Finally, a REAL test. i'm sick of those bullshit "javascript testing" and saying this browser is
faster than that one (and that IE is like a turtle).

btw, #1/#16 and #4, $6 ... are really funny