Relative Performance of Rich Media Content across Browsers and Operating Systems
Two of the things that Flash is often criticized for is that it:
- Uses too much CPU
- Performs significantly worse on the Mac than on Windows
This got me thinking about whether some quick tests would bear this out, and if so, whether it was isolated to just Flash content.
Below are some raw numbers showing CPU usage of various rich content across different browsers and operating systems. This includes video deployed via HTML 5, JavaScript / Canvas animations, Flash Video, and Flash animations. This is by no means a scientific study, but I do think the results show that:
- Flash does not necessarily perform worse on Mac as opposed to PC
- All rich media content, including that created with JavaScript / HTML 5 content show (sometimes widely) varying levels of performance across browsers and operating systems
All numbers below show CPU usage as a total of all CPU resources available on the system. This means that if you are running on a machine with more than one core processor, then CPU usage can be over 100%. This is how Mac shows CPU usage, and I feel it gives a clearer picture of how much CPU any individual item is using. Windows CPU usage have been normalized to show CPU usage in terms of this overall CPU usage.
All tests were run on the following hardware:
| Model Identifier: | MacPro3,1 |
| Processor Name: | Quad-Core Intel Xeon |
| Processor Speed: | 3 GHz |
| Number of Processors: | 2 |
| Total Number of Cores: | 8 |
| L2 Cache: | 12 MB |
| Memory: | 8 GB |
| Bus Speed: | 1.6 GHz |
I have both a Windows 7 and OS X partition on this computer, and Windows 7 tests were run while booted into Windows 7.
For browsers, I used the latest release browser versions. For Flash Player, I used the latest labs release (Flash Player 10.1 Beta 3 : 10,1,51,95).
Mac OS X Versions
| Mac OS X | 10.6.2 |
| Google Chrome | 5.0.307.9 beta |
| Safari | 4.0.4 (6531.21.10) |
| Firefox | 3.6 |
| Flash Player | MAC 10,1,51,95 |
Windows 7 Versions
| Windows 7 | 7600 |
| Google Chrome | 4.0.249.89 (38071) |
| Safari | 4.0.4 (531.21.10) |
| Firefox | 3.6 |
| Internet Explorer | 8.0.7600.16385 |
| Flash Player | WIN 10,1,51,95 |
IMPORTANT, THE ONLY RELEVANT COMPARISONS ARE BETWEEN PERFORMANCE OF THE SAME CONTENT ACROSS DIFFERENT BROWSERS AND OPERATING SYSTEMS. COMPARING FLASH CONTENT TO JAVASCRIPT CONTENT IN THIS TEST IS NOT VALID GIVEN THE DIFFERENCES IN THE CONTENT AND / OR FUNCTIONALITY.
Below are the pages / content run, and the amount of CPU they took to run. Lowest CPU usage is highlighted in green, and highest CPU usage is highlighted in red. I have also included a platform delta line, which shows the range in performance between the high and lowest CPU usage per operating system. Note, that as time passes, the content at some of the URLs may change, and not reflect the content tested.
Note: I am looking into why Internet Explorer is reporting 0% CPU usage on some of the JavaScript and Flash examples.
MacHeist Dynamic Canvas / JavaScript animation example
| Mac | Windows | |
|---|---|---|
| Google Chrome | 95% | 80% |
| Safari | 25% | 80% |
| Firefox | 100% | 40% |
| Internet Explorer | NA | 0% |
9elements JavaScript / Canvas Dynamic Animation Example
| Mac | Windows | |
|---|---|---|
| Google Chrome | 100% | 96% |
| Safari | 98% | 104% |
| Firefox | 100% – 170% | 72% |
| Internet Explorer | NA | Did Not Work |
Sublime HTML 5 HD Video Example
| Mac | Windows | |
|---|---|---|
| Google Chrome | 85% | 24% |
| Safari | 21% | 104% |
| Firefox | 85% | 80% |
| Internet Explorer | NA | Did Not Work |
HYPE Dynamic Flash Animation Example
| Mac | Windows | |
|---|---|---|
| Google Chrome | 25% | 16% |
| Safari | 17% | 48% |
| Firefox | 23% | 18% |
| Internet Explorer | NA | 0% |
| Mac | Windows | |
|---|---|---|
| Google Chrome | 88% | 56% |
| Safari | 58% | 64% |
| Firefox | 70% | 64% |
| Internet Explorer | NA | 104% |
Overall Comparative Performance (shows how often each platform performed the best per content type)
| Mac | Windows | |
|---|---|---|
| HTML / JavaScript | 3 | 6 |
| Flash | 2 | 4 |
| All | 5 | 10 |
I am not going to draw any definitive conclusions from this, but I do have some observations based on the results above.
- From these tests, Flash content does not perform consistently worse on Mac than on Windows.
- There is a wide range of CPU usage for HTML 5 video, depending on the browser / operating system it is being played back on, with Mac generally being slower.
- Canvas / JavaScript animations (at least those tested) seem to have high CPU usage, and generally run slower on Mac than on Windows (although not in all cases).
- Some of the HTML / JavaScript content would not run across all browsers.
- There seems to be some bug in the tested Flash Player when playing back video in Firefox. CPU usage would climb very high, and then drop. (I have reported this to the team).
- Availability of hardware acceleration for playing back video looks like it makes a huge difference in CPU usage (duh!).
- Regardless of technology (Flash and JavaScript / HTML), performance of rich media can vary widely depending on which browser / operating system it runs on.
AGAIN, AS I STATED BEFORE THE ONLY RELEVANT COMPARISONS ARE BETWEEN PERFORMANCE OF THE SAME CONTENT ACROSS DIFFERENT BROWSERS AND OPERATING SYSTEMS. COMPARING FLASH CONTENT TO JAVASCRIPT CONTENT IN THESE TESTS IS NOT VALID GIVEN THE DIFFERENCES IN THE CONTENT AND / OR FUNCTIONALITY.
Please keep comments on topic. Off topic comments will be moderated / deleted.
Update (March 10, 2010) : You can find similar (but separate) tests and discussions on this topic here and here.
Update (March 11, 2010) : Updated to make it clearer which Flash Player version was being used.
9elements JavaScript / Canvas Dynamic Animation Example
Firefox 100% – 170%
How do you get 170%?
Henry Ho
1 Mar 10 at 12:25 pm
@Henry
–
How do you get 170%?
–
That is because I tested on a multicore machine (8). 170% means it used 100% of one core, and 70% of another core.
mike chambers
mesh@adobe.com
mikechambers
1 Mar 10 at 12:27 pm
Frame rate tests would be helpful; in my experience the frame rate of identical Flash-player projects varies widely between browsers and platforms, with Mac OSX consistently showing a noticeably and measurably lower frame rate.
Rob Eberhardt
1 Mar 10 at 12:31 pm
I would prefer a normal particle system instead of the Hype example (without drawing into BitmapData). That would give some information about vector rendering performance.
Rumori
1 Mar 10 at 5:33 pm
Well to be fair, you should have compared the performances with Flash player 9 and 10.
As 10.1 is just released and the focus was on improving performances on Mac, yes now it is fine, no doubt.
Flash player bad performances on Mac were quite old already.
Good thing it is fixed.
Anyway HTML 5 canvas will also create huge performance problems once it will be massively used and poorly animated coded. Then Flash would probably be highly recommended… this would amuse me :)
Baz
1 Mar 10 at 7:53 pm
Independendly from the JS vs Flash comparison, it is quite interesting to see that Safari performs much better under Mac OS, IE under Windows and cross platform developped browsers performaces are quite disappointing (especially Firefox).
It seems Microsoft has to work better on video rendering, but it is quite astonished to see the 0% CPU usage in JS animation while Google Chrome takes 80%. I thought it was a mistake and i could check by myself.
So it looks like, the recommended browsers should be the one developed by the same manufactor as the OS for better performances.
The gap is so high that I really wonder where this can come from. If anybody has any idea?
Baz
2 Mar 10 at 5:58 am
Yeah. Im looking into the IE o% CPU thing. I think something weird is going on there.
mike chambers
mesh@adobe.com
mikechambers
2 Mar 10 at 10:18 am
I’ve noticed that since version 3.6 has come out firefox has been using a lot more cpu for everything (on windows) and for the first time in years I’ve had the browser crash because of flash content. There is something wrong in the latest firefox.
david doull
5 Mar 10 at 7:30 pm
Interesting comparisons, I can’t help but wonder how this translates on run-of-the-mill average hardware tho since you’re using a pretty beefy setup (eons ahead of what the normal joe is using).
What video card do you have? Since Flash 10.1 uses hardware acceleration (finally), that bit of info seems relevant.
Jones
7 Mar 10 at 7:48 am
[...] this article informal research seemed to show that, in some circumstances, safari (the browser in iPad) was the [...]
Is Flash a CPU hog? « flash 4 ipad
8 Mar 10 at 10:02 pm
[...] Relative Performance of Rich Media Content across Browsers and Operating Systems Share Tags: flash | html5 | iPad Comments (0) [...]
Delvinia - Flash vs. HTML5, and the iPad «
10 Mar 10 at 11:31 am
The winner in the HYPE example is wrong.
anon
11 Mar 10 at 10:38 am
Which Safari Processes did you include on OS-X? I see 3 at the moment.
David Illsley
11 Mar 10 at 10:43 am
If Flash doesn’t perform worse on OSX, why do my fans spin up the minute that a Flash video is played or a Flash applet or game is run. I can detect the presence of Flash just by audible cues… Not HTML5, not even Silverlight exhibits this behavior. I’m not saying your methodology is flawed, just that you are missing something of the real world in your tests.
John.B
11 Mar 10 at 10:50 am
@anon
–
The winner in the HYPE example is wrong.
–
Good catch. I fixed it.
In cases like that where it is so close, I probably shouldnt even mark one as performing test best.
mike chambers
mesh@adobe.com
mikechambers
11 Mar 10 at 10:55 am
@David
–
Which Safari Processes did you include on OS-X? I see 3 at the moment.
–
I used the one titled:
Flash Player (Safari Internet plug-in)
(btw, I only see one Safari process and one Flash Player Safari process in my tests).
mike chambers
mesh@adobe.com
mikechambers
11 Mar 10 at 10:57 am
MacHeist is an inaccurate example because it used CSS transforms on webkit browsers and JS animation on everyone else. I suspect that you would see much higher CPU numbers if CSS transforms weren’t being used.
Stephen
11 Mar 10 at 11:00 am
@John B
–
If Flash doesn’t perform worse on OSX, why do my fans spin up the minute that a Flash video is played or a Flash applet or game is run.
–
There is nothing in this post that asserts that Flash Content cannot use a lot of CPU. It can, the same as just about any other type of content can.
mike chambers
mikechambers
11 Mar 10 at 11:03 am
@Stephen
–
MacHeist is an inaccurate example because it used CSS transforms on webkit browsers and JS animation on everyone else. I suspect that you would see much higher CPU numbers if CSS transforms weren’t being used.
–
But isnt that the nature of doing HTML / JavaScript development? Having to deal with the differences in feature implementations between browsers (although the libraries do make this much easier now).
mike chambers
mikechambers
11 Mar 10 at 11:05 am
@Mike
Thanks.
The other one I saw was “Safari Webpage Preview Fetcher”. It seems to disappear after a while, so it seems reasonable to omit it.
I’m not so sure about omitting the Safari process. On the Vimeo example the Flash process varies between 60-70% for me and the Safari process sticks around 20%. That’s a fairly substantial cost which I guess shows up in the Firefox result, but which Safari is getting a pass for.
David Illsley
11 Mar 10 at 12:11 pm
I never have a problem with Flash on my quad-core Xeon machine, but Flash in Safari on my G5 machine is a dog. Fine on a fresh restart, but video is unwatchable after a couple hours of uptime, while flash video on Firefox on the same machine is always smooth.
wjcstp
11 Mar 10 at 12:29 pm
Thank you for putting this information together. It helps inform the discussion regarding javascript/flash and browsers. However, I couldn’t follow the logic behind your conclusion table. When I just added up the percents from the various tasks for each browser and then averaged them, this is what I got:
Mac Safari: 43.8%
Windows Chrome: 54.4%
Windows Firefox: 54.8%
Mac Chrome: 78.6%
Windows Safari: 80%
Mac Firefox: 89.6%
What is obvious to me in looking at these numbers is that each browser is really good in one OS and kinda sucky in the other. This makes me wonder why so many Mac users use Firefox.
Justin Garrity
11 Mar 10 at 12:32 pm
Where’s Opera 10.50? It would be interesting to see – they’ve switched to Cocoa on Mac, added acceleration on Windows, and have smokin’ fast canvas implementation.
kwer
11 Mar 10 at 1:32 pm
Flash may PERFORM as well as it eats more CPU. The user experience on a machine with excess capability will be fine even if the CPU has to work harder.
This is a problem in two cases
1) Battery operation, more CPU usage = shorter battery life
2) slow computers, they can peg out the CPU, THEN the user experience is impacted.
- gws
George
11 Mar 10 at 2:07 pm
Maybe there is a larger dispersion between Mac OS X and Windows 7 performance when using a lesser machine. I’m sure most of the anecdotal evidence of “Flash is bad on the Mac and not on the Windows box” is coming from mediocre spec’d systems relative to your test case.
Mark
11 Mar 10 at 2:11 pm
@George
–
Flash may PERFORM as well as it eats more CPU. The user experience on a machine with excess capability will be fine even if the CPU has to work harder.
–
Yes, and the same applies to other content, such as HTML 5 / JavaScript based content.
mike chambers
mikechambers
11 Mar 10 at 2:21 pm
@Mark
–
Maybe there is a larger dispersion between Mac OS X and Windows 7 performance when using a lesser machine.
–
Yeah, there could be. Im hoping to do another, more extensive round of tests with a lower end MacBook Pro and / or MacBook.
mike chambers
mesh@adobe.com
mikechambers
11 Mar 10 at 2:22 pm
Great objective test. Thanks.
As for tests on low end hardware, if you had access to an original Intel Mac Mini (e.g. core solo 1.5 or the dual 1.6) that would be a good test provided it was loaded with enough ram to take that out of the equation. Or a 1st gen Macbook air, or early macbook. Something with old basic Intel GMA graphics. The 9400M’s in current Macbooks have hardware H.264 acceleration under OS X.
Rowan
11 Mar 10 at 2:59 pm
Mike,
CPU utilization comparisons are interesting but tell only part of the story. The real issue is power consumption, of which CPU utilization plays only a fractional part. Flash may offload certain computational tasks to a GPU on certain platforms, which makes CPU utilization appear smaller thereon. But powering the GPU is not free.
More useful would be an analysis of relative power consumption. This is what is truly driving the debate about Flash in the mobile environment, not CPU utilization per se.
Regards,
–Michael
Michael Fischer
12 Mar 10 at 12:13 am
Thanks for your effort. Two observations:
It would make more sense to average CPU-usage across all tests than to declare winners and losers. 16% vs. 17% is very close.
The 9elements animation produces around 90%-100% CPU usage in Safari on my machine, the WebKit nightly only around 50%-60%. Interesting progress.
AceMcLoud
12 Mar 10 at 3:02 am
[...] full of security holes, and a CPU hog has demonstrated that Jobs was totally wrong. As a couple of unbiased and totally scientific tests by people with a vested interest in Flash (oops, that link is to “unscientific” tests [...]
inconsequence › Flash Vindicated. Apple Evil.
12 Mar 10 at 2:28 pm
[...] also links to additional tests indicating that Flash “does not perform consistently worse on Mac than on [...]
A Skeptical Comparison of HTML5 Video Playback To Flash | JetLib News
14 Mar 10 at 2:27 am
The critique isn’t so much between OSX and Windows as it is the fact that it’s a locked format which doesn’t perform consistently between platforms.
Also, as others have said, just looking at cpu cycles instead of actually caring about how it affects your system is faulty.
Try sitting with a macbook in your lap and watch youtube videos, excepting a very cold room, this isn’t exactly a pleasant experience.
I can watch the same videos on my slower vista laptop without noticing any real heat difference.
Also, the choppiness in both linux and OSX is a real issue when using flash.
Klas
14 Mar 10 at 2:57 am
I’ve found a lot of cpu usage in any browser particular when doing processor intensive tasks like slideshow views, even with web optimized image’s.
To my experience this cases slow & annoying switching between webpages. Javascript libs like jquery and extjs which can do intensive tasks can be seen getting disrupted etc. example:
http://www.internaat-aan-zee.be
(slidehow takes al cpu – jquery marquee doesn’t animate properly)
So it does seem to me that Flash is taking all cpu power with the rest of the browser functionality left to die. Maybe a better multitasking solution were cpu usage is better distributed between tasks would be a solution?
Mike?
Regards,
Chris.
chris devriese
15 Mar 10 at 5:24 am
[...] data for the comparison with HTML5 and Flash can be found here Comparison data here. AKPC_IDS += [...]
HTML5 vs. Flash | LogicLounge
16 Mar 10 at 7:06 pm
I did several similar tests myself trying to figure out why all these Mac Fanboys were saying Flash Player is crap.. I was shocked, relieved and mad all at once when I loaded similar animations in actionscript and javascript from Mr. Doob’s site and Flash/AS outperformed HTML5/JS during intense multiple object animations. HTML5 was almost double CPU usage at times. AS also displays physics sooo much more smoothly than JS currently does it’s laughable for anyone to even compare.
My conclusion is that these ‘Apple Fanatics’ are exaggerating the problems to justify the insanity that Jobs is trying to sell as the ‘ultimate web experience’. They’ve gone mad I tell you.. and you cannot reason with them because their heads are so far up their asses they won’t even hear you. :D
_mark
18 Mar 10 at 12:11 pm
re: 9 Elements example.
They mention in the blog it is an Flash/as port. Anyone have a link to the original version?
JP
24 Mar 10 at 5:55 pm
Mike,
Have you used Process Explorer in Win7? It’s much, much better than the default Task Manager, and might help with that IE8 load, as it shows processes in tree, and lots of other useful stuff taskman doesn’t.
m.
analytik
28 Mar 10 at 2:33 pm
[...] Update: Mike Chambers has performed some additional tests that he says shows that “does not perform consistently worse on Mac than on Windows.” Check out the complete results here. [...]
alfdesign blog/creative lab » Blog Archive » HTML5 vs. Flash comparison finds a few surprises
30 Mar 10 at 9:34 am
ok i just followed your two hd video links
i’m on a g4 macbook pro
mac os 10.4.11
Safari 4.0.4 [i think there is a newer version available]
Flash 10.0 r12
and i have 5 open apps
and a download going in another non-browser app
the html5 plays fine
the flash link gives me shuddering bursts of audio with pauses in between with no video
i use menumeters to monitor cpu use
the flash video pegged it [compared to 40% with html 5]
this is not uncommon
i routinely find my system crawling to a halt if i leave a web page with flash conten open in the background
i am no supertechie but my experiences with flash are almost all negative [too many to list]
i won’t miss it at all
ps
do you work for adobe
that’s my real world example
ed andrews
16 Apr 10 at 4:49 pm
@ed andrews
–
i’m on a g4 macbook pro
mac os 10.4.11
Safari 4.0.
–
Yes, that is consistent with my tests above. Currently Flash on Mac running in Safari is not hardware accelerated, while H264 is. That is why you are seeing the discrepancy.
–
do you work for adobe
–
Yes, as is stated in the about section.
mike chambers
mesh@adobe.com
mikechambers
20 Apr 10 at 3:19 pm
@ed andrews
> i’m on a g4 macbook pro
No you aren’t. Such a thing does not exist.
Matt
22 Apr 10 at 12:42 pm
@Matt
>> i’m on a g4 macbook pro
> No you aren’t. Such a thing does not exist.
sorry, you’re right
it was a hypographical error
should have said “PowerBook G4″
i must have been caught up in the reality distortion field
created by my lust for the new mbpro’s
ed andrews
22 Apr 10 at 4:05 pm
one more thing
following my google search on this topic i discovered the html5 beta on youtube
i switched and youtube is usable again
i think proprietary technology on open access media is a mistake
it gives us unusable web pages created by clueless ms tools, crappy video, and process-hungry spam
processor time is processor time whether cpu or gpu
i do enjoy cute e-cards and games tho
ed andrews
22 Apr 10 at 4:29 pm
I’m at a loss. All these people saying “my Mac grinds to a halt every time I visit a webpage with Flash”. I’m on a 13″ Macbook Pro (not exactly the most powerful Mac) and I NEVER experience this!
As a Flash developer, maybe it’s because I normally visit sites created by developers with half-a-clue about how to develop properly.
It’s true that a badly developed Flash website can eat into your processor but I can’t imagine HTML5, when the masses start developing complex applications with it, won’t experience the same problems. We’re seeing interesting HTML5 demos which perform well around the web now but you have to realise these are being developed by HTML5 *experts*. We’ll see how your performance analysis tools average out when a wide array of HTML5 developers start implementing it in their websites…
Dave
23 Apr 10 at 5:10 am
@ed andrews
–
i think proprietary technology on open access media is a mistake
–
You do realize that youtube is delivering the HTML 5 video via a proprietary technology? (H.264)
–
processor time is processor time whether cpu or gpu
–
Sure, and when the video is hardware accelerated, as it currently is with h.264 on Safari, that processor time is on the GPU. When it is not hardware accelerated (as is the case with Flash on Mac), then that processor time is reflected via the CPU.
mike chambers
mesh@adobe.com
mikechambers
23 Apr 10 at 10:30 am
[...] by Apple. There have been many tests run on this subject, and below is a portion of a test run by Mike Chambers that portrays the amount of CPU used within each browser. He has tested each CPU time in both Mac [...]
HTML5: Worth the Hype? | Web Design | instantShift
26 Apr 10 at 6:28 am
[...] by Apple. There have been many tests run on this subject, and below is a portion of a test run by Mike Chambers that portrays the amount of CPU used within each browser. He has tested each CPU time in both Mac [...]
TG Developer » HTML5: Worth the Hype?
27 Apr 10 at 2:45 am
[...] to (and in many cases significantly more than) similar Flash content. For example, I posted some benchmarks of CPU usage for for some popular HTML 5 / Canvas examples. Lets look at the [...]
Top Flash Misperceptions : Flash is a CPU Hog at Mike Chambers
10 May 10 at 12:57 pm
As i look the results it seems obvious that Canvas animation with Javascript are useless comparing to flash capabilities now and CPU usage. Video will be played with HTML 5 obviously.
Alex
11 May 10 at 11:54 pm
[...] You can look for details of the findings on Mike’s popular blog http://www.mikechambers.com/blog/2010/03/01/relative-performance-of-rich-media-content-across-browse… [...]
HTML 5 Vs. Flash « Rthoughtindia's Blog
4 Aug 10 at 7:30 am
Hmmm, this is obviously a pro apple test. Again, as I said before. I have used a laptop with a 700 mhz p3 with 512 ram, and flash had NO TROUBLE! it ran perfect!
Stop supporting apple for overpricing their hardware and grow up. It’s time to do that.
Also, flash is much more then a website tool, it’s an advertising, content, etc…tool that won’t go away. Flash was made for jailbroken ipods/iphones and they work perfect without much battery sacrificed.
pdiddles03
7 Jul 11 at 8:02 am