Flash Player content, Mouse Events, and Touch input
So, the Interwebs is all a buzz again around the latest article that “proves” that Flash will not be useful on mobile devices (much less the iPad). From the article:
Current Flash sites could never be made work well on any touchscreen device, and this cannot be solved by Apple, Adobe, or magical new hardware.
That’s not because of slow mobile performance, battery drain or crashes. It’s because of the hover or mouseover problem.
Basically, the writer is arguing that because some Flash content uses Mouse over / hover events (MouseEvent.MOUSE_OVER), and because there is no such event as a hover on a touch devices, then apparently, most Flash content just flat out wont work on touch devices.
I wanted to make a quick post that corrects some of the false assumptions and conclusions presented in the article, which include:
1. Hover events do not work in Flash content on devices with touch screens. That is false, they do work.
2. Any issue with hover or any other mouse events on a touch screen device, are specific to Flash. They are not. They affect HTML and any other web content as much as Flash.
3. Most Flash content relies on hover events. Wrong. I dont have a scientific breakdown on how much Flash content RELIES on hover events, but at least in my experience with the Nexus one, almost all content I have come across has worked well with touch screen input, including those that use hover events.
First, lets look at all of the Mouse events that Flash developers normally have access to.
- MouseEvent.CLICK
- MouseEvent.DOUBLE_CLICK
- MouseEvent.MIDDLE_CLICK
- MouseEvent.MIDDLE_MOUSE_DOWN
- MouseEvent.MIDDLE_MOUSE_UP
- MouseEvent.MOUSE_DOWN
- MouseEvent.MOUSE_MOVE
- MouseEvent.MOUSE_OUT
- MouseEvent.MOUSE_OVER
- MouseEvent.MOUSE_UP
- MouseEvent.MOUSE_WHEEL
- MouseEvent.RIGHT_CLICK
- MouseEvent.RIGHT_CLICK_DOWN
- MouseEvent.RIGHT_MOUSE_UP
- MouseEvent.ROLL_OUT
- MouseEvent.ROLL_OVER
Now, lets look at a list of events that will not work on a device whose primary input is a touch screen:
- MouseEvent.MIDDLE_CLICK
- MouseEvent.MIDDLE_MOUSE_DOWN
- MouseEvent.MIDDLE_MOUSE_UP
- MouseEvent.MOUSE_WHEEL
- MouseEvent.RIGHT_CLICK
- MouseEvent.RIGHT_CLICK_DOWN
- MouseEvent.RIGHT_MOUSE_UP
As you can see that list doesn’t include the MOUSE_OVER and ROLL_OVER events. It only includes middle and right mouse button events, as well as mouse wheel events, which in general, are used as a secondary means of user input / interaction.
Basically, hover events do work in Flash content on touch screen devices. The main difference is that on a touch device, you will get also get a MouseEvent.MOUSE_DOWN event prior to the MouseEvent.MOUSE_OVER event, where as on a desktop machine, you MAY only get the MouseEvent.MOUSE_OVER event.
I put together a very simple Flash example for the iphone, that shows all of the MouseEvents being generated. You can view the video here:
Here is a list of the events which were generated in the video:
- doubleClick
- mouseUp
- mouseDown
- click
- mouseUp
- mouseOver
- rollOver
- mouseDown
- rollOut
- mouseOver
- rollOver
- rollOut
- mouseOut
- click
- mouseUp
- mouseOver
- rollOver
So, to summarize:
- Hover events do work in Flash content on touch devices.
- Any issues with mouse input on a touch device are not specific to Flash, but affect all content, including HTML based content.
- If your content RELIES on MIDDLE, RIGHT and / or SCROLL_WHEEL mouse events, then you will need to update your content IF you want users on touch devices to be able to use it. This applies just as much to HTML content as it does to Flash content.
Now, this doesn’t mean that all Flash content designed and developed for mouse input on a desktop computer will run well on a mobile device with touch input. Indeed, some content will not run well due to differences in input (such as keyboard input), user interface, or performance. However, again, those are all issues that affect HTML content as much as they affect Flash content.
Again, any potential input issues apply as much to HTML as they do to Flash, and as browsing the web on the iPhone has demonstrated, these have not been a major barrier to consuming content on a touch device.
As a content creator, you should focus on creating content that provides a good user experience across all platforms that you are targeting. However, even though most Flash content on the web today is not designed with mobile viewing and touch input in mind, my experience has been that existing content works surprisingly well when viewed on a mobile device with touch input.
Here are some links to videos of existing Flash content running on mobile devices. A lot of this content is content never designed to be run on a device with touch input.
- Demos of Flash Player 10.1 on various mobile devices (iPhone, Palm Pre, Nexus One, Droid, and more…)
- Flash Player content on Nexus One
- Unofficial Nexus One Flash Demos
Please keep comments constructive and on topic. Off topic comments will be moderated.
Agree. There is no reason why flash should not work on mobile devices. Except CPU requirements. Jobs got that one right. Current Flash is a CPU hog. There are quite some websites where more than 5 flash banner looping continuously, sometimes horribly written. One of my favorite news site when idle uses 40% of CPU if flash turned on, 0% if turned off.
Just having a website like that open for reading articles and it will suck your batteries dry in an hour.
This is actually an issue that should have been addressed years ago.
lyesmith
22 Feb 10 at 1:26 am
@lyesmith
–
Current Flash is a CPU hog.
–
I would adjust that a bit, and say Flash content CAN be a CPU hog. That includes poorly designed / developed content, having too many individual pieces of Flash content on a single page, etc…
While there are areas where Adobe can (and is) working to improve overall performance, regardless of what is done, it will still be possible for developers to create content that performances poorly or uses a lot of CPU. Of course, that also applies to JavaScript and just about any other programming language.
mike chambers
mesh@adobe.com
mikechambers
22 Feb 10 at 1:53 am
Ahck – so many people peddling opinion as fact on the internet. I can’t stand it. Besides, I think – at least as far as my workflow is concerned – I’d have to sit down and _come up_ with a case where a MOUSE_OVER is something more than secondary feedback mechanism -and- is not redundantly covered by a click.
You do have a point in your comment reply regarding the fact that any language can be poorly programmed, but I believe it’s probably more so the case with Flash simply because of it’s interdisciplinary nature – you’ve many designers and other people who wouldn’t normally have to wear the hat of “coder”. Not a bad problem to have, all things considered.
Will
22 Feb 10 at 2:19 am
Love it Mike. The post in question that is circulating Twitter is a bit far fetched. The biggest issue I had with it is all the retweets it got in the Flash community. I think we need to be researching what we tweet before we tweet it.
This article is a great follow up to a couple days of misleading information. I’ll be posting my rebuttal to that post on my blog later this week.
Way to represent!
nothinggrinder
22 Feb 10 at 3:01 am
To anyone who thinks flash on a mobile phone is not possible for any reason whatsoever look at the Nokia N900. It can do it all while multi-tasking. You can switch input modes from touch to a mouse or even trackball-esque interface with a simple touch.
People need to escape their little Jobs controlled iphone bubble and see the real world.
Things have moved on since 2007. 2/3 years is a long time in mobile land.
The author of the original article is nothing but an iphone/ipad apologist who feels the need to spend his days covering for Apple’s mistakes.
Kimble
22 Feb 10 at 3:44 am
Great article. The “other” post is extremely frustrating to read especially considering it preaches the benefits of html5 and css3. Ironically as you so eloquently pointed out, css3 introduces even more :hover events that the iPad will choose to ignore, along with extra javascript events that are used in almost all web applications.
I love the video showing all the events – it instantly quietens those anti-flash evangelists who seem to be vocal yet clueless.
I mean, don’t these cats realise that we can even compile swfs to iPhone App now? I suspect the other article wasn’t written by a very proficient flash developer.. in fact I suspect a Silverlight developer – they always seem to have a chip on their shoulder ;)
Great read. Thanks for the facts.
zen
22 Feb 10 at 3:49 am
[...] to flash design would solve it, seems to be too much of a mountain to climb. [Update] Seems Mike Chambers is with me too. Flash designers (myself included at times) have the same attitude to Accessibility, [...]
Dino 2.0 » Blog Archive » The real reason your iPhone will NEVER get Flash
22 Feb 10 at 4:14 am
@mike chambers
“I would adjust that a bit, and say Flash content CAN be a CPU hog”
But that is a major problem. The internet is full of badly written flash applications. It is impossible to avoid them. I do a lot of flash development but on my laptop(W7) flash is permanently turned off, because it drains battery while browsing. All my friend using OSX turned off flash and it is harder and harder to sell flash sites because the clients has flash turned off.
lyesmith
22 Feb 10 at 4:34 am
Good to see you guys responding to some of the more misinformed ‘commentary’ out there.
Hope he approves your correction in his posts comments.
RE the CPU Hog debate:
Aside from the improvements in performance of the flash player, is the answer better education for non-coding designers and inexperienced developers to produce more efficient interactive content?
I guess at the end of the day, there’s not a lot you can do when developers on any platform will hang themselves given enough rope, so to speak.
Clint
22 Feb 10 at 5:51 am
@lyesmith
–
But that is a major problem.
–
Sure, but I think you missed my point. This is an issue that is not isolated to just Flash. Sure, perhaps is is exposed more via Flash, but that is only because Flash is so widely used.
If some other technology eventually replaces Flash, then you would see a lot of the same issues.
Regardless though, as I stated earlier, there are definitely areas where we can improve things, and this is something in general that we are constantly working on.
mike chambers
mesh@adobe.com
mikechambers
22 Feb 10 at 6:04 am
I see that over/up/click events are triggered all at the same time.
Mouse_over events in flash are used for something different than the click event, say like dropdown-menu’s? or moving objects in to place; to the mouse-pointer or away of the mouse-pointer…?
So how would you interact with a drop-down on an android/iphone or ipad?
Maybe, Click to show the menu, then click the to chooses item? would that be good practices?
Joris
22 Feb 10 at 7:01 am
I think there *is* a point in this, in that Flash content developed with the web in mind could well display behaviour that might make it annoying on a mobile device: a rollOver over a Flash banner that launches an almost fullscreen creative, or a flash game that takes mouse input, for instance.
The problem does not lie in Flash Player’s inherent ability to deal with the events on a mobile platform, rather with the content itself.
But as far as I’m concerned, the ability to surf the “real” Web on mobile devices is still in its infancy, so what the hell did people expect? It’s like saying, radio content is now available on TV but TV broadcasts of radio shows are boring because they don’t contain a picture.
Not the best analogy, but you get the point. It’s great that Flash publishes easily to mobile but to expect to scrape by solely on existing content is unrealistic. It’s a new game.
What I do however take away as a Flash designer/coder is to try and anticipate how the content I create influences the browsing experience of a mobile visitor to the page it sits on.
There may be sense in forwarding the user agent from browser to my swf content, and Adobe might do well in facilitating this information interchange for developers that care about this. I know it’s possible, I’m talking about making it as easy as possible for me as a content developer.
Franz
22 Feb 10 at 9:32 am
@mike chambers is right
Flash CAN be a CPU hog but so can badly written Javascript, which there is a lot more of than badly written Flash.
If HTML5 depends on Javascript to replace what Flash can do, the web is going to become a much buggier and browser crashing place.
Mike
22 Feb 10 at 9:36 am
Excellent post! Love the addition of the video, too. It’s not just a ‘let me tell you why you’re wrong’, but a ‘let me /show/ you how I know you’re wrong’. Keep it up! ;)
Rev4lis
22 Feb 10 at 9:39 am
One look at the portfolio of the individual who originated this latest round of hilarity reminds me of how dangerous the internet can be when it comes to propagating non-expert analysis.
“A lie gets halfway around the world before the truth has a chance to get its pants on.”
- Winston Churchill
Andy
22 Feb 10 at 9:40 am
Off hand, I can’t think of any content I have ever created or contributed to that REQUIRED hover events for core functionality, although it has certainly been used for such things as button and tab states and tool tips. I have seen some applications that probably use it for annoying navigation systems and some games that may or may not use it for certain gameplay mechanics aside from highlighting potential selections, triggering tool tips, and such, but it seems rare.
I think Flash Player’s inconsistent mouse wheel support across browsers and platforms is a far greater concern—even without considering mobile devices.
Brian Sexton
22 Feb 10 at 9:47 am
[...] greatest is reason is that flash doesn’t support the hover event or something. And just in a rebuttal claiming that reason is moot. Doesn’t matter if this is true or not. I think we’re all beating around the wrong [...]
The real reason why Flash is not on the iPhone/iPad « phpslacker
22 Feb 10 at 10:15 am
I agree with you Mike.
I think the original author of that article is not technical / expert enough with Flash :)
// chall3ng3r //
chall3ng3r
22 Feb 10 at 10:17 am
Completely agree – the suggestion is just daft – educator have been using SMART and Promethean boards for years. The use of Flash on such touch screen devices is amongst the most interactive and engaging tools. This is why the lack of Flash on Apple mobile devices is so disappointing – it is an Apple profitability and control issue – not a lack of hover-over status.
Andrew Field
22 Feb 10 at 10:23 am
In the demo MouseOver/RollOver and MouseOut/RollOut are always simultaneous (at some point in the event stream MouseOvers are being generated from RollOvers). So you say hover events work, but they don’t. They are simply duplicated and relabeled drag events.
Two different interaction techniques have been collapsed into one and can no longer be distinguished. There is a certain user discoverability to MouseOver that is totally lost when you have to be in constant contact with the screen.
Good content creation for touch devices should take into account that the MouseOver and MouseOut events really don’t exist.
DaveF
22 Feb 10 at 10:30 am
@DaveF
–
In the demo MouseOver/RollOver and MouseOut/RollOut are always simultaneous (at some point in the event stream MouseOvers are being generated from RollOvers). So you say hover events work, but they don’t.
–
When implementing a hover interaction in ActionScript, you would listen for the mouseOver OR the rollOver event, not both.
You can find more info on the differences in this article (main difference is how each event bubbles):
http://www.zedia.net/2008/difference-between-mouseeventroll_over-and-mouseeventmouse_over-in-as3/
It is the same behavior as in the browser.
mike chambers
mesh@adobe.com
mikechambers
22 Feb 10 at 10:58 am
[...] Chambers posted this response, but really you should take a look at this video from bytearray, to see Flash 10.1 running on the [...]
Confabulor » Blog Archive » Rollovers are going to kill Flash!
22 Feb 10 at 2:22 pm
Thanks for the added info, Mike, but I think that this doesn’t address the essential criticism of Dilger’s post. You may be *able* to generate rollOver and mouseOver events using a touch device, but only by means that contradict the *essential feature* of touch UIs: the user doesn’t have to move a pointer to select a UI element. The ___Over events in Flash work because the user *has to* move a pointer over the element *before* he/she can register a click – it’s built into the operation of the mouse. Your presentation of Flash’s ability to generate ___Over events with a touchscreen completely glosses over that distinction. At the very least, you’d have to code a “warning – mouseover behavior needed” message into every Flash app that users mouseovers, so that touchscreen users would understand that they now have to interact with their device in a non-intuitive way.
Scott
22 Feb 10 at 4:20 pm
[...] FSF Urges Google to Kill Flash Con: An Adobe Flash developer on why the iPad can’t use Flash Pro: Flash Player content, Mouse Events, and Touch input Flash Player 10.1 on Google Nexus One [...]
iPad Links: Monday, February 22, 2010 « Mike Cane's iPad Test
22 Feb 10 at 5:10 pm
Great article, great detail! Thanks.
You may be misunderstanding me with your corrections, though: I never disputed points 1 and 2, and I still don’t. And as for 3, I don’t pretend to have numbers on the Web as a whole with regards to how important mouseovers are, and to which audiences (casual gamers?) but I do maintain that hover functionality in Flash is VERY common, and often very important. (And as my article mentions, JavaScript menus can indeed share the same issue.)
So just to make sure my original point in the Roughly Drafted article is clear: it’s not that these events CAN’T be handled on touchscreens, in any number ways… it’s that they can’t be handled WELL enough.
I’m not talking about the technicalities under the hood, but about the best experience that it’s possible to get from current (not future) Flash web sites on a touchscreen. Even if Flash can on some technical level transmit events that have those names, that doesn’t mean what the user is DOING with those events makes consistent, user-friendly, rapidly-responsive sense across a whole range of different uses. (Games being the most widely-varied.) It’s not about an unseen event NAMED “rollover,” it’s about the user experience of that rollover actually working right.
Your video doesn’t refute my points (and it’s unable to tell rollover and click apart in the independent way desktop Flash can). But it is definitely a valuable contribution to the topic!
I maintain that the user experience will, necessarily, be very poor for a significant number of things people want to do with their favorite Flash-based content. Some current Flash pages could work well, but enough would not that the overall Flash experience would disappoint users greatly.
(And an issue I didn’t even touch on: touchscreens like the iPhone and iPad that rely on swipe to scroll the page. That’s yet another layer of interaction that somehow has to keep working in a way that doesn’t frustrate people. There are solutions. They’re not great.)
Morgan Adams
22 Feb 10 at 9:14 pm
@Scott
–
Thanks for the added info, Mike, but I think that this doesn’t address the essential criticism of Dilger’s post.
–
The essential criticism of the article was:
–
Current Flash sites could never be made to work well on any touchscreen device, and this cannot be solved by Apple, Adobe, or magical new hardware.
That’s not because of slow mobile performance, battery drain or crashes. It’s because of the hover or mouseover problem.
–
Even if I had posted nothing about hover events in Flash, the videos linked at the bottom of the page show that the essential criticism of the article (“Current Flash sites could never be made work well on any touchscreen device … because of the hover of mouseover problem”) is incorrect.
The article doesnt say that hover events are different, it states that they do not work. That is incorrect.
The article asserts that most Flash content RELIES on hover events to function at all. That is incorrect.
It then draws the conclusion (based on its false assumptions) that since hover events dont work in Flash, and since MOST flash content relies on hover events, MOST flash content will not work AT ALL on touch devices. That is what the essence of the article is, and incorrect.
–
At the very least, you’d have to code a “warning – mouseover behavior needed” message into every Flash app that users mouseovers, so that touchscreen users would understand that they now have to interact with their device in a non-intuitive way.
–
I think that should read:
–
At the very least, you’d have to code a “warning – mouseover behavior needed” message into every Flash app that RELIES mouseovers
–
mike chambers
mesh@adobe.com
mikechambers
22 Feb 10 at 9:31 pm
As a non-programer, Several times a day I go to http://news.yahoo.com/ and hold my mouse over headlines of interest and a slightly longer description of the article pops up, At that point I might or might not click on the article. Is that Flash feature? How would that work on a touch screen? Of course I do not NEED that feature, but if it is the kind of flash feature that would not be available on a touch screen, I would rather see touch screen programming evolve an alternative way to give me that functionality.
Jim
22 Feb 10 at 10:09 pm
This Flash inquisition bullshit will end once tablet devices, such as Dell Mini 8, or mobile device like Nexus One, supporting version 10.1, will be more visible on the market.
People will realize that using an iphone/ipad is very limiting their so called “web experienced” compared to other devices.
Again, this happened only because an angry CEO decided that the whole world doesn’t need Flash.
In my opinion it might have been the best Flash promotion in fact.
People keeps talking about the h264 codec and video html tag, but nobody talks about music which is widely played online thanks to Flash from a protected stream way (ref: myspace.com)
The mouseover controversy is quite funny though :)
I am also amused on how people get amazed by HTML 5 beta features which just duplicates what Flash is doing since years.
baz
23 Feb 10 at 12:07 am
[...] Mouse Events and hover in Flash Player content on touch devices, John Gruber (daringfireball.net) comments that: The problem, though, for a hypothetic Flash plugin that renders pages within web pages (as on [...]
Scrolling HTML with Flash Content on Touch Devices at Mike Chambers
23 Feb 10 at 1:12 am
Well, he got what he wanted. Momentum… 120 replies to an assumption post. Hell, even @smashingmag is ridding the “Bash Flash” bandwagon with this one… Geez…
Mauro
23 Feb 10 at 1:28 am
One thing a lot of articles talking about flash and touchscreen devices seem to neglect is that there are a myriad of webpages with mouseover javascript. Same problems there as in flash….
Webreaper
23 Feb 10 at 1:47 am
Now please can you show me that demo on a jailbroken iPhone with the Flash Player installed so we can see what is really happening?
You’re using a compiled app so much of the stuff you’re doing would be translated as part of the compilation however the stuff on the web won’t be.
I might agree with you if you show me that.
Loweded Wookie
23 Feb 10 at 1:59 am
I don’t seem to see how the video actually contradicts the earlier article.
The point was that on a touch device it is not possible to get mouseover-related events without inherently triggering clicks or other already meaningful touch events at he same time.
And the video merely confirms that.
Which essentially renders mouseover programming useless and inaccessible, doesn’t it?
The inherent difference between a mouse and a touch screen is that a mouse implies a mouse pointer which (ideally) moves continuously – it does not jump and it is present and movable even while no user actions (click/drag/scroll/zoom) are triggered.
A touch screen inherently has no mouse pointer movement without user actions and inherently uses random access positioning (even multiple random access locations with multi-touch).
Maybe I have just missed something in the presentation, but how are you supposed to access mouseover controls on a touch screen in practice? The video doesn’t seem to answer that at all.
The demo seems to have assumed that you could touch-and-drag without the UI interpreting that drag as a scroll event, which is not true in mobile Safari or any related browser.
debug
23 Feb 10 at 3:14 am
@debug
–
I don’t seem to see how the video actually contradicts the earlier article.
The point was that on a touch device it is not possible to get mouseover-related events without inherently triggering clicks or other already meaningful touch events at he same time.
–
I addressed this here:
http://www.mikechambers.com/blog/2010/02/22/flash-player-content-mouse-events-and-touch-input/#comment-18556
–
The demo seems to have assumed that you could touch-and-drag without the UI interpreting that drag as a scroll event, which is not true in mobile Safari or any related browser.
–
I actually made a separate post today which discusses this:
http://www.mikechambers.com/blog/2010/02/23/scrolling-html-with-flash-content-on-touch-devices/
mike chambers
mesh@adobe.com
mikechambers
23 Feb 10 at 7:34 am
Thanks for the response, but the second video does not seem to address the issue either.
It shows isolated scrolling, but no actual mouseover / click action in combination in a conventional Flash applet.
But that is exactly the issue here: Is there a way that existing Flash applets which make use of mouseover for informational purposes but which also employ actual clicks on the same elements will work without re-coding the Flash applet?
That is what all of this eventually hinges on, and that is what the article questioned:
An Adobe Flash developer on why the iPad can’t use Flash
The primary problem is this: If you already have an applet which uses mouseover, can it be used on a touch device in addition to direct click action? Neither of your videos really demonstrated how that could be the case, but if it was in fact possible, why not just show it with one in actual practice?
Funneling touch point movement (which normally means scrolling) into Flash is not even the actual problem there – the entire interaction model is, and the problem is that a touch UI has one interaction dimension less than a mouse UI in this regard. (It also can have multi-touch, but that doesn’t help us here.)
Of course these fundamental problems apply to JavaScript mechanisms just as well – the tactical problem for Flash is just: When the applets will have to be reworked for a touch UI anyway, then why stay with Flash, given the other known difficulties?
That is the core of the article, and I have not yet seen that issue addressed in any practical way so far.
debug
23 Feb 10 at 8:29 am
@debug
We seem to be going in circles here.
The second video was for your comments around drag gestures in the page and how they would be handled when there was flash content.
This blog posts address the assertions made in the originally referenced post:
————–
Many (if not most) current Flash games, menus, and even video players require a visible mouse pointer. They are coded to rely on the difference between hovering over something (mouseover) vs. actually clicking. This distinction is not rare. It’s pervasive, fundamental to interactive design, and vital to the basic use of Flash content.
…
Current Flash sites could never be made work well on any touchscreen device, and this cannot be solved by Apple, Adobe, or magical new hardware.
…
All that Apple and Adobe could ever do is make current Flash content visible. It would be seen, but very often would not work
…
So it’s not just that Apple has refused to support Flash. It cannot, logically, be done.
…
Even if slow performance, battery drain and crashes weren’t problems with Flash (and they truly are), nothing can give users of any touchscreen, from any company, an acceptable experience with today’s Flash sites
————–
Now, that does not mean I am suggesting that all Flash / web content will work fine on a touch screen device, it wont (just as all html based content wont work). But, you can get hover events in Flash content. In SOME cases it may be different that what is in the browser (as you point out), but again, my experience has been that it 1) works fine or is 2) secondary to functionality of the content.
The original article suggests that MOST Flash content RELIES on hover events, and that hover events NEVER work in Flash, and thus MOST Flash content CANNOT function AT ALL on a touch device. As I have already commented, the videos at the end of the post show that is not the case.
mike chambers
mesh@adobe.com
mikechambers
23 Feb 10 at 9:37 am
@debug
–
The primary problem is this: If you already have an applet which uses mouseover, can it be used on a touch device in addition to direct click action?
–
I have uploaded another simple example (this time on a Nexus One) that shows hover events and click events on the same content.
Green is up state. Purple is hover state. Orange is click state.
http://www.youtube.com/watch?v=poueEn0SjzI
Hope that helps clarify…
mike chambers
mesh@adobe.com
mikechambers
23 Feb 10 at 10:06 am
mikechambers: I have uploaded another simple example (this time on a Nexus One) that shows hover events and click events on the same content.
Green is up state. Purple is hover state. Orange is click state.
Thank you, but it was really already clear from your earlier example.
The problem is that you effectively have to hijack the regular scroll UI event in order to simulate hovering over nearby elements.
The expressiveness of a touch UI simply doesn’t have any directly usable equivalent to “hover”, and all the plausible gestures are already reserved for regular browser control, so the only chance would be to wedge this in sideways.
In your example a user would normally expect the content to scroll (or to “bounce back” in case the content is fully visible), not to perform a completely different and in this environment unintuitive operation which is inconsistently masking the entrenched and generally expected behaviour.
I don’t say it isn’t doable, but that is exactly where I must agree with the article that it would be steep uphill climbing trying to establish this with the majority of (usually casual) users.
debug
23 Feb 10 at 10:25 am
@debug
–
The problem is that you effectively have to hijack the regular scroll UI event in order to simulate hovering over nearby elements.
–
Ahh, I think I see where the disconnect is now.
For Flash 10.1 content, the user can double click the content to go into a full screen mode. In that mode, a drag gesture would not trigger scrolling the page (since all of the content is in view).
I show this behavior in the video i referred to earlier:
http://www.mikechambers.com/blog/2010/02/23/scrolling-html-with-flash-content-on-touch-devices/
Regardless though, as shown in the videos I linked at the end of the article, most content DOES NOT REQUIRE any hover events to work, and thus most content works fine on these devices (contrary to the assertions of the original post).
mike chambers
mesh@adobe.com
mikechambers
23 Feb 10 at 10:39 am
well, a lot of hair splitting going on here.
but what matters is desktop Flash cannot be replicated on a touchscreen, tho it can be adapted. so websites assuming desktop Flash and using all its options (like rollover) may present difficulties to touchscreen users. workarounds are possible (like the above demos) but are not great UI.
this obviously calls for a new version of Flash that is optimized for touchscreens, and/or a new version of mobile Flash that solves the issues with a good UI somehow. the ball is in Adobe’s court to come up with this.
the problem is the older format of Flash is so well established across the web, and Flash is proprietary to Adobe. so if/until Adobe doesn’t fix it, it will never get fixed.
whereas the alternatives, which can share the same UI issues, are not so entrenched and belong to no one. so developers can adjust with touchscreen-friendly site design and programming that avoid the issues.
AlfieJr
23 Feb 10 at 12:52 pm
Mike,
I am fairly new to Flex and this article and accompanying video helped explain how flash would do on a touch device. I was under the impression that since Flash is very mouse centric and since a touch device does not have a mouse pointer that nothing would work but your video easily showed that not to be true. Thanks.
Brandon Giromini
23 Feb 10 at 8:17 pm
@alfejr
–
but what matters is desktop Flash cannot be replicated on a touchscreen
–
Im not sure what you are getting at here, but as the demos that I linked show, current desktop Flash content works fine on the touch devices.
mike chambers
mesh@adobe.com
mikechambers
23 Feb 10 at 10:42 pm
@Wookie
–
Now please can you show me that demo on a jailbroken iPhone with the Flash Player installed so we can see what is really happening?
You’re using a compiled app so much of the stuff you’re doing would be translated as part of the compilation however the stuff on the web won’t be.
–
Im not really sure what it being a “compiled app” has anything to do with the behavior, but regardless, you can see a similar demo on a Nexus One device here:
http://www.youtube.com/watch?v=poueEn0SjzI
mike chambers
mesh@adobe.com
mikechambers
23 Feb 10 at 10:46 pm
@Jim
–
As a non-programer, Several times a day I go to http://news.yahoo.com/ and hold my mouse over headlines of interest and a slightly longer description of the article pops up
–
Hmm, that is odd. That mouse over doesnt work for my on Chrome on Mac. Yet, the site is still completely functional for me. i.e. the hover functionality is secondary to the site, and the site does not REQUIRE it in order to be useful.
That is the same for most Flash content that utilizes hover events.
mike chambers
mesh@adobe.com
mikechambers
23 Feb 10 at 10:49 pm
The same issue occurs with HTML content…
Awesome montage of apps built by Flash on mobile devices: http://www.youtube.com/watch?v=aTZaIrZLDp0
leef
24 Feb 10 at 12:00 am
[...] Chambers has also completed a great study on the Touch and Mouse events, and in particular how you can begin to optimize your content for this huge array of new platforms; [...]
Battery Performance with Flash Player 10.1 on Nexus One | www.flashmobileblog.com
24 Feb 10 at 12:07 pm
[...] top of this Mike Chambers recently debunked the idea that Flash wouldn’t work with touch screen devices, like the [...]
Everything you thought about mobile Flash might be wrong
24 Feb 10 at 5:57 pm
[...] of how much bigger a finger is than a mouse pointer and you’ll see the problems. I think the “hover” issues are completely overblown, but there is a big difference between Flash content in a mobile browser and Flash content in a web [...]
Platform Shifts | Ryan Stewart – Rich Internet Application Mountaineer
4 Mar 10 at 4:45 pm
[...] MikeChambers.com – on flash touch screen [...]
Would Flash work on a touch screen? « flash 4 ipad
9 Mar 10 at 4:18 pm
@Mike,
Thanks for a hype-free treatment of the subject. You’re responding to a sensationalized article, so I take the purpose of this article as combatting the spin surrounding the “Flash can never work on touch devices” meme. There’s a kernel of truth to that meme if only it were stated differently.
-
Current Flash sites could never be made work well on any touchscreen device, and this cannot be solved by Apple, Adobe, or magical new hardware.
-
That’s the sensational part. The article does elaborate on it, however. The point is that it’s impractical to the point of being impossible to expect all sites which currently use Flash to be updated to work with touch devices.
As you pointed out this depends on a possibly false assumption, that most Flash content relies on hover in a primary way making that content unpleasant to use on a touch device (though not impossible as your videos have demonstrated). Dilger treats this assumption as true. You suggest it’s false. Both assertions are subjective. Which position a reader believes is probably predicted by which side of the debate they wish to be on.
Personally I believe that it may be true that “enough” current flash content would be unpleasant to use. Ignoring this possibility is as much a mistake as blind faith that background apps are inherently superior while ignoring the challenges they present.
It’s equally foolish to assume that because Flash may be a poor experience today on touch devices that that will be true for all time. Believing that blocking Flash promotes creation of touch compatible apps with newer tech (JS+Canvas) ignores that a similar “clearing the field” could be accomplished by only supporting a Flash version with acceptable levels of touch support.
I dislike Flash for the same reason I dislike Blu-ray. So the “Flash can never work” meme is an appealing siren’s song. Thanks for cutting through the hype and taking a practitioners approach to the debate. Maybe you can wash away the hype from Dilger’s article and find the kernels of truth. I feel that you’ve glossed over some of that because you felt the need to defend Flash against inaccuracies in the article.
toolbear74
6 Apr 10 at 9:56 am
[...] content, tuning your code will be central to a good experience. Three, as Mike Chambers showed, hover content works, but I found it kind of annoying. The way it’s currently set up, the hover event fires after [...]
PhysicalFoursquare – A Flash Player 10.1 Demo App Using the Foursquare API | Ryan Stewart – Rich Internet Application Mountaineer
28 Apr 10 at 12:15 am
Great summary! Thanks! This again proves that Jobs is full of shit, and just want’s to discredit Flash. It’s a way of leading consumers away from the real issue why Apple does not want Flash on their devices. That issue is greed and the control-nazi stance of Apple.
Rob
29 Apr 10 at 11:51 am
[...] Chambers already proved that most Mouse Event from Flash works fine on the iPhone. Even MOUSE_OVER and ROLL_OVER works perfectly! The only problems are with middle click, right [...]
André Gil's Blog » Steve Jobs: From Genius to… Crazy and Liar?
29 Apr 10 at 7:24 pm
Touch screen problems are not flash specific.
Mouse overs are usually just aux UI augmentation (highlighting touchability).
Interfaces that allow drag and drop by trackin mouse down move up events have problems on iPhone and other
UIs that capture these events for their own use (scrolling)! So the trick is how control who consumes a click drag event OS vs APP. IF bubbling events from APP to OS then APP can INTERFEAR with scrolling OH NO. If OS Consumes it (iPhone) then you cant drag your objects around the page! There is
not hardware limitation only a limitation in Apples embrace of the free market. Web Apps should be able to get tese messages first. You want quality control, you dont need a single point of dist to do it, you only need certs. Users decide what they run. You only need a good power tracking monitor in the OS to highlight which apps use what amount of power and let the consumer decide again. I am tired.
Apple is the perfect smug storm. Hackers fix all.
James Johnson
5 May 10 at 8:05 am
[...] The myth began with the publishing of this article. I have already covered this topic in another post, but I wanted to address it again as it seems to have spread [...]
Top Flash Misperceptions : Flash cannot run on touch devices at Mike Chambers
12 May 10 at 10:47 am
[...] User inputs need to be large enough to be used, but also ones that don’t need the non-existant hover state [...]
Big Orange Software Blog » Designing for mobile touch devices – learning from YouTube
13 Aug 10 at 9:12 am
The hover/rollover debate is one that will run for some time, and there will always be differences between mouse usabilty and touch usability. Flash on android 2.2 in my experience however does work. Hover actions can be performed by touching said object and swiping off the object to execute rollover function, not as intuitive but it does work (it also works on standard rollover menus too (non-flash).
HATIRO
7 Feb 11 at 6:22 am
Has anyone ever had this problem when developing flash cs5 applications for a touchscreen:
This doesn’t happen on a regular computer with a mouse, however, on the touchscreen when I “press” a button, it briefly highlights the area of the button (rectangular box) in white (can see if on a colored background).
I have no idea why or how to solve this – any help would be appreciated.
Thanks.
Lynne
13 Jul 11 at 11:00 am
Above problem only seems to happen if I code the button with AS3 – doesn’t happen in AS2
AS3 code:
test.addEventListener(MouseEvent.MOUSE_UP,testClick);
function testClick(event:MouseEvent){
gotoAndPlay(“testit”);
}
AS2 code:
on(release){
gotoAndPlay(“testit”);
}
Lynne
13 Jul 11 at 11:58 am