Scrolling HTML with Flash Content on Touch Devices
In response to my post on Mouse Events and hover in Flash Player content on touch devices, John Gruber (daringfireball.net) raises an issue:
The problem, though, for a hypothetic Flash plugin that renders pages within web pages (as on traditional desktop browsers), is how to tell whether a tap-and-drag within a Flash element is supposed to scroll the entire web page or be passed as a mouse movement event to the Flash element. It can’t do both, and it can’t read the user’s mind.
This is a potential issue, and as John notes, is one not isolated to Flash content (scrolling textareas have similar issues on a touch device).
Here is how the current pre-release builds of Flash Player 10.1 handle this on the Nexus One (I havent had a chance to confirm for the other devices).
When the page is loaded the Flash content is embedded within the HTML and laid out as specified by that HTML. If all of the web page content cannot fit on the screen (i.e. you can scroll) and if the user swipes / drags their finger (including over the Flash content), it will scroll the page.
However, if the user double taps the Flash content, it will go into a scaled full screen mode, and the swipe / drag gesture will then be passed to the Flash content. In the case of the Nexus One, the user presses the device back button, which takes the content out of fullscreen mode.
If the scroll position is at the top of the page, and the user drags down on the Flash content, then the drag will be passed to the Flash content. This makes sense to me, but I am not sure if that is a feature or a bug at this point.
In practice, this works very well, and at least among the people I have watched interact with the content, appears to be pretty intuitive.
You can see a video of the interactions here:
Now, this is all pre-release software, and could change. I would be interested in hearing any other suggestions on how this particular interaction could be made better (for Flash and HTML content). Again though, this is just as much a potential HTML issue, as it is a potential Flash issue, so think about solutions in that context.
Please keep comments constructive and on topic. Off topic comments will be assassinated.
hi,
On my Nokia N900 the Flash content in browser only get tap event for sure. If I drag the content, the browser takes it as page scroll event.
So, it’s pretty limited on N900.
Additinally, I think content authors should also detect if content is running in touch device, and accordingly make the controlls suitable for touch input. For now, youtube player controls are very hard to use.
For making touch experience even better on FP10.1, the dev should have option to allow event to be passed to system. for example, if i don’t use MouseWheel event, or I pass additional parameter, the system should handle it do what it suppose to do.
// chall3ng3r //
chall3ng3r
23 Feb 10 at 1:59 am
Looks good, although it would seem natural that a double tap should exit full screen mode rather than having to use the back button
Nick Brown
23 Feb 10 at 4:20 am
I’ve been experimenting also with Flash Lite 3 on a couple of phones (S60 5th Ed, WM6.5) and think this is an unavoidable problem, unless you do the double-tap to go full screen. This is an excellent feature (not present on my Nokia) – the issue is, how to go back from there. Double-tap on the Nokia browser and you jump to 200% zoom on the page…
Most touchscreen devices have a concept of a right-click, to bring up the context menu (Android may be an exception but they have more hardware keys from what I’ve seen). So this could have a menu item to go fullscreen/normalscreen and would solve everything :-) Hooking in the long-press for the context menu seems to work okay on WM.
cheers
Andrew
Andrew Frost
23 Feb 10 at 4:54 am
Great video. I just hope that Adobe gets this right. It seems to be a very make or break situation that needs to address the points being made and show why the choice is important.
The part I don’t get is that I firmly believe that methods used to convey an interactive product depend on what is required from an overall toolkit. I certainly do not want to be told no I can not use this platform because some Jobsworth said I can’t. If it’s the right process for that design then that’s what platform I’ll use, whether that is flash, java, css etc.
People talk about replacement but at the end of the day there’s lots of things one platform can do that the other can’t and vica verca, and I think the overall point to be made is they are all part of an overall digital toolkit and can all be misused or used properly.
Now, what I’d like to see is people expressing these all in unison as opposed to saying no to one or the other they, as they can all form a part of a richer user experience.
Adam
23 Feb 10 at 5:10 am
Scrolling in textareas works fine if you two-finger scroll on the iPhone.
Jemaleddin
23 Feb 10 at 11:36 am
Is there a difference between the mobile browser scaled full screen mode and the regular full screen mode? I am mostly concerned about keyboard input. Will users be able to input text in this mode?
Josh On
23 Feb 10 at 5:33 pm
Loving this anti-FUD article series.
As Jemaleddin suggests, multi-touch can solve this, by using two fingers in concert to effectively simulate a mouse wheel.
I would bet that all of the missing middle & right click|wheel events from your previous article could potentially be emulated using 2-3 fingers in a touchscreen browser with multi-touch event access from the w3c device api.
rdza
23 Feb 10 at 9:47 pm
What I’m wondering is how to distinguish between a tap-and-swipe and a rollover using a touch interface. Or even if that really matters seeing as rollovers in HTML/JavaScript are not a giant issue on touch devices. But in certain Flash apps, there could be a crucial differentiation between mouseover and click-and-drag events as there’s no obvious way to come up with two distinct gestures for a multitouch UI since one involves adding a mousedown event to instantiate a concurrent click event with a drag. Thoughts? Anyone?
Gary Horsman
24 Feb 10 at 6:08 pm
I hope you can see where Apple would find this solution objectionable for a Safari plug-in.
You can change the meaning of standard gestures in your own applications and Apple will probably not give you much of a hard time. But changing the meaning of gestures in one mode versus another inside Safari is probably not going to fly with Apple’s UI guardians.
This solution has little hope of being adopted even if Apple was willing to allow a Flash plug-in.
James
24 Feb 10 at 6:13 pm
“However, if the user double taps the Flash content, it will go into a scaled full screen mode, and the swipe / drag gesture will then be passed to the Flash content.”
I have a question. When setting the scale mode to NO_SCALE, what happens exactly when double tapping the flash content? Will the movie will still be scaled based on the original (previously displayed) size? Or will that action throw a resize event so the application can resize itself to the new fullscreen size?
Patrick W.
25 Feb 10 at 5:35 am
Yeah. Where a gesture would conflict with a particular OS / Browser, we could of course provide a different solution gesture, one that is consitent with that particular OS and Browser.
mike chambers
mesh@adobe.com
mikechambers
1 Mar 10 at 12:24 pm
Hmm – I did some testing with the nexus one running 10.1. Multi-touch gestures didn’t work at all. Not sure if they are not implemented yet or if they conflict with the zooming-feature of the flash player. Does Adobe plan to implement multi-touch on Mobile devices?
Robin B.
11 Mar 10 at 10:17 am
@Robin
–
Hmm – I did some testing with the nexus one running 10.1. Multi-touch gestures didn’t work at all.
–
Yeah, multi-touch isnt working in the current development FP android build.
Yes, we will be adding multi-touch.
mike chambers
mesh@adobe.com
mikechambers
11 Mar 10 at 11:09 am
[...] Scrolling HTML with Flash Content on Touch Devices [...]
Top Flash Misperceptions : Flash cannot run on touch devices at Mike Chambers
12 May 10 at 11:49 pm
Hi
I’ve problems on my HTC legend (Android 2.1) with scrollbars within Flash-websites …..
The multitouch makes it impossible to drag the scrollbar :-(
Any workarounds?? or any solutions in next player??
Thanks Lars, Copenhagen
Lars
17 Nov 10 at 3:22 am