Daniel Eran Dilger
Random header image... Refresh for more!

HTML5 assault on Adobe Flash heats up with ClickToFlash

History of Flash Player

Daniel Eran Dilger

As Adobe works to port its full Flash Player to mobile platforms and highlights its upcoming support in CS5 for building iPhone apps using Flash tools, an open source group is leading a drive to kill Flash on the desktop using a WebKit plugin named ClickToFlash.

HTML5 assault on Adobe Flash heats up with ClickToFlash
El asalto de HTML5 a Adobe Flash se calienta con ClickToFlash
.

ClickToFlash allows Safari users to isolate Flash content on the web so that it only plays when they choose to allow it. Flash content is replaced with a bounding box that enables the user to ignore the item (such as with ads) or to click the placeholder to activate Flash playback as desired.

Additionally, the plugin can convert requests for YouTube Flash videos into requests for higher quality H.264 videos, allowing desktop users to bypass Flash the same way the iPhone does, and simply play any YouTube videos using the browser’s own built-in HTML5 support for direct H.264 playback.

The examples below compare the same HD clip served by YouTube, first rendered using Flash with the standard grey YouTube playback controls, and then presented in H.264 using ClickToFlash to request the non-Flash version from Google. The native HTML5 version results in Safari using its own native QuickTime X playback controls rather than those created by Google using Flash. During playback, Safari’s native black playback bar disappears; using the ClickToFlash plugin, the user can also present H.264 videos using full screen playback.

YouTube via Flash
YouTube via HTML5
In addition to offering better quality video and more functional and accurate playback controls for smoothly navigating through the video clip, ClickToFlash’s direct playback using HTML5 prevents the Flash plugin from ever engaging and consuming CPU cycles and notebook battery life. Outside of YouTube, this means that loading web pages won’t max out your processor just to animate Flash ads in the background. Any essential Flash elements, such as navigation components, can be activated by clicking on the placeholder created by ClickToFlash as desired.

In the example below, Safari with ClickToFlash does not load Flash content on the New York Times site by default, but will load individual items at the user’s request. On the iPhone, no Flash content is ever loaded nor can be. Most ad networks using Flash now sense the iPhone’s user agent and supply alternative, non-Flash ad banners. As more users opt out of Flash, the ad market will follow, just as it has accommodated the iPhone. Videos will also move to H.264 in order to support modern browsers that don’t need Flash just to present video clips.

NYT ClickToFlash
“One of ClickToFlash’s primary goals is to eliminate as much Flash from the web as possible, allowing users to choose only the Flash they want to see,” the plugin’s development website says. The group asks for help in adding direct video support for other sites that currently use Flash to facilitate video playback.

As support for H.264 video delivered by simple HTML5 video tags erodes the primary demand for Flash on the web, and as web developers become familiar with using HTML5’s new Canvas feature to perform sophisticated drawing effects within web pages without using a plugin such as Flash or Silverlight, many observers report that Adobe will face an uphill battle to continue to push closed Flash development as an alternative to using open web standards, particularly as new initiatives such as WebGL hardware accelerated 3D begin to gain traction.

Adobe’s mobile strategy for Flash

While ClickToFlash works to kill Flash on the desktop, the existing battle between Adobe’s Flash and HTML5 has been waged squarely in the mobile realm. Apple introduced the iPhone without support for either the desktop version of Flash or Flash Lite, a subset of Flash aimed at smartphone devices. As Apple became increasingly adamant in opposing Flash on the iPhone, Adobe reported that it had a Flash Player for the iPhone under wraps and nearing completion, and suggested that Apple was even involved in working on the project.

That changed with the announcement this week that Adobe would be releasing a new version of Flash Player 10 for other mobile platforms over the next year, with an initial public beta for Windows Mobile and Palm’s WebOS by the end of the year, followed by betas for Android, Symbian, and RIM’s BlackBerry OS sometime in 2010. Missing from those announcements was the hottest mobile device on the market: the iPhone/iPod touch.

(Updated since original publication to note that RIM is working with the Open Screen Project; Adobe says “The collaboration is expected to bring the full Flash Player browser runtime to BlackBerry smartphones.”)

Adobe addressed iPhone development with the announcement that its upcoming CS5 Flash development tools would enable the production of native iPhone apps. The resulting applications are not Flash, and this does nothing to enable Flash playback on the iPhone, either within standalone apps or embedded in web pages.

Creating iPhone apps using Flash CS5

Flash “applications” replace open and standard web content created using HTML, CSS, and JavaScript with Adobe’s proprietary .swf, a closed binary file that wraps up web content files (such as graphics and movies) with the company’s own variant of ECMAScript (JavaScript), called ActionScript. In CS5, the Flash development tools will allow new Flash projects to be exported as standard iPhone apps rather than .swf files.

Adobe is doing this using LLVM (Low Level Virtual Machine), an open source compiler technology supported by Apple and used in its Xcode Mac and iPhone development tools. The next version of Adobe’s Flash development app will simply compile Flash ActionScript into native iPhone code, much as existing tools already allow iPhone developers to write their code using Java, Scheme, or other languages, and then compile the code into C or Objective-C as a native iPhone app.

The iPhone is designed not to support any alternative languages via any sort of virtual machine, which prevents it from running “raw” Java, Flash, .Net, Silverlight, or anything else apart from its native C/Objective-C compiled to the ARM processor. This is enforced in the terms of Apple’s SDK agreement. There is however no restriction against compiling any existing code into native C/Objective-C and creating an iPhone app from it.

Making iPhone apps with Flash CS5
Porting Flash to the mobile

Existing Flash projects will require some additional work to get them on the iPhone; Adobe notes the iPhone’s 320×480 resolution is not the target resolution of most Flash apps aimed at web users, and that touch navigation will not work with content designed around the use of a precise mouse pointer.

“Certain behaviors that you may often employ in desktop application development will not necessarily apply to the iPhone,” writes Adobe spokesman Aditya Bansod in a blog posting announcing the new tools. He also warns, “The iPhone is most decidedly not a desktop computer. It has very powerful and sophisticated hardware, but there is a wide spectrum of capabilities between the different generations of device, the amount of memory available, and the amount of processing power your application has at its disposal.”

Adobe’s Applications for iPhone FAQ further notes, “The iPhone and iPod touch have processors significantly slower than those found on most desktop PCs and Macs. Thus, content may run slower than it does when running on a desktop personal computer. However, the exact differences will depend on the specific content.”

The many faces of Flash

Adobe also states that any Flash content intended for conversion to an iPhone app must use ActionScript 3.0, which debuted with Flash 9. Previous versions of Flash used ActionScript 1.0 or 2.0, which are run using an entirely different virtual machine compared to ActionScript 3.0. The desktop Flash plugin supplies two different virtual machines, one for playing existing ActionScript 1/2 bytecode, and a second for executing ActionScript 3.0.

Adobe’s existing version of Flash Lite 3.0 for mobiles only supports ActionScript 2.0, which means Flash Lite can’t run modern Flash 9 or 10 content. The new Flash 10 player Adobe is aiming at mobile users over the next year will only run ActionScript 3.0, which conversely means that legacy Flash content designed to work with Flash Lite won’t work on new devices.

This diversity helps balloon the desktop version of Flash, which makes no effort to force the adoption of modern Flash content because it has the resources available to run legacy content using multiple virtual machines. But that doesn’t translate to the mobile realm, where battery life and processing power constraints make the fat desktop Flash plugin and its pair of virtual machines unworkable.

Adobe’s plans to ship a “full” mobile version of Flash 10 that only supports new 3.0 bytecode will suddenly obsolesce all existing Flash content that is more than a couple years old. That includes anything designed to work with Flash Lite, Adobe’s current mobile strategy. That will force mobile Flash developers to rewrite all of their Flash code in order for their content to work with the new mobile player, undermining a key reason for wanting to use Flash in the first place: its wide installed base.

History of Flash Player

Additionally, with the iPhone consuming around half of all mobile web traffic, mobile Flash developers will have to contemplate whether it’s worth it to rewrite all their existing Flash content in ActionScript 3.0 just to target the non-iPhone platforms that should be able to run Flash Player 10 by sometime over next year.

The alternative is to build HTML5 apps using standard JavaScript, which already runs on the iPhone and on other WebKit mobile browsers such as those for Nokia’s Symbian, Google’s Android, RIM’s BlackBerry OS, and Palm’s WebOS.

Such open content will not only work on the iPhone and BlackBerry, but will also run on desktop browsers including WebKit browsers such as Safari and Chrome; Mozilla’s Firefox; Opera; and even Internet Explorer using Google’s Chrome Frame plugin.

  • JohnWatkins

    No link to readable resolution image of first diagram.

  • JohnWatkins

    Interesting article. Thanks for the explanation and thanks for the info on “Click to Flash” (just installed it.) Never new so much of the web “content” (ads) was Flash, even static stuff. I’ll never miss it.

  • Pingback: gruchalski.com » Archive » I can now start using Safari, ClickToFlash FTW()

  • bartfat

    Interesting that Adobe is the bad guy this time around, because usually they end up pushing the state-of-the-art, such as PDF and Photoshop. But unfortunately here their interests lie with trying to keep developers buying that bloated Adobe Flash program to develop for Flash because god forbid, they’ll actually have to start competing with other companies when developers use standard Javascript and HTML. No one will end up buying Dreamweaver when they can do the same thing better and faster with Coda.

    But yeah, ClickToFlash is AWESOME. I’ll never let go of it, I actually get annoyed if I’m on a computer that doesn’t have it. But thankfully I don’t have to deal with Flash on the iPhone :) Can you imagine being harassed by an ad that drains your battery and forces you to scroll to close it because it was designed for a PC resolution to view on Flash? IMHO i believe Apple has more reasons to abandon Flash than simply that it’s resource-intensive (as if that was bad enough)… it has accessibility issues, can’t be gracefully degraded for text-based browsers, etc, etc.

    But thanks for the article, it confirmed what I already knew in this particular case… but I’m probably in that low low 0.1% of the population that actually reads tech news for hours on end daily.

  • BrianS

    Thanks for the info on ClickToFlash. I have been hoping for the day when I can get rid of Flash and it’s finally here.
    You write above, “using the ClickToFlash plugin, the user can also present H.264 videos using full screen playback.” I can’t find a way to view embedded H.264 video in full screen mode. Am I missing something?

    [Right click on the gear icon in the placeholder box. The option is presented to play full screen using Quicktime – Dan]

  • Pingback: Flash Sucks! – Elemental Media()

  • BrianS

    I did find the full screen option after some more digging. Thanks.

  • pedrocandrade

    I actually thought Adobe’s solution to be quite clever. At least it tries to provide a reason for some people to buy CS5 and to continue to believe in Flash as a cross platform development alternative. (Not that it is any good at that!) It sure beats standing still as the iPhone bandwagon gains ever more momentum.

    But some things are not clear to me. Any chance of Flash conversion working to lower people’s incentive to learn OS X’s native development methods? Are there significant drawbacks here to what you can do with Flash on the iPhone when compared to using the Apple SDK? I can imagine no 3D, a hard time doing multitouch and maybe trouble with location services, but to be honest it’s kind of foggy to me.

    Any clue on what would be the best Adobe could accomplish with this strategy (output quality wise), and how much it could eventually turn developer interest away from Apple’s own tools?

  • mdwittenberg

    I tried installing both 1.5.3 and the beta 1.6b2 but I could not get the standard Quicktime controls for youtube.

    Do you have any further steps to replicate this much needed behavior?

  • mdwittenberg

    Turns out that GlimmerBlocker disturbs ClickToFlash. You must disable the YouTube fix under Site enhancements.

  • Neil Edwards

    @ pedrocandrade

    As a Flash & iPhone developer I can see this as only being of very limited use and as Adobe has demonstrated probably only useful to games.

    As far as I can glean there are currently several limitations, namely no UIKit support, you can only build using the native flash components which bear no visual or functional resemblance to the Apple ones or simply dont have counterparts. No hardware 3D, although there are good 3D solutions (Papervision etc) for Flash the GPU linked version is not complete and I seriously doubt the software one will run anywhere fast enough.

    Multitouch is supported but no GPS.

    Also no info on debugging, although Flash has a excellent debugger in Flex Builder, there will be no Simulator support and it seems the export is just that an ipa, anyone who’s done iPhone dev work knows that you need to debug on the device, I see no way to do this. So yes it might not crash in FlashPlayer but any iPhone crashes will be tricky to fix.

    Most twitter traffic seems to be along the lines of not learning ObjC which is plain lazy or I dont have to buy a Mac now which is plain stupid.

    Obviously Adobe want everyone to use Flash everywhere so maybe this will only be the beginning, but they have a lot to do to catch up ObjC development.

  • mailjohannes

    Flash is a pain in the ass. Especially with the latest version of Safari because no flash killer I know of works.
    Thanks for the info, this is really helpful.

    J.

  • Pingback: Flash 10.1 Mobile, Air Mobile and Flash Lite content?! | biskero()

  • luisd

    @JohnWatkins ->

    There are 2 solutions if you are using a mac. With Safari 4 press cmmnd and “=” or “+” to make the whole page bigger. This makes the images bigger too and readable.

    The other solution, if you have a scrollable mouse (MM, or logitec) use ctrl and scroll up, this zooms in into the screen. I use this all the time.

  • http://www.adviespraktijk.info Berend Schotanus

    Thanks for the link to ClickToFlash.

    For some time I have been dreaming of surfing the web on my MacBook Air the way I do on my iPhone. As long as you are surfing content that is (more or less) free of Flash the MacBook Air is a very power efficient machine. But as soon as you hit a Flash rich site an oven starts heating up on our lap and you literally see the battery indicator move down to zero.
    A Windows machine probably produces so much heat anyway you won’t notice the difference.
    So I started hating those Flash rich sites that drained my battery for annoying advertisements I didn’t want to see in the first place.

    About the Flash – HTML5 battle, I think it is interesting to follow. The ability of Flash CS5 to output Objective-C I think is a cool feature. The Flash authoring environment might appeal to a different audience than the regular Apple tools. When you are a graphics designer with limited programming skills Flash might be an option to do cool things on the iPhone.

    Still it is disappointing Flash CS5 does not output to HTML5, although… for my own browsing comfort it might be a good idea all those annoying ads remain in proprietary Flash format for some time so they can be easily filtered out by ClickToFlash.

  • Josh

    Doesn’t Roughly Drafted have Flash ads?

  • http://KitwanaAkil.com kakil

    Thanks for bringing ClickToFlash to our attention. I’ve installed the plugin and I swear it seems like my PowerBook G4’s Safari browser is loading pages faster. Maybe my imagination, but it does seem a little snappier. We’ll see what happens with further use.

  • Thomas Menguy

    Just a technical correction : I’m at Adobe Max at this time in LA, and I’ve followed the technical sessions around the LLVM compiler … and this doesn’t output C/C++/Objective-C code but direct ARM assembly, same as an input it takes an SWF file and not directly the ActionScript code. (if you want to look at an AS to C/C++ compiler check http://developer.openplug.com/ , also announced at MAX)
    It brings also iphone development to windows box, and the reception here has been tremendous from the designers crowd.
    Just to let you know.
    Good article by the way and I’m gonna use the ClickToFlash magic plugin!

  • Neil Edwards

    More MAX news, seen Flex mobile framework for iPhone, the demo showed very UIKit like functionality! No debuuger news yet, seems only trace at best.

  • http://crankyoldnutcase.blogspot.com/ The Mad Hatter

    And of course if Flash is pushed into Obsolescence than so will be SilverBlight. A nice one, two, punch.

    I wonder if this is why Microsoft was so set on getting the Mono development environment ported to the IPhone? Daniel, you should check into this. I can supply you some links. You might want to check the forums as well, I posted a thread about Mono there, to celebrate being named to the official Mono Enemies List.

  • enzos

    Nice one. I’ve been using it for a day and no probs. Amazed at how much I thought was gifs is actually resource-sapping Flash. Cheers!

  • Pingback: El asalto de HTML5 a Adobe Flash se calienta con ClickToFlash, por Daniel Eran Dilger()

  • deemery

    Let’s not forget the bugs and vulnerabilities that Flash introduces!! I’ve been running Firefox for a lot of websites (e.g. cnn.com) to use the Flash blocker plug-in, so this is welcome functionality for Safari.

  • Pingback: mysocialbrain: 08-10-2009 : protagonist()

  • Pingback: Flash 10.1, Flash Mobile, Flash on iPhone, Air Mobile more information | biskero()

  • Pingback: Segnalibri: 8 ottobre 2009 – 15 ottobre 2009 - Noi (u)siamo la macchina()

  • Pingback: Ten Myth of Apple’s iPad: 2. iPad needs Adobe Flash — RoughlyDrafted Magazine()