Hurley's World

The latest Platform Preview Build includes two great interoperable features for working with the DOM – DOM Traversal and Element Traversal. These features provide web developers with simple, flexible, and fast ways of traversing through a document using the same markup across browsers. These features come in the form of flat enumeration, simplifying the DOM tree to an iterative list, and filtering which enables you to tailor the set of nodes you traverse. These features work with the same markup across browsers – you can try out any of the code here in the IE9 platform preview and other browsers.

Without these features, finding an element of interest on a page requires you to do one or more depth-first traversals of the document using firstChild and nextSibling. This is usually accomplished with complex code that runs slowly. With the DOM and Element Traversal features, there are new and better ways of solving the problem. This blog post is a primer and provides a few best practices to get you on your way.

I’ll start with Element Traversal, since it’s the simplest of the interfaces and follows familiar patterns for enumerating elements in the DOM. Element Traversal is essentially a version of DOM Core optimized for Elements . Instead of calling firstChild and nextSibling, you call firstElementChild and nextElementSibling. For example:

if (elm.firstElementChild)
{
elm = elm.firstElementChild;

while (elm.nextElementSibling)
{
// Do work...
}
}

This is faster and more convenient, saving you the trouble of having to check for text and comment nodes when you’re really only interested in elements.

DOM Traversal is designed for much broader use cases. First, you create a NodeIterator or a TreeWalker. Then you can use one of the iteration methods to traverse the tree:

var iter = document.createNodeIterator(elm, NodeFilter.SHOW_ELEMENT, null, false); // This would work fine with createTreeWalker, as well

var node = iter.nextNode();
while (node = iter.nextNode())
{
node.style.display = "none";
}

The codepath above iterates through a flat list of all nodes in the tree. This can be incredibly useful since in many cases you don’t care whether something is a child or sibling of something else, just whether it occurs before or after your current position in the document.

A big benefit of DOM Traversal is that it introduces the idea of filtering, so that you only traverse the nodes you care about. While nodeIterator only performs flat iterations, TreeWalker has some additional methods, like firstChild(), that let you see as much or as little of the tree structure as you want.

The SHOW_* family of constants provides a way to include broad classes of nodes, such as text or elements (like SHOW_ELEMENT in the earlier example). In many cases, this will be enough. But when you need the most precise control, you can write your own filter via the NodeFilter interface. The NodeFilter interface uses a callback function to filter each node, as in the following example:

 

var iter = document.createNodeIterator(elm, NodeFilter.SHOW_ALL, keywordFilter, false);

function keywordFilter(node)
{
var altStr = node.getAttribute('alt').toLowerCase();

if (altStr.indexOf("flight") != -1 || altStr.indexOf("space") != -1)
return NodeFilter.FILTER_ACCEPT;
else
return NodeFilter.FILTER_REJECT;
}

For a live example, check out my demo for DOM Traversal -- I used NodeFilter extensively there. Complex filtering operations on the list of media elements were as simple as using a NodeFilter callback like the one above.

In this post, I showed that you have options in how to traverse a document. Here are suggested best practices for when you should use the various interfaces:

• If the structure of the document is important – and you’re only interested in elements – consider Element Traversal. It’s fast and won’t leave a big footprint in your code.
• If you don’t care about document structure, use NodeIterator instead of TreeWalker. That way, it’s obvious in your code that you’re only going to be using a flat list. NodeIterator also tends to be faster, which becomes important when traversing large sets of nodes.
• If the SHOW_* constants do what you need for filtering, use them. Using constants makes your code simpler, as well as having slightly better performance. However, if you need fine-grained filtering, NodeFilter callbacks become indispensable.

I’ve already found these features to be a great help in my own coding, so I’m really excited to see what you do with them. Download the latest Platform Preview, try out the APIs, and let us know what you think.

Thanks!
Jonathan Seitel
Program Manager

Here are four interesting mobile articles that caught my eye in the past 24 hours:

• Your mobile app is spying on you. About the danger in free apps.

  The study also found that a large proportion of apps contain third-party code with the capability to interact with sensitive data in a way that may not be apparent to users or the developers of the apps themselves. The third-party code is generally used for advertising or analytics. The project found that 47 percent of free Android apps included this third-party code, while 23 percent of free iPhone apps use it. Third-party code represents a security risk because it is difficult to update (and patch a vulnerability) on a global basis. Apple changed its terms of service for the iPhone recently because of its concerns about what third-party analytics and other companies were doing with private data.

• Nokia’s JavaScript Peformance Best Practices. Contains a lot that was taken from Steve Souders’s research and common knowledge, but also a few things I’ve never heard before. I’d really have to do an evaluation of some of the practices. Also contains a few ugly-but-necessary tricks for Symbian WebKit (S60).
• Also from Nokia: Introducing Ovi Browser Beta for Series 40. Nokia, too, has produced a mini-browser that relies on a server for the actual rendering and sends out the fully-rendered page, just as Opera Mini does. Now I need to lay my hands on an S40 device so that I can test it.
• From Bruce Lawson at Opera, finally, comes Mobile-friendly: The mobile web optimization guide with useful tips and tricks for making your website mobile-friendly.

  [...] mobile users are in a hurry; they're on the go and want to perform one specific task and then finish. A common example cited is that of a restaurant site. The mobile user wants to find the location, the menu and the opening hours so, the argument goes, the mobile site should contain this and nothing else.

  This is a good argument, but it's only half true. If it were 100% true, what would be on the "full" website? Presumably, a movie of the decor, some atmospheric music, animated representations of the house special dishes, and a downloadable menu in some fancy font. The fallacy here is that users of desktop computers are not task-focussed and have time to waste on an immersive branding experience.

In the mobile browser space all the advanced browsers are based on WebKit. That’s fine — WebKit is an excellent rendering engine — but if all browsers were based on WebKit I would start to worry about a monoculture. At least some browsers should be based on other rendering engines, as far as I’m concerned.

The only serious mobile candidate for “other rendering engine” is Opera. But I’m starting to wonder whether it can keep up with the WebKit browsers. With the recent release of Samsung Dolfin Opera Mobile has firmly dropped from third-best to fourth-best mobile browser on my list.

The problem is not that Opera isn’t innovating. It is. But I’m starting to wonder about the direction that innovation is taking.

Witness the recent release of Opera Mobile 10.1 beta for Symbian (on my 40th birthday; thanks, guys). I went through the press release, which highlights the following changes:

• Geolocation is now supported (although it doesn’t give the coordinates in my test page). That’s cool; it’s definitely something that’s extremely important on mobile.
• Vega graphics library and Carakan JavaScript engine. That’s fun to have, but I’m not sure if it’s the most important step Opera could have taken right now. I’ll get back to this.
• Various improvements to the user interface.

And that’s it, according to the press release. (border-radius is supported, maybe some other stuff too, and there will be some bug fixes here and there, but I haven’t yet studied it sufficiently to give you a full list.)

Relative importance

Nowadays all desktop browsers are innovating like mad in their JavaScript engines. For about a year and a half or so, significant improvements in JavaScript speed have been the Holy Grail of browser making, and every new release of every browser now routinely claims to be the fastest browser available.

(So routine has this claim become that I do not believe any of them any more. Besides, as far as I can tell they’re all testing JavaScript Core speed, while the biggest bottleneck is the speed with which DOM changes are made.)

Now on desktop this is a very important development. On mobile it’s also important, but I feel there are several other topics that are even more important than JavaScript speed, and that Opera is ignoring these topics.

Point is, a super-fast JavaScript engine is relatively worthless if you can’t properly set up a JavaScript-heavy interface. And the very first requirement for a JavaScript-heavy interface is that you always know exactly what the user is doing, so that you can react to his actions naturally and great UX is born.

It’s in the latter part of that equation that Opera Mobile has serious problems. I feel that, while innovating in JavaScript engines, Opera is forgetting basic mobile functionality without which a fast JavaScript engine is pointless.

Touch and viewport

There’s a reason I started my foray into mobile browsing by studying the touch events and the viewport dimensions. These two functionalities are absolutely vital to building compelling mobile interfaces.

The touch events allow us to follow exactly what the user is doing to the touchscreen. Without these events you can only detect whether the user clicks somewhere. That is enough for some interfaces, but others will definitely want to react to more subtle actions from the user.

The viewport dimensions are important for knowing how much of your site the user is currently seeing on his screen. An example will clarify why this is important.

I’m currently working on a little side project that involves writing the perfect JavaScript touch-scroll interface. This presupposes the touch events: without them it’s absolutely impossible to create a compelling UX.

However, I want several scrollable areas on the same page, and with that comes the problem of deciding when to allow a user to scroll. If the user has zoomed in and sees a little bit of one scrollable area on the edge of his screen, it makes sense not to scroll that area when he touches it, because it might confuse him. Instead, we should wait until the scrollable area covers most of the screen before allowing a scroll action. My prototype with this behaviour works pretty intuitively.

However, in order to know whether the scrollable area is on the screen I must be able to read out the dimensions of the visual viewport. I need to know which part of the page the user is currently viewing, compare it to the scrollable area, check their coordinates, and decide whether the user is allowed to scroll. That’s something that few browsers support right now.

Android and Samsung support the touch events, but not the visual viewport dimensions. Symbian, BlackBerry and IE support the visual viewport dimensions, but not the touch events. All other browsers, including Opera Mobile, support neither. So this script only works on the iPhone, which is the single browser to support both.

So this interface, which I feel is an example of a mobile-specific UX, will not work on Opera Mobile, regardless of the speed of its JavaScript engine. That’s what I mean when I say that I’m wondering about Opera’s direction of innovation.

Opera Mobile simply must support the touch events and the viewport dimensions.

Zooming

But Opera Mobile has an even larger problem, and that is zooming. Basically it has the worst zooming functionality of any mobile browser I tested. It seems Opera ignored the emerging standard for touch-based zooming, and instead created its own system. That’s fine, as long as the system works. But it doesn’t.

So what exactly is wrong with Opera’s zooming? Basically everything.

Two levels

Opera has only two zooming levels: in and out. Initially you see the entire page, just as on all other mobile browsers. When you zoom in, however, you go to one single pre-defined zooming level, and once you’ve arrived there your only option is to zoom out again.

This behaviour is not unique to Opera Mobile. Many other second-tier mobile browsers, such as Symbian WebKit on touchscreen Nokias, or IE, have only two zoom levels.

But there are two other characteristics that make the Opera zooming experience uniquely lousy.

Pre-emptive narrowing of text

Opera kind of pre-emptively narrows your text into columns that fit snugly in the screen when you zoom in. Although that’s excellent behaviour to display when the user actually zooms in (as Android does), doing this beforehand makes no sense and has the ability to destroy your page’s graphic design.

Here’s my compatibility master page as it should display on mobile (Dolfin on Samsung Wave). Note that the text in the table stretches all the way from left to right, as it should:

And here is the same page on Opera Mobile 10.10 (Nokia N97). Here the text in the table is narrowed to the width Opera is going to take later on, when you zoom in:

In this particular case I don’t mind much, but this feature has the ability to completely destroy a well-designed page. As far as I know you can’t do anything against this effect and will have to accept the damage it does to your page.

Single tap

But by far the worst feature of Opera’s zoom is the user interaction. On touchscreens an industry standard is emerging for zooming. You either pinch-zoom or you double-tap. Some browsers don’t support pinch-zoom, but the double-tap works pretty well, too.

However, Opera in its wisdom uses not a double-tap but a single-tap interface. Tap once when zoomed out and you will zoom in on roughly the area you tapped on.

This is terrible. In order to understand why, take a look at my Touch test page. It’s fully zoomed-out:

This is how I designed the page: I want it to be usable even fully zoomed-out. Thus I can load the page, hit Record, and record an event sequence of my choice.

But what happens in Opera Mobile when I hit Record with a single tap in order to start recording? It. Bloody. Zooms. In! That makes my test page much harder to use.

To be fair, Opera Mobile 10.10 does not zoom at all in pages that have a <meta viewport>, as my test page has. That solves this particular issue, but I feel it’s still a stop-gap solution. All other browsers do allow the user to zoom in on a <meta viewport>-enhanced page. (Some even allow you to zoom out.)

Opera’s problems

Concluding, Opera Mobile has a serious zoom problem that must be addressed in addition to the touch event and viewport dimension problem.

I feel that addressing these issues is more important than yet another faster JavaScript engine or yet another UX upgrade. Besides, as long as zoom remains lousy the UX can be upgraded only so much. Zooming is a fundamental part of the mobile UX; without it many other innovations just don’t make sense.

So in order to remain relevant on the highest tier of mobile browsing, Opera will have to concentrate on three things:

1. Fix zooming: double-tap interaction, many more zoom levels, no pre-emptive narrowing of text columns.
2. Support the touch events.
3. Allow us to read out the viewport dimensions.

I more-or-less expect the busy engineers in Oslo to be working on these problems, and I assume that the next major Opera Mobile upgrade will bring significant improvements here.

If it doesn’t, I’m afraid Opera is out of the race for the top spot. That is not necessarily bad: it could always forget about the top tier and instead concentrate on the low-end smartphone market. Right now Opera Mobile is available only for Symbian and Windows Mobile, and on those operating systems it’s the best available browser.

Without a serious upgrade of mobile-specific functionality ruling this second tier is the best Opera can hope for. The top tier will be WebKit-only.

This is just in: Google seems to be taking steps to allow operator billing. If that’s true it’s huge news.

Note from the outset that the article doesn’t say in so many words that operator billing is coming, although it certainly gives that impression, and plenty of publications translate it as such.

The basic idea of operator billing is very simple: if you want to buy an app, or access to online content, the price is automatically added to your operator bill (or, I assume, deducted from your pre-paid account).

Now I’m not a mobile billing specialist by any means, but I still want to give you an idea of what we’re talking about. If I make any technical mistakes, please correct them in the comments.

The billing process

Just yesterday I made my first Android Market purchase, and although the process was relatively smooth, I still had to fill in all my credit card stuff, make mistakes, being told off, etc. Besides, when I tried to do the same a few months ago, the Android Market rejected my credit card. Why? Probably because the Dutch market wasn’t active yet — but I thought of that only much later. At the moment I was pretty pissed.

Now with operator billing I wouldn’t have all this hassle. I’d just click on whatever I want to buy, give a one-time confirmation “Yes, I do want to spend € 2.39 on this” and be done. When my next operator bill comes around, the € 2.39 will be added to it.

In addition, operators can verify your identity through your SIM card, without you having to do anything. No more hassle with credit card numbers. (In fact, the only parties that have a lot to lose from operator billing are the credit card companies. Expect resistance from them; they’ll probably say it’s unsafe or something.)

Thus operator billing is by far the most user-friendly way of making mobile purchases. That’s what makes it so important. Besides, it also opens up the mobile marketplace to those that do not have a credit card.

A question of identity

However, Google’s rather vague announcement does leave some questions unanswered. No doubt that’s because Google is still figuring out how to answer those questions. But let’s review them anyway:

• What do I have to do in order to make a purchase? A one-step system is within reach; I hope it will be implemented.
• How will my identity (and thus my bill) be verified?
• How will the various parties communicate?

The last question is probably the most important one. If I want to make a purchase through operator billing, there are three parties involved: me, the selling party, and the operator. Somehow, the selling party has to connect to the operator to figure out exactly who I am, and to make a request to put € 2.39 on my bill for my purchase. In addition, the operator has to pay that money (maybe minus a fee) to the selling party.

The JIL 1.2 API gives us some clues as to how this system is going to work. This API, that will eventually be implemented in Vodafone 360 phones as well as, one hopes, many others, has two properties that are meant specifically for operator billing (p. 16):

Widget.Device.AccountInfo.phoneUserUniqueId Widget.Device.AccountInfo.phoneOperatorName

Thus, when purchase times comes around, the store has some grip on your identity. It will have to send off a communication to the specified operator stating that user with unique ID X wants to make a € 2.39 purchase.

The operator will have to make some effort to verify this information; after all I might be able to hack a phone to send false unique IDs. Thus the operator will probably send me an SMS “Are you sure you want to purchase product X for € 2.39?“ Once I reply to that SMS the purchase is made and downloading can commence.

Still, I hope that the process will become even more user-friendly. The same JIL specification defines a way to send an SMS from a widget. Thus, if I want to purchase something the system could automatically generate an SMS for me and send it off to the operator. Thus the operator will be able to verify my intent by comparing my unique user ID with the SIM card through which the SMS was sent. If they match the purchase is made and downloading can commence.

That’s one step less, and thus more user-friendly. Hell, if it’s implemented correctly I don’t even have to switch to my SMS application. (The operator still has to tell the store “Purchase made, proceed with download.” But a proper system will not bother me with the details.)

Unfortunately the JIL 1.2 spec does not yet contain the methods that will be used for actual payments, nor the exact workflow. Besides, it’s unclear which operators Google is talking to right now. Probably US-based ones, and of those only Verizon is part of the JIL consortium. The others might use other APIs. (Come to think of it, so might Verizon. One never knows.)

Future expansion

Let’s close on a positive note and assume that a system roughly similar to what I describe above will actually be in place in two years or so. Apart from the increased user-friendliness of the purchasing process, what will it bring?

The basic answer is Long Tail. Increased user-friendliness and the scrapping of the requirement to own a credit card may entice more consumers to make a mobile purchase. That would be good.

The real benefit will lie with developers, though. In theory, the system could be set up so that individual developers who offer one or two apps for download on their own site can also use it.

Thus the requirement to offer your wares through one or more app stores might also be scrapped. That could be especially important to cross-platform apps such as W3C Widgets. Whichever phone with whichever operator ends up at the developer’s site, they can all make a purchase, provided they support widgets.

One more nail in the app stores’ coffin would be the opportunity to make in-app purchases; say some articles from a news site or a few new levels for your game. Operator billing is explicitly meant for such purchases, too. And if we can use operator billing in our apps, too, the app store infrastructure is basically not necessary any more.

Picture the following:

1. I write a news reader app as a W3C Widget. Anyone can download it for free from my website.
2. If you like my app you can share it with your friends. Just send over the widget via Bluetooth. No more complicated user-unfriendly Send-To-Friend systems necessary.
3. But how do I make money? By selling access to the actual articles. Every article you want to read costs you, say € 0.02. Alternatively, you can buy a day of unlimited access for, I don’t know, € 0.99.
4. All the billing is done in-app through the operator. My users never have to do anything beyond saying “Yes, I want to buy this article.”

Where’s the app store in this process? Nowhere. We don’t need it any more. Wouldn’t that be something?

(I should note that although sending widgets via Bluetooth is possible nowadays — I’ve done it — the process is not very user-friendly yet. But this functionality is definitely coming; it’s not a pipe dream.)

Waiting for Google

So I’m impatiently waiting for Google to announce more details. Exactly how will their system work? What does the user have to do? Which operators? Questions, questions.

Anyway, the future of mobile payments has come one step closer.

Back in early June I got a Samsung Wave that runs the brand-new bada OS and did some brief tests of the native Dolfin browser. In the past few days I’ve done some more extensive testing, and the verdict is in: good browser, well on the way to becoming excellent.

(Oh, and Dolfin ought not to be confused with Dolphin, which is a skin for Android WebKit.)

It’s Samsung’s philosophy that it will not compete in a market unless it belongs to the top three of that market. In the case of the mobile browsing market Samsung has succeeded: from nothing, Dolfin has become the third-best mobile browser in the world. Only iPhone and Android are better.

If you’re keeping track of the mobile browser landscape you should add Dolfin to your A-list. It’s easily good enough, and Samsung has big plans with the bada operating system. Somewhere in 2011 the installed base of Dolfin will pass that of Safari iPhone, and bada might even become a competitor to Android. (Samsung sure hopes so.)

I have updated my mobile pages with Dolfin data. (By the way, I also tested Android 2.2 while I was at it: few changes. There’s not a single difference with 2.1 in my great WebKit table.)

The good parts

Dolfin is only the third browser in the world to support the touch events, after iPhone and Android, and that’s a large part of the reason I count it as the third-best mobile browser.

Mobile browsers for touchscreen phones just must support these events by the end of the year, or they’re out of the race. (Opera, Firefox, Microsoft, NetFront, Nokia, BlackBerry, you have been warned.)

Dolfin’s implementation is even slightly better than Android’s. If I do a pinch-zoom test on my touch test page I use two fingers, and during replay I want the browser to show the position of both fingers. iPhone does so, Android doesn’t. Now Dolfin also shows both fingers’ position.

Unfortunately Dolfin is not quite up to iPhone quality: my multitouch drag-and-drop does not work. Still, the stuff it does support is quite impressive for a browser that didn’t exist at the start of this year.

Dolfin’s responsiveness to pinch-zooming is good. One friend who’s used to the iPhone even said it’s better than Apple’s implementation. I’m not quite prepared to go that far yet, but it easily gives Android a run for its money.

Dolfin incorporates a very modern WebKit version, and in my great WebKit table it is the best mobile browser, narrowly above Android 2.1/2.2 and comfortably above the iPhone 3.1 (4 not yet tested).

I’m currently working on a little side project: the perfect scrolling layer script for touchscreens. Obviously it should work in all three browsers that support the touch events. Of the three Android gives me the biggest problems right now, with Dolfin smoothly sailing along in the wake of the iPhone without any serious problems.

Dolfin is just a bloody good browser.

The buggy parts

Still, not all is rosy. I found a few bugs in Dolfin, and I’ll enumerate them here for the benefit of the Samsung team:

• The viewport dimension properties are an absolute mess. Not only do they generally give wrong data, but a few of them change meaning when a <meta viewport> is present in the page. This must be addressed as soon as possible; it’s a show-stopper.
• There’s a bug in the touchstart event. When you assign an event handler to it and execute it once, the event handler is removed. See this test page for the effect. Works fine on iPhone and Android, but fails in Dolfin the second time you want to drag the item.
  The workaround, obviously, is setting the touchstart event handler again ontouchend. That works fine.
• Dolfin is over-eager in firing the scroll and contextmenu events. Basically they always fire, even if an action does not result in a scroll or a contextmenu. (In fact, I haven’t yet discovered any contextmenu on Dolfin.)
  These events should only fire if the scroll or contextmenu action actually take place.
• The event coordinates are wrong: clientX/Y and screenX/Y copy the values of pageX/Y, while they should report the coordinates relative to the visual viewport in CSS pixels and in device pixels, respectively.
• A personal peeve: on the browser software keyboard the “Go” button is located on the left. A standard is emerging whereby it’s the right button that says “OK, do it,” while the left button says “Never mind.” I got confused by Samsung’s button placement more than once.
• Samsung bada is supposed to support W3C Widgets, but there is no way for developers to upload test widgets to the phone. In practice that means nobody will test on Samsung bada. This situation must be rectified.

Still, such bugs are only to be expected from a first release. In general Samsung has delivered a good browser that’s only a few small steps away from becoming excellent. I will follow Samsung’s next moves with interest.

Apple continues to startle me, and I do not mean by its iPhone 4. (I haven’t yet seen it, so I can’t say anything useful about it.) No, what I mean is the ongoing Antennagate problems, and even there I do not mean the actual problem, but Apple’s way of dealing with it. And even there I do not mean Antennagate as an isolated PR incident, but as yet another chapter in how Apple spends 2010 to piss off the world at large.

Antennagate

The problem with Antennagate is not the actual technical issue itself. That issue may or may not be severe, may or may not occur on other phones, and is probably solved adequately by the covers Apple is now giving away. Frankly I don’t care about the details.

No, the problem is in how Apple handled the entire affair. Not its acknowledgement of the problem itself, or giving away the covers; those are proper, correct responses to the technical issue.

John Gruber made an interesting remark:

It’s telling that the criticism surrounding this issue has shifted, quickly, from speculation about a technical defect in the iPhone 4 hardware to criticism over the tone of Apple’s response to it.

The hardware defect is only a sideshow, and the way Apple is dealing with it is far more important to the world. Gruber is completely right in pointing this out, yet I believe he draws the wrong conclusions from it.

PR is crucial for Apple. Good PR has created a legion of Apple fanbois and a chattering class of tech commentators that consistently paints Apple in the best light possible. A serious PR flap is far more dangerous than a technical issue because it can cost Apple the support of the tech commentators (and possibly even the fanboi legion, but that will take more than a mere year full of Fuck Yous).

Thus I believe that Gruber is right, yet wrong. Weighing Apple’s tone and response is more important than the technical issue. Antennagate is about PR, and not about the iPhone 4’s antenna.

In that light, Tomi Ahonen points at an important mistake Apple made:

Why include that unnecessary hostile attack on the rest of the industry? That was a red flag to the rivals, and a clear challenge to the press to go compare. This is a lose-lose for Apple, even if others are implicated too, every time Death Grip is mentioned, Apple is the leading culprit of the news story. Every time the iPhone 4 is reminded to have Death Grip problems! And for every story where rivals are also implicated, there will be stories where at least one of the rivals is not shown as bad, or in the worst case, that Apple's iPhone 4 comes out worst at Death Grip. Apple's news coverage will be bad - or worse.

In other words, exactly because Steve Jobs wanted to implicate other phones in this problem, Antennagate will come back to haunt him. We can expect a string of press releases from other mobile players, saying something along the lines of “In contrast to Apple’s iPhone 4, our flagship product X does not suffer from an antenna problem.”

Note that it does not matter who’s right here. Even if Jobs is completely right and the other phone vendors are lying through their teeth, the world will again and again be reminded of Antennagate. That’s not disastrous, but it’s certainly a major annoyance that will hurt the iPhone 4’s sales figures somewhat.

Still, Antennagate alone will not spell doom and disaster for Apple. I would expect the hubbub to die down within a few months — if Antennagate were an isolated incident.

Problem is, it isn’t an isolated incident.

Apple pisses off everybody

I said it before and I’ll say it again: this year, Apple is being serious about pissing off absolutely everybody in sight. I do not think this is a viable long-term strategy, and I do think that if Apple keeps this up long enough it will eventually run into serious problems.

So what exactly did Apple do wrong?

First things first: in 2007 it barged in on the mobile market, changed the rules, and succeeded beyond everybody’s wildest dreams. That’s of course a huge success for Apple, and rightly celebrated as such, but it has also created a host of enemies: basically all other powerful mobile players. That’s not a disaster, but it does mean wolves are on the prowl, and Apple has to keep its defenses in order.

Its main line of defense, as always, is PR. And it’s exactly here that cracks are starting to show. Let’s recap Apple’s year so far:

• It pissed off Adobe by rejecting any kind of Flash on the iPhone. The problem here is not so much Adobe itself, which is facing serious trouble in making the move to mobile, but the Adobe fanboi legion, which got thoroughly angry. Adobe, too, has a powerful PR machine, and it knows how to reach designers. Adobe’s point of view has found powerful defenders. (So has Apple’s, but that’s not the point. It would have been far better to continue to let the Adobe hordes sleep.)
  PR translation: the iPhone is a closed platform, and Apple is proud of it.
• Closely related is the major SDK license flap when Apple banned 3rd-party runtimes from the iPhone. There are solid technical reasons for this, but that’s not the point. It’s one more PR flap and a serious gamble. To quote one analysis:

  The proposition is this: Apple is betting it can grow its platform fast enough, using any means necessary, that developers will stick around despite all the hardships and shoddy treatment. Each time it chooses to do what it thinks is best for the future of the iPhone OS platform instead of what will please developers, Apple is pushing more chips into the pot.

  Besides, once again attention was drawn to Apple’s almost frivolous app rejections. The problem here is not that Apple rejects some apps, but that the rules aren’t clear.
  PR translation: iPhone developers should bloody do what Apple tells them. The iPhone is a closed system that’s wholly dependent on Apple. Start loving it.
• Apple announced a lawsuit against patent infringements by HTC. To quote one analysis:

  Apple could be hoping for a chilling effect from other handset makers looking to avoid a legal battle. But Apple is walking a tightrope. It needs to whack HTC without drawing Google and Microsoft into the fight. Riddle me this: Can you realistically just focus on the hardware here? If so perhaps Apple can prosecute a surgical strike on HTC. The reality is that the smartphone software and hardware are intertwined. You can expect that Google and Microsoft will defend their code at some point.

  In other words, Apple may have bitten off more than it can chew. In any case it has aligned some of its most powerful competitors against it.
  PR translation: Apple will bully its weakest competitors on grounds that are not comprehensible to the general audience.
• Then it pissed off Mac developers by focusing WWDC wholly on iOS.
  PR translation: Fuck off, Mac devs. You’re not important any more.
• The case of the stolen prototype. I don’t really care about the legal aspects. The only thing I see is a large, legendarily closed company suing some journalists and assorted helpers.
  PR translation: Fuck you for not obeying our rules. We’ll send our Gestapo after you.
  I cannot help but be impressed by a closing remark on Gizmodo, the site that unveiled the stolen prototype:

  If you think that Gizmodo shouldn't have shown you the iPhone before Apple wanted you to see it, you’re accepting that Apple should be the one to control news about its products. That's not an irrational position, but let's be honest about what it means.

  [...]

  It hands back the control of the story to Apple because some are more comfortable believing Apple's machinations are infallible than that they’re a company made up of human beings who try to control the news cycle — and that even the best laid plan can fall apart because of a single human mistake.

  This is the flip side of the huge amount of attention the US tech world lavishes on Apple. Now that Apple is on the way to becoming a bully some media will gleefully report on it. It’s a new kind of Apple news, after all, and inherently more interesting that the next release of iWhatever.
• And now there is Antennagate.

(Incidentally, there are two groups that Apple hasn’t pissed off yet, and that I don’t expect them to piss off, either. They are consumers, who get their covers against Antennagate, and web developers, who’re still allowed to play with the most advanced mobile browser in the world. I continue to believe that web development is Apple’s ace in a hole. If everything else goes wrong, Apple will still have the best mobile web platform in the world.)

Is any of these mistakes fatal? Certainly not. Apple handled the technical aspects of Antennagate correctly, its rejection of Flash and the new SDK license agreements are defensible from a UX perspective, its dissing of Mac developers is understandable in the light of its mobile strategy, and I don’t doubt the lawsuits against HTC and the prototype culprits have a solid legal grounding. Finally, the PR flap is manageable in every case taken by itself.

Hubris

However, if we take all these issues together we see Apple spending 2010 by sending a cheerful Fuck You to people of all colours and all nations, and especially its own developers and former allies.

Is Apple suffering from imperial overstretch? Is Apple guilty of an old-fashioned case of hubris?

Judging from the evidence that’s certainly a possibility.

Fortune’s wheel. What goes up must come down. The throne of the once-almighty king is toppled because he grew overconfident.

Those are powerful story lines that have interested humanity for at least 2,500 years. Since they conform to some sort of basic human expectation of how history works, the tech commentators who are Apple’s PR mainstay could start to pursue them in order to present a different perspective and gain more readers. (Come to think of it, so am I.)

Right now Apple is doing everything in its power to push those story lines. In the long run that is not healthy.

I feel Apple has a PR problem, but isn’t aware of it, possibly because it doesn’t want to be aware of it.

Now let’s see how this plays out. The story isn’t done yet, and who knows, Apple might even see the errors of its ways before it’s too late.

In recent weeks Firefox 4 beta 1 and Opera 10.60 were released, and I could also put my hands on a working Chrome 4. I added all these browsers to the compatibility tables, which are now all updated, except for the Events one.

There’s little difference with my earlier reports, except that the new Firefox beta is making a serious effort to support the background module. It’s not there yet, but Firefox is not as disastrously behind as it was a month back.

Firefox

Of course Firefox remains a pain in other respects: still no support for contains(), incorrect indexing of attributes (it’s essentially random), no innerText or outerHTML, no sane second argument for the add() method for select boxes, a colorDepth of 24.

All these bugs are Firefox-only, with the exception of the attribute indexing, which is also a problem in IE. Frankly, I’m starting to get fed up with Firefox and its haughty refusal to implements useful methods and properties found in all other browsers. But maybe that’s a passing phase. Maybe I’ll pick another browser to be annoyed at next time around.

Events

I haven’t yet updated the Events table, partly because it’s such a lot of work (it takes me as much time as all other tables combined), and partly because I do not expect huge changes. I tested various problems in IE9, and sadly found no improvements.

I’m not sure when I’ll have the time to do the Events table; I hope before I leave on holiday in early August.

When we first introduced our plans for Internet Explorer Platform Previews we said that “developers and people interested in standards and web development can try out new platform functionality and provide early feedback.” We are now getting such feedback on a daily basis and are using it to improve IE9. However, sometimes the impact of the feedback extends beyond just IE9. Here is the story of how some recent feedback regarding the third IE9 Platform Preview resulted in a correction to the new ECMAScript 5th Edition (ES5) standard for JavaScript.

The ES5 standard became official in December 2009 and the third IE9 Platform Preview is the first widely distributed implementation of some subtle details of the ES5 specification. ES5 was designed to be highly compatible with existing websites and the Ecma International TC39 technical committee worked to avoid any non-security related changes that might break existing JavaScript code. However, perfection generally does not exist in the world of software so with the third IE9 Platform Preview we were very interested to see if any ES5-related compatibility problems with existing sites would show up.

Soon after releasing this platform preview, we received reports that some web-apps that use the jQuery framework did not work correctly in the preview. We tracked the problem to a specific jQuery API method that in some cases passed a caller provided value to Object.prototype.toString without first checking if the value was null or undefined. Specifically, some calls to this jQuery method:

isFunction: function( obj ) { return toString.call(obj) === "[object Function]"; },

failed with an exception: “TypeError: Object expected”. Further analysis showed that toString in the above code was the built-in method Object.prototpe.toString and that the failures occurred when isFunction was being called with undefined as its argument. Why does an exception occur in IE9 and not in previous versions of IE or other browsers? It is because the third IE9 Platform Preview in standards mode actually conforms to the ES5 specification for Object.prototype.toString.

According to the prior editions of the ECMAScript specification, calling any built-in method using null or undefined as the this value passes the “global object” (in browsers this is the DOM window object) to the method as its this value.  This opens a number of potential security holes for frameworks that aim to support mash-ups in a secure way.

The ES5 specification changed this so that null or undefined is not replaced with the window object and the definition for each built-in method was updated specifically to deal with receiving these values as their this value. The ECMAScript technical committee tried to do this in a way that preserves backwards compatibility for normal usage and throws an exception in cases where that is not possible. Object.prototype.toString was specified in ES5 to throw such an exception. This created the compatibility problem described above.

This problem can be easily corrected by modifying the jQuery code with the additions shown in red:

isFunction: function( obj ) { return obj && toString.call(obj) === "[object Function]"; },

The jQuery team actually intends to make this change. However, such a change will not correct the thousands of locally hosted copies of jQuery that already exist on the Web. With the broad use of jQuery, it is clear that the ES5 specification contains a significant compatibility problem. It is fairly obvious how we can modify the IE9 ES5 implementation to eliminate the problem. We can just return the same string value ("[object Object]") that IE8 returns in this situation. Such a fix does not reintroduce any of the security problems that ES5 strives to eliminate. However, we do not want to unilaterally introduce such a variation into our implementation of a new standard. It does not help either compatibility or interoperability if IE fixes this problem one way, and other browsers either don’t fix it or fixed it a different way.

As soon as we understood the problem and the possible solution, I raised this on the TC39 es5-discuss mailing list as a backwards compatibility issue.  My first post on the issue went out at 5:51 PM on June 25, a Friday. By 10 PM there were responses from TC39 members representing Apple, Mozilla, and Google. We all agreed that this was a compatibility issue that needed to be fixed, and that in this case throwing the exception was unnecessary and undesirable. We also initially agreed that the idea of returning the same string value as ES3 for these cases sounded like a good idea. However, in further messages over the weekend we realized that browsers currently don’t all return "[object Object]" in this situation, some other values that were observed include "[object Window]" and "[object Global]". A proposal was made to return "[object null]" and "[object undefined]". This seemed to be a better solution as it not only fixes use cases such as jQuery’s but it also explicitly distinguishes null and undefined from actual objects. It is also better for browser interoperability because it requires that all browsers produce identical results rather than the ES3 situation that allowed different browsers to produce different results.

By Tuesday, the consensus on the mailing list was to follow this final proposal. As soon as that agreement was reached, I passed the revised Object.prototype.toString specification on to our IE9 JavaScript development team so they could make the fix in time to widely test it with the next IE9 platform preview build.  Mozilla has also indicated that they will be incorporating this fix into a future Firefox beta. I also updated the official TC39 Erratum for ES5  so this specification change is recorded there (section 15.2.4.2).

Web standards are complex software artifacts and like all software, they contain bugs. Sometimes the best way to find and fix compatibility bugs is to implement and deploy the standard on widely used browsers. This generally takes place in the context of early releases such as the IE9 platform preview builds. So, when you as a web developer are providing feedback on such releases you aren’t just providing feedback on a specific browser you are also providing feedback on the new and emerging standards that it implements. Of course, for this feedback to be worthwhile, browser implementers and standards authors need to be able and willing to quickly respond to feedback that identifies significant problems. The rapid response to the ES5 jQuery toString problem and other issues on es5-discuss show how browser implementers and other TC39 members can and do work closely together to create a more compatible and interoperable Web. But it all starts with your feedback, so please keep it coming.

Allen Wirfs-Brock
Microsoft JavaScript Language Architect

Lost In Translation

“Web design is 95% typography”. So much of the content produced and consumed on the web is text yet designers and users have been confined to a set of compatible fonts available across client operating systems. Escaping this typographical island has involved everything from cross-browser CSS workarounds, graphics-based solutions and even plug-ins, with trade-offs ranging from extra storage and bandwidth to reduced accessibility.

As a result, typefaces have too often been one of the first casualties of the translation from Photoshop design mock-up to live web page.

Not The Same CSS

Solving this challenge required an interoperable CSS syntax to describe font resources. While IE added support for CSS2’s @font-face rule as long ago as 1997, the differences between this earliest of implementations and the far more recent ones supporting CSS3 Fonts have given rise to CSS design patterns built to ensure the same rule works for all users. A notable example is the bulletproof @font-face syntax developed by Paul Irish from Google, et al.

But crafting a cross-browser @font-face declaration turned out to be half the problem.

No Common Web Font Format

Internet Explorer’s @font-face implementation supports EOT (Embedded OpenType) fonts, a compressed font encoding submitted to the W3C in 2008. Following WebKit’s lead in 2007, Mozilla and Opera added support for raw TrueType and OpenType fonts. Raw fonts work well if your fonts’ end-user license (EULA) allowed you to serve them from your web server. While this is true for many free fonts, this is not the case for the vast majority of commercial fonts. Web authors are thus effectively cut off from the richest font catalogs available. If they choose EOT they are able to license a number of commercial fonts for web use but only IE renders them.

This leaves a challenge for the industry. The major browsers support two incompatible solutions and commercial font EULAs are generally incompatible with one of them.  This spawned dedicated hosting services like TypeKit to help designers license typefaces for their site and use them across browsers. These services, however, may not offer all or even any of the desired fonts that a web developer needs for their site. In some cases, a font set could even require multiple subscriptions.

WOFF

In late 2009, type designers Tal Leming and Erik van Blokland working together with Mozilla’s Jonathan Kew proposed a new web font container format to address the concerns of font vendors without reducing web author choice. It was well received and submitted to the W3C in April 2010 by Mozilla, Opera and Microsoft. Today, Firefox 3.6 supports WOFF and other browser vendors will be following soon.

Font foundries have also rallied behind the new format and already offer WOFF-encoded fonts for web use. Among them, Monotype Imaging, Ascender Corporation, FontShop and Typotheque provided us with the great WOFF fonts used in our our Testdrive demos.

Same CSS, Same Web Font Format

Starting with Platform Preview 3, IE9’s @font-face implementation conforms to the CSS3 Fonts module; supported font formats include EOT and WOFF as well as raw fonts with embedding permissions set to installable.

In future posts, we will look into IE9’s web font support in more detail, as well as the available techniques for cross-browser typography. In his latest post on the FontBlog, Greg Hitchcock offers rich context on the art and science behind the technologies involved in web typography.

We look forward to your feedback on this Preview release, as well as ongoing work within the W3C to extend the reach of the typographic web.

Sylvain Galineau
Program Manager

The network plays a crucial role in the overall performance of a web browser. The best way to improve network performance is to minimize the volume of network traffic by using HTTP compression and taking advantage of the browser cache.

We’ve made a tremendous number of improvements to the way that Internet Explorer 9 caches content to ensure that as many resources as possible are loaded from the cache. This post describes those improvements which are now available in the third IE9 Platform Preview which was released last month.

Understanding Caching

Let’s start with a quick refresher on how caching works in browsers. At a high level, web browsers make two types of requests over HTTP and HTTPS—conditional requests and unconditional requests.

An unconditional request is made when the client browser does not have a cached copy of the resource available locally. In this case, the server is expected to return the resource with a HTTP/200 OK response. If the response’s headers permit it, the client may cache this response in order to reuse it later.

If the browser later needs a resource which is in the local cache, that resource’s headers are checked to determine if the cached copy is still fresh. If the cached copy is fresh, then no network request is made and the client simply reuses the resource from the cache.

If a cached response is stale (older than its max-age or past the Expires date), then the client will make a conditional request to the server to determine whether the previously cached response is still valid and should be reused. The conditional request contains an If-Modified-Since and/or If-None-Match header that indicates to the server what version of the content the browser cache already contains. The server can indicate that the client’s version is still fresh by returning HTTP/304 Not Modified headers with no body, or it can indicate that the client’s version is obsolete by returning a HTTP/200 OK response with the new version of the content.

Obviously, conditional requests result in better performance than unconditional requests (because the server need not retransmit the entire file if the client’s version is fresh) but the best performance is obtained when the client knows that the version in its cache is fresh and the conditional revalidation can be avoided entirely.

Extremely Long-Life Cache Headers

While RFC2616 recommends that servers limit freshness to one year, some servers send Cache-Control directives specifying a much longer freshness lifetime. Prior to IE9, Internet Explorer would treat as stale any resource with a Cache-Control: max-age value over 2147483648 (2^31) seconds, approximately 68 years.

With Internet Explorer 9, we now accept any value up to 2^63 for the max-age value, although internally the freshness interval will be truncated to 2^31 seconds.

Vary Improvements

The HTTP/1.1 Vary response header allows a server to specify that a fresh cached resource is valid for future reuse without server revalidation only if the specified request headers in the later request match the request headers in the original request.

For example, this enables a server to return content in English with a Vary: Accept-Language header. If the user later changes their browser’s Accept-Language from en-US to ja-JP, the previously cached content will not be reused, because the Accept-Language request header no longer matches the request header at the time that the original English response was cached.

With Internet Explorer 9, we’ve enhanced support for key Vary header scenarios. Specifically, IE9 will no longer require server revalidation for responses which contain Vary: Accept-Encoding and Vary: Host directives.

We can safely support these two directives because:

1. All requests implicitly vary by Host, because the host is a component of the request URL.
2. IE always decompresses HTTP responses in the cache, making Vary: Accept-Encoding redundant.

Like IE6 and above, IE9 will also ignore the Vary: User-Agent directive.

If a response contains a Vary directive that specifies a header other than Accept-Encoding, Host, or User-Agent (or any combination of these) then Internet Explorer will still cache the response if the response contains an ETAG header. However, that response will be treated as stale and a conditional HTTP request will be made before reuse to determine if the cached copy is valid.

Redirect Caching

Internet Explorer 9 now supports caching of HTTP redirect responses, as described by RFC 2616. Responses with Permanent Redirect status (301) are cacheable unless there are headers which forbid it (e.g. Cache-Control: no-cache) and those with Temporary Redirect status (302 or 307) are cacheable if there are headers which permit it (e.g. Cache-Control: max-age=120).

While this significantly improves performance, web applications that are misconfigured might not work as expected after this change. For example, we’ve found a few commonly-visited sites that use a pattern which looks like this:

> GET / HTTP/1.1 < 301 Redirect to /SetCookie.asp > GET /SetCookie.asp HTTP/1.1 < 301 Redirect to /

The site’s goal is to have the homepage determine if the user has a cookie set, and if not, send them to a page that sets the cookie. The problem is that the server has chosen a 301 for this task, and a 301 is cacheable. Hence, IE will simply redirect between these two cached redirects on the client (never again contacting the server) until the user gets bored and closes the browser. Notably, any version of IE would hit a redirect loop in the scenario above if the user had disabled cookie storage for the site in question.

If your site makes use of redirects, you should ensure that it is configured to avoid redirect loops by ensuring that any redirect that relies upon side-effects (e.g. testing or setting a cookie) is marked uncacheable.

HTTPS Caching Improvements

A few months ago, I mentioned that Internet Explorer will not reuse a previously-cached resource delivered over HTTPS until at least one secure connection to the target host has been established by the current process. This can cause previously-cached resources to be ignored, leading to unconditional network requests for content that was already in the local cache.

In IE9, the unnecessary cross-host HTTPS requests are now conditional requests, so the server can simply return a HTTP/304 Not Modified response for unchanged content. While a round-trip cost is still incurred, significant performance improvements are gained because the server does not need to retransmit the entire resource.

Back/Forward Optimization

For IE9, we’ve made improvements so that clicking the back and forward buttons results in faster performance.

RFC2616 specifically states that a browser’s Back/Forward mechanisms are not subject to cache directives:

History mechanisms and caches are different. In particular history mechanisms SHOULD NOT try to show a semantically transparent view of the current state of a resource. Rather, a history mechanism is meant to show exactly what the user saw at the time when the resource was retrieved. By default, an expiration time does not apply to history mechanisms. If the entity is still in storage, a history mechanism SHOULD display it even if the entity has expired, unless the user has specifically configured the agent to refresh expired history documents.

In previous versions of Internet Explorer, when the user navigated back or forward, IE would check the freshness of resources if they had been sent with the must-revalidate cache-control directive, and in numerous other circumstances depending on how recently the resource was downloaded. In IE9, the INTERNET_FLAG_FWD_BACK flag behaves as described on MSDN, and IE will not check the freshness of cached resources when the user navigates Back or Forward.

As a result of this optimization, Internet Explorer 9 can perform far fewer conditional HTTP requests when navigating with Back and Forward. For example, the following table shows the improvement when going Back to a typical article on a popular website:

IE8

IE9

Improvement

Back/Forward Navigation

Request Count: 21

Bytes Sent: 12,475

Bytes Recv: 216,580

Request Count: 1

Bytes Sent: 325

Bytes Recv: 144,617

Request Count: -20 (-95%)

Bytes Sent: -12,150 (-97.4%)

Bytes Recv:-71,963 (-33.3%)

Now, I mentioned that we ignore caching directives when navigating back and forward, so alert readers may be wondering why IE9 still makes one request when clicking Back on this site. The reason is that IE will not commit to the cache any uncacheable resource. An uncacheable resource is one delivered with a Cache-Control: no-cache directive or with an Expires date in the past or an Expires date not later than the Date header. Therefore, the browser is forced to redownload such resources when the user navigates Back and Forward. To improve performance and enable a resource to be reused in Back/Forward navigation while still requiring revalidation for other uses, simply replace Cache-Control: no-cache with Cache-Control: max-age=0.

Unlike the other improvements in described in this post, back/forward optimizations are not visible in the Platform Preview build because it does not have a back button.

Heuristic Cache Improvements

Best practices recommend that web developers should specify an explicit expiration time for their content in order to ensure that the browser is able to reuse the content without making conditional HTTP requests to revalidate the content with the server. However, many sites deliver some content with no expiration information at all, relying upon the browser to use its own rules to judge the content’s freshness.

Internet Explorer allows the user to configure what should happen when content is delivered without expiration information. Inside Tools > Internet Options > Browsing history > Settings, you will see four options:

These four options have the following behavior:

Every time I visit the webpage

Any resource without explicit freshness information is treated as stale and will be revalidated with the server before each reuse.

Every time I start Internet Explorer

Any resource without explicit freshness information is validated with the server at least once per browser session (and every 12 hours thereafter in that session).

Automatically (Default)

Internet Explorer will use heuristics to determine freshness.

Never

Any cached resource will be treated as fresh and will not be revalidated.

These options only control the browser’s behavior when content is delivered without expiration information; if the content specifies an explicit policy (e.g. Cache-Control: max-age=3600 or Cache-Control: no-cache) then the browser will respect the server’s directive and the options here have no effect.

In earlier IE versions, the Automatic Heuristics were simple and only affected cached images, but IE9 improves the heuristics to match RFC2616’s suggested behavior:

if the response does have a Last-Modified time, the heuristic expiration value SHOULD be no more than some fraction of the interval since that time. A typical setting of this fraction might be 10%.

If Internet Explorer 9 retrieves a cacheable resource which does not explicitly specify its freshness lifetime, a heuristic lifetime is calculated as follows:

max-age = (DownloadTime - LastModified) * 0.1

If a Last-Modified header wasn’t present in the server’s response, then Internet Explorer will fall back to the “Once per browser session” revalidation behavior.

As a result of the improvement to heuristic caching, Internet Explorer 9 can perform far fewer conditional HTTP requests when reloading many pages. For example, the following table shows the improvement when revisiting a typical article on a popular website:

IE8

IE9

Improvement

Revisit in new browser session (PLT2)

Request Count: 42

Bytes Sent: 26,050

Bytes Recv: 220,681

Request Count: 2

Bytes Sent: 1,134

Bytes Recv: 145,217

Request Count: -40 (-95.3%)

Bytes Sent: -24,916 (-95.6%)

Bytes Recv: -75,464(-34.2%)

The Caching Inspector in Fiddler will show you when a response expires, based on the headers provided on that response. For instance, here’s what you see for a response which contains an ETAG and Last-Modified header, but no expiration information:

Other Networking Improvements

In this post, I’ve enumerated the improvements in Internet Explorer’s caching code that help ensure web sites can make the most efficient possible use of the network. Of course, web developers should continue to follow best practices and specify their desired cache behavior using the Expires and Cache-Control headers, but even sites that fail to do so will load more quickly in IE9.

In a future post, I’ll describe other improvements we’ve made to the IE9 Networking Stack to further improve page load times.

-Eric Lawrence

A few weeks ago, we talked about the performance characteristics of our Flickr Explorer sample. We showed how hardware acceleration benefits real world scenarios such as browsing photos, and how easily web developers can build these types of applications.

Recently, we released a new set of demos alongside the third IE9 Platform Preview. Today we’re going to discuss the Amazon Shelf concept application (also see the companion Channel 9 video).

Much like Flickr Explorer, Amazon Shelf is written using standard HTML, CSS and Javascript. Amazon Shelf also incorporates a key new HTML5 feature – the canvas element. Canvas is an incredibly powerful way to draw directly to the screen using simple Javascript API calls.

When you launch Amazon Shelf, you’re shown a list of the top selling books from Amazon. This data is retrieved using the Amazon Product Advertising API. You can search for specific books, browse, and “open” books to view detailed information and customer reviews.

This demo uses common patterns that you find across many interactive web applications and games. There is one main loop that updates the books and other objects on the screen, and performs simple hit testing to support interacting with the elements on the canvas.

Canvas, like all graphics in IE9, is fully hardware accelerated by default. When IE9 users browse to a website that uses canvas, IE will automatically leverage the full capabilities of the PC to provide a great experience with levels of performance not possible with today’s browsers. Using IE9, Amazon Shelf is generally able to maintain a responsiveness of 60 frames per second, which is considered realtime. Today’s browsers are only able to achieve framerates of 1-8fps which is a small fraction of the performance provided by IE9.

We recently blogged about using the Windows Performance Tools to analyze browser performance. Using these tools, we’ve taken some measurements of loading Amazon Shelf in the top browsers available today. We used the same hardware and methodology discussed in the past. Let’s look at the CPU and GPU activity graphs to better understand how the demo performs in these browsers.

Note: Internet Explorer 8 is not included in this comparison since it does not support the Canvas element.

First up is Chrome 5. Chrome is able to update the screen once every 0.99 seconds, yielding a frame rate of about 1 FPS during the bookshelf load animation. This results in a very slow, choppy experience. One core on this dual core machine is fully utilized, and the GPU is not employed by the browser at all.

Here are the results for Safari 5. During the load animation, Safari does not attempt to render the scene at all, resulting in an effective 0 frames per second. Again, one core on the CPU is fully utilized and the GPU remains untouched.

Next up, Firefox 4 Beta. We used Minefield 4.0b2pre nightly for this analysis. Again, our tests ran this latest nightly build of Firefox (like all the others) in the default configuration. This means hardware rendering with the GPU was not enabled in Firefox.

Here are the results for Firefox. The animation is rendered properly, and the screen is updated twice every .25 seconds, yielding a frame rate of about 8 FPS.

Finally, let’s take a look at Internet Explorer 9 Platform Preview 3. We see that IE9’s full usage of the GPU results in a steady, smooth frame rate of 60 FPS. The CPU handles the task without any trouble and rests frequently while the GPU renders Amazon Shelf to the screen.

There is a meaningful difference in the experience when running the demo in IE9 compared to other browsers. Check out Amazon Shelf on www.ietestdrive.com to see for yourself.

We’d like to thank Amazon for their help in putting this demo together, and embracing the new GPU powered, standards based markup enabled by Internet Explorer 9.

Our team can’t wait to see what other graphically rich experiences web developers armed with hardware accelerated Canvas will dream up!

Seth McLaughlin
Program Manager for IE Performance

Vision Mobile just released its Mobile Developer Economics report in which it presents the result of a poll of 401 mobile developers across the eight main platforms: Android, iPhone, BlackBerry, Symbian, Windows Mobile, Flash, Java ME, and the mobile web.

If you’re interested in the mobile developer world, download and read the report. It’s free, though a valid email address is required. Below I treat some interesting aspects of the research, including the quote from me Vision Mobile decided to publish.

Platforms

The parity between the installed base of a platform and the amount of apps available is totally absent. As one would expect, the iPhone app store offers way more apps than the OS’s relatively humble market share warrants. No surprise here. (p. 10)

What is surprising is that the most popular platform among the 401 developers is not the iPhone, but Android with 60%. iPhone comes distinctly second at around 50%, and the third most popular platform, Java ME, is not far removed from it. No clear iPhone superiority here.

The mobile web does decently at 40% (i.e. 40% of the developers use the mobile web from time to time). Flash is used by about 20% of developers. The long tail is composed of Linux, BREW, bada, and webOS. It should be noted, however, that bada has only just been released. Thus its meager score is quite understandable. This is not the case for the other three platforms. (p. 12)

It should be noted that developers who used Java ME were rather negative about the platform. One said:

The vast majority of Java ME developers have lost faith in the write-once-run-anywhere vision

Incidentally, which technology do you think has the best chance to take over this write-once-run-anywhere vision? Hint: it has existed for twelve years already and has already been deployed across all smartphones.

Striking it rich

Why do developers choose a platform? The clear number one reason is market penetration at 75%, with revenue potential coming second at 50%. The existence of an app store or a development community is important for only 40% of the developers. (p. 13)

Frankly I believe that reason number two is going to disappear. Mobile development is not a goldmine, although obviously you can earn a decent living by making apps for others.

On page 26 the report explicitly treats the long-tail economics. If you’re just a random developer who tries to make boatloads of money by selling apps, will you succeed? Vision Mobile doesn’t think so.

The economics for long-tail developers - i.e. the per-capita profit for the average developer - remains dubious at best.

In other words, Apple’s App Store is not a shortcut to riches. It can’t take the place of the hard work of founding a succesful startup, making a compelling product that people want to use, and selling it to a large player.

The App Store will dole out only niche income to most developers. Obviously there will be one or two notable exceptions to that rule. It’s a casino, after all, and the luck of the draw can be with you. Wanna try? Have fun, but don’t complain that you don’t make any money.

Why is this so? The report pinpoints two main reasons:

1. First, there are plenty of “garage developers” around who create apps not for the money, but for status among their peers. Usually they offer their work for free or a nominal fee. If such a free app competes with your paid one, tough luck.
2. The huge crowding especially in Apple’s app store forces developers to lower their prices in order to remain competitive.

Mastering a platform

How much time does it take to master a platform? Android clocks in fastest with five months. The mobile web comes second with six (yay!), though it has to share this place with Flash. Then comes iPhone with about 7.5 months. By far the toughest nut to crack is Symbian, where it takes 15 months before you’re proficient enough with the platform to write a couple of moderately-simple apps. (p. 32)

Operators

Finally, the report talks about operators, and this happens to be a subject I’m deeply interested in. The general conclusion is that operators don“t care a hoot about mobile developers, and treat them badly. Questions aren’t answered, there is a complete lack of focus with initiatives and websites cropping up all over the place, and operators aren’t interested in the long tail of niche developers.

In other words, ADHD-suffering marketing people jump hither and tither, and don’t care about anything remotely resembling long-term planning. I don’t think that’s going to change in the near future.

Furthermore the operators refuse to send people to industry conferences, instead electing to hold their own. I’ve been to several such events, and although it always works in the sense that it’s a moderate-good conference with moderate-good speakers, the fact that they’re free to enter but not free from marketing speak means that they’re soon forgotten.

I am convinced that the operators’ approach to conferences is wrong. I hope to prove that within the next year.

Which brings me to the quote from me Vision Mobile decided to include (p. 43/44):

The first mobile company to TRULY reach out to web developers will have an edge over the competition, but right now I don’t see any candidates. Except for Google, obviously. (And Apple, but they’re playing their own game.) If Google became an operator our problems would be solved.

To be honest I completely forgot about this thought until Vision Mobile contacted me for permission to quote me. Reading it back I feel I’m on to something. So yes, I agree with myself.

Wondering what Google’s next step will be? Becoming an operator. Especially in the totally dysfunctional American market. They’ll bring simple mobile billing to the world. Unless the traditional operators do it first, but there’s little chance of that.

Interesting idea, no? Not easy, no sir, not at all. But it’s the obvious next step as far as I’m concerned. Operatorship is where the money is. And where the need for innovation is greatest. We don’t have to expect anything from the current marketing-driven bunch.

var ctx0, ctx1; var canvas0, canvas1; var wave = 1; var direction = 1; var timer; var HEIGHT2 = 100, WIDTH2 = 400; var count = 0; var currentX = 0, currentY = 0, lastX = 0, lastY = 0; var r = Math.floor(Math.random() * 255) + 70; var g = Math.floor(Math.random() * 255) + 70; var b = Math.floor(Math.random() * 255) + 70; window.onload=init; function init() { // Check to see if canvas is supported if (!document.createElement('canvas').getContext) return; // Set up canvas0 ctx0 = document.getElementById("canvas0").getContext("2d"); ctxWidth = document.getElementById("canvas0").width; ctxHeight = document.getElementById("canvas0").height; // Set up canvas1 canvas1 = document.getElementById("canvas1"); ctx1 = canvas1.getContext("2d"); ctx1.lineWidth = 25; ctx1.lineCap = "round"; // Add event handlers if (canvas1.addEventListener) canvas1.addEventListener("mousemove", OnMouseMove, false); else if (canvas1.attachEvent) canvas1.attachEvent("onmousemove", OnMouseMove); // Start the renderLoop timer = setInterval(renderLoop, 16); } function OnMouseMove(e) { if (typeof e == 'undefined') e = canvas1.event; if (typeof e.offsetX != 'undefined' && typeof e.offsetY != 'undefined') { currentX = e.offsetX; currentY = e.offsetY; } else { var relPos = getRelativePos(e.clientX, e.clientY); currentX = relPos[0]; currentY = relPos[1]; } } // My thanks to QuirksMode.org for the insight here function getRelativePos(x, y) { var curleft = curtop = 0; var obj = document.getElementById('canvas1'); if (obj.offsetParent) { do { curleft += obj.offsetLeft; curtop += obj.offsetTop; } while (obj = obj.offsetParent); } // Webkit isn't compliant with CSS OM View here; thus, we need to grab scrollTop from body instead of documentElement if (document.body.scrollLeft > 0) { var scrollLeft = document.body.scrollLeft; } else { scrollLeft = document.documentElement.scrollLeft; } if (document.body.scrollTop > 0) { var scrollTop = document.body.scrollTop; } else { scrollTop = document.documentElement.scrollTop; } return [(x - curleft + scrollLeft), (y - curtop + scrollTop)]; } function drawLines(ctx, x, y) { ctx.save(); ctx.beginPath(); ctx.moveTo(lastX, lastY); ctx.lineTo(x, y); ctx.strokeStyle = "rgba(" + r + "," + g + "," + b + ", 1)"; ctx.stroke(); ctx.restore(); } function clearLines(ctx) { // Clear first ctx.fillStyle = "rgba(0,0,0,0.05)"; ctx.fillRect(0, 0, WIDTH2, HEIGHT2); // Change up color if (count++ > 50) { count = 0; r = Math.floor(Math.random() * 255) + 70; g = Math.floor(Math.random() * 255) + 70; b = Math.floor(Math.random() * 255) + 70; } } function renderLoop() { // Draw lines clearLines(ctx1); drawLines(ctx1, currentX, currentY); lastX = currentX; lastY = currentY; drawSimpleShapes(ctx0); } function drawSimpleShapes(ctx) { // Draw background if (wave >= 60 || wave <= 0) { direction = -direction; } var gradient1 = ctx.createLinearGradient(0, 15, 0, 70); gradient1.addColorStop(0, "blue"); gradient1.addColorStop(1, "white"); ctx.clearRect(0, 0, 300, 70); ctx.fillStyle = gradient1; ctx.beginPath(); ctx.moveTo(0, 35); ctx.quadraticCurveTo(75, 70 - wave, 150, 35); ctx.quadraticCurveTo(225, 0 + wave, 300, 35); ctx.lineTo(300, 70); ctx.lineTo(0, 70); ctx.fill(); wave = wave + direction / 5; // Draw green rectangle ctx.beginPath(); ctx.fillStyle = "rgba(0, 255,0,.3)"; ctx.rect(10, 10, 50, 50); ctx.fill(); ctx.stroke(); // Draw blue circle ctx.beginPath(); ctx.fillStyle = "rgba(0, 0, 255,.3)"; ctx.arc(105, 35, 25, 0, 2 * Math.PI, true); ctx.fill(); ctx.stroke(); // Draw lines with decreasing widths ctx.strokeStyle = "black"; ctx.lineWidth = 10; ctx.beginPath(); ctx.moveTo(155, 15); ctx.lineTo(205, 15); ctx.stroke(); ctx.lineWidth = 8; ctx.beginPath(); ctx.moveTo(155, 29); ctx.lineTo(205, 29); ctx.stroke(); ctx.lineWidth = 6; ctx.beginPath(); ctx.moveTo(155, 41); ctx.lineTo(205, 41); ctx.stroke(); ctx.lineWidth = 4; ctx.beginPath(); ctx.moveTo(155, 51); ctx.lineTo(205, 51); ctx.stroke(); ctx.lineWidth = 2; ctx.beginPath(); ctx.moveTo(155, 59); ctx.lineTo(205, 59); ctx.stroke(); // Draw image var image = document.getElementById("ie-logo"); ctx.drawImage(image, 220, 0, 70, 70); }

With the recent release of the latest IE9 platform preview, we talked about how we’re rebuilding the browser to use the power of your whole PC to browse the web, and to unlock a new class of HTML5 applications. One area that developers are especially excited about is the potential of HTML5 canvas. Like all of the graphics in IE9, canvas is hardware accelerated through Windows and the GPU. In this blog post we discuss some of the details behind canvas and the kinds of things developers can build.

Canvas enables everything from basic shapes to fully interactive graphics

Canvas is a way to program graphics on the web. The <canvas> tag is an immediate mode 2d drawing surface that web developers can use to deliver things like real time graphs, animations or interactive games without requiring any extra downloads.

At the most basic level, canvas enables you to draw primitives like lines, rectangles, arcs, Bezier curves, quadratic curves, images and video like the following:

This image is a simulation of what you'd see in a canvas enabled browser.

Please use the IE9 preview to see these examples running in canvas.

The Canvas Pad demo on the IE test drive site goes into detail on the canvas syntax and enables you to easily experiment with a wide range of examples. Feel free to make changes to any of the samples that are there to see how it works -- for example, try changing colors or sizes of things.

Taking things a step further, you can use JavaScript to animate canvas drawings or make interactive experiences. The next example draws lines as you move your mouse (or as you move your finger on touch enabled devices) over the black box. You could also choose to have your canvas experience react to keyboard input, mouse clicks or any browser event.

This image is a simulation of what you'd see in a canvas enabled browser.

With canvas support in IE9, you can move your mouse over the black box and draw lines.

By utilizing the full power of the PC with hardware acceleration for graphics and fast JavaScript for animation, web developers can use IE9 to build deep, graphically rich experiences. Since canvas is an element like other elements in HTML, it participates in the page layout and its API is exposed to JavaScript so it can be fully incorporated into a web page's design. This makes it possible for sites to include things like live data visualizations, games, splash pages and ads without the need for any extra downloads or plugins.

The IE testdrive site includes several examples that demonstrate the kinds of things that sites are now able to do in an interoperable way.

Shopping

The Amazon Shelf shows what shopping for books could look like when the web site designer is able to use the kind of graphics, animations and smooth transitions that canvas enables.

--> var vid = document.getElementById("myvid"); if(vid && vid.canPlayType && !vid.canPlayType("video/mp4").match(/^(probably|maybe)$/i)) { vid.parentNode.insertBefore(vid.firstElementChild, vid); vid.style.display = "none"; document.getElementById("image1").style.display="none"; document.getElementById("div1").style.width="377px"; } /*script to hide the play image if in IE8 */ if (vid && vid.canPlayType==null) { document.getElementById("image1").style.display="none"; document.getElementById("div1").style.width="377px"; }

Immersive game experiences:

The following demos showcase some gaming concepts like physics, character animation, collision detection and mouse interaction coupled with hardware accelerated graphics. In these demos, you'll notice that not all browsers can update the screen with the same frequency (FPS or frames per second). IE is able to maintain a high FPS by using Windows technologies to make use of your GPU - your computer's hardware that's optimized for rendering graphics.

FishIE Tank

This demo makes use of sprites to animate the fish and basic collision logic to redirect the fish when they hit the edges of the tank. It’s also good for measuring graphics performance because you can change the number of fish to increase or decrease the graphics load.

--> var vid = document.getElementById("myvid2"); if(vid && vid.canPlayType && !vid.canPlayType("video/mp4").match(/^(probably|maybe)$/i)) { vid.parentNode.insertBefore(vid.firstElementChild, vid); vid.style.display = "none"; document.getElementById("image2").style.display="none"; document.getElementById("div2").style.width="377px"; } /*script to hide the play image if in IE8 */ if (vid && vid.canPlayType==null) { document.getElementById("image2").style.display="none"; document.getElementById("div2").style.width="377px"; } Asteroid Belt

The asteroid in the demo follows your mouse, scales and rotates. It’s an example of direct interactivity that you might find in a game.

Mr. Potato Gun

A physics engine in this demo defines how the different parts of Mr. Potato head are launched from the gun and then how they react when they bounce off the ground. Many games use some form of physics engine like this to manage particle movement and their response.

Canvas Zoom

This demo enables you to start with a very wide angle on an image like this mountain range and then zoom in very close image like people at a picnic. For games, it’s an interesting example of scaling and smooth transitions.

Demos from around the web:

There are some pretty amazing demos floating around the web and I'd like to share a couple of our favorites -- there are many more. An important part of implementing canvas is that we do it in an interoperable way so that developers can use the same markup. To help achieve this goal, we're always looking for examples that work and those that don't. A future canvas blog post will go into detail about how we work to be interoperable and what we do when there's an issue reported.

I hope you enjoy some of these canvas examples from people around the web.

Cloth Simulation

This demo is interactive and the cloth is responsive to movement and gravity.

Zwibbler

The shapes in this drawing app are preserved so you can select and then move, resize, or change their styling.

Liquid Particles

The particles in this demo are drawn to or repelled from the mouse.

Kaleidoscope

This one does a nice job of drawing you in – it’s engaging and interesting to watch the patterns as they evolve.

Nebula Visualization

The alpha blending used by this demo are really well done. The result is a cloudy atmospheric look. It’s graphics intensive and it’s still very fast and smooth in IE9.

Animated Reflection

The author of this demo says, “The script is currently using 80% of my cpu so it’s not really practical. Hopefully we will be getting JIT'd javascript sometime soon.” Well, now JavaScript is compiled in IE9. It generally uses about 1% of my CPU.

Asteroids in Canvas

This is a full game with nice graphics, collision detection, keyboard interactivity, score keeping and… green lasers.

Particle Animation

See your name in lights. This is another demo that includes a particle system. You can run this with 300 or 1500 sprites. Go ahead and bump it up to 1500.

We're looking forward to seeing the kinds visual experiences web developers will be able to build with a fully hardware accelerated browser.

Give it a try yourself. Watch the videos, get the latest platfrom preview, try out the canvas demos and build some examples of your own. If you find a bug in how canvas works or where the same markup behaves differently, please report bugs on Connect.

- Thanks, Paul Cutsinger and Jatinder Mann

'Under the Sea!' by Eric Stoltz

'Make 'em Proud!' by Michael McAghon and Scotty Reifsnyder

'Orchid Beauty' by Kevin Addison

'Oceanscape' by Justin Gray

'CSS Co., Ltd.' by Benjamin Klemm

'Sakura' by Tatsuya Uchida

'Kyoto Forest' by John Politowski