Daniel Eran Dilger in San Francisco
Random header image... Refresh for more!

Apple adds Flyover, deletes Google’s Street View from iOS 6 Maps

Daniel Eran Dilger

Apple is incorporating advanced 3D imaging to bring a variety of new features to iOS users, from building outlines to topographical terrain to fully rendered 3D models that not only replace Google’s StreetView, but offer to provide 360 degree views across neighborhoods, behind buildings and even off roads.
..
Apple’s strategy of taking its Maps solo in iOS 6 was described in Apple wants to wipe Google off the map with iOS 6, while a second segment, How Apple’s new vector-based Maps leave Google Maps looking jittery detailed how the company will be levering vector graphics to greatly improve the Maps experience in iOS.

However, there are a variety of attractive features Google has bundled into its web and app-based Google Maps portfolio, providing a tall order for anyone hoping to replicate all those features. Apple certainly has big shoes to fill in the maps department. Among these are:

Google’s 3D models and StreetView maps

Google Earth began mapping the world with 3D building outlines, a feature Google later brought into the Android version of its mobile maps client. Apple never adopted these 3D models however, so all iOS users see in the current iOS 5 Maps client is a 2D depiction of the building outlines Google has added to the bit mapped tiles it serves Apple’s Maps client, without any ability to freely rotate the map (below, in contrast with Google’s 3D massing models visible in Google Maps for Android).


More recently, Google has introduced 3D features that essentially paint images from aerial photography onto building outlines, resulting in a nearly photorealistic 3D model that users can explore from different angles. These features require significant client-side processing power to render, and just as with Google’s basic 3D outlines, Apple hasn’t ever incorporated them into its iOS Maps client.

In order to provide an explorable map that is efficient in both processing and bandwidth requirements (compared to 3D model rendering), Google first began its StreetView project, which sends out vehicles to all major roads, taking panoramic images that users can then access in a “you are there” view of virtually any block face visible from virtually any significant road Google’s camera-equipped cars have traveled.

Conceptually, StreetView can be dated back to an ARPA project developed at MIT, which in 1978 began mapping the town of Aspen, Colorado, using photographs stored on interactive Laserdisc, allowing users to virtually walk down streets, and even bring up seasonal views of the town from multiple perspectives.

The Aspen Movie Map was one of the first advanced examples of hypermedia, and was described by a young Steve Jobs in a 1983 speech describing the future potential of personal computing.

“It’s really amazing,” Jobs said, describing the project in contrast with conventional, static forms of old media. “It’s not incredibly useful,” he added as the audience laughed, “but it points to some of the interactive nature of this new medium which is just starting to break out from movies, and will take another five to ten years to evolve.”

Ten years later in 1994, Apple would release QuickTime VR, which allowed photographers to stitch together photos taken from a single location into a node that viewers could later explore dynamically, creating a new type of movie where playback was no longer fixed along a linear progression. Users could also jump between nodes, effectively exploring a virtual model of a real or imagined universe one spot at a time (or, alternatively, view an object from multiple perspectives).

QuickTime VR finds a use, 13 years later, in StreetView

While a variety of companies copied or extended upon Apple’s work on QuickTime VR, there was never a huge market for the technology. That is, until Google realized that panoramic photography would make an ideal way to efficiently allow Google Maps users to explore the world right from their web browser.

Thus, Google’s StreetView exploited a useful application for the QuickTime VR technology Apple pioneered commercially in 1994, four years before Google was founded as a company and 13 years before Google debuted StreetView in 2007.

While not without controversy from privacy advocates (and with government stopping Google’s work of StreetView mapping the world in countries like Australia, Germany and India), Google has effectively covered vast areas of Europe, Russia, North America, Brazil, Japan, Thailand, South Korea and South Africa with its camera cars.

Apple incorporated support for StreetView in iOS 2.2 Maps in 2008, but accessing the feature is not necessarily obvious; users must drop a location pin, bring up its information panel and then tap the StreetView icon to view a local panorama image of the spot (if its is available, and not obscured by passing vehicles or other obstructions).

Users can then step along the street in increments, turn down other mapped streets, and get close enough to houses to see right in homeowners’ windows.

On the web, Google still relies upon the Adobe Flash plugin to view StreetView panoramas, making them inaccessible to iOS users outside of Apple’s specially designed Maps client. In iOS 6, Apple has removed the ability to access Google’s StreetView images.

Duplicating Google’s four years’ worth of efforts in mapping streets from the ground level would be an enormous task (although other mapping companies have started doing this). But instead of skating to where the puck was in 2007, Apple has started promoting a new, overlapping feature for Maps in iOS 6: Flyover.

Flyover’s virtual StreetView

In August 2011, Apple bought C3 Technologies, known for its work in developing 3D images based on aerial or satellite images. By melding building model outlines with photos taken from satellites or low flying aircraft, C3 figured out how to create virtual models without necessarily sending cameras down the street.

While StreetView is tailored to discovering what an exact address looks like from the perspective of a passing car, Apple’s Flyover technology can provide both a bird’s eye view of building faces and an interactive, 360 view of panoramas from rooftops or hilltops or virtually any arbitrary, midair vantage point.

Another distinct advantage of Flyover over StreetView is that in StreetView, the user must proceed through panoramic nodes one step at a time. With Flyover, users can hover above a location, viewing the entire street and surrounding streets all at the same time, seeing a continuous representation of an entire path through a given neighborhood.

This continuous pan and zoom navigation is more akin to the approach of Google Earth rather than the “one step at a time” StreetView, but Flyover (below top) is distinctly sharper than Google Earth (below bottom) without going so far as offering a view into people’s windows.

One downside to the Flyover/Google Earth approach is that it is far more computationally intensive; rather than looking at a relatively simple, distorted and stitched photograph of a fixed panorama, the system must model thousands of 3D objects (buildings, tree and other structures) within a given scene, then wrap them with photographic skins.

Every 3D scene is dynamically active, meaning it can be rotated around 360 degrees, up and down, and at a wide range of viewing angles relative to the horizon in a vast 3D space. This provides a modeled city view reminiscent of video games such as “InFamous” (or “SimCity,” if you were born before 1985), except with far more detailed complexity and a viewing area a big as the world itself.

So far, there are a limited number of cities that have fully modeled support, but the infrastructure is in place to deliver a Flyover experience anywhere. Apple is also reportedly adding support for new cities (like Portland, Oregon, below) at a regular clip. There’s also a technology curve that taxes the resources of even the newest iOS hardware, but things can only improve as mobile devices get faster.

Google (among others) is also deploying its own 3D mobile maps, and would realistically be expected to be far ahead of Apple given its distant head start in mapping and 3D modeling with Google Earth. However, developers report that Google’s 3D maps are in many cases inferior to the developer builds visible in Apple’s iOS 6 Maps app.

Additionally, Apple is already providing models for areas Google hasn’t yet, such as this view of San Francisco’s Treasure Island, where Google Earth simply presents the map rendered as a flat surface (which Apple also does in areas where there are not yet models).

3D model view

If Flyover’s full 3D modeling seems too graphically intensive for today’s hardware, consider instead the highly efficient 3D model mode in iOS 6 Maps’ standard street view.

When viewing a location top down, you get a view similar to that provided by the current Google Maps server, except that building masses are rendered as 2D outlines rather than as flat 3D with a slight perspective effect. Below, San Francisco’s downtown Union Square area in standard 2D.

You can touch the 3D button to get a perspective view, where buildings and other structures pop up from the surface of the map. Drag two fingers up or down and the perspective changes from straight down to about 45 degrees downward. Below, the same view as above, but in standard 3D.

Spin the map with two fingers and you can look down actual street corridors, very similar to the views in Google Earth. Of course, building models only exist for a limited number of major cities. Even in areas where there are not yet models, 3D views of Map’s 2D vector outlines provide an additional level of perspective from an airplane-like view. Below, the same map, rotated to look up Market Street.

Where building models do exist, 3D “standard view” provides a useful overview of what a neighborhood’s structures look like. It isn’t really comparable to StreetView, but it is a very fast and responsive way to explore around neighborhoods. Below, Apple’s Cupertino campus in standard 3D mode.

3D Flyover with satellite images

Add in photo surface textures and of course rendering slows down, but things also get very interesting. While Google’s StreetView can be used to zoom into a particular block face, looking into a house’s windows, Apple’s Flyover approach provides a more “estimated” view, essentially a distorted depiction of an aerial photograph. Below, the same map view above, but with satellite images turned on, and below that, zoomed in to the Apple campus.

Flyover certainly isn’t the same as StreetView; the detail is rougher and often morphed or distorted. Apple doesn’t have to blur faces or license places in Flyover because you can’t make out details at that level anyway. However, Flyover provides a more continuous view of an area, and lets you observe it from an omniscient vantage point, floating upward or spinning the earth below you.

As you zoom around, Flyover loads estimated buildings and fills in details as needed, accounting for the rendering speed of your device and the throughput of your mobile network.

Like StreetView, Flyover provides an idea of what an area looks like before you get there. Unlike StreetView, it’s not tied to a series of points along primary streets, and there’s rarely a case where your StreetView is blocked by a passing truck or other street level obstructions (as often occurs to Google’s street view camera vehicles).

Which is creepier?

Flyover also allows you to see the backside of buildings, down alleys, and explore open spaces that Google can’t reach from its StreetView vehicles. Unlike StreetView, with Flyover you (and anyone else!) can explore your own backyard. Or, for example, the exclusive rooftop deck of the Paramount at Yerba Buena Gardens, where Apple hosts its WWDC after party.

However, with Flyover, you’re getting a massaged version of what is essentially aerial photography mapped to structure dimensions. StreetView shows what someone driving through your neighborhood would see: clear details of the front of your house, cars, pedestrians and so on, details that have generated significant controversy.

Apple is less likely to incite the same response from privacy advocates because, while Flyover lets you see more, it also shows you less. You can’t really spy in windows or see paparazzi-scale details. While both views are useful, StreetView is clearly more privacy-infringing simply due to its photographic realism shot from street-level spy cameras.

Flyover is unnerving in its own way, but it feels more like navigating a realistic map. Satellite and aerial images also raise some eyebrows among privacy advocates, but not to the same extent StreetView has. And Flyover is, essentially, just dynamic aerial imagery, essentially a product name Apple has applied to satellite maps when viewed from the 3D angle.

This also makes Flyover more discoverable as a feature, because it’s tightly integrated into the view options of Maps rather than being a special view mode you have to manually enter.

Google Maps for iOS 6?

Until iOS 6 ships, we won’t know whether Google will successfully introduce its own alternative Maps client for iOS users (it will likely want to) and whether Apple will approve it in the App Store (it may not, but Apple has already approved Google Earth, in addition to a variety of other apps it once held up on premise of “user confusion,” including Google’s Latitude and Google Voice).

If it does, Google and Apple will compete head to head on iOS in delivering vector maps, 3D views, turn by turn navigation and local information, although Apple will have a distinct home field advantage.

Not only is Apple’s Maps app going to be the default, bundled option, but there’s also no way to users (or an installed app) to specify a third party app as a default choice for handling maps. Further, Apple’s iOS will direct all third party apps to the company’s own servers when they make mapping API requests to draw a map in their own apps.

Besides vectors, 3D models and Flyover, there’s another new technology angle Apple will be leveraging in its own iOS 6 Maps app, which will be discussed in a forthcoming segment.

15 comments

1 stevelee { 08.31.12 at 12:44 pm }

I was recently in Seattle for a few days and used the Maps app frequently find the bus stop to walk to and the bus to take to my destination. I’m not traveling any time soon, but if Apple doesn’t implement that function by the next time I visit a city of any size, I’ll really miss that capability in iOS 6 after I upgrade my phone.

2 The Mad Hatter { 08.31.12 at 1:33 pm }

I’ve found the current Maps application incredibly useful, especially the “Walk” button, when I’m in Toronto (Toronto’s traffic isn’t as bad as New York’s, but it comes close, and walking is often the most efficient way to get places.)

Exactly how useful Flyover will be I don’t know. I may not even get to see it on my 3GS, which would be a pity because it looks really neat.

Wayne

3 gslusher { 09.01.12 at 1:15 am }

@stevelee:

I’m not sure, but I’ve read that transit systems and companies consider their routes, stops, etc, to be proprietary data. No one can use it without their permission, which may mean paying them. As long as Apple was using Google maps, it could probably rely upon Google’s licenses. Now, it will have to get its own licenses.

4 broadbean { 09.01.12 at 6:45 am }

iOS 6 flyovers is like a town run by zombies in the day time. I do like what Google StreetView offers, but as it is not updated regularly, it won’t be a deal breaker.

5 jmfree { 09.01.12 at 9:14 am }

Fascinating and amazing.

“Omniscient” and “creepy” perhaps apply even more.

A couple of years ago, there was a rash of break-ins in my neighborhood. They seemed rather systematic and targeted.

It occurred to me to request that Street View images of my house be removed from the Google database. The process was easy and successful. Now, when you take a virtual stroll down my quiet street to case all the homes and check out all the strategic entry and exit points, it’s as if my house doesn’t exist. A few neighbors did the same.

This is a starting point for thinking about “public” and “private”. Yes, my home is on a public street shared by all. But exactly why should all of its details be free subject matter for a publisher, including a zoomable high-definition tour of how my back yard connects to those of my surrounding neighbors? Need to plan the quickest path on foot to three different streets where the escape car can be parked? Be our guest.

I don’t see this as paranoia. I see it as simple prudence and common sense, like not leaving your tax returns in the recycling bin. The “creepy” feeling comes when you have zero control over how much people know about you. For this reason, I don’t have a sign in my front yard saying “Please enter and wander around.”

The maps technology is fabulous, but the demos always involve very public spaces, and for a reason: few people want to talk about what it means to private spaces.

Yeah, I’m being a whimpering wet blanket I suppose. After all, they caught the guy who had been raiding our neighborhood, showing there’s no substitute for good police work. (I didn’t get to interview him on his Google Maps skills.)

But, throughout the whole ordeal, the police task force kept advising everyone to lock their doors at night and keep an eye out for suspicious people or vehicles frequenting the area.

That’s when I realized that, whoever the creep was, he could already know everything he needs to know without setting foot in my neighborhood.

It’s interesting that the homeland security types have similar concerns about public spaces. But that just brings us back to thinking about what’s the difference between public and private.

Educational as always, Daniel. Thank you.

6 The Mad Hatter { 09.01.12 at 1:08 pm }

*jmfree,

Or maybe he lived in your neibourhood, or the one next door, or worked in the area. Criminals tend to work in zones where they feel safe, in other words places that they know. I suspect your thief had ties to the local area.

As to the criminal uses of mobile mapping technology, the phone system is also used for criminal acts, as is the postal system, and the Internet. Everything is used for crime. Criminals will take whatever advantage that they can get.

Which is why we have cops. There’s good cops, bad cops, and some who are a bit of both (do a search on Constable Bubbles, there’s a couple of YouTube videos catching him in action, and then remember that the same guy provided evidence against a cop accused of beating a protestor).

Wayne

7 lahaina { 09.01.12 at 1:25 pm }

Seems to me that the flyover view is more directly comparable to (and probably better than) the Bing bird’s eye view. It really is a great feature and will be fun, but street view is street view and another view that is not from street level is not comparable, at least to me.

8 jmfree { 09.01.12 at 2:43 pm }

“Everything is used for crime. Criminals will take whatever advantage that they can get.”

Oh, Wayne, please.

You’re saying exactly what I’m saying: it’s not the technology, it’s the use of the technology.

But we need more thinking about what’s an allowable use of the technology. I hope in your world it’s not ok to beam lasers at airplanes.

Techno-utopians love to dismiss any such discussion with a wave of the hand and some variant of “Guns don’t kill people — people kill people.” This leaves more time to talk about features and compare screen sizes on the playground.

It also makes it really obvious who doesn’t want to think about it at all.

9 MarkyMark { 09.01.12 at 9:35 pm }

Can’t wait! Tho I do use Street View occasionally and it can be very useful. Like recently an old friend who lives far away in Canada purchased a house, and I could use Street View to see it in surprising detail, and ask her about it. On the other hand I sometimes run into images that have been totally ruined by lens flare.

10 warlock7 { 09.04.12 at 8:12 am }

@jmfree:
Honestly?!? It’s not “techno-utopians” dismissing your paranoia. It’s anyone with a decent sense of right and wrong and a working sense of what freedom really means. Freedom in your world isn’t about freedom at all, based upon what you are saying. Where do you propose to draw the line in your world? Restrict use of everything?
It continually spirals until you cannot have anything or do anything. The fact still remains that it isn’t the tech that causes harm, it’s the people using the tech. Still a completely valid and true point to any reasonable and rational person.

11 jmfree { 09.04.12 at 2:14 pm }

It’s pretty simple, warlock7.

I want to opt out of people creeping around my home. Physically. Virtually.

Especially people who have no understanding of boundaries or privacy. Which includes, from what I’ve seen, your average techno-utopian.

Now, if you have a problem with that, then you’re who I’m talking about.

To my pleasant surprise, Google already agrees with me, and with many of my neighbors. Maybe I’m not such a wacko, after all.

Happy creeping… while you can.

12 warlock7 { 09.04.12 at 11:22 pm }

Wow. Interesting that you decide that I’m somehow out to “creep” around you and yours. Kind of gets back to my point about your paranoia.

Google agrees with you due to their being threatened with litigation. Private Road signs didn’t deter them. Did you have wifi on at your house when they did their drive-by? Well, guess what, they got a lot more than just pictures of your house and backyard. That’s not so much the people using the tech, but the company supplying the tech that’s at question. Besides which, I can make out very clearly from satellite photos what’s in my backyard, my neighbors, and everybody else’s for that matter. Even the ones that don’t allow the street view.

You’re the one making wild connections here. How does beaming lasers at airplanes have anything at all to do with photographs of the outside of your house? In what way does potentially risking the lives of hundreds of people compare with a street view photograph of your house? Again, WOW!

I don’t believe that I ever suggested anything other than the pure and simple fact that you are blaming technology and some perceived lack of restrictions on crime which you have no supporting evidence for. You stated that you blame Google street view for compromising your safety. How about somebody walking down your street? Couldn’t they accomplish the same thing? And with better accuracy too? Street view images don’t get updated all that frequently, but walking down your street will give me an accurate idea rather than a guess. You’re scared that somebody is going to see into your backyard via street view and it’s going to give them some sort of knowledge that wasn’t available to them before? You don’t just sound paranoid, you sound like you’re quite naive too.

13 jmfree { 09.05.12 at 8:46 am }

Jeez, warlock7, sorry to get you in such a tizzy.

But I can’t really follow what you’re trying to say. Something about me being paranoid and naive.

It’s funny because, during the peak of the same mini crime wave, I proposed to the neighborhood crime watch group that a good video camera aimed at the street might have a chance of capturing the perp’s car as it left the scene. There was quite a bit of shouting about how doing so on a public street would be an illegal invasion of privacy. It’s very much the same attitude around here toward red light cameras, which would probably save lives.

So, you see, there are several levels of sensitivity about all this, regarding both public and private spaces, for many, many people.

Saying “wow” a lot and claiming that it’s just me doesn’t change that.

But if it upsets you to discover that some are more sensitive to privacy than you, I suppose there’s nothing I can really do about that.

It’s already perfectly clear to me that some don’t want to allow the discussion because they themselves are “paranoid” that their precious toys will be taken from them.

In every other respect, have a great day.

14 The Mad Hatter { 09.06.12 at 10:16 am }

You forgot to include guns in there @jmfree.

I know a lot of Canadians who won’t visit certain U.S. states because of the guns, places like Oklahoma where 1 in 5 adults are carrying concealed weapons. You are right, limits do need to be set.

I’m just another sure your limit is the right one. If I wanted to find a place with lots of valuables, one option would be to check out the places WHICH don’t show up in StreetView, after all, they must have something to hide!

Wayne

15 jmfree { 09.08.12 at 5:22 pm }

True enough, Wayne.

But when enough people opt out, then the “No! I am Spartacus!” effect prevails. So, when you opt out, perhaps you’re not just helping yourself but others as well.

In the American West, it took a while to persuade cowboys that they would benefit by checking in their guns with the sheriff when they rode into town: no gun, you’re not a legal target for a duel.

Maybe we’ll get there.

You must log in to post a comment.