Why is “Look Around” rolling out so slowly?
After years of data collecting, why doesn’t Apple’s Street View cover more areas?
February 2021
⚠️ Tap or click any image to enlarge
After four years of driving, Apple announced in June 2019 that its imagery collection vehicles had driven “over four million miles”.
What makes this number interesting is that Apple appears to be driving the world faster than Google originally did:1
But when it comes to releasing this imagery, Apple has been releasing it much slower.
For instance, roughly one year after Google published its first Street View imagery, it had coverage in seventy-four metros and eleven major parks—while Apple, after its first year, had coverage in just ten metros:2, 3
These differences become even more dramatic after a year and a half. At that point, Google had Street View coverage in hundreds of metros:
While Apple had coverage in just twenty-one:
Here’s Google’s U.S. Street View coverage at that point:
And here’s Apple’s:
Additionally, many of the twenty-one metros that Apple had released were only partially covered.
Apple’s Look Around coverage for the New York City area, for example, covered less than half of New York City’s metropolitan population (just the city itself and a few smaller cities in New Jersey):
Meanwhile, Los Angeles’s Look Around area only covered the southern half of Los Angeles County—omitting large parts of Greater Los Angeles, including Orange County:
And Apple’s Bay Area coverage excluded nearly all of the North Bay and large portions of the East Bay:
So given the pace and extent of Apple’s Look Around releases, it was surprising to see Apple release Look Around for every metropolitan area in Canada—thirty-five in total—shortly after passing the one and a half year mark on December 10, 2020.
But what made this release so surprising was that Apple had not only covered every Canadian metropolitan area, in full—but that it had also covered the thousands of rural towns and villages between them:
Covering nearly all of Southern Canada (and 99% of Canada’s population)...
...this new Look Around area was more than one hundred times larger than the largest of Apple’s earlier Look Around areas—and more than twenty-seven times larger than all of Apple’s other Look Around areas combined:
But even more surprising than the size of Apple’s Canadian release was the speed at which Apple had delivered it.
Up until this point, Apple seemed to take four to five years to process its imagery. Apple, for instance, started driving the Bay Area in June 2015 and then first published its Bay Area imagery in June 2019—a four year turnaround from collection to release.4
And we see similar turnaround times for other Look Around cities:
But Canada, in contrast, took Apple just a year and seven months from collection to release.
What made this all even more surprising was that Apple hadn’t even started driving in Canada until nearly four years after it had started driving in the U.S.
And by that time, Apple had already driven most of the U.S.:
So given that Apple had been driving the U.S. for years before it began driving Canada, why was so much of U.S. still without Look Around coverage?
After all, it has now been more than five and half years since Apple first visited many U.S. cities that it hasn’t released Look Around for:
If Apple was able to collect and release nearly all of Southern Canada in under two years, then why hadn’t it covered more of the U.S. in five?
And that’s not even considering the eleven other countries that Apple had also been driving before it had started driving in Canada:
Apple has released imagery for parts of just three of these countries (Ireland, Japan, and the United Kingdom). So why does Look Around cover so much of Canada and so few of these others?
Whatever the reason, Apple’s Canadian release made it seem as if Apple was finally speeding up. But if that was the case, it didn’t entirely explain what happened next...
That’s because less than a week after Apple released its imagery for Canada, it released Look Around for three new U.S. cities (Denver, Detroit, and Miami):
But unlike Canada, each of these cities seemed to take Apple more than five years to process—Apple’s longest turnaround times yet:
So was Apple speeding up? Or slowing down?
While the turnaround times suggested Apple was slowing down, the imagery itself suggested the opposite—that Apple was processing imagery even faster. For instance, looking at Apple’s Miami area imagery, you see media stages for last year’s Super Bowl:
This suggests that Apple’s Miami area imagery was captured sometime in early 2020—not 2015.
And the imagery’s timestamp confirms it was “January 2020”:
And if we revisit other Look Around cities and look at their timestamps, we see relatively recent collection dates:
So why is Apple only using imagery from June 2018 onward?
And what happened to all of the imagery that Apple had collected between June 2015 and June 2018?
Was Apple not able to use it? Was there some sort of issue with it?
Or perhaps Apple wasn’t collecting imagery at all?
But if Apple wasn’t collecting imagery, then that doesn’t explain TechCrunch’s June 2018 report—the one that revealed that Apple was rebuilding Apple Maps:
In that story, TechCrunch reported that Apple’s data collection vans were “capped with sensors and cameras”, including “four LiDAR arrays mounted at the corners and eight cameras shooting overlapping high-resolution images”:
And when the story’s author rode inside of one of Apple’s vans, he observed that “each drive captures thousands of images as well as a full point cloud”.
TechCrunch’s report is from the same month as Apple’s oldest Look Around imagery (June 2018). And perhaps not coincidentally, this was also the same month that Apple was first spotted driving a new type of data collection vehicle, the Subaru Impreza:
And if you look closely at Apple’s Look Around imagery (especially in the reflections of windows), you’ll notice that it seems to be taken with these Suburus—not Apple’s vans:
So if Apple’s Look Around imagery is coming from the Subarus (and not its vans)—and all of it was taken after June 2018—then what were Apple’s vans doing during the three years that they were driving around the U.S. and eleven other countries?7, 8
And what prompted the switch to the Subarus in the first place?9
It all seems to suggest that there might have been an issue with the imagery from Apple’s vans. And if that’s the case, it helps explain why Apple seems to have spent the past year re-driving the U.S.:10
In any event, the switch to the Subarus—together with the imagery timestamps—shows us that Apple is processing imagery much faster than it seems. And Apple’s most recent releases (Miami, etc.) suggest that Apple can release imagery within a year of collecting it.
But digging even deeper into the imagery, it appears that Apple can go faster yet. For instance, if you look closely at Apple’s New York imagery, you’ll see that the glass cube at Apple Fifth Avenue was covered in a rainbow-tinted film at the time of the imagery’s capture:
According to media reports, this rainbow-tinted film was added on Friday, September 6, 2019:
What makes this interesting is that Apple went on to release its Look Around imagery for New York on September 30, 2019—twenty-four days after the rainbow-tinted film was added to Apple 5th Avenue.
In other words, Apple is able to process and release Look Around imagery within twenty-four days of collecting it.
As of January of last year, Apple has now collected imagery in all of the U.S.’s counties:
So if Apple can release imagery within twenty-four days of collecting it—and if Apple has collected imagery in every U.S. county—then why does Look Around still cover so little of the U.S.?
* * *
From time to time, I notice misplaced POIs on Apple Maps.
Early last year, for instance, an apartment building in my neighborhood was marked as a restaurant (“Sorrel”)—a restaurant that, in reality, was located twelve blocks away.
But when I went to look at the issue in Look Around, I didn’t see any label or icon for Sorrel in front of the apartment building:
And this has become a pattern: whenever I find a misplaced POI on Apple’s map, it’s always missing in Look Around.
For instance, right off of Patricia’s Green in San Francisco’s Hayes Valley neighborhood, Apple currently labels a restaurant called “Burgers 355”:
Except that “Burgers 355” doesn’t exist—and to my memory, never has.
Here’s Apple’s Look Around imagery—the building that’s labeled as “Burgers 355” on Apple’s map is actually the location of a popular bar (Brass Tacks) that’s been there for at least seven years:
Notice, though, that there’s no label for “Burgers 355” in Look Around—even though it’s labeled on Apple’s main map.
But misplaced POIs aren’t the only things missing from Look Around. Even stranger, businesses that have opened since Apple first published its Look Around imagery of the Bay Area (back in June 2019) are also missing.
A good example of this is San Francisco’s first and only Shake Shack location, opened in January 2020. Even though it’s labeled on Apple’s map...
...there’s no label for it in Look Around:
About a mile away from Shake Shack, a popular bakery that opened in September 2019—Susie Cakes—is labeled on Apple’s map...
...but not in Look Around:
And it’s not just small businesses. Chase Center, the new home of the Golden State Warriors (opened in September 2019), isn’t labeled in Look Around:
Nor is San Francisco’s newest hospital:
Something else peculiar about Look Around is that, unlike the main map, it shows labels and icons for businesses that have permanently closed. You may have noticed this earlier with Susie Cakes:
Instead of labeling Susie Cakes, Look Around labels the location’s former business (“Toss Designs”), which closed in 2018. And you can often find neighborhoods filled with greyed out labels for closed businesses:
Old businesses aren’t removed, new businesses aren’t added, and places shown on the main map are often missing. It all makes it seem as if Look Around is using a different POI database than the main map—one that’s frozen in time (sometime around late 2018).
But that’s not the only thing that makes it seem as if Look Around is using a different POI database. Another thing is that Look Around’s label placements often differ from the main map.
San Francisco’s Huntington Hotel is a good example of this. In Look Around, the hotel’s label is placed directly at its front door on California Street:
But on the map, the label is placed along Taylor Street where there’s no public entrance:
A similar situation exists at San Francisco’s Ferry Building. In Look Around, the Ferry Building is labeled at the end of its south wing:
But on the map, it’s labeled at the building’s center, away from its entrances:
Look Around’s label placements always seem to be at doors and entrances—and they’re often way better than those on Apple’s main map. And oddly, I’ve never been able to find a misplaced label in Look Around—even though I’ve seen many on Apple’s main map.
In fact, everything is labeled so perfectly in Look Around that it almost seems as if it was done manually, by hand.11 But this seems unlikely, given that it would take Apple weeks/months to do per city.
Then again, Apple’s Look Around releases have often been months apart:
The only exception to this is Apple’s recent releases, which started arriving more frequently last August. But if you look closely at Apple’s recent releases, you’ll notice something peculiar: they’re often missing POI labels.
The first Look Around release with missing POIs was Release #7 (August 2020), which covered metropolitan Tokyo, Osaka, and Nagoya. Two of these areas (Osaka and Nagoya) shipped without any POI labels at all:
And though there are POI labels in the center of Tokyo’s Look Around area...
...other parts of Tokyo, like Shinjuku, are missing them:
Apple’s next two releases after Release #7 had POI labels—but then Release #10 (December 2020), which covered Fukuoka, Hiroshima, and Takamatsu, also shipped without them:
Perhaps Apple’s extraction algorithms hadn’t yet been trained for Japanese? But if that was the case, it didn’t explain why some parts of Japan had POI labels, while others didn’t.
Nor did it explain why, a week later, most of Canada (a predominantly English-speaking region) also shipped without POIs:
It was bizarre. Canada’s three largest metros (Toronto, Montreal, and Vancouver) had POI labels...
...but everywhere else in Canada (including Calgary, Edmonton, and Winnipeg) didn’t:
It almost seemed as if Apple had been manually adding POI labels and then ran out of time (because its Canadian release covered such a large area).12
December 2020 was the first month with more than two Look Around releases. But were these releases arriving faster because Apple was taking shortcuts?
...suggest that Apple might be having issues algorithmically identifying POI information in its imagery. And if that’s the case, perhaps Apple is in fact manually adding POI annotations to Look Around?
That would certainly explain why Look Around has been rolling out so slowly. And it would also explain why Look Around appears to be using a different POI database than Apple’s main map.
But it also raises questions regarding Apple’s stated ambitions in AR.
Specifically, if Apple is unable to algorithmically identify POI information in static images (taken in ideal sunlight and weather conditions), then will it be able to successfully augment reality?15
And what happens if Apple’s rumored AR headset is ready before its map is?
__
1 Google first started collecting imagery in 2006 and by mid 2012 (six years later), its Street View vehicles had driven more than five million miles.
Five million miles over six years (Google) is slower than four million over four (Apple)—so Apple’s rate is faster.
(Sources for Google’s Street View vehicle mileage: 5 million miles in 2012, 7 million miles in 2014, 10 million miles in 2019.) ↩︎
2 By “roughly one year”, I mean one year and twelve days.
Google’s early Street View releases were often months apart. And Google had a major release just twelve days after hitting its one year anniversary:
Given how close this release was to Google’s one year anniversary, I’ve included it toward Google’s total (as it was likely the truest measure of Google’s progress at that point).
To be as fair as possible, I gave Apple the same extra twelve days I gave Google while calculating Apple’s total. ↩︎
3 Apple’s first three Look Around areas (Honolulu, Las Vegas, and the San Francisco Bay Area) were published on June 4, 2019 as part of the iOS 13 beta. Given that Apple’s imagery and coverage of these three areas was unchanged at the time of iOS 13’s wide release in September 2019, I’m considering June 4, 2019 as the date Apple first started publishing Look Around imagery. ↩︎
4 It’s worth noting here that Google seems to have published its initial Street View imagery for the San Francisco Bay Area less than a year after it began collecting it. (Google started collecting its Bay Area imagery in “late 2006” and then released it in May 2007—a turnaround time of somewhere between six and nine months.)
One would think that, with all of the technological advances since Street View was launched (e.g., bigger hard drives, faster processors, etc.), Apple would be able to process imagery faster than Google did in 2006/2007.
Then again, Apple’s imagery is of a higher quality than Google’s. And Google didn’t begin blurring faces until a year after Street View first shipped. ↩︎
5 Given that San Francisco and Los Angeles have the earliest capture dates, it’s surprising that Los Angeles—the U.S.’s second largest city—wasn’t among the initial Look Around areas that Apple released. ↩︎
6 Even though Apple’s imagery is relatively recent, it’s already older than Google’s in some areas, including parts of San Francisco and Los Angeles.
For instance, Apple’s imagery of its flagship store in San Francisco was taken in June 2018:
But Google’s current imagery was taken in June 2019, one year later than Apple’s:
Meanwhile in Los Angeles, Apple’s imagery of the Walt Disney Concert Hall was taken in October 2018:
While Google’s was taken in April 2019, six months later:
Given the extent of Street View’s coverage...
...it’s remarkable that Google’s imagery is more up-to-date.↩︎
7 The obvious answer is that Apple was building its map during these years, but here’s yet another peculiarity: Apple has been doing the opposite of what Google originally did.
Google shipped Street View first and then released its own map derived from Street View data two and a half years later. But Apple started by making its own map and then added Look Around later. ↩︎
8 According to its website, Apple offically began collecting data on June 15, 2015. But its vans were actually spotted driving around the Bay Area in February 2015:
And according to MacRumors, Apple’s vans were spotted as far back as August 2014 in New York City. So Apple may have been driving around even earlier. ↩︎
9 The switch to the Suburus in June 2018 also helps explain why notoriously secretive Apple gave TechCrunch such a deep look inside its vans earlier that month—i.e., because it was about to start phasing them out and switch to the Subarus.
If you look back at TechCrunch’s article you’ll see no mention of the Suburus—just Apple’s vans. So it seems that Apple never showed them to TechCrunch.
(Upon revealing all of details about Apple’s vans, TechCrunch declared: “This is their coming out party.” But in retrospect, it seems to have been their going away—and I can’t find any reports of Apple Maps van sightings after 2018.) ↩︎
10 Given how many years of imagery that Apple has captured, will Apple eventually offer a feature similar to Street View’s “Time Machine” feature?
11 Back in 2018, there were numerous hints of manual work in Apple’s initial map of California. And in the time since, there have been reports about Apple Maps “black sites”, which employ thousands of contractors who manually edit Apple’s map.
There has also been evidence of manual work across other Apple services, like Siri. ↩︎
12 If Apple is in fact manually adding the POI labels to its imagery, it would certainly explain why Look Around has been rolling out so slowly.
It also might explain why many of Apple’s Look Around areas cover such small parts of their respective metropolitan areas:
13 Given their relatively recent appearance, I’ve often wondered if the Look Around areas without POI labels are some sort of test by Apple.
In other words, is Apple deliberately omitting POIs to see if anyone notices that they’re missing? And if no one notices, will Apple omit them in future releases (so that it can release Look Around areas more frequently)? ↩︎
14 Google didn’t add POI labels to Street View until April 2010—three years after Street View’s initial release:
And then Google quietly removed them a couple years later.
So why did Google remove this feature so soon after adding it? Was there some sort of technical challenge?
Whatever the reason, Google added POI labels back to Street View last year:
15 One of the things that makes Apple’s Look Around imagery so interesting is the kind of features that can be built out of it. Google, for instance, uses Street View imagery to help power its AR navigation feature, “Live View”:
And Street View imagery is so important to Google’s AR implementation that Live View isn’t available in areas without it:
During WWDC 2020, we saw hints that Apple is taking a similar approach. Specifically, ARKit’s new “Location Anchor” feature appears to be enhanced by—and maybe even directly dependent upon—Apple Maps’s Look Around imagery for localization:
To place location anchors with precision, geo tracking requires a better understanding of the user’s geographic location than is possible with GPS alone. Based on a particular GPS coordinate, ARKit downloads batches of imagery that depict the physical environment in that area and assist the session with determining the user’s precise geographic location.
This localization imagery captures the view mostly from public streets and routes accessible by car. As a result, geo tracking doesn’t support areas within the city that are gated or accessible only to pedestrians, as ARKit lacks localization imagery there.
Because localization imagery depicts specific regions on the map, geo tracking only supports areas only from which Apple has collected localization imagery in advance.
Given this, it’s likely that Look Around imagery will form the backbone of any future Apple AR navigation product. ↩︎
* * *
UPDATE #1 | February 11, 2021
On February 11, 2021, Apple corrected the “Burgers 355” POI issue mentioned earlier:
Apple also added POI labels to its Look Around imagery of central San Diego:
However, other parts of the San Diego Look Around area, like Oceanside, are still missing POI labels:
* * *
UPDATE #2 | February 12, 2021
A U.K.-based reader points us toward a support document on Apple’s U.K. website, which states that Apple’s U.S. and U.K. driving during 2015 was for testing purposes only:
To ensure that Apple was applying the best image blurring technology possible, Apple conducted an entire year of test drives through the US and Europe in 2015, where it did not publish any of the imagery collected during these drives but used it instead to improve Apple’s technology such as image blurring techniques on such objects as faces and license plates.
This tells us that Apple did not start collecting imagery intended for production until the start of 2016 (i.e., five years ago).
This also seems to confirm MacRumors’s reporting (see Footnote #8) that Apple was, in fact, driving the U.S. earlier than stated on its website. In other words, Apple has been driving the U.S. for at least six years now.