Top Ad unit 728 × 90

Google Tango review: Promising Google tech debuts on crappy Lenovo hardware

The 3D sensing abilities are exciting, but not fully baked yet.

Ron Amadeo
http://dwgyu36up6iuz.cloudfront.net/heru80fdn/image/upload/c_fill,d_placeholder_arstechnica.png,fl_progressive,g_face,h_450,q_80,w_800/v1482374949/arstechnica_what-can-a-3d-sensing-smartphone-do.jpg
A look at some Tango apps. Video produced by Jennifer Hahn.

Remember Project Tango? Google's 3D-sensing smartphone project started development three years ago and first became public in 2014. The goal was to equip a smartphone with a depth sensor and other computer vision abilities. Why? To see what developers could come up with. It's a bit like strapping an Xbox Kinect to the back of a smartphone.
Now, Tango is finally out in a consumer device, but this is not the triumphant launch of a new flagship product. After years of buildup, Tango is quietly debuting on a mid-range Lenovo phone, the Phab 2 Pro. The bloated, ugly hardware Lenovo created for the Phab 2 feels more like a barely designed developer kit than a slick consumer product.
Even Google doesn't seem interested in the product. Contrary to most Google initiatives, Tango didn't get a Google-led launch event or press outreach. The most the company could muster was a blog post.
Ars still cares about Project Tango, though. So let's see what fun we can have with a 3D-sensing smartphone. While the hardware is a non-starter, the technology shows promise.

The hardware—Ewww


Let's not waste much time on the actual smartphone parts. Just know that the Phab 2 Pro is a disaster. First of all, it's gigantic—the Phab 2 Pro is the Canyonero of smartphones. The combination of a 6.4-inch screen and huge bezels means this "phone" is 7 inches tall. It weighs over half a pound—double the weight of an iPhone 7—and it's so thick Lenovo doesn't list an official depth on its website.
SPECS AT A GLANCE: Lenovo Phab 2 Pro
SCREEN 2560×1440 6.4" (459ppi) IPS LCD
OS Android 6.0
CPU Octa-core Qualcomm Snapdragon 652 (four 1.8 GHz Cortex A72 cores and four 1.4 GHz Cortex A53 cores)
RAM 4GB
GPU Adreno 510
STORAGE 64GB
NETWORKING 802.11b/g/n/ac, Bluetooth 4.0, GPS, NFC
PORTS MicroUSB, 3.5mm headphone jack
CAMERA 16MP rear camera, 8MP front camera
SIZE 179.8 x 88.6 x 11 mm (7.08 x 3.49 x 0.43 in)
WEIGHT 259 g (9.12 oz.)
BATTERY 4050 mAh
STARTING PRICE $499
OTHER PERKS Fingerprint sensor, notification LED, Tango sensors
The as-basic-as-possible rectangular design with minimally rounded corners contributes to the "ridiculously big" vibe the phone gives off. Compared to other really big phones like the Motorola Nexus 6 and the Xiaomi Mi Mix, the Phab 2 Pro feels like Lenovo didn't even try to make it look nice or compact. The Mi Mix is a great comparison since both phones have a 6.4-inch screen. Xiaomi tried to slim down and made a 158.8mm (6.25 inches) tall device, while Lenovo's design is a whopping 21mm (0.83 inches) taller.
Much of the Phab's girth is presumably to make room for the hefty 4050 mAh battery. Even that heifer can only run a Tango app for about an hour before draining to zero. Don't let the size of the battery fool you: the Phab 2 Pro is grossly underpowered. For some reason, the heavy 3D processing of Tango has been paired with a mid-range Snapdragon 652 processor. One of the coolest Tango apps, a 3D scanner, quickly bogs down to a few FPS thanks to the anemic processor. Tango, especially the scanner app, is a product that easily pushes the limits of top-of-the-line processors. The Snapdragon 652 doesn't stand a chance.
The Phab 2 Pro also feels old for an end-of-2016 device. I don't just mean the weight and massive bezels—the phone still has a Micro USB port on the bottom. The entire smartphone industry moved to USB Type C this year. Even low-end, sub-$200 devices are using the reversible plug. We're sure a decent amount of money went into the extra Tango sensors, but for $500 we expect an overall better package. The P2P's body is aluminum, at least.
Home screen looks fine. Recent apps looks fine. The Notification panel looks... oh wow. Dark AND light text on a transparent background? I can't even.
Enlarge / Home screen looks fine. Recent apps looks fine. The Notification panel looks... oh wow. Dark AND light text on a transparent background? I can't even.
Lenovo mostly stayed away from the software. Sure, there is a custom (and easily replaceable) home screen and custom app icons, but recent apps and the settings are intact. The one thing Lenovo touched was the notification panel (and by extension, the lock screen notifications). And wow, did Lenovo ruin it.
Notifications that pull from Lenovo's custom UI don't have the usual solid white background. Instead, they're directly stuck onto a frosty transparent panel. This makes app notifications basically go crazy, with some picking white text, some picking black text, and some picking gray text. This chromatic free-for-all leads to all sorts of readability issues.
Things are further complicated by how apps don't use the built-in notification style. Instead, they roll their own. That AccuWeather notification (which came with the phone) would look right at home on a device that follows Material Design, but it's a hot mess next to the rest of the notifications. I don't think this transparent design is against the Android compatibility docs, but it needs to be.

How Tango works

The back of the Phab 2 Pro. Three cameras, two light emitters, a microphone, and a fingerprint sensor.
The back of the Phab 2 Pro. Three cameras, two light emitters, a microphone, and a fingerprint sensor.
Ron Amadeo
On the back of the device you'll find a tower of sensors. Some of them are just the usual smartphone parts, and some are the special sauce that makes Tango work. Let's start at the bottom: just above the fingerprint reader is a huge motion sensor camera that gives off a HAL 9000 vibe, and above that is a rear pinhole microphone. At the bottom of the top sensor cluster is a standard dual-tone LED flash, followed by the depth detection setup: an infrared emitter and a time-of-flight depth camera. At the very top is the regular 16MP RGB camera.
With the extra sensors, Tango can do three things for developers: motion tracking, area learning, and depth perception.
  • Motion tracking allows the device to understand its motion as it moves through an area. This happens in full "3D" with six degrees of freedom. Not only can you tilt and spin, but you can walk around or kneel down and you'll still be tracked. Most developers use this mode to make the phone a "portal to another world" that works like a VR headset without any of the stereoscopic 3D or immersion. Motion tracking compares the images from the big fish-eye camera against movement data from the "inertial measurement unit" (a fancy combo of gyroscopes and accelerometers). The camera is used for "feature tracking"—visually identifying features like edges and corners as they move from frame to frame. Google's Tango stack can do this 100 times a second, if the hardware can keep up.
  • Depth perception allows Tango to see in 3D. Tango OEMs have a few options for doing this, but the Phab 2 Pro uses a Time of Flight (ToF) camera and an IR projector. The process works like LIDAR—tiny points of light are sent out of the IR projector and reflected into the ToF camera, which calculates the distance via the travel time. Depth perception apps can make a 3D point cloud of an area, which is neat in and of itself but also good for augmented reality apps that want to place something in 3D space.
  • Area learning combines motion tracking and depth perception to create a map of the area and identify your location within that map. Once a baseline scan has been completed, area learning works as a super-precise indoor GPS. You can stick a virtual object on a shelf, walk away from it, and it will stay there. Or—if someone ever creates an indoor map of something—area learning could work as an indoor navigation system. Tango saves these area scans in an Area Description File (ADF), which can be shared across apps.
That's basically it. Google provides this extra baseline of sensors and leaves figuring out what to do with them to the apps. Now that your phone can "see in 3D," what can you do with it?

Matterport Scene, a 3D scanner app

Matterport Scene is easily the coolest Tango app, and it gives us an idea of how Tango sees the world. It makes a 3D point cloud of whatever you point it at and saves that image in a viewable, explorable 3D scene. An app like Matterport Scene is impossible on any other smartphone.
All the Tango sensors go full blast with Matterport Scene. The time-of-flight camera and IR projector constantly take 3D measurements of the world in front of them. They blast out points of light and measure how long they take to bounce back. This system doesn't see color, only depth, so the RGB camera is active, too. It applies some color to the 3D point cloud. You'll want to scan an object from multiple angles, so your movements are tracked by the fish-eye camera, which allows every new depth pass to be correctly applied to the growing point cloud.
Once you're done scanning, Matterport Scene saves your 3D scans to an internal gallery that's easy to navigate. A one-finger drag rotates the view, pinch-zoom zooms, and a two-finger drag pans. The app also has nifty presets like "3/4 overhead," "First person," and "Blueprint." You can measure your scans with a built-in ruler, there's a 3D crop tool, and the "share" button can send the full 3D data to another Matterport user. You can also snap a picture to send to everyone else.
Listing image by Ron Amadeo
Matterport Scene exposes many of the limitations of the Phab 2 Pro. First of all, an app like this is extremely taxing on the processors. As you build your 3D point cloud, the frame rate drops like a rock. You'll start at a smooth frame rate, but after about 10 seconds of scanning, the Phab 2 Pro drops to single-digit FPS. The data capture rate slows down accordingly, so even though you can capture a ton of data at the start, you capture less and less as time goes on. After 30 seconds, you'll be at a frame every two seconds or so, which makes continuing to scan a challenge. If you move too quickly, Matterport Scene will pause and tell you to go back to where you were. This gets more and more sensitive the slower the phone gets.
In the room scan, above, I didn't stop because I got bored. But after about 5 minutes, the Phab 2 Pro ran out of memory and the app stopped recording—even with 4 gigs of memory available! There are some power saving resolution tricks here, too. Google's developer tools seem to have a higher resolution than Matterport Scene records, but clearly Matterport is trying to not immediately kill the Phab 2 Pro SoC. When exploring a model, panning-and-zooming happens on a lower-resolution model than what is available, with the full res version only kicking in for a stationary view. Apps like this for Tango feel like they need all the power in the world.
Motion tracking isn't exactly accurate, which is especially visible on small objects. Take our Yoshi doll example, above. When you first start the scanner and don't move, you get a fairly crisp image. Introduce movement by taking a quick 360 spin around Yoshi, and suddenly he has three eyes. Since Tango's motion tracking isn't perfectly accurate, there is drift, and the more you scan, the less accurate it is.
The 3D scanner app also exposes another limitation—Tango can't see black objects. Remember how the depth sensor works? Infrared light is fired out and reflected back into a sensor on the phone. Black or dark objects absorb too much of the infrared light, though, so Tango can't see them. In the scanner they are just invisible gaps of nothingness. You can see this in the Yoshi scan. He's sitting on my black couch, and in a single shot you can capture him, the floor, and the wall behind him, but not the couch.
Mirrors are fun, too. Tango will happily map out the mirror universe, which might be 2D to you but is a portal to another dimension for Tango.

Measure, an augmented reality measuring tape

Measure is an augmented reality tape measure.
Measure is an augmented reality tape measure. 
 
A thin green line shows what the edge detection algorithms are seeing, while a white plane detects surface.
12 inches in the app, just over 12 1/4 in real life. That's not super accurate, but it's alright. 
 
Measuring 4 feet on a measuring tape gives us... 3 feet 11 inches.  
 
You can measure multiple points, which is pretty nice. Also 6 foot 10 inches for that door frame is dead on. 
 
You can even have it generate blueprints. I measured the corners of the room, but due to inaccuracies, it didn't come out square. 
I'm a big proponent of the "smartphone-as-a-Swiss-Army-Knife" use case. My smartphone is a phone, a Web browser, a texting machine, a camera, a map, a compass, a notepad, a wallet, an mp3 player, a game console, a TV, and a million other things. How about if we add one more function to the pile: a measuring tape. Measure by Google is an augmented reality measuring tape app that's also impossible on any other smartphone.
The way it works is simple. Using Tango's depth sensor and motion tracking, you can target two points in the camera feed and the phone will tell you how far apart those two points are. In between logging "Point A" and "Point B," you can go for a walk and measure an extremely long distance. You don't even have to stop at two points—you can keep tagging points in the real world and Measure will keep measuring. It can even generate blueprints for a room.
If you want a benchmark for Tango's accuracy, you can't ask for anything better than a ruler app. The catch here is that the accuracy of the Phab 2 Pro's 3D sensing setup leaves something to be desired. You can expect short measurements to have a margin of error of about an inch, and longer measurements—which would be one of the big advantages of a "virtual" ruler—will fluctuate several inches (I tried 28 feet. It was off by three inches). Every Tango app has some amount of "drift" to it, and that affects the measurement here. If you mark a beginning point, mark an end point, and move back to the beginning, you'll find the beginning point has moved.
Measure doesn't have the ease of use of a real-world measuring tape either—there is plenty of room for user error. The targeting system works by detecting edges, which it marks with a green line. It only detects one edge at a time though, and getting it to stick to the edge you want can be a challenge. Getting your edges marked correctly is key to accuracy, which means you can't just hand it to someone and expect them to get accurate measurements.
Also, keep in mind that Tango's depth sensor works by blasting a tiny IR LED at something and timing how long it takes to bounce back, so the range is not great. While it would be awesome for Measure to stand in for things where a measuring tape is not feasible like, say, measuring a telephone pole, the top is just too far away. You have to be able to put the phone within a few feet of your start and end point.
Still, if you didn't have a tape measurer but you really needed one, the Phab 2 Pro can serve in a pinch. Just make sure you follow the old adage of "measure twice, cut once." Measure isn't good enough for precision work, but it's probably better than guessing. If future Tango hardware is a bit more accurate, this could be a useful addition to a smartphone toolkit.

Games and everything else

 An augmented reality domino game. It's a lot easier than setting up all these dominoes in real life.

The dominoes are on the floor, and the table is above the floor, so the table will block the dominoes, right? Wrong. For most games, real life objects can't be in front of virtual objects. 
 
Woorld is the only game I could find that attempts "Area learning." You scan the room before the game starts. 
After the scan it has a basic model of the room, and then you can have virtual objects under real objects. Here the little house is under my table. (It makes a shadow.) 
 
Hang a cloud on the wall? Sure Woorld. Whatever. Again, though, this was the only app to interact with the floor AND the wall.
 
Amazon's TV shopping app. Virtually size up a TV on any wall in the house.
It's actually fun walking around a virtual object. 
 
Lowes' shopping app. Should I keep my stove or get a new stove? Hmmm. 
 
Oh... a box. What's in the box? 
 
Oh boy! It's an undefined mess of blue pixels! Just what I always wanted! 
 
This game is basically SimTower. You give it a surface for the virtual area and then the augmented reality stuff stops.
 
 
You control the camera. Let's zoom in... 
 
Hi! We can get right in the face of our little people. 
 
A Hot Wheels game. This isn't "augmented reality" at all, Tango just acts as the camera. 
 
We can zoom right into our little car.
 
A god game where you control the camera. Every Tango game is a god game where you control the camera. 

Most of the "augmented reality" Tango apps I tried were simple. Basically, they all defined a single plane—a floor, table top, or wall—and then the "augmented reality" processing stopped. For floor and tabletop games, other objects in the real world weren't respected. If you start your domino line on top of a table and go off the table, nothing happens; the dominoes just float in midair. Virtual pets, balls, and other virtual objects happy plowed through my real-life walls and furniture like they weren't there. For most of Tango's apps, augmented reality processing seems to be limited to treating one surface as a flat, infinite, unobstructed plane.
The phone viewport wasn't smart about things either. Real-life objects didn't block virtual objects, even when they should have. If we go back to the dominoes game, I could still see dominoes on the floor through the table top. Most virtual objects floated above the camera feed and never actually interacted with the real world. My experience wasn't much better than Pokemon Go, which just plasters Pikachu on top of a camera feed and calls it "augmented reality."
The one exception to "crappy augmented reality" mode was Woorld, a decorating "game" (?) from Katamari Damacy creator Keita Takahashi. Woorld actually used Area Learning to map out the basic geometry of the room before starting. With an idea of your surroundings, Woorld let me do things like put a tiny VR house under my table, and then the 3D model was actually blocked by the table. A little box character climbed up my walls instead of going through them, and I could hang stuff from the walls and ceilings.
None of it was perfect, but the real world occlusion of virtual objects was critical to even temporarily tricking me into believing these virtual objects had a place in the real world. I still got a significant amount of drift in Woorld, and, while things spatially made sense when I started, everything was out of alignment after a few minutes.
Let's get away from games for a moment: a crop of "shopping" apps from the likes of Lowes and Amazon have a database of objects with real-life measurements, and you can virtually place them in your house. Lowes has a selection of appliances, while Amazon oddly only has televisions. Here, the "dumb" augmented reality of a single, infinite plane worked great. Assuming the data is correct and Tango is properly scaling everything, these apps were fine.
Most of the Tango apps are games, and every game (save for Woorld) I tried was basically the same thing. You have a birds-eye view of the action, the phone is the camera, and you point and tap at everything with a center-locked reticule. Some games are "augmented reality" and happen over the top of a camera feed, while some are completely virtual, working sort of like virtual reality (you ARE the camera!) without the 3D goggles. Since the augmented reality interactions are so limited, the differences between "augmented" and "virtual" games are negligible. Gameplay always happens in a virtual space that doesn't interact with the real world, so what does it matter if that virtual space is on top of a camera feed or not?
The camera controls are excellent with Tango, though. If you duck down, the game will zoom in, with basically no limit. Usually there is some kind of little character on screen, and you can get right in its face if you want. You can take the camera anywhere. Good camera controls don't make for a good game, though, and they never felt like they added to the experience. Oftentimes, kneeling, standing, or bending to get the right camera angle to do what you need to do is uncomfortable. You're also trying to do this while not dropping the phone and not blocking the sensors.
Google has already started experimenting with loading Tango into a VR headset. Tango is basically an early version of the "inside-out tracking" that the virtual reality heavyweights are working toward right now. Basically, instead of the pair of IR base stations the HTC Vive and Oculus Rift currently require (those are "outside-in" tracking), inside-out tracking puts all the 3D sensing technology on the headset—basically strapping a Tango phone into a Daydream headset or Gear VR. Of course, VR tracking is all about ultra-low latency and super precise tracking, which Tango isn't set up for. VR is another possible use case that starts with "maybe for the next version..."

Tango app fragmentation (yes, already)

The Tango Explorer developer app has some compatibility issues on the Phab 2 Pro. Thumbs up for the RGB camera orientation; thumbs down for the point cloud orientation. 

 
Some apps crash immediately, as opposed to all the other Tango apps, which can usually go for a few minutes before they crash.
Tango apps that don't run on my Tango phone. 
 
As a new platform, Tango suffers from the usual problem of there not being enough "Tango compatible" apps. But the most frustrating thing about the Phab 2 Pro is that it isn't even compatible with all the Tango apps that are out there. For instance, Google's developer tools don't work on the Phab 2 Pro. Google's "Tango Explorer Tool" is frequently referenced in the developer documentation, and it allows you to see how Area Learning works, view a raw 3D point cloud, or run diagnostics. Besides crashing constantly on the Phab 2 Pro, the point cloud is upside down! Google's "Tango Area Manager," another key developer tool, instantly crashes.
I don't think I'm complaining about day-one problems. The Phab 2 Pro launched November 1. Android devices have a tradition of being easily turned into developer devices, but Google doesn't seem interested in letting people use the Phab 2 Pro as a development device.
Many of Google's example apps don't work either. "Wizard Demo Experiment" looks like a fun little area to explore, but the X and Y motion tracking axes are swapped, making it impossible to navigate. The orientation is wrong, too—everything is sideways. Google's "Cube Mover Experiment" has the same problems. Jonny Lee, the technical program lead for Tango, has a Play Store page with eight Tango apps, none of which work on the Phab 2 Pro.
This raises questions about how future Tango devices work. Will every Tango app need to be updated for each new device, or are these broken apps just made using an old version of the Tango platform?

Performance—Tango is thirsty for more power

Maybe you can excuse the Phab 2 Pro's large size and ugly looks with how it's the first-ever commercial 3D sensing smartphone. What is hard to reconcile is the fact that if you want this cool new thing, you're stuck buying it attached to mid-range hardware. This is especially disappointing considering that the current Tango apps are begging for more power. They already feel like they could easily tax next year's hardware. Lots of heavy 3D processing is going on here, and the Phab 2 Pro just isn't fast enough to run some of these apps, particularly the 3D scanner app.
The Phab 2 Pro is stuck with a Snapdragon 652. The CPU performance isn't terrible compared to a high-end smartphone, but the GPU performance is pretty bad. In GFXBench, we'd have to go back to high-end phone 2014 devices to see scores like this.
The original Tango prototype used a special co-processor from Movidius to help with the vision processing, but the Phab 2 Pro has no extra co-processor. Qualcomm and Google apparently worked together to get Tango's processing stack working on the Snapdragon line without taxing the CPU too much. Basically, Qualcomm rounded up some of the smaller, "lazier" processors inside a phone—the DSP, ISP, and sensor hub—and recruited them for general compute work. When a Tango app turns on, it's all hands on deck. By offloading work to the smaller processors, Qualcomm claims you'll be able to run all of Tango's algorithms and sensors with a "less than 10%" CPU overhead.
So Qualcomm and Google got Tango running on the Snapdragon 652 with such a low hit to the CPU. Good for them. But Tango still runs like a dog. It is all-around slow and chuggy, and some apps, like the Matterport Scene 3D scanner, quickly leave the "frames per second" measurement scale and move to a "seconds per frame" number. Then the app shuts off because it ran out of memory.

 


Tango is a great idea. Here's hoping for a 2nd generation

The Kyocera VP-210, released in 1999, was one of the first camera cell phones.
Enlarge / The Kyocera VP-210, released in 1999, was one of the first camera cell phones.
On the right you'll find the Kyocera VP-210, one of the first-ever camera cell phones. It shares many of the problems you could also attribute to the Phab 2 Pro. It was hideous, bulky, and probably didn't work very well.
The 0.11 MP sensor probably took horrible photos. If you want to count "two frames per second" as "video," then the video it took was no doubt awful, too. Anyone reviewing the VP-210 back in 1999 would be required to point that awfulness out, just as I am pointing out problems with the Phab 2 Pro now.
None of VP-210 problems meant putting a camera on a phone was a bad idea though. As tiny cameras continued to be developed, they got better and better. Today, every phone has a camera on it, and sometimes they take beautiful pictures. For most people, a smartphone is now their primary camera.
Tango is in the same situation. It's a new set of abilities being added to a phone. Of course the first phone with 3D sensing is bad—the first version of everything is bad. Tango is still a promising technology that I would like to see developed. Generation two will probably be more accurate, probably have less drift, and probably fit into a smaller package, all of which will be a big improvement.
Tango, in theory, feels like an exciting, fun, useful addition to a smartphone. A pocket measuring tape is a good idea. A 3D scanner would be great for building virtual environments or hooking up to a 3D printer. The hardware is an enabling feature for indoor mapping, so if the Google Maps team could get to work on that, that would easily be a killer app for malls, airports, and other complicated indoor places. Tango has an endless amount of gaming applications, and the "virtual shopping" apps for furniture seem useful, too. With a little more accuracy, this technology could also be used for an "inside-out" VR headset, which would make the next generation of Google's Daydream headsets better. These are all solid use cases. I hope the Tango team keeps working on it.
For now though, I don't think anyone should buy the Phab 2 Pro. It's ugly, slow, and huge. Its battery life is awful, especially if you're using a Tango app. The Tango technology is imprecise, and Tango apps aren't there yet. Developers can't even buy the Phab 2 Pro because the developer tools don't work on it.
In the future, 3D sensing seems like something that will eventually be built into most high-end smartphones. Niche, single-purpose hardware regularly gets slapped onto high-end devices. An inaccurate heart rate sensor still gets built into every Samsung flagship. LG and Apple both added an entire second camera to the back of their smartphones just for a fancy zoom feature, while Samsung added a second camera and an IR projector to the front of the Note 7 just for a duplicate biometric login method. Tango seems much more useful and multipurpose than these other hardware add-ons. It just needs a little more time and love to be perfected.

The Good

  • Tango is genuinely interesting, and it opens up exciting new use cases for a smartphone.

The Bad

  • The Phab 2 Pro looks like a developer kit. It's huge, ugly, and heavy.
  • The mid-range Snapdragon 652 SoC isn't fast enough for some of the Tango apps.
  • Tango is not super accurate right now. There's some drift, and the measuring tape app is often off by several inches.
  • Tango has the usual "new ecosystem" app problems—you can't do much to do with it right now.
  • Tango apps will suck your battery down in about an hour.
  • Lenovo ruined the notification panel.

The Ugly

  • The phone. Just the whole thing. Damn.

 

 

 

Google Tango review: Promising Google tech debuts on crappy Lenovo hardware Reviewed by Chidinma C Amadi on 3:14 PM Rating: 5

No comments:

Kogonuso © All Rights Reserved!

Contact Form

Name

Email *

Message *

Powered by Blogger.