Tinkering with Google Glass to expand wearable tech

From golfing to facial recognition, inventors experiment with new technology

The developers at Palo Alto-based Mercedes-Benz Research and Development are constantly watching technology trends that develop around them. And they've seen a lot of trends that have faded into the abyss.

But when Senior Engineering Director Kal Mos and his team walked back from Google I/O — the company's annual developer conference — last year, they knew that one trend they saw was not going away: Google Glass.

"We saw (that) people started hacking those devices and developing things for them," Mos said, referring to the tech giant's latest sensation: a wearable computer that looks like glasses but is equipped with smartphone-like capabilities (such as taking a photo or browsing the Internet) that can be used hands-free.

Palo Alto developers like those at Mercedes-Benz Research and Development have begun tinkering, hacking and inventing with the device to expand the world of wearable technologies. The Bay Area Glass-buzz has convinced Mos that even if Glass is not the next big thing, wearable technology as a whole will indeed take off.

After Mos and his team decided that the hype around Google Glass was not just a fad, they immediately began brainstorming how they could incorporate this native technology into their cars.

"There is one black spot for Google and that's certainly in the vehicle experience," said Thomas Winkler, an engineer on Mos' team. "Every other part of your life, they have pretty much integrated into. The vehicle is one void."

The group brainstormed multiple ideas involving Google Glass and cars and decided to focus on one function that is fundamental to the driver: navigation.

The team calls their Google Glass prototype "door-to-door navigation." A user can speak a desired destination to Google Glass, get in his or her car, plug a phone in to the car and — voila! — the destination is transferred to the car's GPS. After the user parks the car and leaves, the destination is transferred back from the car to the user's Google Glass for the final stretch on foot.

"The wearable computing devices ... are extremely useful. The car itself also has a lot of capabilities," he said. "We looked into how we can have these devices talk to each other, work together to find the most comfortable user experience and non-distracting user experience."

The "non-distracting experience" is very important to Mos and his team. He said he saw that there were many concerns regarding Google Glass and potentially distracted driving, so the consistent goal for his team is to transfer any potential user distractions to the car.

"We want people to focus on the driving experience," he said. "We see this as a way for people to have it both ways. Have the Glass experience. Have the car experience."

Worldwide, people are working on Google Glass applications geared toward journalists, surgeons, the disabled and more. Palo Alto resident Stephen Balaban is one of them. He founded Lamda Labs, a wearable computing company that is developing custom software for devices like Google Glass.

Before Google released Glass, the 24-year-old tech enthusiast had been pondering facial recognition for a while. Imagine a life, he said, where you could look at someone, search through your social network on the spot and find out who that person was.

"When I was really initially thinking about this problem, I thought, 'Man, I'm going to have to make custom hardware. Nothing like this even exists,'" Balaban said. "And then when Google announced (Glass), I felt, 'Wow, maybe I don't have to do all the custom hardware and all the hard work.'"

Unfortunately, Balaban had to root (translation: re-engineer to allow unrestricted access) his Glass, since Google has decided not to support facial recognition in response to privacy concerns. Balaban, though, believes facial recognition with Google Glass has tremendous potential in business and service industries.

"(Google) really to some degree can't control what runs on their device," he said. "They can control what's on their app store certainly, but when you have root access to the device, all bets are off."

After Balaban released a Facial Recognition API (application program interface) in June, Google changed its terms of service to state that it no longer would approve facial recognition applications. Balaban said that by doing so, Google is missing out on significant opportunities.

"My reaction to that was, well, certainly it was time to root the device," Balaban said, adding that he became much more jaded towards Google Glass after that point.

Still, Balaban created a Google Glass app called MindCap, which takes a picture every few seconds, even if the screen is turned off — a capability that would not be possible with a non-rooted device.

Balaban recently looked through all the pictures that his Google Glass took of his day, which turned out to be mostly programming at his home office. He calls it "life logging" and has considered creating a daily report out of it, with a review of the conversations he had that day and even quantitative summaries, such as the amount of time spent outside or in front of a computer. If he could run facial recognition over the live stream of his pictures, he could include a list of people he met.

"Those are the things Lambda can do: machine learning and wearable computing software," he said, "and if you can bring that all together, I think that it could be really cool."

When Google restricts this development stage, Balaban said, it cuts out a core segment of the population.

"The people who really like (Glass) are the hackers and tinkerers who like taking things apart and putting them back together and playing around with them," he said. "I think Google is making a big tactical mistake by not having that as their core demographic."

Catalin Voss, an 18-year-old Stanford University student from Germany, is a major tinkerer. He's been playing with iPhone applications since he was 12 years old, downloading Apple code and experimenting with it to create new applications. He's also the co-founder of Sension, a Mountain View-based start-up that creates visual interface technology.

Sension was originally inspired by a drive to reform online education, but with Google Glass, it's going beyond that: treating people with autism.

Voss, who has a cousin with autism, has built face-tracking technology for Glass that allows the user to recognize another person's emotion. A person can put on Glass, look at someone he or she is talking to and, if they smile, "Happy" pops up on the Glass screen. It can also recognize sadness, anger and fearfulness. The facial recognition tool works geometrically, by analyzing the movement of various points on a person's face. There are 78 total possible points, such as the corners of the mouth or each eye.

"Right now the way autistic children learn emotions, if they have, say, Asperger's, is they sit in a room like this one," Voss said on a recent afternoon, motioning to the conference room he was sitting in. "And basically there's a doctor or a behavioral psychologist or somebody like that in the room with them and they do flashcards. They look at people smiling, they look at images, they look at cartoons, they look at smileys. Sometimes they bring relatives in. But the way I smile and the way you smile is very different, at least to my cousin it is. The problem is, they don't learn the sort of data about the people they interact with. It's not real-life data, and they can't learn it on the people they most care about, which is who they encounter in school, in college, at work."

But with Sension technology, autistic people could wear Glass for a month or two (or however long is necessary) so they become familiarized, in real-time, with real people.

"And then eventually are able to take it off and start recognizing emotions on the people that they learned it on," Voss said.

Using Glass in this way is part of Voss' long-term vision for emotion recognition technologies, which is that devices should respond to and interact with their users. But he said that long-term vision, however far off, has been difficult to reach due to certain constraints with Google and Glass.

Sension only has two units, and Voss said he has "struggled" to get their hands on more.

His company receives many requests from people who want to try out the new technology, including a mother from Berkeley with an autistic son. Sension can't accommodate these kind of requests with only two units, Voss said.

But, "We're playing by Google's rules," Voss said. "I don't know beyond that."

David Grieshaber, founder and CEO of TechGolf, said he missed the mobile transformation. He doesn't want to make that mistake again. That's why his next focus is wearable technology.

Since two years ago, Grieshaber — based in Brisbane, Calif. — has been working to integrate technology and golf in his company TechGolf, with high-tech driving ranges and, now, a Google Glass app called iCaddy.

"Google Glass ... is perfect for golf," he said. "Golf is nonviolent and slow moving, and it is simple to connect to a network on a golf course."

The app displays yardage, elevation, temperature, wind direction, suggested clubs and past scores while the golfer is playing.

"Our goal is to lower the score of every player," Grieshaber said. "Our goal is to make it as simple as possible so you don't have to think about all those numbers."

Grieshaber said that he is surprised at the rate at which Google Glass has gained steam in Silicon Valley and that he doesn't see this product tanking.

"The amount of money that is being poured into the technology now will make it have some sort of success no matter what," he said. "Once it becomes socially acceptable, just like the smartphone, with three or four or five iterations down the road, they could sell 100 million or 200 million."

With the Glass technology as nascent as it is, hackers agree no one's yet developed the "killer app."

William Hurley, who goes by Whurley and is the co-founder of the mobile software studio Chaotic Moon Studios, said there are many great ideas milling about, but not the big one.

"I've seen ideas in every vertical," he wrote in an email. "From medical to aerospace, even law enforcement. All of them show promise, but I'm not ready to award any of them the 'cool' title yet. Simply put, no one has nailed the killer app."

Despite that, he believes wearable technology is the next smartphone and will be everywhere in five years.

"Don't believe that? Just take your time machine five years back and see what the state of today's booming smartphone market looked like. There are literally dozens of parallels," he wrote.

Barg Upender — co-founder of the San Francisco-based Google Glass application developer dSky9 — is a little more skeptical about how mainstream the device can become.

Even though Google has built up a red-carpet feel to the product, he said: "It's still yet to be seen how well it would be adopted by the lay person."

Nonetheless, Upender — who has worked in the technology world for two decades — said that the hype around Glass will provide the engine for other smaller companies working with the macro-trend of wearable technologies.

Whurley agreed that Google Glass will push the macro-trend forward.

"It almost doesn't matter if Google Glass is successful or not, as it has already put enough sunlight on the ecosystem that literally dozens of similar/competing products have received funding (or additional funding) in the last year," he wrote.

Therefore, dSky9 has poured its focus into a Google Glass app called FaveStar.

"It's bookmarks for your life," he said. The user can tell Glass to "favorite this" and their Glass will take a quick picture, which the user can tag and store away in a memory bank. Then, he or she can share it with friends, find recommendations or search for the food, music, events, scenes and people that the user likes.

DSky9 also developed three demo apps: StarFinder to show constellations, UltraRun to show a user real-time training data and PathFinder to show hidden data about the landscape around a user.

Upender said any new technology product needs early adopters and developers to risk their effort.

"It's almost like a petri dish," he said. "If the tech guys are giving the thumbs up, eventually it will move to the rest of the world."

These early adopters, he said, are there to see the vision of what this technology can do. And as they continue, they all will strive to build the next killer app.

Editorial Assistant Elena Kadvany contributed to this report


Like this comment
Posted by
a resident of Barron Park
on Sep 28, 2013 at 9:54 am

I hope they launch their self driving car at the same time as these Google glasses, else drivers beware!

Like this comment
Posted by businessdecision
a resident of another community
on Sep 30, 2013 at 7:53 am

needs to be outlawed.

Like this comment
Posted by musical
a resident of Palo Verde
on Sep 30, 2013 at 8:38 am

Another differentiator between the haves and have nots.

Like this comment
Posted by $$$
a resident of Southgate
on Sep 30, 2013 at 9:00 am

This technology thing is getting stupid and petty, making us lazier and lazier as we burn fewer and fewer calories. I often feel like I am being forced to buy products I don't need so I don't look like a Luddite

Like this comment
Posted by $$$
a resident of Southgate
on Sep 30, 2013 at 9:02 am

How much of this Google glass thing is the result of Sergey Brin promoting his new girlfriend's invention, anyway?

Like this comment
Posted by Noel
a resident of Crescent Park
on Oct 1, 2013 at 11:41 am

Why such grumpy comments? This is way cool and will eventually be cheap enough for everyone. The Internet has been a great social leveler, distributing information and educational opportunities globally. This is the next step. Parts of it are very creepy, just like the Internet, but we will figure that out and deal with it.

Like this comment
Posted by KP
a resident of South of Midtown
on Oct 1, 2013 at 12:47 pm

I want this!
I saw a lady wearing them at the Paly graduation this past year. She was able to take pictures, of whoever she was there to see, without having to stand up in front of people.
I want this!
It's only a matter of time before we can ALL afford these "glasses" - EVERYTHING starts out expensive and comes down in price eventually - well...except for houses in PA! LOL!

Like this comment
Posted by businessdecision
a resident of another community
on Oct 2, 2013 at 1:08 pm

No, this is an awful thing that never should be allowed.

Sorry, but further commenting on this topic has been closed.

Burger chain Shake Shack to open in Palo Alto
By Elena Kadvany | 16 comments | 4,176 views

The Cost of Service
By Aldis Petriceks | 1 comment | 954 views

Couples: When Wrong Admit It; When Right; Shut Up
By Chandrama Anderson | 0 comments | 397 views

This time we're not lying. HONEST! No, really!
By Douglas Moran | 4 comments | 396 views

One-on-one time
By Cheryl Bac | 0 comments | 378 views