Large language models (LLMs) and new voice models have paved the way for a horde of new kinds of wearables ranging from wristbands, pins, credit-card-styled devices that stick on the back of your phone, rings, earrings, earbuds, and smart glasses.
Google is preparing to launch a range of new smart glasses. Apple is rumored to launch its own glasses in the coming years, and OpenAI is working on a smart speaker.
Qualcomm, the chip company, is powering a lot of those new AI devices. Recently, the company’s CEO, Cristiano Amon, said that it is working on over 40 new AI devices with different form factors. In June, it also released platforms for mixed reality and a turnkey toolkit for AI devices, allowing it to white-label its tech.
I sat down with Ziad Asghar, Qualcomm’s SVP for XR, Wearables and Personal AI, to discuss the future of wearables and whether voice is the right interface for it.
What kind of AI devices are you seeing getting traction? Why do you think those form factors are winning over others?
Today, we are seeing amazing pickup on these devices. For example, perhaps the first form factor coming up with AI native devices is smart glasses. With smart glasses, we're seeing amazing traction as they are growing almost 3x every year.
A couple of points to keep in mind. Number one: glasses are an established form factor, as many people wear them daily. I think glasses provide the best input for AI. If you have a camera over here [pointed to the place near your temple], it sees what you see and hears what you hear. I say probably the best form factor for agent AI is a device like glasses.
Second great thing about glasses is that they can actually augment your field of view. For example, when I was in China with one of the customers looking at a demo of glasses based on our Snapdragon AR1, they were speaking in Mandarin, and I could see the full translation coming up on my display, separated by speakers. It shows what is possible with these devices.
We have the ability we call the audio bubble. With that, the device only hears sound within a certain radius of the person, even if there is a lot of noise around the user.
How do you think glasses and other devices will work with an AI assistant?
You want a device that’s able to understand the context of your query. Today, you use ChatGPT or some other agent that’s in the cloud, and it doesn’t know why you are asking a question. Devices like glasses can give you context. From my glasses, I can show the assistant a sign or a plaque that I am looking at and augment my prompt with context.
I think our personal AI devices can augment your AI experience. For example, your watch monitors your heart rate and activity level. Glasses can tell you about the weather conditions; your phone knows your location, calendar, and searches. If you can bring and aggregate all this context, you will have the best agentic experiences.
The new AI devices often don’t have screens. How do you think about interacting with them?
We have done research on human-machine interfaces for the last one and a half decades, but from a mixed reality angle. This includes hand tracking and understanding gestures well. We have a good understanding of head positioning and tracking. We have taken these algorithms and hard-coded them in transistors and silicon, which means we can do these use cases at low power consumption. We really have all the pieces to be able to do these human-machine interfaces.
We have worked with a ring maker called Kiwear, which works on both voice and gestures. At times, in public, you don’t want to use your voice for commands.
Voice as an interaction medium is very popular right now. What are some of its limitations?
The first limitation is being able to capture voices in public spaces. Another challenge is filtering out the noise even when you’re not in public.
Voice capture is difficult in noisy environments. How do you handle that?
We are developing some additional unique technologies for this. We have the ability we call the audio bubble. With that, the device only hears sound within a certain radius of the person, even if there is a lot of noise around the user. We have some amazing echo and noise cancellation technologies. So you will not be able to hear anything else except what the person is saying in that small radius.
We are also working on silent speech technologies where nobody can hear what you are saying but the device.
We will need a new interaction mechanism for different environments to work with newer devices. For example, in industrial settings, voice doesn’t work at all.
For smart glasses, there are multiple approaches. There are glasses with cameras that would give AI assistants more context, and there are glasses with LED screens, which can aid productivity. What do you think about that?
What we are seeing is that people are saying the agent that I might need for productivity work is different from the agent I would need for my personal life.
I was on a vacation trip, and I started to take a lot of pictures with my smart glasses. I got used to actually doing that and also started invoking my agent.
[On the enterprise side] We showed this demo for oil and gas companies where you can bring in any equipment, and the glasses can recognize it and give you more information. However, that’s cloud-based AI. We have also shown off demos where small models are running directly on glasses.
I hear once in a while that glasses are gonna replace the smartphone. I really don't think about it that way. You might still have some AI in the smartwatch, in the earbud, in the glasses, and in the phone. And I call this the constellation of devices.
One thing about AI agents and AI applications is they're always context hungry. These devices ask of you that you will provide more context to the agents. But it comes with a privacy trade-off. We've especially seen kind of a restraint towards smart glasses because people don't want to be recorded all the time. With more devices that have the potential to record slyly, some people will cross boundaries of privacy.
I think that companies that have launched glasses with us have been fairly responsible in the sense that an LED light comes on. When you are able to when you take a picture, the LED light blinks, for the most part. When they are making a video, for example.
The second thing is, I see people walking around with the camera like this [made a gesture holding a phone up] in public spaces all the time here in the US now. And there isn't even a light blinking. So my point is, some of that is actually going to get resolved in time.
Qualcomm is very focused on making sure that the consumer is enabled to make that choice [about privacy]. So what we are doing is, for example, in our solution, we call it an always-sensing camera. When my glasses take the picture as the context for AI, it can actually obfuscate all the faces. Because, really, for context, I don't need the faces. What I need is basically all the information that's beyond the faces.
Number three, as we do more of the AI processing on the device, we're actually able to take away the concern of processing data in the cloud.
I asked you earlier about the friction around context capture and privacy. Do you feel that factor is somewhat of a risk to these devices growing? Apart from the protections that you talked about, what is some of the work that you are seeing?
I actually think that the protections will help immensely. But I also do think people are used to other cameras in the public sphere already, like security cameras. And they're capturing people all the time. The same thing goes for phones as well. I feel that, yes, there is a concern, and it's an important thing where we need to do it in a responsible manner. But I think it has not been as big as what I might have expected, which means people are actually getting comfortable with the technology as we provide more of these mechanisms like face obfuscation and other ideas.
Beyond smart glasses, what are some of the form factors that are picking up?
We are working on more than 40 devices of these types beyond glasses. One form factor that I see repeatedly is a pin or a pendant. The reason is that there are people who don't wear glasses. But they still want to have that same sort of multimodality experience with an AI native device.
I also like the idea of headphones with cameras, because a lot of young people wear headphones even in public to listen to music or podcasts. With headphones, there is also space for bigger batteries.
There's a third form factor that I think is very interesting: helmets. In countries like India, there are a lot of people who ride two-wheelers with a helmet. It also has ample room to put a camera or batteries.