Signals
I was at the first Figma Config in India last week, and I saw the prominence of good voice technology on display. First, the live captions displayed on the screens next to the stage were much more accurate than they have been in years prior. There were speakers who didn’t have a Western accent, and yet the captions were good enough. I mentioned how the HumanX conference used voice tools well. Conferences might be a big unlock for voice tools.
Second, most of the demos by Figma staff showed them voice-prompting the Figma web app and getting good results. I am not sure if this was a demo thing or not, but certainly voice-based commands and executions are getting popular.
In Focus
There is a rise in ambient apps and devices, but the productivity angle is not justified enough
Roughly two years ago, I tried Bee’s first device, back when the company wasn’t a part of Amazon. The device was meant to ambiently listen to you and your conversations all the time, and churn out action items and insights. I used this device for a week, and there were often a lot of useless and inaccurate insights.
Last week, Amazon released two new Bee devices, Frontier and Eclipse. The first one is a pendant similar to the one above, and the second one is a Plaud competitor, which can stick on the back of your phone.
Frontier’s premise is that it can listen to the conversation ambiently and weed out useless stuff, such as noise around you. The first version included a lot of useless information, including mixing in details of a video or a show you might be watching. The new version supposedly is better in that regard.
Separately, a former Plaud exec’s company Ambi, which works on a similar premise of listening to users all the time, launched its apps across Apple devices.
Apps and devices in this category promise that if you let them listen to your conversations or ramblings all the time, you won’t forget important stuff, and the AI behind them will also give you insights that may be worthwhile to you.
There is a big trade-off in letting these devices capture all the information in your speech and the speech of people around you. This is a big privacy concern. The companies want to offset this by promising that data generated by their AI is so valuable that you wouldn’t mind keeping the mic on all the time.
If I double tap on the argument of listening ambiently all the time, their point is that you forget a lot of things in your day or a week. If you keep your devices or apps on all the time, you could become a better human being by remembering things and getting insights from AI.
That argument stands for certain parts of life, such as meetings or shopping reminders. To-do lists, reminder apps, and even notetaking rings like the Pebble ring are useful. For meetings, there are Plaud, Pocket, Amazon’s new Eclipse, and numerous other options.
At the moment, despite the investment of millions into startups working in this area, the upside is not yet visible. With AI, the argument often is that as models get better, users will see results. Then there is the big question of privacy.
Who wants to be “recorded” all the time?
Before we get into the social construct of conversations being recorded all the time, I want to touch on how companies are playing around with the term recording. Traditionally, any kind of audio storage of a conversation has counted as recording, and that is the angle from the new-age ambient startups.
There are laws in various regions about obtaining someone’s consent for recording audio. Some new startups get around that by saying, “We don’t keep recordings.” Even Apple made arguments about not keeping audio data and processing shredded transcripts in a secure cloud environment for its new Voice Intelligence features released for Apple Watch.
There are two approaches to these new features. One set keeps transcripts; another set even discards them and retains summaries or key points. Despite these measures, there is always a chance that some detail in the final summary is identifiable with the person who said it. That particular detail can end up in a lawsuit and brew trouble for the device or the app maker.
Alongside privacy, there is a social aspect of conversations. When we meet someone, we establish a social contract of trust and what exchanged information stay between us. If a device or an app comes into the picture, that trust might change unless all people are fine with it.
In the past few months, I have seen people being wary of recording devices or apps. A lot of them don’t want to be near a recording device and won’t get as comfortable having a conversation with someone with that device. There is already a distaste towards video recording glasses such as Ray-Ban Meta smart glasses. Ambient meeting notetakers can add to the anxiety of users that they are being recorded all the time.
In the coming years, we will see lawmakers scrutinize non-consensual information capture by devices and notetakers, and we will see changes in the way the technology works, or rather, indicates that it is gathering information.
More importantly, when these devices reach average users who might not be automating their whole lives, we would get a true gauge of how much AI people want to accept and how much will be booted out.
Numbers game
xAI reported that Grok handles over 20,000 Starlink support calls daily. Plus, using voice AI, its sales conversion has gone from 20% to 25%.
A survey from AI receptionist startup Upfirst indicated that only 2.4% of solo entrepreneurs have adopted AI despite the promise of making things easier for them.
Image Credit: Upfirst
Model Behaviour
Fish Audio released its speech generation model Drama 3, a potential competitor to ElevenLabs’ v4. Users can direct the new model’s expression with natural language, and edit a part of an audio recording in the same way.
xAI released Grok Voice 3.0, a speech-to-speech model, which is priced at $4.80 per hour.
OpenAI launched GPT-Live-1.1 with support for more languages and the ability to create custom voices through prompts.
AI company HeyGen released its own voice model tuned to be used in videos that retains context and personality throughout clips.
Indian voice AI company Gnani released Timbre v3.0 speech generation model with support for 10 Indic languages.
Quick Bytes
Baroness Jane Campbell used ElevenLabs to recreate her own voice and gave a speech in the House of Lords.
Sony Music and UMG are forcing indie label DistroKid to disclose how many AI tracks it uploads.
YouTube will stick a label on tracks that are fully generated or altered using AI tools.
Deezer will release a toggle by the end of the year to let users filter out AI music.
Deals Corner
Ambi (Mistlabs) ($26 million): Ex-Plaud exec Peter Mo’s startup is working on ambient apps that can listen to you and provide insights.
Investors: Luminous Ventures and Huaqin (co-leads)
Othello ($8 million): The New York startup builds AI for sales coaching.
Investors: a16z speedrun, Owl Capital, and Long Journey Ventures (co-leads); angels include Andy Fang, Chris Larsen, Thomas Wolf, David Cramer, Waseem AlShikh, Dean Sysman, Alex Bouaziz, Philippe Bouaziz, and Kevin Stein
Desible.ai ($3.7 million): The Bengaluru-based startup is making agentic voice AI for banking, financial services, and insurance.
Investors: Prime Venture Partners (lead); Invention Engine (participating)
FLÆ Robotics (€1 million+): The Prague startup’s AI hospitality agent completed over 3,500 bookings without involving any humans.
Investors: Purple Ventures and Garage Angels
GasLab ($1 million+): The Netherlands startup, founded by Dutch techno DJ Reinier Zonneveld, is working on getting musicians paid.
Investors: Angel investors
Thank you for tuning in. Keep listening.
This newsletter is by Ivan Mehta, a freelance reporter at TechCrunch. It covers AI and technology in voice, audio, and music. Email: voiceaiweek@gmail.com or im@ivanmehta.com