Signals
Last week, I wrote about ambient AI and how companies are trying to make use of conversational data. During the week, another company, Nirva, got added to the list. The company has a jewelry-like pendant and $33 million in funding from Lightspeed and South Park Commons. There is a growing trend amongst companies believing that people would be okay telling more about their lives to AI apps and gadgets for either productivity or personal insight gains. All of this comes with a privacy trade-off.
In Focus
The voice AI world is a strange place where there is euphoria around every new model release, which is touted as the best thing since sliced bread or language itself. Rarely does the wider world get amazed by a model. This weekend was one of those rare moments when Jack Dorsey’s Block released a new model called Dessa.
Before we talk about the model’s technical abilities, people are amazed by how expressive it sounds. I am saying expressive because I know it is a model, not a human. If someone gives me a blind test and makes me interact with Dessa, would I know it’s a model? Maybe.
Comparisons between models can be illusory, but here is Dessa talking to GPT-Live-1. Dessa’s voice sounds fuller and more expressive, giving listeners the impression that it is a more “human” voice between the two.
One Dessa team member, Aydin Polat, posted a demo of the model being emotional and almost breaking down while speaking. With the premise that this is an AI model, the demo feels unsettling.
Dessa’s technical feat of making a voice model sound human is impressive, but opens up a can of worms around safety. If this model, or equivalent ones, are opened up, miscreants can call and pretend to be humans to scam people. We will explore both parts one after another below.
Behind Dessa
While Dessa is the new voice model’s name, it is also the name of the startup Block acquired six years ago. Even at that time, the company focused on building AI for enterprise automation and performing machine learning research.
One of the startup’s co-founders, Ragavan, has led the research for the voice model. We know what hardware it was trained on, but we don’t know how much actual data the model needed, and how much time it took to train it.
Trained on one NVIDIA rack
Used only conversational data to train the model
Conversational data, not text, provides the knowledge base for the model
Can run locally on a MacBook M4 Max or better
What’s more, it’s unclear if Block used data from Square customers to train its model.
The research lab made an argument in its blog post that the cascaded method of voice training and deployment, where text is involved, doesn’t inherently represent human emotion. In a blog post, it said:
Speech never repeats. Say a sentence twice, exactly the same way, and the data still comes out different. Text is identical no matter who types it. And a single spoken word takes thousands of times more bytes than the same word in text. People learn to talk so easily that we forget how hard it is. For a machine with no text to lean on, it’s brutal.
The team also called current AI “deaf to emotion in our voices.” To be clear, there are startups working on emotional understanding of voice from different angles, including Hume, Modulate, and Amplifier Health. What Dessa meant was that there are no speech models that carry that understanding.
Block is a financial company, so it can deploy a human-sounding model for an AI receptionist, customer support, and also on its newly launched Square Kiosk.
The technical feat is commendable, but there is a big social downside to it.
The argument against non-human ‘human’ voices
In his 2023 essay, American philosopher Daniel Dennett warned us against creating artificial people and identities in The Atlantic:
Today, for the first time in history, thanks to artificial intelligence, it is possible for anybody to make counterfeit people who can pass for real in many of the new digital environments we have created. These counterfeit people are the most dangerous artifacts in human history, capable of destroying not just economies but human freedom itself.
His argument was that creating fake people will allow mistrust to seep into society and allow big corporations and governments to control us, as counterfeit voices could be manipulative.
“The moment has arrived to insist on making anybody who even thinks of counterfeiting people feel ashamed—and duly deterred from committing such an antisocial act of vandalism. If we spread the word now that such acts will be against the law as soon as we can arrange it, people will have no excuse for persisting in their activities,” he said.
While this might sound doomerism to many, Dessa’s demo did make people uncomfortable at the possibility of human-like voices scamming or tricking others, especially older people who might not be fluent in telling artificial and real voices apart.
Neuroscientist Anil Seth agrees with this in a piece he wrote earlier this month.
“Cheating senior citizens out of their savings is bad enough, but the threat to society goes much deeper. Already, on social media platforms like X, conversations between real humans are increasingly peppered with AI-generated posts. When we cannot tell whether there is a real mind behind the words we read, it becomes hard to take anything on trust, and it might not be long before humans check out entirely. Bring audio and video into the picture, and it could be game over,” he said.
Reactions on X in relation to Dessa’s launch are reflective of the reservations people feel.
One employee of Pangram, the service that detects AI-generated text, said the company might have to look at the audio space next, after hearing the demo.
Thankfully, Dessa is not a publicly available model yet, and Block is only giving access to safety researchers at the moment. I’ll be keen to see observations from that community.
Dessa’s demo is an indicator that companies have progressed infinitely towards making AI sound like humans, and that might not always be a good thing.
Model Behavior
ElevenLabs launched Eleven Realtime (mini and max) speech-to-speech model with a cascaded architecture. The company also released Realtime Translate this week.
Mistral released its Voxtral 3 suite with both transcription and speech generation models.
Scaled Cognition released its enterprise-focused text-to-speech models with pricing of $1 per million characters.
Salesforce detailed its Luna Omni voice model, which is a text model moulded for audio.
Gradium’s new Photon open-source speech generation model can run on any device and fit into 64.5 MB.
IndiaAI Mission-backed startup Chariot releases speech recognition model with support for 10 languages.
Quick Bytes
A new paper indicates that AI had a negative effect on assisting salespeople in China. Notably, this study was conducted two years ago, so it’s not clear if newer models would have any impact on it.
Indie music publisher Reservoir Media sued both Suno and Anthropic, alleging the companies used its materials to train AI.
Florida police said that scammers used AI to trick citizens with a sheriff’s cloned voice.
Apple acquired audio editing and sampling software Samplab, which is known for converting audio into MIDI. This acquisition could mean features in GarageBand.
Spotify said that it has removed over 50 million AI-generated songs it considered to be spam.
Deals Corner
Total funding this week: $354.6 million
Otter AI ($320 million non-dilutive financing): The AI meeting assistant company that transcribes, summarizes, and turns meetings into actions.
Investors: General Catalyst’s Customer Value Fund (sole financier)
Nirva ($25 million pre-Series A): The company behind a $249 pendant that focuses on you and gives you insight about both personal and professional patterns. Total funding now stands at $33 million.
Investors: Lightspeed Venture Partners and South Park Commons
Karta AI ($3 million seed): The agentic voice AI company handling inbound customer calls for banks, lenders, fintechs, and insurers, founded by ex-Razorpay operators from India.
Investors: PeerCapital and Cedar Hill Capital (co-leads); Wyser VC, CDM Capital, and Seedstars
ArtMaster ($2.4 million seed): The Prague company behind Artie, an AI piano coach that listens to you play and teaches in real time. Total funding now stands at roughly $4 million.
Investors: Factorial Capital, Reflex Capital, and Moises (new); angels Nir Zicherman, Grant Davis, and Eliška Vorel
Hunar.AI ($4.2 million Series A): The voice AI company that screens, interviews, and onboards frontline workers over phone calls, WhatsApp, and chat in 20+ Indian languages with clients like Swiggy and Flipkart.
Investors: Blume Ventures (lead); Hyderabad Angels Fund, Together Fund, Seedstars, AVNM Ventures, and PhysicsWallah co-founders Prateek Maheshwari and Alakh Pandey (existing)
Thank you for tuning in. Keep listening.
This newsletter is by Ivan Mehta, a freelance reporter at TechCrunch. It covers AI and technology in voice, audio, and music. Email: voiceaiweek@gmail.com or im@ivanmehta.com