Signals
Granola released a dictation feature months after Wispr Flow released a meeting notetaker. I wrote about that clash in detail over the weekend. The key trend is that we will see voice AI consumer use cases and apps converging towards the same goals over time. Expect others to release dictation, hardware, meeting notetaking, assistants, and more. This can very well lead to acquisitions like Superhuman’s Fathom deal in September.
In Focus
Enterprise voice AI has had a few playbooks to scale up to now. You make a good model, and then scale it through product-led growth or aggressively hire teams of sales, go-to-market, and forward-deployed engineers in all markets while raising a lot of money — just like ElevenLabs did.
Last week, Deepgram added a slightly different strategy of owning an infrastructure layer by acquiring Pipecat maker Daily.
Both Deepgram and Daily come from the same Y Combinator batch, and have engineering-focused leaders in Scott Stephenson and Kwindla Kramer. The message about the continuation of commitment to open-source and supporting the developer ecosystem is clear.
In a post, Kramer said that his title at Deepgram is going to be “head of open source,” indicating that Pipecat will have an open-source focus in the coming years.
He has been a proponent of getting many people involved to figure out problems in voice AI since Daily started working on Pipecat.
“The natural thing to do there [referring to the early days of setting up Pipecat] is just like make everything you’re doing that makes this stuff easier open source because you want a ton of people to like come into this big tent with you and like figure it all out together. So we made that decision about Pipecat pretty early and just thought, okay, we’re gonna put some of our efforts into this ecosystem, this community,” he told me in a conversation.
Both companies said that there is a new Pipecat foundation that also has leading AI companies to maintain the open-source nature of the framework
Infrastructure play
With this acquisition, Deepgram can also reposition itself in the long run. Until now, it was known as a good voice model maker (especially on the speech-to-text front), which could be an option for developers to build their applications on or enterprises to run their calling stack.
Daily being in the organization would allow it to become an infrastructure company. If there is a healthy developer ecosystem, there is going to be a very iterative and quick feedback loop about holes in models and infrastructure.
Director of developer relations at infrastructure company Agora, Hermes Frangoudis, said that we are bound to see consolidation in Voice AI.
I slightly disagree on the model front, given Deepgram has released a text-to-speech model, but we don’t know how popular it is just yet. The company is likely to release other models. But Frangoudis is right in observing that just being a model maker might not be enough long-term.
It is also worth looking outside the Voice AI sphere. Daily started as an infrastructure company for real-time communication over the WebRTC standard. This core structure can enable any kind of data pipeline to flow end-to-end. Even if Deepgram doesn’t end up developing models for different modalities (such as video), it can become a crucial piece for communication.
“Pipecat is well on its way to becoming what one of our partners describes as “Linux for real-time AI,” Kramer said in his post on X.
“Our goal is to create the standards and the standard toolkit for all software that has a human in the loop,” he said later.
Operator speak
Before the acquisition was announced, I spoke to both Stephenson and Kramer over a call. Here are some of the key takeaways from the conversation.
Scott Stephenson
Supporting the voice AI developer ecosystem through Pipecat:
The builder is the one who’s going to build this new future. We just give them really great tools and infrastructure and models in order to build that future on a really reliable infrastructure and with great patterns that they can, you know, base their products on. And that’s one thing that Pipecat has done so well.
Enabling real-time experiences:
I am very excited about the fact that there is already a lot of AI usage in the world, but most of it is batch and what I would describe as interactive mode. So maybe you wait 10 seconds or a minute or two or something for a result to come back. But I think almost all of that, and more, because there will be growth, will move to real time in the future.
Building different voice AI systems
I think most people building voice agents feel like they have only two choices. They could build a cascaded system that sort of doesn’t have all of the context but. And then they look at another way to build systems, like the end-to-end speech models, which are expressive but often miss the control and bring uncertainty.
This is why we came together to create this real-time context standard because you actually can have both. You can get the control, and you can get the expressivity.
Kwindla Kramer
What Pipecat enabled for Voice AI:
Things like being interruptible and steerable, doing things in the background without blocking the main conversation, context engineering under the hood that lets the agent deal with multiple threads of conversation without you as the user having to start a new thread and click around between threads manually. All that stuff has been in Pipecat since the beginning.
Why voice AI is not a solved problem:
Everything’s always a systems engineering problem. You build better components, then you can put them together in new and more interesting ways, which means you need even better components. It also means you can build something more, you know, aggressive and interesting and ambitious, and then you need better components. And that’s true for mechanical engineering; it’s true for chemistry; it’s true for, you know, programming. You’re never finished.
Experiments
India is a complex landscape where you might not know the local language, which sometimes makes it difficult to get houses and transport. A user called Rajvi Shah created a Cartesia agent to call brokers in Kannada and negotiate rents and deposits on her behalf.
Companies like DeepL and Krisp have been working on call-based translations. However, there is a need for this feature in the consumer sector. Y Combinator-backed EzDubs was working on this layer, but got acquired by Cisco last year.
Model Behaviour
ElevenLabs launches a new model for sound effects to create multiple sounds in a video at the same time.
Nvidia released a new open transcription model called Nemotron 3.5, which supports 40 languages.
Cekura released a new text-to-speech benchmark comparing over 26 models
Quick Bytes
Spotify plans to introduce an AI remix feature next year. Until now, startups like Hook and platforms like TikTok and YouTube have experimented with this feature, but not always centered around music.
Native Instruments unveils AI-powered preset generation.
Nikki detailed how a New Zealand dubbing company is helping Japanese execs reach global audiences.
Regal launches agents that behave according to the goal set by operators.
Election Commission in South Africa warned voters about a Nelson Mandela deepfake.
Note: My deal tracker is now updated until October, so go and have a look.
Deals Corner (Weekly funding: $29.5 million)
Phonic ($27.5 million Series A): Builds real-time speech-to-speech voice AI models for developers. Total funding now stands at $31.5 million.
Investors: First Round Capital (lead); Lux Capital, Felicis Ventures, Definition, Essence VC (existing)
EDGE Sound Research ($2 million funding round): Builds audio perception infrastructure that lets AI understand the physical world through sound. Round type not disclosed.
Thank you for tuning in. Keep listening.
This newsletter is by Ivan Mehta, a freelance reporter at TechCrunch. It covers AI and technology in voice, audio, and music. Email: voiceaiweek@gmail.com or im@ivanmehta.com