Spotify Using AI to Translate Podcasters' Original Voices to Additional Languages

[MacRumors]

Spotify this week announced an AI translation feature for podcasters that could give it an edge over Apple's podcasting platform. Voice Translation for podcasts uses artificial intelligence to translate podcasts into additional languages using the original podcaster's voice.

While Spotify developed the tool, it is using OpenAI's latest voice generation technology. It learns the original speaker's voice and style and then translates the podcast into another language. Spotify says that this system will provide a "more authentic listening experience" that is more natural and personal than standard dubbing. Podcasts translated this way will retain the speaker's distinctive speech characteristics.

Spotify is testing Voice Translation at the current time, and is working with podcasters like Dax Shepard, Monica Padman, Bill Simmons, Steven Bartlett, and Lex Fridman to create AI-powered voice translations in languages like Spanish, French, and German for both existing episodes and future podcast episodes.

Voice-translated episodes are available worldwide to Premium and Free users. An initial bundle of translated episodes in Spanish are available now, with French and German rolling out "in the coming days and weeks." According to Spotify, the pilot program will provide "important insights" for future expansion and iteration.

While Apple does not have a competing tool at this time for the Podcasts platform, it is experimenting with AI voice technology. In iOS 17, Apple added a Personal Voice feature that allows you to use AI to create a replica of your voice. Right now, this is an accessibility feature that Apple has designed for those who are at risk of losing their ability to speak, but it stands to reason that Apple could use voice replication in other areas in the future.

Article Link: Spotify Using AI to Translate Podcasters' Original Voices to Additional Languages

 

[icanhazmac]

No way I would ever allow another entity to "create" content in my voice. Before you know it you are getting canceled and apologizing for an AI error that caused "you" to say some horrific stuff in a language you don't even speak.

 

[LeeW]

I mean if it works reasonably well, why not? Wider audience and more revenue for the podcast.

 

[KanosWRX]

How else will we get the voice of Darth Vader for the next 100 years! In all seriousness, James Earl Jones agreed to just that. If it helps you reach new audiences and you have control over it, why not! Don't they use Google Cloud, makes sense they would have access to some cool new stuff!

 

[NEPOBABY]

I mean if it works reasonably well, why not? Wider audience and more revenue for the podcast.

There will definitely be many bad and messed up translations.

Relying on algorithms alone to translate languages is notoriously and famously difficult.

Anyone who has used YouTube or Instagram's translators knows how bad it is. Sometimes the meaning is changed completely.

Instagram translates my Chinese and Japanese friends posts wrong 50% of the time and converts their words into dumbed down American ghetto slang, which not only is a mistranslation but also paints people in a bad light.

 

[zakarhino]

Great technology, obviously there will be improvements to this over time so it's more accurate. This is the kind of thing Google will add system wide into Android soon, Apple should follow them. Anything that opens up access to more foreign content is a big win.

 

[Analog Kid]

My god, the wonders we now possess…

 

[Mrkevinfinnerty]

Great technology. This is the kind of thing Google will add system wide into Android soon, Apple should follow them. Anything that opens up access to more foreign content is a big win.

What about privacy, everything on your iPhone is private. Lol

 

[zakarhino]

What about privacy, everything on your iPhone is private. Lol

Apple's translate app already works on device so does Google's on device system wide caption software.

 

[Love-hate 🍏 relationship]

google's working on it with lip-sync

 

[Love-hate 🍏 relationship]

Great technology, obviously there will be improvements to this over time so it's more accurate. This is the kind of thing Google will add system wide into Android soon, Apple should follow them. Anything that opens up access to more foreign content is a big win.

any source about the android system implementation ?

 

[zakarhino]

any source about the android system implementation ?

None, I'm speculating this feature will be the next step up from the on device captions they already do. Apple added this last year or this year I think. Google have been doing it for years.

"All captions are processed locally, never stored, and never leave your device."

Live Caption: Caption media & calls on your device - Android Accessibility Help

With a single tap, Live Caption automatically captions speech on your device. You can use it on media like videos, podcasts, phone calls, video calls, and audio messages. Important: Some of

support.google.com

 

[mrfunnypenguin]

Have they completely forgot about HiFi?

 

[cbaca51]

I don’t see an issue with this. Everyone’s issue with AI (including mine) is lack of consent and getting payed for these AI companies stealing your work. If an artist agrees and is payed for their work getting used, I don’t see an issue.

 

[HobeSoundDarryl]

I don’t see an issue with this. Everyone issue with AI (including mine) is lack of consent and getting payed for these AI companies stealing your work. If an artist agrees and is payed for their work getting used, I don’t see an issue.

Apple people see issues because this is a Spotify story. Take this one down, wait a day or two and put it back up as an Apple Podcasts story and we'll be gushing at Apple genius for the same thing. "We" are a very fair weather crowd. If it's an Apple innovation, it is genius. If it is an Apple competitor innovation, no one needs it, it's buggy as h*ll, it will never work right, security, privacy, risk of death/murder/mayhem/et all.

If it works pretty well, this seems great to me: great for Spotify, great for the podcasters, great for listeners who would like to hear some podcasts but in their own supported languages (even if there are some translation errors). Hopefully competition moves Apple to come up with the same or better ASAP. Competition is always good.

 

[jdawgnoonan]

I cannot understand wanting to listen to this.

 

[jdawgnoonan]

Apple people see issues because this is a Spotify story. Take this one down, wait a day or two and put it back up as an Apple Podcasts story and we'll be gushing at Apple genius for the same thing. "We" are a very fair weather crowd. If it's an Apple innovation, it is genius. If it is an Apple competitor innovation, no one needs it, it's buggy as h*ll, it will never work right, security, privacy, risk of death/murder/mayhem/et all.

If it works pretty well, this seems great to me: great for Spotify, great for the podcasters, great for listeners who would like to hear some podcasts but in their own supported languages (even if there are some translation errors). Hopefully competition moves Apple to come up with the same or better ASAP. Competition is always good.

I don't think people are reacting negatively to this simply because it is Spotify. I personally would have as much interest in listening to an AI version of a podcast emulating the host in a different language as I would in listening to a screen reader reading a web page to me. No interest at all. If the hosts of the shows agreed to this being okay then it is. But I struggle to picture what would make it worth listening to an AI version of content.

 

[HobeSoundDarryl]

I cannot understand wanting to listen to this.

Imagine a really desirable podcast you'd love to listen to but it's not in your native language. Click a button to listen to it in your language. You get what you want instead of not being able to listen at all because you don't understand the primary language of the podcast.

More simply: think of those old Godzilla monster movies shot in Japan. Imagine no translations and no subtitles. Probably not as fun to watch, eh? Someone goes to the trouble of overdubbing them in your own language and you likely enjoy them a bit more. Apply the same to any foreign language film now.

 

[HobeSoundDarryl]

I don't think people are reacting negatively to this simply because it is Spotify. I personally would have as much interest in listening to an AI version of a podcast emulating the host in a different language as I would in listening to a screen reader reading a web page to me. No interest at all. If the hosts of the shows agreed to this being okay then it is. But I struggle to picture what would make it worth listening to an AI version of content.

If these are paid podcasts, the hosts will enjoy making MORE money because it widens the audience.

I completely missed how this is being implemented without the hosts consent. Of course, if that was true and this is only for Spotify benefit, the host could quit offering their podcast for Spotify and shift it to other channels.

As to the AI variable, love it or hate it, it's a part of tech for the foreseeable future. This particular use doesn't seem to be plotting to overthrow humanity. There are some apps available now that can do some translations of foreign content and I've found them quite useful in places where I don't know the language. I see this as simply more of the same. Seems like a great OPTION to someone wanting to listen to a podcast but does not know the language.

For example, I'm planning a trip to Iceland. I don't know the language at all. I'd love to get some insiders tips on where to go, what to do, what to see, etc. Maybe there is an Icelandic podcast offering the ultimate insider's tips. This would plausibly let me tap into that information vs. not being able to at all. If so, what a great use for A.I. translation tools! And if that podcast makes money, the podcast creator could now make more money from me than having no chance at all to make money from me.

 

[jdawgnoonan]

I cannot stand watching movies where non native speakers do voiceovers. I much prefer subtitles.

 

[jdawgnoonan]

If these are paid podcasts, the hosts will enjoy making MORE money because it widens the audience.

I completely missed how this is being implemented without the hosts consent. Of course, if that was true and this is only for Spotify benefit, the host could quit offering their podcast for Spotify and shift it to other channels.

Since Podcasts have overall not created growth for Spotify I presume that at some point they might drop them.

 

[ikramerica]

No way I would ever allow another entity to "create" content in my voice. Before you know it you are getting canceled and apologizing for an AI error that caused "you" to say some horrific stuff in a language you don't even speak.

It also means all confessions should be tossed out in court, written or spoken.

 

[RodThePlod]

I do wonder how accurate these translations will be. The company I work for do a lot of automated document localisation. From English to French/German/Spanish, etc. (18 different languages in total).

The resulting output needs heavy cleanup - and this is text-based localisation tech that’s been around for many years.

Doing voice accurately seems like a much more difficult task that would be pretty error-prone!

 

[HobeSoundDarryl]

I cannot stand watching movies where non native speakers do voiceovers. I much prefer subtitles.

Since Podcasts have overall not created growth for Spotify I presume that at some point they might drop them.

OK and OK. Apparently this has no benefit for you. Last I checked, there are other people in the world. Perhaps it's something THEY might appreciate while Spotify continues to offer them?

 

[jdawgnoonan]

OK and OK. Apparently this has no benefit for you. Last I checked, there are other people in the world. Perhaps it's something THEY might appreciate while Spotify continues to offer them?

Yeah, for me. You take this stuff pretty personally evidently.