The hardest part of speech-to-speech translation is language processing, not signal processing.

Evidence

One application that is also actively being studied is a telephone service that translates the language of a speaker into another language. Although this service has not yet been achieved, translation in limited cases has been possible. Optimally, the characteristics of the voice of the original speaker should be preserved in the translated speech output. The biggest challenges are, however, in the processing of language, and not in the processing of speech signals. — Ville Pulkki, Matti Karjalainen, Communication Acoustics- An…, 2275

Categorization

• XREF: Connects to modern LLM-based real-time translation apps and voice-cloning preservation of speaker characteristics.
• SEED: Article angle: why speaking-to-speaking translation remains unsolved decades later — the bottleneck is linguistic, not acoustic.