不能否认,微软Azure在TTS(text-to-speech文字转语音)这个人工智能细分领域的影响力是统治级的,一如ChatGPT在NLP领域的随心所欲,予取予求。君不见几乎所有的抖音营销号口播均采用微软的语音合成技术,其影响力由此可见一斑,仅有的白璧微瑕之处就是价格略高,虽然国内也可以使用科大讯飞语音合成进行平替,但我们只想要最好的那一个,本次我们使用免费的开源库edge-tts来实现文本转语音操作,薅微软edge的羊毛。
TTS文本转语音基础使用方式
首先安装edge-tts库:
pip3 install edge-tts
安装成功后,直接在终端运行edge-tts命令:
edge-tts
显示帮助菜单即代表安装成功:
➜ Downloads edge-tts
usage: edge-tts [-h] [-t TEXT] [-f FILE] [-v VOICE] [-l] [--rate RATE] [--volume VOLUME] [-O OVERLAPPING] [--write-media WRITE_MEDIA] [--write-subtitles WRITE_SUBTITLES] [--proxy PROXY]
随后输入命令:
edge-tts --list-voices
该命令可以将Edge浏览器中,内置的语言角色列表列出来:
Name: af-ZA-AdriNeural
Gender: Female Name: af-ZA-WillemNeural
Gender: Male Name: am-ET-AmehaNeural
Gender: Male Name: am-ET-MekdesNeural
Gender: Female Name: ar-AE-FatimaNeural
Gender: Female Name: ar-AE-HamdanNeural
Gender: Male Name: ar-BH-AliNeural
Gender: Male Name: ar-BH-LailaNeural
Gender: Female Name: ar-DZ-AminaNeural
Gender: Female Name: ar-DZ-IsmaelNeural
Gender: Male Name: ar-EG-SalmaNeural
Gender: Female Name: ar-EG-ShakirNeural
Gender: Male Name: ar-IQ-BasselNeural
Gender: Male Name: ar-IQ-RanaNeural
Gender: Female Name: ar-JO-SanaNeural
Gender: Female Name: ar-JO-TaimNeural
Gender: Male Name: ar-KW-FahedNeural
Gender: Male Name: ar-KW-NouraNeural
Gender: Female Name: ar-LB-LaylaNeural
Gender: Female Name: ar-LB-RamiNeural
Gender: Male Name: ar-LY-ImanNeural
Gender: Female Name: ar-LY-OmarNeural
Gender: Male Name: ar-MA-JamalNeural
Gender: Male Name: ar-MA-MounaNeural
Gender: Female Name: ar-OM-AbdullahNeural
Gender: Male Name: ar-OM-AyshaNeural
Gender: Female Name: ar-QA-AmalNeural
Gender: Female Name: ar-QA-MoazNeural
Gender: Male Name: ar-SA-HamedNeural
Gender: Male Name: ar-SA-ZariyahNeural
Gender: Female Name: ar-SY-AmanyNeural
Gender: Female Name: ar-SY-LaithNeural
Gender: Male Name: ar-TN-HediNeural
Gender: Male Name: ar-TN-ReemNeural
Gender: Female Name: ar-YE-MaryamNeural
Gender: Female Name: ar-YE-SalehNeural
Gender: Male Name: az-AZ-BabekNeural
Gender: Male Name: az-AZ-BanuNeural
Gender: Female Name: bg-BG-BorislavNeural
Gender: Male Name: bg-BG-KalinaNeural
Gender: Female Name: bn-BD-NabanitaNeural
Gender: Female Name: bn-BD-PradeepNeural
Gender: Male Name: bn-IN-BashkarNeural
Gender: Male Name: bn-IN-TanishaaNeural
Gender: Female Name: bs-BA-GoranNeural
Gender: Male Name: bs-BA-VesnaNeural
Gender: Female Name: ca-ES-EnricNeural
Gender: Male Name: ca-ES-JoanaNeural
Gender: Female Name: cs-CZ-AntoninNeural
Gender: Male Name: cs-CZ-VlastaNeural
Gender: Female Name: cy-GB-AledNeural
Gender: Male Name: cy-GB-NiaNeural
Gender: Female Name: da-DK-ChristelNeural
Gender: Female Name: da-DK-JeppeNeural
Gender: Male Name: de-AT-IngridNeural
Gender: Female Name: de-AT-JonasNeural
Gender: Male Name: de-CH-JanNeural
Gender: Male Name: de-CH-LeniNeural
Gender: Female Name: de-DE-AmalaNeural
Gender: Female Name: de-DE-ConradNeural
Gender: Male Name: de-DE-KatjaNeural
Gender: Female Name: de-DE-KillianNeural
Gender: Male Name: el-GR-AthinaNeural
Gender: Female Name: el-GR-NestorasNeural
Gender: Male Name: en-AU-NatashaNeural
Gender: Female Name: en-AU-WilliamNeural
Gender: Male Name: en-CA-ClaraNeural
Gender: Female Name: en-CA-LiamNeural
Gender: Male Name: en-GB-LibbyNeural
Gender: Female Name: en-GB-MaisieNeural
Gender: Female Name: en-GB-RyanNeural
Gender: Male Name: en-GB-SoniaNeural
Gender: Female Name: en-GB-ThomasNeural
Gender: Male Name: en-HK-SamNeural
Gender: Male Name: en-HK-YanNeural
Gender: Female Name: en-IE-ConnorNeural
Gender: Male Name: en-IE-EmilyNeural
Gender: Female Name: en-IN-NeerjaExpressiveNeural
Gender: Female Name: en-IN-NeerjaNeural
Gender: Female Name: en-IN-PrabhatNeural
Gender: Male Name: en-KE-AsiliaNeural
Gender: Female Name: en-KE-ChilembaNeural
Gender: Male Name: en-NG-AbeoNeural
Gender: Male Name: en-NG-EzinneNeural
Gender: Female Name: en-NZ-MitchellNeural
Gender: Male Name: en-NZ-MollyNeural
Gender: Female Name: en-PH-JamesNeural
Gender: Male Name: en-PH-RosaNeural
Gender: Female Name: en-SG-LunaNeural
Gender: Female Name: en-SG-WayneNeural
Gender: Male Name: en-TZ-ElimuNeural
Gender: Male Name: en-TZ-ImaniNeural
Gender: Female Name: en-US-AnaNeural
Gender: Female Name: en-US-AriaNeural
Gender: Female Name: en-US-ChristopherNeural
Gender: Male Name: en-US-EricNeural
Gender: Male Name: en-US-GuyNeural
Gender: Male Name: en-US-JennyNeural
Gender: Female Name: en-US-MichelleNeural
Gender: Female Name: en-US-RogerNeural
Gender: Male Name: en-US-SteffanNeural
Gender: Male Name: en-ZA-LeahNeural
Gender: Female Name: en-ZA-LukeNeural
Gender: Male Name: es-AR-ElenaNeural
Gender: Female Name: es-AR-TomasNeural
Gender: Male Name: es-BO-MarceloNeural
Gender: Male Name: es-BO-SofiaNeural
Gender: Female Name: es-CL-CatalinaNeural
Gender: Female Name: es-CL-LorenzoNeural
Gender: Male Name: es-CO-GonzaloNeural
Gender: Male Name: es-CO-SalomeNeural
Gender: Female Name: es-CR-JuanNeural
Gender: Male Name: es-CR-MariaNeural
Gender: Female Name: es-CU-BelkysNeural
Gender: Female Name: es-CU-ManuelNeural
Gender: Male Name: es-DO-EmilioNeural
Gender: Male Name: es-DO-RamonaNeural
Gender: Female Name: es-EC-AndreaNeural
Gender: Female Name: es-EC-LuisNeural
Gender: Male Name: es-ES-AlvaroNeural
Gender: Male Name: es-ES-ElviraNeural
Gender: Female Name: es-ES-ManuelEsCUNeural
Gender: Male Name: es-GQ-JavierNeural
Gender: Male Name: es-GQ-TeresaNeural
Gender: Female Name: es-GT-AndresNeural
Gender: Male Name: es-GT-MartaNeural
Gender: Female Name: es-HN-CarlosNeural
Gender: Male Name: es-HN-KarlaNeural
Gender: Female Name: es-MX-DaliaNeural
Gender: Female Name: es-MX-JorgeNeural
Gender: Male Name: es-MX-LorenzoEsCLNeural
Gender: Male Name: es-NI-FedericoNeural
Gender: Male Name: es-NI-YolandaNeural
Gender: Female Name: es-PA-MargaritaNeural
Gender: Female Name: es-PA-RobertoNeural
Gender: Male Name: es-PE-AlexNeural
Gender: Male Name: es-PE-CamilaNeural
Gender: Female Name: es-PR-KarinaNeural
Gender: Female Name: es-PR-VictorNeural
Gender: Male Name: es-PY-MarioNeural
Gender: Male Name: es-PY-TaniaNeural
Gender: Female Name: es-SV-LorenaNeural
Gender: Female Name: es-SV-RodrigoNeural
Gender: Male Name: es-US-AlonsoNeural
Gender: Male Name: es-US-PalomaNeural
Gender: Female Name: es-UY-MateoNeural
Gender: Male Name: es-UY-ValentinaNeural
Gender: Female Name: es-VE-PaolaNeural
Gender: Female Name: es-VE-SebastianNeural
Gender: Male Name: et-EE-AnuNeural
Gender: Female Name: et-EE-KertNeural
Gender: Male Name: fa-IR-DilaraNeural
Gender: Female Name: fa-IR-FaridNeural
Gender: Male Name: fi-FI-HarriNeural
Gender: Male Name: fi-FI-NooraNeural
Gender: Female Name: fil-PH-AngeloNeural
Gender: Male Name: fil-PH-BlessicaNeural
Gender: Female Name: fr-BE-CharlineNeural
Gender: Female Name: fr-BE-GerardNeural
Gender: Male Name: fr-CA-AntoineNeural
Gender: Male Name: fr-CA-JeanNeural
Gender: Male Name: fr-CA-SylvieNeural
Gender: Female Name: fr-CH-ArianeNeural
Gender: Female Name: fr-CH-FabriceNeural
Gender: Male Name: fr-FR-DeniseNeural
Gender: Female Name: fr-FR-EloiseNeural
Gender: Female Name: fr-FR-HenriNeural
Gender: Male Name: ga-IE-ColmNeural
Gender: Male Name: ga-IE-OrlaNeural
Gender: Female Name: gl-ES-RoiNeural
Gender: Male Name: gl-ES-SabelaNeural
Gender: Female Name: gu-IN-DhwaniNeural
Gender: Female Name: gu-IN-NiranjanNeural
Gender: Male Name: he-IL-AvriNeural
Gender: Male Name: he-IL-HilaNeural
Gender: Female Name: hi-IN-MadhurNeural
Gender: Male Name: hi-IN-SwaraNeural
Gender: Female Name: hr-HR-GabrijelaNeural
Gender: Female Name: hr-HR-SreckoNeural
Gender: Male Name: hu-HU-NoemiNeural
Gender: Female Name: hu-HU-TamasNeural
Gender: Male Name: id-ID-ArdiNeural
Gender: Male Name: id-ID-GadisNeural
Gender: Female Name: is-IS-GudrunNeural
Gender: Female Name: is-IS-GunnarNeural
Gender: Male Name: it-IT-DiegoNeural
Gender: Male Name: it-IT-ElsaNeural
Gender: Female Name: it-IT-IsabellaNeural
Gender: Female Name: ja-JP-KeitaNeural
Gender: Male Name: ja-JP-NanamiNeural
Gender: Female Name: jv-ID-DimasNeural
Gender: Male Name: jv-ID-SitiNeural
Gender: Female Name: ka-GE-EkaNeural
Gender: Female Name: ka-GE-GiorgiNeural
Gender: Male Name: kk-KZ-AigulNeural
Gender: Female Name: kk-KZ-DauletNeural
Gender: Male Name: km-KH-PisethNeural
Gender: Male Name: km-KH-SreymomNeural
Gender: Female Name: kn-IN-GaganNeural
Gender: Male Name: kn-IN-SapnaNeural
Gender: Female Name: ko-KR-InJoonNeural
Gender: Male Name: ko-KR-SunHiNeural
Gender: Female Name: lo-LA-ChanthavongNeural
Gender: Male Name: lo-LA-KeomanyNeural
Gender: Female Name: lt-LT-LeonasNeural
Gender: Male Name: lt-LT-OnaNeural
Gender: Female Name: lv-LV-EveritaNeural
Gender: Female Name: lv-LV-NilsNeural
Gender: Male Name: mk-MK-AleksandarNeural
Gender: Male Name: mk-MK-MarijaNeural
Gender: Female Name: ml-IN-MidhunNeural
Gender: Male Name: ml-IN-SobhanaNeural
Gender: Female Name: mn-MN-BataaNeural
Gender: Male Name: mn-MN-YesuiNeural
Gender: Female Name: mr-IN-AarohiNeural
Gender: Female Name: mr-IN-ManoharNeural
Gender: Male Name: ms-MY-OsmanNeural
Gender: Male Name: ms-MY-YasminNeural
Gender: Female Name: mt-MT-GraceNeural
Gender: Female Name: mt-MT-JosephNeural
Gender: Male Name: my-MM-NilarNeural
Gender: Female Name: my-MM-ThihaNeural
Gender: Male Name: nb-NO-FinnNeural
Gender: Male Name: nb-NO-PernilleNeural
Gender: Female Name: ne-NP-HemkalaNeural
Gender: Female Name: ne-NP-SagarNeural
Gender: Male Name: nl-BE-ArnaudNeural
Gender: Male Name: nl-BE-DenaNeural
Gender: Female Name: nl-NL-ColetteNeural
Gender: Female Name: nl-NL-FennaNeural
Gender: Female Name: nl-NL-MaartenNeural
Gender: Male Name: pl-PL-MarekNeural
Gender: Male Name: pl-PL-ZofiaNeural
Gender: Female Name: ps-AF-GulNawazNeural
Gender: Male Name: ps-AF-LatifaNeural
Gender: Female Name: pt-BR-AntonioNeural
Gender: Male Name: pt-BR-FranciscaNeural
Gender: Female Name: pt-PT-DuarteNeural
Gender: Male Name: pt-PT-RaquelNeural
Gender: Female Name: ro-RO-AlinaNeural
Gender: Female Name: ro-RO-EmilNeural
Gender: Male Name: ru-RU-DmitryNeural
Gender: Male Name: ru-RU-SvetlanaNeural
Gender: Female Name: si-LK-SameeraNeural
Gender: Male Name: si-LK-ThiliniNeural
Gender: Female Name: sk-SK-LukasNeural
Gender: Male Name: sk-SK-ViktoriaNeural
Gender: Female Name: sl-SI-PetraNeural
Gender: Female Name: sl-SI-RokNeural
Gender: Male Name: so-SO-MuuseNeural
Gender: Male Name: so-SO-UbaxNeural
Gender: Female Name: sq-AL-AnilaNeural
Gender: Female Name: sq-AL-IlirNeural
Gender: Male Name: sr-RS-NicholasNeural
Gender: Male Name: sr-RS-SophieNeural
Gender: Female Name: su-ID-JajangNeural
Gender: Male Name: su-ID-TutiNeural
Gender: Female Name: sv-SE-MattiasNeural
Gender: Male Name: sv-SE-SofieNeural
Gender: Female Name: sw-KE-RafikiNeural
Gender: Male Name: sw-KE-ZuriNeural
Gender: Female Name: sw-TZ-DaudiNeural
Gender: Male Name: sw-TZ-RehemaNeural
Gender: Female Name: ta-IN-PallaviNeural
Gender: Female Name: ta-IN-ValluvarNeural
Gender: Male Name: ta-LK-KumarNeural
Gender: Male Name: ta-LK-SaranyaNeural
Gender: Female Name: ta-MY-KaniNeural
Gender: Female Name: ta-MY-SuryaNeural
Gender: Male Name: ta-SG-AnbuNeural
Gender: Male Name: ta-SG-VenbaNeural
Gender: Female Name: te-IN-MohanNeural
Gender: Male Name: te-IN-ShrutiNeural
Gender: Female Name: th-TH-NiwatNeural
Gender: Male Name: th-TH-PremwadeeNeural
Gender: Female Name: tr-TR-AhmetNeural
Gender: Male Name: tr-TR-EmelNeural
Gender: Female Name: uk-UA-OstapNeural
Gender: Male Name: uk-UA-PolinaNeural
Gender: Female Name: ur-IN-GulNeural
Gender: Female Name: ur-IN-SalmanNeural
Gender: Male Name: ur-PK-AsadNeural
Gender: Male Name: ur-PK-UzmaNeural
Gender: Female Name: uz-UZ-MadinaNeural
Gender: Female Name: uz-UZ-SardorNeural
Gender: Male Name: vi-VN-HoaiMyNeural
Gender: Female Name: vi-VN-NamMinhNeural
Gender: Male Name: zh-CN-XiaoxiaoNeural
Gender: Female Name: zh-CN-XiaoyiNeural
Gender: Female Name: zh-CN-YunjianNeural
Gender: Male Name: zh-CN-YunxiNeural
Gender: Male Name: zh-CN-YunxiaNeural
Gender: Male Name: zh-CN-YunyangNeural
Gender: Male Name: zh-CN-liaoning-XiaobeiNeural
Gender: Female Name: zh-CN-shaanxi-XiaoniNeural
Gender: Female Name: zh-HK-HiuGaaiNeural
Gender: Female Name: zh-HK-HiuMaanNeural
Gender: Female Name: zh-HK-WanLungNeural
Gender: Male Name: zh-TW-HsiaoChenNeural
Gender: Female Name: zh-TW-HsiaoYuNeural
Gender: Female Name: zh-TW-YunJheNeural
Gender: Male Name: zu-ZA-ThandoNeural
Gender: Female Name: zu-ZA-ThembaNeural
Gender: Male
一望而知,几乎支持所有主流的通用语,Gender字段为合成语音的性别,Male代表男性,Female代表女性,zh开头的就是中文语音角色,这里以微软的小伊为例子:
edge-tts --voice zh-CN-XiaoyiNeural --text "你好啊,我是智能语音助手" --write-media hello_in_cn.mp3
该命令含义是通过zh-CN-XiaoyiNeural角色合成语音:"你好啊,我是智能语音助手"的内容,随后将音频流写入hello_in_cn.mp3文件。
程序返回:
Downloads edge-tts --voice zh-CN-XiaoyiNeural --text "你好啊,我是智能语音助手" --write-media hello_in_cn.mp3
WEBVTT 00:00:00.100 --> 00:00:00.525
你好 00:00:00.525 --> 00:00:00.912
啊 00:00:01.050 --> 00:00:01.238
我 00:00:01.238 --> 00:00:01.375
是 00:00:01.387 --> 00:00:01.700
智能 00:00:01.700 --> 00:00:02.050
语音 00:00:02.062 --> 00:00:02.550
助手
程序会自动将时间轴和语音文本匹配输出,如此一来,连字幕文件也有了,可谓是一举两得,一箭双雕。
与此同时,我们也可以调整合成语音的语速:
edge-tts --rate=-50% --voice zh-CN-XiaoyiNeural --text "你好啊,我是智能语音助手" --write-media hello_in_cn.mp3
–rate参数可以通过加号或者减号同步加快或者减慢合成语音的语速。
亦或者,调整合成语音的音量:
edge-tts --volume=-50% --voice zh-CN-XiaoyiNeural --text "你好啊,我是智能语音助手" --write-media hello_in_cn.mp3
–volume参数可以调整语音的音量。
遗憾的是,和微软Azure官方的语音合成库相比,开源的语音合成库并不支持基于标记语言 (SSML)的语音调优,比如语调、情绪的调整,但这毕竟是免费的,要求也不能太高了。
Python脚本语音合成
除了通过命令进行语音合成,edge-tts也支持在Python脚本,编辑test.py文件:
import asyncio import edge_tts TEXT = "你好哟,我是智能语音助手,小伊"
VOICE = "zh-CN-XiaoyiNeural"
OUTPUT_FILE = "/Users/liuyue/Downloads/test.mp3" async def _main() -> None: communicate = edge_tts.Communicate(TEXT, VOICE) await communicate.save(OUTPUT_FILE) if __name__ == "__main__": asyncio.run(_main())
这里我们直接通过异步模式调用communicate实例的save方法,就可以并发异步生成语音合成的音频文件,非常方便。
也可以通过语音管理库来自动寻找我们需要的语言:
import asyncio
import random import edge_tts
from edge_tts import VoicesManager TEXT = "中文语音测试"
OUTPUT_FILE ="china.mp3" async def _main() -> None: voices = await VoicesManager.create() voice = voices.find(Gender="Female", Language="zh") communicate = edge_tts.Communicate(TEXT, random.choice(voice)["Name"]) await communicate.save(OUTPUT_FILE) if __name__ == "__main__": asyncio.run(_main())
这里通过内置的VoicesManager库来随机挑选中文语音角色完成语音合成操作。
除此之外,也可以通过脚本将语音流和字母同步进行生成:
import asyncio import edge_tts TEXT = "这里是语音流测试"
VOICE = "zh-CN-XiaoyiNeural"
OUTPUT_FILE = "test.mp3"
WEBVTT_FILE = "test.vtt" async def _main() -> None: communicate = edge_tts.Communicate(TEXT, VOICE) submaker = edge_tts.SubMaker() with open(OUTPUT_FILE, "wb") as file: async for chunk in communicate.stream(): if chunk["type"] == "audio": file.write(chunk["data"]) elif chunk["type"] == "WordBoundary": submaker.create_sub((chunk["offset"], chunk["duration"]), chunk["text"]) with open(WEBVTT_FILE, "w", encoding="utf-8") as file: file.write(submaker.generate_subs()) if __name__ == "__main__": asyncio.run(_main())
这里异步调用之后,音频会写入test.mp3,而字幕文件则会写入test.vtt。后续则可以通过ffmpeg将生成的字幕文件叠加到视频中即可,请参见:基于Python3(Autosub)以及Ffmpeg配合GoogleTranslation(谷歌翻译)为你的影片实现双语版字幕(逐字稿)
结语
开源语音合成edge-tts库可以提高语音合成效率,并且极大地降低了语音合成门槛,为自动化视频剪辑铺平了道路,未来结合基于Stable-Diffusion算法的AI绘图框架,人工智能AI一键式绘制、配音、上字幕的一条龙服务指日可待。