Giving Our Mother Tongue a Digital Future
Note: TaiwaneseAI.org is temporarily under maintenance. You've been redirected here."
Taiwanese is defined by the Tone Sandhi Circle.
Unlike non-tonal languages such as English and Japanese, where intonation is the primary carrier of
affect, Taiwanese requires ”Tonal-Safe Emotional Mapping.”
MTL is a phonetically consistent, multi-syllable orthography based on POJ. MTL
is sandhi-aware, meaning the
written form of a multi-syllable word reflects the tone of the front syllable after
Tone Sandhi has applied. This overcame the homonym ambiguity in character-based writing system.
Historically, this system functioned as a structural checksum;
our research suggests that native speakers utilize a form
of ”Perfect Pitch” to
decode these melodic headers. When
universal AI models fail to respect this melodic
continuity, intelligibility drops significantly.
By treating Taiwanese as a formal signal-processing system
rather than a collection of statistical probabilities, we enable
a ”Sovereign AI” that outperforms universal models in both
accuracy and cultural resonance.
MTL-TTS model utilized new modeling technologies:
-
MTLTag — The AI’s Linguistic Map. We utilize
XLM-RoBERTa for predicting the words should keep the lexical tone or not. With 20,000 labeled
sentences, we are able to achieve 98% of accuracy. For simple
conversation sentences, the best model
achieved 99% of accuracy.
🗣 Y khvoax tiøh si cidee cyn zeng ee koniuu.
🗣 Y khvoax tiøh cidee cyn zeng ee koniuu.
The only difference between the above two sentences is 'si'. And yet, the 1st
sentence meant: She looks like a quite lady. 2nd: He saw a quite lady.
Computers don't inherently understand the musicality of Taiwanese. The TTS system
needs a robust
Part-of-Speech (POS) Tagger. Not only
there is scarce resources of POS in Taiwanese, and it's complexity cannot be
fully covered. Traditionally, applying sandhi-rules proves it's limitation. (70% of
accuracy). MTL-Tag chose labeling sentences with binary labels. 0 (tone change)
and 1 (citation tone). It acts as a "script supervisor" for the AI:
Precision Tone Sandhi: Taiwanese is famous for words changing tones
based on context. MTLTag pre-calculates these shifts so the AI doesn't
have to guess. We continue striving for the best
with POS and sandhi-rules
to reduce the error percentages.
Prosody Guidance: It marks where a sentence should breathe, pause,
or rise in pitch, moving away from robotic, word-by-word synthesis.
-
Harnji segmentation: The input of MTL pipeline is able to take 'harnji' or MTL.
It is advised to use Taiwanese harnji for MTL lookup with the best match for pronunciation.
We utilized backward greedy match to look up MTL's 88,000 vocabularies (V1). When look up has multiple
choices for MTL vocabularies, we use words usage frequency and also lookup Sinica dictionary.
When there are two ambiguities, the model prefer V2 which refers to Sinica dictionary
on Hugging Face. The synthesized audio files, tone sandhi, and input text will be recoded.
MTL database include KIP, POJ, and TOJ. Hence, the saved input text can be converted KIP, POJ or TOJ
if desired.
-
MTL-Number: This module converts Taiwanese numeric number to literal change tone. such as telephone number and calendar year.
-
MTL-Converter: A critical role in MTL-TTS's pipeline process for converting the
input sentence to surface tone based tokens upon the predicted tags in MTLTag.
It also takes KIP/POJ input for converting to MTL into the pipeline process.
-
MTL-tag_rules: We are tag rules which utilize POS and sandhi-rules to reduce
the last invalid 2% tags.
-
VITS Acoustic Synthesis — The "Soul" of the Voice.
-
MTL-Whisper: There have been many Whisper-based AI models for Taiwanese.
With the success of the above pipeline approach, we've trained
MTL-based ASR (4.07% CER) which fits better than
'harnji' based ASR(30% CER).
Currently, we have integrated the following two Taiwanese ASRs from Hugging Face into the app for comparison:
Ideally we would like to integrate MOE's audio input method
which achieved
more than 95% of accuracy in providing Taiwanese 'harnji'.
We also incorporated QWen2.5 to translate English into Taiwanese 'harnji'
for the pipeline process.
-
Demo may be found at TaiwaneseAI.org.
Why This Project Matters for the Next Generation
- From "Canned" to "Conversational": We aren't just making a translator; we are recreating the soul of the language.
- Tech-Forward Preservation: We are using RoBERTa and XLM (Cross-lingual Language Models) to help the AI "understand" Taiwanese at a semantic level, not just a phonetic one.
- Building a Bridge: This project connects the deep linguistic knowledge of the 1st generation with the engineering skills of the 2nd generation.
- Get Involved: We Need Your Skills!
We have the data, we have the pipeline, and we have a model that is already "impressing the neighbors." But there is more to build.
Are you a linguist or Python developer? Help us refine our Class-based converters for KIP/POJ.
Are you into ML/NLP? Help us optimize our RoBERTa/XLM tagging and loss functions (loss_mel, loss_kl, etc).
Are you a Heritage Speaker? Help us review and "slice" audio to
ensure the highest quality training data. Or better yet, help to design a model that can
review synthesized audio file for correctness.
"I can't do this
alone, but together we can fix it."
Let’s ensure the language of our parents and grandparents has
a seat at the table in the future of AI.
Synthesized Audio Samples
Note: The latest best model was trained from a pool of three females and
one male with 28,000 sentences.
Modern Taiwanese Language (MTL) |
Washington DC Taiwan School