The First Traditional Chinese Inference Model: Formosa-1

Developed by APMIC and the Twinkle AI community, backed by the National Center for High-Performance Computing (NCHC), Formosa-1 is Taiwan's first 3B parameter large language model that can be run on mobiles. The model was trained via PrivAI, an NVIDIA NeMo framework-based distillation technology. Formosa-1 is MIT-licensed. Its introduction marks a significant milestone for Traditional Chinese open-source AI.

To enhance its inference capability, Formosa-1 was trained on data tailored to the chain of thought under the context of Taiwanese Mandarin. Moreover, training with NVIDIA NeMo Data Curator enables the model to understand and reason in legal situations, think logically and solve math problems. Formosa-1's training data covers 100B-scale high-quality Traditional Chinese Tokens across a variety of topics, including news, laws, essays, social discussions, and more; this ensures the model's accurate comprehension and application of the Traditional Chinese language context.

This is the first time Taiwan has a 3B parameter large language model that can run on mobile phones and comprehend the Traditional Chinese language context.

Twinkle Eval, an Evaluation Tool for Large Language Model Inference

A robust LLM depends on the evaluation output, but the traditional process handles questions one by one, which requires a long inference time and high computing power when many questions are involved. This limits both evaluation performance and resource efficiency. To effectively solve this problem, APMIC and Twinkle AI jointly developed Twinkle Eval, an open-source evaluation framework. Twinkle Eval is deeply integrated with the NVIDIA NeMo Evaluator model, supporting large-scale parallel testing to ensure model stability and accuracy across different knowledge domains.

Twinkle Eval is designed to make AI model testing fair and reliable by randomizing the order of answer choices, preventing the model from remembering a fixed pattern, it then repeatedly tests to check if the model gives consistent answers across multiple tries. The tool incorporates important benchmarks like TMMLU+, covering liberal studies and professional knowledge in Taiwan, tw-legal-benchmark-v1, and MMLU. This ensures a wide and accurate test scope. With Twinkle Eval, you can improve the model since the tool refines test formats and fixes common errors, resulting in consistent answers and reducing mistakes.

💡Did you know?

In the MMLU (Massive Multitask Language Understanding) challenge, Formosa-1 was evaluated by Twinkle Eval and scored 52 points, which is 46 points higher than the Llama-3.2-3B-Instruct, demonstrating its advantages for inference capability. This proves that Taiwan's AI technology has been on par with international standards and can provide efficient solutions for industrial applications.

APMIC's PrivAI, a Solution For Localized AI Solutions and Their Applications

APMIC helps businesses build their own AI solutions with its new tool: PrivAI, an enterprise-grade, headless AI training system that supports on-premises models ranging from 1B to 500B parameters, tailored to the needs of various industries. PrivAI uses different methods and approaches like pre-training, supervised fine-tuning, and distillation to help businesses deploy AI models that are both accurate and adaptable for different needs. Looking ahead to 2025, businesses that adopt AI solutions will be able to improve computing performance and reduce deployment costs through Formosa-1 and Twinkle Eval. APMIC and Twinkle AI will continue to deepen their cooperation to promote the development of the Traditional Chinese AI ecosystem.

For more information, please refer to:

• Formosa-1: https://huggingface.co/twinkle-ai/Llama-3.2-3B-F1-Instruct

• Twinkle Eval: https://github.com/ai-twinkle/Eval

  
  

About APMIC

Founded in Taiwan in 2017 and as one of NVIDIA's ecosystem partners, APMIC (Accelerate Private Machine Intelligence Company) is on its way to delivering enterprise AI solutions. Possessing expertise in Natural Language Understand, we have developed CaiGunn, an AI training platform, and PrivAI, a private AI training solution. APMIC's products are already in use by 1,100 enterprises and government departments across industries. APMIC is committed to facilitating businesses participating in the AI era.

Learn more about APMIC.