ByteDance pushes ahead with 10-trillion-parameter AI model
Development of large-scale model
ByteDance, China's internet giant, has begun developing an AI model on a scale comparable to Claude Mythos 5 from US AI startup Anthropic, the Financial Times reported on the 7th, citing people familiar with the matter.
The model under development is expected to have as many as 10 trillion parameters, a measure of AI scale. That would be three times the size of Kimi K3, the latest model from Chinese startup Moonshot AI.
Long-term development ordered internally
Anthropic has not disclosed the number of parameters, but Mythos 5 is said to have about 8 trillion, while Claude Fable 5 is said to have about 5 trillion. ByteDance's model is currently in the pre-training stage, a process that usually takes three to six months.
Chinese media outlet The Paper reported on the 6th that ByteDance founder Zhang Yiming told the company's AI development team in an internal meeting to pursue development with a long-term perspective. It also reported that he had banned a method known as distillation, which uses output data, the responses of existing AI systems, to efficiently develop another AI.
Enjoyed this article? Share it with your network!