AI startup Writer is trying to make large language models more accessible in non-English-speaking countries.
Writer revealed on Wednesday that its family of LLMs, Palmyra, now supports text generation and translation in more than 30 languages, including Spanish, French, Hindi, Arabic and Russian.
With a total funding amount of $126 million, according to Crunchbase, Writer provides customers with a full-stack generative AI platform focused on helping businesses use LLMs to generate written content.
Palmyra LLMs are trained with business content and marketing data. The small and base models are open source and available for free on Hugging Face.
Need for multilingual LLMs
Writer's introduction of its multilingual LLM capabilities speaks to the need that enterprises in the non-Western world have for more AI models that support languages other than English and Spanish. Other large tech vendors also recently introduced multilingual models.
Last August, Meta introduced SeamlessM4T, a multilingual AI translation model that supports speech-to-text for about 100 languages. Plus, AI startup Krutrim introduced India's first multilingual LLM in December.
Multilingual translation is a huge use case for a ton of companies.
Rowan CurranAnalyst, Forrester Research
"Multilingual translation is a huge use case for a ton of companies," Forrester Research analyst Rowan Curran said. He added that this is what began a trend Forrester noticed years ago where employees dumped company data into Google Translate.
"There's certainly an ongoing and potentially increasing need for multilingual translation capabilities at enterprises, particularly ones that are operating in global environments," Curran said.
While general-purpose large language models have tried to support enterprise needs, especially with the launch of GPT-4, they've also been costly for enterprises, Gartner analyst Arun Chandrasekaran said.
"If I post a question in English, the cost of answering the question is going to be 10 cents," Chandrasekaran said. "The cost of answering the same question in my mother tongue is often two and a half, three times the cost."
That's where vendors with multilingual LLMs step in, he continued.
"They're trying to make generative AI products more widely available outside developed countries, particularly in Western Europe, Asia and Latin America," Chandrasekaran said.
Making multilingual models accurate
However, a challenge for a vendor such as Writer is to make sure its multilingual capabilities are accurate, Chandrasekaran said.
"One of the challenges with the other languages is there are so many dialects in these languages that are the informal and formal usage," he said. "Some level of human feedback into the model training process, I think, could be very helpful."
It also helps to train the models with not just written text, but also speech, he added. This is because written text can often be outdated from spoken language.
LLM creators such as Writer could also benefit from direct feedback from enterprise customers; the feedback loop would be used to improve the model.
Writer provides multilingual support through a chat interface called Ask Writer, both on desktops and in custom applications.
Esther Ajao is a TechTarget Editorial news writer covering artificial intelligence software and systems.
