Department of Computer Science, University of Copenhagen - DIKU’s Post

We’re gathering one of the most unique native language datasets ever, and we’ve invited you to partake . http://www.langeasy.ai/ Read this article, to learn more on how we aim to build the largest data set of native Nigerian languages, to fuel the development of AI models and applications for Nigeria. http://disq.us/t/4o8nqkz

📝 Introducing our latest report – “AI speaks Polish. The ecosystem of open language models in Poland”! It presents research and conclusions of Alek Tarkowski (Open Future Foundation), Kuba Piwowar (Fundacja Centrum Cyfrowe), and Michał Owczarek (Uniwersytet SWPS). The goal of the report is to provide a case study of Poland’s ecosystem for creating open AI models for the Polish language 🇵🇱 Small language models are filling the gap left by large commercial models, which are not adapted to the Polish language or cultural nuances. The work on these models serves as an example of effectively creating alternatives to dominant entities. The report focuses on two key projects: building the SpeakLeash | Spichlerz language corpus and using it to create the Bielik model, as well as the activities of the #PLLuM consortium (Polish Large Language Model). Based on interviews with the creators of Polish models, the authors outline the development processes and the challenges they presented, and summarise the lessons learned from the achievements so far. 👉 See the full report on our website – in Polish now, and in English next week! 🔗 https://lnkd.in/d7JTr4G8 __ Image: Portrait of Adam Mickiewicz, Austrian National Library [Public Domain, via Europeana.eu]

I recommend reading the report on the development of genAI in Poland. I was involved in writing it with Kuba Piwowar and Alek Tarkowski, we talked to leaders and experts in the field. My favorite conclusion is why it is worthwhile to build Polish LLMs: because the process creates know-how in the industry and allows institutions to store and process their data locally.

There's a lot going on today - so I'll throw in this small AI tidbit that advances the development of open AI language models in Europe. That's right. International researchers have compiled MOSEL - Massive Open-source compliant Speech data for European Languages. ✴ A comprehensive open-source (Evan's favorite two words) speech data collection for the 24 official EU languages. ✅ The collected data comes from 18 different sources, including projects like CommonVoice, LibriSpeech, and VoxPopuli. ✅ It includes both transcribed speech recordings and unlabeled audio data (particularly valuable are the 505,000 hours of transcribed data from 18 different sources.) ✅ The problem -- the distribution among languages is VERY uneven. Example--Over 437,000 hours of labeled data are available for English, BUT languages like Maltese or Irish have only a few hours. 🤔 How do you improve the situation for "resource-poor languages"? ➡ Try to transcribe an additional 441,000 hours of previously unlabeled audio data (they used OpenAI's Whisper AI model for this purpose). ➡ Transcribing Maltese is a challenge! The Whisper model achieved a word error rate over 80% 😲 👉 four out of five words were incorrectly recognized! Do you want the entire data collection? Here it is 👉 GitHub 👉 https://lnkd.in/eCHwyUzH You're welcome. 😊

🌐 AI technologies should benefit all European languages, yet many are underrepresented in large AI models, causing significant language inequality.    By winning the Large AI Grand Challenge, we aim to advance 𝗹𝗶𝗻𝗴𝘂𝗶𝘀𝘁𝗶𝗰 𝗶𝗻𝗰𝗹𝘂𝘀𝗶𝘃𝗶𝘁𝘆 in Europe and develop a foundational Large Language Model for less widely spoken #EU languages.    We commit to following open science principles and ethical data handling, making resources freely accessible to the research community and beyond ➡️ https://lnkd.in/d8hTPieJ #LLM #AI #LargeAIGrandChallenge #FoundationalLLMs European Commission EuroHPC Joint Undertaking (EuroHPC JU) AI BOOST

🌍 How can technology truly serve local communities?, The Distributed AI Research Institute (DAIR) believes that tools created by and for communities are far more impactful than those imposed from outside. Their latest blog shares examples, like how Māori researchers ensure their language is used to benefit their own people first. Read our post reflecting on these ideas and join the conversation on how DAIR is rethinking AI to prioritize people, culture, and community needs. https://lnkd.in/d3bMhkAR

Government communication can be complex, and accessibility is key. In this blog series, Florian Kunneman and Joleen van der Zwan van der explain how AI and language technology may help simplify communication for Dutch citizens. Learn about the collaboration in research between RVO and Utrecht University to make government letters more accessible using Large Language Models (LLMs). #AI #GovernmentCommunication #LLM

Right on point … IMHO. “In this paper we argue that key, often sensational and misleading, claims regarding linguistic capabilities of Large Language Models (#LLMs) are based on at least two unfounded assumptions: the assumption of language completeness and the assumption of data completeness. Language completeness assumes that a distinct and complete thing such as “a natural language” exists, the essential characteristics of which can be effectively and comprehensively modelled by an #LLM. The assumption of data completeness relies on the belief that a language can be quantified and wholly captured by data. “ #AI https://lnkd.in/dBDMajcq

As Dutch speakers, it’s unfortunate to see our language often overlooked in the development of open Large Language Models (LLMs).    To change this, Matthieu and I built ChocoLlama, a family of 6 Dutch LLMs based on Meta’s Llama. We’re excited to release all models on Hugging Face (https://lnkd.in/ePSzc6Yy), along with our paper (https://lnkd.in/espCMSfu).   Technical TL;DR: - We train Llama-2 on all Dutch data we could find (32B tokens). We find that LoRA scales well for language adaptation and modifying the tokenizer boosts performance.  - In the meantime, the new Llama-3 turned out to be better in Dutch than our Llama-2 adaptations– showing how hard it is to keep up in this field! - Repeating our method on Llama-3 yields limited returns, suggesting that more multilingual models may benefit more from posttraining than pretraining.    Big thanks to our co-authors (François REMY, Pieter Delobelle, Jens-Joris Decorte, Thomas Demeester) advisors (Els Lefever, Veronique Hoste, @Cédric Goemaere, Karel D'Oosterlinck, @Loic De Langhe, Pranaydeep Singh and Bram Vanroy) and industry partners who helped us expand our Dutch data (TechWolf, Jeroen Van Hautte 🐺, Bizzy, Hendrik Keeris and ML6 | Your partner in AI, Michiel De Koninck). And a special thanks to the VSC | Vlaams Supercomputer Centrum for extending us the grant making this project possible. It's amazing to see initiatives like this being supported in Flanders.

I've often taken for granted how the internet has been primarily designed and developed with a preference for English language speakers. However, this bias and the recent push for Artificial Intelligence (AI) and Large Language Models (LLMs) have the potential to destroy entire cultural identities surrounding language. The Gradient recently published an insightful article that delves into this issue in depth. One striking example mentioned is how an AI model mistook a language from Benin for something fictional, highlighting the alarming lack of representation and understanding of lesser-known languages within these systems. We must recognize this issue now and take proactive measures to preserve and promote linguistic diversity in the digital age. By raising awareness and advocating for inclusive language practices in technology, we can work towards creating a digital landscape that celebrates the culture and diversity of human languages. It is our responsibility to ensure that no language is left behind in the era of AI and digital transformation. https://lnkd.in/ebgK5MH4