🚀 We release Kan-LLaMA [ಕನ್-LLama] — A 7B Llama-2 model, LoRA PreTrained and FineTuned on "Kannada" tokens🚀 One of the most powerful OSS LLMs can now speak Kannada! Problem? 🔴 One of the most sought out OSS LLMs — Meta's Llama-2 suffers a severe flaw. It was only trained on English tokens ! 🟡 This makes it inherently bad at generating any other language apart from english. To fix this, we've develop and release A LoRA pre-trained and fine-tuned version of Llama-2 to expand its capabilities to Kannada. 🚀🚀 We expand Llama-2's existing linguistic capabilities for Low Resource Indic languages and specifically Kannada by fine tuning on 600 Million Kannada tokens and subsequently fine-tune on SOTA Instruction Datasets. Read the blog & test out the models today! Paper and code dropping soon! Blog: https://lnkd.in/giUnpWhJ Models and Datasets: https://lnkd.in/gp_Xu-kb Contributors: Adarsh Shirawalmath, Adithya Kamath, Bharat Shetty Barkur & Raghav Ravishankar (alphabetical) #opensource #llms #kannada #multilingual #llama2
Awesome! Looking forward to get my hands on this one.
Very surprised and excited to see this. Great job, will play with your model and go over your blog.
Congratulations Team. Your effort in fine tuning the Llama2 model for indic language is truly remarkable. It has intrigued me more due to the Kannada language and I'm excited to make an inference from your custom model
Looks interesting. Surely will check it out.
#tesonic 💪🙏🫶🏻
Incredible! 👏👏Can't wait to try it out.
Kudos to Adarsh Shirawalmath, Adithya Kamath, Bharat Shetty Barkur & Raghav Ravishankar
Great work team!
Thank you
Machine Learning Scientist | past@Mila | IIT Bombay | AU
9moCongratulations! Apparently, for a short period, the CulturaX ds was unavailable and I ended up discovering your Kannada dataset so was anticipating some Kannada Llama model release. Also a nice selection of images for concept art, can recognize Jain manuscripts in the bottom row.