Llama Guard is Off Duty 😲
We benchmarked popular toxicity datasets spanning languages like Portuguese, Ukrainian, and Turkish, and found that Llama Guard has a very high false negative rate for toxic content! We found that base models like Llama 3.1 do all the heavy lifting on toxicity filtering, and that the joint usage of Llama Guard might be redundant. 🤔
At Patronus AI, we rigorously benchmark all things AI to help engineers trust what they use. Reach out to contact@patronus.ai to learn more! Llama Guard might be off duty today, but you don't have to be 🎯
—
Read more in our blog post here: https://lnkd.in/eayCX4ct