Knowledgator

🚀 Let’s transform LLMs into encoders 🚀 Auto-regressive LMs have ruled, but encoder-based architectures like GLiNER are proving to be just as powerful for information extraction while offering better efficiency and interpretability. 🔍✨ Past encoder backbones were limited by small pre-training datasets and old techniques, but with innovations like LLM2Vec, we've transformed decoders into high-performing encoders! 🔄💡 What’s New? 🔹Converted Llama & Qwen decoders to advanced encoders 🔹Improved GLiNER architecture to be able to work with rotary positional encoding 🔹New GLiNER (zero-shot NER) & GLiClass (zero-shot classification) models 🔥 Check it out: New models: https://lnkd.in/eu7HPNGG GLiNER package: https://lnkd.in/eBYG-64S GLiClass package: https://lnkd.in/eqVQyUDh 💻 Read our blog for more insights, and stay tuned for what’s next! https://lnkd.in/eJ32TJab Many thanks to all open-source contributors who made these developments possible: Parishad BehnamGhader Vaibhav Adlakha Urchade Zaratiana Tom Aarsen Thanks to many more, who work in this domain! #NLP #AI #MachineLearning #DeepLearning #GLiNER #GLiClass

Many people asked about the technical nuances of our bi-encoder #GLiNER architecture. If you want to explore the intrinsic details of this work or you just seeking efficient fine-tuning tips here is a blog post for you: https://lnkd.in/eqhhuNsP

🚀 Meet the new GLiNER architecture 🚀 GLiNER revolutionized zero-shot #NER by demonstrating that lightweight encoders can achieve excellent results.  We're excited to continue R&D with this spirit 🔥. Our new bi-encoder and poly-encoder architectures were developed to address the main limitations of the original GLiNER architecture and bring the following new possibilities: 🔹 An unlimited number of entities can be recognized at once. 🔹Faster inference when entity embeddings are preprocessed. 🔹Better generalization to unseen entities. While the bi-encoder architecture can lack inter-label understanding, we developed a poly-encoder architecture with post-fusion. It achieves the same or even better results on many benchmarking datasets compared to the original GLiNER, while still offering the listed advantages of bi-encoders. Now, it’s possible to run GLiNER with hundreds of entities much faster and more reliably. 📌 Try the new models here: https://lnkd.in/eHJGjSEK

🚀 Missed our performance analysis of the #GLiClass model line in #fewshot settings? Catch up here! 🎉 We rigorously tested our models on the most complex and diverse datasets typically used for zero-shot classification, covering areas like sentiment analysis, spam detection, and topic-based classification with up to 77 classes. 📊 To fine-tune, we provided just 8 examples per label. The results? While our models matched the zero-shot performance of embedding models they significantly outperformed them in few-shot scenarios, especially with larger models. 🔍 GLiClass stands out for its computational efficiency during both training and inference, surpassing cross-encoder models. 💡 Check models here: 🔗 Explore our models: https://lnkd.in/enqHmKGm 🖥️ Try demo: https://lnkd.in/eDUJczGF 📚 Official repository with training script: https://lnkd.in/eqVQyUDh

🚀 Meet Our New Line of Efficient and Accurate Zero-Shot and Few-Shot Classifiers! 🚀 We were inspired by #GLiNER architecture and now it’s time to introduce our new contribution to this architecture opening new possibilities in zero-shot sequence classification. It fixes cross-encoder drawbacks such as lack of efficiency due to the need to process all pairs of texts and labels. Plus our architecture brings better inter-label understanding that both cross-encoders and embedding-base approaches lack. As a result, we get accurate models as cross-encoders and efficient ones as embedding-based ones. Key Applications: ✅ Multi-class classification (up to 100 classes in a single run)  ✅ Topic classification  ✅ Sentiment analysis  ✅ Event classification  ✅ Prompt-based constrained classification  ✅ Natural Language Inference  ✅ Multi- and single-label classification Our models excel in few-shot classification, achieving notable performance with as few as 8 examples per label. We’ve observed a more than 20% increase in F1 score on complex datasets! 🎉 We're excited about our new architectural contribution that enables context-dependent inter-layer feature selection, enhancing generalization capabilities. This innovation results in an average increase of 3.5% in F1 scores across tested datasets and models. 📈 Check models here:  🔗 Explore our models: https://lnkd.in/enqHmKGm 🖥️ Try demo: https://lnkd.in/eDUJczGF 📚 Official repository: https://lnkd.in/eqVQyUDh #NLP #MachineLearning #TextClassification #GLiClass #ZeroShot #FewShot

✨ Small but exciting news for the #GLiNER community ✨ Many people asked us how to fine-tune GLiNER models. So we decided to prepare an end-to-end notebook. It harmonically combines Gradio interfaces with code parts. Here’s what you can expect: 📋 Auto-Annotation: Save time and effort with automatic labeling! 🏷️ 🔍 Manual Checking: Ensure accuracy with easy manual verification. ✅ 🧠 Fine-Tuning: Tailor the model to your specific needs with a few tweaks. 🛠️ 📈 Evaluation: Assess the performance with robust metrics and diagnostics. 📊 💡 Inference: Apply your finely tuned model to real-world data effortlessly! 🌐 Link: https://lnkd.in/e_22ifhd Video-tutorial: https://lnkd.in/eFKch6Gh Official repo: https://lnkd.in/e6jFYg-8

We’re thrilled to share our latest technical paper on the multi-task GLiNER model, showcasing its impressive performance, efficiency, and controllability compared to traditional LLMs. Our research dives into the following exciting and forward-thinking topics: 🔍 Zero-shot NER & Information Extraction: We demonstrate that with diverse and ample data, paired with the right architecture, encoders can achieve impressive results across various extraction tasks, such as NER, relation extraction, summarization, etc. 🛠️ Synthetic Data Generation: Leveraging open labelling by LLMs like Llama, we generated high-quality training data. Our student model even outperformed the teacher model, highlighting the potential of this approach. 🤖 Self-Learning: Our model showed consistent improvements in performance without labelled data, achieving up to a 12% increase in F1 score for initially challenging topics. This ability to learn and improve autonomously is a very perspective direction of future research! A huge thanks to Urchade Zaratiana for his support throughout this research 🌟 📄 Read the paper: https://lnkd.in/gg6gdc7N ⚙️Try the model: https://lnkd.in/eGZRN-wb 💻 Try the demo: https://lnkd.in/ebfaHu7k 📌Explore the repo: https://lnkd.in/e6jFYg-8

🚀 Big News from the GLiNER Ecosystem! 🚀 We’re thrilled to announce our significant contributions to the #GLiNER infrastructure that are set to elevate your experience and open new possibilities for large-scale information extraction projects! Code: https://lnkd.in/e6jFYg-8 Here’s what’s new: ✨ Performance Boost: Achieve up to 2x faster inference and up to 3x faster, more memory-efficient training! Our latest updates ensure smoother and quicker model execution. 🤖 Enhanced Hugging Face Integration: Benefit from better integration with Hugging Face Transformers, offering a wider range of training parameters and advanced features like comprehensive reporting. 🔄 ONNX Conversion Support: Now, it's much easier to bring GLiNER to new platforms with support for ONNX conversion, broadening the horizons for model deployment. 📦 Reduced Dependencies: We’ve minimized dependencies on external packages and models, making GLiNER leaner and easier to work with. 💡 Introducing a New Line of GLiNER Models: The latest small model matches the performance of the previous large version, ensuring top-tier efficiency without the bulk. Link: https://lnkd.in/e9M8gc_A

We are thrilled to see that our ⚡ Space at HuggingFace with GLiNER multi-task (https://lnkd.in/ebfaHu7k) was visited more than 3k times and people created their own demo applications based on it. Huu Nguyen has showcased the capabilities of our model in NER tasks (https://lnkd.in/e3cw7wHU) and Information Extraction (https://lnkd.in/e4NaAYgD).

We are thrilled to see that our ⚡ Space at HuggingFace with GLiNER multi-task (https://lnkd.in/ebfaHu7k) was visited more than 3k times and people created their own demo applications based on it. Huu Nguyen has showcased the capabilities of our model in NER tasks (https://lnkd.in/e3cw7wHU) and Information Extraction (https://lnkd.in/e4NaAYgD).