Kabilan Kb’s Post

Bridging Language and Action with AI-Powered Robotics! I recently integrated the powerful LLaMA language model with a robot control system to bring language-based commands to life! 🦾✨ By combining LLaMA's natural language processing capabilities with a JSON configuration file, we're able to make the robot understand and execute commands like "move forward" or "turn left" with precision. The JSON file provides specific action parameters, allowing the robot to perform the task accurately. Chat history maintains context so that commands are relevant and conversational. 💡 This setup is more than just movement—it's about making the human-robot interaction feel natural and engaging. Imagine a robot that not only moves forward but also explains what that command means in real life! Exciting times in AI and robotics! 🚀🤖 #AI #Robotics #MachineLearning #NaturalLanguageProcessing #LLM #RobotControl #Innovation #Technology Ninad Madhab Jigar Halani Dustin Franklin Avinash Chakravarthi Tanna TechBiz LLP

Just wrapped up a session on my NVIDIA Isaac Kaya robot project with Jetson Orin Nano! Leveraging transformer models to give it a deeper understanding of the world. Exciting times for AI and robotics! #NVIDIAIsaac #JetsonOrinNano #AI #Robotics #TechInnovation Ninad Madhab Jigar Halani Dustin Franklin Avinash Chakravarthi Shreyans Dhankhar

Bringing the NVIDIA Kaya robot back to life with Jetson Orin and a Vision Transformer model! Now equipped with advanced generative AI, it's ready to interact and understand the world in a whole new way. #AI #Robot #NVIDIA #JetsonOrin #GenerativeAI #VisionTransformer NVIDIA Robotics Ninad Madhab Jigar Halani Dustin Franklin Avinash Chakravarthi Shreyans Dhankhar Tanna TechBiz LLP

Excited to share my latest blog post on integrating NVIDIA Isaac Sim with Jetson AGX Orin and the LLaMA language model! In this project, I implemented a natural language-based waypoint navigation system, allowing a robot to interpret and respond to verbal commands for autonomous movement. This fusion of simulation and AI is a step forward in advancing human-robot interaction. Dive into the blog to see how this technology is shaping the future of robotics! #AI #Robotics #IsaacSim #Jetson #NLP #ROS2 Ninad Madhab Jigar Halani Dustin Franklin Avinash Chakravarthi Tanna TechBiz LLP https://lnkd.in/gbfkHSGm

Save the date: October 9th in #Bangalore. This event offers a fantastic opportunity for students, faculty, and researchers to explore NVIDIA’s cutting-edge #AI and #robotics technologies. Immerse yourself in Generative AI and learn how to build intelligent robots with #Jetson, #IsaacSim, and #Metropolis Microservices. Event Details: 🗓 Date: 9th October 2024 📍 Venue: NVIDIA Bangalore Secure your spot by filling out the form here: 🔗 [Form]: https://lnkd.in/gsVS8zTZ

Carter can move by Audio to Action, where a robot listens to audio commands and takes action using a large language model (LLaMA 3) on a Jetson AGX Orin, with JSON and Nav2 for seamless navigation. The robot listens to instructions like "Move to the pallet area" or "Go to the charging station," processes them through the LLaMA model, and translates them into structured JSON data for execution. It also provides conversational feedback, such as, "I’m starting to navigate to the pallet area. Please wait!" Using the Nav2 stack, the robot autonomously moves to the specified location. Additionally, this system has been simulated in NVIDIA Isaac Sim, showcasing how AI-powered conversation and advanced navigation can enhance the interactivity of the Carter robot, making it more intuitive and responsive to users.NVIDIA Robotics Ninad Madhab Jigar Halani Dustin Franklin Avinash Chakravarthi Shreyans Dhankhar Tanna TechBiz LLP

Event Success: NVIDIA Robotics Day - Bangalore 🤖 I'm thrilled to announce that the "Build Your Gen-AI Robot in a Day with NVIDIA!" event on 27th September was a huge success! A event organizer, alongside our amazing event coordinator Ninad Madhab , we showcased the future of AI and robotics in an unforgettable way. Attendees got hands-on experience with cutting-edge NVIDIA technologies, including live demos of Generative AI models like LLMs, VLMs, and VLA models running directly on robots. It was an exciting deep dive into intelligent robot creation using the Jetson platform, Isaac Sim, and Metropolis Microservices. A massive thank you to all the developers, researchers, and robotics enthusiasts who joined us and made this day remarkable. The energy and enthusiasm in the room were incredible, and it's inspiring to see how we're pushing the boundaries of AI and robotics together. This event marks just the beginning of many more exciting innovations and collaborations. Stay tuned for what's next as we continue building the future of robotics. Jigar Halani Dustin Franklin Avinash Chakravarthi Tanna TechBiz LLP #NVIDIA #Robotics #GenerativeAI #LLM #VLM #VLAModel #Jetson #IsaacSim #MetropolisMicroservices #Innovation #AICommunity #BangaloreEvents #OrganizingSuccess #EventCoordinator #FutureOfRobotics

🚀 Exciting advancements in human-robot interaction! 🤖✨ We're leveraging a Vision-Language Model (VLM) to create an intelligent robot nvidia Carter that can seamlessly understand and respond to voice commands. Whether it’s monitoring warehouse activity or assisting with tasks, Carter processes audio and visual data in real-time, enhancing operational efficiency and safety. This technology opens new horizons for intelligent automation, making robots not just tools but collaborative partners in various industries. Curious to learn more about the methodology and potential applications? Check out the full article on Medium! #Robotics #AI #VLM #Automation #Innovation #MachineLearning Ninad Madhab Jigar Halani Dustin Franklin Avinash Chakravarthi NVIDIA Robotics Tanna TechBiz LLP

Carter, the robot, can see and understand its environment using a Vision-Language Model (VLM) deployed on Jetson AGX. Through cameras,NVIDIA Robotics Carter visually processes its surroundings and explains what's happening in real time. Using Isaac Sim for simulation. Users can easily communicate with Carter through a Web UI, either by typing or using voice commands like "Hey Carter, what's going on?". The VLM interprets the visual input and responds to these natural language queries, offering immediate situational insights. The system is deployed on Jetson AGX, enabling Carter to perform in real-world environments, improving warehouse automation and human-robot collaboration. This integration of AI, simulation, and robotics brings a natural and efficient way for humans to interact with robots. For more information, read the full article on Medium. https://lnkd.in/greMXi-g Ninad Madhab Jigar Halani Dustin Franklin Avinash Chakravarthi Mandar Rikame Tanna TechBiz LLP

Catch up with the Jetson AI Lab Research Group tomorrow! 🚀 I’ll be presenting the Isaac Sim2Real VLM workflow. Big thanks to Dustin Franklin and the amazing team! Join us on September 17th at 9 AM PST to dive into AI in robotics! 🤖