Navigating the Maze

Welcome back to the Robotic Touch!

Successful robotic piece-picking involves bringing technology to bear that is capable of seeing items, moving items, grasping items, and placing items. In the previous two editions of this newsletter, we focused on computer vision, exploring technologies that enable robotic systems to perceive their surroundings and the challenges involved in applying them to solve real-world problems.

The next step after visualizing the workspace is to move the robotic arm and end effector into position for grasping. Humans effortlessly reach for objects, with our brains calculating the optimal, collision-free path almost instantly. Teaching robots to perform this same task is a challenging feat!

Avoiding the Buzz

If you’ve ever played the game "Operation," you know how challenging it can be to remove small plastic pieces from tiny spaces. A slip triggers a buzz that can make anyone jump out of their skin! Robotic piece-picking shares similarities with this game, where precision and control are crucial in avoiding damage to the robot, the items being picked, or, in the worst-case scenario, nearby humans.

Several variables play crucial roles in planning a robot’s path and avoiding collisions. The static components of the workspace, like equipment and structures, are relatively straightforward to map out using CAD models.

The greater challenge lies in the dynamic elements, which require real-time sensing and tracking for reliable collision avoidance. In piece-picking, the four most important dynamic variables are totes, the robotic arm, the end effector, and the items in the tote. Totes are constantly moved in and out of the workspace, sometimes constrained and other times not. The arm and end effector are also in constant motion, and their position must always be known. Sophisticated end-effectors like RightHand Robotics’ patented gripper system also need to understand the orientation of the fingers. Finally, the items in the tote must be accounted for to prevent damage. Tote fill levels and object orientation are random, adding to the complexity.

Inverse Kinematics and Path Planning

Inverse kinematics involves taking the location of an object and determining the sequence of arm joint movements needed to reach it. Planning the robotic arm’s path requires accounting for static and dynamic elements in the workspace, ensuring collision-free execution. RightHand Robotics’ RightPick software uses sensors, computer vision, and artificial intelligence to quickly compute flawless motion plans. Each motion requires collecting numerous inputs and evaluating hundreds of plans within milliseconds. The system then selects the optimal plan using a ranking algorithm.

More From an Expert

Senior Software Engineer Dr. Michelle Victora has a PhD in physics from the University of Illinois at Urbana-Champaign, and has been at RightHand Robotics since 2021. She has applied her analytical expertise in a variety of areas at RightHand, building out solutions for accurate grasping, multipick detection, and path planning. I asked Michelle to elaborate on the complexities of the topic.

“Our planner is able to take in a segmented depth cloud from our vision system and determine the most stable grasp, as well as the safest plan of approach to avoid obstacles. Inputs to our program include the object itself, as well as the position and height of the object relative to other objects, the quality of the object’s depth data, and where it is with respect to the walls of the tote. We estimate the physics of different objects based on what we know about weight and deformability, and use these inputs to control our movement speed and height, as well as how we hold the object so as to avoid dropping or damaging the item. We also must consider the characteristics of our robot, as cables can provide a potential ensnarement if we are not thoughtful about how we position ourselves.”

“Sometimes there are issues where items are in extremely hard-to-pick edges and corners of the tote, and we must be very intentional about how we approach and pick these items. For example, when picking a small flat surface in isolation, approaching an object at normal incidence will maximize our chances of a strong seal on the item, as our entire suction cup is guaranteed to be flush with the surface. However, when that item is right up against the edge of the tote wall, we don’t have the luxury of approaching the object straight-on. In order to achieve a successful pick without risking a collision, we need to model our system accurately and efficiently, capturing the appropriate amount of dynamicity involved in a moving arm and gripper. With proper modeling, we can adjust our approaches with minimal sacrifice to our pick success.” 

Dr. Victora’s expertise in path planning and motion control, combined with the innovative work of our many other smart team members, has helped optimize robotic navigation in complex environments. Leveraging advanced algorithms and sensor data, our system can adapt to dynamic workspaces, ensuring accuracy and efficiency in every pick. The result is a highly reliable solution that can navigate around obstacles with remarkable precision, setting a new standard in robotic piece-picking.

Up Next

In the next edition of The Robotic Touch, we’ll begin a multi-part discussion about grasping. We’ll explore important factors like target object shape, size, weight, and packaging. We’ll highlight the most challenging failure modes and reveal solutions to overcome them. We might even feature a current customer to discuss why piece-picking is a game changer for their business operations.