Navigating the Maze
From Chaos to Control: The Art of Robotic Path Planning

Navigating the Maze

Welcome back to the Robotic Touch!

Successful robotic piece-picking involves bringing technology to bear that is capable of seeing items, moving items, grasping items, and placing items. In the previous two editions of this newsletter, we focused on computer vision, exploring technologies that enable robotic systems to perceive their surroundings and the challenges involved in applying them to solve real-world problems.

The next step after visualizing the workspace is to move the robotic arm and end effector into position for grasping. Humans effortlessly reach for objects, with our brains calculating the optimal, collision-free path almost instantly. Teaching robots to perform this same task is a challenging feat!

So frustrating!

Avoiding the Buzz

If you’ve ever played the game "Operation," you know how challenging it can be to remove small plastic pieces from tiny spaces. A slip triggers a buzz that can make anyone jump out of their skin! Robotic piece-picking shares similarities with this game, where precision and control are crucial in avoiding damage to the robot, the items being picked, or, in the worst-case scenario, nearby humans.

A complex production workspace presents many obstacles to avoid

Several variables play crucial roles in planning a robot’s path and avoiding collisions. The static components of the workspace, like equipment and structures, are relatively straightforward to map out using CAD models.

The greater challenge lies in the dynamic elements, which require real-time sensing and tracking for reliable collision avoidance. In piece-picking, the four most important dynamic variables are totes, the robotic arm, the end effector, and the items in the tote. Totes are constantly moved in and out of the workspace, sometimes constrained and other times not. The arm and end effector are also in constant motion, and their position must always be known. Sophisticated end-effectors like RightHand Robotics’ patented gripper system also need to understand the orientation of the fingers. Finally, the items in the tote must be accounted for to prevent damage. Tote fill levels and object orientation are random, adding to the complexity.

Unconstrained carton detection

Inverse Kinematics and Path Planning

How do we find θ1 and θ2 for some x and y?

Inverse kinematics involves taking the location of an object and determining the sequence of arm joint movements needed to reach it. Planning the robotic arm’s path requires accounting for static and dynamic elements in the workspace, ensuring collision-free execution. RightHand Robotics’ RightPick software uses sensors, computer vision, and artificial intelligence to quickly compute flawless motion plans. Each motion requires collecting numerous inputs and evaluating hundreds of plans within milliseconds. The system then selects the optimal plan using a ranking algorithm.

More From an Expert

Dr. Victora poses with the RightPick Robot

Senior Software Engineer Dr. Michelle Victora has a PhD in physics from the University of Illinois at Urbana-Champaign, and has been at RightHand Robotics since 2021. She has applied her analytical expertise in a variety of areas at RightHand, building out solutions for accurate grasping, multipick detection, and path planning. I asked Michelle to elaborate on the complexities of the topic.

“Our planner is able to take in a segmented depth cloud from our vision system and determine the most stable grasp, as well as the safest plan of approach to avoid obstacles. Inputs to our program include the object itself, as well as the position and height of the object relative to other objects, the quality of the object’s depth data, and where it is with respect to the walls of the tote. We estimate the physics of different objects based on what we know about weight and deformability, and use these inputs to control our movement speed and height, as well as how we hold the object so as to avoid dropping or damaging the item. We also must consider the characteristics of our robot, as cables can provide a potential ensnarement if we are not thoughtful about how we position ourselves.”

Approach angles can make all the difference

“Sometimes there are issues where items are in extremely hard-to-pick edges and corners of the tote, and we must be very intentional about how we approach and pick these items. For example, when picking a small flat surface in isolation, approaching an object at normal incidence will maximize our chances of a strong seal on the item, as our entire suction cup is guaranteed to be flush with the surface. However, when that item is right up against the edge of the tote wall, we don’t have the luxury of approaching the object straight-on. In order to achieve a successful pick without risking a collision, we need to model our system accurately and efficiently, capturing the appropriate amount of dynamicity involved in a moving arm and gripper. With proper modeling, we can adjust our approaches with minimal sacrifice to our pick success.” 

Dr. Victora’s expertise in path planning and motion control, combined with the innovative work of our many other smart team members, has helped optimize robotic navigation in complex environments. Leveraging advanced algorithms and sensor data, our system can adapt to dynamic workspaces, ensuring accuracy and efficiency in every pick. The result is a highly reliable solution that can navigate around obstacles with remarkable precision, setting a new standard in robotic piece-picking.

Up Next

In the next edition of The Robotic Touch, we’ll begin a multi-part discussion about grasping. We’ll explore important factors like target object shape, size, weight, and packaging. We’ll highlight the most challenging failure modes and reveal solutions to overcome them. We might even feature a current customer to discuss why piece-picking is a game changer for their business operations.

Venkatesh Kumar

Data transcends boundaries | Data Platforms, Advance Analytics & Generative AI

5mo

Perfect, would love to know more about this in detail in next part

Lance Cooper

Associate Head for Graduate Programs at University of Illinois at Urbana-Champaign

5mo

Great work, Michelle!

Paul Else

Talent Acquisition leader - DE&I Advocate - Start-up growth consultant

5mo

Great read as always Joshua Dion

To view or add a comment, sign in

More articles by Joshua Dion

  • Beyond Human Hands Part 1

    Beyond Human Hands Part 1

    Comprising 27 bones, 27 joints, 34 muscles, and more than 100 ligaments and tendons, the hand is capable of an…

    2 Comments
  • Machines That See - Part 2

    Machines That See - Part 2

    Welcome back to The Robotic Touch. Today’s discussion is the final installment of a two-part series exploring how…

    1 Comment
  • Machines That See - Part 1

    Machines That See - Part 1

    Welcome back, everyone! I'm thrilled to share with you our latest newsletter, part one of a two-part series, focused on…

    4 Comments
  • The Robotic Touch

    The Robotic Touch

    The human brain boasts astonishing power. Often, we overlook what our bodies and minds achieve, underestimating the…

    6 Comments
  • Our Culture and Values

    Our Culture and Values

    Company culture plays a significant role in how happy we are at work. Company values is one key ingredient in how the…

  • The Dos and Don’ts of Hiring – Part 2

    The Dos and Don’ts of Hiring – Part 2

    Hiring the right people is one of the most important responsibilities of a manager. Who we decide to bring into our…

    5 Comments
  • The Dos and Don’ts of Hiring – Part 1

    The Dos and Don’ts of Hiring – Part 1

    Hiring the right people is one of the most important responsibilities of a manager. Who we decide to bring into our…

    10 Comments
  • Four Ways To Improve Predictability (Tip #4)

    Four Ways To Improve Predictability (Tip #4)

    This is a continuation of a four-part series. Part #1: Narrow your planning horizon Part #2: Use empirical data Part…

  • Four Ways To Improve Predictability (Tip #3)

    Four Ways To Improve Predictability (Tip #3)

    This is a continuation of a four-part series. Part #1: Narrow your planning horizon Part #2: Use empirical data You are…

    1 Comment
  • Four Ways To Improve Predictability (Tip #2)

    Four Ways To Improve Predictability (Tip #2)

    This is a continuation of a four-part series. Part #1: Narrow your planning horizon You are here --> Part #2: Use…

Insights from the community

Others also viewed

Explore topics