Grocery Store Robot: Arm Manipulation

Summary

Modern grocery stores and warehouses still rely heavily on manual labor to fulfill item retrieval tasks from shelves. This process is time-consuming, labor-intensive, and prone to inefficiencies as order volumes continue to grow. There is an increasing demand for automated systems that can improve operational efficiency while maintaining safety and precision in retail and warehouse environments.

This project focuses on the development of an autonomous robotic arm capable of retrieving items from shelves in a grocery store setting. The system aims to perform precise, collision-free grasping of target items while operating in a constrained shelf environment. By automating repetitive picking tasks, the project seeks to improve efficiency and reduce the reliance on manual labor in grocery fulfillment operations.

The project is sponsored by Professor Solmaz Kia and developed by a student team in the MAE capstone design program. The primary stakeholders include grocery store operators, warehouse automation systems, and future robotics developers who may build upon this platform. The outcome of this project will contribute to research and development in robotic manipulation and autonomous retail automation.

Technical Approach/Methodology

Our project develops a robotic manipulation system capable of autonomously retrieving items from shelves in a grocery store environment. The system integrates robotic arm control, perception, and motion planning to achieve safe and precise item grasping.

A depth camera is used to capture 3D information of the shelf environment and the target objects. The system applies the Grasp Pose Detection (GPD) algorithm to analyze the point cloud data and generate feasible grasp candidates for the robotic gripper. This approach enables the robot to identify stable grasp poses even in cluttered shelf environments.

The robotic arm is modeled and simulated using ROS 2 and the Gazebo simulation environment. Forward and inverse kinematics are used to control the arm’s motion and compute valid grasp configurations. Motion planning algorithms ensure collision-free trajectories so the arm can safely interact with shelves, objects, and the robot base.

Outcomes

By the end of the project, our team successfully developed a robotic arm manipulation system capable of autonomously detecting and grasping objects from grocery store shelves. The system integrates depth-based perception, grasp pose detection, and robotic motion planning to perform reliable item retrieval.

A depth camera is used to capture 3D information of the environment, and the Grasp Pose Detection (GPD) algorithm is applied to generate feasible grasp candidates. Through a graphical user interface (UI), the user can select a target item, after which the robotic arm automatically plans and executes the grasping motion.

The system was implemented using ROS 2 and tested within a Gazebo simulation environment. Motion planning and kinematic control enable the arm to execute collision-free trajectories while interacting with shelf structures and surrounding objects.

The final deliverables include the robotic arm control software, the perception and grasp detection pipeline, the simulation environment, and system integration with the user interface. The completed system demonstrates coordinated perception, planning, and manipulation for autonomous item retrieval in a simulated grocery store environment.

Project Media

Project Poster