Student Thesis / Bachelor's Thesis / Master's Thesis: LLM-Controlled Robot Manipulation

Fraunhofer-Gesellschaft · Augsburg

ThesisGerman needed: not said

Still open on 1 Nov

You apply on the company's own website.

What you will do

Building a modular demonstrator in which a large language model (LLM) orchestrates computer vision functions and robot skills for pick-and-place and bin-picking tasks via tool calling (smolagents/HuggingFace)
Integrating and benchmarking perception modules, such as FoundationPose, and grasp-planning methods for camera-based grasp-point determination
Connecting all components (UR5e, Intel RealSense, Jetson Orin, cloud) via ROS2 Jazzy with a containerized deployment strategy (Docker)
Systematically comparing cloud-based and edge-based LLM backends in terms of latency, task success rate, and planning correctness, and scientifically documenting the results

Building a modular demonstrator in which a large language model (LLM) orchestrates computer vision functions and robot skills for pick-and-place and bin-picking tasks via tool calling (smolagents/HuggingFace)
Integrating and benchmarking perception modules, such as FoundationPose, and grasp-planning methods for camera-based grasp-point determination
Connecting all components (UR5e, Intel RealSense, Jetson Orin, cloud) via ROS2 Jazzy with a containerized deployment strategy (Docker)
Systematically comparing cloud-based and edge-based LLM backends in terms of latency, task success rate, and planning correctness, and scientifically documenting the results