L-6

From Perception to Action: AI in Robotics

L5 Exit Ticket Results

DEMO 1 · L5 Exit Ticket Results

Learning Through Trial and Error

Learning Model Estimates the value of actions Simulation A virtual environment for trials Possible Actions Rotate either joint in either direction Reward Function +6−4 Scores the outcome of an action Our Crawler: Table ΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣ Deep Reinforcement Learning A deep neural networklearns from reward

The Perception–Action Loop

Interactive

How Does a Robot Sense?

Sense DigitalRepresentation Instruction Action Physical World Sensor AI Actuator

Beyond Human Vision

NearFar NearFar RadioMicrowavesInfraredVisibleUltravioletX-raysGamma rays Electromagnetic Spectrum
Interactive

Robot Sensors

Proprioceptive sensors measure the body, while exteroceptive sensors measure the environment

Interactive

Sensor Fusion

Camera＋ LiDAR＋ Radar
Road SceneEstimated World Initial Uncertainty Possible Path Estimated Position Darkness
Camera: uncertain position and motion

Predicting Possible Futures

WaitProceed How Does a Robot Car Learn to Act?

How Does a Robot Learn to Act?

Sense DigitalRepresentation Instruction Action Physical World Sensor AI Actuator

Two Common Ways to Learn Actions

Imitation LearningReinforcement Learning Examples show which actions to take Consequences guide action values The two approaches can be combined

Which Action Is Better: A or B?

Action AAction B

Was Your First Choice Best?

Action Sequence AAction Sequence B

Learn From the Whole Outcome

Action AAction B Previous Sequence Total: +26Previous Sequence Total: +32

Learning to Move in Simulation

How Does a Robot Act?

Sense DigitalRepresentation Instruction Action Physical World Sensor AI Actuator

From Simulation to the Physical World

SimulationPhysical World Slipping Sensor NoiseSensor Reading Slow MotorExpected Arm Position Train under varied conditions. Test on the real robot.

From Images and Instructions to Actions

Image and Instruction Deep Neural Network Physical Action “Place the cupon its saucer” ΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣ VLA ModelVision–Language–Action Updated Camera Image Images + language → robot actions Demonstrations → imitation learningReward feedback → reinforcement learning

Putting It All Together

Sense DigitalRepresentation Instruction Action Physical World Sensor AI Actuator When Is Human Oversight Needed?

Human Oversight

Sense DigitalRepresentation Instruction Action Physical World Sensor AI Actuator Set Limits Monitor Intervene What is the robot allowed to do? How will someone notice a problem? How can someone stop or correct it? If the robot causes harm, who is responsible?

Who Is Responsible for Driving?

What must the person still do? Who does what? PersonCar Hover over a level to explore Before relying on automation, check its limits, your role, and the legal requirements
Interactive

Robotics Through the Years

PR1 (2008) · Remote-Controlled

PR2 (2010) · Folding Towels

Atlas (2016) · Keeping Its Balance

Unitree (2026) · Jumping and Sprinting

Which task in your field could robots change?

Activity

Please reach out to a TA/STA if you encounter any technical problems before leaving.