Head-Motion & Blink Robot Control

UR5e robot arm and gripper at the table with the optical-flow backdrop — **Fig.** The UR5e arm and gripper controlled by the wearable.

6-DOF

UR5e arm + gripper

30 fps

head-camera tracking

input signals: flow + EMG

Motivation

Duchenne muscular dystrophy (DMD) is one of the most common genetic conditions, affecting about 1 in 3,300 male births. It weakens every voluntary muscle over time, but even in severe cases people usually keep small head movements and facial expressions. Life expectancy for people with DMD keeps rising, yet most wearables are designed for rehabilitation rather than everyday tasks. Eye-signal (EOG) control systems also tend to be bulky, expensive and hard to use.

This project combines two inexpensive signals, optical flow from a head-mounted camera and EMG from the eyelid, to control a full robotic arm-gripper system.

Wearable design

The goal was a device that stays out of the way and is easy to learn. I explored headset, headband and glasses form factors. The proposed design is a lightweight 3D-printed PLA piece that sits over one eye and rests on the ear. It holds three electrode pads, a camera, a battery, a microcontroller and a Bluetooth module. The study prototype used a headband, a USB camera and sensor cables.

Headband prototype with USB camera, sensor cables and electrodes — **Fig.** Study prototype: headband, USB camera, sensor cables and electrodes.

Wearable rendered on a head model — Wearable on a head model

Electrode pads on the wearable — Inner face with electrode pads

User interface

User wearing the headband prototype with electrodes, with the EMG signal on a laptop — **Fig.** Headband prototype in use, with a blink spike in the live EMG signal.

The arm moves on a plane 10 cm above the table. When the user looks left, right, up or down, the gripper moves the same way, and the farther they turn their head, the faster it goes. A deliberate strong blink, a tight voluntary squeeze that the system can tell apart from a normal blink, starts a pick-and-place sequence. The arm drops, grips, lifts, moves the object to a spot in front of the user, releases it and returns home.

Look right, left, up, down velocity control — **Fig.** Head direction maps to arm velocity.

Eyes open versus blink control — **Fig.** Eyes open = steer; strong blink = grasp.

System architecture

Two data streams feed the system. EMG comes from electrodes around the eye through an Arduino, and video comes from the head-mounted camera. Processing runs across Arduino C++, Python ROS nodes and a Python visualizer.

ROS graph

/eog_node publishes blink booleans.
/optical_flow_node publishes the camera's x/y displacement.
/controller_node runs motion planning and publishes end-effector targets and gripper state.
/ur5_arm_node solves inverse kinematics, and /gripper_node actuates the gripper.

ROS node graph — **Fig.** ROS nodes and topics.

Motion-planning state machine

A double blink sets the reference frame. After that, the flow node tracks features and streams total displacement for velocity control. A blink, or reaching the edge of the table, stops the arm. The controller then sends a timed sequence of end-effector waypoints: descend 7 cm, grip, ascend, move to the drop-off point, release and return.

Control architecture flowchart — **Fig.** Control architecture.

Optical-flow head tracking

The camera records at 30 fps. On initialization, Shi-Tomasi corner detection (cv.goodFeaturesToTrack) picks 30 features at least 6 px apart, and pyramidal Lucas-Kanade (cv.calcOpticalFlowPyrLK) tracks them frame to frame. Per-frame displacement is clipped at 8 px, and total displacement is capped at 500 px. That range maps linearly to arm speeds of 0 to 6 mm/s.

Optical flow module architecture — **Fig.** Optical-flow module.

I compared Shi-Tomasi against Harris corner detection. Harris was too slow for real-time control, so I chose Shi-Tomasi.

Shi-Tomasi corners — Shi-Tomasi, 30 corners

Optical flow visualization with tracked features — **Fig.** Live flow visualization tracking the corners of a high-contrast backdrop.

EMG blink detection

A MyoWare 2.0 muscle sensor and Link Shield on an Arduino Uno read three electrodes. Two sit on the levator palpebrae superioris, the muscle that opens and closes the upper eyelid, and a reference electrode sits behind the ear. Voltage spikes are classified as blinks and published to ROS. I also tested placement on the frontalis, the eyebrow muscle.

Hardware architecture — **Fig.** Hardware: MyoWare 2.0, Arduino Uno, USB camera.

**Fig.** Candidate electrode placements.

Experimental setup & results

Preliminary experiments validated velocity control of a UR5e from head motion and blink-triggered pick-and-place. A black backdrop with high-contrast markers made feature detection more reliable. Arm control used Clark Teeple's Universal Robots ROS interface: its Cartesian trajectory handler ran the pick-and-place waypoints, and its twist handler sent velocity commands.

Experimental setup diagram and photo — **Fig.** Experimental setup: optical-flow backdrop, UR5e arm and table.

Challenges & future work

Signal quality: eyelid EMG is in the microvolt range. Traditional EOG uses wet Ag/AgCl electrodes, which are hard for users with DMD to maintain. Fabricated electrodes (inkjet-printed, metal 3D-printed or silver-nanoparticle tattoo electrodes) could improve the signal-to-noise ratio.
Placement robustness: the system needs to tolerate small electrode shifts. Machine-learning classifiers for facial-muscle activity are one option.
Integration: a fully 3D-printed, wireless headset that holds the electrodes in a fixed position.