Handsy AI
Plug an ESP32 board (about $7) into your iPhone and let AI watch the screen via AirPlay and tap via USB — no jailbreak required.
About Handsy AI
Handsy AI is an AI agent project for the iPhone: plug an ESP32 board (about $7) into the phone, and the board watches the screen via AirPlay and taps and swipes through USB like a mouse and keyboard, while the companion app decides each step. The whole process requires no jailbreak; a computer is only needed the first time you flash the board.
Best for
Best use cases
Let an AI agent perform taps, swipes, and other actions on your iPhone for you
Explore iPhone automation without jailbreaking
DIY a screen-control agent with an off-the-shelf microcontroller
Features
Watch the screen via AirPlay
The board starts its own Wi-Fi network and appears in screen mirroring, passing the AirPlay video stream unchanged to the app over Bluetooth.
Tap like a human
When plugged into USB-C, the board acts as an ordinary mouse and keyboard; with AssistiveTouch enabled, a click becomes a tap and a drag becomes a swipe.
Thinks in the app
The app sends each screenshot to the vision model you choose, which plans the next step and checks whether the screen has actually changed.
Pros
- No need to jailbreak the iPhone
- Low hardware cost for the board
- You can choose your own vision model
- The project is open source on GitHub
Cons
- Requires providing your own ESP32-S3 board and assembling and flashing the firmware yourself
- A computer is needed to flash the board
- Requires enabling AssistiveTouch and screen mirroring on the iPhone