Project Motivation

The ESP32 Control Studio was born out of the need for a low-latency, responsive, and customizable mobile interface to control embedded robotics projects. Existing solutions were either too rigid, had high latency (like Bluetooth classic or standard HTTP REST APIs), or required complex setups. The goal was to build a universal, highly responsive dual-joystick controller that could integrate seamlessly with any ESP32-based hardware.

Problem Statement

Building custom robotics or remote-controlled vehicles usually requires a custom transmitter or a complicated mobile app. Developers spend too much time building the controller rather than focusing on the robot's logic. Our problem was: How do we create a generalized, ultra-low latency mobile controller that works out-of-the-box with ESP32?

Brainstorming Phase

We considered various control mechanisms:

• Web dashboard (too much latency, hard to handle multi-touch joysticks on mobile browsers).

• Custom hardware transmitter (expensive, not scalable).

• Native mobile applications (iOS/Android separately - too much maintenance).

• Cross-platform mobile app (Flutter - write once, run everywhere).

We settled on Flutter for the mobile app due to its strong UI capabilities and cross-platform nature.

Technology Selection

Flutter vs Native

Flutter was chosen over building separate native apps because it provides excellent 60fps rendering, custom widget support (vital for joysticks), and unified logic for both Android and iOS.

UDP vs Bluetooth vs WebSockets

• Bluetooth/BLE: Often has pairing issues, range limitations, and varying latency based on the device stack.

• WebSockets/TCP: Reliable but suffers from "Head-of-Line Blocking." If a packet drops, subsequent packets are delayed, causing "stutter" in real-time control.

• UDP (User Datagram Protocol): Fire-and-forget. It doesn't guarantee delivery, which is exactly what we want for real-time joysticks. If a packet is lost, the next packet (sent milliseconds later) will contain the latest state anyway.

Decision: UDP for control packets, with a unified 8-byte payload.

System Architecture Design

The architecture was split into three main parts:

1. The Mobile Controller (Flutter): Reads user inputs at 60Hz and blasts state packets.

2. The Network Transport (UDP over WiFi): Handles rapid, stateless transmission.

3. The Embedded Receiver (ESP32): Listens on a dedicated port, decodes the 8-byte packet, and commands hardware (motors/servos/LEDs).

Control Protocol Design

To minimize processing overhead on the ESP32, we designed a hyper-compact 8-byte protocol:

• Byte 0: Header (0xAA) for synchronization.

• Byte 1-4: Joysticks (X1, Y1, X2, Y2) mapped to 0-255 (128 is center).

• Byte 5: Push Buttons state (Bitmask).

• Byte 6: Toggle Switches state (Bitmask).

• Byte 7: Simple XOR Checksum for basic error rejection.

UI/UX Design Decisions

We prioritized a dark-mode, futuristic "Control Studio" aesthetic.

• Dashboard: Dual joysticks placed ergonomically for thumbs.

• Telemetry: Real-time feedback overlaid cleanly, avoiding clutter.

• Connection Screen: Simple IP entry with offline validation before attempting UDP.

Flutter Implementation

We utilized:

• Provider State Management: To decouple the UI from the UDP networking logic. The ControllerState class manages joystick positions, buttons, and handles the network dispatch simultaneously.

• CustomPainter / Gestures: For smooth joystick rendering and multi-touch handling.

Networking Layer Development

The UDP client binds to a random local port and sends datagrams to the target ESP32 IP on port 4210. A background telemetry listener is set up on a return port to catch hardware updates like battery voltage or sensor readings.

ESP32 Firmware Development

The firmware uses the standard WiFiUdp.h library. It sits in a tight loop checking udp.parsePacket(). When a valid 8-byte packet with the correct header and checksum arrives, it updates motor PWM channels and toggles GPIOs instantaneously.

Testing Strategy

• Unit Testing: Flutter network logic was unit-tested with mocked UDP sockets.

• Mock Server: A Python mock server was built to validate packet structures without needing actual hardware on hand.

• Live Hardware: Finally tested with an ESP32 connected to an oscilloscope to measure GPIO response time against screen taps.

Hardware Validation

Validation proved that the Flutter -> WiFi -> ESP32 pipeline achieved an end-to-end latency of <15ms on a local network, well within the threshold for human-perceptible instant response.

Performance Optimization

• Reduced packet send rate to 50Hz (20ms). Sending faster overwhelmed the ESP32's basic WiFi stack.

• Implemented state delta checks: buttons only send updates on change, or regularly alongside the 50Hz joystick loop to ensure state consistency even if a packet is lost.

Final Production Build

The final output is a standalone APK capable of connecting to any ESP32, enabling rapid robotics prototyping with zero custom Android development required for the end user.

for understanding in depth

system architecture: https://github.com/ShravanaHS/ESP32-UDP-Controller/blob/main/docs/architecture.md

protocol implemented: https://github.com/ShravanaHS/ESP32-UDP-Controller/blob/main/docs/protocol.md

How to use it

App usage: https://github.com/ShravanaHS/ESP32-UDP-Controller/blob/main/docs/app-usage.md

Example Code: https://github.com/ShravanaHS/ESP32-UDP-Controller/blob/main/docs/esp32-firmware.md