edge-ainative-integrationIoT

Integrating Edge AI with React Native: Prototyping with Raspberry Pi 5 and the AI HAT+ 2

rreactnative

2026-01-23

10 min read

Hands-on guide to connect React Native to a Raspberry Pi 5 + AI HAT+ 2 for low-latency local inference, native bridges, and performance tuning.

Hook — Fast local inference, low-latency mobile UX, and reliable native integration: this is what teams building cross-platform apps with React Native actually need in 2026.

If you’re shipping features that call an AI model on a networked Raspberry Pi 5 with the new AI HAT+ 2, you’ve hit two hard problems: minimizing round-trip latency and maintaining a robust native integration in a cross-platform app. This guide gives a practical, example-driven path to connect a React Native app to a Raspberry Pi 5 running local models (TensorFlow Lite or ONNX), optimize for low latency, and implement the native bridge patterns you’ll need for production.

What you’ll get in this guide

Hardware and OS checklist for Raspberry Pi 5 + AI HAT+ 2 (2026 context)
How to run TensorFlow Lite inference on the Pi and expose a low-latency API
Design choices: WebSocket vs. gRPC vs. WebRTC DataChannel for lowest latency
React Native native-bridge samples (Kotlin + Swift surface) for binary, streaming inference
Profiling and latency tuning steps for both Pi and mobile app
Advanced strategies and 2026 trends to future-proof your edge-AI integration

1) Hardware & OS checklist (Raspberry Pi 5 + AI HAT+ 2)

Before diving into code, validate your stack. In late 2025–2026 the ecosystem matured fast: NPUs on HAT devices, better delegate support in TensorFlow Lite, and optimized SDKs that expose both HTTP and streaming WebSocket/gRPC endpoints. Make sure you have:

Raspberry Pi 5 with a recent Raspberry Pi OS (2025/2026 build). Update packages: sudo apt update && sudo apt upgrade -y.
AI HAT+ 2 mounted and firmware updated to the vendor’s late-2025 release (firmware often fixes latency regressions).
Python 3.11+ or the vendor-recommended runtime and the latest TensorFlow Lite runtime package with delegate support installed.
Good network: Gigabit Ethernet or 5GHz Wi‑Fi with low interference — the network often dominates latency.

Quick OS commands

sudo apt update
sudo apt upgrade -y
sudo apt install -y python3-pip git build-essential
# Install tflite runtime (example, vendor may provide wheel)
python3 -m pip install --upgrade pip
python3 -m pip install tflite-runtime

2) Running local inference on the Pi (TensorFlow Lite)

In 2026 the practical default for edge inference remains TensorFlow Lite for small/medium models and ONNX Runtime for some operator coverage. The AI HAT+ 2 typically exposes a hardware-accelerated delegate; you want to use that delegate for speed.

Python example: load TFLite model with delegate

from tflite_runtime.interpreter import Interpreter, load_delegate
# use delegate if available
delegate = load_delegate('libai_hat_delegate.so')
interpreter = Interpreter(model_path='model.tflite', experimental_delegates=[delegate])
interpreter.allocate_tensors()
# run inference on a preprocessed input
interpreter.set_tensor(input_index, input_data)
interpreter.invoke()
output = interpreter.get_tensor(output_index)

Tip: Always match input preprocessing (normalization, dimensions) on-device and in the mobile client when you do any local pre-filtering — mismatches are a common source of “it worked on desktop but not on Pi” bugs.

3) Expose a low-latency API on the Pi

Design decision: REST is easy but high-latency. For sub-100ms round-trips you want streaming approaches.

Options

WebSocket — simple to implement, bi-directional, works well for small binary frames and request/response patterns.
gRPC (HTTP/2) — strong typing, streaming RPC, slightly more complex to set up but gives headroom for performance and protobuf binary payloads.
WebRTC DataChannel — lowest latency over lossy networks and ideal for peer-to-peer mobile-to-Pi flows, but more complex (ICE/STUN setup).

For most teams shipping quickly, WebSocket or gRPC streaming are the practical choices. Below is a minimal WebSocket server in Python using FastAPI + websockets that returns inference results with low overhead.

FastAPI + WebSocket server (minimal)

from fastapi import FastAPI, WebSocket
import uvicorn
import numpy as np
# assume interpreter initialized
app = FastAPI()

@app.websocket('/ws/infer')
async def infer_ws(websocket: WebSocket):
    await websocket.accept()
    try:
        while True:
            # receive a binary frame with serialized input (e.g. float32 bytes)
            data = await websocket.receive_bytes()
            input_arr = np.frombuffer(data, dtype=np.float32).reshape(1,224,224,3)
            interpreter.set_tensor(input_index, input_arr)
            interpreter.invoke()
            out = interpreter.get_tensor(output_index)
            # send back bytes for minimal overhead
            await websocket.send_bytes(out.tobytes())
    finally:
        await websocket.close()

if __name__ == '__main__':
    uvicorn.run(app, host='0.0.0.0', port=8000)

Why binary frames? JSON adds serialization cost. Sending float32 bytes (or quantized int8) cuts CPU and serialization latency.

4) Choosing the right protocol for your React Native app

Which option is best depends on your latency budget and network topology:

Wi‑Fi local network, sub-100ms target: WebSocket with binary frames, or gRPC streaming with protobuf. WebSocket is simpler; protobuf + gRPC reduces parsing overhead and improves schema evolution.
Peer-to-peer over public networks: WebRTC DataChannel if you need sub-50ms and can manage NAT traversal.
Very high throughput (video frames): Use a binary streaming pipeline with quantized inputs and dynamic batching on the Pi; consider gRPC for backpressure control.

5) React Native native bridge — why you need it and what it should do

React Native JS is great for app logic, but for low-latency binary streaming you want a native layer handling sockets and binary buffers. Reasons:

Native WebSocket APIs (OkHttp on Android, URLSessionWebSocketTask on iOS) are more reliable and have lower overhead for binary frames than JS-layer libraries.
Native code can use zero-copy buffers and expose them to JS as an event stream, minimizing copies.
Allows integration with platform-native audio/video capture for on-device preprocessing.

This guide shows a small native bridge surface: start, stop, sendFrame, onInference. The mobile app sends preprocessed frames as binary, the Pi returns a binary response (e.g., float32 or protobuf), and the bridge emits events back to JS.

Bridge API (JavaScript surface)

// JS API expected by app
EdgeAI.start({ url: 'ws://192.168.0.42:8000/ws/infer' });
EdgeAI.sendFrame(binaryBuffer); // ArrayBuffer or native buffer wrapper
EdgeAI.on('inference', (payload) => { /* payload: ArrayBuffer or parsed object */ });
EdgeAI.stop();

Android (Kotlin) — core ideas

Use OkHttp's WebSocket with ByteString for binary frames. Buffer data using ByteBuffer and avoid extra copies when emitting to JS via ReactContext.getJSModule(RCTDeviceEventEmitter.class).

class EdgeAiModule(reactContext: ReactApplicationContext): ReactContextBaseJavaModule(reactContext) {
  private var ws: WebSocket? = null
  private val client = OkHttpClient()

  @ReactMethod
  fun start(url: String) {
    val req = Request.Builder().url(url).build()
    ws = client.newWebSocket(req, object: WebSocketListener() {
      override fun onMessage(webSocket: WebSocket, bytes: ByteString) {
        // send bytes to JS as base64 or as a native buffer (prefer native buffer libs)
        val arr = bytes.toByteArray()
        // emit event
        val params = Arguments.createMap()
        params.putString("payload", Base64.encodeToString(arr, Base64.NO_WRAP))
        reactApplicationContext
          .getJSModule(DeviceEventManagerModule.RCTDeviceEventEmitter::class.java)
          .emit("onInference", params)
      }
    })
  }

  @ReactMethod
  fun sendFrame(base64: String) { /* decode and send bytes */ }
}

Notes: Prefer to use shared memory / RN buffer libraries (e.g., react-native-reanimated-style-native-buffer patterns) to avoid base64 where possible. Base64 increases payload size ~33% and CPU.

iOS (Swift) — core ideas

@objc(EdgeAiModule)
class EdgeAiModule: RCTEventEmitter {
  var webSocket: URLSessionWebSocketTask?

  @objc(start:)
  func start(url: String) {
    let session = URLSession(configuration: .default)
    webSocket = session.webSocketTask(with: URL(string: url)!)
    webSocket?.resume()
    receiveLoop()
  }

  func receiveLoop(){
    webSocket?.receive { [weak self] result in
      switch result {
      case .success(.data(let data)):
        // send raw data to JS via sendEvent
        self?.sendEvent(withName: "onInference", body: ["payload": data.base64EncodedString()])
      default: break
      }
      self?.receiveLoop()
    }
  }
}

Important: Use native binary types (Data/ByteBuffer) where possible and only cross to JS as minimized representations. In RN 0.70+ and into 2026, community buffer modules and Hermes improvements make binary exchanges more efficient — use them.

6) Latency profiling & tuning (Pi and mobile)

Measure first — then optimize. Use these steps to find where time is going:

Measure raw network RTT: ping and iperf3 for throughput.
Time the server-side inference end-to-end (preprocess & inference & postprocess) using Python timers or Prometheus metrics.
On mobile, measure JS-to-native bridge times (Flipper, console.time, and add timestamps in events).
Measure serialization overhead — compare JSON/base64 vs. raw binary vs. protobuf.

Common latency-saving wins

Quantize the model: int8 or float16 models reduce inference time and network bytes. See practical Edge-AI examples like Edge AI for small shops for quantization trade-offs.
Use hardware delegate: ensure the AI HAT+ 2 delegate is used by TFLite.
Batch on Pi: accumulate multiple mobile frames into one inference when latency budget allows amortizing overhead.
Zero-copy buffers: avoid base64 and reduce copies across native/JS boundaries (see tooling and buffer patterns in practical tool rundowns).
Network QoS: prioritize traffic or use local Wi‑Fi direct links for consistent latency.

7) Example latency budget: live camera labeler

Goal: show labels for camera frames at interactive speed. Budget: 120ms total. Example breakdown:

Capture + preprocess on device: 20ms
Network transmit (binary ~20KB): 10–30ms (local Wi‑Fi)
Pi inference (TFLite float16 + delegate): 20–40ms
Response parse and render on mobile: 10–20ms

If totals exceed target, apply strategies above: quantize to reduce transmit size and inference time, batch, or run a tiny on-device pre-filter to avoid sending frames that don't need inference.

8) Security, reliability, and production hardening

TLS / mTLS on WebSocket/gRPC for sensitive data. Use lightweight cert management for fleet devices.
Reconnect & backpressure — implement exponential backoff and drop old frames if queue backs up.
Failover — gracefully fall back to on-device fallback models on the phone if Pi is unreachable.
Monitoring — export inference latency, queue depth, memory use (on Pi) and JS event latencies (mobile) to a centralized dashboard.

9) Advanced strategies and 2026 trends

As of 2026 several trends are shaping edge-AI mobile integrations:

Hybrid on-device + edge inference: TinyML locally on the phone (keyword spotting, triage models) plus heavier models on the Pi when needed.
Model shipping & dynamic delegates: Devices increasingly support hot-swappable models and delegates over-the-air — design your Pi server to accept model updates atomically and integrate with modern edge orchestration patterns.
On-device LLMs & compressed models: Smaller contextual models on Pi-class hardware are now feasible — plan for variable model performance per device.
Edge orchestration: Tools in 2025–2026 improve deployment and version tracking for fleets of AI HAT devices. Integrate health checks into your pipeline and consider compact gateways and field-tested hardware reviews like those at compact gateways.

10) Troubleshooting checklist (fast)

No connection? Check firewall, Pi listening port, and whether Wi‑Fi client isolation is enabled on access point.
High latency spikes? Run traceroute, check CPU frequency scaling, and verify delegate is used (TFLite logs show delegate chosen).
Memory OOM on Pi? Reduce batch size, swap out to zram, or use smaller quantized models.
JS thread jank? Move heavy preprocessing into the native bridge or use WebWorkers (Hermes + JS engines limits).

11) Minimal end-to-end example (quick flow)

On Pi: run FastAPI WebSocket server exposing binary infer endpoint.
On mobile: native bridge opens WebSocket, sends a preprocessed binary frame (uint8 or int8), and listens for binary result.
On Pi: deserialize bytes, convert to model input, run TFLite with delegate, serialize output (protobuf or raw bytes), send back.
On mobile: bridge receives bytes, converts to JS objects or typed array, UI updates immediately.

Actionable checklist (do this now)

Update Pi OS & AI HAT+ 2 firmware to latest vendor release (late-2025 build or newer).
Quantize and test your model locally on the Pi; verify delegate is active.
Build a small WebSocket server that accepts binary frames and returns binary outputs — test end-to-end with a desktop WebSocket client first.
Create a lightweight native bridge for your React Native app that handles binary frames natively (avoid base64 if possible).
Measure latency at each hop and iterate: network > server > bridge > UI.

Parting notes — future-proof your edge-AI strategy

Edge AI in 2026 is about balancing compute where it makes the most sense. The Raspberry Pi 5 + AI HAT+ 2 combination makes powerful local inference possible, but the mobile integration layer is where user experience is won or lost. A small native bridge, binary streaming, and disciplined profiling will get you from prototype to production quickly and reliably.

“Optimize for the slowest link first — usually the network or unnecessary serialization.”

Call to action

Ready to build it? Clone a starter repo that contains the Pi WebSocket server, a quantized TFLite sample, and a React Native native module scaffold (Android + iOS) — start with a single frame path and iterate with the profiling checklist above. Share your latency numbers and problems in the community; I’ll outline further optimization patterns (WebRTC, grpc-based flow control, quantization-aware training) in a follow-up.

Next steps: 1) Update your Pi/HAT firmware. 2) Run the TFLite sample with delegate. 3) Wire a native bridge for binary WebSocket. Then measure — and come back to optimize.

reactnative

Contributor

Senior editor and content strategist. Writing about technology, design, and the future of digital media. Follow along for deep dives into the industry's moving parts.