I’m so excited to show my another open-source project here. It is a PoC project.
Distributed Inference is a project to demonstrate an approach to designing cross-language and distributed pipeline in deep learning/machine learning domain, using WebRTC and Redis Streams.
This project consists of multiple services, which are written in Go, Python, and TypeScript, running on Docker. It allows setting up multiple inference services in multiple host machines, in a distributed manner. It does RPC-like calls and service discovery via my other open-source projects, go-inventa and py-inventa, you can find them in my profile too.
Also includes a monitoring stack configuration using Grafana, InfluxDB, Telegraf, and Prometheus.
The main idea is:
- Webcam video will be streamed to the Media Bridge service via WebRTC, - Media Bridge service will capture frame images from the video as JPEG images, pushes them to Redis Streams, - One of available Inference services will pop a JPEG image data from Redis Streams stream, execute YOLOX inference model, push detected objects' name, box coordinates, prediction score, and resolution to other Redis Streams stream, - The Signaling service listens and consumes the Redis Streams stream (predictions), sends the results to relevant participant (by participantId in the JSON) via WebSockets. - Web client will draw boxes for each prediction, and writes results to the browser console.
Please check it out and I’d love to read your thoughts!