Hi, I gather different methods how to deploy Tensorflow model to cater different processing, https://github.com/huseinzol05/Gather-Deployment

1 . Object Detection. Flask SocketIO + WebRTC

• Stream from webcam using WebRTC -> Flask SocketIO to detect objects -> WebRTC -> Website.

2 . Object Detection. Flask SocketIO + opencv

• Stream from OpenCV -> Flask SocketIO to detect objects -> OpenCV.

3 . Speech streaming. Flask SocketIO

• Stream speech from microphone -> Flask SocketIO to do realtime speech recognition.

4 . Text classification. Flask + Gunicorn

• Serve Tensorflow text model using Flask multiworker + Gunicorn.

5 . Image classification. TF Serving

• Serve image classification model using TF Serving.

6 . Image Classification using Inception. Flask SocketIO

• Stream image using SocketIO -> Flask SocketIO to classify.

7 . Object Detection. Flask + opencv

• Webcam -> Opencv -> Flask -> web dashboard.

8 . Face-detection using MTCNN. Flask SocketIO + opencv

• Stream from OpenCV -> Flask SocketIO to detect faces -> OpenCV.

9 . Face-detection using MTCNN. opencv

• Webcam -> Opencv.

10 . Image classification using Inception. Flask + Docker

• Serve Tensorflow image model using Flask multiworker + Gunicorn on Docker container.

11 . Image classification using Inception. Flask + EC2 Docker Swarm + Nginx load balancer

• Serve inception on multiple AWS EC2, scale using Docker Swarm, balancing using Nginx.

12 . Text classification. Hadoop streaming MapReduce

• Batch processing to classify texts using Tensorflow text model on Hadoop MapReduce.

13 . Text classification. Kafka

• Stream text to Kafka producer and classify using Kafka consumer.

14 . Text classification. Distributed TF using Flask + Gunicorn + Eventlet

• Serve text model on multiple machines using Distributed TF + Flask + Gunicorn + Eventlet. Means that, Distributed TF will split a single neural network model to multiple machines to do feed-forward.

15 . Text classification. Tornado + Gunicorn

• Serve Tensorflow text model using Tornado + Gunicorn.

16 . Text classification. Flask + Celery + Hadoop

• Submit large texts using Flask, signal queue celery job to process using Hadoop, delay Hadoop MapReduce.

17 . Text classification. Luigi scheduler + Hadoop

• Submit large texts on Luigi scheduler, run Hadoop inside Luigi, event based Hadoop MapReduce.

18 . Text classification. Luigi scheduler + Distributed Celery

• Submit large texts on Luigi scheduler, run Hadoop inside Luigi, delay processing.

19 . Text classification. Airflow scheduler + elasticsearch + Flask

• Scheduling based processing using Airflow, store inside elasticsearch, serve it using Flask.

20 . Text classification. Apache Kafka + Apache Storm

• Stream from twitter -> Kafka Producer -> Apache Storm, to do distributed minibatch realtime processing.

21 . Text classification. Dask

• Batch processing to classify texts using Tensorflow text model on Dask.

22 . Text classification. Pyspark

• Batch processing to classify texts using Tensorflow text model on Pyspark.

23 . Text classification. Pyspark streaming + Kafka

• Stream texts to Kafka Producer -> Pyspark Streaming, to do minibatch realtime processing.

24 . Text classification. PyFlink

• Batch processing to classify texts using Tensorflow text model on Flink batch processing.

25 . Text classification. Streamz + Dask + Kafka

• Stream texts to Kafka Producer -> Streamz -> Dask, to do minibatch realtime processing.

Discussion

1. I am waiting for official binary released flink 1.10, support custom Python function, right now stable released 1.9 cannot.
2. Just realized, I missed FastAPI, will going to add later.
3. Kubeflow required at least minikube, maybe will add later.
4. Will add imagezmq -> nginx load balancer -> process multiple video sources on multiple machines, display on single machine.

Feel free to comment!