How to use TensorFlow Serving?

Install TensorFlow Serving

• First, ensure you have Docker installed on your machine since it's one of the easiest ways to run TensorFlow Serving. Verify Docker installation by running `docker --version`.

 

• Pull the TensorFlow Serving Docker image using the command:

docker pull tensorflow/serving

Export Your Model to SavedModel Format

• TensorFlow Serving uses the SavedModel format. Ensure your model is converted to this format. If you're using a Keras model, you can export it as shown below:

model.save('/model/path/my_model', save_format='tf')

Run TensorFlow Serving with Docker

• Now that your model is saved, serve the model using TensorFlow Serving. Mount the model directory to Docker and run the container:

docker run -p 8501:8501 --name=tf_serving \
  --mount type=bind,source=/model/path,my_model,target=/models/my_model \
  -e MODEL_NAME=my_model -t tensorflow/serving

Test Your Model Server

• Use an HTTP client such as `curl` to send a JSON request to the model. Be sure to replace `YOUR_DATA` with appropriate input data:

curl -d '{"signature_name":"serving_default", "instances":[YOUR_DATA]}' \
  -H "Content-Type: application/json" \
  -X POST http://localhost:8501/v1/models/my_model:predict

Integrate TensorFlow Serving into Your Application

• To use the served model, you can integrate the HTTP requests into your application code. Here's an example using Python's `requests` library:

import requests
import json

url = "http://localhost:8501/v1/models/my_model:predict"
headers = {"content-type": "application/json"}
data = json.dumps({"signature_name": "serving_default", "instances": [YOUR_DATA]})

json_response = requests.post(url, data=data, headers=headers)
predictions = json_response.json()["predictions"]
print(predictions)