Image processing through API works on the principle of sending the image via HTTP request and getting a response with the result (contains Context by default, but there are also other response types available). Only one image can be sent per request.
If you didn’t configure you project first check Project Configuration.
Request
PEKAT can handle two types of requests:
POST request - for analyzing images
GET request - for getting the last analyzed image and pinging the project
Both requests take an URL as an argument, that needs to be constructed from the target projects IP, port, requested command, response type and possibly some additional arguments.
The POST and GET requests and their different uses will be explained later on this page, but first we’ll focus on how to find out the IP address and port number of the target project.
Example of a request
url = f"http://{ip}:{port}/{command}&{required_arguments}&{optional_arguments}" response = requests.post( url, data=image, headers={"Content-Type": "application/octet-stream"}, timeout=timeout, )
Each request consists of at least:
URL that specifies where to send the request to the project and what is the project supposed to do
Timeout specifying how long to wait for response before aborting (for example, if the project isn’t running, the request would wait indefinitely)
Furthermore as you can see, the request can have additional arguments:
data - they are present only in case of the POST request. The data argument contains data of the image for processing
headers - in case that the request returns data as a result, the headers argument specifies the format of the data sent
URL
The URL for the requests would generally look like this:
url = f"http://{ip}:{port}/{command}"
But depending on the response type you may need to add arguments to the URL, in which case it would be modified to look like this:
url = f"http://{ip}:{port}/{command}?{argument1}={value1}&{argument2}={value2}&..."
In total, URL consists of:
IP address of the computer when the project is running
Port on which the project is running
Command that is telling the project what to do
required arguments - information that the project needs to perform the requested task
optional arguments - additional arguments that can be used to change the response type of the project as well as other things
IP address
The IP address that needs to be in the URL is the address of the machine on which the project is running. Alternatively, if the project is running on the same machine on from which you’ll be sending the requests, you can use address 127.0.0.1.
Port
The port in the URL is the port of the PEKAT project you want to connect to. The port can be found in Projects manager and it can be modified before starting the project.
Once the project is running, both IP address and port are shown in the project window. The localhost is equivalent top IP “127.0.0.1“:
URL Command
Commands differ based on the used request. With POST request, the command specifies the format in which the image data is sent, whereas the commands with GET request can allow you to stop or ping the project or get the last analyzed image.
URL Arguments
URL arguments provide additional context to PEKAT, either required or optional and they differ based on the request and command.
POST request
Post request is used to send images for analyzing to PEKAT project and get result. A general POST request to PEKAT project would look like this:
url = f"http://{ip}:{port}/{command}&{required_arguments}&{optional_arguments}" response = requests.post( url, data=image, headers={"Content-Type": "application/octet-stream"}, timeout=timeout, )
POST request commands
POST request has two different commands, depending on the type of data you’re sending for analyzing. Based on the command you also need to provide certain arguments and optional arguments. The two commands available to the POST requests are “analyze_image” and “analyze_raw_image”.
analyze_image
Is used for sending byte stream with data about .png image. This byte stream data also contain information about the width and height of the image, so it requires less arguments compared to the analyze_raw_image command.
It’s recommended to load the .png image data using the Path class from pathlib and simply use the .read_bytes() method like this:
from pathlib import Path image = Path("{image_path}").read_bytes()
The sent data must be of a .png file. The project won’t be able to process other formats.
The URL of this command doesn’t nee any other arguments, but it can be provided with some optional ones, that will be explained later.
Example of a code sending an image for analyzing using the analyze_image command:
from pathlib import Path image = Path("img.png").read_bytes() url = f"http://127.0.0.1:8000/analyze_image" response = requests.post( url, data=image, headers={"Content-Type": "application/octet-stream"}, timeout=5, )
analyze_image command is useful when you don’t want to preprocess the image, because it allows you to load and send images without the need to install and import any additional libraries.
analyze_raw_image
Is used for sending image as a raw numpy byte stream. This byte stream of a numpy array doesn’t contain information about the height and width of the image, so that has to be provided as additional required arguments in the URL.
You can load an image into a numpy image using the cv2 library like this:
import cv2 image = cv2.imread({"image_path"})
After reading the image using cv2, you’ll have to find out it’s dimensions like this:
height, width = image.shape[:2]
For testing, you can also generate random numpy array that could represent an image (in this example, the image size is 512x512 and it’s an rgb image) like this:
import numpy as np image = np.random.randint(0, 256, (512, 512, 3), dtype=np.uint8)
Additional arguments
The additional arguments
Continue from here
The sending address consists of the IP address, port, type, and optional further query parameters. The request method must be of ‘POST’ type and the content type of ‘application/octet-stream’ type.
Find a practical API demonstration in Python at
https://github.com/pekat-vision/examples/tree/master/image_analyze_api/pythonIP Address
If the project runs on a local computer, the address is 127.0.0.1. If a remote computer is accessed, then you need to use the IP address of the remote computer.
It is necessary to specify the IP address of the PC that is going to send the images. To do this navigate to Input → Host panel and modify the IP address.
Port
This is selected when starting the project, the default is 8000.
Data
You can add extra information to the request. This string will be added to the context (the key is data). It is available only in the Code module (context variable).
'http://127.0.0.1:8000/analyze_image?data=SomeInfo'
In the Code module, you can access this string using:
context['data']
Types
analyze_raw_image
Used for sending the image as raw data. For example, the NumPy array in Python is converted to binary format.
You need to send the image dimensions in query parameters width and height.
Example of a 1920x1024 image:
'http://127.0.0.1:8000/analyze_raw_image?width=1920&height=1024'
If you are sending Bayer RG 8 data, it is necessary to add an argument bayer.
'http://127.0.0.1:8000/analyze_raw_image?width=1920&height=1024&bayer'
Example of a Python code sending requests with raw images from the camera:
import cv2 import requests cap = cv2.VideoCapture(0) # you can set frame size # cap.set(cv2.CAP_PROP_FRAME_WIDTH, 960) # cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 540) # keep alive request_session = requests.Session() while True: # get frame from camera ret, frame = cap.read() # get image shape shape = frame.shape # send frame to PEKAT VISION response = requests.post( url='http://127.0.0.1:8000/analyze_raw_image?width='+str(shape[1])+'&height='+str(shape[0]), data=frame.tobytes(), headers={'Content-Type': 'application/octet-stream'} ) print(response.json())
analyze_image
Is used for sending an image in ‘jpg’ or ‘png’ formats.
Basic example
'http://127.0.0.1:8000/analyze_image'
Example Python code that sends images from a folder to the Pekat project with an API key:
import requests import os request_session = requests.Session() for image in os.listdir('images_folder'): with open(os.path.join('images_folder', image), 'rb') as image: response = requests.post( url='http://127.0.0.1:8000/analyze_image?api_key=728a9180-8357-11ec-b645-e917eb5f5d27', data=image.read(), headers={'Content-Type': 'application/octet-stream'} ) print(response.json())
last_image
Gets the last image/context that was analyzed in the given project before.
Basic example (gets just the context)
http://127.0.0.1:8000/last_image
Python code which gets the last processed image with heatmap and shows it then prints out the context.
import requests import cv2 import numpy as np import base64 import json port = '8000' request_session = requests.Session() response = requests.get( url=f'http://127.0.0.1:{port}/last_image?response_type=annotated_image', headers={'Content-Type': 'application/octet-stream'} ) # Decode and show image with heatmap img = cv2.imdecode(np.frombuffer(response.content, np.uint8), 1) cv2.imshow("Image", img) cv2.waitKey(0) # Decode context from Base64 context_str = base64.b64decode(response.headers['ContextBase64utf']) # Load json from string context = json.loads(context_str) print(context)
Response type
Query parameter ‘response_type’ determines what content will be sent in the response that is returned from PEKAT VISION after processing the request. The request URL changes in the following way (example for annotated_image response type):
response = requests.post( url='http://127.0.0.1:8000/analyze_image?response_type=annotated_image', data=image.read(), headers={'Content-Type': 'application/octet-stream'} )
Context
context – a serialized context in JSON format. The contents are explained on the Context page. You can access the context json from the response using:
j = response.json()
Images
Further options return a PNG image in binary form in the response. It needs to be decoded afterward to further work with it as an image.
Example of decoding and showing the image with the PIL library:
from PIL import Image from io import BytesIO img = Image.open(BytesIO(response.content)) img.show()
Example of decoding and showing the image with OpenCV library:
import cv2 import numpy as np img = cv2.imdecode(np.frombuffer(response.content, np.uint8), 1) cv2.imshow("Image", img) cv2.waitKey(0)
image – processed image - output image that passed through the flow, but without any heatmaps or rectangles (e.g. if the image is scaled during the flow, it returns a scaled image).
annotated_image – processed image with annotations - heatmaps over the image.
heatmap - response contains only the heatmaps over the image (without the image).
For all those image response types, the serialized context is by default added to the headers with the header title 'ContextBase64utf'. To get it to JSON form, you can use this code:
import base64 import json # Decode from Base64 context_str = base64.b64decode(response.headers['ContextBase64utf']) # Load json from string context = json.loads(context_str)
Context in body
When getting an image in the response, by default the context is sent in headers. However, with a very large number of defects, the maximum header length limit can be reached. To solve this, you can use the option to send context in the body of the response together with the image and get the image length in a header 'ImageLen' to be able to divide the two parts after. To activate this mode, add the following to the request:
context_in_body=t
Example of the usage in Python code when getting the last image:
import requests import cv2 import numpy as np import json port = '8000' request_session = requests.Session() response = requests.get( url=f'http://127.0.0.1:{port}/last_image?response_type=annotated_image&context_in_body=t', headers={'Content-Type': 'application/octet-stream'} ) # Get image length img_len = int(response.headers['ImageLen']) # Decode and show image with heatmap img = cv2.imdecode(np.frombuffer(response.content[:img_len], np.uint8), 1) cv2.imshow("Image", img) cv2.waitKey(0) # Get and print context context = json.loads(response.content[img_len:]) print(context)
Testing the API functionality
If you have a running project on a port e.g. 8000, you can test the API by accessing this address in your browser (if PEKAT is running on the local computer, otherwise replace localhost with the IP address of the remote PC):
This displays the testing app, where you can choose to display the last processed image or send a new image to analyze. You can choose the Response type - Context, Image, Annotated image (image with a heatmap), or Heatmap. If the project has Secure image analyze enabled, you also need to enter the API key.
If Analyze image is chosen, you can also select the image that will be sent through the API. If the response type is different than the Context, the default behavior is that the image is sent in the body of the response and the context is sent in the headers - to change this and send both in the body, check the box with Context in the body. The detail of the request structure will be displayed.
After you click Send, you will also see the structure of the response in the blue info box, context (as you would see it in Inspection), and optionally an image based on the response type.