AI Service Development Guide

1. Overview

This document guides developers on the standards needed to develop an AI service for integration into the AI marketplace.The overall architecture is described in Figure 1:

• The AI service is an asynchronous service, therefore, developers need to design a queuing mechanism so that when a request is received, an immediate response is sent back to the client. In case there are no protocol-related errors or parameter violations, an AI task is created (associated with an ID) and pushed into the queue for processing. The AI service then responds to the client, indicating that the task has been created and is awaiting processing, with an errorCode indicating the "pending" status.
• Once the task is sent to the worker for processing, the task status changes from "pending" to "in progress". If at this point the client calls an API to retrieve the result based on the taskID, they will receive a response indicating that the task is currently in the "in progress" state.
• If the task is processed successfully, the task status will change to "success", and the result will be sent to the marketplace via a webhook. If the task encounters an error for any reason, it will be logged with an errorCode and a specific error description. In both cases of success or failure, information about the request is stored in the database for future analysis purposes.
• In case the marketplace does not find the result on the webhook, it can call an API to retrieve the result based on the taskID.

Developers can choose any technology to develop the AI service. However, they need to follow:

• The AI service will communicate with the marketplace via a RESTful API, which includes the following aspects:Endpoint, Number of Endpoints, Status Code, Header, Body Format, Error Code… (The following sections will provide detailed explanations for each item)
• All non-text data such as images, videos, audio files, etc., uploaded by users will be stored in S3. The client (Marketplace) will provide a shareable link for the AI service to download and process user requests. Therefore, developers need to specifically define the details in the body of the /call API.
• For all non-text results such as images, videos, audio files, etc. The AI service will store them in S3 and share a link via the API response for the marketplace to download the results.
• Developers need to follow a database schema to store the necessary information for use in the reporting process
• The AI service is packaged and run as a container.

2. Design API

CREATE REQUEST FLOW

Information response rules

• If there are any protocol-related errors in the API, such as missing fields in the header or body, or incorrect data types defined, the AI service will return an HTTP status code indicating an error along with details.
• The AI service will validate the content of the body: if any field does not comply with the predefined rules, it will return an HTTP status code of 200 along with an error code defined by the developer (more details in section 2.5)
• The AI service will create a task and push it into a queue for processing by a worker. Simultaneously, it will immediately respond to the client with an HTTP status code of 200, providing a taskId that the client can use later to retrieve the results. This response will include an errorCode of "xxx_001" and a reason for "pending".
• Once the request has been processed, the results will be pushed back to the client (using the API provided by the client). This response will include an errorCode of "xxx_000" and a reason for "success".
• When a request is successfully processed, the request information will be saved in the database for future statistical purposes.

GET RESULT FLOW

Information response rules

• If there are any protocol-related errors in the API, such as missing fields in the header or body, or incorrect data types defined, the AI service will return an HTTP status code indicating an error along with details.
• The AI service will validate the content of the body: if any field does not comply with the predefined rules, it will return an HTTP status code of 200 along with an error code defined by the developer (more details in section 2.5)
• The AI service returns the result with an HTTP_STATUS_CODE=200 along with an errorCode in the following cases:
  • If the task is still in progress: errorCode=”xxx_002”, reason =”in progress”
  • If the task successfully processed.: errorCode=”xxx_000”, reason =”success”
  • If the task was not processed due to any specific error: Proper error code and error description.

GET STATISTICS FLOW

Figure 2.3. Get statistics API sequence diagram

Information response rules

• If there are any protocol-related errors in the API, such as missing fields in the header or body, or incorrect data types defined, the AI service will return an HTTP status code indicating an error along with details.
• The AI service will validate the content of the body: if any field does not comply with the predefined rules, it will return an HTTP status code of 200 along with an error code defined by the developer (more details in section 2.5)
• The AI service will return the statistical information on the number of successful or failed requests from the user

2.1 Call API

This API is designed for marketplaces to call and create a request to process an AI task

2.1.1 Endpoint

2.1.2 Request

Header

Body

Example data

# Body
{
  "method": "music_gen",
  "payload": {
    "promt": "energetic EDM",
    "duration": 5
  }
}

# CURL command
curl -X 'POST' 
'host:port/call' 
-H 'accept: application/json' 
-H 'x-marketplace-token: token_value' 
-H 'x-request-id: request_id_value' 
-H 'x-user-id: user_id_value' 
-H 'Content-Type: application/json' 
-d '{
"method": "music_gen",
"payload": {
  "promt": "energetic EDM",
  "duration": 5
}
}'

2.1.3 Response

Example data

# Successful Request Response
# taskId used for calling the /result API.
{
    "requestId": "test-request",
    "traceId": "0242d77c-6079-4214-b0e3-676c35afbe22",
    "apiVersion": "1.0.1",
    "service": "AudioCraft",
    "datetime": "2024-05-16T07:16:12.689022",
    "isResponseImmediate": "false",
    "extraType": "others",
    "response": {
        "taskId": "45e667c3-75ac-41c5-b4d9-538e05cc88d6",
    },
    "errorCode": {
        "status": "AC_001",
        "reason": "pending"
    }
}

# Error Response: Incorrect method used
# Only support audio_gen or music_gen. “music” is not supported
{
    "requestId": "",
    "traceId": "",
    "apiVersion": "1.0.1",
    "service": "AudioCraft",
    "datetime": "2024-05-16T07:16:12.689022",
    "isResponseImmediate": "false",
    "extraType": "others",
    "response": {
        "taskId": ""
    },
    "errorCode": {
        "status": "AC_403",
        "reason": "unsupported method music"
    }
}

2.2 Callback API

After an AI task is processed, an API callback is triggered to push the processing results back to the marketplace. The API callback is defined and deployed by the marketplace. The developer needs to define certain fields in the request body that align with the application's requirements.

2.2.1 Endpoint

2.2.2 Request

Header

Body

Example data

# body
{
    "apiVersion": "1.0.1",
    "service": "AudioCraft",
    "datetime": "2024-05-16T07: 30: 08.592007",
    "processDuration": 5812.790632247925,
    "taskId": "15204bff-02cb-4b04-8ab0-f0dffd9c44fd",
    "isResponseImmediate": "false",
    "extraType": "others",
    "response": {
        "dataType": "S3_OBJECT",
        "data": "A shared link of music file saved on S3"
    },
    "errorCode": {
        "status": "AC_000",
        "reason": "success"
    }
}

# CURL example
curl -X POST 
  https://<The_market_place_webhook_endpoint> 
  -H "x-marketplace-token: token_value" 
  -H "x-request-id: request_id_value" 
  -H "x-user-id: user_id_value" 
  -H "Content-Type: application/json" 
  -d '{
    "apiVersion": "1.0.1",
    "service": "AudioCraft",
    "datetime": "2024-05-16T07:30:08.592007",
    "processDuration": 5812.790632247925,
    "taskId": "15204bff-02cb-4b04-8ab0-f0dffd9c44fd",
    "isResponseImmediate": "false",
    "extraType": "others",
    "response": {
            "dataType": "S3_OBJECT",
            "data": "A shared link of music file saved on S3"
    },
    "errorCode": {
      "status": "AC_000",
      "reason": "success"
    }
  }'

# Any errors encountered during processing will be described in the errorCode.

2.2.3 Response

If the request is successful, it will return an HTTP status code 201.

2.3 Result API

This API is used by the marketplace to retrieve results based on the taskId returned in the /call API.

2.3.1 Endpoint

2.3.2 Request

Header

Body

Example data

# body
{
  "taskId": "the_taskId_from_call_api_response"
}

# CURL example
curl -X 'POST' 
  'host:port/result' 
  -H 'accept: application/json' 
  -H 'x-marketplace-token: token_value' 
  -H 'x-user-id: user_id_value' 
  -H 'Content-Type: application/json' 
  -d '{
  "taskId": "the_taskId_from_call_api_response"
}'

2.3.3 Response

Example data

# Successful Request Response
{
    "apiVersion": "1.0.1",
    "service": "AudioCraft",
    "datetime": "2024-05-16T07:30:08.591039",
    "processDuration": 5812.790632247925,
    "isResponseImmediate": "false",
    "extraType": "others",
    "response": {
        "dataType": "S3_OBJECT",
        "data": ""
    },
    "errorCode": {
        "status": "AC_000",
        "reason": "success"
    }
}
# Response for a request with a non-existent taskId.
{
    "apiVersion": "1.0.1",
    "service": "AudioCraft",
    "datetime": "2024-05-16T07:30:08.591039",
    "processDuration": -1,
    "isResponseImmediate": "false",
    "extraType": "others",
    "response": {
        "dataType": "S3_OBJECT",
        "data": null
    },
    "errorCode": {
        "status": "AC_404",
        "reason": "taskId: a_random_taskid is not exist"
    }
}

2.4 Stats API

The API is designed to allow the marketplace to retrieve statistics about user requests.

2.4.1 Endpoint

2.4.2 Request

Header

Body

2.4.3 Response

2.5 Accession

2.5.1 Upload object

Get presigned

curl -X 'POST' 
  'https://marketplace-api-user.dev.devsaitech.com/api/v1/ai-resource/presigned-upload' 
  -H 'accept: application/json' 
  -H 'x-publisher-key: UK24XC7qjGpeUqPo69tL6fxyt4cSXvmwZ7sYFu4nH7mKjXZyHeyHXTMVtup48hSf' 
  -H 'Content-Type: application/json' 
  -d '{
  "s3Key": "file_name.extension"
}'

Response

{
  "code": 0,
  "message": "success",
  "data": {
    "presignedUrl": "https://dev-aitech-marketplace-blob-storage.s3.ap-southeast-1.amazonaws.com/0029781988958338526/8a19ac42-7ace-463e-9e0f-a70f165964fb-file_name.extension?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIAVRUVRKXLOODYDPP7%2F20240620%2Fap-southeast-1%2Fs3%2Faws4_request&X-Amz-Date=20240620T102419Z&X-Amz-Expires=3600&X-Amz-Signature=ef59496065d0a9f147c9743037fb8849e559da3e19d74b03f509d11ebc709dba&X-Amz-SignedHeaders=host&x-id=PutObject",
    "key": "0029781988958338526/8a19ac42-7ace-463e-9e0f-a70f165964fb-file_name.extension"
  }
}

Use presignedUrl to create an uploading request. The object is stored as 0029781988958338526/8a19ac42-7ace-463e-9e0f-a70f165964fb-file_name.extension

2.5.2 Download object

Get presigned

curl -X 'GET' 
  'https://marketplace-api-user.dev.devsaitech.com/api/v1/ai-resource/presigned-download?s3Key=0029781988958338526%2F8a19ac42-7ace-463e-9e0f-a70f165964fb-file_name.extension' 
  -H 'accept: application/json' 
  -H 'x-publisher-key: UK24XC7qjGpeUqPo69tL6fxyt4cSXvmwZ7sYFu4nH7mKjXZyHeyHXTMVtup48hSf'

Response

{
  "code": 0,
  "message": "success",
  "data": {
    "url": "https://dev-aitech-marketplace-blob-storage.s3.ap-southeast-1.amazonaws.com/0029781988958338526/8a19ac42-7ace-463e-9e0f-a70f165964fb-file_name.extension?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIAVRUVRKXLOODYDPP7%2F20240620%2Fap-southeast-1%2Fs3%2Faws4_request&X-Amz-Date=20240620T103611Z&X-Amz-Expires=3600&X-Amz-Signature=9442371e55e2c8ca35e14112595c8e008412f4434877b9fe739e79c026813cea&X-Amz-SignedHeaders=host&x-id=GetObject"
  }
}

Use url in response to download object

2.5.3 Delete object

curl -X 'POST' 
  'https://marketplace-api-user.dev.devsaitech.com/api/v1/ai-resource/delete' 
  -H 'accept: application/json' 
  -H 'x-publisher-key: UK24XC7qjGpeUqPo69tL6fxyt4cSXvmwZ7sYFu4nH7mKjXZyHeyHXTMVtup48hSf' 
  -H 'Content-Type: application/json' 
  -d '{
  "s3Key": "0029781988958338526/8a19ac42-7ace-463e-9e0f-a70f165964fb-file_name.extension"
}'

2.6 Error Code

All errors related to the API protocol will return an HTTP status code. However, system-related errors such as exceeding size limits or incorrect parameter content will be returned based on the defined error codes below.

Note that: The error code represents a group of errors, so developers need to describe specific errors in the "reason" field to facilitate easier debugging in case of errors.

XXX: Write the abbreviation of the service name. For example FaceDetection will be FD

3. Design database

The database is used to store information about requests to facilitate reporting (via the /stats API).

Stack: Developers can use databases like MongoDB, MySQL, or others for this purpose. However, the required fields of information to be met are as shown in the table below.

4. Appendix