The Service object defines a logical unit within the AI Gateway that represents either a direct backend connection (endpoint) or a reference to an Upstream configuration. This abstraction allows for flexible routing and credential management strategies depending on the nature of the service.
Each Service is associated with a specific Gateway and can include additional configuration such as retries, tags, and optional metadata.
name (string
)
A unique name for the service within the context of the gateway. Used for logging, routing, and identification.
type (string
)
Indicates how this service routes traffic. Valid values are:
upstream
: Delegates routing to a defined Upstream configuration.endpoint
: Connects directly to a host and port.description (string
)
Optional human-readable description of the service’s purpose or scope.
tags (array of strings
)
Optional metadata tags to support organization, filtering, and categorization.
upstream_id (string
)
References the id
of an existing Upstream object.
If the type
is set to endpoint
, the service connects directly to a backend without relying on an upstream definition.
host (string
)
The hostname or IP address of the backend.
port (number
)
The TCP port used to communicate with the backend service.
protocol (string
)
The protocol used to communicate (http
, https
, grpc
, etc).
path (string
)
Optional path to append to the base URL.
headers (object
)
Optional key-value pairs representing custom headers to include in requests to the endpoint.
credentials (object
)
Authentication and parameter settings for accessing the endpoint. See Credentials below.
number
)
Specifies how many retry attempts should be made if the request fails. Applies to both upstream and direct endpoint types.The credentials
field supports authentication with backend services.
aws_access_key_id
, aws_secret_access_key
azure_client_id
, azure_client_secret
, etc.gcp_service_account_json
, header_name
, param_name
, etc.type: "upstream"
when centralizing routing logic across multiple services via an upstream object.type: "endpoint"
when the service talks directly to a backend with its own host, port, and credentials.tags
to filter, search, or categorize services for large-scale gateway setups.The Service object defines a logical unit within the AI Gateway that represents either a direct backend connection (endpoint) or a reference to an Upstream configuration. This abstraction allows for flexible routing and credential management strategies depending on the nature of the service.
Each Service is associated with a specific Gateway and can include additional configuration such as retries, tags, and optional metadata.
name (string
)
A unique name for the service within the context of the gateway. Used for logging, routing, and identification.
type (string
)
Indicates how this service routes traffic. Valid values are:
upstream
: Delegates routing to a defined Upstream configuration.endpoint
: Connects directly to a host and port.description (string
)
Optional human-readable description of the service’s purpose or scope.
tags (array of strings
)
Optional metadata tags to support organization, filtering, and categorization.
upstream_id (string
)
References the id
of an existing Upstream object.
If the type
is set to endpoint
, the service connects directly to a backend without relying on an upstream definition.
host (string
)
The hostname or IP address of the backend.
port (number
)
The TCP port used to communicate with the backend service.
protocol (string
)
The protocol used to communicate (http
, https
, grpc
, etc).
path (string
)
Optional path to append to the base URL.
headers (object
)
Optional key-value pairs representing custom headers to include in requests to the endpoint.
credentials (object
)
Authentication and parameter settings for accessing the endpoint. See Credentials below.
number
)
Specifies how many retry attempts should be made if the request fails. Applies to both upstream and direct endpoint types.The credentials
field supports authentication with backend services.
aws_access_key_id
, aws_secret_access_key
azure_client_id
, azure_client_secret
, etc.gcp_service_account_json
, header_name
, param_name
, etc.type: "upstream"
when centralizing routing logic across multiple services via an upstream object.type: "endpoint"
when the service talks directly to a backend with its own host, port, and credentials.tags
to filter, search, or categorize services for large-scale gateway setups.