The Service object defines a logical unit within the AI Gateway that represents either a direct backend connection (endpoint) or a reference to an Upstream configuration. This abstraction allows for flexible routing and credential management strategies depending on the nature of the service.

Each Service is associated with a specific Gateway and can include additional configuration such as retries, tags, and optional metadata.

​

Fields Reference

​

Top-Level Fields

• name (string) A unique name for the service within the context of the gateway. Used for logging, routing, and identification.

• type (string) Indicates how this service routes traffic. Valid values are:

  • upstream: Delegates routing to a defined Upstream configuration.
  • endpoint: Connects directly to a host and port.

• description (string) Optional human-readable description of the service’s purpose or scope.

• tags (array of strings) Optional metadata tags to support organization, filtering, and categorization.

• upstream_id (string) References the id of an existing Upstream object.

​

Direct Endpoint Configuration

If the type is set to endpoint, the service connects directly to a backend without relying on an upstream definition.

• host (string) The hostname or IP address of the backend.

• port (number) The TCP port used to communicate with the backend service.

• protocol (string) The protocol used to communicate (http, https, grpc, etc).

• path (string) Optional path to append to the base URL.

• headers (object) Optional key-value pairs representing custom headers to include in requests to the endpoint.

• credentials (object) Authentication and parameter settings for accessing the endpoint. See Credentials below.

​

Common Settings

• retries (number) Specifies how many retry attempts should be made if the request fails. Applies to both upstream and direct endpoint types.

​

Credentials

The credentials field supports authentication with backend services.

• See the Upstream Credentials section for a complete breakdown of supported fields such as:
• aws_access_key_id, aws_secret_access_key
• azure_client_id, azure_client_secret, etc.
• gcp_service_account_json, header_name, param_name, etc.

​

Usage Notes

​

Upstream vs. Endpoint

• Use type: "upstream" when centralizing routing logic across multiple services via an upstream object.
• Use type: "endpoint" when the service talks directly to a backend with its own host, port, and credentials.

​

Tags & Organization

• Leverage tags to filter, search, or categorize services for large-scale gateway setups.