The Service object defines a logical unit within the AI Gateway that represents either a direct backend connection (endpoint) or a reference to an Upstream configuration. This abstraction allows for flexible routing and credential management strategies depending on the nature of the service.

Each Service is associated with a specific Gateway and can include additional configuration such as retries, tags, and optional metadata.

​
Fields Reference

​
Top-Level Fields

  • name (string) A unique name for the service within the context of the gateway. Used for logging, routing, and identification.

  • type (string) Indicates how this service routes traffic. Valid values are:

    • upstream: Delegates routing to a defined Upstream configuration.
    • endpoint: Connects directly to a host and port.

  • description (string) Optional human-readable description of the service’s purpose or scope.

  • tags (array of strings) Optional metadata tags to support organization, filtering, and categorization.

  • upstream_id (string) References the id of an existing Upstream object.

​
Direct Endpoint Configuration

If the type is set to endpoint, the service connects directly to a backend without relying on an upstream definition.

  • host (string) The hostname or IP address of the backend.

  • port (number) The TCP port used to communicate with the backend service.

  • protocol (string) The protocol used to communicate (http, https, grpc, etc).

  • path (string) Optional path to append to the base URL.

  • headers (object) Optional key-value pairs representing custom headers to include in requests to the endpoint.

  • credentials (object) Authentication and parameter settings for accessing the endpoint. See Credentials below.

​
Common Settings

  • retries (number) Specifies how many retry attempts should be made if the request fails. Applies to both upstream and direct endpoint types.

​
Credentials

The credentials field supports authentication with backend services.

  • See the Upstream Credentials section for a complete breakdown of supported fields such as:
  • aws_access_key_id, aws_secret_access_key
  • azure_client_id, azure_client_secret, etc.
  • gcp_service_account_json, header_name, param_name, etc.

​
Usage Notes

​
Upstream vs. Endpoint

  • Use type: "upstream" when centralizing routing logic across multiple services via an upstream object.
  • Use type: "endpoint" when the service talks directly to a backend with its own host, port, and credentials.

​
Tags & Organization

  • Leverage tags to filter, search, or categorize services for large-scale gateway setups.