NeuralTrust | The leading security platform for generative AI

After installing AI Gateway, the next step is to create your first gateway instance. This guide will walk you through creating and configuring a basic gateway.

Step 1: Create a Gateway

Set up your first gateway instance with rate limiting configuration

Use the Admin API to create your first gateway:

# Create a gateway
curl -X POST http://localhost:8080/api/v1/gateways \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer jwt example" \
  -d '{
    "name": "my-first-gateway",
    "subdomain": "api",
    "required_plugins": [
        {
            "name": "rate_limiter",
            "enabled": true,
            "stage": "pre_request",
            "priority": 1,
            "settings": {
                "limits": {
                    "global": {
                        "limit": 15,
                        "window": "1m"
                    },
                    "per_ip": {
                        "limit": 5,
                        "window": "1m"
                    },
                    "per_user": {
                        "limit": 5,
                        "window": "1m"
                    }
                },
                "actions": {
                    "type": "reject",
                    "retry_after": "60"
                }
            }
        }
    ]
  }'

Verify Gateway Creation

Check if your gateway was created successfully:

# Get gateway details
curl http://localhost:8080/api/v1/gateways/{gateway-id} \
  -H "Authorization: Bearer jwt example"

# List all gateways
curl http://localhost:8080/api/v1/gateways \
  -H "Authorization: Bearer jwt example"

Step 2: Create an API Key

Generate an API key for authentication and access control

# Create a test API key
curl -X POST http://localhost:8080/api/v1/gateways/{gateway-id}/keys \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer jwt example" \
  -d '{
    "name": "test-key",
    "expires_at": "2027-12-31T23:59:59Z"
  }'

Step 3: Create The Upstream

Configure upstream providers like OpenAI and Anthropic with load balancing

curl -X POST http://localhost:8080/api/v1/gateways/{gateway-id}/upstreams \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer jwt example" \
  -d '{
    "name": "ai-providers-upstream",
    "algorithm": "round-robin",
    "targets": [
      {
        "path": "/v1/chat/completions",
        "provider": "openai",
        "weight": 50,
        "priority": 1,
        "default_model": "gpt-4o-mini",
        "models": ["gpt-3.5-turbo", "gpt-4", "gpt-4o-mini"],
        "credentials": {
          "header_name": "Authorization",
          "header_value": "Bearer your-openai-key"
        }
      },
      {
        "path": "/v1/messages",
        "provider": "anthropic",
        "weight": 50,
        "priority": 1,
        "default_model": "claude-3-5-sonnet-20241022",
        "models": ["claude-3-5-sonnet-20241022"],
        "headers": {
          "anthropic-version": "2023-06-01"
        },
        "credentials": {
          "header_name": "x-api-key",
          "header_value": "your-anthropic-key"
        }
      }
    ],
    "health_checks": {
      "passive": true,
      "threshold": 3,
      "interval": 60
    }
  }'

Verify Configuration

Check that your upstream is properly configured:

curl http://localhost:8080/api/v1/gateways/{gateway-id}/upstreams/{upstream-id} \
  -H "Authorization: Bearer jwt example"

Step 4: Create the Service

Set up a service to handle routing and manage upstream connections

Create your service using the Admin API:

curl -X POST http://localhost:8080/api/v1/gateways/{gateway-id}/services \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer jwt example" \
  -d '{
    "name": "my-service",
    "upstream_id": "{upstream-id}",
    "type": "upstream",
    "description": "My Service"
  }'

Verify Configuration

Check that your service is properly configured:

curl http://localhost:8080/api/v1/gateways/{gateway-id}/services/{service-id} \
  -H "Authorization: Bearer jwt example"

Step 5: Create a Rule

Define routing rules to direct traffic to your service

Create your first rule using the Admin API:

# Exact matching
curl -X POST http://localhost:8080/api/v1/gateways/{gateway-id}/rules \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer jwt example" \
  -d '{
    "service_id": "{service-id}",
    "path": "/test",
    "methods": ["POST"],
    "strip_path": false,
    "preserve_host": false
  }'

Verify Rules

Check that your rules are properly configured:

curl http://localhost:8080/api/v1/gateways/{gateway-id}/rules \
  -H "Authorization: Bearer jwt example"

Step 6: Test the Rules

Verify your configuration by making test requests to the gateway

Test your rule configurations:API Key: You need to set the x-api-key header to the API key of your gateway, that you created in the Step 2.

# Test a route
curl -X POST http://localhost:8080/test \
  -H "Content-Type: application/json" \
  -H "X-TG-API-Key: your-key" \
  -H "Authorization: Bearer jwt example" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [{"role": "user", "content": "Hello"}],
    "stream": true,
    "stream_options": {
      "include_usage": true
    },
    "max_tokens": 1024
  }'

Next steps:

Now that you have configured your rules, read more about:

Learn about Load Balancing
Explore Rate Limiting

Troubleshooting

Common issues and solutions:

Routing Issues

Verify path patterns
Check method restrictions
Review rule priorities

Path Handling

Check strip_path setting
Verify preserve_host
Test path transformations

Header Problems

Verify header requirements
Check header values
Test header matching

Getting Started

Core Concepts

Traffic Management

Non-REST Connectivity

Rate Limiting & Request Control

Content Security

Application Security

Server Security

Data masking

Extending Functionality

Observability & Monitoring

Benchmark

API Reference

Quickstart: Hello Gateway

Verify Gateway Creation

Verify Configuration

Verify Configuration

Verify Rules

Next steps:

Troubleshooting

Getting Started

Core Concepts

Traffic Management

Non-REST Connectivity

Rate Limiting & Request Control

Content Security

Application Security

Server Security

Data masking

Extending Functionality

Observability & Monitoring

Benchmark

API Reference

​Next steps:

​Troubleshooting

Next steps:

Troubleshooting