Arize AX - Portkey Docs

Arize AX is the full-featured observability and evaluation platform from Arize AI for production teams, AI-native companies, and enterprises. It is available as managed cloud or enterprise self-hosted deployment. Portkey is a production-grade AI Gateway and Observability platform for AI applications. The OpenInference Portkey instrumentation lets Arize AX capture Portkey-routed calls, including gateway requests and the underlying LLM completions.

Use Arize AX for production AI observability and evaluation. If you want an open-source path for local development, experimentation, or single-container self-hosting, use the separate Arize Phoenix integration.

Why Portkey + Arize AX?

Thanks to OpenInference instrumentation, Portkey can emit structured traces automatically. This gives you visibility into each LLM call routed through the gateway, making it easier to debug behavior, inspect token usage, and evaluate production traffic in Arize AX. For production evaluation patterns, see Arize’s agent evaluation guide and LLM evaluation guide.

AI Gateway Features

1600+ LLM Providers: Single API for OpenAI, Anthropic, AWS Bedrock, and more
Advanced Routing: Fallbacks, load balancing, conditional routing
Cost Optimization: Semantic caching and request controls
Security: PII detection, content filtering, compliance controls

Built-in Observability

40+ Key Metrics: Cost, latency, tokens, error rates
Detailed Logs & Traces: Request/response bodies and custom tracing
Custom Metadata: Attach custom metadata to your requests
Custom Alerts: Real-time monitoring and notifications

With this integration, you can route LLM traffic through Portkey and gain deep observability in Arize AX, bringing together gateway orchestration and ML observability.

Getting Started

Prerequisites

Python 3.10+
An Arize AX account
A Portkey API key
An OpenAI API key, or another provider key routed through Portkey

Step 1: Install dependencies

Install the required packages for Arize AX and Portkey:

pip install arize-otel openinference-instrumentation-portkey portkey-ai openai

Step 2: Configure credentials

Set the Arize AX and Portkey credentials in the same shell that runs your application:

export ARIZE_SPACE_ID="your-space-id"
export ARIZE_API_KEY="your-arize-api-key"
export ARIZE_PROJECT_NAME="portkey-gateway"
export PORTKEY_API_KEY="your-portkey-api-key"
export OPENAI_API_KEY="your-openai-api-key"

Step 3: Register Arize AX and instrument Portkey

Configure Arize AX

First, set up the Arize AX OpenTelemetry configuration:

import os
from arize.otel import register
from openinference.instrumentation.portkey import PortkeyInstrumentor

tracer_provider = register(
    space_id=os.environ["ARIZE_SPACE_ID"],
    api_key=os.environ["ARIZE_API_KEY"],
    project_name=os.environ["ARIZE_PROJECT_NAME"],
)

PortkeyInstrumentor().instrument(tracer_provider=tracer_provider)

Configure Portkey AI Gateway

Set up Portkey in the same process after registering the PortkeyInstrumentor:

import os
from portkey_ai import Portkey

portkey = Portkey(
    api_key=os.environ["PORTKEY_API_KEY"],
    provider="openai",
    Authorization=f"Bearer {os.environ['OPENAI_API_KEY']}",
)

response = portkey.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Explain machine learning"}]
)

print(response.choices[0].message.content)

Complete Integration Example

Here’s a complete working example that connects Portkey’s AI Gateway with Arize AX for centralized monitoring.

import os
from arize.otel import register
from openinference.instrumentation.portkey import PortkeyInstrumentor

tracer_provider = register(
    space_id=os.environ["ARIZE_SPACE_ID"],
    api_key=os.environ["ARIZE_API_KEY"],
    project_name=os.getenv("ARIZE_PROJECT_NAME", "portkey-production"),
)

PortkeyInstrumentor().instrument(tracer_provider=tracer_provider)

# Import Portkey after instrumentation is registered.
from portkey_ai import Portkey

portkey = Portkey(
    api_key=os.environ["PORTKEY_API_KEY"],
    provider="openai",
    Authorization=f"Bearer {os.environ['OPENAI_API_KEY']}",
)

response = portkey.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "user", "content": "Explain quantum computing in simple terms."}
    ],
)

print(response.choices[0].message.content)

Verify in Arize AX

Open your Arize AX space and select the project configured by ARIZE_PROJECT_NAME. You should see a trace for the Portkey request, including the LLM span, prompt, response, token usage, model, and provider metadata.

Arize AX Portkey Tracing Guide

See the Arize-side Portkey tracing setup, verification steps, and troubleshooting guidance.

Cookbook on Portkey x Arize

Learn how to use Portkey’s Universal API to orchestrate multiple LLMs in a structured debate while tracking performance and evaluating outputs with Arize.

Portkey AI Gateway Features

While Arize AX provides observability and evaluation, Portkey delivers a complete AI infrastructure platform. Here’s everything you get with Portkey:

🚀 Core Gateway Capabilities

1600+ LLM Providers

Access OpenAI, Anthropic, Google, Cohere, Mistral, Llama, and 1600+ models through a single unified API. No more managing different SDKs or endpoints.

Universal API

Use the same code to call any LLM provider. Switch between models and providers without changing your application code.

LLM Integrations

Secure vault for API keys with budget limits, rate limiting, and access controls. Never expose raw API keys in your code.

Advanced Configs

Define routing strategies, model parameters, and reliability settings in reusable configurations. Version control your AI infrastructure.

🛡️ Reliability & Performance

Smart Fallbacks

Automatically switch to backup providers when primary fails. Define fallback chains across multiple providers.

Load Balancing

Distribute requests across multiple API keys or providers based on custom weights and strategies.

Automatic Retries

Configurable retry logic with exponential backoff for transient failures and rate limits.

Request Timeouts

Set custom timeouts to prevent hanging requests and improve application responsiveness.

Conditional Routing

Route requests to different models based on content, metadata, or custom conditions.

Canary Testing

Gradually roll out new models or providers with percentage-based traffic splitting.

💰 Cost Optimization

Semantic Caching

Intelligent caching that understands semantic similarity. Reduce costs by up to 90% on repeated queries.

Budget Limits

Set spending limits per provider, team, or project. Get alerts before hitting limits.

Rate Limits

Set rate limits per provider, team, or project.

Cost Analytics

Real-time cost tracking across all providers with detailed breakdowns by model, user, and feature.

📊 Built-in Observability

Comprehensive Metrics

Track 40+ metrics including latency, tokens, costs, cache hits, error rates, and more in real-time.

Detailed Logs

Full request/response logging with advanced filtering, search, and export capabilities.

Distributed Tracing

Trace requests across your entire AI pipeline with correlation IDs and custom metadata.

Custom Alerts

Set up alerts on any metric with webhook, email, or Slack notifications.

🔒 Security & Compliance

PII Detection

Automatically detect and redact sensitive information like SSN, credit cards, and personal data.

Content Filtering

Block harmful, toxic, or inappropriate content in real-time based on custom policies.

Access Controls

Fine-grained RBAC with team management, user permissions, and audit logs.

SOC2 Compliance

Enterprise-grade security with SOC2 Type II certification and GDPR compliance.

Audit Logs

Complete audit trail of all API usage, configuration changes, and user actions.

Data Privacy

Zero data retention options and deployment in your own VPC for maximum privacy.

🏢 Enterprise Features

SSO Integration

SAML 2.0 support for Okta, Azure AD, Google Workspace, and custom IdPs.

Organization Management

Multi-workspace support with hierarchical teams and department-level controls.

SLA Guarantees

99.9% uptime SLA with dedicated support and custom deployment options.

Private Deployments

Deploy Portkey in your own AWS, Azure, or GCP environment with full control.

Next Steps

Explore Portkey Features

Discover all AI Gateway capabilities beyond observability

LLM Integrations

Secure your API keys and set budgets

Advanced Routing

Configure fallbacks, load balancing, and more

Built-in Analytics

Use Portkey’s native observability features

Need help? Join our Discord community

​Why Portkey + Arize AX?

AI Gateway Features

Built-in Observability

​Getting Started

​Prerequisites

​Step 1: Install dependencies

​Step 2: Configure credentials

​Step 3: Register Arize AX and instrument Portkey

​Complete Integration Example

​Verify in Arize AX

Arize AX Portkey Tracing Guide

Cookbook on Portkey x Arize

​Portkey AI Gateway Features

​🚀 Core Gateway Capabilities

1600+ LLM Providers

Universal API

LLM Integrations

Advanced Configs

​🛡️ Reliability & Performance

Smart Fallbacks

Load Balancing

Automatic Retries

Request Timeouts

Conditional Routing

Canary Testing

​💰 Cost Optimization

Semantic Caching

Budget Limits

Rate Limits

Cost Analytics

​📊 Built-in Observability

Comprehensive Metrics

Detailed Logs

Distributed Tracing

Custom Alerts

​🔒 Security & Compliance

PII Detection

Content Filtering

Access Controls

SOC2 Compliance

Audit Logs

Data Privacy

​🏢 Enterprise Features

SSO Integration

Organization Management

SLA Guarantees

Private Deployments

​Next Steps

Explore Portkey Features

LLM Integrations

Advanced Routing

Built-in Analytics

Portkey is now PRISMA AIRS AI Gateway. See it in action.

Why Portkey + Arize AX?

Getting Started

Prerequisites

Step 1: Install dependencies

Step 2: Configure credentials

Step 3: Register Arize AX and instrument Portkey

Complete Integration Example

Verify in Arize AX

Portkey AI Gateway Features

🚀 Core Gateway Capabilities

🛡️ Reliability & Performance

💰 Cost Optimization

📊 Built-in Observability

🔒 Security & Compliance

🏢 Enterprise Features

Next Steps