Multi-Provider AI Architecture with Redis Cache | Azure OpenAI + OpenAI

As the field of artificial intelligence (AI) continues to evolve, businesses are increasingly looking for flexible and scalable solutions that can integrate seamlessly into their existing infrastructure. This blog post explores how to design a multi-provider AI architecture using Redis cache as an intermediary between your application and different AI service providers like Azure OpenAI and OpenAI. We’ll delve into the technical details, including code examples and step-by-step implementation guides.

📹 Watch the Video Tutorial

Watch the complete walkthrough in the video above for step-by-step guidance.

🎯 Introduction to Multi-Provider AI Architecture

A multi-provider AI architecture allows your application to leverage multiple AI services based on specific use cases or requirements. This approach provides flexibility in terms of cost, performance, and feature availability. By using Redis cache as an intermediary, you can improve the responsiveness and reliability of your AI-powered applications.

Redis, a popular in-memory data structure store, serves as an effective caching layer that reduces latency by storing frequently accessed data close to the application. This is particularly useful for AI models that require high throughput and low response times.

Setting Up Redis Cache

To get started with Redis cache, you’ll need to set up a Redis instance. Depending on your environment, you can choose from various options such as a local Redis server, a managed Redis service on cloud platforms like Azure or AWS, or a Redis-as-a-Service solution.

For this example, we’ll use the popular Redis open-source version. Here’s how you can install and configure it:

# Install Redis
sudo apt-get update
sudo apt-get install redis-server

# Start Redis server
sudo systemctl start redis-server

# Enable Redis to start on boot
sudo systemctl enable redis-server

Once Redis is installed, you can configure it according to your needs. For example, setting a maximum memory limit or configuring persistence options.

Integrating with Azure OpenAI and OpenAI

In this section, we’ll discuss how to integrate your application with both Azure OpenAI and OpenAI services using Redis cache as an intermediary. We’ll focus on the implementation steps and code examples for each provider.

Azure OpenAI Integration

Azure OpenAI provides a range of AI models and APIs that can be integrated into your applications. To integrate with Azure OpenAI, you’ll need to set up an Azure account, create an OpenAI resource, and obtain the API key.

Here’s a simple example of how to use the Azure OpenAI API in C#:

using System;
using System.Net.Http;
using System.Text.Json;
using System.Threading.Tasks;


public class AzureOpenAIClient
{
    private readonly HttpClient _httpClient;
    private readonly string _apiKey;

public AzureOpenAIClient(string apiKey)
    {
        _httpClient = new HttpClient();
        _apiKey = apiKey;
    }

public async Task GenerateTextAsync(string prompt)
    {
        var request = new HttpRequestMessage(HttpMethod.Post, "https://api.openai.com/v1/engines/davinci-codex/completions");
        request.Headers.Add("Authorization", "Bearer " + _apiKey);
        request.Content = new StringContent(JsonSerializer.Serialize(new
        {
            prompt,
            max_tokens = 50
        }), System.Text.Encoding.UTF8, "application/json");

var response = await _httpClient.SendAsync(request);
        if (!response.IsSuccessStatusCode)
        {
            throw new Exception("Error calling Azure OpenAI API");
        }

var result = await response.Content.ReadAsStringAsync();
        var completion = JsonSerializer.Deserialize(result);
        return completion.Choices[0].Text;
    }
}

OpenAI Integration

OpenAI offers a similar range of AI models and APIs. To integrate with OpenAI, you’ll need to sign up for an account, create an API key, and follow the documentation provided.

Here’s a basic example of how to use the OpenAI API in C#:

using System;
using System.Net.Http;
using System.Text.Json;
using System.Threading.Tasks;


public class OpenAIClient
{
    private readonly HttpClient _httpClient;
    private readonly string _apiKey;

public OpenAIClient(string apiKey)
    {
        _httpClient = new HttpClient();
        _apiKey = apiKey;
    }

public async Task GenerateTextAsync(string prompt)
    {
        var request = new HttpRequestMessage(HttpMethod.Post, "https://api.openai.com/v1/engines/davinci-codex/completions");
        request.Headers.Add("Authorization", "Bearer " + _apiKey);
        request.Content = new StringContent(JsonSerializer.Serialize(new
        {
            prompt,
            max_tokens = 50
        }), System.Text.Encoding.UTF8, "application/json");

var response = await _httpClient.SendAsync(request);
        if (!response.IsSuccessStatusCode)
        {
            throw new Exception("Error calling OpenAI API");
        }

var result = await response.Content.ReadAsStringAsync();
        var completion = JsonSerializer.Deserialize(result);
        return completion.Choices[0].Text;
    }
}

Using Redis as a Cache Layer

Now that we have integrated with both Azure OpenAI and OpenAI, let’s explore how to use Redis as a cache layer. The idea is to store the results of AI model predictions in Redis, so subsequent requests for the same input can be served from the cache instead of hitting the AI service again.

Here’s an example of how you can implement this using C#:

using StackExchange.Redis;
using System.Threading.Tasks;


public class RedisCacheService
{
    private readonly ConnectionMultiplexer _redisConnection;
    private readonly IDatabase _db;
Related: Implementing the Circuit Breaker Pattern in .NET Core ...


public RedisCacheService(string connectionString)
    {
        _redisConnection = ConnectionMultiplexer.Connect(connectionString);
        _db = _redisConnection.GetDatabase();
    }

public async Task GetCachedResponseAsync(string key)
    {
        return await _db.StringGetAsync(key);
    }

public async Task SetCachedResponseAsync(string key, string value, TimeSpan expiry)
    {
        await _db.StringSetAsync(key, value, expiry);
    }
}

With this setup, you can modify your AI client classes to first check the Redis cache before making a call to the AI service. If the response is found in the cache, it’s returned immediately; otherwise, the AI service is called, and the result is stored in the cache for future requests.

✨ Benefits and Challenges

A multi-provider AI architecture with Redis cache offers several benefits:

• Flexibility: Easily switch between different AI providers based on requirements.
• Scalability: Handle a high volume of requests with reduced latency using caching.
• Cost-effectiveness: Optimize costs by using the most suitable provider for each use case.