Sign in to confirm you’re not a bot
This helps protect our community. Learn more

Azure OpenAI scalability using API Management

Microsoft DevRadio
12.4K subscribers
<__slot-el>
<__slot-el>
6.6K views 1 year ago
This demo shows how Azure API Management when deployed in front of one or more OpenAI instances, can handle errors gracefully, implement retry logic or even switch backends for an improved user experience. You’ll also see how to improve security using Azure AD access tokens and create a chargeback policy using Event Hub as a central logging mechanism and Stream Analytics as an aggregator of Azure OpenAI consumed tokens. https://github.com/Prosenjitd-84/Azur... (This repository helps in exposing Azure OpenAI through an enterprise governance structure implemented using Azure API Management.) ...more
...more
Azure OpenAI scalability using API Management
69Likes
6,640Views
2023Jun 27
This demo shows how Azure API Management when deployed in front of one or more OpenAI instances, can handle errors gracefully, implement retry logic or even switch backends for an improved user experience. You’ll also see how to improve security using Azure AD access tokens and create a chargeback policy using Event Hub as a central logging mechanism and Stream Analytics as an aggregator of Azure OpenAI consumed tokens. https://github.com/Prosenjitd-84/Azur... (This repository helps in exposing Azure OpenAI through an enterprise governance structure implemented using Azure API Management.)

Follow along using the transcript.

Microsoft DevRadio

12.4K subscribers