1. Background
In the cloud environment, it's very important to securely protect sensitive APIs from unauthorized access and potential security threats originating from the public internet. Azure Function App provide Access Restriction and Private Endpoint feature to safeguard your Function App from unauthorized inbound requests.
In order to safeguard the Function App, one customer enable the private endpoint in Function App and implement the Event Grid trigger function, sending a message to Event Grid when the blog storage file changes and bring up the Azure Function. However, the Ip Forbidden (403) issue occurred from Azure Function.
2. The IP Forbidden (403) issue from Event Grid trigger function
Assessment:
1.Function App aspect
There is Ip Forbidden (403) issue in Front End. The Event Grid's IP (20.xxx.xxx.130) is denied.
- The IP (20.xxx.xxx.130) is from Event Grid. (MicrosoftIPs (csstoolkit.azurewebsites.net)
- The Function App didn’t have Access Restriction configured, but the Private Endpoint is enabled.
2.From Event Grid aspect
There is a limitation for Event Grid which is unable to support virtual network (VNet) integration for outbound traffic. With push delivery in Event Grid, your application cannot receive events over private IP address. (Referring to the document https://learn.microsoft.com/en-us/azure/event-grid/consume-private-endpoints)
Investigation Result:
Therefore, it's a well-known limitation that the Event Grid trigger function would encounter Ip Forbidden (403) issue when the private endpoint being enabled in Function App.
3. How to safeguard the Event Grid trigger function?
There are two options as a workaround:
3.1 Enable Access Restriction
Enable Access Restriction in Function App and allow the source from Event Grid's service tag (AzureEventGrid).
3.2 Enable Private Endpoint and configure the intermediate components
Enable Private Endpoint in Function App and configure the intermediate components between Event Grid and Function App.
Under above configuration, we can make sure entire traffic is secured. (Referring to the document https://learn.microsoft.com/en-us/azure/event-grid/consume-private-endpoints#use-managed-identity)
The First traffic: Event Grid -> Intermediate Component, using Managed identity for secured.
The secured traffic from Event Grid to an intermediate component such as Event Hubs, Service Bus, or Azure Storage, stays on the Microsoft backbone and a managed identity of Event Grid is used.
The Second traffic: Intermediate Component -> Function App, using Private Link Service for secured.
Configuring your Azure Function from within your virtual network to use an intermediate component such as Event Hubs, Service Bus, or Azure Storage via private link ensures the traffic between those services and your function stays within your virtual network perimeter.
3.2.1 How to deliver events from Event Grid to the intermediate component?
For example:
- Deliver events from Event Grid to Service Bus Queue using managed identity, please refer to the following steps for setup:
3.2.1.1 Event Grid Part
Step#1: Create an Event Grid topic with system-assigned or user-assigned managed identity.
If there is existing Event Grid topic, just assign a system-assigned or user-assigned managed identity.
Step#2: Configure the event subscription that uses a Service Bus queue or topic as an endpoint to use the system-assigned or user-assigned managed identity.
3.2.1.2 Managed Identity Part
Add the Managed Identity to the Azure Service Bus Data Sender role on the Service Bus namespace.
3.2.1.3 Service Bus Part
Enable the Allow trusted Microsoft services to bypass this firewall setting on your Service Bus namespace.
4. Reference
https://learn.microsoft.com/en-us/azure/event-grid/consume-private-endpoints