Deploying an Event-Driven-Job with Azure Container App Job and Azure Service Bus

[karthi]

Azure Container Apps jobs let you perform specific tasks inside containers that start, run for a short time, and then stop. You can make these tasks happen whenever you want, following a schedule, or when something specific occurs. These jobs work well for things like dealing with data, teaching computers, or situations where you need quick computing power without leaving a trace.

This blog helps you in deploying an event driven job using Azure Container apps and Azure Service Bus. In this example, the trigger will be a message in Service bus Queue and the Azure container app Job reads the message present in the queue and prints the message in container app console log.

Below are the steps to be performed:

• Create a Container Apps environment to deploy your container apps.
• Create an Azure Service Bus to send messages to the container app.
• Build a container image that runs a job.
• Deploy the job to the Container Apps environment.
• Verify that the Service Bus messages are processed by the container app.

The job you create starts an execution for each message that is sent to an Azure Service Bus Queue. Each job execution runs a container that performs the following steps:

• Dequeues one message from the Service Bus queue.
• Logs the message to the job execution logs.
• Deletes the message from the Service Bus queue.
• Job completes its one execution.

If there is an existing Container App Environment, you can use the same or you can create a new environment by following the steps outlined here.

Build and deploy the container app job:

First ensure to login to Azure CLI and set the subscription, you can use PowerShell/CMD/Cloud shell on Azure portal.

Command to login to Azure CLI

az login

Command to set the subscription: Replace the empty quotes with your subscription.

az account set --subscription ""

To deploy the job, you must first build a container image for the job and push it to any container registry. In this blog post, we will be using Azure Container Registry.

Define a name for your container image and registry.

CONTAINER_IMAGE_NAME="servicebus-reader-job:1.0"
CONTAINER_REGISTRY_NAME="<CONTAINER_REGISTRY_NAME>"

Replace <CONTAINER_REGISTRY_NAME> with a unique name for your container registry. Container registry names must be unique within Azure and should be between 5 and 50 characters in length, containing only numbers and lowercase letters.

Create an Azure Container Registry:

az acr create \
    --name "$CONTAINER_REGISTRY_NAME"\
    --resource-group "$RESOURCE_GROUP"\
    --location "$LOCATION"\
    --sku Basic \
    --admin-enabled true

Explanation: This command creates an Azure Container Registry (ACR) with the specified name, resource group and location. The --sku Basic specifies the pricing tier for the registry, and --admin-enabled true enables admin access to the registry.

The source code for the job is available on GitHub. Run the following command to clone the repository and build the container image in the cloud using the az acr build command.

az acr build \
    --registry"$CONTAINER_REGISTRY_NAME"\
    --image"$CONTAINER_IMAGE_NAME"\
    "https://github.com/karthimalyala/ServiceBusEventDriven.git"

Explanation: This command builds a container image from the specified GitHub repository and pushes it to the Azure Container Registry ($CONTAINER_REGISTRY_NAME). The image will have the name and version specified in $CONTAINER_IMAGE_NAME. The image can later be used to deploy the container app job. 

Create a job in the Container Apps environment.

Command to create a job in the specified resource group and environment:

az containerapp job create \
    --name "$JOB_NAME"\
    --resource-group "$RESOURCE_GROUP"\
    --environment "$ENVIRONMENT"\
    --trigger-type "Event"\
    --replica-timeout "1800"\
    --replica-retry-limit "1"\
    --replica-completion-count "1"\
    --parallelism "1"\
    --min-executions "0"\
    --max-executions "10"\
    --polling-interval "60"\
    --scale-rule-name "queue"\
    --scale-rule-type "azure-servicebus"\
    --scale-rule-metadata "queueName=xxxxx" "namespace=xxxxxx" "messageCount=1"\
    --scale-rule-auth "connection=connection-string-secret"\
    --image "$CONTAINER_REGISTRY_NAME.azurecr.io/$CONTAINER_IMAGE_NAME"\
    --cpu "0.5"\
    --memory "1Gi"\
    --secrets "connection-string-secret=$QUEUE_CONNECTION_STRING"\
    --registry-server "$CONTAINER_REGISTRY_NAME.azurecr.io"\
    --env-vars "AZURE_STORAGE_QUEUE_NAME=$QUEUE_NAME" "AZURE_SERVICE_BUS_CONNECTION_STRING=secretref:connection

Explanation: This command creates a job within the Azure Container Apps environment. The job is configured to trigger when a message arrives in Azure Service Bus. To know more about the parameters used, refer to this document.

In the above CLI command -

1. --scale-rule-type is set to azure-service bus which is a KEDA based scaler. For more details, refer here
2. QUEUE_NAME is the queue name you created in your Azure Service Bus.
3. QUEUE_CONNECTION_STRING is the connection string for your Storage Account.
4. --scale-rule-metadata: the queueName is the Azure Service Bus queue name and the namespace is the Azure Service Bus name.

Verify the deployment:

To verify that the job was configured correctly, you can send some messages to the Azure Service Bus, confirm that job executions are started, and check if the messages are logged in the job execution logs.

Send a message to the queue and Peek from start in Azure Service Bus:

List the executions of a job:

You can refer to the Execution History of the Azure Container App job, we can notice that the job execution has started.

You can also verify the logs from the Log Analytics workspace, which will display the processing of the Azure Service Bus messages by the Azure Container App job, as shown below:

Thank you for accompanying us on this journey through the world of event-driven processing with Azure Service Bus and Container App Jobs. We trust this guide has not only expanded your knowledge but also empowered you to harness the capabilities of these technologies in your projects.

Should you have any inquiries, insights to share, or suggestions for improvements, please don't hesitate to reach out in the comments section below. We value your feedback and are committed to refining our content based on your input.

Happy learning and coding, and may your innovative endeavors continue to flourish in the dynamic realm of technology!