Azure
Microsoft's public cloud offering.
Web Resources
Background
This article describes how to migrate all Azure Storage Queues data between two different storage accounts.
For this, we will use Azure Storage SDK for Python to copy all queues (and the respective data) from one Azure Storage Queue to another Azure Storage Queue. This approach will keep the data in the source queues, and will create new queues with the respective data in the destination Azure Storage Queue.
This script was developed and tested using the following versions but it is expected to work with previous versions:
Python 3.11.7
azure-identity (version:...
Background
This article describes how to copy all Azure Storage Queues data between two different storage accounts.
For this, we will use Azure Storage SDK for Python to copy all queues (and the respective data) from one Azure Storage Queue to another Azure Storage Queue. This approach will keep the data in the source queues, and will create new queues with the respective data in the destination Azure Storage Queue.
This script was developed and tested using the following versions but it is expected to work with previous versions:
Python 3.11.7
azure-identity (version:...
BlobFuse2 is a virtual file system driver available for Azure Blob Storage which helps accessing the containers/blobs on the Linux file system as a virtual file system. At present, we have Blobfuse2, which is great improvement over blobfuse1 and is generally available for all major Linux distributions.
In this article, we will talk about various common troubleshooting scenarios that are seen while using blobfuse2.
We assume that you have mounted blobfuse2 on your Linux VM. For blobfuse2 installation and mount process, you can refer to the below links:...
Assign an existing User-Assigned Managed Identity to VM from another subscription using Azure Policy
Azure provides a comprehensive list of built-in policy definitions (grouped by the category property defined in the metadata) which are owned and maintained by Microsoft (where the azure-policy repository contains the direct representation of these).
Built-in policy definitions usually cover a specific scenario although some flexibility might be provided through parameters that can be configured. However, customers might have specific needs not covered by an available built-in policy definition and a custom policy definition might be needed.
As a rule of thumb, if there is a built-in...
Connectivity from IaaS(VM) and PaaS service(ADF) to ADLS account with public network access disabled
In this blog post, we will investigate the impacts of disabling public network access in the storage account and learn how to establish connections to an Azure Data Lake Storage account from both IaaS (Virtual Machine) and PaaS services (Azure Data Factory), when public network access is disabled. Additionally, we will address common troubleshooting issues that may arise due to this configuration.
Introduction
Azure storage accounts provide a scalable and secure way to store and access data in the cloud. This model enables you to control the level of access to your storage accounts that...
On Feb 29th, 2024, the certificate feature in Azure Batch Account will be retired. To continue using the certificate with Batch task, user will need to migrate the Batch account certificates to Azure Key Vault. But since there is not an existing example provided, that might be a little difficult.
This blog will mainly focus on providing an example about how to recreate the Batch pool with required user assigned Managed Identity and Key Vault extension. Currently the Batch pool with user assigned Managed Identity and extension is only supported by ARM template and REST API call. Creating...
The blog talks about how to leverage Redis Insights GUI tool while working with Azure Cache for Redis. We will look at some of the option what will help us with some high-level connectivity troubleshooting and insights to our data present inside the cache.
To start with, we can leverage this for testing the connectivity to our Redis cache instance. After clicking on Add Redis Database button, we can fill in the other fields ahead:
Host: Complete FQDN or the completed Redis cache Endpoint
For Basic, Standard & Premium Tier - <Cachename>.redis.windows.net
For Enterprise Tier -...
With the impending retirement of the classic Cloud Service (CS) on August 31st, 2024, an increasing number of users have initiated the migration of their classic Cloud Service to Cloud Service Extended Support (CSES). To facilitate this transition, an official feature known as in-place migration has been introduced, enabling the seamless migration of classic CS to CSES without incurring any downtime. However, certain limitations exist, with the VM size used by the CS role being a notable factor. As per documentation, the A-series, encompassing Small, Medium, and Large VM sizes, is no...
The blog talks about how you can leverage the existing metrics and diagnostic logging to monitor or track the execution of lifecycle management policies.
To start, with the Lifecycle management, you will be mainly transitioning blobs from one tier to another or delete the blobs based on the specified rule configuration. So, either Set Blob Tier (REST API) - Azure Storage | Microsoft Learn or Delete Blob (REST API) - Azure Storage | Microsoft Learn will be called underneath respectively.
Now, the policy executes as part of backend scheduling and so the exact timings as to when the...
The blog aims to perform simple File storage REST API operations such as Create, Copy, Rename and Delete operations using CURL utility.Let’s look at some of the CURL command syntax to perform REST API operations and will be making use of SAS as the auth scheme.
We need to take care of the pointers below while performing the operations via CURL:
Ensure the URL is formed correctly as per the operation you are trying to perform.
The mandatory header needs to be passed and can be done using “-H” parameter.
Ensure you are appending/removing extra ‘?’ to the SAS token in the URLs...
Background:
This blog covers Permanent Delete of Soft deleted Snapshot and Versions in Azure storage without disabling Soft Delete option before the retention period ends.
Permanent Delete of Blob Snapshot and Version:
With version 2020-02-10 and later, you can permanently delete a soft-deleted snapshot or version.
Permanent delete enables you to permanently delete a soft-deleted snapshot or blob version before the retention period ends.
Note
The storage account must have versioning or snapshots enabled. Soft-delete must also be enabled on the storage account to soft-delete versions...
Who Deleted a Blob?It is sometimes useful to know who created/modified/deleted a storage blob. For that information to be recorded in a log the authentication must be done with Azure AD to populate the user information correctly.
There are two sets of logs that can help, Azure Monitor logs and classic Analytics logs1. Azure monitor storage columns - https://learn.microsoft.com/en-us/azure/azure-monitor/reference/tables/storagebloblogs
Columns of interest
CallerIpAddress - The IP address of the requester, including the port number
OperationName - The type of REST operation that was...
The previous article explained how user could configure and limit the access to Cloud Service Extended Support (CSES) only from specific Virtual Network(s) by using internal Load Balancer feature. But for some scenarios, user does not really want to fully block the traffic from public Internet, but just wants to make the CSES safer. In this scenario, adding an Azure Application Gateway in front of CSES will be a good idea. It can provide following benefits:
From public Internet, the traffic is only able to reach CSES through Application Gateway
From Virtual Network, the traffic is not...
Agenda
This article will provide a demonstration on how to utilize either SAS token authentication or managed identity from API Management to make requests to Azure Storage. Furthermore, it will explore and compare the differences between these two options.
Comparision
The choice between Managed Identity and SAS Token depends on factors such as the level of control required, the duration of access, and the specific security requirements of your application. Both options offer different levels of access control and security features for accessing Azure Storage.
Azure Managed Identity...
Scenario: PowerShell script to disable Public Network access for Azure Relay
In today’s azure world, it is always better to do automations rather than do a Manual work. It becomes a headache for all of us to navigate to Azure Portal
Here is the simple way to disable Public Network access by using PowerShell script.
Solution:
Connect-AzAccount
Set-AzContext -SubscriptionId "<Sub ID>"
$accessToken = $(Get-AzAccessToken).Token
$body = '{"location": "xxxxxx","properties": {"publicNetworkAccess": "Disabled","defaultAction": "Deny","virtualNetworkRules": [],"ipRules": []}}' |...
The blog provides guidance to perform simple File storage REST API operations such as Create, Copy, Rename and Delete operations using Postman.
Let’s look at some of the command syntax to perform REST API operations and will be making use of SAS as the auth scheme. We need to take care of the pointers below while performing the operations via Postman:
Ensure the URL is formed correctly as per the operation you are trying to perform.
The mandatory header needs to be passed.
Ensure you are appending/removing extra ‘?’ to the SAS token in the URLs accordingly.
Http verb can be GET, PUT...
The blog is inclined towards troubleshooting clientothererrors reflecting in the metrics. It is a step-by-step process to understand what these errors signify and potential reasons. This will also help in analyzing if these are legitimate ones and also deciding on the action plan ahead.
ClientOtherError usually means expected client-side errors which are not fatal errors. These operations have been completed successfully and therefore don't affect other metrics, such as availability. Some examples of operations that execute successfully but that can result in unsuccessful HTTP status...
Azure Storage supports for Secure File Transfer (SFTP) protocol in Azure Storage Account. We can use a SFTP client to securely connect to the Blob Storage endpoint of your Azure Storage account, and then perform upload and download operations over the account. Please note that SFTP feature support is only available for hierarchical namespace (ADLS Gen2) enabled accounts.
In this article, we will discuss about how to troubleshoot and isolate connectivity issues to SFTP storage account from your machine to understand whether this is due to port blockage, firewall issues, connectivity using...
Azure Storage Explorer is a powerful tool that simplifies working with Azure Storage services such as Blob, Queue, Table, and File storage. While it provides a user-friendly interface, there may be times when you need to inspect the requests it sends and the responses it receives from Azure. In such cases, Fiddler, a widely used web debugging proxy, can come to your rescue.
This step-by-step guide will walk you through the process of capturing a trace that you can use to evaluate such requests and responses using Fiddler.
Prerequisites:
Install Azure Storage Explorer: Download and...
Background
This article describes how to identify the blobs with index tags and how to remove those tags using the Blob Inventory Service and Python SDK.
Approach
This article is divided into two sections. These sections are independent, which means that you can perform the steps in section 1 and not perform the steps in section 2, or vice versa:
Use the Blob Inventory service to identify the blobs with index tags
You should follow the steps in this section if you want to know/identify which blobs in your Storage Account have index tags, and to know which index tags are...
Overview
StackExchange.Redis is a high performance general purpose redis client for .NET languages (C#, etc.), and is designed for Windows / .NET SDK environments.On other hand, it is possible now install .NET SDK and run PowerShell commands on Linux environments.This article describes how to install and link all together to have a Linux environment to run some commands and connect to any Redis service using StackExchange.Redis client library.This can be used for test proposes, and there are no support to run this on production.Try and test this at your own risk and responsibility. This...
Managing Azure Policies through Python SDK
Azure Policy helps to enforce organizational standards and to assess compliance at-scale. It also helps to bring your resources to compliance through bulk remediation for existing resources and automatic remediation for added resources.
Common use cases for Azure Policy include implementing governance for resource consistency, regulatory compliance, security, cost, and management. Policy definitions for these common use cases are already available in your Azure environment as built-ins to help you get started.
Specifically, some useful...
Scenario
Suppose you have a specific requirement wherein the user should not have access to view the File Share, Queue, and Table Storage Services Data. The user should only be able to access and view the containers within the storage account. In this blog, we will delve into the methods and techniques to fulfil this requirement.
Assessment:
Azure Storage provides several predefined roles built-in roles that can be utilized to restrict users from executing certain operations on the Azure Storage Account. However, in order to fulfil the aforementioned requirement, it is necessary to...
Introduction:
In the ever-evolving landscape of API development, dealing with legacy systems and formats is a common challenge. This blog post aims to assist you in leveraging Azure API Management (APIM) to address a specific scenario: transforming XML data to JSON and vice versa. We'll delve into the techniques supported by Azure API Management that allow you to bridge the gap between these formats seamlessly.
Understanding the Challenge:
Legacy systems often communicate using XML, while modern APIs tend to rely on JSON due to its simplicity and widespread support. In scenarios...
The current Service Fabric clusters are currently equipped with the MicrosoftMonitoringAgent (MMA) as the default installation. However, it is essential to note that MMA will be deprecated in August 2024, for more details refer- We're retiring the Log Analytics agent in Azure Monitor on 31 August 2024 | Azure updates | Microsoft Azure. Therefore, if you are currently utilizing MMA, it is imperative to initiate the migration process to AzureMonitoringAgent (AMA).
Installation and Linking of AzureMonitoringAgent to a Log Analytics Workspace:
Create a Log Analytics Workspace (if not...
In Storage Account, you can set diagnostic log to EventHub. However, did you ever wonder if you could have a centralized EventHub to take care of these logs? Instead of having multiple EventHubs across each Subscription within the same Management Group? In this article, it will explain to you how to centralize your EventHub.
I. What is the required Permission configuration?
Before reading through this article, be sure to have multiple subscriptions. As this will not explain the steps to add subscriptions to your Management Group. This article is based on the fact that you already have...
The blog points to performing simple Queue REST API operations such as List, Create, Get, Put and Delete using CURL utility.
Let’s look at some of the CURL command syntax to perform REST API operations and will be making use of SAS as the auth scheme for most of the operations. We need to take care of the pointers below while performing the operations via CURL:
Ensure the URL is formed correctly as per the operation you are trying to perform.
The header needs to be passed and can be done using “-H” parameter.
Ensure you are appending/removing extra ‘?’ to the SAS token in the URLs...
Ensuring adequate access control is of utmost importance for effectively safeguarding your Azure Storage assets. The Access Control Lists (ACLs) of your Azure Storage Account assume a pivotal role in establishing permissions for both containers and blobs situated within the storage account. The inheritance of ACLs is already in place for newly generated child items beneath a parent directory. However, it's noteworthy that now you also have the capability to set, update, or remove ACLs recursively for the pre-existing child items of a parent directory, thereby eliminating the need to...
The Put Block operation is used in conjunction with other operations to upload data as blocks to a block blob. This operation is mainly used for uploading large files or data streams in smaller blocks, rather than uploading the entire content in a single request.
There can be different types of errors that you might encounter while uploading blobs via Put Block and Put Block List and has been discussed under blog - Troubleshooting InvalidBlock 'The specified block list is invalid’ based errors - Microsoft Community Hub.
In this blog, we will mainly focus on how to upload a block blob...
Background Information
While the our work environments became more and more complex, the need of automation is a fact. Therefore ,I made a demo on how you can automate various administrative tasks performed on a Service Fabric Cluster using the sfctl module and trigger them via Azure Function. On Function, we deploy a Docker Container that has all the necessary tools and permissions to access SF Cluster. The Azure Service Fabric command-line interface (CLI) is a command-line utility for interacting with and managing Service Fabric entities, like cluster, applications and services....
Introduction:
The Azure PaaS product API management allows to expose APIs hosted on Azure App Services, also known as webapps. The process of importing and publishing the App service is described in document: Import Azure Web App to Azure API Management ; however, on this article we will analyze what actions Azure Portal performs behind the scenes to complete the import and figure out the cause of common scenarios.
Steps to collect a Browser Trace: HAR
Troubleshooting:
Background steps to import a WebApp to API Management:
Once you clicked on the Azure Portal > APIM > APIs >...
Background introduction
API Management allow us to import a function app as an API, the UI offers a straightforward process to import the function app as described on this Microsoft public document. However, in this article we want to go beyond and explore what actions are performed in the background, it will allow us to understand the import process better and will help us dealing with issues while importing the function app.
Steps
Please review all the steps carefully, they are executed in the following order.
Note: The easiest way to see these steps is collecting a HAR trace.
1-...
This article describes how to perform a REST API request in Azure using RBAC authentication with Postman. I will use as example the Get Blob (REST API) request.
Please see below how to perform a REST API request in Azure using RBAC authentication:
Open the Azure Portal and go to Azure Active Directory.
On left side, please create a new App registration by clicking on App registration (left side bar) and then New registration. Fill in the Name and all the information required.
Inside the new app:
Click on Overview and and collect the Application (client) ID value, and the Directory...
Scenario: PowerShell script to get the total number of partitions in Event Hub Namespace.
You must be wondering how to find out total number of partition count for any Event Hub namespace??
On the Azure portal you can only see partition count per Event Hub Entity level but not Namespace level. It’s tiresome job to manually sum up partition counts of all the entities especially when you have hundreds of event hub entities under the namespace.
Here is a simple way to find out total number of Partitions under Event Hub namespace using PowerShell script.
Solution:
Use below...
What are Deferred messages:
Deferred messages refer to messages that a queue or subscription client is unable to process at the moment due to certain circumstances. Instead of processing it immediately, the client can defer the retrieval of the message to a later time, while the message remains in the queue or subscription.
Message Deferral | Azure Service Bus
Unlike dead-letter messages that are stored in a subqueue, deferred messages are kept in the main queue along with other active messages. However, these messages cannot be received using regular receive operations. If an...
Sometimes, user will need to send out REST API request to manage their resources in Azure platform, for example, when the REST API is upgraded but corresponding PowerShell module or User Interface are not upgraded yet. In this blog, it will contain three main parts:
Common points of both ways to send request
Send request manually with User Interface such as Postman
Send request manually with command line, such as PowerShell in Windows and Curl in Linux
Common points of both ways to send request:
No matter user wants to send request by which way, the next three points are always the...
In this blog post, I will elaborate on some troubleshooting steps in Azure storage static websites. Azure storage static websites are a feature that allows you to host static content (HTML, CSS, JavaScript, and image files) directly from a storage container named $web. This is a great option for scenarios where you don't need a web server to render content, but you still want to leverage Azure services such as Functions, App Service, or CDN.
However, sometimes you may encounter some issues when setting up or deploying your static website. Here are some common problems and how to solve...
Issue:
Unable to delete Azure EventHub Cluster/Namespace/ Entity from Portal/ PowerShell/ CLI.
Case 1:
When EventHub tier is Premium and you are unable to delete Azure EventHub namespace and getting conflict operation error 409.
Sample error message : "statusMessage":"{\"error\":{\"code\":\"Conflict\",\"message\":\"Namespace provisioning in transition. For more information visit https://aka.ms/eventhubsarmexceptions.
Reason:
The reason for stuck state of EventHub namespace or its provisioning failure is due to a known race condition which initiates between two or more internal...
With the evolution of Azure Cache for Redis enterprise and the growing number of Redis Enterprise cache users let us look at few steps to troubleshoot if you are stuck in connecting to Azure cache for Redis. Few thumb rule for Enterprise Cache are below.
Are you connecting to the right port. Enterprise cache uses port 10000, not 63xx like basic, standard, Premium?
Are Enterprise cache instance configured to require TLS or non-TLS? It only supports one of them at a time.
Are the client configured to use the TLS or non-TLS setting correctly? (Client must be configured the same as with...
It’s common that a part of the data of a web page is saved in another service such as storage account or SQL server. When the website is hosted on Azure Cloud Service, when we visit the page, the w3wp process of IIS component will need to send out a request to the target remote server to read the needed data.
But when the Cloud Service fails to read the data from the remote server and developer wants to troubleshoot this issue, it will be difficult as by default users are unable to track the outbound traffic from Cloud Service to these remote servers. If user can capture trace of...
Few platform logs and metrics go missing when streaming them from Diagnostic Setting to Event Hub...
Issue: Few platform logs and metrics go missing when streaming them from Diagnostic Setting to Event Hub especially when Event Hub is Throttling.
Scenario: Using Diagnostic setting users route platform logs and metrics to Azure Event Hub (EH). These events are consumed by partner SIEM and Monitoring tools. At times users complain that events that occurred in the system or certain metrics\logs\traces has not reached the Event Hub.
Validation:
Validate if the event\metric\log has been generated by the platform\application. Also check if the Diagnostic setting is properly configured to...
Object replication asynchronously copies block blobs between a source storage account and a destination account. Because block blob data is replicated asynchronously, the source account and destination account are not immediately in sync. There's currently no SLA on how long it takes to replicate data to the destination account. In some cases, you might need to check the replication status, in this article we will go over the different methods that you can use to check and monitor the object replication status for the storage account.
Using Azure Portal
To check the replication status...
Disclaimer: Please test this solution before implementing it for your critical data.
Scenario:
This article explains step by step procedures to accomplish the requirements in Azure Storage using built in RBAC roles:
Read/Download and write/upload operation for containers and blobs should be allowed for the users.
Delete operations should be restricted.
Pre-Requisites:
Azure Storage GPV2 / ADLS Gen 2 Storage account
Make sure to have enough permissions(Microsoft.Authorization/roleAssignments/write permissions) to assign roles to users , such as Owner or User Access...
Scenario: Calls to Service Bus and Event Hub failing with Server Error \ Internal Server Error \ HTTP Error 500 Errors.
Issue Description: SB\EH API calls by the client application (usually sender or consumer) fails with Internal Server Error. The corresponding exception messages are something like this – “The server was unable to process the request due to an internal error”. The exception type and exception messages may vary somewhat based on the programming language of the SDK, SDK version, Wrappers, etc. There are a lot of frequently asked questions related to the Server Error and I...
Service Fabric Explorer (SFX) is the web client used when accessing a Service Fabric (SF) cluster from a web browser. The version of SFX used is determined by the version of your SF cluster. We are providing this blog to make customers aware that running Service Fabric versions 9.1.1436.9590 and below are affected. These versions could potentially allow unwanted code execution in the cluster if an attacker can successfully convince a victim to click a malicious link and perform additional actions in the Service Fabric Explorer interface. This issue has been resolved in Service Fabric...
When using Azure Cognitive Search to index data from an Azure SQL database, it's important to understand the different authentication methods that can be used to connect to the database. In this blog post, we'll explain the different authentication methods that the Search service can use to access an Azure SQL database.
Authentication Methods in SQL DB
There are several authentication methods that can be used to connect to an Azure SQL database, including:
SQL Authentication
SQL authentication is a method of authentication that uses a username and password to connect to an Azure SQL...
This article describes how to scale up a Service Fabric cluster primary node type with minimal downtime. In-place SKU upgrades are not recommended on Service Fabric cluster nodes, as such operations potentially involve data and availability loss so you can refer below for below method for scaling up a Service Fabric node type.
Note: We will be making use of PowerShell cmdlets for all the changes we are going to perform in the cluster, so in case you want to go with the ARM template approach to add a VMSS you can refer Scale up an Azure Service Fabric primary node type - Azure Service...
Background Introduction
Azure Application Insights is a powerful tool for monitoring the performance and availability of your applications. By sending web requests to your application at regular intervals from Application Insight’s servers around the world, it can alert you if your application isn't responding or responds too slowly. Since it is a useful tool to check the server's availability, some users may integrate the availability test feature of Application Insight with the API management service.
As the Application Insights server is in the public network, we need to ensure...
Sometimes, users will need to install some necessary software/packages in the environment before the task is executed. This can be easily done by using Start task feature of Azure Batch.
But when there are many dependencies to be installed, for example 20 packages in Linux and the installation of some of them will take long time such as Tensorflow, it will cause additional problems such as long start task running time when Azure Batch starts Batch node every time or even possible timeout issue or start task failure issue.
In order to avoid this issue, user has two options: Custom...
Background:
There would be scenarios where Customers would require to find the Storage Account capacity with segregation of soft delete, Version, snapshot and Active data.
Calculate the size of a Blob storage Blob Inventory:
The Azure Storage blob inventory feature provides an overview of your containers, blobs, snapshots, and blob versions within a storage account. Use the inventory report to understand various attributes of blobs and containers such as your total data size, age, encryption status, immutability policy, and legal hold and so on. The report provides an overview of your...
The Blob Inventory feature provides an overview of your containers, blobs, snapshots, and blob versions within a storage account. You can make use of the inventory report to understand various attributes of blobs and containers such as your total data size, age etc. depending upon the available fields for the inventory reports.
When it comes to gaining insights on breakup of blobs based on the type, access tier etc., there are certain limitations when it comes to the current available metrics. However, you can make use of the inventory report and merging that with the power of running...
Are you getting 50002 Error or 50008 Error return code in your EventHub? If that’s the case, you are in the right place.
In this documentation, we will go through what these two return codes are and how to prevent your EventHub to throttle.
[50002 Error - ServerBusyException]
Pre-requisite:
Are you getting 50002 Error from your EventHub? Is your Throughput appropriately configured? Is your load evenly distributed across all partitions? If so, keep on scrolling down, as you may find the answer to your problem.
I. What is a Throughput Unit?
The throughput capacity of Event Hubs is...
The blog talks about controlling SAS in terms of permissions and expiry time via Stored Access Policy.
When working with SAS, in case a SAS gets compromised, to prevent any un-authorized access via that SAS, the only option to revoke SAS is to recycle the Access Keys being used to sign the SAS. This in turn will affect all the SAS that have been signed using those set of keys. Also, if you are using those keys anywhere in the application, you will have to update the configuration on all those places.
To prevent this, we can make use of a stored access policy. The advantage of using...
Background Information
To prevent unauthorized access to a Service Fabric cluster, you must secure the cluster. Security is especially important when the cluster runs production workloads. This article describes how to configure node-to-node and client-to-node security by using Windows security, where gMSA is the preferred security model.
A standalone Managed Service Account (sMSA) is a managed domain account that provides automatic password management, simplified service principal name (SPN) management and the ability to delegate the management to other administrators.
The group...
Service Fabric Explorer (SFX) is an open-source tool for inspecting and managing Azure Service Fabric clusters. Service Fabric Explorer is a desktop application for Windows, macOS and Linux. To launch SFX in a web browser, browse to the cluster's HTTP management endpoint from any browser - for example https://clusterFQDN:19080. Service Fabric explorer may not load for numerous reasons. Most frequent reasons could be access denied while trying to access or unable to choose the right certificate. Following steps provide some useful insights on investigation steps and mitigations to be...
Symptoms:
Uploading files using Java SDK class DataLakeFileClient fails to initiate the ADF trigger.
Uploading files using Azure Portal initiates the trigger as expected.
ADF trigger is configured to ignore empty blobs.
Analysis:
Whenever a file is created using ADLS Gen2 REST API, there are 3 operations called at the backend i.e. CreateFile, AppendFile and FlushFile.
Microsoft.Storage.BlobCreated event is triggered when a blob is created or replaced. Specifically, this event is triggered when clients use the CreateFile and FlushWithClose operations that are available...
Recently, a new feature called “Manual effect” popped out to users, but it is still in the PREVIEW stage. It allows users to self-attest the compliance of resources or scopes. In another word, it gives users a chance to determine the evaluation result of the specific resource and the reason. Currently, it is mostly used inside the Security related built-in policies and initiatives. You may check more details from the following doc.
Understand how effects work - Azure Policy | Microsoft Learn
There are multiple built-in policies under the Security category to check the subscription...
Case: In a situation, you observe your EventHub namespace is still incurring the cost, even though you are not doing any operations (read/write/management) on it and your EventHub namespace has no active EventHub instances present.
Root Cause of the behavior:
At the time of creating the Azure EventHub namespace (Basic/Standard/Premium/Dedicated tier), you may have to give pre-purchased units of capacity in terms of Throughput units/Processing units/Capacity units based on the tier you choose. These pre-purchased units of capacity controls throughput capacity of the EventHub...
Background Information
As we all know, the API management service is nothing but a proxy that helps to forward the request and response between the client and the backend server. Ideally, the API Management server will maintain two connections: the one with the client side, and the one with the backend server. The backend server proceeds the request from APIM when SSL/TLS connection is successfully established between APIM and the backend server. However, if any of the elements within the SSL/TLS connection go wrong, the client server may receive a 500 BackendConnectionFailure error...
Azure cloud service extended support(CSES) is a new Azure Resource Manager based deployment model for Azure Cloud Services product. Cloud Services (extended support) has the primary benefit of providing regional resiliency along with feature parity with Azure Cloud Services deployed using Azure Service Manager. It also offers some ARM capabilities such as role-based access and control (RBAC), tags, policy, and supports deployment templates.
For the classic cloud service, we have Azure DevOps built-in pipeline task Azure Cloud Service Deployment task - Azure Pipelines | Microsoft...
Since, well, the beginning of Azure API Management, you've been able to validate that the Json Web Token (JWT) coming into your Azure API Management service is valid before passing it onto the backend service. The validate-jwt policy is one of the most used policies within Azure API Management, will happily ensure your client applications are using the right client IDs, and have the right audiences and claims. Meanwhile, Azure Active Directory (AAD) is a mainstay of enterprise APIs, providing authentication and authorization controls for a wide variety of APIs from M365 APIs to...
Background
This article describes how to get container level stats in Azure Blob Storage, and how to work with the information provided by blob inventory.
The approach presented here uses Azure Databricks and is most suited to be used in storage accounts with a huge amount of data.
At the end of this article, you would be able to create a script to calculate:
The total number of blobs in the container
The total container capacity (in bytes)
The total number of snapshots in the container
The total container snapshots capacity (in bytes)
The total number of versions in the...
Background:
Many times, we have a requirement to delete the older logs from $logs as the Retention days as it was not supported for ADLS gen2 storage account.
Please note Product team already worked on this fix and in case automatic deletion is not working can follow the below steps
Please refer to article https://docs.microsoft.com/en-us/azure/storage/common/manage-storage-analytics-logs?tabs=azure-portal#modify-log-data-retention-period for more information
Deletion of logs from the $logs can be achieved from the following options
Azure Storage explorer
REST...
This blog is the complement of another blog which is related to Azure Event Hub Schema Registry. As we known, it’s not supported to migrate Confluent(Kafka) schema registry into Azure schema registry directly. We need to create and manage the scheme registry in the azure event hub separately. The good news is Azure Event Hub supplies multiple client sdks for us to use to serialize and deserialize payloads containing schema registry identifiers and avro-encoded data. In this section, I’d like to share how to use python sdk to send or receive events with schema registry in the Azure Event...
Scenario:
You have created a Search Service and you want to import your data from the Storage account which is behind a firewall.
There are certain times when you need to update your Indexer for additional properties for a particular search service. So, while trying to update the Indexer, you navigate to “Add Indexer” option and then under Settings tab, you update the required properties.
While updating the Indexer, suddenly you encounter that it failed with the below error message.
Failed to update indexer ‘xxxxxx-indexer', error: 'Error with data source: Credentials provided in the...