The Put Block operation is used in conjunction with other operations to upload data as blocks to a block blob. This operation is mainly used for uploading large files or data streams in smaller blocks, rather than uploading the entire content in a single request.
There can be different types of errors that you might encounter while uploading blobs via Put Block and Put Block List and has been discussed under blog - Troubleshooting InvalidBlock 'The specified block list is invalid’ based errors - Microsoft Communit....
In this blog, we will mainly focus on how to upload a block blob by PUT Block & PUT Block List APIs using POSTMAN, how to deal with “Invalid Block/Blob 'The specified blob or block content is invalid'” error due to incorrect BlockID and make use of GetBlockList API to identify Uncommitted and Committed blocks.
Put Block
This API creates a new block to be committed as part of a blob. Before uploading, each block is given a unique block ID. The block ID is a Base64-encoded string that helps identify each block. Before it's encoded, the string must be less than or equal to 64 bytes in size. For a specified block, the length of the value before encoding, for the block ID parameter must be the same size for each block.
Making a Put Block call using POSTMAN
Sample URL: https://<storageaccountname>.blob.core.windows.net/<containername>/<blobpath>?comp=block&blockid=id&SAS
Blockid = base64 of any number/string (convert using url encoding - Search (bing.com))
Here, I have used blockid = base64 of 2 characters e.g. 10.
Add content of the block in the Body section.
Once you hit the API, you will get response code as 201. This shall confirm that currently the block is in uncommitted state and is ready to be committed to be a part of the blob by PUT Block List API call further.
This is the first uncommitted block that we uploaded using PUT Block, so the block ID that we used to upload this block can be any 64 bytes or lesser value that is base64 encoded. Like in this case we used base64 of 10 for uncommitting the first block.
Now, the next aligned blocks that are to be uploaded should ideally have the blockID that is base64 of any 2-digit character only. If you try to upload a block with blockID other than 2 digits, it will throw the below error-
“400 – The specified blob or block content is invalid.”
In the example below, we have used blockID= base64 of 101.
Therefore, all the blocks should have the block Id of the same length before encoding it to base64 format in order to avoid the above error.
Reference Doc: Uploading blob or block content fails in Azure Blob Storage - Azure | Microsoft Learn
The next is to identify if the blocks are in committed or uncommitted state.
Get Block List
The Get Block List operation retrieves the list of blocks that have been uploaded in chunks as part of a block blob. It gives the list of
- Uncommitted Block List- The list of blocks that have been successfully uploaded for the specified blob by using Put Block but are not committed yet using Put Block List
- Committed Block List- The list of blocks that have been successfully committed to a specified blob by using Put Block List
Making a Get Block List call using POSTMAN-
Sample URL: https://<storageaccountname>.blob.core.windows.net/<containername>/<blobname>?comp=blocklist&blocklisttype=all&SAS
This shall return the list of committed and uncommitted blocks and you can review the output for different sizes of the blocks. Each block can be of different size but with a max limit on content size. I have uploaded three blocks using PUT Block and the block IDs are – base64 of 10, 11 & 12.
The following table describes the maximum permitted block and blob sizes, by service version:
|
Service version |
Maximum block size (via Put Block) |
Maximum blob size (via Put Block List) |
Maximum blob size via single write operation (via Put Blob) |
|
Version 2019-12-12 and later |
4,000 mebibytes (MiB) |
Approximately 190.7 tebibytes (TiB) (4,000 MiB × 50,000 blocks) |
5,000 MiB |
|
Versions 2016-05-31 through 2019-07-07 |
100 MiB |
Approximately 4.75 TiB (100 MiB × 50,000 blocks) |
256 MiB |
|
Versions earlier than 2016-05-31 |
4 MiB |
Approximately 195 GiB (4 MiB × 50,000 blocks) |
64 MiB |
Reference Doc: Get Block List (REST API) - Azure Storage | Microsoft Learn
Once you have verified the blocks and they appear to be in sync, we can proceed further with the next API to commit the blocks further.
Put Block List
This operation is used to commit a list of blocks in a specific order to form or update a block blob. It essentially tells Azure how to arrange the uploaded uncommitted blocks by Put Block to construct the final blob.
Making a Put Block List call using POSTMAN-
Sample URL: https://<storageaccountname>.blob.core.windows.net/<containername>/<blobpath>?comp=blocklist&SAS
In the Request Body of the Put Block List method, add the list of BlockIDs, maintaining the order of the blocks.
Request Body:
<?xml version="1.0" encoding="utf-8"?>
<BlockList>
<Latest>BlockID1</Latest>
<Latest>BlockID2</Latest>
<Latest>BlockID3</Latest>
</BlockList>
Reference Doc: Put Block List (REST API) - Azure Storage | Microsoft Learn
We can also verify this by again running Get Block List. By blockID we can verify that the block has been successfully committed.
We can also verify the same from the Azure Portal-
Please note there is a time limit of nearly 7 days for committing all the blocks into a blob after uploading them. If the Put Block List operation is not performed within this time frame, the uploaded blocks become eligible for garbage collection.
Reference Doc: Uploading blob or block content fails in Azure Blob Storage - Azure | Microsoft Learn
Hope this helps!