Intro

In the previous post I described TableOperation class and showed how to perform various operations on individual entities. In this post I am going to show how to work with TableBatchOperation class and how to perform operations on a set of items.

Prerequisites

Refer to previous posts to get started with Azure Storage Table Service.

Code

The TableBatchOperation class represents multiple TableOperation objects which are executed as a single atomic action within the table service. Thats it, TableBatchOperations, or Entity Group Transactions, are executed atomically meaning that either all operations will succeed or if there is an error caused by one of the individual operations the entire batch will fail. There are couple of limitations that we will discuss in this post.

Consider the following example:

var batch = new TableBatchOperation();
for (var i = 0; i < 100; i++)
{
     batch.Insert(new Profile("AU", $"EMP000{i}", $"John Doe {i}", $"The {i} employee of the company"));
}

table.ExecuteBatch(batch);

This code will sucessfuly insert 100 items into a given table. If you try to insert more that 100 entities, you will get the following exception:

The maximum number of operations allowed in one batch has been exceeded.

The exception is self-explanatory.

Consider the next example:

var batch = new TableBatchOperation();
for (var i = 0; i < 50; i++)
{
    batch.Insert(new Profile("AU", $"EMP000{i}", $"John Doe {i}", $"The {i} employee of the company"));
}

for (var i = 50; i < 100; i++)
{
    batch.Insert(new Profile("NZ", $"EMP000{i}", $"John Doe {i}", $"The {i} employee of the company"));
}

table.ExecuteBatch(batch);

This code tries to insert 50 entities to AU partition and another 50 entities to NZ partition. This code will fail with the following exception:

All entities in a given batch must have the same partition key.

This exception is also self-explanatory.

Consider the following example:

var batch = new TableBatchOperation();
for (var i = 0; i < 50; i++)
{
    batch.Insert(new Profile("AU", $"EMP000{i}", $"John Doe {i}", new string('*', 1024 * 1024)));
}

table.ExecuteBatch(batch);

This code will fail with the following exception:

The remote server returned an error: (413) The request body is too large and exceeds the maximum permissible limit..

According to documentation, the batch operation is limited to a 4MB data payload.

You can mix different operations inside single batch. Consider the following example:

var batch = new TableBatchOperation();
for (var i = 0; i < 100; i++)
{
    batch.Insert(new Profile("AU", $"EMP000{i}", $"John Doe {i}", $"The {i} employee of the company"));
}

table.ExecuteBatch(batch);

var newBatch = new TableBatchOperation();
var profiles = table.CreateQuery<Profile>().Execute().ToList();
for (var i = 0; i < 30; i++)
{
    profiles[i].Description = "Updated";
    newBatch.Merge(profiles[i]);
}

for (var i = 30; i < 60; i++)
{
    newBatch.Delete(profiles[i]);
}

for (var i = 100; i < 120; i++)
{
    newBatch.Insert(new Profile("AU", $"EMP000{i}", $"John Doe {i}", $"The {i} employee of the company"));
}

table.ExecuteBatch(newBatch);

First batch inserts 100 entities and the second batch does Merge/Delete/Insert. It is ok to mix different operation on different entities in a single batch. But it is not allowed to invoke more than operation in a batch for the single entity:

var newBatch = new TableBatchOperation();
var profiles = table.CreateQuery<Profile>().Execute().ToList();
for (var i = 0; i < 30; i++)
{
    profiles[i].Description = "Updated 1";
    newBatch.Merge(profiles[i]);

    profiles[i].Description = "Updated 2";
    newBatch.Merge(profiles[i]);
}

table.ExecuteBatch(newBatch);

This code throws exception:

Element 1 in the batch returned an unexpected response code.

The other limitation is related to Retrieve operations. Only one Retrieve operation allowed per batch and it is not permitted to have any other operations in the same batch. So the following code will fail:

var newBatch = new TableBatchOperation();
var profiles = table.CreateQuery<Profile>().Execute().ToList();

profiles[0].Description = "Updated 1";
newBatch.Merge(profiles[0]);

newBatch.Retrieve<Profile>("AU", "EMP0010");
table.ExecuteBatch(newBatch);

The exception is self-explanatory:

A batch transaction with a retrieve operation cannot contain any other operations.

This is the most frustrating part of Azure Table.

Also, one should remember that batch operations are atomic. Consider the following example:

var batch = new TableBatchOperation();
for (var i = 0; i < 99; i++)
{
    batch.Insert(new Profile("AU", $"EMP000{i}", $"John Doe {i}", $"The {i} employee of the company"));
}

batch.Replace(new Profile("AU", "EMP9999", "John Doe", "The  employee of the company") { ETag = "*" });
table.ExecuteBatch(batch);

This code throws the following exception:

Element 99 in the batch returned an unexpected response code.

If any operation in a batch fails, a whole batch will be canceled.

Summary

In this post I have showed how to use batch operations with Azure Table. I don't know why Microsoft doesn't allow multiple Retrieve operations in a single batch, but it is very frustrating. There are lots of use-cases when one need to retrieve multiple entities from the storage by their keys. In any case, in the next post I will show a couple different ways of how to query data from Azure Table.


;