Executing serverless background tasks in Azure using WebJobs

Introduction

A need that often emerges while working on cloud solutions is to execute background tasks. That is, tasks that are executed without user intervention. These types of tasks may involve parsing and converting files, doing some batch processing on data, etc.

For this and as part of its offering in serverless computing, Azure allows us to deploy WebJobs. The Azure WebJobs are deployed within an Azure App Service, and can be developed in .NET, batch files, PowerShell, PHP, Python, JavaScript and Java.

Getting started with WebJobs

All we need to do to have a WebJob running on Azure is to follow these steps:

• Create an App Service.
• Upload an artifact of one of the supported types to the WebJobs tab within the App Service.

In the Azure Portal we can specify if the WebJob is going to be executed continuously, on a schedule (specified using a crontab expression) or if it will be manually triggered.

Another supported option is to trigger the WebJob using a WebHook. For that, we can send an HTTP POST call to a provided URL, including in our request a username and password. This information can be obtained from the Properties popup for the WebJob in the Azure Portal.

Being able to trigger a WebJob using a WebHook is useful to integrate it with another WebJob or with an Azure Function, in the case of more complex processes.

The page we use to deploy a WebJob in the Azure Portal looks like this:

Using the Azure WebJobs SDK

When developing a WebJob in .NET Core, we can leverage the features of the Azure WebJobs SDK. This SDK makes it easy to get started with WebJobs, allowing us to declare methods to be called in response to different events. For that, the SDK provides different trigger classes that can be used to decorate the methods we want to be called.

For example, we can declare a method to be executed every time a new message is delivered to the queue named “testqueue”. The content of the message delivered to the queue is made available to the method through the parameter “message”. The code for this method would look as follows:

public static void ProcessQueueMessage([QueueTrigger("testqueue")] string message, ILogger logger)
{
    logger.LogInformation(message);
}

We can also declare a method to be executed periodically, on an interval defined by a crontab expression. The code for this method would look as follows:

public static void ProcessTrigger([TimerTrigger("0 * * * * *")] TimerInfo timer)
{
    Console.WriteLine("Timer: " + DateTime.Now);
}

The trigger classes currently available in the Azure WebJobs SDK are the following:

• BlobTrigger
• FileTrigger
• NoAutomaticTrigger
• QueueTrigger
• TimerTrigger

Besides declaring the different methods decorated with trigger classes, we have to add code to the Main method to initialize the infrastructure provided in the SDK. In this case, we’re adding the Azure Storage core services, Azure Storage (to support the first example, of a method using the QueueTrigger) and Timers (to support the second example, of a method using the TimerTrigger).

static async Task Main()
{
    var builder = new HostBuilder();
    builder.ConfigureWebJobs(b =>
    {
        b.AddAzureStorageCoreServices();
        b.AddAzureStorage();
        b.AddTimers();
    });
    builder.ConfigureLogging(b =>
    {
        b.AddConsole();
    });
    var host = builder.Build();
    using (host)
    {
        await host.RunAsync();
    }
}

For this code to run properly, we’ll need to add a setting named “AzureWebJobsStorage” in our appsettings.json file.

One thing worth pointing out is that the main project that we have to create is a .NET console application. However, in that application, we can reference as many other projects and third-party libraries as needed.

This implies three different things:

• We can design our WebJob as we usually design any other .NET solution, defining as many layers and components as needed.
• It is possible to share components between different WebJobs and Web Applications.
• We don’t have to decide upfront if we want to develop a WebJob. We can develop our logic and even data access or service calls with its own unit and integration tests, without having that code tied to anything WebJobs related.

For this code to run properly, we’ll need to add a setting named “AzureWebJobsStorage” in our appsettings.json file.

For more information please go to:
https://docs.microsoft.com/en-us/azure/app-service/webjobs-sdk-get-started
https://docs.microsoft.com/en-us/azure/app-service/webjobs-sdk-how-to

Testing our WebJobs with Azurite

As it was pointed out in the previous section, we’ll need to have a Storage Account provisioned in our Azure Subscription in order to run the WebJob. Additionally, for continuous or scheduled WebJobs we need to set the Always On feature in the App Service. This feature is not available in the free tier of App Services.

This means there are costs associated with developing and testing a WebJob.

However, there’s an emulator for Azure Storage called Azurite (https://github.com/Azure/Azurite) that we can use to test our WebJobs locally, without requiring the provision of a Storage Account or a paid App Service.

We can run this emulator in different ways. For example, we can run it in a Docker Container, just by using the following command:
docker run -p 10000:10000 -p 10001:10001 mcr.microsoft.com/azure-storage/azurite

Once we executed this command, we should get an output like the following:

To be able to use Azurite on our WebJob, we should go to the appsettings.json file to change the value of the “AzureWebJobsStorage” setting. The connection string we have to set is available from the Cloud Explorer in Visual Studio:

Conclusion

Azure WebJobs offers a way to execute background tasks while leveraging the power and convenience of serverless computing in the cloud, avoiding the provisioning and maintenance of a dedicated Virtual Machine.

WebJobs can be used for tasks that take a long time to complete, and since the SDK and WebHooks make it easy to trigger WebJobs, we can design our processes to be composed of several WebJobs and Azure Functions. This means we can communicate through message queues and use an event-driven architecture.