Let's say you have a process (or maybe an external partner) that drops files onto an Azure Storage account for you to consume. The first step in that process might be to move the files from the drop folder to another, 'in process', folder. How to do that? You could have the partner or process email you every time a file is added. Another option would be to just manually check the drop folder every morning or hour or some other time interval. While either option would work, they manual and I don't like manual processes. If manual processes are out, what about an Azure Timer Function? We could configure it to run at a set interval of our choosing, have it copy the file to a new location and then finally delete original file so we don't process it twice. I like this approach. The timer function will always be running and can watch the inbound folder 24/7. Then we might be able to have a second function that watches the in process folder to apply whatever logic we need, maybe updating a database. So let's see how easy it is to build an Azure Timer Function to do the copying for us.
In this example I'm using Visual Studio 2019 for Mac and C#. To start create a new function app in C#.
Next give your function a name and select Timer Function
Next you set your schedule. The timer schedule is a CRON expression. The default is at every '5' minute (00, 05, 10, 15, 25, etc). See here for more CRON examples.
Finally you set the Project & Solution names, the location on disk and any version control options.
Now that we created the blank timer function it's time to get coding. Here is the finished product. Let's go through everything and see how it all works.
We left the default time of 5 minute, but during development I don't want to wait several minutes to test my changes. Also, it would be useful when recovering from errors to run the function as soon as it starts, no matter the time. So we added the RunOnStartup attribute for that purpose. Lines 15-19 are getting values from local.settings.json and creating the basic objects we'll be working with. It can be helpful to think of Cloud File and Cloud File Directory as the classes that replace System.IO.File and System.IO.Directory in Azure. See here for the MSDN documentation for CloudFile and CloudFileDirectory. Line 20 is when things start to get interesting. We are doing everything async but need to make sure certain actions happen after other actions complete. Line 20 is making sure that our Inprocess directory exists in Azure before we try writing to it. Next we are getting a list of all files and directories under the Inbound directory. In our example there are only a few files so we aren't worried about segmenting the result set. If you need to segment your results see the documentation here. Let's zoom in a bit on lines 23-44.
The for each loop does exactly what you think it does, the magic starts at the if statement. New to C# 7 is Pattern Matching. That allows me to test for type and declare a variable in the same line. We have to test for type because directories could be nested under our inbound directory. We want to create a subdirectory in the Inprocess directory for each file. So first we are getting a reference to the directory and then creating it if it does not exist. Once we know the directory exists it is time to create the new file.
Much like the directory we start by getting a reference to the final location of the new file. The creation process is different for a file than it is for a directory. We start by creating a new file with the same length as the original file. Next we start copying the data from the original file to the new file. Once both steps have successfully completed we need to delete the original file so we don't process it again later. That's really it. In about 30 lines we have created a function that will run on its own and copy files from the partner directory to another directory of our choosing for processing. Since Azure Functions are designed as small discrete units of work it is recommended practice to create additional function(s) to process the files, maybe even a durable function that is called from this function after each file is created? Hope that helps and as always Happy Coding. Sean Wernimont The Blind Squirrel Copyright 2015-2020
|
AuthorWelcome to The Blind Squirrel (because even a blind squirrel occasionally finds a nut). I'm a full-stack web and mobile developer that writes about tips and tricks that I've learned in Swift, C#, Azure, F# and more. Archives
April 2018
Categories
All
|