Today I'm introducing a new project. Over the coming days and weeks I'll post more details about specific steps, features & functionality, and deployment to Azure. This post is to explain the high level details and workflow. The app is going to be a simplified payroll processing engine. There is going to be a very simple web front end, an API, a database, message queue, and the actual payroll engine. The main goals for this project are 1) to determine the difficulty in integrating multiple different languages together and 2) how to use Octopus to deploy everything to Microsoft Azure. I mentioned that I want to use different languages, but which languages? My plan is to use simple HTML and Javascript on the front end, C# and Web API for the middle API, Microsoft SQL Server for the Database, Azure Service Bus as the message queue, and finally I want to write the Payroll Engine in F#. The main focus of the project is the API, Message Queue, and Engine. So I chose to just so something simple with the front end so I could focus more on the back end and deployment aspects of the project. With the language basics out of the way let me explain the overall design. Imagine this as a payroll system for a small business. The employees of the business can have 1 of 2 roles. They can be either a regular employee or they can be an administrator. The admin role is a super set of the regular employee so they have all the same functionality as the employee plus the ability to update employee rates and run payroll batches.
For the standard employee role they can only get the details for a single check and view a list of all their past checks. Since these are simple read operations the user makes their request via the front end which calls to the API and which queries the database. The admin user can also update employee hours worked and also their current hourly rate. These are also simple CRUD operations so the API will talk directly to the database. So far nothing new or exciting here. Things start to get interesting when the admin wants to run a payroll batch for N number of employees in the company. When the time comes to run payroll so that employees can be paid things start to get more complicated. Running a payroll for an entire company or even a subset of employees can be a very long running process. This is were having a separate payroll engine can really improve things. Imaging you are the admin in charge of running payroll so everyone gets paid. You know that it takes awhile to calculate everyone's taxes and deductions but you don't want to bring down the API with such a long running process. This is where the message queue comes into play. When the admin wants to run a payroll, they will select all the employees and click the magic go button. The API will then add a message to the queue and stop there. The API won't be blocked waiting for all the payrolls to run and other employees can still view their check details and history as normal. It is very important that you use the message queue rather than a service bus with topics and subscriptions. The reason is that a queue will guarantee that only 1 worker (the payroll engine in this example) picks up the message and runs the payroll. Once the message is on the queue the payroll engine picks it up and gets the information it needs from the database. Then it begins the process of calculating the payroll. Once it is complete it publishes another message to the queue which is picked up by the API and then the customer is alerted that payroll is complete. We could have the API calculate the payroll to save design complexity, but I think splitting the workload like this provides more benefits than the drawbacks. There were two main problems that I wanted to solve, the first was scale. By splitting the engine out of the API you can scale both pieces independently of the other. The second problem that needed to be solved was time. Running a payroll is very complex process and it can take a very long time to complete. Using the message queue decouples the payroll engine from the front end and the API. The engine can take a long as it needs without impacting other users and the admin can close the browser window and go about their day until they get the message that payroll is complete. That's it for the introduction. Thanks for sticking with me this far. In the upcoming posts I'll go into more detail about each piece of the application and why I chose the language that I did. That's it for now. As always, Happy Coding. |
AuthorWelcome to The Blind Squirrel (because even a blind squirrel occasionally finds a nut). I'm a full-stack web and mobile developer that writes about tips and tricks that I've learned in Swift, C#, Azure, F# and more. Archives
April 2018
Categories
All
|