With the recent announcement of the AWS Step Functions AWS SDK Service Integration, it felt like an appropriate time to delve into what Step Functions are, what the announcement really means and hopefully give readers some ideas of where they could potentially implement AWS Step Functions to improve their orchestration processes.
The TLDR on the announcement
Previously, when you wanted to interface with particular AWS services as part of your Step Function workflow, if the service didn’t natively support direct integration with the AWS Step Functions Amazon States Language (ASL), you would essentially need to write a lambda function to handle that process and in some cases multiple lambda functions (and steps) to handle each step of that single service interaction.
With Amazon now providing direct integration to the AWS SDK via the ASL for over 200 services, AWS Step Function users will no longer need to write custom code for certain AWS service integrations/interactions and instead be able to simplify those interactions by using the ASL. This would then result in less custom code to manage (and test) and while also providing an easier visual representation of your workflow versus parsing code that may not necessarily make it apparent what the task is actually doing.
What is AWS Step Functions?
Without going too deep, AWS Step Functions is a highly configurable AWS workflow service that allows teams to handle simple or complex orchestration workflows with native integration to AWS Services using the ASL.
A lot of use cases found online tend to represent e-commerce/ordering systems, where distrubution workflows are necessary from when the order has been placed, to the item being packed and ready for shipment, shipped and the finally delivered. While this is a common use case AWS Step Functions can be treated and used in many different ways, including extremely useful customisations when it comes to complex workload deployments or configurations that may have become complicated and hard to visualise in CI/CD tools such as AWS Code Pipeline coupled with Amazon CodeBuild.
Basically it lets you define a number of steps for the workflow to follow, where some steps can be bypassed or redirected depending on the previous outcome as well as having the functionality to wait for other information to become available such as waiting for a service to respond or a human to confirm/deny an action.
Amazon have a number of use cases you can view to see if any align to current projects or tooling you may be supporting or working on.
Why integrating the SDK is a good move
As briefly alluded to in the TLDR, one pain point for a lot of engineers was service integration when using AWS Step Functions. For example, at a previous employer we had to orchestrate a number of configurations and CloudFormation deployments for multiple AWS Accounts in multiple regions with highly dynamic and point-in-time parameter values that made going for a more traditional approach difficult.
This resulted in having a custom function built to handle CloudFormation deployments in a more programmatic manner written in Python, however for every instance of needing to perform a CloudFormation deployment, this would need to be invoked in full each time to handle the service interaction. Now with the SDK integration, it would be a far less code heavy task and you could simply pass some variables and parameter values via the ASL from the previous step which could then natively interface with the CloudFormation service using the new integrated SDK.
That’s just one example, with over 200 services available, there is less need to write a function or actual code and focus on the ASL and native service integration benefits to remove those needs and simplify AWS service based tasks as part of your step function workflow.
Lambda functions? You no longer need them (sort of)
Well you still might for very complex tasks, however there are benefits to simplifying service interactions by just using the ASL with the SDK;
In the example Amazon provided in the announcement blog post, the workflow was to transcribe the audible language of an English video/audio file and then translate it into Spanish text. Before the integration of the AWS SDK, you would need to;
Write a function to copy the file from it’s source bucket to the transcribe bucket
Write a function to transcribe the file and another function to validate it’s status/had completed (technically you could have a the single function do both with variables assisting progressing)
Write a further three functions for the translating process
Thus you would be looking at a minimum of six Lambda functions plus the supporting infrastructure and deployment configurations to get your code into AWS to complete this workflow versus a 125 line CloudFormation stack which is easily deployable via many means.
Now with the AWS SDK integration, all of this is handled using the State Machine ASL with direct and native interaction with each required service and necessary API calls, resulting in (in this use case) no need to even write a single lambda function!
This also results in your costs only going towards your AWS Step Function State Machine execution costs without the added costs of Lambda execution(s), for every task that may have needed a function.
To conclude
With Amazon making the SDK available to AWS Step Functions, it will provide teams the ability to simplify their code and reduce complexity within their workflows going forward.
Innablr’s skilled and available team of cloud specialists are experts at quickly and effectively merging with customer cloud teams to augment skills and delivery bandwidth capability. Our engineers are recognised Google Cloud and Amazon Web Services experts. Leveraging our proven and repeatable frameworks for continuous integration and continuous deployment pipelines combined with a range of native cloud services to build reliable cloud foundations and support rich cloud based applications we accelerate time to market.