The best features AWS Step Functions offers include the ability to deploy it via code. It helps us orchestrate anything that we want. Even if we have Glue jobs for another use case or Lambda that we are currently looking forward to, we are using Lambda to take DynamoDB data for real-time sales monitoring, then it goes to our S3 bucket. We have multiple workflows that are orchestrated by AWS Step Functions, and at the core, the main functionality is based on different frameworks. We have an Athena complete workflow, Glue job-based workflow, and a separate Lambda-based workflow. All of these are separately handled in AWS Step Functions. This is a good point and again with the inbuilt facilities of it, including how we can construct those workflows and add up a catch state or roll back or do multiple iterations using the Map state, that is very helpful for us.The Map state feature has specifically helped my team create a rollback state of the data since we were dropping some of our data assets. In case we want them back, we just wanted to do this. We took a particular layer and provided the data list, including the tables list, and using the Map state, we iterated over this list and one by one we did the backups of this data at the concurrency of five tables in a single execution. The Catch state, on the other hand, allows us to identify any error immediately. If we encounter an error, the Catch state immediately picks it up and we integrated webhooks or email notifications. That is how we are able to identify immediately if we face any error. At the core of our data engineering platform, AWS Step Functions sit at the topmost core of whatever pipelines we have. It is a main orchestration tool for us and helps us save costs. It is a key component of our engineering platform.
