What is our primary use case?
We process data mostly by scraping from retail chains on behalf of our customers, and then we homologate and clean that data for our customers to use in analytics displays and decision-making tools. The original architecture was very basic as we used applications such as Pentaho for all ETL workloads running on just fixed VMs from EC2, and then we moved these workloads into AWS Fargate. We didn't use any sophisticated Kubernetes clustering integration, so it was very basic in that case.
Currently, we are deploying workloads because this company has been going through a lot of merging and acquisition strategies. The company I am speaking of was a Mexican company that had 100% of the infrastructure over AWS, but as we integrated to manage the other companies we bought, we had to manage a different set of AWS services, such as Redshift and so on.
In our case, the workloads were not oriented to micro-services with AWS Fargate, just servers. The ETL flows we moved into Fargate using Pentaho weren't designed for micro-service architecture. The front end was very monolithic rather than micro-services oriented, and we had some simple API services, but not a formal micro-service architecture, so we didn't use AWS Fargate for that.
What is most valuable?
One of the best features of AWS Fargate is that it was useful for us because we didn't require to run container workloads and we didn't need to deal with the management of a Kubernetes cluster directly, and the ability to run those workloads just in a scheduled manner is also a great feature. Additionally, we reduced costs because the distribution over time of our workloads was not the whole day, so we had fixed VMs and some capacity that we weren't using.
AWS Fargate has definitely impacted our organization positively because before that, we couldn't accommodate the workload requirements as flexibly as we can with Fargate. We were able to process all the workload in a shorter time as we could increase the number of containers processing data by schedule. We provided operations because, in some cases, they required special workloads processed outside the schedule, and we offered some automations for them to just go to an internal application and configure their schedule and the amount of horsepower they require to increase in AWS Fargate, and it worked very easily. I understand that this could be accomplished with the auto scaling capabilities of Fargate, but as mentioned, the DevOps were just learning to use AWS Fargate and chose a simpler approach they considered less risky.
What needs improvement?
Currently, I think that the program is great the way it is, and maybe we use less than 50% of the current features of the platform. For example, we have been evaluating Dask with Python to work with distributed processing, and I don't know if AWS Fargate could be used to build Dask clusters, which would be great because there are many people on our team working with Python workloads. Having something that can manage distributed processing using Dask in a containerized deployment could be valuable. For a company that does not require complexity or managing Kubernetes clusters, AWS Fargate is a great way to go with the use of containers in a simple way, providing the power of containers and scalability without the complexity of going into Kubernetes.
For how long have I used the solution?
I started to use AWS Fargate about two to three years ago because we had some loads on fixed virtual machines, and we needed to scale based on a schedule.
How are customer service and support?
I would rate AWS support more than perfect because when I started working with the Mexican company, which was 100% AWS, we weren't that big so we didn't qualify as high-value customers, but even so, the AWS team in Mexico gave us special attention that went beyond our service level agreement. Even though we didn't contract support, every two weeks I had a 30-minute meeting with a cloud architect from AWS to help our team use different products of AWS, especially with SageMaker for a forecasting algorithm we were developing.
How would you rate customer service and support?
What was our ROI?
The pay-as-you-go pricing model of AWS Fargate was one of the major drivers for us to move there because we reduced costs while increasing the quality of the processing services by about 30%.
What other advice do I have?
My team hasn't used AWS Fargate's capability for automatic scaling in our applications, even when they knew about it; they were very new to using these features.
AWS Fargate's security features weren't the most important criteria or value we got from AWS Fargate.
I would recommend AWS Fargate to others; I would use it myself if I were in a startup or a medium-sized team because of the simplicity it offers without having to manage Kubernetes clusters. As we started using Kubernetes, I found it complex and only justified when you have a larger team that can manage certified cloud architects and cloud engineers. For a small team, I would go blindly using AWS Fargate if I want to take advantage of containers in a scalable way.
I rate AWS Fargate overall as 10 out of 10. I don't have any regrets using it. We have never had any problem using it, even when we used it with very basic knowledge of the cloud.
Which deployment model are you using for this solution?
Public Cloud
If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?
Amazon Web Services (AWS)
Disclosure: My company does not have a business relationship with this vendor other than being a customer.