What is our primary use case?
I am currently working with services that deal with real-time analytics, primarily
Amazon Kinesis for streaming,
AWS Lambda, and DynamoDB for serverless applications, and
AWS EKS for
CloudFormation. In this project, we use
Amazon Kinesis for real-time streaming of incoming messages and process them in near real-time.
What is most valuable?
Amazon Kinesis is easy to get started with, provides good documentation, and has a multilang daemon interface that makes it programming-language agnostic. The throughput is convenient for processing volumes out of the box and does not require complex configurations. It also provides auto-scaling with different partition keys into various shards. Lambda's scalability, seamless integration with other
AWS services, and support for multiple programming languages are very beneficial.
What needs improvement?
Amazon Kinesis could improve its pricing to be more competitive, especially for large volumes. Also, the KCL library's documentation could be improved to better explain the configuration parameters. For Lambda, improving the documentation around Lambda extensions would be beneficial.
For how long have I used the solution?
I have been using Amazon Kinesis for less than six months, but I have used EKS and serverless services like Lambda and DynamoDB for more than a year. I have been using Lambda for more than two years for serverless use cases.
What was my experience with deployment of the solution?
I found setting up Amazon Kinesis very easy with
CloudFormation templates. Lambda deployment involves taking the source code from a repository, zipping it, and uploading it to an
S3 bucket.
What do I think about the stability of the solution?
We have not encountered any stability issues with Amazon Kinesis or Lambda. The performance is as per AWS benchmarks.
What do I think about the scalability of the solution?
Amazon Kinesis provides auto-scaling with streams that handle large volumes well, and Lambda's scalability is highly efficient. AWS takes care of scaling automatically, which is convenient.
How are customer service and support?
Our company has an ongoing engagement with AWS, allowing us to reach out for help anytime. We receive prompt support from AWS solution architects or TAMs. We raised some tickets for service quota increases, which resolved quickly.
How would you rate customer service and support?
Which solution did I use previously and why did I switch?
We evaluated Kafka but decided on Amazon Kinesis due to its ease of implementation and integration with AWS services. We have not used other compute services extensively.
How was the initial setup?
Setting up Amazon Kinesis was straightforward with CloudFormation, involving around ten to fifteen lines of configuration. Setting up the consumer process took more time due to writing the processing logic.
What was our ROI?
Running the same workloads on
EC2 would have been costlier due to additional charges. With Lambda, there is no need for data transfer charges, which is beneficial for less frequent workloads.
What's my experience with pricing, setup cost, and licensing?
Amazon Kinesis and Lambda pricing is competitive, but we noticed that scaling and large volumes could potentially increase costs significantly.
Which other solutions did I evaluate?
We considered using Kafka but did not proceed. I have minimal experience with
Azure.
What other advice do I have?
Constantly track the costs of Lambda as applications scale. An unexpected rise could make
EC2 a more viable option in the future. Keep an eye on the cost and scale usage to maintain control. On a scale of one to ten, I would rate it eight.
Which deployment model are you using for this solution?
Public Cloud
If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?
Amazon Web Services (AWS)