We use the solution for streaming data, in simpler terms. For example, there is a backend application; we need to make that data available for analysis. On the backend side, we don't store the history. We get all the events regarding changes incrementally. If something changes, an event is generated. This is a convenient way to keep track of all the changes.
Amazon Kinesis is similar to Kafka, another type of streaming technology, which can be referred to as a queue service to exchange data. Setting Amazon Kinesis up is quick and easy; it only takes a few minutes to configure the necessary settings and start using it. In comparison, Kafka requires setting up a cluster, even if it is available in the cloud, which can be time-consuming. Amazon Kinesis has a user-friendly interface, making it easy to adjust and scale up the number of shards if needed. The cloud is especially useful when starting something new and not needing a lot of resources initially, but with the potential to upgrade later when there is a larger load. Although there is a cost associated with using the cloud, Amazon Kinesis is very flexible and can be easily adjusted when necessary, making it a great advantage.
Kinesis can be expensive, especially when dealing with large volumes of data.
I have been using the solution for two years.
The solution is stable and I don't recall any issues. Once we set the solution up, it usually works and we only investigate if we encounter a problem. However, if there is a large number of events to process, due to limited capacity for example with the shards, then some events may be delayed. This can be easily resolved by adjusting the configuration to provide more capacity.
The solution is scalable, but this also comes with a financial cost. If we want to increase throughput, we can simply increase the number of shards or adjust some config parameters, which can be done in a matter of minutes if we know how to do it. We can scale the solution almost without limitation.
There are a lot of details involved with the initial setup, so if we need something at the outset, we can set up the solution easily. However, the details are important since they are related to how much money we pay and we need to tailor the solution to our needs. If we want to do something more sophisticated, then we need to spend more time comprehending all the details. Initially, we can easily set something up, but eventually, we need to understand it better and adjust it more to our needs.
Cloud services are often cheaper in the beginning, but when the amount of data and needed resources grows, they cost more and more. In my opinion, it is sometimes simpler to use an existing service rather than having to maintain our own internal infrastructure. This way, we can focus on the things we are good at and can make money from, rather than having to employ people to support the infrastructure. In general, cloud services are very convenient to use, even if we have to pay a bit more, as we know what we are paying for and can focus on other tasks. However, if the scale is large, I would consider making changes depending on the situation.
I give the solution a nine out of ten. Amazon Kinesis is easy to use and configure, especially in the beginning. The solution is stable and I have not encountered any issues with it, nor am I aware of any. The solution is effective.
I don't see any missing features in Amazon Kinesis. I haven't spent a lot of time with this interface, as I have only configured it once. If any changes need to be made, I simply adjust Amazon Kinesis and it works. I only go into Amazon Kinesis if there is a need for a new data stream to be included or if the throughput needs to be increased. This doesn't happen very often.
Depending on the requirements, if there is a need to stream data and access it in real time, then I would consider Amazon Kinesis. However, if there is no need for real-time data access, then I will look for some other cheaper options. Companies such as Redshift, Snowflake, and BigQuery are developing databases with built-in streaming functionality. Depending on the case, this may be an option to consider. It also depends on the target; sometimes it is better to use the mechanisms available in the target tool. If we want to have the data on a stream or some hot stories, then I would consider Amazon Kinesis in that case.
