We manage log processing with Grafana because we found that it is much easier for us to manage it on our infrastructure on AWS. We can maintain all the things we do not need. DataDog told us we have to wait and that they need to provide features we need to develop, so it does not suit our needs.
We are still using DataDog, but for important assets that we need to analyze the logs, we send it to Grafana.
The challenges we face with DataDog compared to Grafana include the need to analyze very important brands, network trafficking, and maintaining many websites, most of which are very important domains that cost a lot of money, so we are getting attacked each day and we need to analyze all of the logs. Sometimes we have false positives and things similar to that, so we have to make sure that we are doing the correct decision of blocking or trying to mitigate attacks. Using the logs with Grafana it is much easier for us to analyze rather than DataDog. DataDog has their own language and they want you to plot things with their own vocabulary. We do not have time to memorize things. We especially wanted to use something that was open source at the beginning, and then other people started using it, took that product and modified it for extra cost, but it is a better solution for us.
We switched from DataDog to Grafana because we wanted to reduce the logs costs, as we are streaming approximately five million logs or even less.