What is our primary use case?
My main use case for Lumigo involves operating a high throughput machine learning inference pipeline and asynchronous microservices combining AWS Lambda, EKS containers, DynamoDB, API gateways, and event-driven queues. As the head of testing, my primary challenge was diagnosing non-deterministic failures, latency spikes, and silent drops during the automated end-to-end testing and regression and performance test suite.
Lumigo has helped across our staging, pre-production, and production environments to achieve end-to-end distributed transaction tracing, and it has allowed our QA and SDET teams to immediately pinpoint the root cause down to specific payloads broken down, stream the API contract, and the database queries timeouts. It has also helped in digging through the fragmented logs and providing us the information we needed.
Lumigo fits into my daily workflow by utilizing zero-code and auto-instrumentation, which has helped us to roll out the distributed tracing in literally minutes, so we did not have to mandate the tedious SDK boilerplates or manual scan creations across dozens of microservices repos. Lumigo helped to inject itself natively on the cloud with OpenTelemetry integration.
How has it helped my organization?
Lumigo has positively impacted our organization with transformative ROI for our quality engineering and delivery cycles, where the meantime to defect and resolve due to debugging failed end-to-end integration test runs has dropped over 68.3%. What previously took an SDET two to four hours of log plunking is now identified within three to five minutes directly via the Lumigo trace, representing a huge improvement on numbers. Additionally, the release velocity with blocked QA pipelines due to flaky or non-reproducible asynchronous microservices errors has decreased by 73.5%, directly accelerating our sprint release cycles. I have seen the infrastructure defect production catch dozens of silent failure states that led to queue overflows and third-party API rate limits during pre-prod load testing before they hit production, which has reduced the infrastructure defect rate by 73.88%.
The improvements in my team's workflow and morale are evident, as the velocity of both the engineering and QA teams has improved, leading to a 48% improvement in internal collaboration, which is a game changer for our overall project, allowing us to collaborate better, understand each other better, and work better.
What is most valuable?
The best features Lumigo offers include the live stream architecture and transaction mapping, which helps to automatically generate and update the visual execution topology for every single test transaction. After seeing the exact flow from the edge API through intermediate queues and containerized microservices down to managed storage, it gives our QA engineers instant context. Additionally, I love the request-response payload inspection feature, which captures input and output parameters, request headers, and invocation payload safely. When an automated integration test fails, the exact offending payload is visible right inside the trace, helping to save our QA and engineering teams a lot of time, leading to at least a 48.33% drop in wasted time previously spent finding the root cause.
With automation saved, automated root cause correlation has helped to correlate logs, metrics, and trace spans across asynchronous boundaries, which is notoriously difficult to debug in event-driven AI pipelines. The live stream architecture and transaction mapping make such a difference for my team, as it generates updates regularly via visual execution and the topology for every single test transaction, providing us with a view through intermediate queues and microservices to managed storage, giving engineers instant context of transactions and payloads, which saves at least 49% of our time and allows overall resources to be used more effectively.
Regarding Lumigo's AI capabilities, I find the governance of security to be really good, and we have seen a positive impact due to the AI capabilities alongside strict compliance regulations such as SOC 2 and ISO 27001, proving it is compliant with applicable regulations and policies. Lumigo's accuracy is close to 100%, making it very reliable. Lumigo integrates seamlessly with other AWS services we use, and we have not faced any issues.
What needs improvement?
Everything works as expected with Lumigo, so I do not have any challenges to report.
For how long have I used the solution?
I have been using Lumigo for almost two and a half years.
What do I think about the stability of the solution?
Lumigo is a 100% stable solution, and I have not faced any challenges or lags.
What do I think about the scalability of the solution?
Lumigo's scalability is impressive, as during high stress testing where our system scales up to tens of millions of invocations in hours, it ingests and processes telemetry without dropped traces or noticeable latency lag in the dashboard UI, making it a great scalable solution.
How are customer service and support?
Customer support is really great, and when I had a query regarding optimizing the tracing rules or simplifying sampling behaviors, their engineering team responded within 45 minutes and resolved my queries. I rate customer support a 10 on a scale of one to ten.
Which solution did I use previously and why did I switch?
I did not use a different solution before choosing Lumigo.
How was the initial setup?
The initial deployment of Lumigo was straightforward and frictionless, as it was onboarded via our AWS account using CloudFormation, taking less than an hour. From a testing perspective, I was able to link trace IDs into my Playwright, Cypress, and Postman automated testing reports within the very first week, making the initial deployment really easy to do.
I was able to configure Lumigo within a week, so it was easy to integrate into the environment we have.
Which other solutions did I evaluate?
Before choosing Lumigo, I evaluated AWS X-Ray, New Relic, and Datadog.
What other advice do I have?
I rate Lumigo a five. I choose a five for my rating because it is a great solution. My advice for others looking into using Lumigo is that anyone looking for an exceptional solution should definitely get them on board.
Which deployment model are you using for this solution?
Public Cloud
If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?
Amazon Web Services (AWS)