What is our primary use case?
My main use case for WhyLabs is its robust data observability capabilities; it provides continuous monitoring of data pipelines and ML models, allowing my team members to quickly identify issues like data drift, model degradation, and training-serving skew. The platform's privacy-preserving integration ensures that the data can be analyzed without moving or duplicating it, which is critical for maintaining security and privacy in sensitive industries like ours.
A particular time when WhyLabs helped me catch an issue early was when it detected and addressed data drift and data integrity problems, which is crucial for maintaining accurate and reliable machine learning models.
The monitoring capabilities of WhyLabs AI Observatory are the main reason why we use it, as we have very limited control of external LLM and services. WhyLabs provides alerts and insights into those LLM models, enabling us to identify and address potential issues promptly.
How has it helped my organization?
Setting up the monitoring has impacted my team positively, as monitoring model performance and input data quality in one place has helped us to catch issues very fast. I love it because it helps detect and address data drift and data integrity problems early, which is crucial for maintaining accurate and reliable machine learning models.
WhyLabs solves most of the machine learning model monitoring needs that models often have while being affordable.
In terms of accuracy and reliability, consistent, relevant outputs help us improve productivity, streamline our workflows, and support better decision-making.
What is most valuable?
The best features WhyLabs offers include very helpful and proactive customer support, a tool that allows for easy ingestion of a big number of features and setting up initial monitoring on them, and the ability to raise alerts to a specific group of users via specific channels such as Slack, which is very helpful. One of the features that I heavily rely on is the robust data observability capabilities.
What makes the data observability capabilities stand out compared to others I have tried is that it provides continuous monitoring of data pipelines and machine models, allowing my team to quickly identify issues such as data drift, model degradation, and training-serving skew.
I also love the platform's privacy-preserving integration, which ensures data can be analyzed without moving or duplicating it.
I love the alerts where we are able to raise them to a specific group of users via a specific channel, and we use Slack, which is very helpful.
Regarding WhyLabs's AI capabilities, I think its governance and security features are very promising; our data is handled very well with a lot of security and care.
What needs improvement?
It can be challenging to set up monitoring in the correct way when it comes to sensitivity, requiring a lot of trial and error.
A few cons include that the dashboards are in beta, and while functional, they lack polish in terms of user interface.
In the dashboards, I wish to see specific functionalities or visualizations that they are working actively on; probably a few months after this review, they may get it ready to fix it.
Additionally, defining groupings by variables must be done at ingestion time, limiting flexibility for post-ingestion analysis.
For how long have I used the solution?
I have been using WhyLabs for two years.
What do I think about the stability of the solution?
WhyLabs is very stable; it focuses on monitoring data quality, detecting anomalies, and tracking model performance, which helps my team identify potential issues early and maintain a reliable AI workflow.
What do I think about the scalability of the solution?
WhyLabs's scalability has been very promising because it offers strong scalability potential for my organization, enabling us to manage our growing AI and machine learning workloads efficiently.
How are customer service and support?
I have had to reach out to customer support several times, and they are very easy to reach out to and very responsive and professional. My experience overall has been very positive, especially because of the available documentation and technical resources, which were helpful for understanding the platform, configuring monitoring, and addressing implementation questions. They were quick to analyze and offer solutions, and I appreciate responsive technical assistance and clear guidance, especially when integrating our AI monitoring into our existing AWS workflows.
A nine out of ten would be a fair rating for customer support.
Which solution did I use previously and why did I switch?
We did not switch from any solution.
How was the initial setup?
Deploying WhyLabs in our environment is easy and straightforward; the setup process is manageable, and the documentation helps us understand the configuration and integration requirements.
My experience with the configuration process is relatively straightforward and easy because the documentation and available configuration options help simplify integration with our existing environment.
WhyLabs integrates well and very seamlessly with our AWS-based environment, supporting our AI observability and helping us monitor data quality and model performance.
What was our ROI?
We have seen a positive return on investments mainly by reducing manual monitoring to save on time and cost, identifying data quality issues earlier, and minimizing time spent troubleshooting AI models. This has improved our productivity, reduced operational overhead, and helped our engineers focus on higher-value tasks, and this is promising to our customers because the projects are delivered on time or hitting their deadlines.
What's my experience with pricing, setup cost, and licensing?
The procurement process of WhyLabs was relatively straightforward due to the available product information and licensing options that helped us understand the requirements and evaluate its suitability for our AI monitoring needs.
WhyLabs offers a promising approach to managing AI monitoring controls and costs; I value the transparent billing, clear usage metrics, and predictable pricing because they help with budgeting and resource planning.
WhyLabs offers good potential value for any organization looking to improve AI observability and model monitoring; the pricing and licensing have been manageable overall.
Which other solutions did I evaluate?
Before choosing WhyLabs, we considered two other AI observability tools and monitoring solutions, which included Evidently AI and Fiddler AI.
What other advice do I have?
I would rate WhyLabs a four out of five.
The reason why I chose a four is that WhyLabs delivers all the standard features metrics accurately; tracking data and model drift is very straightforward using monitors. The platform also supports custom metrics creation during or after ingestion.
My advice for organizations looking to improve AI observability, data quality, and machine learning model monitoring is to go for WhyLabs.
If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?
