What is our primary use case?
I have been working in my current field for over seven years as a DevOps and site reliability engineer, and my primary experience involves managing the reliability of infrastructure platforms hosted in multi-cloud and on-premises environments. I have predominantly worked with systems hosted in AWS services, setting up infrastructure, CI/CD, observability, and completely establishing the release process where I utilize PagerDuty Operations Cloud for triage and other SRE operations.
I have been using PagerDuty Operations Cloud for over four years, and I have utilized it in multiple ways. One involves using PagerDuty Operations Cloud through enterprise services via a subscription model, and I have also used it in a project at Intel where I utilized PagerDuty Operations Cloud from AWS for approximately one to one and a half years. After that period, I have been using it as a subscription currently at IBM.
One of the main use cases for PagerDuty Operations Cloud involves handling the operation center, particularly concerning incident resolutions and triaging different incidents as part of the score platform engineering team within a central IBM cloud where various IBM cloud services are hosted. To ensure continuous reliability, automated incidents are created in PagerDuty Operations Cloud and incident management automation is heavily utilized as part of the project. Previously, I worked on integrating PagerDuty Operations Cloud with default AWS services to create incidents for different AWS services as part of the host infrastructure at Intel. Currently, I am creating different incident workflows within IBM internal cloud operations to ensure an effective incident management process, utilizing integrations with different LLMs as part of incident management, along with agentic SRE tasks that have arisen in the project.
Since I am part of a larger platform engineering team and SRE operations team, there are many incidents and services that my team handles. I handle over 26 IBM cloud score services hosted in our internal platform, where there have been many incidents related to service downtime, reliability issues, and update issues. A dedicated SRE team handles end-to-end incident management, and we wanted to automate the incident management process, especially since we receive hundreds of incidents per day, up to thousands of incidents during critical release times of different services. Thus, the manual on-call process has been automated through utilizing PagerDuty Operations Cloud.
What is most valuable?
One of the features I find valuable in PagerDuty Operations Cloud, which is part of our current migration activities, involves automating the entire incident management process by integrating all service incidents into a single incident management page. In IBM cloud, since many services and incidents occur, I utilize the runbook automation feature where I create runbooks for each service and common issues that facilitate incident management for common incidents.
The runbook automation positively impacts my team's workflow by significantly speeding up the incident resolution process. In IBM cloud, we have different services hosted such as IBM Schematics and IBM Kubernetes Service, with thousands of concurrent global users. We face several issues during multiple incidents, particularly in reliability and infrastructure side issues. Basic level zero incidents often require simple commands run in kubectl. Therefore, I created runbooks to address these common issues, allowing SRE team members to refer to the runbook and manually fix the issues. However, before using PagerDuty Operations Cloud's runbook automation, it took over 20 minutes to resolve these issues. After implementing PagerDuty Operations Cloud's runbook automation, I have reduced the response time from over 20 minutes to less than two minutes, saving approximately 80 to 90 percent of the time and making mean time to resolve significantly faster. I review the runbooks quarterly to update them with any new steps necessary.
PagerDuty Operations Cloud has greatly improved our productivity. Previously, I handled many incidents with manual automation runbooks, leading to substantial toil for the SRE teams in resolving even minor incidents and complicating our on-call schedule. Once I adopted PagerDuty Operations Cloud and heavily utilized the runbook automation, I provided a list of common incidents to PagerDuty Operations Cloud, which then did the heavy lifting in fixing basic incidents automatically. This allowed my team to focus more on development activities related to platform engineering. Overall, it has reduced our toil by at least 50 to 60 percent and improved our efficiency, enabling us to onboard more services. We increased from onboarding seven core IBM cloud services to over 28 services now hosted.
The expansion of services impacts our organization's goals and customer experience by allowing all IBM cloud internal services to be hosted on a dedicated platform engineering service called Rednote. Before using PagerDuty Operations Cloud, I utilized Nagios and Sysdig, and after migrating to PagerDuty Operations Cloud last year, I prepared a set of runbooks, automating on-call schedules and incident management. This automation has led to a significant increase in incident closure rates from around 40 to 45 percent, improving efficiency and reducing manual toil for basic incidents by about 60 percent. This enables my team to focus more on development activities as basic incidents that can be managed through simpler runbooks are now handled automatically by PagerDuty Operations Cloud. Additionally, incorporating AIOps into our on-call scheduling and notifications helps it learn from previous incidents and proactively address issues. This scalability has allowed me to grow from handling four cloud services to 28 services, increasing from 200 to 300 customers to over 1800 plus customers, thanks to PagerDuty Operations Cloud.
What needs improvement?
Since I host our internal services, I want more customization relating to our specific use case.
The needed improvements include the configuration process, as new team members face a steep learning curve to understand the platform. With many new members, they need training to set up runbook workflows, event orchestration, and manage complex on-call schedules across 23 services, making it a challenge for new users. Additionally, I feel the web interface requires improvements.
I would rate PagerDuty Operations Cloud as eight out of ten because the cons include a complex configuration process and high costs for each add-on that I try to obtain, making subscriptions costly, along with limited customization in certain incident workflows.
The primary reasons for rating it an eight include the complex configuration which makes it challenging for new users, as well as their difficulty in setting up advanced runbook workflows and managing complex on-call setups. The web user interface also requires improvement. Although I receive alerts via the mobile app, which is beneficial for handling schedule maintenance, the same features should be added to the web interface. Customization issues persist, such as the inability to clone entire schedules as part of the workflows, and I want to keep incidents open for a specified duration, neither of which I can currently customize. Thus, I raised a ticket with PagerDuty Operations Cloud to address these concerns. Furthermore, the cost is high, making it one of the more expensive incident management solutions.
For how long have I used the solution?
I have been using PagerDuty Operations Cloud for over four years.
What do I think about the stability of the solution?
PagerDuty Operations Cloud is definitely stable, providing faster incident management and making managing our on-call roster easy along with effective escalation and notification channels.
What do I think about the scalability of the solution?
Regarding scalability, I do not find many issues. PagerDuty Operations Cloud effectively handles concurrent incidents, and incidents are fixed properly and on time.
How are customer service and support?
My interaction with PagerDuty Operations Cloud's customer support mainly focused on customizing our workflows. They understand our concerns and are willing to implement solutions that integrate into PagerDuty Operations Cloud effectively. From a reliability perspective, I have not faced any issues, and their support provides timely assistance for custom integration and workflows.
How would you rate customer service and support?
Which solution did I use previously and why did I switch?
Previously, I used RunDeck automation, which is why I switched to PagerDuty Operations Cloud after PagerDuty Operations Cloud acquired RunDeck.
How was the initial setup?
Before using PagerDuty Operations Cloud, I utilized Nagios and Sysdig, and after migrating to PagerDuty Operations Cloud last year, I prepared a set of runbooks, automating on-call schedules and incident management. This automation led to a significant increase in incident closure rates from around 40 to 45 percent, improving efficiency and reducing manual toil for basic incidents by about 60 percent.
What's my experience with pricing, setup cost, and licensing?
I purchased PagerDuty Operations Cloud through AWS Marketplace while at Intel, and my experience has been positive regarding pricing, setup costs, and licensing. I had around seven users part of it for a base pricing of around $450 per user, primarily for custom workflows and the ITSM part. Currently, I am utilizing the runbook automation part, which costs around $2000 per year, and in the last three months, I have also used the AIOps feature for approximately $700 to $800 per month, resulting in a cumulative cost of around $3000 per month.
Which other solutions did I evaluate?
Before choosing PagerDuty Operations Cloud, I evaluated other options, particularly competitors such as ZenDuty due to cost effectiveness, but I favored PagerDuty Operations Cloud for its RunDeck features and automation capabilities for incident workflows, despite the required migration.
What other advice do I have?
Utilizing PagerDuty Operations Cloud allows me to save a significant amount of time, not only on routine incidents but also in focusing on onboarding additional services. This significantly aids me in spending less time on routine operational activities, quantified by the reduced personnel needed to manage routine tasks.
I highly recommend using PagerDuty Operations Cloud if you have numerous operational incidents to handle daily, especially if you prioritize reliability, particularly in critical projects such as IBM's core cloud services where outages must be avoided to ensure compliance and reliability.
I urge potential users to adopt PagerDuty Operations Cloud if reliability is a priority and they have a sufficient budget, as it is suited for larger infrastructures and effectively manages redundant incidents, standing out as the number one option in its market segment. I have rated PagerDuty Operations Cloud as eight out of ten overall.
Which deployment model are you using for this solution?
Hybrid Cloud
If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?
Amazon Web Services (AWS)