Microsoft Azure DevOps Reviews

When I was working at Microsoft, I was one of the core influencers on the feature set and had deployed this solution internally across several organizations. We used it for anything from its CoreALM feature sets to inventory tracking and workflow management and operation support and finance management. There were a bunch of other scenarios. At its core, it is a database with a front end that easily makes it so that you can create new forms and stuff. Then they expose an API, which means you can do a lot of things with it beyond its core use cases.

It becomes a single source of truth for whatever operation is implemented within it. You can have your product definition in there from a requirements management standpoint and then log in bugs and defect management and RPNs and a bunch of other things. You have this single source of truth as they provide an analytics service, and then also easily tie into Power BI. It's really easy to just look at the health and overall operation of your entire business from a single source.

The extensibility of the work item forms and customizations as well as the backend API to query the data, et cetera, and manipulate the data programmatically are all very valuable aspects of the product.

The UI, the user experience, is challenging for newcomers. Once you get it, you get it, and it's not too bad. However, it takes some effort to learn how to work with the system. There's a moderate learning curve. I've used both Jira and Azure DevOps, and I would have that same feedback for both tools.

The biggest challenge has been that both Azure DevOps and Jira tend to pivot more towards software development and the industry is going more towards full end-to-end product development - hardware and software. These platforms could do a lot more to help support the mechanical, electrical, controls, robotics, and more of the hardware side of things.

I started using the solution in 2010. It's been about 12 years. 

I haven't had a problem with stability. There aren't bugs or glitches. It doesn't crash or freeze.

When you surpass a terabyte of data from a work item standpoint, certainly there are some limitations in performance as the running is querying that large data set in the backend.

I've done multiple different deployments. Sometimes, the smaller, smaller deployments have been a handful of five people, and it might be three software devs, a test engineer, a hardware engineer, and a PM. 

I've also done larger deployments where it's 8,300 people. That was a mix of hardware, software, PMs, firmware engineers, front-end, full-stack devs, mechanical engineers, electrical engineers, et cetera. 

In the deployment that was 8,300, it's actually still deployed there and growing. Another deployment I was a part of that was a medium deployment - 20 users - has since reduced more due to politics and going back to the front-end, ease of use. There are some folks that it was too high of friction to use it. You can scale it up or down to match your needs.

I don't deal with technical support in the traditional sense. I know the developers who've developed it, so I just go talk to them.

I've also used Jira.

I previously used SharePoint and Microsoft decided the direction of SharePoint to be less workflow-oriented and less list-oriented and more as a document store. As their roadmap moved away from work management, I've moved over into the TFS/Azure DevOps world.

I was a Microsoft employee. There was some natural tendency to just go with the Microsoft tool, however, it wasn't a hard, fast requirement when we just looked at the feature sets and stuff. 

I've been a developer on the backend. In terms of setting up the product, my answer would be highly complex. If I were just doing it for a core user, set of users, then I would say the setup was relatively frictionless. I would say the one point of ambiguity is for some newcomers if they don't understand the difference between CMMI templates versus Agile, versus Scrum, they'll find it complex. I've seen a lot of new users create dummy projects to then go in and see how each of those is configured from a template standpoint. Work could be done there to just reduce that level of friction.

In terms of deployment times, I've been on multiple different sides of levels of deployment. From the simple side, I've seen deployments take as little as a couple of minutes. If it's teams of five, for example, they go into the web app, they start up a new project, and boom they're in. They get all the requirements and user stories and all that stuff done. 

I've also been on the other side where it's been nine months with 22 people working full-time to configure and deploy this system across thousands and thousands of users. It just depends on the size.

It's hard to put a number on managing the plan of record. I haven't tried to calculate an ROI. It does what it's supposed to and it's more accessible than an Excel sheet.

The solution costs $5 or $10 per user, per month. It's a nominal fee.

I would actually prefer it in that they give you the first five users for free. That little bit of free users goes a long way, just from an initial trial and adoption standpoint. I would encourage them to keep doing things like that.

If you use the other services - if you use their build and compute engines and stuff like that, they charge some amount for computing and some of their extensions. These are not necessarily Microsoft's extensions. They are third-party ones and they'll charge. Depending on if the feature is core to the product, or it's an extension, it might or might not cost you something extra.

I've evaluated pretty much every ALM.

We're just a customer and an end-user.

I've used all the versions, starting back with Server 2005. Now I'm just using their online version.

In terms of advice I'd give to new users, I'd say it goes back to basic change management. Understand who your attractors and detractors are. Lay out the feature sets, ease of use, and things like that. That at least will help detractors become a neutral party as there are always going to be people that create friction within a deployment. Just have an effective change management plan. I've looked at over 12 different ALMs and they all have their pros and cons. It really just comes down to just picking one and going forward with it and learning it.

I would rate the product at an eight out of ten.

We use Azure Repos for most things. It integrates with Azure Boards. 

It's good for the ticketing part and for saving the Azure Repos. We use Terraform.

Azure DevOps integrations with other tools have streamlined our workflows.

The centralized code repository is a major part. We store code and collaborate, and everyone can see what others are doing and what code they're adding. We can review the code and make changes if needed. 

The same code used for implementation is visible to other team members, allowing them to contribute. Additionally, the Azure board interface helps create tickets and assign workloads, keeping everyone informed about progress.

Azure DevOps is effective for repository management and code collaboration. We can create branches, differentiate between code versions, save and review code later, get PR approvals, and more. All these features are valuable.

There is room for improvement in customer service and support. Sometimes, I create a ticket for a specific issue, and they tell me it's not relevant to that ticket. 

They ask me to create a different one, basically saying they can't help with the current one. It's a communication gap. We're troubleshooting, so we don't always know the exact issue. They should let us stick with the same ticket and maybe assign a different engineer if needed. These areas definitely need improvement.

Another area is the Azure monitoring agent for Citrix machines. There's room for improvement there too.

I have been using this solution for four months. 

It's a very refined tool from Microsoft, so there haven't been any problems with stability for us.

Scalability is great. We can add as much code as we need, and the deployed code volume can also be scaled. 

The scalability comes in the sense of creating virtual machines using IaaS, which works really well within the suite.

I'm on the DevOps team, and around 25 to 30 of us in the team itself use Azure DevOps. So, overall, there are a lot of end users in my company. 

The customer service and support are good, but there are some issues when we're doing a deployment and need clarification. They're not very helpful there. Maybe there's a separate team for that. 

But in general, for project creation and work, after everything is deployed, Microsoft can help with a general support ticket. But they won't help with the planning phase. They're more like big management. So, there's a gap in the assistance we need for new project deployments.

Neutral

For ticketing, we use ServiceNow. Both solve the same problem in that regard. And for code repositories, we use GitHub. So, it's not an all-in-one solution like Azure DevOps, which has a lot of features bundled together.

Since Azure DevOps offers everything in one place, it feels more convenient. It does a bit of everything. So it's a good option because it replaces various individual tools and provides all their services in one package.

It is very straightforward. It was easy for me to deploy. 

The deployment depends on the workloads we have. For example, deployment for setting up Azure DevOps or the application itself isn't the same.

It takes a lot of time to get the setup ready. 

Regarding setup, connecting it to Visual Studio was smooth on both Mac and Windows. Integrations are quite good. 

Deployments depend on the workload. We create virtual machines using Terraform, so it's usually fast, especially when downloading repos from Azure Repos.

Smaller tasks like deploying a prepared code for a single service wouldn't require additional engineers. Many workflows can be handled by one person. 

Architecture is different, as the architect designs the infrastructure, which needs to be followed.

ROI depends on the cost optimization we can achieve. Sometimes, clients use heavier resources than they actually need. So it depends. 

If the environment is fully optimized, there can be significant savings, leading to a good return on investment. But they would also be paying for partner management. 

For Microsoft, it can get expensive when you need heavy-duty machines. But compared to on-premises solutions or the hardware we used before, it's still much more convenient for us. So even though it can be pricey, the benefits outweigh the cost in our case.

Maybe some more flexible payment options could improve the pricing. 

Overall, I would rate the solution an eight out of ten. 

We are a team of 12 consultants specializing in this solution. We have customers with approximately 200 users each, and we have been able to reduce our customers' deployment time by using Azure DevOps or TFS.

Additionally, our customers have been able to measure and improve their development process, by generating some KPI's as 'average defect fixing time', 'effort deviation', and 'velocity' among others, due to the continuous use of Azure DevOps.

The integration of the whole cycle is the main strength of the tool. If I want to control the cycle with other tools on the market, I would have to use several tools. However, this is not the case with Azure DevOps or TFS.

The ability to extend work items was more flexible than it is now. Talking version control, one of our customers had some issues because they found it very difficult to manage more than 1000 repositories for one team project.

Additionally, I would like to see more powerful dashboards that could be used instead of Power BI. Azure DevOps or TFS does not support graphics from hierarchy queries. It would be powerful to have because some customers don't have Power BI licenses.

We have been working with Azure DevOps or TFS for the past 11 years.

It is totally stable.

The documentation is good and thus, we have not had to contact technical support very often. However, when we have escalated an issue to technical support, they have been good.

The initial setup can be straightforward if the consultant is knowledgeable.

The pricing is very competitive because of the whole development cycle by Azure DevOps. You don't have to buy and integrate several different tools.

Azure DevOps or TFS is a very good tool for development teams. It's easy to use. However, you would need the help of a consultant who has a lot of experience with the tool. This is because the tool let's you do things in many ways, but not all of them will be the right way to do it. It would be better to invest in the services of an experienced consultant.

Overall, I would rate TFS or Azure DevOps at nine out of ten.

There are many valuable features including dashboards, sprints, queries, pipelines, artifacts, and the cover repository.

I would like to see improvement in the metrics and the dependencies. I would also like to see the option to define the dependencies within all teams in the same project. We have a team level, a program level, and a portfolio level. For the metrics portion, I would like to see some drag-and-drop features for the dashboards that would make it possible to aggregate data from the different teams.

The plugin for the iteration walls can also use improvement as it does not work well.

I have been using the solution for three years.

It is stable. Whenever there has been an issue the solution slowed down but there was never any data loss.

The solution appears to be scalable.

I am not familiar with the initial setup.

The implementation was done in-house.

The cost is reasonable. For the basic license, it is around five euros per month. 

I rate the solution nine out of ten.

If someone is considering developing and deploying the infrastructure in this solution, then using this tool is perfect because it's fully integrated with the pipelines and with a server core repository. When you are building infrastructure, then you are able to use the same tool to deploy the server, deploy infrastructure, and all within a few minutes. This gives you access to the same tool, task management, dashboards, pipelines, and the server core repository. So everything you need to develop a server is integrated into the same tool.

We're using Azure DevOps Services for three things: First, for project management, second, for storing the source code, similar to GitHub Repository, and finally, we use it as our CICD build server or build environment, which builds for us and runs tests and so on. In general, these are the three main use cases for this product. We are large customers of Microsoft and we're on a corporate level with them. We pay extra for support. I'm a software engineer manager. 

I like that this solution is all-in-one, a one-stop-shop, it's the killer feature. I haven't seen anything that comes close. I guess GitHub will be close soon, but that's it, there's really nothing right now for that full integration. Other solutions require three tools so this is really a great feature. The solution has a better user interface and better CICD tools compared to what we used previously when we ran TeamCity. I think it scores higher on most things, including better developer ergonomics. Since it's Git-based, there's no training because everyone uses Git. I've found it to also be very customizable so that on all points it's better. This is an important tool for us. 

This solution is not as good as Jira when it comes to project management and I think they know it, but it's good enough. I'm very used to it now, so I can work more quickly, but I've had colleagues who are very Jira-focused and they don't like Azure DevOps at all. When it comes to the handling of tickets or tasks or the product backlog, Jira is much more customizable and more intuitive. It's an area that Microsoft could improve. 

The instructions could be a little better. We are doing some weird stuff where we're building some things, including embedded firmware. It wasn't super intuitive to set that up which was an issue although it's something minor and we managed to solve it. I just expected it to be a little easier, although it's not what the solution is built for. We're going a little out of the normal use case. It is a little clunky compared to Jira and hosting your own builds could be a little easier.

I'm aware that they're putting money into GitHub to add more features around vulnerability scans and statical analysis and so on, basically taking on cloud and what have you, as well as Vericode that we are using. It would be great if it was built into the tool. I get things from other vendors that are provided out of the box, and it would be awesome for me to have that with DevOps. 

I've been using this solution for several years. 

The stability of the solution is good. We've had a couple of dashboards out and they have a nice page share where they show what's out and what's not. A few months back they had some issues with the Active Directory and we were pretty much locked out of some things. We lost Teams for a while and we use that a lot in Azure DevOps. It was quickly fixed. Otherwise, I'm very happy with the stability. 

The scalability of the solution is good and there's no maintenance required. We're a small operation and we could grow by a factor of 10 and it wouldn't be a problem. This is an SaaS and if you need to take care of it, there's something wrong. We use the solution extensively and soon we'll have almost every piece of software, including all our test automation and embedded firmware there so we'll be increasing usage. 

The company previously used TeamCity, and I have used Jenkins in the past, the grandfather of everything. Azure DevOps is nicer. Jenkins is very configurable, but a pain. I like Azure a lot more and I think this or something like it, GitHub Actions, for example, is the future.

The initial setup is very intuitive. What I think they could work on is the whole permissions model where you have projects and other things which require permissions and which is not very intuitive. You can do almost everything but I want a more granular permissions model that's also easy to maintain. I don't quite like the way it's set up so there's some work to be done there. I think I'd rather do it in text because it's hard to see everything clearly otherwise. If you have a complex permissions system, it's complex to set up and it's not super intuitive. Compared to AWS, which is a very different system, that aspect of Azure is not very intuitive.

I work in an engineering department so we didn't feel the need to get any help with deployment. If you read the manual, create the sandbox, and test it out you're able to roll it out. It's not that hard. 

We're not paying a lot for this product. As developers, we have a Visual Studio license which is basically free. That's how their licensing model works. Then we have a number of stakeholders who need to do edits in the system, but not work with code necessarily. I believe they're paying $5 a month per user. We also have users who only need to read things and don't need code so I set that up for everyone who needs it. We're probably paying a few hundred dollars per month altogether. That's a minor cost for us; we're not currently hosting anything on cloud, so it's a small cost compared to hosting a solution. 

We ran into a few things where we had to pay more because of the number of concurrent building agents. We had capped it low and the developer was unhappy so we paid a little more to get what we needed and that's been good. I don't like it when you get a big bill and you don't know about it. 

I'm somewhat critical of the documentation for certain things, but overall, the documentation is really good. In general, Microsoft is really good at documentation. It's worth taking a few hours to read it and then you'll know a little about how Access works. If you set up a sandbox, you're not going to destroy anything and you'll learn by trying things out. I would still read the documentation and go in parallel so you can at least know enough and be aware that it's safe to get in there.

We are very heavy users in creating small projects and then sometimes deleting them because they weren't useful but I like that model. Create a little sandbox and go build. We have done our own workflows and they are always tested in a sandbox before going live. That would be my suggestion. 

I rate the solution eight out of 10.  

It is used to manage our projects. We basically maintain what would be the equivalent of our project schedules for various projects. So, we capture or create user stories to identify elements that need to be accomplished for the delivery of a project and to track who is responsible for it and the level of effort. We aggregate that within the tool and report out to leadership about the status of when we anticipate completion.

We are using its latest version.

Its integration with different functions has been very helpful. Previously, we had Microsoft Project schedules, and we did our reporting by using Excel and PowerPoint presentations. We also did testing tracking in other tools, such as HP ALM. Our source code was on Teams Foundation Server. All that can now be done within DevOps, which is a huge benefit. Things that we used to do in different tools can now be done in one tool. 

I like the fact that there is built-in Power BI. Both are Microsoft tools. So, you can incorporate dashboard capabilities. 

I also like the integration with the other toolsets, such as Outlook and GitHub. You can do your testing and check your source code within the same tool. That's definitely something really good.

The tool was developed for Agile project methodology, but I've noticed that there has also been a try to incorporate what is typically done in MS Project, which is for more sequential Waterfall projects. The problem with that is that it is half-baked for Waterfall projects. If you're going to do it, then either go all the way and allow us to use the tool for both or don't do it at all.

One thing we had to customize ourselves was to create the critical path. You can't do your project dependencies within the tool. We tried using the tool for a Waterfall project, and we had to find a custom approach to do that because. There should be some functionality for the reporting and dependency tracking for the Waterfall projects.

I have been working with this solution for two to three years.

So far, so good. It has definitely been sized appropriately for our use. We haven't had any issues with it.

We've only been using it for about three years, and so far, it seems to be able to adapt to our growth. We're maturing into it. We're moving in the direction of using it more, and I feel confident that it'll scale appropriately.

We have at least a hundred people using the tool. There are different degrees of people who are using it. Some people are using it in the read mode or view mode to keep themselves informed of where things are. We have some project managers who actually use the tool, and then we have a couple of administrators. I'm one of the administrators for our program. I have a couple of vendor or partner folks who are also administrators. We also have a development team that does some customizations on the dashboard and the Power BI reports that we do. These are pretty much different roles or layers that we have.

We do grant developers access to be able to make their own updates within the tool. Typically, project managers or scrum masters do that, but we also have some team members who are on these projects and have enough understanding of how the tool works and how we're using it. They are able to do their own reporting and their own updates on their statuses.

In terms of plans to increase its usage, we're moving in that direction. Most of our projects are done in Microsoft waterfall project management schedules, but we are being encouraged to move over to more of an Agile approach on our project methodology. Our mandate is that if you're going to do anything Agile, use the DevOps tool.

I have not interacted with them. We have a sort of layer for support. I have had to reach out to one of the three resources that we have. He is our true admin at the company who had to reach out to their support, but it has been seldom, at least from my experience.

I used Jira while working with a vendor that we had here for one of our projects. They brought that tool from their practice. We were doing that because we had not yet moved to DevOps. After they rolled it out at the organization level, the mandate was to stop using Jira and switch over to Azure DevOps. There are a lot of benefits to Azure DevOps over Jira, but Jira is the one that has a lot of market share on that side.

I wasn't involved in that, but I do know that, just like many tools, there is a learning curve that was associated with that. I have used Jira before, so I had more or less an understanding because it is very similar to Jira, but I know that for other people I work with, it was a completely new concept to use something like this.

For its maintenance, we have a small team. We have about three individuals who do the backend support. So, it is minimal. Obviously, if they have any escalations, then they do go to Microsoft, but we haven't had that happen. It was very minimal. There are plugins that are available to enhance kind of some capabilities of the tool. When we ask for that type of functionality, these three individuals have been able to implement plugins for us.

It is an Agile tool. We were using the tool calling that we were Agile, but we were really doing things in the Waterfall methodology. It was our square peg in the round hole, and that's where I realized that we didn't have the capabilities in DevOps to use it as a Waterfall tool, which makes sense because Agile is a different approach. We've evolved since then, and now, we're doing a bit more Agile when we use the tool. So, a tool is just a tool. There has to be that thinking alignment. Otherwise, it is a square peg in a round hole, and it doesn't quite fit. Your organization and your team have to understand that. Just using the tool doesn't make you agile.

The only problem we had was when we rolled this out, we didn't realize how Waterfall we really were. So, I had to go back and have PMs create additional data elements for us to capture what we really wanted to capture to report in Waterfall. Dependencies weren't tracked, and we had to go back. It almost felt like we had to do rework, and people weren't too happy about that.

I haven't used its mobile device capabilities, but that's definitely something that I would hope to evaluate in the future. 

I would rate Microsoft Azure DevOps an eight out of 10. Overall, I'm pleased with the tool, but there is definitely some room for improvement.

We use Microsoft Azure DevOps for CICD, and to organize it in order to visualize the ongoing work.

It allows you to save time while also providing a governance visualization of ongoing activities and transparency.

The most valuable feature of this solution is that it saves time.

The price could be reduced. It is expensive, especially when it comes to infrastructure.

The integration could be better. Being more technology-agnostic through ease of integration would be beneficial. Once you start working for Microsoft, you are frequently tied to Microsoft.

I have been using Microsoft Azure DevOps for the last ten years.

Microsoft Azure DevOps is a stable solution.

Microsoft Azure DevOps is scalable.

I would say the technical support is fine, but I have not had any trouble with the solution.

I have some experience using Jira.

It is very expensive in comparison to others.

As the cost structure is per user, I would recommend paying the cost structure based on the amount of data you use rather than the number of users.

I have recently researched Jira, Microsoft DevOps, TFS, and Micro Focus.

Mostly, because of the pricing, I would rate Microsoft Azure DevOps a seven out of ten.

We use it for requirements, development work, and testing. 

We're doing an implementation at the moment with the client. So, it is the latest version that would've been uploaded.

It is a cloud-based system. So, it is stable and scalable.

Azure DevOps is set up more for development and less for testing. If it is set up correctly, everyone can use it better, but it was set up from a development point of view, which means it is lacking in what I need from a testing perspective. Just like any other tool, it depends on how it is configured. I am not happy with the way it is set up. It is configured more from a development side, and it doesn't necessarily cater to all the other areas that probably need to use it, such as testing data, etc.

I have been using this solution for about 15 months.

Its stability and performance are okay. It is on the cloud. As long as you have got access to it, it is stable.

It is a cloud-based system. So, you can add more bandwidth. It is scalable on the cloud.

We have about a hundred users who are using this solution. It is used on a daily basis.

A third party deals with the technical support of it at the moment.

I wasn't around when they initially set it up, but the way it is set up, it is too complex. It is probably good for developers, but it is not good for the testing side. 

It is a third party that sets it up. I don't know about its maintenance because I'm not that close to it.

I would advise organizing and doing the right assessment for all teams that are going to use it. When it is being set up, more people within the program need to be involved in the setup, not just the developers. You need to know about the requirements for design, development, testing, integrations, and architecture. You need to solicit requirements on what each one of these teams needs from the tool before the tool is configured. When you set something up only from the development perspective, you forget that there would be a need to extract information for data testing and training. So, you need to assess who all are going to use the tool so that you set it up for maximum usage.

At this time, I'd rather not recommend it because it wasn't set up correctly. It wasn't set up with other teams involved. In a year's time, if I'm working on it again, I may have a different opinion.

I would rate it a five out of 10.