Coming October 25: PeerSpot Awards will be announced! Learn more
Program Test Manager at B and H Designs
Real User
Top 5
A cloud-based, scalable solution but it is set up more for development and less for testing
Pros and Cons
  • "It is a cloud-based system. So, it is stable and scalable."
  • "Azure DevOps is set up more for development and less for testing. If it is set up correctly, everyone can use it better, but it was set up from a development point of view, which means it is lacking in what I need from a testing perspective. Just like any other tool, it depends on how it is configured. I am not happy with the way it is set up. It is configured more from a development side, and it doesn't necessarily cater to all the other areas that probably need to use it, such as testing data, etc."

What is our primary use case?

We use it for requirements, development work, and testing. 

We're doing an implementation at the moment with the client. So, it is the latest version that would've been uploaded.

What is most valuable?

It is a cloud-based system. So, it is stable and scalable.

What needs improvement?

Azure DevOps is set up more for development and less for testing. If it is set up correctly, everyone can use it better, but it was set up from a development point of view, which means it is lacking in what I need from a testing perspective. Just like any other tool, it depends on how it is configured. I am not happy with the way it is set up. It is configured more from a development side, and it doesn't necessarily cater to all the other areas that probably need to use it, such as testing data, etc.

For how long have I used the solution?

I have been using this solution for about 15 months.

Buyer's Guide
Microsoft Azure DevOps
September 2022
Learn what your peers think about Microsoft Azure DevOps. Get advice and tips from experienced pros sharing their opinions. Updated: September 2022.
634,325 professionals have used our research since 2012.

What do I think about the stability of the solution?

Its stability and performance are okay. It is on the cloud. As long as you have got access to it, it is stable.

What do I think about the scalability of the solution?

It is a cloud-based system. So, you can add more bandwidth. It is scalable on the cloud.

We have about a hundred users who are using this solution. It is used on a daily basis.

How are customer service and support?

A third party deals with the technical support of it at the moment.

How was the initial setup?

I wasn't around when they initially set it up, but the way it is set up, it is too complex. It is probably good for developers, but it is not good for the testing side. 

What about the implementation team?

It is a third party that sets it up. I don't know about its maintenance because I'm not that close to it.

What other advice do I have?

I would advise organizing and doing the right assessment for all teams that are going to use it. When it is being set up, more people within the program need to be involved in the setup, not just the developers. You need to know about the requirements for design, development, testing, integrations, and architecture. You need to solicit requirements on what each one of these teams needs from the tool before the tool is configured. When you set something up only from the development perspective, you forget that there would be a need to extract information for data testing and training. So, you need to assess who all are going to use the tool so that you set it up for maximum usage.

At this time, I'd rather not recommend it because it wasn't set up correctly. It wasn't set up with other teams involved. In a year's time, if I'm working on it again, I may have a different opinion.

I would rate it a five out of 10.

Which deployment model are you using for this solution?

Public Cloud

If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?

Microsoft Azure
Disclosure: My company has a business relationship with this vendor other than being a customer: Partner
Flag as inappropriate
PeerSpot user
Director at a computer software company with 10,001+ employees
Real User
Top 5
Easy to comprehend and easy to use but the pricing should be easier to manage
Pros and Cons
  • "The simplicity is very good and the customer experience is also great."
  • "It should be easier to manage Licenses especially because it's in the cloud."

What is our primary use case?

My team uses this solution for the CI/CD deployment, and code check-ins.

We are also using Azure Boards for tracking our work, all of the requirements, the backlogs the sprints, and the release planning.

What is most valuable?

What I like the most is that it is easy to comprehend, and it's easy to use.

The simplicity is very good and the customer experience is also great.

What needs improvement?

I am not suggesting this solution should be cheaper. I would like to see a bucket of licenses. for example, 10 licenses or 100 licenses that could be monitored to know how many of those licenses have been used. The price would be deducted accordingly.

I should not have to contact Microsoft daily to request a license or two. There should be a mechanism in place where you are able to find out where you are out of 100 licenses, or that you have used 90 licenses in that year.

I should be given a credit line of 10 licenses at the end of the year or at the end of the month.

It should be easier to manage licenses, especially because it's in the cloud. You should know the usage and based on the usage, you should be able to make decisions.

For how long have I used the solution?

I have been using Microsoft Azure DevOps for one year.

We are using the latest version.

What do I think about the stability of the solution?

It's a stable product. I have not experienced any issues.

What do I think about the scalability of the solution?

This is an area that has yet to be explored fully. We haven't taken it to this level.

We are a team of 25 to 30 members, which is fine for us. Every time we scale up we need a new license and that takes time, it's not just a click and it's done.

I don't know if it will be scalable for 200 to 300 people.

Which solution did I use previously and why did I switch?

Previously, I was working with Jira.

How was the initial setup?

I was not involved in the installation. My team completed it.

What's my experience with pricing, setup cost, and licensing?

Price is an area that could be improved. There are products on the market with a fixed price of 50 or 100 people, you are a bucket price. 

With Azure, you have to pay for every user.

It's good to have a bucket such as 50 to 100, or 100 to 200, and flexible pricing.

The issue may be from having more than one license. When you procure one license or two licenses, it becomes difficult.

It should be easier to procure a license, it should not be one by one. We don't know how many members I will have on my team three months from now.

What other advice do I have?

We plan to continue using this solution.

I would recommend this solution, but I would not know their business needs.

Based on the various features, the deployments, licensing, pricing, and the customer experience, I would rate Microsoft Azure DevOps a seven out of ten.

Which deployment model are you using for this solution?

Public Cloud
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Buyer's Guide
Microsoft Azure DevOps
September 2022
Learn what your peers think about Microsoft Azure DevOps. Get advice and tips from experienced pros sharing their opinions. Updated: September 2022.
634,325 professionals have used our research since 2012.
Derek Smith - PeerSpot reviewer
Development and Release Compliance Officer at a computer software company with 1,001-5,000 employees
Real User
Top 5
A reliable ecosystem with the capability to expand as needed
Pros and Cons
  • "The solution is scalable."
  • "The interface is very bad."

What is our primary use case?

We produce multiple different software for different markets in different countries. It's really for everything you can possibly think of, from online games to financial systems, to payment gateways, to APIs, to service desks, back-office analysis tools, admin tools, et cetera. We use it for anything and everything really.

What is most valuable?

The solution is great due to the fact that it's kind of an entire ecosystem. I'd say the work items are probably the most valuable aspect.

The stability has been great.

The solution is scalable. 

What needs improvement?

The interface is very bad. It's an aspect that really needs to be worked on. It is going to probably get the developers to start abandoning work items.

Our initial setup was quite complex. 

The presence of Microsoft in the country is limited.

For how long have I used the solution?

I've used the solution since it came on the upgraded from Team Foundation Server - about ten years. We've used it for a very long time. 

What do I think about the stability of the solution?

The stability has always been really good. There aren't bugs or glitches and it doesn't crash or freeze.

What do I think about the scalability of the solution?

We've found the scalability to be good. We have about 2,000 users on it right now. We haven't had any issues with scaling as needed. 

How was the initial setup?

For us, the initial setup was extremely complex due to the multiple organizations we had on the go. We actually had to abandon our initial rollout and rethink the whole process though. 

I'm in compliance, and therefore cannot speak to what the maintenance process is like for the solution. 

What about the implementation team?

We had a combination of both in-house and vendor help with the setup process. We've got enterprise agreements with Microsoft, so they always give us resources to assist with our deployments. We already had TFS deployed, so it was an upgrade, really.

What's my experience with pricing, setup cost, and licensing?

I don't deal with licensing. I can't speak to how much the solution costs.

What other advice do I have?

I'm just a customer and an end-user.

I'd advise new users to plan very carefully the way that they would like to set up projects. The model that you choose is extremely important and you might have to do a lot of rework if you don't get it right the first time.

I'd rate the solution an eight out of ten. It's Microsoft. It's backed by the mighty, mega Microsoft. You can't get fired for choosing the top product run by a top multinational company. The downside for us here is, in South Africa, there are only two points of presence. For the data, if it's fully hosted, we only have Capetown and Johannesburg. When it comes to back hall speeds for internet, it could be better if we had points of presence in all the major cities, however, I suppose it's too much investment for Microsoft to make.

Which deployment model are you using for this solution?

Hybrid Cloud
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Flag as inappropriate
PeerSpot user
Manager with 1-10 employees
Real User
Top 5
Good CI/CD pipeline and testing automation that is both stable and scalable
Pros and Cons
  • "The most valuable features of this solution are the CI/CD pipeline, and the testing automation."
  • "They have to add more features such as schedules and more flexibility in the platform."

What is our primary use case?

We are using this solution for CI/CD projects, for Scrum, Agile planning, testing, and business management system solutions. 

We are also using it for continuous integration, and continuous delivery of DevOps.

I am also using the Git Repositories, which is the main source control for me in the organization. We were using it on-premises and now planning to move it to the Cloud.

They are calling it repository and they are supporting an old protocol, which is a popular protocol called Git repository.

What is most valuable?

The most valuable features of this solution are the CI/CD pipeline, and the testing automation.

It's good that they are bringing more options to this environment, and to deploy to virtual machines.

What needs improvement?

The testing agents needs to be improved, they need to work on it.

They can enhance the areas related to continuous integration and build automation. They can add more technologies for the build.

The integration with the Azure infrastructure as a code and include more templates for infrastructure as a code in the continuous integration phase and the build phase.

They can add more flexibility to the continuous deployment.

Artifact management is also an area that needs some enhancements.

I think automation and integration with the CI/CD lifecycle it's good but it's using agents.

I am using a different tool for testing automation because I have not found any use cases on it. This feature is not covered. I'm using only testing and business management. I'm not using the testing automation feature that is provided by Azure DevOps.

I would like to see more integration with other platforms and more flexibility.

They have to add more features such as schedules and more flexibility in the platform. This will allow me to build my space and time based on the schedule and based on a calendar.

I would also like to see a release calendar so that I can plan my releases based on my sprints. 

This is required and it would give this solution more maturity. Between Scrum, Agile Planning, or Project planning with the build and deploy prophecies.

For how long have I used the solution?

I have been using Microsoft Azure DevOps for three years.

We are using the latest version of this solution.

What do I think about the stability of the solution?

It's a stable solution. We have not experienced any issues.

What do I think about the scalability of the solution?

It's a scalable product. We have more than 50 users in our organization using this solution.

I plan to continue using it in the future.

Which solution did I use previously and why did I switch?

Previously, I used TeamCity, Jenkins, I used Chef, and many other technologies.

How was the initial setup?

The initial setup is straightforward, it's a cloud service, so it's plug and play.

What other advice do I have?

I would recommend Microsoft Azure DevOps for other users who are interested in using it.

I would rate this solution an eight out of ten.

Which deployment model are you using for this solution?

Public Cloud
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
CTO at Cantier Inc
Real User
Top 10
We use it for all of our development activities, and it is a complete solution with good reliability and scalability
Pros and Cons
  • "You get a complete solution with Azure DevOps. You can do everything in one place, starting from requirement gathering until you release the product. It is a reliable, scalable, and handy product."
  • "Its testing features are limited and can be improved a little more. They should provide more options from the testing and build perspective. Currently, we have to use a third-party product for testing. It would be great if it has more than one testing tool."
  • "They have brought a lot of new collaboration features in the latest version. We haven't used those features, but they should continue to expand it more on the collaboration front so that two developers can look at the code and work at the same time. It will be helpful for working from home."
  • "Its UI can be easier and more customer-friendly. The UI can be improved from the project management and agile perspective."
  • "Its price can also be reduced. It would be great if they are a little bit more competitive in terms of pricing because many open-source products are currently available in the market."

What is our primary use case?

We are a product company, and we use Azure DevOps for product development. It is our backend platform, and we use it for all of our development activities.

What is most valuable?

You get a complete solution with Azure DevOps. You can do everything in one place, starting from requirement gathering until you release the product. It is a reliable, scalable, and handy product.

What needs improvement?

Its testing features are limited and can be improved a little more. They should provide more options from the testing and build perspective. Currently, we have to use a third-party product for testing. It would be great if it has more than one testing tool. 

They have brought a lot of new collaboration features in the latest version. We haven't used those features, but they should continue to expand it more on the collaboration front so that two developers can look at the code and work at the same time. It will be helpful for working from home.

Its UI can be easier and more customer-friendly. The UI can be improved from the project management and agile perspective.

Its price can also be reduced. It would be great if they are a little bit more competitive in terms of pricing because many open-source products are currently available in the market.

For how long have I used the solution?

I have been using DevOps within the last 12 months.

What do I think about the stability of the solution?

It is definitely a reliable solution. That's why we have been using it for quite some time.

What do I think about the scalability of the solution?

It is scalable. There are no issues in terms of scalability. Currently, we have around 100 users. Its usage will definitely increase because a lot of new projects are coming.

How are customer service and technical support?

When migrating from TFS to Azure DevOps, we didn't get great support and response from Microsoft.

How was the initial setup?

Its initial setup was a little complex for us. We had been using TFS for quite some time, and migration was a bit difficult for us. We didn't get great support and response from Microsoft. We had limited resources in terms of support, and that's why it took us longer to figure out how to migrate from TFS.

What's my experience with pricing, setup cost, and licensing?

It has an annual subscription. It can be cheaper for partners. Many open-source products are available in the market, and it would be great if they can be a little bit more competitive in terms of pricing. A lot of startups are looking for an open-source, free, or cheap solution. If they can accommodate such requirements, it will be good for the product in the long run.

What other advice do I have?

I would rate Microsoft Azure DevOps an eight out of ten.

Which deployment model are you using for this solution?

Public Cloud
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Senior Consultant at CMEx
Consultant
Top 20
User-friendly with good reporting features assist with managing my test plans, and the support is responsive
Pros and Cons
  • "The creation of test plans is valuable and I like the reporting features."
  • "I have not been able to use the integration with automation features, such as test management automation, with a framework that is written in Java."

What is our primary use case?

We are a small product team and we do software development. I'm part of the testing team and I use Azure DevOps to manage all of my test activities. This includes my test plan, where I create my test cases, write test scripts, execute them, and publish the results using the dashboards.

What is most valuable?

I like a lot of the features in Azure DevOps.

The creation of test plans is valuable and I like the reporting features that it offers. For example, I like the charts, as well as reporting progress.

The interface is user-friendly.

What needs improvement?

I have not been able to use the integration with automation features, such as test management automation, with a framework that is written in Java. Currently, Azure offers frameworks that are written in C#. I have not found anything in the Microsoft documentation to say that it supports other languages.

For how long have I used the solution?

I have been using Microsoft Azure DevOps for approximately two years.

What do I think about the stability of the solution?

Being a SaaS product, sometimes we experience hangs. However, that is okay because in general, it is very much stable. If there are any outages then I get prior notification of such events.

What do I think about the scalability of the solution?

I have not tried scaling it. In my team, we have approximately 30 people who are using it.

How are customer service and technical support?

For me, the first point of contact is my internal organization and not Microsoft directly. However, if there are any changes that require Microsoft's intervention then there is an intermediate team who coordinates it. The response has been pretty good and I am happy with the support.

Which solution did I use previously and why did I switch?

I have used several tools in the past including Zephyr and the Rational Quality Manager.

How was the initial setup?

The initial setup is pretty straightforward. We needed to set up a Microsoft account, and that gives access to the projects. Basically, the IT department or IT administrators take care of the access.

As a member of the team, once access is provided, I can just start using it. It is plug-and-play, as far as I'm concerned.

What's my experience with pricing, setup cost, and licensing?

It is a subscription model and I only pay for what I use.

Which other solutions did I evaluate?

Very recently, I was doing comparative studies of available test management solutions on the market and I find Azure DevOps to be pretty good.

What other advice do I have?

Overall, this is a good product and I recommend it.

I would rate this solution an eight out of ten.

Which deployment model are you using for this solution?

Public Cloud

If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
DevOps Engineer at Babel Software
Real User
Top 5
Great serverless technology that makes it quicker and faster to deploy .NET applications
Pros and Cons
  • "It's a pretty problem-free solution."
  • "When we don't have some permissions, we have to research how to get them."

What is our primary use case?

I use it to, for example, build applications. Not just Subnet - also Java or in OGS. I deploy Windows applications and also mobile applications with Visual Studio App Center and Azure DevOps.

What is most valuable?

Azure services like serverless technology make it quicker and faster to deploy .NET applications, for example. Similarly, the Azure Portal is faster to put in services, for example, with specific functionality. These kinds of features are faster to deploy and to put in functionalities that are specific, which is great.

I like the Azure keyboard. Keyboard permits use, for example, connection streams in secret, in a safe manner. 

Sometimes I use also Azure SQL Database cloud, the serverless functionality. It's also very attractive because you can use something like a GP, like the protection of the data on the cloud. You can put a lot of information there, maybe not with banks, however, with other kinds of clients. 

The ability to have cognitive services is good. For example, I use a lot of Azure DevOps, for example, Cloud, Azure DevOps Cloud, and analytics views with, for example, Power BI on-premise if you have Power BI as a service. There is a lot of pipeline potential with Azure DevOps deploying, for example, Containers, to Kubernetes, Azure Kubernetes Service, and also a lot of Azure Container repositories and Azure Container instances. This is great for quickly deploying SonarQube in a container. These kinds of things are very attractive.

It's a pretty problem-free solution.

What needs improvement?

When we don't have some permissions, we have to research how to get them. It's not obvious. For example, deploying in order to do the connections, we need permissions for quotes or quotas. A specific quota might depend on an account. Sometimes we don't have the kind of support we need to work things out easily.

When you deploy in Yaml Pilot, for example, you need to note in Yaml how to associate the piling, the release with the word items. We need more information about how this is possible and more flexibility to make it happen. 

What other advice do I have?

My organization is a Microsoft partner.

I have Microsoft certifications. I always develop in Visual Studio, .NET, or Core, and now I work with Azure DevOps and also in Azure Portal.

I deployed in a hybrid environment as the clients have a lot of things on-premises. For example, the databases I use, for example, Redgate, plus Azure DevOps, or Apex with Azure DevOps, to deploy databases. In some cases, a lot of banks prefer their data on-premise. In other cases, I deploy to Azure App Services, for example, in the cloud. Also, I see things with Amazon Web Services and telephone, like multi-cloud.

I would rate the solution at a ten out of ten as it never gives me any problems. 

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Flag as inappropriate
PeerSpot user
SeniorOwnder67 - PeerSpot reviewer
Owner and Senior Consultant at a tech services company with 1-10 employees
Real User
Simple to use, numerous add-ons, and versatile work items, but user management is difficult.
Pros and Cons
  • "The work items option is incredibly flexible."
  • "When you compare with Jira, there is a lack of progress features."

What is our primary use case?

A common use case of Microsoft Azure DevOps is the use of work items and then connecting them to Git repositories and their updates. It is primarily used for item and code management.

What is most valuable?

The work items option is incredibly flexible.

There are multiple add-ons available.

It is easy to use.

The entry point is adequate.

What needs improvement?

When you compare with Jira, there is a lack of progress features.

I would like to be able to customize the product using add-ons or a similar mechanism.

Scalability is an area where they could advance and make changes.

Unfortunately, managing users in Azure is a very complicated issue. We also have a problem with one of their other tools, which is Teams. The Team messaging has caused us some trouble because they have what is called organization, in addition to the users, but it doesn't work very well.

Technical support needs improvement.

I would like to see scalability, dashboards, KPIs, measurements, and some visual management assistance improved in the next release.

For how long have I used the solution?

I have been working with Microsoft Azure DevOps for one year.

We are using the latest version.

What do I think about the stability of the solution?

Microsoft Azure DevOps is a stable product.

What do I think about the scalability of the solution?

This solution is used by 20 people in our organization.

It appears to be simple, but based on the work items and project management aspects, I believe there are few options for scaling it up in terms of dashboards and KPIs.

How are customer service and support?

I contacted technical support about managing users in Azure. Unfortunately, they were able to resolve this issue for us.

We tried several times and we did not get the answer we expected to get.

Which solution did I use previously and why did I switch?

I use Jira as well as TFS.

How was the initial setup?

The initial setup is straightforward. It is simple to install.

Other than the initial setup, it does not require any maintenance.

What about the implementation team?

It was implemented internally.

What's my experience with pricing, setup cost, and licensing?

The majority of the components are reasonably priced. Testing is one of the more expensive components. When you compare it, it is approximately $3 per month for the other components and $45 or $50 for the Testing component. It costs ten times as much as the other components.

What other advice do I have?

I would recommend this solution to others who may be interested in using it.

I would rate Microsoft Azure DevOps a seven out of ten.

Which deployment model are you using for this solution?

Public Cloud

If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?

Microsoft Azure
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Buyer's Guide
Download our free Microsoft Azure DevOps Report and get advice and tips from experienced pros sharing their opinions.
Updated: September 2022
Buyer's Guide
Download our free Microsoft Azure DevOps Report and get advice and tips from experienced pros sharing their opinions.