Coming October 25: PeerSpot Awards will be announced! Learn more
2014-05-21T19:00:00Z
PeerSpot user
User at a transportation company with 51-200 employees
  • 6
  • 404

Specflow vs Selenium

What are the pros and cons of Specflow vs Selenium?

I'm evaluating Specflow vs Selenium. Our project involves development/test efforts leading to a Web application (.net) that interacts with a windows application (Windows Service) to ultimately communicate with a field-deployed device (such as a handheld vending machine, traffic camera and similar transportation/fleet devices). 

We would like to have a suite of nUnit test code bases, most preferably integrated with a UI-enabled test tool. 

5
PeerSpot user
5 Answers
PeerSpot user
User at a transportation company with 51-200 employees
Vendor
2014-06-14T08:34:16Z
14 June 14

Hello fellow members of this community,

Thanks so much for your suggestions. I hadn't proceeded with my automation plans since the posting of this question but now I am having to make a decision and accordingly coming back to the enquiry only now. It is interesting to note that there appear to be two schools of thought a) Use Selenium and b) Use Spec-flow with the a) line of suggestion stemming primarily from lack of exposure to Specflow possibly because its use is still not widespread. We do have a requirement to maintain reusable framework suitable for repeated regression testing. The idea is to integrate the framework to our agile-delivered incremental builds of the windows application so that sanity test scenarios will be automatically run during installation of every new version. So I may just go with SpecFlow.

Ian, just to clarify why we are not comparing apples to oranges, pls note that the journey that we envisage for the tool to make is to execute unit test codebase before and as part of navigating across different pages/features of the application driven purely by a record and playback-based actions. Both Specflow and Selenium seem capable of doing that as a nUnit developed codebase could be invoked from the test tool UI.

PeerSpot user
Head of Performance with 51-200 employees
Vendor
2014-05-26T15:15:12Z
26 May 14

Specflow and Selenium are really targeted at different requirements so difficult to make a meaningful comparison. (Sort of like comparing a car and a boat.)

· Nunit - make sure the code does what it should.

· SpecFlow - make sure that the individual user requirements (features) are met.

· Selenium - make sure that the user journey is correct.

gagneet - PeerSpot reviewer
Continuous Delivery Lead at SAI Global
Consultant
Top 5Leaderboard
2014-05-25T01:45:06Z
25 May 14

Hi,

SpecFlow is a .NET based testing tool and follows the BDD methodology from
the start, as it employs Cucumber to create and deliver results. As you
have mentioned that you are going to test on an application built on .NET,
this would suit your purposes nicely, as the developers themselves will
also know how to handle and work on any issues with the tests and can
incorporate them into their Unit tests also.

Selenium is a robust and stable framework for creating tests and can be
overlaid with Cucumber along with other xUnit test frameworks to provide
you with the same set of functionality and features. The only difference
here is that SpecFlow already contains the required features and
functionality.

Overall, if you wish to ease work and also have a maintainable framework
for future use, I would recommend you to SpecFlow with Cucumber over
Selenium with Cucumber. They have a good site with all the required forums
and FAQs and a sensible community for feedback and solutions.

Selenium is a good framework, and if you have prior experience with it then
it makes better sense to use it, but if not, pick up SpecFlow for your .NET
application testing.

UI with Web and App tests can all be easily created and integrated within
the SpecFlow tests.

Cheers

it_user98394 - PeerSpot reviewer
Engineer at a tech company with 51-200 employees
Vendor
2014-05-22T17:23:21Z
22 May 14

I haven't used Specflow so cannot compare it with Selenium.

Selenium is good for testing web applications i.e. applications accessed
via web browser. And for writing unit test cases, I would say use TestNg as
it has some more advance features than JUnit e.g. concept of data provider.

Essentially, use the TestNg framework to write both, UI tests (selenium
webdriver) and unit tests.

Hope this helps.

Kevin Surace - PeerSpot reviewer
CEO at Appvance
Vendor
2014-05-22T14:59:33Z
22 May 14

Selenium is a very popular scripting language today and what most people are using. That will do what you want. To automate the tests and also repurpose for performance there are tools like Appvance that do this with Selenium.

Related Questions
Netanya Carmi - PeerSpot reviewer
Content Manager at PeerSpot (formerly IT Central Station)
May 18, 2022
How do I choose one?
See 2 answers
Janet Staver - PeerSpot reviewer
Tech Blogger
17 May 22
Performance issues and website crashes can cause constant disruptions or major disasters for your organization. In order to prevent them, you need a performance testing tool. Performance testing is necessary to check the behavior of an application across various situations. A system can work effectively with a particular number of concurrent users, but might become dysfunctional during peak traffic. However, if you are using a performance testing tool, this helps establish speed, scalability, and stability. Because performance testing tools create better website speed, it helps keep customers more engaged. If your website or application tends to be slow and lousy, it's unlikely that you will draw a good audience and more likely that customers will turn to your competitor instead. More important than keeping customers engaged, good website performance leads to increased revenue. Essentially, by implementing a good performance testing tool, you are setting your organization up for success. Performance testing tools are particularly important in helping to resolve glitches and eliminate bottlenecks. By using a performance testing tool, areas that cause bugs or other problems can be identified and remedied by notifying the dev team promptly. In addition, for obvious reasons, you want your application to work at all times. However, there are times when your organization will experience downtime, and it can be extremely costly. With a performance testing tool, though, stress tests are performed to help ensure stability. Enterprises across different industries often require their websites, web apps & mobile apps to perform seamlessly. In order to do so, performance testing tools should be implemented. Another good thing about performance testing tools is that you can find both commercial options as well as open-source ones. Most of them have features for extensive test automation to save time. Additionally, performance testing tools are generally very easy to download and set up. Furthermore, performance testing tools offer a host of useful features. Some of these include report generation, multithreading, log debugging, and sampling. Most solutions also support Web UI and API services and can read test data from Excel files. Whatmore is the solutions usually don’t require a lot of resources. For example, tests can be run with a normal PC. Some are even very customizable, allowing you to use your own Java code, while others can run from a console without any user interface. And for when it’s necessary, most performance testing tools offer documentation available online. Employing performance testing tools is a must — whether it is for a mobile app or a desktop application. Aside from offering scalability and speed, performance testing tools are reliable and robust, and have impeccable detection capabilities. If you are undecided and not sure if you need to use a performance testing tool because you think you can get off easy by doing things manually, you may want to think again. Either way, investing in a performance testing tool is certainly a good choice.
Ravi Suvvari - PeerSpot reviewer
Performance and Fault-tolerance Architect with 1,001-5,000 employees
18 May 22
It depends on various factors you test, how long, how many times, ROI, number of controllers needed, cloud support, budgets, protocols or bundles needed.
Netanya Carmi - PeerSpot reviewer
Content Manager at PeerSpot (formerly IT Central Station)
May 17, 2022
Why
2 out of 5 answers
Evgeny Belenky - PeerSpot reviewer
Director of Community at PeerSpot (formerly IT Central Station)
11 May 22
Hi @KashifJamil, @Ravi Suvvari , @David Collier ​and @reviewer1322280, Can you please share your professional opinion?  Thanks.
David Collier - PeerSpot reviewer
Co-Founder at Nobius IT
11 May 22
Hi all, I'm afraid the question is rather too generic, but I'll try and provide some pointers. First we need to understand what you mean by "performance testing". To a network manager, they are interested in how quickly, efficiently and securely a packet of data travels from point A to point B on a network. To an application manager, they are interested in how quickly, efficiently and securely a person can complete a unit of work using a given application. To a DBA, it's how quickly, efficiently and securely data can be accessed and stored. To a server manager they're interested in how quickly, efficiently and securely data can be processed. Possibly more relevant in this instance is the needs of the application developer / application tester. To a developer, they are interested in understanding how their application will perform under various loads. An application is likely to perform differently under different loads. For instance, a web application with 2 users is going to behave differently than if it has 2 million users. In fact, I'd go as far as to say that understanding your potential user base should influence server, network, database design in the first place. Effectively, they all want to optimise speed, efficiency and security - and also COST in order to deliver the best outcome for the business they are supporting. From this common set of requirements their needs diverge rapidly into a massive variety of metrics that need to be collected. As you can see it's a VAST area for discussion. With such a variety of requirements it's probably impossible to name a single product that can meet ALL requirements of ALL interested parties. This very site has a good report https://www.peerspot.com/landi... that outlines application performance testing solutions. These range from free, open source solutions all the way through to enterprise class solutions. As an ex-Micro Focus employee, I can only comment on LoadRunner. It's probably one of, if not the leading enterprise solutions out there. It supports a variety of application protocols and can simulate many many simultaneous users. I also understand that some of the simulation scripts in LoadRunner can be used by other MicroFocus tools to monitor application performance when live. However, my main point remains. You need to understand performance at all levels of service delivery not just the application code. This may mean instrumenting code, network probes, database analytics, server performance monitoring and so forth to get a complete picture. This is where cost comes to bear. Instrumenting every layer of service delivery is a complex process resulting in massive amounts of data collection and processing. The overhead of managing this data collection can be prohibitive. And this is where I learned a lesson from an old boss who told me..... "We can instrument the application, we can put network probes everywhere, we can setup monitoring tools. It'll take us a year, cost us hundreds of thousands of pounds to find out where the problem really is. We've got an application we've written with 120 users that runs like a sloth on sleeping tablets. I've ordered a new server with faster CPU's, SSD disks and more and faster memory. It's arriving in 2 weeks and costing me £15k". I replied that he's simply sweeping the problem under the carpet and not fixing the root cause of the issue. His reply to me was simply "I know. I'm simply making an inefficient system be inefficient faster. And by doing that, I'm saving £200k and improving things inside a month." My point here: his priority was to get 120 people doing their given units of work faster and as quickly as possible which was different to the "techie" need to define efficiency in terms of maximm throughput for least usage of resources. Performance testing in time of development is very important, performance monitoring in live operation is critical. The solutions to problems though don't always need to be more testing. Just my 50p worth. Dave..
Related Articles
Janet Staver - PeerSpot reviewer
Tech Blogger
Jul 21, 2022
SmartBear LoadNinja is a great all-in-one test automation tool. Here are some reasons why users like it: Speed. It is fast on almost any machine. Configuration consistency. Accuracy of tests with fantastic results. Great AI features with good cloud scaling. Artificial intelligence. The automation is very good. An application can be tested across many programs with ease. For example, ...
Netanya Carmi - PeerSpot reviewer
Content Manager at PeerSpot (formerly IT Central Station)
Apr 07, 2022
PeerSpot’s crowdsourced user review platform helps technology decision-makers around the world to better connect with peers and other independent experts who provide advice without vendor bias. Our users have ranked these solutions according to their valuable features, and discuss which features they like most and why. You can read user reviews for the Top 5 Performance Testing Tools to help ...
Related Articles
Janet Staver - PeerSpot reviewer
Tech Blogger
Jul 21, 2022
What is SmartBear LoadNinja?
SmartBear LoadNinja is a great all-in-one test automation tool. Here are some reasons why users l...
Netanya Carmi - PeerSpot reviewer
Content Manager at PeerSpot (formerly IT Central Station)
Apr 07, 2022
Top 5 Performance Testing Tools 2022
PeerSpot’s crowdsourced user review platform helps technology decision-makers around the world to...
Related Categories
Download Free Report
Download our free Performance Testing Tools Report and find out what your peers are saying about Apache, Micro Focus, Perforce, and more! Updated: October 2022.
DOWNLOAD NOW
634,550 professionals have used our research since 2012.