Tricentis NeoLoad's mobile platform acts as a stand-alone application but needs to be integrated with the main interface. 

I'm not ready to share what areas of Tricentis NeoLoad have room for improvement now.

The price could be more friendly, and it was impossible to continue using the same version of Tricentis NeoLoad, as we were forced to move to the next version. Sometimes there were compatibility problems, and that was a major problem with backward compatibility issues.

Since the ownership of NeoLoad has changed to Tricentis, they have done a very poor job with license management. They changed the license policy very abruptly. The effect of the new license policy is that NeoLoad becomes more and more unattractive for smaller companies, and only bigger companies are interested or find the license fee fair. The smallest license fee is very high, and there is no starter package at the moment.

If one person opens any script, another person won't be able to work on it simultaneously. If they can improve that feature, it would be helpful for everyone. I found that incorporating all the scripts into a single project was the challenging part. This is because we are working on different domains—I'm on one domain, and others are on another. We need to handle all these scripts cautiously.

Consistent and robust support needs to be readily available throughout the implementation and initial setup of the solution.

I really didn't work on the cloud-based [version]. NeoLoad still has a cloud [offering], and it has pretty good integration. I heard that it's possible to integrate with JMeter as a tool as well.

Maybe I could suggest: I wanted to know more about the integration with DevOps for performance testing. The automatic integration process – how can we run the scripts automatically within a CI/CD pipeline? 

So maybe I wanted to know how to integrate with DevOps, actually. I'm not sure whether that option is there with the tool or not. 

In future releases, it would be good if extra added features for integration are added into NeoLoad.

I didn't like much of the support that you get from the Tricentis group unless it was after it integrated with Tricentis; the support is not that good. But earlier, the support was actually very wonderful. I started using NeoLoad right from 2011.

So, there is room for improvement in customer service and support. It requires a lot of justification and a lot of emails that you need to send back and forth. But earlier, when I was working with Siemens, the integration of the NeoLoad team and Siemens team was very good, and the support was excellent. As soon as you raised a ticket, we had very good support, but that changed after Tricentis.

The coverage provided by Tricentis NeoLoad is not much when compared to LoadRunner. Cost-wise, Tricentis NeoLoad is cheaper than LoadRunner. Both the aforementioned mentioned points create a balance between Tricentis NeoLoad and LoadRunner. If a person has to specifically look into a certain product, then that person can go ahead with Tricentis NeoLoad. There isn't much when it comes to the licensing models offered by Tricentis NeoLoad, while LoadRunner offers different licensing options. Considering some of our company's requirements, we would consider Tricentis NeoLoad since it is price-wise a cheap solution. When it comes to LoadRunner, they offer cloud licenses along with different sets of licenses. In certain areas, the licensing models offered by LoadRunner are better, while at other times, the licensing models offered by Tricentis NeoLoad are better. If Tricentis NeoLoad offers different kinds of licensing versions, then the solution's price can be much lower than LoadRunner and the other solutions in the market.

The product must improve the features that allow integration with CI/CD pipelines. The integration with self-service portals and tools must be improved.

The solution could be cheaper.

An area for improvement in Tricentis NeoLoad is its integration with third-party tools because, at the moment, it's a bit complicated. Per Tricentis, you can integrate Tricentis NeoLoad with different monitoring tools such as Dynatrace and New Relic, but that requires installing an additional tool to make that integration happen, rather than being able to pull in Tricentis NeoLoad from the different tools and servers, and make integration simpler and easier.

In its next release, I'd like to see more compatibility with new protocols or applications from Tricentis NeoLoad.

The solution can be improved by introducing a secure testing feature.

Many applications these days have dashboards, video images, and the like. The rendering of heavy-weighted applications can't be scripted via NeoLoad, so we have to use LoadRunner as an extra client. I think that component could be added to NeoLoad where the UI has more object rendering. For now, we cannot test that in NeoLoad and so people tend to go for the two client protocol of LoadRunner. For now, the GUA still sits on Windows so we do the scripting on our local machine and then transfer them to the cloud. It would be great if we could do everything on the cloud with NeoLoad. 

The overall stability of the GUI should be improved. The GUI component is not stable enough. We have observed crashes several times. 

There should also be support for more protocols for desktop applications.

They should improve the reporting feature of the solution. It is easier to comprehend the analysis on its on-premise setup but not on its on-cloud setup.

The SAP area could be improved. Not the GUI applications, SAP log-on, or something like that. We can see features and use NeoLoad properly in all the normal SAP log-on areas. I haven't tried it, but I have heard that Citrix-based applications don't work properly in the SAP application. In those types of different areas, there could be more protocol flexibility.

If there is a high number of users on NeoLoad—like around 5,000 or 6,000—it sometimes feels like that might be a bottleneck for the performance. That could be improved. With more users, there shouldn't be any issues from the tool side.

With scripting in a different browser, sometimes there are compatibility issues and it doesn't generate the script properly. Those types of crashes could be removed.

LoadRunner supports multiple protocols, whereas NeoLoad supports only three protocols. With NeoLoad, we can go for the SAP technology, web-based HTTP, and Remote Desktop Protocol (RDP). If I have to simulate .NET application-based traffic, I won't be able to do that. So, protocol support is a limitation for NeoLoad. It should support more protocols.

One issue that we faced was that multiple users weren't able to work on the same application. We used to create multiple scripts based on the application and based on the projects, and then we used to integrate all of them in a single place. With NeoLoad, if you have to do this activity, to import a script, one user has to log off, and the second user has to copy the script and improve it, which is a time-consuming activity. These are the things that can block any further activity.

NeoLoad can improve the correlation templates, which are specific to frameworks. There's room for improvement in that area.

In future releases, I think they definitely need a full UI revamp because I believe it's built on Java technology, which feels a bit outdated. I would like to see that being improved. Additionally, increasing the coverage of the correlation templates capabilities would be very useful.

There is room for improvement with the support and community documentation as it can be difficult to find answers to questions quickly. 

The protocol support area could be improved.

Tricentis NeoLoad could improve the range of codeless scripting such as with terminal emulation apps. 

There were some features that were lacking in Tricentis NeoLoad, e.g. those were more into Citrix and other complicated protocols, which were supported easily by a competitor: Micro Focus LoadRunner.

What we want to see in this solution is how it integrates with the entire suite of Tricentis Solutions. Tricentis has a very successful product in Tricentis Tosca, and that is a product that is more focused on test automation. They have test management solutions. They have different management. Performance management or performance engineering within one solution would be a big winner for Tricentis NeoLoad, so this is an area for improvement for this solution. Otherwise, it is a really good tool.

I still need to take a closer look at the integration of Tricentis NeoLoad. I want to see how it integrates with other Tricentis products, because Tricentis did not have a good performance testing tool until now. This is a recent acquisition, so we want to see how these results come together: test automation, performance testing, etc., so that's what I'm looking for.

We would like NeoLoad to be able to support more protocols.

Testing can also be a little tricky at times.

The debugging part of Tricentis NeoLoad takes time. Fixing the scripts takes time due to the problem with the debugging part. The debugging part of the product is an area that requires improvement.

We would like to see the addition of one-to-one integrations with the Tricentis Tosca suite to this product, which would then cover the end-to-end needs of our customers who are looking for a single vendor solution.

An area for improvement in Tricentis NeoLoad is its price, as it has a hefty price tag.

There are still some glitches that they need to improve. We have given support feedback as well when we have some issues. They're very responsive and they do work to fix and improve issues. 

LoadRunner offers a full protocol, whereas, with this product, only a few of the protocols are supported - not all.

The solution’s pricing is higher compared to other tools. Though the product’s reports are accurate, it needs to be more detailed like other tools.

The price of this solution could be reduced to make it more competitive.

We would like to see more focus on support for other types of endpoints. Most people focus on HTTPS or TCP, but it would be good to have support for a variety of different protocols.

Tricentis NeoLoad crashes if an application contains more than 1,000 scripts. Tricentis needs to improve in this area, and the solution should run smoothly even if thousands of scripts are imported into it.

Some users may find NeoLoad too technical, while other users may prefer a scripting language instead of a UI with figures and forms they have to fill in.

Our issues with the solution were related to the license and the support. We had a technical issue related to our software and they weren't able to solve the problem. 

Overall, we're quite satisfied with the product.

Sometimes it's complicated to maintain the test cases. It's much easier than in JMeter, however. I'm not sure if this depends so much on NeoLoad, or is more based on the environment that we are testing.

I would like to see support for auto-correlations.

While importing the scripts from backup it should not create the new variables because it has created some issues for us.

NeoLoad does not support Citrix-based applications. Neotys should take a look into this protocol.

It needs improvement with post-production. 

It needs improvements in the UI. It's currently not as friendly as it should be.

Analysis can be further improved. I have been using LoadRunner analysis and it has been a very powerful tool.

It would be good to make some updates on the reporting side. For example, correlating the performance of server, throughput and hit count in a specific time interval would be a nice functionality.

Team Server - Neotys Team Server - This currently stores project data on the local drive. I suggest having Neotys Team Server Data stored on an actual server.

The ability to show transactions per second during the test run. Currently, we have to eyeball the TPS using the graph. NeoLoad 5.2 will have TPS reading after the test is completed but not during the test run.

The only improvement that I wanted is support for load tests in agile mode. Neoload 5.2 makes this possible by updating the old versions of scripts with data from the new HTTP requests.

It needs improvements when handling binary values.

The weakest feature of most of the load test tools is result analysis. It is important we can customise graphs to visualise results in such a way that stakeholders can understand the outcome of testing. To be able to visualise results better, we extended NeoLoad with Tableau Software.

With Tableau, you can centralize performance related data in a database. Data can come from any tool, from log files or RUM tools and system utilities. Tableau provides you with the ability to connect to any data source, analyse patters and find bottlenecks more easily.  NeoLoad has made huge improvements over the last few years and finally has the ability to view the raw data (and not only aggregated results). This is fundamental for any type of performance engineering.  Aggregated results hide bottlenecks. 

We have raised a few support tickets for certain feature sets/improvement areas.
These include permanent fixes for a few stability issues discussed in the following question and implementation of more robust solutions for media streaming protocols, TCP/IP and websocket protocols.

Also, there are a few changes in the latest version, 5.2, which have brought in many improvements but have removed certain features. For example, v5.2 has done away with the 90th percentile reporting for response times, which was present in earlier versions. While this is a small deviation, it still is important for organizations using it.

The tool can hang and take a long time to open projects if the size is large and if many graphs and result templates are stored. Certain protocols although supported by NeoLoad need to have custom solutions built (e.g. web socket connections) while solutions for certain other protocols such as media streaming protocols like HDS, HLS, etc. are still in the nascent stage.

The product keeps its pace with the updates, however I would like them to offer online advance training certification and better support material.

It wasn't easy to learn how to use NeoLoad and it has a bit of a learning curve. The language used to code scripts was very specific to NeoLoad. It would be better if they incorporated Java and/or other popular coding languages as this would help smooth out the learning curve.

It would be nice if they could make it so that the dynamic parameterization in the requests came back without the values from the previous responses. Also, it needs the ability to capture SQL queries that are being fired.

The documentation for the new JMS protocol is in its initial stages and Neotys needs to update their online support documentation with more information about it.

The current product is very good and the roadmap which is laid out for Neoload is sound. However, they need to offer more protocol support.

If there was ever a way to bring the same level of ease to advanced scripting with Javascript to make it as easy as a regular test in NeoLoad, I would love to see it.

Some type of global test scheduler where different tests could be arranged on a calendar to automatically execute would help scheduling and provide visibility into the test environment.

Tighter integration with NeoSense and perhaps TFS integration to open defects for performance issues would also be nice.

Neotys was extremely promising when they launched and it had features that older versions of HP LoadRunner and HP Performance Center did not (automatic correlation, for example). However with newer releases of HP ALM (especially 11.52 onward), these features were readily available with HP products.

I also ran into installation issues with NeoLoad when the installation never completed.

Reporting needs to be better. I would like to see information such as Send Time, Receive Time, SSL, Waiting Time, and Blocking Time in my response times.

It needs to support SAP GUI-based applications, which forms 60-70% of our portfolio, as well as Windows (client install) and Citrix-based applications.

Almost everything about NewLoad looks good, but I would like to have an indication of the variables for error messages appearing in the error tab. This would help us identify the users triggering the errors.

The support needs to be improved, especially when you are looking for a quick turnaround because of the fast-paced environment we work in.

We would like the product to be able to handle large project suits with large number of test scripts. Currently, we are not able to upload/publish huge projects back and forth from the team server to the NeoLoad controller. There is no support for VT3270 terminal emulation to support mainframe-hosted applications. There is no summary graph that shows the distribution of all of the response times on a single graph. We currently export the data as a coma separated variable (CSV) file and import into Excel to generate this summary graph.

We had some initial issues with the number of tests, memory leaks, and the controller itself.

It would be nice if this could be cheaper.

Some of the documentation is geared more towards Java. We are an all-Windows shop and it would be great to see some C# examples on how to integrate a custom performance counter from functional testing tools into NeoLoad.

They are coming out soon with NeoSense. It's a hybrid between app management tools and a testing tool. This will be helpful for us because we'll be able to monitor applications within various different environments. This is precisely what we have wanted.

NeoLoad’s lack of support for a complete protocol suite is somewhat limiting. While they do offer some protocols (Kronos, GWT V2.6) that others do not, they are not a one stop shop for protocol testing. Neotys is considered a forward looking company in the software performance testing world and legacy protocols such as RTE, SAP GUI and RDP are not included in their suite of protocols. I would like to see a more comprehensive offering of protocols with future releases.

They have done well with patching and enhancements in each version. More of a “community” of shared experiences, help, and collaboration would be a good enhancement to their site.