IT Central Station is now PeerSpot: Here's why

What's the best way to trial log management tools?

Nurit Sherman - PeerSpot reviewer
Content Operations Manager at PeerSpot (formerly IT Central Station)

Hi community members,

We know it's important to conduct a trial and/or proof of concept as part of the buying process. 

Do you have any advice for our community about the best way to conduct a trial or PoC? How do you conduct a trial effectively? 

Are there any mistakes to avoid?

PeerSpot user
99 Answers

reviewer1195575 - PeerSpot reviewer
Top 5Real User

Log management can be simple or very involved. The more they are VERY elaborate, the more maintenance is going to be an issue. I'll explain

Vendors change their software, and when they do they don't all document those changes. Thus if you develop lots of distributed collection mechanisms that also filter, pre-processes and correlation extracted messages, they will break, and you'll have tpo issues versions of them. If you have 10 000 sources that need very careful change management and version control of your monitoring mechanisms.
So you need a lab/Dev environment to test the newer version of software, firmware, etc hopefully before it gets deployed. This assumes you have the means to insert into well defines test schedules. that's not always the case especially if the newer versions must be deployed to fix issues impacting the business. Monitoring is often 2nd or lower priority.

It is best to log and process at the edge or source. If you DON'T use any processing at the source, then you'll bring tons of log data to a central or intermediate store, a big data requirement will emerge with the usual overhead of data management over time PLUS the network bandwidth issue it could generate between the objects and the centralized management.

Our advice is to choose tools that CAN provide local processing at the edge/source.
Use regex tools to test complicated expressions or a rules-based engine, etc to perform the filtering and correlation. 

Ensure you have effective IT processes to manage versioning of the monitoring artifacts, as they WILL need to change. When you have to change just a few chars in the regex or rule itself, you'll have a hard time managing that over time per device/app type and version if you rely on your memory. Those modifications will be hard to find visually, and you'll waste a lot of time doing

Define strategies to store logs (like any other monitored data really).

Log generation is component instrumentation, typically created by developers. Guess what developers change, the quality of the instrumentation can vary immensely. 
Having the means to compare older logs with newer logs can be vital to proving to a vendor that he's the source of an issue when monitoring fails or behaves differently. Save yourself time when claiming support by providing that evidence to force them to admit they've changed things BTW they will. Some VERY large and famous vendors are notorious for changing instrumentation content and API behavior WITHOUT letting you know. be warned and forearmed.

Finally, consider carefully whether the entity doing the log centralisation for you is impacted by issues that might mean the information it provides is erroneous.

For example, you may use an expert management system that centralises logs for 100 000s of devices. Examples are building management systems, network management software, Camera VMS systems. Using these systems to centralise logs and simpler integration for data collection, may make data collection easier, but consider your log collection and analysis may also need to solicit the device directly as well. This is particularly important if the result of certain messages captured is meant to be escalated to busy and expensive rare resources or even service providers that will be billing you for each alert sent

Doing a proof of concept of all these and probably more will help to refine the production monitoring, trigger the creation of missing IT best practices, and avoid unnecessary expense.

Carl Phillips - PeerSpot reviewer

At the risk of sounding flippant,  I personally believe that the best way to trial log management tools is best encapsulated in these three words, "clarity, clarity, clarity".  Clarity as in having an agreed upon internal understanding of exactly what problem(s) you and your team are trying to solve.  This is critically important because a fair amount of consumers inadvertently conflate Log Management with the components of a SIEM, i.e., Security Event Management, Security Information Management, etc.
If there is currently no one on staff that possesses the background and experience to develop and internally socialize a list of criteria for the trial / PoC, I would definitely leverage my interactions with the Vendors to develop my list of criteria.  If budget and internal strategies permits, I would recommend utilizing a 3rd party resource with demonstrated real-world experience and knowledge in this space to assist with providing the thought leadership and direction for the effort.
Some suggestions for decision / evaluation points that you many want to consider as part of your trial / PoC.
1.  Scope 
2.  A defined set of Use Cases
3.  Metrics     
a.  response time      
b.  retention requirements     
c.  other
4.  Log Sources      
a.  COTS - (diversity of log sources, Windows, Unix, Linux, Mainframe, Routers, Firewalls, etc)     
b.  In-house developed      
5.  Problem Set(s)     
a.  Regulatory Compliance     
b.  Security Monitoring / Analysis / Response     
c.  Security Audit Remediation     
6.   Key indicators for success / product selection

Some items to think about:
1.  If you perform a POC / Trial with vendor equipment, do you have a policy / plan that prevents the unintentional retention of internal log data on vendor's hardware, i.e., disks, etc?  ( you absolutely       want to ensure that critical / sensitive log data does not go out the door with the vendor! ) 

it_user756744 - PeerSpot reviewer
Real User

For trial, you need to define,

1. Use case

2. Budget

3. Resources

Use case is very important as withing same budget you may get multiple products.

Understand use case like,

What do we need to achieve?

a) Asset Logs

b) Network Logs

c) What kind of logs

d) Space Requirement

e) Vulnerability Scanning

f) Events alert Custom

g) 0 Day Protection

Tim Dasch - PeerSpot reviewer

The rigor required to adequately test a SIEM tool can be overwhelming. 30 or 45 days may not be enough time, particularly if log management is just your part-time role. check the recommendations of others in your industry and lean toward those tools. you will have to spend some bucks to get something useable without tons of setup and configuration time. and it's not a set it and forget it situation; it requires lots of tuning, else you have wasted your money for a tool you don't use.

Michael SCHLEICH - PeerSpot reviewer
Real User


Choosing a Log Management solution depends strongly on its purpose.
What will you do with this solution?
If a mandatory feature is missing, you will have no other choice than to eliminate the product from your final choice.

This is why it is important to know which feature is mandatory, critical, important or nice to have.

There is one method in 4 steps:
Firstly you have to build a list of all the features you need to have or you would like to have.

You have to do this list for all aspects like (this list is not in a specific order):

Logs collection methods
Logs parsing capabilities
Logs format supported
Inter-compatibility (very important if you have already a SIEM by example)
Product Support (vendor or open-source)
Chart types
Searches capability
Searches regex
Searches operator
Concurrent searches limitation
Data retention
Available Statistics information (like latest logs received, EPS, top values, bottom values)
Static correlation like lookup feature
High Availability
Load balancing
There is maybe other features or capabilities that are very important to you, do not forget them.

You have to do this exercise again and again with your team like a brainstorming.
I don't think you will think about every point in one shout.

This will permit you to build a spider chart with those features (I advise you to create 6 or 8 main categories and you put the above features in their related category to build the chart) that will be your reference to choose the product.
I recommend you to be very focused on the most important points for your business.
If a feature is not mandatory put a lower value this will permit you to increase your choices between different Log Management Solutions.

Thirdly, I will start to evaluate the different products in starting with the one with a free version like Splunk, ArcSight Loggers, etc...

In the last step, for each product, you will evaluate, build also a spider chart then compare it with your reference to know which solution will be the best for you.

To do the test, I recommend sending a bunch of all logs type you will collect to do not have a bad surprise.
Windows, Firewall, Proxy, AV, IDS, IAM, etc. . . And Custom logs, this last one is for me highly important.

This answer concerns only Log Management, I have not talked about SIEM.

Personally, we have chosen ArcSight Loggers just because we had already ArcSight SIEM. The inter-compatibility feature was so important that we have not tested Splunk which is for me the best current log management solution.

I hope I have successfully answered your question.
Please, do not hesitate to contact me if necessary.

UmbertoAlloni - PeerSpot reviewer


for my experience a good Log management POC task must include:

- POC Business Requirements and Drivers of customer

- POC Scope agreement

- POC Success Criteria agreement

o Use Cases to proof for primary interest of the customer

- POC Collecting environment acquisition and definition

o Customer IT Environment

o Technology in POC requirements for installation and configuration

- POC Archiving environment acquisition and definition

o Customer IT Environment

o Technology in POC requirements for installation and configuration

- POC Tasks plan


- Make an evaluation score matrix for primary Log Management capability

o Log Collection capabilities – Using an Agent-based approach or Agentless approach, out-of-the-box log collection support for 3 rd party commercial IT


o Parsing & Normalization capabilities – Collected logs will be parsed and normalized to

o CIA guarantee capabilities - Information security's primary focus is the balanced protection of the confidentiality, integrity and availability of data (also known as the CIA triad) while maintaining a focus on efficient policy implementation, all without hampering organization productivity

- Avoid to proof capability or feature out of success criteria (wasted time)

Kent Gladstone-USA - PeerSpot reviewer

Mark is correct but there are things to look for. Do you have a set of requirements? Not all log managers collect the information, not all log managers are easy to navigate, and not all log managers provide the reports your are looking for. Check to see how much data it collects so you can plan storage. Does the log manager compress the data or does it dependent on a third party tool? Do you know what you are collecting, and why? Are the logs used for security, sox audits or something else? My advise, before testing, is to gather and review you're requirements and test against that. There are lots of free trials. In fact if there isn't one on the web, contact the vendor and they'll give you something to try out for 30 days.

Elena Stefanovska - PeerSpot reviewer
Top 20Reseller

A true SIEM product should be chosen in order to have real time correlation of the gathered logs (or even network flows with some of the vendors). You can start with the Gartner suggested visionaries and leaders, these are proven SIEM vendors that all have free trial of their full version product. In order to choose the best one to start, the most important thing is to clarify your requirements and to know what do you want to achieve with the Log Manager (SIEM). In this case you can choose the most appropriate one to start with a PoC, as most of them achieve the same, but have slightly different advantages like single management console, additional features included in the base product, etc.

it_user791052 - PeerSpot reviewer

Stick with the tried and true SIEM/Log Management Vendor that offers a free, online download and trial, easy to install and operate piece of software with proper documentation. A good example of that can be found here:

Buyer's Guide
Log Management
July 2022
Find out what your peers are saying about Splunk, Datadog, IBM and others in Log Management. Updated: July 2022.
621,548 professionals have used our research since 2012.