Practical Assurance Blog

OSSEC & ELK Stack Integration

OSSEC is the leading open-source host-based intrusion detection system (HIDS) software on the market today. OSSEC performs log analysis, integrity checking, Windows registry monitoring, and much more. It is setup in a server client configuration that can be installed and setup from simple scripts within minutes.

OSSEC offers an open-source web user interface (Web UI) that is very basic and not very customizable. To change this, companies started to integrate with Elasticsearch, Logstash, and Kibana (ELK Stack) giving users more freedom to customize dashboards and find the data they needed faster. This integration has been lead by the open-source project from the team over at Wazuh. They have made a customized version of OSSEC that is configured to automatically integrate with the ELK Stack which also includes some custom OSSEC rules that helps you monitor your systems in regards to PCI and CIS compliance.

This post will guide you through the process of installing OSSEC Server and guide you how to integrate OSSEC with with the ELK Stack on Ubuntu 14.04 Server. We will also describe how to import the custom PCI and CIS Wazuh dashboards and custom rules.

OSSEC Server Installation

Copy scripts folder to server using a secure copy command

scp -r {{PATH_TO_TEMP_FOLDER}} {{ubuntu_user}}@{{Server_IP}}:/home/{{ubuntu_user}}/

SSH into your Ubuntu 14.04 Server and go to the OSSEC_ELK_Temp directory in your home directory. Run the OSSEC_Server_Install_Step1.sh bash script.

sudo bash OSSEC_Server_Install_Step1.sh

Answer Y to confirm the installation from apt-get and then enter your preferred language.

On the next screen, press ENTER to continue and then type “server” when the installation asks what kind of installation do you want.

For the proceeding questions, use the following answers:

Question 2 choose the default option [var/ossec]
Question 3.1, enter your email address and confirm the SMTP server.
Question 3.2 choose the default option [y]
Question 3.3 choose the default option [y]
Question 3.4 choose the default option [y] for both questions
Question 3.4 Part 3 choose the default option [n]
Question 3.5 choose the default option [y]

After answering all the questions, press ENTER to run the installation and your final result should state “Configuration finished properly.”

Press ENTER and the bash script will start OSSEC. A result of “Completed.” should be shown

Next you will want to add an agent to get some logs flowing into OSSEC. Press A to add an agent and then fill out the Name of the new agent, the IP, and the agent id. Confirm adding the new agent by entering Y.

NOTE: The bash script automatically opens the agent manager but to add more agents in the future run /var/ossec/bin/manage_agents

OSSEC Agent Installation

Open the OSSEC Agent Manager console if it is not open already

sudo /var/ossec/bin/manage_agents

Enter “E” stating you want to extract an agent key and then enter the agent ID that you want to extract. Highlight and copy the key to be used after the agent has been installed.

Copy Scripts folder to the agent server using a secure copy command

scp -r {{PATH_TO_TEMP_FOLDER}} {{ubuntu_user}}@{{Server_IP}}:/home/{{ubuntu_user}}/

On the Agent Server, change into the OSSEC_ELK_TEMP folder and then run the OSSEC_Agent_Install_Step2.sh bash script to install the OSSEC Agent

sudo bash OSSEC_Agent_Install_Step2.sh

A configuration window will appear and enter the IP Address of your OSSEC Server

In the Agent Manager, enter “I” stating you want to add an agent key, and then paste the key into the agent manager. Press enter, and confirm the entry by entering “y”.

ELK Stack Architecture

Elasticsearch, Logstash, and Kibana can be configured in a multitude of ways. This guide will be using the single host configuration where all components of the ELK Stack including OSSEC is installed on the same virtual machine. The ELK Stack can be distributed across multiple hosts and this configuration can be explained more in detail here in the Wazuh project documentation.

ELK Stack Prerequisites

Elasticsearch, Logstash, and Kibana require the Java 8 JRE to be installed. To install this, on the OSSEC Server machine go to the scripts folder that you copied over earlier and run the ELK_Stack_Prereqs_step3.sh bash script to install Java 8 JRE.

sudo bash ELK_Stack_Prerecs_step3.sh

Logstash Installation

To install Logstash, on the OSSEC Server machine go to the scripts folder that you copied earlier and run the ELK_Stack_Logstash_Install_step4.sh bash script.

sudo bash ELK_Stack_Logstash_Install_step4.sh

Install Elasticsearch

To install Elasticsearch, on the OSSEC Server machine go to the scripts folder that you copied earlier and run the ELK_Stack_Elasticsearch_Install_step5.sh bash script.

sudo bash ELK_Stack_Elasticsearch_Install_step5.sh

The script will automatically open up the elasticsearch.yml file, in the file set the cluster.name equal to “ossec”, and the node.name equal to “ossec_node1”

At the end of the elasticsearch.yml file, add the following lines shown in the image below to the end of the file. Then save the file and exit out of it.

Next, the script will open the limits.conf file. Add the following lines shown in the image below to the end of the file. Then save the file and exit out of it.

Next, the script will open the default/elasticsearch file. In this file set the ES_HEAP_SIZE equal to half the amount of RAM on the system.

In the same file, uncomment the following lines shown in the image below. Then save the file and exit out of it.

Kibana Installation

To install Kibana, on the OSSEC Server machine go to the scripts folder that you copied earlier and run the ELK_Stack_Kibana_Install_step6.sh bash script.

sudo bash ELK_Stack_Kibana_Install_step6.sh

Once the script has completed, in your web browser on your local machine navigate to http://{{OSSEC/ELK Host IP}}:5601. This should open up the Kibana web interface.

Next you will need to import the OSSEC index pattern by typing “ossec-*” into the textbox and then click create.

In the navigation select “Discover”. This shows all of the recent OSSEC alerts.

Wazuh Custom Dashboards

By default, the custom Wazuh dashboards are not imported into Kibana. To import them, navigate to this link and download the JSON file to your local machine.

In Kibana, go to settings, objects, and then click on import and select the JSON file you just downloaded.

3 dashboard should appear in the list. Click on the eye icon next to the OSSEC Alerts Dashboard to open the OSSEC Alerts Dashboard.

Wazuh Custom Rules

To import Wazuh’s custom OSSEC rules, on the OSSEC/ELK server, navigate to the scripts folder that you copied earlier and run the Wazuh_Rulesets.sh bash script.

sudo bash Wazuh_Rulesets.sh

When crontab opens, add this line to the bottom of your crontab file to update the Wazuh rules on a weekly basis, then save and exit the crontab file.

@weekly root cd /var/ossec/update/ruleset &amp;&amp; ./ossec_ruleset.py -s

Cleanup

In the root directory of your user’s home folder on the OSSEC/ELK server, run the following commands to remove the temporary files.

sudo rm -rf ossec_tmp/
sudo rm -rf OSSEC_ELK_TEMP/

Conclusion

Completing this guide and getting OSSEC and the ELK Stack running is just the start of what can be accomplished. Custom Kibana dashboards can be created that allows you to monitor the alerts that are most important to your company. OSSEC can also be customized allowing you to create custom rules and rulesets that could be integrated into your custom dashboards. As more agents are added to the OSSEC server and more user start accessing Kibana, using the distributed architecture would become more important allowing the system to grow and expand.

We would like to thank the Wazuh project for all the hard work and dedication they have put in making the integration of OSSEC and the ELK Stack quick and simple. If you would like more information about the Wazuh Project, the ELK Stack, or OSSEC, view the links below.

Wazuh Project

ELK Stack

OSSEC

Host Based Intrusion Detection Systems

Host based intrusion detection systems (HIDS) is a intrusion detection system that is placed on a single host system. This involves an agent being installed on the host system that monitors and reports the system configuration and application activity. Some common abilities include log analysis, integrity checking, policy enforcement and, rootkit detection. Most HIDS can be customized for specific use-cases allowing you to build custom rules right out of the box.

In the market today, there are many HIDS available to be implemented into your infrastructure. In this post will be focusing on solutions that could be used by both startups and enterprise level businesses. The three HIDS that we will be discussing are Threat Stack, Tripwire, and OSSEC. We decided to focus on these solutions due to the market share that they hold, and if they included an open-source solution.

Threat Stack

Threat Stack is a SaaS offering that strives to provide “continuous security monitoring for public, private, and hybrid cloud infrastructures…”. This includes protecting servers and data protection. Treat Stack features workload insights, infrastructure monitoring, compliance reporting, threat intelligence, and vulnerability management. It works by creating a Threat Stack account and then installing the agent on one of the supported Linux Operating Systems (Windows not supported). All logs and alerts are sent back to the Threat Stack server and then can be viewed using the Threat Stack web interface.

Tripwire

Tripwire is a “software security and data integrity tool useful for monitoring and alerting on specific file change(s)..”. Unlink Threat Stack, Tripwire does work on Windows Operating Systems, but requires the paid enterprise version. Tripwire creates a baseline of all files in an encrypted file and then monitors the files for changes. It works by installing the Tripwire server application on a Linux server for the open-source version, and Windows/Linux for the enterprise version. Agents are installed on the servers and configured to know what files should be monitored.

Tripwire is available in an enterprise and open-source version. The open-source version is very limited and does not generate real-time alerts. The enterprise version is a full-version of the software and can be setup to send out real time alerts upon intrusion detection. Tripwire Enterprise starts are $699 plus a node licensing fee on top of that.

OSSEC

OSSEC is a “scalable, multi-platform, open-source Host-based Intrusion Detection System…”. OSSEC has a powerful correlation and analysis engine that integrates log analysis, file integrity checking, Windows registry monitoring, and much more. OSSEC has a real-time alerting engine that can send notifications a variety of ways including Email, Slack, and PagerDuty. It works by installing the OSSEC server application on a Linux based host, and then installing the agents of a variety of host Operating Systems. The agent can be installed on Windows, Linux, and macOS. The agents and server communicate by sending encrypted messages using the Blowfish algorithm and compressing using zlib.

OSSEC is completely open-source and has a very active community. OSSEC can also be integrated with the ELK Stack giving you a powerful search and web ui. Alternatively, OSSEC has created its own web ui that can be downloaded and configured from the OSSEC Github account.

Comparison

Platform	Pros	Cons
Threat Stack	– Workload Insights – Infrastructure Monitoring – Deployment using Chef/Puppet/Ansible/Salt – Compliance Reporting (Advanced/Pro Version) – Vulerability Management	– Only Supports Linux – No open-source version
Tripwire	– Monitors files and reports unauthorized chanegs – Creates baseline of all the files and then monitors for changes	– Open-source version is Linux only – Open-source version does not generate real-time alerts upon intrusion dection – All aleart for the open-source version are saved in log files – Open-source version cannot detect any intrusinos already on the system prior to installation.
OSSEC	– Agent runs on Windows, Linux, and macOS – Server and agent communicates via encrypted messages – Advance log analysis engine – Can be integrated with Slack and PagerDuty – Can be integrated with the ELK Stack	– Upgrading to newer versions can be difficult – When upgrading, previous defined rules are overwritten by default values

Conclusion

Depending on the specific use-cases and the level of service that you are needing will guide you to which solution would be best for your company. If you are looking for a very good all around solution that has community support but no enterprise level support then OSSEC would be the best option. If you are needing the peace-of-mind of having a support team available to you at anytime then Threat Stack would probably be the next best option. Either way, adding a host-based intrusion detection system would not only help in your compliance efforts, it would also add an extra layer of monitoring and security giving you a deeper insight into your infrastructure.

HELP: I have my first security questionnaire – now what?

Don’t panic. We’re here to help!

In this post you’ll get the quick rundown on the first steps to take on how to respond to a security questionnaire when bidding for IT work from a large company and also insight into the process. While the security questionnaire may look intimidating, we’re here to break it down in easy to digest pieces.

But wait – why are these questionnaires even necessary and what is the company really hoping for? While best practices recommend tech founders to consider information security and compliance when writing their business plan – we know it’s not always possible. Large companies know that security is at the heart of trust and good business. Security matters to the customers, but also to the investors, employees, and partners.

What do security questionnaires ask?
The length of security questionnaires will differ – but some can be up to 300 pages long. They will ask for an in-depth description of your security controls, business continuity, change management and security policies.

Here are some sample questions:

Is there an enterprise level system in place to detect and remove malware, and what is the regular schedule of operating system and application patching on all equivalent systems?
There is a report structure in place that can be generated that can cross-reference authorized staff and physical access permissions so that Company can be assured that only properly authorized persons have direct contact with data, systems or other information.
Vendor provides a multi-level backup process that provides Company with redundant systems for business continuity and disaster recovery. These are included in the incident discovery and response plan that measures the mean time to recovery (MTTR).
Security awareness training is applied to all Vendor personnel working for, with, or on behalf of Vendor on a regular basis (at least yearly and upon hire).

To top it off, it’s not enough to just answer these questions – a lot of them will require you to take action and fix the gaps in your security protocols. Yet, you’re working on getting your startup off the ground and start selling product.

At Practical Assurance we know that it’s not always possible to have a dedicated security person on staff who is knowledgeable and can navigate through the security process. It’s not enough to just identify and fix the security gaps – you need someone who can preempt future ones.

9 Best Internal Control Examples

When developing a compliance plan for your company one of the first tasks is identifying how your information security management system operates. Below we have provided several internal controls examples to demonstrate the types of polices, procedures, and technical configurations a company may establish to build a strong control environment. Ideally, a pre-cursor to establishing internal controls is a risk analysis.

Controls are a means to mitigate risk. Adding a control could be seen as slowing down business, so it’s necessary to ensure that only the right controls are prioritized and implemented. You may be asking, what are internal controls? It can be anything from a policy that directs what should be done, a procedure which describes how something should be done to reduce risk, a technical configuration to prevent information exposure, or monitoring to detect malicious activity. Controls are generally categorized as preventive or detective.

Below are 9 examples of common internal controls:

Information Security Policy – a foundational document that defines the administrative, technical, and physical security requirements of an organization. It is a document that defines how information confidentiality, integrity, and availability is protected.

Annual Security Policy Review – a procedure to ensure that the information security policy remains up to date. Over time company goals change, there are personnel changes, and new threats emerge. Reviewing your information security policy annually will keep your company current.

Confidentiality Agreement – a legal document that employees typically sign that requires them to keep all company and customer data confidential. The purpose of this is to prevent information leakage.

Encryption Policy – a document that describes how and when a company uses encryption. An example encryption policy may state that all customer data in transit or at rest must be encrypted. Policies typically also specify encryption algorithms and key lengths.

Change Management – a process that enables the secure and structured approach to management changes to system configurations or application code. Change management is a category that often includes controls as testing and QA, source code versioning, peer review, and segregation of duties between developers and production engineers.

Backup and Recovery – a process that ensures that data remains available when needed. Companies often focus a lot on backup but fall short when developing recovery plans. Backups should be tested on a regular basis.

Security Awareness Training – people are often the weakest link in any information security program. Regular security training, reminders, and documentation to prove it occurred goes a long way in keeping auditors happy.

Semi-Annual Review – just as policies and procedures go stale, this control ensures that accounts and configurations on systems remain up to date. New employees are hired, job responsibilities change, and terminations happen. This ensures that system access control remains consistent with the workforce.

Vendor Patching – keeping software such as applications and operating systems up to date is one of the best ways to prevent getting hacked. Software patching should occur on a regular basis for normal updates and immediately for critical updates.

Our cloud provider already has a SOC 2 and other certifications, do we still need to do it?

If your company is using an IaaS (Infrastructure as a Service) provider such as AWS (Amazon Web Services), you’re probably impressed with number of certifications they have collected. A SOC 2 Type II from an IaaS provider will often cover most of the physical security requirements. Depending on how your system is configured, it may cover backup & recovery, and disaster recovery portions. A SOC 2 Type II from your cloud provider will not cover your application, your internal policies, etc. Using cloud services are helpful, but will not give you 100% coverage.

How long does it take to prepare for a SOC 2 audit?

On average, going from zero to SOC 2 Type II will take from 8 months to a year. Smaller companies that don’t have many systems can often complete the process faster. To further expedite the process, it is advisable to not create all policies and procedures from scratch. Many security & compliance consultants have built vast libraries of policies and procedures that can be customized for your business and make your life easier.

We already have good security, is that enough for SOC 2?

Having good security practices in place is certainly a good start, but often not sufficient for compliance. Security does not equal compliance, and vice versa. Preparing for SOC 2 may include Security (logical & physical), Availability, Integrity, Confidentiality, and Privacy. Newer/smaller companies often prepare for a SOC 2 by creating many of these policies for the first time. The creation of new policy will often lead to the implementation of new preventative and detective controls.

Who typically leads a SOC 2 compliance effort in a company?

Large organizations typically appoint a Chief Security or Chief Compliance Officer to manage audits from beginning to end. Smaller companies tend to outsource expertise and form a team to prepare for compliance. It is best implemented as a team effort because policies changes will impact everyone in your company. As with any major project, executive buy-in is key. The value of compliance isn’t always apparent and having the right people on board will help immensely.

How do companies prepare for SOC 2?

SOC 2 preparation usually happens in a few stages. First, your company should identify all “key systems” and perform a gap analysis against all requirements documented in the Trust Services Principles and Criteria. Next, existing security controls should be identified and policies and procedures should be written to meet all requirements. This can take anywhere from a few weeks to up to 6 months, depending on the size and maturity of your company. At this point you are ready for the SOC 1 Type I audit. A SOC 2 Type II audit is typically performed 6 months later.

What is the difference between a SOC 2 Type I and SOC 2 Type II audit?

A SOC 2 Type I audit is an audit reporting on the policies and procedures a company has established at a particular point in time. It is generally the first step taken and is often referred to as “test of design.” It will answer the question, “are the controls properly in place?” A SOC 2 Type II audit is a “test of effectiveness” over a period of time. The “period of time” is generally no less than 6 months and no more than a year. It will answer, “is your company following it’s own policies?”