Nemesida WAF: The WAF That DevOps Love

When developing or supporting web applications, at some point in time you have to deal with the need to use WAF (Web Application Firewall). If you have no experience working with such a class solution or you are tired of constant false positives, I will tell you how to simplify the task, as well as share tips and tricks. As a tool, we will use Nemesida WAF Free — the free version of Nemesida WAF.

Visualization, or let’s start from the end

You can monitor the work of Nemesida WAF Free through a browser, so after a short setup of the system, we will get access to the web interface, which will provide information about blocked attacks, reasons for blocking, information about IP addresses, etc. In addition, there will be sections with summary statistics in the form of graphs, charts and traffic data from the VTS module (if it is used).

Demo: demo.lk.nemesida-security.com (demo@pentestit.ru / pentestit)

Now let’s start the installation.

Installing Nemesida WAF Free

Despite the fact that the free version is just a part of a full version, it contains a large set of features for detecting, blocking and visualizing attacks on web applications. By a web application, we mean everything that is built on the HTTP Protocol: websites, personal accounts, online stores, training platforms, APIs, and everything else.

In the previous paragraph, I purposely divided the attack blocking functionality into detection and blocking, since there are 2 (even three) modes of operation of the product: IDS, IPS, and PseudoIDS (LM mode).

IDS mode

IDS mode allows you to use WAF on a copy of traffic, detecting but not blocking attacks. This mode of operation is useful, for example, for primary start-up or for passive monitoring, in order to avoid any blocking of requests or an increase, even insignificant response time. For an example of configuration, we will use Nginx for the transmitting server (although you can use any other one, for example, Apache2 or IIS).

The configuration of the transmitting server:

(instead of 192.168.0.1, you must specify the address of the server with Nemesida WAF installed)

After making changes and restarting the web server, incoming requests to this server will be broadcast to the server 192.168.0.1 with Nemesida WAF installed (its configuration is simple and will be described below). This scheme of work allows you to monitor attacks without blocking them, but at the same time, without affecting the main server.

IPS and PseudoIDS mode

The other 2 modes of operation assume the use of WAF “contrary”, while in IPS mode detected security incidents are blocked, in PseudoIDS mode — they are fixed, but not blocked. The latter mode is convenient because switching between these two modes occurs with simple options: the ability to switch to PseudoIDS mode both by server name (option nwaf_host_lm) and by client IP address (option nwaf_ip_lm).

In general, Nemesida WAF Free provides many parameters for “fine-tuning” the system: flexible functionality for creating your own rules for blocking and exceptions, the ability to add the client’s IP to the “list of exceptions”, the option to configure a ban for all and for each individual virtual host, etc. All of this can be managed through the configuration file in the free version, and in the full version — also through API calls.

Let’s go back to the installation procedure. Nemesida WAF is presented in the form of several components:

  • Dynamic module for Nginx
  • Nemesida WAF API (accepts events from Nemesida WAF and puts them in Postgres for further output to the PA or integration with SIEM systems)
  • Personal Account (web interface for monitoring incidents)
  • Nemesida AI machine learning module
  • Nemesida WAF vulnerability Scanner
  • Nemesida WAF Signtest — web interface for managing the machine learning module

In Nemesida WAF Free, we only need the first three — the dynamic module itself, the Nemesida WAF API, and the Personal Account. All components are available as installation distributions and allow you to connect Nemesida WAF to an already installed Nginx instance, starting with version 1.12 (Stable, Mainline and Plus versions of Nginx are supported).

Dynamic module Nemesida WAF

For those who install the distribution not for the first time, the process of installing and running the dynamic module takes about 5–10 minutes. The dynamic Nemesida WAF module can be connected to an already installed Nginx (or compiled from source with its own modules).

Nemesida WAF repositories are available for the following operating systems: Debian 9/10, Ubuntu 16.04/18.04/20.04, Centos 7/8. Video instructions for installing and initial configuration of components are published on the Youtube channel. We recommend you to see one of them, but it is better to install and configure it according to the documentation on the main site, since some parameters may become outdated, while others may be added.

After Nginx is configured, connect the Nemesida WAF repository that is appropriate for your OS and start installing. The product is also updated from the repository. Installation instructions.

Nemesida WAF API and Personal Account

After the dynamic module is installed and launched, it is time to start installing the remaining two components: Nemesida WAF API and Personal account.

The Nemesida WAF API is presented as an API written using Flask, and is designed to receive events from Nemesida WAF, Nemesida WAF Scanner, and Nemesida AI, and then place these events in the database. PostgreSQL is used as a DBMS. In the free version of Nemesida WAF, only information about blocked requests will be transmitted to the database. Manual.

Once the Nemesida WAF API is configured and connected to PostgreSQL, it’s time to start launching your Personal account. According to the documentation, we install, configure, perform migration, specify the user and password to log in. Manual.

From experience, installing the last two components causes more difficulties (usually some steps are skipped, for example, they forget to make a migration or allow connection to Postgres), so for a quick start, we prepared a Virtual Appliance (a virtual disk with Debian 10 and Nemesida WAF components, 3GB before unpacking), and also made 2 Docker images: for the dynamic module and for the Nemesida WAF API/Personal account.

Well, the most boring part is over, now we can check out the WAF in action.

First hack

To test the operation of an already configured WAF, it is not necessary to remember different variations of attacks. We have created a test signature that will allow you to check whether Nemesida WAF works and whether blocked attacks are displayed in the PA. The current set of used signatures available by link: rlinfo.nemesida-security.com.

We send a request (I did it through the console, but it’s better to do it through the browser for clarity):

or, if you want something more close to reality:

We get the response code 403:

And in the PA in a few seconds, an attack should appear:

If the request was not blocked — the WAF is incorrectly connected or configured (perhaps the address or host was added to WL/LM), if the request was blocked, but there is no information in the PA — check that the interaction with the Nemesida WAF API and the PA is configured correctly. In any case, you can always ask a question on the forum.

Custom 403 page

By default, the 403 page (a page with a 403 response code) is nondescript and stingy with information. Nemesida WAF in conjunction with Nginx allows you to make it beautiful and more informative.

In order for your server to return such a page, you must:

  1. Create a configuration file for custom pages (for example, in /etc/nginx/snippets/custom_pages.conf);

Add required parameters to Nginx:

Description:

error_page 403 405 = 222 /403.html; – set a custom 222 response code for response codes 403 and 405 and return the location /403.html;

Determining the location /403.html as an internal (that is, if you refer to it example.com/403.html — it will not be available), adding special headers for displaying the request ID ($request_id), the virtual host ($host) that we are accessing, the visitor’s IP ($remote_addr), and the response (reason for blocking) Nemesida WAF ($nwaf_block_type). Nemesida WAF has several types of blocking, for example, 1 and 2 — the request is blocked by signature analysis, 3 — by the machine learning module, 4 — by antivirus, etc.

2. Connect the created file:

Connect the created file to the Nginx configuration:

Example of connecting a custom page configuration file in a file with a virtual host (for example, in /etc/nginx/conf.d/example.com.conf):

3. Create a custom page (for example, /var/www/custom_pages/403.html) as follows (for example):

Example of a custom page 403

After restarting Nginx (with Nemesida WAF installed) all pages with response codes 403 and 405 will look like this:

In this case, the custom page will be updated every 7 seconds, and if the client’s IP is not banned, the root page of the site will return.

Automatic ban

You probably have heard or even use automatic ban systems like Fail2ban and know about their advantages and disadvantages. Nemesida WAF uses a built-in and easy-to-configure ban mechanism that allows you to set an arbitrary block period for server virtual hosts. Blocking occurs by IP address, and the following options are available for management:

  • The number of IP attacks that result in blocking;
  • The cutoff period;
  • Virtual host to target (optional).

The automatic blocking parameter is controlled using the nwaf_limit parameter available in the /etc/nginx/nwaf/conf/global/nwaf.conf file. Using this parameter will be useful in cases where the site is being scanned for vulnerabilities or when trying to promote a detected vulnerability.

Exception lists (While lists)

The work of WAF is based on the principle of analyzing incoming requests to the server and responding in cases where they contain signs of an attack or anomalies. The use of machine learning algorithms coupled with improved normalization technology in the full version of Nemesida WAF allows you to detect such attacks accurately and with an ultra-minimal number of false positives (about 0.01%), but in the free version, to reduce the number of false positives, we rest against the limitations inherent in the signature analysis architecture. Thus, the free version has a higher number of false positives, and to solve this problem, you have to use exception lists (or ”white lists”). Creating exception rules is also available in Nemesida WAF.

Most often, false positives appear when the administrator/moderator of a web resource makes an update or change through the web interface, passing in the request body atypical constructions for the user:


$html = curl_exec($ch);
curl_close($ch);
return json_decode($html,true);

A legitimate request will be blocked because it contains an occurrence of the json_decode() function, and such requests are not typical for ordinary visitors to the web resource.

In cases when administrators can not interact with it by bypassing the WAF, you can add the IP address from which they access the resource to the exception list, or switch the address to PseudoIDS mode (the nwaf_ip_lm option) to capture events without blocking. But with such actions, you always need to be quite careful.

By the way, Nemesida WAF allows you to add not only IP addresses,but also subnets, if necessary.

Conclusion

No matter how well you think the code is written, it applies the input filtering and special frameworks like HTML Purifier, designed to be removed if not all malicious code, most of it, you must use the WAF to improve security level.

If you are going to use WAF for the first time, or you are tired of the endless addition of exclusion rules, we recommend you to try the free Nemesida WAF Free. For professional use (blocking complex attacks, brute-force attacks, SMS flooding, vulnerability search, virtual patching system, etc.), a full version of Nemesida WAF with a machine learning module and a vulnerability scanner is required. However, for most non-targeted attacks and mass scans, Nemesida WAF Free will be a good and convenient tool.

In this article, I tried to reveal the situations that most often arise when using solutions of this class. I will be glad if you remember more in the comments.

From Information Security With Love

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store