1 of 32

v1.4.0

Overview

The following documentation is for version 1.4.x; different versions are available at the top left of the sidebar. ("Next" are the most recent changes which have not yet been bundled into a release.)

Curiefense is an API-first, DevOps oriented web-defense HTTP-Filter adapter for Envoy and NGINX. It provides multiple security technologies (WAF, application-layer DDoS protection, bot management, and more) along with real-time traffic monitoring and transparency.

Curiefense is fully controllable programmatically. All configuration data (security rulesets, policies, etc.) can be maintained singularly, or as different branches for different environments, as you choose. All changes are versioned, and reverts can be done at any time.

Curiefense also has a UI console, discussed in this Manual beginning in the Settings section.

Version

This documentation is for version 1.4.0.

(To view docs for a different version, choose it at the top of the left sidebar.)

Architecture and Components

Curiefense provides traffic filtering that can be configured differently for multiple environments (e.g. dev/qa/prod), all of which can be administered from one central cluster if desired. Here is an overview of its components.

In the diagram above, the Server represents a resource protected by Curiefense (a site, app, service, or API). The User is a traffic source attempting to access that resource.

Incoming traffic passes through Curiefense. Hostile requests are blocked.

The other components in the diagram represent the Curiefense platform, as follows:

Curiefense proxy (represented by the column with the Curiefense logo): Integrated with Envoy or NGINX; performs traffic filtering.
Elasticsearch stores access logs.
Access Logs: Traffic data viewable via Kibana.
Metrics. A Prometheus store of traffic metrics.
Dashboard. Grafana dashboard(s) with visual displays of traffic metrics.
Web Console. Curiefense's web UI for configuring the platform.
Config Server: A service which:
- Receives configuration edits from the Web Console.
- Receives configuration edits from API calls (not shown in the diagram).
- Creates a new configuration version in response to edits.
- Stores the new version in one or more Cloud Storage buckets.
Cloud Storage: Stores versioned configurations. Each Curiefense proxy periodically checks Cloud Storage: when a new version is found there, the proxy downloads it and updates its security posture.

For detailed information about the specific containers and services which perform the roles described above, see the reference page on Services and Container Images.

Deployment Options

Curiefense can run in a variety of environments, depending on your specific needs. It can be adapted to many different use cases.

Deployment instructions for several different environments are available in the Installation section of this manual and on the Getting Started page. More will be added in the future.

If you create an installation workflow for a situation that is not currently described in this manual, please feel free to submit it for inclusion.

Three Primary Roles

Conceptually, there are three primary roles performed by Curiefense:

Configuration (allowing admins to define security policies, assign them to URLs, etc.)
Filtering (applying the defined Configurations to incoming traffic and blocking hostile requests)
Monitoring (displaying traffic data in real-time and in historical logs).

Each is discussed below.

Configuration

Data Structures

Curiefense maintains its security parameters as Entries, which are contained in Documents, which are contained in Configurations.

A Configuration is a complete definition of Curiefense's behavior for a specific environment. An organization can maintain multiple Configurations (e.g., development, staging, and production).

Each Configuration contains six Documents (one of each type: ACL Profiles, Rate Limits, etc.) Each Document contains at least one Entry, i.e., an individual security rule or definition. Documents are edited and managed in the Policies & Rules UI or via API.

A Configuration also includes data blobs, which currently are used to store the Maxmind geolocation database. This is where Curiefense obtains its geolocation data and ASN for each request it processes.

Configuration Administration

A Configuration is the atomic unit for all of Curiefense's parameters. Any edits to a Configuration result in a new Configuration being committed. Configurations are versioned, and can be reverted at any time.

When a Configuration is created or modified (whether by the UI console or an API call), the admin pushes it to a Cloud Storage bucket. An important feature of Curiefense is simultaneous publishing to multiple environments.

When a Configuration is published, it can be pushed to multiple buckets (each of which can be monitored by one or more environments) all at once, from a single button-push or API call.

Filtering

Traffic filtering is performed by the Curiefense proxy, as shown in the first diagram above. In other words, this is where the security policies defined in the Configurations are enforced.

Some activities (such as rate limiting) require local data storage. Internally, Curiefense uses Redis for this. Other storage methods can be used if desired.

Monitoring

Each time a request goes through Curiefense, a detailed log message is pushed to elasticsearch.

Traffic data is available in several ways:

The Curiefense graphical client provides a Kibana Access Log which provides comprehensive details for requests.
Curiefense is also integrated with Grafana and Prometheus, for traffic dashboards and other displays.

Installation

Getting Started with Curiefense

This Quick Start guide shows how to quickly deploy Curiefense, set up some basic security policies, and test them.

Prerequisite: Ubuntu 21.04

In this Quick Start guide, we will deploy Curiefense using Docker Compose, then test and configure.

Or, you can:

Go through the process at Docker Compose deployment in depth for more control over the deployment options.
Use Helm instead, by following the instructions here: Istio via Helm.
Deploy for NGINX instead, by following the instructions here: NGINX deployment.

If you choose any of these options, after you follow the instructions on the appropriate page, return here to the Verify the Deployment section below, and continue.

Let's begin

To ensure we have all tools in place, we will run:

sudo apt update
sudo apt install -y git docker.io docker-compose libpq-dev python3-dev gcc python3-psycopg2 vim
sudo usermod -aG docker $(whoami) && sudo -i -u $(whoami)

Grab the latest code from GitHub to get started:

git clone https://github.com/curiefense/curiefense.git
cd curiefense/deploy/compose/
docker-compose up

💡 ProTip: Try Curiefense out on Katacoda, no install necessary.

Verify the Deployment

This tutorial assumes that host names curie.demo and api.curie.demo are mapped to the host machine. You may replace it with the IP of your host, or set your/etc/hostsfile accordingly. For example, if docker-compose was executed locally, then

127.0.0.1 curie.demo api.curie.demo

At this point you should have the following http interfaces:

Make sure everything is working by testing the echo server:

curl curie.demo:30081

Request served by echo

HTTP/1.1 GET /

Host: curie.demo:30081
X-Forwarded-For: <redacted>
X-Forwarded-Proto: http
X-Envoy-Internal: true
X-Request-Id: 0bced763-5d06-4724-875d-c5936b948d80
X-Envoy-Expected-Rq-Timeout-Ms: 15000
User-Agent: curl/7.74.0
Accept: */*

Deployed Containers Overview

This section describes the newly-described components of Curiefense. Feel free to skip it if desired, and go straight to the instructions for policy and rules configuration.

What Have We Just Deployed?

This diagram will help us understand the containers we just deployed, their connections, and the data flow:

Terminology and Concepts

Before diving in and making changes, let's discuss a few concepts of Curiefense's configuration.

Git is the storage management used to keep track of changes. This means:
1. Data can be stored anywhere a git repository can (local, remote, hosted, etc).
2. Every change you made can be reverted.
3. You can automate deployments based on tagging.
4. Using a single configuration server, you can maintain configurations of multiple deployments (e.g. production, devops, qa, rc, etc.) by keeping each in a separate branch, and you can merge them the git way at any point in time, via the API and/or UI.
Configurations are organized in Documents and Entries. More on this here.

Policy and Rules Configuration

Curiefense runs every incoming request (and in some cases, responses as well) through a series of mechanisms. We will now walk through some of them to understand how traffic is handled and processed.

During the procedures described below, you will set up some simple rules and then run some requests through Curiefense. By the end of this process, you will understand how to create security rules and policies, you will observe them being applied, and you will see how Curiefense reports on incoming requests and its reactions to them.

Open the UI

Open the UI management console by going to http://curie.demo:30080/. In the left sidebar, select Policies & Rules if it is not already selected.

At the top left of the page, in the second pulldown control, select Tag Rules.

Create a Tag Rule

Tag Rules attach tags to requests and sessions based on various criteria, from matching headers, cookies, arguments or URLs, to traffic sources such as geolocations, IP addresses, CIDRs, and ASNs. Subsequently, the tags are then used to make decisions about how the requests are handled.

Start by creating a new Tag Rule by selecting the "+" button at the top:

Next, in the Tags text box on the left, enter the value hdr-test .

Then, at the top of the (currently empty) list to the right, add a new entry by selecting the Create New Section button. An empty rule will appear.

In the left pulldown, select Header. Enter foo for its name, and test for its value. Then select "Add".

Your screen should look similar to this:

We have created a simple tag rule. Every request that contains a header named foo which matches the regex (PCRE) test will receive a tag of hdr-test.

Now save the new configuration:

And then publish it by going to "Publish Changes" in the left sidebar, and selecting Publish configuration:

After publishing, your changes need time to propagate. Waiting 10-15 seconds should be enough.

Now it is time to test our configuration. Let's run the following curl commands:

curl http://curie.demo:30081/no/header
curl http://curie.demo:30081/with/header -H "foo: test"

Navigating to Kibana in the left sidebar should show a screen similar to this:

This shows the two requests you just sent, in order of receipt (with the newest on top).

Click on the ">" to expand the top entry:

All the information about this request is available here. Scroll down until you see the "tags" field:

You should see that Curiefense added the hdr-test tag.

Notice also that along the hdr-test tag that you defined, Curiefense attached a number of tags that were generated automatically. More information about these.

Tag Rules Lists are a powerful feature of Curiefense (and are explained in depth here). We just demonstrated the ability to create a single-entry self-managed list that characterizes incoming requests based on a header. Curiefense allows you to attach tags based on complex combinations of headers, arguments, cookies, geolocation, methods, paths, and more. External data sources are also supported; sessions can be profiled based on data and rules defined by a third party, such as blocklists and whitelists.

Now that we know how to attach tags to incoming requests, let's tell Curiefense how to react to them.

Create an ACL (Access Control List)

We're going to block all requests with the hdr-test tag.

In the left menu, navigate to Policies & Rules. By default, ACL Profiles should already be selected.

Enter hdr-test into the DENY column. Your screen should look similar to this:

Save your changes, then publish the new configuration again (and wait 15 seconds for the changes to propagate).

Test the ACL

Run this command (note the addition of the -i option):

curl -i http://curie.demo:30081/with/header -H "foo: test"

You should see output that looks something like this:

HTTP/1.1 200 OK
content-type: text/plain
date: Tue, 03 Aug 2021 01:34:34 GMT
content-length: 308
x-envoy-upstream-service-time: 0
server: envoy

Request served by echo

HTTP/1.1 GET /with/header

Host: curie.demo:30081
X-Request-Id: dde00ccb-776a-4953-9011-51f7c52ed342
User-Agent: curl/7.64.1
Accept: */*
Foo: test
X-Forwarded-Proto: http
X-Envoy-Expected-Rq-Timeout-Ms: 15000

The -i allows us to see the response headers. Note that we received HTTP status code 200: "request succeeded." In other words, Curiefense did not block the request.

This behavior is expected. By default, Curiefense's security profiles are in report/monitor mode; requests will be flagged but not blocked. This mode allows for testing and fine-tuning of new configurations without affecting traffic.

This information is also visible in the access log. The log shows that Curiefense is configured to block the request, and the reason why this would happen:

And it also shows that the request was not actually blocked, because status code 200 was returned:

Now let's assume that we've tested our new policies and we want to make the ACL active.

Activate the ACL

Navigate to Policies & Rules and then choose to edit URL Maps in the upper dropdown list.

URL Maps assign security policies to paths within the protected application. You can assign policies at any scale, from globally down to individual URLs. (They are explained in depth here.)

We're going to edit Curiefense's default security profile: the one that applies to every URL which does not otherwise have any policies assigned to it.

Expand the default profile (the one assigned to path /) by selecting it. Then activate the ACL Policy by checking its Active Mode checkbox. Note that the name of the ACL Policy changes from red to green.

Save your changes and publish the configuration again.

After the change propagates, send the request again:

curl -i http://curie.demo:30081/with/header -H "foo: bar"

This time, Curiefense should block the request:

HTTP/1.1 403 Forbidden
x-curiefense: response
content-length: 13
date: Tue, 03 Aug 2021 01:44:57 GMT
server: envoy

access denied

As before, the blocking reason appears in the traffic log:

And so does the response status code:

Add Rate Limiting

You have seen how to create security policies that filter hostile requests, and how to assign these policies to paths within your application.

Now we'll see how to filter requests that are not obviously hostile at first, but which display hostile intent with increased volume. For example, a user who fails a login attempt might have mistyped their password—but a user who fails ten login attempts in a short time is probably trying to guess the password for an account they do not own.

Add a Rate Limit

Return to URL Maps and select the default profile again. At the bottom of its map, there is the Rate Limit Rules section. Currently, it is empty.

Attach an existing rule by select the first "here" link. A pulldown list of available Rate Limit Rules will be displayed.

Open the list, and you should see a default entry that comes with the system. This rule limits requests from a given IP address to a maximum of 5 requests per 60 seconds.

If you don't see any default entries, click on the second "here" link to create a new Rate Limit. In the window that appears, create the rule shown below, save it, and then return to the URL Map.

Select the "add" link to add this rule to the default URL Map. The rule will then be displayed without the pulldown list:

Save your changes, and then publish.

After propagation, we can test it:

while true; 
do 
    curl -s -o /dev/null -w "%{http_code}" http://curie.demo:30081/with/header -H "foo: test";
    printf ", "; 
    sleep 1s; 
done

Wait briefly, and you should see that on the 6th request, the response changes from 403 (which means the ACL Policy is blocking the request) to 503 (which means the Rate Limit is blocking it).

403, 403, 403, 403, 403, 503, 503, 503, 503, 503, 503, 503, 503...

In Curiefense, Rate Limit Rules are enforced before the ACL Policies (as explained further here: Multi-Stage Traffic Filtering). Thus, when a request would violate both, it is the Rate Limit Rule that blocks it.

Add Multiple Rate Limits

Now that we're somewhat familiar with the system, let's set up multi-layered rate limiting to protect against a variety of attacks.

Rate limits in Curiefense are reusable 'stand-alone' rules that can be attached to different paths in URL Maps.

Previously, we used the default Rate Limit that comes with Curiefense, and applied it to the entire domain. Now we will create some specific Rate Limits for the login process of an API, and attach them to the relevant endpoints.

Creating a URL Map for the API

Create a new URL Map by duplicating the default one:

Set its name (the unlabeled field at the top) to API. Set Matching Names to api.curie.demo.

Adding Profiles

As did the original URL Map, this new one contains a default profile. As you might expect, this will apply to every path within the scope set by the Matching Names entry. (In this tutorial, the Matching Names is a specific subdomain. In production use, this might be a regex describing a range of domains, subdomains, or URLs.)

Our hypothetical API has two versions: /api/v1/ and /api/v2/. Let's say that the default profile is good enough for each of them. (If it weren't, we could set up separate profiles for one or both instead.)

However, there are two endpoints where we want to set up stricter Rate Limiting: /api/v1/login and /api/v2/login. To do this, let's create a dedicated profile for the login endpoints. (We could set up a separate profile for each endpoint, but in this case, we can accomplish this task with only one.)

To add a profile to a URL Map, open an existing profile (in this case, the default) and select the Fork profile button.

Edit the Name and Match condition of the profile as follows:

Now save your changes (but you don't need to publish them yet).

At this point, we have two profiles: the default / which we left in place, and an entry for /login that controls login access to both versions of the API.

Setting Rate Limits

Both profiles currently have the default Rate Limit assigned to them.

For /api/v[1-2]/login, we will create two new rate limit rules.

Create New Rate Limits

Open the Rate Limits section of the UI by selecting Rate Limits at the top of the window (in the pulldown list that currently says URL Maps).

The default Rate Limit rule should be displayed. Add a new Rate Limit by duplicating it, using the Duplicate Document button in the top right part of the interface.

An attacker who is trying to brute-force a login will get caught by the default Rate Limit rule. To evade this, attackers will often rotate their geolocations. So, if the same "user" attempts to login from different countries within a short time, it's probably an attack. Let's set up a rate limit to prevent this.

Edit the new Rate Limit so that it limits a given userid argument to a single country within a one-hour timespan. When you are done, save it.

In production use, you can use a similar tactic and limit the number of ASNs (shown in the Curiefense interface as "Companies") per User ID. However, you'd want the rule to be less strict than the one shown here, to accommodate users who can legitimately change ASNs (for example, a user who switches from coffee shop WiFi to a cellular data connection).

Rule #2: Limit number of User IDs

Another common threat is credential stuffing, where an attacker submits valid credential sets (usually stolen from other sites) in an attempt to see which ones will work, and then take over those accounts. Let's set up a Rate Limit for this.

Create a second Rate Limit by duplicating an existing one. Edit it so that it limits each IP to submitting a maximum of two unique User IDs per minute. When you are done, save it.

Using the new Rate Limit rules

Return to the URL Maps section of the interface; the API map that you created earlier should be opened for editing (since it's the first one in the alphabetical list). Let's attach the new rules to the /api/v[1-2]/login matching path.

Expand the API Login profile, select "+" to add a rule, select one of the two new rules you created, then select "add". Repeat for the second rule. When you're finished, you should see something similar to this:

Save your changes, publish them, and wait 15-20 seconds for propagation.

Testing the new Rate Limits

Next, let's try to brute-force the login endpoint, using the same user id and IP, but with a new password each time:

while true; 
do 
  curl -s -o /dev/null -w "%{http_code}" \
     http://api.curie.demo:30081/api/v1/login \
     --data "userid=me@gmail.com&password=$RANDOM" ; 
  printf ", "; 
  sleep 1s; 
done

The first five attempts will succeed, and then the default Rate Limit will be triggered.

200, 200, 200, 200, 200, 503, 503, 503, 503, ...

After waiting one minute to let the Rate Limit reset, we'll test the Credential Stuffing limit by having curl send a unique User ID each time:

while true; 
do 
  curl -s -o /dev/null -w "%{http_code}" \
     http://api.curie.demo:30081/api/v1/login \
     --data "userid=$RANDOM@gmail.com&password=$RANDOM" ; 
  printf ", "; 
  sleep 1s; 
done

This time, only two login attempts are allowed:

200, 200, 503, 503, 503, 503, ...

As before, the details of these events are available in the traffic log.

Advanced Rate Limiting

The demonstration above is only the beginning of what can be done with Rate Limiting. You can configure Curiefense to limit complex combinations of conditions and events, customize its reactions (including blocking, redirecting, monitoring, responding with custom codes, adding headers to requests for backend evaluation), and more.

Here's a common example. Rate Limits like the ones demonstrated above will block a requestor who exceeds a defined limit within the given timespan. However, once the time period resets, the requestor would be able to try again, and could repeat this cycle as often as desired. To prevent this, you can configure Curiefense to ban a requestor (and block all of their requests) when multiple rate limits are violated.

A full explanation of Curiefense's Rate Limits and their capabilities is available here.

Grafana Dashboards

The docker-compose deployment process will create Grafana visualizations for Curiefense's traffic data.

Curiefense comes with two dashboards out of the box: Traffic Overview and Top Activities. They are available at http://curie.demo:30300/.

To login, the default username and password are admin and admin. Then you should see this:

On the Dashboards menu, select "Manage":

This will reveal the Provisioned folder:

Opening the Provisioned folder will reveal the default Curiefense dashboards:

Here's one of them:

Traffic Overview Dashboard

Onwards

You now have a working Curiefense installation to experiment with. Its capabilities go far beyond those demonstrated in this tutorial; browsing through the rest of this Manual (especially the Policies & Rules section) will give you some ideas to test, so you can see what the platform can do.

Note that this tutorial was a Quick Start guide, and therefore, it used many default options for the deployment. You might want to change some of them; for example, you might want Curiefense to use TLS for its UI server and for communicating with the backend.

Modifying the deployment is straightforward. For Docker Compose, just go through the procedures described here and then re-run docker-compose up. For Helm, go through the procedures here.

Deployment in Depth

Curiefense has multiple deployment options:

Docker Compose
Istio via Helm
NGINX

Docker Compose

Introduction

This page describes the tasks necessary to deploy Curiefense using Docker Compose. The tasks are described sequentially below:

Clone the Repository
TLS Setup
Set Deployment Variables
Deploy Curiefense
Test the Deployment
Clean Up

During this process, you might find it helpful to read the descriptions (which include the purpose, secrets, and network/port details) of the services and their containers: Services and Container Images

If during this process you need to rebuild an image, see the instructions here: Building/Rebuilding an Image.

Clone the Repository

Clone the repository, if you have not already done so:

git clone https://github.com/curiefense/curiefense.git

This documentation assumes it has been cloned to ~/curiefense.

TLS Setup

A Docker Compose deployment can use TLS for communication with Curiefense's UI server and also for the protected service, but this is optional. (If you do not choose to set it up, HTTPS will be disabled.)

If you do not want Curiefense to use TLS, then skip this step and proceed to the next section. Otherwise, generate the certificate(s) and key(s) now.

To enable TLS for the protected site/application, go to curiefense/deploy/compose/curiesecrets/curieproxy_ssl/ and do the following:

Edit site.crt and add the certificate.
Edit site.key and add the key.

To enable TLS for the nginx server that is used by uiserver, go to curiefense/deploy/compose/curiesecrets/uiserver_ssl/and do the following:

Edit ui.crt and add the certificate.
Edit ui.key and add the key.

Set Deployment Variables

Docker Compose deployments can be configured in two ways:

By setting values for variables in deploy/compose/.env
Or by setting OS environment variables (which will override any variables set in.env)

These variables are described below.

CURIE_BUCKET_LINK

Curiefense uses the storage defined here for synchronizing configuration changes between confserver and the Curiefense sidecars.

By default, this points to the local_bucket Docker volume:

$ grep CURIE_BUCKET_LINK .env
CURIE_BUCKET_LINK=file:///bucket/prod/manifest.json

For multi-node deployments, or to use S3 for a single node, replace this value with the URL of an S3 bucket:

CURIE_BUCKET_LINK=s3:///BUCKETNAME/prod/manifest.json

In that case, you will need to supply AWS credentials in deploy/compose/curiesecrets/s3cfg, following this template:

[default]
access_key = AAAAAAAAAAAAAAAAAAAA
secret_key = AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

TARGET_ADDRESS and TARGET_PORT

The address of the destination service for which Curiefense acts as a reverse proxy. By default, this points to the echo container, which simply echoes the HTTP requests it receives.

DOCKER_TAG

Defaults to main (the latest stable image, automatically built from the main branch). To run a version that matches the contents of your working directory, use the following command:

DOCKER_TAG="$(git describe --tag --long --dirty)-$(git rev-parse --short=12 HEAD:curiefense)"

Deploy Curiefense

Once the tasks above are completed, run these commands:

cd curiefense/deploy/compose/
docker-compose up

Test the Deployment

After deployment, the Echo service should be running and protected behind Curiefense. You can test the success of the deployment by querying it:

$ curl http://localhost:30081/
Request served by echo

HTTP/1.1 GET /

Host: localhost:30081
X-Envoy-Internal: true
X-Request-Id: 57dd8be5-6040-491a-903e-7ef3734ab9db
X-Envoy-Expected-Rq-Timeout-Ms: 15000
User-Agent: curl/7.74.0
Accept: */*
X-Forwarded-For: 172.18.0.1
X-Forwarded-Proto: http

Also verify the following:

The UIServer is now available at http://localhost:30080
Grafana is now available at http://localhost:30300
The confserver is now available at http://localhost:30000/api/v1/

Clean Up

To stop all containers and remove any persistent data stored in volumes, run the following commands:

docker-compose rm -f && docker volume prune -f

Istio via Helm

Introduction

The instructions below show how to install Curiefense on a Kubernetes cluster, embedded in an Istio service mesh.

The following tasks, each described below in sequence, should be performed:

Clone the Helm Repository
Setup Synchronization
Create a Kubernetes Cluster Running Helm
Reset State
Create Namespaces
Setup Secrets
Setup TLS
Deploy Istio and Curiefense Images
Deploy the (Sample) App
Expose Curiefense Services Using NodePorts
Access Curiefense Services

At the bottom of this page is a Reference section describing the charts and configuration variables.

Clone the Helm Repository

Clone the repository, if you have not already done so:

git clone https://github.com/curiefense/curiefense-helm.git

This documentation assumes it has been cloned to ~/curiefense-helm.

Setup Synchronization

An AWS S3 bucket must be available to synchronize configurations between the confserver and the Curiefense Istio sidecars. The following Curiefense variables must be set:

In ~/curiefense-helm/istio-helm/charts/gateways/istio-ingress/values.yaml:
- Setcurieconf_manifest_url to the bucket URL.
In ~/curiefense-helm/curiefense-helm/curiefense/values.yaml:
- Set curieconf_manifest_url to the bucket URL.

Create a Kubernetes Cluster

Access to a Kubernetes cluster is required. Dynamic provisioning of persistent volumes must be supported. To set a StorageClass other than the default, change or override variable storage_class_name in ~/curiefense-helm/curiefense-helm/curiefense/values.yaml.

Below are instructions for several ways to achieve this:

Using minikube, Kubernetes 1.20.2 (dynamic provisioning is enabled by default)
Using Google GKE, Kubernetes 1.16.13 (RBAC and dynamic provisioning are enabled by default)
Using Amazon EKS, Kubernetes 1.18 (RBAC and dynamic provisioning are enabled by default)

You will need to install the following clients:

Install kubectl (https://kubernetes.io/docs/tasks/tools/install-kubectl/) -- use the same version as your cluster.
Install Helm v3 (https://helm.sh/docs/intro/install/)

Option 1: Using minikube

This section describes the install for a single-node test setup (which is generally not useful for production).

Install minikube

Starting from a fresh ubuntu 21.04 VM:

Install docker (https://docs.docker.com/engine/install/ubuntu/), and allow your user to interact with docker with sudo usermod -aG docker $USER && newgrp docker
Install minikube (https://minikube.sigs.k8s.io/docs/start/)

minikube start --kubernetes-version=v1.20.2 --driver=docker --memory='8g' --cpus 6
minikube addons enable ingress

Start a screen or tmux, and keep the following command running:

minikube tunnel

Option 2: Using Google GKE

Create a cluster

gcloud container clusters create curiefense-gks --num-nodes=1 --machine-type=n1-standard-4
gcloud container clusters get-credentials curiefense-gks

Option 3: Using Amazon EKS

Create a cluster

eksctl create cluster --name curiefense-eks-2 --version 1.18 --nodes 1 --nodes-max 1 --managed --region us-east-2 --node-type m5.xlarge

Reset State

If you have a clean machine where Curiefense has never been installed, skip this step and go to the next.

Otherwise, run these commands:

helm delete curiefense
helm delete -n curiefense curiefense
helm delete -n istio-system istio-ingress
helm delete -n istio-system istiod
helm delete -n istio-system istio-base

Ensure that helm ls -a --all-namespaces outputs nothing.

Create Namespaces

Run the following commands:

kubectl create namespace curiefense
kubectl create namespace istio-system

Setup Secrets

AWS credentials

Encode the AWS S3 credentials that have r/w access to the S3 bucket. This yields a base64 string:

cat << EOF | base64 -w0
[default]
access_key = xxxxxxxxxxxxxxxxxxxx
secret_key = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
EOF

Create a local file called s3cfg.yaml, with the contents below, replacing both occurrences of BASE64_S3CFG with the previously obtained base64 string:

---
apiVersion: v1
kind: Secret
data:
  s3cfg: "BASE64_S3CFG"
metadata:
  namespace: curiefense
  labels:
    app.kubernetes.io/name: s3cfg
  name: s3cfg
type: Opaque
---
apiVersion: v1
kind: Secret
data:
  s3cfg: "BASE64_S3CFG"
metadata:
  namespace: istio-system
  labels:
    app.kubernetes.io/name: s3cfg
  name: s3cfg
type: Opaque

Deploy this secret to the cluster:

kubectl apply -f s3cfg.yaml

Alternative to AWS: GS credentials

Create a bucket, and a service account that has read/write access to the bucket. Obtain a private key for this account, which should look like this:

{
  "type": "service_account",
  "project_id": "PROJECT",
  "private_key_id": "1234abcd1234abcd1234abcd1234abcd1234abcd",
  "private_key": "-----BEGIN PRIVATE KEY-----\nMIIE.....ABCD=\n-----END PRIVATE KEY-----\n",
  "client_email": "....@PROJECT.iam.gserviceaccount.com",
  "client_id": "123412341234123412341",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/....%40PROJECT.iam.gserviceaccount.com"
}

Create a local file called gs.yaml, with the contents below, replacing both occurrences of BASE64_GS_PRIVATE_KEY with the previously obtained base64 string:

---
apiVersion: v1
kind: Secret
data:
  gs.json: "BASE64_GS_PRIVATE_KEY"
metadata:
  labels:
    app.kubernetes.io/name: gs
  name: gs
  namespace: curiefense
type: Opaque
---
apiVersion: v1
kind: Secret
data:
  gs.json: "BASE64_GS_PRIVATE_KEY"
metadata:
  labels:
    app.kubernetes.io/name: gs
  name: gs
  namespace: istio-system
type: Opaque

Deploy this secret to the cluster:

kubectl apply -f gs.yaml

Set the curieconf_manifest_url variables in curiefense-helm/curiefense/values.yaml and istio-helm/charts/gateways/istio-ingress/values.yaml to the following URL: gs://BUCKET_NAME/prod/manifest.json (replace BUCKET_NAME with the actual name of the bucket).

Also set the curiefense_bucket_type variables in the same values.yaml files to gs.

Setup TLS for the UI server

Using TLS is optional. Follow these steps if only if you want to use TLS for communicating with the UI server, and you do not rely on istio to manage TLS.

The UIServer can be made to be reachable over HTTPS. To do that, two secrets have to be created to hold the TLS certificate and TLS key.

Create a local file called uiserver-tls.yaml, replacing TLS_CERT_BASE64 with the base64-encoded PEM X509 TLS certificate, and TLS_KEY_BASE64 with the base64-encoded TLS key.

---
apiVersion: v1
data:
  uisslcrt: TLS_CERT_BASE64
kind: Secret
metadata:
  labels:
    app.kubernetes.io/name: uisslcrt
  name: uisslcrt
  namespace: curiefense
type: Opaque
---
apiVersion: v1
data:
  uisslkey: TLS_KEY_BASE64
kind: Secret
metadata:
  labels:
    app.kubernetes.io/name: uisslkey
  name: uisslkey
  namespace: curiefense
type: Opaque

Deploy this secret to the cluster:

kubectl apply -f uiserver-tls.yaml

An example file with self-signed certificates is provided at ~/curiefense-helm/curiefense-helm/example-uiserver-tls.yaml.

Deploy Istio and Curiefense Images

Deploy the Istio service mesh:

cd ~/curiefense-helm/istio-helm 
DOCKER_TAG=main ./deploy.sh

And then the Curiefense components:

cd ~/curiefense-helm/curiefense-helm
DOCKER_TAG=main ./deploy.sh

Deploy the (Sample) App

The application to be protected by Curiefense should now be deployed. These instructions are for the sample application bookinfo which is deployed in the default kubernetes namespace. Installation instructions are summarized below. More detailed instruction are available on the istio website.

Enable Istio injection

Add the istio-injection=enabled label that will make Istio automatically inject necessary sidecars to applications that are deployed in the default namespace.

kubectl label namespace default istio-injection=enabled

Install the application

cd ~
wget 'https://github.com/istio/istio/releases/download/1.9.3/istio-1.9.3-linux-amd64.tar.gz'
tar -xf istio-1.9.3-linux-amd64.tar.gz
cd ~/istio-1.9.3/
kubectl apply -f samples/bookinfo/platform/kube/bookinfo.yaml
kubectl apply -f samples/bookinfo/networking/bookinfo-gateway.yaml

Test bookinfo

Check that bookinfo Pods are running (wait a bit if they are not):

kubectl get pod -l app=ratings

Sample output example:

NAME                         READY   STATUS    RESTARTS   AGE
ratings-v1-f745cf57b-cjg69   2/2     Running   0          79s

Check that the application is working by querying its API directly without going through the Istio service mesh:

kubectl exec "$(kubectl get pod -l app=ratings -o jsonpath='{.items[0].metadata.name}')" -c ratings -- curl -sS productpage:9080/productpage | grep -o "<title>.*</title>"

Expected output:

<title>Simple Bookstore App</title>

Test access to bookinfo through Istio

Set the GATEWAY_URL variable by following instructions on the Istio website.

Alternatively, with minikube, this command can be used instead:

export GATEWAY_URL=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}'):80

Check that bookinfo is reachable through Istio:

curl -sS http://$GATEWAY_URL/productpage | grep -o "<title>.*</title>"

Expected output:

<title>Simple Bookstore App</title>

If this error occurs: Could not resolve host: a6fdxxxxxxxxxxxxxxxxxxxxxxxxxxxx-xxxxxxxx.us-west-2.elb.amazonaws.com ...the ELB is not ready yet. Wait and retry until it becomes available (typically a few minutes).

Check that logs stored in the Elasticsearch cluster

Run this query to access the protected website, bookinfo, and thus generate an access log entry:

curl http://$GATEWAY_URL/TEST_STRING

Run this to ensure that the logs have been recorded:

kubectl exec -ti -n curiefense elasticsearch-0 -- curl http://127.0.0.1:9200/_search -H "Content-Type: application/json" -d '{"query": {"bool": {"must":{"match":{"request.attributes.uri": "/TEST_STRING"}}}}}'|grep -Eo '"uri":"/TEST_STRING"'

Expected output:.

"uri":"/TEST_STRING"

Expose Curiefense Services Using NodePorts

Run the following commands to expose Curiefense services through NodePorts. Warning: if the machine has a public IP, the services will be exposed on the Internet.

Start with this command:

kubectl apply -f ~/curiefense-helm/curiefense-helm/expose-services.yaml

The following command can be used to determine the IP address of your cluster nodes on which services will be exposed:

kubectl get nodes -o wide

For minikube only:

If you are using minikube, also run the following commands on the host in order to expose services on the Internet (ex. if you are running this on a cloud VM):

sudo iptables -t nat -A PREROUTING -p tcp --match multiport --dports 30000,30080,30300,30443 -j DNAT --to $(minikube ip)
sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j DNAT --to $(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
sudo iptables -I FORWARD -p tcp --match multiport --dports 80,30000,30080,30300,30443,30444 -j  ACCEPT

For Amazon EKS only:

If you are using Amazon EKS, you will also need to allow inbound connections for port range 30000-30500 from your IP. Go to the EC2 page in the AWS console, select the EC2 instance for the cluster (named curiefense-eks-...-Node), select the "Security" pane, select the security group (named eks-cluster-sg-curiefense-eks-[0-9]+), then add the incoming rule.

Access Curiefense Services

The UIServer is now available on port 30080 over HTTP, and on port 30443 over HTTPS.

Grafana is now available on port 30300 over HTTP.

For the bookinfo sample app, the Book Review product page is now available on port 80 over HTTP, and on port 30444 over HTTPS. Try reaching http://IP/productpage.

The confserver is now available on port 30000 over HTTP: try reaching http://IP:30000/api/v1/.

For a full list of ports used by Curiefense containers, see the Reference page on services and containers.

Reference: Description of Helm Charts

Curiefense charts

Helm charts are divided as follows:

curiefense-admin - confserver, and UIServer.
curiefense-dashboards - Grafana and Prometheus.
curiefense-log - elasticsearch, filebeat, fluentd, kibana, logstash.
curiefense-proxy - curielogger and redis.

Chart configuration variables

Configuration variables in ~/curiefense-helm/curiefense-helm/curiefense/values.yaml can be modified or overridden to fit your deployment needs:

Variables in the images section define the Docker image names for each component. Override this if you want to host images on your own private registry.
storage_class_name is the StorageClass that is used for dynamic provisioning of Persistent Volumes. It defaults to null (default storage class, which works by default on EKS, GKE and minikube).
..._storage_size variables define the size of persistent volumes. The defaults are fine for a test or small-scale deployment.
curieconf_manifest_url is the URL of the AWS S3 or Google Cloud Storage bucket that is used to synchronize configurations between the confserver and the Curiefense Istio sidecars.
docker_tag defines the image tag versions that should be used. deploy.sh will override this to deploy a version that matches the current working directory, unless the DOCKER_TAG environment variable is set.

Istio chart

Components added or modified by Curiefense are defined in ~/curiefense-helm/istio-helm/charts/gateways/istio-ingress/. Compared to the upstream Istio Kubernetes distribution, we add or change the following Pods:

An initContainer called curiesync-initialpull has been added. It synchronizes configuration before running Envoy.
A container called curiesync has been added. It periodically fetches the configuration that should be applied from an S3 or GS bucket (configurable with the curieconf_manifest_url variable), and makes it available to Envoy. This configuration is used by the LUA code that inspects traffic.
The container called istio-proxy now uses our custom Docker image, embedding our HTTP Filter, written in Lua.
An EnvoyFilter has been added. It forwards access logs to curielogger (see curiefense_access_logs_filter.yaml).
An EnvoyFilter has been added. It runs Curiefense's Lua code to inspect incoming traffic on the Ingress Gateways (see curiefense_lua_filter.yaml).

Chart configuration variables

Configuration variables in ~/curiefense-helm/istio-helm/charts/gateways/istio-ingress/values.yaml can be modified or overridden to fit your deployment needs:

gw_image defines the name of the image that contains our filtering code and modified Envoy binary.
curiesync_image defines the name of the image that contains scripts that synchronize local Envoy configuration with the AWS S3 bucket defined in curieconf_manifest_url.
curieconf_manifest_url is the URL of the AWS S3 bucket that is used to synchronize configurations between the confserver and the Curiefense Istio sidecars.
curiefense_namespace should contain the name of the namespace where Curiefense components defined in ~/curiefense-helm/curiefense-helm/ are running.
redis_host defines the hostname of the redis server that will be used by curieproxy. Defaults to the provided redis StatefulSet. Override this to replace the redis instance with one you supply.
initial_curieconf_pull defines whether a configuration should be pulled from the AWS S3 bucket before running Envoy (true), or if traffic should be allowed to flow with a default configuration until the next synchronization (typically every 10s).

NGINX

This document describes how Curiefense can be integrated into an existing NGINX-based reverse proxy.

This guide describes a basic integration, and it cannot cover the wide variety of possible use cases and configurations. For specific questions about this, or other Curiefense-related topics, feel free to join our Slack at https://join.slack.com/t/curiefense/shared_invite/zt-nc8lyrjo-JJoY2mwrqNOfkmoA6ycTHg.

Scope

This page describes the installation of the Curiefense filtering component for an environment where NGINX is running in a container.

The other components of Curiefense will need to be installed separately, according to the specific instructions for each situation (e.g., Docker and Istio). This can be done either before or after completing the instructions below.

Dependencies

If OpenResty is not installed yet, please follow the instructions on the OpenResty website.

You will also need the Hyperscan library, version 4 or 5. For example, on Ubuntu 20.04:

sudo apt install libhyperscan5

Curiefense installation

cd /tmp
git clone https://github.com/curiefense/curiefense
cp -r /tmp/curiefense/curiefense/curieproxy/lua /
cp /tmp/curiefense/curiefense/curieproxy/lua/shared-objects/grasshopper.so /usr/local/openresty/luajit/lib/lua/5.1/

Next, build the Curiefense shared object. This needs to be done on a Linux system that runs the same major libc and libhyperscan versions as your NGINX server.

On the build machine, first install the Rust compiler.

Then run the following:

cd /tmp/curiefense/curiefense/curieproxy/rust
cargo build --release
mv target/release/libcuriefense_lua.so target/release/curiefense.so

Move the new curiefense.so file on the build machine to this location on the proxy machine: /usr/local/openresty/luajit/lib/lua/5.1/curiefense.so

Configuration setup

mkdir -p /config/current
cp -r /tmp/curiefense/curiefense/images/confserver/bootstrap/confdb-initial-data/master/config/ /config/current/

OpenResty configuration

In the http block of the configuration, the following directives must be set:

# set the package path for the Curiefense logic
lua_package_path '/lua/?.lua;;';

# access log is now completely handled by Curiefense, stored in the $request_map variable
log_format curiefenselog escape=none '$request_map';

# Curiefense specific variables.
# Make sure request_map exists
map $status $request_map { default '-'; }

For each server block that must be protected with Curiefense:

server {
    # original configuration directives
    (...)

    
    # Logging: choose one option.
 
    # -- If curielogger is used:
    access_log syslog:server=unix:/dev/log curiefenselog;
    
    # -- If curielogger is not used:
    access_log logs/access.log curiefenselog;


    location / {
        # add these in each location block:
        access_by_lua_block {
            local session = require "lua.session_nginx"
            session.inspect(ngx)
        }
        log_by_lua_block {
            local session = require "lua.session_nginx"
            session.log(ngx)
        }

        # original location directives
        (...)
    }
}

Testing the installation

The default configuration does not block any requests, so the following steps should be performed to ensure proper integration:

Traffic should be served as usual.
No errors appear in the error logs.
JSON data appears in the access logs.

Settings

Policies & Rules

Curiefense maintains its security parameters in Documents, which contain Entries. (Read more about Curiefense's data structures.)

The Policies & Rules section is where Documents and their Entries are created, defined, and administered via the UI.

Documents and traffic flow

Curiefense processes incoming requests according to this traffic flow:

Each incoming request is inspected, and tags are assigned to it. For example, if the request's IP was found on the Spamhaus DROP list, it could be assigned a "spamhaus" tag. Some tags are generated automatically, while others are user-defined. (Read more about tags.)
Next, Rate Limits are enforced.
Then, Session Flow Control is enforced.
Next, Curiefense determines the security ruleset(s) that have been assigned to the request's target URL, and which match the tags. For example, there might be a ruleset defined for the "spamhaus" tag, or for the "devops" tag.
Curiefense then enforces the ruleset(s), and takes the defined Action(s). For example, "block requests from Spamhaus-listed IPs", or "bypass devops requests from further filtering."

This process is based on the Documents as follows:

Tag Rules is the Document which defines tags for external lists and custom lists.
Rate Limits and Flow Control parameters define session-based policies.
ACL Policies and WAF Policies define the actions to take when specific tags and/or other criteria are observed. URL Maps assign these actions to internal URLs.

Policies & Rules interface

This page is divided into three vertical sections. From top to bottom, they are:

Administration
Entry editing
Versioning

Each is discussed below.

After editing anything on this page, you must save your changes (with the Save button on the upper right) and then publish them.

Entry Administration

The top section contains a toolbar with input controls. On the left, there are these:

Configuration pulldown: Selects the branch/configuration for editing.
Document pulldown: Selects the Document to display for editing.
Download button: Downloads the currently displayed parameters.

On the right are these:

Entry pulldown: Selects the Entry (the ruleset) that is being displayed for editing.
Duplicate button: Makes a copy of the currently displayed Entry.
Add button: Creates a new Entry of the currently selected type.
Save button: Saves all changes that were made since the last Save action.
Delete button. Deletes the currently selected Entry.

Entry Editing

The UI in this section will vary, depending on the type of Document being edited. Each is discussed in more depth here:

ACL Policies
Flow Control
Tag Rules
Rate Limits
URL Maps
WAF Policies
WAF Rules

At the bottom of this section, the URL of the current entry is shown in gray. It reflects the structure of the data.

For example, /conf/api/v1/configs/devops/d/urlmaps/e/__default__ shows that:

The current Configuration (or branch) is devops.
The current Document is urlmaps.
The current Entry is __default__.

Document Versioning

The bottom section of this page shows a history of versions for this Document.

To revert this Document to a previous version, hover the cursor over the end of its listing. As shown above, a button will appear; selecting it will restore that version.

Reversions are also available in the API and in the Publish Configuration section of the UI.

ACL Policies

An ACL (Access Control List) Policy is a set of sequential criteria by which a request is evaluated. The input controls at the top of this page are described here: Policies & Rules Entry Administration. Specific editing of an ACL Policy is described below.

This page defines ACL Policies. The Policies are assigned to URLs within the web application via URL Maps.

Out of the box, Curiefense includes a default ACL Policy. It can (and usually should) be edited, but it cannot be deleted. It is used for all URLs where no other Policy has been assigned.

The top part of the interface shows the ACL Policy that is currently being edited. Each Policy consists of one or more tags. Each tag is listed under the action which will be triggered when a request matches that particular tag.

In the interface, the tags are listed below their associated actions. Evaluation is performed from left to right:

The incoming request is evaluated to see if it matches any of the tags in the far left column.
If a match is found, that column's action is performed, and no further evaluation is done.
If a match is not found, then the next column to the right is evaluated. A match will trigger that column's action; otherwise the next column is evaluated.
This process continues until an action is taken or all columns have been evaluated.
If a request does not match any tags in this Policy, it is passed to the WAF for further inspection.

Action columns

An ACL Policy that blocks requests via the Enforce Deny Action will only apply to the paths/URLs in the URL Map where this ACL Policy has been assigned. If you want to block requests globally for specific Tags, this can be done by defining the appropriate Action within the relevant Tag Rules.

Be very cautious when adding the "all" tag to an action column. In most situations, that action will be taken for all requests for which a match was not found in any columns to the left.

The most common use is the "Deny Bot" column, as noted above. This means that all requestors (except for those previously Allowed or Bypassed) will be challenged and verified to be humans.

Another possible use is to place "all" in the Deny column, to create a positive security model. (This will block all requests which did not explicitly meet any of the conditions to be Bypassed or Allowed.) But even this usage can be dangerous. Unless all the conditions for Bypassing or Allowing requests are fully and correctly defined, this can result in False Positive errors, and some legitimate requests will be blocked and filtered.

In other columns, "all" can have serious consequences. For example, placing it in the Enforce Deny column will block all incoming traffic to which this ACL Policy is applied. On the other hand, placing it in the Bypass column will allow all incoming traffic (and exempt it from being scrubbed by the WAF!), except for those requests which matched a tag in the Enforce Deny column.

Summary: before using the "all" tag in an ACL Policy, carefully consider its ramifications.

Tag Rules

The input controls at the top of this page are described here: Policies & Rules Entry Administration. Specific editing of a Tag Rules List is described below.

Overview

Early in the traffic evaluation process, Curiefense assigns Tags to an incoming request. Subsequently, the Tags can be used to make decisions about how the request is processed. After processing, a request's Tags remain associated with it, and they are available for display in the Access Log.

This page allows you to administer Tag Rule Lists, which are combinations of user-defined Tags and the criteria for assigning them to requests.

Each Tag Rule List consists of:

Match conditions: Definitions of possible characteristics that a request can match (e.g., a list of IP addresses that it might originate from), plus one or more logical operators to use when evaluating the match.
One or more Tags to assign when a match occurs.
An Action to apply to the request.

If the Action defined here is Tag Only or Header, then the request will undergo further processing after the Tags are assigned. The post-processing actions that are performed as a result of the Tag assignments are administered in ACL Policies and Rate Limits.

For each incoming request, Curiefense will evaluate all active Tag Rule Lists. A single request will receive Tags from all Lists which match it.

Metadata

Action

The default Action for a Tag Rules List is Tag Only. This means that the specified tag(s) will be applied, and the traffic evaluation process will continue.

A related Action is Header. The specified tag(s) will be applied, and the specified header will be added. Then the traffic evaluation process will continue.

Sometimes the Match Conditions describe a request which should be rejected immediately; if so, there is no need for the complete evaluation process to occur. In this situation, one of the other Actions can be set here, and it will be applied immediately whenever a request meets the Match Conditions.

Note that the Actions defined here are global for their Tags. To define tag-related actions for a set of paths/URLs, set up the Actions within an ACL Policy and then assign the ACL Policy to the paths in a URL Map.

Source

Tag Rule Lists can be either Internet-sourced or self-managed.

Internet-sourced Lists are based upon online sources (e.g., Spamhaus DROP lists). To use an Internet-sourced list, enter its URL into the Source field. For example, to create a list based on the Spamhaus ASN DROP list, you would enter https://www.spamhaus.org/drop/asndrop.txt and select the update now control that will appear. Curiefense will then populate the list automatically.
Self-managed Lists are created manually. The Source field entry will be self-managed.

Note that Internet-sourced lists are not editable within the interface, because they are obtained automatically; Curiefense updates them every 24 hours (or you can force an immediate update by selecting update now). Typically, these will be Single-Section Lists.

Specifying Match Conditions

Match conditions are lists of criteria and the logical operators (AND or OR) to use when evaluating them.

Match conditions are combined into sections. An additional logical operator defines the relationship between each section.

Single-Section Lists

Many Tag Rules Lists will have only section. Here's an example.

In the example, the Sections relation toggle is set to AND. For a single-section list, this toggle is ignored.

Within each section, there is an additional logical operator, shown at the beginning of each list entry after the first. This is used when evaluating the criteria within the list:

OR means the Tags will be attached to, and the Action taken for, each request that matches any of the entries in the list.
AND means the Tags will only be attached to, and the Action taken for, requests that match all of the entries in the list.

This operator can be toggled by selecting it.

Adding a Match Condition

To add a match condition, select the add control ("+") at the bottom of the list. The following dialog will appear.

For most of the categories (IP Address, Method, etc.), the dialog will appear as it is above. Individual entries can be added by entering their value into the first text box, and an annotation (a label for display within the Curiefense interface) in the second text box, then selecting the Add control.

Multiple entries can be made at once, with each entry on a separate line. Each line contains the value, plus a pound sign (#) followed by the annotation. Example:

Selecting the Add control would then produce the following:

For some categories, one entry can be made at a time, with no annotations.

Once created, these entries cannot be edited. If one needs to be modified, remove it and re-create it.

Here are some sample entries for the various categories. Notice that boolean operators are available.

Match criteria are case-insensitive.

Multiple-Section Lists

Match conditions can be combined into multiple sections. To add a section to an existing Tag Rules List, select Create new section.

Here's an example:

Each additional section works the same as the initial section. However, a multiple-section list also includes a logical operator for the relationship between the individual sections. It is shown between the sections, and can be changed by toggling the Sections Relation control on the left.

In the example above, this is AND. Therefore, a request will receive the tag of internal if it matches any of the IPs in the first section, AND it also contains one of the HTTP methods specified in the second section.

Rate Limits

Rate Limits are rules which define the number of requests with certain characteristics that are allowed within defined time frames. When a request is received that exceeds a Rate Limit, a specified action is performed.

The input controls at the top of this page are described here: Policies & Rules Entry Administration. Specific editing of a Rate Limit is described below.

Rate Limit concepts

A Rate Limit defines the number of times that requests can match certain conditions within a certain time frame. Once that limit has been reached, subsequent requests matching those conditions within the same time frame will trigger an action.

The matching conditions are specified with the parameters in the "Count by" section and the optional Event. See Matching Conditions below.

After a Rate Limit is defined on this page, it will not be active until it is assigned to one or more URLs. This is done in the URL Maps Document.

By default, a Rate Limit will be enforced for all requests for the URL(s) to which it is assigned.

Enforcement can be further limited to a subset of these requests: see Limiting the scope of a Rate Limit below.

Meta Parameters

Rate Limit

Matching Conditions (the "Count by" input controls)

A condition consists of a field and a value. Within a Rate Limit, they play a role like this:

"More than <THRESHOLD> requests with the same <CONDITION-VALUE> <CONDITION-FIELD> sent to <ASSIGNED-LOCATION> within <TTL-PERIOD> will cause <ACTION>."

Example: "More than three requests with the same username argument sent to the login form within one hour will cause the requestor to be banned for six hours."

Note that the <ASSIGNED-LOCATION> mentioned above is included for illustration purposes only. This value is not part of a Rate Limit definition—it is defined in a URL Map.

Condition Definition

A condition can be built upon any one of these four categories:

Multiple Conditions

Multiple conditions can be defined within the same Rate Limit. To create a new condition and open it for editing, select "New entry" below the list of conditions.

If multiple conditions are defined, they are evaluated by combining them together with a logical AND. In other words, the cumulative count toward the Threshold will be incremented whenever a request is seen that matches all of the conditions simultaneously. Different combinations of conditions will have separate Threshold counts maintained for them.

Example. A Rule contains two conditions: "Attribute / IP Address" and "Argument / Username". When the first request is received, an internal counter is created (set to a value of one) for this unique combination of IP Address and Username. A second request is then received, originating from the same IP and for the same Username; this causes the internal counter to be incremented up to two. A third request is then received from the same IP but with a different Username; this causes a new internal counter to be created (and set to a value of one) for this combination.

The "Event" option: Changing the meaning of the Rate Limit

Below the list of condition(s), there is another condition named "Event."

By default, this is set to "HTTP request,", which simply means to increment a counter each time a request is received that matches the conditions.

However, if the Event condition is changed to a different value, then the following applies.

In the discussion below, "Count Condition" will refer to the condition (or combination of conditions) defined by the Count by input controls.

"Event Condition" will refer to the optional, additional condition defined by the Event input controls.

Adding an Event Condition changes the evaluation process. An Event Condition is not logically combined with the preceding Count Condition; it is always evaluated separately.

More importantly, adding an Event Condition changes the meaning of the Rate Limit.

If an Event Condition is not defined—in other words, if "HTTP request" is selected—then as discussed above, an internal counter is maintained for each Count Condition value, and incremented each time that value is encountered in a request.
If an Event Condition is defined—in other words, if something other than "HTTP request" is selected—an internal counter is maintained for each Count Condition Value, and incremented each time a new, previously unobserved Event Condition value is encountered in a request.

Therefore, if an Event Condition is defined, the Rate Limit constrains the number of allowable Event Condition values for any given Count Condition value.

So, the evaluation becomes something like this:

"More than <THRESHOLD> <EVENT-CONDITION-VALUE> <EVENT-CONDITION-FIELD>s per any one <COUNT-CONDITION-VALUE><COUNT-CONDITION-FIELD> sent to <ASSIGNED-LOCATION> within <TTL-PERIOD> will cause <ACTION>."

Note that the number of Count Condition values is not being limited here. The limit is on the number of Event Condition values that each Count Condition value is allowed.

Example: Let's say we want to allow an individual user to login from a maximum of two ASNs within one hour. (Perhaps the user is accessing our web application from a coffee shop's WiFi, and then a few minutes later, leaves the coffee shop and begins using the cell network instead.) We want to allow this possibility; however, if we receive requests from the same user originating from three or more ASNs within an hour, we want to treat this traffic more suspiciously. This is not possible merely by specifying two Count Conditions, as described earlier in the "Multiple Conditions" section. If we set up two conditions ("Argument / Username" and "Attribute / Organization") with a Threshold of 2, and assign it to our login form, then this will merely limit the number of times that the user can login from each ASN within an hour. Instead, we can set up one Count Condition ("Argument / Username") and then set up the Event Condition ("Attribute / Organization"). Now the Threshold will apply to the number of Organizations that are observed for each specific Username.

Action

When an incoming request exceeds the Threshold, the Action specified here will occur.

The Ban action

Most of the Actions listed above will not fully exclude an attacker that continues pressing the attack.

Example: Access to a login form is rate-limited to three requests per minute. An attacker tries to brute-force the login, and sends 60 requests per minute. The Rate Limit allows the first three requests, but then blocks the next 57 requests with a 503 error. However, after the minute has passed, the Rate Limit resets. The attacker is allowed another three attempts before being temporarily blocked again. This cycle can continue for as long as the attacker wishes. In effect, the Rate Limit is not preventing the attack; it is merely slowing it from 60 attempts per minute down to three attempts per minute.

The Ban action can be used to block (or take some other Action in response to) a Rate Limit violator for an extended period of time.

Example: As described above, a Rate Limit is created to allow three requests per minute, with an Action of 503 Service Unavailable. However, an additional Rate Limit rule is also defined: nine requests per three minutes, with an Action of Ban. The Ban has an Action of 503 Service Unavailable, and a duration of one hour. URL Maps allow for multiple Rate Limits to be assigned to a single URL. Thus, both of the above rules can be assigned to the login form. Now an attacker tries to brute-force the login form, sending 60 requests per minute. The first three requests are allowed. The next six requests are blocked (and a 503 error is issued) by the first Rate Limit. The tenth request triggers the second Rate Limit, and the Ban occurs. For the next hour, the attacker's requests will be blocked with a 503 error.

Second example: A hostile bot receives a bot challenge, which it fails. Curiefense will block the request. If the bot keeps re-submitting its request, it will continue to fail the challenges. However, each time the bot tries again, Curiefense has to issue a new challenge . To solve this problem, a second Rate Limit is created with a Ban action. Now a persistent bot will simply be Banned, saving Curiefense the overhead of issuing continuous challenges.

Note that when setting up a Ban, the most common choices for its Action are to deny the violator's requests (via 503 Service Unavailable, Response, or Redirect). However, you can also choose Tag Only (to observe the violator's actions during the ban period), Challenge (to verify that the violating activity is not being done by bots), or Header (to mark the requests for further scrutiny by the backend).

Limiting the scope of a Rate Limit

By default, an active Rate Limit rule will be enforced upon all incoming requests targeting URLs to which this rule has been assigned.

To change this behavior, you can add Include and/or Exclude parameters. These define the portion of the incoming traffic stream that will be evaluated for possible violation of the Rate Limit. In other words, they limit the scope of the Rate Limit's enforcement:

The Include filter will limit enforcement to requests matching its parameters. All other requests in the traffic stream will not have this Rate Limit enforced upon them.
The Exclude filter will exempt requests from enforcement that otherwise would have been subject to it.

(Internally, Curiefense evaluates Exclude parameters first, and then Include parameters.)

To add one or more filters, select New entry, define the parameters, then select the "+" button. If more than one Include filter is specified, they are combined with a logical AND.

URL Maps

The input controls at the top of this page are described here: Policies & Rules Entry Administration. Specific editing of a URL Map is described below.

Overview

This page specifies a list of URLs and the security policies assigned to them.

Every incoming HTTP/S request targets a specific URL. Curiefense finds the best match for that URL in the URL Maps, and applies the security policies defined for it.

The "best match" is determined by regex evaluation. The order in which the URLs are listed in the interface does not matter.

Components of a URL Map

A URL Map consists of:

Host definition: The (sub)domain(s) within which the Path Maps will be found.
Path Maps: one or more paths, and the security policies which will be applied to them.

The Default URL Map

Every Curiefense deployment includes a default URL Map. If a request does not match any other URL Map, the default one is applied.

To ensure that a default always exists, the Matching Name and Path Map for this URL Map are not editable.

Creating a URL Map

To add a new URL Map, use the buttons at the top of the window to duplicate an existing one or create a new one. Then fill in these fields.

When you create or revise a URL Map, each combination of Matching Name and Path Map must be unique. For this reason, when a new URL Map is created, the UI generates a unique Matching Name.

This should be changed to a correct value before the URL Map is saved.

Editing its Path Maps

A new URL Map will include a default Path Map. Clicking on it, or on the expand button at the end of its listing, will expand it for editing.

To add a new Path Map, select an existing one, expand it, and select Fork Profile at the bottom. The existing one will be cloned, and the new one will be displayed for editing.

Note that the buttons at the top of the window are for administering URL Maps (which generally correspond to domains). Administering Path Maps (for paths and URLs within the specified domain) is done in the middle of the window.

Path Map Fields

Ensure that URL Maps do not overlap. In other words, for every domain defined in a Matching Name, ensure that every possible path within it cannot match more than one Match expression. If an incoming request matches more than one Path Map definition, there is no way to predict which of them Curiefense will use when enforcing security policies.

In addition to editing the fields discussed above, the Path Mapping dialog also provides the ability to:

Activate or deactivate the WAF Policy (by toggling its Active Mode checkbox).
Activate or deactivate the ACL Policy (by toggling its Active Mode checkbox).
Assign an existing Rate Limit rule to this Path Map, via the + button or selecting the link ("To attach an existing rule, click here."), then selecting add. (The + button will only be shown if there are unassigned Rate Limit rules available.)
Create a new Rate Limit rule for this Path Map, by selecting the link ("To create a new rate-limit rule, click here.")
Remove an assigned Rate Limit Rule by selecting remove.
Create a copy of this Path Map, and open it for editing, via the Fork Profile button.

WAF Policies

The input controls at the top of this page are described here: Policies & Rules Entry Administration. Specific editing of a WAF Profile is described below.

Overview

A WAF Profile is a set of security policies that are used by the Curiefense WAF (Web Application Firewall). Every deployment includes a default WAF Profile, and additional Profiles can be created.

Every URL that Curiefense protects has a WAF Profile assigned to it in URL Maps. (If none is assigned explicitly, the default is used.) A request sent to a URL might, or might not, be filtered according to the assigned Profile.

Reasons why WAF filtering might not occur:

The request was blocked before WAF filtering would have occurred. (Before the WAF is used, several other stages of filtering occur first.)
The applicable ACL Profile resulted in an Action of Bypass, which exempts the request from WAF filtering.
The WAF is not in Active Mode for this URL.

Input Characteristics

At the top of the page, the following values are defined for incoming requests.

Content Filtering and Whitelisting

By default, an incoming request will be compared to all the WAF Rules. If any parameter (any header, cookie, or argument within it) fails this evaluation, the request will be blocked.

However, parameters can be whitelisted and exempted from this filtering. For each parameter, this can be done in two ways:

Full exemption is available by specifying a regex pattern which, if it matches the parameter's value, will exempt that parameter from WAF Rule evaluation.
Partial exemption is available by specifying a list of WAF Rules which will not be evaluated, even if the regex pattern is not matched.

Along with this content whitelisting, a "positive security" form of content filtering is also available. Curiefense can be configured to require certain content in a specified parameter, and reject requests that do not contain it.

Parameter Content Constraints

The bottom part of the UI defines Curiefense's behavior for both whitelisting and content filtering for each parameter.

In the following discussion, a constraint refers to the values in the UI input controls (Parameter, Matching Value, Restrict?, and Exclude WAF Rule) that have been specified for one parameter.

Each incoming request is processed like this:

This behavior is defined in the following fields in the UI.

Constraints are defined in the same way for Headers, Cookies, and Arguments within their respective tabs.

WAF Rules

This section displays all the WAF Rules defined within Curiefense. This has various uses; for example, looking up the underlying signature for a tag that appears in the Access Log, or getting the ID of a Rule to exclude in a WAF Policy.

Curiefense comes with many WAF Rules. You can view them by choosing different IDs in the pulldown on the upper right.

Note that this list is for reference only. The entries cannot be edited.

Search

This page offers a search capability globally across Curiefense, or for a specific scope (Document Type, ID, Name, Description, Tags, or Connections).

As shown above, hovering the cursor at the end of a search result will display a button that will bring you directly to the document.

Databases

This page displays the current System Database in a JSON editor. These settings are used to populate the interface for Publish Configuration.

The settings are:

buckets: the complete list of buckets to which a Configuration can be pushed.
branch_buckets: the default list of buckets for each branch. When a Configuration is published, the default buckets for the branch will be initially selected in the Publish Configuration interface (but they can be changed before publishing).

Publish Changes

As the name suggests, this page allows you to publish a new or revised Configuration to the cloud, or revert to a previous version.

To do so, follow these steps:

In the pulldown on the top left, select the Configuration.
Below it is the Version History: a list of Configurations, with the most recently edited version at the top. If you wish to publish your most recent edits, ensure that the top entry is selected. If instead you wish to revert to a previous entry, find it in the list (using the View More option if necessary) and select it.
On the right is a list of Target Buckets. The default buckets for this branch (which are defined in System Settings) should be selected already. If different buckets are needed, select them instead.
Select the Publish Configuration button on the top right.

curl Examples

Here are some examples of sending API requests to Curiefense using curl.

This list is not meant to be exhaustive, and might not reflect recent changes to the API. For current and canonical information about API operations and data structures, the Swagger interface is recommended.

Operations on Configurations

Get the detailed list of existing configurations:

curl -XGET $api_url/configs/

Retrieve a complete configuration:

curl -XGET $api_url/configs/{config}/

Create a new configuration, name is in the posted data:

curl -XPOST $api_url/configs/

Create a new configuration, name is provided and overrides posted data:

curl -XPOST $api_url/configs/{config}/

Update an existing configuration:

curl -XPUT $api_url/configs/{config}/

Delete a configuration:

curl -XDELETE $api_url/configs/{config}/

Clone a configuration, new name is in POST data:

curl -XPOST $api_url/configs/{config}/clone/

Clone a configuration, new name is in the URL:

curl -XPOST $api_url/configs/{config}/clone/{new_name}/

Get all versions of a given configuration:

curl -XGET $api_url/configs/{config}/v/

Retrieve a specific version of a configuration:

curl -XGET $api_url/configs/{config}/v/{version}/

Create a new version for a configuration from an old version:

curl -XPUT $api_url/configs/{config}/v/{version}/revert/

Format for full config dumps and batch updates:

{
    "config": { <fields of config metadata> }

    "resources": {
        "<resname>": [],
        ...
    }
    "blobs": {
        "dataname": {"format":"<format of blob, ex: raw, base64, gzip+base64, etc.>", 
                     "blob": "<blob>"
         }
    }
    "delete_resources": {
        "<resname>": {
            "<resid>": true,
            ...
        }
        ...
    }
    "delete_blobs": {
        "<blobname>": true,
        ...
    }
}

Operations on Documents

Retrieve the list of existing documents in this configuration:

curl -XGET $api_url/configs/{config}/d/

Get a complete document:

curl -XGET $api_url/configs/{config}/d/{document}/

Get a given version of a document:

curl -XGET $api_url/configs/{config}/d/{document}/v/{version}/

Retrieve the existing versions of a given document:

curl -XGET $api_url/configs/{config}/d/{document}/v/

Create a new complete document:

curl -XPOST $api_url/configs/{config}/d/{document}/

Update an existing document:

curl -XPUT $api_url/configs/{config}/d/{document}/

Delete/empty a document:

curl -XDELETE $api_url/configs/{config}/d/{document}/

Create a new version for a document from an old version:

curl -XPUT $api_url/configs/{config}/d/{document}/v/{version}/revert/

Operations on Entries

Retrieve the list of entries in a document:

curl -XGET $api_url/configs/{config}/d/{document}/e/

Retrieve an entry from a document:

curl -XGET $api_url/configs/{config}/d/{document}/e/{entry}/

Get the list of existing versions of a given entry in a document:

curl -XGET $api_url/configs/{config}/d/{document}/e/{entry}/v/

Get a given version of a document entry:

curl -XGET $api_url/configs/{config}/r/{document}/i/{entry}/v/{version}/

Create an entry in a document:

curl -XPOST $api_url/configs/{config}/d/{document}/e/

Update an entry in a document:

curl -XPUT $api_url/configs/{config}/d/{document}/e/{entry}/

Delete an entry from a document:

curl -XDELETE $api_url/configs/{config}/d/{document}/e/{entry}/

Operations on Blobs

Retrieve the list of available blobs:

curl -XGET $api_url/configs/{config}/b/

Retrieve a blob:

curl -XGET $api_url/configs/{config}/b/{blob}/

Create a new blob:

curl -XPOST $api_url/configs/{config}/b/{blob}/

Replace a blob with new data:

curl -XPUT $api_url/configs/{config}/b/{blob}/

Delete a blob:

curl -XDELETE $api_url/configs/{config}/b/{blob}/

Retrieve the list of versions of a given blob:

curl -XGET $api_url/configs/{config}/b/{blob}/v/

Retrieve the given version of a blob:

curl -XGET $api_url/configs/{config}/b/{blob}/v/{version}/

Create a new version for a blob from an old version:

curl -XPUT $api_url/configs/{config}/b/{blob}/v/{version}/revert/

Analytics

Kibana

The Access Log displays requests that were received in the specified time period.

Filtering the Display

By default, this page shows all requests in the time period. To change the time period, use the controls at the top right of the window. You can also zoom into a subset of the time period currently being shown, by dragging the cursor across the columns in the graph. The controls above the graph define its granularity.

To filter the display, type a search string into the Search box. For example, to see only those requests that include the substring "script", enter it into the box.

To search for (or exclude) the value of an entire parameter, it is often faster to hover the cursor over an existing request with that value. For example, to show all requests originating from IP address 13.212.190.213, you could manually enter that address into the Search box. Or, find an existing request with that value and hover the mouse over it. A "Filter for value" button will appear; selecting it will create the filter for you automatically.

More powerful filtering capabilities are available immediately below the Search box, by selecting Add filter and manually constructing one or more filtering criteria.

The access log display will change as filters are constructed and edited. To remove all filters, select Reset search at the top of the graph.

It's helpful to spend a few minutes experimenting with these filtering capabilities. You can quickly drill down through large swaths of traffic, discovering events and patterns that can reveal many insights about your traffic. This is helpful when constructing and fine-tuning security policies, especially during attacks.

Viewing Request Details

The primary display shows a summary of each request. To view more information about a request, click on the twirl control (>) at the beginning of its listing. The display will expand to show its full details.

All the details of the request are shown. Hovering the cursor over a parameter, as shown for the headers.request_method example above, shows controls where you can easily construct a filter for it.

Within the expanded display, the tag field can be especially useful.

This shows all the tags that were assigned to this request. It can be helpful in understanding how Curiefense evaluated this request, and the decisions that it made.

Grafana

Introduction

Out of the box, Curiefense stores metrics using , and provides dashboards and alerts via .

By default, Curiefense includes several Grafana dashboards, or you can create your own. See also the Community-Supported dashboards, described in a later section below.

Accessing the Dashboards

To login, the default username and password are admin and admin.

After logging into Grafana, you will see this:

On the Dashboards menu, select "Manage":

This will reveal the Provisioned folder:

Opening the Provisioned folder will reveal the default Curiefense dashboards:

Selecting a dashboard will open it. Then specify the date/time range to display.

Default Dashboards

Curiefense includes several default dashboards, described below. It is also recommended that you evaluate the community dashboards, described in the next section.

Curiefense Overview

This dashboard shows:

Total active connections
Total requests
Downstream total active connections
Downstream total active requests
Upstream network traffic
Downstream network traffic
Downstream members

Curiefense System Metrics

This dashboard currently shows a heatmap of Curielogger's SQL query duration.

Top Activities

This dashboard provides the ability to monitor the top metrics and sources of various security issues, including blocked requests, problematic traffic sources, URLs, and so on

Top Countries
URL maps
URLs
Methods
Blocked Methods
Blocking WAF IDs
Blocking ACL IDs

Traffic Overview

HTTP Status
Incoming requests bandwidth
Outgoing requests bandwidth
Blocked vs Passed
HTTP Origin Status

Community-Supported Dashboards

The following dashboards are worth investigating, to evaluate them for your use case.

GIT

Version Control

This page displays previous versions of configurations, and allows you to revert/restore the system to any saved configuration.

To do this, hover the cursor over the desired configuration. The "Restore Version" button will appear, as shown above.

Select this button. Once the process is complete, publish. Curiefense will push the selected configuration out to all deployments.

Reference

Services and Container Images

Containers are built from recipes contained incuriefense/images/. Descriptions of the various images are below.

Image Tagging

Images can also have two-part tags to identify what is in the image. The parts are:

the output of git describe --tag --long --dirty which contains the latest git tag, number of commits since last tag, and abbreviated current commit hash
the shortened hash of the git tree HEAD:curiefense/ where Docker images are stored. It helps to see quickly whether image sources from two different commits are identical.

Common Images

confserver

A REST API server to read, write, and maintain versioning of Curiefense's configuration.
Flask is the web interface. Git is the storage engine for historical and versioned configuration. Nginx serves as the frontend for the Flask web application.
Secrets:
- If an S3 bucket of Google Storage bucket is used as a synchronization mechanism between confserver and curieproxy instances (S3 is the default with Helm deployments), then this container requires credentials to access it.
- In Docker Compose environments, S3 credentials are defined in deploy/compose/curiesecrets/s3cfg before deployment, then mounted in /var/run/secrets/s3cfg. By default, Docker Compose environments use a "local bucket" (shared volume) so that an S3 bucket is not necessary to test curiefense locally.
- In Helm environments, S3 credentials are . They are expected to be in a Secret object called s3cfg, in the same namespace as confserver, which is mounted to make credentials available in /var/run/secrets/s3cfg/s3cfg. Google Storage bucket credentials are mounted to /var/run/secrets/gs/.
Network details:
- Port 80 is the configuration API. A Swagger interface is available at endpoint /api/v1/ (reachable at in the sample Docker Compose and Helm deployments).

curielogger

Receives access logs from Envoy (curieproxyimages) over gRPC or from nginx over syslog, and does the following:
- Pushes logs to elasticsearch through either fluentd or filebeat,
- Aggregates metrics,
- Serves metrics over HTTP port 2112 for the Prometheus scraper.
Network details:
- Port 9001 receives logs from Envoy over GRPC
- Port 9514 receives logs from nginx over syslog
- Port 2112 exposes Prometheus metrics over HTTP

curiesync

Periodically (every 10 seconds) polls the bucket located at CURIE_BUCKET_LINK, and extracts the active configuration to /config, which is shared with curiefense.
In Helm deployments, curiesync runs first as an initContainer to fetch the correct configuration before istio is started. It then runs in a regular container, to sync periodically.
Secrets: the same as described in confserver, above.
Network details:
- No exposed network service

grafana

Grafana provides visualization for the metrics stored in Prometheus, and sends alerts based on anomaly thresholds.
Its datasource is prometheus. Its basic dashboards are in the dashboards directory.
Secrets
- Default user: admin
- Default password: admin
- These can be changed upon the first connection.
Network details:

prometheus

Prometheus scrapes metrics from Envoy, curielogger, and Prometheus itself. These are defined in curiefense/images/prometheus/prometheus.yml.
curielogger metrics are exposed as follows:
- curiemetric_http_request_total: total number of requests.
- curiemetric_request_bytes: total inbound (request) bytes.
- curiemetric_response_bytes: total outbound (response) bytes.
- curiemetric_session_details_total: Static labels are a fixed set of labels created for each request, such as "method", "path", "geo", etc.
- curiemetric_session_tags_total: this metric stores the "dynamic" labels: tags and labels created dynamically on a request basis, and per context. For example, while the "blocked" label is set to 0 or 1 for each request, an actual blocked request may carry additional tags such as the block reasons and origin.
Network details:
- Port 9090 allows access to the Prometheus user interface over HTTP

redis

Redis is accessed by curieproxy, and is used to synchronize Curiefense's advanced rate limiting and session control mechanisms.
Network details:
- Port 6379 receives Redis client queries

UIServer

Serves the user interface. A Vue js app developed as single page app with NodeJS and serves the management console UI.
The UI displays access logs to the user, and displays Curiefense's configuration for editing. API calls for configuration and access logs are routed to confserver by the Nginx inside the container. Nginx also serves the static parts of the UI such as HTML, CSS and JS.
Secrets: This image will enable TLS on the nginx server if a TLS certificate and key are provided:
- For Kubernetes (e.g. Helm) deployments, the certificate is expected at /run/secrets/uisslcrt/uisslcrt and the key at /run/secrets/uisslkey/uisslkey
- For Docker Compose deployments, the certificate is expected at /run/secrets/uisslcrt and the key at /run/secrets/uisslkey
Network details:

Additional Images for Docker-Compose Deployments

curieproxy-envoy

Acts as a reverse proxy to TARGET_ADDRESS:TARGET_PORT.
Filters traffic according to the active configuration.
Sends access logs over GRPC tocurielogserver.
Network details:
- Port 8001 is the Envoy administration interface

echo

Simple http server that echoes received HTTP requests.
Uses the public jmalloc/echo-server image.
Network details:
- Listens on port 5678

Additional Images for Istio-Helm Deployments

curieproxy-istio

Acts as a reverse proxy to TARGET_ADDRESS:TARGET_PORT.
Filters traffic according to the active configuration.
Sends access logs over GRPC tocurielogserver.
In Helm deployments, two EnvoyFilters are defined:
Network details:
- Port 8001 is the Envoy administration interface

Building/Rebuilding Images

If for some reason you need to rebuild the images, run the following command:

To build images with a custom tag, the DOCKER_TAG environment variable may be set:

The Challenge Process

Detecting Bots, Verifying Humans

Much of the Curiefense interface focuses on configuration for:

The detection and mitigation of hostile requests.
The detection and mitigation of hostile behavior of requestors.

The challenge process is different. It is built into Curiefense, and provides a third approach to security. It mitigates threats based on the requestor's identity, environment, and behavior.

Bot challenges are included in the Premium version of Curiefense.

When Curiefense receives the first request from a previously unknown traffic source (below described as the "user") for a browser-based web application, this is the typical process that is followed.

Curiefense challenges the user's browsing environment. Curiefense uses a variety of proprietary, multi-faceted techniques to verify that this requestor is a human using a browser, instead of a bot using a headless browser or emulator.
- The user’s browser is subjected to several dozen tests, verifying the features known to be supported by that browser. This includes hidden canvases, video and audio in various formats, WebRTC and other advanced networking protocols, screen resolution, and more.
- The browser is subjected to an invisible “attack”: subtle errors are injected into the environment, and the browser engine’s reactions are captured and analyzed. Curiefense verifies that the exceptions and error messages are those which should be generated, if the browser is what it claims to be. (It is very difficult for threat actors to spoof this behavior using headless browsers and emulators, since there is an infinite number of possible errors to which any browser can be subjected, and threat actors need to replicate the actual reactions for each possible input.)
- The above process only takes milliseconds, and it is completely transparent to the end user.
- Once a browser has passed these tests, Curiefense signs it cryptographically. This signature accompanies all subsequent activity.
If the challenge is not passed, the request is suspected to be a bot, and another challenge is issued. This process continues until a challenge is passed, or a threshold is reached (e.g., via a Rate Limit) to ban the requestor.
If the challenge is passed, the browser's session is authenticated, and the browser receives cookies from Curiefense.
The browser then automatically resubmits the original request, but this time, the cookies are included. The user is granted access to the requested URL, resources, etc.
Subsequent requests will also include the cookies, and thus, they are not challenged.

This process happens quickly (in a few milliseconds), and is invisible to the user.

Active Challenges versus Passive Challenges

The process described above is the active challenge process. Out of the box, this is the challenge process that Curiefense uses.

We recommend that whenever possible, users also enable passive challenges.

Passive challenges add three additional benefits:

They enable biometric behavioral profiling: a much more powerful means of identifying automated traffic. More on this below.
In some situations, active challenges can interfere with certain metrics such as those provided by Google Analytics. (The initial referrer information is lost.) If this is a problem, active challenges can be disabled. In this situation, passive challenges can provide effective bot protection instead.
When caching is being done by a CDN, active challenges will not occur for pages being served from the cache. Passive challenges are necessary for Curiefense to perform bot detection in this situation.

If possible, we recommend that users use both active and passive challenges.

Biometric Behavioral Profiling

With Biometric Bot Detection, Curiefense continually gathers and analyzes stats such as client-side I/O events, triggered by the user’s keyboard, mouse, scroll, touch, zoom, device orientation, movements, and more. Based on these metrics, the platform uses Machine Learning to construct and maintain behavioral profiles of legitimate human visitors. Curiefense learns and understands how actual humans interact with the applications and services it is protecting. Continuous multivariate analysis verifies that each user is indeed conforming to expected behavioral patterns, and is thus a human user with legitimate intentions.

Using this approach, Curiefense bot detection accuracy is not only high, it is also robust and resistant to reverse-engineering by threat actors. Behavioral profiles are constructed based on private analytics data, and threat actors have no realistic way of obtaining this information.

This is an important reason why we recommend that all users enable Passive Challenges if it is possible to do so. Biometric Behavioral Profiling provides much more robust detection of automated traffic than is possible without it.

Implementation

Implementing Passive Challenges is simple. Place this Javascript code within the pages of your web applications:

<script src="/c3650cdf-216a-4ba2-80b0-9d6c540b105e58d2670b-ea0f-484e-b88c-0e2c1499ec9bd71e4b42-8570-44e3-89b6-845326fa43b6" type="text/javascript"></script>

The code snippet can go into the header or at the end of the page. The best practice is to place it within the header. This ensures that subsequent calls contain authenticating cookies.

(This matters for the first page served to a visitor. Subsequent pages will already have the authenticating cookies to include with their requests.)

Most users set up the code snippet as a tag within their tag manager. This makes it simple to install it instantly across their entire site/application.

If desired, the script code can include async and/or defer attributes:

<script async src="/c3650cdf-216a-4ba2-80b0-9d6c540b105e58d2670b-ea0f-484e-b88c-0e2c1499ec9bd71e4b42-8570-44e3-89b6-845326fa43b6" type="text/javascript"></script>

<script defer src="/c3650cdf-216a-4ba2-80b0-9d6c540b105e58d2670b-ea0f-484e-b88c-0e2c1499ec9bd71e4b42-8570-44e3-89b6-845326fa43b6" type="text/javascript"></script>

These usually are not necessary, and their effect will depend on the placement of the script within the page. Their use is left to your discretion.

Testing

To test the implementation, use a browser to visit a page containing the Javascript snippet. Once it runs, the browser should have a cookie named rbzid.

Disabling Active Challenges (only if necessary)

There are two primary situations where users sometimes want to disable Active Challenges:

When a user needs site analytics to correctly reflect all referrers. (Active Challenges can interfere with this.)
For API endpoints. Active Challenges are designed to verify the client's browser environment; for most API calls, there is no browser environment to verify.

Other than those situations, Active Challenges can be very beneficial.

We recommend that you keep Active Challenges enabled if possible. They automatically eliminate almost all DDoS traffic, scanning tools, and other hostile bot traffic.

If you wish to turn off Active Challenges, remove all tags from the "Deny Bot" column within ACL Profiles.

If you have not enabled Passive Challenges (and successfully tested them), disabling Active Challenges is not recommended.

Multi-Stage Traffic Filtering

Curiefense evaluates incoming traffic in a multi-stage filtering process. An HTTP/S request which passes all security tests will be forwarded to the backend.

When constructing its security posture, it can be helpful to understand the filtering stages. They are:

Tag Rules: Curiefense assigns and to the requests. During this process, requests will be blocked immediately if they match a defined for it.
enforcement.
enforcement.
enforcement.
and enforcement.
Transmission of request to the backend.

Release Notes

All notable changes to this project will be documented in this file. This project adheres to Semantic Versioning.

Version 1.4.0

Added

[e2e] Add back test_ipv4 which passes
[e2e] Add support for fork repositories in github workflows
[helm] Add curiefense to Istio-helm charts
[docker] Add missing packages to curielogger (to run contrib scripts)
[ui] Add options to configure links to Kibana & Grafana
[curielogger] Add docker-compose e2e tests
[e2e] Add tests to last missing components, fix referral bug in url maps editor, chang coverage thresholds, remove unused code
[ui] Add autocomplete support to WAF Policies editor and resolve a bug in URL Maps editor
[ui] Add a requirement of at least one tag for Tag Rule tags list in Tag Rules json schema
[e2e] Add test of flow control editor in case of multiple limit option keys
[helm] Add v2 deployment tests
[e2e] Add test on fluentd
[helm] Add filebeat to the helm deployment
[curielogger] Add logrotate container
[e2e] Add a testcase for pairwith limits
[e2e] Add Rust formatting tests to Makefile
Add configs and templates for Elasticsearch 6.x
Add an nginx-ingress container
Add map to define request_map
Add knob to disable Kibana initialization (es6 init script)

Updated

[ui] Update dependencies with found security vulnerabilities.
[ui] Update version to 1.3.0 to match the achieved milestone and overall* system version
[docker] Update Envoy configuration version to v3
[e2e] Update log patterns
[docker] Update Istio image to use Envoy binary for 1.9.2
[helm] Update curiefense EnvoyFilters to v3
[docker] Update Envoy binary for Istio
[ci] Update minikube to fix CI
[e2e] Update Rust unit tests to include urldecode
[curieproxy] Update iptools.so in curieproxy with new url decode function
Update iptools.so for lua
Update iptools.so with fixed urldecode
Update with new urldecode algorithm

Improved

[e2e] Improve general coverage of UI unit tests in DocumentEditor.vue and Publish.Vue for a total coverage of 89%+
[e2e] Improve general coverage of UI unit tests, add types to unit tests, fix small issues throughout the UI

Removed

[helm] Remove helm install
[e2e] Remove test for feature that does not exist anymore
[helm] Remove references & variables for postgres & curielogserver
[deploy] Remove remaining postgres configuration values
Remove the ROADMAP.md file in favor of RELEASES.md
Remove ILM for ES 6.x as it was added in 7.x
Remove logstashs' from e2e-ci.yml

Fixed

[ci] use more recent shellcheck version, fix remaining errors
[e2e] Fix ratelimit countby tests
[e2e] Fix WAF Rules tests
[e2e] Fix arguments passed to deploy.sh. Fixes e2e tests.
[e2e] Fix elasticsearch port for tests on minikube
[ci] Fix deployment & tests following Istio update
[e2e] Fix latency tests (deploy-gke.sh)
[ci] Fix environment for rust & lua tests
[docker-compose] Fix curieproxy metrics scrape
[ui] Fix referral bug in url maps editor
[curielogger] Fix test_logs Elasticsearch query
[docker-compose] Fix CI
[curielogger] Fix tag rules logging
[curieproxy] fix geo-related ratelimit counters
[curieproxy] fix geo-related ratelimit scope checks
Fix challenge in flow control
Fix start_curiefense script
Fix flow checks tags
Fix default return codes
Fix nginx failure with unknown remote ip
Fix curiefense/images/uiserver/Dockerfile to reduce vulnerabilities

Enhanced

Istio via Helm

Introduction

The instructions below show how to install Curiefense on a Kubernetes cluster, embedded in an Istio service mesh.

The following tasks, each described below in sequence, should be performed:

Clone the Helm Repository
Setup Synchronization
Create a Kubernetes Cluster Running Helm
Reset State
Create Namespaces
Setup Secrets
Setup TLS
Deploy Istio and Curiefense Images
Deploy the (Sample) App
Expose Curiefense Services Using NodePorts
Access Curiefense Services

At the bottom of this page is a Reference section describing the charts and configuration variables.

Clone the Helm Repository

Clone the repository, if you have not already done so:

git clone https://github.com/curiefense/curiefense-helm.git

This documentation assumes it has been cloned to ~/curiefense-helm.

Setup Synchronization

An AWS S3 bucket must be available to synchronize configurations between the confserver and the Curiefense Istio sidecars. The following Curiefense variables must be set:

In ~/curiefense-helm/istio-helm/charts/gateways/istio-ingress/values.yaml:
- Setcurieconf_manifest_url to the bucket URL.
In ~/curiefense-helm/curiefense-helm/curiefense/values.yaml:
- Set curieconf_manifest_url to the bucket URL.

Create a Kubernetes Cluster

Below are instructions for several ways to achieve this:

Using minikube, Kubernetes 1.20.2 (dynamic provisioning is enabled by default)
Using Google GKE, Kubernetes 1.16.13 (RBAC and dynamic provisioning are enabled by default)
Using Amazon EKS, Kubernetes 1.18 (RBAC and dynamic provisioning are enabled by default)

You will need to install the following clients:

Install kubectl (https://kubernetes.io/docs/tasks/tools/install-kubectl/) -- use the same version as your cluster.
Install Helm v3 (https://helm.sh/docs/intro/install/)

Option 1: Using minikube

This section describes the install for a single-node test setup (which is generally not useful for production).

Install minikube

Starting from a fresh ubuntu 21.04 VM:

Install docker (https://docs.docker.com/engine/install/ubuntu/), and allow your user to interact with docker with sudo usermod -aG docker $USER && newgrp docker
Install minikube (https://minikube.sigs.k8s.io/docs/start/)

minikube start --kubernetes-version=v1.20.2 --driver=docker --memory='8g' --cpus 6
minikube addons enable ingress

Start a screen or tmux, and keep the following command running:

minikube tunnel

Option 2: Using Google GKE

Create a cluster

gcloud container clusters create curiefense-gks --num-nodes=1 --machine-type=n1-standard-4
gcloud container clusters get-credentials curiefense-gks

Option 3: Using Amazon EKS

Create a cluster

eksctl create cluster --name curiefense-eks-2 --version 1.18 --nodes 1 --nodes-max 1 --managed --region us-east-2 --node-type m5.xlarge

Reset State

If you have a clean machine where Curiefense has never been installed, skip this step and go to the next.

Otherwise, run these commands:

helm delete curiefense
helm delete -n curiefense curiefense
helm delete -n istio-system istio-ingress
helm delete -n istio-system istiod
helm delete -n istio-system istio-base

Ensure that helm ls -a --all-namespaces outputs nothing.

Create Namespaces

Run the following commands:

kubectl create namespace curiefense
kubectl create namespace istio-system

Setup Secrets

AWS credentials

Encode the AWS S3 credentials that have r/w access to the S3 bucket. This yields a base64 string:

cat << EOF | base64 -w0
[default]
access_key = xxxxxxxxxxxxxxxxxxxx
secret_key = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
EOF

Create a local file called s3cfg.yaml, with the contents below, replacing both occurrences of BASE64_S3CFG with the previously obtained base64 string:

---
apiVersion: v1
kind: Secret
data:
  s3cfg: "BASE64_S3CFG"
metadata:
  namespace: curiefense
  labels:
    app.kubernetes.io/name: s3cfg
  name: s3cfg
type: Opaque
---
apiVersion: v1
kind: Secret
data:
  s3cfg: "BASE64_S3CFG"
metadata:
  namespace: istio-system
  labels:
    app.kubernetes.io/name: s3cfg
  name: s3cfg
type: Opaque

Deploy this secret to the cluster:

kubectl apply -f s3cfg.yaml

Alternative to AWS: GS credentials

Create a bucket, and a service account that has read/write access to the bucket. Obtain a private key for this account, which should look like this:

{
  "type": "service_account",
  "project_id": "PROJECT",
  "private_key_id": "1234abcd1234abcd1234abcd1234abcd1234abcd",
  "private_key": "-----BEGIN PRIVATE KEY-----\nMIIE.....ABCD=\n-----END PRIVATE KEY-----\n",
  "client_email": "....@PROJECT.iam.gserviceaccount.com",
  "client_id": "123412341234123412341",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/....%40PROJECT.iam.gserviceaccount.com"
}

Create a local file called gs.yaml, with the contents below, replacing both occurrences of BASE64_GS_PRIVATE_KEY with the previously obtained base64 string:

---
apiVersion: v1
kind: Secret
data:
  gs.json: "BASE64_GS_PRIVATE_KEY"
metadata:
  labels:
    app.kubernetes.io/name: gs
  name: gs
  namespace: curiefense
type: Opaque
---
apiVersion: v1
kind: Secret
data:
  gs.json: "BASE64_GS_PRIVATE_KEY"
metadata:
  labels:
    app.kubernetes.io/name: gs
  name: gs
  namespace: istio-system
type: Opaque

Deploy this secret to the cluster:

kubectl apply -f gs.yaml

Also set the curiefense_bucket_type variables in the same values.yaml files to gs.

Setup TLS for the UI server

Using TLS is optional. Follow these steps if only if you want to use TLS for communicating with the UI server, and you do not rely on istio to manage TLS.

The UIServer can be made to be reachable over HTTPS. To do that, two secrets have to be created to hold the TLS certificate and TLS key.

Create a local file called uiserver-tls.yaml, replacing TLS_CERT_BASE64 with the base64-encoded PEM X509 TLS certificate, and TLS_KEY_BASE64 with the base64-encoded TLS key.

---
apiVersion: v1
data:
  uisslcrt: TLS_CERT_BASE64
kind: Secret
metadata:
  labels:
    app.kubernetes.io/name: uisslcrt
  name: uisslcrt
  namespace: curiefense
type: Opaque
---
apiVersion: v1
data:
  uisslkey: TLS_KEY_BASE64
kind: Secret
metadata:
  labels:
    app.kubernetes.io/name: uisslkey
  name: uisslkey
  namespace: curiefense
type: Opaque

Deploy this secret to the cluster:

kubectl apply -f uiserver-tls.yaml

An example file with self-signed certificates is provided at ~/curiefense-helm/curiefense-helm/example-uiserver-tls.yaml.

Deploy Istio and Curiefense Images

Deploy the Istio service mesh:

cd ~/curiefense-helm/istio-helm 
DOCKER_TAG=main ./deploy.sh

And then the Curiefense components:

cd ~/curiefense-helm/curiefense-helm
DOCKER_TAG=main ./deploy.sh

Deploy the (Sample) App

Enable Istio injection

Add the istio-injection=enabled label that will make Istio automatically inject necessary sidecars to applications that are deployed in the default namespace.

kubectl label namespace default istio-injection=enabled

Install the application

cd ~
wget 'https://github.com/istio/istio/releases/download/1.9.3/istio-1.9.3-linux-amd64.tar.gz'
tar -xf istio-1.9.3-linux-amd64.tar.gz
cd ~/istio-1.9.3/
kubectl apply -f samples/bookinfo/platform/kube/bookinfo.yaml
kubectl apply -f samples/bookinfo/networking/bookinfo-gateway.yaml

Test bookinfo

Check that bookinfo Pods are running (wait a bit if they are not):

kubectl get pod -l app=ratings

Sample output example:

NAME                         READY   STATUS    RESTARTS   AGE
ratings-v1-f745cf57b-cjg69   2/2     Running   0          79s

Check that the application is working by querying its API directly without going through the Istio service mesh:

kubectl exec "$(kubectl get pod -l app=ratings -o jsonpath='{.items[0].metadata.name}')" -c ratings -- curl -sS productpage:9080/productpage | grep -o "<title>.*</title>"

Expected output:

<title>Simple Bookstore App</title>

Test access to bookinfo through Istio

Set the GATEWAY_URL variable by following instructions on the Istio website.

Alternatively, with minikube, this command can be used instead:

export GATEWAY_URL=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}'):80

Check that bookinfo is reachable through Istio:

curl -sS http://$GATEWAY_URL/productpage | grep -o "<title>.*</title>"

Expected output:

<title>Simple Bookstore App</title>

Check that logs stored in the Elasticsearch cluster

Run this query to access the protected website, bookinfo, and thus generate an access log entry:

curl http://$GATEWAY_URL/TEST_STRING

Run this to ensure that the logs have been recorded:

kubectl exec -ti -n curiefense elasticsearch-0 -- curl http://127.0.0.1:9200/_search -H "Content-Type: application/json" -d '{"query": {"bool": {"must":{"match":{"request.attributes.uri": "/TEST_STRING"}}}}}'|grep -Eo '"uri":"/TEST_STRING"'

Expected output:.

"uri":"/TEST_STRING"

Expose Curiefense Services Using NodePorts

Run the following commands to expose Curiefense services through NodePorts. Warning: if the machine has a public IP, the services will be exposed on the Internet.

Start with this command:

kubectl apply -f ~/curiefense-helm/curiefense-helm/expose-services.yaml

The following command can be used to determine the IP address of your cluster nodes on which services will be exposed:

kubectl get nodes -o wide

For minikube only:

If you are using minikube, also run the following commands on the host in order to expose services on the Internet (ex. if you are running this on a cloud VM):

sudo iptables -t nat -A PREROUTING -p tcp --match multiport --dports 30000,30080,30300,30443 -j DNAT --to $(minikube ip)
sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j DNAT --to $(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
sudo iptables -I FORWARD -p tcp --match multiport --dports 80,30000,30080,30300,30443,30444 -j  ACCEPT

For Amazon EKS only:

Access Curiefense Services

The UIServer is now available on port 30080 over HTTP, and on port 30443 over HTTPS.

Grafana is now available on port 30300 over HTTP.

For the bookinfo sample app, the Book Review product page is now available on port 80 over HTTP, and on port 30444 over HTTPS. Try reaching http://IP/productpage.

The confserver is now available on port 30000 over HTTP: try reaching http://IP:30000/api/v1/.

For a full list of ports used by Curiefense containers, see the Reference page on services and containers.

Reference: Description of Helm Charts

Curiefense charts

Helm charts are divided as follows:

curiefense-admin - confserver, and UIServer.
curiefense-dashboards - Grafana and Prometheus.
curiefense-log - elasticsearch, filebeat, fluentd, kibana, logstash.
curiefense-proxy - curielogger and redis.

Chart configuration variables

Configuration variables in ~/curiefense-helm/curiefense-helm/curiefense/values.yaml can be modified or overridden to fit your deployment needs:

Variables in the images section define the Docker image names for each component. Override this if you want to host images on your own private registry.
storage_class_name is the StorageClass that is used for dynamic provisioning of Persistent Volumes. It defaults to null (default storage class, which works by default on EKS, GKE and minikube).
..._storage_size variables define the size of persistent volumes. The defaults are fine for a test or small-scale deployment.
curieconf_manifest_url is the URL of the AWS S3 or Google Cloud Storage bucket that is used to synchronize configurations between the confserver and the Curiefense Istio sidecars.
docker_tag defines the image tag versions that should be used. deploy.sh will override this to deploy a version that matches the current working directory, unless the DOCKER_TAG environment variable is set.

Istio chart

An initContainer called curiesync-initialpull has been added. It synchronizes configuration before running Envoy.
A container called curiesync has been added. It periodically fetches the configuration that should be applied from an S3 or GS bucket (configurable with the curieconf_manifest_url variable), and makes it available to Envoy. This configuration is used by the LUA code that inspects traffic.
The container called istio-proxy now uses our custom Docker image, embedding our HTTP Filter, written in Lua.
An EnvoyFilter has been added. It forwards access logs to curielogger (see curiefense_access_logs_filter.yaml).
An EnvoyFilter has been added. It runs Curiefense's Lua code to inspect incoming traffic on the Ingress Gateways (see curiefense_lua_filter.yaml).

Chart configuration variables

Configuration variables in ~/curiefense-helm/istio-helm/charts/gateways/istio-ingress/values.yaml can be modified or overridden to fit your deployment needs:

gw_image defines the name of the image that contains our filtering code and modified Envoy binary.
curiesync_image defines the name of the image that contains scripts that synchronize local Envoy configuration with the AWS S3 bucket defined in curieconf_manifest_url.
curieconf_manifest_url is the URL of the AWS S3 bucket that is used to synchronize configurations between the confserver and the Curiefense Istio sidecars.
curiefense_namespace should contain the name of the namespace where Curiefense components defined in ~/curiefense-helm/curiefense-helm/ are running.
redis_host defines the hostname of the redis server that will be used by curieproxy. Defaults to the provided redis StatefulSet. Override this to replace the redis instance with one you supply.
initial_curieconf_pull defines whether a configuration should be pulled from the AWS S3 bucket before running Envoy (true), or if traffic should be allowed to flow with a default configuration until the next synchronization (typically every 10s).