The full deployment process for Curiefense:
First, complete the general tasks described in First Tasks
Then complete the specific tasks for your environment:
This page describes the initial tasks that must be performed when deploying Curiefense:
After completing the tasks below, proceed to the tasks for the specific method being used:
During this process, you might find it helpful to read the descriptions (which include the purpose, secrets, and network/port details) of the services and their containers:
Clone the repository, if you have not already done so:
This documentation assumes it has been cloned to ~/curiefense
.
Curiefense can use TLS, but this is optional. (If you do not choose to set it up, HTTPS will be disabled.)
At this point in the setup process, you should decide whether or not to do so:
An Istio Helm deployment can use TLS for communication with Curiefense's UI server.
A Docker Compose deployment can use TLS for communication with Curiefense's UI server and also for the protected service.
If you do not want Curiefense to use TLS, then skip this step and proceed to the next section.
If you want Curiefense to use TLS, generate the certificate(s) and key(s) now. You will add them to Curiefense later.
After performing the previous tasks, perform the tasks for the specific method being used:
This page describes the tasks necessary to deploy Curiefense using Docker Compose. It assumes that the instructions described in First Tasks have been completed successfully.
This process consists of the following tasks, described sequentially below:
If during this process you need to rebuild an image, see the instructions here: Building/Rebuilding an Image.
If you want Curiefense to use TLS, then you should have already generated the certificates and keys.
To enable TLS for the protected site/application, go to curiefense/deploy/compose/curiesecrets/curieproxy_ssl/
and do the following:
Edit site.crt
and add the certificate.
Edit site.key
and add the key.
To enable TLS for the nginx server that is used by uiserver
, go to curiefense/deploy/compose/curiesecrets/uiserver_ssl/
and do the following:
Edit ui.crt
and add the certificate.
Edit ui.key
and add the key.
The logdb database has two accounts:
Thepostgres
account has write access, and is used by curielogger
.
Thelogserver_ro
account has read-only access, and is used by curielogserver
.
If you wish to change the default passwords for these accounts, you must edit the files in which they are defined:
The password for postgres
is defined in curiesecrets/logdb/postgres_password.txt
.
The password for logserver_ro
is defined in curiesecrets/logdb/ro_password.txt
.
Docker Compose deployments can be configured in two ways:
By setting values for variables in deploy/compose/.env
Or by setting OS environment variables (which will override any variables set in.env
)
These variables are described below.
Curiefense uses the storage defined here for synchronizing configuration changes between confserver
and the Curiefense sidecars.
By default, this points to the local_bucket
Docker volume:
For multi-node deployments, or to use S3 for a single node, replace this value with the URL of an S3 bucket:
In that case, you will need to supply AWS credentials in deploy/compose/curiesecrets/s3cfg
, following this template:
The address of the destination service for which Curiefense acts as a reverse proxy. By default, this points to the echo
container, which simply echoes the HTTP requests it receives.
Defaults to latest
(the latest stable image). To run a version that matches the contents of your working directory, use the following command:
Once the tasks above are completed, run these commands:
After deployment, the Echo service should be running and protected behind Curiefense. You can test the success of the deployment by querying it:
Also verify the following:
The UIServer is now available at http://localhost:30080 (access logs are at http://localhost:30080/accesslog)
Grafana is now available at http://localhost:30300
The confserver
is now available at http://localhost:30000/api/v1/
To stop all containers and remove any persistent data stored in volumes, run the following commands:
The instructions below show how to install Curiefense on a Kubernetes cluster, embedded in an Istio service mesh. It assumes that the instructions described in First Tasks have been completed successfully.
The following tasks, each described below in sequence, should be performed:
At the bottom of this page is a Reference section describing the charts and configuration variables.
An AWS S3 bucket must be available to synchronize configurations between the confserver
and the Curiefense Istio sidecars. The following Curiefense variables must be set:
In deploy/istio-helm/chart/values.yaml
:
Setcurieconf_manifest_url
to the bucket URL.
In deploy/curiefense-helm/curiefense/values.yaml
:
Set curieconf_manifest_url
to the bucket URL.
Set curiefense_db_hostname
to the hostname (or FQDN) of the database server if you choose to supply your own database. If you choose to use the instance from our curiefense
chart, use the default logdb
value.
Access to a Kubernetes cluster running Helm v2 is required. Dynamic provisioning of persistent volumes must be supported. To set a StorageClass other than the default, change or override variable storage_class_name
in deploy/curiefense-helm/curiefense/values.yaml
.
Below are instructions for several ways to achieve this:
Using minikube, Kubernetes 1.14.9 and Helm v2.13.1 (dynamic provisioning is enabled by default)
Using Google GKE, Kubernetes 1.16.13 (with RBAC) and Helm v2.16.7 (dynamic provisioning is enabled by default)
Using Amazon EKS, Kubernetes 1.18 (with RBAC) and Helm v2.16.7 (dynamic provisioning is enabled by default)
This section describes the install for a single-node test setup (which is generally not useful for production).
Starting from a fresh ubuntu 20.04 VM:
Install docker (https://docs.docker.com/engine/install/ubuntu/)
Install kubectl (https://kubernetes.io/docs/tasks/tools/install-kubectl/) -- use version 1.14.9.
Install minikube (https://minikube.sigs.k8s.io/docs/start/)
Allow your user to interact with docker: sudo usermod -aG docker $USER && newgrp docker
Start a screen
or tmux
, and keep the following command running:
Run the following commands:
(Alternately, Helm can be manually downloaded as a binary release, as explained at https://helm.sh/docs/intro/install/. If you choose to do this, ensure that you obtain v2.13.1.)
Now install Helm to the Kubernetes cluster:
This option uses a more recent Kubernetes, with RBAC enabled.
Follow instructions at https://kubernetes.io/docs/tasks/tools/install-kubectl/. Use version 1.16.13.
(Alternately, Helm can be manually downloaded as a binary release, as explained at https://helm.sh/docs/intro/install/. If you choose to do this, ensure that you obtain v2.16.7.)
Now we must define RBAC authorizations. Helm needs to be able to deploy applications to both the curiefense
and istio-system
namespaces.
To do that, we provide an example configuration, which installs Tiller in the kube-system
namespaces, and grants it cluster-admin permissions.
Finally, install Helm to the Kubernetes cluster:
This option uses a more recent Kubernetes, with RBAC enabled.
Follow instructions at https://kubernetes.io/docs/tasks/tools/install-kubectl/. Use version 1.18.
Create a cluster
Install Helm v2.16.7
Follow all the "Install Helm v2.16.7" instructions shown above in the Google GKE section.
If you have a clean machine where Curiefense has never been installed, skip this step and go to the next.
Otherwise, run these commands:
Ensure that helm ls -a
outputs nothing.
Run the following commands:
Encode the AWS S3 credentials that have r/w access to the S3 bucket. This yields a base64 string:
Create a local file called s3cfg.yaml
, with the contents below, replacing both occurrences of BASE64_S3CFG
with the previously obtained base64 string:
Deploy this secret to the cluster:
Curiefense requires two database accounts. They will be automatically provisioned by the logdb
container:
One with read/write authorization (described below as BASE64_READWRITE_USERNAME
andBASE64_READWRITE_PASSWORD
). If the logdb
container is used (which is the default), BASE64_READWRITE_USERNAME
must be set to postgres
.
One with read-only permissions (described below as
BASE64_READONLY_USERNAME
and BASE64_READONLY_PASSWORD
). If the logdb
container is used (which is the default), BASE64_READONLY_USERNAME
must be set to logserver_ro
.
Create a local file called dbsecret.yaml
, with the contents below, containing database credentials. Replace BASE64_READWRITE_USERNAME
, BASE64_READONLY_USERNAME
, BASE64_READWRITE_PASSWORD
and BASE64_READONLY_PASSWORD
with the correct base64-encoded values.
Deploy this secret to the cluster:
An example file with weak default credentials is provided at deploy/curiefense-helm/example-dbsecret.yaml
.
Using TLS is optional.
The UIServer can be made to be reachable over HTTPS. To do that, two secrets have to be created to hold the TLS certificate and TLS key.
Create a local file called uiserver-tls.yaml
, replacing TLS_CERT_BASE64
with the base64-encoded PEM X509 TLS certificate, and TLS_KEY_BASE64
with the base64-encoded TLS key.
Deploy this secret to the cluster:
An example file with self-signed certificates is provided at deploy/curiefense-helm/example-uiserver-tls.yaml
.
Deploy the Istio service mesh:
And then the Curiefense components:
The application to be protected by Curiefense should now be deployed. These instructions are for the sample application bookinfo
.
Create the Kubernetes namespace, and add the istio-injection=enabled
label that will make Istio automatically inject necessary sidecars to applications that are deployed in this namespace.
Check that bookinfo
Pods are running (wait a bit if they are not):
Sample output example:
Check that the application is working by querying its API directly without going through the Istio service mesh:
Expected output:
(Replace "ip" with "hostname" if running in an environment where the LoadBalancer yields a FQDN, as is the case with Amazon's ELB.)
Expected output:
If this error occurs: Could not resolve host: a6fdxxxxxxxxxxxxxxxxxxxxxxxxxxxx-xxxxxxxx.us-west-2.elb.amazonaws.com
...the ELB is not ready yet. Wait and retry until it becomes available (typically a few minutes).
Run this query (add random characters at the end)
(Replace "ip" with "hostname" if running in an environment where the LoadBalancer yields a FQDN, as is the case with Amazon's ELB.)
Run this to ensure that the logs have been recorded and are reachable from the UI server:
Check that TEST_STRING
appears in the output.
Run the following commands to expose Curiefense services through NodePorts. Warning: if the machine has a public IP, the services will be exposed on the Internet.
Start with this command:
The following command can be used to determine the IP address of your cluster nodes on which services will be exposed:
If you are using minikube, also run the following commands on the host in order to expose services on the Internet:
If you are using Amazon EKS, you will also need to allow inbound connections for port range 30000-30500 from your IP. Go to the EC2 page in the AWS console, select the EC2 instance for the cluster (named curiefense-eks-...-Node
), select the "Security" pane, select the security group (named eks-cluster-sg-curiefense-eks-[0-9]+
), then add the incoming rule.
The UIServer is now available on port 30080 over HTTP, and on port 30443 over HTTPS.
Grafana is now available on port 30300 over HTTP.
For the bookinfo
sample app, the Book Review product page is now available on port 30081 over HTTP, and on port 30444 over HTTPS.
The confserver is now available on port 30000 over HTTP.
For a full list of ports used by Curiefense containers, see the Reference page on services and containers.
Helm charts are divided as follows:
curiefense-admin
- confserver, curielogserver and UIServer.
curiefense-dashboards
- Grafana and Prometheus.
curiefense-log
- logdb, namely: PostgreSQL.
curiefense-proxy
- curielogger, curiesync and redis.
Configuration variables in deploy/curiefense-helm/curiefense/values.yaml
can be modified or overridden to fit your deployment needs:
Variables in the images
section define the Docker image names for each component. Override this if you want to host images on your own private registry.
storage_class_name
is the StorageClass that is used for dynamic provisioning of Persistent Volumes. It defaults to null
(default storage class, which works by default on EKS, GKE and minikube).
..._storage_size
variables define the size of persistent volumes. The defaults are fine for a test or small-scale deployment.
curieconf_manifest_url
is the URL of the AWS S3 bucket that is used to synchronize configurations between the confserver
and the Curiefense Istio sidecars.
curiefense_db_hostname
defines the hostname of the postgres server that will be used to store logs. Defaults to the provided logdb
StatefulSet. Override this to replace the postgres instance with one you supply, or an AWS Aurora instance.
docker_tag
defines the image tag versions that should be used. deploy.sh
will override this to deploy a version that matches the current working directory, unless the DOCKER_TAG
environment variable is set.
Components added or modified by Curiefense are defined in deploy/istio-helm/chart/charts/gateways
. Compared to the upstream Istio Kubernetes distribution, we add or change the following Pods:
An initContainer
called curiesync-initialpull
has been added. It synchronizes configuration before running Envoy.
A container called curiesync
has been added. It periodically fetches the configuration that should be applied from an S3 bucket (configurable with the curieconf_manifest_url
variable), and makes it available to Envoy. This configuration is used by the LUA code that inspects traffic.
The container called istio-proxy
now uses our custom Docker image, embedding our HTTP Filter, written in Lua.
An EnvoyFilter
has been added. It forwards access logs to curielogger
(see curiefense_access_logs_filter.yaml
).
An EnvoyFilter
has been added. It runs Curiefense's Lua code to inspect incoming traffic on the Ingress Gateways (see curiefense_lua_filter.yaml
).
Configuration variables in deploy/istio-helm/chart/values.yaml
can be modified or overridden to fit your deployment needs:
gw_image
defines the name of the image that contains our filtering code and modified Envoy binary.
curiesync_image
defines the name of the image that contains scripts that synchronize local Envoy configuration with the AWS S3 bucket defined in curieconf_manifest_url
.
curieconf_manifest_url
is the URL of the AWS S3 bucket that is used to synchronize configurations between the confserver
and the Curiefense Istio sidecars.
curiefense_namespace
should contain the name of the namespace where Curiefense components defined in deploy/curiefense-helm/
are running.
redis_host
defines the hostname of the redis server that will be used by curieproxy
. Defaults to the provided redis StatefulSet. Override this to replace the redis instance with one you supply.
initial_curieconf_pull
defines whether a configuration should be pulled from the AWS S3 bucket before running Envoy (true
), or if traffic should be allowed to flow with a default configuration until the next synchronization (typically every 10s).