NGINX Ingress
- Kubernetes cluster (I have one running on AWS)
- Bucket (I'll use S3)
Note: the Kubernetes cluster should be large enough to run all dependencies (including ElasticSearch).
On AWS, you can create a bucket using the AWS CLI:
# Use your bucket name
aws s3 mb s3://my-curiefense-test
Create a new user for Curiefense:
aws iam create-user --user-name my-curiefense-test
Create new credentials for the user:
aws iam create-access-key --user-name my-curiefense-test
Take note of the
AccessKeyId and SecretAccessKey fields.Create a
policy.json file with the following content:{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Sid0",
"Effect": "Allow",
"Action": "s3:*",
"Resource": "arn:aws:s3:::my-curiefense-test/*"
},
{
"Sid": "Sid1",
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": "arn:aws:s3:::my-curiefense-test"
},
{
"Sid": "Sid2",
"Effect": "Allow",
"Action": [
"s3:ListAllMyBuckets"
],
"Resource": "*"
}
]
}
Attach the policy to the user:
aws iam put-user-policy --user-name my-curiefense-test --policy-name CuriefenseS3Bucket --policy-document file://policy.json
Due to some limitations (syslog config in nginx image, etc) every component have to be installed in the same namespace.
Create a
curiefense namespace:kubectl create namespace curiefense
Create a
curiesync-secret.yaml file with the following content:apiVersion: v1
kind: Secret
metadata:
name: curiesync
data:
curiesync.env: |
export CURIE_BUCKET_LINK=s3://my-curiefense-test/prod/manifest.json
export CURIE_S3_ACCESS_KEY=YOUR_ACCESS_KEY_ID
export CURIE_S3_SECRET_KEY=YOUR_SECRET_ACCESS_KEY
Create the secret:
kubectl -n curiefense apply -f curiesync-secret.yaml
Create a
values.ingress.yaml file with the following content:controller:
image:
repository: curiefense/curiefense-nginx-ingress
tag: e2bd0d43d9ecd7c6544a8457cf74ef1df85547c2
volumes:
- name: curiesync
secret:
secretName: curiesync
volumeMounts:
- name: curiesync
mountPath: /etc/curiefense
If you don't already have the
nginx-stable repo added to Helm, run the following commands:helm repo add nginx-stable https://helm.nginx.com/stable
helm repo update
Install the ingress controller:
# This particular chart version installs the latest supported curiefense nginx ingress image
helm -n curiefense install --version 0.9.3 -f values.ingress.yaml ingress nginx-stable/nginx-ingress
Create a
s3cfg-secret.yaml file with the following content:apiVersion: v1
kind: Secret
metadata:
name: s3cfg
type: Opaque
stringData:
s3cfg: |
[default]
access_key = YOUR_ACCESS_KEY_ID
secret_key = YOUR_SECRET_ACCESS_KEY
Create the secret:
kubectl -n curiefense apply -f s3cfg-secret.yaml
Create a
values.curiefense.yaml file with the following content:global:
proxy:
frontend: "nginx"
settings:
curieconf_manifest_url: "s3://my-curiefense-test/prod/manifest.json"
Clone the Curiefense Helm repository:
git clone [email protected]:curiefense/curiefense-helm.git
Install Curiefense:
helm install -n curiefense -f values.curiefense.yaml curiefense ./curiefense-helm/curiefense-helm/curiefense
Open a port forward to the UI server and start hacking:
kubectl -n curiefense port-forward deploy/uiserver 8080:80
open http://localhost:8080
Make some changes then head to the "Publish Changes" section and click "Publish configuration".
It's time to put Curiefense to the test.
Create an
echoserver.yaml file with the following content:apiVersion: apps/v1
kind: Deployment
metadata:
name: echoserver
labels:
app.kubernetes.io/part-of: "curiefense"
spec:
replicas: 1
selector:
matchLabels:
app: echoserver
template:
metadata:
labels:
app: echoserver
app.kubernetes.io/part-of: "curiefense"
spec:
containers:
- image: gcr.io/google_containers/echoserver:1.10
imagePullPolicy: IfNotPresent
name: echoserver
ports:
- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: echoserver
labels:
app: echoserver
service: echoserver
app.kubernetes.io/part-of: "curiefense"
spec:
ports:
- port: 8080
name: http
selector:
app: echoserver
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: echoserver
labels:
app.kubernetes.io/part-of: "curiefense"
annotations:
nginx.org/location-snippets: |
access_by_lua_block {
local session = require "lua.session_nginx"
session.inspect(ngx)
}
log_by_lua_block {
local session = require "lua.session_nginx"
session.log(ngx)
}
spec:
ingressClassName: nginx
rules:
- host: YOUR_HOST
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: echoserver
port:
number: 8080
Based on how you configured the ingress controller and DNS, you should be able to access the echoserver at the host of your choosing.
Be careful, these commands are destructive!
Once you are done, you can cleanup the created resources from the cluster with the following commands:
kubectl delete -f echoserver.yaml
kubectl delete namespace curiefense
To delete all AWS resource:
aws iam delete-user-policy --user-name my-curiefense-test --policy-name CuriefenseS3Bucket
aws iam delete-access-key --user-name my-curiefense-test --access-key-id YOUR_ACCESS_KEY_ID
aws iam delete-user --user-name my-curiefense-test
aws s3 rb s3://my-curiefense-test --force
- Curiefense nginx ingress image should be updated to the latest version (to support the latest Ingress API)
- Ingress needs to be deployed in the same namespace at the moment (in order to push logs to curielogger)
- ElasticSearch doesn't work out of the box
Last modified 8mo ago