Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
The input controls at the top of this page are described here: Policies & Rules Entry Administration. Specific editing of a URL Map is described below.
This page specifies a list of URLs and the security policies assigned to them.
Every incoming HTTP/S request targets a specific URL. Curiefense finds the best match for that URL in the URL Maps, and applies the security policies defined for it.
The "best match" is determined by regex evaluation. The order in which the URLs are listed in the interface does not matter.
A URL Map consists of:
Host definition: The (sub)domain(s) within which the Path Maps will be found.
Path Maps: one or more paths, and the security policies which will be applied to them.
Every Curiefense deployment includes a default URL Map. If a request does not match any other URL Map, the default one is applied.
To ensure that a default always exists, the Matching Name and Path Map for this URL Map are not editable.
To add a new URL Map, use the buttons at the top of the window to duplicate an existing one or create a new one. Then fill in these fields.
When you create or revise a URL Map, each combination of Matching Name and Path Map must be unique. For this reason, when a new URL Map is created, the UI generates a unique Matching Name.
This should be changed to a correct value before the URL Map is saved.
A new URL Map will include a default Path Map. Clicking on it, or on the expand button at the end of its listing, will expand it for editing.
To add a new Path Map, select an existing one, expand it, and select Fork Profile at the bottom. The existing one will be cloned, and the new one will be displayed for editing.
Note that the buttons at the top of the window are for administering URL Maps (which generally correspond to domains). Administering Path Maps (for paths and URLs within the specified domain) is done in the middle of the window.
Ensure that URL Maps do not overlap. In other words, for every domain defined in a Matching Name, ensure that every possible path within it cannot match more than one Match expression. If an incoming request matches more than one Path Map definition, there is no way to predict which of them Curiefense will use when enforcing security policies.
In addition to editing the fields discussed above, the Path Mapping dialog also provides the ability to:
Activate or deactivate the WAF Policy (by toggling its Active Mode checkbox).
Activate or deactivate the ACL Policy (by toggling its Active Mode checkbox).
Assign an existing Rate Limit rule to this Path Map, via the + button or selecting the link ("To attach an existing rule, click here."), then selecting add. (The + button will only be shown if there are unassigned Rate Limit rules available.)
Create a new Rate Limit rule for this Path Map, by selecting the link ("To create a new rate-limit rule, click here.")
Remove an assigned Rate Limit Rule by selecting remove.
Create a copy of this Path Map, and open it for editing, via the Fork Profile button.
Rate Limits are rules which define the number of requests with certain characteristics that are allowed within defined time frames. When a request is received that exceeds a Rate Limit, a specified action is performed.
The input controls at the top of this page are described here: Policies & Rules Entry Administration. Specific editing of a Rate Limit is described below.
A Rate Limit defines the number of times that requests can match certain conditions within a certain time frame. Once that limit has been reached, subsequent requests matching those conditions within the same time frame will trigger an action.
The matching conditions are specified with the parameters in the "Count by" section and the optional Event. See Matching Conditions below.
After a Rate Limit is defined on this page, it will not be active until it is assigned to one or more URLs. This is done in the URL Maps Document.
By default, a Rate Limit will be enforced for all requests for the URL(s) to which it is assigned.
Enforcement can be further limited to a subset of these requests: see Limiting the scope of a Rate Limit below.
A condition consists of a field and a value. Within a Rate Limit, they play a role like this:
"More than <THRESHOLD> requests with the same <CONDITION-VALUE> <CONDITION-FIELD> sent to <ASSIGNED-LOCATION> within <TTL-PERIOD> will cause <ACTION>."
Example: "More than three requests with the same username argument sent to the login form within one hour will cause the requestor to be banned for six hours."
Note that the <ASSIGNED-LOCATION> mentioned above is included for illustration purposes only. This value is not part of a Rate Limit definition—it is defined in a URL Map.
A condition can be built upon any one of these four categories:
Multiple conditions can be defined within the same Rate Limit. To create a new condition and open it for editing, select "New entry" below the list of conditions.
If multiple conditions are defined, they are evaluated by combining them together with a logical AND. In other words, the cumulative count toward the Threshold will be incremented whenever a request is seen that matches all of the conditions simultaneously. Different combinations of conditions will have separate Threshold counts maintained for them.
Example. A Rule contains two conditions: "Attribute / Remote Address" and "Argument / Username". When the first request is received, an internal counter is created (set to a value of one) for this unique combination of Remote Address and Username. A second request is then received, originating from the same Remote Address and for the same Username; this causes the internal counter to be incremented up to two. A third request is then received from the same Remote Address but with a different Username; this causes a new internal counter to be created (and set to a value of one) for this combination.
Below the list of condition(s), there is another condition named "Event."
By default, this is set to "HTTP request,", which simply means to increment a counter each time a request is received that matches the conditions.
However, if the Event condition is changed to a different value, then the following applies.
In the discussion below, "Count Condition" will refer to the condition (or combination of conditions) defined by the Count by input controls.
"Event Condition" will refer to the optional, additional condition defined by the Event input controls.
Adding an Event Condition changes the evaluation process. An Event Condition is not logically combined with the preceding Count Condition; it is always evaluated separately.
More importantly, adding an Event Condition changes the meaning of the Rate Limit.
If an Event Condition is not defined—in other words, if "HTTP request" is selected—then as discussed above, an internal counter is maintained for each Count Condition value, and incremented each time that value is encountered in a request.
If an Event Condition is defined—in other words, if something other than "HTTP request" is selected—an internal counter is maintained for each Count Condition Value, and incremented each time a new, previously unobserved Event Condition value is encountered in a request.
Therefore, if an Event Condition is defined, the Rate Limit constrains the number of allowable Event Condition values for any given Count Condition value.
So, the evaluation becomes something like this:
"More than <THRESHOLD> <EVENT-CONDITION-VALUE> <EVENT-CONDITION-FIELD>s per any one <COUNT-CONDITION-VALUE><COUNT-CONDITION-FIELD> sent to <ASSIGNED-LOCATION> within <TTL-PERIOD> will cause <ACTION>."
Note that the number of Count Condition values is not being limited here. The limit is on the number of Event Condition values that each Count Condition value is allowed.
Example: Let's say we want to allow an individual user to login from a maximum of two ASNs within one hour. (Perhaps the user is accessing our web application from a coffee shop's WiFi, and then a few minutes later, leaves the coffee shop and begins using the cell network instead.) We want to allow this possibility; however, if we receive requests from the same user originating from three or more ASNs within an hour, we want to treat this traffic more suspiciously. This is not possible merely by specifying two Count Conditions, as described earlier in the "Multiple Conditions" section. If we set up two conditions ("Argument / Username" and "Attribute / Organization") with a Threshold of 2, and assign it to our login form, then this will merely limit the number of times that the user can login from each ASN within an hour. Instead, we can set up one Count Condition ("Argument / Username") and then set up the Event Condition ("Attribute / Organization"). Now the Threshold will apply to the number of Organizations that are observed for each specific Username.
When an incoming request exceeds the Threshold, the Action specified here will occur.
Most of the Actions listed above will not fully exclude an attacker that continues pressing the attack.
Example: Access to a login form is rate-limited to three requests per minute. An attacker tries to brute-force the login, and sends 60 requests per minute. The Rate Limit allows the first three requests, but then blocks the next 57 requests with a 503 error. However, after the minute has passed, the Rate Limit resets. The attacker is allowed another three attempts before being temporarily blocked again. This cycle can continue for as long as the attacker wishes. In effect, the Rate Limit is not preventing the attack; it is merely slowing it from 60 attempts per minute down to three attempts per minute.
The Ban action can be used to block (or take some other Action in response to) a Rate Limit violator for an extended period of time.
Example: As described above, a Rate Limit is created to allow three requests per minute, with an Action of 503 Service Unavailable. However, an additional Rate Limit rule is also defined: nine requests per three minutes, with an Action of Ban. The Ban has an Action of 503 Service Unavailable, and a duration of one hour. URL Maps allow for multiple Rate Limits to be assigned to a single URL. Thus, both of the above rules can be assigned to the login form. Now an attacker tries to brute-force the login form, sending 60 requests per minute. The first three requests are allowed. The next six requests are blocked (and a 503 error is issued) by the first Rate Limit. The tenth request triggers the second Rate Limit, and the Ban occurs. For the next hour, the attacker's requests will be blocked with a 503 error.
Second example: A hostile bot receives a bot challenge, which it fails. Curiefense will block the request. If the bot keeps re-submitting its request, it will continue to fail the challenges. However, each time the bot tries again, Curiefense has to issue a new challenge . To solve this problem, a second Rate Limit is created with a Ban action. Now a persistent bot will simply be Banned, saving Curiefense the overhead of issuing continuous challenges.
Note that when setting up a Ban, the most common choices for its Action are to deny the violator's requests (via 503 Service Unavailable, Response, or Redirect). However, you can also choose Tag Only (to observe the violator's actions during the ban period), Challenge (to verify that the violating activity is not being done by bots), or Header (to mark the requests for further scrutiny by the backend).
By default, an active Rate Limit rule will be enforced upon all incoming requests targeting URLs to which this rule has been assigned.
To change this behavior, you can add Include and/or Exclude parameters. These define the portion of the incoming traffic stream that will be evaluated for possible violation of the Rate Limit. In other words, they limit the scope of the Rate Limit's enforcement:
The Include filter will limit enforcement to requests matching its parameters. All other requests in the traffic stream will not have this Rate Limit enforced upon them.
The Exclude filter will exempt requests from enforcement that otherwise would have been subject to it.
(Internally, Curiefense evaluates Exclude parameters first, and then Include parameters.)
To add one or more filters, select New entry, define the parameters, then select the "+" button. If more than one Include filter is specified, they are combined with a logical AND.
The input controls at the top of this page are described here: Policies & Rules Entry Administration. Specific editing of a WAF Profile is described below.
A WAF Profile is a set of security policies that are used by the Curiefense WAF (Web Application Firewall). Every deployment includes a default WAF Profile, and additional Profiles can be created.
Every URL that Curiefense protects has a WAF Profile assigned to it in URL Maps. (If none is assigned explicitly, the default is used.) A request sent to a URL might, or might not, be filtered according to the assigned Profile.
The request was blocked before WAF filtering would have occurred. (Before the WAF is used, several other stages of filtering occur first.)
The applicable ACL Profile resulted in an Action of Bypass, which exempts the request from WAF filtering.
The WAF is not in Active Mode for this URL.
At the top of the page, the following values are defined for incoming requests.
By default, an incoming request will be compared to all the WAF Rules. If any parameter (any header, cookie, or argument within it) fails this evaluation, the request will be blocked.
However, parameters can be whitelisted and exempted from this filtering. For each parameter, this can be done in two ways:
Full exemption is available by specifying a regex pattern which, if it matches the parameter's value, will exempt that parameter from WAF Rule evaluation.
Partial exemption is available by specifying a list of WAF Rules which will not be evaluated, even if the regex pattern is not matched.
Along with this content whitelisting, a "positive security" form of content filtering is also available. Curiefense can be configured to require certain content in a specified parameter, and reject requests that do not contain it.
The bottom part of the UI defines Curiefense's behavior for both whitelisting and content filtering for each parameter.
In the following discussion, a constraint refers to the values in the UI input controls (Parameter, Matching Value, Restrict?, and Exclude WAF Rule) that have been specified for one parameter.
Each incoming request is processed like this:
This behavior is defined in the following fields in the UI.
Constraints are defined in the same way for Headers, Cookies, and Arguments within their respective tabs.
The input controls at the top of this page are described here: Policies & Rules Entry Administration. Specific editing of a Session Flow Control entry is described below.
The Session Flow Control module blocks hostile activity based on defined sequences of session flow.
Threat actors usually behave quite differently than legitimate users. For speed and efficiency, they tend to deviate from normal patterns of activity. The Session Flow Control capabilities of Curiefense allow you to define the expected patterns of behavior, and block access attempts that deviate from them.
For example, when a legitimate user attempts to log into a web application, the initial access of the login page will generate a GET request. Subsequently, a POST request will arrive with the login credentials.
However, a hostile bot that's attempting a credential stuffing attack has no need to issue a GET, and often, will not bother to do so. Therefore, if a POST request arrives that was not preceded by a GET, this is anomalous behavior, and Curiefense can block it.
These parameters define the sequence of requests that will be enforced. The sequence consists of several sequence sections. They must be fulfilled in the order defined here.
By default, a new sequence contains two sections. Additional sections can be added by selecting the "Create new sequence section" button.
A request will fulfill this Sequence Section if it matches all of these parameters:
The HTTP method specified in Method
The domain or host specified in Host
The path specified in Path
And the optional parameters, if any. Optional parameters can be added by selecting the "+" button; each parameter includes matching characteristics for a header, cookie, or argument.
Curiefense maintains its security parameters in Documents, which contain Entries. (Read more about Curiefense's data structures.)
The Policies & Rules section is where Documents and their Entries are created, defined, and administered via the UI.
Curiefense processes incoming requests according to this traffic flow:
Each incoming request is inspected, and tags are assigned to it. For example, if the request's IP was found on the Spamhaus DROP list, it could be assigned a "spamhaus" tag. Some tags are generated automatically, while others are user-defined. (Read more about tags.)
Next, Rate Limits are enforced.
Then, Session Flow Control is enforced.
Next, Curiefense determines the security ruleset(s) that have been assigned to the request's target URL, and which match the tags. For example, there might be a ruleset defined for the "spamhaus" tag, or for the "devops" tag.
Curiefense then enforces the ruleset(s), and takes the defined Action(s). For example, "block requests from Spamhaus-listed IPs", or "bypass devops requests from further filtering."
This process is based on the Documents as follows:
Tag Rules is the Document which defines tags for external lists and custom lists.
Rate Limits and Flow Control parameters define session-based policies.
ACL Policies and WAF Policies define the actions to take when specific tags and/or other criteria are observed. URL Maps assign these actions to internal URLs.
This page is divided into three vertical sections. From top to bottom, they are:
Administration
Entry editing
Versioning
Each is discussed below.
After editing anything on this page, you must save your changes (with the Save button on the upper right) and then publish them.
The top section contains a toolbar with input controls. On the left, there are these:
Configuration pulldown: Selects the branch/configuration for editing.
Document pulldown: Selects the Document to display for editing.
Download button: Downloads the currently displayed parameters.
On the right are these:
Entry pulldown: Selects the Entry (the ruleset) that is being displayed for editing.
Duplicate button: Makes a copy of the currently displayed Entry.
Add button: Creates a new Entry of the currently selected type.
Save button: Saves all changes that were made since the last Save action.
Delete button. Deletes the currently selected Entry.
The UI in this section will vary, depending on the type of Document being edited. Each is discussed in more depth here:
At the bottom of this section, the URL of the current entry is shown in gray. It reflects the structure of the data.
For example, /conf/api/v1/configs/devops/d/urlmaps/e/__default__
shows that:
The current Configuration (or branch) is devops.
The current Document is urlmaps.
The current Entry is __default__.
The bottom section of this page shows a history of versions for this Document.
To revert this Document to a previous version, hover the cursor over the end of its listing. As shown above, a button will appear; selecting it will restore that version.
Reversions are also available in the API and in the Publish Configuration section of the UI.
An ACL (Access Control List) Policy is a set of sequential criteria by which a request is evaluated. The input controls at the top of this page are described here: . Specific editing of an ACL Policy is described below.
This page defines ACL Policies. The Policies are assigned to URLs within the web application via .
Out of the box, Curiefense includes a default ACL Policy. It can (and usually should) be edited, but it cannot be deleted. It is used for all URLs where no other Policy has been assigned.
In the interface, the tags are listed below their associated actions. Evaluation is performed from left to right:
The incoming request is evaluated to see if it matches any of the tags in the far left column.
If a match is found, that column's action is performed, and no further evaluation is done.
If a match is not found, then the next column to the right is evaluated. A match will trigger that column's action; otherwise the next column is evaluated.
This process continues until an action is taken or all columns have been evaluated.
If a request does not match any tags in this Policy, it is passed to the WAF for further inspection.
Be very cautious when adding the "all" tag to an action column. In most situations, that action will be taken for all requests for which a match was not found in any columns to the left.
Another possible use is to place "all" in the Deny column, to create a positive security model. (This will block all requests which did not explicitly meet any of the conditions to be Bypassed or Allowed.) But even this usage can be dangerous. Unless all the conditions for Bypassing or Allowing requests are fully and correctly defined, this can result in False Positive errors, and some legitimate requests will be blocked and filtered.
In other columns, "all" can have serious consequences. For example, placing it in the Enforce Deny column will block all incoming traffic to which this ACL Policy is applied. On the other hand, placing it in the Bypass column will allow all incoming traffic (and exempt it from being scrubbed by the WAF!), except for those requests which matched a tag in the Enforce Deny column.
Summary: before using the "all" tag in an ACL Policy, carefully consider its ramifications.
This section displays all the WAF Rules defined within Curiefense. This has various uses; for example, looking up the underlying signature for a tag that appears in the Access Log, or getting the ID of a Rule to exclude in a WAF Policy.
Curiefense comes with many WAF Rules. You can view them by choosing different IDs in the pulldown on the upper right.
Note that this list is for reference only. The entries cannot be edited.
The top part of the interface shows the ACL Policy that is currently being edited. Each Policy consists of one or more . Each tag is listed under the action which will be triggered when a request matches that particular tag.
An ACL Policy that blocks requests via the Enforce Deny Action will only apply to the paths/URLs in the where this ACL Policy has been assigned. If you want to block requests globally for specific Tags, this can be done by defining the appropriate within the relevant .
The most common use is the "Deny Bot" column, as noted above. This means that all requestors (except for those previously Allowed or Bypassed) will be and verified to be humans.