Rate Limits are rules which define the number of requests with certain characteristics that are allowed within defined time frames. When a request is received that exceeds a Rate Limit, a specified action is performed.
The input controls at the top of this page are described here: Policies & Rules Entry Administration. Specific editing of a Rate Limit is described below.
A Rate Limit defines the number of times that requests can match certain conditions within a certain time frame. Once that limit has been reached, subsequent requests matching those conditions within the same time frame will trigger an action.
The matching conditions are specified with the parameters in the "Count by" section and the optional Event. See Matching Conditions below.
By default, a Rate Limit will be enforced for all requests for the URL(s) to which it is assigned.
Enforcement can be further limited to a subset of these requests: see Limiting the scope of a Rate Limit below.
A name that will be used within the Curiefense interface, and will also be used to create a tag that will be assigned to requests which trigger the Rate Limit. It is recommended that the name summarizes the rule; for example, a rule with a Threshold of 5 and a TTL of 60 could be named "Rate limit rule 5/60".
A description that will be used within the Curiefense interface.
The maximum number of allowable requests within the specified TTL. Subsequent requests within the TTL will trigger the Action.
Time to Live: the period (specified in seconds) within which the Threshold is enforced.
A condition consists of a field and a value. Within a Rate Limit, they play a role like this:
"More than <THRESHOLD> requests with the same <CONDITION-VALUE> <CONDITION-FIELD> sent to <ASSIGNED-LOCATION> within <TTL-PERIOD> will cause <ACTION>."
A condition can be built upon any one of these four categories:
All requests with the same value for the specified header will be counted together toward the Threshold.
All requests with the same value for the specified cookie will be counted together toward the Threshold.
All requests with the same value for the specified argument will be counted together toward the Threshold.
All requests with the same value for the specified attribute will be counted together toward the Threshold.
Multiple conditions can be defined within the same Rate Limit. To create a new condition and open it for editing, select "New entry" below the list of conditions.
If multiple conditions are defined, they are evaluated by combining them together with a logical AND. In other words, the cumulative count toward the Threshold will be incremented whenever a request is seen that matches all of the conditions simultaneously. Different combinations of conditions will have separate Threshold counts maintained for them.
Below the list of condition(s), there is another condition named "Event."
By default, this is set to "HTTP request,", which simply means to increment a counter each time a request is received that matches the conditions.
However, if the Event condition is changed to a different value, then the following applies.
Adding an Event Condition changes the evaluation process. An Event Condition is not logically combined with the preceding Count Condition; it is always evaluated separately.
More importantly, adding an Event Condition changes the meaning of the Rate Limit.
If an Event Condition is not defined—in other words, if "HTTP request" is selected—then as discussed above, an internal counter is maintained for each Count Condition value, and incremented each time that value is encountered in a request.
If an Event Condition is defined—in other words, if something other than "HTTP request" is selected—an internal counter is maintained for each Count Condition Value, and incremented each time a new, previously unobserved Event Condition value is encountered in a request.
Therefore, if an Event Condition is defined, the Rate Limit constrains the number of allowable Event Condition values for any given Count Condition value.
So, the evaluation becomes something like this:
"More than <THRESHOLD> <EVENT-CONDITION-VALUE> <EVENT-CONDITION-FIELD>s per any one <COUNT-CONDITION-VALUE><COUNT-CONDITION-FIELD> sent to <ASSIGNED-LOCATION> within <TTL-PERIOD> will cause <ACTION>."
Note that the number of Count Condition values is not being limited here. The limit is on the number of Event Condition values that each Count Condition value is allowed.
When an incoming request exceeds the Threshold, the Action specified here will occur.
503 Service Unavailable
The request will be blocked and the requestor will receive a response of "503 Service Unavailable".
For a browser-based web application, a bot challenge will be issued to verify that the requestor is a human using a browser, and not a bot using a headless browser or emulator. If the challenge is failed, the request is blocked.
The request will not be blocked; it will merely be tagged with the Rate Limit's name, for subsequent viewing in the Access Log and other places. This Action is useful for testing new Rate Limit rules without actually affecting incoming traffic.
Blocks the request, and responds with a custom error code (0-999) and response body.
Blocks the request with a custom error code, and redirects the requestor to a specified URL. For example, the URL might be a page that says, "Your activity appears suspicious, and your access has been restricted. Contact support if you think this decision was made in error."
Blocks the requestor for the specified amount of time. See further discussion below.
Does not block the request, but adds headers to it (indicating the Rate Limit rule name and the threshold) for receipt and evaluation by the user's backend.
Most of the Actions listed above will not fully exclude an attacker that continues pressing the attack.
The Ban action can be used to block (or take some other Action in response to) a Rate Limit violator for an extended period of time.
Note that when setting up a Ban, the most common choices for its Action are to deny the violator's requests (via 503 Service Unavailable, Response, or Redirect). However, you can also choose Tag Only (to observe the violator's actions during the ban period), Challenge (to verify that the violating activity is not being done by bots), or Header (to mark the requests for further scrutiny by the backend).
By default, an active Rate Limit rule will be enforced upon all incoming requests targeting URLs to which this rule has been assigned.
To change this behavior, you can add Include and/or Exclude parameters. These define the portion of the incoming traffic stream that will be evaluated for possible violation of the Rate Limit. In other words, they limit the scope of the Rate Limit's enforcement:
The Include filter will limit enforcement to requests matching its parameters. All other requests in the traffic stream will not have this Rate Limit enforced upon them.
The Exclude filter will exempt requests from enforcement that otherwise would have been subject to it.
(Internally, Curiefense evaluates Exclude parameters first, and then Include parameters.)
To add one or more filters, select New entry, define the parameters, then select the "+" button. If more than one Include filter is specified, they are combined with a logical AND.