Ceptor - Get a Free Trial

Scripts and Macros

Owned by Kim Rasmussen
Last updated: Jan 26, 2021 by Kim Rasmussen (Unlicensed)
12 min read

Javascript

Javascript code is executed to generate a resulting value. Full scripts can be executed, or a macro can contain script code which is run.

A script is assumed, if the value is prefixed with %{script} then the entire rest of the value is executed as a script.
When using the GUI to enter values, a radio button allows you to select which language to use for scripting in the editor handles prefixing the string, and changing the editor to javascript mode for syntax hiliting.

If you start a string with %{file} then the actual contents of the script will be loaded from the file (meaning the file needs to exist on the gateway where it is executed).

Groovy scripts

Like with javascript, you can use groovy scripts to execute code - if the value is prefixed with %{script:groovy} then the rest of the value is executed as a groovy script. You can find more information about the groovy language here: http://groovy-lang.org/documentation.html and this page: http://groovy-lang.org/differences.html contains a quick list of the main differences between groovy and java languages.

Python scripts

By prefixing with %{script:python} you can execute python code - note that "jython" (http://www.jython.org/) is used for executing python scripts within the gateway - you can find very good information on integration with java here: https://wiki.python.org/jython/UserGuide

Request context

Script code has access to a variable called context which is of an instance of the class StateHolder - see Plugins for a description of its contents. It contains access to all data that the gateway uses internally, so it allows you to do anything (and also to mess up anything so be careful about what you do). 

Remember the saying - With Great Power comes Great Responsibility - take care what you do and ensure that you do not do stupid things like calling eval() on a string originating from outside, or constructing code dynamically based upon input from a browser query field.

If you e.g. call context.macro(context.getQueryOrPostParam('country')) and someone posts country with the value "%{script}java.lang.System.exit(1)" then you have just created a remote code execution vulnerability.

As long as you do not trust input from the outside, you are perfectly safe.


Global variables

All script code has access to a java.util.concurrent.ConcurrentHashMap called sharedState in which they can store data between requests - this data is globally shared between all scripts and all requests, so it can be used for caching data.

Request attributes within the context (accessed via context.requestAttributes is a hashMap containing request specific attributes - this is local for the request, and disappears once the request is finished.

All scripts run within separate thread contexts, so any global variables a script assigns and reuses between runs are not shared between threads.

This is to ensure separation of requests so ensuring that scripts that are not thread-safe will work anyway when executed concurrently by multiple threads.
If your scripts contain variables that must be shared between threads, such as cached values, place them in the sharedState variable which is accessible from all scripts, for all requests and all script languages.

Return values

A script can return the value it generates, it can be e.g. true or false for scripts that run inside a condition, or it can be a string for a script that outputs an HTTP header value. In some cases (like when using python scripts that call a function to generate a result) it is easier to assign the result to a variable, to support this, Ceptor's script engine checks the return value from the script - if that is null, it will look in the variable "__result__" (two underscores + the word result + two underscores) - and if that exists after executing the script - its value will be used.

Using macros from scripts

If you need to use a macro from a script, you can call context.macro("xxxxx") - e.g. context.macro("https://%{HTTP_HOST}/myurl") will parse the macro and replace %{HTTP_HOST} with the appropriate value.

A word on performance

Executing scripts is significantly slower (usually several orders of magnitude slower) than using macros - so try to keep things simple and use macros whenever you can.

Ceptor precompiles the scripts on the first run, and reuses the precompiled version - When using Python scripts, this compilation can take several seconds on the first startup, since the Jython engine is quite expensive to start.

Javascript tends to be faster than both python and groovy after tends of thousands invocations once the JIT has optimized it

Literal values

If you (like with %{script}) prefix the string with %{literal} then the rest of the string is taken literally, meaning it is not parsed or processed in any way.
Note that this requires minimum version 5.66 to work.

Functions to call from scripts

These are some of the built-in functions that can be called from your scripts;

contex.respond(<http response code>, <http response reason>, <content type>, <body>)

Sends a response to the client, the response body can be a String, a byte[] or a ByteBuffer.


context.gateway.sendAndLogError(context, <http response code>, <http response reason>, <exception text>);
context.gateway.sendAndLogError(context, <http response code>, <http response reason>, <java throwable>);

Sends an error response back to the client, the exception text / throwable will appear in the access log, and a warning will be logged in the system log with error send back.


context.gateway.sendError(context, <http response code>, <http response reason>, <exception text>);
context.gateway.sendError(context, <http response code>, <http response reason>, <java throwable>);

Sends an error response back to the client, the exception text / throwable will appear in the access log, no error/warning will be logged in the system log.


context.gateway.isFromSSLAccelerator(context);

Returns true if request went through a configured SSL accelerator, false if not.


context.gateway.getCanonicalPath(context);

Returns the canonical path - where /./ and /../ are canonicalized and resolved/removed.


context.gateway.getEncodedCertificates(context, forceRenegotiateFlag);

Return the client SSL certificate - if forceRenegotiateFlag is set to true, SSL renegotiation will be done.


context.gateway.isSecure(context);

Returns true if request is considered secure (meaning HTTPS was used, or request passed through SSL accelerator where HTTPS was used).


context.gateway.getGeoIP(context);

Returns LocatorInfo object, containing GeoIP information about the client. Note that this call blocks while looking GeoIP info up in the database via the session controller, so it can take a few milliseconds to resolve.


context.gateway.isIPMatchingRange(context, range name, ip address);

Checks if the provided IP address is within the configured IP Range or not


context.isResponseStartedOrSent();

Returns true if a response is already started or sent - if a response has started, the HTTP headers cannot be modified since they are already sent to the client.


context.getQueryOrPostParam(<parameter name>);

Returns the value of the POST or query parameter - note that POST parameter content is only available if a previous location has been configured to read the request content / POST data.


context.getSessionVariable(<variable name>);

Returns the value (or null if not found) of the state variable in the Ceptor Session.

context.proxyTo(long maxRequestTimeMsecs, Config.Destination destination);
context.proxyTo(long maxRequestTimeMsecs, String destinationName);
context.proxyToServer(long maxRequestTimeMsecs, String scheme, String hostname, int port);
context.proxyToServer(long maxRequestTimeMsecs, String scheme, String hostname, int port, boolean ignoreSSLCert); // Requires minimum Ceptor v6.4.1
context.proxyToServer(long maxRequestTimeMsecs, String scheme, String hostname, int port, boolean ignoreSSLCert, int timeoutSeconds); // Requires minimum Ceptor v6.4.8
context.proxyToServer(long maxRequestTimeMsecs, String scheme, String hostname, int port, boolean ignoreSSLCert, int timeoutSeconds, String templateJSON); // Requires minimum Ceptor v6.4.8


Proxy requests to a server immediately.

When proxying from a script, all other actions for a location not yet completed are aborted and not done - so it depends on where you invoke it.

Since a location script is run as one of the first things when processing a location, any headers, authorization, authentication checks etc. that would otherwise occur later will not be done if proxying is invoked from a location script.
If proxying is invoked from e.g. an authorization script however, that is one of the last actions done as part of processing a location, so anything else (authentication, parameter checking, http header modification etc.) will already be done or queued up if proxying is invoked from such a script.

If in doubt, make sure all actions needed are done from an earlier location before you have a script which starts proxying programmatically.

Also note that compression of responses is not done when proxying is initated from a script.



context.agent

Allows access to the java API for Ceptor Agent - see Ceptor Agent for Java

When accessing state variables from within a session, note that by default not all state variables are visible to gateway/dispatcher agents. These agents typically use a "restricted" connection to the session controller - this means that certain content is removed to reduce the risk of compromising sensitive data in case a gateway machine in a DMZ zone is compromised for any reason.

To make state variables visible for gateways, add them to safeStateVariables in the session controller configuration - see Session Controller Properties for more details.


context.trace.trace(<message>);
context.trace.trace(<message>, <exception>);

Allows access to the trace, if Gateway Tracing is enabled for the request. Call with message and eventual exception.


context.addToMDC(key, value);

Stores a value in the SLF4J MDC which is then available for logging (requires minimum Ceptor v6.0.1).


context.getDeployedAPIsForLocation(<location>);

/**
* Returns the list of deployed APIs for a given location - often <code>currentLocation</code> can be used to choose the current active location - otherwise you can get to locations via <code>config.locationList</code>
* 
* @param location Location to return APIs from
* 
* @return List of API versions currently deployed
*/
public List<Config.APIManagement.APIVersion> getDeployedAPIsForLocation(Config.Location location);

Returns the list of deployed APIs for a given location - often currentLocation can be used to choose the current active location - otherwise you can get to locations via config.locationList

This can be used to find which APIs are available for use within a Location - i.e. the ones that are deployed in the given environment.

Config.APIManagement.APIVersion useful attributes:
/** API Version ID */
public String id;
/** Name of API Version */
public String name;
/** API Version description */
public String description;
/** Base Path */
public String basePath;
/** Type of API - wsdl, openapi or plainhttp */
public io.ceptor.apimanager.Config.APIType type;
/** For openapi's this contains the swagger/openapi spec */
public JSONObject openAPISpec;
/** Contains the wsdl for soap APIs */
public String wsdl;
/** List of environments this API is deployed in */
public Set<String> deployedEnvironments;
/** Name of API Group this API version is within */
public String apiGroupName;
/** The ID of the API containing this version */
public String apiId;
/** The name of the API containing this API version */
public String apiName;
/** Printable name containing both the API Group, API and API version naming */
public String prettyName;


/** True, if CORS is enabled */
public boolean corsEnabled;
/** List of header parameter names contained by all operations within this API */
public Set<String> headerParameterNames;


/** True, if subscriptions are required to access this API */
public boolean subscriptionRequired;
			
/** If OpenAPI spec is loaded from remote, this is the URL it is loaded from */
public String remoteUrl;
/** Time of last update (milliseconds since 1/1 1970 */
public long remoteLastUpdatedAt;
/** Exception that occured if loading of the remote spec failed */
public Throwable lastRemoteLoadError;

/** Full API Version JSON */
public JSONObject json;


Examples

Examples
// Send response to client
context.gateway.sendResponse(context, 200, "OK", "application/json", '{"status": "ok"}');

// Send error back to client, error will be logged in access log and a warning in the system log
context.gateway.sendAndLogError(context, 403, "Access denied", "Didn't provide the magic password");

var locator = context.gateway.getGeoIP(context);
if (locator) {
  var countryCode = locator.getCountryCode();
  var countryName = locator.getCountryName();
  var city = locator.getCity();
  var postalCode = locator.getPostalCode();
  var region = locator.getRegion();
  var latitude = locator.getLatitude();
  var longitude = locator.gteLongitude();
  var isp = locator.getISP();
  var organisation = locator.getOrganization();
  var ip = locator.getIP();
  // Distance in kilometers from central london
  var distanceInKilometers = locator.distanceFrom(51.509865, -0.118092);
}

// If gateway trace is enabled, add this text to the trace
context.trace.trace("Some tracing information here");


Macros

Macros such as %{HTTP_HOST} are replaced with the contents they represent - in this case, the contents of the Host HTTP header sent from the client.
a macro is in the format %{<name>} or %{<prefix>:<name>}
The following prefixes exist:

  • base64 - base64 the value of the macro name.
  • urlencode - urlencode the value of the macro name.
  • htmlencode - htmlencode the value of the macro name.
  • requestcookie - Lookup the request cookie.
  • requestheader - Lookup the request HTTP header.
  • responseheader - Replace with response HTTP header.
  • requestattribute - Replace with named request attribute value, set in the location.
  • responsecookie - Replace with value of response cookie as set from server.
  • statevariable - Replace with state variable from Users Session.
  • query - Replace with the value of the specified query parameter.
  • compreq.cancelled - true, if the completed request was cancelled, e.g. because of timeout, and false if not.
  • compreq.exception - If the completed request failed with an exception, this contains the toString() output from the exception.
  • compreq.response - The response for the completed request, converted to a string.
  • compreq.response.code - Response code for a completed request, e.g. 200.
  • compreq.response.reason - Response reason for a completed request, e.g. OK.
  • compreq.response.headers - A JSON object containing all the response headers.
  • compreq.response.header - The header value (or JSON array of values) of the response header - the name of the header stake from the # in the name, e.g. %{compreq.response.header:sample#Content-Length} returns the Content-Length header for the completed response named "sample".

For information about compreq.* prefixes refer to the documentation for the RetrieveURL Task (also called "Task - Call service / URL").

and the following macros exist

  • HTTP_USER_AGENT - User Agent HTTP request header.
  • HTTP_REFERER - Referer HTTP request header.
  • HTTP_COOKIE - Cookie HTTP request header.
  • HTTP_FORWARDED - Forwarded HTTP request header.
  • HTTP_HOST - Host HTTP request header.
  • HTTP_HOST_NOPORT - Host HTTP request header, but without the port number if one was present.
  • HTTP_SCHEME - Request scheme (https/http).
  • HTTP_PROXY_CONNECTION - Proxy-Connection HTTP request header.
  • HTTP_ACCEPT - Accept HTTP request header.
  • REMOTE_ADDR - Remote Client IP address.
  • REMOTE_PORT - Remote Client TCP port.
  • REQUEST_ID_BASICAUTH - Request (Session and Transaction) ID and "password" usable in basichauth header.
  • REQUEST_ID - Request ID containing Session ID and Transaction ID.
  • SESSION_ID - Session ID.
  • REMOTE_USER - Userid of remote user, if authenticated - empty if not.
  • REMOTE_IDENT - Userid of remote user, if authenticated - empty if not.
  • REQUEST_METHOD HTTP request method-
  • REQUEST_PATH - Path in the request, without the query string.
  • QUERY_STRING - HTTP request query string.
  • PATH_WITH_QUERY - HTTP request path and query string.
  • AUTH_TYPE - ID of authentication plugin which authenticated the user.
  • AUTH_LEVEL - Authentication Level - the higher the level, the stronger authentication.
  • SESSION_USERNAME - Name of the authenticated user, if any.
  • SESSION_ISAUTHENTICATED - true if the user is authenticated, false if not.
  • SESSION_ISINTERNAL - true if the user is an internal user, false if not.
  • SESSION_AGREEMENTID - The agreement ID from the session, if set.
  • SESSION_CUSTOMERID - The customer ID from the session, if set.
  • SESSION_GROUPS - Comma separated list of user groups from the session, or null if no groups.
  • GATEWAY_NAME - Name of gateway as it is specified in server configuration - e.g. gateway1.
  • SERVER_NAME - Hostname of interface on the gateway on which the request was received (if hostname is specified on listen settings - otherwise IP).
  • SERVER_ADDR - Local IP address of server interface which the request was received on.
  • SERVER_PORT - TCP port the gateway received the request on.
  • SERVER_PROTOCOL - HTTP Protocol name and version - e.g. HTTP/1.1
  • SERVER_PROTOCOL_VERSION - HTTP Protocol version - e.g. 1.1 or 2.0
  • SERVER_SOFTWARE - Name of server software.
  • SERVER_SOFTWARE_VERSION - Version of server software.
  • THE_REQUEST - Entire request with HTTP method, path, query params and protocol.
  • REQUEST_URL - Request URL with scheme, hostname, path and parameters.
  • REQUEST_URI - Request URI with path and parameters.
  • REQUEST_BODY - The prefetched request body - only available if a location has "Preload Request content" set for a location.
  • HTTPS - "on" if request is secure, "off" if not.
  • TIME_YEAR - Current year.
  • TIME_MON - Current month.
  • TIME_DAY - Current day of month.
  • TIME_HOUR - Current hour.
  • TIME_MIN - Current minute.
  • TIME_SEC - Current second.
  • TIME_WDAY - Current day of week.
  • TIME - Request intiation time in format: "EEE, dd MMM yyyy HH:mm:ss z"
  • TIME_LOGFORMAT - Current time in common log format: "[dd/MMM/yyyy:HH:mm:ss Z]"
  • GEOIP_ISP - Client geoip ISP.
  • GEOIP_ORGANIZATION - Client geoip organization.
  • GEOIP_COUNTRYCODE - Client geoip country code.
  • GEOIP_COUNTRYNAME - Client geoip country name.
  • GEOIP_CITY - Client geoip city.
  • GEOIP_POSTALCODE - Client geoip postal code.
  • GEOIP_REGION - Client geoip region.
  • GEOIP_LATITUDE - Client geoip latitude.
  • GEOIP_LONGITUDE - Client geoip longitude.
  • ORIGINAL_REQUEST - This contain the entire original first line of the request before any eventual rewrite.
  • ORIGINAL_REQUEST_URL - Request URL before any eventual rewrite.
  • ORIGINAL_REQUEST_PATH - Request Path (decoded and canonicalized) before any eventual rewrite.
  • ORIGINAL_REQUEST_URI - Request URI (without query string) not decode, as it was before any eventual rewrite.
  • ORIGINAL_REQUEST_SCHEME - Request scheme, before any eventual rewrite.
  • ORIGINAL_HOST - Request host, before any eventual rewrite.
  • EXCEPTION_LOG - empty if no exception, otherwise a new line, and the full exception with stacktrace.
  • GENERATE_FORWARDED_HEADER - Generates an RFC7239 HTTP Forwarded header - including any eventual existing header that was already present in the request

Examples:
%{requestcookie:sessionid} for the request cookie named sessionid.
%{HTTP_HOST} for the HTTP Host request header.
%{urlencode:REQUEST_PATH} for an URL encoded version of the request path.

Macros for Logging

The following macros are only valid after the response has been written, so they are mostly useful for access logging:

  • HTTP_RESPONSECODE - The HTTP response code written.
  • HTTP_BYTESSENT - The number of bytes sent, excluding the HTTP header.
  • HTTP_RESPONSETIME - HTTP Response time in milliseconds.

Macros specific for API Gateway

These macros are for use with the API Gateway.

  • API_VERSION_ID - ID of the API Version that was called.
  • API_ID - ID of the API that the API Version is inside.
  • API_PARTNER_ID - ID of the API Partner.
  • API_PARTNER_NAME - Name of API Partner.
  • API_PARTNER_APP_ID - API Partner Application ID.
  • API_PARTNER_APP_NAME - API Partner Application Name.

Special macros

In addition to the regular macros, there are also two special macros

%{rewrite:....} macro

Rewrite is used to change a value into another value by rewriting it if it matches a specified regular expression.
It is in this format:

%{rewrite:_name_;"_regex";"_flags_";"_value_"}

_name_ is replaced with a macro name, e.g. REMOTE_USER or responseheader:location.
_flags is replaced with flags as input to regular expression construction, these flags exist: (refer to java.util.regex.Pattern javadoc for details)

  • UNIX_LINES - Unix lines mode.
  • CASE_INSENSITIVE - Case insensitive comparison.
  • COMMENTS - Comments in pattern are ignored.
  • MULTILINE - Enable multiline mode.
  • LITERAL - Enables literal parsing of the pattern.
  • DOTALL - In dotall mode, the expression . matches any character, including a line terminator.
  • UNICODE_CASE - Enables Unicode-aware case folding.
  • CANON_EQ - Enables canonical equivalence.
  • UNICODE_CHARACTER_CLASS - Enables the Unicode version of Predefined character classes and POSIX character classes.

_value_ is replaced by the value that the regular expression is rewritten to.
Example:

%{rewrite:responseheader:location;"http\:\/\/(.*)$";"CASE_INSENSITIVE";"https://$1"}

This example rewrites the response header to https://xxxxx if it matches the pattern http://xxxxx - notice that $1 is used in the value which is replaced by the regex expression in the first set of parenthesis - please refer to the documentation on java regex patterns for additional information.

%{script:.....} macro

The script macro is used to embed javascript code into expressions for more complex expression handling that the one available in the macros

Example:
%{script:if (context.httpExchange.getRequestHeaders().getFirst('Content-Type') == 'application/json') 'application/json'; else 'text/html';} evaluates to application/json if the request Content-Type header was set to the same, and text/html otherwise.

Another example:
%{script:if (context.httpExchange.getRequestHeaders().getFirst('Content-Type') == 'application/json') '{"error":"access.denied"\}'; else '<html><head><title>No access</title><body><h1>Invalid userid/password</h1></body></html>';} returns an html or json error message depending on the request content-type.

Note that when running scripts embedded within the %{script:.....} macro, only the javascript language can be used - to run python or groovy scripts, use %{script:python}..... or %{script:groovy}..... instead.

© Ceptor ApS. All Rights Reserved.