What is a Pipeline

A pipeline is an ordered collection of tasks executed in sequence. This can be used to chain together tasks that are executed in order, e.g. call a service, transform its output from XML to JSON and return the response.

What is a Task

A task is an action performed within a pipeline - many different tasks exists, ranging from branches (if...then...else) task, to validations, convertions, logging, tracing to service callouts, proxying requests or returning responses.

Tasks can be anything from doing base64 or html encoding/decoding of a block of text to a call to a service to retrieve data, or run custom scripts to e.g. merge the output of multiple separate service calls with each other. There are also tasks for performing validation of input requests or output data, writing data to the log or transformations of data format between XML and JSON.

Example

Example flow of tasks within a pipeline.

In this example, we have a pipeline with a number of tasks, first we call a service, then update the value of a request attribute with the content of the HTTP response headers, after this, we transfer from JSON to XML, and then in case of failure we send a failure response, otherwise we update the JSON content by running a script (here, we add the XML as an attribute in the response) which we then send in the final task.

This is just one example - any number of tasks can be executed within the pipeline, and other pipelines can be called from within one to allow reuse of often-used sequence of tasks.

Here is a screenshot of how these tasks are edited in the Ceptor Console. You can reorder tasks by dragging them around in the list, and right-click them to add/remove more. You can also drag new tasks into the pipeline from the Task Palette.

Are tasks always needed ?

No, tasks are not always needed - you might be able to perform similar custom actions in an authentication or authorization script within the gateway, but tasks are often a more elegant and more readable way of archiving results.
Also, while a script needs to block waiting for external events to complete, a task can be asynchronous, so some operations especially those that do IO work will perform better when split into async tasks.

List of tasks

The following tasks are available:

Branches

• If...then...else
  Allows you to branch out, based upon a list of configured conditions - if conditions match, one branch of tasks are executed, if not - another branch is executed instead.
• If failure then... else
  If any of the previous tasks has failed with an error (unless the error has been cleared) then execute X tasks, if not, execute Y other tasks.

Transformations

• Base64 encode/decode
  Base64 encode or decode any value - result is stored in a request attribute.
• URL encode/decode
  Allows you to URL encode/decode any value, variable or macro - result is stored in a request attribute.
• HTML encode
  Allows you to HTML encode a text - escapes any HTML characters ensuring the result is safe to display in a web page.
• Transform JSON to XML
  Transforms JSON into XML tags.
• Transform XML to JSON
  Transforms XML into JSON, note that some information will be lost in transit, such as attributes, order and namespaces.
• Transform JSONML to XML
  Transforms from JSON Markup Language into XML - see http://www.jsonml.org/ for information on JSON ML
• Transform XML to JSONML
  Transform from XML into JSON Markup Language - this transformation can be reverted back into JSON without loss of data, but note that the format is quite verbose and may take up a lot of memory space.

Attributes

• Set Request Attribute
  Sets or deletes the value of a request attribute to a specific value or the result of a macro/script.
• Clear Exception / Failure
  Clears any exception/failure that might previously have occurred during task execution.

Connectivity

• Call service / URL
  Perform an HTTP request to call a service, or retrieve data from a resource. Calls can be made to arbitrary URLs or to servers configured in Destinations within the gateway. The result of the call, e.g. the data read from the remote server is stored in a request attribute.
• Send Response
  Sends a response back to the client - note that this is the last task that can be executed within a pipeline - once done, no further processing is done for this request.
• Proxy to Destination
  Proxy the request onwards to a destination server - like "Send Response", this is the last task in the pipeline that can be executed. This is equivalent to proxying to a destination from within a Location.

Validations

• Validate JSON format
  Allows you to do validation on JSON structure, such as ensuring valid JSON content, defining maximum size, limiting number or depth of elements, array size, key or value sizes etc.
• Validate XML format
  Allows you to do validation an XML, such as ensuring valid XML content, defining maximum size, limiting number or depth of elements, attribute count, element or attribute name/value etc.
  You can also specify multiple XSD schemas which the XML is validated against, the XSD schemas can be embedded in the configuration or loaded from URLs. When loading for URLs you can specify a refresh interval after which they will be reloaded to allow for changes to be automatically used.

Actions

• Execute script
  Execute a script - allows the script to do any processing it wants by running custom code.
• Call pipeline
  Call another pipeline and run through all its tasks - this allows you to chain pipelines and reuse collections of tasks from other pipelines.

Logging

• Write to Trace
  If request trace is enabled within the gateway, this writes a value to it
• Write to Log
  Writes a value to the log at the given log level - logger name can also be specified.
• Write to Transaction Log
  Allows you to write a message to the transaction log - this log resides at the Ceptor Session Controller - implemented by individual authentication plugins so writing here is a remote call.