Ceptor - Get a Free Trial

Getting Started with Ceptor API Management

Owned by Kim Rasmussen
Last updated: Jul 05, 2022
14 min read

Prerequisites

Ceptor API Management is an addon to Ceptor - see Ceptor Getting Started for general information.

APIs are defined in the Ceptor Console, or by using the Management APIs.

Installation

The default distribution contains example configuration and setup that uses Ceptor.

API Management is not installed separately but included in the core functionality - however it needs to be configured.

Initial Configuration

If you need to upgrade an existing Ceptor configuration without API Management support to one that supports API Management, you will need to do the following changes:

  • Optionally add Apache Ignite to ceptor_launch.xml
  • Optionally add ignite configuration to ceptor-configuration.xml
  • Add datastore sections to ceptor-configuration.xml
  • Decide whether to store API data in ignite or a database
  • Add apimanagement section to ceptor-configuration.xml
  • Add API Gateway to location within Ceptor Gateway

Optionally add Apache Ignite to ceptor_launch.xml

Using Ignite

Using Apache Ignite is deprecated and not encouraged due to potential stability issues with the Ignite cluster, please use a database instead.

In ceptor_launch.xml, you need to add the following XMLs if you want to use Apache Ignite for API Usage, Rate limiting or for storing access/refresh tokens.

	<!-- Apache Ignite -->
	<jvm name="ignite" vmargs="-Xmx2048M -Djava.awt.headless=true -Xnoclassgc -XX:+HeapDumpOnOutOfMemoryError -XX:+ExitOnOutOfMemoryError -XX:MaxDirectMemorySize=2G -XX:+DisableExplicitGC -DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true" systemclasspath="">
		<config servers="loadbalance:nios://localhost:21233?validateservercert=false;nios://localhost:21234?validateservercert=false" />
		<!-- Split into multiple classloaders -->
		<classloader name="ig">
			<service name="ignite1" launcherclass="io.ceptor.ignite.IgniteLauncher">
			</service>
		</classloader>
	</jvm>
	<!-- Apache Ignite -->
	<jvm name="ignite2" vmargs="-Xmx2048M -Djava.awt.headless=true -Xnoclassgc -XX:+HeapDumpOnOutOfMemoryError -XX:+ExitOnOutOfMemoryError -XX:MaxDirectMemorySize=2G -XX:+DisableExplicitGC -DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true" systemclasspath="">
		<config servers="loadbalance:nios://localhost:21233?validateservercert=false;nios://localhost:21234?validateservercert=false" />
		<!-- Split into multiple classloaders -->
		<classloader name="ig">
			<service name="ignite2" launcherclass="io.ceptor.ignite.IgniteLauncher">
			</service>
		</classloader>
	</jvm>

This starts two Apache Ignite instances, named ignite1 and ignite2 - they will run in the same cluster.

You need to adjust the configuration according to your needs; e.g. increasing the memory size or starting the instances on multiple machines.

Optionally Add Ignite Configuration to ceptor-configuration.xml

You need to add the following to ceptor-configuration.xml to use Apache Ignite:

	<server name="ignite-server" type="abstract" description="Ignite cluster server">
		<group name="_JSON_" description="JSON configuration">
			<property name="ignite_JSON_" description="Ignite cluster configuration">
<![CDATA[{
  "ignite.init": "%{script}load(\"nashorn:mozilla_compat.js\");\nimportPackage(org.apache.ignite);\nimportPackage(org.apache.ignite.configuration);\n\ncontext.cfg.setIgniteInstanceName(input);\n",
  "ignite.iplist": ["127.0.0.1:47500..47501"]
}]]></property>
		</group>
		<group name="agent" description="Agent Settings">
			<property name="sessioncontrollers" value="" description="No session controllers"/>
		</group>
	</server>
	<server name="ignite1" type="ignite" description="Ignite server" extends="ignite-server">
	</server>
	<server name="ignite2" type="ignite" description="Ignite server" extends="ignite-server">
	</server>

this sets up configuration for the two newly added ignite servers - change ignite.iplist if you use multiple machines to cover all server nodes in the cluster, e.g.

"ignite.iplist": [
  "192.168.1.100:47500..47501",
  "192.168.1.101:47500..47501"
]

This assumes two instances (port 47500 to port 47501) running on both 192.168.1.100 and 192.168.1.101 - so 4 in total.

The script in ignite.init can be used to modify additional attributes in the ignite configuration, e.g. to change the directory that persistent data is stored within (defaults to ./work) - see https://apacheignite.readme.io/docs for details.

Add Datastore Sections to ceptor-configuration.xml

Now, you need to add datastore configuration to ceptor-configuration.xml, like this:

Datastore Configuration 
	<server name="datastore-ignite-persistent-partitioned" type="abstract" description="Settings for a common ignite persistent partitioned datastore" extends="">
		<group name="_JSON_" description="JSON configuration">
			<property name="datastore-ignite-persistent-partitioned_JSON_" description="Datastore related configuration and metadata">
<![CDATA[{
  "datastore": {
      "name": "Ignite partitioned datastore",
      "description": "Datastore implementation for a common partitioned ignite datastore",
      "factoryclass": "io.ceptor.datastore.ignite.IgniteDataStoreFactory",
      "configuration": {
          "replicated": "false",
  	      "ignite.init": "%{script}load(\"nashorn:mozilla_compat.js\");\nimportPackage(org.apache.ignite);\nimportPackage(org.apache.ignite.configuration);\n\ncontext.cfg.setIgniteInstanceName(input);\ncontext.cfg.setClientMode(true);\n",
          "ignite.iplist": ["127.0.0.1:47500..47501"]
       }
    }
}]]></property>
		</group>
	</server>
	<server name="datastore-ignite-persistent-replicated" type="abstract" description="Settings for a common ignite persistent replicated datastore" extends="">
		<group name="_JSON_" description="JSON configuration">
			<property name="datastore-ignite-persistent-replicated_JSON_" description="Datastore related configuration and metadata">
<![CDATA[{
  "datastore": {
      "name": "Ignite replicated datastore",
      "description": "Datastore implementation for a common replicated ignite datastore",
      "factoryclass": "io.ceptor.datastore.ignite.IgniteDataStoreFactory",
      "configuration": {
          "replicated": "true",
  	      "ignite.init": "%{script}load(\"nashorn:mozilla_compat.js\");\nimportPackage(org.apache.ignite);\nimportPackage(org.apache.ignite.configuration);\n\ncontext.cfg.setIgniteInstanceName(input);\ncontext.cfg.setClientMode(true);\n",
          "ignite.iplist": ["127.0.0.1:47500..47501"]
       }
    }
}]]></property>
		</group>
	</server>
	<server name="datastore-primary" type="abstract" description="Settings for a common derby datastore" extends="">
		<group name="_JSON_" description="JSON configuration">
			<property name="datastore-primary_JSON_" description="Datastore related configuration and metadata">
<![CDATA[{"datastore": {
  "name": "Derby sample datastore",
  "description": "Datastore implementation using a derby database",
  "factoryclass": "io.ceptor.datastore.db.JDBCDataStoreFactory",
  "configuration": {
    "databasetype": "DERBY",
    "pool": {
      "db.checkexecuteupdate": true,
      "db.connectionurl": "jdbc:derby://localhost:1527/datastore;create=true",
      "db.credentials": "password",
      "db.debug": false,
      "db.delaygetconnection": "0",
      "db.drivername": "org.apache.derby.jdbc.ClientDriver",
      "db.getconnectiontimeout": 60000,
      "db.initialpoolsize": 5,
      "db.maxconnectionlife": 600,
      "db.maxconnectionusage": 5000,
      "db.searchstartwithwildcard": false,
      "db.testonreserve": false,
      "db.username": "admin"
    }
  },
  "id": "datastore-primary"
}}]]></property>
		</group>
	</server>

Note again the values of ignite.iplist which is the list of servers in a cluster to connect to for each datastore definition. The two above, are for replicated and partitioned datastores (replicated means same data is replicated to all nodes in the cluster, partitioned means data is distributed across the cluster, with backup copies but each node does not have all data available).

The two above, are for setting up the Apache Ignite Client to connect to the servers we added earlier.

Now, you need to tell the various modules to use this configuration, so you need to change configuration for the configserver (which is used for APIs) and the session controller (used for authentication plugins, e.g. storing access tokens/refresh tokens, reading Partner information).

For the configuration servers, add extends="datastore-ignite-persistent-replicated" to get the configuration for replicated datastore added into the configuration server

<server name="configservers" type="abstract" description="config servers shared configuration" extends="datastore-primary,datastore-ignite-persistent-replicated">

and (again for the config servers, add this:)

<group name="apimanagement" description="API Management datastore">
	<property name="datastore-apimanagement" value="datastore-ignite-persistent-replicated" description=""/>
</group>

This selects the datastore implementation to use for API management.

To use database instead of Apache Ignite, add this instead of the above:

<group name="apimanagement" description="API Management datastore">
	<property name="datastore-apimanagement" value="datastore-primary" description=""/>
</group>

For the session controllers where the authentication plugins (e.g. JWT Authentication plugin using OAuth2 Ignite datastores) exists, you need similar changes:

<server name="sessioncontrollers" type="abstract" description="Session Controllers" extends="x509,eticket,ntlm,datastore-ignite-persistent-replicated,datastore-ignite-persistent-partitioned,datastore-primary">

and add this group to select a datastore:

<group name="apimanagement" description="API Management datastore">
	<property name="datastore-apimanagement" value="datastore-primary" description=""/>
</group>

If you use the Ignite Rate limiter in the gateway, you also need to let it extend from datastore-primary (or datastore-ignite-persistent-partitioned if you use Apache Ignite), like this:

<server name="gateways" type="abstract" description="gateway server" extends="datastore-primary">


Decide whether to store API data in ignite or a database

You can store the API data in a database instead of ignite. Ignite can still be used for non-permanent data like statistics, counters, tokens, etc in relation to API handling. If you want to store API management data in a database, you need to add an additional datastore to the ceptor-configuration.xml. Below is an example from the distribution, where it is setup for use with a Derby database. You can configure and use your own database as well. 

If you run into issues with your database - please contact support so we can verify these issues. The database datastore support have been tested towards DB/2, Oracle, MySQL and Derby at this point. But should work towards most industry standard databases.

Derby Datastore Configuration 
	<server name="datastore-primary" type="abstract" description="Settings for a common derby datastore" extends="">
		<group name="_JSON_" description="JSON configuration">
			<property name="datastore-primary_JSON_" description="Datastore related configuration and metadata">
<![CDATA[{
  "datastore": {
	  "name": "Derby sample datastore",
      "description": "Datastore implementation using a derby database",
      "factoryclass": "io.ceptor.datastore.db.JDBCDataStoreFactory",
      "configuration": {
          "databasetype": "default",          
           "pool": {
            "db.caching.acl.time": "300",
			"db.caching.authmethod.time": "300",
			"db.caching.group.time": "300",
			"db.caching.profile.time": "300",
			"db.caching.status.time": "300",
			"db.checkexecuteupdate": "true",
			"db.connectionurl": "jdbc:derby://localhost:1527/datastore;create=true",
			"db.credentials": "password",
			"db.debug": "false",
			"db.delaygetconnection": "0",
			"db.drivername": "org.apache.derby.jdbc.ClientDriver",
			"db.getconnectiontimeout": "60000",
			"db.initialpoolsize": "5",
			"db.maxconnectionlife": "600",
			"db.maxconnectionusage": "5000",
			"db.searchstartwithwildcard": "false",
			"db.testonreserve": "false",
			"db.testtable": "",
			"db.username": "admin"
           }      
       }
    }
}]]></property>
		</group>
	</server>

Another datastore example - here using Postgres:

PostgreSQL Datastore Configuration 
		<server name="datastore-primary" type="abstract" description="" extends="">
		<group name="_JSON_" description="JSON configuration">
			<property name="datastore-primary_JSON_" description="Datastore related configuration and metadata">
<![CDATA[{"datastore": {
  "name": "Postgres API Management",
  "description": "Datastore for API mgmtm in Postgres",
  "factoryclass": "io.ceptor.datastore.db.JDBCDataStoreFactory",
  "configuration": {
    "databasetype": "postgres",
    "pool": {
      "db.drivername": "org.postgresql.Driver",
      "db.connectionurl": "jdbc:postgresql://localhost:5432/apimgmt",
      "db.username": "apimgmt",
      "db.credentials": "<secret>",
      "db.initialpoolsize": 5,
      "db.getconnectiontimeout": 60000,
      "db.maxconnectionlife": 600,
      "db.maxconnectionusage": 5000,
      "db.testonreserve": false,
      "db.debug": false,
      "db.searchstartwithwildcard": false,
      "db.checkexecuteupdate": false
    }
  },
  "id": "datastore-primary"
}}]]></property>
		</group>
	</server>


Again - as mentioned above - for the configuration servers, the datastore needs to be added. Using the example above add extends="datastore-derby" to get the configuration for a database datastore added into the configuration server

<server name="configservers" type="abstract" description="config servers shared configuration" extends="datastore-priimary,datastore-ignite-persistent-replicated">

and for the API management configuration to use this datastore, add this::

<group name="apimanagement" description="API Management datastore">
	<property name="datastore-apimanagement" value="datastore-primary" description=""/>
</group>


For the sake of this example with the Ceptor distribution there is a derby launcher prepared in the pp-launch.xml that is used. So if the example above is used - please enable that launch configuration - or a similar derby server must be started.

Add apimanagement section to ceptor-configuration.xml

You can also add an API Management section to ceptor-configuration.xml - note that this step can be done from the Ceptor Console using the "API Management → Configuration" instead, so you only need to copy this configuration if you want to reuse our default environment, rate limit configurations and subscription plans.

Optional API Management configuration
	<server name="apimanagement" type="abstract" description="Settings for API management" extends="">
		<group name="_JSON_" description="JSON configuration">
			<property name="apimanagement_JSON_" description="API management related configuration and metadata">
<![CDATA[{
  "environments": [
    {
      "name": "Sandbox",
      "description": "Sandbox environment, used for initial testing of APIs, playing around with new versions",
      "baseurl": "https://localhost:8443/",
      "oauth2.authorizationurl": "https://localhost:8443/oauth2/auth",
      "oauth2.tokenurl": "https://localhost:8443/oauth2/token",
      "oauth2.refreshurl": "https://localhost:8443/oauth2/refresh",
      "openidconnect.discoveryurl": "https://localhost:8443/.well-known/openid-configuration"
    },
    {
      "name": "Test",
      "description": "Test environment, used as a stable test environment",
      "baseurl": "https://test.ceptor.io:8443"
    },
    {
      "name": "Preprod",
      "description": "Pre-Production environment",
      "baseurl": "https://api.ceptor.io"
    }
  ],
  "ratelimitgroups": [
    {
      "id": "7933225d-6593-4201-bf14-c848226c770b",
      "name": "Basic",
      "description": "Basic rate limit",
      "limits": [
        {
          "value": "100",
          "unit": "hour"
        },
        {
          "value": "5",
          "unit": "minute"
        }
      ]
    },
    {
      "id": "20bc445b-a29b-4fcf-90ac-f861a56ab9ad",
      "name": "Premium",
      "description": "Plan for premium customers",
      "limits": [
        {
          "value": "10000",
          "unit": "hour"
        },
        {
          "value": "100",
          "unit": "second"
        }
      ]
    },
    {
      "id": "c268c2ba-4d80-499f-a097-806b9c364ed6",
      "name": "Unlimited",
      "description": "No limits",
      "limits": []
    }
  ],
  "subscriptionplans": [
    {
      "id": "4841e9f5-2af5-4e42-ac1d-6b19be04c446",
      "name": "Free",
      "description": "Free plan",
      "ratelimitgroup": "7933225d-6593-4201-bf14-c848226c770b",
      "default": true
    },
    {
      "id": "109ec49f-2399-410a-8bd9-975a5e9d48a3",
      "name": "Premium",
      "description": "Subscription plan for premium customers",
      "ratelimitgroup": "20bc445b-a29b-4fcf-90ac-f861a56ab9ad"
    },
    {
      "id": "1feed83f-ed74-4c36-bb07-4ebf2220fd86",
      "name": "Unlimited",
      "ratelimitgroup": "c268c2ba-4d80-499f-a097-806b9c364ed6",
      "default": false
    }
  ]
}]]></property>
		</group>
	</server>

See API Management Configuration for information on how to set this up using the API Management Configuration within the Console

Add API Gateway to Location Within Ceptor Gateway

You can add this location within the gateway to serve APIs from the "Sandbox" environment.

Example API Gateway location
    {
      "name": "API Gateway - Sandbox",
      "description": "Serve any APIs defined in API Management which are deployed to the Sandbox environment",
      "content.preload": true,
      "session.needed": false,
      "response.compress": true,
      "conditions.type": "or",
      "conditions": [
        {
          "type": "header",
          "name": "Accept",
          "deny": false,
          "values": [
            "application/json*",
            "application/xml"
          ],
          "lowercase": true
        },
        {
          "type": "method",
          "deny": false,
          "lowercase": false,
          "values": ["OPTIONS"]
        }
      ],
      "apigateway": {
        "environment": "Sandbox",
        "failure": {
          "action": "respond",
          "response.status": 500
        },
        "failifnotfound": true,
        "apiusage": [
          "io.ceptor.gateway.analytics.log.APIUsageLog",
          "io.ceptor.gateway.analytics.stat.APIUsageStatistics"
        ],
        "ratelimiter": "io.ceptor.gateway.analytics.memory.RateLimiterMemory"
      },
      "location.enabled": true,
      "session.override": false,
      "action": "serveapi",
      "cookiesnapper": {},
      "plugin": {},
      "response.headers": [],
      "session.afterconditionsmatch": false
    },

See Setting up API Gateway for information on how to configure this using the Gateway configuration within the Console.

Session Resolver

Protecting access to these APIs is important - here is a copy from the default distribution of the session resolver setup for the internal gateway serving the Internal APIs

  "session": {
    "cookie.not.for.uri": "*.crl",
    "resolvers": ["io.ceptor.session.SessionResolverScript"],
    "cookie.no.cachecontrol.header.for": "*.crl|*.pdf",
    "http.cookiename": "sessionid",
    "https.cookiename": "sslsessionid",
    "cookie.path": "/",
    "sessionfixation.cookiename": "sslsessionid_sf",
    "cookie.use.httponly": true,
    "sessionfixation.addcookie": true,
    "sessionfixation.defense": true,
    "cookie.obfuscate": true,
    "cookie.use.domain": true,
    "cookie.samesite": "none",
    "resolve.script": "%{script:groovy}import dk.itp.security.passticket.PTException;\nimport io.undertow.util.Methods;\nimport io.undertow.util.Headers;\nimport io.undertow.util.HttpString;\n\nHttpString ACCESS_CONTROL_REQUEST_METHOD = HttpString.tryFromString(\"Access-Control-Request-Method\");\n\n// For useradmin API, just used the session ID in the query parameter, if present\nif (context.macro('%{REQUEST_PATH}').startsWith(\"/useradmin/\")) {\n    String sid = context.macro('%{query:session}');\n    if (sid?.trim()) {\n        try {\n            context.sessionId = new dk.itp.security.utils.UniqueId(sid);\n        } catch(Exception e) {\n            context.httpExchange.getResponseHeaders().add(io.undertow.util.HttpString.tryFromString(\"Access-Control-Allow-Origin\"), \"*\");\n            context.gateway.sendAndLogError(context, 401, \"Invalid session\", e)\n        }\n        return;\n    }\n}\n\nString authHeader = context.macro('%{requestheader:Authorization}');\n\nif (authHeader && authHeader.toLowerCase().startsWith(\"basic \")) {\n    try {\n        String sessionid = context.agent.getSessionFromTicket(54, authHeader.substring(6), context.config.gateway.segmentId, context.config.gateway.clusterId, context.gateway.getClientSourceIP(context), true);\n        if (sessionid) {\n            context.sessionId = new dk.itp.security.utils.UniqueId(sessionid);\n        }\n    } catch(PTException e) {\n        context.httpExchange.getResponseHeaders().add(io.undertow.util.HttpString.tryFromString(\"Access-Control-Allow-Origin\"), \"*\");\n        context.gateway.sendAndLogError(context, 401, \"Invalid credentials\", e)\n    }\n} else if (context.httpExchange.getRequestMethod() == Methods.OPTIONS && context.httpExchange.getRequestHeaders().contains(Headers.ORIGIN) && context.httpExchange.getRequestHeaders().contains(ACCESS_CONTROL_REQUEST_METHOD)) {\n    context.trace.trace(\"Allowing OPTIONS preflight request through without session\");\n} else {\n    context.httpExchange.getResponseHeaders().add(io.undertow.util.HttpString.tryFromString(\"Access-Control-Allow-Origin\"), \"*\");\n    context.httpExchange.getResponseHeaders().add(io.undertow.util.Headers.WWW_AUTHENTICATE, \"basic\");\n    context.gateway.sendAndLogError(context, 401, \"Authentication Required\", \"Basic auth required\")\n}"
  },

Here is the session resolver script as it looks in Groovy:

import dk.itp.security.passticket.PTException;
import io.undertow.util.Methods;
import io.undertow.util.Headers;
import io.undertow.util.HttpString;

HttpString ACCESS_CONTROL_REQUEST_METHOD = HttpString.tryFromString("Access-Control-Request-Method");

// For useradmin API, just used the session ID in the query parameter, if present
if (context.macro('%{REQUEST_PATH}').startsWith("/useradmin/")) {
    String sid = context.macro('%{query:session}');
    if (sid?.trim()) {
        try {
            context.sessionId = new dk.itp.security.utils.UniqueId(sid);
        } catch(Exception e) {
            context.httpExchange.getResponseHeaders().add(io.undertow.util.HttpString.tryFromString("Access-Control-Allow-Origin"), "*");
            context.gateway.sendAndLogError(context, 401, "Invalid session", e)
        }
        return;
    }
}

String authHeader = context.macro('%{requestheader:Authorization}');

if (authHeader && authHeader.toLowerCase().startsWith("basic ")) {
    try {
        String sessionid = context.agent.getSessionFromTicket(54, authHeader.substring(6), context.config.gateway.segmentId, context.config.gateway.clusterId, context.gateway.getClientSourceIP(context), true);
        if (sessionid) {
            context.sessionId = new dk.itp.security.utils.UniqueId(sessionid);
        }
    } catch(PTException e) {
        context.httpExchange.getResponseHeaders().add(io.undertow.util.HttpString.tryFromString("Access-Control-Allow-Origin"), "*");
        context.gateway.sendAndLogError(context, 401, "Invalid credentials", e)
    }
} else if (context.httpExchange.getRequestMethod() == Methods.OPTIONS && context.httpExchange.getRequestHeaders().contains(Headers.ORIGIN) && context.httpExchange.getRequestHeaders().contains(ACCESS_CONTROL_REQUEST_METHOD)) {
    context.trace.trace("Allowing OPTIONS preflight request through without session");
} else {
    context.httpExchange.getResponseHeaders().add(io.undertow.util.HttpString.tryFromString("Access-Control-Allow-Origin"), "*");
    context.httpExchange.getResponseHeaders().add(io.undertow.util.Headers.WWW_AUTHENTICATE, "basic");
    context.gateway.sendAndLogError(context, 401, "Authentication Required", "Basic auth required")
}

The script uses basic authentication and expects same users as the ones who can access the console - see the call in line 26 - for this to work, the authentication plugin; dk.itp.security.passticket.server.ConfigServerAuthenticationPlugin must be installed in the Session Controller.


Datastores

By default, APIs, Partners, Partner Applications and Developers (see Glossary for info) are stored in a Derby database, running on the local machine.

You can also choose to use Apache Ignite instead, but we do not recommend doing so due to Apache Ignite cluster instability issues.

Apache Ignite is an Open-Source high-performance distributed database / in-memory store, which Ceptor can use for things like storing OAuth2 access/refresh tokens, Rate-Limiting information, as well as API information.

It is possible to switch to different other datastores, such as a regular database, and we have an API available that allows you to customize it and replace with your preferred implementation.

Datastores are configured in Ceptor Console



© Ceptor ApS. All Rights Reserved.