Installation on some Linux distributions can be troublesome, here are a few things to ensure when running on linux.
Unable to create native thread
You might get an error like this: java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
in the logs - this is not an indication that Ceptor has run out of memory (despite the name of the exception) but rather that a new thread cannot be created because a limit of processes, tasks or maximum file handles has been reached.
See below for information on how to fix these kind of problems:
Setting ulimit flags correctly
Default limits for maximum file handles are often too low for most services on some linux distribution.
You can view current limits by running ulimit -a
to display current settings.
These limits can be changed by running ulimit -n
to change max number of file handles (open files, sockets etc.) and ulimit -u
to change the limits for processes/threads.
You can also edit the file /etc/security/limits.conf
and change the settings permanently there - refer to the documentation for your particular linux distribution for details.
Below is an example of setting the values for all users (replace * with specific username that you use to launch Ceptor)
* soft nproc 8192 * hard nproc 8192 * soft nofile 30000 * hard nofile 30000
In this example the maximum number of processes/threads is set to 8192 and max number of filehandles to 30000 - your needs will vary so change them according to load.
Pay special attention to number of concurrently open socket connections to clients - on very heavily loaded systems you might need the number of filehandles increased beyond these example values.
When installing from RPM or DEB images
When installing from RPM or DEB images, a service (pplauncher_service) is automatically created and set to start as a systemd service
This service is limited by systemd settings that can affect it - e.g. the DefaultMaxTasks=512 will cause it to fail since no more than 512 threads can be started across all processes within the same service.
One way to fix this, is to edit /etc/systemd/system.conf
and set DefaultTasksMax=4096
- another is to edit the service file for the pplauncher directly /etc/systemd/system/pplauncher_service.service
and set TasksMax=4096
within it in the [Service] section.
Ceptor versions newer than 6.5.3 will set this value by default.