Ceptor - Get a Free Trial

Linux Troubleshooting

Owned by Kim Rasmussen
Last updated: May 23, 2022
2 min read

Installation on some Linux distributions can be troublesome, here are a few things to ensure when running on linux.

Unable to create native thread

You might get an error like this: java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached in the logs - this is not an indication that Ceptor has run out of memory (despite the name of the exception) but rather that a new thread cannot be created because a limit of processes, tasks or maximum file handles has been reached.

See below for information on how to fix these kind of problems:

Setting ulimit flags correctly

Default limits for maximum file handles are often too low for most services on some linux distribution.

You can view current limits by running ulimit -a to display current settings.

These limits can be changed by running ulimit -n to change max number of file handles (open files, sockets etc.) and ulimit -u to change the limits for processes/threads.

You can also edit the file /etc/security/limits.conf and change the settings permanently there - refer to the documentation for your particular linux distribution for details.

Below is an example of setting the values for all users (replace * with specific username that you use to launch Ceptor)

* soft nproc 8192 * hard nproc 8192 * soft nofile 30000 * hard nofile 30000

In this example the maximum number of processes/threads is set to 8192 and max number of filehandles to 30000 - your needs will vary so change them according to load.

Pay special attention to number of concurrently open socket connections to clients - on very heavily loaded systems you might need the number of filehandles increased beyond these example values.

When installing from RPM or DEB images

Before upgrading RPM or DEP images make sure to backup your entire installation first.

Due to the nature of RPM/DEP installations, configuration files are overwritten with default value so ensure that you save any changes you have made.

Backups are available in the configuration-backup subdirectory but ensure that you have saved the most recent configuration before updating.

In general we recommend using the .sh installer which guides you through the installation and gives you options for customizing what to install, overwrite etc.

When installing from RPM or DEB images, a service (pplauncher_service) is automatically created and set to start as a systemd service

This service is limited by systemd settings that can affect it - e.g. the DefaultMaxTasks=512 will cause it to fail since no more than 512 threads can be started across all processes within the same service.

One way to fix this, is to edit /etc/systemd/system.conf and set DefaultTasksMax=4096 - another is to edit the service file for the pplauncher directly /etc/systemd/system/pplauncher_service.service and set TasksMax=4096 within it in the [Service] section.

Ceptor versions newer than 6.5.3 will set this value by default.

 

© Ceptor ApS. All Rights Reserved.