Consultor Eletrônico

Status: Verified

SYMPTOM(s):

Stateless AppServer hung in a SENDING state due to java.lang.OutOfMemoryError: unable to create new native thread errors

AppServer Broker log shows AppServer Broker is running out of memory

Unhandled exception caught in L-8301. (8419)

java.lang.OutOfMemoryError: unable to create new native thread

FACT(s) (Environment):

OpenEdge 10.1B0309
Multiple AppServer Brokers are on the machine.
Issue is only being experienced on their main AppServer Broker where user logins are validated and where most of their data processing occurs.
Errors do not occur in Test environment
No persistent procedures are being ran on this Application Server.
Does not appear to be associated with a memory leak.
Sufficient memory available to support number of users and processes running on this machine.
swapinfo command shows that only 2% of swap space is being used at the time of the issue.
kcusage command showed history for the max_proc_thread exceeding its top value during the period where the AppServer Broker was issuing the java.lang.OutOfMemoryError: unable to create new native thread message.
OpenEdge 10.1B
HP-UX
OpenEdge 10.1B03 Service Pack

CHANGE:

Migrated 3rd party Vantage Application 8.03.404A from Test environment to Production environment.

CAUSE:

The AppServer Broker was attempting to exceed the maximum number of threads (max_thread_proc) kernel parameter that limits the number of threads a process can create.

FIX:

Increased the Operating System kernel parameter:

max_thread_proc - (specifies the maximum number of threads a single process is allowed to create).


Based on information from HP forums on Java tuning the kernel setting max_thread_proc is inter-related to other kernel settings and all should be reviewed when making a change.

Changes to any of the following kernel values may require all of the following to be re-evaluated.
maxusers
nproc
max_thread_proc
nkthread
nfile
maxfiles
maxfiles_lim
ncallout
maxdsiz