Kbase P149086: Performance issues with SQL92 clients since installing 10.1B0347 Hot Fix.
| Autor |
Progress Software Corporation - Progress |
| Acesso |
Público |
| Publicação |
7/17/2009 |
|
Status: Unverified
SYMPTOM(s):
Performance issues with SQL92 clients since installing 10.1B0347 Hot Fix.
Lots of context switching occurring sporadically.
When running 10.1B0347 Hot Fix, promon gather script showing 1.4 million waits on semaphore when _sqlsrv2 processes go runaway.
_sqlsrv2 server processes going runaway
_sqlsrv2 processes are the processes that are doing most of the context switching.
_sqlsrv2 processes that are runaway are using 100% of CPU
_sqlsrv2 processes that are runaway do not have any users connected to them according to the database log file.
_sqlsrv2 processes that are runaway do not have any users connected to them according to promon.
Database log file showing that the _sqlsrv2 processes that are runaway have a pending user connection.
Broker process is cleaning up the pending user connection but users still cannot connect to these _sqlsrv2 processes.
_sqlsrv2 processes that have gone runaway have 8 threads associated with the them according to Task Manager
Userdump output from _sqlsrv2 process that is runaway is showing that the process is looping and then waiting on a socket every 5 seconds for multiple threads.
Terminating the rogue _sqlsrv2 processes with promon does not resolve the performance issue.
After terminating rogue _sqlsrv2 processes, new _sqlsrv2 processes are spawned and users immediately begin to pend for these _sqlsrv2 processes.
Issue occurs on multiple server machines sporadically.
Issue occurs on source database server machine
Issue occurs on target database server machine
Issue does not occur on source and target database server machines at the same time.
Running 10.1B0347 Hot Fix
When running 10.1B0325 Hot Fix, promon gather script showing 1100 - 1400 waits on semaphore when _sqlsrv2 processes go runaway.
When running 10.1B0325 Hot Fix, _sqlsrv2 processes still go runaway but only have 4 threads when they have gone runaway.
With 10.1B0325 can terminated the rogue _sqlsrv2 processes using promon and performance returns to normal.
FACT(s) (Environment):
Fathom Replication is being used at this site.
Multiple CPUs installed on server machines.
Has the fix for PendConnTime for _sqlsrv2 processes so user count is decremented appropriately when a client cannot connect to the _sqlsrv2 process.
Conmgr.properties file confirms that the database makes use of secondary login brokers.
The secondary login broker is configured to have 1 client per _sqlsrv2 process.
OpenEdge Category: Database
OpenEdge Database Category: Performance
Windows
Windows Server 2003
OpenEdge 10.1B03 Service Pack
CAUSE:
Bug# OE00187265
CAUSE:
The _sqlsrv2 process is spawning threads for clients that should be connecting to these _sqlsrv2 processes. When clients are not able to connect to the _sqlsrv2 process due to some 3rd party application or networking issue, the threads the _sqlsrv2 process spawned for these clients continue to remain active. These threads will wait on a socket and then loop every 5 seconds looking for any incoming messages. This looping and waiting for the non-utilized threads may be causing the context switching. This is a suspicion at this point in time. The exact cause is unknown at the time of this writing.
FIX:
Workaround:
1. Stop all OpenEdge processes and services.
2. Remove 10.1B0347 Hot fix from the machine either by deleting the directory, or renaming the directory to something else.
3. Restore Operating System backup of the OpenEdge installation (in this case 10.1B0325) that existed prior to installing the 10.1B0347 hotfix.
4. Terminate the rogue _sqlsrv2 processes using promon as the runaway _sqlsrv2 processes occur.