On most sites the using Oracle Applications the common practice of an Oracle Apps DBA is to check the Unix Process of the Internal Concurrent Manager using the command:
ps –ef | grep CPMGR
or
ps –ef | grep FNDCPMBR
applmgr 24988 24984 0 Dec 26 ? 5:16 FNDLIBR FND CPMGR FNDCPMBR sysmgr="" sleep=30 logfile=/Ora_base/thor/rpt/co
applmgr 20050 20040 0 Dec 16 ? 3:30 FNDLIBR FND CPMGR FNDCPMBR sysmgr="" sleep=30 logfile=/Ora_base/thor/rpt/co
applmgr 19900 19889 0 Dec 16 ? 4:53 FNDLIBR FND CPMGR FNDCPMBR sysmgr="" sleep=30 logfile=/Ora_base/thor/rpt/co
applmgr 22629 22624 0 Dec 16 ? 0:56 FNDLIBR FND CPMGR FNDCPMBR sysmgr="" sleep=30 logfile=/Ora_base/thor/applmg
applmgr 24757 24753 0 Dec 26 ? 5:04 FNDLIBR FND CPMGR FNDCPMBR sysmgr="" sleep=30 logfile=/Ora_base/thor/rpt/co
Well, either way the result of the output will be the same, because the search string exists in the output. If you have a single instance, then you would get just one output and that would help to know that the Internal Concurrent Manager Process is up (Note: Not necessarily running or active).
In case you have different releases of Oracle Apps on the same server and if the output of the command above is as shown, then it would be difficult to find out which background process belongs to which Application.
The goal of this Paper is to find out not only if the background process is alive, but also to find out if the Internal Manager is really active and to find the same across releases. Note different releases have different ways of addressing this problem, which you would see as we go along.
According to Oracle Notes:
Internal Concurrent Manager's function is only to run 'queue control' requests, which are requests to
startup or shutdown other managers. It is responsible for startup and shutdown of the whole
concurrent processing facility, and it also monitors the other managers periodically, and restarts them
if they should go down. It can also take over the Conflict Resolution manager's job, and
resolve incompatibilities. If the ICM itself should go down, requests will continue to run normally,
except for 'queue control' requests. If the ICM should go down, you can restart it with 'startmgr'.
You do not need to kill the other managers first.
From the above note it is clear that, if the Internal Concurrent Manager is not active, then the background process would still exist and might become a zombie. The Activation of the other managers and failed managers would not happen and the normal symptom would be the Conflict Resolution Manager piling up all the requests. An example of which would be when the Internal Manager goes down with an error :
APP-01167 Internal Concurrent Manager has encountered an error.
This error would be in the ORACLE_SID.mgr file in the directory $APPLCSF/$APPLLOG , where all the Manager information log goes into, provided you have set the environment correctly in the APPLSYS.env file. This error is logged when an Internal Manager encounters various errors and deactivates the Internal Concurrent Manager and sets its Active process to zero.
You can check if the the Active process is zero by logging in to the Application and navigate to,
Concurrent Manager Administor
You would see that the Internal Manager’s Active Process would be zero and the Target Process would be one. The background CPMGR related process would be running, so this does not reflect the correct status of the Internal Concurrent Manager.
You can also check which process belongs to which database by picking the OS Process ID ORACLE_SID.mgr file in the directory $APPLCSF/$APPLLOG , where the OS Process ID gets logged when the Internal Manager comes up but it is more work, example logfile output,
========================================================================
Starting dev3 internal concurrent manager -- shell process ID 24984
logfile=/Ora_base/thor/rpt/common/logdev3/dev3.mgr
PRINTER=ioo341
mailto=applmgr
restart=N
diag=N
sleep=30
pmon=20 (default)
quesiz=1 (default)
+---------------------------------------------------------------------------+
ps -ef | grep 24984
applmgr 24988 24984 0 Dec 26 ? 5:20 FNDLIBR FND CPMGR FNDCPMBR sysmgr="" sleep=30 ogfile=/Ora_base/thor/rpt/co
From the above command we know that the Internal Manager Process exists and is running for Database DEV3, but does not show dev3 anywhere at the backend when you grep the PPID or the PID. You can then verify the Internal Manager from the front-end application to check if it is running.
This shows that it is imperative that we not only check the Internal Manager Background process, but also to make sure that the Actual Process of the Internal Manager is not set to zero and running.
From the above ps –ef | grep CPMGR command we got an output of Five Internal Manager Background process and we have to now figure out which process belongs to which database, which is essentially our goal.
10.7 SC/ NCA
In this release the Oracle Home was common for both the database and the Application, so the Internal Concurrent Manager background process was always spawned by a parent process that had an ORACLE_SID attached to it. We could easily check which process belongs to which database by comparing the parent process with the Internal Manager process like:
ps -ef | grep -v grep | grep FNDCPMBR | awk '{print $2}' | while read CMPID
do
ps -ef | nawk -v CMPPID=$CMPID '{if (CMPPID == $3) {print $0} }' | grep oracle
$ORACLE_SID >/dev/null
if [ $? == 0 ] ; then
echo "Conc Mgr for database=$ORACLE_SID is UP"
Here we check the background process only and presume that the Internal Manager is up and running and incase its Active process gets set to zero due to errors, it doesn’t raise a flag, only the users and developers would call up to say that all the request are clogging.
10.7 NCA Server Partition Mode:
In the server partition mode, the parent process drops off once the Internal Manager Process is spawned, so it becomes difficult to compare like we did in the previous script to find out which process belongs to which database. We have to manually check the log file and get the OS Process ID for the Internal Manager and then do a grep on the Process ID. Then use the Unix “ pstree ID“ command to trace the process and pick the parent ID and its corresponding ORACLE_SID.
The better way to do that would be to log on to the Application and pick the same OS Process ID of the Internal Concurrent Manager that gets logged in the $APPLCSF/$APPLLOG/ORACLE_SID.mgr file from the database by querying the table FND_CONCURRENT_PROCESSES. The table registers the OS Process ID in the field OS_PROCESS_ID and we can also check the Running_Processes to find out if the Actual Process is not equal to zero.
Once the OS Process Id and the active Process is checked we can then pick the corresponding ORACLE_PORCESS_ID from the same table and compare it with the SID in the dynamic view v$session. The FND_CONCURENT_QUEUE would identify if the process is the Internal Manager Process, and hence it would check comparing the APPLICATION_ID’s. The paddr of the v$session for the SID would then be compared with the addr of V$Process and the corresponding SPID can be retrieved. This SPID would be the spawned parent ID that would have the ORACLE_SID attached to it.
The script below exactly does the same:
********************************** CUT HERE **********************************************
#!/bin/ksh
#===========================================================================
# Filename: icmtest107
# Author : Shankar Govindan.
# Created : 12/25/2001
# Good for : Release 107 Server Partition Mode only
# Description : This script is used for checking out the status of Internal Manager in Oracle APPS,
# It picks up the OS_PROCESS_ID from FND_CONCURRENT_PROCESSES, spools
# them and then looks for the actual OS process with its corresponding ORACLE_SID.
# The grep looks for SPID and LOCAL = NO because of Oracle version 7x.
# Note : I have used a password file with system/manager which is in my /dba/etc dir.
#
# 1st Argument: status
# 2nd Argument: ORACLE_SID (optional)
#
#===========================================================================
if ( [ "$1" != "status" ] )
then
echo "Usage: $0 status "
echo " Eg.: $0 status OSID"
exit
fi
CMD=$1
OSID=$2
ORACLE_SID=`echo $ORACLE_SID`
TEMPDIR=`pwd`
GROUP=`groups | cut -d" " -f1`
function _icmtestcmd
{
echo "==========================================================================="
echo "checking Internal concurrent Manager status for Database ${ORACLE_SID} "
date
echo "==========================================================================="
echo
PWD_FILE=/dba/etc/.${GROUP}_${ORACLE_SID}.pwd
SYSTEM_PWD=`grep -i "^system/" $PWD_FILE`
case $CMD in
status)
if [ "$SYSTEM_PWD" = "" ] ; then
echo "\nERROR: userid=system does not exist in ${PWD_FILE}"
return 1
fi
# echo "** Sending status for Oracle database=${ORACLE_SID}"
(echo ${SYSTEM_PWD}; echo set pages 0 pause off verify off feedback off termout off; echo "select p
.spid from v\$process p, v\$session s, apps.fnd_concurrent_processes f, apps.fnd_concurrent_queues
q where p.addr = s.paddr and s.sid = f.oracle_process_id and f.concurrent_queue_id = q.concurrent_q
ueue_id and q.concurrent_queue_name = 'Internal Manager' and q.running_processes = 1 group by p.spi
d order by 1 ;") | sqlplus -s > $TEMPDIR/icmppid
ps -ef | grep -i `cat $TEMPDIR/icmppid` |grep NO | while read LINE
do
MATCH=`echo $LINE | awk '{print $9}'`
if [ "$MATCH" = "oracle${ORACLE_SID}" ] ; then
echo "For Database=$ORACLE_SID , Internal Concurrent Manager is UP and RUNNING !!"
echo
return 1
echo
else
echo "Database=$ORACLE_SID Internal Concurrent Manager is DOWN"
echo
fi
done
return 0
;;
*) echo "\nERROR: Invalid option=$CMD (in function _icmtestcmd)"
return 2
;;
esac
}
#
#MAIN
#
if [ "$CMD" = "status" ];
then
_icmtestcmd $CMD $OSID
else
echo "Check the Command again "
wait
return 1
fi
********************************** CUT HERE **********************************************
Oracle Apps Release 11 :
In Release 11, the setup is similar to 10.7 NCA without the server partition mode, the old scripts would hold good, but it does not the check the active process from the database to compare and hence the script of 10.7 NCA server Partition Mode would be a better one.
Although the concept for getting the information is the same, the stored information in the AOL tables has changed from 10.7 to Rel 11. The FND_CONCURRENT_QUEUES table does not store the CONCURRENT_QUEUE_NAME = ‘Internal Manager’ but as ‘FNDICM’, so we pick the information from the view FND_CONCURENT_QUEUE_VL.USER_CONCURRENT_QUEUE_NAME = ‘Internal Concurrent Manager ‘ and all the other info from the same view. The rest of the information are retrieved just like the above script used for 10.7 NCA Server Partition Mode.
In the Rel 11 we use Net 8 Adapter and according to Oracle,
The BEQUEATH adapter is essentially a true Net8 adapter though it has one unique distinction in
that it does not connect to Oracle kernel via a Listener process, it connects directly to the kernel.
Oracle 8 applications use the BEQUEATH adapter by default though you can specify a TNS
alias for BEQUEATH. The BEQUEATH adapter has been around since Oracle 7.3.3. However,
there were problems using 7.3.3. and Net 2.3.3 to connect to a local Oracle 8 kernel. These
were addressed in the 7.3.4/2.3.4 release.
In 10.7 Sever Partition Mode script we grep not only for the Process ID, but also grep for LOCAL=NO, since that identifies the connectivity for the process. In Release 11 we grep LOCAL = YES because of the above explanation for Net 8 adapter for Oracle 8 Kernel, the connection would look like:
oracle 22637 22633 0 Dec 16 ? 0:04 oracledev4 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))
The script used for Release 11 Internal Manager status check:
************************************* Cut Here *********************************************
#!/bin/ksh
#===========================================================================
# Filename: icmtest_rel11
# Author : Shankar Govindan.
# Created : 01/04/2002
# Good for : Release 11 only
# Description : This script is used for checking out the status of Internal Manager in Oracle APPS,
# It picks up the OS_PROCESS_ID from FND_CONCURRENT_PROCESSES, spools them
# and then looks for the actual OS process with its corresponding ORACLE_SID.
# The grep looks for SPID and LOCAL = YES because of REL 11 SqlNet beq default mode.
# Note : I have used a password file with system/manager.
#
# 1st Argument: status
# 2nd Argument: ORACLE_SID (optional)
#
#===========================================================================
if ( [ "$1" != "status" ] )
then
echo "Usage: $0 status "
echo " Eg.: $0 status "
exit
fi
CMD=$1
OSID=$2
ORACLE_SID=`echo $ORACLE_SID`
TEMPDIR=`pwd`
GROUP=`groups | cut -d" " -f1`
function icmtestcmd_rel11
{
echo "==========================================================================="
echo "checking Internal concurrent Manager status for Database ${ORACLE_SID} "
date
echo "==========================================================================="
echo
PWD_FILE=/dba/etc/.${GROUP}_${ORACLE_SID}.pwd
SYSTEM_PWD=`grep -i "^system/" $PWD_FILE`
case $CMD in
status)
if [ "$SYSTEM_PWD" = "" ] ; then
echo "\nERROR: userid=system does not exist in ${PWD_FILE}"
return 1
fi
# echo "** Sending status for Oracle database=${ORACLE_SID}"
(echo ${SYSTEM_PWD}; echo set pages 0 pause off verify off feedback off termout off; echo "select p
.spid from v\$process p, v\$session s, apps.fnd_concurrent_processes f, apps.fnd_concurrent_queues_
vl q where p.addr = s.paddr and s.audsid = f.session_id and f.concurrent_queue_id = q.concurrent_qu
eue_id and f.queue_application_id = q.application_id and q.user_concurrent_queue_name = 'Internal M
anager' and q.manager_type = f.manager_type and q.running_processes = 1 group by p.spid order by 1
;") | sqlplus -s > $TEMPDIR/icmppid
ps -ef | grep -i `cat $TEMPDIR/icmppid` | grep YES | while read LINE
do
MATCH=`echo $LINE | awk '{print $9}'`
if [ "$MATCH" = "oracle${ORACLE_SID}" ] ; then
echo "For Database=$ORACLE_SID , Internal Concurrent Manager is UP and RUNNING !!"
echo
return 1
echo
else
echo "Database=$ORACLE_SID Internal Cocnurrent Manager is DOWN"
echo
fi
done
return 0
;;
*) echo "\nERROR: Invalid option=$CMD (in function _icmtestcmd)"
return 2
;;
esac
}
#
#MAIN
#
if [ "$CMD" = "status" ];
then
icmtestcmd_rel11 $CMD $OSID
else
echo " Check your command "
wait
return 1
fi
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment