Question:Error:
========
After submitting the concurrent program, i am getting the status as inactive and no manager and when i clicked on diagnostics button its showing a message like
This request will not be processed because there are currently no managers running that can process it.
This is an error condition. There are one or more managers defined that can process this request that should be running at this time, but do not appear to be running.
Contact your system administrator and/or check the manager log files for errors.
Solution:
=========
to resolve that, just u simple startup concurrent manager by executing the ./adcmctl.sh script
Question:
=========
want to create a new “MANAGER”
Ans:
====
sysadmin guide for that
http://download-uk.oracle.com/docs/cd/B25516_11/current/html/docset.html
To figure out the issue.
1) Check your CRM is showing any jobs in conflicts.
2) Check for locks.
3)280295.1 REQUESTS.sql Script for Parent/Child Request IDs and Trace File IDs
ANALYZEPENDING.SQL - Analyze all Pending Requests Doc ID: Note:134033.1
Question :
===========
problem is My technical consultant has submitted two concurrent requests. But it is long time running. the status is running. There is nothing to find in the log file. Where can i know what is exact problem
Solution
========
are the concurrent requests are custom or default concurrent programs....did you check the concurrent managers status.....
(a) check the concurrent managers status
(b) check the concurrent manager logfiles
if the concurrent requests are custom...i mean....if it is the report based...check the report query and tune the query wether custom tables are having indexes or not.......check the schema statistics of apps schema
2:
Try enabling trace on the particular concurrent request,may be that should help to diagnose the problem?
Question:
=========
What is environment variable APPLCSF in Apps environment?
Ans:
==
If environment variable APPLCSF is set then u can
get the logfiles of concurrent managers under $APPLCSF/$APPLLOG.
If APPLCSF is not set, Request Log files will go to
$<MODULE>_TOP/$APPLLOG
and output files will go to
$<MODULE>_TOP/$APPLOUT
Here <MODULE_TOP> is GL_TOP, AP_TOP, AR_TOP etc etc.
QUESTION:
=========
How and why to configure PCP(Parallel Concurrent Processing)?
Answer:
========
PCP is nothing but a load balancing methodology for concurrent processing server. To load balance on concurrent processing server, this is required. In a multinode architecture, it is being used.
Check the following metalink notes
How to Set Up Parallel Concurrent Processing (PCP) in Apps 11i?
Doc ID: Note:388495.1
Oracle E-Business Suite Release 11i with 9i RAC: Installation and Configuration using AutoConfig
Doc ID: Note:279956.1
Question:
=========
CM is not working after cloning?
Answer
======
1.Bounce CM
2.check FND_NODES correctlly updated for concurent manager
3.if is there custom tops ..if so means...add those to any env file..after that set env
then Bounce the concurrnet managers..
Question:
=========
Concurrent manager "Inactive, No Manager"
1. We are facing this problem even when no other job is running
2. We dont face this problem on daily basis- it can happen sometimes thrice in a day and sometimes once in a month.
3. This program generally runs atleast 10 times in a day.
4. Configuration of the concurrent manager is (this is not standard manager):
No. of processes is: 8, Cache Size: 10, Sleep time: 10
Answer:
=========
sol_1
====
This problem seems to be inactive ICM.
When ICM is not up u will get error inactive-no manager.
Just check with manager and make it up.
ps -ef|grep LIBR
if it shows the manager is down then start the CM using script adcmctl.sh.
And if the manager is up but it is showing the inactive-nomanager then check for the log file and do the needful.
As my idea y dont u try with cmclean.sql.(not supported by oracle),Check in UAT before trying in prod.
1.Bring down CM.
2.run cmclean.sql
3.Bring up CM.
Sol_2:
======
Most importantly, see try reassigning the request to a different manager instead of the Standard Manager (maybe a custom one which only it belongs to) and see if you get the same behavior. Also check to see there are no unnecessary incompatibilities listed for the Program under the Program definitions.
sol_3:
========
I'd like to add that we checked the work schedule which is set to standard (24X7) and specialised rules against the standard Concurrent manager in the define manager screen. In adition we recreated the Conc mgr views, also created our own manager which gave the same message Phase Inactive No Manager.
Sol_4:
=======
What is the value of your GSM Enabled profile setting?
We had it set at Y but since changed it to N. We have also changed the appldcp context value to NO. Nothing seems to make any difference
Sol_5:
=======
after cloning you are getting inactive manager
go to applmgr user
$TNS_ADMIN / path
change the tns file as
edit
(SID=FNDSM) to (SID=FNDSM_VIS)
Sol_6:
=====
check the disc space of this instance, and increase
QUESTION:
=========
What are the differences between the functionality of purging concurrent request, FNDCPPUR and CMCLEAN.
Answer:
=======
FNDCPPUR is executable name of Purge Concurrent Request and/or Manager Data Program.
When you run this request, it will delete all the requests from Tables and log, req and manager log files from OS leve.
cmclean.sql is used to clean the concurrent manager tables, when there is any problem with managers starting up..
for more check the following note
CMCLEAN.SQL - Non Destructive Script to Clean Concurrent Manager Tables
Doc ID: Note:134007.1
Question:
=======
If ICM stop the othar manager ST which are already
started will continue but the new manager will not start
At the same time confict resolution manager will pickup all the request
How to know the ICM is down
Answer:
========
FNDSM is ICMs agent which will be doing the job for ICM(starting stoppin managers etc..). if that goes down everything else related to concurrent processing shuts off.
So you could grep for this FNDSM process continously at OS level to check ICMs status using a script
LOCKING:
========
Check V$session and V$locked_object. You can formulate a query by joining these two views to get the required info..Do a describe on them and try to create a query.
>How to resolve locking
using the two views mentioned above you can find out which session is blockin which sessions.If the culprit session is INACTIVE, then you may kill it using alter system kill session command. That would release the blockage
Question:
=========
what is PMON cyle,QUEUE size and sleep time:
ANS:
===
While the basic ICM definition should not be changed, you can if required modify the sleep time (number of seconds the ICM waits between checking for new concurrent requests), PMON (process monitor) cycle time (number of sleep cycles the ICM waits between checking for failed workers), and queue size (duration between checks for number of active workers, measured in PMON cycles). If Parallel Concurrent Processing (described below) is being used, you can also set some options for this.
what does these failed "workers" refer to?? What are these workers?
A/These workers are nothing but the Operating System processes associated with the concurrent managers.
Question:
==========
Reasons when concurrent manager fail?
In concurrent manager -> administer
Actual /target values different? Wat do you mean by it?
Answer:
=========
Reasons when concurrent manager fail?
There can be N number of reason for CM failure , check CM log file for exact message and cause
Few of them are --
Database listener down,
wrong password while starting CM,
corruption in FND_NODES ...
In concurrent manager -> administer Actual /target values different?
Wat do you mean by it?
Actual is actual number of manager running at that time and target is the number of Manager should be running at that particular time.
Qestion:
========
How to check the CM is up?
Answer:
========
ps -ef|grep FNDLIBR
Question:
========
which concurrent manager will run, which concurrent programs?
ANS_!:
========
Basically there are four types of Conc Mgrs
1 Internal Concurrent Manager
2 Conflict resoluction manager
3 Standard Manager
4 Transactional Managers
ICM - monitors other managers
SM picks up all the requests which or not there in it's exclusion list
Each transaction manager can process only the programs contained in its program library.
In the Concurrent Program Definition there is a column called REQUEST_TYPE by which we can assign the program to a particular manager.
You can use include and exclude options for the managers depending upon the your requirement if you want the program to be run by only one user.
You can navigate to Administer Concurrent Managers from System Adminsrator reqponsiblity, select maanger and view it's requests to see what all requests it is running.
Document which gives you more details is "System Administrator’s Guide"
Ans_2
=====
you can know this by navigate to concurrent>>manager>>define and there just query for that manager what you search for. There , the manager has been defined that it will run which requests or programs.
ANS_3:
========
From concurrent manager define and administrator screen we can find out what are all the concurrent programs that manager is running.
We can find out currently running request and which manager queue this request is running from the script $FND_TOP/sql/afrqrun.sql.
But my question is, I want general script or query to find from back end,
Which request (completed, errored, schedule, running etc) goes to which manager?
ANS_4:
=======
Use this query. Provide short name as concurrent_program_name.
SELECT A.INCLUDE_FLAG, A.QUEUE_APPLICATION_ID, C.USER_CONCURRENT_QUEUE_NAME,a
B.CONCURRENT_PROGRAM_NAME
FROM APPLSYS.FND_CONCURRENT_QUEUE_CONTENT A, APPLSYS.FND_CONCURRENT_PROGRAMS B, APPS.FND_CONCURRENT_QUEUES_VL C
WHERE type_id = b.concurrent_program_id and b.concurrent_program_name = '&SHORT_NAME' and c.concurrent_queue_id = a.concurrent_queue_id
/
Output will show you Which program is excluded from standard manager and included in other manager.
Question:
=========
What to do when Concurrent request is queing up and not being completed.
Did u started the concurrent manager...
ANSWER:
=========
A/Start the conc manger .. $COMMON_TOP/admin/scripts/adcmctl.sh start apps/apps
But still it not complited
B/Are there any errors in the Standard Manager log?
$APPLCSF/log/<Context Name>/FNDSM9999.mgr
Question
==========
What cmclean.sql will do?
Answer:
=========
cmclean.sql is non destructive script to clean CM tables. To run this script, CM has to be down since it does some updates on the tables. The script resets the flags of the requests to 'COMPLETED' and clean the CM tables so allowing the concurrent managers to come up. You need to run this script whenever you have an issue with the CM startup.
EXEC FND_CONC_CLONE.SETUP_CLEAN;commit ==> to clean fnd_tables
SQL> select NODE_NAME,SERVER_ADDRESS from fnd_nodes;
DESCRIPTION
Concurrent managers cleanup script.
To be called by rapidinstall ONLY.
- Delete all site specific information (nodes, services, printers, etc).
- Delete requests that are in Completed and Running states.
- Delete/reset all concurrent manager state information.
- Cancel all Pending Oracle Reports based requests.
Question:
========
What is CONCSUB
Ans:
=====
CONCSUB is a utility run at the operating system level to submit a concurrent program to the concurrent manager.
For more check the metalink note...Using The CONCSUB Utility
Doc ID: Note:9611.1
Question:
=========
How to purge concurrent request espcially those who has .req and .log extension name on OS side, is it safe to purge concurrent request?
Answer:
========
A/To avoid running out of space on your disk drives, you should periodically
delete Oracle Applications log files, output files and purge these tables
with the FNDCPPUR program (Purge Concurrent Request and/or Manager Data).
The FNDCPUR purge program maintains the number of log and output files the operating system retains, and manage tables that store information about concurrent requests and concurrent manager processes.
You can run the program FNDCPPUR once and automatically resubmit the program for your specific time intervals.
Depending on the usage you might have a different strategy for purging the concurrent requests log/out files eg.
- every 30 days for normal usage
- every two weeks (14 days) for heavy usage
- if you are using the AGE mode, set the Mode Value to 7 or higher to retain the most recent days of concurrent request data, log files, and report output files.
WARNING : When you purge concurrent request information, you lose audit details which are used by the Signon Audit Concurrent Requests report.
If your business requires to see the requests run back in time then you need to keep archive on disk or tape.
B/The log and out files go to a common location for all modules.
The concurrent manager first looks for the environment variable
$APPLCSF. If this is set, it creates a path using two other
environment variables: $APPLLOG and $APPLOUT
It places log files in $APPLCSF/$APPLLOG, output files go in
$APPLCSF/$APPLOUT
So for example, if you have this environment set:
$APPLCSF = /u01/appl/common
$APPLLOG = log
$APPLOUT = out
The concurrent manager will place log files in /u01/appl/common/log,
and output files in /u01/appl/common/out
Note that $APPLCSF must be a full, absolute path, and the other two
are directory names.
If $APPLCSF is not set, it places the files under the product top of
the application associated with the request. For example, a PO report
would go under $PO_TOP/$APPLLOG and $PO_TOP/$APPLOUT
Logfiles go to: /u01/appl/po/9.0/log
Output files to: /u01/appl/po/9.0/out
All these directories must exist and have the correct permissions.
Note that all concurrent requests produce a log file, but not necessarily
an output file.
Concurrent manager logfiles follow the same convention, and will be
found in the $APPLLOG directory.
Use the find command to move files which are more than N days old to another directory and zip them up or archive to tape. You can also use utilities like TSM (Trivoli Storage Manager) for backing up the filesystem.
eg. find $APPLCSF/$APPLLOG -type f -mtime +45|xargs -i mv -f {} /tmp/45more_log
Question:
=========
15 custom CM's , in which one is down . so what is the way to troubleshoot it to bring it up.
Answer:
=====
you can go with anyone:
1. 1st check the log file of the cm. if your error message is not enough to find out the problem, then change DIAG_PARAM ="" to DIAG_PARAM ="Y" in adcmctl.sh.
2.you can use cmclean.sql and then check the status of that cm.
3. check the values of target and actual column for that cm in frontend.
Question:
===========
CM is down Status Inactive and No Manager?
Ans:
===
1Check in icm log file to see issue
$APPLCSF/$APPLLOG/<SID>_<timestamp>.log Check in icm log file to see issue
$APPLCSF/$APPLLOG/<SID>_<timestamp>.log
Ans_2:
======
This is way too big of a question to be answered in one post but in general:
1. Ensure you have enough Standard Manager processes to handle all of the requests. The default number is 3, this can be increased in the Manager Definitions (Workshift).
2. Ensure that the number of rows in FND_CONCURRENT_PROCESSES and FND_CONCURRENT_REQUESTS don't exceed 50K or there will be performance issues.
3. Ensure that you don't have any unnecessary manager processes running in the manager queues.
4. Set a low priority if possible on the jobs which generally run the longest. If you are consistently getting slow performance on jobs, check to see if they also execute slowly from the SQL PLUS. If they do, the issue is with the code, not the managers.
DBA_EXTENTS or DBA_OBJECTS Table..there you can get the SIZE of the Objects
Question:
=========
submitted a request and it completed with the status as completed normal
But not able to see log file?
Answer:
========
A, RRA profile agaent & apps listener adalnctl.sh
B, whatever you have operating system , you need to restart the listener again even if it's working status and if the users are connected to the Apps. then you have to restart the Apatche first then the listener of the application 8.0.6.
Question:
========
DB tier and Concurrent Manager on one node
Apps Tier(Forms, Apache) on another node.
How to move the Concurrent Manager to a new node
Ans:
====
Theoretically, you'll need to add a node to the Apps Tier using the Rapid Clone addnode.pl
This should give you an overall idea.You'll need to tweak this to your requirement.
So, 1) you run preclone on DB node, CM node, Web/Forms node
2) Deregister existing CM node
3) Disable security option, run autoconfig on DB node, restart listener
4) Copy files to the new node
5) Run post clone on DB tier
6) Run post clone on Web/Forms Tier
7) Run post clone on new node to configure for CM node
8 ) Run addclone.pl on Apps tier.
How do I deregister an application tier from the Net Services Topology Data Model?
Answer:
To deregister the current application tier from the Net Services Topology Data Model, invoke the following command:
perl <AD_TOP>/bin/adgentns.pl appspass=<APPSpwd> contextfile=<CONTEXT> -removeserver
Refer Note: Note:218089.1 Autoconfig FAQ
Regarding step ( 8 ) : This would need to be run from Forms/Web node I believe. Not familiar with it's prompts, but it should be straightforward as we have the NEW CM node already ready by the time we execute this. Please ensure that once this is run, the entries in the context file on this node reflect your new node. I would check entries in the following section in context XML
<oa_admin_server>
<oa_cp_server>
For addnode.pl refer to note: Note:230672.1 Cloning Oracle Applications Release 11i
After everything is done, I would also check FND tables:
FND_NODES
FND_APP_SERVERS
The entries here should match with what we needed to start with.
1) Entry for DB node
2) Entry for Forms/Web Node
3) Entry for *NEW* CM node
4) No entries for *OLD* CM node
I hope you are doing this on test first? Let me know how it goes.
Question:
========
Want to configur PCP?
Ans:
===
241370.1 Concurrent Manager Setup and Configuration Requirements in an 11i RAC Environment
312731.1 Configuring Oracle Applications Release 11i with
10g Release1 Real Application Clusters and Automatic Storage Management
Section 3.11 Configuring Parallel Concurrent Processing
Question:
=========
Why only for Concurrent Manager, we specify apps/apps password & not for other?
Ans:
======
Concurrent Manager is used for processing requests..it means...for example...I want to run one report based on Current Month GL Balance...So we have to run one report to show balance...this balance will be stored in the Database tables..right.....
Concurrent Manager runs this request by connecting to the database using apps user password.....
Got it..hope it is clear.....'
Question:
========
which situations we need to bounce the concurrent manager ? and application services ?
Ans:
====
doc id: 105133.1(given in teachme oracle-Metalink Notes Section).
When you modify the Printer Driver you have to restart the Manager which runs the request which is attached to that Printer Driver, however,if you do not know which manager then you have to restart the Internal manager because the printer driver can be used by multiple managers and multiple requests. If only a concurrent program definition is modified, running a verify on the Internal Manager will pick up the changes without the need for bouncing the manager.
QUESTION:
==========
I have defined a new manager and submitted request in it.
But the job is always running and it never stops.
ANS:
====
issue initially the process specified for the new ASAP manager is below 10.
But as per the thumb rule for that specified application and the manager
the process should be more than 20.Thumb rule for the manager which we created was
thumbrule= 4 + (2 * profile value).
According to the above thumbrule the intial pocess of 8 was increased to 25 then the manager runs normally.
q/a
Below is the relevant content from adcmctl.sh:
printf "Shutting down concurrent managers for $TWO_TASK ...\n" >> $LOGFILE
$FND_TOP/bin/CONCSUB "$unpw" SYSADMIN $WAIT_PARAM 'System Administrator' SYSADMIN CONCURRENT FND SHUTDOWN
exit_code=$?
QUESTION
==========
CM is not shuting down.
Ans?
===
As u defined that CM is hanged due to some pending request,
the one option that i think would be feasible is to make cancel that request from backend
keeping status_code='E' and phase_code='C' ,
So , once the request got completed the CM will automatically come down with no problem
run Cmclean.sql and start the Cm
precisley knowing your FNDLIBR process you can try this in command line:
ps -aex|grep FNDLIBR|grep ORCL
The above command will list out FNDLIBR process related to instance ORCL only.
Question :
=======
CM is up but Oracle Applications Manager. it shows down?
Ans:
======
- Dashborad collection manager is down
- GSM might be disabled
- Monitoring/SErvice Update in OAM might be diabled
References:
Basic information about Concurrent Managers (Doc ID 69336.1)
Concurrent Processing - Product Information Center (PIC) (Doc ID 1304305.1)
Note:134007.1 CMCLEAN.SQL
Note:388495.1 Parallel Concurrent Processing (PCP) in Apps 11i?
Note:279956.1 Oracle E-Business Suite Release 11i with 9i RAC: Installation and Configuration using AutoConfig
Note:9611.1 CONCSUB Utility
Note:241370.1 Concurrent Manager Setup and Configuration Requirements in an 11i RAC Environment
Note:312731.1 Configuring Oracle Applications Release 11i with 10g Release1 Real Application Clusters and Automatic Storage Management Section 3.11 Configuring Parallel Concurrent Processing
Note:105133.1 when to bounce Cm
Note:392246.1 Requests Stuck with Inactive No Manager
Note:2181143
Note:233428.1 merge the APPL_TOP on the DB tier with the APPL_TOP on the apps tier
280295.1 REQUESTS.sql Script for Parent/Child Request IDs
Note:134033.1 Analyze all Pending Requests
Note 189708.1 DBA to configure two files i.e. uiprint.txt & default.ppd
Note 91151.1 Concurrent Managers Rel. 10.7 Tuning and Installation Issues
NOTE:74717.1 - TROUBLESHOOTING CONCURRENT MANAGER STARTUP PROBLEMS - Applications 11.0
NOTE:97798.1 - ORA-904 starting Concurrent Managers
NOTE:1057802.1 - Concurrent Processing - Best Practices for Performance for Concurrent Managers in E-Business Suite
NOTE:200360.1 - 11i : Oracle Application Object Library Concurrent Manager Setup Test
NOTE:168071.1 - ICMLOG.sql - Script for Retrieving Only the Internal Manager Logfile
NOTE:171855.1 - Concurrent Processing - CCM.sql Diagnostic Script to Diagnose Common Concurrent Manager Issues
NOTE:179661.1 - E-Business Suite Diagnostics 11i Test Catalog
NOTE:185036.1 - Unable to Startup or Shutdown the Concurrent Managers in 11.5.5 by Using the Syntax adcmctl.sh apps/<password> Start or Stop
NOTE:1059201.6 - Status Pending With 'Inactive No Manager' for Concurrent Requests
NOTE:113135.1 - Conflict Resolution Manager FNDCRM Does Not Start; No Error Message
NOTE:123607.1 - CONCSUB SYSADMIN 'System Administrator' Gives Invalid Responsibility Error
NOTE:1304305.1 - Concurrent Processing - Product Information Center (PIC)
NOTE:2055375.6 - Concurrent Managers and PMON Method
NOTE:2120154.6 - Uable to Start the CCM after Installation - ICM Core Dumps with Status 139
NOTE:213021.1 - Concurrent Processing (CP) / APPS Reporting Scripts
NOTE:230121.1 - Routine AFPEIM Encountered an Error While Starting Concurrent Manager on Cloned Instance
NOTE:421245.1 - E-Business Suite Diagnostics References for R12
Tips:
===
To trace CM request:
select v.request_id,
s.sid,
s.serial#,
s.event,
p.spid
from apps.fnd_conc_requests_form_v v,
v$process p,
v$session s
where v.request_id = ‘5029302'
and v.oracle_process_id = p.spid(+)
and p.addr = s.paddr(+)
/
Form the session ID find the PID
SQL> col sid for 999999
SQL> col serial# for 999999
SQL> col username for a20
SQL> col osuser for a20
SQL> select s.sid,s.serial#,s.username,s.osuser
from v$session s,v$process p
where s.paddr=p.addr;
Set the trace as below what is happening/executing we can comes to know
SQL> -- set the OS PID
SQL> oradebug setospid 864
Windows thread id: 864, image: ORACLE.EXE
SQL> -- or set the Oracle pid
SQL> oradebug setorapid 13
Windows thread id: 864, image: ORACLE.EXE
SQL> -- set the trace file size to unlimitd
SQL> oradebug unlimit
Statement processed.
SQL> -- now turn on trace for SCOTT
SQL> oradebug event 10046 trace name context forever, level 12
Statement processed.
SQL> -- run some queries in another session and then turn trace off
SQL> oradebug event 10046 trace name context off
Statement processed.
Find the trace in UDUMP
http://www.petefinnigan.com/ramblings/how_to_set_trace.htm==> to set trace using oradebug
453527.1
===================================================================
Query to find the Request id complete date
select request_id,
to_char(REQUESTED_START_DATE, 'mm/dd HH24:MI:ss') req_dt,
to_char(actual_START_DATE, 'mm/dd HH24:MI:ss') start_dt,
phase_code,
status_code,
to_char(ACTUAL_COMPLETION_DATE , 'mm/dd HH24:MI:ss') end_dt
from FND_CONCURRENT_REQUESTS
where (REQUESTED_START_DATE
between to_date('06/18 05:04:59','mm/dd HH24:MI:ss')
and to_date('06/18 05:10:00','mm/dd HH24:MI:ss'))
order by REQUESTED_START_DATE
/
