Monday, November 18, 2019

12.2 E-Business Suite Steps To Enable FND: Debug and AF Logging When Access To Change Profile Options Is Not Possible (Doc ID 1563168.1)

In this Document
Goal
Solution


APPLIES TO:

Oracle E-Business Suite Technology Stack - Version 12.2 and later
Information in this document applies to any platform.

GOAL

E-Business Suite Technology Stack, Weblogic Server related issues




Document the different steps to enable FND Debug and / or AF Logging in E-Business Suite.
The standard method to enable this type of debug is via the following profile options:
FND: Debug Log Enabled
FND: Debug Log Filename for Middle-Tier
FND: Debug Log Level
FND: Debug Log Module
However, in some circumstances you may be unable to login and set these profiles.
This note explains the alternative method, which is to enable the same level of debug via the Fusion Middleware configuration options.

SOLUTION

Test the following steps to enable FND Debug and / or AF Logging in E-Business Suite:
1. Source the run file system environment file.
2. Backup file:
   $INST_TOP/appl/admin/oacore_wls.properties
3. Edit $INST_TOP/appl/admin/oacore_wls.properties and add the following lines (adjust the variables accordingly):
AFLOG_ENABLED=true
AFLOG_LEVEL=statement
AFLOG_MODULE=%
AFLOG_FILENAME=/tmp/aflog.txt
4. Re-start Middle tier services.
5. Reproduce the issue.
6. Collect the log file.
7. Disable logging by removing the lines which were addded to oacore_wls.properties and re-start services.
Didn't find what you are looking for?

Oracle Applications E-Business Suite 12.2 Fusion Middleware Log Files: Locate,View, and Control (Doc ID 1366187.1)

APPLIES TO:

Oracle E-Business Suite Technology Stack - Version 12.2 to 12.2.3 [Release 12.2]
Information in this document applies to any platform.

ABSTRACT

 Oracle Applications E-Business Suite 12.2 is integrated with Oracle Fusion Middleware (FMW)11g, at initial release FMW 11.1.1.6 and WebLogic Server 10.3.6 with additional patching; the relevant components for this article are:

Oracle Process Manager and Notifications Server
Manages HTTP Server
Oracle HTTP Server (Apache)
Single Entry Point Proxy Server
WebLogic Server
Containing Oracle E-Business Suite Application Server Deployments

Each of the above components write to specific logs files that may be used for diagnostic purposes. This article describes these files and their location within Oracle E-Business Suite . For Information on creating and collecting diagnostic data specific to Oracle Forms within Applications E-Business Suite please refer to NOTE:373548.1.

HISTORY


DETAILS

Environment Variables*Description
LOG_HOMEApplication Instance Specific Log Directory
IAS_ORACLE_HOMEFMW Web Tier Home Directory
CONTEXT_FILESource for information populating template files (autoconfig)
EBS_DOMAIN_HOMEWLS Deployment of Oracle E-Business Suite 12.2 Domain (instance specific)
ADMIN_SCRIPTS_HOMEShell scripts to control processes associated to the Applications Instance
EBS_ORACLE_HOMEOracle E-Business Suite 12.2 FMW Deployment directory

*Access the variables by sourcing: <s_base>/EBSapps.env and choosing either Run or Patch environment. s_base is the base installation level of the Application E-Business Suite instance.

This article contains

Oracle E-Business Suite Fusion Middleware 12.2 Control Script Logs

Oracle E-Business Suite 12.2 uses instance specific control scripts to initiate and stop application processes. The scripts are located in $ADMIN_SCRIPTS_HOME and output from these scripts will provide insight into failures during the action attempted. The text based script output files are located in $LOG_HOME/appl/admin/log containing informational and error messages created during startup or shutdown of the associated process The table below details output files relevant to Fusion Middleware Application 12.2 processes
Log File NameDescription
adnodemgrctl.txtWLS Node Manager controls server instances within a domain providing automatic restart functionality.
adadminsrvctl.txtWLS AdminServer controls access to administrative pages within WLS for system wide configuration
adoacorectl.txtWLS Oracle E-Business Suite 12.2 Application Server for Self Service web based products
adformsctl.txtWLS Oracle E-Business Suite 12.2 Application Forms Server
adoafmctl.txtWLS Oracle E-Business Suite 12.2 Application Server for Oracle Transport Agent XML transactions, Secure Enterprise Search (SES) and Web Service
adforms-c4wsctl.txtWLS Oracle E-Business Suite 12.2 Application Server utilized by WebService products using java API for Forms
adopmnctl.txtFMW Oracle Process Manager Notification Server (Controls Apache processes)
adapcctl.txtFMW OHS (Apache) Service

OPMN and OHS

Oracle E-Business Suite 12.2 is configured with Apache (OHS Plugin) as a proxy server for its Application specific WebLogic components. The Apache process is started and monitored by the Oracle Process Manager and Notification Server (OPMN). The log files for OPMN and OHS are located under $IAS_ORACLE_HOME/instances/<ohs_instance>/diagnostics/logs with additional sub-directories for the specific Module i.e OPMN/opmn or OHS/<ohs_component>.


Log File NameDescription:View and Control:
opmn.logTracks command execution and operation progress. It contains messages useful for monitoring the operations of the OPMN server. Output written to the opmn.log file contains the exit status of a child OPMN process. A status code of 4 indicates a normal reload of OPMN. All other non-zero status codes indicate an abnormal termination of the child OPMN processControl: opmn.xml3
  • Log File Rotation
  • Log levels

View: o/s level access required.
debug.logContains OPMN debug log messages (English only) for ONS and PM. Review the error codes and messages that are shown in the debug.log file. The PM portion of OPMN generates and outputs the error messages in this file. The debug.log file tracks command execution and operation progress
opmn.outContains the standard output (stdout) and standard error (stderr) logs (normally 0 bytes) Also referred to as the OPMN "console log". After a certain point in OPMN initialization, nothing else is written to this fileControl: None

View: o/s level access required.
provision.logWhen written contains information about registration and deletion of instances. This log file is not created during normal day to day activities. Upgrade or specific scripts invoking opmn activities will write this file to the default location, if not redirected.Control:Execution of opmnctl either manually or through Scripts.

View: o/s level access required.
access.logBasic information about HTTP transactions handled by the serverControlFusion Middleware Control Console (EM)
  • WebTier > instance right click
  • Administration > Log Configuration

View: Fusion Middleware Control Console (EM)
  • WebTier > instance right click
  • Logs > View Logs
EBS_web_component.logError Log: startup and shutdown information, errors, warning messages
console~OHS~#.logStdout /Std Error messages
mod_rewrite.logFor recording redirection used for SSL enabled instancesControl:Applications Template file: ssl_conf_FMW.tmp on by default RewriteLogLevel 0

View: o/s level access required.
ssl_request_logRecords user activity when SSL enabled (custom log)Control:Applications Template file: ssl_conf_FMW.tmp on by default RewriteLogLevel 0

View: o/s level access required.
sec_debug_logProvides detailed information on issues related to security settingsControl:Applications Template file: security2_conf_FMW.tmp SecDebugLogLevel

View: o/s level access required.
sec_audit_logRecords information related to connections and security functionsControl:Applications Template file: security2_conf_FMW.tmp (off by default) SecAuditEngine

View: o/s level access required.

WebLogic (WLS)

Oracle E-Business Suite 12.2 is integrated with the FMW WebLogic Server. Server processes are initiated using a NodeManger providing automated restart functionality. Application Servers communicate with an AdminServer allowing monitoring of Oracle E-Business Suite server performance and administration. The log files are located within the WLS server deployment: $EBS_DOMAIN_HOME, each instance will have its own domain directory structure and associated log files, with the exception of the nodemanager (first row), where the complete path is given.
Log File Name
Description

View and Control
<s_base1>/fs<1/2>/FMW_Home/wlserver_10.3/ common/nodemanager/nodemanager.logNodeMgr information for Java-based Node Manager only; it is not created when using a script-based Node Manager.
Log file is generated by Node Manager and contains data for all domains that are controlled by Node Manager on a given physical machine.
Log rotation is disabled by default, but can be enabled by setting LogCount, LogLimit in nodemanager.properties
Control: LogLevel3
  • nodemanager.properties
View:WebLogic Server Administration Console
  • Expand Environment
  • Select Machines > MachineName > Monitoring
  • Node Manager Log tab
WLST Scripts
  • Using the WLST nmLog command
servers/<server_name2>/logs/<server_name2>>.outStdout and stderr messages generated by the server instance at NOTICE severity level or higher written by Node ManagerControl: Node Manager debug property enabled
  • LogLevel3
  • nodemanager.properties
View:WLST Scripts
  • Using the WLST nmServerLog command (must be connected to nodemgr) Requires additional configuration. Refer to Node Manager Administrators Guide Chapter 3
  • view: o/s level access Required
sysman/log/emoms.log
AdminServer/sysman/emoms.log
servers/<server_name2>/sysman/log/emoms.log
Oracle Management Service log file: start/stop and error logging.
Initial settings AdminServer and Domain level information is written. Files under $EBS_DOMAIN_HOME/sysman/log are viewed only on the instance o/s access required.
ControlFusion Middleware Control Console (EM)
  • Expand WebLogic Domain
  • Right click on AdminServer
  • Logs => Log Configuration
View:Fusion Middleware Control Console (EM)
  • Expand WebLogic Domain
  • Right click on AdminServer
  • Logs >View Logs / Log Configuration
WLST Scripts
  • Using the WLST displayLog command
sysman/log/emoms.trc
AdminServer/sysman/emoms.trc
servers/<server_name2>/sysman/log/emoms.trc
Oracle Management Service trace file: Default: contains critical and warning messages.
Initial deployment AdminServer and Domain level information is written.
ControlFusion Middleware Control Console (EM)
  • Expand WebLogic Domain
  • Right click on AdminServer
  • Logs => Log Configuration


View: o/s level access required
servers/<server_name2>/logs/<server_name2>.logInformation about events such as the start up and shut down of servers, the deployment of new applications, or the failure of one or more subsystemControl:WebLogic Server Administration Console
  • Expand Environment
  • Choose Servers > Defined Server
  • Logging or Debug Tab based on need.
View:Fusion Middleware Control Console (EM)
  • Expand WebLogic Domain
  • Right click on Server
  • Logs >View Logs / Log Configuration

WebLogic Server Administration Console
  • Select Diagnostics > Log Files.
WLST Scripts
  • Using the WLST displayLog command
servers/<server_name2>/logs/ <server_name2>-diagnostic.logSpecifies the volume of diagnostic data that is automatically produced by WebLogic Server at run time. Note that the WLDF diagnostic volume setting does not affect explicitly configured diagnostic modules. For example, this controls the volume of events generated for JRockit Flight RecorderControlWebLogic Server Administration Console
  • Expand Environment
  • Choose Servers > Defined Server
  • Select Configuration > General >Diagnostic Volume
View:Fusion Middleware Control Console (EM)
  • Expand WebLogic Domain
  • Right click on Server
  • Logs >View Logs / Log Configuration


WebLogic Server Administration Console
  • Select Diagnostics > Log Files.

WLST Scripts
  • Using the WLST displayLog command
servers/AdminServer/logs/<DomainName>.logContains forwarded information from the defined servers within a domain. Default: servers forward only messages of severity level NOTICE or higher,Control:WebLogic Server Administration Console
  • Expand Environment
  • Choose Servers > Defined Server
  • Configuration>General
  • Advanced Options: Domain Log Broadcaster: Security Level
ViewWebLogic Server Administration Console
  • Select Diagnostics > Log Files>DomainLog
servers/AdminServer/logs/owsm/msglogging/diagnostic.logOracle WebServices Manager log:
Oracle Web Services Manager (WSM) security and management has been integrated into the Oracle WebLogic Server along with Oracle WSM Agent functionality
.[domain]/servers/<server_name2 >/logs/owsm/msglogging/diagnostic.log
A default deployment is found within FMW AdminServer within Applications E-Business Suite 12.2
No services are deployed 0 byte file

Control: Fusion Middleware Control Console (EM)
  • Expand WebLogic Domain
  • right click on AdminServer
  • select Log Configuration
  • Highlight log file
  • Select Edit Configuration

View:Fusion Middleware Control Console (EM)
  • Expand WebLogic Domain
  • Right click on Server
  • Logs >View Log Messages >target Log Files
servers/<server_name2>/logs/access.logLog of all HTTP transactions in a text file, common log formatControl: WebLogic Server Administration Console
  • Expand Environment > Servers
  • select specific server
  • tab: Logging > sub tab HTTP
ViewWebLogic Server Administration Console
  • Select Diagnostics > Log Files
servers/AdminServer/data/ldap/log/EmbeddedLDAP.logAuthorization Policy Manager
The access log file is only written to when access is from an external client.
Control: Manual
  • servers/<Server_Name>/data/ldap/conf/vde.prop
View: o/s level access required
servers/AdminServer/data/ldap/log/EmbeddedLDAPAccess.log

1 $CONTEXT_FILE (repository of physical values for Oracle E-Business Suite instance used to instantiate template files. View this file as needed to determine the actual value of references in this document. As an example s_base path value is the default base directory of installation.
2server_name values for a standard Oracle E-Business Suite instance with a brief description:
  • AdminServer - WLS Administrative Server
  • forms_server1 - Oracle Forms 10.1.2.3 Servlet
  • forms-c4ws_server - Web Service Forms Servlet
  • oacore_server1 : Self Service Applications
  • oafm_server1: WebServices, AppsSearch, Transport Agent
3 Manual Edit

Log File Configuraton:

Log file size limit,rotation and retention are controlled within their various configuration files for Fusion Middleware. Reference:Oracle® Fusion Middleware Administrator's Guide Chapter 12 Managing Log Files and Diagnostic Data for details.

CONTROL TOOLS

Control, view and download logs associated to Fusion Middleware and WebLogic Servers within an Oracle E-Business Suite instance, without direct command line manipulation of the configuration files. Also available but not discussed in this article are Oracle E-Business Suite products that have integrated features to accomplish similar functions.

Enterprise Manager: Fusion Middleware Control:

Provides the ability to configure, view, search and download log files related to the Oracle HTTP Server. It may be used to Format file output, location and control the detail level. How to information is found in: Oracle Fusion Middleware Administrator's Guide for Oracle HTTP Server 11g, Chapter 7

Example: View error Logs

URL Access: http(s)://<s_webentryhost>.<domain>:<s_wls_adminport>/em
  • WebTier > instance right click
  • Logs > View EBS_web_component.log
Image of EM Console Log View Page

WebLogic Server Administrative Console

Available to configure, view, search and download log files related to the WebLogic Server. The Administrative Console provides an interactive GUI interface. How to information is found in: Oracle Fusion Middleware Administrator's Guide 11g Chapter 12

Example: view of available log files from WebLogic Server Administration Console

URL Access: http(s)://<s_webentryhost>.<domain>:<s_wls_adminport>/console
  • Select Diagnostics > Log Files
WebLogic AdminConsole Log View

Oracle WebLogic Scripting Tool (WLST)

WLST is a command-line interactive scripting environment that provides the ability to create, manage, and monitor the Oracle WebLogic Server domain How to information is found in: Oracle Fusion Middleware Oracle WebLogic Scripting Tool 11g Release 1

Unix command line example displayLogs or listLogs may be used, consult the reference above to select scope of output :
-bash-3.2$ cd $FMW_HOME/oracle_common/common/bin
-bash-3.2$./wlst.sh
<stdout: shows environment being setup >

wls:/offline>
wls:/offline> connect ('<userid>', '<password>', 't3://<s_webhost.s_webdomain1>:<s_wls_adminport1>')
Connecting to t3://host:port with userid <userid>...

Successfully connected to Admin Server 'AdminServer' that belongs to domain 'EBS_domain_vision'.
Warning: An insecure protocol was used to connect to the
server. To ensure on-the-wire security, the SSL port or
Admin port should be used instead.
wls:/EBS_domain_vision/serverConfig> listLogs()

Alternative sample commands:
  • List OHS logs:
wls:/EBS_domain_vision/serverConfig>listLogs (target="opmn:<s_ohs_instance>1/<s_ohs_component1>")
  • Display last 15 minutes OHS Logs
wls:/EBS_domain_vision/serverConfig>displayLogs (target="opmn:<s_ohs_instance>1/<s_ohs_component1>",last='15m')
  • List specific logs for WLS Server: oacore_server1:
wls:/EBS_domain_vision/serverConfig> listLogs(oracleInstance="<s_base1>/FMW_Home/user_projects/domains/EBS_domain_<s_dbSid1>",target="oacore_server1")
  • Display last 100 lines of oacore_server1 log
wls:/EBS_domain_vision/serverConfig> displayLogs(oracleInstance="<s_base1>/FMW_Home/user_projects/domains/EBS_domain_<s_dbSid1>",target="oacore_server1",tail=100)

ORACLE FUSION MIDDLEWARE DIAGNOSTIC FRAMEWORK

Oracle E-Business Suite 12.2 includes a Diagnostic Framework installed within Oracle Fusion Middleware which aids in detecting, diagnosing, and resolving problems. When a critical error occurs, it is assigned an Incident number, and diagnostic data for the error (thread dumps, log/configuration files) are captured at the time of an Incident (single occurrence of a problem). The Automatic Diagnostic Repository (ADR)stores the data in a file-base. By default an Incident is initiated when a log message is logged at the INCIDENT_ERROR log level. Additional configurations may be configured to allow fine grained collection of diagnostics specific to help in analysis.

Default Detection Levels:
  • Uncaught exceptions
  • Thread deadlocks
  • Stuck threads
  • OutOfMemory Errors when JRockit Flight Recorder is enabled (default)
Incidents are automatically detected in two ways:
  • By the incident detection log filter, automatically configured to detect critical errors.
  • By the WLDF Watch and Notification component. The Diagnostics Framework listens for a predefined notification type and creates Incidents when it receives such notifications.
    • Default Watch and Notification Module-FMWDFW may be altered to enable additional logging or alter default collection settings. Alternatively a new Module may be created for a specific purpose.
    • Reference: Oracle Fusion Middleware Administrator's Guide
      11g Release 1
       Chapter 13
The collected files from an Incident are written to: $EBS_DOMAIN_HOME/servers/server_name/adr/diag/ofm/EBS_domain_<sid>/<server_name>/incident

Command-Line Utility (ADRCI)

Is provided by Oracle Fusion Middleware as a utility to package an Incident for upload to support. The instructions below are an example, executed within a Linux operating system:
Note: Do Not Source the Oracle E-Business Suite Applications environment otherwise the following error is noted and the zip is not created:
DDE encountered the following error:
DIA-48312: Sweep incident 43 staging file failed
DIA-48001: internal error code, arguments: [dbgxtkInitXmlCtx:4], [0x0], [201], [], [], [], [], []


Replace all values within <> with instance specific information.
$pwd
<s_fmw_home1>/wlserver_10.3/server/adr
export ORACLE_HOME=<full value of present working directory>
export LD_LIBRARY_PATH=<full value of present working directory>
./adrci
No ADR base is set
adrci> SET BASE <s_fmw_home1>/user_projects/domains/EBS_domain_<s_dbSid1>/servers/<server_name2>/adr
adrci> SET HOMEPATH diag/ofm/EBS_domain_<s_dbSid1>/<server_name2>
adrci> ips create package incident 1
Created package 6 based on incident id 1, correlation level typical
adrci> ips generate package 6 in /tmp
Generated package 6 in file /tmp/EXAMPLE_20111024114658_COM_1.zip, mode complete
quit

File Structure for Oracle E-Business Suite 12.2 Fusion Middleware

Below is a visual representation of the log file locations for an instance of Oracle E-Business Suite 12.2 FMW installation. It does not show the complete directory structure or all log files locations with the instance.
FMW Visual Diagram of Log Locations

Patching Tools for Oracle E-Business Suite 12.2 Fusion Middleware

4Information on the FMW patching tools and commands can be found in Note:1355068.1

REFERENCES

NOTE:1362900.1 - R12.2 Ebusiness Suite - Collecting Fusion Middleware Log Files@ NOTE:373548.1 - How To Collect And Use Forms Trace (FRD) in Oracle Applications Release 12
http://www.oracle.com/technetwork/indexes/documentation/index.html#middleware
   Oracle Fusion Middleware Administrator's Guide 11g Release
   Oracle Fusion Middleware Administrator's Guide for Oracle HTTP Server 11g
   Oracle Fusion Middleware Oracle Process Manager and Notification Server Administrator's Guide
   Oracle Fusion Middleware Oracle WebLogic Scripting Tool 11g
   Oracle Enterprise Manager Advanced Configuration 10g Release 1 (10.1)
   Oracle Fusion Middleware Node Manager Administrator's Guide for Oracle WebLogic Server

REFERENCES

NOTE:373548.1 - How To Collect And Use Forms Trace (FRD) in Oracle Applications Release 12

12.2 E-Business Suite - Collecting Fusion Middleware Log Files (Doc ID 1362900.1)

In this Document
Goal
Solution
 Section 1: Steps to collect log files
 Alternative...
 Section 2: Steps to configure higher Debug levels and collect log files
 Section 4: Collecting configuration Information
 
 Appendix A: Script for automatically clearing contents of log files
References

APPLIES TO:

Oracle E-Business Suite Technology Stack - Version 12.2 to 12.2.6 [Release 12.2]
Oracle E-Business Suite Performance - Version 12.2 to 12.2 [Release 12.2]
Information in this document applies to any platform.

GOAL

This note is intended to be used as a debugging tool for the Fusion Middleware 11g components of E-Business Suite R12.2 which include OPMN, OHS and WebLogic components.

These include:
  • Oracle Process Manager and Notifications Server which manages the HTTP Server
  • Oracle HTTP Server (Apache) which acts as a single Entry Point Proxy Server
  • WebLogic Server which contains Oracle E-Business Suite Application Server Deployments
This note provides steps to enable extra debug and automate collection of log files required for debugging issues. Any changes made to increase log levels should be reverted once the information is collected. Oracle does not recommend detailed logging be turned on for extended periods, especially in production.

SOLUTION

Section 1: Steps to collect log files

This section should be followed by customers who are creating service requests for issues that have perhaps already occurred or which are not easily reproducible. This section does not enable extra debug - it simply collects log files at whatever debug levels are currently configured. In some cases the debug collected here may not suffice and in that case you may be directed by support to follow the steps in Section 2 to enable extra debug.

a) OPMN and OHS log files

Run this script as the owner of the applications file system to gather the log files for OPMN and OHS services:
zip -r /tmp/`uname -n`_`date +%m%d%y.%H%M`_OPMN_OHS.zip $IAS_ORACLE_HOME/instances/*/diagnostics/logs/OHS/*/*log* $IAS_ORACLE_HOME/instances/*/diagnostics/logs/OPMN/opmn/*

b) WebLogic Server

Run this script as the owner of the applications file system to gather the log files for Fusion Middleware components such as NodeManager, AdminServer, forms, oacore and oafm services:
zip -r /tmp/`uname -n`_`date +%m%d%y.%H%M`_FMW.zip $IAS_ORACLE_HOME/../wlserver_10.3/common/nodemanager $EBS_DOMAIN_HOME/servers/oa*/logs/*out* $EBS_DOMAIN_HOME/servers/oa*/logs/*log* $EBS_DOMAIN_HOME/servers/forms*/logs/*out* $EBS_DOMAIN_HOME/servers/forms*/logs/*log* $EBS_DOMAIN_HOME/servers/AdminServer/logs/*out* $EBS_DOMAIN_HOME/servers/AdminServer/logs/*log* $EBS_DOMAIN_HOME/sysman/log/* $EBS_DOMAIN_HOME/servers/oac*/adr/diag/ofm/EBS_domain*/oac*/incident/* $EBS_DOMAIN_HOME/servers/forms_s*/adr/diag/ofm/EBS_domain*/forms_s*/incident/*

These 2 commands will create zip files in the /tmp directory with names including server name and date. e.g. server.customer.com_092811.2005_OPMN_OHS.zip.  Please collect these files from the /tmp directory and upload them to your service request.

Alternative...

Use the following script to gather the Apache, WLS and Apps log files which have been modified in the last 1 day

#####################################################################################
####     Start of script
#####################################################################################
(
# pick up files which have been modified in the last 1 day only
HowManyDaysOld=1
echo "Picking up files which have been modified in the last ${HowManyDaysOld} days"
set -x
find $LOG_HOME -type f -mtime -${HowManyDaysOld} > m.tmp
find $FMW_HOME/webtier/instances/*/diagnostics/logs -type f -mtime -${HowManyDaysOld} >> m.tmp
find $FMW_HOME/wlserver_10.3/common/nodemanager/nmHome*/*.log -type f -mtime -${HowManyDaysOld} >> m.tmp
##  Get log files for only the WLS servers needed.   Valid server names are one or more of:
##      AdminServer forms-c4ws_server  forms_server  oacore_server  oaea_server  oafm_server
for SERVERNAME in AdminServer oacore_server forms_server oafm_server
do
   find $EBS_DOMAIN_HOME/servers/${SERVERNAME}*/logs -type f -mtime -${HowManyDaysOld} >> m.tmp
   find $EBS_DOMAIN_HOME/servers/${SERVERNAME}*/adr/diag/ofm/EBS_domain*/${SERVERNAME}*/incident -type f -mtime -${HowManyDaysOld} >> m.tmp
done
zip -r mzAppsLogFiles_`hostname`_`date '+%m%d%y'`.zip -@ < m.tmp
rm m.tmp
) 2>&1 | tee mzLogZip.out
#####################################################################################
####     End of script
#####################################################################################


Section 2: Steps to configure higher Debug levels and collect log files

This section explains the steps to enable higher logging levels to capture debug information required for resolving problems. You may be directed to this section by Oracle Support. Follow the steps below to enable debug and collect the log files generated after reproducing the issue. Generally speaking, increasing the log levels as per below should have only a minimal impact on performance (max 5% degradation)

a) OPMN
To enable debug for opmn, edit $IAS_ORACLE_HOME/instances/<instance>/config/OPMN/opmn/opmn.xml

Change the following line
<debug comp="" rotation-size="1500000"/>
to
<debug comp="ons[all];pm[all]" rotation-size="1500000"/>

Alternatively you can increase OPMN logging for the current session by issuing command
$IAS_ORACLE_HOME/instances/<yourinstance>/bin/opmnctl set target=debug comp="ons[all];pm[all]"


This would enable debug for all components running on the OPMN process. However, this may not be practical in most situations.


b) Increasing OHS Logging
To adjust HTTP logging levels access the Enterprise Manager console as follows
http://server.domain:wls_admin_port/em
login as weblogic/<weblogic admin password>

Expand the Web Tier menu
Right click EBS_web_component and choose administration - log configuration. Here you have the option to set the Log level. For debug purposes you can set this to "Trace:32"

c) Node Manager
To adjust log levels for Node Manager edit $IAS_ORACLE_HOME/../wlserver_10.3/common/nodemanager/nodemanager.properties
adjust LogLevel=INFO to LogLevel=FINEST


d) Start and Stop LoggingTo adjust start/stop log levels access the Enterprise Manager console as follows
http://server.domain:wls_admin_port/em
login as weblogic/welcome1
Expand WebLogic Domain then expand EBS_domain_X
Right click AdminServer and select logs - log configuration
Change the logging level to Trace:32

e) Web Logic Logging

Step 1: Enable log recording levels
To adjust WebLogic log levels access the WebLogic console as follows
http://server.domain:wls_admin_port/console
login as weblogic/<password>

Expand environment - servers

Select the servers to enable debug for. In most cases this will be the oacore servers. If you have more than one server defined in a cluster you will need to enable debug for each server individually. Only enable logging for the managed servers that are having issues.

Choose the logging tab, select the advanced hyperlink
Change severity level to Trace
Change Severity levels for standard out and domain log broadcaster to Debug

On this same configuration page you will see the setting "limit number of retained files". You may need to increase this value as extra logging may result in log files being overwritten due to excessive log file rotation.

Step 2: Enable debug areas
Continuing from step 1 above, select the debug tab
Expand the Weblogic section
Select the Servlet group which you wish to enable debug for (see below)
Choose enable to activate debug for these components.
ServletEnable this for all types of issues
JDBCEnable logging for this component if your issue relates to JDBC connection problems
ClassloaderEnable logging for this component for issues relating to classloading. Such as missing classes on startup of services

Note: If support or the customer is directed to enable any other component besides JDBC, servlet or classloader please inform the note author so these can be considered for addition to this note.


f) Restart Application Tier Services and clear log filesStop application tier services using the scripts located in $ADMIN_SCRIPTS_HOME. Clear the log files and then restart services
Note: Ideally at this stage it is a good idea to clear log files so that the problem can be pinpointed easily in the log files.This is especially true of production systems where logs can be large and almost un-manageable to debug. Log files should be cleared when services are down. You can do this manually or use the script provided in Appendix A

g) Reproduce issue and upload log files
Reproduce the issue and use the scripts in Section 1 to gather the log files. Upload the log files to your service request

Section 4: Collecting configuration Information

a) Run the technology stack inventory report to collect a list of patches applied to all middle tier homes (besides Weblogic). This script will also report the patch level for each oracle home:
$ADPERLPRG $FND_TOP/patch/115/bin/TXKScript.pl -script=$FND_TOP/patch/115/bin/txkInventory.pl -txktop=$APPLTMP -contextfile=$CONTEXT_FILE -appspass=<appspassword> -outfile=$APPLTMP/Report_App_Inventory.html

b) Provide the following files which contain managed server configuration information:
$IAS_ORACLE_HOME/instances/EBS_web_<SID>_OHS1/config/OHS/EBS_web_<SID>  (ZIP all files)
$INST_TOP/appl/admin/*.properties
$EBS_DOMAIN_HOME/config/config.xml (contains managed server start / stop options and memory settings)
$EBS_APPS_DEPLOYMENT_DIR/[forms  forms-c4ws  oacore  oafm] /APP-INF/node_info.txt


Appendix A: Script for automatically clearing contents of log files

The following script can be run as the applmgr user with applications environment sourced. This script will automatically clear the contents of the log files so that fresh logs can be captured. All application tier services should be shutdown prior to running this script.
for files in $IAS_ORACLE_HOME/instances/*/diagnostics/logs/OPMN/opmn $EBS_DOMAIN_HOME/sysman/log
do
for file in `ls $files/*`
do
echo "Clearing file - " $file
cat /dev/null > $file
done
done
for files in $IAS_ORACLE_HOME/instances/*/diagnostics/logs/OHS/EBS_web_component $EBS_DOMAIN_HOME/servers/*/logs $IAS_ORACLE_HOME/../wlserver_10.3/common/nodemanager
do
for file in `ls $files/*log* $files/*out*| grep -v lck`
do
echo "Clearing file - " $file
cat /dev/null > $file
done
done



Note: one can comment the following line
#cat /dev/null > $file
so that the command identifies the files without making any changes. Once comfortable with the changes the comment can be removed and the script run again to clear the files


References
For more information on logging definition, rotation, location and control review Document 1366187.1 - Oracle Applications E-Business Suite 12.2 Fusion Middleware Log Files: Locate,View, and Control

Database Options/Management Packs Usage Reporting for Oracle Databases 11.2 and later (Doc ID 1317265.1)

  Database Options/Management Packs Usage Report You can determine whether an option is currently in use in a database by running options_pa...