OracleAppsDBAKK1: E-Business Suite Applications DBA Top Adpatch Patch Application Issues For Release 11i And 12.x (Doc ID 1444322.1)

In this Document

APPLIES TO:

Oracle Order Management - Version 12.2.7 to 12.2.7 [Release 12.2]
Oracle Applications DBA - Version 11.5.10.2 to 12.2 [Release 11.5.10 to 12.2Cloud]
Information in this document applies to any platform.

PURPOSE

The goal of this document is to describe the commonly used best practices about patching for E-business Suite Release 11i (11.5.10.2) and Release 12 (up to 12.1.3).

This document includes:

Patching Strategy
Troubleshooting Guide for Top Patching Issues
Useful information that should be loaded in Community Threads or Service requests

Main Steps for Troubleshooting Errors:

Review logs (location of logs included, default log names included)
Verify that one followed the read me and installed prerequisites
Search My Oracle Support (How To Search Included)
Known Issues
Investigation Steps
What Information/Logs to Load in a Community Thread/Service Requests

For more information on patching, see the following reference documentation:

Release 11i

Release 12

TROUBLESHOOTING STEPS

1. How To Minimize The Risk When Applying A Patch

1. Always apply a patch in a test environment first
To perform an analysis of the effects of applying a patch, always apply it first to a test system.

2. Clone your Production System
In order to be able to analyze the patch impact, your test system must be similar to you production system. The application of the patch should be done on a test environment that is a very recent clone or copy of your Production environment.

3. Always have a valid backup
of the file system and database before applying any patch.
If the patching process fails when running the database portion of the unified driver, you will not be able to back out the patch and the only solution is to restore from a valid backup

Review Note 343987.1 How to Uninstall Backup Rollback an Oracle Applications 11i or R12 Patch

4. Analyze the impact of the patch
The use of Patch Wizard Utility to analyze patches before they are applied on your system.

Use the OAM Timing Report to analyze the patch application
From OAM : Site Map > Maintenance > Timing Reports

- Provides accurate method to measure requirements for maintenance windows
- Useful when applying patches between environments

Review:
Note 976188.1 Patch Wizard Utility
Note 976688.1 Patch Wizard FAQ
Note 1077813.1 Patch Wizard Overview
Note 352843.1 How to Run a Patch Impact Analysis In OAM

5. Use test mode

In reactive or emergency patching situations where testing is not feasible, the patch may be applied on your Production system by using the AutoPatch (adpatch) test mode argument (apply=no)

Review: Note 1078973.1 AD Command Line Options for Release R12

6. Test intensively the effect of the patch

Repeat the patch application until all issues are resolved and the system downtime is satisfactory.

7. Review Community threads

Oracle Communities has forum for patch reviews where you can get information from other customers about patches they have applied and their results.

2. How To Reduce System Downtime

1. Be sure to review and apply the best practices from Note 225165.1 Patching Best Practices And Reducing Downtime.

2. Staged Applications System
All of the applications filesystem patches are applied to a clone of your production Apps environment. This can be done while your production system is still running. The production system is down only for the time needed to apply database patches. Review: Note 242480.1 Using a Staged Applications 11i System to Reduce Patching Downtime

3. Merge Patches
Before running AutoPatch, use AD Merge Patch to merge multiple patches into a single, integrated patch so that the required patching tasks and processes are performed only once.

In general, you can safely merge any Oracle Applications patch with another Oracle Applications patch.
Patches should be merged with their listed prerequisite patches to make the application of the patch easier.
However, patches that affect the Applications DBA (AD) product may change the AutoPatch utility itself.
So, they can be merged only with other AD patches and must be applied separately, before applying any non-AD patches.
If the system uses multiple languages, one can use AD Merge Patch to create merged patches in the following ways:

       - A single, merged patch that contains all languages (including US).
       - One merged patch for US and a second merged patch for all other languages.
       - A separate merged patch for each language.For option 2 and 3 you can apply the US patches first during downtime, then, you can apply the merged NLS translation patches during uptime.

Each of these options has advantages and disadvantages.
    - The first option is the simplest to apply because there is only one patch involved but it requires the longest system downtime of the three options, as the patch can be large.
    - The second option is relatively easy to apply, and allows you to bring US users back online while you apply the patch for non-US languages.
    - The third option is the most difficult to apply, but allows you to bring users for various non-US languages back online in a phased approach, which could be useful for multi-national company environment The second option because provides the best compromise between easy application and minimum downtime.

Review Note 228779.1 How to Merge Patches Using admrgpch

4. Translation Synchronization patch
You don't need to apply all NLS patches, just one patch to synchronize NLS.
The Translation Synchronization patch feature provides a quick way for you to synchronize your existing translations with the American English file versions on your Applications instance. By applying just one patch for each language, you will be able to bring your translations up to your Applications patch level. You can also choose to get the latest translations to bring your translations up-to-date.

5. Shared application tier

The best improvement that can be made to a multi node environment is to share the application tier file system.
Considering the measures that have already been introduced, sharing the application tier file system is the one major and significant improvement that will reduce downtime. It will also simplify and reduce maintenance. Available documentation:

Note 384248.1 Sharing the Application Tier File System in Oracle E-Business Suite Release 12
Note 233428.1 Sharing the Application Tier File System in Oracle Applications Release 11i
Note 745580.1 How To Apply Patches On Shared Application Tier File System Environment
Note 243880.1 Shared APPL_TOP FAQ

6. Correctly select the number of workers:

Carefully monitor your server resources, especially the memory usage when you use a high number of workers.
Use the number of workers employed to apply the patch on the test system to extrapolate the number required for your production system.
Use Distributed AD. Distributed AD is a parallel processing feature that can reduce downtime by efficiently utilizing all the available resources on a shared application file system. AD Administration and AutoPatch run on the primary node and direct workers running on that node and other nodes in the system. The AD Controller utility controls and monitors the actions of the workers that you specify. For complete information, see Distributing Processing Tasks in Oracle Applications Maintenance Procedures (11i and R12). Check available documentation:

Note 800024.1 How Does Adpatch Determine The Number Of Workers To Recommend?
Note 226191.1 How To Select Number of Workers Based on Number of CPUs When Running ADPATCH
Note 756063.1 How to Troubleshoot "adpatch" Performance Issues: Slow, Hanging or Crashes

7. Patch Wizard
Patch Wizard is a Web-based utility in Oracle Applications Manager (OAM).
It provides a one-stop place for administrators to perform all the patching related tasks:
        - View previously applied recommended patches on the system
        - Determine new patches to uptake for the system
        - Understand impact of patch application on system
        - Download patches to be applied
        - Automatic updates on relevant new patches from Oracle

Review:
Note 976188.1 Patch Wizard Utility
Note 976688.1 Patch Wizard FAQ
Note 1077813.1 Patch Wizard Overview

8. Run Adpatch in non-interactive mode

Applying a set of patches using AD Patch in non-interactive mode eliminates the delay between successive tasks.

9. Defer system-wide database tasks until the end

Using adpatch options=nocompiledb,nomaintainmrc defers system-wide database tasks such as "Compile APPS schema" and "Maintain MRC" until after all patches have been applied. Review Note 1078973.1 AD Command Line Options for Release R12.

Recommendation: Check available webcast on the subject: Minimizing E-Business Suite Maintenance Downtimes

3. How To Plan A Smart Patching Strategy

We will provide best practices patching strategy for all EBS environments. In order of priority, one should:

1. Apply the latest EBS Release Update Pack.

For example: 12.1.3 for Release 12.1, 12.0.6 for Release 12.0, 11.5.10.2 for Release 11i.

Note 1080973.1 Oracle E-Business Suite Release 12.1.3 Readme
Note 743368.1 Oracle E-Business Suite Release Update Pack Readme, Release 12.0.6
Note 316366.1 11.5.10 Oracle E-Business Suite Consolidated Update 2 (CU2)

2. Apply the latest EBS Family Packs and all patches on the Recommended Patch List.

This includes ATG Release Update Packs and AutoConfig updates.
Check section How to identify recommended patches for a product?

3. Upgrade all technology stack components to the latest certified levels.

For example, as of today, the latest certified levels for EBS 12 are Database 12cR1, Forms 10.1.2.3, OC4J 10.1.3.5, Oracle Internet Directory 11.1.1.3. Check the Certifications database on My Oracle Support of EBS certifications for a snapshot of the latest certified patchsets for technology stack components.

Note 1524398.1 Interoperability Notes EBS 12.0 or 12.1 with RDBMS 12cR1
Note 1524399.1 Interoperability Notes EBS 11i with RDBMS 12cR1
Note 437878.1 Upgrading OracleAS 10g Forms and Reports in Oracle E-Business Suite Release 12
Note 454811.1 Upgrading to the Latest OracleAS 10g 10.1.3.x Patch Set in Oracle E-Business Suite Release 12

4. Apply the latest Critical Patch Updates (CPU)

One should always apply the latest security patches. Always. These are released quarterly.
Regularly check: CPU link
At this moment latest CPU is:

Note 1559732.1 Oracle E-Business Suite Releases 11i and 12 Critical Patch Update Knowledge Document (July 2013)

5. Apply all mandatory RDBMS patches

Depending on your your database level check the interoperability note that applies to you.
Check Note 1072409.1 Database Documentation Resources for EBS Release 11i and R12
For example for R12 with 11.2.0.3 database check: Note 1058763.1 Interoperability Notes EBS R12 with Database 11gR2
section "8. Apply additional 11.2.0.3 RDBMS patches"

6. <Optional> Apply the latest Database Patch Set Updates (PSU) and the required EBS prerequisites

It's safe to apply Database PSUs to the EBS environment. Some customers like the convenience of PSUs. Some don't. It's up to you.
While not mandatory for the interoperability of Oracle E-Business Suite with the Oracle Database, customers may choose to apply Database Patch Set Updates (PSU) on their Oracle E-Business Suite Database. If so, please see Note 1147107.1 ('Database Patch Set Update Overlay Patches Required for Use with PSUs and Oracle E-Business Suite') on My Oracle Support for more information.

7. Apply specific one-off and interim patches...

... but only if they're really required for your environment. This applies to both EBS and technology stack component patches. You're generally better-off waiting for patches to be rolled into the bigger release vehicles listed in #1 to #4, above. Those consolidated release vehicles are tested more comprehensively with all Apps modules and configurations than one-off or interim patches.

8. Use tools like Oracle E-Business Suite Plug-in

This Enterprise Manager plug-in automates the process of checking for required patches, staging those patches, then applying those patches to multiple EBS environments.

Note 1434392.1 Getting Started with Oracle E-Business Suite Plug-in, Release 12.1.0.1.0
Note 1224313.1 Getting Started with Oracle E-Business Suite Plug-in, Release 4.0

9. Check advise from: Note 313.1 Patching & Maintenance Advisor: E-Business Suite (EBS) 11i and R12

Reference blog: What's the Best Way to Patch an E-Business Suite Environment?

4. How To Identify The Location And Meaning Of Patching Logs

Where to find the log files:

The log files are written to:
Unix :
$APPL_TOP/admin/<SID>/log (UNIX),
where <SID> is the value of the ORACLE_SID or TWO_TASK variable,
Windows :
%APPL_TOP%\admin\<SID>\log (Windows),
where <SID> is the value of ORACLE_SID or LOCAL.

Different kind of patch log files

Log file name	Log File Used For
adpatch.log	Main AutoPatch log file (default name) Recommended to rename it : <patchnum>.log
adworkxxx.log	Database worker for operations run in parallel When AutoPatch is using parallel processing and an error occurs, the job fails. Review the main log file (adpatch.log) and the adworkxxx.log file to determine the source of the error, resolve the issues and continue. Restart AutoPatch using the adctrl command.
adpatch.lgi	AutoPatch informational messages (default name) important file it contains the steps done by adpatch and also the backup operations that have been executed Very important file if you need to check the files copied during the patch application
adrelink.log	Main Relink log file
adlibin.log	Additional Relink log : file Moving C object files into the C library of a product
adlibout.log	Additional Relink log : Moving C object files out of the C library of a product
<language>_<filename>_ldt.log	FNDLOAD : Seed data loader files
<patchnumber>preenv.html	List of the invalids before the patch application
<patchnumber>postenv.html	List of the invalids after the patch application
job timing report	Located in $APPL_TOP/admin/<SID>/out whereas other files are in $APPL_TOP/admin/<SID>/log

Notes:

One can also review log files using the View Log Files feature of OAM.
If AutoPatch does not perform an action, it does not generate the log file associated with that type of action.

For addressing invalid objects the following utility has been released to help identify known issues:

Note 2214169.1 - EBS Invalid Object Utility (Doc ID 2214169.1)

Method to Investigate
- Get the patch log files
- Check for the error in the main adpatch log file
- Identify the error and so the log file containing the error
- Drill down to the corresponding file to check the detailed error

Additional Information

Patch Backup directory

During the copy portion of a patch
adpatch takes a backup of any files the patch will replace (with a more recent version)
For example, if <patch_dir> is the patch directory, the patch backup directory is:

<patch_dir>/backup/….

Successful Completion message

AutoPatch displays messages such as the following when processing is complete. If one does not see a completion message, always investigate and identify the reason

A job timing report has been generated for the current session.
You should check the file
$APPL_TOP/admin/PROD/out/adt323790.lst
for details.
Purging timing information for prior sessions.
sqlplus -s APPS/*****
@$APPL_TOP/ad/12.0.0/sql/adtpurge.sql 10 1000
Done purging timing information for prior sessions.

AutoPatch is complete.

AutoPatch may have written informational messages to the file
$APPL_TOP/admin/PROD/log/adpatch.lgi
Errors and warnings are listed in the log file
$APPL_TOP/admin/PROD/log/adpatch.log
and in other log files in the same directory.

Database Alert log file

The database alert log file is very important as it can contain information related to the error in the database. The alert_instance.log (where instance is the Oracle SID) is located in the bdump directory. To locate it please log in to SQL Plus as the apps or system user and run the following command:

SQL> show parameter background_dump_dest

5. How To Identify If A Patch Is Already Applied

How to check if a certain Patch was applied to Oracle Applications instance using 'adpatch'?
Very quick check is:

select bug_number from ad_bugs where bug_number='xxxxxx';

Please review next documents:
Note 443761.1 How to check if a certain Patch was applied to Oracle Applications instance (11i or R12) ?
Note 390065.1 How to check if a NLS Patch for a base US Patch has been applied?

Usual questions:

1. There are cases where few patches are stored in ad_bugs and few in ad_applied_patches. What is the difference?

AD_APPLIED_PATCHES holds information about the "distinct" Oracle Applications patches that have been applied.
If 2 patches happen to have the same name but are different in content (eg. "merged" patches), then they are considered distinct and this table will therefore hold 2 records.
So it contains information on patches you installed directly (running adpatch driver=u<patch>.drv).
AD_BUGS holds information about the various Oracle Applications bugs whose fixes have been applied (ie. patched) in the Oracle Applications installation.
So this table holds information about all bug fixes that have been applied. Even if this patch have been included in other patch.

2. Which fixes are included in a patch?
You can perform patch impact analysis as per Note 352843.1 How to Run a Patch Impact Analysis In OAM
When you apply a patch there are 2 files provided by each patch :

b<patchnum>.ldt provides the list of the bugs fixed by the patch. adpatch will record this information in ad_bugs table
f<patchnum>.ldt provides the list of the files and their versions delivered by the patch. adpatch will record this information in ad_files table

Those 2 files are used by OAM to perform the patch analysis and other reports.

In order to identify if Oracle Home patches applied via opatch are available in the system you have to use next procedure:

Source related ORACLE_HOME environment (can be 8.1.7, 10.1.2, 10.1.3, 9i, 10g, 11g)

cd $ORACLE_HOME/OPatch
export $PATH=$ORACLE_HOME/OPatch:$PATH
opatch lsinventory -invPtrLoc $ORACLE_HOME/oraInst.loc

Check if patch is listed.

6. How To Solve Performance Issues During A Patch

To avoid performance issues during patch check next steps:

1. Use correct number of parallel workers

The issue can be an incorrect number of workers used at patching.

Check: Note 756063.1 How to Troubleshoot "adpatch" Performance Issues: Slow, Hanging or Crashes

2. Before starting patch, relink ad executables:

On Linux:

adrelink.sh force=y "ad all"

On Windows:

Run %APPL_TOP%\relinkenv.cmd
In the command window that results, change directory to %APPL_TOP% and run apps.sh to set up all required environment variables. (Note there is a space between the dots in this command.)
     . ./apps.sh
Run command
sh adrelink.sh force=y "ad all"

3. If specified in documentation/read me of patch, disable fast validation for PL/SQL recompilations:

_disable_fast_validate=TRUE

The line should be commented out during the normal operation of the Applications system. Use it only during patching/upgrade.

Example: Note 761570.1 Database Preparation Guidelines for an E-Business Suite Release 12.1 Upgrade

4. Correct database initialization parameters as per:
Note 396009.1 Database Initialization Parameters for Oracle Applications Release 12
Note 216205.1 Database Initialization Parameters for Oracle Applications Release 11i
use:
Note 174605.1 bde_chk_cbo.sql - EBS initialization parameters - Healthcheck
This will help you to identify the difference between actual values and required values

Check if an SGA resizing is needed.

5. Ensure one schedules the Gather Schema statistics concurrent request periodically.
Note 419728.1 Concurrent Processing - How To Gather Statistics On Oracle Applications Release 11i and/or Release 12 - Concurrent Process,Temp Tables, Manually

6. Try to increase adpatch batchsize to 10000 from 100.

7. If one did not identify the issue please raise a Community thread or service request with:

- The operating system with version

- Output of: Note 174605.1 bde_chk_cbo.sql - EBS initialization parameters - Healthcheck

- Patch Log

7. You Are Not Able To Find 64bit Version Of An Applications Patch

Is this a patch to apply with adpatch? Then use 32bit version of patch even you are on 64bit operating system.
Is this a patch for 10.1.2 ORACLE_HOME or for 10.1.3 ORACLE_HOME? Then use 32bit version of patch even you are on 64bit operating system.
The 64bit version of patches are needed only for your RDBMS ORACLE_HOME.

This is documented in:

Note 343917.1 Frequently Asked Questions: Oracle E-Business Suite Support on x86-64
"Are there separate patches for Oracle E-Business Suite on x86-64 with a 64-bit OS?
Answer:
There are really two answers to this question - for the Database Tier, the answer is yes. Since we bundle the 64-bit Database for use on 64-bit OS'es, the database tier's patches may indeed differ for the 64-bit distribution vs the 32-bit.
On the application tier, the answer is no. Users should apply 32-bit patches for components on the application tier (E-Business Suite applications, Fusion Middleware components, 3rd party patches) since there is not a separate 64-bit distribution generally for E-Business Suite.
"

There is a difference in the 'bitness' of the OS, and the 'bitness' of the code/applications that reside on that OS.
32-bit applications can, generally speaking, run on both 32 and 64-bit versions of the OS while 64-bit apps can only be built and run on 64-bit OS'es.
Bitness of the OS can be checked by various OS-specific commands ('uname -a' on Linux, 'isainfo -kv' on Solaris SPARC, 'getconf' on HP-UX and AIX, for example)

Applications patches are 32 bits even on 64 bits platform
The application tier of the E-Business Suite is always 32 bit software and therefore all Apps patches (patches applied using adpatch) and application tier technology stack patches (applied using opatch) are 32 bit.

This means that although the patch may be listed as only available for Solaris-64 or HP-UX-64 the patch will actually be 32 bit and it can be applied to a 32 bit application tier.

Operating systems such as Solaris, HP-UX and AIX have been 64 bit only for a long time and EBS R12 is no longer actually even certified on the 32 bit version of these platforms.

As conclusion the patch may be applied to both 32-bit and 64-bit operating systems.

8. How To Solve Relinking Errors During Patching

In your patch log one will see:

STRT_TASK: [Relink executables] []

Relinking executables...

An error occurred while relinking application programs.
Continue as if it were successful [No] :

1. First root cause can be missing rpms in your operating system.
In order to check your OS prerequisites in automatic way please use Note 250262.1 RDA 4 - Health Check / Validation Engine Guide .
After following “Installation Instructions” please follow “Instructions for UNIX type operating systems”.
You have to run

[applmgr@soatest rda]$ ./rda.sh -T hcve
Choose:  Oracle E-Business Suite Release 12 (11.5.10.2) Preinstall

An html diagnostic will be generated, You have to review and ensure there is no failure.

2. Verify $APPL_TOP/admin/<db_name>/log/adrelink.log
Verify the exact executable failed to relink.

You will NOT see the error in adpatch log. You have to look in adrelink.log and do a search with "failed".

3. Do a search in MOS using:

- executable name
- error found in adrelink.log

Search after your error in Note 1523589.1 Master Note - Troubleshooting Adpatch Relinking Errors

4. Check if this executable could be relinked prior to this patch (just search in adrelink.log for an older session).

5. Let us know if this is a clone? Relink issues usually occurs for an incorrect clone preparation (related to OS rpms)
If it is a clone, do you have the same issue on source system?

6. Review Oracle E-Business Suite Installation and Upgrade Notes for your release.

Many relink issue are solved if you review all the steps from this note.

Here are few examples:

6.1. There are known issue with relinking the Advanced Supply Chain Planning (ASCP) executables MSO, MSC, MSR and FEM

You will see errors like:

libgcc_s.so.1
/usr/bin/ld: `.gnu.linkonce.t._ZN17SCOProfileOptionsC1Ev' referenced in
section `.rodata' of
.../mso/12.0.0/lib/libmso.a(msopomdl.o): defined in
discarded section `.gnu.linkonce.t._ZN17SCOProfileOptionsC1Ev' of
.../mso/12.0.0/lib/libmso.a(msopomdl.o)

/usr/bin/ld: `.gnu.linkonce.t._ZN17SCOProfileOptionsC1Ev' referenced in
section `.rodata' of
.../mso/12.0.0/lib/libmso.a(msopomdl.o): defined in
discarded section `.gnu.linkonce.t._ZN17SCOProfileOptionsC1Ev' of
.../mso/12.0.0/lib/libmso.a(msopomdl.o)

Review and implement the steps from installation update note for your system. For example:

Note 761566.1 Oracle E-Business Suite Installation and Upgrade Notes Release 12 (12.1.1) for Linux x86-64
or
Note 316806.1 Oracle Applications Installation Update Notes, Release 11i (11.5.10.2)
"Relink Advanced Supply Chain Planning executables (for SLES 10, Oracle Linux/RHEL 5.4 or higher and Oracle Linux 6 only)

During the relink phase of the installation of EBS Release 12 (12.1.1) on SLES 10, Oracle Linux/RHEL 5.4 (Update 4 or higher) or Oracle Linux 6 failures will result while relinking the Advanced Supply Chain Planning (ASCP) executables MSO, MSC, MSR and FEM. To fix this problem, users are required to replace the following line under the Linux section of the $AD_TOP/bin/adrelinknew.sh:
•CPP_LDFLAGS=' -L$(ORACLE_HOME)/lib -L$(ORACLE_HOME)/lib/stubs -lclntsh'
•with
•CPP_LDFLAGS=' -L$(ORACLE_HOME)/lib -L$(ORACLE_HOME)/lib/stubs -lclntsh -Wl,--noinhibit-exec'
"

Permanent solution for R12.1 is: Patch 10349415:R12.AD.B

6.2. You have relink errors like "aCC: not found"
Then Install the needed C++ compiler
As per Note 316806.1 Oracle Applications Installation Update Notes, Release 11i (11.5.10.2)
"C++ Compiler C.03.27 (or higher) or HP C/aC++ Developer's Bundle"

6.3. If you have error:
/usr/lib/gcc/x86_64-redhat-linux/4.4.6/32/libgcc_s.so: undefined reference to `__stack_chk_fail@GLIBC_2.4'

Ensure you followed Note 761566.1 Oracle E-Business Suite Installation and Upgrade Notes Release 12 (12.1.1) for Linux x86-64
Section "After Installing or Upgrading"
Patch AS 10g (10.1.2 and 10.1.3) Oracle Homes (Oracle Linux 6 and RHEL 6 only)

Reference Note 1525823.1 Relink From Adadmin Giving Error: libgcc_s.so: undefined reference to `__stack_chk_fail@GLIBC_2.4'

7. If you have relink errors like: The file is in use and cannot be overwritten

For example:

ld: 0711-851 SEVERE ERROR: Output file:
/u02/appldg/fmsdgappl/po/11.5.0/bin/RCVOLTM
    The file is in use and cannot be overwritten.
make: The error code from the last command is 12.

Stop.
Done with link of po executable 'RCVOLTM'

- Stop all services of application tier:
$ADMIN_SCRIPTS_HOME/adstpall.sh

- Make sure all the application tier processes are down, which means the command below returns no records:
ps -ef | grep <applmgr>
Note: One should replace <applmgr> with your own application tier operation system user.

Make sure no FNDLIBR, FNDSM, or other dead processes are running.

- Retest relink.

- If the error mentioned is thrown, then try to find if any users are actively using the executables. If any, kill them using the following command:

fuser -ki <filename>

For example:
fuser -ki /u02/appldg/fmsdgappl/po/11.5.0/bin/RCVOLTM

Identify what is using the executable

- If the fuser does not list any user actively using the file, then login as root user:

su - root
/usr/sbin/slibclean
Note! The command is for AIX

8. If you did not identified the issue please raise a Community thread or service request with:

- your operating system and version. Relink issue usually differs from an OS to another

- how the instance was created (fresh install, clone, patching, upgrade)

- upload $APPL_TOP/admin/<db_name>/log/adrelink.log

- the query to prove that all prerequisites are applied.
Use next query connected as apps user:

SQL> select bug_number from ad_bugs where bug_number in ('&patch_number1','&patch_number2',...);

- upload the html generated by "./rda.sh -T hcve" from Note 250262.1 RDA 4 - Health Check / Validation Engine Guide

9. How To Solve Invalid Objects After Patching

Here you can find generic steps useful for solving invalid objects after any patching or upgrade:

1. Ensure that all known bug-fixes are installed by following EBS interoperability notes that applies to your current version of RDBMS
Note 1524398.1 Interoperability Notes EBS 12.0 or 12.1 with RDBMS 12cR1
Note 1524399.1 Interoperability Notes EBS 11i with RDBMS 12cR1
Note 1058763.1 Interoperability Notes - Oracle E-Business Suite Release 12 with Oracle Database 11g Release 2 (11.2.0)
Note 735276.1 Interoperability Notes E-Business Suite R12 with Oracle database 11gR1 (11.1.0)
Note 812362.1 Interoperability Notes Oracle EBS 12 with Oracle Database 10gR2 (10.2.0)
In every interoperability note there is a list called additional patches for database version xxx
Be sure you apply them all!

Also you will find in prerequisites a patch called "interoperability patch for Release xxx"
Be sure is applied in your system!

2. Ensure there are no relink issues for RDBMS executables.
Tip: Follow Note 356878.1 How to relink an Applications Installation of Release 11i and Release 12
to relink RDBMS' executables.

3. Ensure that database initialization parameters are correctly set as per notes:
Note 396009.1 Database Initialization Parameters for Oracle E-Business Suite Release 12
or
Note 216205.1 Database Initialization Parameters for Oracle Applications Release 11i
In order to verify use the diagnostic from:
Note 174605.1 bde_chk_cbo.sql - EBS initialization parameters - Healthcheck

4. Use adadmin. From the Main AD Administration menu, go to the Maintain Applications Database Entities menu. Select the “Validate APPS schema" task.
It produces a report named <APPS schemaname>.lst that lists issues and potential issues

5. Ensure that APPS has required privileges to access RDBMS' objects by performing the post upgrade steps.
Note: There is also a script named $AD_TOP/patch/115/sql/ademusr.sql that might need to be run to grant required privileges.

6. You have errors like:

ORA-04065: not executed, altered or dropped stored procedure
"<procedure name>"
ORA-06508: PL/SQL: could not find program unit being called:
"<procedure name>"

Then compile RDBMS invalids using
utlirp.sql
utlrp.sql
See: Note 370137.1 After Upgrade, Some Packages Intermittently Fail with ORA- 04065

7. Use adadmin. From the Main AD Administration menu, go to the Maintain Applications Database Entities menu. Select the "Recreate grants and synonyms for APPS schema" task.

8. Use adadmin. From the Main AD Administration menu, go to the Maintain Applications Files menu. Choose the " Check for missing files" task.
Find a newer patch that brings those missing files and apply it.

9. Use adadmin. From the Main AD Administration menu, go to the Compile/Reload Database Entities menu. Choose the "Compile APPS schema" task.
Use the following SQL statement to see if the number of invalid objects decreases:

SQL> select count(*)
from  dba_objects
where status<>'VALID';

10 In order to obtain the list of invalids with related error

sqlplus apps/apps_pass @$AD_TOP/sql/aderrchk.sql apps apps_pass % invalids.log NOFAIL

invalids.log will display all invalids and the associated error.

11. Check Note 472937.1 Information On Installed Database Components and Schemas
for more information about reloading missing/invalid RDBMS’ components.

12. If you have java classes invalid?
Use Note 468565.1 How To Reload The APPS Java Class Objects In An Oracle Applications Environment 11i and R12
as an additional step if the JVM was rebuilt for some reason

13. If after/during upgrade you have to De-Install/ Reinstall/Upgrade XMLDB this please use:
Note 402785.1 iSetup dependency with Deinstall and Reinstall of XMLDB
This is the correct procedure to be followed in order to have isetup ok.
If this was not followed, many AZ invalid will appear, so you will have to follow:
Note 832459.1 How to Cleanup Invalid Oracle iSetup (AZ) Tables
Note 1221233.1 Invalid CSR_RULES_B Table And Invalid Objects With CSRRSREG.sql

14. Use Note 551325.1 How to verify or create a Database Object using a odf (adodfcmp) or xdf (FndXdfCmp) file ?
if for some reason a table/view is missing.

15. If for some reason a queue is invalid, see:
Note 275571.1 How to validate an invalid Database Queue in an E-Business Suite Environment

16. For any Intermedia invalid use:
Note 743720.1 Oracle Text: Re-installation and Rebuilding of Applications R12 Oracle Text Indexes
Note 312640.1 Oracle Text: Re-installation of Applications 11i (11.5.10) Oracle Text Indexes

17. The following errors suggest that package was not loaded into the database:
PLS-00201: identifier '<Name>' must be declared
PLS-00304: cannot compile body of '<Package>' without its specification
To solve the issue reload the package using the scripts found on the file system.

18. The following error suggest that package Spec & Body are not in sync:
PLS-00306: wrong number or types of arguments in call to ‘<Identifier>'
To solve the issue find a newer patch that brings both files and apply it.

19. The following errors suggest that some packages that are reported as valid are in fact invalid:
PLS-00907: cannot load library unit <Package_X> (referenced by <Package_Y>)
In this case follow
Note 370137.1 After Upgrade, Some Packages Intermittently Fail with ORA-04065

20. If you have error:
ORA-00904: <table>/<view>.column invalid identifier

or

PLS-00302: component 'column' must be declared

This means that the referred table or view is missing a column.

Open the invalid package .pls file and you will find out the table and the column missing.

Example A. For example you have error:

ORA-00904: "BIS_LEVELS"."ENABLED": invalid identifier

This means the object is invalid because column ENABLED in not available in BIS_LEVELS table.

We need to found out how BIS_LEVELS table is created.

All tables/views/synonyms are created using a file available in:
$PROD_TOP/patch/115/odf
or
$PROD_TOP/patch/115/xdf

Go there and find the file:

Find the object using os command "grep"
grep '<what table you are looking after>' *

For our example:

cd $PROD_TOP/patch/115/xdf
grep BIS_LEVELS *
=> will identify: bis_levels.xdf

So BIS_LEVELS table is created via bis_levels.xdf

Check the file and see if the column is in the file.

If the column is in the file, then you have a de synchronization between your database and your file system.
You have to reload the view/table using Note 551325.1 How to verify or create a Database Object using a odf (adodfcmp) or xdf (FndXdfCmp) file ?
If the column is not in the file, then you need a patch with higher version of bis_levels.xdf.
Ask Oracle Support to give you a patch.
Or check using Patch Wizard for recommended patches for that product, as per Note 976188.1 Patch Wizard

Example B. For example you have error:

PLS-00302: component 'ENT_REFUND_AMOUNT' must be declared

Open the package and do a search after ENT_REFUND_AMOUNT. We will find that offending issue is in:

 INSERT INTO fv_extract_detail_gt_logs
        (
          event_id,
          line_number,
....
          fund_type,
          fed_non_fed_ind,
          acc_refund_amount,
          ent_refund_amount,
          advance_required

The issue is caused by the fact that several columns like ENT_REFUND_AMOUNT are not available in table FV_EXTRACT_DETAIL_GT_LOGS.

We need to find the file that creates fv_extract_detail_gt_logs

$FV_TOP/patch/115/odf
grep FV_EXTRACT_DETAIL_GT_LOGS *

=> we will receive:
fvfet.odf

So this table is created using $FV_TOP/patch/115/odf/fvfet.odf

We need to find out the version of the file:

adident Header $FV_TOP/patch/115/odf/fvfet.odf

We need to find the version of fvfet.odf where column ENT_REFUND_AMOUNT is available in table FV_EXTRACT_DETAIL_GT_LOGS

Check the file and see if the column is in the fvfet.odf file.

If the column is in the file, then you have a de synchronization between your database and your file system.
You have to reload the view/table using Note 551325.1 How to verify or create a Database Object using a odf (adodfcmp) or xdf (FndXdfCmp) file ?
If the column is not in the file, then you need a patch with higher version of fvfet.odf .
Ask Oracle Support to give you a patch.
Or check using Patch Wizard for recommended patches for that product, as per Note 976188.1 Patch Wizard

21. Follow the steps from:
Note 1325394.1 Troubleshooting Guide - invalid objects in the E-Business Suite Environment 11i and 12

22. If you still have the issue raise a Community thread or Service request with:

- Provide the list with all invalids and related error:
Run:

sqlplus apps/apps_pass @$AD_TOP/sql/aderrchk.sql apps apps_pass % invalids.log NOFAIL

Upload invalids.log

- For invalid packages please provide:
- database version

sqlplus apps/...
SQL> select text from user_source where name = upper('<package name>') and line = 2;
example:
SQL> select text from user_source where name = 'AS_ACCESSES_PKG' and line = 2;

- Filesystem version
Take the files generated by the previous select and check the version in filesystem:

adident Header $PROD_TOP/patch/115/sql/……..s.pls
adident Header $PROD_TOP/patch/115/sql/……..b.pls
Example :
adident Header $AS_TOP/patch/115/sql/asxacacs.pls
adident Header $AS_TOP/patch/115/sql/asxacacb.pls

- Give the code creating the error :
Identify the line where the error is encountered.

sqlplus apps/...
SQL> select a.line,a.text from user_source a , user_errors b
where a.name='<PACKAGE NAME>'
and a.name=b.name and a.line=b.line
Example :
select a.line,a.text from user_source a , user_errors b
where a.name= 'AR_UPGRADE_CASH_ACCRUAL'
and a.name=b.name and a.line=b.line

- The query to prove that all prerequisites are applied.
Use next query connected as apps user:

sqlplus apps/...
SQL> select bug_number from ad_bugs where bug_number in ('&patch_number1','&patch_number2',...);

10. How To Solve Failures For Libraries (.pll/.plx)

In patch log you will see errors like:

The following Oracle Forms objects did not generate successfully

<here you will have the list with failing libraries and forms>

1. Ensure you applied all prerequisites for the patch you applied now. 50% of cases are caused by missing prerequisites.

Use next query connected as apps user:

SQL> select bug_number from ad_bugs where bug_number in ('&patch_number1','&patch_number2',...);

2. Check the error in adwork log. In patch log you will find the worker where the job was assigned. Review the adwork log to identify the error.

Or use adadmin and generate the library.

Check adpatch.log/adadmin.log (only last part with this session) and related adworker.log (only last part with this session).
You can use: Note 178722.1 How to Generate a Specific Form Through AD utility adadmin

Note! The error will NOT be found in adpatch or adadmin log. You have to look in related adworker log to identify the real error!

The log files are written to $APPL_TOP/admin/<SID>/log (UNIX), where <SID> is the value of your ORACLE_SID or TWO_TASK variable
or %APPL_TOP%\admin \<SID>\log (Windows), where <SID> is the value of ORACLE_SID or LOCAL.

Libraries can be generated with the same option like forms: adadmin - Generate form files

Enter list of products ('all' for all products) [all] :  
Generate specific forms objects for each selected product [No] ? No
Do you want to regenerate Oracle Forms PL/SQL library files [Yes] ? Yes
Enter libraries and menus to generate, or enter 'all' [all]:

Verify the version of library and compiled library:

adident Header $AU_TOP/resource/<library_name>.pll
adident Header $AU_TOP/resource/<library_name>.plx

If you have the same version then the issue is solved and no need for further investigations.

If not, try next steps.

3. Do a search in MOS using:

- <library_name>.pll
- error identified in adworker log.

4. Check is if you have invalid objects for that product.

SQL> SELECT object_name, object_type, owner
   FROM dba_objects
   WHERE status ='INVALID'
   AND object_name like '<PROD>%';

example:

SQL> SELECT object_name, object_type, owner
   FROM dba_objects
   WHERE status ='INVALID'
   AND object_name like 'AP%';

Try to solve the invalids. For this please use section How to solve invalid objects after patching.

5. If you have errors like:

cannot load library unit APPS.xxx (referenced by APPS.xxx)

then there are dependency timestamp discrepancies between the affected objects.
Please follow:
Note 370137.1 After Upgrade, Some Packages Intermittently Fail with ORA- 04065

6. If you have issues like:

ERROR [code=11] generating library "resource/<library_name>.plx" from input file
$APPL_TOP/au/12.0.0/resource/<library_name>.pll

These type of errors occurs when custom.pll is customized.
So please use the default standard custom.pll. Then to compile it to generate custom.plx.

frmcmp_batch module=$AU_TOP/resource/CUSTOM.pll userid=apps/apps_password output_file=$AU_TOP/resource/CUSTOM.plx module_type=library batch=no compile_all=special

Retest generation of the failing libraries.

If it is working, please ask you developers to review the customizations. These are not anymore inline with your patching level.

7. If you have errors like:

ORA-06501: PL/SQL: program error

then you have a desynchronization with one of the packages in your database.

A very easy way to investigate is to use a text editor (like notepad) and open the .pll failing.

Search after the function/procedure from the error message. Then you can find the de-synchronized package. Check his version:
- database version

SQL> select text from dba_source where name = upper('<package name>') and line < 3;
example:
SQL> select text from dba_source where name = 'AS_ACCESSES_PKG' and line < 3;

- file system version
Take the files generated by the previous select and check the version in file system:

adident Header $PROD_TOP/patch/115/sql/……..s.pls
adident Header $PROD_TOP/patch/115/sql/……..b.pls
Example
adident Header $AS_TOP/patch/115/sql/asxacacs.pls
adident Header $AS_TOP/patch/115/sql/asxacacb.pls

If there is a de-synchronization between database versions and file system versions, ry to recreate it running in sqlplus connected as apps user:

@$PROD_TOP/patch/115/sql/……..s.pls
@$PROD_TOP/patch/115/sql/……..b.pls

Support will identify a patch to synchronize this package with the form.

8. If your patch is finished you can manually generate the library. How to do manual regeneration of library?
Important note! Manual generation of the library will generally NOT help if adadmin failed to generate it!
Manual generation is not doing something magic! If adadmin is not able to generate a library, then manual generation will fail also.
But of you really want to manually generate the form be sure you do this first:

export FORMS_PATH=$AU_TOP/resource:$AU_TOP/forms/US:$AU_TOP/resource/US

The command to manually generate the library is:

$ORACLE_HOME/bin/frmcmp_batch module=$APPL_TOP/au/12.0.0/resource/<library_name>.pll userid=APPS/***** output_file=$APPL_TOP/au/12.0.0/resource/<library_name>.plx module_type=library batch=no compile_all=special

(frmcmp.exe for Windows)

Example:

$ORACLE_HOME/bin/frmcmp_batch module=$APPL_TOP/au/12.0.0/resource/FNDSQF.pll userid=APPS/***** output_file=$APPL_TOP/au/12.0.0/resource/FNDSQF.plx module_type=library batch=no compile_all=special

Note! Use option batch=NO in order to see the error on screen

For 11i use:

export FORMS60_PATH=$AU_TOP/resource:$AU_TOP/forms/US:$AU_TOP/resource/US

f60gen (Windows  ifcmp60.exe)
f60gen userid=APPS/<apps_password> module=$AU_TOP/resource/<library_name>.pll module_type=library output_file=$AU_TOP/resource/<library_name.plx> compile_all=Yes debug=no

Example:

f60gen userid=APPS/<apps_password> module=$AU_TOP/resource/BENACRPT.pll module_type=library output_file=$AU_TOP/resource/BENACRPT.plx compile_all=Yes debug=no

9. If you still have the issue raise a Community thread or Service request with:

- The patch where the issue occurs

- The query to prove that all prerequisites are applied.
Use next query connected as apps user:

SQL> select bug_number from ad_bugs where bug_number in ('&patch_number1','&patch_number2',...);

- adworker log where the library is generated.
(investigation can be better performed if you provide adpatch/adadmin logs and not manual generation)

- library version:

adident Header $AU_TOP/resource/<library_name>.pll

- Provide the list with all invalids and related error:
Run:

sqlplus apps/apps_pass @$AD_TOP/sql/aderrchk.sql apps apps_pass % invalids.log NOFAIL

Upload invalids.log

11. How To Solve Failures For US Forms (.fmb)

In patch log you will see errors like:

The following Oracle Forms objects did not generate successfully

<here you will have the list with failing forms and libraries>

1. Ensure you applied all prerequisites for the patch.
Use next query connected as apps user:

SQL> select bug_number from ad_bugs where bug_number in ('&patch_number1','&patch_number2',...);

2. Be sure that you first solve libraries issues. The form can fail because a library failed before. Use section How to solve failures for libraries (.pll/.plx)

3. Check the error in adwork log. In patch log you will find the worker where the job was assigned. Review the adwork log to identify the error.

Or use adadmin and generate the form.

Check adpatch.log/adadmin.log (only last part with this session) and related adworker.log (only last part with this session).
You can use: Note 178722.1 How to Generate a Specific Form Through AD utility adadmin.

Note! The error will NOT be found in adpatch or adadmin log. You have to look in related adworker log to identify the real error!

The log files are written to $APPL_TOP/admin/<SID>/log (UNIX), where <SID> is the value of your ORACLE_SID or TWO_TASK variable
or %APPL_TOP%\admin \<SID>\log (Windows), where <SID> is the value of ORACLE_SID or LOCAL.

Run adadmin and choose:

Enter list of products ('all' for all products) [all] :  
Generate specific forms objects for each selected product [No] ? Yes
Enter list of languages ('all' for all of the above)[all]:
Choose form: <form_name>.fmx

Verify the version of form. Focus on US forms failures first!

adident Header $AU_TOP/forms/US/<form_name>.fmb
adident Header $PROD_TOP/forms/US/<form_name>.fmx

Note! You will find all the .fmb files in your $AU_TOP
You will find .fmx files in your product top, for example $CSC_TOP or $AP_TOP or $AR_TOP

Depending on the results, there are different investigation paths:

A. Same version for .fmb and .fmx, then it was compiled successfully
Issue is solved, so there is no need for other investigation steps.

B. Different version for US fmx and US fmb
We need to start the investigation in next steps.

4. Do a search in MOS using:

- <form_name>.fmb
- error you identified in adwoker.log

5. Ensure you don't have invalid objects for that specific product. For this you can use section How to solve invalid objects after patching

6. The issue can occurs due to missing rpms for your operating system.
To verify you can use Note 250262.1 "RDA 4 - Health Check / Validation Engine Guide“
After following "Installation Instructions" please follow "Instructions for UNIX type operating systems".
You have to run:

[applmgr@soatest rda]$ ./rda.sh -T hcve
Choose: Oracle E-Business Suite Release 12 (or 11.5.10.2) Preinstall

An html diagnostic will be generated, you have to review and ensure there is no failure.

7. If you have issues with NLS form failing to generate, like:

$PROD_TOP/forms/<NLS>/<form_name>.fmx

where NLS is not US, then please check all the steps from section How to solve failures for NLS forms (.fmb)

8. In many cases the issue is caused by a de-synchronization between form version and packages from database.

A very easy way to investigate is to use a text editor (like notepad) and open the .fmb failing.
Search after the function/procedure from the error message. Then you can find the de-synchronized package. Check his version:

- database version

SQL> select text from dba_source where name = upper('<package name>') and line < 3;
example:
SQL> select text from dba_source where name = 'AS_ACCESSES_PKG' and line < 3;

- filesystem version
Take the files generated by the previous select and check the version in filesystem:

adident Header $PROD_TOP/patch/115/sql/……..s.pls
adident Header $PROD_TOP/patch/115/sql/……..b.pls
Example
adident Header $AS_TOP/patch/115/sql/asxacacs.pls
adident Header $AS_TOP/patch/115/sql/asxacacb.pls

Support will identify a patch to synchronize this package with the form.

9. If you need to manually generate a form:

Important note! Manual generation of the form will generally NOT help if adadmin failed to generate it!

Manual generation is not doing something magic! If adadmin is not able to generate a form, then manual generation will usually fail also.
But of you really want to manually generate the form be sure you do this first:

export FORMS_PATH=$AU_TOP/resource:$AU_TOP/forms/US:$AU_TOP/resource/US

The command to manually generate the form is:

frmcmp_batch module=$AU_TOP/forms/US/<form_name>.fmb userid=APPS/<insert your password> output_file=$PROD_TOP/forms/US/<form_name>.fmx module_type=form batch=NO compile_all=all

(frmcmp.exe for Windows)

Example:

frmcmp_batch module=$AU_TOP/forms/US/CSTFQRAE.fmb userid=APPS/<insert your password> output_file=$BOM_TOP/forms/US/CSTFQRAE.fmx module_type=form batch=NO compile_all=all

Note! Use batch=NO in order to see the error on screen

For 11i use:

export FORMS60_PATH=$AU_TOP/resource:$AU_TOP/forms/US:$AU_TOP/resource/US

for example:

export FORMS60_PATH=$AU_TOP/resource:$AU_TOP/forms/US:$AU_TOP/resource/US

f60gen (Windows  ifcmp60.exe)
f60gen userid=APPS/<apps_password> module=$AU_TOP/forms/US/<form_name>.fmb module_type=form output_file=$PROD_TOP/forms/US/<form_name.fmx compile_all=Yes debug=no

Example:
f60gen userid=APPS/<apps_password> module=$AU_TOP/forms/US/APXINWKB.fmb module_type=form output_file=$AP_TOP/forms/US/APXINWKB.fmx compile_all

10. If you still have the issue raise a Community thread or Service request with:

- the patch where the issue occurs

- the query to prove that all prerequisites are applied.
Use next query connected as apps user:

SQL> select bug_number from ad_bugs where bug_number in ('&patch_number1','&patch_number2',...);

- confirmation that there were no skipped jobs during this patch or prerequisites

- adworker logs where the form is generated.

- forms version for US forms:

adident Header $AU_TOP/forms/US/<form_name>.fmb
adident Header $PROD_TOP/forms/US/<form_name>.fmx

12. How To Solve Failures For NLS Forms (.fmb)

In patch log you will see errors like:

The following Oracle Forms objects did not generate successfully

<here you will have the list with failing forms for a specific language>

1. As a general rule solve first US forms failing.

This will probably solve your NLS forms failing. Use section How to solve failures for US forms (.fmb)

2. Be sure that you solve libraries issues. The form can fail because a library failed before. Use section How to solve failures for libraries (/pll/plx)

3. Check adworker logs.

If not available, use adadmin and generate the form. Check adadmin.log (only last part with this session) and related adworker.log (only last part with this session).

You can use: Note 178722.1 How to Generate a Specific Form Through AD utility adadmin.

Note! The error will NOT be found in adpatch or adadmin log. You have to look in related adworker log to identify the real error!

Compile both US and NLS form:
Run adadmin and choose:

Enter list of products ('all' for all products) [all] :  
Generate specific forms objects for each selected product [No] ? Yes
Enter list of languages ('all' for all of the above)[all]: <NLS>
Choose form: <form_name>.fmx

Verify the version of US and NLS form:

adident Header $AU_TOP/forms/US/<form_name>.fmb
adident Header $PROD_TOP/forms/US/<form_name>.fmx
adident Header $AU_TOP/forms/<NLS>/<form_name>.fmb
adident Header $PROD_TOP/forms/<NLS>/<form_name>.fmx

Note! You will find all the .fmb files in your $AU_TOP
You will find .fmx files in your product top, for example $CSC_TOP or $AP_TOP or $AR_TOP

Depending on the results, different investigation paths:

A. Same version for .fmb and .fmx, then it was compiled successfully
Issue is solved, so there is no need for other investigation steps.

B. Different version for US fmx and US fmb
Then the issue is not related to NLS. It needs investigation on US patch.

C. Different version for US form fmb and NLS form fmb
You can identify what patch was applied only for US language and not for NLS.

This is the best way to ensure that ALL your NLS files are in synchronization with US files!!!
This can save a lot of your time to perform other investigations.
Oracle will generate for you a custom specific patch that will ONLY contain the desynchronized files (so it is not a big patch). Take in account that you can usually apply NLS patches in hotmode and you will not need to bring your production down.

Check:
Note 1478859.1 Troubleshooting NLS issues with Oracle Applications
section 3. NLS / MLS Forms Compilation Issues
Note 1537973.1 How To Troubleshoot NLS Forms Generation Errors When Applying An NLS Patch In R12 Or 11i

4. Do a search in MOS using:

- <form_name>.fmb
- error you identified in adwoker.log

5. The issue can occurs due to missing rpms for your operating system. To verify you can use Note 250262.1 "RDA 4 - Health Check / Validation Engine Guide“

After following "Installation Instructions" please follow "Instructions for UNIX type operating systems".

You have to run:

[applmgr@soatest rda]$ ./rda.sh -T hcve
Choose: Oracle E-Business Suite Release 12  or 11.5.10.2 Preinstall

An html diagnostic will be generated, you have to review and ensure there is no failure.

6. If you still have the issue raise a Community thread or Service request with:

- the patch where the issue occurs

- the query to prove that all prerequisites are applied.
Use next query connected as apps user:

SQL> select bug_number from ad_bugs where bug_number in ('&patch_number1','&patch_number2',...);

- adworker log where the form is generated.

- forms version for US and NLS:

adident Header $AU_TOP/forms/US/<form_name>.fmb
adident Header $PROD_TOP/forms/US/<form_name>.fmx
adident Header $AU_TOP/forms/<NLS>/<form_name>.fmb
adident Header $PROD_TOP/forms/<NLS>/<form_name>.fmx

13. How To Solve Failures For .ldt (FNDLOAD) Files

In the patch log you will see errors like:

FAILED: file <filename>.ldt on worker ....

In adworker log you will see:

FNDLOAD APPS/***** 0 Y UPLOAD @PRODUCT:patch/115/import/<lct_file>.lct   @PRODUCT:patch/115/import/US/<filename>.ldt -
Connecting to APPS......Connected successfully.
Calling FNDLOAD function.
Returned from FNDLOAD function.
Log file: apps_st/appl/admin/<SID>/log/US_<filename>_ldt.log
Error calling FNDLOAD function.

1. Ensure you applied all prerequisites for the patch.
Use next query connected as apps user:

SQL> select bug_number from ad_bugs where bug_number in ('&patch_number1','&patch_number2',...);

2. How to identify the error:

In the main patch log you will see:

FAILED: file <filename1>.ldt on worker 1.
FAILED: file <filename2>.ldt on worker 2.
FAILED: file <filename3>.ldt on worker 3.

for example:

FAILED: file hxczzhxcmpcm0077.ldt on worker 1.
FAILED: file hxczzhxcmpcm0080.ldt on worker 2.
FAILED: file hxczzhxcmpcm0081.ldt on worker 3.

In the related adworker log you will see:

FNDLOAD APPS/***** 0 Y UPLOAD @PRODUCT:patch/115/import/<lct_file>.lct   @PRODUCT:patch/115/import/US/<filename>.ldt -
Connecting to APPS......Connected successfully.
Calling FNDLOAD function.
Returned from FNDLOAD function.
Log file: apps_st/appl/admin/<SID>/log/US_<filename>_ldt.log
Error calling FNDLOAD function.

In order to identify the error you have to check:
$APPL_TOP/admin/<SID>/log/US_<filename>_ldt.log
or
$APPL_TOP/admin/<SID>/log/NLS_<filename>_ldt.log
for example:
$APPL_TOP/admin/<SID>/log/F_<filename>_ldt.log

So for our example please check:
$APPL_TOP/admin/<SID>/log/US_hxczzhxcmpcm0077_ldt.log

Note! In R11 the log is has format: lxxxxxx.req, for example
$APPL_TOP/admin/<SID>/log/l936179.req

3. Do a search on MOS using:

- filename.ldt
- error from $APPL_TOP/admin/<SID>/log/<language>_<filename>_ldt.log or lxxxxxx.req

Take the error from <language>_<filename>_ ldt.log or lxxxxxx.req and do a simple search in MOS and Patching Community.
For example, let's say you have itaauditschema.ld failing with:

Error loading seed data for FND_AUDIT_SCHEMAS: ORACLE_USERNAME = APPLSYS,
ORA-01400: cannot insert NULL into ("FND"."FND_AUDIT_SCHEMAS"."CREATION_DATE")
ORA-01403: no data found

Do a simple search using:
- itaauditschema.ldt
- ORA-01400
- ORA-01403
and you will find the note to solve your error.

4. If you have error:

A database error occurred:
ORA-24816: Expanded non LONG bind data supplied after actual LONG or LOB column

Review and implement: Note 452095.1 ORA-24816: Expanded Non Long Bind Data Supplied After Actual Long Or Lob Column

5. If you have error:

Altering database NLS_LANGUAGE environment to FRENCH/CANADIAN FRENCH
A database error occurred:
ORA-20001: Oracle error -20001: ORA-20001: Oracle error -12705: ORA-12705: Cannot access NLS data files or invalid environment specified
has been detected in fnd_global.set_nls.set_parameter('NLS_LANGUAGE','FRENCH').
has been detected in fnd_global.set_nls.
ORA-06512: at "APPS.APP_EXCEPTION", line 72
ORA-06512: at "APPS.FND_GLOBAL", line 245
ORA-06512: at "APPS.FND_GLOBAL", line 1426
ORA-06512: at "APPS.FND_GLOBAL", line 1454
ORA-06512: at line 1

Unable to determine the language code for the current session

Then you have to edit the Database Tier context XML file to have the following :
<ORA_NLS oa_var="s_db_oranls" osd="unix">RDBMS_ORACLE_HOME/nls/data/9idata</ORA_NLS>
After run autoconfig on Database Tier.

Reference Note 1273367.1 Translation Synchronization Patch For French Language Failing With ORA-12705

6. Please check alert.log for the moment .ldt file was loaded. You can see there additional errors.

7. Open the .ldt file failing and look at the beginning of the file.
You will see there the .lct file and minimum version needed to correctly load the .ldt file.
Check the .lct file version in your system:
adident Header $PRODUCT_TOP/patch/115/import/<lct_file>.lct
If this is lower than the one you found in the .ldt file, then you missed a prerequisite.

8. Enable FNDLOAD debug.

Run FNDLOAD with FLOAD_DEBUG=TRUE option and upload the trace and request files.

FLOAD_DEBUG=TRUE Indicating running FNDLOAD in debug mode, only used for optimized FNDLOAD. In this mode, FNDLOAD will output the dynamically generated wrapper code into a .pls file, and also dump seed data into permanent staging tables for debug use.

For example:

$FND_TOP/bin/FNDLOAD apps/apps 0 Y UPLOAD  $FND_TOP/patch/115/import/afscprof.lct $BNE_TOP/patch/115/import/US/bnegldi.ldt - FLOAD_DEBUG=TRUE

9. Enable additional debug like:

SQL> alter system set events '10046 TRACE NAME CONTEXT FOREVER, LEVEL 4';

Useful notes:
Note 21154.1 EVENT: 10046 "enable SQL statement tracing (including binds/waits)"
-> With SYSTEM and SESSION settings
Note 218105.1 Introduction to ORACLE Diagnostic EVENTS
-> How to see details for one ORA-xxxx
Note 376442.1 Master Note: Recommended Method for Obtaining 10046 trace
Note 219968.1 SQL*Net, Net8, Oracle Net Services - Tracing and Logging

After trace is enabled, restart failed worker and look after additional errors.

10. If you still have the issue raise a Community thread or Service request with:
- the patch where the issue occurs

- the query to prove that all prerequisites are applied.
Use next query connected as apps user:

select bug_number from ad_bugs where bug_number in ('&patch_number1','&patch_number2',...);

- confirmation that there were no skipped jobs during this patch or prerequisites
- adworker logs where the .ldt is failing
- <language>_<filename>_ ldt.log (this is very important! Here we will find the error)
- .ldt file version:

adident Header $PRODUCT_TOP/patch/115/import/US/<filename>.ldt
or
adident Header $PRODUCT_TOP/patch/115/import/<NLS>/<filename>.ldt

- .lct file version:

adident Header $PRODUCT_TOP/patch/115/import/<lct_file>.lct

- check if the product is used:

SQL> select decode(nvl(a.APPLICATION_short_name,'Not Found'),
        'SQLAP','AP','SQLGL','GL','OFA','FA',
        'Not Found','id '||to_char(fpi.application_id),
        a.APPLICATION_short_name) apps,
        decode(fpi.status,'I','Installed','S','Shared',
               'N','Inactive',fpi.status) status,
        fpi.product_version,
        nvl(fpi.patch_level,'-- Not Available --') Patchset,
        to_char(fpi.last_update_date,'dd-Mon-RRRR') "Update Date"
from fnd_oracle_userid o, fnd_application a, fnd_product_installations fpi
where fpi.application_id = a.application_id(+)
  and fpi.oracle_id = o.oracle_id(+)
and a.APPLICATION_short_name ='&product'

For example if gmebrrui.ldt is failing check if GME is Installed/Shared.

14. How To Solve Failures For .sql Files

In patch log you will see errors like:

FAILED: file <filename1>.sql on worker 1.

1. Ensure you applied all prerequisites for the patch you applied now. 50% of cases are caused by missing prerequisites.

Use next query connected as apps user:

SQL> select bug_number from ad_bugs where bug_number in ('&patch_number1','&patch_number2',...);

2. Check the related adworker.log (only last part with this session).

Important note! The error can be found ONLY in adworker logs (not in patch log).

The log files are written to $APPL_TOP/admin/<SID>/log (UNIX), where <SID> is the value of your ORACLE_SID or TWO_TASK variable
or %APPL_TOP%\admin \<SID>\log (Windows), where <SID> is the value of ORACLE_SID or LOCAL.

3. Search MOS using:
- <file_name>.sql
- error identified in adworker log

4. Run the script manually. You can take the entire syntax and parameters looking in adwork log file.
If manually running the script is ok, then you can skip the worker.

5. Check alert.log for any new indication about the error.

6. Enable additional debug like:

SQL> alter system set events '10046 TRACE NAME CONTEXT FOREVER, LEVEL 4';

After trace is enabled, restart failed worker and look after additional errors.

7. Check if you have invalid objects for that product. You can use:

SQL> select owner, object_name, object_type, last_ddl_time
from dba_objects
where status=’INVALID’ and
object_name like '<PRODUCT>%'

Try to fix invalid object for that product. Use section How to solve invalid objects after patching

8. If you still have the issue raise a Community thread or Service request with:
- The patch where the issue occurs

- the query to prove that all prerequisites are applied.
Use next query connected as apps user:

SQL> select bug_number from ad_bugs where bug_number in ('&patch_number1','&patch_number2',...);

- adworker log where the sql fails.

- .sql file version:

adident Header $<PRODUCT>_TOP/patch/115/sql/<filename>.sql

- Provide the list with all invalids and related error:
Run:

SQL> sqlplus apps/apps_pass @$AD_TOP/sql/aderrchk.sql apps apps_pass % invalids.log NOFAIL

Upload invalids.log

15. How To Solve Failures For .odf Files

In patch log you will see errors like:

ATTENTION: All workers either have failed or are waiting:
FAILED: file <filename>.odf on worker #

1. Ensure you applied all prerequisites for the patch.
Use next query connected as apps user:

SQL> select bug_number from ad_bugs where bug_number in ('&patch_number1','&patch_number2',...);

2. Check the error in adwork log. In patch log you will find the worker where the job was assigned. Review the adwork log to identify the error.

Note! The error will NOT be found in adpatch log. You have to look in related adworker log to identify the real error!

The log files are written to $APPL_TOP/admin/<SID>/log (UNIX), where <SID> is the value of your ORACLE_SID or TWO_TASK variable
or %APPL_TOP%\admin \<SID>\log (Windows), where <SID> is the value of ORACLE_SID or LOCAL.

You will find something similar with:

Comparing objects in username <user> with ODF file $PRODUCT_TOP/patch/115/odf/<odf file>.odf
....
Start time for statement below is: date
ALTER TABLE <table_name> STORAGE (FREELISTS 4)

Statement executed.

=> then the alter object commands
-> then the error

3. Do a search in MOS using:
- <file_name>.odf
- error from adwork log

4. Review alert.log.
You can find additional information to solve the issue

5. If you have error:

The CREATE INDEX statement above failed because there is another index on the same columns.
Looking for the existing index on the same columns...
AD Worker error:
Unable to find existing index on the same columns

This is caused by impossibility of creating another index if an index already exists on these columns. Usually custom index are created by customer, so standard index can not be created on the same columns.
You should drop your custom index.
Review Note 1549195.1 Adpatch fails with: "The CREATE INDEX statement above failed because there is another index on the same columns"

6. If you have error:

The index cannot be created as the table has duplicate keys.
Use the following SQL statement to identify the duplicate keys:
SELECT ...
FROM ...
GROUP BY ...
HAVING count(*)>1
AD Worker error:
Unable to compare or correct tables or indexes or keys because of the error above

This is data corruption. The issue occurs if you have duplicate entries for the column where you want to create an index. You should find a note on MOS.
The solution differs, could be:
- delete the duplicate rows
- run a concurrent request to delete the duplicate rows
- rename the unique key value

For example:
Note 987878.1 Create Index CSF_PHONETIC_VALUES_U3 Fails As The Table Has Duplicate Keys
Note 430673.1 icxwtab.odf is unable to create index ICX_TRANSACTIONS_U1

7. If you have error:

AD Worker - aduobbrt2: INFO: ORA-03211: The segment does not exist or is not in a valid state
A segment specified in the DBMS_SPACE_ADMIN operation does exist or is not in a state appropriate for this operation.
This usually occurs for a temporary table without a segment allocated. But during execution it is invoked package DBMS_ADMIN_SPACE which requires the segment to exist.
Usually we have to drop the temporary table.

Check:
Note 372945.1 Applying 3480000 csdcsd1.odf Aduobbrt2: Info: ORA-03211: The Segment Does Not Exist
Note 1546496.1 ORA-03211 During 12.1 Driver In Cstcbom.odf
Note 457866.1 Ad Worker Fails For Wiphdr.Odf In Patch.6116755 The segment does not exist or is not in a valid state
Note 1286823.1 Worker GMDCOA.odf Fails When Upgrading To R12.1 from Rel 11i

8. If you have error:

ORA-00054: resource busy and acquire with NOWAIT specified
The object is locked by another process.
Try next steps:
- use adctrl to restart the failed worker
- make sure that services, specially Concurrent Managers are down during patch application
- identify the lock and release it:

SQL> select * from v$locked_object;

- bounce database: shutdown immediate/normal and startup

9. If you have poxcom.odf failing with ORA-00911: invalid character
The view po_lines_xml definition has a ';' character which is not allowed in odf files.
To solve this, apply any patch that delivers poxcom.odf version 120.18.12000000.33 or higher, for example:
Patch 16369996:R12.PO.A
Patch 16214305:R12.PO.A
Patch 16061415:R12.PO.A
Patch 15893161:R12.PO.A
Patch 15843328:R12.PO.A

10. If you need to manually load an .odf file, you can use:

Note 551325.1 How to verify or create a Database Object using a odf (adodfcmp) or xdf (FndXdfCmp) file ?

adodfcmp odffile=<Filename> mode=<Objects to be checked/updated> changedb=yes  userid=<User>/<Password> touser=<User>/<Password> priv_schema=SYSTEM/<Password>
where mode can be :
baseonly, tables, indexes, noindexes, sequences, views, grants

For exemple:
adodfcmp userid=as/ars mode=views odffile=$AS_TOP/patch/115/odf/asdss.odf changedb=yes touser=apps/apps_password priv_schema=apps/apps_password
 
adodfcmp userid=ar/ar mode=tables  odffile=$AR_TOP/patch/115/odf/arhz.odf changedb=yes touser=apps/apps_password priv_schema=apps/apps_password

11. If you still have the issue raise a Community thread or Service request with:

- the patch where the issue occurs

- the query to prove that all prerequisites are applied.
Use next query connected as apps user:

SQL> select bug_number from ad_bugs where bug_number in ('&patch_number1','&patch_number2',...);

- adworker log where the .odf is failing.

- .odf version:

adident Header $<PRODUCT>_TOP/patch/115/odf/<filename>.odf

16. How To Solve Failures For .xdf Files (oracle.apps.fnd.odf2.FndXdfCmp)

In patch log you will see errors like:

ATTENTION: All workers either have failed or are waiting:
FAILED: file <filename1>.xdf on worker #.

1. Ensure you applied all prerequisites for the patch.
Use next query connected as apps user:

SQL> select bug_number from ad_bugs where bug_number in ('&patch_number1','&patch_number2',...);

Invoking Utility FndXdfCmp ...
Class: oracle.apps.fnd.odf2.FndXdfCmp
Method: applyXDF
Arguments: &un_gl &pw_gl &un_apps &pw_apps &jdbc_protocol &jdbc_db_addr table &fullpath_<product>_patch/115/xdf_<filename>.xdf &fullpath_fnd_patch/115/xdf_xsl index_category=large parallel_index_threshold=20000
==========================================================

XDF file application started.

================================================================================
Applying XDF file : /$APPL_TOP/<product>/12.0.0/patch/115/xdf/<filename>.xdf
================================================================================

=> there comes the error.

3. Do a search in MOS using:
- <file_name>.xdf
- error from adwork log

4. Review alert.log.
You can find additional information to solve the issue, for example is not able to extend tablespace.
the error you will see is: No AOL metadata present in the XDF file

If you find in alert.log next error: ORA-1652: unable to extend temp segment by
we recommend to increased the TEMP space availability.

5. If you have error: Error message is ORA-01430
like:

Invoking Utility FndXdfCmp ...
Class: oracle.apps.fnd.odf2.FndXdfCmp
Method: applyXDF
...
  ALTER TABLE table_name ADD ( column_name column_type )

Start time for statement above is date
End time for statement above is date
Error in executing statement ALTER TABLE table_name ADD ( column_name type )
Error message is ORA-01430: la colonna che si sta aggiungendo esiste giÃ  nella tabella

Unset the locale environment variable on the Applications and Database tiers. Enter the following on the servers to confirm it's not set as the example in the cause above:
$locale
Bounce database after the confirming the locale environment variable is not set.
Re-apply the patch from beginning.

Review:
Note 1504251.1 Patch Is Failing Running FndXdfCmp With ORA-01430

6. If you have error like:

Exception in thread "main" java.sql.SQLException: Listener refused the connection with the following error:
ORA-12514, TNS:listener does not currently know of service requested in connect descriptor
The Connection descriptor used by the client was:
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=t106pldbms02.nehr.dev)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=HTBSITB2)))

at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:112)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:261)
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:387)
at oracle.jdbc.driver.PhysicalConnection.(PhysicalConnection.java:439)
at oracle.jdbc.driver.T4CConnection.(T4CConnection.java:165)
at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:35)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:801)
at java.sql.DriverManager.getConnection(DriverManager.java:525)
at java.sql.DriverManager.getConnection(DriverManager.java:171)
at oracle.apps.ad.worker.AdJavaWorker.getAppsConnection(AdJavaWorker.java:1027)
at oracle.apps.ad.worker.AdJavaWorker.main(AdJavaWorker.java:276)

Review:
Note 1461916.1 ADPATCH Appears To Hang When Processing xdf files. Connection Error Reported In Worker Logs

7. If you have error like: A Different index with same column name exists
you have custom indexes that should be dropped.
You can double check if the index is custom on: ETRM site (here you have all standard objects).
Please check: Note 1514897.1 Patch Application Fails Running FndXdfCmp To Create An Index With Error: A Different Index With Same Column Name Exists

8. There are many errors caused by incorrect java settings.

You can experience errors like:
- Exception occurred :C: Unable to resolve type
- Exception occurred :Fail to construct descriptor: Unable to resolve type

You have incorrect settings for java.
Please review the checks (depending on your java version and EBS version) from:

Note 418664.1 Overview of Using Java with Oracle E-Business Suite Release 12
Note 455492.1 Using Latest Update of Java 6.0 with Oracle E-Business Suite Release 12
Note 384249.1 Using Latest Update of JDK 5.0 with Oracle E-Business Suite Release 12
Note 300482.1 Overview of Using Java with Oracle E-Business Suite Release 11i
Note 401561.1 Using J2SE Version 6 with Oracle E-Business Suite 11i
Note 304099.1 Using J2SE Version 5.0 with Oracle E-Business Suite 11i, Release 11.5.10
Note 246105.1 Upgrading to J2SE 1.4.2 with Oracle Applications 11i
Note 130091.1 Upgrading Oracle Applications 11i to use JDK 1.3

Similar issues are discussed in:

Note 294457.1 Avoiding XDF Errors by Verifying CLASSPATH Settings

9. If you need to manually load an .xdf file, you can use:

Note 308427.1 The XDF Comparison Utility (FndXdfCmp)
Note 551325.1 How to verify or create a Database Object using a odf (adodfcmp) or xdf (FndXdfCmp) file ?

You have to source apps environment and then perform:

adjava -ms128m -mx256m -nojit oracle.apps.fnd.odf2.FndXdfCmp apps <password_for_apps> apps <password_for_apps>   THIN "HOST:port:SID"  type $FND_TOP/patch/115/xdf/jtf_pf_tabletype.xdf  $FND_TOP/patch/115/xdf/xsl

for example:
adjava -ms128m -mx256m -nojit oracle.apps.fnd.odf2.FndXdfCmp ego <password_for_ego> apps <password_for_apps>   THIN "testro15.ro.oracle.com:1526:VIS"  table $EGO_TOP/patch/115/xdf/ego_mtl_sy_items_ext_b.xdf  $FND_TOP/patch/115/xdf/xsl

10. If you still have the issue raise a Community thread or Service request with:

- the patch where the issue occurs

- the query to prove that all prerequisites are applied.
Use next query connected as apps user:

SQL> select bug_number from ad_bugs where bug_number in ('&patch_number1','&patch_number2',...);

- adworker log where the .xdf is failing.

- .xdf version:

adident Header $<PRODUCT>_TOP/patch/115/xdf/<filename>.xdf

17. How To Solve Failures For XLIFFImporter (.xlf) Files

In the patch log you will see errors like:

ATTENTION: All workers either have failed or are waiting:

          FAILED: file XLIFFImporter.class on worker #.

1. The error are displayed in adwork*.log located in $APPL_TOP/admin/$TWO_TASK/log directory.
You need to check the related adworker log for further details.

2. Most encountered errors are like:
"Could not import translations in repository"
when NLS patch is applied.

Invoking Utility XLIFFImporter ...
Class: oracle.jrad.tools.trans.imp.XLIFFImporter
Method: processXLIFF
Arguments: &fullpath:<product>:<path>:<filename>.xlf -username &un_apps -password &pw_apps -dbconnection &jdbc_db_addr
TimeStamp : [date]
==========================================================
Could not import translations in repository  : "<path>/<filename>" not found in repository.
==========================================================
Done calling the utility function. Return Code = [1] TimeStamp = date
Updating task with status 1

AD Worker error:
The utility XLIFFImporter returned error for the above task.

The error "Could not import translations in repository : "<path>/<filename>" not found in repository" typically means the NLS patch was not applied immediately after the base install/upgrade.
The impact of this failure (can not load the .xdf file) is that you can get an error message "No page found" when using that specific product in that specific language.
A potential cause is that the US xml file is just missing in the repository.

Let's say you have error:

adjava -Xmx512M -nojit oracle.jrad.tools.trans.imp.XLIFFImporter &fullpath:ap:mds/oie/policy/category/meals/webui/E:OIE_POL_MEALS_RULES_PAGE.xlf -username &un_apps -password &pw_apps -dbconnection &jdbc_db_addr

Reading product information from file...

Reading language and territory information from file...

Reading language information from applUS.txt ...
 Temporarily resetting CLASSPATH to: ...

 Calling $COMMON_TOP/util/java/jdk1.5.0/bin/java ...
Could not import translations in repository  : "/oracle/apps/ap/oie/policy/category/meals/webui/OIE_POL_MEALS_RULES_PAGE" not found in repository.

So you can try to reload it in the repository as follow :

2.1. Ensure that related .xml file is available in US directory
For example in our test case ensure you have this file: $AP_TOP/mds/oie/policy/category/meals/webui/OIE_POL_MEALS_RULES_PAGE.xml exists

2.2. Reload XML (US data)

java oracle.jrad.tools.xml.importer.XMLImporter \
$AP_TOP/mds/oie/policy/category/meals/webui/OIE_POL_MEALS_RULES_PAGE.xml \
-username APPS -password <APPS Password> -dbconnection \
"(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=<hostname.domain>) \
PORT=<port number>))(CONNECT_DATA=(SID=<your SID>)))" \
-rootdir $PRODUCT_TOP/mds

Please replace the dbconnection HOST/PORT/SID with the string valid for your environment in the above command, like:

java oracle.jrad.tools.xml.importer.XMLImporter \
$AP_TOP/mds/oie/policy/category/meals/webui/OIE_POL_MEALS_RULES_PAGE.xml \
-username APPS -password APPS
-dbconnection "(DESCRIPTION=(LOAD_BALANCE=YES)(FAILOVER=YES)(ADDRESS_LIST=(ADDRESS=(PROTOCOL=tcp)(HOST=hostname.domain.com)(PORT=1521)))(CONNECT_DATA=(SID=TEST)))"
-rootdir $AP_TOP/mds -rootPackage /oracle/apps

2.3. restart the failed job
or manually reload XLF (NLS data)

java oracle.jrad.tools.trans.imp.XLIFFImporter \
-username APPS -password <APPS password> -dbconnection \
"(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=<hostname. domain>) \
PORT=<port number>))(CONNECT_DATA=(SID=<your SID>)))“ \
$<path>/<filename>.xlf

In order to see what is loaded in the database you can use:

SQL> select RPAD(f.filename,30), f.file_id, RPAD(v.version,20), f.creation_date, f.last_update_date
from AD_FILES f, AD_FILE_VERSIONS v
where f.file_id = v.file_id
and f.filename LIKE '<filename_failing>';

For example:
SQL> select RPAD(f.filename,30), f.file_id, RPAD(v.version,20), f.creation_date, f.last_update_date
from AD_FILES f, AD_FILE_VERSIONS v
where f.file_id = v.file_id
and f.filename LIKE 'OIE_POL_MEALS_RULES_PAGE%';

If reloading the US related file is not solving the failure you can:
- skip this failed job and continue your patching activity
- after you finish patching, request a Translation Synchronization Patch (TSP)
This patch will synchronize your NLS with your US files and should not raise this failure anymore.

3. If you have errors like: jre: No such file or directory

For example:

FAILED: file XLIFFImporter.class on worker 1.

adjava -Xmx512M -nojit oracle.jrad.tools.trans.imp.XLIFFImporter &fullpath:icx:
mds/por/rcv/webui/ZHS:IcxPorRcvRvwPG.xlf -username &un_apps -password &pw_apps -
dbconnection &jdbc_db_addr

Reading product information from file...

Reading language and territory information from file...

Reading  language information from applUS.txt ...
  Temporarily resetting CLASSPATH to:
....
  Calling /.../j2sdk1.4.2_13/bin/jre ...
/bin/sh: line 1: /.../j2sdk1.4.2_13/bin/jre: No such file or directory

You have to verify java/jre settings. Use next notes depending on your EBS version and java version:

4. If you have Performance issues on all XLIFFImporter.class jobs which take a lot of time to complete it is possible that you are missing Patch 8576725 "12.1.1 NLS PATCHING PERFORMANCE FIX"

Check Note 839978.1 12.1.1 NLS Upgrade Patch 6678700 : Performance Installation Issues With XLIFFImporter Java Jobs

5. If you have errors like:

oracle.jdbc.driver.T4CSocketInputStreamWrapper.readNextPacket

  at oracle.jdbc.driver.T4CSocketInputStreamWrapper.readNextPacket(T4CSocketInputStreamWrapper.java:122)
  at oracle.jdbc.driver.T4CSocketInputStreamWrapper.read(T4CSocketInputStreamWrapper.java:78)
  at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1179)
  at oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:1155)
  at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:279)
  at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:186)
  at oracle.jdbc.driver.T4CTTIoauthenticate.doOAUTH(T4CTTIoauthenticate.java:366)
  at oracle.jdbc.driver.T4CTTIoauthenticate.doOAUTH(T4CTTIoauthenticate.java:752)
  at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:359)
  ... 8 more

Edit the sqlnet.ora and set the parameter:
SQLNET.INBOUND_CONNECT_TIME=0

Edit the listener.ora and set the parameter
connect_timeout_ = 0

Check Note 1461326.1 Patch 13612777 Arabic Hangs While Running XLIFFLoader.class

6. Ensure you have correct synchronization between DB character set and apps character set.
Please review the following document which discusses the details of globalization configuration and character set selection for R12:
Note 393861.1 Globalization Guide for Oracle Applications Release 12
Note 333785.1 Oracle Applications 11i Internationalization Guide
Application Tier (Middle Tier) - APPL_TOP Character Set and Language

Detailed explanation is available in Note 1478859.1 Troubleshooting NLS issues with Oracle Applications
section: 1. Invalid/junk characters

Note! This is a mandatory step that should be checked before starting patching session!

7. If you still have the issue raise a Community thread or Service request with:

- the patch failing

- the query to prove that all prerequisites are applied.
Use next query connected as apps user:

SQL> select bug_number from ad_bugs where bug_number in ('&patch_number1','&patch_number2',...);

- is this a failure or a hanging issue?
- provide details about patching history (fresh install, upgrade, NLS patches applied)
- related adworker log available in in $APPL_TOP/admin/$TWO_TASK/log directory.
- results of:

SQL> select RPAD(f.filename,30), f.file_id, RPAD(v.version,20), f.creation_date, f.last_update_date
from AD_FILES f, AD_FILE_VERSIONS v
where f.file_id = v.file_id
and f.filename LIKE '<filename_failing>';

18. How To Solve Failures For XLIFFLoader (.xlf) files

In patch log you will see errors like:

ATTENTION: All workers either have failed or are waiting:

FAILED: file XLIFFLoader.class on worker #.

1. If you have errors like:

No template found with application short name ... and template code ...

Parameters passed to XLIFFLoader...
[FILE_NAME] [$PRODUCT_TOP/patch/115/publisher/templates/<NLS>/<filename>.xlf]
[APPS_SHORT_NAME] [<product>]
[UPLOAD] [UPLOAD]
[DB_USERNAME] [APPS]
[JDBC_CONNECTION] [(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=host)(PORT=port))(CONNECT_DATA=(SERVICE_NAME=sid)))]
[DB_PASSWORD] [******]
[TEMPLATE_CODE] [<template_code>]

Target file: <filename>.xlf
Start uploading...
oracle.apps.xdo.XDOException: No template found with application short name ...and template code ...
at oracle.apps.xdo.oa.util.XLIFFLoader.processUpload(XLIFFLoader.java:681)
at oracle.apps.xdo.oa.util.XLIFFLoader.process(XLIFFLoader.java:565)
at oracle.apps.xdo.oa.util.XLIFFLoader.main(XLIFFLoader.java:1073)

The error reports that the required based US template is not loaded.
US template should be loaded before the same NLS templates can be loaded successfully.

To identify if that specific US template is loaded please use:

SQL> select application_short_name, template_code, mls_language, mls_territory from xdo_templates_b
where application_short_name ='<apps_short_name>' and template_code='<template_code>';

For example if your error is with:

[FILE_NAME] [$GFM_TOP/patch/115/publisher/templates/<NLS>/GMFDSUR.xlf]
[APPS_SHORT_NAME] [GMF]
...
[TEMPLATE_CODE] [GMFDSUR]

you should use:

SQL> select application_short_name, template_code, mls_language, mls_territory from xdo_templates_b
where application_short_name ='GMF' and template_code='GMFDSUR';

Here you can see if US template is loaded or not.

Correct display should be:

APPLICATION_SHORT_NAME
--------------------------------------------------
TEMPLATE_CODE
--------------------------------------------------------------------------------
MLS_LA MLS_TE
------ ------
GMF
GMFDSUR
en     US

If you don't have here the file loaded for US, then you have to load it first using FNDLOAD.
It should be possible to identify the required ldt to load the data by searching for the template name:

cd $<apps_short_name>_TOP/patch/115/import/US
grep <template_code> *

=> you will find here the file that delivers the template.

For our example:

cd $GMF_TOP/patch/115/import/US
grep GMFDSUR *

=> you will find that US template is loaded via gmfxdodefn.ldt

In order to load the file use:

FNDLOAD apps/apps_password 0 Y UPLOAD $XDO_TOP/patch/115/import/xdotmpl.lct $<apps_short_name>_TOP/patch/115/import/US/<file_with_US_template>.ldt

For our example:
FNDLOAD apps/apps 0 Y UPLOAD $XDO_TOP/patch/115/import/xdotmpl.lct $GMF_TOP/patch/115/import/US/gmfxdodefn.ldt

Now

SQL> select application_short_name, template_code, mls_language, mls_territory from xdo_templates_b
where application_short_name ='GMF' and template_code='GMFDSUR';

will display:

APPLICATION_SHORT_NAME
--------------------------------------------------
TEMPLATE_CODE
--------------------------------------------------------------------------------
MLS_LA MLS_TE
------ ------
GMF
GMFDSUR
en     US

If above SQL now returns rows, then the xlf file can be successfully loaded.
Use adctrl to restart failed worker.
If workers were skipped then the NLS template data can be loaded manually using:

adjava -ms128m -mx256m -nojit oracle.apps.xdo.oa.util.XLIFFLoader UPLOAD -DB_USERNAME apps -DB_PASSWORD <pw_apps> -JDBC_CONNECTION <hostname>:<port>:<sid> -APPS_SHORT_NAME <apps_short_name> -TEMPLATE_CODE <template_code> -FILE_NAME $<apps_short_name>_TOP/patch/115/publisher/templates/<NLS>/<filename>.xlf

For our example:
adjava -ms128m -mx256m -nojit oracle.apps.xdo.oa.util.XLIFFLoader UPLOAD -DB_USERNAME apps -DB_PASSWORD <pw_apps> -JDBC_CONNECTION <hostname>:<port>:<sid> -APPS_SHORT_NAME GMF -TEMPLATE_CODE GMFDSUR -FILE_NAME $GMF_TOP/patch/115/publisher/templates/<NLS>/GMFDSUR.xlf

Similar steps in: Note 1357075.1 Xliffloader Errors When Applying 6678700(NLS) Driver Patch

If you have this specific error:
No template found with application short name OFAand template code FAXRASIM_en_US
FAXRASIM_en_US.xlf should not be included in an NLS patch as this is US file. There is a problem with the patch you are applying.
You will have to apply another replacement patch. Please review:
Note 1265621.1 Patch 9553921 (NLS) Fails Application: No Template Found With Template Codes FAXRASUN_en_US And FAXRASIM_en_US
Note 1351833.1 NLS Patch 11853917:R12.FA.B OR NLS Patch 12810804:R12.FA.B Fails

If you have this specific error:
No template found with application short name SQLAPand template code APTRXFEE_XML-LANGUAGE
No template found with application short name SQLAPand template code APOBRR
The patch you are applying is missing the ldt file which is responsible for updating the xdo_templates where template code is updated.
APTRXFEE_XML definition is delivered via $AP_TOP/patch/115/import/US/ap12axdo.ldt
This file was not loaded in the system. You need a patch to deliver correct version of $AP_TOP/patch/115/import/US/ap12axdo.ldt
Please review:
Note 1224165.1 Patch 9465448 (French) Failed At XLIFFLoader.class : No template found APTRXFEE_XML-LANGUAGE
Note 1321286.1 NLS Patch 7303033 fails with "No template found with application short name SQLAPand template code APTRXFEE_XML"
Note 1330843.1 Patches Failure Due To Unavailability Of Template Data Through ap12axdo.ldt

2. If you have errors like:

java.io.UTFDataFormatException: Invalid UTF8 encoding
oracle.apps.xdo.XDOException: Exception while parsing base template

java.io.UTFDataFormatException: Invalid UTF8 encoding.
at oracle.xml.parser.v2.XMLUTF8Reader.checkUTF8Byte(XMLUTF8Reader.java:160)
at oracle.xml.parser.v2.XMLUTF8Reader.readUTF8Char(XMLUTF8Reader.java:203)
at oracle.xml.parser.v2.XMLUTF8Reader.fillBuffer(XMLUTF8Reader.java:120)
at oracle.xml.parser.v2.XMLByteReader.saveBuffer(XMLByteReader.java:450)
at oracle.xml.parser.v2.XMLReader.fillBuffer(XMLReader.java:2488)
at oracle.xml.parser.v2.XMLReader.tryRead(XMLReader.java:1089)
at oracle.xml.parser.v2.XMLReader.scanXMLDecl(XMLReader.java:3047)
at oracle.xml.parser.v2.XMLReader.pushXMLReader(XMLReader.java:521)
at oracle.xml.parser.v2.XMLParser.parse(XMLParser.java:288)
at oracle.apps.xdo.oa.util.TemplateTranslator.
createMLSTemplates(TemplateTranslator.java:238)
at oracle.apps.xdo.oa.util.XLIFFLoader.processUpload(XLIFFLoader.java:709)
at oracle.apps.xdo.oa.util.XLIFFLoader.process(XLIFFLoader.java:565)
at oracle.apps.xdo.oa.util.XLIFFLoader.main(XLIFFLoader.java:1073)

oracle.apps.xdo.XDOException: Exception while parsing base template
at oracle.apps.xdo.oa.util.TemplateTranslator.
createMLSTemplates(TemplateTranslator.java:241)
at oracle.apps.xdo.oa.util.XLIFFLoader.processUpload(XLIFFLoader.java:709)
at oracle.apps.xdo.oa.util.XLIFFLoader.process(XLIFFLoader.java:565)
at oracle.apps.xdo.oa.util.XLIFFLoader.main(XLIFFLoader.java:1073)

AD Worker error:
The above program failed. See the error messages listed
above, if any, or see the log and output files for the program.

US file is not loaded in the database, as already stated in the error:

"Exception while parsing base template"

= To reload XML

adjava -ms128m -mx256m -nojit oracle.apps.xdo.oa.util.XDOLoader UPLOAD
-DB_USERNAME <apps_un> -DB_PASSWORD <apps_pw> -JDBC_CONNECTION
"(DESCRIPTION=(ADDRESS_LIST=(LOAD_BALANCE=YES)(FAILOVER=YES)(ADDRESS=(PROTOCOL
=tcp)(HOST=<your hostname with domain>)(PORT=<your port>)))(CONNECT_DATA=(SID=<your sid>)))"
-LOB_TYPE DATA_TEMPLATE -APPS_SHORT_NAME OFA -LOB_CODE FADTXD -LANGUAGE 00
-XDO_FILE_TYPE XML-DATA-TEMPLATE -FILE_NAME
$FA_TOP/patch/115/publisher/defs/FADTXD.xml

= To reload RTF

adjava -ms128m -mx256m -nojit oracle.apps.xdo.oa.util.XDOLoader UPLOAD
-DB_USERNAME <apps_un> -DB_PASSWORD <apps_pw> -JDBC_CONNECTION
"(DESCRIPTION=(ADDRESS_LIST=(LOAD_BALANCE=YES)(FAILOVER=YES)(ADDRESS=(PROTOCOL
=tcp)(HOST=<your hostname with domain>)(PORT=<your port>)))(CONNECT_DATA=(SID=<your sid>)))"
-LOB_TYPE TEMPLATE_SOURCE -APPS_SHORT_NAME OFA -LOB_CODE FADTXD -LANGUAGE en
-TERRITORY US -XDO_FILE_TYPE RTF -TRANSLATE Y -FILE_NAME
$FA_TOP/patch/115/publisher/templates/US/FADTXD.rtf

Check Note 880149.1 Patch 6678700 Fails With UTFDataFormatException Xliffloader Error On EBS R12 Upgrade

3. If you have error:

No translatable template defined for this template code

[FILE_NAME] [$PRODUCT_TOP/patch/115/publisher/templates/<NLS>/<filename>.xlf]
[APPS_SHORT_NAME] [<product>]
[UPLOAD] [UPLOAD]
[DB_USERNAME] [APPS]
[JDBC_CONNECTION] [(DESCRIPTION=(ADDRESS_LIST=(LOAD_BALANCE=YES)(FAILOVER=YES)(ADDRESS=(PROTOCOL=tcp)(HOST=host)(PORT=port)))(CONNECT_DATA=(SERVICE_NAME=sid)))]
[DB_PASSWORD] [******]
[TEMPLATE_CODE] [template_code]

Target file: <filename>.xlf
No translatable template defined for this template code.

Check what templates are loaded in database for that specific APPS_SHORT_NAME.

To identify if that specific US template is loaded please use:

SQL> select application_short_name, template_code, mls_language, mls_territory from xdo_templates_b
where application_short_name ='<apps_short_name>' and template_code='<template_code>';

Verify that the results has:
MLS_LANGUAGE = 'en' and MLS_TERRITORY = 'US' for the template.

For example if your error is with:

[FILE_NAME] [$AR_TOP/patch/115/publisher/templates/<NLS>/ARATAPPM.xlf]
[APPS_SHORT_NAME] [AR]
...
[TEMPLATE_CODE] [ARATAPPM]

SQL> select application_short_name, template_code, mls_language, mls_territory from xdo_templates_b
where application_short_name ='AR' and template_code='ARATAPPM';

should display:

APPLICATION_SHORT_NAME
--------------------------------------------------
TEMPLATE_CODE
--------------------------------------------------------------------------------
MLS_LA MLS_TE
------ ------
AR
ARATAPPM
en     US

As you can see this is the correct result:
- template is available
- MLS_LANGUAGE = 'en'
- MLS_TERRITORY = 'US'

If in your case you don't see something similar then please review:
Note 1073816.1 Patch Error: All NLS Patches Fail Running XLIFFLoader For File ARATAPPM.xlf
Note 1500668.1 Patch 14408571 Error: Target File: PAYNLABPCSV.xlf No Translatable Template Defined
Note 1505842.1 Patch 14526314 Failure - FAILED: file XLIFFLoader.class IEXSTRDIG.xlf No translatable template defined for this template code.

4. If you have error:

Exception in thread "main" java.lang.NoSuchMethodError: sun.io.ByteToCharMS936.getIndex1()[S
at sun.nio.cs.ext.MS936$Decoder.<init>(MS936.java:50)
at sun.nio.cs.ext.MS936.newDecoder(MS936.java:38)
at java.lang.StringCoding$CharsetSD.<init>(StringCoding.java:172)
at java.lang.StringCoding$CharsetSD.<init>(StringCoding.java:163)
at java.lang.StringCoding.decode(StringCoding.java:219)
at java.lang.String.<init>(String.java:320)
at java.lang.String.<init>(String.java:346)
at oracle.apps.xdo.template.rtf.basic.RTFText.convertString(RTFText.java:425)
at oracle.apps.xdo.template.rtf.basic.RTFText.convertString(RTFText.java:336)
at oracle.apps.xdo.template.rtf.RTFParser.handleText(RTFParser.java:516)
at oracle.apps.xdo.template.rtf.io.RTFFilter.parseCurrentChars(RTFFilter.java:108)
at oracle.apps.xdo.template.rtf.io.RTFFilter.write(RTFFilter.java:202)
at oracle.apps.xdo.template.rtf.io.Filter.write(Filter.java:124)
at oracle.apps.xdo.template.rtf.RTF2XSLParser.write(RTF2XSLParser.java:170)
at oracle.apps.xdo.template.rtf.io.Filter.readFromStream(Filter.java:64)
at oracle.apps.xdo.template.rtf.RTFParser.generate(RTFParser.java:134)
at oracle.apps.xdo.template.rtf.RTF2XSLParser.generateXSL(RTF2XSLParser.java:295)
at oracle.apps.xdo.oa.util.CoreHelper.runRtfParser(CoreHelper.java:130)
at oracle.apps.xdo.oa.util.CoreHelper.runRtfParser(CoreHelper.java:70)
at oracle.apps.xdo.oa.util.XDOLoader.processUpload(XDOLoader.java:1157)
at oracle.apps.xdo.oa.util.XDOLoader.process(XDOLoader.java:1002)
at oracle.apps.xdo.oa.util.XDOLoader.main(XDOLoader.java:2245)
AD Run Java Command is complete.

Ensure you have supported JDK version. Actual supported JDK version is 1.6, but this issue is fixed also in 1.5
Note 455492.1 Using Latest Update of Java 6.0 with Oracle E-Business Suite Release 12
Note 384249.1 Using Latest Update of JDK 5.0 with Oracle E-Business Suite Release 12
Note 401561.1 Using J2SE Version 6 with Oracle E-Business Suite 11i
Note 304099.1 Using J2SE Version 5.0 with Oracle E-Business Suite 11i, Release 11.5.10

5. If you still have the issue raise a Community thread or Service request with:

- patch that you are applying

- US patch log file

- NLS patch log file

- related adworker log with the error

- the query to prove that all prerequisites are applied.
Use next query connected as apps user:

SQL> select bug_number from ad_bugs where bug_number in ('&patch_number1','&patch_number2',...);

- results of:

SQL> select application_short_name, template_code, mls_language, mls_territory from xdo_templates_b
where application_short_name ='<apps_short_name>' and template_code='<template_code>';

19. How To Solve Error: "File in patch is not a known Oracle Applications file"

In patch log you will see:

AutoPatch found some files which it will not apply.
These files are listed in the AutoPatch informational message file.

AutoPatch error:
File in patch is not a known Oracle Applications file:

<here comes a list of missing files>

1. Cause: This error occurs because the patch cannot find the file referenced in the file driver.
Under each PRODUCT_TOP is a directory admin, where a file <product_short_name>file.drv is stored to reference the files used.

For example :

    * $FND_TOP/admin/driver/fndfile.drv
    * $PO_TOP/admin/driver/arfile.drv
    * $GL_TOP/admin/driver/glfile.drv

Those files also have include statements, to have other files checked. In these driver files it records the files that are meant to be present, and their location.
So in $<PRODUCT>_TOP/admin/driver/
there are several files called:
<product>xxx.drv

The patch is checking this file to see if the patch file is newer than the one present in the apps file system and since the file is not listed, it cannot compare what is already in place on the file system, to the patch version.

For example, if you have error:

AutoPatch error:
File in patch is not a known Oracle Applications file:
'pay html pyshpslp.jsp'

This means in your $PAY_TOP/admin/driver
one of the <file>.drv should include the file: pyshpslp.jsp
If not, the patch will fail.

Then you would need to review each failing file to be present in their driver files as described above.

2. The issue is usually caused by missing prerequisite.
Ensure you applied all prerequisites for the patch.
Use next query connected as apps user:

SQL> select bug_number from ad_bugs where bug_number in ('&patch_number1','&patch_number2',...);

3. Do a search on MOS using:
- filename
- "File in patch is not a known Oracle Applications file"

4. Support can identify a new version of $<PRODUCT>_TOP/admin/driver/<product_short_name>file.drv or other .drv file and include the missing file.
If the .drv includes the file, patch will not fail anymore.

5. There are specific cases when the workaround is to edit to include the missing references the patch is complaining about.
You will have to backup $<PRODUCT>_TOP/admin/driver/<product_short_name>file.drv or other .drv file and include the missing file.
Note! This are only particular cases and the workaround should not be generally used.

Review:
Note 1433312.1 Translation Synchronization Patch fails with "File in patch is not a known Oracle Applications file:GLEFCREP.rdf, XTREFCAN.rdf, XTREFCOM.rdf"
Note 461836.1 Applying 11.5.10.2 Patch 3480000 Errors With "File in patch is not a known Oracle Applications file: Bne Conf Example.Txt."

6. If you still have the issue raise a Community thread or Service request with:
- patch failing

- adpatch log with the error

- the query to prove that all prerequisites are applied.
Use next query connected as apps user:

SQL> select bug_number from ad_bugs where bug_number in ('&patch_number1','&patch_number2',...);

- version of all files under $<PRODUCT>_TOP/admin/driver/
For example if you have error for:
'pay html pyshpslp.jsp'
please provide the version of all files under $PAY_TOP/admin/driver/

cd $PAY_TOP/admin/driver/
adident Header * > version.txt

Upload versions.txt

20. How To Solve Error: "This patch is not compatible with your current codelines"

In patch log you will see errors like:

This patch is not compatible with your current codelines.

This patch is compatible with: entity 'ibe' - codeline 'R12.<PRODUCT>.A'.
Your current on-site codeline for the entity 'ibe' is: 'R12.<PRODUCT>.B'.

You should not apply this patch.

Apply an equivalent patch that is compatible with your current codelines instead.

1. Ensure that you are applying the correct patch for your release.

If you are on R12.1.X this means you are codelevel B and you can only apply B patches (like R12.JL.B, R12.ZX.B, R12.TXK.B.delta.2).
If you are on R12.0.x this means you are codelevel A and you can only apply A patches (like R12.JL.A, R12.ZX.A, R12.TXK.A.delta.2).
If you are on R12.2.x this means you are codelevel C and you can only apply C patches (like R12.JL.C, R12.ZX.C, R12.TXK.C.delta.2).

In order to identify your release you can use:

select product_group_name,release_name,multi_org_flag,
multi_lingual_flag,multi_currency_flag
from FND_PRODUCT_GROUPS

In order to distinguish Between E-Business Suite 12.1 and 12.0 Patches, in MOS the "Release" column clearly distinguishes between the patches for each Release 12 codeline.

If you are applying the correct codelevel patch for your release then please check next steps.

2. Ensure that correct ORACLE_HOME is sourced.

echo $ORACLE_HOME

should point to your 10.1.2 ORACLE_HOME.

3. The issue can be caused by several tables locked. Bounce the database and retest.

4. Run adadmin / maintain snapshot information and retest.

5. There are few cases when the error will point to C codelevel like:

This patch is compatible with: entity 'po' - codeline 'R12.PO.B'.
Your current on-site codeline for the entity 'po' is: 'R12.PO.C'.

Please search on MOS using:
- patch number
- "This patch is not compatible with your current codelines"
and see if this is a known bug. Otherwise please raise a new thread on community or a new SR.

Only few products have published codelevel C patches.
In order to identify which products in your environment have already this code level you can use:

select decode(nvl(a.APPLICATION_short_name,'Not Found'),
'SQLAP','AP','SQLGL','GL','OFA','FA',
'Not Found','id '||to_char(fpi.application_id),
a.APPLICATION_short_name) apps,
decode(fpi.status,'I','Installed','S','Shared',
'N','Inactive',fpi.status) status,
fpi.product_version,
nvl(fpi.patch_level,'-- Not Available --') Patchset,
to_char(fpi.last_update_date,'dd-Mon-RRRR') "Update Date"
from fnd_oracle_userid o, fnd_application a, fnd_product_installations fpi
where fpi.application_id = a.application_id(+)
and fpi.oracle_id = o.oracle_id(+)
and nvl(fpi.patch_level,'x') like ('%.C.%')
order by 1,2;

Review:
Note 1465165.1 Error applying patch 10338643:R12.ONT.B. This patch is compatible with: entity 'icx' 'R12.ICX.C'
Note 1470344.1 Problems Applying Patch 12997784:R12.AP.B

6. If you still have the issue raise a Community thread or Service request with:
- patch failing

- adpatch log with the error

- adadmin - mainatain snapshot log

7. One appears to have another driver (*.drv) file in the same directory where one downloaded the patch.
It would appear adpatch is picking up / recognising other drivers. One may have a directory something like -> '/home/ismapp/Patches/7597660". Suggestion is to download the patch to a different directory and try to install from there.

21. How To Solve After Patch is Applied, Java Class Version Is Not Correct (As Per Patch Read Me)

After the patch is applied you identify that the java class version is not changed. How to solve the situation?

1. Cause

In the E-business Suite, the file versions for java files are stored in the JRIMETA.DAT.

When a patch containing a java class file is applied, adpatch looks in the JRIMETA.DAT and compares this with the version of the class file contained within the patch.
If the patch file is newer then it replaces the actual class file and the JRIMETA.DAT file is updated. adpatch does not look at the actual class file within $JAVA_TOP for comparison with the corresponding file in the patch.

So in your case this means the JRIMETA.DAT is not correct and has not the correct information, so the java files are not copied.
We need to synchronized it with the versions from the java files in the APPL_TOP.

Please review Note 459832.1 An Explanation of the JRIMETA.DAT file as used with Oracle Applications

2. Action plan

- As usual before applying any patch please take a backup before of your database and APPL_TOP(s).

- Verify the synchronization

$ adjava -mx512m oracle.apps.ad.jri.adjcopy -masterArchive $JAVA_TOP -sync -mode CHECK_ONLY

- If there is any synchronization issue reported, then perform:

adjava -mx512m oracle.apps.ad.jri.adjcopy -masterArchive $JAVA_TOP -sync -mode APPLY

Important Note! The above command should only be used if it is not possible to revert to a previous known, good environment. If it is suspected there is a mismatch between the JRIMETA.DAT and the $JAVA_TOP and this is discovered when patches are being tested (as part of an upgrade for example), it is preferable to return to a known, good JRIMETA.DAT and then repeat the patches or maintenance and identify where the inconsistency occurs. The above command only corrects the JRIMETA.DAT to reflect the current $JAVA_TOP - it is more important that the source of the problem is identified and how and why the problem initially occurred. It will usually only be possible to identify the source of the mismatch if there is a known, good situation that can be reverted to.

- Re-apply the patch

3. If you still have the issue raise a Community thread or Service request with:
- patch failing

- adpatch log with the error

- log generated by

$ adjava -mx512m oracle.apps.ad.jri.adjcopy -masterArchive $JAVA_TOP -sync -mode CHECK_ONLY

22. How To Solve Incorrect Fonts In Self Service Pages After Patching

After applying OAF patches 11894708, 12630484, 14069503 and 12687346, Fonts are messed up (appear bigger) on Self Service Framework pages which is also causing some links (such as Favorites, Update, ... and others) to be missing. Also seeing missing images, and buttons (Update, Cancel, Next,...) on some of the pages.

It was also found that:
Workflow Admin Web Application
Business Events
Name: %Task%
Go
Should be "Update Button" for each item and Test icon, but these are missing.

Several patches where the issue was reported are:

11894708 OA FRAMEWORK R12.1.3.1
12630484 XSLPROCESSOR.VALUEOF() FAILING AFTER PATCH 11894708
12687346 R:PPR CHANGES READONLY COLUMNS TO EDITABLE
6390830 ADPATCH FAILED WHILE APPLYING HR GLOBAL PATCH IN RUP3
14111441 OAF BATCH ELEMENT ENTRY
14146401 HRMSR12: FWD PORT OF 14137458
14262825 HRMSR12: FWD PORT OF 14192643
14351767 1OFF:14273944:12.1.5::AFTER APPLYING 11IRUP7 XFER TO BEE IS MPNG HRS
13418800 R12.HR_PF.B.DELTA.5
14069503 ATG CONSOLIDATED PATCH FOR R12.1 HRMS RUP5
13979377 ORACLE APPLICATIONS RELEASE 12.1: CPU PATCH FOR JUL 2012

NOTE! This issue is not specific to these patches, these are just some examples of patches that have caused this issue. This issue could occur with other patches as well.
NOTE! Usually the issue occurs only in Internet Explorer, not in other browsers.

This is caused by cabo caching issue.
In order to solve the issue please follow: Note 1348791.1 "R12: Font And Links Have Changed After Patching "

In order to clear all cache please check:
Note 742107.1 How To Clear Caches (Apache/iAS, Cabo, Modplsql, Browser, Jinitiator, Java, Portal, WebADI) for E-Business Suite?

23. How To Identify Recommended Patches For A Product

Review:
Note 1400757.1 How to Find E-Business Suite Recommended Patches
Note 550654.1 How to Get The Patchset Level of Oracle Applications Products in R12

On Patch wizard, also please see:
Note 1077813.1 Patch Wizard Overview
Note 976188.1 Patch Wizard Utility
Note 976688.1 Patch Wizard FAQ
Note 1085668.1 Patch Wizard Training

24. How to determine the number of adpatch workers ?

Monitor carefully your server ressources, especially the memory usage when you use a high number of workers.
If you select a higher number of workers than CPUs*2 you can experience performance issues.
If you are seeing this problem regularly, please bring the number of Workers down until
resources are no longer an issue.

To summarize, the suggested default is a best estimate of how many Workers should be
able to run in parallel on your server, but that value is not guaranteed as usage levels change.

Review the following documents which discuss this topic:

Note 756063.1 - How to Troubleshoot "adpatch" Performance Issues: Slow, Hanging or Crashes
Note 800024.1 - How Does Adpatch Determine The Number Of Workers To Recommend ?
Note 226191.1 - How To Select Number of Workers Based on Number of CPUs When Running ADPATCH

Still Have Questions?

To discuss this information further with Oracle experts and industry peers, we encourage you to review, join or start a discussion in the E-Business Suite Patching Community.

To provide feedback on this note, click on the Rate this document link.

REFERENCES

NOTE:356878.1 - 11i / R12 E-Business Suite Applications Technology Stack Steps To Relink E-Business Suite Binaries After An Upgrade Or RPM O/S Patching
NOTE:21154.1 - EVENT: 10046 "enable SQL statement tracing (including binds/waits)"
NOTE:225165.1 - Patching Best Practices And Reducing Downtime
NOTE:1523589.1 - Master Note - Troubleshooting Adpatch Relinking Errors
NOTE:881505.1 - Interoperability Notes Oracle EBS 11i with Oracle Database 11gR2 (11.2.0)
NOTE:1400757.1 - How to Find E-Business Suite Recommended Patches
NOTE:219968.1 - SQL*Net & Oracle Net Services - Tracing and Logging at a Glance
NOTE:384249.1 - Using Latest Update of JDK 5.0 with Oracle E-Business Suite Release 12
NOTE:384248.1 - Sharing The Application Tier File System in Oracle E-Business Suite Release 12.0 and 12.1
NOTE:551325.1 - How To Verify Or Create A Database Object Using An ODF (Adodfcmp) Or XDF (FndXdfCmp) File
NOTE:243880.1 - Shared APPL_TOP FAQ
NOTE:1549195.1 - Adpatch fails with: "The CREATE INDEX statement above failed because there is another index on the same columns"
NOTE:308427.1 - The XDF Comparison Utility (FndXdfCmp)
NOTE:250262.1 - RDA - Health Check / Validation Engine Guide
NOTE:312640.1 - Oracle Text: Re-installation of Applications 11i (11.5.10) Oracle Text Indexes
NOTE:468565.1 - How To Reload The APPS Java Class Objects In An Oracle Applications Environment 11i and R12
NOTE:2214169.1 - EBS Invalid Object Utility
NOTE:343917.1 - Frequently Asked Questions: Oracle E-Business Suite Support on x86-64
NOTE:178722.1 - How to Generate a Specific Form Through AD utility adadmin
NOTE:1325394.1 - Troubleshooting Guide - Invalid Objects in the E-Business Suite Environment 11i and 12
NOTE:275571.1 - How to validate an invalid Database Queue in an E-Business Suite Environment
NOTE:304099.1 - Using J2SE Version 5.0 with Oracle E-Business Suite 11i
NOTE:761566.1 - Oracle E-Business Suite Installation and Upgrade Notes Release 12 (12.1.1) for Linux x86-64
NOTE:800024.1 - How Does Adpatch Determine The Number Of Workers To Recommend?
NOTE:401561.1 - Using J2SE Version 6 with Oracle E-Business Suite 11i
NOTE:226191.1 - How To Select Number of Workers Based on Number of CPUs When Running ADPATCH
NOTE:343987.1 - How to uninstall backout rollback an Oracle Applications 11i or R12 Patch
NOTE:370137.1 - After Upgrade, Some Packages Intermittently Fail with ORA-04065 ORA-06508
NOTE:396009.1 - Database Initialization Parameters for Oracle E-Business Suite Release 12
NOTE:880149.1 - Patch 6678700 Fails With UTFDataFormatException Xliffloader Error On EBS R12 Upgrade
NOTE:402785.1 - iSetup dependency with Deinstall and Reinstall of XMLDB
NOTE:418664.1 - Overview of Using Java with Oracle E-Business Suite Release 12.x
NOTE:452095.1 - ORA-24816: Expanded Non Long Bind Data Supplied After Actual Long Or Lob Column
NOTE:352843.1 - How to Run a Patch Impact Analysis In OAM
NOTE:745580.1 - How To Apply Patches On Shared Application Tier File System Environment
NOTE:1078973.1 - AD Command Line Options for Release R12
NOTE:756063.1 - How to Troubleshoot The "adpatch" Performance Issues: Slow, Hanging or Crashes
NOTE:393861.1 - Globalization Guide for Oracle Applications Release 12
NOTE:228779.1 - How to Merge Patches Using admrgpch
NOTE:459832.1 - An Explanation of the JRIMETA.DAT file as used with Oracle Applications 11i
NOTE:1224313.1 - Getting Started with Oracle E-Business Suite Plug-in, Release 4.0
NOTE:1537973.1 - How To Troubleshoot NLS Forms Generation Errors When Applying An NLS Patch In R12 Or 11i
NOTE:832459.1 - How To Cleanup Invalid Oracle iSetup (AZ) Tables And Recreate
NOTE:313.1 - Patching & Maintenance Advisor: E-Business Suite (11i, 12.0 & 12.1)
NOTE:1058763.1 - Interoperability Notes EBS 12.0 and 12.1 with Database 11gR2
NOTE:1524398.1 - Interoperability Notes EBS 12.0 or 12.1 with RDBMS 12cR1
NOTE:1348791.1 - R12: Font And Links Have Changed After Patching
NOTE:742107.1 - How To Clear Apache/iAS, Cabo, Modplsql, Browser, Jinitiator, Java, Portal, WebADI Caches in E-Business Suite?
NOTE:1434392.1 - R11i / R12.0 / R12.1: Getting Started with Oracle Application Management Pack (AMP) for Oracle E-Business Suite, Release 12.1.0.1.0
NOTE:376442.1 - * How To Collect 10046 Trace (SQL_TRACE) Diagnostics for Performance Issues
NOTE:976188.1 - R11i / R12 : Patch Wizard Utility [Video]
NOTE:174605.1 - bde_chk_cbo.sql - EBS initialization parameters - Healthcheck
NOTE:316806.1 - Oracle Applications Installation Update Notes, Release 11i (11.5.10.2)
NOTE:1357075.1 - Xliffloader Errors When Applying 6678700(NLS) Driver Patch.
NOTE:333785.1 - Oracle Applications 11i Internationalization Guide
NOTE:472937.1 - Information On Installed Database Components and Schemas
NOTE:300482.1 - Overview of Using Java with Oracle E-Business Suite Release 11i
NOTE:550654.1 - E-Business Suite Applications Manager Steps To Get The Patchset Level of Each Specific Oracle Applications Functional Product in R12
NOTE:1478859.1 - Troubleshooting NLS issues with Oracle Applications
NOTE:976688.1 - Patch Wizard FAQ
NOTE:437878.1 - Upgrading OracleAS 10g Forms and Reports in Oracle E-Business Suite Release 12
NOTE:216205.1 - Database Initialization Parameters for Oracle Applications Release 11i
NOTE:1524399.1 - Interoperability Notes EBS 11i with RDBMS 12cR1
NOTE:455492.1 - Using Latest Java 6.0 Update With Oracle E-Business Suite Release 12
NOTE:743720.1 - Oracle Text: Re-installation and Rebuilding of Applications R12 Oracle Text Indexes
NOTE:454811.1 - Upgrading to the Latest OracleAS 10g 10.1.3.x Patch Set in Oracle E-Business Suite Release 12
NOTE:218105.1 - Introduction to ORACLE Diagnostic EVENTS
NOTE:233428.1 - Sharing the Application Tier File System in Oracle Applications Release 11i
NOTE:419728.1 - Concurrent Processing - How To Gather Statistics On Oracle Applications Release 11i and/or Release 12 - Concurrent Process,Temp Tables, Manually
NOTE:1077813.1 - E-Business Suite Applications Manager Patch Wizard Video Overview Of Navigation And Basic Features [VIDEO]

OracleAppsDBAKK1

Wednesday, November 6, 2019

E-Business Suite Applications DBA Top Adpatch Patch Application Issues For Release 11i And 12.x (Doc ID 1444322.1)

APPLIES TO:

PURPOSE

TROUBLESHOOTING STEPS

1. How To Minimize The Risk When Applying A Patch

2. How To Reduce System Downtime

3. How To Plan A Smart Patching Strategy

4. How To Identify The Location And Meaning Of Patching Logs

5. How To Identify If A Patch Is Already Applied

6. How To Solve Performance Issues During A Patch

7. You Are Not Able To Find 64bit Version Of An Applications Patch

8. How To Solve Relinking Errors During Patching

9. How To Solve Invalid Objects After Patching

10. How To Solve Failures For Libraries (.pll/.plx)

11. How To Solve Failures For US Forms (.fmb)

12. How To Solve Failures For NLS Forms (.fmb)

13. How To Solve Failures For .ldt (FNDLOAD) Files

14. How To Solve Failures For .sql Files

15. How To Solve Failures For .odf Files

16. How To Solve Failures For .xdf Files (oracle.apps.fnd.odf2.FndXdfCmp)

17. How To Solve Failures For XLIFFImporter (.xlf) Files

18. How To Solve Failures For XLIFFLoader (.xlf) files

19. How To Solve Error: "File in patch is not a known Oracle Applications file"

20. How To Solve Error: "This patch is not compatible with your current codelines"

21. How To Solve After Patch is Applied, Java Class Version Is Not Correct (As Per Patch Read Me)

22. How To Solve Incorrect Fonts In Self Service Pages After Patching

23. How To Identify Recommended Patches For A Product

24. How to determine the number of adpatch workers ?

Still Have Questions?

REFERENCES

No comments:

Post a Comment

Database Options/Management Packs Usage Reporting for Oracle Databases 11.2 and later (Doc ID 1317265.1)

Search This Blog