Monday, August 21, 2023

Failed hostname verification check when using SSL Certificates with Wildcards in WebLogic Server (Doc ID 1377479.1)

 

SYMPTOMS

 When using SSL Certificates with Wild Cards in WebLogic Server from weblogic 9.2 to 10.3.5. as below

CN=*.<Fully-Qualified-Domain-Name>

you can see the following error:

Caused by: javax.net.ssl.SSLKeyException: [Security:090504]Certificate chain received from <HOSTNAME> - <IP> failed hostname verification check. Certificate contained *.domain but check expected <HOSTNAME>

 

 

CHANGES

 

CAUSE

This issue is caused due to Default BEA hostname verifier is configured with WLS. Weblogic does not handle/support wildcard certificates out of the box. When weblogic initiates a connection over SSL, it makes a call to the HostnameVerifier. The default BEA hostname verifier matches the hostname you are connecting to and the CN of the certificate.

SOLUTION

To fix this issue there are three solutions:

1. Disable the Default BEA hostname verifier in WLS.

2. Write a custom hostname verifier and configure it with WLS.

3. For some WLS versions there is also patch for Bug 10215257 available for download.

 

Disable the Default BEA hostname verifier in WLS

Login to Admin console --> Click on servers --> SSL --> Advanced --> from the dropdown menu next to Hostname verification select --> None

Alternatively, you can add to JAVA_OPTIONS -Dweblogic.security.SSL.ignoreHostnameVerification=true to achieve the same.

If you are using Node Manager, please check this document:

 

WebLogic Server: Procedure for configuring Node Manager with SSL. (Doc ID 1142995.1)

 

Write a custom hostname verifier and configure it with WLS

This is described in Oracle Documentation:

Configure a custom host name verifier
Using a Custom Hostname Verifier


Hostname Verifier Sample Code Fragment
:


package examples.security.sslclient;

import java.io.ByteArrayInputStream;
import java.security.cert.Certificate;
import java.security.cert.CertificateFactory;
import java.security.cert.X509Certificate;
import java.util.logging.Logger;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

import javax.net.ssl.SSLSession;

public class TestHostnameVerifier implements weblogic.security.SSL.HostnameVerifier {

    private final static Logger log = Logger.getLogger(TestHostnameVerifier.class.getCanonicalName());

    @Override
    public boolean verify(String hostname, SSLSession session) {


        try {
            Certificate cert = session.getPeerCertificates()[0];

            byte[] encoded = cert.getEncoded();

            CertificateFactory cf = CertificateFactory.getInstance("X.509");
            ByteArrayInputStream bais = new ByteArrayInputStream(encoded);

            X509Certificate xcert = (X509Certificate) cf.generateCertificate(bais);

            String cn = getCanonicalName(xcert.getSubjectDN().getName());

            log.info("CN: '" + cn + "'" + " HOSTNAME: '" + hostname + "'");

           
            // If CN is equals to Hostname then it is approved
            if (cn.equals(hostname)) {
                return true;
            }

            // Compile regular expression
            // here we set the wildcard and the rest of the URL inside, could be more generic too.
            
            cn = cn.replace(".", "\\.");
            cn = cn.replace("-", "\\-");
            cn = cn.replace("*", "[-.*aA-zZ0-9]+");
           
            Pattern pattern = Pattern.compile(cn);
           
            // Determine if there is an exact match
            CharSequence inputStr = hostname;
            Matcher matcher = pattern.matcher(inputStr);
            boolean matchFound = matcher.matches();
           
            if (matchFound==false) {
                log.info("pattern doesn't match hostname");
            }
           
            //return boolean value
            return matchFound;       

        } catch (Exception e) {
            e.printStackTrace();
        }

        return true;
    }

    /**
     * Returns just the canonical name from the distinguishedName on the cert.
     *
     *
     */
    private String getCanonicalName(String subjectDN) {

        Pattern pattern = Pattern.compile("CN=([-.*aA-zZ0-9]*)");
        Matcher matcher = pattern.matcher(subjectDN);

        if (matcher.find()) {
            return matcher.group(1);
        }

        log.info("Couldn't find match for CN in subject");
        return subjectDN;

    }

}

To compile and package into a jar file

i. Navigate to <WLS_HOME>/user_projects/domains/<DOMAINNAME>/bin
ii. Set up an environment by executing: $ . ./setDomainEnv.sh This will include weblogic.jar into classpath, in order to use any of the libraries included under package weblogic.*
iii. Compile the class by copying the content of the code above and naming the file as:


TestHostnameVerifier.java


iv. Run javac to compile the class.


$javac TestHostnameVerifier.java

v. Package the compiled class into a jar file by executing:


$jar cvf TestHostnameVerifier.jar TestHostnameVerifier.class


Expected output is:


stnameVerifier.class
added manifest
adding:TestHostnameVerifier.class(in = 363) (out= 363)(stored 0%)



vi. This will produce a file called:

TestHostnameVerifier.jar

 

When attempting to use the Custom Hostname Verifier, you might experience the following error:

<Error> <WebLogicServer> <AdminServer> <[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'> <> <> <ECID> <BEA-000297> <Inconsistent security configuration, weblogic.utils.NestedRuntimeException: [Security:090563]Cannot create instance of Hostname Verifier WildcardHostnameVerifier.WildcardHostnameVerifier.>
<Emergency> <Security> <AdminServer> <[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'> <> <> <ECID> <BEA-090034> <Not listening for SSL, java.io.IOException: [Security:090563]Cannot create instance of Hostname Verifier WildcardHostnameVerifier.WildcardHostnameVerifier..>

The issue is caused by Custom Hostname Verifier is not in the classpath.

To fix this issue:

i. Modify setDomainEnv.cmd/setDomainEnv.sh adding:

set CLASSPATH=%CUSTOMHOSTNAMEVERIFIERCLASSPATH%;%CLASSPATH%

ii. Restart WebLogic Server and re test the issue.  

For some WLS versions there is also patch for Bug 10215257 available for download

Patch 10215257  is available to enhance the existing default BEA hostname verifier. This means that you do not need to turn off Hostname verification OR write a custom hostname verifier to use Wildcard certificates from WLS. Once the patch is applied wildcard certificates are accepted by WLS.

This patch extends the "default" hostname verifier (SSLWLSHostnameVerifier) to support a certificate's Subject Alternative Name (SAN) field, DNS Name type only.  As an example, please go to this URL: 

https://www.digicert.com/subject-alternative-name.htm

 and view the certificate using your browser, select the Subject Alternative Name field, to view a list of domains secured by a single SSL certificate.  List provided here for convenience (DNS Name=www.digicert.com, DNS Name=www.origin.digicert.com, DNS Name=content.digicert.com, DNS Name=digicert.com).  SANs cannot be wildcarded because the standards that define SAN functionality do not support wildcarded values.
 
Additionally, a new custom hostname verifier (SSLWLSWildcardHostnameVerifier) was implemented, derived from the default hostname verifier, so that it supports everything the default hostname verifier does, including SANs.   You must  configure your WebLogic server to use this custom hostname verifier if support for wildcard certificates is required during SSL handshake.  One option is to use the following WebLogic property: 

-Dweblogic.security.SSL.hostnameVerifier=weblogic.security.utils.SSLWLSWildcardHostnameVerifier

Additional information can be found in Oracle Documentation:
 http://download.oracle.com/docs/cd/E21764_01/web.1111/e13711/ssl_client.htm#i1029782

 

Patches are available for Bug 10215257:

PATCH INFORMATION
WLS VersionPatch Number
10.3Patch: 10215257
10.3.2Patch: 10215257
10.3.4Patch: 10215257
10.3.5Patch: 10215257

 This issue is fixed in 10.3.6.

To apply one of these patches, download the patch from My Oracle Support by searching for the patch number for your WLS version: for detailed instructions, please see Primary Note: How to Locate and Download Patches for WebLogic Server Using My Oracle Support Note:1302053.1. For instructions on how to apply these patches to your system, please see How to Apply WebLogic Server (WLS) Patches Using Smart Update Note:876004.1. For other issues relating to Smart Update, please see Primary Note on Troubleshooting Smart Update Issues Note:1230725.1

Patches are specifically tied to a particular release and maintenance pack of WebLogic Server (WLS). A patch for WLS 10.3.6, for example, very likely would not work on WLS 10.3.5. In a few cases, patches are also specific to particular operating systems. If you think you are experiencing the problem outlined in this note, and you are running a WLS version which is eligible for error correction (see Note:950131.1; for more about the Oracle Error Correction Policy), but your WLS version is not included in the list of patches provided here, please contact support and ask for a version of the patch appropriate for you. Please reference this note as well as this bug number to help speed our service for you.

 

REFERENCES

http://download.oracle.com/docs/cd/E21764_01/web.1111/e13711/ssl_client.htm#i1029782
NOTE:1142995.1 - WebLogic Server: Procedure for Configuring Node Manager with SSL
http://docs.oracle.com/cd/E21764_01/web.1111/e13711/ssl_client.htm#i1029782
http://docs.oracle.com/cd/E21764_01/apirefs.1111/e13952/taskhelp/security/ConfigureACustomHostNameVerifier.html



Friday, August 11, 2023

Upgrade your Oracle E-Business Suite database on Exadata Database Service Dedicated

 For Oracle E-Business Suite Release 12.2 and Oracle Database 12.1.0.2:

Document 2662718.1, Upgrading Oracle Database 12c to 19c for Oracle E-Business Suite Release 12.2 on Oracle Exadata Cloud Services and Oracle Exadata Cloud@Customer Gen 2
For Oracle E-Business Suite Release 12.2 and Oracle Database 11.2.0.4:
Document 2567103.1, Upgrading Oracle Database 11g to 19c for Oracle E-Business Suite Release 12.2 on Oracle Exadata Cloud Services
For Oracle E-Business Suite Release 12.1.3 and Oracle Database 12.1.0.2:
Document 2662717.1, Upgrading Oracle Database 12c to 19c for Oracle E-Business Suite Release 12.1 on Oracle Exadata Cloud Services and Oracle Exadata Cloud@Customer Gen 2
For Oracle E-Business Suite Release 12.1.3 and Oracle Database 11.2.0.4:
Document 2617850.1, Upgrading Oracle Database 11g to 19c for Oracle E-Business Suite Release 12.1 on Oracle Exadata Cloud Services

Interoperability Notes: Oracle E-Business Suite Release 12.1 with Oracle Database 19c (Doc ID 2580629.1)
Using Oracle 19c RAC Multitenant (Single PDB) with Oracle E-Business Suite Release 12.1 (Doc ID 2530680.1)

How To Automatically Set the Current Run or Patch Edition / File System for EBS 12.2

 For 12.2.2:

Change directory to the Base directory and run script EBSapps.env giving "run" or "patch" as argument, eg:

cd /. ./EBSapps.env run

  E-Business Suite Environment Information
  ----------------------------------------
  RUN File System : <EBS base dir>/fs1/EBSapps/appl
  PATCH File System : <EBS base dir>/fs2/EBSapps/appl
  Non-Editioned File System : <EBS base dir>/fs_ne
  DB Host: <hostname.domain name>  Service/SID: <SID>

  Sourcing the RUN File System ...

 If EBSapps.env is run without the file system as argument this will be prompted for, eg:

. ./EBSapps.env

  E-Business Suite Environment Information
  ----------------------------------------
  RUN File System : <EBS base dir>/fs1/EBSapps/appl
  PATCH File System : <EBS base dir>/fs2/EBSapps/appl
  Non-Editioned File System : <EBS base dir>/fs_ne
  DB Host: <hostname.domain name>  Service/SID: <SID>

  E-Business Suite Environment Setting
  ------------------------------------
  - Enter [R/r] for sourcing Run File System Environment file, or
  - Enter [P/p] for sourcing Patch File System Environment file, or
  - Enter anything else to exit

  Would you like to set the E-Business Suite environment [R/P]:R

  Sourcing the RUN File System ...

 

For 12.2.0:

Note 12.2.0 is not certified, so should be upgraded as soon as possible to 12.2.2 using the procedure of Note 1506669.1 : "Oracle E-Business Suite Release 12.2.2 Readme"

Download, setup and run script fsauto as per the header part of the script:

# Script to set the current RUN Edition APPL_TOP environment automatically.
# Available through My Oracle Support Note 1545584.1
# Requirements: Set the XMLfile variable as specified below. This needs
#   to be set only once and not again after a file system cutover in adop.
# Save the script as fsauto<SID>.env in a non-EBS directory, eg.:
#   /.../.../fsautoPROD.env
# Usage: . <directory>/fsauto<SID>.env
#   Eg.: . /.../.../fsautoPROD.env
#     note the space ' ' character between the leading period '.' and first
#       forward-slash '/' characters
#   Alternatively use: source <directory>/fsauto<SID>.env
# If needed make the script runnable using command: chmod u+x fsautoPROD.env
# Carlo den Otter, 02-APR-2013
# Note this script is for educational purposes only, and as such is not supported.
# It was created on Linux and not tested on other platforms.
# Use at own risk.

 

Sample output:

[oracle@host ~]$ . ./fsautoPROD.sh
Running fsauto 1.0 :
&nbsp;Context Name (&lt;SID_host&gt;) : <SID>_<host>
&nbsp;RUN Edition APPL_TOP&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; : ..../fs1
&nbsp;PATCH Edition APPL_TOP&nbsp;&nbsp;&nbsp; : ..../fs2
&nbsp;Non-Editioned File System : .../fs_ne/EBSapps/appl
&nbsp;Instance Top Directory&nbsp;&nbsp;&nbsp; : .../fs1/inst/apps/<SID>_<host>
&nbsp;APPL_TOP environment file : ..../fs1/EBSapps/appl/APPS<SID>_<host>.env

Setting RUN Edition APPL_TOP environment for <...>/fs1 ...
Done.

  


How To Automatically Set the Current Run or Patch Edition / File System for EBS 12.2 (Doc ID 1545584.1)

12.2: Functionality Disabled During An Online Patching Cycle

 

SOLUTION

What happens on Oracle E-Business Suite releases 12.2.6 and above?

During the execution of the adop phases (prepare, apply, finalize, cutover) some product functionality may be disabled, but this holds true just for releases 12.2.3, 12.2.4 and 12.2.5. From Oracle E-Business Suite 12.2.6 and above, no product functionality will be disabled when an online patching cycle is active, users will remain unaffected while performing their daily tasks. If you review the following documents, you will find that the Readme files for Releases 12.2.3, 12.2.4 and 12.2.5 contain information about product functionality that is disabled during adop execution for that particular release, but the same information is not found in 12.2.6 or higher, meaning that no product functionality will be disabled from 12.2.6 and above.

Oracle E-Business Suite Release 12.2.7 Readme (Doc ID 2230783.1)
Oracle E-Business Suite Release 12.2.6 Readme (Doc ID 2114016.1)
Oracle E-Business Suite Release 12.2.5 Readme (Doc ID 1983050.1)
Oracle E-Business Suite Release 12.2.4 Readme (Doc ID 1617458.1)
Oracle E-Business Suite Release 12.2.3 Readme (Doc ID 1586214.1)

What happens on Oracle E-Business Suite releases 12.2.3, 12.2.4 and 12.2.5?

During the prepare phase the ADZDPATCH concurrent program is run and will continue to run through the entire patching cycle (prepare, apply, finalize, cutover).

This program prevents certain concurrent programs from being run.

These programs are made incompatible with ADZDPATCH by functional developers to avoid data inconsistencies.

Important: Not all programs which are incompatible with ADZDPATCH are defined incompatible by concurrent manager definition. In other words, you may not find a defined incompatibility via the UI. Instead,
some development teams have built the incompatibility within the code of the program definition itself. An example of this is:
XLAABACR module: Validate Application Accounting Definitions.

If you attempt to run a concurrent program that is incompatible with ADZDPATCH the program will error with something similar to the following seen in the request log file:

MSG-00000: OPT Currently patch application is in progress. <Name Of Failing Program> after patch application.
MSG-00000:  101504: non-ORACLE exception
REP-1419: MSG-00000: OPTCurrently patch application is in progress. Validate Application Accounting Definition after patch application

 


Oracle E-Business Suite Release 12.2: Recovery Guidelines For Failed Online Patching (adop) Cutover

 This document describes how you can, if required, restore an Oracle E-Business Suite Release 12.2 system to the state it was in before an Online Patching cutover phase was run. It is only intended for use in specific scenarios where cutover has failed for some reason.

Note: This document supplements the main Online Patching documentation, which is primarily provided in Oracle E-Business Suite Concepts and Oracle E-Business Suite Maintenance Guide. Both these books are available as part of the Release 12.2 Documentation Library.

In This Document

  1. Prerequisites and Restrictions
  2. Setting Up Flashback
  3. Problem Scenario
  4. Flashing Back the Database
  5. Restoring the File Systems
  6. References
  7. Change Record
  8. Documentation Notices

1. Prerequisites and Restrictions

Oracle E-Business Suite Release 12.2 supports Online Patching, which allows patches to be applied to a copy of the system while users are working on the existing system. The Online Patching cycle consists of several phases, with the critical phase where the changes are committed being called cutover. Up to this phase, you can run a special phase called abort, which will undo the changes made so far in the patching cycle. After cutover is complete, however, you cannot do this.

Note: The procedure described in this document is only intended for use in the event of a failed cutover, and as a last resort. It is not supported for use as a rollback (restoration) option after a successful cutover.

If a cutover error occurs, you should first check the error message and try to determine if the problem can be fixed easily, or (as is true in many cases) cutover can be made to succeed simply by running the command again. Restoring to a point before cutover via Flashback recovery should only be done when the error cannot easily be fixed, and continues to fail on subsequent cutover attempts.

So, before proceeding further with the instructions in this document:

  1. Review failure messages and cutover logs, identify problems, and make corrections as applicable. Issues such as running out of disk space can be corrected easily, whilst issues such as timeouts, deadlocks, and network issues may prove to be transient.
  2. Retry the cutover command.
  3. If cutover still fails, follow the instructions in the rest of this document to restore system availability while you take further diagnostic and corrective actions.

If after cutover you want to revert to the state of the system before the patching cycle was started, you can use the Oracle Database Flashback feature to go back to a designated point in time (a restore point). You should create the restore point just before running the cutover phase.

Note: Before creating the restore point, it is advisable to issue a suitable downtime notification and shut down the web services. This will ensure you do not lose any transactional data, and in effect simply extends slightly the cutover downtime.

Depending on exactly when the failure occurred, you may also need to restore the application tier file systems.

Note: The following prequisites must be met before you proceed with the instructions in this document:

  1. You are ready to perform cutover.
  2. All concurrent managers have been shut down cleanly.
  3. There are no current database transactions being performed by any third-party applications.

Top


2. Setting Up Flashback

All the steps in this section are performed on the database tier, as sysdba.

2.1 Set ARCHIVELOG mode

Ensure the database is in ARCHIVELOG mode. Refer to the documentation on Changing the Database Archiving Mode.

2.2 Enable Fast Recovery Area

You enable the Fast Recovery Area (FRA) by setting two database initialization parameters:

  • DB_RECOVERY_FILE_DEST_SIZE - Specifies the size of the Fast Recovery Area.
  • DB_RECOVERY_FILE_DEST - Specifies the physical location of the Flashback recovery files.
Note: If you are using the Fast Recovery Area to store RMAN backups to disk, the FRA needs to be sized to hold all your datafiles, any incremental backups, and the flashback logs generated during cutover.

If you are not using the Fast Recovery Area to store RMAN backups, the FRA only needs to be large enough to contain the flashback logs for the duration of cutover. The size will depend on your selected retention policy (based on the time needed to perform cutover), and should be sufficient to accommodate your online redo log files. If you set the size too small, you may experience space availability issues, resulting in restoration failure. It is therefore advisable to monitor space utilization: any shortages will be recorded in the database alert log.

To minimize FRA space requirements, the Online Patching cutover phase should be scheduled for a time when there are few online transactions, and batch processing is minimal.

SQL>alter system set db_recovery_file_dest_size = 10G scope=BOTH SID='*';
System altered.
SQL>alter system set db_recovery_file_dest = '/d1/oracle/test/ARCH' scope=BOTH SID='*';
System altered.

Note: It is generally advisable to use a server parameter file ("spfile") for managing the database initialization parameters. Refer to the documentation on Managing Initialization Parameters Using a Server Parameter File.

2.3 Specify maximum flashback time

The next command specifies the maximum number of minutes the database may be flashed back. This parameter determines how much flashback log data is kept in the recovery area.

SQL>alter system set db_flashback_retention_target=120;
System altered.

Note: The amount of retention time and space needed will be governed by the amount of time required for cutover. Setting the flashback retention target too high may result in issues if DB_RECOVERY_FILE_DEST_SIZE is set to a large value.

2.4 Activate Flashback

Flashback is activated using the following command:

SQL>alter database flashback on;
Database altered.

2.5 Create restore point

This step will create a restore point called BEFORE_CUTOVER. As shown in the example below, it is also recommended to force a logfile switch both before and after the restore point is created.

SQL>alter system switch logfile;
System altered.
SQL>create restore point BEFORE_CUTOVER guarantee flashback database;
Restore point created.

SQL>alter system switch logfile;
System altered.

You are now ready to flashback the database if needed.

Note: As noted under the FRA description, the Online Patching cutover phase should be scheduled for a time when there are few online transactions and batch processing is minimal. You should confirm that critical concurrent requests are not executing during cutover.  You should also consider putting scheduled concurrent requests on hold prior to creating the BEFORE_CUTOVER flashback restore point.

Top


3. Problem Scenario

You are running an Online Patching cycle:

$ adop phase=prepare
...
$ adop phase=apply patches=11111111,22222222
...
$ adop phase=finalize
...
$ adop phase=cutover

Cutover fails, and you need to go back to the state of the system before you ran the cutover phase.

Note: If you had not run the cutover phase, you would have been able to roll back the patch application process by running the adop abort phase. However, this is not possible once cutover has been run.

There are two main parts to the restore procedure:

  • You will at least need to restore the database using the Flashback feature (described in Section 4)
  • Depending on when cutover failed, you may also need to restore the application tier file systems (described in Section 5).

These activities will be considered in the next two sections.

Top


4. Flashing Back the Database

  1. First, shut down the database, then start it up in mount state:
    SQL>shutdown immediate
    Database closed.
    Database dismounted.
    ORACLE instance shut down.
    SQL>startup mount
    ORACLE instance started.
  2. Restore the flashback to the specified restore point:
    SQL>flashback database to restore point BEFORE_CUTOVER;
    Flashback complete.
  3. Start the database in read-only mode:
    SQL>alter database open read only;
    Database altered.
    Check all looks as expected.

  4. Shut down the database, start it up in mount state, then open it with the resetlogs option:
    SQL>shutdown immediate
    Database closed.
    Database dismounted.
    ORACLE instance shut down.
    SQL>startup mount
    ORACLE instance started.
    Total System Global Area 2142679040 bytes
    Fixed Size 1346140 bytes
    Variable Size 520095140 bytes
    Database Buffers 1593835520 bytes
    Redo Buffers 27402240 bytes
    Database mounted.
    SQL>alter database open resetlogs;
    Database altered.
  5. Disable flashback:
    SQL>alter database flashback off;
    Database altered.
  6. Drop the restore point:
    SQL>drop restore point BEFORE_CUTOVER;
    Restore point dropped.
  7. Set recovery file destination:
    SQL>alter system set db_recovery_file_dest='';
    System altered.
  8. Confirm that Flashback has been deactivated:
    SQL>select FLASHBACK_ON from v$database;
    FLASHBACK_ON
    ------------
    NO

This concludes the Flashback activities.

Note: If you are not using FRA, you may want to clean up the flashback space (unless you intend to keep it for the next adop cycle).

Top


5. Restoring the File Systems

Whether you need to perform this step is conditional, depending on whether cutover failed before the file systems were switched. You can identify which of these cases applies by referring to the cutover logs in $NE_BASE/EBSapps/log/adop/<current_session_id>/cutover_<timestamp>/ for your current session id.

Case 1 - If the log messages indicate that cutover failed before the file systems were switched, do a clean shutdown of any services that are running. Then restart all the services using the normal startup script, and go to Section 6.

Case 2 - If the log messages indicate that cutover failed after the file systems were switched, follow Step 5.1 to shut down any services that have started from the new run file system, then follow Step 5.2 to switch the file systems back. After that, go to Section 6.

5.1 Shut down services started from new run file system

  1. Source the environment on the new run file system.
  2. From $ADMIN_SCRIPTS_HOME, shut down all the services (using adstpall.sh on UNIX).
  3. In a multi-node environment, repeat the preceding two steps on all nodes, leaving the admin node until after all the secondary nodes.

5.2 Switch file systems back

  1. On all nodes where file systems have been switched, run the following command to switch the file systems back:
    $ perl $AD_TOP/patch/115/bin/txkADOPCutOverPhaseCtrlScript.pl \
    -action=ctxupdate \
    -contextfile=<full path to new run context file> \
    -patchcontextfile=<full path to new patch file system context file> \
    -outdir=<full path to out directory>
  2. Start up all services from the old run file system (using adstrtal.sh on UNIX).
  3. In a multi-node environment, repeat the preceding two steps on all nodes, starting with the admin node and then proceeding to the secondary nodes.

Top


6. Options and Next Steps

After the restore is complete, you have two basic options for proceeding:

  • Abort the current patching cycle, if the issue that required you to restore was caused by the patches you were attempting to apply.
  • Identify and fix any other issues in the current patching cycle, and proceed with patching.

Top


7. References

For more information on the database features involved, plus guidance on setting the parameters to meet your specific requirements, refer to:

  • Chapter 5, Configuring the RMAN Environment, in Oracle Database Backup and Recovery User's Guide 11g Release 2 (11.2), Part No. E10642.

For more information on the Oracle E-Business Suite Online Patching cycle, refer to:

  • Chapter 3, Patching Procedures, in Oracle E-Business Suite Maintenance Guide, Part No.E22954.

Oracle E-Business Suite Release 12.2: Backup and Recovery Guidelines For Online Patching (adop) Cutover (Doc ID 1584097.1)

Database Options/Management Packs Usage Reporting for Oracle Databases 11.2 and later (Doc ID 1317265.1)

  Database Options/Management Packs Usage Report You can determine whether an option is currently in use in a database by running options_pa...