Saturday, May 29, 2021

Concurrent Processing - Best Practices for Performance for Concurrent Managers in E-Business Suite (Doc ID 1057802.1)

 

APPLIES TO:

Oracle Concurrent Processing - Version 11.5.0 to 12.2.5 [Release 11.5 to 12.2]
Oracle Application Object Library - Version 12.1.3 to 12.1.3 [Release 12.1]
Oracle Concurrent Processing - Version 12.2.6 to 12.2.6 [Release 12.2]
Information in this document applies to any platform.

PURPOSE

Provide the best practices to achieve better performance for concurrent manager in Oracle E-Business Suite.

Please also visit the Concurrent Processing Product Information Center (Note 1304305.1) for the latest in CP recommendations and solutions.

SCOPE

Applications DBAs, System Administrators involved  in configuration and administration of Oracle E-Business Suite.

DETAILS

Best Practices for Performance for Concurrent Managers in E-Business Suite

This Document contains 5 topics

1. Generic Tips

2. Transaction Manager (TM).

3. Parallel Concurrent Processing (PCP) Environment.

4. Tuning Output Post Processor (OPP).

5. Concurrent Processing Server Tuning.

Generic Tips

 
1) Sleep Seconds -  is the number of seconds your Concurrent manager waits between checking the list of pending concurrent requests (concurrent requests waiting to be started). A manager only sleeps if there are no runnable jobs in the queue.

Tip: During peak time, when the number of requests submitted is expected to be high, Set the sleep time to a reasonable wait time(e.g. 30 seconds) dependent on the average run time and to prevent backlog. Otherwise set the sleep time to a high number (e.g. 2 minutes). This avoids constant polls to check  for new requests. 

2) Increase the cache size (number of requests cached) to at least twice the number of target processes.

For example, if a manager's work shift has 1 target process and a cache value of 3, it will read three requests, and try to run those  three requests before reading any new requests

 
Tip: Enter a value of 1 when defining a manager that runs long, time-consuming jobs, and a value of 3 or 4 for managers that run small, quick jobs.
This is only guidance and a balance needs to struck in tuning the cache, so with fast jobs you need to cache to get enough work for a few minutes. With slow jobs, a small queue helps should you need to reprioritize requests.

4) To maximize throughput consider reducing the sleep time of the Conflict Resolution Manager (CRM).  The default value is 60 seconds. You can consider setting to 5 or 10 seconds.

5) Avoid enabling an excessive number of standard or specialized managers. It can degrade the performance due polling on queue tables (FND_CONCURRENT_REQUESTS...). You need to create specialized managers only if there is a real need.

6) Set the system profile option "Concurrent: Force Local Output File Mode" to "Yes" if required  . You need to apply Patch 7530490 for R12 (or) Patch 7834670 for 11i to get this profile.

Refer to Note 822368.1 - 'Purge Concurrent Request FNDCPPUR Does Not Delete Files From File System or Slow performance'.

Note:- The profile option "Concurrent: Force Local Output File Mode" is set to "No" by default. After applying the patch, set the profile option to YES will cause FNDCPPUR to always access files on the local file system, hence FNDCPPUR will remove the OS files faster.To enable this feature, All Concurrent Manager nodes must be able to access the output file location via the local filesystem.

7) Truncate the reports.log file in log directory.   Refer to Note 844976.1 for more details.

Truncation of file "reports.log" is a regular maintenance work of Application DBA. Make sure that reports log file size should not increase to its maximum limit of 2 GB. There is no purge program to truncate file "reports.log". This maintenance needs to be done manually and regularly depending on number of concurrent program which uses "reports.log". You can safely truncate "reports.log".

The "reports.log" file can be located under $APPLCSF/$APPLLOG.

8) Ensure "Purge Concurrent Request and/or Manager Data, FNDCPPUR,"  is run at regular intervals with "Entity" parameter as "ALL".  A high number of records in FND_CONCURRENT tables can degrade the performance.

Additionally, the following are very good methods to follow for optimizing the process:
  • Run the job in hours with low workload. Doing this after hours will lessen the contention on the tables from running against your daily processing.
  • To get the requests under control, run the FNDCPPUR program with Age=20 or Age=18 would be a good method. That means, all requests older than 18 or 20 days will be purged.
  • Once the requests are under control, run the FNDCPPUR program with Age=7 to maintain an efficient process. This would solely depend on the level of processing that is performed at your site
9) Ensure that the log/out files are removed from the locations shown below as you run "Purge Concurrent Request and/or Manager Data program".
 $APPLCSF/$APPLLOG
 $APPLCSF/$APPLOUT

In the event that it does not remove the log/out files, over a period of time it will slow down the performance.  Please refer to the following note which suggests the patch which fixes it.

  • Note 822368.1 - 'Purge Concurrent Request FNDCPPUR Does Not Delete Files From File System or Slow performance'.
  • Note 1616827.1  Managing Concurrent Manager Log and Out Directories 
10) Defragment the tables periodically to reclaim unused space / improve performance.  
  • FND_CONCURRENT_REQUESTS
  • FND_CONCURRENT_PROCESSES
  • FND_CRM_HISTORY
  • FND_ENV_CONTEXT
  • FND_TEMP_FILES
  • FND_CONFLICTS_DOMAIN
 

HOW TO DEFRAGMENT


10.1) alter table <owner>.<table_name> move;

10.2)Note that, some indexes might become unusable after table is moved, check the index status from dba_indexes for the table moved and rebuild them too as explained in next bullet.
select owner, index_name, status from dba_indexes
where table_owner = upper('&OWNER') and
table_name = upper('&SEGMENT_NAME');

10.3) alter index <owner>.<index_name> rebuild online;

Note:

Ensure Concurrent Manager is down before you defragment.

Ensure the tablespace in which the object currently exists has got sufficient space before you move/defragment.

Always take backup of the tables before moving the data. It is recommended to perform ths action on Test instance initially then test it thoroughly before performing it on Production instance.


10.4) You will need to re-collect the statistics for the tables.

For example:
exec fnd_stats.gather_table_stats ('APPLSYS','FND_CONCURRENT_REQUESTS',PERCENT=>99);

11) Ensure you upgraded to the latest code to avoid below known Performance , Availablity & Deadlock related issues issues.

Performance

  • Note 1615981.1 - Java Concurrent Managers Are Running Slower In Newer Servers Due To Lack Of Entropy In The System
  • Note 1075684.1 - 'Concurrent Managers are consuming high CPU and memory'
  • Note 1492893.1 - 'R12: Performance Issue When Standard Managers Waiting for "enq: TX - row lock contention" Held By ICM'
  • Note 1360118.1 - 'Performance: Concurrent Requests Hang in Pending Status For Long Time'
  • Note 1541526.1 - 'Performance: Concurrent Requests Hang in Pending Standby Status For Long Time'
  • Patch 10065439 When multiple JVMs run as in Concurrent Program can affect the performance

Dead Lock

  • Note 1060736.1 - 'Deadlock Error During Concurrent Request Termination'
  • Note 866298.1 - 'Concurrent Processing - ORA-00060: Deadlock Detected - UPDATE FND_CONCURRENT_QUEUES'

Availablity

  • Note 1604300.1 Concurrent Manager FNDCRM Down / Crashed All Scheduled Concurrent Programs Are Stuck Pending/Scheduled Pending/Standby
  • Note 1577982.1 Concurrent Manager FNDSM intermittently Crashes / Shutting Down Abnormally When a Concurrent Request is Cancelled
  • Note 1567057.1 Request Submitted By Custom Responsibility Associated With Custom Data Group Causes FNDLIBR To Coredump 
  • Note 1506643.1 Concurrent Manager Crashes FNDLIBR or INVLIBR Core dumps Continuously and get Terminated 
  • Note 1601174.1 Concurrent Manager Standard Manager Crashes / Terminated ; FNDLIBR Segfault In Operating System Log 
  • Note 1457414.1 Faulting application FNDLIBR.exe, version 0.0.0.0, faulting module oranls10.dll 
  • Note 1542216.1 Concurrent Requests fail Due to FNDSM Log File Size Grows 2 GB ; SQL*Loader-101: Invalid Argument for username/password
  • Note 1413393.1  12.1.3 PO Document Approval Manager (POXCON), Receiving Transaction Manager (RCVOLTM) and INV Remote Procedure Manager (INCTM) Do Not Start / Die After Restart 

Latest Performance Patches (RDBMS)

  • RDBMS Patches Patch 20355502 (OR) Patch 22521733 should be applied based on the RDBMS version to resolve issues with fnd_concurrent_worker_requests view and/or screen FNDCPVCM Administer Concurrent Manager
  • Reference Note 2106106.1 - 12.2 E-Business Suite Conflict Resolution Concurrent Manager Not Picking Up Requests Due To FND_CONCURRENT_CRM_REQUESTS View Performance Issue (Doc ID 2106106.1)

Transaction Manager(TM)

12 ) Profile Concurrent:Wait for Available TM  -  Total time to wait for a TM before switchover to next available TM.  Consider setting this  to 1 (second).

13) Ensure enough TMs exist to service the incoming request load.

14) When the load is high, set the following profile to optimum values to achieve better results.

  PO: Approval Timeout Value  -  Total time for workflow call (When initiated from Forms) to time out.

15)  Set the sleep time on the Transaction Manager to a high number (e.g. 10 minutes), this avoids constant polls to check for shutdown requests.

Parallel Concurrent Processing (PCP) Environment

16) If the failover of managers is taking too long refer to Note 551895.1 - 'Failover Of Concurrent Manager Processes Takes More than 30 Minutes'.
 
17) To avoid known issues, apply Patch 15900099 (11i), Patch 15981173 (12.0 ), Patch 15981176 (12.1.3) along with pre-requisites , when you are in the process of implementing PCP. Also refer to Note 1389261.1.
 
18) Set profile option 'Concurrent: PCP Instance Check' to 'OFF' if instance-sensitive failover is not required. Setting it to 'ON' means that concurrent managers will fail over to a secondary application tier node if the database instance to which it is connected goes down.
 
19)Transaction Manager uses DBMS_PIPE to communicate with application session prior to 11i.ATG_PF.H RUP3. DBMS_PIPE in turn uses OS Pipe.We might use Advance Queue(AQ) with 11i.ATG_PF.H RUP3 by setting System Profile Concurrent: TM Transport Type to QUEUE.
 
Note  Pipes are more efficient but require a Transaction Manager to be running on each DB Instance (RAC). So you might want to use "Queue" for easy maintenance.
 
20) Add these parameters depends on your Database version
 
                + _lm_global_posts=TRUE
                + _immediate_commit_propagation=TRUE  (11g RAC)
                + max_commit_propagation_delay=0  (9i RAC)
 
21) To speed up the PCP Failover ,Tune the below parameters.
  • Kernel parameters (Find the analogous parameter for your platform)

tcp_keepalive_intvl
tcp_keepalive_probes
tcp_keepalive_time ( Do not set this value to low; since it will then use up your network resources with unnecessary traffic)

  • DCD (Dead connection detection); To setup update sqlnet.ora from the Database Tier.

sqlnet.expire_time

  • Set the environment Variable at Concurrent Manager Tier.

FDCPTRW

  • Check PMON Cycle & Sleep Intervals for ICM (internal Concurrent Manager) setup.

Navigation OAM -> SiteMap -> Monitoring -> Internal Concurrent Manager Link(Under Availability) -> "View Status" -> "Edit ICM Runtime Parameters"

  • Tune Failover Processes.

In the case of node failover, the maximum number of processes that the work shift can run simultaneously.

Nodes can become overloaded when a middle-tier node fails and service instances on that node failover to their secondary nodes. The Failover Processes value should be smaller than the normal Processes value, to lessen the impact on the existing resources allocated on a secondary node. When failover occurs, the ICM uses the Failover Processes value in place of the normal running processes value as it iterates through service instances to perform queue sizing.

Navigation System Administration > Concurrent Managers > Standard Manager > Edit > Failover Processes

                    System Administration > Concurrent Managers > Output Post Processor > Edit > Failover Processes

  • Enable Reviver.

What is FNDREVIVER and How Is It Set? Note 466752.1

  • Ensure Internal Monitor is up and running on all the PCP Nodes. Ensure it has a valid workshift.

            Concurrent > Manager > Define > Query "Internal Monitor" > workshift

            Note:- The sole job of an Internal Monitor Process is to monitor the Internal Concurrent Manager and to restart that manager should it fail. The first Internal Monitor Process to detect that the Internal Concurrent Manager has failed restarts that manager on its own node.

PCP references:
NOTE: Multiple active managers are capable of processing the same jobs and can be  defined to run at the same time.
How To Run a Concurrent Program Against a Specific RAC Instance with PCP/RAC Setup? (Note 1129203.1)
How to Activate Parallel Concurrent Processing - Background Facts and Setup Steps (Note 602899.1)

  

Tuning Output Post Processor (OPP)

In order to tune the OPP to improve performance refer the below Note. It discusses  How to monitor the workload of OPP and Suggests you how to tune Output Post Processor (OPP) to improve performance and avoid java.lang.OutOfMemoryError exception. 

Note 1399454.1 - 'Tuning Output Post Processor (OPP) to Improve Performance'.

Concurrent Processing Server Tuning

1. Any Concurrent Processing (CP) server tuning or load balancing needs are to be addressed by Oracle Consulting. There are way too many site specific factors that needs to be considered for optimum CP throughput: from machine hardware, to user request volume, to required Work Shifts, to programs run time characteristics (long / short running)--not to mention also testing and benchmarking. Such a tasks, is beyond the scope of ATG Support.

ATG support would be glad to investigate a failing manager or program issue; however, CP performance issues due to increased concurrent request volume or due to a new installation needs to be addressed by Oracle Consulting.

2. The "Tuning Concurrent Processing" chapter of the white paper "A Holistic Approach To Performance Tuning Oracle Applications Systems Release 11 and 11i" Note 69565.1 may provide some basic insight. Also reference the "Defining Concurrent Managers" and the "Setting Up and Starting Concurrent Managers" chapters of the "Oracle Applications System Administrator's Guide - Configuration".

3. As per Note 69565.1 "A Holistic Approach to Performance Tuning Oracle Applications Systems", "50% of concurrent processing performance tuning is in the business!"

4. Visit the Concurrent Processing Product Information Center (PIC) Note 1304305.1 for additional performance and setup documentation.

Information Center, Diagnostics, & Community

  • E-Business Concurrent Processing Information Center Note 1304305.1
    Please reference this document regularly to review current offerings for Concurrent Processing needs.
  • Diagnostics
    For additional help, please refer to one of the following documents on diagnostics to address current needs. Providing diagnostic output on an issue for support when logging a service request is very helpful.

    Note 179661.1 for 11i or Note 421245.1 for Rel 12.x
  • Core Concurrent Processing Community
    Visit the Core Concurrent Processing community for help from industry experts or to share knowledge.

REFERENCES

NOTE:1360118.1 - Performance: Concurrent Requests Hang in Pending Status For Long Time
NOTE:1060736.1 - Deadlock Error During Concurrent Request Termination
NOTE:1492893.1 - Performance Issue When Standard Managers Waiting for "enq: TX - row lock contention" Held By ICM
NOTE:866298.1 - Concurrent Processing - ORA-00060: Deadlock Detected - UPDATE FND_CONCURRENT_QUEUES
NOTE:1399454.1 - Tuning Output Post Processor (OPP) to Improve Performance
NOTE:1616827.1 - Managing Concurrent Manager Log and Out Directories
NOTE:844976.1 - Concurrent Processing - Concurrent Reports Failing With Errors REP-0004,REP-0082 and REP-0104
NOTE:466752.1 - Concurrent Processing - What is FNDREVIVER and How Is It Set?
NOTE:551895.1 - Concurrent Processing - Failover Of Concurrent Manager Processes Takes More than 30 Minutes
NOTE:1389261.1 - PCP Concurrent Manager Failover/Failback Does Not Work When Application Listener is Down On Primary Node
NOTE:822368.1 - Concurrent Processing - How To Run the Purge Concurrent Request FNDCPPUR, Which Tables Are Purged, And Known Issues Like Files Are Not Deleted From File System
NOTE:104452.1 - Concurrent Processing - Troubleshooting Concurrent Manager Issues (Unix specific)
NOTE:1060707.1 - Purge Concurrent Requests/Manager Data, FNDCPPUR, Not Removing Files From Filesystem
NOTE:1075684.1 - Concurrent Managers are consuming high CPU and memory
NOTE:1615981.1 - Java Programs Are Running Slower In Newer Servers Due To Lack Of Entropy In The System

Example Manual Migration of Oracle E-Business Suite Release 12.2 with Oracle Database 12.1.0.2 to VM DB Systems and Compute Cloud Service in Oracle Cloud Infrastructure (Doc ID 2673057.1)

 

Overview

This document provides one example of how you can manually migrate an Oracle E-Business Suite Linux on-premises installation to Oracle Cloud Infrastructure and VM DB Systems, in cases where the Oracle E-Business Suite Cloud Tools cannot be used.

The example migration shown is for Oracle E-Business Suite Release 12.2 with Oracle Database 12.1.0.2.

Notes:
  • We strongly recommend that you use the available automation to lift and shift Oracle E-Business Suite environments to Oracle Cloud Infrastructure and VM DB Systems, as this will facilitate environment management operations such as backing up, copying, and deleting.

    For information about using the Oracle E-Business Suite Cloud Tools to migrate your environment, refer to "4.2.2 Lift and Shift Oracle E-Business Suite from On-Premises" in My Oracle Support Knowledge Document 2517025.1Getting Started with Oracle E-Business Suite on Oracle Cloud Infrastructure.

    If you are unable to use Oracle E-Business Suite Cloud Manager, please tell us why by following the instructions in Document 2181340.2Obtaining Support for Oracle Applications on Oracle Cloud - Infrastructure as a Service (IaaS) and Platform as a Service (PaaS), to log a service request (choosing Problem Type: Manual Cloud Procedures).

  • When migrating your Oracle E-Business Suite system from on-premises to Oracle Cloud, it is essential to ensure that the performance of key operations will continue to meet your requirements after the transition. We recommend that you first migrate a test environment, and then conduct a benchmark test with a configuration, product mix, and user load that simulates your expected workload. These controlled conditions will help verify performance before you migrate your production environment. For more information, refer to Document 2528000.1Performance Best Practices for Oracle E-Business Suite on Oracle Cloud Infrastructure and On-Premises.

The steps in this example migration adhere to the standards listed in My Oracle Support Knowledge Document 2656874.1Standards Used by the Oracle E-Business Suite Cloud Manager for Provisioning Oracle E-Business Suite on Oracle Cloud Infrastructure.

Be aware that Oracle is unable to validate custom migration procedures. Therefore, when performing your own procedures you should always undertake multiple trial runs first, and verify that the resulting environment functions properly and meets the higher of the minimum requirements as noted in the following My Oracle Support knowledge documents:

  • Table 1 - Certified Combinations for Linux x86-64 in Document 2517025.1Getting Started with Oracle E-Business Suite on Oracle Cloud Infrastructure
  • Document 1195034.1Oracle E-Business Suite Error Correction Support Policy (V.5)

In This Document

There is a change log at the end of this document.

The current version of this document can be obtained in My Oracle Support Knowledge Document 2673057.1.

Section 1: Before You Begin

1.1 Prerequisites for the Migration

To follow the procedure in this document successfully, you will need:

  • An on-premises Oracle E-Business Suite environment on Linux x86-64 with:
    • The application tier on Oracle E-Business Suite Release 12.2.3 or later.
    • The database tier on Oracle Database 12.1.0.2, with a PSU/bundle patch of April 2021 or earlier.
  • Subscriptions to the following cloud services (with associated tenancy administrator privileges):
  • A tool (such as ssh-keygen) for generating SSH key pairs on the client machine to connect to Oracle Compute Cloud Service.

Note that Oracle Database 12.1.0.2 Bundle Patches are automatically included when you create a new VM DB System. The current patch level for new provisioning is April 2021.

1.2 Software Prerequisites

The procedure described in this document requires the following versions of Oracle E-Business Suite and Oracle Database, and the associated patches for each:

    Software ComponentApplicable ReleasesAdditional Information
    Oracle E-Business Suite

    12.2.3 and later

    Document 1594274.1Oracle E-Business Suite Release 12.2: Consolidated List of Patches and Technology Bug Fixes

    Apply the latest AD-TXK code level and other patches

    12.2.3 and later

    Operating system for the source database

    Oracle Linux 6 or 7

    N/A
    Operating system for the target database

    Oracle Linux 7

    N/A
    Source Oracle Database12.1.0.2

    Oracle Database Bundle Patch April 2021 or earlier

    Target Oracle Database

    12.1.0.2

    Oracle Database Bundle Patch April 2021

 

Section 2: Establish Compartment, Group, and User

In this section, the tenancy administrator will perform tasks in order to establish a compartment, group, and users.

2.1 Create Compartment

Note: If you already have an existing compartment you want to use, skip this step and proceed to 2.2 Create Group.

As the tenancy administrator in the Oracle Cloud Infrastructure console, create a new compartment that will be associated with a Virtual Cloud Network (VCN) where your Oracle E-Business Suite environment will reside (for example, "ebs-compartment").

2.2 Create Group

As the tenancy administrator, sign in to the Oracle Cloud Infrastructure Console and add a new group. In this example, we will use "oci-ebsadmin." Create a policy for the oci-ebsadmin group in the root compartment with the following policy statements:

  • Allow group <group_name> to manage all-resources in compartment <compartment_name>
  • Allow group <group_name> to manage buckets in tenancy

2.3 Create User

Create a user in the Oracle Cloud Infrastructure console as the tenancy administrator. Generate the user's password and provide it to the user, who will need it create network resources and perform the remainder of the tasks in this document. Add the user to the oci-ebsadmin group.

Section 3: Establish a Virtual Cloud Network and Associated Network Resources

In this section, the Oracle E-Business Suite Cloud Administrator will perform the tasks in this section to establish a Virtual Cloud Network (VCN) and associated network resources (internet gateway, route tables, security list, and subnets).

3.1 Create a Virtual Cloud Network

Note: If you already have an existing Virtual Cloud Network (VCN) that you want to use, skip this step and proceed to 2.3.
  1. From the Oracle Cloud Infrastructure Service Console, click on the menu icon at the top left the open the navigation menu. Under CORE INFRASTRUCTURE, go to Networking , and click Virtual Cloud Networks.
  2. Click Create Virtual Cloud Network.
  3. Enter the required details about the Virtual Cloud Network (VCN).
    1. CREATE IN COMPARTMENT: Select your compartment (such as "ebs-compartment").
    2. NAME: ebs-vcn
    3. CIDR BLOCK: Specify your choice of CIDR.
  4. Click Create VCN.

For more information, see "VCNs and Subnets" on the Oracle Cloud Infrastructure Documentation web page.

3.2 Create an Internet Gateway or NAT Gateway

The type of gateway chosen is based on how the application will be used. The following diagram illustrates a basic network layout. As you see in the example, arrows indicate that in an internet gateway, connections can be initiated in both directions from within the VCN and from the internet. In a NAT gateway, connections are only initiated in one direction from the VCN.

A NAT gateway is recommended for subnets in your VCN that do not require ingress connections from the internet.

3.2.1 Create Internet Gateway

To create an internet gateway:

  1. On the Virtual Cloud Networks screen, click on the link with the name of your VCN (such as "ebs-vcn").
  2. Under Resources on the navigation menu at the left, select Internet Gateways.
  3. Click Create Internet Gateway:
    1. CREATE IN COMPARTMENT: Select your compartment name (such as "ebs-compartment").
    2. NAME: Optional, but you can specify a name (such as "ebs-igw").
    3. Click Create Internet Gateway at the bottom of the window.

3.2.2 Create NAT Gateway

To create a NAT gateway:

  1. On the Virtual Cloud Networks screen, click on the link with the name of your VCN, (such as "ebs-vcn").
  2. Under Resources on the navigation menu at the left, select NAT Gateway.
  3. Click Create NAT Gateway:
    1. CREATE IN COMPARTMENT: Select your compartment name, such as ebs-compartment.
    2. NAME: Optional, but you can specify a name (such as "ebs-ngw").
    3. Click Create NAT Gateway at the bottom of the window.

3.3 Create Route Tables

Create four separate route tables. Their roles as well as example names are shown in the following table:

Component Route Table Needed ForExample Route Table Name
Bastion Server VMbastvm-RouteTable
Load Balancerebslbaas-RouteTable
Oracle E-Business Suite Application Tierapps-RouteTable
Oracle E-Business Suite Database Tierdb-RouteTable

The steps you will take depend on whether you are creating a public subnet or a private subnet. Follow the steps in either 3.3.1 or 3.3.2, as appropriate.

3.3.1 Create Route Tables for Public Subnet

To create the route tables for a public subnet:

  1. On the Virtual Cloud Networks screen, click on the link with the name of your VCN (such as "ebs-vcn").
  2. Under Resources on the navigation menu at the left, select Route Tables.
  3. Click Create Route Table:
    1. CREATE IN COMPARTMENT: Select your compartment name (such as "ebs-compartment").
    2. NAME: Specify a name (such as "bastvm-RouteTable", "ebslbaas-RouteTable", "apps-RouteTable", or "db-RouteTable").
    3. Click on + Additional Route Rule.
    4. Enter Route Rules details as follows:
      1. TARGET TYPE: Select Internet Gateway.
      2. DESTINATION: 0.0.0.0/0
      3. COMPARTMENT: Select the previously identified compartment.
      4. TARGET INTERNET GATEWAY: Select the previously created gateway.
    5. Click Create Route Table at the bottom of the window.

Repeat these steps for the four route tables you will create.

3.3.2 Create Route Tables for Private Subnet

To create the route tables for a private subnet:

  1. On the Virtual Cloud Networks screen, click on the link with the name of your VCN (such as "ebs-vcn").
  2. Under Resources on the navigation menu at the left, select Route Tables.
  3. Click Create Route Table:
    1. CREATE IN COMPARTMENT: Select your compartment name (such as "ebs-compartment").
    2. NAME: Specify a name (such as "bastvm-RouteTable", "ebslbaas-RouteTable", "apps-RouteTable", or "db-RouteTable").
    3. Enter Route Rules details as follows:
      1. TARGET TYPE: Select NAT Gateway.
      2. DESTINATION: Select "All Services In Oracle Services Network".
      3. COMPARTMENT: Select the previously identified compartment.
      4. TARGET NAT GATEWAY: Select the previously created gateway.
    4. Click Create Route Table at the bottom of the window.

Repeat these steps for the four route tables you will create.

3.4 Create Security Lists

Create four separate security lists. Their roles as well as example names are shown in the following table:

Component Security List Needed ForExample Security List Name
Bastion Server VMbastvm-seclist
Load Balancerebslbaas-seclist
Oracle E-Business Suite Application Tierapps-seclist
Oracle E-Business Suite Database Tierdb-seclist

Repeat the following steps for each of the four security lists:

  1. On the Virtual Cloud Networks screen, click on the link with the name of your VCN (such as "ebs-vcn").
  2. Under Resources on the navigation menu at the left, select Security Lists.
  3. Click Create Security List:
    1. CREATE IN COMPARTMENT: Select your compartment name (such as "ebs-compartment").
    2. SECURITY LIST NAME: Specify a name (such as "bastvm-seclist", "ebslbaas-seclist", "apps-seclist", or "db-seclist").
    3. Click Create Security List at the bottom of the window.

3.5 Create Subnets

Create five new subnets specifying your own names and parameters. You can use the examples in the following table for guidance:

Subnet NameAvailability Domain (AD)CIDR BlockRoute TableSubnet AccessSecurity Lists
bastvm-subnet-ad1AD-110.0.0.0/24bastvm-RouteTablePublic or private subnetbastvm-seclist
ebslbaas-subnet-ad1AD-110.0.1.0/24ebslbaas-RouteTablePublic or private subnetebslbaas-seclist
apps-subnet-ad1AD-110.0.2.0/24apps-RouteTablePublic or private subnetapps-seclist
ebslbaas-subnet-ad2AD-210.0.3.0/24ebslbaas-RouteTablePublic or private subnetebslbaas-seclist
db-subnet-ad1AD-110.0.4.0/24db-RouteTablePublic or private subnetdb-seclist

Repeat the following steps for each of the five subnets:

  1. On the Virtual Cloud Networks screen, click on the link with the name of your VCN (such as "ebs-vcn").
  2. Under Resources in the navigation menu on the left, select Subnets.
  3. Click Create Subnet.
  4. Select "Availability Domain Specific".
  5. Select the Availability Domain from the drop-down list.
  6. Specify your choice for the following parameters:
    1. CIDR BLOCK
    2. ROUTE TABLE
    3. SUBNET ACCESS
    4. SECURITY LIST
  7. Click Create at the bottom of the window.

For more information, see "VCNs and Subnets" on the Oracle Cloud Infrastructure Documentation web page.

3.6 Create Security Rules

In this section, you will add ingress and egress security rules to the security lists you created, to allow secure communication between the instances assigned to the associated subnets.

3.6.1 Bastion Server Security List

To add ingress and egress rules to the bastion server security list:

  1. Navigate to Networking, then select Virtual Cloud Networks.
  2. Select your VCN.
  3. Click Security Lists in the left menu underneath Resources.
  4. Select the security list associated with the bastion server (for example, "bastvm-seclist").
  5. Click Add Ingress Rules.
  6. Ensure that an ingress rule exists with the following attributes:
    1. SOURCE TYPE: CIDR
    2. SOURCE CIDR: The CIDR address representing your network IP range in the form a.b.c.d/e (where a through e are integers from 1 to 255)
    3. IP PROTOCOL: SSH (TCP/22)
    4. DESTINATION PORT: 22
  7. Click Add Ingress Rules.
  8. Click the link "Egress Rules", then click Add Egress Rules.
  9. Ensure that an egress rule exists with the following attributes:
    1. DESTINATION TYPE: CIDR
    2. DESTINATION CIDR: The CIDR address representing your network IP range in the form a.b.c.d/e (where a through e are integers from 1 to 255)
    3. IP PROTOCOL: TCP
    4. SOURCE PORT RANGE: All
    5. DESTINATION PORT RANGE: All
  10. Click Add Egress Rules.

3.6.2 Load Balancer Security List

If you choose to implement a load balancer, then you will add the ingress and egress rules to the load balancer security list as shown:

  1. Navigate to Networking, then select Virtual Cloud Networks.
  2. Select your VCN.
  3. Click Security Lists in the left menu underneath Resources.
  4. Select the security list associated with the load balancer (for example, "ebslbaas-seclist").
  5. Click Add Ingress Rules or Add Egress Rules to add the security rules:
  • Ingress Rules for Both Public and Private Subnets

    Source TypeSourceProtocolSource Port Range/ Type and CodeDestination Port Range / Type and Code
    CIDRCIDR that describes the IP range users will use to access your Oracle E-Business Suite environments.TCPAllDepends on the web entry port you will use during the provisioning of your environment.

  • Egress Rule When Using a Public Subnet

    Destination TypeDestinationProtocolSourceDestination
    CIDR0.0.0.0/0TCPAllAll
    CIDR0.0.0.0/0ICMPN/AN/A

  • Egress Rule When Using a Private Subnet

    Destination TypeDestinationProtocolSourceDestination
    CIDR<Internal application tier subnet CIDR>TCPAllAll
    CIDR0.0.0.0/0ICMPN/AN/A

3.6.3 Application Tier Security List

Add the following ingress and egress rules to the application tier security list as shown:

  1. Navigate to Networking, then select Virtual Cloud Networks.
  2. Select your VCN.
  3. Click Security Lists in the left menu underneath Resources.
  4. Select the security list associated with the application tier (for example, "apps-seclist").
  5. Click Add Ingress Rules or Add Egress Rules to add the application tier security list:
  • Ingress Rules for Both Public and Private Subnets

  • Source TypeSourceProtocolSource Port Range / Type and CodeDestination Port Range / Type and Code
    CIDR<Internal application tier subnet CIDR>TCPAllAll
    CIDR<EBS Cloud Manager subnet CIDR>ICMPN/A (leave Type and Code blank)N/A (leave Type and Code blank)
    CIDR<Internal load balancer subnet CIDR>ICMPN/A (leave Type and Code blank)N/A (leave Type and Code blank)
    CIDR<EBS Cloud Manager subnet CIDR>TCPAll22
    CIDR<External application tier subnet CIDR>TCPAll111
    CIDR<External application tier subnet CIDR>TCPAll2049
    CIDR<Database tier subnet CIDR>ICMPN/A (leave Type and Code blank)N/A (leave Type and Code blank)
    CIDR<Internal application tier subnet CIDR>ICMPN/A (leave Type and Code blank)N/A (leave Type and Code blank)
    CIDR<External application tier subnet CIDR>TCPAll7001-7003
    CIDR<External application tier subnet CIDR>TCPAll6801-6802
    CIDR<External application tier subnet CIDR>TCPAll16801-16802
    CIDR<External application tier subnet CIDR>TCPAll12345
    CIDR<External application tier subnet CIDR>TCPAll36501-36550
    CIDR<Internal load balancer subnet CIDR>TCPAll8000

  • Egress Rule When Using a Public Subnet

    Destination TypeDestinationProtocolSourceDestination
    CIDR0.0.0.0/0TCPAllAll
    CIDR0.0.0.0/0ICMPN/AN/A

  • Egress Rule When Using a Private Subnet

    Destination TypeDestinationProtocolSourceDestination
    CIDR134.70.0.0/17TCPAllAll
    ServiceAll <XXX> Services in the Oracle Services Network
    (XXX is a region-specific code, such as IAD or LHR)
    TCPAllAll
    ServiceAll <XXX> Services in the Oracle Services Network
    (XXX is a region-specific code, such as IAD or LHR)
    ICMPN/AN/A
    CIDR<External application tier subnet CIDR>TCPAllAll
    CIDR<Internal application tier subnet CIDR>TCPAllAll
    CIDR<Database tier subnet CIDR>ICMPAll1521-1524
    CIDR<EBS Cloud Manager subnet CIDR>TCPAll443
    CIDR0.0.0.0/0ICMPN/AN/A

 

3.6.4 Database Tier Security List

Add the ingress and egress rules to the database tier security list as shown:

  1. Navigate to Networking, then select Virtual Cloud Networks.
  2. Select your VCN.
  3. Click Security Lists in the left menu underneath Resources.
  4. Select the security list associated with the database tier (for example, "db-seclist").
  5. Click Add Ingress Rules or Add Egress Rules to add the database tier security list:
  • Ingress Rules for Both Public and Private Subnets

  • Source TypeSourceProtocolSource Port Range / Type and CodeDestination Port Range / Type and Code
    CIDR<EBS Cloud Manager subnet CIDR>ICMPN/A (leave Type and Code blank)N/A (leave Type and Code blank)
    CIDR<Database tier subnet CIDR>ICMPN/A (leave Type and Code blank)N/A (leave Type and Code blank)
    CIDR<EBS Cloud Manager subnet CIDR>TCPAll22
    CIDR<Internal application tier subnet CIDR>TCPAll1521-1524
    CIDR<Internal application tier subnet CIDR>ICMPN/A (leave Type and Code blank)N/A (leave Type and Code blank)
    CIDR<External application tier subnet CIDR>TCPAll1521-1524
    CIDR<External application tier subnet CIDR>ICMPN/A (leave Type and Code blank)N/A (leave Type and Code blank)
    CIDR<Database tier subnet CIDR>TCPAll22
    CIDR<Database tier subnet CIDR>TCPAll1521-1524

  • Egress Rule When Using a Public Subnet

    Destination TypeDestinationProtocolSourceDestination
    CIDR0.0.0.0/0TCPAllAll
    CIDR0.0.0.0/0ICMPN/AN/A

  • Egress Rule When Using a Private Subnet

    Destination TypeDestinationProtocolSourceDestination
    CIDR134.70.0.0/17TCPAllAll
    CIDR<EBS Cloud Manager subnet CIDR>TCPAll443
    CIDR<Database tier subnet CIDR>TCPAll1521-1524
    CIDR<Database tier subnet CIDR>TCPAll22
    CIDR0.0.0.0/0ICMPN/AN/A
    ServiceAll <XXX> Services in the Oracle Services Network
    (XXX is a region-specific code, such as IAD or LHR)
    TCPAllAll
    ServiceAll <XXX> Services in the Oracle Services NetworkICMPAllAll

Section 4: Complete Prerequisite Tasks

The procedure in this document incorporates the creation of a bastion host. The bastion host is used as a barrier between an external network (which may or may not be the public internet) and your application and database tiers. For more information on the use of bastion hosts, refer to "Bastion Hosts: Protected Access for Virtual Cloud Networks".

As a prerequisite for creating your bastion server instance and other cloud instances, you must generate an SSH key pair.

  1. Follow the instructions in "Managing Key Pairs on Linux Instances" to generate an SSH key on your local system (UNIX or Windows).
  2. Plan to supply the public SSH key to the Create Instance and Launch Instance wizards when creating your VMs in the following sets of instructions:
    1. Section 5: Create a Bastion Server Instance (on the Create Instance screen when prompted)
    2. Section 7: Create a Database Instance on Oracle Cloud Infrastructure VM DB System
    3. Section 9: Create a Compute Cloud Instance to Host the Oracle E-Business Suite Application Tier
  3. Copy the private key to the on-premises primary application tier server and to the on-premises database server. Ensure that the SSH configuration files on both servers (~/.ssh/config) have the entry "ServerAliveInterval 100".

These steps are necessary to establish connectivity between your client machine and your on-premises servers to your cloud instances.

Section 5: Create a Bastion Server Instance

To create a bastion server instance, use the following steps.

  1. Log in to the Oracle Cloud Infrastructure Service Console.
    1. Select Menu, then Compute, and then Instances in the left panel under the heading CORE INFRASTRUCTURE.
    2. Click Create Instance.
  2. In the Create Compute Instance dialog box, specify the following:
    1. Under Name your instance, enter a name for your bastion server VM (for example, "bastionvm").
    2. Under Choose an operating system or image source, leave the default selected (currently Oracle Linux 7.8).
    3. Under Availability domain, select the availability domain where you will place your bastion server VM.
    4. Under Choose instance shape, you will see a default shape, such as VM.Standard.E3.Flex shape. If you want to specify another shape, click on Change Shape.
    5. Under Configure Boot Volume, accept the default volume size.
    6. Under Add SSH Key, choose one of the following two options, either (i) or (ii):
      1. Select the Choose SSH key file radio button, and then click Choose Files to specify the file containing your SSH public key generated previously.
      2. Alternatively, select the Paste SSH keys radio button and paste the SSH public key content in the text field provided using the content of the public key generated previously.
    7. Under Configure Networking:
      1. From the Virtual cloud network compartment drop-down list, select the compartment where the bastion server will reside (for example, "ebs-compartment").
      2. From the Virtual cloud network drop-down list, choose your VCN. For example, ebs-vcn.
      3. Under Subnet compartment, specify the subnet compartment. For example, ebs-compartment.
      4. Under Subnet, specify the bastion server subnet (for example, bastvm-subnet-ad1".)

        Note: If the VM is associated with a public subnet and you want to assign a public IP address, select the Assign public IP address check box.

  3. Click the Create button at the bottom of your screen.

Section 6: Prepare the On-Premises Oracle E-Business Suite Application Tier

Perform the following steps on the application tier of the on-premises Oracle E-Business Suite instance:

  1. Source the run file system, such as in the following example:
    cd <Base Directory>
    . ./EBSapp.env RUN
  2. Then, run the following command:
    perl <AD_TOP>/bin/admkappsutil.pl
  3. Copy the appsutil.zip into the <ORACLE_HOME> of the on-premises Oracle E-Business Suite database tier, and unzip it using the following command:
    unzip -o appsutil.zip
  4. Synchronize the run and patch file systems:
    adop phase=fs_clone
    Note: Online patching cycle should be complete before synchronizing the run and patch file systems.

  5. Prior to proceeding with the steps in this document, download and run hcheck.sql from My Oracle Support Knowledge Document 136697.1hcheck.sql - Script to Check for Known Problems in Oracle8i, Oracle9i, Oracle10g, Oracle 11g and Oracle 12c and Above, which looks for some known common Data Dictionary problems.
    sqlplus / as sysdba
    SQL> @hcheck.sql
    Note: Any identified issues must be addressed before continuing with the database migration. Contact Oracle Support for assistance if required.

  6. Validate the time zone settings on the on-premises database tier.

    Note: The default time zone for the Oracle Cloud Infrastructure is UTC, but you can specify a different time zone. When you migrate your Oracle E-Business Suite environments to Oracle Cloud Infrastructure, you must ensure that the time zone settings are the same as they are on your source environment during the creation of the VM . For more information on creating a VM and specifying the required time zone refer to DB System Time Zone in the Oracle Cloud Infrastructure Documentation.

    1. On the on-premises database tier, source the Oracle environment file and run the following SQL query to identify the time zone settings:
      sqlplus / as sysdba
      SQL> select dbtimezone from dual;
    2. If the time zone value returned from the previous query is null, then execute the following command to retrieve the value from the operating system:
      /usr/sbin/hwclock --localtime

Section 7: Create a Database Instance on Oracle Cloud Infrastructure VM DB System

Perform the steps in this section to create a VM DB System instance. Ensure that the service name matches the database name that you want to keep. For example, use the database instance name of your on-premises instance.

  1. Log in to Oracle Cloud My Services using your Oracle Cloud Infrastructure credentials.

  2. Select the menu at the upper left corner of the page. Then under Oracle Database, select Bare Metal, VM, and Exadata to display the Database Cloud Service console.

  3. Click Create DB System to create a DB system with the values indicated as follows. Refer to "Creating a DB System" for more details on each of the fields:
    1. Select a compartment: By default, the DB system is created in your current compartment and you can use the network resources in that compartment.
    2. Name your DB system: A non-unique, display name for the DB system. An Oracle Cloud Identifier (OCID) uniquely identifies the DB system.
    3. Select an availability domain: The availability domain in which the DB system resides.
    4. Select a shape type: The shape type you select sets the default shape and filters the shape options in the next field. Select "Virtual Machine".
    5. Select a shape: The shape determines the type of DB system created and the resources allocated to the system. To specify a shape other than the default, click Change Shape, and select an available shape from the list. The recommendation for Oracle E-Business Suite is to use a minimum of VM.Standard2.x shape for the OCI VM DB System.
    6. Configure the DB system:
      1. Total node count: The number of nodes in the DB system, which depends on the shape you select. For virtual machine DB systems, you can specify either one or two nodes, except for VM.Standard2.1 and VM.Standard1.1, which are single-node DB systems.
      2. Oracle Database software edition: The database edition supported by the DB system. The database edition cannot be changed and applies to all the databases in this DB system. Virtual machine systems support only one database. Select Software Edition as Enterprise Edition High Performance at a minimum.
    7. Choose Storage Management Software: 1-node virtual machine DB systems only. Select Oracle Grid Infrastructure to use Oracle Automatic Storage Management (ASM), which is recommended for production workloads.
    8. Configure storage: Specify the following:
      1. Available storage (GB): Virtual machine only. The amount of Block Storage in GB to allocate to the virtual machine DB system. Available storage can be scaled up or down as needed after provisioning your DB system.
      2. Total storage(GB): Virtual machine only. The total Block Storage in GB used by the virtual machine DB system. The amount of available storage you select determines this value. Oracle charges for the total storage used.
      3. Cluster name: (Optional) A unique cluster name for a multi-node DB system. The name must begin with a letter and contain only letters (a-z and A-Z), numbers (0-9) and hyphens (-). The cluster name can be no longer than 11 characters and is not case sensitive.
    9. Add public SSH keys: The public key portion of each key pair you want to use for SSH access to the DB system. You can browse or drag and drop .pub files, or paste in individual public keys. To paste multiple keys, click + Another SSH Key, and supply a single key for each entry.
    10. Choose a license type: The type of license you want to use for the DB system. Your choice affects metering for billing.
      1. License Included means the cost of this Oracle Cloud Infrastructure Database service resource will include both the Oracle Database software licenses and the service.
      2. Bring Your Own License (BYOL) means you will use your organization's Oracle Database software licenses for this Oracle Cloud Infrastructure Database service resource. See Bring Your Own License for more information.
    11. Specify the network information.
      1. Virtual cloud network: The VCN in which to create the DB system. Click Change Compartment to select a VCN in a different compartment.
      2. Client Subnet: The subnet to which the DB system should attach. For 1- and 2-node RAC DB systems: Do not use a subnet that overlaps with 192.168.16.16/28, which is used by the Oracle Clusterware private interconnect on the database instance. Specifying an overlapping subnet will cause the private interconnect to malfunction.
      3. Hostname prefix: Your choice of host name for the bare metal or virtual machine DB system. The host name must begin with an alphabetic character, and can contain only alphanumeric characters and hyphens (-). The maximum number of characters allowed for bare metal and virtual machine DB systems is 16.
      4. Host domain name: The domain name for the DB system. If the selected subnet uses the Oracle-provided Internet and VCN Resolver for DNS name resolution, then this field displays the domain name for the subnet and it can't be changed. Otherwise, you can provide your choice of a domain name. Hyphens (-) are not permitted.
      5. Host and domain URL: Combines the host and domain names to display the fully qualified domain name (FQDN) for the database. The maximum length is 64 characters.
    12. Click on "Show Advanced Options" and ensure that the time zone settings are the same as they are on the source on-premises database. Then, click Next.
    13. Database Name: The name for the database. The database name must begin with an alphabetic character and can contain a maximum of eight alphanumeric characters. Special characters are not permitted.
    14. Database image: This controls the version of the initial database created on the DB system. By default, the latest available Oracle Database version is selected. You can also choose an older Oracle Database version, or choose a customized database software image that you have previously created in your current region with your choice of updates and one-off (interim) patches. See Oracle Database Software Images for information on creating and working with database software images. Select Oracle Database 12.1
    15. PDB name: (Required) The name of the default pluggable database (PDB). As part of an Oracle Database 12c deployment, a PDB name is mandatory, but the actual pluggable database created at this point will not be used by Oracle E-Business Suite. Therefore, enter a dummy value such as "DUMMYPDB." It will be removed later in this section.
    16. Create administrator credentials: A database administrator SYS user will be created with the password you supply.
      1. Username: SYS
      2. Password: A strong password for SYS, SYSTEM, TDE wallet, and PDB Admin. The password must be 9 to 30 characters and contain at least two uppercase, two lowercase, two numeric, and two special characters. The special characters must be _, #, or -. The password must not contain the username (SYS, SYSTEM, and so on) or the word "oracle" either in forward or reversed order and regardless of casing.
      3. Confirm password: Re-enter the SYS password you specified.
    17. Select workload type: Select Online Transactional Processing (OLTP).
    18. Configure database backups: Optionally, specify the settings for backing up the database to Object Storage.
    19. Click on Show Advanced Options to specify advanced options for the initial database:
      1. Character Set: The character set for the database. This value must match the value of the on-premises Oracle E-Business Suite database.
      2. National Character Set: The national character set for the database. This value must match the value of the on-premises Oracle E-Business Suite database.
    20. Click Create DB System.

  4. Wait for the DB System creation to complete and note the public and private IP address of the DB System created as these values will be needed later. This step may take some time.

    The ORACLE_HOME is located at /u01/app/oracle/product/12.1.0.2/dbhome_1 on both nodes. The storage is ASM.

  5. Ensure the TZ environment variable is not set in the current shell using unset TZ on the command line and remove this entry from any environment files sourced to set the environment when starting the database or TNS listener. You must use the OS time zone settings (not TZ) for the database time zone.

  6. During creation of the 12.1.0.2 database, you entered a dummy value for the PDB name. As this pluggable database is not needed, you will now drop it by performing the following steps:

    Note: In the case of a 2-Node VM DB System, you must close the database on each Oracle RAC node before dropping the database; the drop command can be run from either node.

    1. Using your private key, log in to the DB System as the oracle user.
      ssh -i <absolute path of the private key> oracle@<VM DB IP address>
    2. Connect to the CDB using sqlplus as sysdba.
      sqlplus / as sysdba
    3. Close the dummy PDB that is going to be removed. In this example, the PDB to be removed is DUMMYPDB.
      SQL> alter pluggable database DUMMYPDB close immediate;
    4. Drop the pluggable database.
      SQL> drop pluggable database DUMMYPDB including datafiles;
  7. Run the following query to validate the time zone on the Oracle Cloud Infrastructure database tier:
    sqlplus / as sysdba
    SQL> select dbtimezone from dual;

    Note: The time zone value returned by the above query on the Oracle Cloud Infrastructure database tier must match the time zone settings on the on-premises database tier, as described in Section 6.

  8. Refer to Appendix A to verify the Oracle Cloud Infrastructure VM DB PSU/Bundle level and apply the relevant Oracle E-Business Suite patches listed in the appendix to your Oracle Cloud Infrastructure VM DB instance. For a 2-Node VM DB System, the patches must be applied on both nodes, individually. Post-patch steps should not be executed at this point. They will be run after the database is restored.

Section 8: Clone and Configure an Oracle E-Business Suite Database Tier on the VM DB System Instance

8.1 Review and Reduce Number of Editions

Before you conduct the lift and shift, we recommend that you review the number of database editions. If there are more than 20 database editions, drop the old editions by running the adop actualize_all phase and then performing a full cleanup.

For detailed information and instructions, refer to "Dropping Old Editions With the actualize_all Phase" in the section "The Online Patching Cycle" of the Oracle E-Business Suite Maintenance Guide.

8.2 Prepare the On-Premises Database for Cloning

Perform the following steps on both nodes to clone the on-premises database tier to the OCI VM DB instance.

Note: These steps must be performed on both nodes in an Oracle RAC configuration.
  1. Using your private key, log in to the VM DB System through the bastion server, and then switch to the root user, as shown in the following example (which uses ssh):
    ssh -i <absolute path of private key> -J <Bastion VM IP address> opc@<VM DB IP address>
    sudo -i
    Note: The exact method you use to establish this connection to the VM DB System from the client machine through the bastion server host may vary. If you are using ssh, older SSH clients may not have the -J or ProxyJump option, so you can use an alternative method such as stdio forwarding using ProxyCommand to achieve the connection via proxy. Similarly, if you are using PuTTY or another tool, follow instructions in the product documentation to connect via proxy.

  2. Update the SSH configuration parameters on the instance.
    1. To allow the multi-threaded transfer of files and to prevent client timeout, edit /etc/ssh/sshd_config and modify (or add, if necessary) the settings shown:
      # To allow multi-threaded transfer of files.
      MaxStartups 100
      # To prevent client timeout.
      ClientAliveInterval 100
      ClientAliveCountMax 99999
    2. Still as the root user, restart the SSHD service:
      systemctl restart sshd
  3. It is recommended that you always start the database using srvctl, which automatically runs setasmgidwrap and resets the group to asmadmin. Alternatively, you can manually run setasmgidwrap as the grid user to revert the group to asmadmin using the following commands:
    export GRID_HOME=/u01/app/19.0.0.0/grid
    export ORACLE_SID=+ASM1
    export PATH=$GRID_HOME/bin:$PATH
    setasmgidwrap -o /u01/app/oracle/product/12.1.0.2/dbhome_1/bin/oracle
  4. Prepare the on-premises database tier by performing the following:
    1. Identify the version of the OJVM patch on the OCI VM DB 12.1.0.2 Oracle home.
      $ORACLE_HOME/OPatch/opatch lsinventory | grep 'Oracle JavaVM Component'
    2. If OJVM is enabled on the database, verify whether its version is earlier than the version on the OCI VM DB 12.1.0.2 Oracle home. If so, the OJVM version needs to be upgraded to the version used in the OCI VM DB 12.1.0.2 Oracle home. Refer to the OJVM readme file for instructions.

    3. On the on-premises database tier, source the Oracle environment file and execute the following SQL query to identify the DST version. This value will be needed later.
      sqlplus / as sysdba
      SQL> select * from v$timezone_file;
    4. Run the following SQL commands as the SYS user in order to avoid errors later when running datapatch on the restored database.
      SQL> call dbms_java.grant_policy_permission('JAVA_ADMIN','SYS','javax.management.MBeanServerPermission','*');
      SQL> call dbms_java.grant_policy_permission('JAVA_ADMIN','SYS','javax.management.MBeanPermission','*');
    5. (Conditional) If you have Transparent Data Encryption (TDE) enabled in the on-premises Oracle E-Business Suite database, preform the following steps to export the TDE master encryption key:
      1. Rename the auto-login keystore file cwallet.sso.
        mv cwallet.sso cwallet.sso_bkup
      2. Set ORACLE_UNQNAME.
        export ORACLE_UNQNAME=<NONCDB_DBUNIQUE_NAME>
      3. Close the auto-login software keystore.
        SQL> ADMINISTER KEY MANAGEMENT SET KEYSTORE CLOSE;
      4. Open the password-based software keystore.
        SQL> ADMINISTER KEY MANAGEMENT SET KEYSTORE OPEN IDENTIFIED BY "<WALLET_PASSWORD>";
      5. Export the master encryption key from a password-based software keystore to a file.
        SQL> ADMINISTER KEY MANAGEMENT EXPORT ENCRYPTION KEYS WITH SECRET "<NONCDB_WALLET_PASSWORD>" to '<NONCDB_WALLET_KEY_BACKUP_LOCATION>/tdepdb.exp' IDENTIFIED BY "<NONCDB_WALLET_PASSWORD>";
      6. Rename the auto-login file cwallet.sso_bkup to the original file name.
        mv cwallet.sso_bkup cwallet.sso
      7. Copy the export file tdepdb.exp to the OCI VM DB.
        cd <NONCDB_WALLET_KEY_BACKUP_LOCATION>
        $ scp -i <absolute path of private key> -o "ProxyJump opc@<Bastion VM IP address>" \
        tdepdb.exp oracle@<VM DB IP address>:/u01/app/oracle/product/12.1.0.2/dbhome_1/<CDB_WALLET_KEY_BACKUP_LOCATION>

    6. Run the following command to generate the pluggable database (PDB) description file:
      { echo <APPS_PASSWORD>; } | perl $ORACLE_HOME/appsutil/bin/txkPrePDBCreationTasks.pl -dbcontextfile=<CONTEXT_FILE> \
      -dboraclehome=$ORACLE_HOME -outdir=<OUT DIRECTORY> -skipdbshutdown=Yes -promptmsg=hide
    7. Run the following command to create a zip of the appsutil directory:
      cd $ORACLE_HOME
      zip -ry appsutil.zip appsutil
    8. Transfer the PDB description file and appsutil.zip to the OCI VM DB instance through the bastion server, as shown in this example:
      1. Copy the PDB description file to the OCI VM DB instance.
        scp -i <absolute path of private key> -o "ProxyJump opc@<Bastion VM IP address>" \
        <ORACLE_HOME>/dbs/<PDB_SID>_PDBDesc.xml oracle@<VM DB IP address>:/u01/app/oracle/product/12.1.0.2/dbhome_1/dbs
        Note: The exact method you use to copy files from the on-premises system through the bastion server to the VM DB System may vary. If you are using an older version of scp, you may not have the ProxyJump option, so you can use an alternative method such as stdio forwarding using ProxyCommand instead. Follow instructions in the product documentation for guidance.

      2. Copy appsutil.zip to the OCI VM DB instance.
        cd $ORACLE_HOME
        scp -i <absolute path of the private key> -o "ProxyJump opc@<Bastion VM IP address>" \
        appsutil.zip oracle@<VM DB IP address>:/u01/app/oracle/product/12.1.0.2/dbhome_1
      3. Create a zip file named sqlpatch_patch.zip containing all patch directories within the <ORACLE_HOME>/sqlpatch directory.
        cd $ORACLE_HOME
        zip -ry sqlpatch_patch.zip sqlpatch/
      4. Then, copy the zip file to the OCI VM DB instance.
        scp -i <absolute path of the private key> -o "ProxyJump opc@<Bastion VM IP address>" \
        sqlpatch_patch.zip oracle@<VM DB IP address>:/u01/app/oracle/product/12.1.0.2/dbhome_1
  5. Establish whether there any plug-in violations and fix them before migrating the database from the on-premises instance to the OCI VM DB instance. Log in to the OCI VM DB instance switch to the oracle user to perform the following steps:
    1. If the on-premises Oracle E-Business Suite database Oracle home has a later DST patch, you must patch the OCI VM DB Oracle home with an appropriate DST patch. If the OCI VM DB oracle home has a later DST patch, you must patch the on-premises Oracle E-Business Suite database Oracle home with an appropriate DST patch.

      For more information, see Document 563019.1Complying with Daylight Saving Time (DST) and Time Zone Rule Changes in E-Business Suite 12.x.

    2. Unzip the appsutil.zip file, as shown in the following example:
      cd /u01/app/oracle/product/12.1.0.2/dbhome_1
      unzip appsutil.zip
    3. Take a backup of the <ORACLE_HOME>/sqlpatch/lib directory and the <ORACLE_HOME>/sqlpatch/sqlpatch* files. For example:
      mkdir <ORACLE_HOME>/sqlpatch/sqlpatch_bkup
      cp -R <ORACLE_HOME>/sqlpatch/lib <ORACLE_HOME>/sqlpatch/sqlpatch_bkup
      cp <ORACLE_HOME>/sqlpatch/sqlpatch* <ORACLE_HOME>/sqlpatch/sqlpatch_bkup
    4. Unzip sqlpatch_patch.zip using the '-o' option, as shown in the following example:
      cd <ORACLE_HOME>
      unzip -o sqlpatch_patch.zip
    5. Restore the <ORACLE_HOME>/sqlpatch/lib and <ORACLE_HOME>/sqlpatch/sqlpatch* files from the backup:
      rm -rf <ORACLE_HOME>/sqlpatch/lib
      mv <ORACLE_HOME>/sqlpatch/sqlpatch_bkup/lib <ORACLE_HOME>/sqlpatch/lib
      mv <ORACLE_HOME>/sqlpatch/sqlpatch_bkup/sqlpatch* <ORACLE_HOME>/sqlpatch/
      rm -rf <ORACLE_HOME>/sqlpatch/sqlpatch_bkup
    6. Source the environment as follows:
      cd /u01/app/oracle/product/12.1.0.2/dbhome_1/appsutil
      . ./txkSetCfgCDB.env dboraclehome=/u01/app/oracle/product/12.1.0.2/dbhome_1
      export TNS_ADMIN=/u01/app/oracle/product/12.1.0.2/dbhome_1/network/admin
    7. Ensure the CDB (container database) is up and running, and then run the following command to identify any plug-in violations when plugging in the non-CDB into the CDB on the OCI VM DB instance.
      perl $ORACLE_HOME/appsutil/bin/txkChkPDBCompatability.pl -dboraclehome=$ORACLE_HOME \
      -outdir=<OUT_DIRECTORY> -cdbsid=<CDB_SID> -pdbsid=<SOURCE_SID>

      Note that in this command:

      1. CDB_SID is the DB name (SID) that was provided at the time of creation of OCI VM DB.
      2. PDB_SID is the DB name (SID) of the on-premises Oracle E-Business Suite database.

    8. Identify and fix any plug-in violations. Errors reported for components installed on the PDB but not on the CDB need to be resolved before proceeding further. In this case, the missing component will need to be installed on CDB. For example, if any plug-in violations were reported as an MGD mismatch, where MGD was reported to be installed in the PDB but not in CDB, execute the following commands on OCI VM DB:
      sqlplus / as sysdba
      SQL> @?/rdbms/admin/catmgd.sql
      Notes:
      • Errors reported for components installed on the CDB but not on the PDB can be ignored. These will be resolved when the Oracle E-Business Suite database is plugged into the CDB.
      • SQL patch errors can also be ignored at this point.
      • Review warnings regarding mismatched database parameters, and update any that are critical for your environment.
        • Ensure that the "compatible" initialization parameter is set to "12.1.0".
        • If your database parameters (for example, db_files) are set to a higher value than the previous value, ignore the warnings displayed.
        • For more information on database initialization parameter settings, see Document 396009.1Database Initialization Parameters for Oracle E-Business Suite Release 12.
      • If you made any changes to the on-premises database instance to resolve the plug-in violations, regenerate the PDB description file on the on-premises database and copy it to the OCI VM DB instance. Repeat the previous step to check for further plug-in violations.

8.3 Clone the On-Premises Database to the OCI VM DB Instance

Perform the following steps on only one of the nodes to clone the on-premises database tier to the OCI VM DB instance.

Note: These steps must be performed on only one of the nodes in an Oracle RAC configuration.
  1. Create an Oracle Cloud storage bucket to host the on-premises database backup.
    1. From the Oracle Cloud Infrastructure console, click on the menu icon at the top left to open the navigation menu.
    2. Click Storage, then under Object Storage, click Bucket.
    3. Select your compartment from the compartment list.
    4. Click Create Bucket.
    5. Provide a valid Bucket Name and click Create Bucket.

  2. Download and install the backup module on the on-premises instance using the following steps. For more details, refer to "Installing the Oracle Database Cloud Backup Module for OCI".
    1. Create the following directories:
      1. <Any Directory>/cloudbackup
      2. <Any Directory>/cloudbackup/lib - This is to store the libraries of the OCI installer.
      3. <Any Directory>/cloudbackup/wallet - This is to store the wallet files for the OCI installer.

    2. Download the Oracle Database Cloud Backup Module opc_installer.zip into a newly created cloudbackup directory. You can download the module from the following link: https://www.oracle.com/database/technologies/oracle-cloud-backup-downloads.html.

    3. Unzip the opc_installer.zip file. This contains oci_install.jar and a readme file.

    4. Change to the directory where the oci_install.jar file was unzipped and execute the following command to install the module:
      java -jar oci_install.jar \
      -host https://objectstorage.<region>.oraclecloud.com \
      -pvtKeyFile <oci_private_key> \
      -pubFingerPrint <oci_public_fingerprint> \
      -uOCID <user_ocid> -tOCID <tenancy_ocid> \
      -bucket <bucket_name> \
      -walletDir <wallet_dir_path_created_in_8 3.2.a> \
      -libDir <lib_dir_path_created_in_8.3 2.a> \
      -configfile $ORACLE_HOME/dbs/opc$ORACLE_SID.ora

      This creates an opc<SID>.ora file under the $ORACLE_HOME/dbs directory.

      Refer to the readme file of the module for more details.

  3. Ensure that the on-premises database is in ARCHIVELOG mode. If not, enable it before starting the backup:
    sqlplus / as sysdba
    SQL> archive log list;
    Database log mode   Archive Mode
    -----------------   ------------
    Automatic archival  Enabled
  4. Back up the on-premises database to the Cloud Storage bucket with the following steps.
    1. On the on-premises database, connect to the database using RMAN, as shown:
      rman target /
      Note: Store the DBID value for this connection. This will be used later during the restoration of the database on the VM DB System.

    2. Run the following RMAN commands to back up the database to the Cloud Storage bucket.
      • If the source on-premises database is non-TDE, this requires a password to be set for the database encryption.
        RMAN> SET ENCRYPTION ON IDENTIFIED BY '<ENCRYPTION PASSWORD>' ONLY;
        RMAN> CONFIGURE COMPRESSION ALGORITHM 'MEDIUM';
        RMAN> CONFIGURE DEFAULT DEVICE TYPE TO 'SBT_TAPE';
        RMAN> CONFIGURE CHANNEL DEVICE TYPE DISK FORMAT '%U';
        RMAN> CONFIGURE CHANNEL DEVICE TYPE sbt PARMS='SBT_LIBRARY=<OPC INSTALLER LIBRARY DIR>/libopc.so,SBT_PARMS=(OPC_PFILE=<Absolute path of $ORACLE_HOME>/dbs/opc<SID>.ora)';
        RMAN> CONFIGURE DEVICE TYPE sbt PARALLELISM 10 BACKUP TYPE TO COMPRESSED BACKUPSET;
        RMAN> CONFIGURE MAXSETSIZE to 20480m;
        RMAN> sql 'ALTER SYSTEM ARCHIVE LOG CURRENT';
        RMAN> BACKUP DATABASE PLUS ARCHIVELOG;
        RMAN> BACKUP CURRENT CONTROLFILE FORMAT 'ct_%d_%s_%T_dbid%I.rman' TAG <BUCKET NAME>;
        Note: The encryption password will be used during the restore of the non-TDE database.
      • If the source on-premises database is TDE enabled, run the following commands:
        RMAN> SET ENCRYPTION ON
        RMAN> CONFIGURE COMPRESSION ALGORITHM 'MEDIUM';
        RMAN> CONFIGURE DEFAULT DEVICE TYPE TO 'SBT_TAPE';
        RMAN> CONFIGURE CHANNEL DEVICE TYPE DISK FORMAT '%U';
        RMAN> CONFIGURE CHANNEL DEVICE TYPE sbt PARMS='SBT_LIBRARY=<OPC INSTALLER LIBRARY DIR>/libopc.so,SBT_PARMS=(OPC_PFILE=<Absolute path of $ORACLE_HOME>/opcop1228.ora)';
        RMAN> CONFIGURE DEVICE TYPE sbt PARALLELISM 10 BACKUP TYPE TO COMPRESSED BACKUPSET;
        RMAN> CONFIGURE MAXSETSIZE to 20480m;
        RMAN> sql 'ALTER SYSTEM ARCHIVE LOG CURRENT';
        RMAN> BACKUP DATABASE PLUS ARCHIVELOG;
        RMAN> BACKUP CURRENT CONTROLFILE FORMAT 'ct_%d_%s_%T_dbid%I.rman' TAG <BUCKET NAME>;

      Note: From the output above, note the name of the control file backup, which will be in the format ct_XXXXXXXXX_YYYYMMDD_XX.rman.
  5. Create the following directories on the VM DB System node as the root user and change the ownership to oracle:oinstall, as shown:
    mkdir /u01/app/oracle/product/cloudbackup
    chown oracle:oinstall /u01/app/oracle/product/cloudbackup
    mkdir /u01/app/oracle/product/cloudbackup/lib
    chown oracle:oinstall /u01/app/oracle/product/cloudbackup/lib
    mkdir /u01/app/oracle/product/cloudbackup/wallet
    chown oracle:oinstall /u01/app/oracle/product/cloudbackup/wallet
  6. (Conditional) Merge non-CDB keystore into the existing CDB keystore on OCI VM DB.

    If you have TDE enabled in the on-premises Oracle E-Business Suite database, preform the following steps:

    1. Copy the wallet file (ewallet.p12) from the on-premises server to the OCI VM DB.
      scp -i <absolute path of private key> -o "ProxyJump opc@<Bastion VM IP address>" \ <ONPREM_WALLET_LOCATION>/ewallet.p12 oracle@<VM DB IP address>:<WALLET_BACKUP_LOCATION>/ewallet.p12
    2. Rename the auto-login keystore file cwallet.sso on OCI VM DB.
      mv cwallet.sso cwallet.sso_bkup
    3. Set the following environment variables:
      export LD_LIBRARY_PATH=<ORACLE_HOME/lib>
      export ORACLE_UNQNAME=<CDB_DBUNIQUE_NAME>
    4. Close the auto-login software keystore.
      SQL> ADMINISTER KEY MANAGEMENT SET KEYSTORE CLOSE;
    5. Open the password-based software keystore.
      SQL> ADMINISTER KEY MANAGEMENT SET KEYSTORE OPEN IDENTIFIED BY "<CDB_WALLET_PASSWORD>";
    6. Merge non-CDB with CDB keystore on OCI VM DB.
      SQL> ADMINISTER KEY MANAGEMENT MERGE KEYSTORE '<WALLET_BACKUP_LOCATION>' IDENTIFIED BY "<NONCDB_WALLET_PASSWORD>" INTO EXISTING KEYSTORE '<CDB_WALLET_LOCATION>' IDENTIFIED BY "<CDB_WALLET_PASSWORD>" WITH BACKUP;
    7. Re-enable the auto-login software keystore.
      SQL> ADMINISTER KEY MANAGEMENT CREATE AUTO_LOGIN KEYSTORE FROM KEYSTORE '<CDB_WALLET_LOCATION>' identified by "<CDB_WALLET_PASSWORD>";
  7. Restore the database from the backup and configure the VM DB System instance.

    Note: The pluggable database name should have the same name as that of source database SID.

    1. Create the following directories as the oracle user:
      1. $ORACLE_HOME/<PDB_SID>
      2. $ORACLE_HOME/<PDB_SID>/adump

    2. Log in as the opc user, then switch to the grid user, and source the grid home. Then using the asmcmd command, create the following directories:
      sudo -i -u grid
      asmcmd

      ASMCMD> mkdir +DATA/<CDB_SID>
      ASMCMD> mkdir +DATA/<CDB_SID>/<PDB_SID>
      ASMCMD> mkdir +RECO/<CDB_SID>
      ASMCMD> mkdir +RECO/<CDB_SID>/<PDB_SID>
    3. Log in as the opc user, then switch to the oracle user, and create a temporary database initialization parameter file (for example: $ORACLE_HOME/dbs/init<PDB_SID>.ora) with the following parameters which will be used to restore the database:
      db_name='<PDB_SID>'
      memory_target=1G
      processes=300
      audit_file_dest='<ORACLE_HOME>/<PDB_SID>/adump'
      audit_trail ='DB'
      db_block_size=8192
      db_files=512
      db_domain=''
      db_recovery_file_dest='+RECO'
      db_recovery_file_dest_size=45G
      diagnostic_dest='<ORACLE_HOME>'
      open_cursors=300
      remote_login_passwordfile='EXCLUSIVE'
      undo_tablespace='APPS_UNDOTS1'
      sec_case_sensitive_logon=FALSE/TRUE

      # You may want to ensure that control files are created on separate physical devices
      control_files = (+DATA/<CDB_SID>/<PDB_SID>/control01.ctl, +DATA/<CDB_SID>/<PDB_SID>/control02.ctl)
      compatible ='12.1.0.2.0'
      Notes:
      • db_name must be same as the source SID.
      • db_files size must be sufficient to hold the source database files.
      • undo_tablespace must be the same as the source undo tablespace name.
      • sec_case_sensitive_logon should be set to the same value as your on-premises database for this procedure.


    4. Start up the database in NOMOUNT mode after setting the ORACLE_SID to the PDB name, as shown:
      export ORACLE_SID=<PDB_SID>
      sqlplus / as sysdba

      SQL> startup nomount pfile=$ORACLE_HOME/dbs/init<PDB_SID>.ora
    5. Install the backup module:
      1. Download the Oracle Database Cloud Backup Module opc_installer.zip into a newly created cloud backup directory. You can download the module from the following link: https://www.oracle.com/database/technologies/oracle-cloud-backup-downloads.html
      2. Unzip the opc_installer.zip file. This contains oci_install.jar and a readme file.
      3. Change to the directory where the oci_install.jar file was unzipped and run the following command to install the module:
        java -jar oci_install.jar \
        -host https://objectstorage.<region>.oraclecloud.com \
        -pvtKeyFile <oci_private_key> \
        -pubFingerPrint <oci_public_fingerprint> \
        -uOCID <user_ocid> -tOCID <tenancy_ocid> \
        -bucket <bucket_name> \
        -walletDir /u01/app/oracle/product/cloudbackup/wallet \
        -libDir /u01/app/oracle/product/cloudbackup/lib \
        -configfile $ORACLE_HOME/dbs/opc$ORACLE_SID.ora
    6. Restore the control file from the Oracle Cloud Storage bucket:
      rman target /
      RMAN> SET DECRYPTION IDENTIFIED BY '<ENCRYPTED PASSWORD>';
      RMAN> SET DBID=<DB ID from the rman connection for RMAN backup>;
      RMAN> RUN { ALLOCATE CHANNEL t1 DEVICE TYPE sbt PARMS 'SBT_LIBRARY=/u01/app/oracle/product/cloudbackup/lib/libopc.so, SBT_PARMS=(OPC_PFILE=/u01/app/oracle/product/12.1.0.2/dbhome_1/dbs/opc<PDB_SID>.ora)';
      RESTORE CONTROLFILE FROM '<CONTROL FILE BACKUP PIECE>'; }

      For example, the control file will be of the type ct_XXXXXXXXX_YYYYMMDD_XX.rman.

    7. Mount the database.
      RMAN> alter database mount;
      RMAN> exit;
    8. Restore the database from the Oracle Cloud storage bucket.

      Note: Ensure that the user group of $ORACLE_HOME/bin/oracle is asmadmin.
      ls -ls /u01/app/oracle/product/12.1.0.2/dbhome_1/bin/oracle
      xxxxxx -rwsr-s--x 1 oracle asmadmin 331976437 Jul 30 05:39 /u01/app/oracle/product/12.1.0.2/dbhome_1/bin/oracle
      Otherwise, run the following command as the root user on both nodes.
      /u01/app/12.2.0.1/grid/bin/setasmgidwrap o=/u01/app/oracle/product/12.1.0.2/dbhome_1/bin/oracle
      rman target /
      RMAN> SET DECRYPTION IDENTIFIED BY '<ENCRYPTED PASSWORD>';
      RMAN> RUN { SET NEWNAME FOR DATABASE TO '+DATA/<CDB_SID>/<PDB_SID>/%U';
      ALLOCATE CHANNEL t1 DEVICE TYPE sbt PARMS 'SBT_LIBRARY=/u01/app/oracle/product/cloudbackup/lib/libopc.so, SBT_PARMS=(OPC_PFILE=/u01/app/oracle/product/12.1.0.2/dbhome_1/dbs/opc<PDB_SID>.ora)';
      ALLOCATE CHANNEL t2 DEVICE TYPE sbt PARMS 'SBT_LIBRARY=/u01/app/oracle/product/cloudbackup/lib/libopc.so, SBT_PARMS=(OPC_PFILE=/u01/app/oracle/product/12.1.0.2/dbhome_1/dbs/opc<PDB_SID>.ora)';
      ALLOCATE CHANNEL t3 DEVICE TYPE sbt PARMS 'SBT_LIBRARY=/u01/app/oracle/product/cloudbackup/lib/libopc.so, SBT_PARMS=(OPC_PFILE=/u01/app/oracle/product/12.1.0.2/dbhome_1/dbs/opc<PDB_SID>.ora)';
      RESTORE DATABASE;
      switch datafile all;
      switch tempfile all;
      SQL "ALTER DATABASE RENAME FILE ''<data directory of on-premises database>/log1.dbf'' TO ''+RECO/<CDB_NAME>/<PDB_NAME>/log1.dbf''";
      SQL "ALTER DATABASE RENAME FILE ''<data directory of on-premises database>/log2.dbf'' TO ''+RECO/<CDB_NAME>/<PDB_NAME>/log2.dbf''";
      SQL "ALTER DATABASE RENAME FILE ''<data directory of on-premises database>/log3.dbf'' TO ''+RECO/<CDB_NAME>/<PDB_NAME>/log3.dbf''"; }
      Note: This log rename command is just an example. Rename all of the logs that are listed in the output of the SQL query: 'select member from v$logfile'. The actual log file names and paths should also be derived from the same output.

    9. Recover the database:
      RMAN> SET DECRYPTION IDENTIFIED BY '<ENCRYPTED PASSWORD>';
      RMAN> RUN { ALLOCATE CHANNEL t1 DEVICE TYPE sbt PARMS 'SBT_LIBRARY=/u01/app/oracle/product/cloudbackup/lib/libopc.so, SBT_PARMS=(OPC_PFILE=/u01/app/oracle/product/12.1.0.2/dbhome_1/dbs/opc<PDB_SID>.ora)';
      recover database;}

      RMAN> exit;
      Note: Ignore the following archive log errors:
      • RMAN-03002: failure of recover command at 08/18/2018 15:35:42
      • RMAN-06054: media recovery requesting unknown archived log for thread 1 with sequence 12 and starting SCN of xxxxxxxxxxxxx.


    10. Run the following commands to first disable block change tracking (if enabled), and then enable block change tracking. This will avoid errors later when opening the database.
      SQL> alter database disable block change tracking;
      SQL> alter database enable block change tracking using file '+DATA'
      ;
      Note: When disabling block change tracking, ignore the following message as it is for informational purposes only: "ORA-19759: block change tracking is not enabled".

    11. Open the database with the resetlogs option.
      SQL> alter database open resetlogs;
    12. Run the adgrants.sql script.
      cd $ORACLE_HOME/appsutil/admin
      sqlplus "/ as sysdba" @adgrants.sql <APPS Schema Name>
    13. Open the database (PDB_SID) in upgrade mode and run datapatch:
      sqlplus / as sysdba
      SQL> shutdown immediate;
      SQL> startup upgrade;
      $ORACLE_HOME/OPatch/datapatch -verbose
    14. Open the database (PDB_SID) in read-only mode:
      SQL> shutdown immediate;
      SQL> startup mount;
      SQL> alter database open read only;
    15. Execute the following command to generate the PDB description file:
      sqlplus / as sysdba
      SQL> BEGIN DBMS_PDB.DESCRIBE(pdb_descr_file => '/u01/app/oracle/product/12.1.0.2/dbhome_1/dbs/<PDB_SID>_PDBDesc.xml');
      end;

      /
    16. Source the environment, as shown:
      cd /u01/app/oracle/product/12.1.0.2/dbhome_1/appsutil
      . ./txkSetCfgCDB.env dboraclehome=/u01/app/oracle/product/12.1.0.2/dbhome_1
      export ORACLE_SID=<CDB_SID>
      export TNS_ADMIN=/u01/app/oracle/product/12.1.0.2/dbhome_1/network/admin

    17. Run the following command to identify whether there are any plug-in violations when plugging in the non-CDB database into the CDB on the VM DB System instance:
      perl $ORACLE_HOME/appsutil/bin/txkChkPDBCompatability.pl \
      -dboraclehome=$ORACLE_HOME -outdir=<OUT_DIRECTORY> \
      -cdbsid=<CDB_SID> -pdbsid=<PDB_SID>
      Notes:
      • CDB_SID is the DB Name (SID) that was provided at the time of creation of the VM DB System.
      • PDB_SID is the DB Name (SID) of the on-premises Oracle E-Business Suite database.


    18. Identify and fix any plug-in violations, keeping the following guidelines in mind:
      1. Errors reported for components installed on the PDB but not on the CDB need to be resolved before proceeding further. In this case, the missing component will need to be installed on the CDB.

      2. Errors reported for components installed on the CDB but not on the PDB can be ignored. These will be resolved when the Oracle E-Business Suite database is plugged into the CDB.

      3. SQL patch errors can be ignored at this point.

      4. Review warnings regarding mismatched database parameters, and update any that are critical for your environment. For more information, refer to Document 396009.1Database Initialization Parameters for Oracle E-Business Suite Release 12.

    19. Proceed with the migration of the on-premises database into the VM DB System. Source the environment and execute the following script to plug the Oracle E-Business Suite database into the CDB:
      1. For a 2-Node VM DB System, shut down the second instance to avoid errors while disabling the archive log mode during the PDB creation.
        srvctl stop instance -d <CDB_UNIQNAME> -i <CDB_INSTANCE_2>
      2. Source the environment as follows:
        $ cd /u01/app/oracle/product/12.1.0.2/dbhome_1/appsutil
        $ . ./txkSetCfgCDB.env dboraclehome=/u01/app/oracle/product/12.1.0.2/dbhome_1
        $ export ORACLE_SID=<CDB_SID>
        $ export TNS_ADMIN=/u01/app/oracle/product/12.1.0.2/dbhome_1/network/admin
      3. Run the txkCreatePDB.pl script to plug in the Oracle E-Business Suite database into the container database. Additional parameters are required if the non-CDB database is TDE enabled.

        Note: Depending on the size of the database, this script may take several hours to run.

        • If the source on-premises database is non-TDE, run txkCreatePDB.pl as follows:
          perl $ORACLE_HOME/appsutil/bin/txkCreatePDB.pl \
          -dboraclehome=$ORACLE_HOME -outdir=$ORACLE_HOME/appsutil/log \
          -cdbsid=<CDB_SID> -pdbsid=<PDB_SID> \
          -noncdbdatadir=<DATA DIRECTORY LOCATION ON TARGET> \
          -pdbdatadir=<DATA_DIRECTORY_LOCATION_ON_TARGET>
        • If the source on-premises database is TDE enabled, run the following commands:

          Note: Ensure the container database is running on a password wallet. Auto-login wallet should be disabled before running the txkCreatePDB.pl script.
          perl $ORACLE_HOME/appsutil/bin/txkCreatePDB.pl \
          -dboraclehome=$ORACLE_HOME -outdir=$ORACLE_HOME/appsutil/log \
          -cdbsid=<CDB_SID> -pdbsid=<PDB_SID> \
          -noncdbdatadir=<DATA DIRECTORY LOCATION ON TARGET> \
          -pdbdatadir=<DATA_DIRECTORY_LOCATION_ON_TARGET> \
          -istdeenabled=yes \
          -secretkeyfile=<CDB_WALLET_KEY_BACKUP_LOCATION>/tdepdb.exp \
          -keystoreloc=<CDB_WALLET_LOCATION> \
          -promptsecretkey=yes \
          -backupwallet=yes
    20. If the txkCreatePDB.pl script fails when it executes the noncdb_to_pdb.sql script, perform the following steps and then re-run txkCreatePDB.pl:
      1. Unplug the pluggable database by running the following commands:
        sqlplus / as sysdba
        SQL> alter pluggable database <PDB_SID> close immediate;
        SQL> alter pluggable database <PDB_SID> unplug into '/u01/app/oracle/product/12.1.0.2/dbhome_1/dbs/unplug<PDB_SID>.xml';
        SQL> drop pluggable database <PDB_SID>;
      2. Repeat steps starting with "d. Start up the database in NOMOUNT mode after setting the ORACLE_SID to the PDB name" to "s. Proceed with the migration of the on-premises database into the VM DB System".

    21. During the PDB creation, the parameters log_mode and flashback_on are disabled and set to the values NOARCHIVELOG and NO, respectively. Review these parameters and change them if desired.
      sqlplus / as sysdba
      SQL> select log_mode from v$database;
      SQL> select flashback_on from v$database;
    22. If any plug-in violations were reported for any database patches installed in the PDB but not in the CDB, apply those patches to the CDB. Refer to the respective patch README for specific steps.

    23. If any plug-in violations were reported for database patches that are installed in the CDB but not in the PDB, run datapatch as follows.
      1. For a 2-Node VM DB System:
        sqlplus / as sysdba
        SQL> alter system set cluster_database=FALSE scope=spfile;
        SQL> shutdown immediate;
        SQL> startup upgrade;
        SQL> alter pluggable database all open upgrade;
        SQL> exit
        $ORACLE_HOME/OPatch/datapatch -verbose
      2. For a 1-Node VM DB System:
        export DB_UNIQUE_NAME=<DB_UNIQUE_NAME>
        sqlplus / as sysdba
        SQL> shutdown immediate
        SQL> startup upgrade
        SQL> alter pluggable database all open upgrade;
        SQL> exit
        $ORACLE_HOME/OPatch/datapatch -verbose

      If datapatch reports an ORA-01435 error when applying any database patch, run the following commands to re-execute datapatch and ignore the error:

      cd <PATCH_DIRECTORY>
      ./datapatch -apply <PATCH_NUMBER> -verbose -ignorable_errors=ORA-01435,Warning
    24. Log in as the SYSDBA user and run the utlrp.sql script:
      sqlplus / as sysdba
      SQL> @?/rdbms/admin/utlrp.sql
    25. Stop and restart the CDB and PDB databases and verify that the Oracle E-Business Suite pluggable database is opened in non-restricted mode.
      1. For a 2-Node VM DB System:
        sqlplus / as sysdba
        SQL> alter system set cluster_database=TRUE scope=spfile;
        SQL> shutdown immediate;
        SQL> startup;
        SQL> alter pluggable database all open;
      2. For a 1-Node VM DB System:
        sqlplus / as sysdba
        SQL> shutdown immediate
        SQL> startup
        SQL> alter pluggable database all open;
    26. Run the following to save the state of the PDB once it is open:
      SQL> alter pluggable database <PDB_SID> save state;
      This preserves the PDB open mode when you restart the CDB. You can find the current saved state by querying DBA_PDB_SAVED_STATES, as shown:
      sqlplus / as sysdba
      SQL> select con_name, state from dba_pdb_saved_states;

8.4 Update the Grid Listener and Enable AutoConfig

Perform the following steps on both nodes to update the grid listener and enable AutoConfig.

Note: These steps must be performed on both nodes in an Oracle RAC configuration.
  1. Log in as the opc user and switch to the grid user. Then, add the use_sid_as_service parameter to the grid listener.ora file using a text editor such as vi.

    If you are using a 1-Node VM DB system, perform this step on the single node database.

    If you are using a 2-Node VM DB System, perform this step on each Oracle RAC node.
    sudo -i -u grid
    vi $ORACLE_HOME/network/admin/listener.ora

    [ Add the following ]
    USE_SID_AS_SERVICE_LISTENER=ON
  2. Now, restart the listener.
    lsnrctl stop
    lsnrctl start
  3. Complete the configuration of the Oracle E-Business Suite database. Execute the following commands for context file generation and execution of AutoConfig.
    1. On the 1-Node system, execute the following commands and provide the apps and system user passwords when prompted. Note that the system password will be the one specified during the VM provisioning process:
      cd /u01/app/oracle/product/12.1.0.2/dbhome_1/appsutil
      source txkSetCfgCDB.env dboraclehome=/u01/app/oracle/product/12.1.0.2/dbhome_1
      export ORACLE_SID=<CDB_SID>
      export TNS_ADMIN=$ORACLE_HOME/network/admin
      perl $ORACLE_HOME/appsutil/bin/txkPostPDBCreationTasks.pl \
      -dboraclehome=<ORACLE_HOME> -outdir=<ORACLE_HOME>/appsutil/log \
      -cdbsid=<CDB SID> -pdbsid=<PDB SID> \
      -appsuser=<APPS schema user name> -dbport=<Database port> -servicetype=dbsystem
    2. On a 2-Node system:
      1. Execute the following commands and provide the apps and system user passwords when prompted. Note that the system password will be the one specified during the VM provisioning process:
        cd /u01/app/oracle/product/12.1.0.2/dbhome_1/appsutil
        source txkSetCfgCDB.env dboraclehome=/u01/app/oracle/product/12.1.0.2/dbhome_1
        export ORACLE_SID=<CDB_SID>
        export TNS_ADMIN=$ORACLE_HOME/network/admin
        perl $ORACLE_HOME/appsutil/bin/txkPostPDBCreationTasks.pl \
        -dboraclehome=<ORACLE_HOME> -outdir=<ORACLE_HOME>/appsutil/log \
        -cdbname=<CDB Name> -cdbsid=<CDB SID> -pdbsid=<PDB SID> \
        -appsuser=<APPS schema user name> -dbport=<Database port> \
        -servicetype=dbsystem -israc=yes \
        -virtualhostname=<Virtual hostname for RAC> \
        -scanhostname=<Scan hostname for RAC> -scanport=<Scan port for RAC>
      2. On the second node of your 2-Node system, copy the /appsutil directory from first node to second node:
        scp -r /u01/app/oracle/product/12.1.0.2/dbhome_1/appsutil <Node2>:/u01/app/oracle/product/12.1.0.2/dbhome_1/
      3. Execute the following command on the second node and provide the apps user password when prompted:
        cd /u01/app/oracle/product/12.1.0.2/dbhome_1/appsutil
        source txkSetCfgCDB.env dboraclehome=/u01/app/oracle/product/12.1.0.2/dbhome_1
        export ORACLE_SID=<CDB_SID>
        export TNS_ADMIN=$ORACLE_HOME/network/admin
        perl $ORACLE_HOME/appsutil/bin/ txkCfgPDBRACNode.pl \
        -dboraclehome=<ORACLE_HOME> -outdir=<RDBMS_ORACLE_HOME>/appsutil/log \
        -cdbname=<CDB_Name> -cdbsid=<CDB_SID instance 2> -pdbsid=<PDB_SID> \
        -appsuser=<APPS_USER> -virtualhostname=<Virtual Host Name> \
        -scanhostname=<Scan Name> -scanport=<Scan Port> -israc=yes -dbport=1521

8.5 Update the PDB Master Encryption Key

  1. Review the STATUS column in the v$encryption_wallet view for the CDB and PDB, as shown in the following example:
    SQL> alter session set container=<PDB>;
    SQL> select WRL_TYPE, WRL_PARAMETER,STATUS, WALLET_TYPE from v$encryption_wallet;

    Note: In a multitenant environment, each pluggable database (PDB) has its own master encryption key, which is stored in a single keystore used by all containers. After you create or plug in a new PDB, you must create and activate a master encryption key for it. If you do not do so, the STATUS column in the v$encryption_wallet view shows the value OPEN_NO_MASTER_KEY.

  2. Confirm that the PDB is in READ WRITE open mode and is not restricted, as shown in the following example:
    SQL> show pdbs;
    The PDB cannot be open in restricted mode (the RESTRICTED column must show NO).

  3. Run the following DBCLI commands to change the status to OPEN:
    sudo -i
    dbcli list-databases
    dbcli update-tdekey -i <database_ID> -n <PDB_name> -p

    Note: The update-tdekey command shown will prompt you for the admin password.

  4. Confirm that the status of the wallet has changed from OPEN_NO_MASTER_KEY to OPEN by querying the v$encryption_wallet view as shown in step 3.

8.6  Configure Transparent Data Encryption for the Pluggable Database (Conditional)

Note: Skip this section if the source database is encrypted.

Refer to Appendix B to perform Transparent Data Encryption (TDE) tablespace offline encryption for the pluggable database (PDB).

8.7 Run the Oracle E-Business Suite Technology Codelevel Checker (ETCC)

The EBS Technology Codelevel Checker (ETCC) utility is a set of scripts you can run to help ensure that you have the necessary database and application tier bug fixes installed on your Oracle E-Business Suite Release 12.2 system. ETCC maps missing bug fixes to the default corresponding patches and displays them in a patch recommendation summary.

The checkDBpatch.sh script is the Oracle Database EBS Technology Codelevel Checker (DB-ETCC), which when used with the option cloud=y determines if all the needed bug fixes exist in the specified database ORACLE_HOME. You should run this utility to ensure that all required database bugs fixes have been installed.

Note: Always use the latest version of ETCC, which can be downloaded via Patch 17537119. In the case of OCI VM DB RAC, ETCC should be run on both nodes.

Section 9: Create an Instance to Host the Oracle E-Business Suite Application Tier

Perform the following steps to create an OCI Compute instance and prepare the instance to host the Oracle E-Business Suite application tier:

  1. You can create the Compute instance for the target application tier in one of the following two ways:

    • Option 1: By Using a Standard Oracle Linux Image
      1. Log in to Oracle Cloud My Services using your Oracle Cloud Infrastructure credentials.
      2. In the navigation menu under Core Infrastructure, go to Compute, and click Instances.
      3. Click Create Instance and provide the following details on the Create Compute Instance page:
        1. Name: Provide a suitable name in the form <env name>app01. In the case of a multi-node non-shared file system, instance names should be in the form <env name>app<seqno> where <seqno> is 01, 02, 03, etc.
        2. Create in Compartment: Select the compartment in which to install the instance in the drop-down list.
        3. Image or Operating System: Click Change Image. In the Browse All Images window, under the Platform Images tab, select "Oracle Linux 7.8" as an image operating system. Click Select Image.
        4. Availability Domain: Select the availability domain where you have chosen to place your application tier.
        5. Shape: Select a valid shape. To do so, click Change Shape. Then select your desired shape (for example, select Intel Skylake and then "VM.Standard 2.2").
        6. Configure Networking: Select the VCN and application tier subnet you established previously (for example, "ebs-vcn" and "apps-subnet-ad1").
        7. SSH Keys: Provide values for either the VM public key or the file that contains the VM public key contents.
      4. Click Create.

    For more details refer to "Creating an Instance."

    • Option 2: By Using an Oracle WebLogic Suite UCM Image

      Follow the instructions in Document 2766854.1Using the Oracle WebLogic Suite UCM Image for the Oracle E-Business Suite Application Tier.

  2. Now, create a block volume by performing the following steps:
    1. Log in to Oracle Cloud My Services using your Oracle Cloud Infrastructure credentials.
    2. In the navigation menu under Core Infrastructure, go to Block Storage, and click Block Volumes.
    3. Click Create Block Volume and provide the following details:
      1. Name: Provide a valid name.
      2. Create in Compartment: Select the compartment that you established earlier (such as "ebs-compartment") from the drop-down list.
      3. Availability Domain: Select the same availability domain as was selected for the Compute instance created previously from the drop-down list.
      4. Volume Size and Performance: Select the Custom radio button and enter a suitable size to hold the application tier instance and select "Balanced" for the Volume Performance.
    4. Click Create Block Volume.

    For more information, refer to "Creating a Volume."

  3. Attach the block volume to the instance by performing the following steps:
    1. Log in to Oracle Cloud My Services using your Oracle Cloud Infrastructure credentials.
    2. In the navigation menu under Core Infrastructure, go to Compute, and click Instances.
    3. In the Instances list, click the instance that you want to attach a volume to.
    4. In the Resources section, click on the "Attached Block Volumes" link.
    5. Click Attach Block Volume and provide the following details:
      1. Volume Attachment Type: Select ISCSI.
      2. Block Volume: In the Block Volume drop-down list, select the block volume previously created.
      3. Device Path: Select the appropriate device path from the drop-down list.
      4. Access: Select READ/WRITE (the default option for volume attachments).
      5. Click Attach. Wait for the status to change to "Attached" before proceeding.
    6. Click on the options icon for the attached block volume and select ISCSI Commands & Information.
    7. Copy the Attach Commands into a text file for later use.

    For more information, refer to "Attaching a Volume."

  4. In this step, you will attach the block volume and install prerequisites for Oracle E-Business Suite on the Compute instance.   
    1. Using your private key, log in to the application tier instance and then switch to the root user:
      ssh -i <absolute path of the private key> opc@<IPaddress>
      sudo -i
    2. Execute the previously recorded Attach Commands in step 3g.

    3. Execute the following steps as the root user to mount the newly added volume:
      fdisk -l [Note the new volume /dev/sdb]
      fdisk /dev/sdb
    4. When prompted, provide the following values for the command parameters:
      Command (m for help): n
      Partition type:
        p primary (0 primary, 0 extended, 4 free)
        e extended
      Select (default p): p
      Partition number (1-4, default 1): 1
      First sector (2048-419430399, default 2048): 2048
      Last sector, +sectors or +size{K,M,G} (2048-419430399, default 419430399):
      Using default value 419430399 Partition 1 of type Linux and of size 200 GiB is set

      Command (m for help): w
      mkfs -t ext4 /dev/sdb1
      mkdir /u01
      mount /dev/sdb1 /u01
      sudo yum -y install oracle-ebs-server-R12-preinstall.x86_64
      chown -R oracle:oinstall /u01
    5. Edit /etc/fstab and add an entry for the block volume. Ensure that you include the _netdevnoatime, and nofail options. For example:
      lsblk -o NAME,UUID
      NAME UUID
      sda
      sda1 xxxx-xxxx
      sda2 xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
      sda3 xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
      sdb
      sdb1 xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
      cat /etc/fstab
      UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx /u01 ext4 defaults,noatime,_netdev,nofail 0 2
  5. As the root user:
    1. Edit /etc/hosts and add entries for the VM DB System node (in the case of a 1-Node VM DB System) or nodes (in the case of a 2-Node VM DB System), using the private IPs.
    2. Edit /etc/oci-hostname.conf and change the value of PRESERVE_HOSTINFO from "0" to "2" to preserve the user configured hostname across instance reboots.

Section 10: Configure the Oracle E-Business Suite Application Tier on the Previously-Created Instance

10.1 Validate the Application Tier Time Zone

Perform the following steps to validate the time zone settings on the application tier:

  1. Run the following command on the application tier:
    /usr/sbin/hwclock --localtime
    Note: The time zone value returned from this command for the application tier must match the time zone settings on the Oracle OCI VM DB, as described in Section 7.

  2. Ensure the TZ environment variable is not set in the current shell using unset TZ on the command line and remove this entry from any environment files sourced to set the environment when starting the application services. You must use the OS time zone settings (not TZ) for the application tier time zone.

10.2 Configure the Application Tier

Perform the following steps to clone the on-premises application tier to the instance you created earlier to hold the application tier:

  1. As the root user, perform the following steps:
    1. Create /etc/oraInst.loc with the following contents:
      inst_group=oinstall
      inventory_loc=/u01/install/APPS/oraInventory
    2. Ensure the /etc/ssh/sshd_config is altered to allow the oracle user to log in to the instance using a private key. For example, sshd_config must contain the following line:
      AllowUsers oracle opc
    3. Uncomment and edit the following parameters in /etc/ssh/sshd_config to allow multi-threaded transfer of files and prevent client timeout:
      # To allow multi-threaded transfer of files.
      MaxStartups 100
      # To prevent client timeout.
      ClientAliveInterval 100
      ClientAliveCountMax 99999
    4. Restart the SSHD services as follows:
      systemctl restart sshd
  2. As the root user, create the /u01/install/APPS/oraInventory directory and ensure the oracle user has permission to write to it.

  3. Prepare the on-premises application tier:
    1. Source the environment file of the run edition file system.

      You can use the following command to confirm that the environment variable FILE_EDITION points to the run edition file system:
      echo $FILE_EDITION
      It should return the value run.

    2. Execute the following commands:
      cd $ADMIN_SCRIPTS_HOME
      perl adpreclone.pl appsTier
    3. Tar and transfer the Oracle E-Business Suite application tier run file system from on-premises to the application tier instance.
      1. Tar the Oracle E-Business Suite application tier run file system from on-premises to the application tier instance.
        tar -czvsf EBSapps.tgz EBSapps
      2. Ensure that the oracle user has permission to write to /u01/install/APPS directory and copy the tar file to the application tier node:
        scp -i <absolute path of the private key> -o "ProxyJump opc@<Bastion VM IP address>" \
        EBSapps.tgz oracle@<application tier IP address>:/u01/install/APPS
  4. Log in and switch to the oracle user on the application tier instance. Perform the following steps:
    1. Identify the run file system name (for example, fs1) on the source and create the same on the cloud application tier instance. For example:
      mkdir -p /u01/install/APPS/fs1
      mv /u01/install/APPS/EBSapps.tgz /u01/install/APPS/fs1

    2. Untar the archives transferred to create the relevant directory structure, as shown in the following commands:
      cd /u01/install/APPS/fs1
      tar -xzvf EBSapps.tgz
    3. Configure the application tier instance. For a single node database, ensure that port pool is selected as "0" as the database tier by default is configured to use 1521.

      Note: Accept the default value 'n' when prompted for whether or not to add a node.
      cd /u01/install/APPS/fs1/EBSapps/comn/clone/bin
      perl adcfgclone.pl appsTier dualfs

      Note that the database SID <PDB_SID> should be entered in UPPER case, as the database name is always stored in upper case in a multitenant database setup.

  5. Edit the application context file and update the value of the "s_fnd_cache_port_range" context variable to "36501-36550".
    <fndcache_port_range oa_var="s_fnd_cache_port_range">36501-36550</fndcache_port_range>
  6. Run AutoConfig.

  7. For multitenancy support, apply Patch 25452805:R12.FND.C as well as the following patch or patches dependent on your release of Oracle E-Business Suite:
    1. Oracle E-Business Suite Release 12.2.9 - Patch 24300571:12.2.0
    2. Oracle E-Business Suite Release 12.2.8 - Patch 24300571:12.2.0
    3. Oracle E-Business Suite Release 12.2.7 - Patch 24300571:12.2.0
    4. Oracle E-Business Suite Release 12.2.6 - Patch 24300571:12.2.0
    5. Oracle E-Business Suite Release 12.2.5 - Patch 24300571:12.2.0 and Patch 23560508:R12.MSC.C
    6. Oracle E-Business Suite Release 12.2.4 - Patch 24300571:12.2.0 and Patch 23588491:R12.MSC.C
    7. Oracle E-Business Suite Release 12.2.3 - Patch 24300571:12.2.0 and Patch 23588492:R12.MSC.C

Section 11: Deploy and Configure an Oracle Cloud Infrastructure Load Balancer (Optional)

If you are deploying multiple Oracle E-Business Suite application tier nodes, a load balancer is required. For single-node application tier node deployments, a load balancer provides a layer of security as incoming traffic can be directed to it instead of directly to an application tier node. We also recommend off-loading TLS encryption to the load balancer.

In this section, you will deploy Load Balancer as a Service (LBaaS). You will also perform the required configuration for the web entry point and Transport Layer Security (TLS).

11.1 Deploy a Load Balancer
11.2 Configure the Web Entry Point and Transport Layer Security (TLS)

11.1 Deploy a Load Balancer

Note: Only HTTP or HTTPS protocols have been certified for Oracle E-Business Suite at this time. Oracle E-Business Suite generates its own cookies and expects session cookie persistence through HTTP/HTTPS.

To begin the process of deploying a load balancer in your environment, navigate to the Create Load Balancer page by following these steps:

  1. Open the Oracle Cloud Infrastructure Console. Click Networking, and then Load Balancers.

  2. Choose a compartment that you established previously (such as "ebs-compartment"), and then click Create Load Balancer.

The workflow to create a load balancer includes four waypoints to configure your load balancer.

11.1.1 Add Details
11.1.2 Configure Backend Sets
11.1.3 Create Virtual Host Name (Optional)
11.1.4 Configure Listener

Further information can be found in "Creating Load Balancers" under "Load Balancer Management" of the Oracle Cloud Infrastructure Documentation.

11.1.1 Add Details

Add details to specify the attributes of the load balancer. Under Load Balancer Information, specify the following:

  1. LOAD BALANCER NAME: (Required) Specify a friendly name for the load balancer. It does not have to be unique, but it cannot be changed in the console. Instead, you can change it using the API. Avoid entering confidential information.

  2. CHOOSE VISIBILITY TYPE: (Required) Specify whether your load balancer is public or private.
    1. PUBLIC LOAD BALANCER: Choose this option to create a public load balancer. You can use the assigned public IP address as a front end for incoming traffic and to balance that traffic across all of the backend servers.
    2. PRIVATE LOAD BALANCER: Choose this option to create a private load balancer. You can use the assigned private IP address as a front end for incoming internal VCN traffic and to balance that traffic across all the backend servers.

  3. CHOOSE THE MAXIMUM TOTAL BANDWIDTH: (Required) Specify a shape to provision the maximum total bandwidth (ingress and egress) for your load balancer. Available shapes include: Small (100 Mbps), Medium (400 Mbps), and Large (8000 Mbps).

    If you are not an Always Free user, you can adjust the size of the bandwidth to one of the other predefined sizes. For more information, refer to Changing the Load Balancer Bandwidth.

  4. CHOOSE NETWORKING. If the current compartment contains at least one VCN, the Console provides a drop-down list of VCNs for you to choose from.
    1. VIRTUAL CLOUD NETWORK in <COMPARTMENT NAME>: (Required) Specify a VCN for the load balancer.
    2. SUBNET in <COMPARTMENT NAME>: (Required) Select an available subnet. For a public load balancer, it must be a public subnet.

      By default, the Console shows a list of subnets in the compartment you're currently working in. Click the Change Compartment link to select a subnet from a different compartment.

      Note: If your region supports more than one availability domain, you must select the second subnet created in 3.5 Create Subnets.

  5. Click Next.

11.1.2 Configure Backend Sets

A load balancer distributes traffic to backend servers within a backend set. A backend set is a logical entity defined by a load balancing policy, a health check policy, and a list of backend servers (Compute instances). For more information, refer to the instructions in "To create a backend set" in the Oracle Cloud Infrastructure Documentation.

  1. Under the Resources menu, select Backend Sets. Then click Create Backend Set.

  2. Enter a name for the backend set. This cannot be modified.

  3. Under the TRAFFIC DISTRIBUTION POLICY section, enter the following information:
    1. Choose "Weighted Round Robin" for the load balancer policy for the backend set.
      For more information on these policies, see "How Load Balancing Policies Work" in the Oracle Cloud Infrastructure Documentation.
    2. Ensure that the "Use SSL" check box is not selected. SSL will be configured optionally in 11.2.

  4. Under the HEALTH CHECK POLICY section, specify the test parameters that confirm the health of your backend servers. You must configure your health check protocol to match your application or service; only HTTP protocol is certified for Oracle E-Business Suite, not TCP. All the entries marked as "HTTP only" are then required with the chosen HTTP protocol.
    1. PROTOCOL: (Required) Specify the protocol to use for health check queries. Select HTTP (as required by Oracle E-Business Suite).
    2. PORT: (Required) Specify the backend server port against which to run the health check. You can enter the value of "0" (which equates to 'zero' without the double quotes) to have the health check use the backend server's traffic port.
    3. URL PATH (URI): (Required) Specify a URL endpoint against which to run the health check. HTTP only.
    4. INTERVAL IN MS: (Required) Specify how frequently to run the health check, in milliseconds. The default is 10000 (10 seconds). Note that the interval must be greater than the timeout.
    5. TIMEOUT IN MS: (Required) Specify the maximum time in milliseconds to wait for a reply to a health check. A health check is successful only if a reply returns within this timeout period. The default is 3000 (3 seconds). Note that the timeout must be less than the interval.
    6. NUMBER OF RETRIES: (Required) Specify the number of retries to attempt before a backend server is considered "unhealthy". This number also applies when recovering a server to a "healthy" state. The default is 3.
    7. STATUS CODE: (Required) Specify the status code a healthy backend server must return. For Oracle E-Business Suite, set this to 200. HTTP only.
    8. RESPONSE BODY REGEX: (Optional) Provide a regular expression for parsing the response body from the backend server. For Oracle E-Business Suite, set this to ".*" (which equates to 'dot' followed by 'asterisk' without the double quotes).

    Note: To allow for the LBaaS Health Check Monitor to access the Oracle E-Business Suite OHS port, the port must be marked as open for the iptables or firewall. If not, the health check monitor will not function and/or attempts to access the login page will fail with a '502 Bad Gateway' error message. Configuration of the iptables/firewall requires root access of the applications node. An example of what is set on Linux for the firewall would be as follows:
    firewall-cmd --zone=public --add-port 8000/tcp
    firewall-cmd --zone=public --add-port 8000/tcp --permanent
    In this example, the OHS port here is port 8000.

  5. Click Show Advanced Options and enter a Backend Set Name.

  6. Within the SESSION PERSISTENCE tab:
    1. Select the "Enable Application Cookie Persistence" radio button. By default, session persistence is disabled.
    2. Set the cookie name to "*" (which equates to only an asterisk without the quotes).
    3. Ensure that the "Disable Fallback" check box is not selected.

    For Oracle E-Business Suite integration with LBaaS, the recommendation is to use Application Cookie Persistence with SSL termination either at the load balancer or TLS end-to-end to web servers.

  7. Click Next Step.

11.1.3 Create Virtual Host Name (optional)

An advantage of virtual host names include multiple host names, backed by DNS entries, can point to the same load balancer IP address.

To ensure a web entry point for Oracle E-Business Suite made of a single application name, you must create a virtual host name for the listener.

For example, instead of clients accessing http://app_lbr.example.com:8000, you could have http://ebs.example.com.

Note: The virtual host names feature supports HTTP and HTTPS listeners only, but does not support TCP listeners; Oracle E-Business Suite has not been certified for TCP protocol.
  1. In the Resources menu, click Hostnames. Then click Create Hostname.

  2. Enter the following information:
    1. NAME: (Required) Specify a virtual host name friendly name
    2. HOSTNAME: (Required) Specify virtual host name for Oracle E-Business Suite (for example, ebs.example.com)

11.1.4 Configure Listener

If you would like to use SSL with the load balancer listener, you will first need to generate a digital certificate using the instructions in "Upload TLS Certificate" in the Oracle E-Business Suite Cloud Manager Guide. Doing so will allow you to then select the "Use SSL" check box, making the load balancer the SSL end-point termination.

  1. Under the Resources menu, select Listeners. Then click Create Listener.

  2. Enter the following information:
    1. NAME: (Required) Specify a friendly name for the listener. The name must be unique and cannot be changed. Avoid entering confidential information. If you do not specify a name, the load balancing service creates one for you.
    2. HOSTNAMES: (Optional) This is only needed if you defined a virtual hostname in 11.1.3 Create Virtual Host Name (optional).
    3. PROTOCOL: (Required) Select HTTP from the drop-down list to specify the protocol to use for listener ingress traffic.
    4. PORT: (Required) Defaults are 443 for HTTPS and 80 for HTTP. TCP/22 is not certified.
    5. USE SSL: (Optional) Select this check box to associate an SSL certificate bundle with the listener and select the certificate uploaded previously (See "Upload TLS Certificate" in the Oracle E-Business Suite Cloud Manager Guide for details.).
    6. BACKEND SET: (Required) Choose the Backend Set Name to service.
    7. IDLE TIMEOUT IN SECONDS: (Optional) Specify the maximum idle time in seconds. This setting applies to the time allowed between two successive receive or two successive send network input/output operations during the HTTP request-response phase.

  3. Configure the path route or rule sets (optional).

For more information regarding the Oracle Cloud Infrastructure Load Balancing service, as well as creating and configuring a load balancer, refer to "Overview of Load Balancing" and "Load Balancer Management" in the Oracle Cloud Infrastructure Documentation.

11.2 Configure the Web Entry Point and Transport Layer Security (TLS)

There are two different options for TLS, the first being TLS endpoint, and the other being the end-to-end TLS.

For the TLS endpoint, you only need to set the 'Use SSL' as part of the listener configuration, and thus only need to have already uploaded the certificate issued to the load balancer.

For the end-to-end TLS, in addition to what is required for the TLS endpoint, you also need to check the 'Use SSL' within the backend set configuration, and have already uploaded the root CA certificate matching the backend servers' certificate chain.

Now, perform the required configuration for the web entry point and TLS by following these steps found in the Oracle E-Business Suite Cloud Manager Guide:

  1. If not already done so previously, generate the digital certificates for the load balancer (and optionally for the Oracle E-Business Suite web servers). See the instructions in "Upload TLS Certificate (Conditionally Required)" in the Oracle E-Business Suite Cloud Manager Guide.

  2. Follow the steps in "Manually Enable TLS When Using Oracle HTTP Server on the Application Tier Node as the Web Entry Point" in the Oracle E-Business Suite Cloud Manager Guide. This will configure Oracle E-Business Suite to use HTTPS (HTTP over TLS).

    The web entry point needs to be set appropriately to enable access to the AppsLogin page. To complete the configuration, set the following context variables in the context file of the run file system:
    • s_webentryhost - Set the value to the LBaaS name
    • s_webentrydomain - Set the value to the LBaaS domain name
    • s_external_url - Set the value to the URL https://<LBaaS name>.<LBaaS domain name>
    • s_login_page - Set the value to the URL https://<LBaaS name>.<LBaaS domain name>/OA_HTML/AppsLogin
    • s_active_webport - Set the value to 443
    • s_webentryurlprotocol - Set this to https
    • s_enable_sslterminator - Remove the '#' to use ssl_terminator.conf
    • s_url_protocol - HTTP unencrypted traffic to Oracle E-Business Suite application tiers

  3. Optionally, you can enable TLS on the Oracle E-Business Suite application tiers by following steps 2 and 3 of
    "Enable TLS for Manually Configured Load Balancer (Conditionally Required)" in the Oracle E-Business Suite Cloud Manager Guide.

Section 12: Configure Transport Layer Security and Web Entry Point Access if Not Using a Load Balancer (Conditional)

If you have deployed your Oracle E-Business Suite Release 12.2 environment without using a load balancer, we highly recommend that you encrypt the traffic between the client and the Oracle HTTP Server. Then after the encryption setup is complete, you must configure the Oracle E-Business Suite web entry point.

For instructions to set up encryption and to configure the web entry point, see the following sections in the Oracle E-Business Suite Cloud Manager Guide:

Section 13: Configure Secure Access for WebLogic Server Admin Port

Additional configuration is required to securely access the administration port used for the Fusion Middleware Control and WebLogic Server Administration Console.

  1. Update the security list for the primary application tier node by adding a security rule that allows inbound communication on ports 7001 and 7002 from the bastion server VM. See Working with Security Lists in the Oracle Cloud Infrastructure Documentation.

    In the Oracle Cloud Infrastructure console, open the security list for the Oracle E-Business Suite application tier subnet and add a new entry under Allow rules for ingress with the following properties:
    • Source CIDR - The CIDR for the bastion server VM
    • Protocol - TCP
    • Destination Port Range - 7001-7002

  2. Create firewall rules on the primary application tier node that allow inbound communication on ports 7001 and 7002 from the subnet that contains the bastion server VM. First, log on to the Oracle Cloud Infrastructure instance that hosts your Oracle E-Business Suite environment, using SSH. See Connecting to an Instance in the Oracle Cloud Infrastructure Documentation.

    Then, switch to the root user.
    sudo -i
    Execute the following commands to create the required firewall rules:
    firewall-cmd --zone=public --add-rich-rule \
    'rule family=ipv4 source address=<EBS_Cloud_Admin_Tool_VM_CIDR> \
    port port=7001 protocol=tcp accept' --permanent ;

    firewall-cmd --zone=public --add-rich-rule \
    'rule family=ipv4 source address=<EBS_Cloud_Admin_Tool_VM_CIDR> \
    port port=7002 protocol=tcp accept' --permanent ;

    firewall-cmd --zone=public --add-rich-rule \
    'rule family=ipv4 source address=<EBS_Cloud_Admin_Tool_VM_CIDR> \
    port port=7001 protocol=tcp accept';

    firewall-cmd --zone=public --add-rich-rule \
    'rule family=ipv4 source address=<EBS_Cloud_Admin_Tool_VM_CIDR> \
    port port=7002 protocol=tcp accept';

Once the configuration for the secure access to the WLS admin port is complete, a system administrator can use the following steps detailed in the Oracle E-Business Suite Cloud Manager Guide to access the Fusion Middleware Control or WebLogic Server Administration Console: Accessing the Fusion Middleware Control and WebLogic Server Administration Console with SSH Port Forwarding for Oracle E-Business Suite on Oracle Cloud Infrastructure.

Section 14: Configure Forms and Concurrent Processing for Oracle RAC (Conditionally Required)

Follow the steps in this section if you are running a 2-Node VM DB System.

  1. Implement Oracle RAC load balancing by following the instructions provided in Document 2029173.1Configuring and Managing Oracle E-Business Suite Release 12.2.x Forms and Concurrent Processing for Oracle RAC.

  2. After completing the instructions in Document 2029173.1, restart all application tier services, as shown in the example below:
    sh <ADMIN_SCRIPTS_HOME>/adstpall.sh
    sh <ADMIN_SCRIPTS_HOME>/adstrtal.sh

Section 15: Apply Quarterly Database Bundle Patch (Optional)

If desired, you can apply an available quarterly database bundle patch to get the latest updates. Refer to the appropriate document below for instructions:

  • For a 1-Node VM DB System (Single Instance)Document 2360215.1Installing Database Patch Updates for Oracle E-Business Suite on Single Instance VM DB System or DB System in Oracle Cloud Infrastructure

  • For a 2-Node VM DB System (Oracle RAC)Document 2476292.1Installing Database Patch Updates for Oracle E-Business Suite on Oracle RAC VM DB Systems in Oracle Cloud Infrastructure

Appendix A: Database Patches

This appendix lists the database patches that need to be applied to the container database (CDB) of the 1-Node or 2-Node VM DB System before plugging in the Oracle E-Business Suite database.

The patch list that should be applied is based on the OCI VM DB PSU/bundle level on the provisioned OCI VM DB and Oracle E-Business Suite version.

A consolidated zip file with the required patches is available on My Oracle Support through the patches as listed in the following sections.

Note: The patches listed in the following sections must be applied to the database ORACLE_HOME. Skip any instructions in the patch README files related to the Grid Infrastructure software (GI_HOME).

A1. 1-Node VM DB System (Single Instance) Database Patches

Apply Patch 32878787 - CLOUD EBS RELEASE 12.2 CONSOLIDATED DATABASE FIXES FOR VMDB SI JAN2020 AND APR2021. This corresponds to 12.1.0.2.210420 which will be used when selecting your database version when creating a VM DB system in Section 7.

Ensure the following patches have been applied. All patches listed below are located in the /etcc-bundle/Cloud/DBSI/LINUX_X86-64/database/12.1.0.2.210420ProactiveBP of the patch directory.

  • p32808059_12102210420ProactiveBP_Linux-x86-64.zip
  • p19239846_12102210420ProactiveBP_Linux-x86-64.zip
  • p31404014_12102180717ProactiveBP_Linux-x86-64.zip
  • p19472320_12102200714ProactiveBP_Linux-x86-64.zip
  • p21322448_12102190716ProactiveBP_Linux-x86-64.zip
  • p22731026_121027DBEngSysandDBIM_Generic.zip
  • p21967332_12102160119DBEngSysandDBIM_Linux-x86-64.zip
  • p27051384_12102191015ProactiveBP_Linux-x86-64.zip
  • p24007737_12102170418ProactiveBP_Linux-x86-64.zip
  • p21321429_12102170718ProactiveBP_Generic.zip
  • p21864513_121020_Linux-x86-64.zip
  • p22828765_121020_Generic.zip
  • p22747454_121020_Generic.zip

A2. 2-Node VM DB System (Oracle RAC) Database Patches

Apply Patch 32878784 - CLOUD EBS REL 12.2 CONSOLIDATED DATABASE FIXES FOR VMDB RAC JAN2020 AND APR2021. This corresponds to 12.1.0.2.210420 which will be used when selecting your database version when creating a VM DB system in Section 7.

Ensure the following patches have been applied. All patches listed below are located in the /etcc-bundle/Cloud/VMDB_RAC/LINUX_X86-64/database/12.1.0.2.210420ProactiveBP of the patch directory.

  • p32808059_12102210420ProactiveBP_Linux-x86-64.zip
  • p19239846_12102210420ProactiveBP_Linux-x86-64.zip
  • p31404014_12102180717ProactiveBP_Linux-x86-64.zip
  • p19472320_12102200714ProactiveBP_Linux-x86-64.zip
  • p21322448_12102190716ProactiveBP_Linux-x86-64.zip
  • p22731026_121027DBEngSysandDBIM_Generic.zip
  • p21967332_12102160119DBEngSysandDBIM_Linux-x86-64.zip
  • p27051384_12102191015ProactiveBP_Linux-x86-64.zip
  • p24007737_12102170418ProactiveBP_Linux-x86-64.zip
  • p21321429_12102170718ProactiveBP_Generic.zip
  • p21864513_121020_Linux-x86-64.zip
  • p22828765_121020_Generic.zip
  • p22747454_121020_Generic.zip

Appendix B: Transparent Data Encryption Tablespace Offline Encryption

This appendix describes the steps needed to perform TDE tablespace offline encryption for an Oracle E-Business Suite database.

  1. Verify the wallet status in PDB:
    SQL> alter session set container=<PDB>;
    SQL> select WRL_TYPE, WRL_PARAMETER,STATUS, WALLET_TYPE from gv$encryption_wallet;
    Note: The status column in v$encryption_wallet should show the value OPEN.

  2. Identify all the temporary and undo tablespaces in the database:
    SQL> alter session set container=<PDB>;
    SQL> select tablespace_name from dba_tablespaces where contents='TEMPORARY' and STATUS='ONLINE';

    SQL> select tablespace_name from dba_tablespaces where contents='UNDO' and STATUS='ONLINE';
  3. Create a script called tbsp_offline.sql to bring tablespaces other than SYSTEM, SYSAUX, TEMP and UNDO offline:
    sqlplus / as sysdba
    SQL> alter session set container=<PDB>;
    SQL> set heading off
    SQL> spool tbsp_offline.sql
    SQL> select 'alter tablespace '||tablespace_name|| ' offline;' from dba_tablespaces where tablespace_name not in ('SYSTEM','SYSAUX','TEMP','APPS_UNDOTS1');
    SQL> exit

    Now, edit the tbsp_offline.sql script to remove all lines other than alter tablespace commands.

  4. Bring all the specified tablespaces offline by connecting to SQL*Plus as sysdba, and running the script tbsp_offline.sql:
    sqlplus / as sysdba
    SQL> alter session set container=<PDB>;
    SQL> @tbsp_offline.sql
  5. Create a script called datafiles_encrypt.sql containing the commands to encrypt your datafiles, except SYSTEM, SYSAUX, TEMP and UNDO. Include all TEMP and UNDO tablespace names from the database in your EBS instance, in the exclusion list.
    sqlplus / as sysdba
    SQL> alter session set container=<PDB>;
    SQL> set heading off
    SQL> set linesize 150
    SQL> spool datafiles_encrypt.sql
    SQL> select 'alter database datafile ''' || file_name ||''' encrypt;' from dba_data_files where tablespace_name not in ('SYSTEM','SYSAUX','TEMP','APPS_UNDOTS1');
    SQL> exit

    Now, edit the datafiles_encrypt.sql script to remove all lines other than alter tablespace commands.

  6. Encrypt your datafiles by running the datafiles_encrypt.sql encryption script from SQL*Plus as sysdba:
    sqlplus / as sysdba
    SQL> alter session set container=<PDB>;
    SQL> @datafiles_encrypt.sql
  7. Take the offline tablespaces online using the following steps:
    1. Copy the file tbsp_offline.sql and rename it as tbsp_online.sql.

    2. Edit the tbsp_online.sql file, change the word "offline" to "online", then save the file.

    3. Connect to SQL*Plus as sysdba and run the tbsp_online.sql file to bring the tablespaces online:
      sqlplus / as sysdba
      SQL> alter session set container=<PDB>;
      SQL> @tbsp_online.sql
      Note: Some tablespaces may take time to show as online. These are probably tablespaces that are encrypted.

    4. Check the status of tablespace encryption by connecting to SQL*Plus as sysdba and running the query:
      sqlplus / as sysdba
      SQL> alter session set container=<PDB>;
      SQL> select tablespace_name, encrypted from dba_tablespaces;

Oracle E-Business Suite Release 12.2 System Schema Migration

In This Document Section 1: Overview of the EBS System Schema Section 2: Requirements for Using the EBS System Schema Section 3: Migrating t...