How to Synchronize the AD tables FND_NODES, ADOP_VALID_NODES, and FND_OAM_CONTEXT_FILES in 12.2 when adop fails. (Doc ID 2064223.1)

APPLIES TO:

Oracle Applications DBA - Version 12.2 to 12.2.8 [Release 12.2Cloud to 12.2]
Oracle Concurrent Processing - Version 12.2.6 to 12.2.8 [Release 12.2]
Information in this document applies to any platform.

SYMPTOMS

CASE 1: adop phase=prepare

$ adop phase=prepare

  Enter the APPS password:
  Enter the SYSTEM password:
  Enter the WLSADMIN password:

Validating credentials...

Initializing...
  Run Edition context : /appl/admin/PROD_prodapp1.xml
  Patch edition context: /appl/admin/PROD_prodapp1.xml
*******FATAL ERROR*******
PROGRAM : (/appl/ad/12.0.0/bin/adzdoptl.pl)
TIME : Fri Oct 2 22:06:45 2015
FUNCTION: ADOP::GlobalVars::_GetMandatoryArgs [ Level 1 ]
ERRORMSG: adop is not able to detect any valid application tier nodes in ADOP_VALID_NODES table.

Ensure autoconfig is run on all nodes.

[STATEMENT] Please run adopscanlog utility, using the command
"adopscanlog -latest=yes"
to get the list of the log files along with snippet of the error message corresponding to each log file.
adop exiting with status = 255 (Fail)

CAUSE

CASE 1: adop phase=prepare

In this case, The fnd_nodes table has NULL for domain entry of this valid server.

Example Data Supporting the Conclusion
---------------
[table.applsys.fnd_nodes]

NODE     PLAT    D C A F W    NODE_NAME    SERVER_ADDRESS     DOMAIN WEBHOST                              VIRTUAL_IP    S
-------  -----    ----------   -------------   ------------------   ---------   --------- -------------------                    ---------------  --
12,121     226    N Y Y Y Y    NEW122APP1   ###.###.###.103     NULL     new122app1.somewhere.com  NULL             Y
 

SOLUTION

Test the following steps in a cloned instance first, and then migrate accordingly once the desired result is confirmed:
 

NOTE: 1. Due to the method required for "cleaning out" / "re-synchronizing" the following tables, it is EXPECTED / REQUIRED that the Applications have been shutdown.
                  The only thing running should be the Database Tier.
           2. A full backup should be taken before any testing begins.

1. Backup the fnd_oam_context_files, fnd_nodes, and adop_valid_nodes tables, and if on AD/TXK 8 or higher; ad_nodes_config_status in the EBS env nodes:

      Linux#> sqlplus applsys/<pwd>

      SQLPlus#> create table fnd_oam_context_files_bkp as select * from fnd_oam_context_files;
      SQLPlus#> create table fnd_nodes_bk as select * from fnd_nodes;
        SQLPlus#> create table adop_valid_nodes_bk as select * from adop_valid_nodes;
      If on AD/TXK 8 or higher:
      SQLPlus#> create table ad_nodes_config_status_bk as select * from ad_nodes_config_status;

 2. Truncate the following tables (Continue from step 1):

      SQLPlus#> truncate table fnd_oam_context_files;
      SQLPlus#> truncate table fnd_nodes;
      SQLPlus#> truncate table adop_valid_nodes;
      If on AD/TXK 8 or higher:
      SQLPlus#> truncate table ad_nodes_config_status;

3.  Run AutoConfig on the DB tier:

Source the <RDBMS_ORACLE_HOME> home.

Linux#> cd <RDBMS_ORACLE_HOME>/appsutil/scripts/<SID>_<HOSTNAME>/

Linux#> ./adautocfg.sh

              ... Confirm Autoconfig completes successfully.

              ... If RAC, Repeat step 3 on all RAC nodes.

4.  Run Autoconfig on the run file system.

Linux#> source <EBS_BASE>/EBSapps.env run

Linux#> cd $ADMIN_SCRIPTS_HOME

Linux#> ./adautocfg.sh

      ... Confirm Autoconfig completes successfully.

      ... If more than one EBS node, repeat step 4 on all EBS nodes.

5.  Run Autoconfig on the patch file system

Before running Autoconfig on the patch file system the ebs_login trigger MUST be disabled
... After the successful completion of Autoconfig the ebs_login trigger MUST be re-enabled.

a. Disable the ebs_login trigger using the following SQL.

Linux#> sqlplus system/<pwd>

SQLPlus#> alter trigger ebs_logon disable;

b. At this time Run autoconfig with the patch env sourced .        

Linux#> source <EBS_BASE>/EBSapps.env patch

Linux#> cd $ADMIN_SCRIPTS_HOME

Linux#> ./adautocfg.sh

... Confirm Autoconfig completes successfully

... If more than one EBS node, repeat step 5.b on all EBS nodes.

c. Enable the ebs_login trigger using the following SQL.

Linux#> sqlplus system/<pwd>

SQLPlus#> alter trigger ebs_logon enable;

 6.  After Autoconfig has been run successfully on all nodes, run the following two (2) queries in order to verify the tables have been correctly populated:

SQL>    set pagesize 5
        set linesize 132
        col node_name format a15
        col server_id format a8
        col server_address format a15
        col platform_code format a4
        col webhost format a12
        col domain format a20
        col virtual_ip format a12

        select node_id, platform_code, support_db D, support_cp C, support_admin A, support_forms F, support_web W, node_name, server_id,
               server_address, domain, webhost, virtual_ip, status
          from fnd_nodes order by node_id;

SQL>    set pagesize 5
        set linesize 132
        col NAME format A20
        col VERSION format A12
        col PATH format A110
        col STATUS format A10

        select NAME,VERSION,PATH, STATUS from FND_OAM_CONTEXT_FILES;