Monday, September 2, 2013

RMAN 12c: System trigger to open PDBs causing table recovery to fail with ORA-65019, RMAN-11003

In Oracle Database 12c, while trying to perform a table recovery, I hit the below error:

RMAN-11003: failure during parse/execution of SQL statement: alter pluggable database  PDB_RMAN open
ORA-65019: pluggable database PDB_RMAN already open

The command used was:

RECOVER TABLE TEST.TEST OF PLUGGABLE DATABASE PDB_RMAN UNTIL SCN 3106312 AUXILIARY DESTINATION '/u01/app/oracle/test';

I had all the pre-requisites validated:
The target database was in read-write mode.
The target database was in ARCHIVELOG mode.
RMAN full backup taken previously the drop command

For the information, I had created an empty pluggable database, created a table in it, performed full backup at CDB level.
I then dropped the table and tried to recover it.

I have previously created a system trigger to start all my pluggable database when CDB is open.

Full section of the error is the below:

contents of Memory Script:
{
sql clone 'alter pluggable database  PDB_RMAN open';
}
executing Memory Script

sql statement: alter pluggable database  PDB_RMAN open

Removing automatic instance
shutting down automatic instance
Oracle instance shut down
Automatic instance removed
auxiliary instance file /u01/app/oracle/test/JVGF_PITR_PDB_RMAN_CDB1/datafile/o1_mf_test_916xmk0w_.dbf deleted
auxiliary instance file /u01/app/oracle/test/CDB1/datafile/o1_mf_sysaux_916xkgf6_.dbf deleted
auxiliary instance file /u01/app/oracle/test/CDB1/datafile/o1_mf_system_916xkgh9_.dbf deleted
auxiliary instance file /u01/app/oracle/test/CDB1/datafile/o1_mf_sysaux_916xhq2h_.dbf deleted
auxiliary instance file /u01/app/oracle/test/CDB1/datafile/o1_mf_undotbs1_916xhq31_.dbf deleted
auxiliary instance file /u01/app/oracle/test/CDB1/datafile/o1_mf_system_916xhq2x_.dbf deleted
auxiliary instance file /u01/app/oracle/test/CDB1/controlfile/o1_mf_916xhgl9_.ctl deleted
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of recover command at 08/20/2013 15:00:54
RMAN-03015: error occurred in stored script Memory Script
RMAN-03009: failure of sql command on clone_default channel at 08/20/2013 15:00:54
RMAN-11003: failure during parse/execution of SQL statement: alter pluggable database  PDB_RMAN open
ORA-65019: pluggable database PDB_RMAN already open


In short, what happens during table recovery is the following:
- Oracle creates an automatic (auxiliary) instance
- restores the backup into it and performes a media recovery
- exports from the auxiliary instance using EXPDP the table you want to recover
- imports into your pluggable database using IMPDP


The issue appears when you have a system trigger at CDB level which opens all PDB or at least the pluggable database where you have the issue with
the table you want to recover.

The workaround is to create an auxiliary parameter file /tmp/initaux.ora with the following line in it:
_system_trig_enabled=FALSE

Then, while invoking RMAN, set this auxiliary instance parameter file before recovering the table

RMAN> set auxiliary instance parameter file to '/tmp/initaux.ora';
RMAN> recover table test.test....

The error will no longer appear and you will be able to complete the recovery of the table.

This is documented as the following bug:
Bug 16822622 : PDB TABLESPACE PITR FAILS WITH EXISTING TRIGGER OPENING PDBS

No comments:

Post a Comment