By Mouhamadou Diaw

Last day I found a curious message in the backup alert log in a dataguard environment.
The message was related to an issue with the resync catalog and was something like

1
2
3
4
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03009: failure of resync command on default channel at 02/18/2016 17:48:42
ORA-00001: unique constraint (RMAN.TS_P2) violated
After few researches, it seems that this error is due to a mismatch of tempfiles between Primary and Standby.
Some tempfiles were missing in the standby database.

How is it be possible as we have the parameter StandbyFileManagement set  to auto on both sides

Primary database
1
2
3
4
5
6
7
8
9
10
NAME                     TYPE    VALUE
------------------------------------ ----------- ------------------------------
db_unique_name               string  HRVA_SITE1
SQL> show parameter standby_file
NAME                     TYPE    VALUE
------------------------------------ ----------- ------------------------------
standby_file_management          string  AUTO
SQL>
Standby database
1
2
3
4
5
6
7
8
9
10
NAME                     TYPE    VALUE
------------------------------------ ----------- ------------------------------
db_unique_name               string  HRVA_SITE2
SQL> show parameter standby_file
NAME                     TYPE    VALUE
------------------------------------ ----------- ------------------------------
standby_file_management          string  AUTO
SQL>

Let’s do some tests to better understand how oracle manages datafiles and tempfiles in dataguard environment

First we add a datafile to the primary database

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
SQL> show parameter db_uni
NAME                     TYPE    VALUE
------------------------------------ ----------- --------------------
db_unique_name               string  HRVA_SITE1
SQL> alter tablespace USERS add datafile '/u01/app/oracle/oradata/HRVA/users02.dbf' size 5M;
Tablespace altered.
SQL> select name from v$datafile;
NAME
--------------------------------------------------------------------
/u01/app/oracle/oradata/HRVA/system01.dbf
/u01/app/oracle/oradata/HRVA/sysaux01.dbf
/u01/app/oracle/oradata/HRVA/undotbs01.dbf
/u01/app/oracle/oradata/HRVA/users01.dbf
/u01/app/oracle/oradata/HRVA/users02.dbf
Now let's verify that the new datafile is present in the standby database
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
SQL> show parameter db_uni
NAME                     TYPE    VALUE
------------------------------------ ----------- -----------------------
db_unique_name               string  HRVA_SITE2
SQL> select open_mode from v$database;
OPEN_MODE
--------------------
MOUNTED
SQL> select name from v$datafile;
NAME
--------------------------------------------------------------------------------
/u01/app/oracle/oradata/HRVA/system01.dbf
/u01/app/oracle/oradata/HRVA/sysaux01.dbf
/u01/app/oracle/oradata/HRVA/undotbs01.dbf
/u01/app/oracle/oradata/HRVA/users01.dbf
/u01/app/oracle/oradata/HRVA/users02.dbf
SQL>

This confirms that the new datafile was replicated as expected.

Now let’s add a tempfile to the primary database

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
SQL> show parameter db_uni
NAME                     TYPE    VALUE
------------------------------------ ----------- --------------------------
db_unique_name               string  HRVA_SITE1
SQL> select name from v$tempfile;
NAME
---------------------------------------------------------------------------
/u01/app/oracle/oradata/HRVA/temp01.dbf
SQL> alter tablespace temp add tempfile '/u01/app/oracle/oradata/HRVA/temp02.dbf' size 10M;
Tablespace altered.
SQL> select name from v$tempfile;
NAME
--------------------------------------------------------------------------
/u01/app/oracle/oradata/HRVA/temp01.dbf
/u01/app/oracle/oradata/HRVA/temp02.dbf
SQL>

Connecting to the standby database, we can see that the new tempfile was not replicated (although the standby_file_management is set to auto)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
SQL> select open_mode from v$database;
OPEN_MODE
--------------------
MOUNTED
SQL> show parameter db_uni
NAME                     TYPE    VALUE
------------------------------------ ----------- ------------------------------
db_unique_name               string  HRVA_SITE2
SQL> select name from v$tempfile;
NAME
--------------------------------------------------------------------------------
/u01/app/oracle/oradata/HRVA/temp01.dbf
SQL>

In fact when you add tempfiles in the Primary database, new tempfiles are not added automatically in the Physical Standby database

because no redo is generated.  (Oracle documentation  Doc ID 834174.1)

We have to manually create the added tempfile in the standby. For this we have to open the standby database in Read Only mode.

1- First we disable  the redo apply on the standby database to not activate ACTIVE DATAGUARD option if ever we do not have the license

1
2
3
4
5
6
7
8
9
10
DGMGRL> EDIT DATABASE 'HRVA_SITE2' SET STATE='APPLY-OFF';
Succeeded.
DGMGRL> show database verbose 'HRVA_SITE2';
Database - HRVA_SITE2
  Role:            PHYSICAL STANDBY
  Intended State:  APPLY-OFF
  Transport Lag:   0 seconds (computed 1 second ago)
  Apply Lag:       46 seconds (computed 1 second ago)
  Apply Rate:      (unknown)
  Real Time Query: OFF
2- We open the standby database in read only mode
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
SQL> alter database open read only;
Database altered.
********* dbi services Ltd. *********
STATUS                 : OPEN
DB_UNIQUE_NAME         : HRVA_SITE2
OPEN_MODE              : READ ONLY
LOG_MODE               : ARCHIVELOG
DATABASE_ROLE          : PHYSICAL STANDBY
FLASHBACK_ON           : YES
FORCE_LOGGING          : YES
VERSION                : 11.2.0.4.0
*************************************
oracle@rac12e:/home/oracle/ [HRVA]
3- We manually add the missing tempfile in the standby database
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
SQL> select name from v$tempfile;
NAME
---------------------------------------------------------------------------
/u01/app/oracle/oradata/HRVA/temp01.dbf
SQL> alter tablespace temp add tempfile '/u01/app/oracle/oradata/HRVA/temp02.dbf' size 10M;
Tablespace altered.
SQL> select name from v$tempfile;
NAME
---------------------------------------------------------------------------
/u01/app/oracle/oradata/HRVA/temp01.dbf
/u01/app/oracle/oradata/HRVA/temp02.dbf

4- We shut down the standby database and  mount the database

1
2
3
4
5
6
7
8
9
10
11
12
13
14
SQL> shutdown immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup mount;
ORACLE instance started.
Total System Global Area  417546240 bytes
Fixed Size          2253824 bytes
Variable Size         331353088 bytes
Database Buffers       79691776 bytes
Redo Buffers            4247552 bytes
Database mounted.
SQL>
5- We enable the redo apply
1
2
3
4
5
6
7
8
9
10
11
12
DGMGRL> EDIT DATABASE 'HRVA_SITE2' SET STATE='APPLY-ON';
Succeeded.
DGMGRL> show database verbose 'HRVA_SITE2';
Database - HRVA_SITE2
  Role:            PHYSICAL STANDBY
  Intended State:  APPLY-ON
  Transport Lag:   0 seconds (computed 0 seconds ago)
  Apply Lag:       0 seconds (computed 0 seconds ago)
  Apply Rate:      0 Byte/s
  Real Time Query: OFF
  Instance(s):
    HRVA

Conclusion

In a dataguard environment whenever we add tempfiles or temporary tablespaces, we have to create manually the corresponding tempfiles in the standby database.