Infrastructure at your Service

Pierre Sicot

ORA-12547 TNS lost contact on SLES12 and oracle 11.2.0.x

On a client’s site, I have migrated successfully Oracle 12c databases from RedHat to SLES12 without any problem. I encountered a problem doing the same work with Oracle 11.2.0.3 and oracle 11.2.0.4 on SLES 12.

Once I have moved the database data files, redo logs, control files and spfile to the new server, when I try to startup the database, I receive the error message:

oracle@server:/u00/app/oracle/diag/ [db1] sq
SQL*Plus: Release 11.2.0.3.0 Production on Thu Feb 22 09:31:18 2018
Copyright (c) 1982, 2011, Oracle.  All rights reserved.
SQL> startup
ERROR:
ORA-12547: TNS:lost contact

In the alert.log file, we receive the following error:

Exception [type: SIGSEGV, SI_KERNEL(general_protection)] [ADDR:0x0] [PC:0x7F09646224A0, __lll_unlock_elision()+48] [flags: 0x0, count: 1]
ORA-12547 : TNS lost contact

I remembered that some months ago, I had quite a similar problem while installing Oracle Clusterware on SLES12 and after a quick search on Metalink I discovered the Metalink note 2297117.1 explaining that glibc in SuSE 12 makes use of a Hardware Lock Elision (HLE) available in newer Intel Processors.

The solution is equivalent to the one purposed in Metalink node 2270947.1 for the Oracle Clusterware problem with SLES12:

We have to add a line in /etc/ld.so.conf:

/lib64/noelision

And then we have to create a symbolic link as follows:

ln -s /lib64/noelision/libpthread-xxx $ORACLE_HOME/lib/libpthread.so.0

Then we edit /etc/ld.so.conf and we add the necessary line

/lib64/noelision
/usr/local/lib64
/usr/local/lib
include /etc/ld.so.conf.d/*.conf

And we create a symbolic link:

ln -s /lib64/noelision/libpthread-2.22.so $ORACLE_HOME/lib/libpthread.so.0

We can check:

oracle@server:/u00/app/oracle/product/11.2.0.3/dbhome_1/bin/ [db1] ldd sqlplus
        linux-vdso.so.1 (0x00007ffdf8513000)
        libsqlplus.so => /u00/app/oracle/product/11.2.0.3/dbhome_1/lib/libsqlplus.so (0x00007fb948303000)
        libclntsh.so.11.1 => /u00/app/oracle/product/11.2.0.3/dbhome_1/lib/libclntsh.so.11.1 (0x00007fb945944000)
        libnnz11.so => /u00/app/oracle/product/11.2.0.3/dbhome_1/lib/libnnz11.so (0x00007fb945577000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007fb945357000)
        libm.so.6 => /lib64/libm.so.6 (0x00007fb945059000)
        libpthread.so.0 => /u00/app/oracle/product/11.2.0.3/dbhome_1/lib/libpthread.so.0 (0x00007fb944e3c000)
        libnsl.so.1 => /lib64/libnsl.so.1 (0x00007fb944c24000)
        libc.so.6 => /lib64/libc.so.6 (0x00007fb944880000)
        libaio.so.1 => /lib64/libaio.so.1 (0x00007fb94467e000)
        /lib64/ld-linux-x86-64.so.2 (0x000055c70f7fc000)

Normally a simple reconnection should have solved the problem, but I had some semaphores and memory segments blocked.

I had to run sysresv and ipcrm to completely solve my problem:=)

The first connection did not solve the problem:

oracle@server:/u00/app/oracle/ [db1] sq
SQL*Plus: Release 11.2.0.3.0 Production on Thu Feb 22 09:45:41 2018
Copyright (c) 1982, 2011, Oracle.  All rights reserved.
Connected.
SQL> startup
ERROR:
ORA-12547: TNS:lost contact

I had to use the sysresv utility which provides instance status for a specified ORACLE_SID. This utility offers the possibility to identify resources to be deleted, especially if the instance is detected to be dead.

In my case, typically my instance was crashed and memory segments or semaphores were always present.

So running sysresv:

oracle@server:/u00/app/oracle/  [db1] sysresv
IPC Resources for ORACLE_SID "db1" :
Shared Memory:
ID              KEY
1278378019      0x00000000
1278410788      0x00000000
1278443557      0x186a9d18
Semaphores:
ID              KEY
5931036         0x7e57b354
Unable to determine if Oracle instance alivefor sid "db1"

Sysresv detects the shared memory and semaphores id. To delete the memory segments or the semaphores segments, we use ipcrm -m or ipcrm -s:

In my case I removed the memory segments:

ipcrm -m 1278378019
ipcrm -m 1278410788
ipcrm -m 1278443557

Finally after performing the listed actions, I was able to start the database without any problem :=)

 

 

 

 

 

 

 

 

 

Leave a Reply

Pierre Sicot
Pierre Sicot

Senior Consultant