CELL-01547: CELLSRV startup failed due to unknown reasons. ORA-600 [LinuxBlockIO::init]


ISSUE :

[celladmin@Exacell ~]$ cellcli -e alter cell status services all

Stopping the RS, CELLSRV, and MS services...
The SHUTDOWN of services was successful.
Starting the RS, CELLSRV, and MS services...
Getting the state of RS services...  running
Starting CELLSRV services...
The STARTUP of CELLSRV services was not successful.
CELL-01547: CELLSRV startup failed due to unknown reasons.
Starting MS services...
The STARTUP of MS services was successful.


Note : Check the trace files 

/opt/oracle/cell12.1.1.1.0_LINUX.X64_131219/log/diag/asm/cell/Exacell/trace/svtrc_8204_main.trc


----- DDE Diagnostic Information Dump -----
Depth: 1
DDE flags: 0x0
Heap: 0x4c3dee0
Incident Context pointer in diag: 0x7ffd9c902030
Incident ID Cache: 0x4c62170 (PGA)
Invocation Context #: 0
----- Invocation Context Dump -----
Address: 0x4c58f30
Phase: 3
flags: 0x10E0000
Incident ID: 25
Error Descriptor: ORA-600 [LinuxBlockIO::init] [] [] [] [] [] [] [] [] [] [] []
Error class: 0
Problem Key # of args: 0
Number of actions: 7
----- Incident Context Dump -----
Address: 0x7ffd9c902030
Incident ID: 25
Problem Key: ORA 600
Error: ORA-600 [LinuxBlockIO::init] [] [] [] [] [] [] [] [] [] [] []



Alert Log : 

CELLSRV version=12.1.1.1.0,label=OSS_12.1.1.1.0_LINUX.X64_131219,Thu_Dec_19_03:35:39_PST_2013
OS Stats: Physical memory: 3000 MB. Num cores: 2
CELLSRV configuration parameters:
Cellsrv max memory not set. Total physical mem: 3000 MB is less than required minimum: 3891 MB.
Running on simulated hardware in production environment
MS_ALERT HUGEPAGE CLEAR
CELL communication is configured to use 1 interface(s):
    10.0.0.100
IPC version: Oracle RDS/IP (generic)
IPC Vendor 1 Protocol 3
  Version 4.1
Error 18446744073709551605 from io_setup, async ctx = 0x228a8c8
Errors in file /opt/oracle/cell12.1.1.1.0_LINUX.X64_131219/log/diag/asm/cell/Exacell/trace/svtrc_8204_main.trc  (incident=33):
ORA-00600: internal error code, arguments: [LinuxBlockIO::init], [], [], [], [], [], [], [], [], [], [], []
Incident details in: /opt/oracle/cell12.1.1.1.0_LINUX.X64_131219/log/diag/asm/cell/Exacell/incident/incdir_33/svtrc_8204_main_i33.trc
Sweep [inc][33]: completed
Tue Apr  7 20:30:17 2020 State dump completed for CELLSRV<8204> after ORA-600 occurred
CELLSRV error - ORA-600 internal error
Tue Apr 07 20:30:17 2020
Could not connect to MS socket. Communication with MS may be degraded. errno=115
[RS] monitoring process /opt/oracle/cell12.1.1.1.0_LINUX.X64_131219/cellsrv/bin/cellrsomt (pid: 8197) returned with error: 161
Tue Apr 07 20:30:17 2020
[RS] Stopped Service CELLSRV with pid 8204
[RS] Started Service MS with pid 8199


Mainly this is due to memory issue : 


Workaround :

Will be fixed by increasing the memory of vm from 3 to 8 and 

/etc/sysctl.conf

fs.aio-max-nr = 50000000


CellCLI> alter cell shutdown services cellsrv

Stopping CELLSRV services... 
The SHUTDOWN of CELLSRV services was successful.


[celladmin@Exacell ~]$ cellcli -e create cell exacel01 interconnect1=eth1
Cell exacel01 successfully created
Starting CELLSRV services...
The STARTUP of CELLSRV services was successful.
Flash cell disks, FlashCache, and FlashLog will be created...
CellDisk FD_00_exacel01 successfully created
CellDisk FD_01_exacel01 successfully created
CellDisk FD_02_exacel01 successfully created
CellDisk FD_03_exacel01 successfully created
CellDisk FD_04_exacel01 successfully created
CellDisk FD_05_exacel01 successfully created
Flash log exacel01_FLASHLOG successfully created
Flash cache exacel01_FLASHCACHE successfully created

Previous
Next Post »