Monday, 13 April 2015

msStatus stopped : socket open error: Port no: 8888. Received errorno 111. Connection refused

After restarting the Test VM exadata i see msStatus is stopped,So i try to start the msStatus alone and i get "Timed out" error then in log it shows connection refused.

[root@localhost ~]# service celld status
         rsStatus:               running
         msStatus:               stopped
         cellsrvStatus:          running

[root@localhost ~]# cellcli -e "alter cell startup services ms"
Starting MS services...
The STARTUP of MS services was not successful. Error: Start Timed out

While checking the log 


2015-04-13 10:31:15.981305 :00000007: Started monitoring process /opt/oracle/cell12.1.2.1.0_LINUX.X64_141206.1/cellsrv/bin/cellrsmmt with pid 6181
2015-04-13 10:31:15.981864 :00000008: mon_proc_pid oldpid: 0
2015-04-13 10:31:15.985699 :00000009: mon_proc_pid newpid: 0
2015-04-13 10:31:15.985738 :0000000A: MS process is not alive. Pid is missing.
2015-04-13 10:31:15.985755 :0000000B: Missed a heartbeat for process MS or leaking memory, error: -75
2015-04-13 10:31:15.985767 :0000000C: Service MS was not alive, try starting
2015-04-13 10:31:15.985939 :0000000D: Exec new process /opt/oracle/cell12.1.2.1.0_LINUX.X64_141206.1/cellsrv/deploy/msdomain/bin/startWebLogic.sh
2015-04-13 10:31:15.985961 :0000000E: Cmdline: /opt/oracle/cell12.1.2.1.0_LINUX.X64_141206.1/cellsrv/deploy/msdomain/bin/startWebLogic.sh
2015-04-13 10:31:15.985978 :0000000F: Redirect STDOUT from process /opt/oracle/cell12.1.2.1.0_LINUX.X64_141206.1/cellsrv/deploy/msdomain/bin/startWebLogic.sh to MS (tra
ce flag 3)
2015-04-13 10:31:15.986421 :00000010: Trace location after redirection: /opt/oracle/cell12.1.2.1.0_LINUX.X64_141206.1/cellsrv/deploy/log/wls6182.trc
2015-04-13 10:31:15.986668 :00000010: mon_proc_pid oldpid: 0
2015-04-13 10:31:15.990022 :00000011: mon_proc_pid newpid: 0
2015-04-13 10:31:16.092647 :00000012: mon_proc_pid oldpid: 0
2015-04-13 10:31:16.097378 :00000013: mon_proc_pid newpid: 6231
2015-04-13 10:31:16.097714 :00000014: socket open error: Port no: 8888. Received errorno 111. Connection refused
2015-04-13 10:31:16.198637 :00000015: mon_proc_pid oldpid: 6231
2015-04-13 10:31:16.198885 :00000016: socket open error: Port no: 8888. Received errorno 111. Connection refused
2015-04-13 10:31:16.300606 :00000017: mon_proc_pid oldpid: 6231
2015-04-13 10:31:16.300824 :00000018: socket open error: Port no: 8888. Received errorno 111. Connection refused
2015-04-13 10:31:16.401627 :00000019: mon_proc_pid oldpid: 6231
2015-04-13 10:31:16.401905 :0000001A: socket open error: Port no: 8888. Received errorno 111. Connection refused
2015-04-13 10:31:16.502602 :0000001B: mon_proc_pid oldpid: 6231
2015-04-13 10:31:16.502812 :0000001C: socket open error: Port no: 8888. Received errorno 111. Connection refused
2015-04-13 10:31:16.605710 :0000001D: mon_proc_pid oldpid: 6231
2015-04-13 10:31:16.605949 :0000001E: socket open error: Port no: 8888. Received errorno 111. Connection refused
2015-04-13 10:31:16.706607 :0000001F: mon_proc_pid oldpid: 6231
2015-04-13 10:31:16.706847 :00000020: socket open error: Port no: 8888. Received errorno 111. Connection refused


Solution:
Stop all the services and start the cell in order rs,ms and cellsrv service.

[root@localhost ~]# service celld stop

Stopping the RS, CELLSRV, and MS services...
The SHUTDOWN of services was successful.
[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]# cellcli -e "alter cell startup services rs"

Starting the RS services...
Getting the state of RS services...  running
[root@localhost ~]# cellcli -e "alter cell startup services ms"

Starting MS services...
The STARTUP of MS services was successful.
[root@localhost ~]# cellcli -e "alter cell startup services cellsrv"

Starting CELLSRV services...
The STARTUP of CELLSRV services was successful.
[root@localhost ~]#

No comments: