Could not mount Database after restarting the cluster

Could not mount Database after restarting the cluster.

ALERT LOG

ORA-00202: controlfile: '/dev/vx/rdsk/racdg/orcl_raw_controlfile1'

ORA-27041: unable to open file

SVR4 Error: 6: No such device or address

Additional information: 2

Thu Jan 5 20:12:07 2006

ORA-205 signalled during: ALTER DATABASE MOUNT...

DISKDUMP

$ dd if=/dev/vx/rdsk/racdg/orcl_raw_controlfile1 of=/dev/null

dd: /dev/vx/rdsk/racdg/orcl_raw_controlfile1: open: No such device or address

$ dd if=/dev/vx/rdsk/racdg/orcl_raw_tools of=/dev/null bs=1024 count=3

dd: /dev/vx/rdsk/racdg/orcl_raw_tools: open: No such device or address

MESSAGES

Jan 5 19:38:06 icai1 last message repeated 1 time

Jan 5 19:38:06 icai1 Cluster.CCR: [ID 551094 daemon.warning] reservation warning(node_join) - Unable to open device /dev/did/rdsk/d11s2, will retry in 2 seconds

Jan 5 19:38:10 icai1 last message repeated 2 times

Jan 5 19:38:12 icai1 Cluster.CCR: [ID 192619 daemon.warning] reservation error(node_join) - Unable to open device /dev/did/rdsk/d11s2

Jan 5 19:38:17 icai1 Cluster.Framework: [ID 801593 daemon.notice] stdout: resetting scsi buses shared with non-cluster nodes

Jan 5 19:38:17 icai1 Cluster.CCR: [ID 417232 daemon.warning] scvxvmlg warning - /dev/vx/dsk/racdg does not link to /global/.devices/node@1/dev/vx/dsk/racdg, changing it

Jan 5 19:38:17 icai1 Cluster.CCR: [ID 417232 daemon.warning] scvxvmlg warning - /dev/vx/rdsk/racdg does not link to /global/.devices/node@1/dev/vx/rdsk/racdg, changing it

Jan 5 19:38:17 icai1 Cluster.CCR: [ID 869698 daemon.warning] scvxvmlg warning - found no matching volume for device node /global/.devices/node@1/dev/vx/dsk/racdg/orcl_raw_caddatacro, removing it

387150 :scvxvmlg warning - found no matching volume for device node %s, removing it

Description: The program responsible for maintaining the VxVM device namespace has discovered inconsistencies between the VxVM device namespace on this node and the VxVM configuration information stored in the cluster device configuration system. If configuration changes were made recently, then this message should reflect one of the configuration changes. If no changes were made recently or if this message does not correctly reflect a change that has been made, the VxVM device namespace on this node may be in an inconsistent state. VxVM volumes may be inaccessible from this node.

Solution: If this message correctly reflects a configuration change to VxVM diskgroups then no action is required. If the change this message reflects is not correct, then the information stored in the device configuration system for each VxVM diskgroup should be examined for correctness. If the information in the device configuration system is accurate, then executing '/usr/cluster/lib/dcs/scvxvmlg' on this node should restore the device namespace. If the information stored in the device configuration system is not accurate, it must be updated by executing '/usr/cluster/bin/scconf -c -D name=diskgroup_name' for each VxVM diskgroup with inconsistent information.

http://docs.sun.com/app/docs/doc/806-1426/6janbf84j?a=view