RAC Attack - Oracle Cluster Database at Home/Install Grid Infrastructure (Shared Filesystem)

Grid Install (Shared Filesystem) (e)

Several NFS appliances and big-iron cluster filesystems are very common in large cluster database deployments. We will use OCFS2 here to practice 11gR2 RAC with a filesystem.

As the oracle user, launch the grid installer. At the first screen, choose Install and Configure Grid Infrastructure for a Cluster and click NEXT. [oracle@collabn1 ~]$ /mnt/cdrom*/grid/runInstaller
Choose Advanced Installation and click NEXT.
Accept the default language (English) and choose NEXT.
Name the cluster collab and make sure that the SCAN name is collab-scan with port 1521, then click NEXT.
Add node collabn2 with VIP collabn2-vip and choose NEXT to validate the cluster configuration.
Verify that eth0 on subnet 192.168.78.0 is PUBLIC and that eth1 on subnet 172.16.100.0 is PRIVATE, then click NEXT.
Choose to store the Clusterware Files in Shared File System and choose NEXT.
For the OCR, choose External Redundancy and type the path /u61/cluster/ocr. (This is the NFS location from the BUG WORKAROUND.) Click NEXT to continue.
For the Voting Disk, do the same – choose External Redundancy and type the path /u61/cluster/vdsk. (Again, this is the NFS location from the BUG WORKAROUND.) Click NEXT to continue.
Do not use IPMI. Click NEXT.
Set the OSDBA group to asmdba, the OSOPER group to asmoper and the OSASM group to asmadmin. Then click NEXT.
Accept the ORACLE_BASE location of /u01/app/oracle and use the ORACLE_HOME location of /u01/grid/oracle/product/11.2.0/grid_1. Then click NEXT.
Accept the default inventory location and choose NEXT
The prerequisite checks will execute. A warning will be issued saying that three checks failed: physical memory, swap size and network time protocol. Click the CHECK BOX to Ignore All, then click NEXT.
SAVE a response file called grid.rsp in the oracle user home directory. Then click FINISH to install grid infrastructure.

When prompted, open a terminal as the root user and run the two root.sh scripts. Make sure to run BOTH SCRIPTS on BOTH NODES!

[oracle@collabn1 ~]$ su -
Password: racattack
 
[root@collabn1 ~]# /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read, write permissions for group.
Removing read,write,execute permissions for world.
 
Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.
 
[root@collabn1 ~]# /u01/grid/oracle/product/11.2.0/grid_1/root.sh
Running Oracle 11g root.sh script...
 
The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/grid/oracle/product/11.2.0/grid_1
 
Enter the full pathname of the local bin directory: [/usr/local/bin]: /usr/bin
   Copying dbhome to /usr/bin ...
   Copying oraenv to /usr/bin ...
   Copying coraenv to /usr/bin ...
 
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
2011-03-30 16:49:13: Parsing the host name
2011-03-30 16:49:13: Checking for super user privileges
2011-03-30 16:49:13: User has super user privileges
Using configuration parameter file: /u01/grid/oracle/product/11.2.0/grid_1/crs/install/crsconfig_params
Creating trace directory
LOCAL ADD MODE
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
  root wallet
  root wallet cert
  root cert export
  peer wallet
  profile reader wallet
  pa wallet
  peer wallet keys
  pa wallet keys
  peer cert request
  pa cert request
  peer cert
  pa cert
  peer root cert TP
  profile reader root cert TP
  pa root cert TP
  peer pa cert TP
  pa peer cert TP
  profile reader pa cert TP
  profile reader peer cert TP
  peer user cert
  pa user cert
Adding daemon to inittab
CRS-4123: Oracle High Availability Services has been started.
ohasd is starting
CRS-2672: Attempting to start 'ora.gipcd' on 'collabn1'
CRS-2672: Attempting to start 'ora.mdnsd' on 'collabn1'
CRS-2676: Start of 'ora.mdnsd' on 'collabn1' succeeded
CRS-2676: Start of 'ora.gipcd' on 'collabn1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'collabn1'
CRS-2676: Start of 'ora.gpnpd' on 'collabn1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'collabn1'
CRS-2676: Start of 'ora.cssdmonitor' on 'collabn1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'collabn1'
CRS-2672: Attempting to start 'ora.diskmon' on 'collabn1'
CRS-2676: Start of 'ora.diskmon' on 'collabn1' succeeded
CRS-2676: Start of 'ora.cssd' on 'collabn1' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'collabn1'
CRS-2676: Start of 'ora.ctssd' on 'collabn1' succeeded
clscfg: -install mode specified
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-2672: Attempting to start 'ora.crsd' on 'collabn1'
CRS-2676: Start of 'ora.crsd' on 'collabn1' succeeded
Now formatting voting disk: /u61/cluster/vdsk.
CRS-4603: Successful addition of voting disk /u61/cluster/vdsk.
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   59f9f62cc0f44f35bff552ced06e35ee (/u61/cluster/vdsk) []
Located 1 voting disk(s).
CRS-2673: Attempting to stop 'ora.crsd' on 'collabn1'
CRS-2677: Stop of 'ora.crsd' on 'collabn1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'collabn1'
CRS-2677: Stop of 'ora.asm' on 'collabn1' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'collabn1'
CRS-2677: Stop of 'ora.ctssd' on 'collabn1' succeeded
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'collabn1'
CRS-2677: Stop of 'ora.cssdmonitor' on 'collabn1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'collabn1'
CRS-2677: Stop of 'ora.cssd' on 'collabn1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'collabn1'
CRS-2677: Stop of 'ora.gpnpd' on 'collabn1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'collabn1'
CRS-2677: Stop of 'ora.gipcd' on 'collabn1' succeeded
CRS-2673: Attempting to stop 'ora.mdnsd' on 'collabn1'
CRS-2677: Stop of 'ora.mdnsd' on 'collabn1' succeeded
CRS-2672: Attempting to start 'ora.mdnsd' on 'collabn1'
CRS-2676: Start of 'ora.mdnsd' on 'collabn1' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'collabn1'
CRS-2676: Start of 'ora.gipcd' on 'collabn1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'collabn1'
CRS-2676: Start of 'ora.gpnpd' on 'collabn1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'collabn1'
CRS-2676: Start of 'ora.cssdmonitor' on 'collabn1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'collabn1'
CRS-2672: Attempting to start 'ora.diskmon' on 'collabn1'
CRS-2676: Start of 'ora.diskmon' on 'collabn1' succeeded
CRS-2676: Start of 'ora.cssd' on 'collabn1' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'collabn1'
CRS-2676: Start of 'ora.ctssd' on 'collabn1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'collabn1'
CRS-2676: Start of 'ora.asm' on 'collabn1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'collabn1'
CRS-2676: Start of 'ora.crsd' on 'collabn1' succeeded
CRS-2672: Attempting to start 'ora.evmd' on 'collabn1'
CRS-2676: Start of 'ora.evmd' on 'collabn1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'collabn1'
CRS-2676: Start of 'ora.asm' on 'collabn1' succeeded
CRS-2672: Attempting to start 'ora.DATA.dg' on 'collabn1'
CRS-2676: Start of 'ora.DATA.dg' on 'collabn1' succeeded
CRS-2672: Attempting to start 'ora.registry.acfs' on 'collabn1'
CRS-2676: Start of 'ora.registry.acfs' on 'collabn1' succeeded
 
collabn1     2011/03/30 16:59:35     /u01/grid/oracle/product/11.2.0/grid_1/cdata/collabn1/backup_20110330_165935.olr
Preparing packages for installation...
cvuqdisk-1.0.7-1
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
Updating inventory properties for clusterware
Starting Oracle Universal Installer...
 
Checking swap space: must be greater than 500 MB.   Actual 968 MB    Passed
The inventory pointer is located at /etc/oraInst.loc
The inventory is located at /u01/app/oraInventory
'UpdateNodeList' was successful.

[root@collabn1 ~]# ssh collabn2
root@collabn2's password: racattack
-bash: oraenv: No such file or directory
[root@collabn2 ~]# /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read, write permissions for group.
Removing read,write,execute permissions for world.
 
Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.
 
[root@collabn2 ~]# /u01/grid/oracle/product/11.2.0/grid_1/root.sh
Running Oracle 11g root.sh script...
 
The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/grid/oracle/product/11.2.0/grid_1
 
Enter the full pathname of the local bin directory: [/usr/local/bin]: /usr/bin
   Copying dbhome to /usr/bin ...
   Copying oraenv to /usr/bin ...
   Copying coraenv to /usr/bin ...
 
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
2011-03-30 17:04:26: Parsing the host name
2011-03-30 17:04:26: Checking for super user privileges
2011-03-30 17:04:26: User has super user privileges
Using configuration parameter file: /u01/grid/oracle/product/11.2.0/grid_1/crs/install/crsconfig_params
Creating trace directory
LOCAL ADD MODE
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Adding daemon to inittab
CRS-4123: Oracle High Availability Services has been started.
ohasd is starting
CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node collabn1, number 1, and is terminating
An active cluster was found during exclusive startup, restarting to join the cluster
CRS-2672: Attempting to start 'ora.mdnsd' on 'collabn2'
CRS-2676: Start of 'ora.mdnsd' on 'collabn2' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'collabn2'
CRS-2676: Start of 'ora.gipcd' on 'collabn2' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'collabn2'
CRS-2676: Start of 'ora.gpnpd' on 'collabn2' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'collabn2'
CRS-2676: Start of 'ora.cssdmonitor' on 'collabn2' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'collabn2'
CRS-2672: Attempting to start 'ora.diskmon' on 'collabn2'
CRS-2676: Start of 'ora.diskmon' on 'collabn2' succeeded
CRS-2676: Start of 'ora.cssd' on 'collabn2' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'collabn2'
CRS-2676: Start of 'ora.ctssd' on 'collabn2' succeeded
CRS-2672: Attempting to start 'ora.drivers.acfs' on 'collabn2'
CRS-2676: Start of 'ora.drivers.acfs' on 'collabn2' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'collabn2'
CRS-2676: Start of 'ora.asm' on 'collabn2' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'collabn2'
CRS-2676: Start of 'ora.crsd' on 'collabn2' succeeded
CRS-2672: Attempting to start 'ora.evmd' on 'collabn2'
CRS-2676: Start of 'ora.evmd' on 'collabn2' succeeded
 
collabn2     2011/03/30 17:12:32     /u01/grid/oracle/product/11.2.0/grid_1/cdata/collabn2/backup_20110330_171232.olr
Preparing packages for installation...
cvuqdisk-1.0.7-1
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
Updating inventory properties for clusterware
Starting Oracle Universal Installer...
 
Checking swap space: must be greater than 500 MB.   Actual 1205 MB    Passed
The inventory pointer is located at /etc/oraInst.loc
The inventory is located at /u01/app/oraInventory
'UpdateNodeList' was successful.

After running both scripts, return to the installer window and click OK to continue running configuration assistants.
The Cluster Verification Utility will fail because NTP is not running. If you want to, check the error message at the very end of the logfile. Then click OK to close the messagebox and click NEXT to continue.
You should now see the final screen! Click CLOSE to exit the installer.