RAC Attack - Oracle Cluster Database at Home/Patching Grid and Database Software
Patching Overview
[edit | edit source]
- Upgrades to the "base version" are very complicated and always use the full Oracle installer (runInstaller). Major new features are only introduced in new base versions.
- Patch Sets are also installed with the full Oracle installer. Historically, each patchset was installed on top of the base version (top row in the illustration) by using runInstaller. However, starting with 11.2.0.2 the patch sets can be installed as a new installation without the base version. It is now recommended to perform Patch Set upgrades "out-of-place" in this manner. Sometimes new features are also included with Patch Sets (for example RAT data collection).
- PSUs are installed with opatch. They include security updates and important bug fixes. They are released quarterly and always include the latest CPU.
- CPUs are installed with opatch. CPUs include only security updates, and are also released quarterly. They cannot be applied after you have applied any PSU. (Until you upgrade to a new patch set or base version.)
Support Status and Known Issues
[edit | edit source]Before performing any installation or upgrade of Oracle, you should always check the Support Status and Known Issues for the release. Metalink note 161818.1 is always the starting point – open this note and review it. Next, follow the link for 11.2.0.X to metalink note 880782.1 and review that note. Finally, follow the link to note 880707.1 and review the known issues with Oracle 11.2.0.1 which is the version we will be using for this lab.
 | These notes have been saved as HTML files on the virtual DVD provided by the instructor. It is available in your Virtual RAC Nodes at /mnt/cdrom5. |
Practice Rolling Application
[edit | edit source]For this lab, the instructor has provided recent PSUs. PSUs and CPUs are collections of one-off patches. One-off patches can only be applied to an Oracle database in a rolling manner if they have been certified for rolling upgrades.
-
Review the installation instructions. We're going to install three patches and you can find the README files at these locations:
- /mnt/cdrom5/patch/psu6-db-12419378/12419378/README.html
- /mnt/cdrom5/patch/psu2-gi-9655006/README.html
- /mnt/cdrom5/patch/opatch-6880880/README.txt
- First we need to update the OPatch utility. Find patch 6880880 on the instructor-provided CDROM and unzip it directly into both the grid home and the database home. Before unzipping the file, backup the existing OPatch programs. $ oenv SIDs here are: grid +ASM1 RAC1 ORACLE_SID = [RAC1] ? grid $ cd $ORACLE_HOME $ tar czvf ~/gi-opatch-backup_$(date +%Y-%m-%d).tgz OPatch/ ... $ unzip -o /mnt/cdrom*/patch/opatch-6880880/p6880880_112000_LINUX.zip ... $ OPatch/opatch version Invoking OPatch 11.2.0.1.5 OPatch Version: 11.2.0.1.5 OPatch succeeded. $ oenv SIDs here are: grid +ASM1 RAC1 ORACLE_SID = [grid] ? RAC1 $ cd $ORACLE_HOME $ mv OPatch OPatch.backup $ unzip /mnt/cdrom*/patch/opatch-6880880/p6880880_112000_LINUX.zip ... $ OPatch/opatch version Invoking OPatch 11.2.0.1.5 OPatch Version: 11.2.0.1.5 OPatch succeeded.
- Repeat the previous step on node collabn2. $ ssh collabn2 ... repeat previous step ... $ exit
- This new version of OPatch requires an "OCM response file" for certain operations. Use the OCM utility to generate this file. We don't want to configure OCM; leave your username blank and confirm that "YES" you don't want to enter any account information. $ $ORACLE_HOME/OPatch/ocm/bin/emocmrsp -output /home/oracle/ocm.rsp OCM Installation Response Generator 10.3.4.0.0 - Production Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. Provide your email address to be informed of security issues, install and initiate Oracle Configuration Manager. Easier for you if you use your My Oracle Support Email address/User Name. Visit http://www.oracle.com/support/policies.html for details. Email address/User Name: <enter> You have not provided an email address for notification of security issues. Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]: Y The OCM configuration response file (/home/oracle/ocm.rsp) was successfully created.
- Find the Grid Infrastructure and Database PSUs on the instructor-provided DVD. $ cd /mnt/cdrom5/patch $ ls opatch-6880880 psu2-gi-9655006 psu6-db-12419378
- We know that these PSUs can be applied in a rolling manner, but verify this. $ for D in psu*/[0-9]*; do echo -n $D: cd $D $ORACLE_HOME/OPatch/opatch query -is_rolling_patch | grep rolling cd ../.. done psu2-gi-9655006/9654983: Patch is a rolling patch: true psu2-gi-9655006/9655006: Patch is a rolling patch: true psu6-db-12419378/12419378: Patch is a rolling patch: true
- We also know that there won't be any patch conflicts (since these are the first patches we're installing) – but verify this too. $ oenv ORACLE_SID = [RAC1] ? grid [grid]$ cd psu2-gi-9655006 psu2-gi-9655006[grid]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir ./9655006 ... Invoking prereq "checkconflictagainstohwithdetail" Prereq "checkConflictAgainstOHWithDetail" passed. [grid]$ oenv ORACLE_SID = [RAC1] ? RAC1 [RAC1]$ cd ../psu6-db-12419378/ psu6-db-12419378[RAC1]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir ./12419378 ... Invoking prereq "checkconflictagainstohwithdetail" Prereq "checkConflictAgainstOHWithDetail" passed.
-
The automated patch application process will automatically shutdown and restart all database processes on the node. However, we don't want the automatic restart – because we are applying two PSUs (one for grid and one for database). Disable the instance auto-start for node collabn1 and manually shutdown the instance for patch application.
On a production system, all active connections would need to be migrated to the other instance before doing this (for example, with services).collabn1$ srvctl disable instance -d rac -i RAC1 collabn1$ srvctl stop instance -d rac -i RAC1
-
We will use automated patch application to apply the grid infrastructure PSU. First use oenv to enter the grid environment, then switch to the root user and run opatch auto.
- Note: any database processes on this node will be automatically shutdown during patching.
- When prompted for the OCM response file path, enter /home/oracle/ocm.rsp and press enter.
- When prompted about the grid home being shared, type yes to continue.
Make sure to specify the grid home on the command line so that the patch doesn't find the database home and apply itself there in addition.$ oenv SIDs here are: grid +ASM1 RAC1 ORACLE_SID = [RAC1] ? grid The Oracle base for ORACLE_HOME=/u01/grid/oracle/product/11.2.0/grid_1 is /u01/app/oracle [grid]$ cd /mnt/cdrom5/patch/psu2-gi-9655006 psu2-gi-9655006[grid]$ su Password: racattack psu2-gi-9655006# $ORACLE_HOME/OPatch/opatch auto . -oh $ORACLE_HOME Executing /usr/bin/perl /u01/grid/oracle/product/11.2.0/grid_1/OPatch/crs/patch112.pl -patchdir . -patchn . -oh /u01/grid/oracle/product/11.2.0/grid_1 -paramfile /u01/grid/oracle/product/11.2.0/grid_1/crs/install/crsconfig_params opatch auto log file location is /u01/grid/oracle/product/11.2.0/grid_1/OPatch/crs/../../cfgtoollogs/opatchauto2011-04-01_22-44-08.log Detected Oracle Clusterware install Using configuration parameter file: /u01/grid/oracle/product/11.2.0/grid_1/crs/install/crsconfig_params OPatch is bundled with OCM, Enter the absolute OCM response file path: /home/oracle/ocm.rsp Can't change permissions of ./.: Read-only file system Unable to determine if /u01/grid/oracle/product/11.2.0/grid_1 is shared oracle home Enter 'yes' if this is not a shared home or if the prerequiste actions are performed to patch this shared home (yes/no): yes Successfully unlock /u01/grid/oracle/product/11.2.0/grid_1 patch ././9655006 apply successful for home /u01/grid/oracle/product/11.2.0/grid_1 patch ././9654983 apply successful for home /u01/grid/oracle/product/11.2.0/grid_1 ACFS-9300: ADVM/ACFS distribution files found. ACFS-9312: Existing ADVM/ACFS installation detected. ACFS-9314: Removing previous ADVM/ACFS installation. ACFS-9315: Previous ADVM/ACFS components successfully removed. ACFS-9307: Installing requested ADVM/ACFS software. ACFS-9308: Loading installed ADVM/ACFS drivers. ACFS-9321: Creating udev for ADVM/ACFS. ACFS-9323: Creating module dependencies - this may take some time. ACFS-9327: Verifying ADVM/ACFS devices. ACFS-9309: ADVM/ACFS installation correctness verified. CRS-4123: Oracle High Availability Services has been started.
-
Return to the oracle user and find the Database PSU on the instructor-provided DVD. Use oenv to enter the database environment, then run opatch apply.
- Run OPatch in local mode so that it does not attempt an automatic rolling install across the entire cluster.
- When prompted about OCM, leave your username blank and confirm with "y" that you don't want to enter any account information.
- When prompted whether the local home is ready for patching, type "y" to continue.
-
Enable and start the Oracle database instance on node collabn1. After the instance is running, stop and disable the instance on node collabn2. There should be no point at which the database is not running.
$ srvctl enable instance -d rac -i RAC1
$ srvctl start instance -d rac -i RAC1
On a production system, all active connections would need to be migrated to the other instance at this point (for example, with services).
$ srvctl disable instance -d rac -i RAC2 $ srvctl stop instance -d rac -i RAC2
-
Use automated patch application to apply the grid infrastructure PSU to collabn2.
- Note: any database processes on this node will be automatically shutdown during patching.
- Copy the OCM response file (/home/oracle/ocm.rsp) to collabn2.
- When prompted about the grid home being shared, type yes to continue.
Make sure to specify the grid home on the command line so that the patch doesn't find the database home and apply itself there in addition.$ scp /home/oracle/ocm.rsp collabn2:/home/oracle/ $ ssh collabn2 $ oenv SIDs here are: grid +ASM2 RAC2 ORACLE_SID = [RAC2] ? grid The Oracle base for ORACLE_HOME=/u01/grid/oracle/product/11.2.0/grid_1 is /u01/app/oracle [grid]$ cd /mnt/cdrom5/patch/psu2-gi-9655006 psu2-gi-9655006[grid]$ su Password: racattack psu2-gi-9655006# $ORACLE_HOME/OPatch/opatch auto . -oh $ORACLE_HOME Executing /usr/bin/perl /u01/grid/oracle/product/11.2.0/grid_1/OPatch/crs/patch112.pl -patchdir . -patchn . -oh /u01/grid/oracle/product/11.2.0/grid_1 -paramfile /u01/grid/oracle/product/11.2.0/grid_1/crs/install/crsconfig_params opatch auto log file location is /u01/grid/oracle/product/11.2.0/grid_1/OPatch/crs/../../cfgtoollogs/opatchauto2011-04-02_16-42-52.log Detected Oracle Clusterware install Using configuration parameter file: /u01/grid/oracle/product/11.2.0/grid_1/crs/install/crsconfig_params OPatch is bundled with OCM, Enter the absolute OCM response file path: /home/oracle/ocm.rsp Can't change permissions of ./.: Read-only file system Unable to determine if /u01/grid/oracle/product/11.2.0/grid_1 is shared oracle home Enter 'yes' if this is not a shared home or if the prerequiste actions are performed to patch this shared home (yes/no): yes Successfully unlock /u01/grid/oracle/product/11.2.0/grid_1 patch ././9655006 apply successful for home /u01/grid/oracle/product/11.2.0/grid_1 patch ././9654983 apply successful for home /u01/grid/oracle/product/11.2.0/grid_1 ACFS-9300: ADVM/ACFS distribution files found. ACFS-9312: Existing ADVM/ACFS installation detected. ACFS-9314: Removing previous ADVM/ACFS installation. ACFS-9315: Previous ADVM/ACFS components successfully removed. ACFS-9307: Installing requested ADVM/ACFS software. ACFS-9308: Loading installed ADVM/ACFS drivers. ACFS-9321: Creating udev for ADVM/ACFS. ACFS-9323: Creating module dependencies - this may take some time. ACFS-9327: Verifying ADVM/ACFS devices. ACFS-9309: ADVM/ACFS installation correctness verified. CRS-4123: Oracle High Availability Services has been started.
-
Return to the oracle user and find the Database PSU on the instructor-provided DVD. Use oenv to enter the database environment, then run opatch apply.
- Run OPatch in local mode so that it does not attempt an automatic rolling install across the entire cluster.
- When prompted about OCM, leave your username blank and confirm with "y" that you don't want to enter any account information.
- When prompted whether the local home is ready for patching, type "y" to continue.
- Enable and restart the Oracle database instance on node collabn2. $ srvctl enable instance -d rac -i RAC2 $ srvctl start instance -d rac -i RAC2 $ srvctl status database -d rac Instance RAC1 is running on node collabn1 Instance RAC2 is running on node collabn2
- Run the catbundle.sql script. $ ss SQL> @?/rdbms/admin/catbundle.sql psu apply $ less /u01/app/oracle/product/11.2.0/db_1/cfgtoollogs/catbundle/ catbundle_PSU_RAC_APPLY_2011Apr02_17_28_19.log
- Optional: if you want more practice working with patches, then try rolling back the database PSU and then try applying it in automated rolling mode (without local flag) or in the “minimum downtime” mode.
