RAC Attack - Oracle Cluster Database at Home/Setup ASMLIB. From Wikibooks, open books for an open world - Oracle Cluster Database at Home. [root@collabn1 ~]# /etc/init.d/oracleasm createdisk data /dev/sdb1 Marking disk 'data' as an ASM disk: [ OK ] [root@collabn1 ~]#. Oracleasm createdisk VOL1 /dev/sda1. Use the createdisk option to mark a disk device for use with the Oracle Automatic Storage Management library driver and give it a name, where labelname is the name you want to use to mark the device, and devicepath is the path to the device: oracleasm createdisk labelname devicepath.
This appendix provides instructions to complete configuration tasks manually that Cluster Verification Utility (CVU) and Oracle Universal Installer (OUI) normally complete during installation using Fixup scripts. Use this appendix as a guide if you cannot use Fixup scripts.
This appendix contains the following information:
F.1 Configuring SSH Manually on All Cluster Nodes
Passwordless SSH configuration is a mandatory installation requirement. SSH is used during installation to configure cluster member nodes, and SSH is used after installation by configuration assistants, Oracle Enterprise Manager, Opatch, and other features.
Automatic Passwordless SSH configuration using OUI creates RSA encryption keys on all nodes of the cluster. If you have system restrictions that require you to set up SSH manually, such as using DSA keys, then use this procedure as a guide to set up passwordless SSH.
In the examples that follow, the Oracle software owner listed is the
grid
user.If SSH is not available, then OUI attempts to use rsh and rcp instead. However, these services are disabled by default on most Linux systems.
Note:
The supported version of SSH for Linux distributions is OpenSSH.This section contains the following:
F.1.1 Checking Existing SSH Configuration on the System
To determine if SSH is running, enter the following command:
If SSH is running, then the response to this command is one or more process ID numbers. In the home directory of the installation software owner (
grid
, oracle
), use the command ls -al
to ensure that the .ssh
directory is owned and writable only by the user.You need either an RSA or a DSA key for the SSH protocol. RSA is used with the SSH 1.5 protocol, while DSA is the default for the SSH 2.0 protocol. With OpenSSH, you can use either RSA or DSA. The instructions that follow are for SSH1. If you have an SSH2 installation, and you cannot use SSH1, then refer to your SSH distribution documentation to configure SSH1 compatibility or to configure SSH2 with DSA.
F.1.2 Configuring SSH on Cluster Nodes
To configure SSH, you must first create RSA or DSA keys on each cluster node, and then copy all the keys generated on all cluster node members into an authorized keys file that is identical on each node. Note that the SSH files must be readable only by
root
and by the software installation user (oracle
, grid
), as SSH ignores a private key file if it is accessible by others. In the examples that follow, the DSA key is used.You must configure SSH separately for each Oracle software installation owner that you intend to use for installation.
To configure SSH, complete the following:
F.1.2.1 Create SSH Directory, and Create SSH Keys On Each Node
Complete the following steps on each node:
- Log in as the software owner (in this example, the
grid
user). - To ensure that you are logged in as
grid
, and to verify that the user ID matches the expected user ID you have assigned to thegrid
user, enter the commandsid
andid grid
. Ensure that Oracle user group and user and the user terminal window process you are using have group and user IDs are identical. For example: - If necessary, create the
.ssh
directory in thegrid
user's home directory, and set permissions on it to ensure that only theoracle
user has read and write permissions:Note:SSH configuration will fail if the permissions are not set to 700. - Enter the following command:At the prompts, accept the default location for the key file (press Enter).Note:SSH with passphrase is not supported for Oracle Clusterware 11g Release 2 and later releases.This command writes the DSA public key to the
~/.ssh/id_dsa.pub
file and the private key to the~/.ssh/id_dsa
file.Never distribute the private key to anyone not authorized to perform Oracle software installations. - Repeat steps 1 through 4 on each node that you intend to make a member of the cluster, using the DSA key.
F.1.2.2 Add All Keys to a Common authorized_keys File
Complete the following steps:
- On the local node, change directories to the
.ssh
directory in the Oracle Grid Infrastructure owner's home directory (typically, eithergrid
ororacle
).Then, add the DSA key to theauthorized_keys
file using the following commands:In the SSH directory, you should see theid_dsa.pub
keys that you have created, and the fileauthorized_keys
. - On the local node, use SCP (Secure Copy) or SFTP (Secure FTP) to copy the
authorized_keys
file to theoracle
user.ssh
directory on a remote node. The following example is with SCP, on a node called node2, with the Oracle Grid Infrastructure ownergrid
, where thegrid
user path is/home/grid
:You are prompted to accept a DSA key. Enter Yes, and you see that the node you are copying to is added to theknown_hosts
file.When prompted, provide the password for the Grid user, which should be the same on all nodes in the cluster. Theauthorized_keys
file is copied to the remote node.Your output should be similar to the following, wherexxx
represents parts of a valid IP address: - Using SSH, log in to the node where you copied the
authorized_keys
file. Then change to the.ssh
directory, and using thecat
command, add the DSA keys for the second node to theauthorized_keys
file, clicking Enter when you are prompted for a password, so that passwordless SSH is set up:Repeat steps 2 and 3 from each node to each other member node in the cluster.When you have added keys from each cluster node member to theauthorized_keys
file on the last node you want to have as a cluster node member, then usescp
to copy theauthorized_keys
file with the keys from all nodes back to each cluster node member, overwriting the existing version on the other nodes.To confirm that you have all nodes in theauthorized_keys
file, enter the commandmore authorized_keys
, and determine if there is a DSA key for each member node. The file lists the type of key (ssh-dsa
), followed by the key, and then followed by the user and server. For example:Note:Thegrid
user's/.ssh/authorized_keys
file on every node must contain the contents from all of the/.ssh/id_dsa.pub
files that you generated on all cluster nodes.
F.1.3 Enabling SSH User Equivalency on Cluster Nodes
After you have copied the
authorized_keys
file that contains all keys to each node in the cluster, complete the following procedure, in the order listed. In this example, the Oracle Grid Infrastructure software owner is named grid
:- On the system where you want to run OUI, log in as the
grid
user. - Use the following command syntax, where
hostname1
,hostname2
, and so on, are the public host names (alias and fully qualified domain name) of nodes in the cluster to run SSH from the local node to each node, including from the local node to itself, and from each node to each other node:For example:At the end of this process, the public host name for each member node should be registered in theknown_hosts
file for all other cluster nodes.If you are using a remote client to connect to the local node, and you see a message similar to 'Warning: No xauth data; using fake authentication data for X11 forwarding,' then this means that your authorized keys file is configured correctly, but your SSH configuration has X11 forwarding enabled. To correct this issue, proceed to Section 6.2.4, 'Setting Remote Display and X11 Forwarding Configuration.' - Repeat step 2 on each cluster node member.
If you have configured SSH correctly, then you can now use the
ssh
or scp
commands without being prompted for a password. For example:If any node prompts for a password, then verify that the
~/.ssh/authorized_keys
file on that node contains the correct public keys, and that you have created an Oracle software owner with identical group membership and IDs.F.2 Configuring Kernel Parameters
This section contains the following:
Note:
The kernel parameter and shell limit values shown in the following section are recommended values only. For production database systems, Oracle recommends that you tune these values to optimize the performance of the system. Refer to your operating system documentation for more information about tuning kernel parameters.F.2.1 Minimum Parameter Settings for Installation
During installation, or when you run the Cluster Verification Utility (cluvfy) with the flag
-fixup
, a fixup script is generated. This script updates required kernel parameters if necessary to minimum values.If you cannot use the fixup scripts, then review Table F-1 to set values manually:
Table F-1 Minimum Operating System Parameter Settings for Installation on Linux
Parameter | Value | File |
---|---|---|
semmsl semmns semopm semmni | 250 32000 100 128 | /proc/sys/kernel/sem |
shmall | 40 percent of the size of physical memory in pages. If the server supports multiple databases, or uses a large SGA, then set this parameter to a value that is equal to the total amount of shared memory, in 4K pages, that the system can use at one time. | /proc/sys/kernel/shmall |
shmmax | Half the size of physical memory in bytes See My Oracle Support Note 567506.1 for additional information about configuring shmmax . | /proc/sys/kernel/shmmax |
shmmni | 4096 | /proc/sys/kernel/shmmni |
file-max | 6815744 | /proc/sys/fs/file-max |
aio-max-nr | 1048576 Note: This value limits concurrent outstanding requests and should be set to avoid I/O subsystem failures. | /proc/sys/fs/aio-max-nr |
ip_local_port_range | Minimum: 9000 Maximum: 65500 | /proc/sys/net/ipv4/ip_local_port_range |
rmem_default | 262144 | /proc/sys/net/core/rmem_default |
rmem_max | 4194304 | /proc/sys/net/core/rmem_max |
wmem_default | 262144 | /proc/sys/net/core/wmem_default |
wmem_max | 1048576 | /proc/sys/net/core/wmem_max |
panic_on_oops | 1 | /proc/sys/kernel/panic_on_oops |
Note:
If you intend to install Oracle Databases or Oracle RAC databases on the cluster, be aware that the size of the /dev/shm
mount area on each server must be greater than the system global area (SGA) and the program global area (PGA) of the databases on the servers. Review expected SGA and PGA sizes with database administrators to ensure that you do not have to increase /dev/shm
after databases are installed on the cluster.F.2.2 Additional Parameter and Kernel Settings for SUSE Linux Enterprise Server
On SUSE Linux Enterprise Server systems only, complete the following steps as needed:
- Enter the following command to cause the system to read the
/etc/sysctl.conf
file when it restarts: - Enter the GID of the
oinstall
group as the value for the parameter/proc/sys/vm/hugetlb_shm_group
. Doing this grants members ofoinstall
a group permission to create shared memory segments.For example, where the oinstall group GID is 1000:After running this command, usevi
to add the following text to/etc/sysctl.conf
, and enable theboot.sysctl
script to run on system restart:Note:Only one group can be defined as thevm.hugetlb_shm_group
. - Repeat steps 1 through 3 on all other nodes in the cluster.
F.3Setting UDP and TCP Kernel Parameters Manually
If you do not use a Fixup script or CVU to set ephemeral ports, then set TCP/IP ephemeral port range parameters manually to provide enough ephemeral ports for the anticipated server workload. Ensure that the lower range is set to at least 9000 or higher, to avoid Well Known ports, and to avoid ports in the Registered Ports range commonly used by Oracle and other server ports. Set the port range high enough to avoid reserved ports for any applications you may intend to use. If the lower value of the range you have is greater than 9000, and the range is large enough for your anticipated workload, then you can ignore OUI warnings regarding the ephemeral port range.
For example, with IPv4, use the following command to check your current range for ephemeral ports:
In the preceding example, the lowest port (32768) and the highest port (61000) are set to the default range.
If necessary, update the UDP and TCP ephemeral port range to a range high enough for anticipated system workloads, and to ensure that the ephemeral port range starts at 9000 and above. For example:
Oracle recommends that you make these settings permanent. For example, as
root
, use a text editor to open /etc/sysctl.conf
, and add or change to the following: net.ipv4.ip_local_port_range = 9000 65500
, and then restart the network (# /etc/rc.d/init.d/network restart
). Refer to your Linux distribution system administration documentation for detailed information about how to automate this ephemeral port range alteration on system restarts.F.4 Configuring Storage Paths and Disk Devices
For persistent device naming, you can configure ASMLIB or set
udev
rules.This section consists of the following:
F.4.1 Configuring Storage Device Path Persistence Using Oracle ASMLIB
Oracle recommends that you use Oracle ASM Filter Driver (ASMFD) to maintain device persistence. However, you can choose to use ASMLIB for device persistence.
Review the following section to configure Oracle ASMLIB:
Note:
Oracle ASMLIB is not supported on IBM:Linux on System z.F.4.1.1 About Oracle ASM with Oracle ASMLIB
The Oracle Automatic Storage Management (Oracle ASM) library driver (ASMLIB) simplifies the configuration and management of block disk devices by eliminating the need to rebind block disk devices used with Oracle ASM each time the system is restarted.
With ASMLIB, you define the range of disks you want to have made available as Oracle ASM disks. ASMLIB maintains permissions and disk labels that are persistent on the storage device, so that label is available even after an operating system upgrade. You can update storage paths on all cluster member nodes by running one
oracleasm
command on each node, without the need to modify the udev
file manually to provide permissions and path persistence.Note:
If you configure disks using ASMLIB, then you must change the disk discovery string to ORCL:*. If the disk string is set to ORCL:*, or is left empty ('), then the installer discovers these disks.F.4.1.2 Configuring Oracle ASMLIB to Maintain Block Devices
To use the Oracle Automatic Storage Management Library Driver (ASMLIB) to configure Oracle ASM devices, complete the following tasks.
Note:
To create a database during the installation using the Oracle ASM library driver, you must choose an installation method that runs ASMCA in interactive mode. You must also change the default disk discovery string to ORCL:*.
F.4.1.2.1 Installing and Configuring the Oracle ASM Library Driver Software
ASMLIB is already included with Oracle Linux packages, and with SUSE Linux Enterprise Server. If you are a member of the Unbreakable Linux Network, then you can install the ASMLIB RPMs by subscribing to the Oracle Linux channel, and using
yum
to retrieve the most current package for your system and kernel. For additional information, see the following URL:To install and configure the ASMLIB driver software manually, follow these steps:
- Enter the following command to determine the kernel version and architecture of the system:
- Download the required ASMLIB packages from the Oracle Technology Network website:Note:You must install
oracleasm-support
package version 2.0.1 or later to use ASMLIB on Red Hat Enterprise Linux 5 Advanced Server. ASMLIB is already included with SUSE Linux Enterprise Server distributions.See Also:My Oracle Support Note 1089399.1 for information about ASMLIB support with Red Hat distributions: - Switch user to the
root
user: - Install the following packages in sequence, where
version
is the version of the ASMLIB driver,arch
is the system architecture, andkernel
is the version of the kernel that you are using:Enter a command similar to the following to install the packages:For example, if you are using the Red Hat Enterprise Linux 5 AS kernel on an AMD64 system, then enter a command similar to the following: - Enter the following command to run the
oracleasm
initialization script with theconfigure
option:Note:Theoracleasm
command in/usr/sbin
is the command you should use. The/etc/init.d
path is not deprecated, but theoracleasm
binary in that path is now used typically for internal commands. - Enter the following information in response to the prompts that the script displays:Table F-2 ORACLEASM Configure Prompts and Responses
Prompt Suggested Response Default user to own the driver interface:Standard groups and users configuration: Specify the Oracle software owner user (for example,oracle
).Job role separation groups and users configuration: Specify the Oracle Grid Infrastructure software owner user (for example,grid
).Default group to own the driver interface:Standard groups and users configuration: Specify the OSDBA group for the database (for example,dba
).Job role separation groups and users configuration: Specify the OSASM group for storage administration (for example,asmadmin
).Start Oracle ASM Library driver on boot (y/n):Entery
to start the Oracle Automatic Storage Management library driver when the system starts.Scan for Oracle ASM disks on boot (y/n)Entery
to scan for Oracle ASM disks when the system starts.The script completes the following tasks:- Creates the
/etc/sysconfig/oracleasm
configuration file - Creates the
/dev/oracleasm
mount point - Mounts the ASMLIB driver file systemNote:The ASMLIB driver file system is not a regular file system. It is used only by the Oracle ASM library to communicate with the Oracle ASM driver.
- Enter the following command to load the
oracleasm
kernel module: - Repeat this procedure on all nodes in the cluster where you want to install Oracle RAC.
F.4.1.2.2 Configuring Disk Devices to Use Oracle ASM Library Driver on x86 Systems
To configure the disk devices to use in an Oracle ASM disk group, follow these steps:
- If you intend to use IDE, SCSI, or RAID devices in the Oracle ASM disk group, then follow these steps:
- If necessary, install or configure the shared disk devices that you intend to use for the disk group and restart the system.
- Enter the following command to identify the device name for the disks to use, enter the following command:Depending on the type of disk, the device name can vary. Table F-3 describes some types of disk paths:Table F-3 Types of Linux Storage Disk Paths
Disk Type Device Name Format Description IDE diskIn this example,x
is a letter that identifies the IDE disk andn
is the partition number. For example,/dev/hda
is the first disk on the first IDE bus.SCSI diskIn this example,x
is a letter that identifies the SCSI disk andn
is the partition number. For example,/dev/sda
is the first disk on the first SCSI bus.RAID diskDepending on the RAID controller, RAID devices can have different device names. In the examples shown,x
is a number that identifies the controller,y
is a number that identifies the disk, andz
is a number that identifies the partition. For example,/dev/ida/c0d1
is the second logical drive on the first controller.To include devices in a disk group, you can specify either whole-drive device names or partition device names.Note:Oracle recommends that you create a single whole-disk partition on each disk. - Use either
fdisk
orparted
to create a single whole-disk partition on the disk devices.
- Enter a command similar to the following to mark a disk as an Oracle ASM disk:In this example,
DISK1
is the name you assign to the disk.Note:The disk names that you specify can contain uppercase letters, numbers, and the underscore character. They must start with an uppercase letter.If you are using a multi-pathing disk driver with Oracle ASM, then make sure that you specify the correct logical device name for the disk. - To make the disk available on the other nodes in the cluster, enter the following command as
root
on each node:This command identifies shared disks attached to the node that are marked as Oracle ASM disks.
F.4.1.2.3 Configuring Disk Devices to Use ASM Library Driver on IBM zSeries Systems
- If you formatted the DASD with the compatible disk layout, then enter a command similar to the following to create a single whole-disk partition on the device:
- Enter a command similar to the following to mark a disk as an ASM disk:In this example,
DISK1
is a name that you want to assign to the disk.Note:The disk names that you specify can contain uppercase letters, numbers, and the underscore character. They must start with an uppercase letter.If you are using a multi-pathing disk driver with ASM, then make sure that you specify the correct logical device name for the disk. - To make the disk available on the other cluster nodes, enter the following command as root on each node:This command identifies shared disks attached to the node that are marked as ASM disks.
F.4.1.2.4 Administering the Oracle ASM Library Driver and Disks
To administer the Oracle Automatic Storage Management library driver (ASMLIB) and disks, use the
/usr/sbin/oracleasm
initialization script with different options, as described in Table F-4:Table F-4 Disk Management Tasks Using ORACLEASM
Task | Command Example | Description |
---|---|---|
Configure or reconfigure ASMLIB | Use the configure option to reconfigure the Oracle Automatic Storage Management library driver, if necessary.To see command options, enter oracleasm configure without the -i flag. | |
Change system restart load options for ASMLIB | Options are disable and enable .Use the disable and enable options to change the actions of the Oracle Automatic Storage Management library driver when the system starts. The enable option causes the Oracle Automatic Storage Management library driver to load when the system starts | |
Load or unload ASMLIB without restarting the system | Options are start , stop and restart .Use the start , stop , and restart options to load or unload the Oracle Automatic Storage Management library driver without restarting the system. | |
Mark a disk for use with ASMLIB | Use the createdisk option to mark a disk device for use with the Oracle Automatic Storage Management library driver and give it a name, where labelname is the name you want to use to mark the device, and devicepath is the path to the device:oracleasm createdisk labelnamedevicepath | |
Unmark a named disk device | Use the deletedisk option to unmark a named disk device, where diskname is the name of the disk:oracleasm deletedisk diskname Caution: Do not use this command to unmark disks that are being used by an Oracle Automatic Storage Management disk group. You must delete the disk from the Oracle Automatic Storage Management disk group before you unmark it. | |
Determine if ASMLIB is using a disk device | Use the querydisk option to determine if a disk device or disk name is being used by the Oracle Automatic Storage Management library driver, where diskname_devicename is the name of the disk or device that you want to query:oracleasm querydisk diskname_devicename | |
List Oracle ASMLIB disks | Use the listdisks option to list the disk names of marked ASMLIB disks. | |
Identify disks marked as ASMLIB disks | Use the scandisks option to enable cluster nodes to identify which shared disks have been marked as ASMLIB disks on another node. | |
Rename ASMLIB disks | Use the renamedisk option to change the label of an Oracle ASM library driver disk or device by using the following syntax, where manager specifies the manager device, label_device specifies the disk you intend to rename, as specified either by OracleASM label name or by the device path, and new_label specifies the new label you want to use for the disk:oracleasm renamedisk [-l manager] [-v ] label_devicenew_labelUse the -v flag to provide a verbose output for debugging.Caution: You must ensure that all Oracle Database and Oracle ASM instances have ceased using the disk before you relabel the disk. If you do not do this, then you may lose data. |
F.4.1.3 Configuring Oracle ASMLIB for Multipath Disks
Additional configuration is required to use the Oracle Automatic Storage Management library Driver (ASMLIB) with third party vendor multipath disks.
See Also:
My Oracle Support site for updates to supported storage options:F.4.1.3.1 About Using Oracle ASM with Multipath Disks
Oracle ASM requires that each disk is uniquely identified. If the same disk appears under multiple paths, then it causes errors. In a multipath disk configuration, the same disk can appear three times:
- The initial path to the disk
- The second path to the disk
- The multipath disk access point
For example: If you have one local disk,
/dev/sda
, and one disk attached with external storage, then your server shows two connections, or paths, to that external storage. The Linux SCSI driver shows both paths. They appear as /dev/sdb
and /dev/sdc
. The system may access either /dev/sdb
or /dev/sdc
, but the access is to the same disk.If you enable multipathing, then you have a multipath disk (for example,
/dev/multipatha
), which can access both /dev/sdb
and /dev sdc
; any I/O to multipatha
can use either the sdb
or sdc
path. If a system is using the /dev/sdb
path, and that cable is unplugged, then the system shows an error. But the multipath disk will switch from the /dev/sdb
path to the /dev/sdc
path.Most system software is unaware of multipath configurations. They can use any paths (
sdb
, sdc
or multipatha
). ASMLIB also is unaware of multipath configurations.By default, ASMLIB recognizes the first disk path that Linux reports to it, but because it imprints an identity on that disk, it recognizes that disk only under one path. Depending on your storage driver, it may recognize the multipath disk, or it may recognize one of the single disk paths.
Instead of relying on the default, you should configure Oracle ASM to recognize the multipath disk.
F.4.1.3.2 Disk Scan Ordering
The ASMLIB configuration file is located in the path
/etc/sysconfig/oracleasm
. It contains all the startup configuration you specified with the command /etc/init.d/oracleasm configure
. That command cannot configure scan ordering.The configuration file contains many configuration variables. The
ORACLEASM_SCANORDER
variable specifies disks to be scanned first. The ORACLEASM_SCANEXCLUDE
variable specifies the disks that are to be ignored.Configure values for
ORACLEASM_SCANORDER
using space-delimited prefix strings. A prefix string is the common string associated with a type of disk. For example, if you use the prefix string sd
, then this string matches all SCSI devices, including /dev/sda
, /dev/sdb
, /dev/sdc
and so on. Note that these are not globs. They do not use wild cards. They are simple prefixes. Also note that the path is not a part of the prefix. For example, the /dev/
path is not part of the prefix for SCSI disks that are in the path /dev/sd
*.For Oracle Linux and Red Hat Enterprise Linux version 5, when scanning, the kernel sees the devices as
/dev/mapper/
XXX
entries. By default, the device file naming scheme udev
creates the /dev/mapper/
XXX
names for human readability. Any configuration using ORACLEASM_SCANORDER
should use the /dev/mapper/
XXX
entries.F.4.1.3.3 Configuring Disk Scan Ordering to Select Multipath Disks
To configure ASMLIB to select multipath disks first, complete the following procedure:
- Using a text editor, open the ASMLIB configuration file
/etc/sysconfig/oracleasm
. - Edit the ORACLEASM_SCANORDER variable to provide the prefix path of the multipath disks. For example, if the multipath disks use the prefix
multipath
(/dev/mapper/multipatha
,/dev/mapper/multipathb
and so on), and the multipath disks mount SCSI disks, then provide a prefix path similar to the following: - Save the file.
When you have completed this procedure, then when ASMLIB scans disks, it first scans all disks with the prefix string multipath, and labels these disks as Oracle ASM disks using the
/dev/mapper/multipath
X
value. It then scans all disks with the prefix string sd
. However, because ASMLIB recognizes that these disks have already been labeled with the /dev/mapper/multipath
string values, it ignores these disks. After scanning for the prefix strings multipath
and sd
, Oracle ASM then scans for any other disks that do not match the scan order.In the example in step 2, the key word multipath is actually the alias for multipath devices configured in
/etc/multipath.conf
under the multipaths
section. For example:The default device name is in the format /dev/mapper/mpath* (or a similar path).
F.4.1.3.4 Configuring Disk Order Scan to Exclude Single Path Disks
To configure ASMLIB to exclude particular single path disks, complete the following procedure:
- Using a text editor, open the ASMLIB configuration file
/etc/sysconfig/oracleasm
. - Edit the
ORACLEASM_SCANEXCLUDE
variable to provide the prefix path of the single path disks. For example, if you want to exclude the single path disks/dev sdb
and/dev/sdc
, then provide a prefix path similar to the following: - Save the file.
When you have completed this procedure, then when ASMLIB scans disks, it scans all disks except for the disks with the
sdb
and sdc
prefixes, so that it ignores /dev/sdb
and /dev/sdc
. It does not ignore other SCSI disks, nor multipath disks. If you have a multipath disk (for example, /dev/multipatha
), which accesses both /dev/sdb
and /dev/sdc
, but you have configured ASMLIB to ignore sdb
and sdc
, then ASMLIB ignores these disks and instead marks only the multipath disk as an Oracle ASM disk.F.4.1.4 Deinstalling Oracle ASMLIB
If you have Oracle ASMLIB installed but do not use it for storage persistence, you can deinstall it in rolling mode, one node at a time, as follows:
- Login as
root
. - Stop Oracle ASM and any running database instance on the node:To stop the last Oracle Flex ASM instance on the node, stop the Oracle Clusterware stack:
- Stop Oracle ASMLIB:
- Remove
oracleasm
library and tools RPMs: - Remove any
oracleasm
kernel driver RPMs provided by vendors: - Check if any
oracleasm
RPMs remain: - If any
oracleasm
configuration files remain, remove them:Oracle ASMLIB and the associated RPMs are removed. - Start the Oracle Clusterware stack. Optionally, you can install and configure Oracle ASM Filter Driver (Oracle ASMFD) before starting the Oracle Clusterware stack.
See Also:
Oracle Automatic Storage Management Administrator's Guide for more information about configuring storage device path persistence using Oracle ASM Filter DriverF.4.2 Configuring Disk Devices Manually for Oracle ASM
This section contains the following information about preparing disk devices for use by Oracle ASM:
Note:
The operation of udev
depends on the Linux version, vendor, and storage configuration.F.4.2.1 About Device File Names and Ownership for Linux
By default, the device file naming scheme
udev
dynamically creates device file names when the server is started, and assigns ownership of them to root
. If udev
applies default settings, then it changes device file names and owners for voting files or Oracle Cluster Registry partitions, making them inaccessible when the server is restarted. For example, a voting file on a device named /dev/sdd
owned by the user grid
may be on a device named /dev/sdf
owned by root
after restarting the server.If you use ASMFD, then you do not need to ensure permissions and device path persistency in
udev
.If you do not use ASMFD, then you must create a custom rules file. When
udev
is started, it sequentially carries out rules (configuration directives) defined in rules files. These files are in the path /etc/udev/rules.d/
. Rules files are read in lexical order. For example, rules in the file 10-wacom.rules
are parsed and carried out before rules in the rules file 90-ib.rules
.When specifying the device information in the UDEV rules file, ensure that the OWNER, GROUP and MODE are specified before all other characteristics in the order shown. For example, if you want to include the characteristic ACTION on the UDEV line, then specify ACTION after OWNER, GROUP, and MODE.
Where rules files describe the same devices, on the supported Linux kernel versions, the last file read is the one that is applied.
F.4.2.2 Configuring a Permissions File for Disk Devices for Oracle ASM
To configure a permissions file for disk devices for Oracle ASM, complete the following tasks:
- To obtain information about existing block devices, run the command
scsi_id
(/sbin/scsi_id
) on storage devices from one cluster node to obtain their unique device identifiers. When running thescsi_id
command with the-s
argument, the device path and name passed should be that relative to thesysfs
directory/sys
(for example,/block/device
) when referring to/sys/block/device
. For example:Record the unique SCSI identifiers of clusterware devices, so you can provide them when required.Note:The commandscsi_id
should return the same device identifier value for a given device, regardless of which node the command is run from. - Configure SCSI devices as trusted devices (white listed), by editing the
/etc/scsi_id.config
file and addingoptions=-g
to the file. For example: - Using a text editor, create a UDEV rules file for the Oracle ASM devices, setting permissions to 0660 for the installation owner and the group whose members are administrators of the Oracle Grid Infrastructure software. For example, on Oracle Linux, to create a role-based configuration rules.d file where the installation owner is
grid
and the OSASM groupasmadmin
, enter commands similar to the following: - Copy the
rules.d
file to all other nodes on the cluster. For example: - Load updated block device partition tables on all member nodes of the cluster, using
/sbin/partprobe
devicename
. You must do this asroot
. - Run the command
udevtest
(/sbin/udevtest
) to test the UDEV rules configuration you have created. The output should indicate that the devices are available and the rules are applied as expected. For example:In the example output, note that applying the rules renames OCR device/dev/sdb1
to/dev/data1
. - Enter the command to restart the UDEV service.On Oracle Linux and Red Hat Enterprise Linux, the commands are:On SUSE Linux Enterprise Server, the command is:
F.5 Checking OCFS2 Version Manually
To check your OCFS2 version manually, enter the following commands:
Ensure that
ocfs2console
and ocfs2-tools
are at least version 1.2.7, and that the other OCFS2 components correspond to the pattern ocfs2-kernel_version-1.2.7 or greater. If you want to install Oracle RAC on a shared home, then the OCFS2 version must be 1.4.1 or greater.For information about OCFS2, refer to the following website: