Enterprise_Linux_64_X86安装Oracle11gR2_RAC详细.docx
1安装文档安装文档11111111g g g gR2R2R2R2 RACASMRACASMRACASMRACASM forforforfor OEL6.4OEL6.4OEL6.4OEL6.4V V1.01.02013 年 09 月 9 日2文档信息文档信息文挡名称作者电话邮件Oracle 11gR2 RACASM for OEL6.4 安装文档印冲13851698652文档修订记录文档修订记录编号修订内容简述修订日期修订前版本号3一 环境准备.41.1 操作系统.41.2 解决虚拟机安装 RAC 的几个问题.51.2.1 共享存储.51.2.2 安装过程中总死机重启.151.3 修改/etc/hosts 文件（所有节点）.151.4 添加组和用户（所有节点）.161.5 建目录（所有节点）.171.6 修改系统参数（所有节点）.181.7 禁用防火墙和 SELNUX（所有节点）.221.8 配置时间同步（所有节点）.241.9 检查安装包（所有节点）.251.10 配置用户环境变量（所有节点）.261.10.1 Grid 用户.261.10.2 Oracle 用户.281.11 配置用户等效性.301.11.1 Grid 用户：.301.11.2 Oracle 用户：.341.12 配置 ASM.381.12.1 安装 ASM 支持包（两个节点）.381.12.2 配置 ASM（两个节点）.391.12.3 创建 ASM 磁盘.401.13 安装 cvuqdisk 包并验证（两个节点）.401.14 验证安装前的检查：.41二 安装 CRS.502.1 安装 Grid infrastructure.502.2 检查 CRS 状态.612.2.1 资源组状态:.612.2.2 检查集群节点:.622.2.3 检查 Oracle 集群注册表(OCR):.632.2.4 检查表决磁盘:.642.2.5 检查 CRS 状态.642.3 为数据和快速恢复区创建 ASM 磁盘组:asmca.64三 安装 Oracle 软件（不建库）.65四 创建数据库.684.1 安装数据库.684.2 查看数据库进程.694.3 连接数据库.724一一 环境准备环境准备安装环境：本机 32 位 VMware Workstation。操作系统 Oracle Enterprise Linux Release 6 Update 4 for x86(32 Bit)。安装介质：p13390677_112040_LINUX_3of7.zip集群基础包p13390677_112040_LINUX_1of7.zip数据库基础包p13390677_112040_LINUX_2of7.zip数据库基础包oracleasm-support-2.1.8-1.el6.i686.rpmASM 支持包oracleasmlib-2.0.4-1.el6.i686.rpmASM lib 安装包cvuqdisk-1.0.9-1.rpm共享存储IP 分配：主机名主机 IPPRIVVIPrac1192.168.255.101192.168.1.103192.168.1.105rac2192.168.255.102192.168.1.104192.168.1.106sacnIP192.168.255.107安装目录：Oracle 软件/u01/app/oracle/11.2.0/db_1grid 软件/u01/app/grid/11.2.0/CRS 软件+CRS/rac-cluster1.1 操作系统操作系统本安装文档，操作系统为 Oracle Enterprise Linux Release 6 Update 4 for x86(32 Bit)。(以 rac1 节点为例)rootrac1#lsb_release-aLSBVersion:base-4.0-ia32:base-4.0-noarch:core-4.0-ia32:core-4.0-noarch:graphics-4.0-ia32:graphics-4.0-noarch:printing-4.0-ia32:printing-4.0-noarchDistributor ID:OracleServerDescription:Oracle Linux Server release 6.4Release:6.4Codename:n/arootrac1#cat/etc/redhat-releaseRed Hat Enterprise Linux Server release 6.4(Santiago)rootrac1#uname-aLinux rac1 2.6.39-400.17.1.el6uek.i686#1 SMP Fri Feb 22 18:17:46 PST 2013 i686 i686 i386GNU/Linuxrootrac1#grep MemTotal/proc/meminfoMemTotal:1030172 kB5rootrac1#grep SwapTotal/proc/meminfoSwapTotal:2097148 kB1.2 解决虚拟机安装解决虚拟机安装 RAC 的几个问题的几个问题1.2.11.2.1 共享存储共享存储在一号虚拟机创建新的磁盘，打开虚拟机设置:点添加按钮：6选择硬盘，点继续：7选择创建一个新的虚拟磁盘：8选择 SCSI，勾选独立模式，然后继续：9输入磁盘大小，勾选立即分配所有磁盘空间【如果不勾选，安装 CRS 的时候容易报错超时】：10选择文件路径及名称，点完成。1112选中刚才创建的硬盘，点击高级按钮：13选择虚拟设备节点，选取 SCSI 1:0：14点击确定完成。使用记事本编辑.wmx 文件，添加虚拟机参数：disk.locking=falsediskLib.dataCacheMaxSize=0diskLib.dataCacheMaxReadAheadSize=0diskLib.DataCacheMinReadAheadSize=0diskLib.dataCachePageSize=4096diskLib.maxUnsyncedWrites=0tools.syncTime=TRUEscsi1:0.deviceType=disk【刚才添加的盘对应的节点】scsi1:1.deviceType=disk【每添加一个新的共享磁盘，这边就加一句】scsi1:2.deviceType=disk在二号虚拟机里面添加刚才创建好的磁盘15添加好之后点高级，也选择 SCSI 1：0使用记事本编辑.wmx 文件，添加虚拟机参数：disk.locking=falsediskLib.dataCacheMaxSize=0diskLib.dataCacheMaxReadAheadSize=0diskLib.DataCacheMinReadAheadSize=0diskLib.dataCachePageSize=4096diskLib.maxUnsyncedWrites=0tools.syncTime=TRUEscsi1:0.deviceType=disk【刚才添加的盘对应的节点】scsi1:1.deviceType=disk【每添加一个新的共享磁盘，这边就加一句】scsi1:2.deviceType=disk1.2.21.2.2 安装过程中总死机重启安装过程中总死机重启使用记事本编辑.wmx 文件，添加虚拟机参数：timeTracker.lazyApic=FALSE1.3 修改修改/etc/hosts 文件（所有节点）文件（所有节点）rootrac1#vi/etc/hostsrootrac1#cat/etc/hosts16127.0.0.1localhost localhost.localdomain localhost4 localhost4.localdomain4:1localhost localhost.localdomain localhost6 localhost6.localdomain6192.168.1.128udbs01192.168.1.133udbs02192.168.1.103udbs01-priv192.168.1.104udbs02-priv192.168.1.129udbs01-vip192.168.1.134udbs02-vip192.168.1.107udbs01-scanrootrac2#vi/etc/hostsrootrac2#cat/etc/hosts127.0.0.1localhost localhost.localdomain localhost4 localhost4.localdomain4:1localhost localhost.localdomain localhost6 localhost6.localdomain6192.168.1.128udbs01192.168.1.133udbs02192.168.1.103udbs01-priv192.168.1.104udbs02-priv192.168.1.129udbs01-vip192.168.1.134udbs02-vip192.168.1.107udbs01-scan1.4 添加组和用户（所有节点）添加组和用户（所有节点）rootrac1#groupadd-g 500 oinstallrootrac1#groupadd-g 501 dbarootrac1#groupadd-g 502 operrootrac1#groupadd-g 503 asmadminrootrac1#groupadd-g 504 asmoperrootrac1#groupadd-g 505 asmdbarootrac1#useradd-g oinstall-G dba,asmdba,oper oraclerootrac1#useradd-g oinstall-G asmadmin,asmdba,asmoper,oper,dba gridrootrac1#id oracleuid=500(oracle)gid=500(oinstall)groups=500(oinstall),501(dba),502(oper),505(asmdba)rootrac1#id griduid=501(grid)gid=500(oinstall)groups=500(oinstall),501(dba),502(oper),503(asmadmin),504(asmoper),505(asmdba)rootrac1#passwd oraclerootrac1#passwd grid17rootrac2#groupadd-g 500 oinstallrootrac2#groupadd-g 501 dbarootrac2#groupadd-g 502 operrootrac2#groupadd-g 503 asmadminrootrac2#groupadd-g 504 asmoperrootrac2#groupadd-g 505 asmdbarootrac2#useradd-g oinstall-G dba,asmdba,oper oraclerootrac2#useradd-g oinstall-G asmadmin,asmdba,asmoper,oper,dba gridrootrac2#id oracleuid=500(oracle)gid=500(oinstall)groups=500(oinstall),501(dba),502(oper),505(asmdba)rootrac2#id griduid=501(grid)gid=500(oinstall)groups=500(oinstall),501(dba),502(oper),503(asmadmin),504(asmoper),505(asmdba)rootrac2#passwd oraclerootrac2#passwd grid1.5 建目录（所有节点）建目录（所有节点）rootrac1#mkdir/u01/approotrac1#chown-R grid:oinstall/u01/app/rootrac1#chmod-R 775/u01/app/rootrac1#mkdir-p/u01/app/oraInventoryrootrac1#chown-R grid:oinstall/u01/app/oraInventory/rootrac1#chmod-R 775/u01/app/oraInventory/rootrac1#mkdir-p/u01/app/gridrootrac1#mkdir-p/u01/app/oraclerootrac1#chown-R grid:oinstall/u01/app/grid/rootrac1#chown-R oracle:oinstall/u01/app/oracle/rootrac1#chmod-R 775/u01/app/grid/rootrac1#chmod-R 775/u01/app/oracle/rootrac2#mkdir/u01/approotrac2#chown-R grid:oinstall/u01/app/rootrac2#chmod-R 775/u01/app/rootrac2#mkdir-p/u01/app/oraInventoryrootrac2#chown-R grid:oinstall/u01/app/oraInventory/rootrac2#chmod-R 775/u01/app/oraInventory/rootrac2#mkdir-p/u01/app/grid18rootrac2#mkdir-p/u01/app/oraclerootrac2#chown-R grid:oinstall/u01/app/grid/rootrac2#chown-R oracle:oinstall/u01/app/oracle/rootrac2#chmod-R 775/u01/app/grid/rootrac2#chmod-R 775/u01/app/oracle/1.6 修改系统参数（所有节点）修改系统参数（所有节点）rootrac1#vi/etc/security/limits.conf#ORACLE SETTINGgridsoftnproc2047gridhardnproc16384gridsoftnofile1024gridhardnofile65536oraclesoftnproc2047oraclehardnproc16384oraclesoftnofile1024oraclehardnofile65536rootrac2 app#vi/etc/security/limits.conf#ORACLE SETTINGgridsoftnproc2047gridhardnproc16384gridsoftnofile1024gridhardnofile65536oraclesoftnproc2047oraclehardnproc16384oraclesoftnofile1024oraclehardnofile65536rootrac1#vi/etc/pam.d/login#%PAM-1.0auth user_unknown=ignore success=ok ignore=ignore default=bad pam_securetty.soauthincludesystem-authaccountrequiredpam_nologin.soaccountincludesystem-authpasswordincludesystem-auth#pam_selinux.so close should be the first session rulesessionrequiredpam_selinux.so closesessionrequiredpam_loginuid.so19sessionoptionalpam_console.so#pam_selinux.so open should only be followed by sessions to be executed in the user contextsessionrequiredpam_selinux.so opensessionrequiredpam_namespace.sosessionoptionalpam_keyinit.so force revokesessionincludesystem-auth-sessionoptionalpam_ck_connector.so#ORACLE SETTINGsessionrequiredpam_limits.so(增加)rootrac2 app#vi/etc/pam.d/login#%PAM-1.0auth user_unknown=ignore success=ok ignore=ignore default=bad pam_securetty.soauthincludesystem-authaccountrequiredpam_nologin.soaccountincludesystem-authpasswordincludesystem-auth#pam_selinux.so close should be the first session rulesessionrequiredpam_selinux.so closesessionrequiredpam_loginuid.sosessionoptionalpam_console.so#pam_selinux.so open should only be followed by sessions to be executed in the user contextsessionrequiredpam_selinux.so opensessionrequiredpam_namespace.sosessionoptionalpam_keyinit.so force revokesessionincludesystem-auth-sessionoptionalpam_ck_connector.so#ORACLE SETTINGsessionrequiredpam_limits.sorootrac1#vi/etc/sysctl.conf#Kernel sysctl configuration file for Oracle Linux#For binary values,0 is disabled,1 is enabled.See sysctl(8)and20#sysctl.conf(5)for more details.#Controls IP packet forwardingnet.ipv4.ip_forward=0#Controls source route verification#Controls the use of TCP syncookiesnet.ipv4.tcp_syncookies=1#Disable netfilter on .bridge.bridge-nf-call-ip6tables=0net.bridge.bridge-nf-call-iptables=0net.bridge.bridge-nf-call-arptables=0#Controls the default maxmimum size of a mesage queuekernel.msgmnb=65536#Controls the maximum size of a message,in byteskernel.msgmax=65536#Controls the maximum shared segment size,in byteskernel.shmmax=4294967295#Controls the maximum number of shared memory segments,in pageskernel.shmall=268435456#ORACLE SETTINGfs.aio-max-nr=1048576fs.file-max=6815744kernel.shmall=2097152kernel.shmmax=536870912kernel.shmmni=4096kernel.sem=250 32000 100 128net.ipv4.ip_local_port_range=9000 65500net.core.rmem_default=262144net.core.rmem_max=4194304net.core.wmem_default=262144net.core.wmem_max=1048586rootrac2#vi/etc/sysctl.conf#Kernel sysctl configuration file for Oracle Linux#For binary values,0 is disabled,1 is enabled.See sysctl(8)and21#sysctl.conf(5)for more details.#Controls IP packet forwardingnet.ipv4.ip_forward=0#Controls source route verificationnet.ipv4.conf.default.rp_filter=1#Do not accept source routingnet.ipv4.conf.default.accept_source_route=0#Controls the System Request debugging functionality of the kernelkernel.sysrq=0#Controls the use of TCP syncookiesnet.ipv4.tcp_syncookies=1#Disable netfilter on .bridge.bridge-nf-call-ip6tables=0net.bridge.bridge-nf-call-iptables=0net.bridge.bridge-nf-call-arptables=0#Controls the default maxmimum size of a mesage queuekernel.msgmnb=65536#Controls the maximum size of a message,in byteskernel.msgmax=65536#Controls the maximum shared segment size,in byteskernel.shmmax=4294967295#Controls the maximum number of shared memory segments,in pageskernel.shmall=268435456#ORACLE SETTINGfs.aio-max-nr=1048576fs.file-max=6815744#kernel.shmall=2097152#kernel.shmmax=536870912kernel.shmmni=4096kernel.sem=250 32000 100 128net.ipv4.ip_local_port_range=9000 65500net.core.rmem_default=26214422net.core.rmem_max=4194304net.core.wmem_default=262144net.core.wmem_max=10485861.7 禁用防火墙和禁用防火墙和 SELNUX（所有节点）（所有节点）rootrac1#service iptables status表格：filterChain INPUT(policyACCEPT)numtargetprot opt sourcedestination1ACCEPTall-0.0.0.0/00.0.0.0/0stateRELATED,ESTABLISHED2ACCEPTicmp-0.0.0.0/00.0.0.0/03ACCEPTall-0.0.0.0/00.0.0.0/04ACCEPTtcp-0.0.0.0/00.0.0.0/0state NEW tcpdpt:225REJECTall-0.0.0.0/00.0.0.0/0reject-withicmp-host-prohibitedChain FORWARD(policy ACCEPT)numtargetprot opt sourcedestination1REJECTall-0.0.0.0/00.0.0.0/0reject-withicmp-host-prohibitedChain OUTPUT(policy ACCEPT)numtargetprot opt sourcedestinationrootrac1#service iptables stopiptables：清除防火墙规则：确定iptables：将链设置为政策 ACCEPT：filter确定iptables：正在卸载模块：确定rootrac1#chkconfig iptables offrootrac1#chkconfig iptables-listiptables0:关闭1:关闭2:关闭3:关闭4:关闭5:关闭6:关闭rootrac1#vi/etc/selinux/config#This file controls the state of SELinux on the system.#SELINUX=can take one of these three values:#enforcing-SELinux security policy is enforced.#permissive-SELinux prints warnings instead of enforcing.#disabled-No SELinux policy is loaded.23SELINUX=disabled#SELINUXTYPE=can take one of these two values:#targeted-Targeted processes are protected,#mls-Multi Level Security protection.SELINUXTYPE=targetedrootrac2#service iptables status表格：filterChain INPUT(policyACCEPT)numtargetprot opt sourcedestination1ACCEPTall-0.0.0.0/00.0.0.0/0stateRELATED,ESTABLISHED2ACCEPTicmp-0.0.0.0/00.0.0.0/03ACCEPTall-0.0.0.0/00.0.0.0/04ACCEPTtcp-0.0.0.0/00.0.0.0/0state NEW tcpdpt:225REJECTall-0.0.0.0/00.0.0.0/0reject-withicmp-host-prohibitedChain FORWARD(policy ACCEPT)numtargetprot opt sourcedestination1REJECTall-0.0.0.0/00.0.0.0/0reject-withicmp-host-prohibitedChain OUTPUT(policy ACCEPT)numtargetprot opt sourcedestinationrootrac2#service iptables stopiptables：清除防火墙规则：确定iptables：将链设置为政策 ACCEPT：filter确定iptables：正在卸载模块：确定rootrac2#chkconfig iptables offrootrac2#chkconfig iptables-listiptables0:关闭1:关闭2:关闭3:关闭4:关闭5:关闭6:关闭rootrac2#vi/etc/selinux/configrootrac2#vi/etc/selinux/config#This file controls the state of SELinux on the system.#SELINUX=can take one of these three values:#enforcing-SELinux security policy is enforced.#permissive-SELinux prints warnings instead of enforcing.24#disabled-No SELinux policy is loaded.SELINUX=disabled#SELINUXTYPE=can take one of these two values:#targeted-Targeted processes are protected,#mls-Multi Level Security protection.SELINUXTYPE=targeted1.8 配置时间同步（所有节点）配置时间同步（所有节点）在11gR2 中，RAC 在安装的时候，时间同步可以用两种方式来实现：（1）NTP（2）CTSS。当安装程序发现 NTP 协议处于非活动状态时，安装集群时间同步服务将以活动模式自动进行安装并通过所有节点的时间。如果发现配置了 NTP，则以观察者模式启动集群时间同步服务，Oracle Clusterware 不会在集群中进行活动的时间同步。Oracle 集群时间同步服务(ctssd)旨在为那些 Oracle RAC 数据库无法访问 NTP 服务的组织提供服务。目前建议配置NTP。本文档采用CTSS使用集群时间同步服务在集群中提供同步服务，需要卸载网络时间协议(NTP)及其配置。要停用 NTP 服务，必须停止当前的 ntpd 服务，从初始化序列中禁用该服务，并删除 ntp.conf 文件。要在Linux 上完成这些步骤，以 root 用户身份在两个 Oracle RAC 节点上运行以下命令：rootrac1#/sbin/service ntpd stop关闭 ntpd：失败rootrac1#chkconfig ntpd offrootrac1#mv/etc/ntp.conf/etc/ntp.conf.originalrootrac1#chkconfig ntpd-listntpd0:关闭1:关闭2:关闭3:关闭4:关闭5:关闭6:关闭rootrac1#rm/var/run/ntpd.pidrm:无法删除/var/run/ntpd.pid:没有那个文件或目录rootrac2#/sbin/service ntpd stop关闭 ntpd：失败rootrac2#chkconfig ntpd offrootrac2#mv/etc/ntp.conf/etc/ntp.conf.originalrootrac2#chkconfig ntpd-list25ntpd0:关闭1:关闭2:关闭3:关闭4:关闭5:关闭6:关闭rootrac2#rm/var/run/ntpd.pidrm:无法删除/var/run/ntpd.pid:没有那个文件或目录在安装后，要确认 ctssd 处于活动状态，用grid 所有者输入以下命令：gridrac1$crsctl check ctssCRS-4701:The Cluster Time Synchronization Service is in Active mode.CRS-4702:Offset(in msec):0gridrac2$crsctl check ctssCRS-4701:The Cluster Time Synchronization Service is in Active mode.CRS-4702:Offset(in msec):01.9 检查安装包（所有节点）检查安装包（所有节点）rootrac1#rpm-q binutils compat-libcap1 compat-libstdc+gcc gcc-c+glibcglibc-devel ksh libaio libaio-devel libgcc libstdc+libstdc+-devel make sysstat unixODBCunixODBC-develbinutils-2.20.51.0.2-5.36.el6.i686compat-libcap1-1.10-1.i686package compat-libstdc+is not installedgcc-4.4.7-3.el6.i686gcc-c+-4.4.7-3.el6.i686glibc-2.12-1.107.el6.i686glibc-devel-2.12-1.107.el6.i686ksh-20100621-19.el6.i686libaio-0.3.107-10.el6.i686libaio-devel-0.3.107-10.el6.i686libgcc-4.4.7-3.el6.i686libstdc+-4.4.7-3.el6.i686libstdc+-devel-4.4.7-3.el6.i686make-3.81-20.el6.i686sysstat-9.0.4-20.el6.i686unixODBC-2.2.14-12.el6_3.i686unixODBC-devel-2.2.14-12.el6_3.i686rootrac2#rpm-q binutils compat-libcap1 compat-libstdc+gcc gcc-c+glibcglibc-devel ksh libaio libaio-devel libgcc libstdc+libstdc+-devel make sysstat unixODBCunixODBC-develbinutils-2.20.51.0.2-5.36.el6.i686compat-libcap1-1.10-1.i686package compat-libstdc+is not installedgcc-4.4.7-3.el6.i686gcc-c+-4.4.7-3.el6.i68626glibc-2.12-1.107.el6.i686glibc-devel-2.12-1.107.el6.i686ksh-20100621-19.el6.i686libaio-0.3.107-10.el6.i686libaio-devel-0.3.107-10.el6.i686libgcc-4.4.7-3.el6.i686libstdc+-4.4.7-3.el6.i686libstdc+-devel-4.4.7-3.el6.i686make-3.81-20.el6.i686sysstat-9.0.4-20.el6.i686unixODBC-2.2.14-12.el6_3.i686unixODBC-devel-2.2.14-12.el6_3.i6861.10 配置用户环境变量（所有节点）配置用户环境变量（所有节点）1.10.11.10.1 GridGrid 用户用户rootrac1#su-gridgridrac1$vi.bash_profile#.bash_profile#Get the aliases and functionsif -f/.bashrc;then./.bashrcfi#User specific environmen
