linux+oracle 11g 集群的安装

1、内存要求

 RAM 最少1G，最好2G或者2G以上

查看内存大小：

[root@localhost ~]# grep MemTotal /proc/meminfo

MemTotal: 2294584 kB

RAM和swap的大小配置关系

RAM Swap Space

Between 1 GB and 2 GB 1.5 times the size of the RAM

Between 2 GB and 16 GB Equal to the size of the RAM

More than 16 GB 16 GB

查看swap大小：

[root@localhost ~]# grep SwapTotal /proc/meminfo

SwapTotal: 4096564 kB

共享内存的大小取决于：

[root@rac1 ~]# df -h /dev/shm

文件系统 容量 已用 可用 已用% 挂载点

tmpfs 1.1G 0 1.1G 0% /dev/shm

注意：

当使用 LOCK_SGA时并且enable时， MEMORY_MAX_TARGET 和 MEMORY_TARGET 参数不能被使用，而且不能使用大页内存

2、磁盘空间的需求

对于企业版的数据库，软件需要4G，数据文件需要1.7G，所以 安装数据库至少需要5.7G的空间，但是随着数据量的增大，占用的空间会不断增加，以及数据库本身也会不断的产生日志，所以安装数据库的时候最好空间要足够

3、查看操作系统的版本

[root@localhost ~]# uname -rm

2.6.18-194.el5 x86_64

4、软件的安装 

yum install binutils-2.17.50.0.6

yum install compat-libstdc++-33-3.2.3

yum install elfutils-libelf-0.125

yum install elfutils-libelf-devel-0.125

yum install elfutils-libelf-devel-static-0.125

yum install gcc-4.1.2

yum install gcc-c++-4.1.2

yum install glibc-2.5-24

yum install glibc-common-2.5

yum install glibc-devel-2.5

yum install glibc-headers-2.5

yum install kernel-headers-2.6.18

yum install ksh-20060214

yum install libaio-0.3.106

yum install libaio-devel-0.3.106

yum install libgcc-4.1.2

yum install libgomp-4.1.2

yum install libstdc++-4.1.2

yum install libstdc++-devel-4.1.2

yum install make-3.81

yum install sysstat-7.0.2

yum install unixODBC

 yum install unixODBC-devel

yum install  gcc-c++-4.1.2

yum install  libaio-devel-0.3.106 (i386)

yum install  libstdc++-devel-4.1.2 (x86_64)

yum install sysstat-7.0.2

 yum install sysstat

5、创建用户

[root@localhost rpm]# groupadd -g 1000 oinstall

 建立oinstall 组主要是为了让属于这个组的用户能够对 oraInventory目录具有读写权限 ，对数据库能够重启资源，其中oracle和grid用户必须属于oinstall组

 [root@localhost rpm]# groupadd -g 1001 dba

 创建dba组是为了让数据库的管理者oracle用户，能够登陆到asm实例上，dba组能够确定操作系统的用户能够管理数据库的权限

 [root@localhost rpm]# useradd -u 1000 -g oinstall -G dba oracle

 oracle用户主要是管理数据库软件

 [root@localhost rpm]# useradd -u 1001 -g oinstall -G dba grid

给oracle用户修改密码：

 [root@localhost rpm]# passwd oracle

 Changing password for user oracle.

 New UNIX password:

 BAD PASSWORD: it is based on a dictionary word

[root@localhost ~]# passwd grid

Changing password for user grid.

New UNIX password:

BAD PASSWORD: it is based on a dictionary word

Retype new UNIX password:

passwd: all authentication tokens updated successfully.

设置环境变量：

设置grid用户的环境变量：

export ORACLE_BASE=/u01/app/grid

export ORACLE_HOME=/u01/app/11.2/grid

export ORACLE_SID=+ASM1

export NLS_LANG=AMERICAN_AMERICA.ZHS16GBK

export PATH=.:${PATH}:$HOME/bin:$ORACLE_HOME/bin

export PATH=${PATH}:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin

export PATH=${PATH}:$ORACLE_BASE/common/oracle/bin

export ORACLE_PATH=${PATH}:$ORACLE_BASE/common/oracle/sql:.:$ORACLE_HOME/rdbms/admin

export ORACLE_TERM=xterm

export TNS_ADMIN=$ORACLE_HOME/network/admin

export ORA_NLS10=$ORACLE_HOME/nls/data

export LD_LIBRARY_PATH=$ORACLE_HOME/lib

export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$ORACLE_HOME/oracm/lib

export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib

export CLASSPATH=$ORACLE_HOME/JRE

export CLASSPATH=${CLASSPATH}:$ORACLE_HOME/jlib

export CLASSPATH=${CLASSPATH}:$ORACLE_HOME/rdbms/jlib

export CLASSPATH=${CLASSPATH}:$ORACLE_HOME/network/jlib

export PS1=`hostname`:'$PWD'$

设置oracle用户的环境变量：

vi .bash_porfile

export ORACLE_BASE=/u01/app/oracle

export ORACLE_HOME=$ORACLE_BASE/product/11.2

export ORACLE_SID=orcl1

export ORACLE_UNQNAME=rac

export NLS_LANG=AMERICAN_AMERICA.ZHS16GBK

export PATH=.:${PATH}:$HOME/bin:$ORACLE_HOME/bin

export PATH=${PATH}:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin

export PATH=${PATH}:$ORACLE_BASE/common/oracle/bin

export ORACLE_PATH=${PATH}:$ORACLE_BASE/common/oracle/sql:.:$ORACLE_HOME/rdbms/admin

export ORACLE_TERM=xterm

export TNS_ADMIN=$ORACLE_HOME/network/admin

export ORA_NLS10=$ORACLE_HOME/nls/data

export LD_LIBRARY_PATH=$ORACLE_HOME/lib

export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$ORACLE_HOME/oracm/lib

export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib

export CLASSPATH=$ORACLE_HOME/JRE

export CLASSPATH=${CLASSPATH}:$ORACLE_HOME/jlib

export CLASSPATH=${CLASSPATH}:$ORACLE_HOME/rdbms/jlib

export CLASSPATH=${CLASSPATH}:$ORACLE_HOME/network/jlib

export TEMP=/tmp

export TMPDIR=/tmp

export PS1=`hostname`:'$PWD'$

6、设置ip 地址

vi /etc/hosts

# Public

192.168.1.111 rac1

192.168.1.112 rac2

# Private

1.1.1.2 rac1-priv

1.1.1.3 rac2-priv

# Virtual

192.168.1.211 rac1-vip

192.168.1.212 rac2-vip

# SCAN

192.168.1.213 rac-scan

scan ip 和virtual ip 必须在同一个网段，

节点一：

cd /etc/sysconfig/network-scripts

[root@rac1 network-scripts]# vi ifcfg-eth0:1

# Intel Corporation 82540EM Gigabit Ethernet Controller

DEVICE=eth0:1

BOOTPROTO=none

HWADDR=08:00:27:1b:c5:39

ONBOOT=yes

IPADDR=1.1.1.2

TYPE=Ethernet

节点二：

[root@rac2 network-scripts]# cp ifcfg-eth0 ifcfg-eth0:1

[root@rac2 network-scripts]# vi ifcfg-eth0:1

# Intel Corporation 82540EM Gigabit Ethernet Controller 

DEVICE=eth0:1

BOOTPROTO=none

HWADDR=08:00:27:a1:86:ef

ONBOOT=yes

IPADDR=1.1.1.3

TYPE=Ethernet

7、设置系统参数

[root@localhost ~]# vi /etc/sysctl.conf 

fs.aio-max-nr = 1048576

fs.file-max = 6815744

kernel.shmall = 2097152

kernel.shmmax = 1054504960

kernel.shmmni = 4096

kernel.sem = 250 32000 100 128

net.ipv4.ip_local_port_range = 9000 65500

net.core.rmem_default=262144

net.core.rmem_max=4194304

net.core.wmem_default=262144

net.core.wmem_max=1048586

sysctl -p

"/etc/security/limits.conf" file.

oracle soft nproc 2047

oracle hard nproc 16384

oracle soft nofile 1024

oracle hard nofile 65536

grid soft nproc 2047

grid hard nproc 16384

grid soft nofile 1024

grid hard nofile 65536

vi /etc/pam.d/login

session required pam_limits.so

8设置目录

mkdir -p /u01/app/11.2.0/grid

mkdir -p /u01/app/grid 

mkdir -p /u01/app/oracle/product/11.2/db_1

chown -R oracle:oinstall /u01

chmod -R 775 /u01/

9、配置集群间的信任关系：

192.168.1.111

rac1:/home/oracle$ssh-keygen -t rsa

Generating public/private rsa key pair.

Enter file in which to save the key (/home/oracle/.ssh/id_rsa):

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /home/oracle/.ssh/id_rsa.

Your public key has been saved in /home/oracle/.ssh/id_rsa.pub.

The key fingerprint is:

bc:c5:94:38:32:c2:2d:a6:02:b5:cf:d0:b7:a9:f5:f7 oracle@rac1

rac1:/home/oracle$ssh-keygen -t dsa

Generating public/private dsa key pair.

Enter file in which to save the key (/home/oracle/.ssh/id_dsa):

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /home/oracle/.ssh/id_dsa.

Your public key has been saved in /home/oracle/.ssh/id_dsa.pub.

The key fingerprint is:

8e:2e:89:c2:9d:fb:95:a7:a5:1f:c9:7e:14:f9:0a:60 oracle@rac1

rac1:/home/oracle$cd .ssh

rac1:/home/oracle/.ssh$cat *.pub>> authorized_keys

rac1:/home/oracle/.ssh$ls

authorized_keys id_dsa id_dsa.pub id_rsa id_rsa.pub

rac1:/home/oracle/.ssh$scp authorized_keys oracle@192.168.1.112:/home/oracle/.ssh/key_dbs

oracle@192.168.1.112's password:

authorized_keys                 

192.168.1.112

rac2:/home/oracle/.ssh$ssh-keydbs -t rsa

-bash: ssh-keydbs: command not found

rac2:/home/oracle/.ssh$ssh-keygen -t rsa

Generating public/private rsa key pair.

Enter file in which to save the key (/home/oracle/.ssh/id_rsa):

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /home/oracle/.ssh/id_rsa.

Your public key has been saved in /home/oracle/.ssh/id_rsa.pub.

The key fingerprint is:

c9:fa:03:30:d7:a8:51:54:5f:9d:77:2c:fc:cb:e4:04 oracle@rac2

rac2:/home/oracle/.ssh$ssh-keygen -t dsa

Generating public/private dsa key pair.

Enter file in which to save the key (/home/oracle/.ssh/id_dsa):

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /home/oracle/.ssh/id_dsa.

Your public key has been saved in /home/oracle/.ssh/id_dsa.pub.

The key fingerprint is:

10:7d:42:ea:7e:0b:6c:dd:92:30:2e:bd:9b:db:13:56 oracle@rac2

rac2:/home/oracle/.ssh$ls

id_dsa id_dsa.pub id_rsa id_rsa.pub

rac2:/home/oracle/.ssh$cat *.pub >> authorized_keys

rac2:/home/oracle/.ssh$ls

authorized_keys id_dsa id_dsa.pub id_rsa id_rsa.pub

rac2:/home/oracle/.ssh$cat key_dbs >> authorized_keys

rac2:/home/oracle/.ssh$ls

authorized_keys id_dsa id_dsa.pub id_rsa id_rsa.pub key_dbs

rac2:/home/oracle/.ssh$scp authorized_keys oracle@192.168.1.111:/home/oracle/.ssh/

The authenticity of host '192.168.1.111 (192.168.1.111)' can't be established.

RSA key fingerprint is 27:ee:30:59:36:4d:07:b8:43:38:f7:1d:bc:e5:9a:f5.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added '192.168.1.111' (RSA) to the list of known hosts.

oracle@192.168.1.111's password:

authorized_keys   

设置共享磁盘：

服务端：

192.168.1.112

[root@rac2 ~]# yum install compat-db*

[root@rac2 ~]# yum install libXp*

[root@rac2 ~]# yum install scsi-target-utils*

[root@rac2 ~]# yum install openmotif*

[root@rac2 ~]# yum install iscsi*

[root@rac2 ~]# service tgtd restart

Stopping SCSI target daemon: Stopping target framework daemon

                                                           [确定]

Starting SCSI target daemon: Starting target framework daemon

客户端：

192.168.1.111

192.168.1.112

[root@rac1 ~]# yum install iscsi-initiator-utils*

[root@rac2 ~]# service iscsi restart

[root@rac1 ~]# iscsiadm -m discovery -t sendtargets -p 192.168.1.112:3260

192.168.1.112:3260,1 iqn.2008-09.com.example:rac

[root@rac2 bin]# tgt-admin --show

[root@rac1 ~]# iscsiadm -m node -T iqn.2008-09.com.example:rac -p 192.168.1.112:3260 -l

Logging in to [iface: default, target: iqn.2008-09.com.example:rac, portal: 192.168.1.112,3260]

Login to [iface: default, target: iqn.2008-09.com.example:rac, portal: 192.168.1.112,3260]: successful

[root@rac2 ~]# iscsiadm -m discovery -t sendtargets -p 192.168.1.112:3260

192.168.1.112:3260,1 iqn.2008-09.com.example:rac

[root@rac2 ~]# iscsiadm -m node -T iqn.2008-09.com.example:rac -p 192.168.1.112:3260 -l

Logging in to [iface: default, target: iqn.2008-09.com.example:rac, portal: 192.168.1.112,3260]

Login to [iface: default, target: iqn.2008-09.com.example:rac, portal: 192.168.1.112,3260]: successful

[root@rac2 ~]# fdisk -l

Disk /dev/sda: 107.4 GB, 107400396800 bytes

255 heads, 63 sectors/track, 13057 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot Start End Blocks Id System

/dev/sda1 * 1 25 200781 83 Linux

/dev/sda2 26 5124 40957717+ 83 Linux

/dev/sda3 5125 5634 4096575 82 Linux swap / Solaris

/dev/sda4 5635 13057 59625247+ 5 Extended

/dev/sda5 5635 11714 48837568+ 83 Linux

Disk /dev/sdb: 50.0 GB, 50009670144 bytes

64 heads, 32 sectors/track, 47692 cylinders

Units = cylinders of 2048 * 512 = 1048576 bytes

Disk /dev/sdb doesn't contain a valid partition table

[root@rac2 ~]# service iscsi restart

[root@rac2 ~]# service tgtd restart

[root@rac2 ~]# chkconfig iscsi on

[root@rac2 ~]# chkconfig tgtd on

[root@rac2 ~]# fdisk -l

Disk /dev/sda: 107.4 GB, 107400396800 bytes

255 heads, 63 sectors/track, 13057 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot Start End Blocks Id System

/dev/sda1 * 1 25 200781 83 Linux

/dev/sda2 26 5124 40957717+ 83 Linux

/dev/sda3 5125 5634 4096575 82 Linux swap / Solaris

/dev/sda4 5635 13057 59625247+ 5 Extended

/dev/sda5 5635 11714 48837568+ 83 Linux

Disk /dev/sdb: 50.0 GB, 50009670144 bytes

64 heads, 32 sectors/track, 47692 cylinders

Units = cylinders of 2048 * 512 = 1048576 bytes

Disk /dev/sdb doesn't contain a valid partition table

对磁盘进行划分

[root@rac1 ~]# fdisk -l

Disk /dev/sda: 48.9 GB, 48968499200 bytes

255 heads, 63 sectors/track, 5953 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot Start End Blocks Id System

/dev/sda1 * 1 25 200781 83 Linux

/dev/sda2 26 5124 40957717+ 83 Linux

/dev/sda3 5125 5634 4096575 82 Linux swap / Solaris

Disk /dev/sdb: 50.0 GB, 50009670144 bytes

64 heads, 32 sectors/track, 47692 cylinders

Units = cylinders of 2048 * 512 = 1048576 bytes

   Device Boot Start End Blocks Id System

/dev/sdb1 1 4769 4883440 83 Linux

/dev/sdb2 4770 9538 4883456 83 Linux

/dev/sdb3 9539 14307 4883456 83 Linux

/dev/sdb4 14308 47692 34186240 83 Linux

此处用udev进行绑定变量，还可以用asmlib，这个两个是同样的方式

 vi /etc/udev/rules.d/60-raw.rules

 ACTION=="add", KERNEL=="sdb1", RUN+="/bin/raw /dev/raw/raw1 %N"

 ACTION=="add", KERNEL=="sdb2", RUN+="/bin/raw /dev/raw/raw2 %N"

 ACTION=="add", KERNEL=="sdb3", RUN+="/bin/raw /dev/raw/raw3 %N"

 ACTION=="add", KERNEL=="sdb4", RUN+="/bin/raw /dev/raw/raw4 %N" 

 KERNEL=="raw[1]", MODE="0660", GROUP="oinstall", OWNER="oracle"

 KERNEL=="raw[2]", MODE="0660", GROUP="oinstall", OWNER="oracle"

 KERNEL=="raw[3]", MODE="0660", GROUP="oinstall", OWNER="oracle"

 KERNEL=="raw[4]", MODE="0660", GROUP="oinstall", OWNER="oracle" 

[root@rac1 ~]# cd /dev/raw

[root@rac1 raw]# ls

raw1 raw2 raw3 raw4

时间同步采用ctss，而不用ntp服务

[root@rac1 ~]# mv /etc/ntp.conf /etc/ntp.conf.bak

到此前期的准备都已经做好，下面我们来安装grid

[root@rac1 cd]# cp database /tmp -a

[root@rac1 cd]# chown oracle:oinstall /tmp/database -R

[root@rac1 cd]# cp grid  /tmp -a

[root@rac1 tmp]# chown grid:oinstall /tmp/grid -R

预检测：

[grid@dba grid]$ ./runcluvfy.sh stage -pre crsinst -n rac1,rac2 -verbose

   集群服务设置 的预检查成功。

在安装中需要注意：

osdba  asm数据库管理员

osoper  实力管理操作者

osasm  实例管理员

在这里了我们三个都用oinstall组

grid的软件目录是： /u01/app/11.2/grid

oracale基目录  ：  /u01/app/grid

用root用户执行远程脚本：

/u01/app/oraInventory/orainstRoot.sh

安装完后点击完成后：

到此grid 安装完成：

安装oracle数据软件 ：

在数据检查定义的时候报了一个错误：

节点连接性的错误，次错误时因为在定义私有网络ip地址使用了1.1.1.0网段，但是网关没有定义：

[root@rac1 app]# route -n

Kernel IP routing table

Destination Gateway Genmask Flags Metric Ref Use Iface

192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0

169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0

1.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 eth0

在两个节点上分别定义一个网关：

[root@rac2 bin]# route add default gw 1.1.1.0

设置好网关检查顺利通过，并且安装数据库软件