一体机配置

一体机网络构架：

一体机配置：

总结：一体机内部有大量的写好的脚步，从上文档发现很多配置都是脚步！

配置：KVM 

1. Pull the KVM tray out from the front of the rack, and open it using the handle.

2. Touch the touch pad.

3. Toggle between the host and KVM interface by pressing the Ctrl key on the left side twice, similar to a double-click on a mouse.

4. Select Target Devices from the Unit View of the user interface. The number of sessions shown should be 22 for Exadata Database Machine Full Rack, 11 for Exadata Database Machine Half Rack, and 5 for Exadata Database Machine Quarter Rack. The number of sessions should be 18 for Oracle Exadata Storage Expansion Full Rack, 9 for Oracle Exadata Storage Expansion Half Rack, and 4 for Oracle Exadata Storage Expansion Quarter Rack.

   Note:

   If all sessions are not shown, then select IQ Adaptors from the Ports heading. Click the table heading, Port, to sort the sessions by port number. Note any missing items. The sessions are numbered from the bottom of the rack to the top.

5. Return to the Target Devices screen.

6. Select Local from User Accounts.

7. Click Admin under Users.

8. Set a password for the Admin account. Do not modify any other parameters.

9. Click Save.

10. Select Network from Appliance Settings. The Network Information screen appears.

11. Enter the values for Address, Subnet, Gateway, and the IP addresses of the DNS servers.

12. Click Save.

13. Connect the KVM LAN1 Ethernet port to the management network.

14. Verify the port has been configured correctly by checking the MAC address on the Network Information screen. The address should match the label next to the LAN1/LAN2 ports on the rear of the KVM switch.

15. Select Overview from Appliance.

16. Enter a name for the KVM switch.

17. Click Save.

18. Restart the KVM switch by selecting Reboot under Overview.

19. Examine the firmware version of the switch by selecting Versions from Appliance Settings. There are two version numbers shown, Application and Boot, as shown in the following:

    Required version is:Application 1.2.10.15038Boot  1.6.15020

配置 KVM Switch to Access a Server

The following procedure describes how to configure the KVM switch to access the servers:

Note:

The KVM switch is only available in Oracle Exadata Database Machine X2-2 racks and Oracle Exadata Storage Expansion Racks.

1. Select Target Devices from Unit View.

2. Power on the server. The power button is on the front panel. If the button seems stuck, then use a small tool to loosen the button.

3. Click the system name in the Name column using the left mouse button.

4. Click Overview, and overwrite the name with the Oracle standard naming format of customer prefix, node type, and number. For example, trnacel03 has the prefixtrna, and storage cell 3 from the bottom of the rack, and trnadb02 has the prefix trna, and database server 2 from the bottom of the rack.

5. Press Save.

6. Repeat steps 2 through 5 for each server in the rack. Each server boots up through BIOS, and boots the operating system with the default factory IP configuration.

Accessing a Server Using the KVM Switch

The following procedure describes how to access a server using the KVM switch:

Note:

The KVM switch is only available in Oracle Exadata Database Machine X2-2 racks and Oracle Exadata Storage Expansion Racks.

1. Select Target Devices from Unit View.

2. Click the system name in the Name column using the left mouse button.

3. Click the KVM session.

Configuring Sun Datacenter InfiniBand Switch 36 Switch

The following procedure describes how to configure the Sun Datacenter InfiniBand Switch 36 switches.

1. Log in to the first database server as the root user. The first database server is the lowest database server in the rack, which is rack position U16.

   Note:

   If you do not have the password for the root user, then contact Oracle Support Services.

2. Use SSH to log in to the Sun Datacenter InfiniBand Switch 36 switch as the root user. The IP address for the Sun Datacenter InfiniBand Switch 36 switch is listed inChapter 3, "Understanding the Network Requirements for Oracle Exadata Database Machine." The following is an example of the command:

   ssh root@192.168.1.201

3. Use spsh to open the Integrated Lights Out Manager (ILOM) command line interface.

4. Configure the network interface using the following commands:

5. set /SP/network pendingipdiscovery=static pendingipaddress=pending_ip \pendingipgateway=pending_gw pendingipnetmask=pending_nmset /SP/network commitpending=true 

   In the preceding commands, pending_ip, pending_gw, and pending_nm are IP addresses defined by the network administrator.

6. Use the exit command to exit the interface.

7. Edit the /etc/hosts file to set the IP address and host name using an editor such as vi. The following is an example of the updated file:

   #Do not remove the following link, or various programs#that require network functionality will fail.127.0.0.1     localhost.localdomain localhost10.7.7.32     dm01sw-ib1-ib2.example.com trnasw-ib2

   It is required that the second line has both fully-qualified and non-fully-qualified names.

8. Edit the /etc/resolv.conf file to set the DNS server and domain name using an editor such as vi. There should be a name server line for each DNS server. The following is an example of the updated file:

   search        example.comnameserver    10.7.7.3

9. Edit the /etc/sysconfig/network file to include the host name. The following is an example of the syntax:

   hostname=switch_value

   In the preceding syntax, switch_value is the fully-qualified domain name for the switch.

10. Use spsh to open the ILOM command line interface.

11. Set the date, time zone, and Network Protocol Time (NTP) using the following commands:

    set /SP/clock timezone=preferred_tzset /SP/clients/ntp/server/1 address=ntp_ip1set /SP/clients/ntp/server/2 address=ntp_ip2set /SP/clock usentpserver=enabled 

    In the preceding commands, preferred_tz is the preferred time zone, and ntp_ip1 and ntp_ip2 are the NTP server IP addresses. It is not necessary to configure both NTP servers, but at least one should be configured.

12. Restart the switch.

13. Examine the firmware version using the following command:

    # version

    See My Oracle Support Note 888828.1 for the current firmware version.

14. Check the health of the switch using the following command:

    # showunhealthyOK - No unhealthy sensors

15. Run the environment test using the following command:

    # env_testNM2 Environment test started:Starting Voltage test:Voltage ECB OKMeasured 3.3V Main = 3.28 VMeasured 3.3V Standby = 3.42 VMeasured 12V =12.06 VMeasured 5V =5.03 VMeasured VBAT =3.06 VMeasured 2.5V =2.53 VMeasured 1.8V =1.79 VMeasured I4 1.2V =1.22 VVoltage test returned OKStarting PSU test:PSU 0 presentPSU 1 presentPSU test returned OKStarting Temperature test:Back temperature 30.50Front temperature 33.88ComEx temperature 34.12I4 temperature 56,maxtemperature 57Temperature test returned OKStarting FAN test:Fan 0 not presentFan 1 running at rpm 12946Fan 2 running at rpm 12684Fan 3 running at rpm 12558Fan 4 not presentFAN test returned OKStarting Connector test:Connector test returned OKStarting I4 test:I4 OKAll I4s OKI4 test returned OKNM2 Environment test PASSED

16. Enable the InfiniBand Subnet Manager using the following command:

    # enablesm

    Note:

    If you get an error indicating the InfiniBand Subnet Manager is already running, then restart it as follows:

    # disablesm# enablesm

17. Verify the IP address is correct using the following command:

    # ifconfig eth0eth0    Link encap:Ethernet HWaddr 00:E0:4B:2A:07:2B        inet addr:172.16.10.32 Bcast:172.16.10.255Mask:255.255.255.0        inet6 addr:fe80::2e0:3a00:fe2a:61e/64 Scope:Link        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1        RX packets:11927 errors:0 dropped:0 overruns:0 frame:0        TX packets:89 errors:0 dropped:0 overruns:0 carrier:0        collisions:0 txqueuelen:1000        RX bytes:720262 (703.3 KiB)TX bytes:11402 (11.1 KiB)

18. Verify the host name is correct using the following command:

    # hostnamedm01sw-ib2.example.com

19. Set the Subnet Manager Master as described in "Setting the Subnet Manager Master on Exadata Database Machine Full Rack and Exadata Database Machine Half Rack". This step is needed for Exadata Database Machine Half Rack and Exadata Database Machine Full Rack.

20. Log out from the InfiniBand switch using the following command:

    # exit

Note:

If the network settings did not appear after restart, then power cycle the switch by removing both power cords for one minute.

Setting the Subnet Manager Master on Exadata Database Machine Full Rack and Exadata Database Machine Half Rack

Exadata Database Machine Full Racks and Oracle Exadata Database Machine X2-2 Half Racks have three Sun Datacenter InfiniBand Switch 36 switches. The switch at rack unit 1 (U1) is referred to as the spine switch. The switches at rack unit 20 (U20) and rack unit 24 (U24) in Oracle Exadata Database Machine X2-2 racks, or unit 21 (U21) and rack unit 23 (U23) in Oracle Exadata Database Machine X2-8 Full Racks are referred to as leaf switches. The spine switch is the Subnet Manager Master for the InfiniBand subnet. The Subnet Manager Master has priority 8, and can be verified using the following procedure:

See Also:

• "Hardware Components of Oracle Exadata Database Machine"

• Appendix E, "Cabling Tables for Oracle Exadata Database Machine"

1. Log in to the spine switch as the root user.

2. Run the setsmpriority list command.

   The command should show that smpriority has a value of 8. If smpriority has a different value, then do the following:

   1. Use the disablesm command to stop the Subnet Manager.

   2. Use the setsmpriority 8 command to set the priority to 8.

   3. Use the enablesm command to restart the Subnet Manager.

The leaf switches are the Standby Subnet Managers with a priority of 5. This can be verified using the preceding procedure, substituting a value of 5 in the command shown in step 2b.

Note:

Exadata Database Machine Half Rack based on Sun Fire X4170 Oracle Database Servers include two Sun Datacenter InfiniBand Switch 36 switches, which are set to priority 5.

To determine the Subnet Manager Master, log in as the root user on any InfiniBand switch, and run the getmaster command. The location of the Subnet Manager Master is displayed. The following is an example of the output from the getmaster command:

# getmaster20100701 11:46:38 OpenSM Master on Switch : 0x0021283a8516a0a0 ports 36 Sun DCS 36QDR switch dm01sw-ib1.example.com enhanced port 0 lid 1 lmc 0

The preceding output shows the proper configuration. The Subnet Master Manager is running on spine switch dm01sw-ib1.example.com.

If the spine switch is not the Subnet Manager Master, then do the following procedure to set the Subnet Manager Master:

1. Use the getmaster command to identify the current location of the Subnet Manager Master.

2. Log in as the root user on the leaf switch that is the Subnet Manager Master.

3. Disable the Subnet Manager on the switch. The Subnet Manager Master relocates to another switch.

   See Also:

   "Disable the Subnet Manager" in Sun Datacenter InfiniBand Switch 36 User's Guide at

   http://docs.oracle.com/cd/E19197-01/835-0784-05/z4001de61813698.html#z40003f12047367

4. Use the getmaster command to identify the current location of the Subnet Manager Master. If the spine switch is not Subnet Manager Master, then repeat steps 2through 3 until the spine switch is the Subnet Manager Master.

5. Enable the Subnet Manager on the leaf switches that were disabled during this procedure.

   See Also:

   "Enable the Subnet Manager" in Sun Datacenter InfiniBand Switch 36 User's Guide at

   http://docs.oracle.com/cd/E19197-01/835-0784-05/z4001de61707660.html#z40003f12047359

Notes:

• If the InfiniBand network consists of four or more racks cabled together, then only the spine switches should run Subnet Manager. The leaf switches should have Subnet Manager disabled on them.

• Exadata Database Machine Half Racks based on Sun Fire X4170 Oracle Database Servers and Exadata Database Machine Quarter Racks have two Sun Datacenter InfiniBand Switch 36 switches, and both are set to priority 5. The master is the one with the lowest GUID.

Configuring the Cisco Ethernet Switch

The Cisco Catalyst 4948 Ethernet switch supplied with Oracle Exadata Rack is minimally configured during installation. The minimal configuration disables IP routing, and sets the following:

• Host name

• IP address

• Subnet mask

• Default gateway

• Domain name

• Name server

• NTP server

• Time

• Time zone

Before configuring the switch, note the following:

• The Cisco Ethernet switch should not be connected until the running configuration has been verified, and any necessary changes have been made by the network administrator.

• The Cisco Ethernet switch should not be connected to the customer network until the IP addresses on all components have been configured in Oracle Exadata Rack. This is to prevent any duplicate IP address conflicts which are possible due to the default addresses set in the components when shipped.

See Also:

• "Network Connection and IP Address Requirements for Oracle Exadata Database Machine"

• "Network Configuration Checklist"

The following procedure describes how to configure the Cisco Ethernet switch. Configuration should be done with the network administrator.

1. Connect a serial cable from the Cisco switch console to a laptop or similar device. An RJ45 to DB9 serial cable is included with the Cisco documentation package.

2. Ensure the terminal session is recorded on the laptop by logging the output. The output can be used as a reference that the switch has been configured correctly. Thedefault serial port speed is 9600 baud, 8 bits, no parity, 1 stop bit, and no handshake.

   Switch con0 is now availablePress RETURN to get started.

3. Change to enable mode using the following command:

   Switch> enable

   Note:

   If you do not have the password for the root user, then contact Oracle Support Services.

4. Configure the network for a single VLAN. The following is an example of the configuration:

   Switch# configure terminalEnter configuration commands,one per line.End with CNTL/Z.Switch(config)# interface vlan 1Switch(config-if)# ip address 10.7.7.34 255.255.255.0Switch(config-if)# endSwitch# *Sep 15 14:12:06.309:%SYS-5-CONFIG_I:Configured from console by consoleSwitch# write memoryBuilding configuration...Compressed configuration from 2474 bytes to 1066 bytes [OK ]

5. (Optional) Disable the default IP routing setting, and configure the default gateway as follows. This step is required if IP routing will not be used on the switch.

   Switch# configure terminalEnter configuration commands,one per line.End with CNTL/Z.Switch(config)# no ip routingSwitch(config)# ip default-gateway 10.7.7.1Switch(config)# end *Sep 15 14:12:46.309:%SYS-5-CONFIG_I:Configured from console by consoleSwitch# write memoryBuilding configuration...Compressed configuration from 2492 bytes to 1070 bytes [OK ]

   Note:

   If IP routing is required on the switch, then leave the IP routing setting as the default, and configure the default gateway as follows:

   Switch# configure terminalEnter configuration commands,one per line.End with CNTL/Z.Switch(config)# ip route 0.0.0.0 0.0.0.0 10.7.7.1Switch(config)# end*Sep 15 14:13:26.013:%SYS-5-CONFIG_I:Configured from console by consoleSwitch# write memoryBuilding configuration...Compressed configuration from 2502 bytes to 1085 bytes [OK ]

6. Set the host name of the switch as follows:

   Switch# configure terminalEnter configuration commands,one per line.End with CNTL/Z.Switch(config)# hostname burxsw-ipburxsw-ip(config)# endburxsw-ip# write memoryBuilding configuration...Compressed configuration from 3789 bytes to 1469 bytes [OK ]

   The system host name is used as the prompt name.

7. Set the password as follows:

   burxsw-ip # configure terminalEnter configuration commands,one per line.End with CNTL/Z.burxsw-ip(config)# enable password *******burxsw-ip(config)# enable secret ******* The enable secret you have chosen is the same as your enable password.This is not recommended.Re-enter the enable secret.burxsw-ip(config)# endburxsw-ip#write memory *Sep 15 14:25:05.893:%SYS-5-CONFIG_I:Configured from console by consoleBuilding configuration...Compressed configuration from 2502 bytes to 1085 bytes [OK ]

8. Set the password for telnet network access. The following is an example:

   Switch# configure terminalEnter configuration commands,one per line.End with CNTL/Z.burxsw-ip(config)# line vty 0 15burxsw-ip(config-line)# login%Login disabled on line 1,until 'password'is set%Login disabled on line 2,until 'password'is set%Login disabled on line 3,until 'password'is set...%Login disabled on line 15,until 'password'is set%Login disabled on line 16,until 'password'is setburxsw-ip(config-line)# password *******burxsw-ip(config-line)# loginburxsw-ip(config-line)# endburxsw-ip#write memoryBuilding configuration...Compressed configuration from 3786 bytes to 1468 bytes [OK ]

   In the preceding example, the first login output shows the password has not been set, and telnet access is disabled. If the login command returns nothing, then the password is set and telnet access is available.

9. Configure up to three DNS servers. The following is an example:

   burxsw-ip# configure terminalEnter configuration commands,one per line.End with CNTL/Z.burxsw-ip(config)# ip domain-name exdm.comburxsw-ip(config)# ip name-server 10.7.7.3burxsw-ip(config)# ip name-server 129.148.5.5burxsw-ip(config)# ip name-server 10.8.160.1burxsw-ip(config)# end *Sep 15 14:26:37.045:%SYS-5-CONFIG_I:Configured from console by consoleburxsw-ip#write memoryBuilding configuration...Compressed configuration from 2603 bytes to 1158 bytes [OK ]

10. Set the clock and time zone as follows. The switch keeps internal time in Coordinated Universal Time (UTC) format.

    • To use UTC, use the following command:

      no clock timezone global configuration

    • To use a time zone, use the following command:

      clock timezone zone hours-offset [minutes-offset]

      In the preceding command, zone is the time zone to display when standard time in effect, hours-offset is the hours offset from UTC, and minutes-offset is the minutes offset from UTC.

    • To set summer time hours, use the following command:

      clock summer-time zone recurring [week day month hh:mm week day month \hh:mm [offset]]

      In the preceding command, zone is the time zone to be displayed when summer time is in effect, week is the week of the month (1 to 5 or last), day is the day of the week, month is the month, hh:mm is the time in 24-hour format, and offset is the number of minutes to add during summer time. The default offset is 60 minutes.

    • To manually set the clock to any time use the following command:

      clock set hh:mm:ss month day year

      In the preceding command, hh:mm:ss is the time in 24-hour format, day is the day of the month, month is the month, and year is the year. The time specified is relative to the configured time zone.

    To set the local time and time zone, ordering is important. The following is an example of setting local time to US Eastern time:

    Switch# configure terminalEnter configuration commands,one per line.End with CNTL/Z.burxsw-ip(config)# clock timezone EST -5 burxsw-ip(config)# clock summer-time EDT recurringburxsw-ip(config)# endburxsw-ip# clock set 21:00:00 August 09 2010burxsw-ip#write memoryBuilding configuration...Compressed configuration from 3784 bytes to 1465 bytes [OK ]burxsw-ip# show clock21:00:06.643 EST Mon Aug 9 2010

11. Configure up to two NTP servers. The following is an example:

    burxsw-ip# configure terminalEnter configuration commands,one per line.End with CNTL/Z.burxsw-ip(config)# ntp server 10.7.7.32 preferburxsw-ip(config)# ntp server 129.148.9.19burxsw-ip(config)# end*Sep 15 14:51:08.665:%SYS-5-CONFIG_I:Configured from console byconsoleburxsw-ip# write memoryBuilding configuration...Compressed configuration from 2654 bytes to 1163 bytes [OK ]burxsw-ip# show ntp status<output will vary per network>burxsw-ip# show clock20:59:06.643 EST Mon Aug 9 2010

    The preceding should show the NTP server synchronized to local time if the Cisco switch is connected to the network and has access to NTP.

12. Verify the configuration using the following command:

    burxsw-ip# show running-config

    The following is an example of the output:

    Building configuration...Current configuration :2654 bytes!version 12.2no service padservice timestamps debug datetime msecservice timestamps log datetime msecno service password-encryptionservice compress-config!hostname Switch!boot-start-markerboot-end-marker!enable secret 5 $1$mS8h$EaJrIECUxavfGH6vLZg1T.enable password *******!no aaa new-modelip subnet-zeroip domain-name sodm.comip name-server 10.7.7.3ip name-server 172.16.5.5ip name-server 10.8.160.1!ip vrf mgmtVrf!vtp mode transparent!power redundancy-mode redundant!!spanning-tree mode pvstspanning-tree extend system-id!vlan internal allocation policy ascending!!interface FastEthernet1 ip vrf forwarding mgmtVrf no ip address speed auto duplex auto!interface GigabitEthernet1/1 spanning-tree portfast!interface GigabitEthernet1/2 spanning-tree portfast!...!interface GigabitEthernet1/47 media-type rj45 spanning-tree portfast!interface GigabitEthernet1/48 media-type rj45!interface Vlan1 ip address 10.7.7.34 255.255.255.0!interface Vlan48 no ip address shutdown!ip default-gateway 10.7.7.1ip http server!!control-plane!!line con 0 stopbits 1line vty 0 4 password ******* loginline vty 5 15 password ******* login!ntp server 10.7.7.32 preferntp server 172.16.9.1end

    If any setting is incorrect, the repeat the appropriate step. To erase a setting, enter no in front of the same command. For example, to erase the default gateway, the following commands would be entered:

    burxsw-ip# no ip default-gateway 10.7.7.1burxsw-ip# endburxsw-ip# write memory

13. Save the current configuration using the following command:

    burxsw-ip# copy running-config startup-config

14. Exit from the session using the following command:

    burxsw-ip# exit

15. Disconnect the cable from the Cisco console.

16. Attach a laptop computer to port 48, and ping the IP address of the internal management network to check the configuration.

Configuring the Power Distribution Units

The power distribution units (PDUs) are configured with a static IP address to connect to the network for monitoring. Ensure you have the following before connecting the PDU to the network:

• Static IP address

• Subnet mask

• Default gateway

• Ethernet cables

• Laptop computer with either Oracle Solaris or Microsoft Windows

The following procedure describes how to configure and connect the PDU to the network:

1. Power off the PDU.

2. Disconnect the power leads from the power source.

3. Select an unused LAN network connection from the available network connections on the laptop as follows:

   • For Oracle Solaris:

     1. Log in as a superuser.

     2. Use the dladm show-link command to find an unused Ethernet interface.

     3. Use the ifconfig -a command to determine which interface is being used.

     4. Use the following command to plumb the unused interface:

        # ifconfig interface plumb up

        In the preceding command, interface is the network interface determined in step c.

     5. Using the following command to assign an IPv4 address and netmask to the interface:

        # ifconfig interface IPv4_address netmask + netmask \  broadcast broadcast up

        In the preceding command, interface is the network interface, IPv4_address is the IP address, netmask is the netmask address, and broadcast is the broadcast address.

        Note:

        The network interfaces configured with the ifconfig command do not persist across system restarts.

   • For Windows:

     1. Select Settings from the Start menu.

     2. Select Network Connections from the Settings menu.

     3. Select an unused local area connection from the Network Connections menu.

     4. Double-click Internet Protocol (TCP/IP) in the Local Area Connection Properties window. The Internet Protocol (TCP/IP) window appears.

     5. Select Use the following IP address.

     6. Enter the IP address in the IP address field. Do not use 192.168.0.1 because that is the default PDU metering unit address.

     7. Enter the IP address in the Subnet mask field.

     8. Leave the Default gateway field blank.

     9. Click OK.

4. Connect the PDU and the laptop to two ports in the Cisco switch. The equipment currently using the Cisco switch must be disconnected temporarily.

5. Connect the PDU power input lead to the power source. Only one PDU power input lead needs to be connected, as follows:

   • On PDUs with one power input lead, connect the one lead to the power source.

   • On PDUs with two power leads, connect the first power lead to the power source. The first power lead is labelled 0.

   • On PDUs with three power input leads, connect the middle power lead to the power source. The middle power lead is labelled 1.

6. Use a Web browser to access the PDU metering unit by entering the factory default IP address for the unit, 192.168.0.1, in the address line of the browser. The Current Measurement page should appear.

7. Click Network Configuration in the upper left of the page.

8. Log in as the admin user on the PDU metering unit. Change the password after configuring the network.

9. Confirm the DHCP Enable check box is not selected.

10. Enter the following network settings for the PDU metering unit:

    • IP address

    • Subnet mask address

    • Default gateway

11. Click Submit to set the network settings, and reset the PDU metering unit.

12. Disconnect the PDU and laptop from the Cisco switch.

13. Reconnect the two cables that were originally connected to the Cisco switch.

14. Connect an Ethernet cable to the PDU metering unit RJ-45 Ethernet port and to the network.

15. Log in to the PDU metering unit using a Web browser. Use the PDU metering unit's static IP address in the browser's address line. If the configuration was successful, then the Current Measurement page is displayed.

16. Connect the remaining PDU power input leads to the power source.

Checking Exadata Storage Servers

The following procedure describes how to check Exadata Storage Servers:

Notes:

• When using the KVM switch, and pressing the ESC key, the BIOS may receive two ESC characters, and prompt to exit. SelectCANCEL.

• If you need to connect to the ILOM serial management port, then the baud rate setting on Exadata Storage Servers is changed from the default ILOM setting 9600 to 115200 baud, 8 bits, no parity, and 1 stop bit.

1. Log in as the root user to the first Exadata Storage Server.

   Note:

   If you do not have the password for the root user, then contact Oracle Support Services.

2. Verify the SysSN label for the cell matches the product_serial_number value using the following command:

   #ipmitool sunoem cli "show /SYS product_serial_number"Connected.Use ^ D to exit.->show /SYS product_serial_number/SYSProperties:product_serial_number =0937XFG03B->Session closedDisconnected

3. Check the hardware and firmware using the following command:

   # /opt/oracle.SupportTools/CheckHWnFWProfile

   If output does not show success, then examine the output for mismatched hardware or firmware and take steps to correct those as explained in next steps in this section.

4. Verify the disks on the server are visible and online using the following command. The disks are numbered slot 0 to slot 11.

   #cd /opt/MegaRAID/MegaCli#./MegaCli 64 -Pdlist -aAll |grep "Slot \|Firmware"Slot Number:0Firmware state:Online, Spun UpSlot Number:1Firmware state:Online, Spun UpSlot Number:2Firmware state:Online, Spun UpSlot Number:3Firmware state:Online, Spun UpSlot Number:4Firmware state:Online, Spun UpSlot Number:5Firmware state:Online, Spun Up

1. Slot Number:6Firmware state:Online, Spun UpSlot Number:7Firmware state:Online, Spun UpSlot Number:8Firmware state:Online, Spun UpSlot Number:9Firmware state:Online, Spun UpSlot Number:10Firmware state:Online, Spun UpSlot Number:11Firmware state:Online, Spun Up[root@cell01 ~]#

2. Verify the flash modules are visible using the following command. There are four cards, and four modules per card.

   # lsscsi |grep -i marvel# cellcli -e list lun where disktype=flashdisk

   The numbering shows the PCI slot number and FMOD number, such as 1_0 is PCI slot 1, FMOD 0 on the card. To confirm the flash disks are mapped that way, use the following command:

   # cellcli -e list physicaldisk attributes name, id, slotnumber where \disktype="flashdisk" and 'status! ="not present"'

   If any FMODs are missing, then power cycle the system, and wait 10 minutes after booting to check. If this does not resolve the problem, then reseat the module and flash card. If the problem continues, then replace the card with the onsite spare.

3. Check the rear of the rack to ensure the flash card LEDs are green all the way across. These are FMOD, 0, 1, 2, 3, and SuperCAP. If any LED is amber, then check again after the next step. If the LED is still amber after restart, then reseat the card and ESM or FMOD. If the problem continues, then replace the card with the onsite spare.

   Note:

   This step can be done when all systems are on. It does not have to be done on a per-server basis.

4. Log out of the system using the following command:

   # logout

5. Return to the User Interface on the KVM switch by pressing the Ctrl key on the left to return to the KVM switch, and select Disconnect Session, if using the KVM switch to access the server.

6. Select Target Devices from Unit View, if using the KVM switch to access the server.

7. Proceed to the next Exadata Storage Server.

Checking Oracle Database Servers

The following procedure describes how to check the Oracle Database servers. At this point, the operating system is Linux.

1. Log in as the root user to the first database server.

   Note:

   If you do not have the password for the root user, then contact Oracle Support Services.

2. Verify the SysSN label for the server matches the product_serial_number value using the following command:

   # ipmitool sunoem cli "show /SYS product_serial_number"Connected.Use ^ D to exit.->show /SYS product_serial_number/SYSProperties:product_serial_number =093ABCD5001->Session closedDisconnected

3. Verify the hardware profile is correct using the following command. The command checks CPU type and count, component firmware, and so on.

   # dcli -l root -g half "/opt/oracle.SupportTools/CheckHWnFWProfile -c  strict" > /tmp/checkhwfw.out# more /tmp/checkhwfw.out

   The following is an example of a check that failed:

   172.108.1.6:[WARNING ]The hardware and firmware are not supported.See details below[DiskControllerPCIeSlotWidth ]Requires:x8Found:x4[WARNING ]The hardware and firmware are not supported.See detailsabove

   If a profile check fails and the firmware needs to be refreshed, then run the following command from the system with the problem.

   # /opt/oracle.SupportTools/CheckHWnFWProfile -U /opt/oracle.cellos/iso/cellbitsNow updating the ILOM and the BIOS …

   The command must be entered on a single line. After running the command, power off the system, reset the ILOM, and wait 10 minutes before powering on the system.

4. Verify the disks on the server are visible and online using the following command. All but one disk should show Spun Up. The one disk that shows Spun Down is the hot spare. For Oracle Exadata Database Machine X2-2, there are four disks, and for Oracle Exadata Database Machine X2-8 Full Rack, there are eight disks.

   # cd /opt/MegaRAID/MegaCli# ./MegaCli 64 -Pdlist -aAll |grep "Slot \|Firmware"Firmware state:Online, Spun UpSlot Number:1Firmware state:Online, Spun UpSlot Number:2Firmware state:Online, Spun UpSlot Number:3Firmware state:Hotspare, Spun Down

5. Examine the output of the following command:

   /opt/MegaRAID/MegaCli/MegaCli64 -LdInfo -LAll -aAll 

   For Oracle Exadata Database Machine X2-2, the shipped configuration is dual boot. Verify there is one RAID-1 and two RAID-0 logical (virtual) drives.

   For Oracle Exadata Database Machine X2-8 Full Rack, the shipped configuration is dual boot. There should be one 3-disk RAID-5 and two 2-disk RAID-1 logical drives with one global hot spare drive.

6. (Oracle Exadata Database Machine based on Sun Fire X4170 Oracle Database Servers only) Check that the ILOM is configured for the proper fan cooling on each database server using the following command:

   # ipmitool sunoem cli "show /SP/policy"

   • If the policy is present and enabled, then no additional steps are needed.

   • If the policy is present and disabled, then enable the policy using the following command:

     # ipmitool sumoem cli "set ?SP/policy FLASH_ACCELERATOR_CARD_ \INSTALLED=enabled"

     The command must be entered as a single line.

7. Log out of the system using the following command:

   # logout

8. Return to the User Interface on the KVM switch by pressing the Ctrl key on the left to return to the KVM switch, and select Disconnect Session, if using the KVM switch to access the server.

9. Select Target Devices from Unit View, and proceed to the next database server, if using the KVM switch to access the server.

10. Verify that NTP servers are configured the same for all servers using the following command:

    dcli -g all_group -l root "ntpq -p"

Performing Additional Checks and Configuration

The following procedure describes additional checks and configuration steps for Oracle Exadata Rack:

1. (Oracle Exadata Database Machine only) Log in to the first database server as the root user.

2. Change to the firstconf directory using the following command:

   # cd /opt/oracle.SupportTools/firstconf

3. Verify the network port for eth0 can be accessed over the Cisco switch using the following command:

   # ./ping_check_ilom_and_eth0.sh rack_size

   In the preceding command, rack_size is the size of the rack. For Oracle Exadata Database Machine X2-2 Full Rack use full, for Oracle Exadata Database Machine X2-2 Half Rack use half, for Oracle Exadata Database Machine X2-2 Quarter Rack use quarter, and for Oracle Exadata Database Machine X2-8 Full Rack use full_v2pp. The following is an example of a failure:

   # ./ping_check_ilom_and_eth0.sh full_v2pp[INFO ]Checking nodes are pingable...[ERROR ]Following nodes are not pingable.Check connections and retry.Unpingable:172.16.10.25 172.16.10.18

   A failure indicates a bad connection, faulty cable, bad network port, or bad switch port.

4. Verify the NTP servers are configured the same for all servers using the following command:

   dcli -g all_group -l root "ntpq -p"

5. Verify the rack master serial number is set correctly using the following command:

   # dcli -l root -g half "ipmitool sunoem cli 'show /SP system_identifier'" >   /tmp/show-rack-csn.out# more /tmp/show-rack-csn.out

   If none of the serial numbers are set, then contact Oracle Support Services. If one serial number is not set correctly, then run the following command to correct it:

   ipmitool sunoem cli 'set /SP system_identifier= \"Exadata Database Machine rack_typexxxxAKyyyy"'

   In the preceding command, rack_type is X2-2 or X2-8, depending on the rack.

6. Ensure disk cache policy is set to Disabled using the following command as the root user on the first database server in the cluster:

   dcli -g /opt/oracle.SupportTools/onecommand/all_group -l root \/opt/MegaRAID/MegaCli/MegaCli64 -LdPdInfo -aALL | grep -i 'Disk Cache Policy'

   The following is an example of the output from the command:

   dm01db01: Disk Cache Policy   : Disableddm01db02: Disk Cache Policy   : Disableddm01db03: Disk Cache Policy   : Disabled...dm01cel01: Disk Cache Policy   : Disableddm01cel02: Disk Cache Policy   : Disabled...

   If any server shows different output, then identify the LUN and use the following command to reset the policy to Disabled:

   MegaCli64 -LDSetProp -DisDskCache -LUNn -a0

   In the preceding command, LUNn is the LUN number, such as L2.

Verifying the InfiniBand Network

The following procedure describes how to verify the InfiniBand network:

1. Visually check all the InfiniBand cable connections within the rack. The port lights should be on, and the LEDs should be on. Do not press each connector to verify connectivity.

2. Log in as the root user on any component in the rack.

3. Verify the InfiniBand topology using the following commands:

   # cd /opt/oracle.SupportTools/idiagtools# ./verify-topology [-t rack_size]

   The following example shows the output when the network components are correct.

   [DB Machine Infiniband Cabling Topology Verification Tool ]Is every external switch connected to every internal switch......[SUCCESS ]Are any external switches connected to each other................[SUCCESS ]Are any hosts connected to spine switch..........................[SUCCESS ]Check if all hosts have 2 CAs to different switches..............[SUCCESS ]Leaf switch check:cardinality and even distribution..............[SUCCESS ]Check if each rack has an valid internal ring....................[SUCCESS ]

   In the preceding command, rack_size is the size of the rack. The -t rack_size option is needed if the rack is Exadata Database Machine Half Rack or Exadata Database Machine Quarter Rack. Use halfrack for Exadata Database Machine Half Rack, and quarterrack for Exadata Database Machine Quarter Rack.

   The following example shows the output when there is a bad InfiniBand switch to InfiniBand cable connection:

   #./verify-topology[DB Machine Infiniband Cabling Topology Verification Tool ]Is every external switch connected to every internal switch......[SUCCESS ]Are any external switches connected to each other................[SUCCESS ]Are any hosts connected to spine switch..........................[SUCCESS ]Check if all hosts have 2 CAs to different switches..............[SUCCESS ]Leaf switch check:cardinality and even distribution..............[SUCCESS ]Check if each rack has an valid internal ring....................[ERROR ]Switches 0x21283a87cba0a0 0x21283a87b8a0a0 have 6 connections between them.They should have at least 7 links between them

   The following example shows the output when there is a bad InfiniBand cable on a database server:

   #./verify-topology[DB Machine Infiniband Cabling Topology Verification Tool ]Is every external switch connected to every internal switch......[SUCCESS ]Are any external switches connected to each other................[SUCCESS ]Are any hosts connected to spine switch..........................[SUCCESS ]Check if all hosts have 2 CAs to different switches..............[ERROR ]Node db01 has 1 endpoints.(Should be 2)Port 2 of this node is not connected to any switch--------fattree End Point Cabling verification failed-----Leaf switch check:cardinality and even distribution..............[ERROR ]Internal QDR Switch 0x21283a87b8a0a0 has fewer than 4 compute nodesIt has only 3 links belonging to compute nodes                  [SUCCESS ]Check if each rack has an valid internal ring...................[SUCCESS ]

   The following example shows the output when there is a bad connection on the switch and the system:

   #./verify-topology[DB Machine Infiniband Cabling Topology Verification Tool ]Is every external switch connected to every internal switch......[SUCCESS ]Are any external switches connected to each other................[SUCCESS ]Are any hosts connected to spine switch..........................[SUCCESS ]Check if all hosts have 2 CAs to different switches..............[ERROR ]Node burxdb01 has 1 endpoints.(Should be 2) Port 2 of this node is not connected to any switch--------fattree End Point Cabling verifation failed-----Leaf switch check:cardinality and even distribution..............[ERROR ]Internal QDR Switch 0x21283a87b8a0a0 has fewer than 4 compute nodes It has only 3 links belonging to compute nodes...................[SUCCESS ]Check if each rack has an valid internal ring....................[ERROR ]Switches 0x21283a87cba0a0 0x21283a87b8a0a0 have 6 connections between themThey should have at least 7 links between them

Selecting the Operating System

Oracle Exadata Database Machine is shipped with the Linux operating system and Oracle Solaris operating system for the Oracle Database servers. Linux is the default operating system. The following procedure describes how to select Oracle Solaris as the operating system:

1. Log in as the root user.

2. Change to the /opt/SupportTools directory.

3. Run the following command:

   defaultOSchoose.pl

   The script will prompt as follows:

   Default OS is : LINUX_BOOT_0Please choose new default OS:[0] LINUX_BOOT_0[1] SOLARIS_BOOT_1[2] SOLARIS_BOOT_2Please type the number you would like to make a new default OS:

4. Enter 1 to select SOLARIS_BOOT_1 for the operating system, and press Enter.

Reclaiming Disk Space After Selecting the Operating System

After selecting an operating system,reclaim the disk space occupied by the other operating system. The disk space is shipped in the following configurations:

• The four-disk database servers in Oracle Exadata Database Machine X2-2 are shipped with Linux using first two disks in hardware RAID-1 with RAID-1 implemented as the disk controller configuration, and the second two disks used by Oracle Solaris in a ZFS RAID-1 configuration.

• For the eight-disk database servers in Oracle Exadata Database Machine X2-8 Full Rack, Linux resides on first four disks configured as three disks in RAID-5 at the disk controller level with the fourth disk as the global hot spare. Oracle Solaris uses the other four disks with two disks making one ZFS RAID-1 for root pool, and two disks making one ZFS RAID-1 for the data pool.

The following sections describe how to reclaim the disk space:

• Reclaiming Disks for the Linux Operating System

• Reclaiming Disks for the Oracle Solaris Operating System

Caution:

Do not restart Oracle Exadata Database Machine or interrupt the reclamation process.

Reclaiming Disks for the Linux Operating System

Reclaiming the disks on a four-disk database server with Linux as the operating system, converts the system to have three disks RAID-5 with the fourth disk as the hot spare at the disk controller. Reclaiming the disks on an eight-disk database server with Linux as the operating system, creates seven-disk RAID-5 with the eighth disk as the hot spare at the disk controller level.

The following procedure describes how reclaim the disk space used by the Oracle Solaris operating system. This procedure should be performed on each database server.

1. Log in as the root user.

2. Change to the /opt/oracle.SupportTools directory.

3. Check the current disk configuration using the following command:

   ./reclaimdisks.sh -check

   The command returns a detailed layout of the logical and physical disks. For Oracle Exadata Database Machine X2-2, the last line of the output should be the following:

   [INFO] Valid dual boot configuration found for Linux: RAID1 from 2 disks 

   For Oracle Exadata Database Machine X2-8 Full Rack, the last line of the output should be the following:

   [INFO] Valid dual boot configuration found for Linux: RAID5 from 3 disks and 1 global hot spare disk 

4. Start the disk reclamation process using the following command:

   ./reclaimdisks.sh -free -reclaim

   The command frees any Oracle Solaris-configured disks, and reclaims all free disks for Linux. The process may take two hours for Oracle Exadata Database Machine X2-2, and five hours for Oracle Exadata Database Machine X2-8 Full Rack. To check the progress of the reclamation process, use the following command:

   tail -f /var/log/cellos/reclaimdisks.bg.log

5. Run the following command when the disk reclamation process is complete:

   ./reclaimdisks.sh -check

   For Oracle Exadata Database Machine X2-2, the last line of the output should be the following:

   INFO] Valid single boot configuration found for Linux: RAID5 from 3 disks and 1 global hot spare disk 

   For Oracle Exadata Database Machine X2-8 Full Rack, the last line of the output should be the following:

   [INFO] Valid single boot configuration found for Linux: RAID5 from 7 disks and1 global hot spare disk 

Reclaiming Disks for the Oracle Solaris Operating System

For four-disk database servers with Oracle Solaris as the operating system, reclaiming disks creates a data pool with two reclaimed disks in ZFS RAID-1 configuration. For the eight-disk database server with Oracle Solaris as the operating system, reclaiming disks results in two ZFS RAID-1 sets created from the reclaimed disks to the existing data pool. The /opt/oracle.SupportTools/disk_map.pl script on Oracle Solaris can assist in mapping the physical enclosure:slot location of disks to the device names, ZFS pools and the mount points for the pools.

The following procedure describes how reclaim the disk space used by the Linux operating system. This procedure should be performed on each database server.

1. Log in as the root user on the database server.

2. Change to the /opt/SupportTools directory.

3. Install the latest reclaimdisks.pl script from patch 12854525.

4. Run the reclaimdisks.pl script as follows:

   reclaimdisks.pl [--unattended]

   The script runs in interactive mode. To run the script in unattended mode, use the --unattended option.

   Note:

   In some cases, the system is unable to restart because the boot disks are not marked as such in the LSI controller. If this occurs, then do one of the following:

   • Boot in single user mode using the Oracle Solaris flash drive, and run the following command:

     # /opt/MegaRAID/MegaCLI - AdpBootDrive set L1 -a0

   • Use the LSI preboot command line interface as follows:

     1. Press CTRL+Y while powering the database server. This brings up the LSI preboot interface.

     2. Run the following command at the prompt:

        $ -adpbootdrive set l1 -a0

5. Respond to the prompts, as needed. The script prompts for confirmation to remove the Linux virtual disks, and to create Oracle Solaris virtual disks.

The following is an example of the output when running the script on Oracle Exadata Database Machine X2-8 Full Rack:

This script changes the Linux virtual and physical disks to Solaris disks. The system has identified unused virtual and physical disks.  The results are as follows:     Unused Virtual Disks:      /opt/MegaRAID/MegaCli -LDInfo -L0 -a0      Virtual Drive: 0 (Target Id: 0)      Name               :DBSYS       RAID Level         : Primary-5, Secondary-0, RAID Level       Qualifier-3      Size               : 557.75 GB      State              : Optimal       Stripe Size        : 1.0 MB       Number Of Drives   : 3      Span Depth         : 1      Default Cache Policy: WriteBack, ReadAheadNone, Direct, No Write        Cache if Bad BBU       Current Cache Policy: WriteBack, ReadAheadNone, Direct, No Write      Cache if Bad BBU       Access Policy      : Read/Write       Disk Cache Policy  : Disabled       Encryption Type    : None       Number of Dedicated Hot Spares: 1   0 : EnclId - 17 SlotId - 3 The virtual disks will be deleted. Do you want to proceed?  (yes/no) yesThis is an irreversible operation. Confirm that you want to proceed. (yes/no)  yesDeleting the virtual disks ...  /opt/MegaRAID/MegaCli -CfgLdDel -L0 -a0 Adapter 0: Deleted Virtual Drive-0(target id-0) Done.Disabling hot spare disks ...  /opt/MegaRAID/MegaCli -pdHSP -Rmv -PhysDrv[17:3] -a0 Adapter: 0: Remove Physical Drive at EnclId-17 SlotId-3 as Hot Spare Success.Unused Physical Drives:/opt/MegaRAID/MegaCli -pdInfo -PhysDrv[17:3] -a0  Slot Number: 3  Device Id: 14  PD Type: SAS  Raw Size: 279.396 GB [0x22ecb25c Sectors]  Inquiry Data: SEAGATE ST930003SSUN300G0868102570CHPY/opt/MegaRAID/MegaCli -pdInfo -PhysDrv[17:0] -a0   Slot Number: 0  ...New Virtual Drives will be created from these Physical Disks. Do you want to proceed? (yes/no) yes/opt/MegaRAID/MegaCli -CfgLdAdd -R0[17:3] WB NORA Direct NoCachedBadBBU -strpsz1024 -a0   Adapter 0: Created VD 0 Adapter 0: Configured the Adapter!!  /opt/MegaRAID/MegaCli -CfgLdAdd -R0[17:0] WB NORA Direct NoCachedBadBBU -strpsz1024 -a0 ...  Adapter 0: Created VD 7  Adapter 0: Configured the Adapter!!  There are 4 disks from which 'data' pool may be created (or extended)  Do you want to proceed? (yes/no) yes  Creating an additional group in 'data' pool zpool add -f data mirror c2t7d0 c2t6d0  Creating an additional group in 'data' pool zpool add -f data mirror c2t5d0 c2t0d0  The disks reconfiguration has completed 

The following is sample output for Oracle Exadata Database Machine X2-2 before reclaiming disks:

zpool listNAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOTrpool 278G 42.9G 235G 15% 1.00x ONLINE -/opt/oracle.SupportTools/disks_map.pl====================================================LOCATION DISK NAME POOL NAME====================================================------------------- mirror-0 ---------------------252:3 c1t2d0s0 rpool252:2 c1t1d0s0 rpool--------------------------------------------------====================================================POOL/FILESYSTEM MOUNTPOIT====================================================rpool /rpoolrpool/ROOT legacyrpool/ROOT/solaris /arpool/dump -rpool/export /exportrpool/export/home /export/homerpool/savecore /var/crash/solarisrpool/swap -rpool/u01 /u01

The following is sample output for Oracle Exadata Database Machine X2-2 after reclaiming disks:

zpool listNAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOTdata 278G 9.34G 269G 3% 1.00x ONLINE -rpool 278G 33.6G 244G 12% 1.00x ONLINE -/opt/oracle.SupportTools/disks_map.pl====================================================LOCATION DISK NAME POOL NAME====================================================------------------- mirror-0 ---------------------252:3 c1t2d0s0 rpool252:2 c1t1d0s0 rpool--------------------------------------------------------------------- mirror-0 ---------------------252:0 c1t0d0 data252:1 c1t3d0 data--------------------------------------------------====================================================POOL/FILESYSTEM MOUNTPOIT====================================================data /datadata/u01 /u01rpool /rpoolrpool/ROOT legacyrpool/ROOT/solaris /arpool/dump -rpool/export /exportrpool/export/home /export/homerpool/savecore /var/crash/solarisrpool/swap

The following is sample output for Oracle Exadata Database Machine X2-8 Full Rack before reclaiming disks:

zpool listNAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOTdata 278G 9.29G 269G 3% 1.00x ONLINE -rpool 278G 39.7G 238G 14% 1.00x ONLINE -/opt/oracle.SupportTools/disks_map.pl====================================================LOCATION DISK NAME POOL NAME====================================================------------------- mirror-0 ---------------------16:5 c1t2d0s0 rpool16:4 c1t1d0s0 rpool--------------------------------------------------------------------- mirror-0 ---------------------16:6 c1t3d0 data16:7 c1t4d0 data--------------------------------------------------====================================================POOL/FILESYSTEM MOUNTPOIT====================================================data /datadata/u01 /u01rpool /rpoolrpool/ROOT legacyrpool/ROOT/solaris /arpool/dump -rpool/export /exportrpool/export/home /export/homerpool/savecore /var/crash/solarisrpool/swap -

The following is sample output for Oracle Exadata Database Machine X2-8 Full Rack after reclaiming disks:

zpool listNAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOTdata 834G 9.14G 825G 1% 1.00x ONLINE -rpool 278G 33.6G 244G 12% 1.00x ONLINE -/opt/oracle.SupportTools/disks_map.pl====================================================LOCATION DISK NAME POOL NAME====================================================------------------- mirror-0 ---------------------16:5 c1t2d0s0 rpool16:4 c1t1d0s0 rpool--------------------------------------------------------------------- mirror-0 ---------------------16:6 c1t3d0 data16:7 c1t4d0 data--------------------------------------------------------------------- mirror-2 ---------------------16:0 c1t5d0 data16:2 c1t6d0 data--------------------------------------------------------------------- mirror-1 ---------------------16:3 c1t0d0 data16:1 c1t7d0 data--------------------------------------------------====================================================POOL/FILESYSTEM MOUNTPOIT====================================================data /datadata/u01 /u01rpool /rpoolrpool/ROOT legacyrpool/ROOT/solaris /arpool/dump -rpool/export /exportrpool/export/home /export/homerpool/savecore /var/crash/solarisrpool/swap -

Performing Initial Configuration of Oracle Exadata Database Machine

The following procedure describes how to perform the initial configuration of Oracle Exadata Database Machine. You will need the configuration files generated by Oracle Exadata Database Machine Configurator Spreadsheet.

1. Open a console session to the first database server. The first database server is the lowest database server in the rack, which is rack position U16.

   Note:

   The network connection and configuration for Oracle Exadata Database Machine has not occurred yet.

   See Also:

   Appendix E, "Cabling Tables for Oracle Exadata Database Machine" for rack layout

2. Log in as the root user on the first database server.

   Note:

   If you do not have the password for the root user, then contact Oracle Support Services.

3. Copy the configuration files generated by Oracle Exadata Database Machine Configurator Spreadsheet to the database server using one of the following methods:

   • Using a USB flash drive on Linux:

     1. Copy the configuration files to a USB flash drive.

     2. Connect the USB drive to the database server.

     3. Locate the USB drive using the following command:

        for x in `ls -1 /sys/block`; do udevinfo -a -p /sys/block/$x | grep \-iq 'DRIVER=="usb-storage"'; if [ $? -eq 0 ]; then echo /dev/${x}1; \fi ; done

     4. Create a directory on the database server using the following command:

        # mkdir /mnt/usb

     5. Mount the device. Use the device name given in step c. The following is an example of the command.

        # mount -t vfat /dev/sdb1 /mnt/usb

     6. Copy the configuration files from the USB flash drive to the onecommand directory on the database server using the following command:

        # cp /mnt/usb/path_to_files/*.* /opt/oracle.SupportTools/onecommand

        In the preceding command, path_to_files is the directory path to the configuration files on the USB flash drive.

     7. Unmount the USB flash drive using the following command:

        # umount /mnt/usb

     8. Remove the USB flash drive from the database server.

   • Using a USB flash drive on Oracle Solaris:

     1. Copy the configuration files to a USB flash drive.

     2. Connect the USB drive to the database server.

     3. Check if the USB flash drive has been automatically mounted to /media/ using a command similar to the following:

        mount | grep -i usb

        The output should be similar to the following:

        /media/USB2FlashStorage on /dev/dsk/c5t0d0p0:1read/write/nosetuid/nodevices/rstchown/hidden/nofoldcase/clamptime/ \noatime/timezone=28800/dev=981190 on Mon Apr 4 12:47:39 2011

        If the USB flash drive is not mounted automatically, then mount it manually.

     4. Identify the device for the USB flash drive using the following command:

        # mformat

        The output should be similar to the following. In this example, the device name is c5t0d0.

        Looking for devices...1. Logical Node: /dev/rdsk/c5t0d0p0Physical Node: /pci@0,0/pci108e,4844@1d,7/hub@3/storage@2/disk@0,0Connected Device: Ut163 USB2FlashStorage 0.00Device Type: RemovableBus: USBSize: 1.9 GBLabel: <Unknown>Access permissions: Medium is not write protected.

     5. Create a directory on the database server using the following command:

        # mkdir /media/USB2FlashsStorage

     6. Mount the device using the device name identified in step d. The following is an example of the command:

        # mount -F pcfs /dev/dsk/c5t0d0p0:c /media/USB2FlashStorage

     7. Copy the configuration files from the USB flash drive to the onecommand directory on the database server using the following command:

        # cp /media/USB2FlashStorage/path_to_files/*.* /opt/oracle.SupportTools/onecommand

        In the preceding command, path_to_files is the directory path to the configuration files on the USB flash drive.

     8. Unmount the USB flash drive using the following command:

        # umount /media/USB2FlashStorage

     9. Remove the USB flash drive from the database server.

   • Using a temporary network connection:

     This option is only needed if there were IP address conflicts, and Oracle Exadata Database Machine is not already connected to the network.

     1. Connect a network cable to the Ethernet port labeled NET0 on the back of the first database server.

     2. Bring up the eth0:1 interface as follows:

        For Linux:

        # ifconfig eth0:1 real_ip netmask real_netmask up

        In the preceding command, real_ip is the IP address to assign to the database server, and real_netmask is the subnet mask to assign to the database server.

        For Oracle Solaris:

        ifconfig eth0:1 plumbifconfig eth0:1 real_ip netmask real_netmask up

     3. Add the default route as follows:

        For Linux:

        # route add -net 0.0.0.0 netmask 0.0.0.0 gw real_gateway

        In the preceding command, real_gateway is the default gateway to use for the database server.

        For Oracle Solaris:

        route add default real_gateway

     4. Copy the configuration files to the /opt/oracle.SupportTools/onecommand directory on the database server using a file transfer utility such as scp.

     5. Disconnect the network cable from Ethernet port NET0.

        Note:

        It is not necessary to reconfigure the network because the network will be reconfigured automatically during the restart step.

4. Determine the Oracle Exadata Storage Server Software release currently installed on Oracle Exadata Database Machine using the following command:

   # imagehistory

   The choice of configuration file depends on the version and imaging mode shown in the output from the command. For Oracle Exadata Database Machine at release 11.2.2.1.0 or later, use the preconf.csv file. For earlier releases, do the following:

   • If the version is 11.2.1.3.0 and the imaging mode is fresh as shown in the following, then use the preconf-11-2-1-2-2.csv file.

     Version                  : 11.2.1.3.0Image activation date    : 2010-08-13 13:12:59 -0700Imaging mode             : freshImaging status           : success

   • If the imaging mode is patch as shown in the following, then use the preconf-11-2-1-1-0.csv file.

     Version                  : 11.2.1.3.0Image activation date    : 2010-08-21 18:27:54 -0500Imaging mode             : patchImaging status           : success

   If the release is release 11.2.1.3.1 or later, then go to step 5. If the release is earlier than release 11.2.1.3.0, then do the following:

   1. Get the network interface MAC addresses for all nodes by running the fetch.macs.sh script. Send the output to grep for the required customer network interfaces and redirect to a file. MAC addresses for eth1 and eth2 are not needed if BONDETH0 is used. When using the script, use full, half, or quarter to reflect the type of rack.

      Note:

      As of release 11.2.2.1.0, the names used for bonding changed. The names are BONDIB0 for the InfiniBand bonding and BONDETH0 for Ethernet bonding. In earlier releases, the names were BOND0 and BOND1, respectively.

      The following is an output example for Exadata Database Machine Full Rack when eth0, eth1 and eth3 are used, and BONDETH0 is not used:

      #cd /opt/oracle.SupportTools/firstconf#fetch_macs.sh full |grep 'eth0\|eth1\|eth3' > mymacs#cat mymacs192.168.1.22: fcell02  eth0  00:14:4F:CA:F8:E6192.168.1.21: fcell01  eth0  00:21:28:47:3D:7E…..192.168.1.9:  fdb02 eth0  00:21:28:47:3D:7E192.168.1.8:  fdb01 eth0  00:14:4F:CA:FA:AE…..192.168.1.9:  fdb02 eth1  00:11:1F:CA:CC:2F192.168.1.8:  fdb01 eth1  00:14:4F:E7:89:89…..192.168.1.9:  fdb02 eth3  00:11:1F:CA:CF:2G192.168.1.8:  fdb01 eth3  00:14:4F:E8:89:89

      The following is an output example for Exadata Database Machine Full Rack when using BONDETH0, eth0, and eth3 are used:

      #cd /opt/oracle.SupportTools/firstconf#fetch_macs.sh full |grep 'eth0\|eth3' > mymacs #cat mymacsexample of output:192.168.1.22: fcell02  eth0  00:14:4F:CA:F8:E6192.168.1.21: fcell01  eth0  00:21:28:47:3D:7E….192.168.1.9:  fdb02 eth0  00:21:28:47:3D:7E192.168.1.8:  fdb01 eth0  00:14:4F:CA:FA:AE192.168.1.9:  fdb02 eth3  00:11:1F:CA:CC:2F192.168.1.8:  fdb01 eth3  00:14:4F:E7:89:89

   2. Use the vi editor to update the preconf-11-2-1-1-0.csv file with the network interface MAC address values in the mymacs file. Make sure to match the node type and number, and avoid typing errors. Network interface eth0 always needs to be updated on the database servers and Exadata Storage Servers. The network MAC addresses needs to be updated for any customer-specified unbonded network interface such as eth1, eth2, and eth3 on the database servers.

5. Validate the preconf.csv file using the following command:

   # /opt/oracle.cellos/ipconf.pl -verify \  -preconf /opt/oracle.SupportTools/onecommand/preconf.csv

   In the preceding command, preconf.csv is the name of the configuration file. The name is preconf-11-2-2-1-0.csv for releases 11.2.2.1 and later, preconf-11-2-1-2-2.csv for release 11.2.1.3.0, and preconf-11-2-1-1-0.csv for releases earlier than release 11.2.1.3.0.

   Note:

   All validation errors must be fixed before continuing.

6. Run the applyconfig.sh script using the following command:

   # cd /opt/oracle.SupportTools/firstconf# ./applyconfig.sh -preconf /opt/oracle.SupportTools/onecommand/preconf.csv \  -dbm-node-ips size

   In the preceding command, size is the size of Oracle Exadata Database Machine. Use full for Exadata Database Machine Full Rack, half for Exadata Database Machine Half Rack, or quarter for Exadata Database Machine Quarter Rack.

   The applyconfig.sh script performs network configuration for all database servers and Exadata Storage Servers. All servers restart during the process.

7. Connect the corporate network cable for the management network to the Cisco Ethernet switch.

8. Connect the corporate network cables for the client access network to the database servers.

9. Restart all database servers and Exadata Storage Servers.

   See Also:

   "Powering On and Off Oracle Exadata Rack" for information about restarting the servers

10. Log in to the first database server to verify network connectivity using the following commands:

    # /opt/oracle.SupportTools/onecommand#./checkip.sh -m post_applyconfig

    Note:

    The preceding command can be run on a Linux server outside of Oracle Exadata Database Machine to verify network connectivity to Oracle Exadata Database Machine. Copy the checkip.sh and dbm.dat files to the external system, and run the command as shown in this step.

11. Use the Oracle OneCommand utility to install and configure the software for Oracle Exadata Database Machine.

Loading the Configuration Information and Installing the Software

The Oracle OneCommand utility loads the network settings, creates the user accounts, installs Oracle Database software, and secures the system based on the information the configuration files. You can also choose to do the process manually. The following sections describe each process.

• Configuring Oracle Exadata Database Machine Using Oracle OneCommand Utility

• Configuring Oracle Database and Oracle ASM Instances for Oracle Exadata Database Machine Manually

See Also:

Table 4-1, "Files for Oracle Exadata Database Machine Configuration" for descriptions of the configuration files generated by Oracle Exadata Database Machine Configurator Spreadsheet

Configuring Oracle Exadata Database Machine Using Oracle OneCommand Utility

The Oracle OneCommand utility is used to configure Oracle Exadata Database Machine. The utility works with the information provided in Oracle Exadata Database Machine Configurator Spreadsheet.

The following procedure describes how to configure Oracle Exadata Database Machine using the Oracle OneCommand utility:

1. Log in as the root user on the first database server.

   Note:

   If you do not have the password for the root user, then contact Oracle Support Services.

2. Go to My Oracle Support at

   https://support.oracle.com

3. Download the latest Oracle OneCommand utility, and follow the steps in the associated README file. See My Oracle Support note 888828.1 for details.

4. Download all necessary Oracle Exadata Storage Server Software and Oracle Database patches. See My Oracle Support note 888828.1 for details.

5. Apply the Oracle OneCommand utility update according to the instructions included in the README file.

6. Change to the following directory:

   # cd /opt/oracle.SupportTools/onecommand

7. Run the appropriate command for your release as follows. Oracle recommends using the -s option to run the script one step at a time.

   • For Oracle Database 11g Release 2 (11.2.0.2)

     ./deploy112.sh -l

   • For Oracle Database 11g Release 2 (11.2.0.3)

     ./deploy11203.sh -l

   In the preceding command, -l is the option to list the configuration steps. The following table lists the options for the script.

   OptionDescription-hDescribe options for the script.-s nRun step n of the Oracle OneCommand utility.-r n-NRun steps n through N of the Oracle OneCommand utility.-lList the steps of the Oracle OneCommand utility.
   

Table 6-41 lists the steps of the Oracle OneCommand utility. The steps before grid disk creation (step 17) can be run more than once. Once the script is run to the end, it cannot be run again.

Table 6-41 Steps of the Oracle OneCommand Utility

Step NumberNameDescription

Step 0

ValidateEnv

Validate the environment.

Step 1

CreateWorkDir

Create work directory.

Step 2

UnzipFiles

Unzip files.

Step 3

setupSSHroot

Setup SSH for the root user.

Step 4

UpdateEtcHosts

Update the /etc/hosts directory.

Step 5

CreateCellipinitora

Create the cellip.ora and cellinit.ora files

Step 6

ValidateIB

Validate the InfiniBand network.

Step 7

ValidateCell

Validate the cells.

Step 8

PingRdsCheck

Check RDS using the ping command.

Step 9

RunCalibrate

Calibrate the system.

Step 10

CreateUsers

Create users.

Step 11

SetupSSHusers

Set up SSH for the user accounts.

Step 12

CreateGridDisks

Create the grid disks.

Step 13

GridSwInstall

Install the grid software.

Step 14

PatchGridHome

Patch the grid home software.

Step 15

RelinkRDSGI

Relink Reliable Data Socket (RDS) for the Grid Infrastructure.

Step 16

GridRootScripts

Run the grid root scripts.

Step 17

DbSwInstall

Install the Oracle Database software.

Step 18

PatchDBHomes

Patch the database software.

Step 19

CreateASMDiskgroups

Create the Oracle ASM disk groups.

Step 20

DbcaDB

Run Oracle Database Configuration Assistant.

Step 21

DoUnlock

Unlock Oracle Grid Infrastructure.

Step 22

RelinkRDSDb

Relink RDS protocol.

Step 23

LockUpGI

Lock Oracle Grid Infrastructure.

Step 24

SetupCellEmailAlerts

Set up e-mail alerts for Exadata Storage Servers.

Step 25

ApplySecurityFixes

Apply any security fixes.

Step 26

ResecureMachine

Secure Oracle Exadata Database Machine.

To configure a database instance to access cell storage, ensure that the COMPATIBLE parameter is set to 11.2.0.2 in the database initialization file.

You can view the initialization parameter file from the operating system or use the SQL*Plus administration command SHOW PARAMETER if the database is running. For example:

SQL> SHOW PARAMETERSQL> SHOW PARAMETER compatible

If necessary, set the COMPATIBLE initialization parameter in the initialization parameter file to 11.2.0.2 or later. For example:

COMPATIBLE='11.2.0.2'

The COMPATIBLE parameter cannot be changed dynamically. If you change the value in the initialization parameter file, then the database must be shut down and restarted for the change to take effect.

Configuring Initialization Parameters for an Oracle ASM Instance

To enable an Oracle ASM instance to discover and access Exadata Storage Server grid disks, you must configure the ASM_DISKSTRING initialization parameter as follows:

• Set the ASM_DISKSTRING initialization parameter to the empty string ('') to discover all grid disks on the cells listed in the cellip.ora file. This causes Oracle ASM discovery for /dev/raw/* and any ASMLIB disks on any non-Exadata Storage Servers.

• Set the ASM_DISKSTRING initialization parameter to restrict the discovery by Oracle ASM to only specific sets of disks, as shown in the examples.

  • To discover only a subset of cells, use a pattern for the ASM_DISKSTRING value such as the following:

    o/cell-connect-info-pattern/griddisk-name-pattern. 

    For example:

    ASM_DISKSTRING = 'o/*/data*'

  • To discover non-Exadata Storage Server disks and all grid disks on all the cells specified in the cellip.ora file, use a wildcard pattern for Exadata Storage Server grid disks. For example:

    ASM_DISKSTRING = '/dev/rdsk/disk*', 'o/*/*'

You can change the ASM_DISKSTRING initialization parameter when the Oracle ASM instance is running with the SQL ALTER SYSTEM command. If you edit the ASM_DISKSTRING initialization parameter in the initialization parameter file when the Oracle ASM instance is running, then the Oracle ASM instance must be shut down and restarted for the change to take effect.

Installing Oracle Enterprise Manager Grid Control on Oracle Exadata Database Machine

The following procedure describes how to install Oracle Enterprise Manager Grid Control on Oracle Exadata Database Machine:

1. Create the /u01/usb directory on a database server.

2. Copy the USB kit to the /u01/usb directory.

3. Change to the /tmp directory.

4. Run the following command:

   zcat /u01/usb/dbm_context_linux_64.tar.Z | tar xBvpf -

5. Change to the /scripts directory using the following command:

   cd usb/scripts

6. Run the setup script using the following command:

   ./setup.sh -em | tee /tmp/setup.log

7. Enter the root password when prompted by the system.

8. Enter y or n when prompted for the Oracle Database home. If the response is n, then the system will prompt for the correct Oracle Database home.

9. Enter the Oracle Enterprise Manager Grid Control user password for each database server when prompted.

10. Enter y or n when prompted to harden the Oracle Database and Oracle ASM passwords.

11. Enter y or n when prompted for the Oracle Database home. If the response is n, then the system will prompt for the correct Oracle Database home.

The script will discover the targets automatically. It may be necessary to manually configure the Oracle Database, Oracle ASM, and Oracle Enterprise Manager Grid Control targets in OMS.

Installing Oracle Enterprise Manager Cloud Control on a Standalone Server to Monitor Oracle Exadata Database Machine

The following procedure describes how to prepare the standalone server to monitor Oracle Exadata Database Machine:

1. Log in as the root user on the standalone server.

2. Download the latest Oracle Enterprise Manager Cloud Control patches, and follow the steps in the associated README file. See My Oracle Support note 888828.1 for details.

3. Download the latest patch available from Oracle bug database. The bug number is 11852869. The following files must be downloaded:

   • p11852869_111010_Linux-x86-64_1of4.zip

   • p11852869_111010_Linux-x86-64_2of4.zip

   • p11852869_111010_Linux-x86-64_3of4.zip

   • p11852869_111010_Linux-x86-64_4of4.zip

4. Copy the downloaded files to the /tmp/emkit directory.

5. Change to the /tmp/emkit directory.

6. Run commands similar to the following to unzip the files:

   unzip p12960610_121010_Linux-x86-64_1of4.zipunzip p12960610_121010_Linux-x86-64_2of4.zipunzip p12960610_121010_Linux-x86-64_3of4.zipunzip p12960610_121010_Linux-x86-64_4of4.zip

7. Run the following command:

   cat CloudControl_Standalone00 CloudControl_Standalone01 \CloudControl_Standalone02 CloudControl_Standalone03 >   \CloudControl_Standalone_12.1.0.1.0_Linux_x86.tar.Z

8. Run the following command as the root user:

   tar xBvpf CloudControl_Standalone_12.1.0.1.0_Linux_x86.tar.Z

9. Change to the /tmp/emkit/common directory.

10. Run the following command as the root user:

    (sh setupem.sh -emOnly -emUser em_user  -emBase em_base|tee /tmp/setupem.log) \3>&1 1>&2 2>&3 | tee /tmp/setupem.err

    In the preceding command, em_user is the user who owns Oracle Management Server (OMS), and em_base is the directory to be used when creating OMS and agent home. The operating system user must exist and belong to the oinstall and dba groups.

    Note:

    If the message Ambiguous output redirect appears after running the command, then change to the bash shell.

    See Also:

    Oracle Configuration Manager Installation and Administration Guide

11. Log in to OMS as the sysman user using the OMS URL, and verify the setup as follows:

    1. Accept the license agreement.

    2. Verify the agent and host targets appear under the Targets tab.

12. Discover the Management Repository instance as follows:

    1. Log in to Oracle Enterprise Manager Cloud Control as the sysman user.

    2. Select Databases from the Targets tab.

    3. Select Search List.

    4. Click Add.

    5. Click the search icon.

    6. Select EM host.

    7. Click Continue.

    8. Click the configuration icon when the database is discovered.

    9. Enter the monitor password on the Configure Database Instance page.

    10. Click Test Connection.

    11. Use the wizard

    12. Save the targets.

13. Enable Oracle Database Lifecycle Management Pack as follows:

    1. Select Management Packs from the Setup page.

    2. Select Management Pack Access.

    3. Select Pack Access Agreed.

    4. Click Apply.

    5. Log out, and then log back in.

14. Change the passwords for the database and OMS to secure the system.

Installing Oracle Enterprise Manager Cloud Control Agents on Oracle Exadata Database Machine

The following procedure describes how to install Oracle Enterprise Manager Cloud Control agents on Oracle Exadata Database Machine.

Note:

The steps in this procedure assume that Oracle Enterprise Manager Cloud Control information was entered into the Oracle Exadata Database Machine Configurator Spreadsheet, and the /opt/oracle.SupportTools/onecommand/em.param file is available. If not, then create the file. Refer to "About the em.param File" for an example of the file.

See Also:

Chapter 4, "Using the Oracle Exadata Database Machine Configurator Spreadsheet"

1. Log in as the root user on the first database server.

2. Create the /tmp/emkit directory on the first database server on Oracle Exadata Database Machine.

3. Download the Oracle Enterprise Manager Cloud Control Agent kit from Oracle, selecting the kit that is appropriate for your system.

4. Copy the downloaded kit to the /tmp/emkit directory.

5. Change to the /tmp/emkit directory.

6. Run the following commands to unzip the file:

   • For Linux:

     unzip  p12960596_121010_Linux-x86-64.ziptar xBvpf CloudControl_Agent_12.1.0.1.0_LINUX_X64.tar.z

   • For Oracle Solaris:

     unzip  p12960596_121010_Solaris86-64.ziptar xBvpf CloudControl_Agent_12.1.0.1.0_SOLARIS_X64.tar.z

7. Change to the /tmp.emkit/common directory.

8. Run the setup script using the following command:

   (sh setupem.sh | tee /tmp/setupag.log) 3>&1 1>&2 2>&3 | tee /tmp/setupag.err

   Setup for SSH for the root user will start on all servers.

9. Enter the password for each server when prompted. The system prompts twice for each password.

10. Enter the Management agent registration password for the agent on the local server.

11. Discover Oracle Exadata Database Machine as follows:

    1. Log in to Oracle Enterprise Manager Cloud Control.

    2. Select Add Non-Host Targets Using Guided Process.

    3. Select Oracle Exadata Database Machine as the target type.

    4. Click Add Using Guided Discovery.

    5. Use the wizard.

12. Discover the grid infrastructure as follows:

    1. Select Add Targets from the Setup page.

    2. Select Add Targets Manually.

    3. Select Add Non-Host Targets Using Guided Process.

    4. Select Oracle Exadata Database Machine as the target type.

    5. Click Add Using Guided Discovery.

    6. Use the wizard.

13. Discover Oracle RAC as follows:

    1. Select Add Targets from the Setup page.

    2. Select Add Targets Manually.

    3. Select Add Non-Host Targets Using Guided Process.

    4. Select Oracle Database, Listener and Automatic Storage Management as the target type.

    5. Click Add Using Guided Discovery.

    6. Use the wizard.

    7. Configure the Oracle RAC databases. The dbsnmp password is required for this step.

    8. Configure Oracle ASM. The asmsnmp password is required for this step.

14. Enter the em_user user password for the server when prompted by the system.

15. Enter y or n when prompted for the Oracle Database home. If the response is n, then the system prompts for the correct Oracle Database home.

16. Respond to the prompts, as needed.

    Note:

    You can configure Oracle Configuration Manager (OCM) using the emocmrsp utility. The utility prompts for My Oracle Support credentials and connection information, and configures OCM collector on all Exadata Storage Servers.

    The utility creates the /opt/oracle.SupportTools/dbmcprov/ccr/exadata_homes.csv file which includes all servers and their Oracle homes. This file can be used with the Mass Deployment utility to re-configure OCM collector.

    Use the software CSI assigned to Oracle Exadata Database Machine when configuring OCM collector.

17. Remove the SSH setup when prompted by the system.

See Also:

My Oracle Support note 1110675.1 for information about configuring plug-ins

About the em.param File

The /opt/oracle.SupportTools/onecommand/em.param file is generated by Oracle Exadata Database Machine Configurator Spreadsheet. The following is an example of the file:

# This is em.param file# Written : 3/30/2011EM_VERSION=1.0OMS_LOCATION=REMOTE_CORP_GCEM_BASE=/myem/myemhomeOMS_HOST=myoms.example.comOMS_PORT=4900EM_CELLS=(cell_node1 cell_node2 cell_node3)EM_COMPUTE_ILOM_NAME=(dbhost1-ilom dbhost2-ilom)EM_COMPUTE_ILOM_IP=(x.x.x.x x.x.x.x)machinemodel="X2-2 Quarter rack"EM_USER=oracleEM_PASSWORD=*******swikvmname=my-kvmswikvmip=x.x.x.xswiipname=myciscoswiipip=x.x.x.xswiib1name=my-ib1swiib1ip=x.x.x.xswiib2name=my-ib2swiib2ip=x.x.x.xswiib3name=my-ib3swiib3ip=x.x.x.xpduaname=my-pduapduaip=x.x.x.xpdubname=my-pdubpdubip=x.x.x.x

In the preceding example, the OMS_LOCATION setting has two options. The setting OMS_LOCATION=REMOTE_CORP_GC means that the agent is configured on this Oracle Exadata Rack. The other setting is OMS_LOCATION=NO_GC, which means the agent will not be configured on Oracle Exadata Rack. The settings are based on the Install 11g Management Agent option in Oracle Exadata Database Machine Configurator Spreadsheet. The OMS_PORT setting is the upload port.

Note:

If Oracle Enterprise Manager Cloud Control configuration is being run after the initial installation of Oracle Exadata Database Machine, then verify the contents of the em.param file are still valid.

The following table lists the em.param switch parameters and their descriptions.

Switch ParameterDescriptionswikvmnameName of the KVM switch.swikvmipIP address of the KVM switch.swiipnameName of the Cisco switch.swiipipIP address of the Cisco switch.swiib1nameName of first InfiniBand switch. This is a leaf switch.swiib1ipIP address of first InfiniBand switch.swiib2nameName of second InfiniBand switch. This is a leaf switch.swiib2ipIP address of second InfiniBand switch.swiib3nameName of third InfiniBand switch. This is a spine switch.swiib3ipIP address of third InfiniBand switch.pduanameName of PDU-A.pduaipIP address of PDU-A.pdubnameName of PDU-B.pdubipIP address of PDU-B.

Removing Oracle Enterprise Manager Grid Control Agents from Oracle Exadata Database Machine

The following procedure describes how to manually remove Oracle Enterprise Manager Grid Control agents from Oracle Exadata Database Machine:

1. Follow the deinstallation instructions in Oracle Enterprise Manager Grid Control Advanced Installation and Configuration Guide.

2. Remove the kit from the /tmp directory using the following command:

   $ rm -rf /tmp/usb

3. Remove dbmcprov from the /opt/Oracle.SupportTools directory using the following command:

   $ cd /opt/Oracle.SupportTools/$ rm -rf dbmcprov

4. Remove the targets as follows:

   • Oracle Enterprise Manager Grid Control was installed on Oracle Exadata Database Machine:

     1. Log in as the root user on the first database server.

     2. Change to the /opt/Oracle.SupportTools/dbmcprov/scripts directory.

     3. Run the following command:

        ./rollback.sh -cleanAgents

     4. Enter the root password when prompted by the system.

   • Oracle Enterprise Manager Grid Control was installed on standalone server:

     1. Log in as the root user on the standalone server.

     2. Change to the /opt/Oracle.SupportTools/dbmcprov/scripts directory.

     3. Run the following command:

        ./rollback.sh -cleanStandAloneOMS

5. Delete the targets from OMS. The following targets should be deleted:

   • Oracle ASM targets for Oracle Exadata Database Machine

   • Oracle Cluster database targets

   • Oracle Database instance targets

   • Cisco switch targets

   • Oracle InfiniBand switch targets

   • KVM switch targets

   • PDU switch targets

   • Exadata Storage Server targets

   • Oracle ILOM server targets

   • Oracle Database server targets

   • Oracle Database agent targets

   • Listeners on Oracle Database server targets

Removing Oracle Enterprise Manager Cloud Control Agents from Oracle Exadata Database Machine When Using a Standalone Server

The following procedure describes how to remove Oracle Enterprise Manager Cloud Control agents from Oracle Exadata Database Machine:

1. Log in as the root user on the first database server. This is the same server that the installation was done on.

2. Change to the /opt/Oracle.SupportTools/emkit/common directory.

3. Run the following command:

   (sh rollback.sh -cleanAgents -emBase em_base | tee /tmp/cleanupag.log) \3>&1 1>&2 2>&3 | tee /tmp/cleanupag.err

   In the preceding command, em_base is the directory for the agent.

4. Remove the Oracle Exadata Database Machine targets from the Oracle Enterprise Manager Cloud Control server. The following targets should be deleted:

   • Oracle ASM targets for Oracle Exadata Database Machine

   • Oracle Cluster database targets

   • Oracle Database instance targets

   • Cisco switch targets

   • Oracle InfiniBand switch targets

   • KVM switch targets

   • PDU switch targets

   • Exadata Storage Server targets

   • Oracle ILOM server targets

   • Listeners on Oracle Database server targets

   • Host targets

   • Oracle agent targets

   • Oracle home targets

Removing Oracle Enterprise Manager Cloud Control from a Standalone Server

The following procedure describes how to remove Oracle Enterprise Manager Cloud Control from a standalone server:

1. Log in as the root user to the first database server where the setup was done.

2. Change to the /opt/Oracle.SupportTools/emkit/common directory.

3. Run the following command:

   (sh rollback.sh -cleanStandAloneOMS -emBase em_base | tee /tmp/cleanupem.log) \3>&1 1>&2 2>&3 | tee /tmp/cleanupem.err

   In the preceding command, em_base is the directory for OMS and agent home.

4. Ensure the directory for OMS and agent home directory, em_base, is empty.

Installing Oracle Enterprise Manager Grid Control on Oracle Exadata Database Machine

The following procedure describes how to install Oracle Enterprise Manager Grid Control on Oracle Exadata Database Machine:

1. Create the /u01/usb directory on a database server.

2. Copy the USB kit to the /u01/usb directory.

3. Change to the /tmp directory.

4. Run the following command:

   zcat /u01/usb/dbm_context_linux_64.tar.Z | tar xBvpf -

5. Change to the /scripts directory using the following command:

   cd usb/scripts

6. Run the setup script using the following command:

   ./setup.sh -em | tee /tmp/setup.log

7. Enter the root password when prompted by the system.

8. Enter y or n when prompted for the Oracle Database home. If the response is n, then the system will prompt for the correct Oracle Database home.

9. Enter the Oracle Enterprise Manager Grid Control user password for each database server when prompted.

10. Enter y or n when prompted to harden the Oracle Database and Oracle ASM passwords.

11. Enter y or n when prompted for the Oracle Database home. If the response is n, then the system will prompt for the correct Oracle Database home.

The script will discover the targets automatically. It may be necessary to manually configure the Oracle Database, Oracle ASM, and Oracle Enterprise Manager Grid Control targets in OMS.

总结：一体机内部有大量的写好的脚步，从上文档发现很多配置都是脚步！