Exadata Installation (only for test and education purpose)

This post is about installing Exadata storage server while I am preparing for exadata certification. This is just for educational and learning purposes to understand exadata components and commands to get some hands on experience before exam.

Pre-requisities

• Install virtualbox
• Install Oracle Linux 5 Update 9 (just install this ersion only not even oracle linux 6, if not exadata may not start properly)
• Add a hard disk with 10gb to your virtualbox

Download the required softwares from OTN

• Oracle Linux 5 (update 9)
• Oracle Middleware software from edelivery.oracle.com
• For convenience , I made bundle of all required things in zip and you can download from here

Install required rpm's (if you have media mounted do rpm -ivh of these packages)

[root@stocell1 ~] yum install oracle-validated
[root@stocell1 ~] yum install perl
[root@stocell1 ~] yum install
[root@stocell1 ~] yum install unzip
[root@stocell1 ~] yum install perl
[root@stocell1 ~] yum install net-snmp-utils
[root@stocell1 ~] yum install "perl(LWP)"

Install required rpms (navigate to media folder and server/packages)

[root@stocell1 ~] rpm -ivh binutils compat-libcap1 compat-libstdc++-33 compat-libstdc++-33.i686 gcc gcc-c++ glibc glibc.i686 glibc-devel glibc-devel.i686 ksh libgcc libgcc.i686 libstdc++ libstdc++.i686 libstdc++-devel libstdc++-devel.i686 libaio libaio.i686 libaio-devel libaio-devel.i686 libXext libXext.i686 libXtst libXtst.i686 libX11 libX11.i686 libXau libXau.i686 libxcb libxcb.i686 libXi libXi.i686 make sysstat unixODBC unixODBC-devel unzip perl net-snmp-utils "perl(LWP)"

              [root@stocell1 ~] rpm -ivh oracle-validated

Create swap file

dd if=/dev/zero of=/swapfile1 bs=1024 count=2097152

Add rds protocol

[celladmin@stocell1 ~] modprobe rds
[celladmin@stocell1 ~] modprobe rds_tcp
[celladmin@stocell1 ~] modprobe rds_rdma
[celladmin@stocell1 ~]  vi /etc/modprobe.d/rds.conf   (insert below line and save the file)
install rds /sbin/modprobe --ignore-install rds && /sbin/modprobe rds_tcp && /sbin/modprobe rds_rdma
[celladmin@stocell1 ~] lsmod | grep rds

Stop Firewall

   [root@stocell1 ~] service iptables off
[root@stocell1 ~]service iptables stop
[root@stocell1 ~] chkconfig iptables off

Create directories & log directories

[root@stocell1 ~] mkdir /var/log/oracle
[root@stocell1 ~] mkdir /opt/oracle
[root@stocell1 ~] chmod 775 /var/log/oracle

Add hostname to /etc/hosts (find your machine ip address through ifconfig command and it to your etc/hosts file)

[root@stocell1 ~]vi /etc/hosts
127.0.0.1           localhost.localdomain  localhost

<ipaddre>  stocell1.geekslabs.com geekslabs

              [root@stocell1 ~] service network restart

Install JDK 1.5.0

[root@stocell1 ~] rpm -ivh jdk-1_5_0_15-linux-amd64.rpm

Install the exadata storage software (this will install the exadata storage software)

[root@stocell1 ~] unzip cell.bin

Just in case if already installed, and removed the folder you can uninstall rpm using following

rpm -e --noscripts cell-11.2.3.2.1_LINUX.X64_130109-1

[root@stocell1 ~] rpm -ivh cell-11.2.3.2.1_LINUX.X64_130109-1.x86_64.rpm

I have 10gb disk, create 4 1024M disks, create following script and ensure change the /dev/xvdh with your disk

[root@stocell1 ~] vi partitions.sh

#!/bin/bash
for i in {1..20}
do
cat <<EOF | fdisk /dev/sdb
n
p
{$i}

+500M
w
EOF
done

Run the script, it will create 10 partitions with 1gb each.

[celladmin@stocell1 ~] sh partition.sh

Create Softlink to the disk partitions and make them as raw disks

cd /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks
mkdir -p disks/raw
cd disks/raw
ln -s /dev/sdb1  cell12_DISK1
ln -s /dev/sdb2  cell12_DISK2
ln -s /dev/sdb3  cell12_DISK3
ln -s /dev/sdb4  cell12_DISK4

ln -s /dev/sdb5  cell12_DISK5
ln -s /dev/sdb6  cell12_DISK6
ln -s /dev/sdb7  cell12_DISK7
ln -s /dev/sdb8  cell12_DISK8
ln -s /dev/sdb9  cell12_DISK9
ln -s /dev/sdb10 cell12_DISK10
ln -s /dev/sdb11  cell12_DISK11

Create the Cell services using cellcli, as celladmin

[celladmin@stocell1 ~]su - celladmin
[celladmin@stocell1 ~]cellcli -e create cell cell12 interconnect1=eth0

Start the cell services

[celladmin@stocell1 ~]$ cellcli -e alter cell restart services all
Stopping the RS, CELLSRV, and MS services...
CELL-01509: Restart Server (RS) not responding.
Starting the RS, CELLSRV, and MS services...
Getting the state of RS services... running
Starting CELLSRV services...
The STARTUP of CELLSRV services was successful.
Starting MS services...
The STARTUP of MS services was successful.

Create Cell Disks, this will create the cell disks for all the raw disks we have created

[celladmin@stocell1 ~]$  cellcli -e create celldisk all

Login and check cell disks created or not,

[celladmin@stocell1 ~]$ cellcli
CellCLI> list celldisk
CD_DISK10_cell12 normal
CD_DISK11_cell12 normal
CD_DISK1_cell12 normal
CD_DISK2_cell12 normal
CD_DISK3_cell12 normal
CD_DISK4_cell12 normal
CD_DISK5_cell12 normal
CD_DISK6_cell12 normal
CD_DISK7_cell12 normal
CD_DISK8_cell12 normal
CD_DISK9_cell12 normal

Create Grid Disks

[celladmin@stocell1 ~]$  cellcli -e create griddisk all harddisk prefix=DATA

Login and check cell disks created or not,

CellCLI> list griddisk
DATA_CD_DISK10_cell12 active
DATA_CD_DISK11_cell12 active
DATA_CD_DISK1_cell12 active
DATA_CD_DISK2_cell12 active
DATA_CD_DISK3_cell12 active
DATA_CD_DISK4_cell12 active
DATA_CD_DISK5_cell12 active
DATA_CD_DISK6_cell12 active
DATA_CD_DISK7_cell12 active
DATA_CD_DISK8_cell12 active
DATA_CD_DISK9_cell12 active

Check Luns (in this case they are just a bunch of raw disks otherwise they point to hard disks)

CellCLI> list lun
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/cell12_DISK1 /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/cell12_DISK1 normal
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/cell12_DISK10 /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/cell12_DISK10 normal
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/cell12_DISK11 /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/cell12_DISK11 normal
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/cell12_DISK2 /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/cell12_DISK2 normal
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/cell12_DISK3 /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/cell12_DISK3 normal
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/cell12_DISK4 /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/cell12_DISK4 normal
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/cell12_DISK5 /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/cell12_DISK5 normal
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/cell12_DISK6 /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/cell12_DISK6 normal
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/cell12_DISK7 /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/cell12_DISK7 normal
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/cell12_DISK8 /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/cell12_DISK8 normal
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/cell12_DISK9 /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/cell12_DISK9 normal

Finally, Check your Cell Service Details

[celladmin@stocell1 ~]$ cellcli
 CellCLI: Release 11.2.3.2.1 - Production on Tue Aug 01 07:48:51 EDT 2017
 Copyright (c) 2007, 2012, Oracle.  All rights reserved.
 Cell Efficiency Ratio: 1
 CellCLI> list cell detail
 name:                   cell12
 bbuTempThreshold:       60
 bbuChargeThreshold:     800
 bmcType:                absent
 cellVersion:            OSS_11.2.3.2.1_LINUX.X64_130109
 cpuCount:               1
 diagHistoryDays:        7
 fanCount:               1/1
 fanStatus:              normal
 flashCacheMode:         WriteThrough
 id:                     29bf45c9-e189-4221-84bd-3683593d0e7a
 interconnectCount:      1
 interconnect1:          eth0
 iormBoost:              0.0
 ipaddress1:             10.0.0.136/24
 kernelVersion:          2.6.39-400.215.11.el5uek
 makeModel:              Fake hardware
 metricHistoryDays:      7
 offloadEfficiency:      1.0
 powerCount:             1/1
 powerStatus:            normal
 releaseVersion:         11.2.3.2.1
 releaseTrackingBug:     14522699
 status:                 online
 temperatureReading:     0.0
 temperatureStatus:      normal
 upTime:                 0 days, 0:30
 cellsrvStatus:          running
 msStatus:               running
 rsStatus:               running

MS and RS Services running and as well as cellsrv.

And if you look above make model show as fake hardware since we run as simulator not original exadata hardware.

This concludes the storage software installation and in next post we will see how to use this grid disks as ASM disks.