Login to PKENA019
1. Check the status of the cluster node by executing a command
#scstat –g
# scswitch -n -M -j Kenact-lh-rs
#scswitch -n -M -j Kenact-hasp-rs
#scswitch -n -M -j ebs-rs
#scswitch -n -M -j Kenadm-DB-rs
#scswitch -n -M -j Kencat-DB-rs
#scswitch -n -M -j Kenlsnr-res
All the above command will disable cluster monitoring.
2. root@PKENA019 # cfgadm -alv |grep permanent
SB2::memory connected configured ok base address 0x22000000000, 16777216 KBytes total, 4734056 KBytes permanent
root@PKENA019 # init 0
Now PKENA019 will be in OK prompt.
Login to SC (10.1.18.122)
su to sms-svc
T-Sky-20K-2-sc1:sms-svc:2> showboards
Retrieving board information. Please wait.
......
Location Pwr Type of Board Board Status Test Status Domain
-------- --- ------------- ------------ ----------- ------
SB0 On V3CPU Active Passed A---------------PKEND021
SB1 On V3CPU Active Passed A---------------PKEND021
SB2 On V3CPU Active Passed B---------------PKENA019
SB3 On V3CPU Active Passed C---------------PSBLD008
SB4 On V3CPU Active Passed C---------------PSBLD008
SB5 On V3CPU Active Passed C---------------PSBLD008
SB6 On V3CPU Active Passed D---------------PSAPA013
SB7 - Empty Slot Available - Isolated
SB8 On V3CPU Active Passed E---------------PEAID015
IO0 On HPCI+ Active Passed A
IO1 On HPCI+ Active Passed A
IO2 On HPCI+ Active Passed B
IO3 On HPCI+ Active Passed B
IO4 On HPCI+ Active Passed C
IO5 On HPCI+ Active Passed C
IO6 On HPCI+ Active Passed D
IO7 On HPCI+ Active Passed D
IO8 On HPCI+ Active Passed E
bash-2.05$ showkeyswitch -d b
Virtual key switch position: ON
Turn off the keyswitch of domain B .
bash-2.05$ setkeyswitch -d b off
Current virtual key switch position is "ON".
Are you sure you want to change to the "OFF" position (yes/no)? yes
Domain is down.
Waiting on exclusive access to EXB(s): 3FFFF.
Powering off: V3CPU at SB2
Powering off: HPCI+ at IO2
Powering off: EXB at EX2
Powering off: HPCI+ at IO3
bash-2.05$
T-Sky-20K-2-sc1:sms-svc:>showplatform -p domains =======Check the status
T-Sky-20K-2-sc1:sms-svc:>deleteboard SB2
SB2 successfully unassigned.
bash-2.05$
Add the board to Domain A
T-Sky-20K-2-sc1:sms-svc:>addboard -d A SB2
assign SB2
.
assign SB2 done
poweron SB2
.............
poweron SB2 done
test SB2 .......... test SB2 done
connect SB2 ...... connect SB2 done
configure SB2
....
configure SB2 done
.
notify online SUNW_cpu/cpu64
notify online SUNW_cpu/cpu68
notify online SUNW_cpu/cpu65
notify online SUNW_cpu/cpu69
notify online SUNW_cpu/cpu66
notify online SUNW_cpu/cpu70
notify online SUNW_cpu/cpu67
notify online SUNW_cpu/cpu71
.
notify add capacity (8 cpus)
notify add capacity (2097152 pages)
notify add capacity SB2 done
T-Sky-20K-2-sc1:sms-svc:>showboards -d A
bash-2.05$ showboards -d A
Retrieving board information. Please wait.
......
Location Pwr Type of Board Board Status Test Status Domain
-------- --- ------------- ------------ ----------- ------
SB0 On V3CPU Active Passed A
SB1 On V3CPU Active Passed A
SB2 On V3CPU Active Passed A
SB7 - Empty Slot Available - Isolated
SB9 - Empty Slot Available - Isolated
SB10 - Empty Slot Available - Isolated
SB11 - Empty Slot Available - Isolated
SB12 - Empty Slot Available - Isolated
SB13 - Empty Slot Available - Isolated
SB14 - Empty Slot Available - Isolated
SB15 - Empty Slot Available - Isolated
SB16 - Empty Slot Available - Isolated
SB17 - Empty Slot Available - Isolated
IO0 On HPCI+ Active Passed A
IO1 On HPCI+ Active Passed A
IO9 - Empty Slot Available - Isolated
IO10 - Empty Slot Available - Isolated
IO11 - Empty Slot Available - Isolated
IO12 - Empty Slot Available - Isolated
IO13 - Empty Slot Available - Isolated
IO14 - Empty Slot Available - Isolated
IO15 - Empty Slot Available - Isolated
IO16 - Empty Slot Available - Isolated
IO17 - Empty Slot Available - Isolated
EXAMPLES
Example 1: Assigning Boards to Domain C
To assign four boards to domain C you must have platform
privileges or domain privileges and the boards must be in
the domain available component list.
sc0:sms-user:> addboard -d C -c assign SB0 IO1 SB1 SB2
SB0 assigned to domain: C
IO1 assigned to domain: C
SB1 assigned to domain: C
SB2 assigned to domain: C
Friday, June 18, 2010
E20K SMS Upgrade from SMS 1.5 to SMS 1.6
SC PREREQUISITES,
Solaris 9 OS Requirements for SCs
As a minimum, the Solaris 9 OS version of SMS 1.6 requires:
■ Solaris 9 4/04 OS release
■ Same version of SMS software on both system controllers
Patches to be installed in SC before starting the upgradation,
■ 113027-03 patch
■ 111712-12 patch (for UltraSPARC® IV+ 1800 MHz only)
Domain PREREQUISITES,
Solaris 9 OS Requirements for Domains
As a minimum, the Solaris 9 OS requires:
■ Solaris 9 4/04 OS release.
■ Patches:
■ 113027-03 patch
■ 112233-09 patch
Solaris 10 OS Requirements for Domains
As a minimum, the Solaris 10 OS requires:
■ Solaris 10 3/05 (minimum for UltraSPARC III+, UltraSPARC III++, and
UltraSPARC IV)
■ Solaris 10 3/05 HW1 (minimum for UltraSPARC IV+ 1500 MHz and 1800 MHz
only)
The Solaris 10 1/06 OS release is supported on the domains on all processors.
1. Gather the superuser passwords for both SCs.
2. Be sure you have platadmn privileges to both SCs.
3. On both SCs, determine the directory into which you will download the SMS
software from the web.
4. Ensure that the SC data is synchronized between the two SCs by typing this
command on the main SC.
root@tatatel12-25k-sc0-eri0 # /opt/SUNWsms/bin/setdatasync backup
5. Ensure that both SC clocks are phase-locked. You can do this by looking at the
most recent messages in the platform logs, which say whether the SC clocks are
locked or not.
Preparing for Installation
1) Login to the Main SC (SC0)
2) Disable failover,
root@tatatel12-25k-sc0-eri0:sms-svc:># /opt/SUNWSMS/bin/setfailover off
3) Backup the current configuration on SC0
1a. Stop SMS
root@tatatel12-25k-sc0-eri0:sms-svc:># /etc/init.d/sms stop
2b. run smsbackup
root@tatatel12-25k-sc0-eri0:sms-svc:># /opt/SUNWSMS/bin/smsbackup directory_name
3c. Start SMS.html
root@tatatel12-25k-sc0-eri0:sms-svc:># /etc/init.d/sms start
4) Login to the Spare SC (SC1) as root
5) Backup the current configuration on SC1
1a. Stop SMS
root@tatatel12-25k-sc1-eri0:sms-svc:># /etc/init.d/sms stop
2b. run smsbackup
root@tatatel12-25k-sc1-eri0:sms-svc:># /opt/SUNWSMS/bin/smsbackup directory_name
SMS Upgrade on Spare SC:
1. Login to spare SC (SC1)
2. ftp Solaris 9 OS–SMS-1_6-S9-sparc.zip to /sun on spare SC (SC1)
3. root@tatatel12-25k-sc1-eri0:# cd /sun
4. root@tatatel12-25k-sc1-eri0:# unzip SMS-1_6-S9-sparc.zip
5. root@tatatel12-25k-sc1-eri0:# cd /sun/sms_1_6_sparc/System_Management_Services_1.6/Tools
6. root@tatatel12-25k-sc1-eri0:# ./smsupgrade /sun/sms_1_6_sparc/System_Management_Services_1.6/Product
7. Install all the SMS 1.6 patches on the spare SC which are available on EIS CD.
Switch Control to the Spare SC
1. Log in to the main system controller (sc0) as superuser.
2. Stop SMS.
root@tatatel12-25k-sc0-eri0:sms-svc:># /etc/init.d/sms stop
3. Log in to the spare SC (sc1) and change to the OpenBoot PROM prompt.
root@tatatel12-25k-sc1eri0:># shutdown -y -g0 -i0
4. Reboot the spare SC.
root@tatatel12-25k-sc1eri0:># boot –rv
To Upgrade the Spare SC Flash PROMs
1. Log in to the spare SC as a user with platadmn privileges.
2. root@tatatel12-25k-sc1-eri0:sms-svc:># flashupdate -f /opt/SUNWSMS/firmware/SCOBPimg.di sc1/fp0
3. root@tatatel12-25k-sc1-eri0:sms-svc:># flashupdate -f /opt/SUNWSMS/firmware/oSSCPOST.di sc1/fp1
4. root@tatatel12-25k-sc1-eri0:sms-svc:># /etc/init.d/sms stop
5. root@tatatel12-25k-sc1-eri0:sms-svc:># shutdown -y -g0 -i0
6. root@tatatel12-25k-sc1-eri0:sms-svc:># boot –rv
SMS Upgrade on Main SC:
1. Login to main SC (SC0)
2. ftp Solaris 9 OS–SMS-1_6-S9-sparc.zip to /sun on spare SC (SC1)
3. root@tatatel12-25k-sc0-eri0:# cd /sun
4. root@tatatel12-25k-sc0-eri0:# unzip SMS-1_6-S9-sparc.zip
5. root@tatatel12-25k-sc0-eri0:# cd /sun/sms_1_6_sparc/System_Management_Services_1.6/Tools
6. root@tatatel12-25k-sc0-eri0:# ./smsupgrade /sun/sms_1_6_sparc/System_Management_Services_1.6/Product
7. Install all the SMS 1.6 patches on the spare SC which are available on EIS CD.
To Upgrade the Main SC Flash PROMs
1. Log in to the spare SC as a user with platadmn privileges.
2. root@tatatel12-25k-sc0-eri0:sms-svc:># flashupdate -f /opt/SUNWSMS/firmware/SCOBPimg.di sc0/fp0
3. root@tatatel12-25k-sc0-eri0:sms-svc:># flashupdate -f /opt/SUNWSMS/firmware/oSSCPOST.di sc0/fp1
To Switch Control Back to the Main SC
1. Log in to the spare SC (sc1) as superuser.
2. root@tatatel12-25k-sc1-eri0:sms-svc:># /etc/init.d/sms stop
3. Log in to the main SC (sc0) and change to the OpenBoot PROM prompt.
4. root@tatatel12-25k-sc0-eri0:># shutdown -y -g0 -i0
5. root@tatatel12-25k-sc0-eri0:># boot –rv
6. Use the /etc/init.d/sms script to restart SMS on the spare SC
root@tatatel12-25k-sc1-eri0:># /etc/init.d/sms start
7. Enable failover
root@tatatel12-25k-sc0-eri0:sms-svc># /opt/SUNWSMS/bin/setfailover on
8. Verify that failover is working.
root@tatatel12-25k-sc0-eri0:sms-svc># /opt/SUNWSMS/bin/showfailover -v
To Upgrade the System Board Flash PROMs
1. Log in to the main SC as a user with platadmn privileges.
2. Use flashupdate to upgrade the CPU flash PROMs in a domain.
root@tatatel12-25k-sc0-eri0:sms-svc># flashupdate -d \/opt/SUNWSMS/hostobjs/sgcpu.flash
Solaris 9 OS Requirements for SCs
As a minimum, the Solaris 9 OS version of SMS 1.6 requires:
■ Solaris 9 4/04 OS release
■ Same version of SMS software on both system controllers
Patches to be installed in SC before starting the upgradation,
■ 113027-03 patch
■ 111712-12 patch (for UltraSPARC® IV+ 1800 MHz only)
Domain PREREQUISITES,
Solaris 9 OS Requirements for Domains
As a minimum, the Solaris 9 OS requires:
■ Solaris 9 4/04 OS release.
■ Patches:
■ 113027-03 patch
■ 112233-09 patch
Solaris 10 OS Requirements for Domains
As a minimum, the Solaris 10 OS requires:
■ Solaris 10 3/05 (minimum for UltraSPARC III+, UltraSPARC III++, and
UltraSPARC IV)
■ Solaris 10 3/05 HW1 (minimum for UltraSPARC IV+ 1500 MHz and 1800 MHz
only)
The Solaris 10 1/06 OS release is supported on the domains on all processors.
1. Gather the superuser passwords for both SCs.
2. Be sure you have platadmn privileges to both SCs.
3. On both SCs, determine the directory into which you will download the SMS
software from the web.
4. Ensure that the SC data is synchronized between the two SCs by typing this
command on the main SC.
root@tatatel12-25k-sc0-eri0 # /opt/SUNWsms/bin/setdatasync backup
5. Ensure that both SC clocks are phase-locked. You can do this by looking at the
most recent messages in the platform logs, which say whether the SC clocks are
locked or not.
Preparing for Installation
1) Login to the Main SC (SC0)
2) Disable failover,
root@tatatel12-25k-sc0-eri0:sms-svc:># /opt/SUNWSMS/bin/setfailover off
3) Backup the current configuration on SC0
1a. Stop SMS
root@tatatel12-25k-sc0-eri0:sms-svc:># /etc/init.d/sms stop
2b. run smsbackup
root@tatatel12-25k-sc0-eri0:sms-svc:># /opt/SUNWSMS/bin/smsbackup directory_name
3c. Start SMS.html
root@tatatel12-25k-sc0-eri0:sms-svc:># /etc/init.d/sms start
4) Login to the Spare SC (SC1) as root
5) Backup the current configuration on SC1
1a. Stop SMS
root@tatatel12-25k-sc1-eri0:sms-svc:># /etc/init.d/sms stop
2b. run smsbackup
root@tatatel12-25k-sc1-eri0:sms-svc:># /opt/SUNWSMS/bin/smsbackup directory_name
SMS Upgrade on Spare SC:
1. Login to spare SC (SC1)
2. ftp Solaris 9 OS–SMS-1_6-S9-sparc.zip to /sun on spare SC (SC1)
3. root@tatatel12-25k-sc1-eri0:# cd /sun
4. root@tatatel12-25k-sc1-eri0:# unzip SMS-1_6-S9-sparc.zip
5. root@tatatel12-25k-sc1-eri0:# cd /sun/sms_1_6_sparc/System_Management_Services_1.6/Tools
6. root@tatatel12-25k-sc1-eri0:# ./smsupgrade /sun/sms_1_6_sparc/System_Management_Services_1.6/Product
7. Install all the SMS 1.6 patches on the spare SC which are available on EIS CD.
Switch Control to the Spare SC
1. Log in to the main system controller (sc0) as superuser.
2. Stop SMS.
root@tatatel12-25k-sc0-eri0:sms-svc:># /etc/init.d/sms stop
3. Log in to the spare SC (sc1) and change to the OpenBoot PROM prompt.
root@tatatel12-25k-sc1eri0:># shutdown -y -g0 -i0
4. Reboot the spare SC.
root@tatatel12-25k-sc1eri0:># boot –rv
To Upgrade the Spare SC Flash PROMs
1. Log in to the spare SC as a user with platadmn privileges.
2. root@tatatel12-25k-sc1-eri0:sms-svc:># flashupdate -f /opt/SUNWSMS/firmware/SCOBPimg.di sc1/fp0
3. root@tatatel12-25k-sc1-eri0:sms-svc:># flashupdate -f /opt/SUNWSMS/firmware/oSSCPOST.di sc1/fp1
4. root@tatatel12-25k-sc1-eri0:sms-svc:># /etc/init.d/sms stop
5. root@tatatel12-25k-sc1-eri0:sms-svc:># shutdown -y -g0 -i0
6. root@tatatel12-25k-sc1-eri0:sms-svc:># boot –rv
SMS Upgrade on Main SC:
1. Login to main SC (SC0)
2. ftp Solaris 9 OS–SMS-1_6-S9-sparc.zip to /sun on spare SC (SC1)
3. root@tatatel12-25k-sc0-eri0:# cd /sun
4. root@tatatel12-25k-sc0-eri0:# unzip SMS-1_6-S9-sparc.zip
5. root@tatatel12-25k-sc0-eri0:# cd /sun/sms_1_6_sparc/System_Management_Services_1.6/Tools
6. root@tatatel12-25k-sc0-eri0:# ./smsupgrade /sun/sms_1_6_sparc/System_Management_Services_1.6/Product
7. Install all the SMS 1.6 patches on the spare SC which are available on EIS CD.
To Upgrade the Main SC Flash PROMs
1. Log in to the spare SC as a user with platadmn privileges.
2. root@tatatel12-25k-sc0-eri0:sms-svc:># flashupdate -f /opt/SUNWSMS/firmware/SCOBPimg.di sc0/fp0
3. root@tatatel12-25k-sc0-eri0:sms-svc:># flashupdate -f /opt/SUNWSMS/firmware/oSSCPOST.di sc0/fp1
To Switch Control Back to the Main SC
1. Log in to the spare SC (sc1) as superuser.
2. root@tatatel12-25k-sc1-eri0:sms-svc:># /etc/init.d/sms stop
3. Log in to the main SC (sc0) and change to the OpenBoot PROM prompt.
4. root@tatatel12-25k-sc0-eri0:># shutdown -y -g0 -i0
5. root@tatatel12-25k-sc0-eri0:># boot –rv
6. Use the /etc/init.d/sms script to restart SMS on the spare SC
root@tatatel12-25k-sc1-eri0:># /etc/init.d/sms start
7. Enable failover
root@tatatel12-25k-sc0-eri0:sms-svc># /opt/SUNWSMS/bin/setfailover on
8. Verify that failover is working.
root@tatatel12-25k-sc0-eri0:sms-svc># /opt/SUNWSMS/bin/showfailover -v
To Upgrade the System Board Flash PROMs
1. Log in to the main SC as a user with platadmn privileges.
2. Use flashupdate to upgrade the CPU flash PROMs in a domain.
root@tatatel12-25k-sc0-eri0:sms-svc># flashupdate -d
Thursday, June 17, 2010
Scsetup - Modifications
#------------------------------------------------------------
# delete a metaset...
# check status
metastat -p -s Kencu1-DG
# remove all soft partitions and concats...
metaclear -s Kencu1-DG d1
metaclear -s Kencu1-DG d2
metaclear -s Kencu1-DG d3
metaclear -s Kencu1-DG d0
# remove all disks from the metaset...
metaset -s Kencu1-DG -d /dev/did/rdsk/d5
# you might have to force it "-f"
metaset -s Kencu1-DG -d -f /dev/did/rdsk/d5
# remove all hosts from diskset
metaset -s Kencu1-DG -d -h test
metaset -s Kencu1-DG -d -h test1
metaset -s Kencu1-DG -d -f -h test2
# check that the diskset no longer exists
metaset
scstat -D
#------------------------------------------------------------
# create a new diskset
# create metaset and mediators
metaset -s new-cu1 -a -h PKEND022 PKEND023
metaset -s new-cu1 -a -m PKEND022 PKENd023
# add disk to the metaset
metaset -s new-cu1 -a /dev/did/rdsk/d5
# check status
metaset -s new-cu1
metadb -s new-cu1
medstat -s new-cu1
# create the first concat
metainit -s new-cu1 d0 1 1 /dev/did/rdsk/d5s0
# create soft partitions (If required In our case its not required )
metainit -s new-cu1 d1 -p d0 100m
metainit -s new-cu1 d2 -p d0 2g
metainit -s new-cu1 d3 -p d0 2g
metainit -s new-cu1 d4 -p d0 1.5g
#------------------------------------------------------------
# create resource group
scrgadm -a -g Kencu1-DG -h PKEND022,PKEND023 -y RG_description="KENCU1-DG"
# create StoragePlus resource
scrgadm -a -j kencu1-hasp-rs -t SUNW.HAStoragePlus -g Kencu1-DG \
-x FileSystemMountPoints=/s1/kencu2/oracle,/s1/kencu2/oradata01,/s1/kencu2/oradata02 \
-x AffinityOn=true
# create logical hostname resource
scrgadm -a -L -g Kencu1-DG -j kencu1-lh-rs -l PKENCU2
# Resource Group commands
# check status of all resource groups/resources
scstat -g
# shutdown a resource group
scswitch -F -g
# start a resource group
scswitch -Z -g
# failover a resource group to another node
scswitch -z -g -h
# restart a resource group
scswitch -R -g -h
# evacuate all resources and resource groups from a node
scswitch -S -h
#------------------------------------------------------------
# Resource commands
# disable a resource and its fault monitor
scswitch -n -j
# enable a resource and its fault monitor
scswitch -e -j
# clear the STOP_FAILED flag of a resource
scswitch -c -j -h -f STOP_FAILED
#------------------------------------------------------------
# Shutdown the ENTIRE cluster
scshutdown
#------------------------------------------------------------
# View properties of Resource Groups/Resources
# Use "-v" to increase verbosity
scrgadm -p -g
scrgadm -p -j
#------------------------------------------------------------
# Add in ipmp tests and maintenance procedures
#------------------------------------------------------------
# Add a new LUN on the fly...
# probe/display all fibre attached devices
cfgadm -al
# look for the WWN numbers given by storage
root@PKEND022 # cfgadm -al
Ap_Id Type Receptacle Occupant Condition
c0 scsi-bus connected configured unknown
c0::dsk/c0t0d0 CD-ROM connected configured unknown
c1 fc-private connected configured unknown
c1::210000008741a927 disk connected configured unknown
c1::2100000087968e21 disk connected configured unknown
c2 fc-fabric connected configured unknown
c2::50060e80042d0a24 disk connected configured unusable
c3 fc-fabric connected unconfigured unknown
c3::50060e80042d0a06 disk connected unconfigured unknown unknown <-- and this one
c4 fc-fabric connected configured unknown
c4::50060e80042d0a34 disk connected configured unusable
c5 fc-fabric connected unconfigured unknown
c5::50060e80042d0a16 disk connected unconfigured unknown
c7 scsi-bus connected unconfigured unknown
c8 scsi-bus connected unconfigured unknown
usb0/1 unknown empty unconfigured ok
usb0/2 unknown empty unconfigured ok
usb0/3 unknown empty unconfigured ok
usb0/4 unknown empty unconfigured ok
root@PKEND022 #
# if it's not already configured then configure the disk
cfgadm -c configure c3::50060e80042d0a06 c5::50060e80042d0a16
# this one shows lun hex id's too!!
cfgadm -al -o show_FCP_dev c3::50060e80042d0a06
# else confirm with...
# format
Searching for disks...done
c6t500060E80000000000009CBB00000484d0: configured with capacity of 96.28GB
c6t500060E80000000000009CBB00000492d0: configured with capacity of 96.28GB
AVAILABLE DISK SELECTIONS:
0. c1t0d0
/ssm@0,0/pci@18,600000/scsi@2/sd@0,0
1. c1t1d0
/ssm@0,0/pci@18,600000/scsi@2/sd@1,0
2. c6t500060E80000000000009CBB0000043Dd0
/scsi_vhci/ssd@g500060e80000000000009cbb0000043d
3. c6t500060E80000000000009CBB00000051d0
/scsi_vhci/ssd@g500060e80000000000009cbb00000051
4. c6t500060E80000000000009CBB00000141d0
/scsi_vhci/ssd@g500060e80000000000009cbb00000141
5. c6t500060E80000000000009CBB00000430d0
/scsi_vhci/ssd@g500060e80000000000009cbb00000430
6. c6t500060E80000000000009CBB00000442d0
/scsi_vhci/ssd@g500060e80000000000009cbb00000442
7. c6t500060E80000000000009CBB00000443d0
/scsi_vhci/ssd@g500060e80000000000009cbb00000443
8. c6t500060E80000000000009CBB00000444d0
/scsi_vhci/ssd@g500060e80000000000009cbb00000444
9. c6t500060E80000000000009CBB00000445d0
/scsi_vhci/ssd@g500060e80000000000009cbb00000445
10. c6t500060E80000000000009CBB00000480d0
/scsi_vhci/ssd@g500060e80000000000009cbb00000480
11. c6t500060E80000000000009CBB00000484d0
/scsi_vhci/ssd@g500060e80000000000009cbb00000484
12. c6t500060E80000000000009CBB00000492d0
/scsi_vhci/ssd@g500060e80000000000009cbb00000492
Specify disk (enter its number):
# the label of the disk should give you an indication of which one you are after (ie 128Gb = OPEN-9*14)
# then confirm your suspicions with...
# luxadm display /dev/rdsk/c6t500060E80000000000009CBB00000492d0s2
DEVICE PROPERTIES for disk: /dev/rdsk/c6t500060E80000000000009CBB00000492d0s2
Vendor: HITACHI
Product ID: OPEN-9*14 -SUN
Revision: 0117
Serial Num: 40123
Unformatted capacity: 103384.352 MBytes
Write Cache: Enabled
Read Cache: Enabled
Minimum prefetch: 0x0
Maximum prefetch: 0x0
Device Type: Disk device
Path(s):
/dev/rdsk/c6t500060E80000000000009CBB00000492d0s2
/devices/scsi_vhci/ssd@g500060e80000000000009cbb00000492:c,raw
Controller /devices/ssm@0,0/pci@18,600000/SUNW,qlc@1/fp@0,0
Device Address 500060e8029cbb08,c <-- check this line
Host controller port WWN 210000e08b0a3bfe
Class primary
State ONLINE
# the Device Address line above should correlate with the WWN and Hex address that
# the Storage boys supply you with.
# create new did instances for these devices on each node
# scdidadm -r
did instance 21 created.
did subpath mulloway:/dev/rdsk/c6t500060E80000000000009CBB00000484d0 created for instance 21.
did instance 22 created.
did subpath mulloway:/dev/rdsk/c6t500060E80000000000009CBB00000492d0 created for instance 22.
root@pkend023:/root
root@pkend022:/root
# scdidadm -r
did subpath /dev/rdsk/c6t500060E80000000000009CBB00000484d0s2 created for instance 21.
did subpath /dev/rdsk/c6t500060E80000000000009CBB00000492d0s2 created for instance 22.
root@pkend022:/root
root@pkend022:/root
# scdidadm -L|grep c6t500060E80000000000009CBB00000484d0
21 manta:/dev/rdsk/c6t500060E80000000000009CBB00000484d0 /dev/did/rdsk/d21
21 mulloway:/dev/rdsk/c6t500060E80000000000009CBB00000484d0 /dev/did/rdsk/d21
21 marlin:/dev/rdsk/c6t500060E80000000000009CBB00000484d0 /dev/did/rdsk/d21
root@pkend022:/root
#
# update the global devices namespace
scgdevs
# if it's a new LUN size you've been given, create a new label in /etc/format.dat.
# These 100Gb ones didn't seem to work using an explicit entry in /etc/format.dat.
# In this case, just add the type manually thru "format".
# Use the Hitachi disk spec manual for values.
# At any rate, label and partition the disks. Make slice 7 20Mb, and add the rest of the disk into
# slice 0.
#------------------------------------------------------------
# Remove a LUN on the fly...
# If you don't know the diskset/device(s) to remove, the storage
# boys will provide you with the WWN and LUN ID
# ie 50060e80042d0a06, LUN x'06'
# Issue a luxadm display using the WWN
# All luns on that port will be displayed.
# Look at the "Device Address" filed to find the right lun.
DEVICE PROPERTIES for disk: 50060e80042d0a06
Vendor: HITACHI
Product ID: OPEN-9 -SUN
Revision: 0119
Serial Num: 40123
Unformatted capacity: 7384.597 MBytes
Write Cache: Enabled
Read Cache: Enabled
Minimum prefetch: 0x0
Maximum prefetch: 0x0
Device Type: Disk device
Path(s):
/dev/rdsk/c6t500060E80000000000009CBB00000444d0s2
/devices/scsi_vhci/ssd@g500060e80000000000009cbb00000444:c,raw
Controller /devices/ssm@0,0/pci@18,600000/SUNW,qlc@1/fp@0,0
Device Address 500060e8029cbb08,6
Host controller port WWN 210000e08b0aa6fd
Class primary
State ONLINE
Controller /devices/ssm@0,0/pci@19,700000/SUNW,qlc@3/fp@0,0
Device Address 500060e8029cbb18,6
Host controller port WWN 210000e08b0e0622
Class primary
State ONLINE
DEVICE PROPERTIES for disk: 500060e8029cbb08
Vendor: HITACHI
Product ID: OPEN-9 -SUN
Revision: 0119
Serial Num: 40123
Unformatted capacity: 7384.597 MBytes
Write Cache: Enabled
Read Cache: Enabled
Minimum prefetch: 0x0
Maximum prefetch: 0x0
Device Type: Disk device
Path(s):
/dev/rdsk/c6t500060E80000000000009CBB00000445d0s2
/devices/scsi_vhci/ssd@g500060e80000000000009cbb00000445:c,raw
Controller /devices/ssm@0,0/pci@18,600000/SUNW,qlc@1/fp@0,0
Device Address 500060e8029cbb08,7
Host controller port WWN 210000e08b0aa6fd
Class primary
State ONLINE
Controller /devices/ssm@0,0/pci@19,700000/SUNW,qlc@3/fp@0,0
Device Address 500060e8029cbb18,7
Host controller port WWN 210000e08b0e0622
Class primary
State ONLINE
# Once you have this info, you have the disk device name like...
/dev/rdsk/c6t500060E80000000000009CBB00000444d0s2
# Use this to find the did device name
scdidadm -L |grep c6t500060E80000000000009CBB00000444d0
4 pkend022:/dev/rdsk/c6t500060E80000000000009CBB00000444d0 /dev/did/rdsk/d4
4 pkend023:/dev/rdsk/c6t500060E80000000000009CBB00000444d0 /dev/did/rdsk/d4
4 pkend024:/dev/rdsk/c6t500060E80000000000009CBB00000444d0 /dev/did/rdsk/d4
scdidadm -L |grep c6t500060E80000000000009CBB00000444d0
9 pkend022:/dev/rdsk/c6t500060E80000000000009CBB00000445d0 /dev/did/rdsk/d9
9 pkend023:/dev/rdsk/c6t500060E80000000000009CBB00000445d0 /dev/did/rdsk/d9
9 pkend024:/dev/rdsk/c6t500060E80000000000009CBB00000445d0 /dev/did/rdsk/d9
# So it's d4 and d9 I want to remove
# Check for their existence in metasets
metaset|grep d9
# If there's any output, you'd better take a close look at the whole output
# to find which metaset it belongs to.
# If it belongs to a metaset, remove all filesystems partitions etc.
# Finally delete the metaset.
# If you have HDS SCSI reserve errors when trying to deallocate the lun...
# Check for SCSI3 reserves using the undocumented utility /usr/cluster/lib/sc/reserve.
# Use either the did or the OS device file.
root@marlin:/usr/cluster/lib/sc
$ ./reserve -c inkeys -z /dev/did/rdsk/d9s2
Reservation keys(3):
0x3f8a0ed500000003
0x3f8a0ed500000001
0x3f8a0ed500000002
root@pkend022:/usr/cluster/lib/sc
$ scdidadm -L|grep d9
9 manta:/dev/rdsk/c6t500060E80000000000009CBB00000445d0 /dev/did/rdsk/d9
9 marlin:/dev/rdsk/c6t500060E80000000000009CBB00000445d0 /dev/did/rdsk/d9
9 mulloway:/dev/rdsk/c6t500060E80000000000009CBB00000445d0 /dev/did/rdsk/d9
root@pkend023:/usr/cluster/lib/sc
$ ./reserve -c inkeys -z /dev/rdsk/c6t500060E80000000000009CBB00000445d0s2
Reservation keys(3):
0x3f8a0ed500000003
0x3f8a0ed500000001
0x3f8a0ed500000002
root@marlin:/usr/cluster/lib/sc
$
root@marlin:/usr/cluster/lib/sc
$ ./reserve -c scrub -z /dev/rdsk/c6t500060E80000000000009CBB00000445d0s2
Reservation keys currently on disk:
0x3f8a0ed500000003
0x3f8a0ed500000001
0x3f8a0ed500000002
Attempting to remove all keys from the disk...
May 26 17:44:57 marlin last message repeated 1 time
May 26 17:46:44 marlin scsi: WARNING: /scsi_vhci/ssd@g500060e80000000000009cbb00000445 (ssd5):
May 26 17:46:44 marlin Error for Command: Error Level: Informational
Scrubbing complete, use 'reserve -c inkeys -z /dev/rdsk/c6t500060E80000000000009CBB00000445d0s2' to verify success
root@marlin:/usr/cluster/lib/sc
$ May 26 17:46:44 marlin scsi: Requested Block: 0 Error Block: 0
May 26 17:46:44 marlin scsi: Vendor: HITACHI Serial Number: 04009CBB0445
May 26 17:46:44 marlin scsi: Sense Key: Unit Attention
May 26 17:46:44 marlin scsi: ASC: 0x2a (), ASCQ: 0x4, FRU: 0x0
root@marlin:/usr/cluster/lib/sc
$ ./reserve -c inkeys -z /dev/rdsk/c6t500060E80000000000009CBB00000445d0s2
Reservation keys(0):
root@marlin:/usr/cluster/lib/sc
# run devfsadm to remove device files
devfsadm -C -c disk
# clean up the did devices
scdidadm -C
#------------------------------------------------------------
# create a new diskset
# create metaset and mediators
metaset -s ds04 -a -h manta mulloway marlin
metaset -s ds04 -a -m mulloway manta
# add disk to the metaset
metaset -s ds04 -a /dev/did/rdsk/d21 /dev/did/rdsk/d22
# check status
metaset -s ds04
metadb -s ds04
medstat -s ds04
# create the first concat
metainit -s ds04 d0 2 1 /dev/did/rdsk/d21s0 1 /dev/did/rdsk/d22s0
# create soft partitions
root@manta:init.d
# metainit -s ds04 d1 -p d0 10g
d1: Soft Partition is setup
root@manta:init.d
# metainit -s ds04 d2 -p d0 10g
d2: Soft Partition is setup
root@manta:init.d
#
# create default ufs filesystems
newfs /dev/md/ds04/rdsk/d1
newfs /dev/md/ds04/rdsk/d2
# check required filesystem settings using...
# mkfs -m /dev/md/ds04/rdsk/d1
mkfs -F ufs -o nsect=120,ntrack=56,bsize=8192,fragsize=1024,cgsize=16,free=1,rps=166,nbpi=8239,opt=t,apc=0,gap=0,nrpos=8,maxcontig=16 /dev/md/ds04/rdsk/d1 20971520
# create resource group
scrgadm -a -g super1 -h manta,mulloway,marlin -y RG_description="Summit Production"
# create StoragePlus resource
scrgadm -a -j super1-ds04 -t SUNW.HAStoragePlus -g super1 \
-x FileSystemMountPoints=/opt/smt,/opt/oraclest \
-x AffinityOn=true
# create logical hostname resource
scrgadm -a -L -g super1 -j super1-ip -l super1
# create the super1 apache application resource
scrgadm -a -j super1-apache -t EUM.super1 -g super1 -y Resource_dependencies=super1-ds04 -x Eum_admin_dir=/opt/smt/admin/bin
# delete a metaset...
# check status
metastat -p -s Kencu1-DG
# remove all soft partitions and concats...
metaclear -s Kencu1-DG d1
metaclear -s Kencu1-DG d2
metaclear -s Kencu1-DG d3
metaclear -s Kencu1-DG d0
# remove all disks from the metaset...
metaset -s Kencu1-DG -d /dev/did/rdsk/d5
# you might have to force it "-f"
metaset -s Kencu1-DG -d -f /dev/did/rdsk/d5
# remove all hosts from diskset
metaset -s Kencu1-DG -d -h test
metaset -s Kencu1-DG -d -h test1
metaset -s Kencu1-DG -d -f -h test2
# check that the diskset no longer exists
metaset
scstat -D
#------------------------------------------------------------
# create a new diskset
# create metaset and mediators
metaset -s new-cu1 -a -h PKEND022 PKEND023
metaset -s new-cu1 -a -m PKEND022 PKENd023
# add disk to the metaset
metaset -s new-cu1 -a /dev/did/rdsk/d5
# check status
metaset -s new-cu1
metadb -s new-cu1
medstat -s new-cu1
# create the first concat
metainit -s new-cu1 d0 1 1 /dev/did/rdsk/d5s0
# create soft partitions (If required In our case its not required )
metainit -s new-cu1 d1 -p d0 100m
metainit -s new-cu1 d2 -p d0 2g
metainit -s new-cu1 d3 -p d0 2g
metainit -s new-cu1 d4 -p d0 1.5g
#------------------------------------------------------------
# create resource group
scrgadm -a -g Kencu1-DG -h PKEND022,PKEND023 -y RG_description="KENCU1-DG"
# create StoragePlus resource
scrgadm -a -j kencu1-hasp-rs -t SUNW.HAStoragePlus -g Kencu1-DG \
-x FileSystemMountPoints=/s1/kencu2/oracle,/s1/kencu2/oradata01,/s1/kencu2/oradata02 \
-x AffinityOn=true
# create logical hostname resource
scrgadm -a -L -g Kencu1-DG -j kencu1-lh-rs -l PKENCU2
# Resource Group commands
# check status of all resource groups/resources
scstat -g
# shutdown a resource group
scswitch -F -g
# start a resource group
scswitch -Z -g
# failover a resource group to another node
scswitch -z -g -h
# restart a resource group
scswitch -R -g -h
# evacuate all resources and resource groups from a node
scswitch -S -h
#------------------------------------------------------------
# Resource commands
# disable a resource and its fault monitor
scswitch -n -j
# enable a resource and its fault monitor
scswitch -e -j
# clear the STOP_FAILED flag of a resource
scswitch -c -j -h -f STOP_FAILED
#------------------------------------------------------------
# Shutdown the ENTIRE cluster
scshutdown
#------------------------------------------------------------
# View properties of Resource Groups/Resources
# Use "-v" to increase verbosity
scrgadm -p -g
scrgadm -p -j
#------------------------------------------------------------
# Add in ipmp tests and maintenance procedures
#------------------------------------------------------------
# Add a new LUN on the fly...
# probe/display all fibre attached devices
cfgadm -al
# look for the WWN numbers given by storage
root@PKEND022 # cfgadm -al
Ap_Id Type Receptacle Occupant Condition
c0 scsi-bus connected configured unknown
c0::dsk/c0t0d0 CD-ROM connected configured unknown
c1 fc-private connected configured unknown
c1::210000008741a927 disk connected configured unknown
c1::2100000087968e21 disk connected configured unknown
c2 fc-fabric connected configured unknown
c2::50060e80042d0a24 disk connected configured unusable
c3 fc-fabric connected unconfigured unknown
c3::50060e80042d0a06 disk connected unconfigured unknown unknown <-- and this one
c4 fc-fabric connected configured unknown
c4::50060e80042d0a34 disk connected configured unusable
c5 fc-fabric connected unconfigured unknown
c5::50060e80042d0a16 disk connected unconfigured unknown
c7 scsi-bus connected unconfigured unknown
c8 scsi-bus connected unconfigured unknown
usb0/1 unknown empty unconfigured ok
usb0/2 unknown empty unconfigured ok
usb0/3 unknown empty unconfigured ok
usb0/4 unknown empty unconfigured ok
root@PKEND022 #
# if it's not already configured then configure the disk
cfgadm -c configure c3::50060e80042d0a06 c5::50060e80042d0a16
# this one shows lun hex id's too!!
cfgadm -al -o show_FCP_dev c3::50060e80042d0a06
# else confirm with...
# format
Searching for disks...done
c6t500060E80000000000009CBB00000484d0: configured with capacity of 96.28GB
c6t500060E80000000000009CBB00000492d0: configured with capacity of 96.28GB
AVAILABLE DISK SELECTIONS:
0. c1t0d0
/ssm@0,0/pci@18,600000/scsi@2/sd@0,0
1. c1t1d0
/ssm@0,0/pci@18,600000/scsi@2/sd@1,0
2. c6t500060E80000000000009CBB0000043Dd0
/scsi_vhci/ssd@g500060e80000000000009cbb0000043d
3. c6t500060E80000000000009CBB00000051d0
/scsi_vhci/ssd@g500060e80000000000009cbb00000051
4. c6t500060E80000000000009CBB00000141d0
/scsi_vhci/ssd@g500060e80000000000009cbb00000141
5. c6t500060E80000000000009CBB00000430d0
/scsi_vhci/ssd@g500060e80000000000009cbb00000430
6. c6t500060E80000000000009CBB00000442d0
/scsi_vhci/ssd@g500060e80000000000009cbb00000442
7. c6t500060E80000000000009CBB00000443d0
/scsi_vhci/ssd@g500060e80000000000009cbb00000443
8. c6t500060E80000000000009CBB00000444d0
/scsi_vhci/ssd@g500060e80000000000009cbb00000444
9. c6t500060E80000000000009CBB00000445d0
/scsi_vhci/ssd@g500060e80000000000009cbb00000445
10. c6t500060E80000000000009CBB00000480d0
/scsi_vhci/ssd@g500060e80000000000009cbb00000480
11. c6t500060E80000000000009CBB00000484d0
/scsi_vhci/ssd@g500060e80000000000009cbb00000484
12. c6t500060E80000000000009CBB00000492d0
/scsi_vhci/ssd@g500060e80000000000009cbb00000492
Specify disk (enter its number):
# the label of the disk should give you an indication of which one you are after (ie 128Gb = OPEN-9*14)
# then confirm your suspicions with...
# luxadm display /dev/rdsk/c6t500060E80000000000009CBB00000492d0s2
DEVICE PROPERTIES for disk: /dev/rdsk/c6t500060E80000000000009CBB00000492d0s2
Vendor: HITACHI
Product ID: OPEN-9*14 -SUN
Revision: 0117
Serial Num: 40123
Unformatted capacity: 103384.352 MBytes
Write Cache: Enabled
Read Cache: Enabled
Minimum prefetch: 0x0
Maximum prefetch: 0x0
Device Type: Disk device
Path(s):
/dev/rdsk/c6t500060E80000000000009CBB00000492d0s2
/devices/scsi_vhci/ssd@g500060e80000000000009cbb00000492:c,raw
Controller /devices/ssm@0,0/pci@18,600000/SUNW,qlc@1/fp@0,0
Device Address 500060e8029cbb08,c <-- check this line
Host controller port WWN 210000e08b0a3bfe
Class primary
State ONLINE
# the Device Address line above should correlate with the WWN and Hex address that
# the Storage boys supply you with.
# create new did instances for these devices on each node
# scdidadm -r
did instance 21 created.
did subpath mulloway:/dev/rdsk/c6t500060E80000000000009CBB00000484d0 created for instance 21.
did instance 22 created.
did subpath mulloway:/dev/rdsk/c6t500060E80000000000009CBB00000492d0 created for instance 22.
root@pkend023:/root
root@pkend022:/root
# scdidadm -r
did subpath /dev/rdsk/c6t500060E80000000000009CBB00000484d0s2 created for instance 21.
did subpath /dev/rdsk/c6t500060E80000000000009CBB00000492d0s2 created for instance 22.
root@pkend022:/root
root@pkend022:/root
# scdidadm -L|grep c6t500060E80000000000009CBB00000484d0
21 manta:/dev/rdsk/c6t500060E80000000000009CBB00000484d0 /dev/did/rdsk/d21
21 mulloway:/dev/rdsk/c6t500060E80000000000009CBB00000484d0 /dev/did/rdsk/d21
21 marlin:/dev/rdsk/c6t500060E80000000000009CBB00000484d0 /dev/did/rdsk/d21
root@pkend022:/root
#
# update the global devices namespace
scgdevs
# if it's a new LUN size you've been given, create a new label in /etc/format.dat.
# These 100Gb ones didn't seem to work using an explicit entry in /etc/format.dat.
# In this case, just add the type manually thru "format".
# Use the Hitachi disk spec manual for values.
# At any rate, label and partition the disks. Make slice 7 20Mb, and add the rest of the disk into
# slice 0.
#------------------------------------------------------------
# Remove a LUN on the fly...
# If you don't know the diskset/device(s) to remove, the storage
# boys will provide you with the WWN and LUN ID
# ie 50060e80042d0a06, LUN x'06'
# Issue a luxadm display using the WWN
# All luns on that port will be displayed.
# Look at the "Device Address" filed to find the right lun.
DEVICE PROPERTIES for disk: 50060e80042d0a06
Vendor: HITACHI
Product ID: OPEN-9 -SUN
Revision: 0119
Serial Num: 40123
Unformatted capacity: 7384.597 MBytes
Write Cache: Enabled
Read Cache: Enabled
Minimum prefetch: 0x0
Maximum prefetch: 0x0
Device Type: Disk device
Path(s):
/dev/rdsk/c6t500060E80000000000009CBB00000444d0s2
/devices/scsi_vhci/ssd@g500060e80000000000009cbb00000444:c,raw
Controller /devices/ssm@0,0/pci@18,600000/SUNW,qlc@1/fp@0,0
Device Address 500060e8029cbb08,6
Host controller port WWN 210000e08b0aa6fd
Class primary
State ONLINE
Controller /devices/ssm@0,0/pci@19,700000/SUNW,qlc@3/fp@0,0
Device Address 500060e8029cbb18,6
Host controller port WWN 210000e08b0e0622
Class primary
State ONLINE
DEVICE PROPERTIES for disk: 500060e8029cbb08
Vendor: HITACHI
Product ID: OPEN-9 -SUN
Revision: 0119
Serial Num: 40123
Unformatted capacity: 7384.597 MBytes
Write Cache: Enabled
Read Cache: Enabled
Minimum prefetch: 0x0
Maximum prefetch: 0x0
Device Type: Disk device
Path(s):
/dev/rdsk/c6t500060E80000000000009CBB00000445d0s2
/devices/scsi_vhci/ssd@g500060e80000000000009cbb00000445:c,raw
Controller /devices/ssm@0,0/pci@18,600000/SUNW,qlc@1/fp@0,0
Device Address 500060e8029cbb08,7
Host controller port WWN 210000e08b0aa6fd
Class primary
State ONLINE
Controller /devices/ssm@0,0/pci@19,700000/SUNW,qlc@3/fp@0,0
Device Address 500060e8029cbb18,7
Host controller port WWN 210000e08b0e0622
Class primary
State ONLINE
# Once you have this info, you have the disk device name like...
/dev/rdsk/c6t500060E80000000000009CBB00000444d0s2
# Use this to find the did device name
scdidadm -L |grep c6t500060E80000000000009CBB00000444d0
4 pkend022:/dev/rdsk/c6t500060E80000000000009CBB00000444d0 /dev/did/rdsk/d4
4 pkend023:/dev/rdsk/c6t500060E80000000000009CBB00000444d0 /dev/did/rdsk/d4
4 pkend024:/dev/rdsk/c6t500060E80000000000009CBB00000444d0 /dev/did/rdsk/d4
scdidadm -L |grep c6t500060E80000000000009CBB00000444d0
9 pkend022:/dev/rdsk/c6t500060E80000000000009CBB00000445d0 /dev/did/rdsk/d9
9 pkend023:/dev/rdsk/c6t500060E80000000000009CBB00000445d0 /dev/did/rdsk/d9
9 pkend024:/dev/rdsk/c6t500060E80000000000009CBB00000445d0 /dev/did/rdsk/d9
# So it's d4 and d9 I want to remove
# Check for their existence in metasets
metaset|grep d9
# If there's any output, you'd better take a close look at the whole output
# to find which metaset it belongs to.
# If it belongs to a metaset, remove all filesystems partitions etc.
# Finally delete the metaset.
# If you have HDS SCSI reserve errors when trying to deallocate the lun...
# Check for SCSI3 reserves using the undocumented utility /usr/cluster/lib/sc/reserve.
# Use either the did or the OS device file.
root@marlin:/usr/cluster/lib/sc
$ ./reserve -c inkeys -z /dev/did/rdsk/d9s2
Reservation keys(3):
0x3f8a0ed500000003
0x3f8a0ed500000001
0x3f8a0ed500000002
root@pkend022:/usr/cluster/lib/sc
$ scdidadm -L|grep d9
9 manta:/dev/rdsk/c6t500060E80000000000009CBB00000445d0 /dev/did/rdsk/d9
9 marlin:/dev/rdsk/c6t500060E80000000000009CBB00000445d0 /dev/did/rdsk/d9
9 mulloway:/dev/rdsk/c6t500060E80000000000009CBB00000445d0 /dev/did/rdsk/d9
root@pkend023:/usr/cluster/lib/sc
$ ./reserve -c inkeys -z /dev/rdsk/c6t500060E80000000000009CBB00000445d0s2
Reservation keys(3):
0x3f8a0ed500000003
0x3f8a0ed500000001
0x3f8a0ed500000002
root@marlin:/usr/cluster/lib/sc
$
root@marlin:/usr/cluster/lib/sc
$ ./reserve -c scrub -z /dev/rdsk/c6t500060E80000000000009CBB00000445d0s2
Reservation keys currently on disk:
0x3f8a0ed500000003
0x3f8a0ed500000001
0x3f8a0ed500000002
Attempting to remove all keys from the disk...
May 26 17:44:57 marlin last message repeated 1 time
May 26 17:46:44 marlin scsi: WARNING: /scsi_vhci/ssd@g500060e80000000000009cbb00000445 (ssd5):
May 26 17:46:44 marlin Error for Command: Error Level: Informational
Scrubbing complete, use 'reserve -c inkeys -z /dev/rdsk/c6t500060E80000000000009CBB00000445d0s2' to verify success
root@marlin:/usr/cluster/lib/sc
$ May 26 17:46:44 marlin scsi: Requested Block: 0 Error Block: 0
May 26 17:46:44 marlin scsi: Vendor: HITACHI Serial Number: 04009CBB0445
May 26 17:46:44 marlin scsi: Sense Key: Unit Attention
May 26 17:46:44 marlin scsi: ASC: 0x2a (), ASCQ: 0x4, FRU: 0x0
root@marlin:/usr/cluster/lib/sc
$ ./reserve -c inkeys -z /dev/rdsk/c6t500060E80000000000009CBB00000445d0s2
Reservation keys(0):
root@marlin:/usr/cluster/lib/sc
# run devfsadm to remove device files
devfsadm -C -c disk
# clean up the did devices
scdidadm -C
#------------------------------------------------------------
# create a new diskset
# create metaset and mediators
metaset -s ds04 -a -h manta mulloway marlin
metaset -s ds04 -a -m mulloway manta
# add disk to the metaset
metaset -s ds04 -a /dev/did/rdsk/d21 /dev/did/rdsk/d22
# check status
metaset -s ds04
metadb -s ds04
medstat -s ds04
# create the first concat
metainit -s ds04 d0 2 1 /dev/did/rdsk/d21s0 1 /dev/did/rdsk/d22s0
# create soft partitions
root@manta:init.d
# metainit -s ds04 d1 -p d0 10g
d1: Soft Partition is setup
root@manta:init.d
# metainit -s ds04 d2 -p d0 10g
d2: Soft Partition is setup
root@manta:init.d
#
# create default ufs filesystems
newfs /dev/md/ds04/rdsk/d1
newfs /dev/md/ds04/rdsk/d2
# check required filesystem settings using...
# mkfs -m /dev/md/ds04/rdsk/d1
mkfs -F ufs -o nsect=120,ntrack=56,bsize=8192,fragsize=1024,cgsize=16,free=1,rps=166,nbpi=8239,opt=t,apc=0,gap=0,nrpos=8,maxcontig=16 /dev/md/ds04/rdsk/d1 20971520
# create resource group
scrgadm -a -g super1 -h manta,mulloway,marlin -y RG_description="Summit Production"
# create StoragePlus resource
scrgadm -a -j super1-ds04 -t SUNW.HAStoragePlus -g super1 \
-x FileSystemMountPoints=/opt/smt,/opt/oraclest \
-x AffinityOn=true
# create logical hostname resource
scrgadm -a -L -g super1 -j super1-ip -l super1
# create the super1 apache application resource
scrgadm -a -j super1-apache -t EUM.super1 -g super1 -y Resource_dependencies=super1-ds04 -x Eum_admin_dir=/opt/smt/admin/bin
Friday, June 11, 2010
Mounting an ISO image in Solaris
The other day I needed to mount an ISO image in Solaris. It turns out there is a simple way to do that using lofiadm command.
Using lofiadm you can link a file with a block device that can be mounted later as a regular filesystem. First the file has to be linked to a block device:
bash-3.00# lofiadm -a /path/to/my/ISO /dev/lofi/1
Now the block device can be mounted as usual:
bash-3.00# mount -F hsfs /dev/lofi/1 /mnt
When done, simply unmount the block device and remove the file-device association using lofiadm
bash-3.00# lofiadm -d /dev/lofi/1
More info can be found in lofiadm man page.
Using lofiadm you can link a file with a block device that can be mounted later as a regular filesystem. First the file has to be linked to a block device:
bash-3.00# lofiadm -a /path/to/my/ISO /dev/lofi/1
Now the block device can be mounted as usual:
bash-3.00# mount -F hsfs /dev/lofi/1 /mnt
When done, simply unmount the block device and remove the file-device association using lofiadm
bash-3.00# lofiadm -d /dev/lofi/1
More info can be found in lofiadm man page.
Quick and dirty SVM cheatsheet
This list focuses mostly on mirror operations. I use Solaris Volume Manager quite a bit when mirroring internal drives. There are tons of additional features and commands, if you use SVM for things other than mirroring. In that case you might want to look at check out Solaris Volume Manager Administration Guide.
Create database replicas:
metadb -f -a -c [number_of_replicas] [device]
metadb -f -a -c 3 c0t0d0s7
Delete all database replicas from device:
metadb -d [device]
metadb -d c0t0d0s7
Display status of database replicas:
metadb -i
metadb -i
Display metadevice status:
metastat
metastat
Create simple concat/stripe metadevice:
metainit -f [concat_metadevice] 1 1 [device]
metainit -f d21 1 1 c0t0d0s1
Create a mirror with one submirror:
metainit [mirror_metadevice] -m [submirror_metadevice]
metainit d20 -m d21
Attach a submirror to one sided mirror:
metattach [mirror_metadevice] [submirror_metadevice]
metattach d20 d22
Detach a submirror from a mirror:
metadetach [mirror_metadevice] [submirror_metadevice]
metadetach d20 d22
Clear a metadevice:
metaclear [metadevice]
metaclear d22
Offline a submirror:
metaoffline [mirror_metadevice] [submirror_metadevice]
metaoffline d20 d22
Online a submirror:
metaonline [mirror_metadevice] [submirror_metadevice]
metaonline d20 d22
Enable a failed component:
metareplace -e [metadevice] [device]
metareplace -e d21 c0t0d0s1
Rename a metadevice:
metarename [old_metadevice] [new_metadevice]
metarename d20 d30
Switch metadevice names:
metarename [metadevice_1] [metadevice_2]
metarename -x d20 d30
Configure system for root metadevice:
metaroot [metadevice]
metaroot d10
Create database replicas:
metadb -f -a -c [number_of_replicas] [device]
metadb -f -a -c 3 c0t0d0s7
Delete all database replicas from device:
metadb -d [device]
metadb -d c0t0d0s7
Display status of database replicas:
metadb -i
metadb -i
Display metadevice status:
metastat
metastat
Create simple concat/stripe metadevice:
metainit -f [concat_metadevice] 1 1 [device]
metainit -f d21 1 1 c0t0d0s1
Create a mirror with one submirror:
metainit [mirror_metadevice] -m [submirror_metadevice]
metainit d20 -m d21
Attach a submirror to one sided mirror:
metattach [mirror_metadevice] [submirror_metadevice]
metattach d20 d22
Detach a submirror from a mirror:
metadetach [mirror_metadevice] [submirror_metadevice]
metadetach d20 d22
Clear a metadevice:
metaclear [metadevice]
metaclear d22
Offline a submirror:
metaoffline [mirror_metadevice] [submirror_metadevice]
metaoffline d20 d22
Online a submirror:
metaonline [mirror_metadevice] [submirror_metadevice]
metaonline d20 d22
Enable a failed component:
metareplace -e [metadevice] [device]
metareplace -e d21 c0t0d0s1
Rename a metadevice:
metarename [old_metadevice] [new_metadevice]
metarename d20 d30
Switch metadevice names:
metarename [metadevice_1] [metadevice_2]
metarename -x d20 d30
Configure system for root metadevice:
metaroot [metadevice]
metaroot d10
Useful Sun Cluster commands
Some useful Sun Cluster commands
Shut down a resource group:
scswitch -F -g [RESOURCE_GROUP_NAME]
Bring up a resource group:
scswitch -Z -g [RESOURCE_GROUP_NAME]
Move failover resource group to node_name:
scswitch -z -g [RESOURCE_GROUP_NAME] -h [NODE_NAME]
Restart resource group:
scswitch -R -h [NODE_NAME] -g [RESOURCE_GROUP_NAME]
Evacuate all resources from node_name:
scswitch -S -h [NODE_NAME]
Disable resource:
scswitch -n -j [RESOURCE]
Enable resource:
scswitch -e -j [RESOURCE]
Clear STOP_FAILED on resource:
scswitch -c -j [RESOURCE] -h [NODE_NAME] -f STOP_FAILED
Disable resource’s fault monitor:
scswitch -n -M -j [RESOURCE]
Enable resource’s fault monitor:
scswitch -e -M -j [RESOURCE]
Enable resource’s fault monitor:
scswitch -e -M -j [RESOURCE]
Lists currently configured DID’s:
scdidadm -L
Put a new device under cluster control:
scgdevs
Displays status of the cluster, resources, resource groups, etc.:
scstat
Display useful setup info about cluster nodes, cluster transport, disksets, etc.:
scconf -p -v
Shut down a resource group:
scswitch -F -g [RESOURCE_GROUP_NAME]
Bring up a resource group:
scswitch -Z -g [RESOURCE_GROUP_NAME]
Move failover resource group to node_name:
scswitch -z -g [RESOURCE_GROUP_NAME] -h [NODE_NAME]
Restart resource group:
scswitch -R -h [NODE_NAME] -g [RESOURCE_GROUP_NAME]
Evacuate all resources from node_name:
scswitch -S -h [NODE_NAME]
Disable resource:
scswitch -n -j [RESOURCE]
Enable resource:
scswitch -e -j [RESOURCE]
Clear STOP_FAILED on resource:
scswitch -c -j [RESOURCE] -h [NODE_NAME] -f STOP_FAILED
Disable resource’s fault monitor:
scswitch -n -M -j [RESOURCE]
Enable resource’s fault monitor:
scswitch -e -M -j [RESOURCE]
Enable resource’s fault monitor:
scswitch -e -M -j [RESOURCE]
Lists currently configured DID’s:
scdidadm -L
Put a new device under cluster control:
scgdevs
Displays status of the cluster, resources, resource groups, etc.:
scstat
Display useful setup info about cluster nodes, cluster transport, disksets, etc.:
scconf -p -v
Growing Solaris Volume Manager mirrors online
You have a SVM mirror and you need to grow it. Online. All steps are straight forward except for the last one. So, in a nutshell:
1.Grow sizes of both submirror LUN’s on SAN
2.Detach the first submirror metadevice
3.Clear the submirror metadevice
4.Make the system see new LUN size
5.Recreate the submirror metadevice, attach it to the mirror and let it sync
6.Repeat the above for the second submirror
7.Not so obvious step
8.Grow the filesystem
You have a mirror metadevice d200 consisting of submirrors d201 and d202. Both submirrors are sliced in such way that all space is allocated in slice 0. At this point both LUN’s should be already grown, but you need to make the system to see the size increase. Here is what d201, one of the submirrors of d200, should look like:
bash-3.00# metastat d201
d201: Submirror of d200
State: Okay
Size: 2064384 blocks (1008 MB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
/dev/dsk/c5t511714A510011C370000C00003410000d0s0 0 No Okay Yes
First, detach d201 from its parent and clear it:
bash-3.00# metadetach d200 d201
bash-3.00# metaclear d201
The next step is likely to alter the LUN slice configuration, so make sure you have it noted.
You need to run format command to get the system to see the grown LUN. So, run format then select type and then select Autoconfigure. At this point format should be reporting the new LUN size. Make sure your slice configuration is correct and that all the new space is tacked onto the end of correct slice. In this case, it’s slice 0.
Now recreate d201, attach it to d200 and let the mirror sync. When checking status of d200 note that overall mirror size is still the same but the size of d201 increased:
bash-3.00# metastat d200
d200: Mirror
Submirror 0: d202
State: Okay
Submirror 1: d201
State: Okay
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 2064384 blocks (1008 MB)
d202: Submirror of d200
State: Okay
Size: 2064384 blocks (1008 MB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
/dev/dsk/c5t511714A510011C370000C00003460000d0s0 0 No Okay Yes
d201: Submirror of d200
State: Okay
Size: 4128768 blocks (2.0 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
/dev/dsk/c5t511714A510011C370000C00003410000d0s0 0 No Okay Yes
Now repeat the same process with d202. After you are done you will have both submirrors bigger, but the size of overall mirror will still be the same:
bash-3.00# metastat d200
d200: Mirror
Submirror 0: d201
State: Okay
Submirror 1: d202
State: Okay
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 2064384 blocks (1008 MB)
d201: Submirror of d200
State: Okay
Size: 4128768 blocks (2.0 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
/dev/dsk/c5t511714A510011C370000C00003410000d0s0 0 No Okay Yes
d202: Submirror of d200
State: Okay
Size: 4161536 blocks (2.0 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
/dev/dsk/c5t511714A510011C370000C00003460000d0s0 0 No Okay Yes
Now comes the “Not so obvious” step. In order to increase overall mirror size you need to run metattach on the mirror itself:
bash-3.00# metattach d200
bash-3.00# metastat d200
d200: Mirror
Submirror 0: d201
State: Okay
Submirror 1: d202
State: Okay
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 4128768 blocks (2.0 GB)
This will grow the size of the entire mirror to the size of submirror devices. Now you can grow the filesystem on it without unmounting it. Keep in mind that for the duration of growfs command there will be no writes to the volume.
bash-3.00# growfs -M /export/home/users /dev/md/rdsk/d200
/dev/md/rdsk/d200: 4128768 sectors in 126 cylinders of 128 tracks, 256 sectors
2016.0MB in 63 cyl groups (2 c/g, 32.00MB/g, 15040 i/g)
super-block backups (for fsck -F ufs -o b=#) at:
32, 65824, 131616, 197408, 263200, 328992, 394784, 460576, 526368, 592160,
3487008, 3552800, 3618592, 3684384, 3750176, 3815968, 3881760, 3947552,
4013344, 4079136
That’s all there is to it. The “Not so obvious” step is mentioned in the metattach man page. But unless you do this stuff every day, this very useful piece of information can be easily forgotten.
1.Grow sizes of both submirror LUN’s on SAN
2.Detach the first submirror metadevice
3.Clear the submirror metadevice
4.Make the system see new LUN size
5.Recreate the submirror metadevice, attach it to the mirror and let it sync
6.Repeat the above for the second submirror
7.Not so obvious step
8.Grow the filesystem
You have a mirror metadevice d200 consisting of submirrors d201 and d202. Both submirrors are sliced in such way that all space is allocated in slice 0. At this point both LUN’s should be already grown, but you need to make the system to see the size increase. Here is what d201, one of the submirrors of d200, should look like:
bash-3.00# metastat d201
d201: Submirror of d200
State: Okay
Size: 2064384 blocks (1008 MB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
/dev/dsk/c5t511714A510011C370000C00003410000d0s0 0 No Okay Yes
First, detach d201 from its parent and clear it:
bash-3.00# metadetach d200 d201
bash-3.00# metaclear d201
The next step is likely to alter the LUN slice configuration, so make sure you have it noted.
You need to run format command to get the system to see the grown LUN. So, run format then select type and then select Autoconfigure. At this point format should be reporting the new LUN size. Make sure your slice configuration is correct and that all the new space is tacked onto the end of correct slice. In this case, it’s slice 0.
Now recreate d201, attach it to d200 and let the mirror sync. When checking status of d200 note that overall mirror size is still the same but the size of d201 increased:
bash-3.00# metastat d200
d200: Mirror
Submirror 0: d202
State: Okay
Submirror 1: d201
State: Okay
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 2064384 blocks (1008 MB)
d202: Submirror of d200
State: Okay
Size: 2064384 blocks (1008 MB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
/dev/dsk/c5t511714A510011C370000C00003460000d0s0 0 No Okay Yes
d201: Submirror of d200
State: Okay
Size: 4128768 blocks (2.0 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
/dev/dsk/c5t511714A510011C370000C00003410000d0s0 0 No Okay Yes
Now repeat the same process with d202. After you are done you will have both submirrors bigger, but the size of overall mirror will still be the same:
bash-3.00# metastat d200
d200: Mirror
Submirror 0: d201
State: Okay
Submirror 1: d202
State: Okay
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 2064384 blocks (1008 MB)
d201: Submirror of d200
State: Okay
Size: 4128768 blocks (2.0 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
/dev/dsk/c5t511714A510011C370000C00003410000d0s0 0 No Okay Yes
d202: Submirror of d200
State: Okay
Size: 4161536 blocks (2.0 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
/dev/dsk/c5t511714A510011C370000C00003460000d0s0 0 No Okay Yes
Now comes the “Not so obvious” step. In order to increase overall mirror size you need to run metattach on the mirror itself:
bash-3.00# metattach d200
bash-3.00# metastat d200
d200: Mirror
Submirror 0: d201
State: Okay
Submirror 1: d202
State: Okay
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 4128768 blocks (2.0 GB)
This will grow the size of the entire mirror to the size of submirror devices. Now you can grow the filesystem on it without unmounting it. Keep in mind that for the duration of growfs command there will be no writes to the volume.
bash-3.00# growfs -M /export/home/users /dev/md/rdsk/d200
/dev/md/rdsk/d200: 4128768 sectors in 126 cylinders of 128 tracks, 256 sectors
2016.0MB in 63 cyl groups (2 c/g, 32.00MB/g, 15040 i/g)
super-block backups (for fsck -F ufs -o b=#) at:
32, 65824, 131616, 197408, 263200, 328992, 394784, 460576, 526368, 592160,
3487008, 3552800, 3618592, 3684384, 3750176, 3815968, 3881760, 3947552,
4013344, 4079136
That’s all there is to it. The “Not so obvious” step is mentioned in the metattach man page. But unless you do this stuff every day, this very useful piece of information can be easily forgotten.
Subscribe to:
Posts (Atom)