Google Search

Custom Search

Wednesday, January 26, 2011

Solaris LDOMS

Steps to verify LDOM services are started
Below are two basic services needs to be running for the ldom to start/work properly.
1. svc:/ldoms/ldmd:default
2. svc:/ldoms/vntsd:default

From the above services we can see the ldmd is ldom service and vntsd is the virtual console service.
The state of the service should be online, rather than disabled /maintenance.If the State of the service is disabled we can try restarting it using /usr/sbin/svcadm (If the state of the service is in maintenance check the

/var/svc/log/ldoms-ldmd:default.log for the ldm logs
/var/svc/log/ldoms-vntsd:default.log for the vntsd logs

Also check in the /var/adm/messages file.Use the ps command to check the process list whether the ldmd  and vntsd  is there in the process list or not

The below is the default configuration of the ldmd service and the vntsd service

svc:> select ldoms/ldmd
svc:/ldoms/ldmd> listprop
ldmd                                    application
ldmd/debug                              integer  0
ldmd/hops                               integer  0
ldmd/nocfg                              boolean  false
ldmd/xmpp                               boolean  false
fmd_config                              application
fmd_config/fmd_to_ldmd_init_timeout     integer  20
fmd_config/fmd_to_ldmd_running_timeout  integer  10
filesystem                              dependency
filesystem/entities                     fmri     svc:/system/filesystem/local
filesystem/grouping                     astring  require_all
filesystem/restart_on                   astring  none
filesystem/type                         astring  service
general                                 framework
general/action_authorization            astring  solaris.smf.manage.ldoms
general/entity_stability                astring  Unstable
general/single_instance                 boolean  true
start                                   method
start/exec                              astring  /opt/SUNWldm/bin/ldmd_start
start/timeout_seconds                   count    120
start/type                              astring  method
stop                                    method
stop/exec                               astring  :kill
stop/timeout_seconds                    count    60
stop/type                               astring  method
tm_common_name                          template
tm_common_name/C                        ustring  "Logical Domain Manager"

svc:> select ldoms/vntsd
svc:/ldoms/vntsd> listprop
vntsd                     application
vntsd/listen_addr         astring  localhost
vntsd/timeout_minutes     integer  0
vntsd/vcc_device          astring  virtual-console-concentrator@0
network                   dependency
network/entities          fmri     svc:/milestone/network
network/grouping          astring  optional_all
network/restart_on        astring  error
network/type              astring  service
syslog                    dependency
syslog/entities           fmri     svc:/system/system-log
syslog/grouping           astring  optional_all
syslog/restart_on         astring  none
syslog/type               astring  service
general                   framework
general/entity_stability  astring  Unstable
start                     method
start/exec                astring  /lib/svc/method/svc-vntsd
start/timeout_seco
Procedure to ensure appropriate LDOM resources are assigned to domains:
Review VCPU/Memory/MAU assigned to a domain
A summary of resources allocated to all domains on the system can be displayed with the ldm(1M) list subcommand.

For example:
# /opt/SUNWldm/bin/ldm list
NAME             STATE    FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
primary          active   -n-cv   SP      8     4G        17%  32d 2h 3m
guestdom1        active   -n---   5001    4     4G       0.5%  38d 13h 44m
guestdom2        active   -n---   5002    4     4G       1.0%  38d 13h 45m
guestdom3        active   -n---   5003    4     4G       0.7%  38d 13h 45m
guestdom4        active   -n---   5004    4     4G       1.2%  38d 13h 46m
guestdom5        active   -n---   5005    4     4G       0.8%  38d 13h 48m
guestdom6        active   -n---   5006    4     4G       0.7%  38d 21h 58m


For an extensive review of the resources allocated to a specific domain, please use the ldm(1M) list-domain sub command. Take note of the VCPU, MEMORY and MAU section for the individual resources assigned to the domain.

For example:
# /opt/SUNWldm/bin/ldm list-domain -l guestdom1
NAME             STATE    FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
guestdom1        active   -n---   5001    4     4G       0.4%  38d 13h 51m
SOFTSTATE
Solaris running
VCPU
VID    PID    UTIL STRAND
0      4      100%   100%
1      5      1.1%   100%
2      6      0.9%   100%
3      7      0.0%   100%
MAU
ID     CPUSET
0      (8, 9, 10, 11, 12, 13, 14, 15)
MEMORY
RA               PA               SIZE
0x8000000        0x108000000      2G
0x408000000      0x688000000      2G


Setting of VCPU/Memory/MAU resources for a domain
Allocation of VCPU
To set the number of VCPUs for a given domain, use the ldm(1M) set-vcpu subcommand:
# /opt/SUNWldm/bin/ldm set-vcpu  ldom

To add additional VCPUs to a given domain, use the ldm(1M) add-vcpu subcommand:
# /opt/SUNWldm/bin/ldm add-vcpu  ldom

There are limited number of VCPUs on a given machine. There are times that one may want to reduce the number of VCPUs from one domain, and then assign them to another domain.

To remove a number of VCPU assigned to a domain, used the ldm(1M) remove-vcpu subcommand.
# /opt/SUNWldm/bin/ldm remove-vcpu  ldom


Allocation of Memory
To set the amount of memory for a given domain, use the ldm(1M) set-memory sub command:
# /opt/SUNWldm/bin/ldm set-memory  ldom

To increase additional memory to a given domain, use the ldm(1M) add-memory subcommand:
# /opt/SUNWldm/bin/ldm add-memory  ldom

There are limited amount of memory on a given machine. There are times that one may want to reduce the amount of memory from one domain, and then assign them to another domain.

To remove certain amount of memory assigned to a domain, used the ldm(1M) remove-memory sub command.
# /opt/SUNWldm/bin/ldm remove-memory  ldom


Allocation of MAU
MAUs can only be assigned to specific domain only if at least of the VCPU from the owning processor core is assigned to the domain. To set the number of MAUs for a given domain, use the ldm(1M) set-mau subcommand:
# /opt/SUNWldm/bin/ldm set-mau  ldom


To add additional MAUs to a given domain, use the ldm(1M) add-mau subcommand:
# /opt/SUNWldm/bin/ldm add-mau  ldom


There are limited number of MAUs on a given machine. There are times that one may want to reduce the number of MAUs from one domain, and then assign them to another domain.

To remove a number of MAU assigned to a domain, used the ldm(1M) remove-mau subcommand.
# /opt/SUNWldm/bin/ldm remove-mau  ldom

Saving the new configuration

The newly configured resource, though may have been assigned to the target domain dynamically, the configurations need to be saved to the system controller (SC) for it to persist across reboot of the primary domain.

In order to store the configuration, use the ldm(1M) add-config subcommand:
# /opt/SUNWldm/bin/ldm add-config config_name


For example, saving the current configuraton as cfg090221: # /opt/SUNWldm/bin/ldm add-config cfg090221

On LDoms 1.0.3 or earlier, issuing the ldm(1M) list-config subcommand will show that configuration named cfg090221 will be the next active configuration(Though the changes may indeed be already active) when the primary domain reboots:

# /opt/SUNWldm/bin/ldm list-config
factory-default
initconfig
cfg090113 [current]
cfg090221 [next]

On LDoms 1.1, issuing the ldm(1M) list-config subcommand will show that configuration named cfg090221 is shown as [current], which provides a better reflection that the changes are already in active and also will be the next active configuration when the primary domain reboots:
# /opt/SUNWldm/bin/ldm list-config
factory-default
initconfig
cfg090113
cfg090221 [current]

If after the above steps fail to make the necessary resource allocation, please contact Sun Support with the following information:
- The latest explorer output using the latest version of explorer script.
- The if there is any specific error, please provide session log illustrating the error concerned.
Replacing motherboard or service processor in LDOM environment
  • Backup of the ldom environment before hardware replacement.
  • Identify the ldom names using the ldm list-domain command
Example:
# ldm list-domain
NAME             STATE    FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
primary          active   -n-cv   SP      8     4G       0.1%  1d 4h 52m
guest1           active -----           8     4G            
·         Capture the LDOM environment
·         Log in as an authorized user or superuser
·         Use each of the following methods to ensure the required information is captured
·         Run the command ldm ls -e -p
·         Run the command ldm list-constraints -x ldom > ldom.xml for each of the ldom names identified earlier
·         If this is not available then the system configuration can be obtained from the file /var/opt/SUNWldm/ldom-db.xml
*See the Appendix to this document for examples of the outputs from these commands. These examples are used in this procedure.
  • Shutdown all the domains and power off the system
  • Replace the motherboard or SC following the procedures shown in the appropriate service guide. See http://docs.sun.com
  • Boot the operating system.
The system will display an errors similar to the following:
Oct 14 12:15:18 svc.startd[7]: svc:/network/physical:default: Method "/lib/svc/method/net-physical" failed with exit status 96.
Oct 14 12:15:18 svc.startd[7]: network/physical:default misconfigured: transitioned to maintenance (see 'svcs -xv' for details)
Oct 14 12:15:34 svc.startd[7]: svc:/ldoms/vntsd:default: Method "/lib/svc/method/svc-vntsd" failed with exit status 1.
Oct 14 12:15:34 svc.startd[7]: svc:/ldoms/vntsd:default: Method "/lib/svc/method/svc-vntsd" failed with exit status 1.
Oct 14 12:15:35 svc.startd[7]: svc:/ldoms/vntsd:default: Method "/lib/svc/method/svc-vntsd" failed with exit status 1.
Oct 14 12:15:35 svc.startd[7]: ldoms/vntsd:default failed: transitioned to maintenance (see 'svcs -xv' for details)
It will still be possible to login to the system, however there will be no network access.
  • Login to the system as root
  • Move the virtual network interface from /etc/hostname.vsw0 to the physical network interface.
    In this example the physical interface is e1000g0
          # mv /etc/hostname.vsw0 /etc/hostname.e1000g0
  • Configure the physical network interface
# ifconfig e1000g0 plumb
# ifconfig e1000g0 ip_of_e1000g0 netmask netmask_of_e1000g0 broadcast + up
  • Reboot the system and then login again.
In some instances the previous svc-vntsd failure messages may still be displayed. However, it will still be possible to login.
Manually re-create the Primary Domain.
The following examples of the commands use the parameters from the example files in the Appendix.
Substitute the true values from the saved files.
Recreate the Virtual Disk Server (vds).
primary
-
-
-
primary-vds0
# ldm add-vds primary-vds0 primary
Re-create the Virtual Console Concentrator service (vcc)
primary-vcc0
5000
5100

# ldm add-vcc port-range=5000-5100 primary-vcc0 primary
Re-create the Virtual Switch service (vsw)
primary-vsw0
0x00144ffb6ff9
e1000g0

# ldm add-vsw net-dev=e1000g0 primary-vsw0 primary

Re-create the cryptographic resource 1
# ldm set-mau 1 primary
Re-create the Virtual CPUs 8
# ldm set-vcpu 8 primary
Re-create the memory 4294967296
# ldm set-memory 4G primary
Save the configuration to the service controller
# ldm add-config initial
List the saved configs
# ldm list-config
factory-default [current]
initial [next]
Perform a re-configuration boot of the system.
# touch /reconfigure
# shutdown -y -g0 -i6
Power off and then power on the platform and allow the system to boot.
Restore the virtual network
Please remember the following are examples , your system may be using different names and values.
# ifconfig -a
# ifconfig vsw0 plumb
# ifconfig e1000g0 down unplumb
# ifconfig vsw0 ip_of_e1000g0 netmask netamsk_of_e1000g0 broadcast + up
# mv /etc/hostname.e1000g0 /etc/hostname.vsw0
Check the virtual network terminal server daemon is enabled.
# svcs -l vntsd
fmri svc:/ldoms/vntsd:default
name virtual network terminal server
enabled true
state online
next_state none
state_time Wed Oct 15 16:03:54 2008
logfile /var/svc/log/ldoms-vntsd:default.log
restarter svc:/system/svc/restarter:default
contract_id 57
dependency optional_all/error svc:/milestone/network (online)
dependency optional_all/none svc:/system/system-log (online)
If this is not enabled use the following command
# svcadm enable vntsd
Restore the virtual disk server to the guest domains.
Please remember the following are examples. Your system may be using different values and names.
primary-vds0


/dev/dsk/c1t1d0s2
vol1

# ldm add-vdsdev /dev/dsk/c1t1d0s2 vol1@primary-vds0
Finally Bind and Start the remaining domains.
In this example the guest domain is named guest1.
# ldm bind-domain guest1
# ldm start-domain guest1
Appendix:
Output from command ldm ls -e -p
VERSION 1.1
DOMAIN|name=primary|state=active|flags=normal,control,vio-service|cons=SP|ncpu=8|mem=4294967296|util=12.8|uptime=684412|softstate=Solaris running
VCPU
|vid=0|pid=0|util= 17|strand=100
|vid=1|pid=1|util=0.4|strand=100
|vid=2|pid=2|util=0.4|strand=100
vid=3|pid=3|util=0.4|strand=100
|vid=4|pid=4|util= 81|strand=100
|vid=5|pid=5|util=1.5|strand=100
|vid=6|pid=6|util=2.1|strand=100
|vid=7|pid=7|util=0.0|strand=100
MAU
|id=0|cpuset=0,1,2,3,4,5,6,7
MEMORY
|ra=0x8000000|pa=0x8000000|size=4294967296
IO
|dev=pci@0|alias=pci
|dev=niu@80|alias=niu
VCC|name=primary-vcc0|nclients=1|port-range=5000-5100
VSW|name=primary-vsw0|nclients=1|mac-addr=00:14:4f:fb:6f:f9|net-dev=e1000g0|dev=switch@0|mode=
VDS|name=primary-vds0|nclients=1
|vol=vol1|opts=|dev=/dev/dsk/c1t1d0s2
VLDC|name=primary-vldc3|nclients=7
VLDC|name=primary-vldc0|nclients=4
VLDCC|name=ds|service=primary-vldc0@primary|desc=domain-services
VLDCC|name=vldcc2|service=SP|desc=spfma
VLDCC|name=vldcc1|service=primary-vldc0@primary|desc=ldmfma
VCONS|type=SP
DOMAIN|name=guest1|state=active|flags=normal|cons=5000|ncpu=8|mem=4294967296|util=0.1|uptime=512444|softstate=Solaris running
VCPU
|vid=0|pid=8|util= 75|strand=100
|vid=1|pid=9|util= 49|strand=100
|vid=2|pid=10|util= 11|strand=100
|vid=3|pid=11|util=0.0|strand=100
|vid=4|pid=12|util=100|strand=100
|vid=5|pid=13|util=0.0|strand=100
|vid=6|pid=14|util=0.0|strand=100
|vid=7|pid=15|util=0.0|strand=100
MEMORY
|ra=0x8000000|pa=0x108000000|size=4294967296
VARIABLES
|auto-boot?=true
|boot-device=vdisk1
|keyboard-layout=US-English
VNET|name=vnet1|dev=network@0|service=primary-vsw0@primary|mac-addr=00:14:4f:f9:15:9e
VDISK|name=vdisk1|vol=vol1@primary-vds0|dev=disk@0|server=primary
VLDCC|name=ds|service=primary-vldc0@primary|desc=domain-services
VCONS|group=guest1|service=primary-vcc0@primary|port=5000

Output from ldm list-constraints command for control/primary domain
ldm list-constraints -x ldom
ldom_info
00:14:4f:ab:e3:16

cpu 8 mau 1 memory 4G

physio_device pci@0

physio_device niu@80

vsw auto-allocated primary-vsw0 e1000g0

vcc primary-vcc0 5000 5100

vds primary-vds0 vds_volume vol1 /dev/dsk/c1t1d0s2 primary-vds0

Output from the file /var/opt/SUNWldm/ldom-db.xml
Primary 8  1 4294967296
pci@0 0 niu@80 0 0x00144fabe316
primary-vsw0 0x00144ffb6ff9 e1000g0
primary-vds0
/dev/dsk/c1t1d0s2 vol1
primary-vcc0 5000 5100
guest1 8 0
4294967296 0x00144ffab8f0
vnet1 primary-vsw0 0x00144ff9159e 1500
vdisk1 primary-vds0 vol1 -1
primary-vcc0 guest1 5000
auto-boot?true
boot-device vdisk1
keyboard-layout US-English








No comments:

Post a Comment