Install Solaris 10 U5 in a guest LDOM from CDROM/DVD or ISO

With the new release of Solaris 10 5/08 (Update 5) we now have the chance to install guest Logical Domains from CDROM/DVD or an ISO image. In the previous releases, the only way to install Solaris 10 in a LDOM was using a Jumpstart Server.

Here we go, fasten your seatbelt ;):


This is the starting point, three standard services with some values (virtual disk service, virtual console service and virtual network service).
# ldm ls-services
VDS
NAME VOLUME OPTIONS DEVICE
primary-vds0 vol0 /disco1

VCC
NAME PORT-RANGE
primary-vcc0 5000-5100

VSW
NAME MAC NET-DEV DEVICE MODE
primary-net0 00:14:4f:fb:2b:da e1000g0 switch@0 prog,promisc

# ldm list-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv SP 4 1G 8.2% 16h 36m
ldom1 bound ----- 5000 4 2G

Now we can start adding some resources to the guest domain we're creating (CPU and Memory):

# ldm add-domain ldom2
# ldm add-vcpu 4 ldom2
# ldm add-memory 4g ldom2

The next commands set up the virtualized network interface and the file that will be used as a virtual disk to install Solaris

# ldm add-vnet vnet0 primary-net0 ldom2

# mkfile 6g /datos1/disco2
# ldm add-vdiskserverdevice /datos1/disco2 vol1@primary-vds0

# ldm add-vdisk vdisk0 vol1@primary-vds0 ldom2
# ldm set-variable auto-boot\?=false ldom2

And now the new feature with Solaris 10 U5, we add the ISO image as a virtual device that will be used as the installation media:

# ldm add-vdiskserverdevice /root/sol-10-u5-ga-sparc-dvd.iso iso@primary-vds0

# ldm add-vdisk iso iso@primary-vds0 ldom2

-- Start of NOTE --
Note: The procedure to install from a CDROM or DVD is the same:
# cdrw -l
Looking for CD devices...
Node Connected Device Device type
----------------------+--------------------------------+-----------------
/dev/rdsk/c0t0d0s2 | MATSHITA CD-RW CW-8124 DZ13 | CD Reader/Writer

# ldm add-vdiskserverdevice /dev/dsk/c0t0d0s2 cdrom@primary-vds0

# ldm add-vdisk cdrom cdrom@primary-vds0 ldom2

----End of NOTE----

Let's bind and start the new domain:

# ldm bind-domain ldom2
# ldm start-domain ldom2
LDom ldom2 started

# telnet localhost 5001
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

Connecting to console "ldom2" in group "ldom2" ....
Press ~? for control options ..

{0} ok show-disks
a) /virtual-devices@100/channel-devices@200/disk@1
b) /virtual-devices@100/channel-devices@200/disk@0
q) NO SELECTION
Enter Selection, q to quit: a
/virtual-devices@100/channel-devices@200/disk@1 has been selected.
Type ^Y ( Control-Y ) to insert it in the command line.
e.g. ok nvalias mydev ^Y
for creating devalias mydev for
/virtual-devices@100/channel-devices@200/disk@1
{0} ok devalias
iso /virtual-devices@100/channel-devices@200/disk@1
vdisk0 /virtual-devices@100/channel-devices@200/disk@0
vnet0 /virtual-devices@100/channel-devices@200/network@0
net /virtual-devices@100/channel-devices@200/network@0
disk /virtual-devices@100/channel-devices@200/disk@0
virtual-console /virtual-devices/console@1
name aliases

We now boot from the virtualized iso image appending the :f (this is to specify the slice 6 of the DVD/ISO image). This can also be done using boot iso:f

{0} ok boot /virtual-devices@100/channel-devices@200/disk@1:f
Boot device: /virtual-devices@100/channel-devices@200/disk@1:f File
and args:
SunOS Release 5.10 Version Generic_127127-11 64-bit
Copyright 1983-2008 Sun Microsystems, Inc. All rights reserved.
Use is subject to license terms.
Configuring devices.
Using RPC Bootparams for network configuration information.
Attempting to configure interface vnet0...
Skipped interface vnet0
Setting up Java. Please wait...
Extracting windowing system. Please wait...
Beginning system identification...


That's all :). Very usefull and good feature that comes from OpenSolaris.

Solaris 10 + Sun Cluster 3.2 + ZFS + QMail (HA - Failover)

Trying to improve our mail system, I decided to change our previous architecture to an edge technology like ZFS + Sun Cluster.

Our previous mail server had a Linux Debian + Qmail LDAP + OpenLDAP + SpamAssassin filters + Antivirus and so on with an internal storage. The performance was quite good but there are some missing features in OpenLDAP and the standard Linux file systems.

Using my experience with Solaris (right now I feel more comfortable in Solaris than in Linux) I proposed a significant change:

Two nodes running Solaris 10, with Sun Cluster 3.2 (quorum device in an external storage... may be some day I'll try the quorum server), a pool of ZFS for the mail spool (with compression enabled as the traffic is not too heavy), SunOne Directory Server 5 (I'd prefer 6, but I'm not the LDAP specialist so...) and QMail with LDAP enabled (with the standard options... SpamAssassin, DKIM, qmail-scanner-st with several antivirus, RBL, etc..). The courier imap is also part of the architecture.

Everything working on HA (ZFS does not support Scalable services yet).

Why QMail?
I love this MTA (if it's patched with LDAP support). It's a little bit tricky to meet all the requirements in Solaris 10 (compiled with Sun Studio, but gcc is also fine) but in my opinion is the best talking about performance.

Solaris 10 (11/06): it offers many good things (SMF, ZFS, DTrace and so on...). I'm a Solaris instructor so...

Sun Cluster 3.2: Support for ZFS... that's enough for me. We have an external storage so there's no need to use the quorum server feature.
I developed a specific QMail data service agent that uses the daemon tools but I guess that with the Generic Data Service Agent is enough.

ZFS: With an external storage (and old one but good enough) I built a Raidz pool with a dataset (compression enabled) for the mail spool. The compression algorithm is fast, but the compress ratio is not like gzip or bzip. Our spool was 20GB in the old system and after the upgrade is 17GB. Yes, in a production environment ;). I wrote a script to manage the snapshot feature (I don't think it's really usefull in this case, but it doesn't disturb). Our mail traffic is not really heavy, so the compression and the snapshots are not a performance problem.

Sun Directory Server 5: The multimaster feature makes this product better for this cluster environment than openldap. I'm not a LDAP specialist but I love version 6... it works quite well with WebConsole (I know... it takes the ram as if it was free...).

The "bad" thing of this is the big amount of packets you need to install before QMail works, but after that you have a fast system delivering mails, with the flexibility of an MTA that rocks. The integration with SpamAssassin and the very good qmail-scanner-st tool is awesome.

Here is the result (sorry... still using the old cli of Sun Cluster):

root@tupolev:/> scstat -g

-- Resource Groups and Resources --

Group Name Resources
---------- ---------
Resources: haqmail-rg haqmail-ds qmail qmail-ds


-- Resource Groups --

Group Name Node Name State Suspended
---------- --------- ----- ---------
Group: haqmail-rg tupolev Online No
Group: haqmail-rg metcha Offline No


-- Resources --

Resource Name Node Name State Status Message
------------- --------- ----- --------------
Resource: haqmail-ds tupolev Online Online
Resource: haqmail-ds metcha Offline Offline

Resource: qmail tupolev Online Online - LogicalHostname online.
Resource: qmail metcha Offline Offline

Resource: qmail-ds tupolev Online Online - Service is online.
Resource: qmail-ds metcha Offline Offline

When it's clean enough, I'll post the Data Service Agent for QMail.