The noise of 1'000'000 inactive mailboxes

We have now migrated all inactive mailboxes (some may obviously be active again) to one 7410 Cluster.

What you can see is the IO generated by these boxes. Even if the mailboxes are abandoned they receive mails (spam, newsletters etc.)

Storage2/HDD4 and storage2/HDD8 are again the mirrored SLOG devices. As we can see here, they don't have any problems at all with the write load. If you look at all the other HDDs you see the low IOPS numbers



Looking at how many bytes per seconds are going through the disks we can see that the SLOGs are busy collecting all synchronous bits and bytes.

The slow 1TB disks get about ~700k of data per second. Looking at e.g. HDD11 we see a low number of IOPS. I would guess the average IO size is about 60-70kB. As a reference, an email is around 4k to 8k.



What this means: We get larger IOs to the disk thanks to the slog.

Thanks, mighty Logzilla :-)

SLOG Latency

After reading Brendan's newest blog entry, I was curious about what kind of slog latency we can see for our data migration load.

To remind you, only synchronous writes go into the slog SSD devices.

As this is a migration running, we can see mostly NFS write operations:



In our configuration the SSD slog is mirrored (HDD4 and HDD8). Hence the same number of IOPS:



The next picture shows us the latency for our SDD SLOG Device HDD 4. We can see here that latencies start at 79 us and are mostly under 200 us. There are some outliers, but approx. 95% are under 500 us:



This matches quite well with the values Brendan blogged about (137-181 us), which includes NFSv3 latency. For reference (no picture here), we can see latencies of about 170-500 us mostly for NFSv4.

By the way. SLOG Devices are mostly one way devices, as shown here. Only if things go really bad, they are read from...

Resilvering progress...found!

Found the CLI menu for resilvering progress :-)

file1:configuration storage (pool-1)> show
Pools:

POOL OWNER DATA PROFILE LOG PROFILE STATUS
pool = pool-1 file1 mirror_nspf log_mirror_nspf degraded

Properties:
pool = pool-1
status = degraded
owner = file1
profile = mirror_nspf
log_profile = log_mirror
cache_profile = cache_stripe
scrub = resilver in progress for 4h46m, 83.85% done, 0h55m to go

High IOPS for 1TB Disks

Wondering how 1TB disk can have such a high IOPS.

HDD8 and HDD4 are Log-Devices (SSD)



UPDATE: For the moment I assume, these are writes to the disk cache.

Being bored....

...while doing 22536 NFS ops per second, and doing gzip-2 at the same time.



HSP really works. NFS synchronous operations get "eaten" by the SSD devices (1133 IOPS).

Trekkie Stuff...

Enterprise Warp Core:





Borg Cube:





PS: We all know who won in the end ;-)

Opensolaris, Huawei E220, Swisscom and Sunrise

I was able to open a 3G connection to Swisscom and Sunrise using a Huawei E220 3G Modem under Opensolaris.

Here are the config files (If PIN is enabled on the SIM, add OK "AT+CPIN=????" to the chat script)

Swisscom:

/etc/ppp/chat-swisscom:

ABORT BUSY
ABORT 'NO CARRIER'
ABORT ERROR
REPORT CONNECT
TIMEOUT 120
"" "AT&F"
OK "ATZ"
OK "ATQ0 V1 E1 S0=0 &C1 &D2"
#OK "AT+CPIN=????"
OK 'AT+CGDCONT=1,"IP","gprs.swisscom.ch"'
SAY "Calling Swisscom"
OK "ATDT*99***1#"
TIMEOUT 120
CONNECT ''


/etc/ppp/peers/swisscom:

term/1
115200
connect "/usr/bin/chat -V -t15 -f /etc/ppp/chat-swisscom"
idle 7200
lock
crtscts
noipdefault
modem
user "gprs"
password "gprs"
noauth
passive
usepeerdns
defaultroute
connect-delay 6000
novj
nodetach


Sunrise:

/etc/ppp/chat-sunrise:

ABORT BUSY
ABORT 'NO CARRIER'
ABORT ERROR
REPORT CONNECT
TIMEOUT 120
"" "AT&F"
OK "ATZ"
OK 'ATS7=60'
#OK "AT+CPIN=????"
OK "AT+COPS?"
OK "AT&V"
OK "AT+CSQ"
OK 'AT+CFUN=?'
OK 'AT+CGDCONT=?'
OK 'AT+CPBR=?'
OK 'AT+CPBS=?'
OK 'AT+CGDCONT=1,"IP","Internet"'
SAY "Calling Sunrise"
OK 'ATDT*99***1#'
TIMEOUT 120
CONNECT ''


/etc/ppp/peers:

term/1
115200
connect "/usr/bin/chat -V -t15 -f /etc/ppp/chat-sunrise"
idle 7200
lock
crtscts
noipdefault
modem
user ""
password ""
noauth
passive
usepeerdns
defaultroute
connect-delay 6000
novj
nodetach


Don't forget to adjust the term/? line to reflect your /dev/term/? device.

After this, just fire up "pppd call swisscom" or "pppd call sunrise" and adjust your resolv.conf. The DNS Servers are in the output. For debugging just add -d to the pppd command.

There are a few problems with the Huawei E220, as it is recognized as a storage device first, and not as a serial device.

See the Opensolaris Forum

Most of the time the modem is recognized correctly when the Modem is plugged in before booting, and doing a reconfiguration boot with grub. ( press "e" -> append "-rv" -> -> press "b" )

There is also a patch for the Eee PC's Serial USB Port driver. See Masafumi's Blog.

Good luck!