Finding out what's using disk

[Gordon]

Thought I'd dig up this old thread as a better place to discuss disk usage than a question whether Excito would support a WD red disk.

Anyway, I just did some monitoring on my system and found some periodic writes that were not discussed here before:

• LMS (squeezeboxserver) does a handshake every 10 minutes on devices connected through WLAN. Judging by the timings these are related to when LMS was started
• Samba updates `browse.dat` every 12 minutes
• Tor complains every 20 minutes that my port 9001 is not accessible from the outside
• Cups changes some type of certificate stored in /var/run/cups/certs/0 every 13 minutes

Probably more to find still, but from these findings I guess you'd have to stop just about every service on the B3 to make it sleep for any time longer than just a few minutes. Obviously I could do without tor, but LMS and Samba and Cups are things that I use, as well as email.

[ingo2]

Ok Gordon,

you are right, I am running only a few Excito-services on my box:

a) FTP-server (seldom used)
b) Samba, but generally there are no clients in my network, because we have all Linux PC's. Just our Smart-TV sometimes connects and wakes the B3 up. Generally I use NFSv4 which is much faster than Samba and easier to configure, so:
c) NFS-Server with several exports via NFSv4. Even if clients are connected but there is no traffic, the HD spins down. Throughput usually is some 15MB/s.
d) DLNA-Steaming is on
e) CUPS is running, but no printers connected.
f) "Up- and Down-loads" is on.

The other Excito-services like Squeezebox, DAAP, E-mail, AFP are all off.

g) 'rsync' is also running as a server with a few modules configured. That I use for backup of my PC's over the network. Have some comfortable scripts written for that which recursively first calculate md5sums of files in the current directory, then rsync the whole directory content to a selectable module on the B3. This usually achieves 18MB/s and syncing a whole Linux system partition usually takes less than 10 minutes.

h) I almost forgot: of course a SSH-server is running

So, that's all and since yesterday I had only 3 spin-up's (Start/Stop Counts according to 'smartctl -A'):
1st supposed by switching TV on, 2nd by cron jobs this morning, 3rd from ssh login now to check and from admin interface to see which services are enabled.

Regards,
Ingo

[Gordon]

heheh...

LCC has been stuck at 548,943 for 457 days now.

[ingo2]

mine reports a few hundred Start/Stop Counts. I don't get LCC's because it is no WD-Green, so it does not unload the heads after a few seconds.

[Gordon]

Well I'm close to owning the B3 exactly two years now, which counts up to 730 days. Minus the current uptime since the 5 minute patch was put on the drive that means I was increasing my LCC, or start/stop count, with around 2,000 each day. Given that the drive is guaranteed to be able to handle an LCC upto 600.000, that would have rendered it unusable within less than a year.

In conclusion this means that the WD green is essentially a bad choice for this device, although in some circumstances it might actually make this a real low power machine. I think Excito would do wise about extending the web frontend with a module that reports the LCC increment rate and will offer to run the 5 minute patch if it exceeds a certain limit (<200 a day would still be good for 8 years, which is probably more than you could ever expect the whole system to last).

[flimflam]

Oh, oh .... so thats a bug in head, if WD Green is not the right choice when I am close to buy it, what is the good choice?

Let´s say 3 TB is limit, which brand than?

So i copy that sleep ability ( WD Green ) is useless while B3 is all the time calling for some service? Or?

[Gordon]

Like I stated in that other thread: looking at the specs for the WD red it seems to me that this drive is exactly the same as the green drive, with the exception of the ridiculously low time setting for unloading the drive. Since we can change that, you can decide for yourself whether your usage of the B3 allows keeping the 8 second green setting or the drive should be converted to behave red. And there is in fact a sourceforge project that gives you the power to enter other timeouts that just 5 minutes, so using that you could also experiment with values like 1 minute (you can increment with 0.1 second steps upto 12.8 and then starting with 30 seconds in 30 second increments). After all if the drive does go into sleep mode that will save power and money.

In any case I will be getting a second B3 in from the summer campaign (with the 1TB green disk) and will be doing some additional monitoring on LCC increments on that one. It will be running less services, although I do intend to experiment with domotica as well. I may decide to get a Pi for that later on though.

[ingo2]

The WD patch tool should still be part of the installation, see here:
http://forum.excito.net/viewtopic.php?f ... 120#p21725

Without that applied I would not recommend a WD-green drive for NAS use.

[Gordon]

Got my new B3 in yesterday and did some first experiments today. Things I found:

• The default setting on the drive is to unload after 12.8 seconds (not 8)
• Cups rewrites /var/run/cups/certs/0 every 5 minutes but you can stop it from doing that by adding the following line to /etc/cups/cupsd.conf

  Code: Select all

  RootCertDuration 0

• I found Samba writing to unexpected.tdb at least once every minute. This appears to be caused by my Dune mediaplayer not behaving exactly according to protocol.
• There's a cron job for clearing old PHP session files every 30 minutes at 09 and 39 and I wonder if that really needs to be run that often on a server that attracts just a couple of users per hour (or even none during such a period).
• Running smartctl seems to trigger a load cycle as well - at least that's what it looks like right now

In any case automated activity seems mostly concentrated around the /var tree and I thought I'd give it a try and move that part of the file system onto solid state. Picking the right thumb drive proved to be difficult though, as none of them appear to have their speeds stamped on them. As I had an old card reader lying around doing nothing I used that instead and plugged a class 6 SD card in (Pi recommends a class 10 card). So far it seems to run smooth and I had an LCC increment of only 2 in the last hour. With 4 hours drive spinning time (smartctl only shows an integer count for this) I'm now up to 160 LCC (I was at 146 after the first 2 hours).

PS I also incremented the drive idle time setting from the original 12.8 seconds to 1 minute.

[ingo2]

Got my new B3 in yesterday and did some first experiments today. Things I found:

• There's a cron job for clearing old PHP session files every 30 minutes at 09 and 39 and I wonder if that really needs to be run that often on a server that attracts just a couple of users per hour (or even none during such a period).

You can safely move this cron.hourly job to cron daily. That's how I did. It is more than enough for our usage of B3.
I created a small cron script "/etc/cron.daily/php5-cleanup":

Code: Select all

#!/bin/sh#
#
# Replaces the cron enty in /etc/cron.d/php5 which removes 
# php5 session cookies every 30 minutes. A daily cleanup is 
# more then sufficient for home use. Disable the cron entry 
# in /etc/cron.d/php5 instead.

[ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find \
/var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -delete

Here this was the only script in cron.hourly. So after above change you can safely disable cron.hourly completely in /etc/crontab.

/Ingo

[Gordon]

What I found out about the B3 is that if you remove a script it will usually come back whenever there's a software update. Therefore I normally choose to alter the original script or change the environment in such a way that it will be ignored. In case of cron the best way to get full control over what is done is to disable every entry in the system crontab and add whatever you do require in root's own crontab (or whatever service account the job should be run as).

The amazing thing is that with my ear to the machine it sounds like the disk keeps on spinning all the time, even though the machine shouldn't be doing anything. Not sure if that's what it is supposed to do...

[ingo2]

The amazing thing is that with my ear to the machine it sounds like the disk keeps on spinning all the time, even though the machine shouldn't be doing anything. Not sure if that's what it is supposed to do...

I recommend (I did this) to manually install 'hdparm' and configure it in

Code: Select all

/etc/hdparm.conf'                (attach at the end):

# Sleep delay = 120 * 5sec, (is not linear - see man page for details).
/dev/sda {
spindown_time = 120
}

and reboot - yes, correct though ots Linux, the HD will "forget the settings" after each powqer cycle.
But maybe the WD-drives stll do not beheave like expected according to ATA settings

To check wether th HD is running or spun down, log in via SSH and exectue:

Code: Select all

# hdparm -C /dev/sda

it will tell you eithe "active/idle" = spinning, or "standby" = spun down. This command does not wake up the HD!

[ingo2]

Yo can also force the HD to spin down:

Code: Select all

hdparm -y /dev/sda

[DanielM]

You can safely move this cron.hourly job to cron daily. That's how I did. It is more than enough for our usage of B3.

Another way is to completely remove that script and in /etc/php5/apache2/php.ini check that session.gc_probability is set to 1, this means that every thousand login old sessions will be removed automatically. I'm not sure what the default for B3 is, but the Debian default is 0 for session.gc_probability and instead removing the old sessions via cron.

/Daniel

[Gordon]

Another way is to completely remove that script and in /etc/php5/apache2/php.ini check that session.gc_probability is set to 1, this means that every thousand login old sessions will be removed automatically. I'm not sure what the default for B3 is, but the Debian default is 0 for session.gc_probability and instead removing the old sessions via cron.

/Daniel

Good catch!

I actually found that for production session.gc_divisor (just below session.gc_probability) should be set at 100. Obviously if you only use web to access the B3 interface the developer setting at 1000 will be just fine. Should probably note though that this B3 interface runs using the fastcgi extension, so that changing the php settings for apache only will likely not be enough to actually trigger the garbage collection.