Gigabit LAN connection with an average speed of 2,5 MB p/s

PaulW · Post by **PaulW** » 26 Jan 2009, 16:00

In my Bubba2 I have a 1 TB WD GreenPower type 1000FYP RE2-GP
http://www.westerndigital.com/en/produc ... riveid=385

The results of my Bubba2 are:

Code: Select all

cp128139-b:/home/storage# dd if=/dev/zero of=bigfile count=5k bs=100k 
5120+0 records in
5120+0 records out
524288000 bytes (524 MB) copied, 64.3138 seconds, 8.2 MB/s

On one of my local desktops: (With a WD5000AACS GreenPower disk)
http://www.wdc.com/en/products/products ... anguage=en

Code: Select all

paul@ven001:~$ dd if=/dev/zero of=bigfile count=5k bs=100k
5120+0 records in
5120+0 records out
524288000 bytes (524 MB) copied, 4,21394 s, 124 MB/s

If this is a disk performance issue why is on the LAN connection just one led blinking iso 2 leds?
This is an indication of a 10/100 Mb speed connection.

Post by **johannes** » 26 Jan 2009, 16:27

This is a strange problem, and we are trying to sort it out. First:

The dd test shows the same results here, around 16 MB/s. This is probably normal for this platform (remember, don't compare it to a PC, this one runs without fans

).

However, 2,5 MB/s isn't at all normal. When I transfer large files over samba I get about 7-8 MByte/s with a 100 Mbit/s link, this is about the maximum speed possible on this link.

When you are trying such a transfer, can you please issue the "top" command and let us know whats eating CPU power? Also, seing what is running when transfers are off might be helpful.

Thanks

carl · Post by **carl** » 26 Jan 2009, 16:34

PaulW wrote:I found something which might have something to do with it.

When I execute the command (as root):

Code: Select all

apt-get update

I get the following errors in my terminal:

Code: Select all

Get:1 http://ftp.se.debian.org etch Release.gpg [386B]
Get:2 http://update.excito.org marielle Release.gpg [189B]
Get:3 http://update.excito.org upstream_etch Release.gpg [189B]
Hit http://update.excito.org marielle Release
Hit http://ftp.se.debian.org etch Release
Hit http://update.excito.org upstream_etch Release
Err http://update.excito.org marielle Release
  
Get:4 http://update.excito.org marielle Release [2970B]
Err http://update.excito.org upstream_etch Release
  
Ign http://ftp.se.debian.org etch/main Packages/DiffIndex            
Get:5 http://update.excito.org upstream_etch Release [2651B]         
Ign http://update.excito.org marielle Release  
Ign http://ftp.se.debian.org etch/contrib Packages/DiffIndex         
Ign http://ftp.se.debian.org etch/non-free Packages/DiffIndex        
Ign http://update.excito.org upstream_etch Release                   
Hit http://update.excito.org marielle/main Packages/DiffIndex        
Hit http://ftp.se.debian.org etch/main Packages
Hit http://ftp.se.debian.org etch/contrib Packages
Ign http://update.excito.org marielle/main Sources/DiffIndex
Hit http://update.excito.org upstream_etch/main Packages/DiffIndex
Hit http://ftp.se.debian.org etch/non-free Packages
Hit http://update.excito.org marielle/main Sources
Fetched 6000B in 1s (4193B/s)
Reading package lists... Done
W: GPG error: http://update.excito.org marielle Release: Unknown error executing gpgv
W: GPG error: http://update.excito.org upstream_etch Release: Unknown error executing gpgv
W: You may want to run apt-get update to correct these problems

Till now I didn't find a solution for this. (Google shows a lot of this...)
I can imagine that because of this some updates are not or not correctly installed which is causing my speed problem.

This could be due to time errors; if the clock in the bubba is too far back in time; try:

Code: Select all

apt-get -c /etc/apt/bubba.conf update

which includes an notion to ingore excessive time errors for gpg.

/Carl

PaulW · Post by **PaulW** » 26 Jan 2009, 17:23

Thanks for both of your quick respons.

@Johannes:
With copying a file to the bubba top gives the results:

Code: Select all

[size=7]top - 23:05:06 up 2 days,  1:51,  1 user,  load average: 0.81, 0.22, 0.13
Tasks:  90 total,   3 running,  87 sleeping,   0 stopped,   0 zombie
Cpu(s): 72.0%us, 19.4%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  8.6%si,  0.0%st
Mem:    256456k total,   253276k used,     3180k free,     3540k buffers
Swap:  1156672k total,   129704k used,  1026968k free,   119176k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                        
 9678 paul      20   0 11624 3164 1224 R 42.7  1.2   0:20.44 sshd                                                                                           
 9682 root      20   0 64040  15m 9292 R 42.0  6.3   0:02.28 php                                                                                            
 9679 paul      20   0  5632 1360 1088 S  8.3  0.5   0:03.98 scp                                                                                            
   99 root      15  -5     0    0    0 S  3.2  0.0   3:33.88 kswapd0                                                                                        
 9674 paul      20   0  2976 1268  996 R  1.9  0.5   0:01.08 top                                                                                            
 2624 squeezec  20   0 75988  23m 3464 S  1.3  9.6  40:09.18 squeezecenter-s                                                                                
   98 root      20   0     0    0    0 S  0.6  0.0   3:03.68 pdflush                                                                                        
 2344 root      20   0 61620 2476 2128 S  0.6  1.0   3:41.09 ftd.bin                                                                                        
 2213 mysql     20   0  128m  11m 3344 S  0.3  4.6   1420:30 mysqld                                                                                         
    1 root      20   0  2436  724  652 S  0.0  0.3   0:05.58 init                                                                                           
    2 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 kthreadd[/size]

Without doing anything to the Bubba:

Code: Select all

top - 22:43:35 up 2 days,  1:30,  1 user,  load average: 0.20, 0.34, 0.24
Tasks:  84 total,   1 running,  83 sleeping,   0 stopped,   0 zombie
Cpu(s):  1.6%us,  0.7%sy,  0.0%ni, 97.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:    256456k total,   200580k used,    55876k free,     2544k buffers
Swap:  1156672k total,   104148k used,  1052524k free,    57476k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                        
 2624 squeezec  20   0 75988  30m 3676 S  1.3 12.3  39:56.02 squeezecenter-s                                                                                
 9607 paul      20   0  2976 1264  996 R  1.3  0.5   0:01.34 top                                                                                            
 2213 mysql     20   0  128m  23m 4728 S  0.7  9.2   1420:26 mysqld                                                                                         
    1 root      20   0  2436  752  656 S  0.0  0.3   0:05.54 init                                                                                           
    2 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 kthreadd                                                                                       
    3 root      15  -5     0    0    0 S  0.0  0.0   0:07.70 ksoftirqd/0                                                                                    
    4 root      15  -5     0    0    0 S  0.0  0.0   0:23.64 events/0                                                                                       
    5 root      15  -5     0    0    0 S  0.0  0.0   0:00.08 khelper                                                                                        
   52 root      15  -5     0    0    0 S  0.0  0.0   1:48.06 kblockd/0

There was one strange behavior of the Bubba2 I discovered a few minutes ago before creating this message. Somehow I couldn't reach your site anymore. A small investigation shows that the eth0 port on the Bubba2 was set to its default value 576. After changing this back in 1500 everything was working again. Something caused the WAN port to reset to its default settings.
Normally the 576 value is set after the Bubba2 is rebooted.

@Carl: You are right. The error messages were a result of a faulty clock value.
The Bubba2 was still living in 2008

After changing the clock was working fine.

jonte · Post by **jonte** » 28 Jan 2009, 06:06

johannes wrote: When you are trying such a transfer, can you please issue the "top" command and let us know whats eating CPU power? Also, seing what is running when transfers are off might be helpful.

Here,s another pair of top from a slow moving B2 (2,5 mb/s smb). Hope this is of any help.

First one is during SMB-filetransfer:

Code: Select all

top - 11:52:06 up 6 days, 18:23,  1 user,  load average: 0.98, 0.36, 0.12
Tasks:  73 total,   2 running,  71 sleeping,   0 stopped,   0 zombie
Cpu(s):  8.1%us, 49.5%sy,  0.0%ni,  1.6%id,  0.0%wa,  7.1%hi, 33.7%si,  0.0%st
Mem:    256456k total,   253360k used,     3096k free,      840k buffers
Swap:  1060280k total,      680k used,  1059600k free,   172868k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 7182 jonte     20   0 12476 3768 2744 R 89.2  1.5   1:05.62 smbd
 7321 root      20   0  2976 1256  996 R  1.9  0.5   0:07.18 top
   52 root      15  -5     0    0    0 S  1.3  0.0   0:19.04 kblockd/0
   99 root      15  -5     0    0    0 S  0.6  0.0   0:51.46 kswapd0
23830 root      20   0 58024 7656 4772 S  0.6  3.0   1:04.34 php5-cgi
    1 root      20   0  2436  760  660 S  0.0  0.3   0:17.68 init
    2 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 kthreadd
    3 root      15  -5     0    0    0 S  0.0  0.0   1:15.72 ksoftirqd/0
    4 root      15  -5     0    0    0 S  0.0  0.0   1:07.04 events/0
    5 root      15  -5     0    0    0 S  0.0  0.0   0:00.08 khelper
   59 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 ata/0
   60 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 ata_aux
   68 root      15  -5     0    0    0 S  0.0  0.0   0:00.02 khubd
   97 root      20   0     0    0    0 S  0.0  0.0   1:21.72 pdflush
   98 root      20   0     0    0    0 S  0.0  0.0   3:24.08 pdflush
  100 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 aio/0
  101 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 nfsiod

This second one is the B2 doing "nothing"

Code: Select all

top - 11:48:08 up 6 days, 18:19,  1 user,  load average: 0.14, 0.10, 0.03
Tasks:  73 total,   1 running,  72 sleeping,   0 stopped,   0 zombie
Cpu(s):  1.3%us,  0.7%sy,  0.0%ni, 98.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:    256456k total,   242800k used,    13656k free,    91884k buffers
Swap:  1060280k total,      664k used,  1059616k free,    49972k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 7321 root      20   0  2976 1256  996 R  1.3  0.5   0:03.78 top
 2117 mysql     20   0  126m  37m 5676 S  0.3 15.0  32:35.58 mysqld
    1 root      20   0  2436  760  660 S  0.0  0.3   0:17.68 init
    2 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 kthreadd
    3 root      15  -5     0    0    0 S  0.0  0.0   1:15.70 ksoftirqd/0
    4 root      15  -5     0    0    0 S  0.0  0.0   1:07.00 events/0
    5 root      15  -5     0    0    0 S  0.0  0.0   0:00.08 khelper
   52 root      15  -5     0    0    0 S  0.0  0.0   0:18.42 kblockd/0
   59 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 ata/0
   60 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 ata_aux
   68 root      15  -5     0    0    0 S  0.0  0.0   0:00.02 khubd
   97 root      20   0     0    0    0 S  0.0  0.0   1:21.70 pdflush
   98 root      20   0     0    0    0 S  0.0  0.0   3:24.08 pdflush
   99 root      15  -5     0    0    0 S  0.0  0.0   0:50.68 kswapd0
  100 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 aio/0
  101 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 nfsiod
  691 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 scsi_eh_0

Hammer · Post by **Hammer** » 28 Jan 2009, 10:30

To cheer you up, I had 2.5 Mbit/s (not byte!) for several weeks, and then it crashed wholly and irrecoverably. Couldn't even reformat with an image on a USB stick.

Maybe your's is going that way as well...

Cheers,
Hammer

davidI · Post by **davidI** » 29 Jan 2009, 13:15

I get the following results with a 100MBit/s network and a Linux intel PC.

Could the disabling of the hardware checksum be a reason? The CPU has to work harder if the checksum has to be calculated with software.

In summary about 7Mb/s with ftp and 2.5Mb/s with scp when the theoretical limit is about 12.5 Mb/s. Maybe ssh isn't configured optimally?

1) 16.7Mbs with the dd command.

2) 2.3Mb/s with scp bubba:bigfile . run on my PC, and sshd shows about 90% CPU on the bubba
3) 2.4Mb/s scp bigfile bubba:bigfile run from my PC and sshd on the bubba shows 83%

4) scp run on bubba from bubba to PC gets 2.8Mb/s and PC's sshd uses 8% CPU, bubba shows ssh using about 85% CPU.
5) scp run on bubba from PC to bubba gets 2.3Mb/s and PC's sshd uses 4% CPU.

6) 7.5MB/s with ftp get run from my PC and proftpd shows 90% CPU.
7) 6.4 Mb/s with "ftp put" run from my PC proftpd shows 86%

pa · Post by pa » 02 Feb 2009, 04:16

Hi,

Trying to shed some light on this.

Expected transferspeeds using samba is 7-8MByte/s, using jumboframes can increase this to 10-12, but jumboframes support is to be considered experimental. This might work in your LAN and it might not, it depends on all of the quipment in your network, not just Bubba and your PC.

Expected transferspeeds using ssh is 2.5-3 MByte/s. This is due to the limited CPU-power available for the encryption of the transfer. Remember not to compare the cpu power of bubba with a regular PC, bubba is build to be quiet and power efficient.

/PA, Excito

Post by **johannes** » 26 Mar 2009, 04:40

Ok, some of you probably suffered from the network performance bug, fixed now. Please see:

http://forum.excito.net/viewtopic.php?t=1685

joost · Post by **joost** » 27 Mar 2009, 13:14

kees wrote:I have the same speed problem. FTP tranfers 4.0 Mb/s and SCP, SMB transfers ca. 2.5 Mb/s. This should be greater then 35Mb/s for a Gigabit network.

And exactly the same problem here.
I've already tried many ways to fix this, but no luck. So I've (grudgingly) tried to accept it and just be a little more patient while transferring data. :(

Post by **johannes** » 28 Mar 2009, 06:14

joost, did you do the upgrade and reboot? No difference?

joost · Post by **joost** » 30 Mar 2009, 18:00

Johannes, yes, I did do the upgrade, but had no luck.

It seems there was a problem with the (probably uncompleted) upgrade first time I tried though, as I upgraded again yesterday and now it seems much better! :)

joost · Post by **joost** » 04 Apr 2009, 10:09

I wrote that it seemed much better, but after checking it for a while transfer speed seems to be only marginally beter.
4,5 MB write and 5 MB read speed. Oh well.

Also a new problem which I hadn't noticed before cropped up: when I cut and paste a directory from Bubba to a pc, it only copies the directory structure but leaves the files where they were. Only remedy is to copy the contents of every directory manually.

yooakim · Post by **yooakim** » 15 May 2009, 02:56

I just upgraded my home network to 1GBits.

When I do a scp of large AVI files from the bubba to a Ubuntu PC I get performance around 2.5MB/s

Is this to be expected?

My Bubba ethernet port has the following config:

Code: Select all

xxxxxx@BUBBA:~$ sudo ethtool eth1
Settings for eth1:
	Supported ports: [ MII ]
	Supported link modes:   10baseT/Half 10baseT/Full 
	                        100baseT/Half 100baseT/Full 
	                        1000baseT/Full 
	Supports auto-negotiation: Yes
	Advertised link modes:  10baseT/Half 10baseT/Full 
	                        100baseT/Half 100baseT/Full 
	                        1000baseT/Full 
	Advertised auto-negotiation: Yes
	Speed: 1000Mb/s
	Duplex: Full
	Port: MII
	PHYAD: 0
	Transceiver: external
	Auto-negotiation: on
	Current message level: 0x0000003f (63)
	Link detected: yes

The disk performance is as follows (which seems OK for a device such as bubba)

Code: Select all

xxxxx@BUBBA:~$ dd if=/dev/zero of=bigfile bs=1M count=1K
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 67.447 seconds, 15.9 MB/s

Copying using SCP from BUBBA to my Macbook (wirelessly) gives me speed of about 2.3MB/s

I would expect to get a little more speed when copying from BUBBA to a Ubuntu PC connected via a 1GB switch...

UPDATE: I just checked with the top command and the process that is using most of the CPU is SSHD, around 77% CPU according to TOP.

Any recommendations for how to fix this?

NB: I did a full update/upgrade earlier today.

Cheers,
Joakim

pa · Post by pa » 15 May 2009, 03:02

If you use scp to copy files yes, this is expected as the crypto is heavy and the transfers are limited by the available cpu-cycles.

Remember that Bubba is designed to be low power with no fans, so it should not be compared to a big noisy PC.

If you are inside your LAN, I suggest using another protocol to transfer files, you should be able to see ~7MB/s using samba.

/PA

forum.excito.org

Gigabit LAN connection with an average speed of 2,5 MB p/s

Re: slow network speed

Status of this issue?