hdd i transfer from is seagate external 4tb drive (STCA4000100)
dir1 - 54k of 8mb files
dir2 - 20k of 8mb files
nas4free box config:
Code: Select all
Intel Pentium G630 Sandy Bridge Dual-Core 2.7GHz
32Gb RAM
4 x 1tb WD Blue ([b]WD10EZEX[/b]) drives in ZFS Striped Vdev
My network is 10gbe Mellanox ConnectX-2 (direct attach).
iperf test yields around 700mb/s between server and client.
when i transfer small files, i get good speed for first ~3500 files, than when my RAM is to 92% usage the speed drops to 80mb/s and continues to drop from there until it drops to around 1.7kbytes/s (it's been transefring for 20+ hours now and still at 90%). when i stop copying, reboot nas, start copying without replace again, it calculates the number of files it needs to skip(which takes a lot of time), then gets back to 1.7kbytes/s speed. if i try to copy to new location, i get good speed until ram is full.
CPU usage is around 30-50% during transfer. when i try to transfer single 40gb file with clean ram, the speed is ~110mb, for some files(.mkv, .mp4) the speed jumps up to 500mb (which is weird).
is this normal behavior with lots of small files? should i go with nfs instead of cifs?
system config:
Code: Select all
nas4free 10.2.0.2 - Prester (revision 2003)
FreeBSD Revision: 199506
FreeBSD 10.2-RELEASE-p7 #0 r290499M: Sat Nov 7 18:14:14 CET 2015
x64-embedded on Intel(R) Pentium(R) CPU G630 @ 2.70GHz
ASRock H77 Pro4-M
UPS disabledCode: Select all
Authentication - Local User
Max Protocol - NT1
Interface - All Interfaces
Dos charset - CP437
Unix charset - UTF8
Log Level - Minimum
Local Master Browser - No
Time Server - No
Guest account - ftp
Map to guest - Bad User
Send Buffer Size - 128480
Receive Buffer Size - 128480
Large read/write - enabled
EA support - disabled
Store DOS attributes - disabled
Mapping DOS attributes - enabled
Null passwords - enabled
Asynchronous I/O (AIO) - enabled
AIO read size - 1024
AIO write size - 1024Code: Select all
Read only - disabled
Browseable - enabled
Guest - enabled
Inherit permissions - enabled
Recycle bin - enabled
Hide dot files - enabled
Shadow Copy - enabled
ZFS ACL - enabled
Inherit ACL - enabled
Store alternate data streams - disabled
Store NTFS acls - disabled
AIO module - aio_pthreadCode: Select all
NAME PROPERTY VALUE SOURCE
pool1 type filesystem -
pool1 creation Fri Sep 21 2:53 2012 -
pool1 used 647G -
pool1 available 2.88T -
pool1 referenced 96K -
pool1 compressratio 1.00x -
pool1 mounted yes -
pool1 quota none default
pool1 reservation none default
pool1 recordsize 128K default
pool1 mountpoint /mnt/pool1 local
pool1 sharenfs off default
pool1 checksum on default
pool1 compression off default
pool1 atime on default
pool1 devices on default
pool1 exec on default
pool1 setuid on default
pool1 readonly off default
pool1 jailed off default
pool1 snapdir hidden default
pool1 aclmode discard default
pool1 aclinherit restricted default
pool1 canmount on default
pool1 xattr off temporary
pool1 copies 1 default
pool1 version 5 -
pool1 utf8only off -
pool1 normalization none -
pool1 casesensitivity sensitive -
pool1 vscan off default
pool1 nbmand off default
pool1 sharesmb off default
pool1 refquota none default
pool1 refreservation none default
pool1 primarycache all default
pool1 secondarycache all default
pool1 usedbysnapshots 0 -
pool1 usedbydataset 96K -
pool1 usedbychildren 647G -
pool1 usedbyrefreservation 0 -
pool1 logbias latency default
pool1 dedup off default
pool1 mlslabel -
pool1 sync standard default
pool1 refcompressratio 1.00x -
pool1 written 96K -
pool1 logicalused 647G -
pool1 logicalreferenced 9.50K -
pool1 volmode default default
pool1 filesystem_limit none default
pool1 snapshot_limit none default
pool1 filesystem_count none default
pool1 snapshot_count none default
pool1 redundant_metadata all default
pool1/dataset1 type filesystem -
pool1/dataset1 creation Fri Sep 21 3:03 2012 -
pool1/dataset1 used 647G -
pool1/dataset1 available 2.88T -
pool1/dataset1 referenced 647G -
pool1/dataset1 compressratio 1.00x -
pool1/dataset1 mounted yes -
pool1/dataset1 quota none default
pool1/dataset1 reservation none default
pool1/dataset1 recordsize 128K default
pool1/dataset1 mountpoint /mnt/pool1/dataset1 inherited from pool1
pool1/dataset1 sharenfs off default
pool1/dataset1 checksum on default
pool1/dataset1 compression off local
pool1/dataset1 atime off local
pool1/dataset1 devices on default
pool1/dataset1 exec on default
pool1/dataset1 setuid on default
pool1/dataset1 readonly off default
pool1/dataset1 jailed off default
pool1/dataset1 snapdir hidden default
pool1/dataset1 aclmode discard local
pool1/dataset1 aclinherit restricted local
pool1/dataset1 canmount on default
pool1/dataset1 xattr off temporary
pool1/dataset1 copies 1 default
pool1/dataset1 version 5 -
pool1/dataset1 utf8only off -
pool1/dataset1 normalization none -
pool1/dataset1 casesensitivity sensitive -
pool1/dataset1 vscan off default
pool1/dataset1 nbmand off default
pool1/dataset1 sharesmb off default
pool1/dataset1 refquota none default
pool1/dataset1 refreservation none default
pool1/dataset1 primarycache all default
pool1/dataset1 secondarycache all default
pool1/dataset1 usedbysnapshots 0 -
pool1/dataset1 usedbydataset 647G -
pool1/dataset1 usedbychildren 0 -
pool1/dataset1 usedbyrefreservation 0 -
pool1/dataset1 logbias latency default
pool1/dataset1 dedup off local
pool1/dataset1 mlslabel -
pool1/dataset1 sync standard local
pool1/dataset1 refcompressratio 1.00x -
pool1/dataset1 written 647G -
pool1/dataset1 logicalused 647G -
pool1/dataset1 logicalreferenced 647G -
pool1/dataset1 volmode default default
pool1/dataset1 filesystem_limit none default
pool1/dataset1 snapshot_limit none default
pool1/dataset1 filesystem_count none default
pool1/dataset1 snapshot_count none default
pool1/dataset1 redundant_metadata all default
