Tuning general/fs performance on a 2GB machine

[halfer]

Hi all

I'm a long-time FreeNAS user, and have recently upgraded to 0.7.2 Sabanda (revision 8191). The performance of my machine has never been great, but with this upgrade (or perhaps the one before it) it has decreased even further. I suspect it is a ZFS problem, since I've only 2GB of RAM - but I am all ears for general suggestions. I'm comfortable with the cli, but am not at all au fait with hardware and os tuning.

My server is the 64bit Atom-powered BBS2, with 2GB RAM and four SATAII drives (2 x 1TB, 2 x 2 TB). I've configured all disks to have a UFS filing system, with Blowfish atop each, and then each of these being used as a virtual device for ZFS. In ZFS I've set up each to be a mirrored pair, so I have two ZFS storage pools: 2TB and 1TB.

I access the server in two ways: from a Linux netbook for music streaming, via read-only SMB using a 200MBit EoP network. Also via a MacBook Pro over N-standard wifi (MCS
index = 15, so I should get 130-300Mbit/sec). Both network types go to a domestic Netgear gigabit wifi/switch, which then connects to the server.

Music reading from the server is now choppy to the Linux machine (I occasionally get a second of silence, though this didn't happen with older versions of FreeNAS). Also, I notice that if I copy a large file from my Mac to the server, via the GUI, the progress indicator is quite 'bursty' - nothing happens for a while, and then 35M appears to have transferred (although this may not reflect an inconsistent write speed, I suppose). rsync on the Mac reports speeds of between 3.5 and 5.5M/sec for a 1 GB file going to the server. Reading from the server, the average rsync speed for a large file is around 3.5M/sec (1M/sec slower than writing). Copying a large file from the server to my Mac is also bursty - it seems fine for the first 10 seconds, updating the progress bar every second, and then appears to read in 10M blocks, and at very inconsistent speeds.

Since both networks are affected by poor performance, I think this a server issue. I'm not looking to get amazing performance (no streaming of full HD here, yet anyway!). But I'd like to achieve at least 10-20M/sec both ways (wireless N should manage that, I reckon, and the EoP should get close to that too).

* Should I upgrade to the nightly of Nas4Free? (I note there's no stable).
* Is my using ZFS with only 2GB of RAM a problem? My mobo can't take any more RAM, although I'm not opposed to getting a new one if I can get one to fit my server (I am not willing to buy a new server however, as my BBS2 is much-cherished, and not ready for flogging off yet). I could just switch to Software RAID and get rid of ZFS entirely - I've not used incremental snapshotting in several years, although I keep saying I'd like to try it! So I'd like to stick with ZFS if poss.
* Should I try a drive with a different encryption? This is a bit of a hassle, so I don't want to try it if it is unlikely to yield results.

(I'm sticking with FN7, btw, since 8.2 doesn't yet have encryption options).

Thanks for any suggestions.

Edit: wow, just noticed that Nas4Free runs ZFS28, while FN8 is only running 15! (And I'm only running a mix of 1 and 3 - all tests above done with the latter).

[raulfg3]

* Should I upgrade to the nightly of Nas4Free? (I note there's no stable).

try it, you allways can do a fresh install on a new USB and import your old pool to test it, if somethings goes wrong go away and use old USB to boot normally.

I suspect it is a ZFS problem, since I've only 2GB of RAM

In general you need 1GB RAM for each 1 TB of DATA disk so if you have 6 TB disk, a good aproach is to have 6GB of RAM (8 is better), but you can tune ZFS to use it with less RAM, to do it use/install daoyama ZFSkerntune , it's not magic, but normally apport stability to your system ( always is better to have more RAM).

I access the server in two ways: from a Linux netbook for music streaming, via read-only SMB using a 200MBit EoP network

try NFS always is better and you forget choppy conections that you comment.

I've configured all disks to have a UFS filing system, with Blowfish atop each, and then each of these being used as a virtual device for ZFS. In ZFS I've set up each to be a mirrored pair, so I have two ZFS storage pools: 2TB and 1TB.

I never seen a config like yours, I think that you are bottleneck your CPU/Disk using this config.

Are you try to do a Zpool of nop devices?

[raulfg3]

Should I upgrade to the nightly of Nas4Free? (I note there's no stable).

Latest nightly are really Stable, so nevermind the folder (Stable or Nightly ) all are really stables.

Should I try a drive with a different encryption? This is a bit of a hassle, so I don't want to try it if it is unlikely to yield results

I think YES, (AES is normally best optimized and use less resources) but good test are need to do in your hardware, so you need to previosly saved your valuable data prior to do some test.

Is my using ZFS with only 2GB of RAM a problem? My mobo can't take any more RAM, although I'm not opposed to getting a new one if I can get one to fit my server (I am not willing to buy a new server however, as my BBS2 is much-cherished, and not ready for flogging off yet). I could just switch to Software RAID and get rid of ZFS entirely - I've not used incremental snapshotting in several years, although I keep saying I'd like to try it! So I'd like to stick with ZFS if poss.

try to install ZFSkerntune and configure it for 2GB of RAM.
viewtopic.php?f=55&t=26

PD: Do you use x64 Version? <- Allways works better ZFS on a native 64bit CPU that in a x86

[halfer]

Ah, a bit more searching reveals this problem - I'd missed that before.

I never seen a config like yours, I think that you are bottleneck your CPU/Disk using this config.

I can't remember where I got it from, but I didn't figure it out myself! I would have googled it, and found out how to do it from a blogpost, or maybe the FN7 wiki. It's just UFS+geli+ZFS mirrors; pretty easy to set up.

Are you try to do a Zpool of nop devices?

I don't know what that is, so probably not

try it, you allways can do a fresh install on a new USB and import your old pool to test it, if somethings goes wrong go away and use old USB to boot normally.

Good idea, will do.

In general you need 1GB RAM for each 1 TB of DATA disk so if you have 6 TB disk, a good aproach is to have 6GB of RAM (8 is better), but you can tune ZFS to use it with less RAM, to do it use/install daoyama ZFSkerntune , it's not magic, but normally apport stability to your system ( always is better to have more RAM).

Ah brilliant, I've not heard of ZFkerntune - will try that.

I didn't think of checking 'top' during a copy - will also give that a whirl. Thanks for your thoughts, will report back!

[raulfg3]

raulfg3 escribió:Are you try to do a Zpool of nop devices?


I don't know what that is, so probably not

at the end I was talking about the same that you has, sorry I don't relationate GELI= UFS Encripted.

Should I try a drive with a different encryption? This is a bit of a hassle, so I don't want to try it if it is unlikely to yield results

I think YES, (AES is normally best optimized and use less resources) but good test are need to do in your hardware, so you need to previosly saved your valuable data prior to do some test.

in the example use 128 bit AES, I think that is fast that BlowFish but not sure ( I don't use it)

0.7.2 Sabanda (revision 7S-5255)
amd64-embedded on Intel(R) Pentium(R) Dual CPU E2160 @ 1.80GHz
6GB of DDR2 RAM
Network: 1 Gbit
GELI: 128-bit AES - default
ZFS: 4 x disk 1,5TB in pool ZRAID1

[lasdem]

From my experiance, if you build a ZFS pool with encrypted disks, you probably have your CPU as the bottleneck (I do).
I have 2GB RAM, 4x2TB encrypted (without any filesystem beneath it) and put them in a RaidZ1 (Raid5) virtual device which is the only device in my zpool.
I have about 5-8 MB/s write and 10-12 MB/s read. Which seams enough to watch 1080p videos without problems.
And If I watch the CPU of my NAS while copieng files, it is always above 97%.

PS: I have not tried ZFkerntune yet because I run embedded install.

[lux]

encryption slows down my NAS from 80-90Mb/s to 20-40Mb/s

so i recommend don't encrypt your Data

[lasdem]

encryption slows down my NAS from 80-90Mb/s to 20-40Mb/s

so i recommend don't encrypt your Data

Well I am an Security Engineer and I have good educated reasons on why I encrypt my data.
Effectivly I specifically looked for a NAS system which is able to combine a RAID5 with encryption and nas4free does it perfectly.

Well I can live with my low data rates, because they are enought to watch 1080p movies from a samba share.
Everybody else has to ask himself the question if you really need and want encryption and is the performance with encryption still high enought for you usecase.

[nix4win]

encryption slows down my NAS from 80-90Mb/s to 20-40Mb/s

so i recommend don't encrypt your Data

Do you have some config information and setup info on this testing?

Thinking of doing an Atom 1.6ghz based embedded box with UFS and encryption. Ideally would want 1 drive/array encrypted for system backups/sensitive data etc and another drive/array un-encrypted for music/movies/bulk etc.

[lux]

sorry, i test this some years ago with RaidZ1 3x1Tb Disc's...

but for /me 20-40Mb/s is too slow! - my Hardware Specs doesn't change since - see my Signature

[Jtcdesigns]

If your speeds are that slow... You should connect your computer and NAS into a gigabit switch and then test the speed. Get a single 1GB file and copy it to the server and then copy it back to your computer and see what it reports. I see you are using EoP.. and depending on the brand and quality you may not get very good speeds at all. Remember.. it is wireless so don't expect anything amazing. I've seen wireless N cards give very poor performance... also, make sure your router is on wireless N ONLY otherwise you will only get legacy speeds.