XigmaNAS

I have two N4F boxes built and running. One has 8 3TB drives on it. The other has 4 4TB drives on it. The 8 and 4 drives groupings are each a single ZFS partition. So that's a total of 2, of course. And they are SO SLOW!!! By comparison, the units they are supposed to be replacing are Synology NAS units. They are quite fast listing directories while the ZFS on N4F is at least 10 times slower. Possibly far slower than that. I have not timed it exactly but from what I have counted myself, there is a minimum of 10 to 1 speed difference.

They are both running on ASRock 960GM/U3S3 FX with American Megatrends Inc. version: P1.40 10/02/2013 pushed along with a x64-embedded on AMD Athlon(tm) II X2 250 Processor. One has 5GB of memory, the other (the one I am watching grind) has 8GB of memory. It is 27% used. No thrashing. And they don't run CPU bound. In fact, I have never seen them more than 50% busy. Right now, while I am getting a size of the entire ZFS partition on one of them and the CPU is only 8% busy. So it is not a CPU problem.

The connection is SMB to both N4S and Synology from the same Windows 7 box on the same 1 Gb connection. It is hardly used for this process.

I started the file sizing process before I started writing this and it is only about 1/3 through. On the Synology it would be through by now. In fact, during the time I wrote this sentence, the Synology already started and finished. N4F is still chugging along about half way done. This is HORRIBLE!!

Why is this so slow? Is there any optimization I should do? Anything to look at? WHY is it running SO SLOW?!?!?!

Then there's the size problem. The Synology is using 9.4 TB. The EXACT same data (mirrored with Robocopy) is on both. The properties windows shows the same number of files and they are almost the same size. But the ZFS info shows 13 TB used to hold this 9.4 TB of data. That's 38% larger! Synology shows 9730 GB used and N4F/ZFS shows 13000 GB used. This is a HUGE difference. No, there is nothing else stored to account for this. Recycle is empty. No web site or anything else. They are both running 4 ea. 4TB drives with 1 disk redundancy. On Synology, it is called SHR and feels a lot like ZFS. The drives don't have to be the same size. It will use what is there and autoexpands when a larger drive is added.

So...

Double mystery here. Would appreciate any help with this dual problematic situation. Slow response and far larger data size on ZFS.

Mike Morrow

Have you tried the ZFS Kern Tune? Search the forum, it can help speed up ZFS quite a bit.

As for the size reporting what do you see from linux / Windows clients?

But the ZFS info shows 13 TB used to hold this 9.4 TB of data.

Where exactly do you get this data from? Can you post a screenshot?

Then you should not use this Asrock 960GM/U3S3 FX mainboard for ZFS

Supports DDR3 1866(OC)/1600(OC)/1333/1066/800 non-ECC, un-buffered memory

as ECC memory is HIGHLY needed for ZFS:
https://forums.freenas.org/index.php?th ... zfs.15449/

And finally:

They are quite fast listing directories while the ZFS on N4F is at least 10 times slower. Possibly far slower than that. I have not timed it exactly but from what I have counted myself, there is a minimum of 10 to 1 speed difference.

This is dependent on how you have configured SMB.
Use sendfile -> turn off
EA support -> turn off
Store DOS attributes -> turn off
Asynchronous I/O (AIO) -> turn on

On SMB shares:
ZFS ACL -> turn off

Did those changes, restarted SMB. No improvement. Still has huge waits between files.

Haven't changed the motherboard of memory yet.

"Free" just got a whole lot less free!

Mike

I documented where I got the data. I guess I could go around to the three systems and capture screen shots that will show exactly what I typed but... will that do any good? I typed exactly what I saw. Due to space limitations, I had to split the one partition into two and I have to get the new two loaded up before I can get the new numbers.

Can we just address the numbers presented? They are true and the sources are documented.

Mike

Since you're using SMB, you could try the config changes I mention in this post here: viewtopic.php?p=52332#p52332

It made a big difference for me when accessing the directory structure (ie listing files in directory etc). Remember to reboot after saving the changes

Just a tip, dont use robocopy, use zfs send and recive to copy or sync zfs.