Page 1 of 1
[SOLVED] Running on i386 with 4GB RAM (hint: don't)
Posted: 19 May 2015 14:41
by tuaris
I know that running ZFS on a 32-bit system is not-recommended and in fact very bad. Unfortunately I have no choice but to use a 32-bit system at the moment (at least temporarily), and indeed I am getting kernel panics when the utilization on the filesystem goes up. (at least I think they are kernel panics, the system just reboots and nothing shows in remote logging).
The Tuning guide (
https://wiki.freebsd.org/ZFSTuningGuide#i386) says I should increase the
kmem_size, but it looks like NAS4Free has already done that:
Code: Select all
# sysctl vm.kmem_size
vm.kmem_size: 1342177280
# sysctl vm.kmem_size_max
vm.kmem_size_max: 1342177280
The guide also has a sample script that will summarize kernel memory utilization, but it does not work on NAS4Free since there is no
dc command installed.
How can I verify that the server is running out of kernel memory and what else can I do to stabilize the system until I can get proper hardware?
Re: Running on i386 with 4GB RAM
Posted: 19 May 2015 20:47
by raulfg3
Re: Running on i386 with 4GB RAM
Posted: 21 May 2015 03:46
by tuaris
So far so good. I used the tuning extension, set it to a conservative 1GB left pre-fetch on. I have a 32 GB SSD as the L2ARC.
Values look okay after rebooting
Code: Select all
sysctl vm.kmem_size vfs.zfs.arc_min vfs.zfs.arc_max
vm.kmem_size: 536870912
vfs.zfs.arc_min: 134217728
vfs.zfs.arc_max: 134217728
Stressed the server as much as I could and no kernel panic yet.
Would be great if that resolves issue, even though 64-bit hardware is on the way, this would make a good backup server.
Re: Running on i386 with 4GB RAM
Posted: 21 May 2015 07:37
by tuaris
Sorry to say that tuning has not resolved the issue.
Solution: Use 64-bit hardware with ZFS because ZFS don't like 32-bit. 
Re: Running on i386 with 4GB RAM
Posted: 21 May 2015 07:44
by raulfg3
sorry to know,
Re: Running on i386 with 4GB RAM
Posted: 21 May 2015 12:46
by erico.bettoni
It should work! Try disabling prefetch and remove ssd. I used zfs in x86 for a long time, with 768mb memory.
My guess would the the l2arc. Since you're going to use the ssd on the new box this could still be a good backup box.
Re: Running on i386 with 4GB RAM
Posted: 21 May 2015 13:03
by tuaris
erico.bettoni wrote:It should work! Try disabling prefetch and remove ssd. I used zfs in x86 for a long time, with 768mb memory.
Mu guess would the the l2arc. Since you're going to use the ssd on the new box this could still be a good backup box.
Okay, set it to 1 GB and disabled the prefetch. Also pulled out the SSD. Lets see what happens.
Re: Running on i386 with 4GB RAM
Posted: 21 May 2015 19:29
by tuaris
tuaris wrote:erico.bettoni wrote:It should work! Try disabling prefetch and remove ssd. I used zfs in x86 for a long time, with 768mb memory.
Mu guess would the the l2arc. Since you're going to use the ssd on the new box this could still be a good backup box.
Okay, set it to 1 GB and disabled the prefetch. Also pulled out the SSD. Lets see what happens.
Sorry to say it's still kernel panics.
Maybe I will try with 512 MB instead of 1 GB.
One this to keep in mind is that this is acting as an NFS server, and it's quite busy. Mostly reads. The graphs section shows a constant 40 MB/s read rate with peaks up to 50 MB/s. Writes remain in the 12 to 20 MB/s range, also constant.
Also like to point out that the kernel panic are more frequent when the NFS server is really busy.
Re: Running on i386 with 4GB RAM
Posted: 21 May 2015 20:13
by erico.bettoni
Could be the memory or the mobo. Try creating on share using UFS instead of ZFS just to be sure.
Also, try memtest to see if you memory is ok.
Re: Running on i386 with 4GB RAM
Posted: 21 May 2015 20:28
by tuaris
erico.bettoni wrote:Could be the memory or the mobo. Try creating on share using UFS instead of ZFS just to be sure.
Also, try memtest to see if you memory is ok.
I used to have this hardware with UFS in NAS4Free 9.2.x (with different HBA and drives). Had issues with the drives dropping off due to me using WD Green Drives and a embedded SATA controller, but never kernel panics.
I decided to give the hardware another chance with 9.3 and put in a new HBA, ARC-1110 and drives, WD Enterprise WD2003FYPS-27Y2B R001.
Going to try UFS once I migrate this data.
Re: Running on i386 with 4GB RAM
Posted: 21 May 2015 23:10
by tuaris
Here is an interesting observation... the less memory I set in the tuner, the more often the system kernel panics.
It also kernel panics less when I leave the prefetch enabled.
I currently have it set to 2GB+ and prefetch enabled. The system has been without a kernel panic for 1 hour 25 minutes. on moderate usage.
Where as if I had it set to 512 MB, and disable prefetch, the system would kernel panic roughly within 20 minutes, on moderate usage.
Re: Running on i386 with 4GB RAM
Posted: 22 May 2015 15:20
by tuaris
So far no kernel panic
Code: Select all
9:19AM up 17:38, 2 users, load averages: 0.22, 0.28, 0.30
However, I did already migrate most of the data to a different server. So it may be due to the lighter load.
Re: Running on i386 with 4GB RAM
Posted: 12 Jul 2015 11:39
by tuaris
The server has continued to kernel panic even with a light load, just not as often as it was with high load. I have since migrated to 64-bit hardware and had no issues with that.
I took this old 32-bit server, setup hardware RAID, and reformatted it to use UFS. It's now a pretty good backups storage host.
Re: [SOLVED] Running on i386 with 4GB RAM (hint: don't)
Posted: 12 Jul 2015 15:48
by raulfg3
sorry to know.