*New 12.1 series Release:
2019-11-08: XigmaNAS 12.1.0.4.7091 - released!

*New 11.3 series Release:
2019-10-19: XigmaNAS 11.3.0.4.7014 - released


We really need "Your" help on XigmaNAS https://translations.launchpad.net/xigmanas translations. Please help today!

Producing and hosting XigmaNAS costs money. Please consider donating for our project so that we can continue to offer you the best.
We need your support! eg: PAYPAL

Web gui reports incorrect zfs disk space usage

Forum rules
Set-Up GuideFAQsForum Rules
Post Reply
luguber
NewUser
NewUser
Posts: 2
Joined: 04 Jul 2012 16:11
Status: Offline

Web gui reports incorrect zfs disk space usage

#1

Post by luguber » 01 Nov 2013 23:43

I'm currently running latest version of nas4free (9.1.0.1 - Sandstorm (revision 847)). But I noticed this issue were present in older builds as well.
I have recently expanded zpool capacity by replacing disks in one of the vdevs. The gui didn't report correctly before that either, but I have no evidence for that.

In nas4free webgui it now show following disk space usage;

Code: Select all

Disks|ZFS|Pools|Management
Name	   Size	  Used	  Free	 Capacity	Dedup	Health	AltRoot	
zpool1 	38.1T 	13.5T 	11.4T 	53% 	1.01x 	ONLINE 	- 
If I ssh into the box and run "zpool list", it shows the correct disk space usage;

Code: Select all

nas4free: ~ # zpool list
NAME     SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
zpool1  38.1T  20.3T  17.7T    53%  1.01x  ONLINE  -
Anyone else have similar issue, and is there anything I can do to make the webgui report the actual disk usage?

kenZ71
Advanced User
Advanced User
Posts: 403
Joined: 27 Jun 2012 20:18
Location: Northeast, USA
Status: Offline

Re: Web gui reports incorrect zfs disk space usage

#2

Post by kenZ71 » 04 Nov 2013 22:17

I have seen similar issues on my system but I don't pay much attention to it.

I am always curious what people have stored in large capacity systems 13Tb is a lot of stuff!!
11.2-RELEASE-p3 | ZFS Mirror - 2 x 8TB WD Red | 28GB ECC Ram
HP ML10v2 x64-embedded on Intel(R) Core(TM) i3-4150 CPU @ 3.50GHz

Extra memory so I can host a couple VMs
1) Unifi Controller on Ubuntu
2) Librenms on Ubuntu

luguber
NewUser
NewUser
Posts: 2
Joined: 04 Jul 2012 16:11
Status: Offline

Re: Web gui reports incorrect zfs disk space usage

#3

Post by luguber » 08 Nov 2013 19:00

Sure, it's not a big deal, just thought I'd mention it to see if it was only my setup or if other had the same issue.

Whether 13TB is much or not depends what you store on it. If you store raw video recordings then it eat up that space rather fast.

User avatar
b0ssman
Forum Moderator
Forum Moderator
Posts: 2458
Joined: 14 Feb 2013 08:34
Location: Munich, Germany
Status: Offline

Re: Web gui reports incorrect zfs disk space usage

#4

Post by b0ssman » 08 Nov 2013 20:44

zpool includes parity
web gui does not
Nas4Free 11.1.0.4.4517. Supermicro X10SLL-F, 16gb ECC, i3 4130, IBM M1015 with IT firmware. 4x 3tb WD Red, 4x 2TB Samsung F4, both GEOM AES 256 encrypted.

serverguy
Starter
Starter
Posts: 22
Joined: 25 Oct 2014 13:18
Contact:
Status: Offline

Re: Web gui reports incorrect zfs disk space usage

#5

Post by serverguy » 01 Dec 2014 02:18

Brand new installation. And NO files NO folders on the BRAND NEW installation. 8 HDs with 3TB (manufacturer claimed) space on them.

Created ZFS1 (1 parity) from these 8 drives. It works! :P Initial space (counting absolutely everything claimed) should be 24 TB.

Here is the report from the GUI under "System Information":
N4FRaid
0% of 21.8TB
Total: 21.8T | Used: 176K | Free: 18.6T | State: ONLINE

Already lost 2 TB (maybe that's MB to MiB conversion - don't know). Then lost another 3.2 TB (maybe that's the parity). SO only 18.6 left from the original (per manufacturer) of 24. Already down to 77.5 % of installed size and nothing loaded yet. Even using the number from the table, below, the total comes to 22.892 TB. The displayed Total is still 1 TB smaller than that.

Is this the way it should be? Seems like an awful lot of lost bytes before anything has been loaded. :roll: Where is all my claimed storage going???

Mike M.

Full info, below:

Device Device model Size Serial number Controller Controller model Standby time File system Status
ada0 WDC WD30EZRX-00MMMB0 2861589MB WD-WCAWZ1422294 ahcich0 ATI IXP700 AHCI SATA controller 5 ZFS storage pool device ONLINE Edit disk Delete disk
ada1 WDC WD30EZRX-00MMMB0 2861589MB WD-WCAWZ2321386 ahcich1 ATI IXP700 AHCI SATA controller 5 ZFS storage pool device ONLINE Edit disk Delete disk
ada2 WDC WD30EZRX-00MMMB0 2861589MB WD-WCAWZ2718829 ahcich2 ATI IXP700 AHCI SATA controller 5 ZFS storage pool device ONLINE Edit disk Delete disk
ada3 WDC WD30EZRX-00MMMB0 2861589MB WD-WMAWZ0339153 ahcich3 ATI IXP700 AHCI SATA controller 5 ZFS storage pool device ONLINE Edit disk Delete disk
ada4 WDC WD30EZRX-00MMMB0 2861589MB WD-WCAWZ0957176 ata4 Promise PDC20319 SATA150 controller 5 ZFS storage pool device ONLINE Edit disk Delete disk
ada5 WDC WD30EZRX-00DC0B0 2861589MB WD-WMC1T0285134 ata5 Promise PDC20319 SATA150 controller 5 ZFS storage pool device ONLINE Edit disk Delete disk
ada6 WDC WD30EZRX-00MMMB0 2861589MB WD-WMAWZ0162162 ata6 Promise PDC20319 SATA150 controller 5 ZFS storage pool device ONLINE Edit disk Delete disk
ada7 WDC WD30EZRX-00MMMB0 2861589MB WD-WCAWZ2460314 ata7 Promise PDC20319 SATA150 controller 5 ZFS storage pool device ONLINE Edit disk Delete disk
da0 PMAP 3824MB n/a umass-sim0 AMD SB7x0/SB8x0/SB9x0 USB 2.0 controller Always on UFS ONLINE Edit disk Delete disk

User avatar
daoyama
Developer
Developer
Posts: 423
Joined: 25 Aug 2012 09:28
Location: Japan
Status: Offline

Re: Web gui reports incorrect zfs disk space usage

#6

Post by daoyama » 01 Dec 2014 07:19

>WD30EZRX-00MMMB0 2861589MB
At least your total is correct.

(2861589MB=2.73TiB) x 8= 21.84TiB

You can see space from cli by:
# zpool list
# zfs list <poolname>

Probably, GUI's Free space is NOT including meta data like filename timestamp checksum parity etc.
So you can put the file of 18.6TB(+ parity in 2.73TiB) on the pool.
I think this size on GUI is very confusing.
Take size from zpool list is better.
NAS4Free 10.2.0.2.2115 (x64-embedded), 10.2.0.2.2258 (arm), 10.2.0.2.2258(dom0)
GIGABYTE 5YASV-RH, Celeron E3400 (Dual 2.6GHz), ECC 8GB, Intel ET/CT/82566DM (on-board), ZFS mirror (2TBx2)
ASRock E350M1/USB3, 16GB, Realtek 8111E (on-board), ZFS mirror (2TBx2)
MSI MS-9666, Core i7-860(Quad 2.8GHz/HT), 32GB, Mellanox ConnectX-2 EN/Intel 82578DM (on-board), ZFS mirror (3TBx2+L2ARC/ZIL:SSD128GB)
Develop/test environment:
VirtualBox 512MB VM, ESXi 512MB-8GB VM, Raspberry Pi, Pi2, ODROID-C1

User avatar
crowi
Forum Moderator
Forum Moderator
Posts: 1189
Joined: 21 Feb 2013 16:18
Location: Munich, Germany
Status: Offline

Re: Web gui reports incorrect zfs disk space usage

#7

Post by crowi » 01 Dec 2014 11:49

Is this the way it should be? Seems like an awful lot of lost bytes before anything has been loaded.
Yes, 2,73 TiB x 8 = 21,84 Total
21,84 TiB - 2,73 TiB (Parity for RAIDZ) = 19,11 TiB minus some overhead for ZFS gives you these 18,6 TiB.

Anyway 8 drives in one RAIDZ1 is not the best idea:
http://www.zdnet.com/blog/storage/why-r ... n-2009/162
you should go for Z2 (the ZFS equivalent of RAID6)

also a good reading:
http://forums.freebsd.org/threads/4k-bl ... t-21271611
NAS 1: Milchkuh: Asrock C2550D4I, Intel Avoton C2550 Quad-Core, 16GB DDR3 ECC, 5x3TB WD Red RaidZ1 +60 GB SSD for ZIL/L2ARC, APC-Back UPS 350 CS, NAS4Free 11.0.0.4.3460 embedded
NAS 2: Backup: HP N54L, 8 GB ECC RAM, 4x4 TB WD Red, RaidZ1, NAS4Free 11.0.0.4.3460 embedded
NAS 3: Office: HP N54L, 8 GB ECC RAM, 2x3 TB WD Red, ZFS Mirror, APC-Back UPS 350 CS NAS4Free 11.0.0.4.3460 embedded

moggy
NewUser
NewUser
Posts: 2
Joined: 21 Jun 2017 07:22
Status: Offline

Re: Web gui reports incorrect zfs disk space usage

#8

Post by moggy » 03 Feb 2020 04:19

Personally, it's not the way it should be IMHO.
The home page gui should report not only total disk space (as used in any parity scheme) but should also report an "estimated" value of actual file usage storage space. The lack of the latter has caused me confusion over the years, and I shouldn't have to go looking at confusing diagnostic pages.

I simply estimate 2 disk spaces lost when using z2 and 1 disk space lost when using z1, etc.
e.g. 4x2TB gives me under 6TB using z2 zfs.

I am not an IT expert so the lack of guis to do things is hard for me. So much trouble getting Plex to work (again) on nas4free.
I now only use xigmanas/nas4f servers as backups for my main unraid server (70TB).
That still means at least 4 xigmanas/nas4free pcs to maintain. -:)
Moggy
The Dunce from Oz

User avatar
ms49434
Developer
Developer
Posts: 805
Joined: 03 Sep 2015 18:49
Location: Neuenkirchen-Vörden, Germany - GMT+1
Contact:
Status: Online

Re: Web gui reports incorrect zfs disk space usage

#9

Post by ms49434 » 03 Feb 2020 09:48

moggy wrote:
03 Feb 2020 04:19
Personally, it's not the way it should be IMHO.
The home page gui should report not only total disk space (as used in any parity scheme) but should also report an "estimated" value of actual file usage storage space. The lack of the latter has caused me confusion over the years, and I shouldn't have to go looking at confusing diagnostic pages.

I simply estimate 2 disk spaces lost when using z2 and 1 disk space lost when using z1, etc.
e.g. 4x2TB gives me under 6TB using z2 zfs.

I am not an IT expert so the lack of guis to do things is hard for me. So much trouble getting Plex to work (again) on nas4free.
I now only use xigmanas/nas4f servers as backups for my main unraid server (70TB).
That still means at least 4 xigmanas/nas4free pcs to maintain. -:)
Disk Space:
Goto Disks > ZFS > Settings and enable "Display Used/Avail information from the filesystem instead of the Alloc/Free information from the pool."
Used/Avail lists storage information after all redundancy is taken into account but is impacted by compression, deduplication and quotas. Alloc/Free lists the raw storage information of a pool.

Plex:
Plex is a community driven project and is available as an Add-On and can be easily installed via OBI: Plex Media Server
1) XigmaNAS 12.1.0.4 amd64-embedded on a Dell T20 running in a VM on ESXi 6.7U3, 22GB out of 32GB ECC RAM, LSI 9300-8i IT mode in passthrough mode. Pool 1: 2x HGST 10TB, mirrored, SLOG: Samsung 850 Pro, L2ARC: Samsung 850 Pro, Pool 2: 1x Samsung 860 EVO 1TB , services: Samba AD, CIFS/SMB, ftp, ctld, rsync, syncthing, zfs snapshots.
2) XigmaNAS 12.1.0.4 amd64-embedded on a Dell T20 running in a VM on ESXi 6.7U3, 8GB out of 32GB ECC RAM, IBM M1215 crossflashed, IT mode, passthrough mode, 2x HGST 10TB , services: rsync.

moggy
NewUser
NewUser
Posts: 2
Joined: 21 Jun 2017 07:22
Status: Offline

Re: Web gui reports incorrect zfs disk space usage

#10

Post by moggy » 07 Feb 2020 04:15

Thanks for the info.
Unfortunately, my nas4free servers don't have that option, only the xigmanas ones (as I discovered). Also, I have to remember what version of disk space is shown, confusing until I update all to xigmanas. Upgrading is always fraught with danger. Even updates in nas4free caused me much much trouble. I have 2x xigma and 2x nas4free.
Also I meant showing "real" space on the home page as well as what's there. A graph is easy to immediately understand.
However, it's something and I'm grateful. Thanks.
Re. Plex.... Yes, I love it. Unfortunately my plex doesn't seem to want to re-start in my nas4free pcs. It says re-started but is actually stopped with a red stopped word. I've tried re-installing with all the so-called deletions required but no joy. I can only think that I'll need to update them to xigmanas to get plex to work. A story for another day.
Moggy
The Dunce from Oz

Post Reply

Return to “ZFS (only!)”