*New 11.4 series Release:
2020-07-03: XigmaNAS 11.4.0.4.7633 - released!

*New 12.1 series Release:
2020-04-17: XigmaNAS 12.1.0.4.7542 - released


We really need "Your" help on XigmaNAS https://translations.launchpad.net/xigmanas translations. Please help today!

Producing and hosting XigmaNAS costs money. Please consider donating for our project so that we can continue to offer you the best.
We need your support! eg: PAYPAL

Link aggregation and vlans

NIC, network controllers, compatibility questions, WOL, wake on lan
Forum rules
Set-Up GuideFAQsForum Rules
Post Reply
nadir.latif
NewUser
NewUser
Posts: 3
Joined: 15 Apr 2014 22:38
Location: Karachi, Pakistan
Contact:
Status: Offline

Link aggregation and vlans

#1

Post by nadir.latif »

Hello,

I am trying to setup setup vlans and laggs on our nas4free server. I did not find any useful documentation on the nas4free wiki. What steps do we need to follow in order to setup vlans and laggs on our server. we have 2 physical nics that we would like to have bonded. We would like the bonded interface to have a single ip on a vlan.

Thanks,

Nadir Latif

User avatar
crowi
Forum Moderator
Forum Moderator
Posts: 1182
Joined: 21 Feb 2013 16:18
Location: Munich, Germany
Status: Offline

Re: Link aggregation and vlans

#2

Post by crowi »

Configuring LAGG I can help, I just tried this little howto from another user:

backup your configuration
go to Network -> Interface Management

in management create
- Interface OPT1 (NIC2)
in LAGG create
- Interface lagg0 with LACP and NIC1 & 2

In Management
- set the order of Interfaces
LAN = lagg0
OPT1 = igb0 (NIC1)
OPT2 = igb1 (NIC2)

reboot

from the console menu start the shell and adjust the rc.conf (folder /etc)

so, go to /etc/

use nano to edit rc.conf:

Code: Select all

nano rc.conf
delete the two lines showing the ipv4: (in my case it was)
ipv4_addrs_igb0="192.168.178.50/24"
ipv4_addrs_res0="192.168.178.50/24"

save file and exit nano (ctrl-x then y to save)

reboot and LAGG is activated

Good luck,
Crowi
NAS 1: Milchkuh: Asrock C2550D4I, Intel Avoton C2550 Quad-Core, 16GB DDR3 ECC, 5x3TB WD Red RaidZ1 +60 GB SSD for ZIL/L2ARC, APC-Back UPS 350 CS, NAS4Free 11.0.0.4.3460 embedded
NAS 2: Backup: HP N54L, 8 GB ECC RAM, 4x4 TB WD Red, RaidZ1, NAS4Free 11.0.0.4.3460 embedded
NAS 3: Office: HP N54L, 8 GB ECC RAM, 2x3 TB WD Red, ZFS Mirror, APC-Back UPS 350 CS NAS4Free 11.0.0.4.3460 embedded

Northwinds
Starter
Starter
Posts: 21
Joined: 03 Sep 2014 01:51
Status: Offline

Re: Link aggregation and vlans

#3

Post by Northwinds »

Hi there
I just decided to go from 9.2 to 10.0.0.2 today and am having real problems setting up my lagg again.
Would you be able to advise me?
Many thanks.
11.0.0.4.4067
x64-embedded on Intel(R) Core(TM)2 CPU 6600 @ 2.40GHz
Supermicro X7SB4/E with 2x Single-port Gigabit PCI-Express Ethernet Controllers
8GB RAM
6 x Seagate ST4000VN000 4TB NAS HDD

Northwinds
Starter
Starter
Posts: 21
Joined: 03 Sep 2014 01:51
Status: Offline

Re: Link aggregation and vlans

#4

Post by Northwinds »

Hi again
After much mucking around I have reverted to 9.3 as 10 has a setting - lacp_strict_mode - that is mucking up my lagg.
Did you come across this problem in 10?
Thanks.
11.0.0.4.4067
x64-embedded on Intel(R) Core(TM)2 CPU 6600 @ 2.40GHz
Supermicro X7SB4/E with 2x Single-port Gigabit PCI-Express Ethernet Controllers
8GB RAM
6 x Seagate ST4000VN000 4TB NAS HDD

jediknightjason
NewUser
NewUser
Posts: 7
Joined: 05 Nov 2015 08:48
Status: Offline

Re: Link aggregation and vlans

#5

Post by jediknightjason »

help i.ve followed your instructions up to editing the rc.conf file but when i edit such file the lines you mention do not exist....at this point my system has set 192.168.1.250 as the lan network ip v4 addresss and the lagg0 as the network interface. and i have bce0 as OPT1 and bce1 as OPT2 but i have no access via web gui and the nas box doesnt even show up in my windows network. only way it works is if i turn off the lagg and just use one port...but that defeats the purpose of the lagg....i have 2 dual port gigabit nics in this machaine and a gigabit lan on the motherboard i want to utilize.

User avatar
Parkcomm
Advanced User
Advanced User
Posts: 388
Joined: 21 Sep 2012 12:58
Location: Australia
Status: Offline

Re: Link aggregation and vlans

#6

Post by Parkcomm »

Hey Jason I don't have the ports to test this now - but are you trying to use VLANs as the title of the thread suggests.
https://plone.lucidsolutions.co.nz/stor ... g-and-vlan

If not I'd suggest you start a new thread - also look here
https://www.freebsd.org/doc/handbook/ne ... ation.html - in particular 30.7.1, it even includes the setup for the cisco router.

Also just a note - I do not know why Crowl has suggested editing the rc.conf, and as I said I cant test it. There are two important stesp Crowl does not mention:
  • You need to have your switch set up matching the LAGG method you use (e.g. LACP), and
  • You need to to configure the ipV4 for the LACP interface, in particular once lagg0 is set up as the LAN interface go to Network|LAN Management and configure it to use DHCP or set up a static ipv4 address
NAS4Free Embedded 10.2.0.2 - Prester (revision 2003), HP N40L Microserver (AMD Turion) with modified BIOS, ZFS Mirror 4 x WD Red + L2ARC 128M Apple SSD, 10G ECC Ram, Intel 1G CT NIC + inbuilt broadcom

User avatar
daoyama
Developer
Developer
Posts: 422
Joined: 25 Aug 2012 09:28
Location: Japan
Status: Offline

Re: Link aggregation and vlans

#7

Post by daoyama »

Parkcomm wrote: Also just a note - I do not know why Crowl has suggested editing the rc.conf, and as I said I cant test it.
I think you don't need to edit files if you don't use lagg member as OPT1, OPT2,...
Using lagg0 as LAN only is enough. (not tested)
NAS4Free 10.2.0.2.2115 (x64-embedded), 10.2.0.2.2258 (arm), 10.2.0.2.2258(dom0)
GIGABYTE 5YASV-RH, Celeron E3400 (Dual 2.6GHz), ECC 8GB, Intel ET/CT/82566DM (on-board), ZFS mirror (2TBx2)
ASRock E350M1/USB3, 16GB, Realtek 8111E (on-board), ZFS mirror (2TBx2)
MSI MS-9666, Core i7-860(Quad 2.8GHz/HT), 32GB, Mellanox ConnectX-2 EN/Intel 82578DM (on-board), ZFS mirror (3TBx2+L2ARC/ZIL:SSD128GB)
Develop/test environment:
VirtualBox 512MB VM, ESXi 512MB-8GB VM, Raspberry Pi, Pi2, ODROID-C1

User avatar
lindsay
Forum Moderator
Forum Moderator
Posts: 299
Joined: 23 Jun 2012 09:59
Location: Steinkjer,Norway
Status: Offline

Re: Link aggregation and vlans

#8

Post by lindsay »

For me it is working without edit a thing with quad intel card connected to an cisco small business switch
Protected by smoothiebox Red,Green,
Purple,Orange Zones/VLAN`s
Powered by AMD A10-6700T


XigmaNAS Box-1 11.2.0.4 - Omnius (revision 6625)
Platform : x64-embedded on 2X Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz
Motherboard: ASUS Z10PA-D8, 2xSocket-2011-3
SATA Controllers : 1X Avago Technologies (LSI) SAS2008 and 1x Avago Technologies (LSI) SAS2308
Pool 1 (Media-Pool) 8X4TB in raidz2
Pool 2 (Media-Pool-2) 4X2TB in raidz2 and 2X2TB in mirror mirror and 2X3TB in mirror
Pool 3 (Media-Pool-3) 2X2TB in mirror and 2X4TB in mirror and 2X1TB in mirror

jediknightjason
NewUser
NewUser
Posts: 7
Joined: 05 Nov 2015 08:48
Status: Offline

Re: Link aggregation and vlans

#9

Post by jediknightjason »

ok so i was configureing it right i just need a compatible switch then?
i have the server plugged into a little 5 port netgear gigabit switch at the moment but i plan to plug it permanently into my TP-LINK 16-Port gigabit Ethernet Switch TL-SG1016D Version 4.0
will it work with that switch or do i need to get something else?

My wired network consists of a cisco wrvs4400n small buisness router and a 16 port gigabit switch and 2 little 5 port netgear gigabit switches, one on my workbench and one in my bedroom...i have 5 pc's on this network and 3 small dlink nas boxes of which i hope to replace with this 12 terabye server i.m building..i also have a line from the big switch run downstairs with a verizon wireless router/extender attacted to the end serving as a hotspot for all my downstairs neighbors phones and tablets!.
jason

User avatar
Parkcomm
Advanced User
Advanced User
Posts: 388
Joined: 21 Sep 2012 12:58
Location: Australia
Status: Offline

Re: Link aggregation and vlans

#10

Post by Parkcomm »

Both ends of a Lagg need to support the prototcol, so it needs to be a managed switch. HP Procurve 1810 / Cisco SG300-10 / Netgear GSM712,

look for lagg or link aggregation in the specs

FYI - with LACP each end device goes through only one of the interfaces (actually each mac address) so you will only get 1G to any one endpoint, but you multiplex several devices through the switch to fill the 2G lagg link.
NAS4Free Embedded 10.2.0.2 - Prester (revision 2003), HP N40L Microserver (AMD Turion) with modified BIOS, ZFS Mirror 4 x WD Red + L2ARC 128M Apple SSD, 10G ECC Ram, Intel 1G CT NIC + inbuilt broadcom

User avatar
lindsay
Forum Moderator
Forum Moderator
Posts: 299
Joined: 23 Jun 2012 09:59
Location: Steinkjer,Norway
Status: Offline

Re: Link aggregation and vlans

#11

Post by lindsay »

As you can see here on my cisco switch

Image

I have also set the interfaces to Trunk

Image
Last edited by lindsay on 26 Jan 2016 12:44, edited 1 time in total.
Protected by smoothiebox Red,Green,
Purple,Orange Zones/VLAN`s
Powered by AMD A10-6700T


XigmaNAS Box-1 11.2.0.4 - Omnius (revision 6625)
Platform : x64-embedded on 2X Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz
Motherboard: ASUS Z10PA-D8, 2xSocket-2011-3
SATA Controllers : 1X Avago Technologies (LSI) SAS2008 and 1x Avago Technologies (LSI) SAS2308
Pool 1 (Media-Pool) 8X4TB in raidz2
Pool 2 (Media-Pool-2) 4X2TB in raidz2 and 2X2TB in mirror mirror and 2X3TB in mirror
Pool 3 (Media-Pool-3) 2X2TB in mirror and 2X4TB in mirror and 2X1TB in mirror

User avatar
daoyama
Developer
Developer
Posts: 422
Joined: 25 Aug 2012 09:28
Location: Japan
Status: Offline

Re: Link aggregation and vlans

#12

Post by daoyama »

lindsay wrote:As you can see here on my cisco switch
Thank you providing image. Here is one of my switches image.
Dell Power Connect 5324/24 port switch.
20151107B.png
Note: I'm going 10GbE, it's only used for old machine which does not have 10GbE card for multiple ESXi servers for iSCSI link.

The lagg shows like this:

Code: Select all

# ifconfig lagg0
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
        options=403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO>
        ether 00:1b:21:49:ec:5c
        inet 172.16.0.98 netmask 0xffff0000 broadcast 172.16.255.255
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect
        status: active
        laggproto lacp lagghash l2,l3,l4
        laggport: igb0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
        laggport: igb1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
ESXi side:
20151107C.png
You do not have the required permissions to view the files attached to this post.
NAS4Free 10.2.0.2.2115 (x64-embedded), 10.2.0.2.2258 (arm), 10.2.0.2.2258(dom0)
GIGABYTE 5YASV-RH, Celeron E3400 (Dual 2.6GHz), ECC 8GB, Intel ET/CT/82566DM (on-board), ZFS mirror (2TBx2)
ASRock E350M1/USB3, 16GB, Realtek 8111E (on-board), ZFS mirror (2TBx2)
MSI MS-9666, Core i7-860(Quad 2.8GHz/HT), 32GB, Mellanox ConnectX-2 EN/Intel 82578DM (on-board), ZFS mirror (3TBx2+L2ARC/ZIL:SSD128GB)
Develop/test environment:
VirtualBox 512MB VM, ESXi 512MB-8GB VM, Raspberry Pi, Pi2, ODROID-C1

User avatar
lindsay
Forum Moderator
Forum Moderator
Posts: 299
Joined: 23 Jun 2012 09:59
Location: Steinkjer,Norway
Status: Offline

Re: Link aggregation and vlans

#13

Post by lindsay »

@daoyama
One tiny problem as i can see it is when using Kodi it starts to buffer content (movies).
Filetransfer though works like a charm, it do not max out at 125MB/s per interface but 110 in average on each interface

Edit: it do not buffering at all on another client, it has to be the client wich cause it ;)
Edit2: It was also connected to an 100Mbps switch :D
Have not split the clients in the livingroom
Protected by smoothiebox Red,Green,
Purple,Orange Zones/VLAN`s
Powered by AMD A10-6700T


XigmaNAS Box-1 11.2.0.4 - Omnius (revision 6625)
Platform : x64-embedded on 2X Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz
Motherboard: ASUS Z10PA-D8, 2xSocket-2011-3
SATA Controllers : 1X Avago Technologies (LSI) SAS2008 and 1x Avago Technologies (LSI) SAS2308
Pool 1 (Media-Pool) 8X4TB in raidz2
Pool 2 (Media-Pool-2) 4X2TB in raidz2 and 2X2TB in mirror mirror and 2X3TB in mirror
Pool 3 (Media-Pool-3) 2X2TB in mirror and 2X4TB in mirror and 2X1TB in mirror

jediknightjason
NewUser
NewUser
Posts: 7
Joined: 05 Nov 2015 08:48
Status: Offline

Re: Link aggregation and vlans

#14

Post by jediknightjason »

thankyou guys for all the info....i've been thinking about upgrading my network to 10gb ethernet and i think that might be the better way to go. thanks again!

User avatar
Parkcomm
Advanced User
Advanced User
Posts: 388
Joined: 21 Sep 2012 12:58
Location: Australia
Status: Offline

Re: Link aggregation and vlans

#15

Post by Parkcomm »

All - I've just been trying to set up LACP on two interfaces, an intel (em0) and a broadcom (bge0). The broadcom does not support jumbo frames.

If I have both interfaces configured, cables plugged in and set the lagg for DHCP, I get cyclic reboots. I think the MTU might be the issue, but I would expect it to config the lagg with the size of the first interface and the not add bge0, not a reboot.

Anyone seen anything similar?
NAS4Free Embedded 10.2.0.2 - Prester (revision 2003), HP N40L Microserver (AMD Turion) with modified BIOS, ZFS Mirror 4 x WD Red + L2ARC 128M Apple SSD, 10G ECC Ram, Intel 1G CT NIC + inbuilt broadcom

User avatar
Parkcomm
Advanced User
Advanced User
Posts: 388
Joined: 21 Sep 2012 12:58
Location: Australia
Status: Offline

Re: Link aggregation and vlans

#16

Post by Parkcomm »

Well - I think I've found the problem, and it in fact stops the work around.

The /etc/rc.conf created by nas4free has the following:

Code: Select all

hostname="mightymouse.lan"
ifconfig_lagg0="syncdhcp media 1000baseTX mediaopt full-duplex laggproto lacp laggport em0 laggport bge0"
cloned_interfaces="lagg0"
ifconfig_em0="up mtu 1500"
ifconfig_bge0="up mtu 1500"
whereas the freebsd handbook says

Code: Select all

ifconfig_fxp0="up"
ifconfig_fxp1="up"
cloned_interfaces="lagg0"
ifconfig_lagg0="laggproto failover laggport fxp0 laggport fxp1 10.0.0.15/24"
the laggports have to be up before you create the lagg interface

this causes the following errors

Code: Select all

can't re-use a leaf (tx_test)!
can't re-use a leaf (rx_test)!
can't re-use a leaf (lacp_strict_mode)!
bge0: interface stopped DISTRIBUTING, possible flapping
and switch sets its trunk port to bge0 to DOWN

I think this is the problem here
Northwinds wrote:Hi again
After much mucking around I have reverted to 9.3 as 10 has a setting - lacp_strict_mode - that is mucking up my lagg.
Did you come across this problem in 10?
Thanks.
anyone else seen this?

I'm also suspicious lagg doesn't play well with vlans, but the problem above got in the way of diagnosing the vlan issue.
Last edited by Parkcomm on 08 Nov 2015 04:53, edited 1 time in total.
NAS4Free Embedded 10.2.0.2 - Prester (revision 2003), HP N40L Microserver (AMD Turion) with modified BIOS, ZFS Mirror 4 x WD Red + L2ARC 128M Apple SSD, 10G ECC Ram, Intel 1G CT NIC + inbuilt broadcom

User avatar
daoyama
Developer
Developer
Posts: 422
Joined: 25 Aug 2012 09:28
Location: Japan
Status: Offline

Re: Link aggregation and vlans

#17

Post by daoyama »

Parkcomm wrote: The /etc/rc.conf created by nas4free has the following:

Code: Select all

hostname="mightymouse.lan"
ifconfig_lagg0="syncdhcp media 1000baseTX mediaopt full-duplex laggproto lacp laggport em0 laggport bge0"
cloned_interfaces="lagg0"
ifconfig_em0="up mtu 1500"
ifconfig_bge0="up mtu 1500"
whereas the freebsd handbook says

Code: Select all

ifconfig_fxp0="up"
ifconfig_fxp1="up"
cloned_interfaces="lagg0"
ifconfig_lagg0="laggproto failover laggport fxp0 laggport fxp1 10.0.0.15/24"
the laggports have to be up before you create the lagg interface
If you say about line order of rc.conf, it's nothing affect running commands.
rc.conf is definition of variables used by rc.d scripts.
NAS4Free 10.2.0.2.2115 (x64-embedded), 10.2.0.2.2258 (arm), 10.2.0.2.2258(dom0)
GIGABYTE 5YASV-RH, Celeron E3400 (Dual 2.6GHz), ECC 8GB, Intel ET/CT/82566DM (on-board), ZFS mirror (2TBx2)
ASRock E350M1/USB3, 16GB, Realtek 8111E (on-board), ZFS mirror (2TBx2)
MSI MS-9666, Core i7-860(Quad 2.8GHz/HT), 32GB, Mellanox ConnectX-2 EN/Intel 82578DM (on-board), ZFS mirror (3TBx2+L2ARC/ZIL:SSD128GB)
Develop/test environment:
VirtualBox 512MB VM, ESXi 512MB-8GB VM, Raspberry Pi, Pi2, ODROID-C1

User avatar
Parkcomm
Advanced User
Advanced User
Posts: 388
Joined: 21 Sep 2012 12:58
Location: Australia
Status: Offline

Re: Link aggregation and vlans

#18

Post by Parkcomm »

So the order does not matter at all?

I can recreate the problem by entering the commands in the same order as rc.conf in the terminal. What I can say is it looks as if the bge0 has not come up before lagg0 and when I do that in the terminal I see a similar problem with bge0.
NAS4Free Embedded 10.2.0.2 - Prester (revision 2003), HP N40L Microserver (AMD Turion) with modified BIOS, ZFS Mirror 4 x WD Red + L2ARC 128M Apple SSD, 10G ECC Ram, Intel 1G CT NIC + inbuilt broadcom

Post Reply

Return to “LAN , Network controllers”