This is the old XigmaNAS forum in read only mode,
it will taken offline by the end of march 2021!


I like to aks Users and Admins to rewrite/take over important post from here into the new fresh main forum!
Its not possible for us to export from here and import it to the main forum!

listen queue overflow error

iSCSI over TCP/IP.
Forum rules
• Set-Up Guide • FAQs • Forum Rules
Post Reply
cjohnson
NewUser
NewUser
Posts: 4
Joined: 04 Jun 2014 07:03
Status: Offline

listen queue overflow error

Post by cjohnson »

Jun 18 02:00:15 istgt[4651]: istgt_iscsi.c:5414:sender: ***ERROR*** iscsi_write_pdu() failed on iqn.2014-06.net.home.dvrstorage:dvrstoragedisk1target,t,0x0001(iqn.home.net.mediaserver,i,0x400001370000)
Jun 18 02:00:15 istgt[4651]: istgt_iscsi.c:1261:istgt_iscsi_write_pdu_internal: ***ERROR*** writev() failed (errno=32,iqn.home.net.mediaserver,time=0)
Jun 18 02:00:15 istgt[4651]: istgt_iscsi.c:5392:sender: ***ERROR*** iscsi_task_response() CmdSN=2287171 failed on iqn.2014-06.net.home.dvrstorage:dvrstoragedisk1target,t,0x0001(iqn.home.net.mediaserver,i,0x400001370000)
Jun 18 02:00:15 istgt[4651]: istgt_iscsi.c:3853:istgt_iscsi_task_response: ***ERROR*** iscsi_transfer_in() failed
Jun 18 02:00:15 istgt[4651]: istgt_iscsi.c:3484:istgt_iscsi_transfer_in_internal: ***ERROR*** iscsi_write_pdu() failed
Jun 18 02:00:15 istgt[4651]: istgt_iscsi.c:1261:istgt_iscsi_write_pdu_internal: ***ERROR*** writev() failed (errno=32,iqn.home.net.mediaserver,time=0)
Jun 18 02:00:15 istgt[4651]: istgt_iscsi.c:5392:sender: ***ERROR*** iscsi_task_response() CmdSN=2287170 failed on iqn.2014-06.net.home.dvrstorage:dvrstoragedisk1target,t,0x0001(iqn.home.net.mediaserver,i,0x400001370000)
Jun 18 02:00:15 istgt[4651]: istgt_iscsi.c:3853:istgt_iscsi_task_response: ***ERROR*** iscsi_transfer_in() failed
Jun 18 02:00:15 istgt[4651]: istgt_iscsi.c:3484:istgt_iscsi_transfer_in_internal: ***ERROR*** iscsi_write_pdu() failed
Jun 18 02:00:15 istgt[4651]: istgt_iscsi.c:1261:istgt_iscsi_write_pdu_internal: ***ERROR*** writev() failed (errno=32,iqn.home.net.mediaserver,time=0)
Jun 18 02:00:15 last message repeated 653 times
Jun 18 02:00:15 kernel: sonewconn: pcb 0xfffffe021523aab8: Listen queue overflow: 2 already in queue awaiting acceptance
Jun 18 02:00:15 kernel: : Listen queue overflow: 2 already in queue awaiting acceptance

this only seems to happen when transferring large files. when it happens the iscsi target disappears from the initiator until i disable/reenable the target in freenas. can anyone out there help me out with this?
Top
cjohnson
NewUser
NewUser
Posts: 4
Joined: 04 Jun 2014 07:03
Status: Offline

Re: listen queue overflow error

Post by cjohnson »

I've been reading some posts here and there, googling etc. Maybe I've not provided enough information to facilitate a reply?

The hardware is a S5000PSL motherboard with two E5450's and 16gigs of ECC RAM. I've attached 17 1TB SATA drives to this board via three h310's flashed with dell IT mode firmware. The disks are arranged into three raidz2's like this......
config:

NAME STATE READ WRITE CKSUM
dvrstoragedisk1 ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
da15 ONLINE 0 0 0
da1 ONLINE 0 0 0
da10 ONLINE 0 0 0
da13 ONLINE 0 0 0
da7 ONLINE 0 0 0
da2 ONLINE 0 0 0
raidz2-1 ONLINE 0 0 0
da9 ONLINE 0 0 0
da14 ONLINE 0 0 0
da6 ONLINE 0 0 0
da11 ONLINE 0 0 0
da17 ONLINE 0 0 0
da3 ONLINE 0 0 0
raidz2-2 ONLINE 0 0 0
da5 ONLINE 0 0 0
da8 ONLINE 0 0 0
da16 ONLINE 0 0 0
da18 ONLINE 0 0 0
da12 ONLINE 0 0 0
da4 ONLINE 0 0 0
cache
da0 ONLINE 0 0 0
ada0 ONLINE 0 0 0
one of the cache disks is attached directly to the motherboards first onboard SATA controller. the total pool utilization is at 68% currently. Does this help?
Top
hvisage
NewUser
NewUser
Posts: 14
Joined: 27 Aug 2014 16:35
Status: Offline

Re: listen queue overflow error

Post by hvisage »

Seeing similar issues:
Aug 27 15:40:08 nas4free kernel: sonewconn: pcb 0xfffffe018b539620: Listen
queue overflow: 2 already in queue awaiting acceptance
12.0.0.4 - Reticulus (revision 6766)
HP N36L/N40L microservers
Top
hvisage
NewUser
NewUser
Posts: 14
Joined: 27 Aug 2014 16:35
Status: Offline

Re: listen queue overflow error

Post by hvisage »

NAS4Free 9.2.0.1 (revision 972) x64-embedded
HP P56 Intel(R) Xeon(R) CPU 5160 @ 3.00GHz 8960MiB RAM

Intel 63XXESB USB 2.0 controller Intel 631XESB/632XESB/3100 USB
controller USB-1 Intel 631XESB/632XESB/3100 USB controller USB-2 Intel
63XXESB2 UDMA100 controller Intel 631XESB/632XESB/3100 USB controller
USB-3 BCM5708C 1000BASE-T media interface Intel 631XESB/632XESB/3100
USB controller USB-4 bus=1 hubaddr=1 port=5 devaddr=2 interface=0
bus=1 hubaddr=1 port=5 devaddr=2 interface=1 bus=1 hubaddr=8 port=4
devaddr=2 interface=0 UHCI (generic) USB controller bus=1 hubaddr=2
port=5 devaddr=3 interface=0
Subject:
iSCSI connections not accepting fast enough?
Description:
On tracking some troubles I had with iSCSI from three Proxmox (Linux 2.6.32
based) hypervisor servers, checking the logs I see an abundant of errors like:

kernel: sonewconn: pcb 0xfffffe018b539620: Listen queue overflow: 2 already in
queue awaiting acceptance

My Googling points me (at best) to: netstat -LaAn of:
fffffe018b6617a0 tcp4 0/0/1 *.3210
fffffe018b53c000 tcp4 0/0/1 *.3260

These I assume/presume is my iSCSI portals I've defined on ports 3260 and 3210

Now I miss the tools like lsof and not that FreeBSD literate as I should be
(Solaris/Linux man :() I do not know how to fix this issue easily.

Please help/advise?

Code: Select all

Aug 27 15:40:09 	nas4free 	istgt[2912]: Logout(discovery) from
iqn.1991-05.com.microsoft:uat-web (10.23.3.203) on (10.23.45.17:3260,1),
ISID=400001370000, TSIH=25, CID=1, HeaderDigest=off, DataDigest=off
Aug 27 15:40:09 	nas4free 	istgt[2912]: Login(discovery) from
iqn.1991-05.com.microsoft:uat-web (10.23.3.203) on (10.23.45.17:3260,1),
ISID=400001370000, TSIH=25, CID=1, HeaderDigest=off, DataDigest=off
Aug 27 15:40:08 	nas4free 	kernel: sonewconn: pcb 0xfffffe018b539620: Listen
queue overflow: 2 already in queue awaiting acceptance
Aug 27 15:40:08 	nas4free 	kernel: sonewconn: pcb 0xfffffe018b539620: Listen
queue overflow: 2 already in queue awaiting acceptance
Aug 27 15:40:06 	nas4free 	istgt[2912]: Login from
iqn.1991-05.com.microsoft:uat-web (10.23.3.203) on
iqn.2013-10.n4f.inside.istgt:uat-web LU8 (10.23.45.17:3260,1),
ISID=400001370001, TSIH=3, CID=1, HeaderDigest=off, DataDigest=off
Aug 27 15:37:53 	nas4free 	istgt[2912]: Login from
iqn.1993-08.org.debian:01:1a9a44b54238 (10.23.3.101) on
iqn.2013-10.n4f.inside.istgt:uat-vms LU7 (10.23.45.17:3260,1), ISID=23d010000,
TSIH=7, CID=0, HeaderDigest=off, DataDigest=off
12.0.0.4 - Reticulus (revision 6766)
HP N36L/N40L microservers
Top
User avatar
ccie4526
NewUser
NewUser
Posts: 12
Joined: 30 Nov 2014 22:25
Status: Offline

Re: listen queue overflow error

Post by ccie4526 »

Bump. Seeing this issue. How to resolve?
NAS1:Nas4Free 10.2.0.2.2235. HP DL380G6, 72GB DDR3, 2x X5560 QuadCore, 2x10Gb NIC, LSI9211-8i & LSI9200-8e with IT firmware. 16x 146GB 10K SAS, 25x 300GB 10K SAS. 2x 100GB SSD SAS, Intel 750 NVMe 400GB.
NAS2:Nas4Free 10.2.0.2.2433. HP N54L, 8GB DDR2, AMD Turion DualCore, 1Gb NIC, 2x 1TB WD Red, 1x 32Gb X25-e SSD
NAS3:Nas4Free 10.2.0.2.2235. Sun X4540, 64GB DDR2, 2x AMD Opteron QuadCore, 2x10Gb NIC, 19x 750GB SATA, Intel 750 NVMe 400GB
Top
simond
NewUser
NewUser
Posts: 1
Joined: 03 Aug 2015 18:06
Status: Offline

Re: listen queue overflow error

Post by simond »

We're seeing that too with 10.1.0.2 and iSCSI backing our VMware servers and the only thing that normally fixes issues is restarting the iSCSI target and then rebooting all our VMs
Top
User avatar
daoyama
Developer
Developer
Posts: 394
Joined: 25 Aug 2012 09:28
Location: Japan
Status: Offline

Re: listen queue overflow error

Post by daoyama »

hvisage wrote: Aug 27 15:40:08 nas4free kernel: sonewconn: pcb 0xfffffe018b539620: Listen
queue overflow: 2 already in queue awaiting acceptance
This is not fatal error. You can ignore it. Client will retry it.
Try QueueDepth 64 or 0, and Timeout 60.
If you use lagg, check your switch setting.
NAS4Free 10.2.0.2.2115 (x64-embedded), 10.2.0.2.2258 (arm), 10.2.0.2.2258(dom0)
GIGABYTE 5YASV-RH, Celeron E3400 (Dual 2.6GHz), ECC 8GB, Intel ET/CT/82566DM (on-board), ZFS mirror (2TBx2)
ASRock E350M1/USB3, 16GB, Realtek 8111E (on-board), ZFS mirror (2TBx2)
MSI MS-9666, Core i7-860(Quad 2.8GHz/HT), 32GB, Mellanox ConnectX-2 EN/Intel 82578DM (on-board), ZFS mirror (3TBx2+L2ARC/ZIL:SSD128GB)
Develop/test environment:
VirtualBox 512MB VM, ESXi 512MB-8GB VM, Raspberry Pi, Pi2, ODROID-C1
Top
allen00se
NewUser
NewUser
Posts: 1
Joined: 15 May 2014 17:37
Status: Offline

Re: listen queue overflow error

Post by allen00se »

daoyama wrote:
This is not fatal error. You can ignore it. Client will retry it.
Try QueueDepth 64 or 0, and Timeout 60.
If you use lagg, check your switch setting.
Where is the QueueDepth setting located?
Top
Post Reply

Return to “iSCSI (Internet Small Computer Systems Interface)”