Page 1 of 1

HPE Gen10 with Smartarray in HBA mode: error in bus_dmamap_load_ccb

Posted: 16 Jan 2020 14:58
by rel
Hello,

We try to use XigmaNAS on HPE DL380 Gen10 with SmartArray P816i-a (via BHA mode).
During writing from network (via CIFS), after a few minutes there is an error put to the console, and any read from zpool hangs server.
Do you know if that can be hardware issues, or FreeBSD does not support Gen10 fully, or ZFS does not support it?

[ERROR]::[177:655.0][0.67.0][CPU 10][pqi_map_request][540]:bus_dmamap_load_ccb failed = 36 count = 131072
[WARN]:[177:655.0][CPU 10][pqisrc_to_start][794]:In progress on 67

No hardware RAID is defined, Smartarray is in midex mode, disks are visible in HBA mode to the system.
Either pool or dataset is made available to external backup system as SMB/CIFS storage.

ZPOOL is created and there are no hardware errors visible in ILO etc.

There are no errors in hardware noticed during extensive tests.

XigmaNAS version used: 12.1.0.4 - Ingva (revision 7091)

Result of command:

Code: Select all

# zpool status -v Pool_1
  pool: Pool_1
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
        still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
  scan: resilvered 1.02G in 0 days 00:00:09 with 0 errors on Wed Jan  8 13:58:47 2020
config:

        NAME        STATE     READ WRITE CKSUM
        Pool_1      ONLINE       0     0     0
          raidz3-0  ONLINE       0     0     0
            da0p1   ONLINE       0     0     0
            da1p1   ONLINE       0     0     0
            da2p1   ONLINE       0     0     0
            da3p1   ONLINE       0     0     0
            da4p1   ONLINE       0     0     0
            da5p1   ONLINE       0     0     0
            da6p1   ONLINE       0     0     0
            da7p1   ONLINE       0     0     0
errors: No known data errors

Best regards,
Piotr

Re: HPE Gen10 with Smartarray in HBA mode: error in bus_dmamap_load_ccb

Posted: 16 Jan 2020 16:27
by raulfg3
some notes:
da0p1 means that you install your pool on partitions not on entire disk.
viewtopic.php?f=56&t=12485&p=78143&hilit=da0p1#p78143
viewtopic.php?f=66&t=3014#p16310

about dmamap_load_ccb please test a 11.3 install to discart problem on latest 12.1 rev. of BSD

Re: HPE Gen10 with Smartarray in HBA mode: error in bus_dmamap_load_ccb

Posted: 20 Jan 2020 15:24
by rel
Same result after change from partitions to disks, and with 11.3:

Code: Select all

# dmesg | tail -2
[ERROR]::[177:655.0][0,65,0][CPU 14][pqi_map_request][541]:bus_dmamap_load_ccb failed = 36 count = 131072
[WARN]:[177:655.0][CPU 14][pqisrc_io_start][795]:In Progress on 65

Code: Select all

# zpool status
  pool: Pool1
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
        still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
  scan: scrub repaired 0 in 0 days 00:00:00 with 0 errors on Mon Jan 20 15:04:46 2020
config:

        NAME        STATE     READ WRITE CKSUM
        Pool1       ONLINE       0     0     0
          raidz3-0  ONLINE       0     0     0
            da0     ONLINE       0     0     0
            da1     ONLINE       0     0     0
            da2     ONLINE       0     0     0
            da3     ONLINE       0     0     0
            da4     ONLINE       0     0     0
            da5     ONLINE       0     0     0
            da6     ONLINE       0     0     0
            da7     ONLINE       0     0     0

errors: No known data errors

Code: Select all

# uname -a
FreeBSD [xxxxxx] 11.3-RELEASE-p3 FreeBSD 11.3-RELEASE-p3 #0 r353706M: Fri Oct 18 02:02:16 CEST 2019     root@dev.xigmanas.com:/usr/obj/xigmanas/usr/src/sys/XIGMANAS-amd64  amd64
Another test with another system - OMV (Linux) passed OK (no errors).

It looks like FreeBSD's smartpqi driver is not compatible with HPE Smart Array P816i-a "Adaptec Smart Storage PQI 12G SAS/PCIe 3"

i. https://bugs.freebsd.org/bugzilla/show_ ... ?id=240145 - here suggestion was buggy RAID card, but how to prove it, if OMV (no ZFS) is working fine.
ii. https://reviews.freebsd.org/D14514 - looks like development is ongoing for the driver

Should I report this to FreeBSD or to HPE, or both?

Re: HPE Gen10 with Smartarray in HBA mode: error in bus_dmamap_load_ccb

Posted: 22 Jan 2020 14:05
by rel
Hello,

just to close the topic: HPE responded, that they do not support FreeBSD operating system
and it is not among certified operating systems:
HPE Servers Support & Certification Matrices: https://techlibrary.hpe.com/us/en/enter ... ndows.aspx
As in the HP Dl380 Gen10 logs, they found no errors, then error reported from FreeBSD will be not taken into their analysis.
Time to look for other system or other hardware.