This is the old XigmaNAS forum in read only mode,
it will taken offline by the end of march 2021!


I like to aks Users and Admins to rewrite/take over important post from here into the new fresh main forum!
Its not possible for us to export from here and import it to the main forum!

Random reboots with NAS4Free 9.2.0.1.925 on a ESXi server.

New installs of XigmaNAS.
Forum rules
• Set-Up Guide • FAQs • Forum Rules
Post Reply
neptunus
experienced User
experienced User
Posts: 79
Joined: 11 Jun 2013 08:50
Status: Offline

Random reboots with NAS4Free 9.2.0.1.925 on a ESXi server.

Post by neptunus »

I have random reboots with NAS4Free 9.2.0.1.925 on a ESXi server. I had no problems with NAS4Free 9.1.0.1.847 Release.

For reference here is my hardware setup:
Xeon CPU
Supermicro X9SRL-F Motherboard
64GB ECC
2x IBM ServeRAID M1015 in TI mode
4GB USB Flash drive (ESXi 5.1)
1x SATADOM 64 GB for 2 VM's other VM shared by iSCSI NAS4Free
2x 120 GB SDD
10 x 2TB Hitachi HDD
Intel PRO/1000 PT Quad port Server NIC

NAS4Free setup:
NAS4Free 9.2.0.1.925 in a ESXi VM (vCPU 2, 32GB RAM, 2x M1015 in passthru to VM, vmx3f0 ethernet to VM)
2x 120 GB SDD
10 x 2TB Hitachi HDD
Using ZFS, Samba, iSCSI

With NAS4Free 9.1.0.1.847 I had a IRQ storm problems with 2x M1015 in passthru to VM:

Code: Select all

run_interrupt_driven_hooks: still waiting after 60 seconds for xpt_config mps_startup
run_interrupt_driven_hooks: still waiting after 120 seconds for xpt_config mps_startup
run_interrupt_driven_hooks: still waiting after 180 seconds for xpt_config mps_startup
Removing the 2x M1015 in passthru to VM and edit loader.conf
hw.pci.enable_msix="0" # Driver Interrupts problem SAS2008
Reboot
Adding 2x M1015 in passthru to VM no problems with the IRQ strom.

Now I have a random reboot problem with NAS4Free 9.2.0.1.925 running on a ESXi server. I just have no idea why, can't figure out why.


Maybe it is a IRQ problem. Can some one help me?

Code: Select all

sideswipe: / # vmstat -i
interrupt                          total       rate
irq1: atkbd0                           6          0
irq15: ata1                          161          0
irq17: mpt0                         3017         16
cpu0:timer                         10026         54
irq256: vmx3f0                      2830         15
irq257: mps0                        2591         14
irq258: mps1                        6753         36
cpu3:timer                          1937         10
cpu1:timer                          1984         10
cpu2:timer                          1896         10
Total                              31201        168

Code: Select all

sideswipe: / # dmesg | grep irq
mps1: <LSI SAS2008> port 0x6000-0x60ff mem 0xd2600000-0xd2603fff,0xd2640000-0xd267ffff irq 16 at device 0.0 on p                                       ci19
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
psm0: <PS/2 Mouse> irq 12 on atkbdc0
vmx3f0: <VMware Vmxnet3 Ethernet Controller> port 0x4000-0x400f mem 0xd2404000-0xd2404fff,0xd2403000-0xd2403fff,                                       0xd2400000-0xd2401fff irq 18 at device 0.0 on pci3
vmx3f0: Allocating MSIx irq failed, error : 6
ioapic0 <Version 1.1> irqs 0-23 on motherboard
attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
vgapci0: <VGA-compatible display> port 0x10d0-0x10df mem 0xd8000000-0xdbffffff,0xd0800000-0xd0ffffff irq 16 at d                                       evice 15.0 on pci0
mpt0: <LSILogic 1030 Ultra4 Adapter> port 0x1400-0x14ff mem 0xd0040000-0xd005ffff,0xd0020000-0xd003ffff irq 17 a                                       t device 16.0 on pci0
vmx3f0: <VMware Vmxnet3 Ethernet Controller> port 0x4000-0x400f mem 0xd2404000-0xd2404fff,0xd2403000-0xd2403fff,                                       0xd2400000-0xd2401fff irq 18 at device 0.0 on pci3
mps0: <LSI SAS2008> port 0x5000-0x50ff mem 0xd2500000-0xd2503fff,0xd2540000-0xd257ffff irq 19 at device 0.0 on p                                       ci11
mps1: <LSI SAS2008> port 0x6000-0x60ff mem 0xd2600000-0xd2603fff,0xd2640000-0xd267ffff irq 16 at device 0.0 on p                                       ci19
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
psm0: <PS/2 Mouse> irq 12 on atkbdc0
ioapic0 <Version 1.1> irqs 0-23 on motherboard
attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
vgapci0: <VGA-compatible display> port 0x10d0-0x10df mem 0xd8000000-0xdbffffff,0xd0800000-0xd0ffffff irq 16 at d                                       evice 15.0 on pci0
mpt0: <LSILogic 1030 Ultra4 Adapter> port 0x1400-0x14ff mem 0xd0040000-0xd005ffff,0xd0020000-0xd003ffff irq 17 a                                       t device 16.0 on pci0
vmx3f0: <VMware Vmxnet3 Ethernet Controller> port 0x4000-0x400f mem 0xd2404000-0xd2404fff,0xd2403000-0xd2403fff,                                       0xd2400000-0xd2401fff irq 18 at device 0.0 on pci3
mps0: <LSI SAS2008> port 0x5000-0x50ff mem 0xd2500000-0xd2503fff,0xd2540000-0xd257ffff irq 19 at device 0.0 on p                                       ci11
mps1: <LSI SAS2008> port 0x6000-0x60ff mem 0xd2600000-0xd2603fff,0xd2640000-0xd267ffff irq 16 at device 0.0 on p                                       ci19
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
psm0: <PS/2 Mouse> irq 12 on atkbdc0
ioapic0 <Version 1.1> irqs 0-23 on motherboard
attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
vgapci0: <VGA-compatible display> port 0x10d0-0x10df mem 0xd8000000-0xdbffffff,0xd0800000-0xd0ffffff irq 16 at d                                       evice 15.0 on pci0
mpt0: <LSILogic 1030 Ultra4 Adapter> port 0x1400-0x14ff mem 0xd0040000-0xd005ffff,0xd0020000-0xd003ffff irq 17 a                                       t device 16.0 on pci0
vmx3f0: <VMware Vmxnet3 Ethernet Controller> port 0x4000-0x400f mem 0xd2404000-0xd2404fff,0xd2403000-0xd2403fff,                                       0xd2400000-0xd2401fff irq 18 at device 0.0 on pci3
mps0: <LSI SAS2008> port 0x5000-0x50ff mem 0xd2500000-0xd2503fff,0xd2540000-0xd257ffff irq 19 at device 0.0 on p                                       ci11
mps1: <LSI SAS2008> port 0x6000-0x60ff mem 0xd2600000-0xd2603fff,0xd2640000-0xd267ffff irq 16 at device 0.0 on p                                       ci19
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
psm0: <PS/2 Mouse> irq 12 on atkbdc0
Below a summary of the IRQ's:
atkbdc0: <Keyboard controller (i8042)> irq 1
atkbd0: <AT Keyboard> irq 1
psm0: <PS/2 Mouse> irq 12
vmx3f0: <VMware Vmxnet3 Ethernet Controller> irq 18
vmx3f0: Allocating MSIx irq failed, error : 6
ioapic0 <Version 1.1> irqs 0-23
attimer0: <AT timer> irq 0
atrtc0: <AT realtime clock> irq 8
vgapci0: <VGA-compatible display> irq 16
mpt0: <LSILogic 1030 Ultra4 Adapter> irq 17
vmx3f0: <VMware Vmxnet3 Ethernet Controller> irq 18
mps0: <LSI SAS2008> irq 19
mps1: <LSI SAS2008> irq 16
atkbdc0: <Keyboard controller (i8042)> irq 1
atkbd0: <AT Keyboard> irq 1
psm0: <PS/2 Mouse> irq 12

Suspects:
- vmx3f0 because of Allocating MSIx irq failed, error : 6
- vgapci0 and mps1 same IRQ.
Top
User avatar
raulfg3
Site Admin
Site Admin
Posts: 4865
Joined: 22 Jun 2012 22:13
Location: Madrid (ESPAÑA)
Contact:
Contact raulfg3
Status: Offline

Re: Random reboots with NAS4Free 9.2.0.1.925 on a ESXi serve

Post by raulfg3 »

some problems detected loading loader.conf from webgui: viewtopic.php?f=78&t=5847&p=32056&hilit ... onf#p32056

Please until developers find a patch, do changes in loader.conf by hand ( no webGUI)
12.1.0.4 - Ingva (revision 7743) on SUPERMICRO X8SIL-F 8GB of ECC RAM, 11x3TB disk in 1 vdev = Vpool = 32TB Raw size , so 29TB usable size (I Have other NAS as Backup)

Wiki
Last changes

HP T510
Top
RedAntz
experienced User
experienced User
Posts: 125
Joined: 11 Jul 2012 07:46
Location: Sydney, Australia
Status: Offline

Re: Random reboots with NAS4Free 9.2.0.1.925 on a ESXi serve

Post by RedAntz »

Hi neptunus,

I don't know how to change irq with ESXi, but I have *similar* setup that runs flawlessly for 1year + and can give you suggestions.

First, remove unnecessary devices from your NAS VM you'll be fine. USB device was the one that causes IRQ storm for me. Attaching a picture of what I have (no interrupt storm, no hw.pci.enable_msix="0" from NAS4Free 9.2 full install )
Image

Second, change your ethernet controller to e1000 instead of vmxnet3 and see if this resolve your issue.
You do not have the required permissions to view the files attached to this post.
Top
neptunus
experienced User
experienced User
Posts: 79
Joined: 11 Jun 2013 08:50
Status: Offline

Re: Random reboots with NAS4Free 9.2.0.1.925 on a ESXi serve

Post by neptunus »

RedAntz wrote:Hi neptunus,

I don't know how to change irq with ESXi, but I have *similar* setup that runs flawlessly for 1year + and can give you suggestions.

First, remove unnecessary devices from your NAS VM you'll be fine. USB device was the one that causes IRQ storm for me. Attaching a picture of what I have (no interrupt storm, no hw.pci.enable_msix="0" from NAS4Free 9.2 full install )

Second, change your ethernet controller to e1000 instead of vmxnet3 and see if this resolve your issue.
Thank, I will upgrade to NAS4Free 9.2.0.1.943 Released (FreeBSD 9.2-RELEASE p3) and will check my devices for the VM in ESXi. I will keep you informed.
Top
neptunus
experienced User
experienced User
Posts: 79
Joined: 11 Jun 2013 08:50
Status: Offline

Re: Random reboots with NAS4Free 9.2.0.1.925 on a ESXi serve

Post by neptunus »

Fresh install of 9.2.0.1.943 but still the random reboot.

ESXi config below:
Image

dmesg after reboot:

Code: Select all

Copyright (c) 1992-2013 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.2-RELEASE-p3 #0 r260900M: Mon Jan 20 07:22:46 CET 2014
    root@dev.nas4free.org:/usr/obj/nas4free/usr/src/sys/NAS4FREE-amd64 amd64
gcc version 4.2.1 20070831 patched [FreeBSD]
CPU: Intel(R) Xeon(R) CPU E5-1620 0 @ 3.60GHz (3600.00-MHz K8-class CPU)
  Origin = "GenuineIntel"  Id = 0x206d7  Family = 0x6  Model = 0x2d  Stepping = 7
  Features=0x1fa3fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,DTS,MMX,FXSR,SSE,SSE2,SS,HTT>
  Features2=0x9e982203<SSE3,PCLMULQDQ,SSSE3,CX16,SSE4.1,SSE4.2,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,HV>
  AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
  AMD Features2=0x1<LAHF>
  TSC: P-state invariant
real memory  = 34359738368 (32768 MB)
avail memory = 33176080384 (31639 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <PTLTD  	 APIC  >
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 2 package(s) x 2 core(s)
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
 cpu2 (AP): APIC ID:  2
 cpu3 (AP): APIC ID:  3
MADT: Forcing active-low polarity and level trigger for SCI
ioapic0 <Version 1.1> irqs 0-23 on motherboard
kbd1 at kbdmux0
cryptosoft0: <software crypto> on motherboard
acpi0: <INTEL 440BX> on motherboard
acpi0: Power Button (fixed)
Timecounter "HPET" frequency 14318180 Hz quality 950
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
Event timer "RTC" frequency 32768 Hz quality 0
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 1.0 on pci0
pci1: <ACPI PCI bus> on pcib1
isab0: <PCI-ISA bridge> at device 7.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel PIIX4 UDMA33 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x10c0-0x10cf at device 7.1 on pci0
ata0: <ATA channel> at channel 0 on atapci0
ata1: <ATA channel> at channel 1 on atapci0
pci0: <bridge> at device 7.3 (no driver attached)
pci0: <base peripheral> at device 7.7 (no driver attached)
vgapci0: <VGA-compatible display> port 0x10d0-0x10df mem 0xd8000000-0xdbffffff,0xd0800000-0xd0ffffff irq 16 at device 15.0 on pci0
mpt0: <LSILogic 1030 Ultra4 Adapter> port 0x1400-0x14ff mem 0xd0040000-0xd005ffff,0xd0020000-0xd003ffff irq 17 at device 16.0 on pci0
mpt0: MPI Version=1.2.0.0
pcib2: <ACPI PCI-PCI bridge> at device 17.0 on pci0
pci2: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> at device 21.0 on pci0
pci3: <ACPI PCI bus> on pcib3
pci3: <network, ethernet> at device 0.0 (no driver attached)
pcib4: <ACPI PCI-PCI bridge> at device 21.1 on pci0
pci4: <ACPI PCI bus> on pcib4
pcib5: <ACPI PCI-PCI bridge> at device 21.2 on pci0
pci5: <ACPI PCI bus> on pcib5
pcib6: <ACPI PCI-PCI bridge> at device 21.3 on pci0
pci6: <ACPI PCI bus> on pcib6
pcib7: <ACPI PCI-PCI bridge> at device 21.4 on pci0
pci7: <ACPI PCI bus> on pcib7
pcib8: <ACPI PCI-PCI bridge> at device 21.5 on pci0
pci8: <ACPI PCI bus> on pcib8
pcib9: <ACPI PCI-PCI bridge> at device 21.6 on pci0
pci9: <ACPI PCI bus> on pcib9
pcib10: <ACPI PCI-PCI bridge> at device 21.7 on pci0
pci10: <ACPI PCI bus> on pcib10
pcib11: <ACPI PCI-PCI bridge> at device 22.0 on pci0
pci11: <ACPI PCI bus> on pcib11
mps0: <LSI SAS2008> port 0x5000-0x50ff mem 0xd2500000-0xd2503fff,0xd2540000-0xd257ffff irq 19 at device 0.0 on pci11
mps0: Firmware: 16.00.00.00, Driver: 14.00.00.01-fbsd
mps0: IOCCapabilities: 1285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,HostDisc>
pcib12: <ACPI PCI-PCI bridge> at device 22.1 on pci0
pci12: <ACPI PCI bus> on pcib12
pcib13: <ACPI PCI-PCI bridge> at device 22.2 on pci0
pci13: <ACPI PCI bus> on pcib13
pcib14: <ACPI PCI-PCI bridge> at device 22.3 on pci0
pci14: <ACPI PCI bus> on pcib14
pcib15: <ACPI PCI-PCI bridge> at device 22.4 on pci0
pci15: <ACPI PCI bus> on pcib15
pcib16: <ACPI PCI-PCI bridge> at device 22.5 on pci0
pci16: <ACPI PCI bus> on pcib16
pcib17: <ACPI PCI-PCI bridge> at device 22.6 on pci0
pci17: <ACPI PCI bus> on pcib17
pcib18: <ACPI PCI-PCI bridge> at device 22.7 on pci0
pci18: <ACPI PCI bus> on pcib18
pcib19: <ACPI PCI-PCI bridge> at device 23.0 on pci0
pci19: <ACPI PCI bus> on pcib19
mps1: <LSI SAS2008> port 0x6000-0x60ff mem 0xd2600000-0xd2603fff,0xd2640000-0xd267ffff irq 16 at device 0.0 on pci19
mps1: Firmware: 16.00.00.00, Driver: 14.00.00.01-fbsd
mps1: IOCCapabilities: 1285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,HostDisc>
pcib20: <ACPI PCI-PCI bridge> at device 23.1 on pci0
pci20: <ACPI PCI bus> on pcib20
pcib21: <ACPI PCI-PCI bridge> at device 23.2 on pci0
pci21: <ACPI PCI bus> on pcib21
pcib22: <ACPI PCI-PCI bridge> at device 23.3 on pci0
pci22: <ACPI PCI bus> on pcib22
pcib23: <ACPI PCI-PCI bridge> at device 23.4 on pci0
pci23: <ACPI PCI bus> on pcib23
pcib24: <ACPI PCI-PCI bridge> at device 23.5 on pci0
pci24: <ACPI PCI bus> on pcib24
pcib25: <ACPI PCI-PCI bridge> at device 23.6 on pci0
pci25: <ACPI PCI bus> on pcib25
pcib26: <ACPI PCI-PCI bridge> at device 23.7 on pci0
pci26: <ACPI PCI bus> on pcib26
pcib27: <ACPI PCI-PCI bridge> at device 24.0 on pci0
pci27: <ACPI PCI bus> on pcib27
pcib28: <ACPI PCI-PCI bridge> at device 24.1 on pci0
pci28: <ACPI PCI bus> on pcib28
pcib29: <ACPI PCI-PCI bridge> at device 24.2 on pci0
pci29: <ACPI PCI bus> on pcib29
pcib30: <ACPI PCI-PCI bridge> at device 24.3 on pci0
pci30: <ACPI PCI bus> on pcib30
pcib31: <ACPI PCI-PCI bridge> at device 24.4 on pci0
pci31: <ACPI PCI bus> on pcib31
pcib32: <ACPI PCI-PCI bridge> at device 24.5 on pci0
pci32: <ACPI PCI bus> on pcib32
pcib33: <ACPI PCI-PCI bridge> at device 24.6 on pci0
pci33: <ACPI PCI bus> on pcib33
pcib34: <ACPI PCI-PCI bridge> at device 24.7 on pci0
pci34: <ACPI PCI bus> on pcib34
acpi_acad0: <AC Adapter> on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: model IntelliMouse, device ID 3
orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xca000-0xcafff,0xdc000-0xdffff,0xe0000-0xe7fff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
ppc0: cannot reserve I/O port range
ctl: CAM Target Layer loaded
coretemp0: <CPU On-Die Thermal Sensors> on cpu0
coretemp0: Tj(target) value 0 does not seem right.
acpi_throttle0: <ACPI CPU Throttling> on cpu0
coretemp1: <CPU On-Die Thermal Sensors> on cpu1
coretemp1: Tj(target) value 0 does not seem right.
acpi_throttle1: <ACPI CPU Throttling> on cpu1
acpi_throttle1: failed to attach P_CNT
device_attach: acpi_throttle1 attach returned 6
coretemp2: <CPU On-Die Thermal Sensors> on cpu2
coretemp2: Tj(target) value 0 does not seem right.
acpi_throttle2: <ACPI CPU Throttling> on cpu2
acpi_throttle2: failed to attach P_CNT
device_attach: acpi_throttle2 attach returned 6
coretemp3: <CPU On-Die Thermal Sensors> on cpu3
coretemp3: Tj(target) value 0 does not seem right.
acpi_throttle3: <ACPI CPU Throttling> on cpu3
acpi_throttle3: failed to attach P_CNT
device_attach: acpi_throttle3 attach returned 6
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
Timecounters tick every 10.000 msec
ipfw2 (+ipv6) initialized, divert loadable, nat loadable, default to accept, logging disabled
iSCSI boot driver version 0.2.6
cd0 at ata1 bus 0 scbus1 target 0 lun 0
cd0: <NECVMWar VMware IDE CDR10 1.00> Removable CD-ROM SCSI-0 device 
cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes)
cd0: Attempt to query device size failed: NOT READY, Medium not present
da0 at mpt0 bus 0 scbus2 target 0 lun 0
da0: <VMware Virtual disk 1.0> Fixed Direct Access SCSI-2 device 
da0: 320.000MB/s transfers (160.000MHz, offset 127, 16bit)
da0: Command Queueing enabled
da0: 4096MB (8388608 512 byte sectors: 255H 63S/T 522C)
da11 at mps1 bus 0 scbus4 target 5 lun 0
da11: <ATA INTEL SSDSA2CW12 0362> Fixed Direct Access SCSI-6 device 
da11: 300.000MB/s transfers
da11: Command Queueing enabled
da11: 114473MB (234441648 512 byte sectors: 255H 63S/T 14593C)
da11: quirks=0x8<4K>
da12 at mps1 bus 0 scbus4 target 6 lun 0
da12: <ATA INTEL SSDSA2CW12 0362> Fixed Direct Access SCSI-6 device 
da12: 300.000MB/s transfers
da12: Command Queueing enabled
da12: 114473MB (234441648 512 byte sectors: 255H 63S/T 14593C)
da12: quirks=0x8<4K>
da1 at mps0 bus 0 scbus3 target 0 lun 0
da1: <ATA Hitachi HUA72302 A840> Fixed Direct Access SCSI-6 device 
da1: 600.000MB/s transfers
da1: Command Queueing enabled
da1: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da2 at mps0 bus 0 scbus3 target 1 lun 0
da2: <ATA Hitachi HUA72302 A840> Fixed Direct Access SCSI-6 device 
da2: 600.000MB/s transfers
da2: Command Queueing enabled
da2: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da3 at mps0 bus 0 scbus3 target 2 lun 0
da3: <ATA Hitachi HUA72302 A840> Fixed Direct Access SCSI-6 device 
da3: 600.000MB/s transfers
da3: Command Queueing enabled
da3: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da4 at mps0 bus 0 scbus3 target 3 lun 0
da4: <ATA Hitachi HUA72302 A840> Fixed Direct Access SCSI-6 device 
da4: 600.000MB/s transfers
da4: Command Queueing enabled
da4: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da5 at mps0 bus 0 scbus3 target 4 lun 0
da5: <ATA Hitachi HUA72302 A840> Fixed Direct Access SCSI-6 device 
da5: 600.000MB/s transfers
da5: Command Queueing enabled
da5: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da7 at mps1 bus 0 scbus4 target 1 lun 0
da7: <ATA Hitachi HUA72302 A840> Fixed Direct Access SCSI-6 device 
da7: 600.000MB/s transfers
da7: Command Queueing enabled
da7: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da6 at mps1 bus 0 scbus4 target 0 lun 0
da6: <ATA Hitachi HUA72302 A840> Fixed Direct Access SCSI-6 device 
da6: 600.000MB/s transfers
da6: Command Queueing enabled
da6: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da9 at mps1 bus 0 scbus4 target 3 lun 0
da9: <ATA Hitachi HUA72302 A840> Fixed Direct Access SCSI-6 device 
da9: 600.000MB/s transfers
da9: Command Queueing enabled
da9: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da8 at mps1 bus 0 scbus4 target 2 lun 0
da8: <ATA Hitachi HUA72302 A840> Fixed Direct Access SCSI-6 device 
da8: 600.000MB/s transfers
da8: Command Queueing enabled
da8: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da10 at mps1 bus 0 scbus4 target 4 lun 0
da10: <ATA Hitachi HUA72302 A840> Fixed Direct Access SCSI-6 device 
da10: 600.000MB/s transfers
da10: Command Queueing enabled
da10: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
SMP: AP CPU #3 Launched!
SMP: AP CPU #2 Launched!
SMP: AP CPU #1 Launched!
Trying to mount root from ufs:/dev/ufsid/52dd563212019309 [rw]...
VMware memory control driver initialized
vmx3f0: <VMware Vmxnet3 Ethernet Controller> port 0x4000-0x400f mem 0xd2404000-0xd2404fff,0xd2403000-0xd2403fff,0xd2400000-0xd2401fff irq 18 at device 0.0 on pci3
vmx3f0: Driver version : 0.1.0.0.
vmx3f0: Capabilities : sg jf vlan csum tso
vmx3f0: Allocating MSIx irq failed, error : 6
vmx3f0: Ethernet address: 00:0c:29:2e:88:20


Fatal trap 12: page fault while in kernel mode
cpuid = 3; apic id = 03
fault virtual address	= 0x368
fault code		= supervisor read data, page not present
instruction pointer	= 0x20:0xffffffff80a152af
stack pointer	        = 0x28:0xffffff84cbcd2fd0
frame pointer	        = 0x28:0xffffff84cbcd3060
code segment		= base 0x0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 10547 (zabbix_agentd)
trap number		= 12
panic: page fault
cpuid = 3
KDB: stack backtrace:
#0 0xffffffff80a46df6 at kdb_backtrace+0x66
#1 0xffffffff80a0ce1e at panic+0x1ce
#2 0xffffffff80e16220 at trap_fatal+0x290
#3 0xffffffff80e16581 at trap_pfault+0x211
#4 0xffffffff80e16b34 at trap+0x344
#5 0xffffffff80dffe63 at calltrap+0x8
#6 0xffffffff80a15d75 at _sx_xlock+0x75
#7 0xffffffff81bdd44f at zfsctl_snapdir_lookup+0x16f
#8 0xffffffff80ee4e0e at VOP_LOOKUP_APV+0x6e
#9 0xffffffff80a9d098 at lookup+0x3f8
#10 0xffffffff80a9e358 at namei+0x4f8
#11 0xffffffff80ab1b64 at kern_statfs+0xa4
#12 0xffffffff80ab1e37 at sys_statfs+0x37
#13 0xffffffff80e159ca at amd64_syscall+0x5ea
#14 0xffffffff80e00147 at Xfast_syscall+0xf7
Uptime: 2h13m30s
Copyright (c) 1992-2013 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.2-RELEASE-p3 #0 r260900M: Mon Jan 20 07:22:46 CET 2014
    root@dev.nas4free.org:/usr/obj/nas4free/usr/src/sys/NAS4FREE-amd64 amd64
gcc version 4.2.1 20070831 patched [FreeBSD]
CPU: Intel(R) Xeon(R) CPU E5-1620 0 @ 3.60GHz (3600.00-MHz K8-class CPU)
  Origin = "GenuineIntel"  Id = 0x206d7  Family = 0x6  Model = 0x2d  Stepping = 7
  Features=0x1fa3fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,DTS,MMX,FXSR,SSE,SSE2,SS,HTT>
  Features2=0x9e982203<SSE3,PCLMULQDQ,SSSE3,CX16,SSE4.1,SSE4.2,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,HV>
  AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
  AMD Features2=0x1<LAHF>
  TSC: P-state invariant
real memory  = 34359738368 (32768 MB)
avail memory = 33176076288 (31639 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <PTLTD  	 APIC  >
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 2 package(s) x 2 core(s)
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
 cpu2 (AP): APIC ID:  2
 cpu3 (AP): APIC ID:  3
MADT: Forcing active-low polarity and level trigger for SCI
ioapic0 <Version 1.1> irqs 0-23 on motherboard
kbd1 at kbdmux0
cryptosoft0: <software crypto> on motherboard
acpi0: <INTEL 440BX> on motherboard
acpi0: Power Button (fixed)
Timecounter "HPET" frequency 14318180 Hz quality 950
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
Event timer "RTC" frequency 32768 Hz quality 0
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 1.0 on pci0
pci1: <ACPI PCI bus> on pcib1
isab0: <PCI-ISA bridge> at device 7.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel PIIX4 UDMA33 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x10c0-0x10cf at device 7.1 on pci0
ata0: <ATA channel> at channel 0 on atapci0
ata1: <ATA channel> at channel 1 on atapci0
pci0: <bridge> at device 7.3 (no driver attached)
pci0: <base peripheral> at device 7.7 (no driver attached)
vgapci0: <VGA-compatible display> port 0x10d0-0x10df mem 0xd8000000-0xdbffffff,0xd0800000-0xd0ffffff irq 16 at device 15.0 on pci0
mpt0: <LSILogic 1030 Ultra4 Adapter> port 0x1400-0x14ff mem 0xd0040000-0xd005ffff,0xd0020000-0xd003ffff irq 17 at device 16.0 on pci0
mpt0: MPI Version=1.2.0.0
pcib2: <ACPI PCI-PCI bridge> at device 17.0 on pci0
pci2: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> at device 21.0 on pci0
pci3: <ACPI PCI bus> on pcib3
pci3: <network, ethernet> at device 0.0 (no driver attached)
pcib4: <ACPI PCI-PCI bridge> at device 21.1 on pci0
pci4: <ACPI PCI bus> on pcib4
pcib5: <ACPI PCI-PCI bridge> at device 21.2 on pci0
pci5: <ACPI PCI bus> on pcib5
pcib6: <ACPI PCI-PCI bridge> at device 21.3 on pci0
pci6: <ACPI PCI bus> on pcib6
pcib7: <ACPI PCI-PCI bridge> at device 21.4 on pci0
pci7: <ACPI PCI bus> on pcib7
pcib8: <ACPI PCI-PCI bridge> at device 21.5 on pci0
pci8: <ACPI PCI bus> on pcib8
pcib9: <ACPI PCI-PCI bridge> at device 21.6 on pci0
pci9: <ACPI PCI bus> on pcib9
pcib10: <ACPI PCI-PCI bridge> at device 21.7 on pci0
pci10: <ACPI PCI bus> on pcib10
pcib11: <ACPI PCI-PCI bridge> at device 22.0 on pci0
pci11: <ACPI PCI bus> on pcib11
mps0: <LSI SAS2008> port 0x5000-0x50ff mem 0xd2500000-0xd2503fff,0xd2540000-0xd257ffff irq 19 at device 0.0 on pci11
mps0: Firmware: 16.00.00.00, Driver: 14.00.00.01-fbsd
mps0: IOCCapabilities: 1285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,HostDisc>
pcib12: <ACPI PCI-PCI bridge> at device 22.1 on pci0
pci12: <ACPI PCI bus> on pcib12
pcib13: <ACPI PCI-PCI bridge> at device 22.2 on pci0
pci13: <ACPI PCI bus> on pcib13
pcib14: <ACPI PCI-PCI bridge> at device 22.3 on pci0
pci14: <ACPI PCI bus> on pcib14
pcib15: <ACPI PCI-PCI bridge> at device 22.4 on pci0
pci15: <ACPI PCI bus> on pcib15
pcib16: <ACPI PCI-PCI bridge> at device 22.5 on pci0
pci16: <ACPI PCI bus> on pcib16
pcib17: <ACPI PCI-PCI bridge> at device 22.6 on pci0
pci17: <ACPI PCI bus> on pcib17
pcib18: <ACPI PCI-PCI bridge> at device 22.7 on pci0
pci18: <ACPI PCI bus> on pcib18
pcib19: <ACPI PCI-PCI bridge> at device 23.0 on pci0
pci19: <ACPI PCI bus> on pcib19
mps1: <LSI SAS2008> port 0x6000-0x60ff mem 0xd2600000-0xd2603fff,0xd2640000-0xd267ffff irq 16 at device 0.0 on pci19
mps1: Firmware: 16.00.00.00, Driver: 14.00.00.01-fbsd
mps1: IOCCapabilities: 1285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,HostDisc>
pcib20: <ACPI PCI-PCI bridge> at device 23.1 on pci0
pci20: <ACPI PCI bus> on pcib20
pcib21: <ACPI PCI-PCI bridge> at device 23.2 on pci0
pci21: <ACPI PCI bus> on pcib21
pcib22: <ACPI PCI-PCI bridge> at device 23.3 on pci0
pci22: <ACPI PCI bus> on pcib22
pcib23: <ACPI PCI-PCI bridge> at device 23.4 on pci0
pci23: <ACPI PCI bus> on pcib23
pcib24: <ACPI PCI-PCI bridge> at device 23.5 on pci0
pci24: <ACPI PCI bus> on pcib24
pcib25: <ACPI PCI-PCI bridge> at device 23.6 on pci0
pci25: <ACPI PCI bus> on pcib25
pcib26: <ACPI PCI-PCI bridge> at device 23.7 on pci0
pci26: <ACPI PCI bus> on pcib26
pcib27: <ACPI PCI-PCI bridge> at device 24.0 on pci0
pci27: <ACPI PCI bus> on pcib27
pcib28: <ACPI PCI-PCI bridge> at device 24.1 on pci0
pci28: <ACPI PCI bus> on pcib28
pcib29: <ACPI PCI-PCI bridge> at device 24.2 on pci0
pci29: <ACPI PCI bus> on pcib29
pcib30: <ACPI PCI-PCI bridge> at device 24.3 on pci0
pci30: <ACPI PCI bus> on pcib30
pcib31: <ACPI PCI-PCI bridge> at device 24.4 on pci0
pci31: <ACPI PCI bus> on pcib31
pcib32: <ACPI PCI-PCI bridge> at device 24.5 on pci0
pci32: <ACPI PCI bus> on pcib32
pcib33: <ACPI PCI-PCI bridge> at device 24.6 on pci0
pci33: <ACPI PCI bus> on pcib33
pcib34: <ACPI PCI-PCI bridge> at device 24.7 on pci0
pci34: <ACPI PCI bus> on pcib34
acpi_acad0: <AC Adapter> on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: model IntelliMouse, device ID 3
orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xca000-0xcafff,0xdc000-0xdffff,0xe0000-0xe7fff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
ppc0: cannot reserve I/O port range
ctl: CAM Target Layer loaded
coretemp0: <CPU On-Die Thermal Sensors> on cpu0
coretemp0: Tj(target) value 0 does not seem right.
acpi_throttle0: <ACPI CPU Throttling> on cpu0
coretemp1: <CPU On-Die Thermal Sensors> on cpu1
coretemp1: Tj(target) value 0 does not seem right.
acpi_throttle1: <ACPI CPU Throttling> on cpu1
acpi_throttle1: failed to attach P_CNT
device_attach: acpi_throttle1 attach returned 6
coretemp2: <CPU On-Die Thermal Sensors> on cpu2
coretemp2: Tj(target) value 0 does not seem right.
acpi_throttle2: <ACPI CPU Throttling> on cpu2
acpi_throttle2: failed to attach P_CNT
device_attach: acpi_throttle2 attach returned 6
coretemp3: <CPU On-Die Thermal Sensors> on cpu3
coretemp3: Tj(target) value 0 does not seem right.
acpi_throttle3: <ACPI CPU Throttling> on cpu3
acpi_throttle3: failed to attach P_CNT
device_attach: acpi_throttle3 attach returned 6
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
Timecounters tick every 10.000 msec
ipfw2 (+ipv6) initialized, divert loadable, nat loadable, default to accept, logging disabled
iSCSI boot driver version 0.2.6
cd0 at ata1 bus 0 scbus1 target 0 lun 0
cd0: <NECVMWar VMware IDE CDR10 1.00> Removable CD-ROM SCSI-0 device 
cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes)
cd0: Attempt to query device size failed: NOT READY, Medium not present
da0 at mpt0 bus 0 scbus2 target 0 lun 0
da0: <VMware Virtual disk 1.0> Fixed Direct Access SCSI-2 device 
da0: 320.000MB/s transfers (160.000MHz, offset 127, 16bit)
da0: Command Queueing enabled
da0: 4096MB (8388608 512 byte sectors: 255H 63S/T 522C)
da11 at mps1 bus 0 scbus4 target 5 lun 0
da11: <ATA INTEL SSDSA2CW12 0362> Fixed Direct Access SCSI-6 device 
da11: 300.000MB/s transfers
da11: Command Queueing enabled
da11: 114473MB (234441648 512 byte sectors: 255H 63S/T 14593C)
da11: quirks=0x8<4K>
da12 at mps1 bus 0 scbus4 target 6 lun 0
da12: <ATA INTEL SSDSA2CW12 0362> Fixed Direct Access SCSI-6 device 
da12: 300.000MB/s transfers
da12: Command Queueing enabled
da12: 114473MB (234441648 512 byte sectors: 255H 63S/T 14593C)
da12: quirks=0x8<4K>
da1 at mps0 bus 0 scbus3 target 0 lun 0
da1: <ATA Hitachi HUA72302 A840> Fixed Direct Access SCSI-6 device 
da1: 600.000MB/s transfers
da1: Command Queueing enabled
da1: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da2 at mps0 bus 0 scbus3 target 1 lun 0
da2: <ATA Hitachi HUA72302 A840> Fixed Direct Access SCSI-6 device 
da2: 600.000MB/s transfers
da2: Command Queueing enabled
da2: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da3 at mps0 bus 0 scbus3 target 2 lun 0
da3: <ATA Hitachi HUA72302 A840> Fixed Direct Access SCSI-6 device 
da3: 600.000MB/s transfers
da3: Command Queueing enabled
da3: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da4 at mps0 bus 0 scbus3 target 3 lun 0
da4: <ATA Hitachi HUA72302 A840> Fixed Direct Access SCSI-6 device 
da4: 600.000MB/s transfers
da4: Command Queueing enabled
da4: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da5 at mps0 bus 0 scbus3 target 4 lun 0
da5: <ATA Hitachi HUA72302 A840> Fixed Direct Access SCSI-6 device 
da5: 600.000MB/s transfers
da5: Command Queueing enabled
da5: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
SMP: AP CPU #3 Launched!
da7 at mps1 bus 0 scbus4 target 1 lun 0
SMP: AP CPU #1 Launched!
da7: <ATA Hitachi HUA72302 A840> Fixed Direct Access SCSI-6 device 
SMP: AP CPU #2 Launched!
da7: 600.000MB/s transfers
da7: Command Queueing enabled
da7: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da8 at mps1 bus 0 scbus4 target 2 lun 0
da8: <ATA Hitachi HUA72302 A840> Fixed Direct Access SCSI-6 device 
da8: 600.000MB/s transfers
da8: Command Queueing enabled
da8: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da6 at mps1 bus 0 scbus4 target 0 lun 0
da6: <ATA Hitachi HUA72302 A840> Fixed Direct Access SCSI-6 device 
da6: 600.000MB/s transfers
da6: Command Queueing enabled
da6: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da10 at mps1 bus 0 scbus4 target 4 lun 0
da10: <ATA Hitachi HUA72302 A840> Fixed Direct Access SCSI-6 device 
da10: 600.000MB/s transfers
da10: Command Queueing enabled
da10: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da9 at mps1 bus 0 scbus4 target 3 lun 0
da9: <ATA Hitachi HUA72302 A840> Fixed Direct Access SCSI-6 device 
da9: 600.000MB/s transfers
da9: Command Queueing enabled
da9: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
Trying to mount root from ufs:/dev/ufsid/52dd563212019309 [rw]...
WARNING: / was not properly dismounted
VMware memory control driver initialized
vmx3f0: <VMware Vmxnet3 Ethernet Controller> port 0x4000-0x400f mem 0xd2404000-0xd2404fff,0xd2403000-0xd2403fff,0xd2400000-0xd2401fff irq 18 at device 0.0 on pci3
vmx3f0: Driver version : 0.1.0.0.
vmx3f0: Capabilities : sg jf vlan csum tso
vmx3f0: Ethernet address: 00:0c:29:2e:88:20


Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address	= 0x368
fault code		= supervisor read data, page not present
instruction pointer	= 0x20:0xffffffff80a152af
stack pointer	        = 0x28:0xffffff84cbcd8fd0
frame pointer	        = 0x28:0xffffff84cbcd9060
code segment		= base 0x0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 1667 (zabbix_agentd)
trap number		= 12
panic: page fault
cpuid = 0
KDB: stack backtrace:
#0 0xffffffff80a46df6 at kdb_backtrace+0x66
#1 0xffffffff80a0ce1e at panic+0x1ce
#2 0xffffffff80e16220 at trap_fatal+0x290
#3 0xffffffff80e16581 at trap_pfault+0x211
#4 0xffffffff80e16b34 at trap+0x344
#5 0xffffffff80dffe63 at calltrap+0x8
#6 0xffffffff80a15d75 at _sx_xlock+0x75
#7 0xffffffff81bdd44f at zfsctl_snapdir_lookup+0x16f
#8 0xffffffff80ee4e0e at VOP_LOOKUP_APV+0x6e
#9 0xffffffff80a9d098 at lookup+0x3f8
#10 0xffffffff80a9e358 at namei+0x4f8
#11 0xffffffff80ab1b64 at kern_statfs+0xa4
#12 0xffffffff80ab1e37 at sys_statfs+0x37
#13 0xffffffff80e159ca at amd64_syscall+0x5ea
#14 0xffffffff80e00147 at Xfast_syscall+0xf7
Uptime: 3m33s
Copyright (c) 1992-2013 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.2-RELEASE-p3 #0 r260900M: Mon Jan 20 07:22:46 CET 2014
    root@dev.nas4free.org:/usr/obj/nas4free/usr/src/sys/NAS4FREE-amd64 amd64
gcc version 4.2.1 20070831 patched [FreeBSD]
CPU: Intel(R) Xeon(R) CPU E5-1620 0 @ 3.60GHz (3600.00-MHz K8-class CPU)
  Origin = "GenuineIntel"  Id = 0x206d7  Family = 0x6  Model = 0x2d  Stepping = 7
  Features=0x1fa3fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,DTS,MMX,FXSR,SSE,SSE2,SS,HTT>
  Features2=0x9e982203<SSE3,PCLMULQDQ,SSSE3,CX16,SSE4.1,SSE4.2,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,HV>
  AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
  AMD Features2=0x1<LAHF>
  TSC: P-state invariant
real memory  = 34359738368 (32768 MB)
avail memory = 33176076288 (31639 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <PTLTD  	 APIC  >
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 2 package(s) x 2 core(s)
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
 cpu2 (AP): APIC ID:  2
 cpu3 (AP): APIC ID:  3
MADT: Forcing active-low polarity and level trigger for SCI
ioapic0 <Version 1.1> irqs 0-23 on motherboard
kbd1 at kbdmux0
cryptosoft0: <software crypto> on motherboard
acpi0: <INTEL 440BX> on motherboard
acpi0: Power Button (fixed)
Timecounter "HPET" frequency 14318180 Hz quality 950
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
Event timer "RTC" frequency 32768 Hz quality 0
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 1.0 on pci0
pci1: <ACPI PCI bus> on pcib1
isab0: <PCI-ISA bridge> at device 7.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel PIIX4 UDMA33 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x10c0-0x10cf at device 7.1 on pci0
ata0: <ATA channel> at channel 0 on atapci0
ata1: <ATA channel> at channel 1 on atapci0
pci0: <bridge> at device 7.3 (no driver attached)
pci0: <base peripheral> at device 7.7 (no driver attached)
vgapci0: <VGA-compatible display> port 0x10d0-0x10df mem 0xd8000000-0xdbffffff,0xd0800000-0xd0ffffff irq 16 at device 15.0 on pci0
mpt0: <LSILogic 1030 Ultra4 Adapter> port 0x1400-0x14ff mem 0xd0040000-0xd005ffff,0xd0020000-0xd003ffff irq 17 at device 16.0 on pci0
mpt0: MPI Version=1.2.0.0
pcib2: <ACPI PCI-PCI bridge> at device 17.0 on pci0
pci2: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> at device 21.0 on pci0
pci3: <ACPI PCI bus> on pcib3
pci3: <network, ethernet> at device 0.0 (no driver attached)
pcib4: <ACPI PCI-PCI bridge> at device 21.1 on pci0
pci4: <ACPI PCI bus> on pcib4
pcib5: <ACPI PCI-PCI bridge> at device 21.2 on pci0
pci5: <ACPI PCI bus> on pcib5
pcib6: <ACPI PCI-PCI bridge> at device 21.3 on pci0
pci6: <ACPI PCI bus> on pcib6
pcib7: <ACPI PCI-PCI bridge> at device 21.4 on pci0
pci7: <ACPI PCI bus> on pcib7
pcib8: <ACPI PCI-PCI bridge> at device 21.5 on pci0
pci8: <ACPI PCI bus> on pcib8
pcib9: <ACPI PCI-PCI bridge> at device 21.6 on pci0
pci9: <ACPI PCI bus> on pcib9
pcib10: <ACPI PCI-PCI bridge> at device 21.7 on pci0
pci10: <ACPI PCI bus> on pcib10
pcib11: <ACPI PCI-PCI bridge> at device 22.0 on pci0
pci11: <ACPI PCI bus> on pcib11
mps0: <LSI SAS2008> port 0x5000-0x50ff mem 0xd2500000-0xd2503fff,0xd2540000-0xd257ffff irq 19 at device 0.0 on pci11
mps0: Firmware: 16.00.00.00, Driver: 14.00.00.01-fbsd
mps0: IOCCapabilities: 1285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,HostDisc>
pcib12: <ACPI PCI-PCI bridge> at device 22.1 on pci0
pci12: <ACPI PCI bus> on pcib12
pcib13: <ACPI PCI-PCI bridge> at device 22.2 on pci0
pci13: <ACPI PCI bus> on pcib13
pcib14: <ACPI PCI-PCI bridge> at device 22.3 on pci0
pci14: <ACPI PCI bus> on pcib14
pcib15: <ACPI PCI-PCI bridge> at device 22.4 on pci0
pci15: <ACPI PCI bus> on pcib15
pcib16: <ACPI PCI-PCI bridge> at device 22.5 on pci0
pci16: <ACPI PCI bus> on pcib16
pcib17: <ACPI PCI-PCI bridge> at device 22.6 on pci0
pci17: <ACPI PCI bus> on pcib17
pcib18: <ACPI PCI-PCI bridge> at device 22.7 on pci0
pci18: <ACPI PCI bus> on pcib18
pcib19: <ACPI PCI-PCI bridge> at device 23.0 on pci0
pci19: <ACPI PCI bus> on pcib19
mps1: <LSI SAS2008> port 0x6000-0x60ff mem 0xd2600000-0xd2603fff,0xd2640000-0xd267ffff irq 16 at device 0.0 on pci19
mps1: Firmware: 16.00.00.00, Driver: 14.00.00.01-fbsd
mps1: IOCCapabilities: 1285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,HostDisc>
pcib20: <ACPI PCI-PCI bridge> at device 23.1 on pci0
pci20: <ACPI PCI bus> on pcib20
pcib21: <ACPI PCI-PCI bridge> at device 23.2 on pci0
pci21: <ACPI PCI bus> on pcib21
pcib22: <ACPI PCI-PCI bridge> at device 23.3 on pci0
pci22: <ACPI PCI bus> on pcib22
pcib23: <ACPI PCI-PCI bridge> at device 23.4 on pci0
pci23: <ACPI PCI bus> on pcib23
pcib24: <ACPI PCI-PCI bridge> at device 23.5 on pci0
pci24: <ACPI PCI bus> on pcib24
pcib25: <ACPI PCI-PCI bridge> at device 23.6 on pci0
pci25: <ACPI PCI bus> on pcib25
pcib26: <ACPI PCI-PCI bridge> at device 23.7 on pci0
pci26: <ACPI PCI bus> on pcib26
pcib27: <ACPI PCI-PCI bridge> at device 24.0 on pci0
pci27: <ACPI PCI bus> on pcib27
pcib28: <ACPI PCI-PCI bridge> at device 24.1 on pci0
pci28: <ACPI PCI bus> on pcib28
pcib29: <ACPI PCI-PCI bridge> at device 24.2 on pci0
pci29: <ACPI PCI bus> on pcib29
pcib30: <ACPI PCI-PCI bridge> at device 24.3 on pci0
pci30: <ACPI PCI bus> on pcib30
pcib31: <ACPI PCI-PCI bridge> at device 24.4 on pci0
pci31: <ACPI PCI bus> on pcib31
pcib32: <ACPI PCI-PCI bridge> at device 24.5 on pci0
pci32: <ACPI PCI bus> on pcib32
pcib33: <ACPI PCI-PCI bridge> at device 24.6 on pci0
pci33: <ACPI PCI bus> on pcib33
pcib34: <ACPI PCI-PCI bridge> at device 24.7 on pci0
pci34: <ACPI PCI bus> on pcib34
acpi_acad0: <AC Adapter> on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: model IntelliMouse, device ID 3
orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xca000-0xcafff,0xdc000-0xdffff,0xe0000-0xe7fff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
ppc0: cannot reserve I/O port range
ctl: CAM Target Layer loaded
coretemp0: <CPU On-Die Thermal Sensors> on cpu0
coretemp0: Tj(target) value 0 does not seem right.
acpi_throttle0: <ACPI CPU Throttling> on cpu0
coretemp1: <CPU On-Die Thermal Sensors> on cpu1
coretemp1: Tj(target) value 0 does not seem right.
acpi_throttle1: <ACPI CPU Throttling> on cpu1
acpi_throttle1: failed to attach P_CNT
device_attach: acpi_throttle1 attach returned 6
coretemp2: <CPU On-Die Thermal Sensors> on cpu2
coretemp2: Tj(target) value 0 does not seem right.
acpi_throttle2: <ACPI CPU Throttling> on cpu2
acpi_throttle2: failed to attach P_CNT
device_attach: acpi_throttle2 attach returned 6
coretemp3: <CPU On-Die Thermal Sensors> on cpu3
coretemp3: Tj(target) value 0 does not seem right.
acpi_throttle3: <ACPI CPU Throttling> on cpu3
acpi_throttle3: failed to attach P_CNT
device_attach: acpi_throttle3 attach returned 6
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
Timecounters tick every 10.000 msec
ipfw2 (+ipv6) initialized, divert loadable, nat loadable, default to accept, logging disabled
iSCSI boot driver version 0.2.6
cd0 at ata1 bus 0 scbus1 target 0 lun 0
cd0: <NECVMWar VMware IDE CDR10 1.00> Removable CD-ROM SCSI-0 device 
cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes)
cd0: Attempt to query device size failed: NOT READY, Medium not present
da0 at mpt0 bus 0 scbus2 target 0 lun 0
da0: <VMware Virtual disk 1.0> Fixed Direct Access SCSI-2 device 
da0: 320.000MB/s transfers (160.000MHz, offset 127, 16bit)
da0: Command Queueing enabled
da0: 4096MB (8388608 512 byte sectors: 255H 63S/T 522C)
da1 at mps0 bus 0 scbus3 target 0 lun 0
da1: <ATA Hitachi HUA72302 A840> Fixed Direct Access SCSI-6 device 
da1: 600.000MB/s transfers
da1: Command Queueing enabled
da1: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da2 at mps0 bus 0 scbus3 target 1 lun 0
da2: <ATA Hitachi HUA72302 A840> Fixed Direct Access SCSI-6 device 
da2: 600.000MB/s transfers
da2: Command Queueing enabled
da2: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da3 at mps0 bus 0 scbus3 target 2 lun 0
da3: <ATA Hitachi HUA72302 A840> Fixed Direct Access SCSI-6 device 
da3: 600.000MB/s transfers
da3: Command Queueing enabled
da3: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da4 at mps0 bus 0 scbus3 target 3 lun 0
da4: <ATA Hitachi HUA72302 A840> Fixed Direct Access SCSI-6 device 
da4: 600.000MB/s transfers
da4: Command Queueing enabled
da4: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da5 at mps0 bus 0 scbus3 target 4 lun 0
da5: <ATA Hitachi HUA72302 A840> Fixed Direct Access SCSI-6 device 
da5: 600.000MB/s transfers
da5: Command Queueing enabled
da5: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da9 at mps1 bus 0 scbus4 target 3 lun 0
da9: <ATA Hitachi HUA72302 A840> Fixed Direct Access SCSI-6 device 
da9: 600.000MB/s transfers
da9: Command Queueing enabled
da9: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da8 at mps1 bus 0 scbus4 target 2 lun 0
da8: <ATA Hitachi HUA72302 A840> Fixed Direct Access SCSI-6 device 
da8: 600.000MB/s transfers
da8: Command Queueing enabled
da8: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da7 at mps1 bus 0 scbus4 target 1 lun 0
da7: <ATA Hitachi HUA72302 A840> Fixed Direct Access SCSI-6 device 
da7: 600.000MB/s transfers
da7: Command Queueing enabled
da7: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da6 at mps1 bus 0 scbus4 target 0 lun 0
da6: <ATA Hitachi HUA72302 A840> Fixed Direct Access SCSI-6 device 
da6: 600.000MB/s transfers
da6: Command Queueing enabled
da6: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da11 at mps1 bus 0 scbus4 target 5 lun 0
da11: <ATA INTEL SSDSA2CW12 0362> Fixed Direct Access SCSI-6 device 
da11: 300.000MB/s transfers
da11: Command Queueing enabled
da11: 114473MB (234441648 512 byte sectors: 255H 63S/T 14593C)
da11: quirks=0x8<4K>
da12 at mps1 bus 0 scbus4 target 6 lun 0
da12: <ATA INTEL SSDSA2CW12 0362> Fixed Direct Access SCSI-6 device 
da12: 300.000MB/s transfers
da12: Command Queueing enabled
da12: 114473MB (234441648 512 byte sectors: 255H 63S/T 14593C)
da12: quirks=0x8<4K>
da10 at mps1 bus 0 scbus4 target 4 lun 0
da10: <ATA Hitachi HUA72302 A840> Fixed Direct Access SCSI-6 device 
da10: 600.000MB/s transfers
da10: Command Queueing enabled
da10: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
SMP: AP CPU #3 Launched!
SMP: AP CPU #1 Launched!
SMP: AP CPU #2 Launched!
Trying to mount root from ufs:/dev/ufsid/52dd563212019309 [rw]...
WARNING: / was not properly dismounted
VMware memory control driver initialized
vmx3f0: <VMware Vmxnet3 Ethernet Controller> port 0x4000-0x400f mem 0xd2404000-0xd2404fff,0xd2403000-0xd2403fff,0xd2400000-0xd2401fff irq 18 at device 0.0 on pci3
vmx3f0: Driver version : 0.1.0.0.
vmx3f0: Capabilities : sg jf vlan csum tso
vmx3f0: Ethernet address: 00:0c:29:2e:88:20
I think zabbix_agentd is the problem. MMmmmm how to fix this.

Code: Select all

Fatal trap 12: page fault while in kernel mode
cpuid = 3; apic id = 03
fault virtual address	= 0x368
fault code		= supervisor read data, page not present
instruction pointer	= 0x20:0xffffffff80a152af
stack pointer	        = 0x28:0xffffff84cbcd2fd0
frame pointer	        = 0x28:0xffffff84cbcd3060
code segment		= base 0x0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 10547 (zabbix_agentd)
trap number		= 12
panic: page fault
cpuid = 3
KDB: stack backtrace:
#0 0xffffffff80a46df6 at kdb_backtrace+0x66
#1 0xffffffff80a0ce1e at panic+0x1ce
#2 0xffffffff80e16220 at trap_fatal+0x290
#3 0xffffffff80e16581 at trap_pfault+0x211
#4 0xffffffff80e16b34 at trap+0x344
#5 0xffffffff80dffe63 at calltrap+0x8
#6 0xffffffff80a15d75 at _sx_xlock+0x75
#7 0xffffffff81bdd44f at zfsctl_snapdir_lookup+0x16f
#8 0xffffffff80ee4e0e at VOP_LOOKUP_APV+0x6e
#9 0xffffffff80a9d098 at lookup+0x3f8
#10 0xffffffff80a9e358 at namei+0x4f8
#11 0xffffffff80ab1b64 at kern_statfs+0xa4
#12 0xffffffff80ab1e37 at sys_statfs+0x37
#13 0xffffffff80e159ca at amd64_syscall+0x5ea
#14 0xffffffff80e00147 at Xfast_syscall+0xf7
Uptime: 2h13m30s

Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address	= 0x368
fault code		= supervisor read data, page not present
instruction pointer	= 0x20:0xffffffff80a152af
stack pointer	        = 0x28:0xffffff84cbcd8fd0
frame pointer	        = 0x28:0xffffff84cbcd9060
code segment		= base 0x0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 1667 (zabbix_agentd)
trap number		= 12
panic: page fault
cpuid = 0
KDB: stack backtrace:
#0 0xffffffff80a46df6 at kdb_backtrace+0x66
#1 0xffffffff80a0ce1e at panic+0x1ce
#2 0xffffffff80e16220 at trap_fatal+0x290
#3 0xffffffff80e16581 at trap_pfault+0x211
#4 0xffffffff80e16b34 at trap+0x344
#5 0xffffffff80dffe63 at calltrap+0x8
#6 0xffffffff80a15d75 at _sx_xlock+0x75
#7 0xffffffff81bdd44f at zfsctl_snapdir_lookup+0x16f
#8 0xffffffff80ee4e0e at VOP_LOOKUP_APV+0x6e
#9 0xffffffff80a9d098 at lookup+0x3f8
#10 0xffffffff80a9e358 at namei+0x4f8
#11 0xffffffff80ab1b64 at kern_statfs+0xa4
#12 0xffffffff80ab1e37 at sys_statfs+0x37
#13 0xffffffff80e159ca at amd64_syscall+0x5ea
#14 0xffffffff80e00147 at Xfast_syscall+0xf7
I think the default pkg_add -r zabbix2-agent is not OKE. But how to find a good pre-compiled one.
Top
RedAntz
experienced User
experienced User
Posts: 125
Joined: 11 Jul 2012 07:46
Location: Sydney, Australia
Status: Offline

Re: Random reboots with NAS4Free 9.2.0.1.925 on a ESXi serve

Post by RedAntz »

Please note that virtualisation is not officially supported here, so does custom setups. It is simply impossible to debug in uncontrolled environment.
Top
neptunus
experienced User
experienced User
Posts: 79
Joined: 11 Jun 2013 08:50
Status: Offline

Re: Random reboots with NAS4Free 9.2.0.1.925 on a ESXi serve

Post by neptunus »

RedAntz wrote:Please note that virtualisation is not officially supported here, so does custom setups. It is simply impossible to debug in uncontrolled environment.
I know, I'm happy with the help you have given me. Zabbix agent was the problem. The system is stable for now.

I use a zabbix server to monitor my hosts in my network. Now I need to find out how to install zabbix agend which is stable on freebsd 9.2.That is why I asked the question here.
Top
Post Reply

Return to “Install XigmaNAS”