Sorry we lost some posts because of database errors!

*New 12.1 series Release:
2020-09-01: XigmaNAS - released

*New 11.4 series Release:
2020-08-27: XigmaNAS - released!

We really need "Your" help on XigmaNAS translations. Please help today!

Producing and hosting XigmaNAS costs money. Please consider donating for our project so that we can continue to offer you the best.
We need your support! eg: PAYPAL

Alternative OS on EMC VNX SP

Green PC, UPS, Power saving, etc.
Forum rules
Set-Up GuideFAQsForum Rules
Post Reply
Posts: 1
Joined: 04 Dec 2017 07:50
Status: Offline

Alternative OS on EMC VNX SP


Post by oddballracing »

Hi all,

I've been a long time user of FreeBSD-based appliance packages (pfSense, FreeNAS->NAS4Free->Xigmanas, etc). I am thoroughly impressed with these and their ability to turn mostly anything into a puposeful appliance intended by the package. I run pfSense and NAS4Free with no stability concerns at all, most of my units are sitting at 2.5+ yeats uptime and humming along supporting a reasonably large virtualized datacenter.

Abit of background on this:

I've been pushing towards a new generation of hardware this year in hopes of improving performance of Windows 10 and CentOS VMs, from both a processor and storage perspective, and have been finding many reasonable prices on used servers and storage equipment. In my seatches i have come across some EMC storage units with Xeon-based storage processors (controllers, for those not familiar with EMC lingo). All of these units found at local recyclers have all their disks removed (yes, even including the OE/OS disks in slots 1-4), so they are effectively useless for anything but parts at that point. This got me thinking - They are X86-based, support FC, SAS, 10GbE with the appropriate modules, and theOE operating system is based upon Windows storage server 2008 R2 (license sticker is even on the DPE (managed storage enclosure) itself. At the price i snagged my VNX550 15x3.5" DPE with 14 caddies, the processors (Xeon E5606) and memory 6 x 4Gb DDR3 12800E alone was owrth what i paid. I just tore into this thing last night and have confirmed the following details so far, i will update as i get more info.

SP Hardware:
Processor: Intel Xeon E5606 ... l-qpi.html
Memory: 3 x DDR3 SDRAM slots, my VNX5500 SPs are loaded with 3x4GB PC3-12800E each (Hynix)
Chipset: Intel 5500 "Tylersburg" IOH ... o-hub.html + ICH 10 (82801J)
FC (Mezzanine): PMC "Tachyon" QE8 PM8032 4x8Gb Fiber Channel Controller
SAS (Mezzanine): PMC SPC 8x6G PM8001(C) (Adaptec 6H controllers use the same PCIe/SAS controller)
LAN: 2 x Broadcom "Tigon3" BCM95751 (5751) Gigabit PCIe Ethernet
SAS Expander: PMC SXP 36x6G SEC SAS Expander
USB: supported by IC, no discoverable headers
RS-232: Micro-D 9-pin x2 (UART A/B) - Jerry rigged console cable does work, i have verified it with a null modem upstream cable, if you have an EMC official console cable, you'll save yourself this headache.

So far i have managed to gain console access and observe some maintenance menus of the unit, and after discovering a menu option that allows PXE boot, i got to work with some basic tests of this (jsut to see if singed boot images would be required with this unit, and at least on the PXE side, that does not appear to be the case.

First attempt confirmed loading of pxelinux.0 and a basic menu i had configured.

Second attempt got me to load a vmlinuz and initrd, but i had forgotten to set a serial console, so i got no further output.

Third attempt, i successfully loaded and began booting CentOS 7 (netinstall) image over PXE (there are still no disks in this unit), but encountered a kernel panic after the following in the boot messages:

Code: Select all

         Starting dracut initqueue hook...

[   49.908341] pm80xx pm8001_chip_soft_rst 1094:TIMEOUT : ToggleVal 0x4,MSGU_SCRATCH_PAD1 = 0xb

[   50.009339] pm80xx pm8001_chip_soft_rst 1098:SCRATCH_PAD0 value = 0x9000e600

[   50.093659] pm80xx pm8001_chip_soft_rst 1102:SCRATCH_PAD2 value = 0x20c

[   50.172787] pm80xx pm8001_chip_soft_rst 1106:SCRATCH_PAD3 value = 0x7

[   50.249875] pm80xx pm8001_chip_init 658:Firmware is not ready!

[   50.319716] pm80xx pm8001_pci_probe 881:chip_init failed [ret: -16]

[   50.395822] pm80xx: probe of 0000:30:00.0 failed with error -16

[   50.470368] 8021q: 802.1Q VLAN Support v1.8

[   50.576939] IPv6: ADDRCONF(NETDEV_UP): enp251s0: link is not ready

[   53.137040] dracut-initqueue[771]: Warning: No carrier detected on interface enp251s0

[   68.601421] dracut-initqueue[[   81.566707] BUG: unable to handle kernel paging request at 0000000000070048

[   81.667642] IP: [<ffffffff9ac6e0b8>] dup_fd+0x138/0x2a0

[   81.730447] PGD 8000000326edb067 PUD 326f4c067 PMD 0

[   81.791592] Oops: 0002 [#1] SMP

[   81.830691] Modules linked in: 8021q garp mrp stp crct10dif_pclmul crct10dif_common ata_generic pata_acpi crc32_pclmul pm80xx crc32c_intel llc ghash_clmulni_intel tg3 libsas aesni_intel scsi_transport_sas ptp ata_piix glue_helper ablk_helper cryptd serio_raw libata pps_core sunrpc xts lrw gf128mul dm_crypt dm_round_robin dm_multipath dm_snapshot dm_bufio dm_mirror dm_region_hash dm_log dm_zero dm_mod linear raid10 raid456 async_raid6_recov async_memcpy async_pq raid6_pq libcrc32c async_xor xor async_tx raid1 raid0 iscsi_boot_sysfs iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi squashfs cramfs

[   82.477674] CPU: 0 PID: 1479 Comm: ifup Not tainted 3.10.0-1127.el7.x86_64 #1

[   82.563138] Hardware name: Intel Corporation Tylersburg & ICH9/ICH10 Chipset/GreenCity/Silkcity CRB, BIOS 6.00 05/10/2013

[   82.694361] task: ffffa110e5bb5230 ti: ffffa110ee310000 task.ti: ffffa110ee310000

[   82.783990] RIP: 0010:[<ffffffff9ac6e0b8>]  [<ffffffff9ac6e0b8>] dup_fd+0x138/0x2a0

[   82.875916] RSP: 0018:ffffa110ee313df0  EFLAGS: 00010202

[   82.939558] RAX: ffffa110e6fe9000 RBX: ffffa10f3b239e00 RCX: 0000000000000002

[   83.025033] RDX: 0000000000000100 RSI: ffffa110e6cb50a0 RDI: 0000000000000000

[   83.110505] RBP: ffffa110ee313e28 R08: 0000000000070010 R09: ffffa10ddf04f800

[   83.195976] R10: 0000000000000100 R11: 00000000000000ff R12: ffffa110ee107240

[   83.281452] R13: ffffa110ee107200 R14: ffffa110ee313e7c R15: ffffa110e6ea8580

[   83.366926] FS:  00007fd7a4c34740(0000) GS:ffffa110ffc00000(0000) knlGS:0000000000000000

[   83.463840] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033

[   83.532672] CR2: 0000000000070048 CR3: 000000032e1b6000 CR4: 00000000000207f0

[   83.618148] Call Trace:

[   83.647475]  [<ffffffff9aa99d71>] copy_process+0xc41/0x1a70

[   83.714228]  [<ffffffff9aa9ad51>] do_fork+0x91/0x330

[   83.773702]  [<ffffffff9b192e09>] ? system_call_after_swapgs+0x96/0x13a

[   83.852931]  [<ffffffff9b192e15>] ? system_call_after_swapgs+0xa2/0x13a

[   83.932167]  [<ffffffff9b192e09>] ? system_call_after_swapgs+0x96/0x13a

[   84.011404]  [<ffffffff9b192e15>] ? system_call_after_swapgs+0xa2/0x13a

[   84.090641]  [<ffffffff9b192e09>] ? system_call_after_swapgs+0x96/0x13a

[   84.169875]  [<ffffffff9aa9b076>] SyS_clone+0x16/0x20

[   84.230392]  [<ffffffff9b1932b4>] stub_clone+0x44/0x70

[   84.291955]  [<ffffffff9b192ed2>] ? system_call_fastpath+0x25/0x2a

[   84.365990] Code: ff fa ff ff 44 8b 55 cc 4c 8b 45 d0 4d 8b 4f 08 45 85 d2 49 8b 40 08 74 57 45 8d 5a ff 31 c9 44 89 da 48 83 c2 01 eb 15 0f 1f 00 <f0> 49 ff 40 38 4d 89 04 c9 48 83 c1 01 48 39 d1 74 2e 4c 8b 04

[   84.605357] RIP  [<ffffffff9ac6e0b8>] dup_fd+0x138/0x2a0

[   84.669203]  RSP <ffffa110ee313df0>

[   84.711006] CR2: 0000000000070048

[   84.750764] ---[ end trace d50d1aed4ae1f5aa ]---

[   84.806106] Kernel panic - not syncing: Fatal exception

771]: Warning: N[   84.868706] Kernel Offset: 0x19a00000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
Fourth attempt was with the PCI riser cage removed, both FlexSlot modules removed, and the mezzanine card (containing the PM8001 chip) removed. I successfully dropped to a dracut shell after i had mucked up a boot option for the installer (apparently my FTP path was unsatisfactory for dracut to use to start anaconda for the install).

Code: Select all

[   80.800832] dracut-initqueue[791]: Warning: Invalid value for 'inst.repo': inst.repo=

[  257.875437] dracut-initqueue[791]: Warning: dracut-initqueue timeout - starting timeout scripts

[  258.391778] dracut-iWarning: /dev/root does not exist

Generating "/run/initramfs/rdsosreport.txt"

Entering emergency mode. Exit the shell to continue.

Type "journalctl" to view system logs.

You might want to save "/run/initramfs/rdsosreport.txt" to a USB stick or /boot

after mounting them and attach it to a bug report.

All I can say is that this is enough for me to continue messign around with this thing, and I cannot seem to find any info on the internet about someone else trying to crack one of these units open. I suspect that being that majority of the units out there are runnign happily and are not worth messing with, or they are prohibitively expensive for this kind of testing. I consider myself lucky for getting a (for the most part) functional hardware assembly for such a low price. I have nothign to lose with this thing, so if i completely bungle it trying to make it run FreeBSD/FreeNAS/XigmaNAS or Linux or Windows, i would feel no pain, just pull the CPU and memory, and fill some empty sockets and slots in some IBM servers.

This leads me to the question:

Is anyone else interested in seeing and EMC Clariion or VNX unit adapted to run FreeNas or XigmaNAS, or even FreeBSD in the least? or am i just wasting my time, do you think?

Let me know, because if anyone is interested, i'd gladly keep the community updated as i progress through turnign this proprietary product shrouded in vendor-only supported "service calls" into yet another piece of iron that can run an OSS/Free software package.


Post Reply

Return to “Other hardware”