Distributing disks in zpool across backplanes or not?

[selulven]

OK, so I have a weird question.

My new SAN server has four SAS backplanes with ports for 4 disks each. Independently, I'm organizing my zpools around pairs of mirrored vdevs. So I have a physical disk layout that's a 4x4 matrix, and I have a logical layout with groups of 4 disks.

One 4 drive zpool could either occupy one whole backplane, or it could be distributed with one disk on each backplane (or potentially 2 disks each in 2 backplanes).

My question is this: is there any benefit to thinking about disk organization across the backplanes?

If a backplane were to die, it would either take out a whole zpool, which I assume in the meantime would be very safe and very offline, or it would take out (redundant) parts of all the zpools, which would degrade all of them but keep them online.

Any performance benefit or detriment to centering traffic to a zpool on one HBA port/one backplane?

Any other considerations?

I realize this is taking it a little far.

[b0ssman]

i would put them on the same backplane.

in a mirrored setup the pool will be split if they are on a different backplane and you will have to destroy one pool before you can resilver.

[selulven]

Hi b0ssman.

Thank you for replying.

I'm not sure I understand what you mean exactly, but I'll take the recommendation to heart and put the pools on the same backplane.

I would expect if a backplane (or anything else) were to fail to take one disk out of a four disk mirrored pool, the pool would be degraded until the backplane was restored, at which point it would just be re-added? Why would there be a split?

[b0ssman]

it really depends on what you want to do.

i just said what i would do.

if you want maximum uptime go for your setup.

[selulven]

Here's a consideration that's relevant for me, as I have both SAS and SATA drives:

You shouldn't mix SATA and SAS drives on the same backplane. Basically, the controller will kick down the speed to SATA to keep compatibility, and freak out the SAS drives. It would be useless.

[selulven]

b0ssman, can you elaborate on why you think there will be a split if a backplane dies and is replaced? I don't know anything about how splits occur, I just reason that a backplane dying would be the same as a disk being removed and reinserted, and I haven't seen the pools splitting over that. Yet.

[b0ssman]

the pool continues to function when a disk is disconnected.
so if the backplane is replaced the drive contains a working set of the array it zfs will consider it a different pool.
so if you want the drive to be part of the original pool you will need to destroy the 2nd pool (and being carefull not to destroy the wrong one)
wipe the drive and then add the drive to the pool again.
for me that is to much hassle.
that is why i prefer for my pool to go down as a whole.

but if you want your pool to keep running above all then do the split.

[selulven]

Now it turns out I really can't, since apparently it would wreck the performance of my SAS drives to distribute them across backplanes with the SATA drives.

But I still don't follow. Maybe it's just my setup, but when I've been pulling and reinserting disks lately (SO much fun), I've never seen the reinserted disk show up as a separate pool. Maybe I just wasn't looking. Also, I never need to destroy any pools -- I could just online the reinserted disk and everything was fine. In what circumstance would the reinserted disk become a separate pool?