LVM on top of linux zfs to use Openstack with nova-volume

LVM on top of linux zfs to use Openstack with nova-volume

LVM on top of linux zfs to use Openstack with nova-volume
Did you ever wanted to know how to setup LVM on top of linux zfs to use Openstack with nova-volume? In this article i will describe this in a few easy steps based on my earlier mentioned how to about zfs on linux.

A common problem when using ZFS as storage for openstack is the lack of plugins that will cover as interface between nova-volume/cinder-volume and the block storage types. The volume service needs an LVM Volume group called nova-volumes as default.

In this how-to i will explain you step by step a workaround how to create an block storage device with zfs and get it into lvm.



Create a ZFS filesystem you will later use as LVM Group:

zfs create -V 500G storage/your_new_volume

The -V switch defines the size of the new created file-system. Change “storage” to you pool name.

Lets see how it looks like:

zfs list
Command Output:
NAME                            USED  AVAIL  REFER  MOUNTPOINT
storage/your_new_volume   516G  1.78T    76K  -

There you see your new created file-system without an mountpoint.

We now need to get its real device path:

file /dev/zvol/storage/your_new_volume
Command output: 
/dev/zvol/storage/your_new_volume: symbolic link to `../../zd0'

So our real device path is: /dev/zd0

Let’s do the magic part:

losetup /dev/loop2 /dev/zd0

And create a partition on that device:

fdisk /dev/loop2
(and hit: n p 1 ENTER ENTER t 8e w)

Finally we can create our LVM volume group:

pvcreate /dev/loop2
vgcreate nova-volumes /dev/loop2


We now need to make clear that the loop device gets mounted on the next boot:

Modify the file called /etc/rc.local in your preferred editor and enter following:

exec losetup /dev/loop2 /dev/zd0

And that’s it!

Rating: 5.0/5. From 2 votes.
Please wait...


Jules is the owner and author of ISPIRE.ME. He's a Linux System Engineer, Tech fanatic and an Open Source fan.

You may also like...

7 Responses

  1. Lorin Hochstein Lorin Hochstein says:

    Have you tried using the native ZFS backend for cinder? It’s called the SolarisISCSIDriver. I’m not sure whether it actually works for ZFS for Linux or if it depends on some pieces that are Solaris-specific:

    No votes yet.
    Please wait...
    • Jules Jules says:

      Hi Lorin,

      yes i did tried this driver some times before but it was very buggy and not reliable enough.
      I’m gonna check if that have changed today.

      No votes yet.
      Please wait...
      • Lorin Hochstein Lorin Hochstein says:

        If you do encounter bugs, please let me know. I’d like to add a section in the OpenStack documentation about running the ZFS backend with Linux, but I have no idea whether it actually works or not.

        Rating: 5.0/5. From 1 vote.
        Please wait...
        • rkaw92 rkaw92 says:

          w00t, thread necro!
          If you’re still interested in the Solaris iSCSI SAN driver, I can confirm that its quality in upstream is indeed bad.
          However, you should be able to use the fixed drivers from our cinder fork as a temporary solution:

          (Specifically, we have patched the san and the solaris drivers. Both were buggy and both are used when connecting to ZFS hosts such as Solaris 11 and OpenIndiana. Note that they require SunOS-specific tools, so they will not work on Linux since it lacks STMF and RBAC, among others. Tested on OpenIndiana 151a8 only. Consult a doctor or read the brochure before use. May contain Pythons.)

          We plan to push the updated drivers to mainstream – trying to make sense of OpenStack’s code review / patch system at the moment.

          Rating: 5.0/5. From 1 vote.
          Please wait...
  2. Blogs Blogs says:

    And just what is the point of this? Would you put this into production? I wouldn’t. ZFS on Linux? Stop kidding yourself. For ZFS in production there is only Solaris 11, so you have support just in case it goes belly up.

    No votes yet.
    Please wait...
    • Jules Jules says:

      ZFSonLinux is a Native port of Solaris version. It supports the same Pool features, so i see no reason why not to try and use it. Since im working with it, it works flawless on my servers and if you ask me it doesn’t make really no difference which OS it’s running on if the port translation is implented well like the project maintainer done it. Btrfs which comes from Oracle and was praised as ZFS alternative isn’t even suitable (buggy/instable/crap) although it was exclusive intended for linux. So there are no alternatives but since zfsonlinux is running that good i don’t really care and need no alternatives.
      Btw. Not just Solaris 11 supports production ready ZFS and if you ask me, solaris isn’t solaris anymore since SUN was bought from Oracle.
      There are also other great OSes which have production ready ZFS support like: FreeBSD which i preffer, OpenSolaris, Illumos, and SmartOS (Illumos fork).

      Rating: 5.0/5. From 1 vote.
      Please wait...
    • Jules Jules says:

      Btw: ZFSOnLinux has just reached the “wide scale deployment” status.
      Which means it’s not experimental anymore.

      Rating: 5.0/5. From 1 vote.
      Please wait...

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Translate »