How to configure ceph as cinder backend ?

In this article I am going to show how to configure ceph as cinder backend. In my last article, I have shown the integration of ceph as glance and nova backend. This integration is almost similar to nova with some changes in configuration file.

Step 1 :  Create one pool with name of volumes with 64 PGs.

[root@ceph1 ~]# ceph osd pool create volumes 64
pool ‘volumes’ created

Step 2 : Created user vole with full access to the pool ‘volumes’ and with read, execution access to the pool ‘vms’ which is used for glance integration.

[root@ceph1 ~]# ceph auth get-or-create client.cinder mon ‘allow r’ osd ‘allow object_prefix class-read rbd_children, allow rwx pool=volumes, allow rx pool=vms’ -o /etc/ceph/ceph.client.cinder.keyring

Step 3 : Copied the keyring and auth-key to the openstack node.

[root@ceph1 ~]# scp -p /etc/ceph/ceph.client.vole.keyring root@192.168.122.245:/etc/ceph/
root@192.168.122.245’s password:
ceph.client.vole.keyring

[root@ceph1 ~]# ceph auth get-key client.vole | ssh root@192.168.122.245 tee /root/test.key

AQAePjRWG5mIJRAAXVRPRSs13ViWWpxlZFluHQ==

Step 4 : Verify on the openstack node that you are able to list the content of the pool volumes.

[root@openstack1 ceph(keystone_admin)]# rbd –user vole -p volumes ls

Step 5 : Define the secret.

[root@openstack1 ceph(keystone_admin)]# virsh secret-define –file /root/cinder1.xml
Secret 14cf1630-801b-4dd6-aef5-2bed37fd086b created

Checking the content of the cinder1.xml file. You need to create this file simply using vi editor and the uuid part you can generate with simple uuidgen command.

[root@openstack1 ceph(keystone_admin)]# virsh secret-dumpxml 14cf1630-801b-4dd6-aef5-2bed37fd086b
<secret ephemeral=’no’ private=’no’>
<uuid>14cf1630-801b-4dd6-aef5-2bed37fd086b</uuid>
<usage type=’ceph’>
<name>client.vole secret</name>
</usage>
</secret>

Set the secret value. Note : test.key contains the key of user vole (recall step 3)

[root@openstack1 ceph(keystone_admin)]# virsh secret-set-value –secret 14cf1630-801b-4dd6-aef5-2bed37fd086b –base64 $(cat /root/test.key)
Secret value set

Step 6 : Take the backup of cinder.conf file before making any change and set the default store to rbd.

[root@openstack1 ceph(keystone_admin)]# cp -p  /etc/cinder/cinder.conf /etc/cinder/cinder.conf.orig
[root@openstack1 ceph(keystone_admin)]# crudini –set /etc/cinder/cinder.conf DEFAULT enabled_backends rbd

After making all changes to configuration file, below is the list of differences between the original and modified configuration file.

[root@openstack1 ceph(keystone_admin)]# diff /etc/cinder/cinder.conf /etc/cinder/cinder.conf.orig
662c662
< enabled_backends=rbd

> enabled_backends=lvm
813c813
< debug=True

> debug=False
2877,2883d2876
<
< [rbd]
< volume_driver=cinder.volume.drivers.rbd.RBDDriver
< rbd_pool=volumes
< rbd_user=vole
< rbd_ceph_conf=/etc/ceph/ceph.conf
< rbd_secret_uuid=14cf1630-801b-4dd6-aef5-2bed37fd086b

After restart of cinder service, make sure that cinder(rbd) service is up and running.

[root@openstack1 ceph(keystone_admin)]# openstack-service restart cinder-volume

[root@openstack1 ceph(keystone_admin)]# cinder service-list
+——————+—————-+——+———+——-+—————————-+—————–+
|      Binary      |      Host      | Zone |  Status | State |         Updated_at         | Disabled Reason |
+——————+—————-+——+———+——-+—————————-+—————–+
|  cinder-backup   |   openstack1   | nova | enabled |  down | 2015-10-31T04:15:46.000000 |       None      |
| cinder-scheduler |   openstack1   | nova | enabled |   up  | 2015-10-31T04:48:16.000000 |       None      |
|  cinder-volume   | openstack1@lvm | nova | enabled |  down | 2015-10-31T04:15:46.000000 |       None      |
|  cinder-volume   | openstack1@rbd | nova | enabled |   up  | 2015-10-31T04:48:08.000000 |       None      |
+——————+—————-+——+———+——-+—————————-+—————–+

Step 7 : Creating the test volume to verify the configuration.

[root@openstack1 ceph(keystone_admin)]# cinder create –display-name=”test1″ 1
+———————+————————————–+
|       Property      |                Value                 |
+———————+————————————–+
|     attachments     |                  []                  |
|  availability_zone  |                 nova                 |
|       bootable      |                false                 |
|      created_at     |      2015-10-31T04:34:41.306901      |
| display_description |                 None                 |
|     display_name    |                test1                 |
|      encrypted      |                False                 |
|          id         | 4fbc1eb7-b2ab-4247-9f09-792021fc505c |
|       metadata      |                  {}                  |
|         size        |                  1                   |
|     snapshot_id     |                 None                 |
|     source_volid    |                 None                 |
|        status       |               creating               |
|     volume_type     |                 None                 |
+———————+————————————–+

[root@openstack1 ceph(keystone_admin)]# cinder list
+————————————–+———–+————–+——+————-+———-+————-+
|                  ID                  |   Status  | Display Name | Size | Volume Type | Bootable | Attached to |
+————————————–+———–+————–+——+————-+———-+————-+
| 4fbc1eb7-b2ab-4247-9f09-792021fc505c | available |    test1     |  1   |     None    |  false   |             |
+————————————–+———–+————–+——+————-+———-+————-+

Okay, our volume is successfully created.

Step 8 : Checking the content of the pool volumes which is configured to store the volume.

[root@openstack1 ceph(keystone_admin)]# rbd –user vole -p volumes ls
volume-4fbc1eb7-b2ab-4247-9f09-792021fc505c

[root@openstack1 ceph(keystone_admin)]# rbd –user vole -p volumes info volume-4fbc1eb7-b2ab-4247-9f09-792021fc505c
rbd image ‘volume-4fbc1eb7-b2ab-4247-9f09-792021fc505c’:
size 1024 MB in 256 objects
order 22 (4096 kB objects)
block_name_prefix: rbd_data.37cf5e5b00de
format: 2
features: layering

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s