CEPH: How to Restart an Install, or How to Reset a Cluster


Hey, are you installing CEPH in your test lab and you screwed it up, or something has gone wrong with your test cluster and you want to start over? Well, the instructions on how to do this are below.

Note that most of this process is actually well documented, but I added in a few extra steps good to run if you want to start from scratch without having to reinstall an OS.

First remove all CEPH rpms from your CEPH hosts, this includes Monitor nodes and OSD nodes. Note that I am in /root/ceph-deploy on my monitor/admin server.  If you have separate admin and monitor nodes then run these commands from your admin node.

# ceph-deploy purge mon01 osd01 osd02 osd03

Now purge all config files.

# ceph-deploy purgedata mon01 osd01 osd02 osd03

Now remove all keys

# ceph-deploy forgetkeys

Now delete any remaining files or keys from  /root/ceph-deploy, if there are any files in here that you may want to keep, just copy them to /root.

# rm -rf /root/ceph-deploy/*

Now remove Calamari. Note: that in my test lab I am running Calamari on my one MON node which also happens to be my admin node, so I run this command there

#yum remove ceph-deploy calamari-server calamari-clients

You can now start over. I am installing Red Hat Ceph 1.3, so I am using the instructions here.

You might also want to uninstall salt from your servers. If that’s the case just look for any of the packages below and rip them out. Your versions will vary.

  • salt-2015.5.3-4.el7.noarch
  • salt-minion-2015.5.3-4.el7.noarch
  • salt-master-2015.5.3-4.el7.noarch
Print Friendly