ceph

1
1. Check or watch cluster health: ceph status || ceph -w
If you want to quickly verify that your cluster is operating normally, use ceph status to get a birds-eye view of cluster status (hint: typically, you want your cluster to be active + clean). You can also watch cluster activity in real-time with ceph -w; you'll typically use this when you add or remove OSDs and want to see the placement groups adjust.

2. Check cluster usage stats: ceph df
To check a cluster’s data usage and data distribution among pools, use ceph df. This provides information on available and used storage space, plus a list of pools and how much storage each pool consumes. Use this often to check that your cluster is not running out of space.

3. Check placement group stats: ceph pg dump
When you need statistics for the placement groups in your cluster, use ceph pg dump. You can get the data in JSON as well in case you want to use it for automatic report generation.

4. View the CRUSH map: ceph osd tree
Need to troubleshoot a cluster by identifying the physical data center, room, row and rack of a failed OSD faster? Use ceph osd tree, which produces an ASCII art CRUSH tree map with a host, its OSDs, whether they are up and their weight.

5. Create or remove OSDs: ceph osd create || ceph osd rm
Use ceph osd create to add a new OSD to the cluster. If no UUID is given, it will be set automatically when the OSD starts up. When you need to remove an OSD from the CRUSH map, use ceph osd rm with the UUID.

6. Create or delete a storage pool: ceph osd pool create || ceph osd pool delete
Create a new storage pool with a name and number of placement groups with ceph osd pool create. Remove it (and wave bye-bye to all the data in it) with ceph osd pool delete.

7. Repair an OSD: ceph osd repair
Ceph is a self-repairing cluster. Tell Ceph to attempt repair of an OSD by calling ceph osd repair with the OSD identifier.

8. Benchmark an OSD: ceph tell osd.* bench
Added an awesome new storage device to your cluster? Use ceph tell to see how well it performs by running a simple throughput benchmark. By default, the test writes 1 GB in total in 4-MB increments.

9. Adjust an OSD’s crush weight: ceph osd crush reweight
Ideally, you want all your OSDs to be the same in terms of thoroughput and capacity...but this isn't always possible. When your OSDs differ in their key attributes, use ceph osd crush reweight to modify their weights in the CRUSH map so that the cluster is properly balanced and OSDs of different types receive an appropriately-adjusted number of I/O requests and data.

10. List cluster keys: ceph auth list
Ceph uses keyrings to store one or more Ceph authentication keys and capability specifications. The ceph auth list command provides an easy way to to keep track of keys and capabilities

-=DRAFT=-

List des auth pour copier la clef

1
root@cephmngnt-01:~# ceph auth list
root@cephmngnt-01:~# cat /etc/ceph/ceph.client.admin.keyring 
root@opennebula-01:~# cat /etc/ceph/ceph.client.admin.keyring

Check info disk

1
root@kvmserver-01:~# cat /etc/libvirt/qemu/client-bdd01.xml

List des disks

1
root@opennebula-01:~# rbd -p opennebula ls
root@opennebula-01:~# qemu-img info rbd:opennebula/one-666-1664-0

Rm et cp

1
root@opennebula-01:~# rbd rm opennebula/one-666-1664-0
root@opennebula-01:~# rbd cp libvirtpool/client-bdd01 opennebula/one-666-1664-0

Jimmy’s TIPS

1
- Visualisation de la configuration des OSD :
ceph -n osd.ID --show-config

- Changement de la taille du réplica et du min_size
ceph osd pool set POOOL size 2
ceph osd pool set POOOL min_size 2

- Augmentation du nombre de Placement Group :
ceph osd pool set NOM_DU_POOL pg_num 64
ceph osd pool set NOM_DU_POOL pgp_num 64

- Visualisation des performances de l'OSD
ceph daemon osd.ID perf dump

- Historique des dernières opérations (I/O) de l'OSD :
ceph daemon osd.ID dump_historic_ops

- Modification de la crush Map :
Cf slide

- Voir l'ensemble de PG
ceph pg dump

- Voir les informations lié à un PH
ceph pg ID query

- Sortir un osd :
ceph osd out ID

- Ajouter un osd déjà existant :
ceph osd in ID

- Mise en place de la maintenance et désactivation de la maintenance :
ceph osd set noout
ceph osd unset noout

- Mirroring RBD
http://docs.ceph.com/docs/jewel/rbd/rbd-mirroring/

- Process de MAJ
http://docs.ceph.com/docs/master/releases