2016-09-21

Ceph常用命令

#创建池
$ceph osd pool create <poolname> <int[0-]> {<int[0-]>} {replicated|erasure} {<int>}

#[123[ 4]]   auid 1234  crush rule 4
$rados mkpool <pool-name> [123[ 4]]

$rados lspools
$rados rmpool <pool-name> [<pool-name> --yes-i-really-really-mean-it]

#创建卷
$rbd create [--order <bits>] [--image-features <features>] [--image-shared]
$rbd create -s 4096 volumes/vol1

#查看卷信息
$rbd info volumes/vol1
rbd image 'vol1':
        size 4096 MB in 1024 objects
        order 22 (4096 kB objects)
        block_name_prefix: rb.0.2a44.238e1f29
        format: 1

#映射卷为块设备
$rbd map volumes/vol1
/dev/rbd0

#取消映射
$rbd unmap /dev/rbd0 

#取消映射
$rbd unmap volumes/vol1

#调整大小
$rbd resize -s 5120 volumes/vol1
Resizing image: 100% complete...done.

$rbd resize -s 4096 volumes/vol1 --allow-shrink
Resizing image: 100% complete...done.

#显示池下所有卷
$rbd ls volumes
vol1
volume-e2a437ef-7eb5-433b-90b4-40e60c353f12

#删除卷
$rbd rm volumes/vol1
Removing image: 100% complete...done.

#卷导出导入
$rbd export volumes/vol1 output.file
Exporting image: 100% complete...done.

$rbd import output.file volumes/vol2
Importing image: 100% complete...done.

通过配置CRUSH实现不同配置的池

#导出CRUSH并转换为可读文件
$ceph osd getcrushmap -o cluster_crush_map
got crush map from osdmap epoch 26

#转化
crushtool -d cluster_crush_map -o cluster_crush_map.txt

配置示例

# begin crush map
tunable choose_local_tries 0
tunable choose_local_fallback_tries 0
tunable choose_total_tries 50
tunable chooseleaf_descend_once 1
tunable straw_calc_version 1

# devices
device 0 osd.0
device 1 osd.1
device 2 osd.2

# types
type 0 osd
type 1 host
type 2 chassis
type 3 rack
type 4 row
type 5 pdu
type 6 pod
type 7 room
type 8 datacenter
type 9 region
type 10 root

# buckets
host node1 {
        id -2           # do not change unnecessarily
        # weight 0.010
        alg straw
        hash 0  # rjenkins1
        item osd.0 weight 0.010
}
host node2 {
        id -3           # do not change unnecessarily
        # weight 0.010
        alg straw
        hash 0  # rjenkins1
        item osd.1 weight 0.010
}
host node3 {
        id -4           # do not change unnecessarily
        # weight 0.010
        alg straw
        hash 0  # rjenkins1
        item osd.2 weight 0.010
}
root default {
        id -1           # do not change unnecessarily
        # weight 0.030
        alg straw
        hash 0  # rjenkins1
        item node1 weight 0.010
        item node2 weight 0.010
        item node3 weight 0.010
}

# rules
rule replicated_ruleset {
        ruleset 0
        type replicated
        min_size 1
        max_size 10
        step take default
        step chooseleaf firstn 0 type host
        step emit
}

# end crush map

修改配置,注意新建条目的id,保持不冲突

  • 修改可读配置文件,删除原host中部分item osd
  • 新建新的host把之前删除的osd放入到新建host中
  • 新建新的root把新建的host放入其中
  • 新建rule把step take参数换成新建的root

导入新的CRUSH配置

#将可读配置转换
$crushtool -c cluster_crush_map.txt -o new_map

#导入
$ceph osd setcrushmap -i new_map 
set crush map

查询相关

#查询池的规则集合
$ceph osd pool get volumes crush_ruleset 
crush_ruleset: 0

#设置池规则集合
$ceph osd pool set volumes crush_ruleset 0
set pool 2 crush_ruleset to 0

#显示树结构
$ceph osd tree
ID WEIGHT  TYPE NAME      UP/DOWN REWEIGHT PRIMARY-AFFINITY 
-1 0.02998 root default                                     
-2 0.00999     host node1                                   
 0 0.00999         osd.0       up  1.00000          1.00000 
-3 0.00999     host node2                                   
 1 0.00999         osd.1       up  1.00000          1.00000 
-4 0.00999     host node3                                   
 2 0.00999         osd.2       up  1.00000          1.00000