Blaine Gardner
c9d99e01a0
mkdocs uses a markdown renderer that is hardcoded to 4 spaces per tab
for detecting indentation levels, including ordered- and
unordered-lists. Since we cannot easily change the renderer, begin using
a markdown linter in CI that will fail if official docs do not adhere to
the spacing rules.
As a starting point, the markdownlint config does not begin with the
default set of checks, which might overwhelm attempts to fix them.
Instead, focus on list-tab-spacing rules and a few other highly useful
checks.
markdownlint also has some gaps in its abilities that allow common Rook
doc issues to pass acceptance. However, it allows creating custom
linting plugins. Create 2 such linting plugins to check 2 things:
- all doc lines (except code blocks) must be aligned to a 4-space
boundary, without exception. This ensures that markdown will render
correctly with mkdocs. This unfortunately makes it possible to create
lists that are internally aligned strangely.
- admonitions must all follow the same format of
```
!!! header
body
```
For the strange lists, this is allowed and renders correctly, but it
looks strange:
```md
- first bullet
- second bullet
still second bullet
- third bullet
has a paragraph
of text inside
- last bullet
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
6.2 KiB
title
| title |
|---|
| Filesystem Mirroring |
Ceph filesystem mirroring is a process of asynchronous replication of snapshots to a remote CephFS file system. Snapshots are synchronized by mirroring snapshot data followed by creating a snapshot with the same name (for a given directory on the remote file system) as the snapshot being synchronized. It is generally useful when planning for Disaster Recovery. Mirroring is for clusters that are geographically distributed and stretching a single cluster is not possible due to high latencies.
Prerequisites
This guide assumes you have created a Rook cluster as explained in the main quickstart guide
Create the Filesystem with Mirroring enabled
The following will enable mirroring on the filesystem:
apiVersion: ceph.rook.io/v1
kind: CephFilesystem
metadata:
name: myfs
namespace: rook-ceph
spec:
metadataPool:
failureDomain: host
replicated:
size: 3
dataPools:
- name: replicated
failureDomain: host
replicated:
size: 3
preserveFilesystemOnDelete: true
metadataServer:
activeCount: 1
activeStandby: true
mirroring:
enabled: true
# list of Kubernetes Secrets containing the peer token
# for more details see: https://docs.ceph.com/en/latest/dev/cephfs-mirroring/#bootstrap-peers
# Add the secret name if it already exists else specify the empty list here.
peers:
secretNames:
#- secondary-cluster-peer
# specify the schedule(s) on which snapshots should be taken
# see the official syntax here https://docs.ceph.com/en/latest/cephfs/snap-schedule/#add-and-remove-schedules
snapshotSchedules:
- path: /
interval: 24h # daily snapshots
# The startTime should be mentioned in the format YYYY-MM-DDTHH:MM:SS
# If startTime is not specified, then by default the start time is considered as midnight UTC.
# see usage here https://docs.ceph.com/en/latest/cephfs/snap-schedule/#usage
# startTime: 2022-07-15T11:55:00
# manage retention policies
# see syntax duration here https://docs.ceph.com/en/latest/cephfs/snap-schedule/#add-and-remove-retention-policies
snapshotRetention:
- path: /
duration: "h 24"
Create the cephfs-mirror daemon
Launch the rook-ceph-fs-mirror pod on the source storage cluster, which deploys the cephfs-mirror daemon in the cluster:
kubectl create -f deploy/examples/filesystem-mirror.yaml
Please refer to Filesystem Mirror CRD for more information.
Configuring mirroring peers
Once mirroring is enabled, Rook will by default create its own bootstrap peer token so that it can be used by another cluster. The bootstrap peer token can be found in a Kubernetes Secret. The name of the Secret is present in the Status field of the CephFilesystem CR:
status:
info:
fsMirrorBootstrapPeerSecretName: fs-peer-token-myfs
This secret can then be fetched like so:
# kubectl get secret -n rook-ceph fs-peer-token-myfs -o jsonpath='{.data.token}'|base64 -d
eyJmc2lkIjoiOTFlYWUwZGQtMDZiMS00ZDJjLTkxZjMtMTMxMWM5ZGYzODJiIiwiY2xpZW50X2lkIjoicmJkLW1pcnJvci1wZWVyIiwia2V5IjoiQVFEN1psOWZ3V1VGRHhBQWdmY0gyZi8xeUhYeGZDUTU5L1N0NEE9PSIsIm1vbl9ob3N0IjoiW3YyOjEwLjEwMS4xOC4yMjM6MzMwMCx2MToxMC4xMDEuMTguMjIzOjY3ODldIn0=
Import the token in the Destination cluster
The decoded secret must be saved in a file before importing.
# ceph fs snapshot mirror peer_bootstrap import <fs_name> <token file path>
See the CephFS mirror documentation on how to add a bootstrap peer.
Further refer to CephFS mirror documentation to configure a directory for snapshot mirroring.
Verify that the snapshots have synced
To check the mirror daemon status, please run the following command from the toolbox pod.
For example :
# ceph fs snapshot mirror daemon status | jq
[
{
"daemon_id": 906790,
"filesystems": [
{
"filesystem_id": 1,
"name": "myfs",
"directory_count": 1,
"peers": [
{
"uuid": "a24a3366-8130-4d55-aada-95fa9d3ff94d",
"remote": {
"client_name": "client.mirror",
"cluster_name": "91046889-a6aa-4f74-9fb0-f7bb111666b4",
"fs_name": "myfs"
},
"stats": {
"failure_count": 0,
"recovery_count": 0
}
}
]
}
]
}
]
Please refer to the --admin-daemon socket commands from the CephFS mirror documentation to verify mirror status and peer synchronization status and run the commands from the rook-ceph-fs-mirror pod:
# kubectl -n rook-ceph exec -it deploy/rook-ceph-fs-mirror -- bash
Fetch the ceph-client.fs-mirror daemon admin socket file from the /var/run/ceph directory:
# ls -lhsa /var/run/ceph/
# ceph --admin-daemon /var/run/ceph/ceph-client.fs-mirror.1.93989418120648.asok fs mirror status myfs@1
{
"rados_inst": "X.X.X.X:0/2286593433",
"peers": {
"a24a3366-8130-4d55-aada-95fa9d3ff94d": {
"remote": {
"client_name": "client.mirror",
"cluster_name": "91046889-a6aa-4f74-9fb0-f7bb111666b4",
"fs_name": "myfs"
}
}
},
"snap_dirs": {
"dir_count": 1
}
}
For getting peer synchronization status:
# ceph --admin-daemon /var/run/ceph/ceph-client.fs-mirror.1.93989418120648.asok fs mirror peer status myfs@1 a24a3366-8130-4d55-aada-95fa9d3ff94d
{
"/volumes/_nogroup/subvol-1": {
"state": "idle",
"last_synced_snap": {
"id": 4,
"name": "snap2"
},
"snaps_synced": 0,
"snaps_deleted": 0,
"snaps_renamed": 0
}
}