nvmeof: use .nvmeof pool for gateway and remove pool field from CRD

Changes: - Use a dedicated .nvmeof pool (via CephBlockPool CR named builtin-nvmeof) for the NVMe-oF gateway internal state. - Use a separate nvmeof pool for the StorageClass data. - Remove the pool field from the CephNVMeOFGateway CRD. The gateway now always uses the .nvmeof pool, hardcoded in the operator. - Add .nvmeof to the allowed CephBlockPool name overrides. - Create a production example nvmeof.yaml (instances: 2, replicas: 3) and a CI-only nvmeof-test.yaml (instances: 1, replicas: 1). - Update documentation and CI test script accordingly. Signed-off-by: Oded Viner <oviner@redhat.com> (cherry picked from commit ec502c3f80)
2026-07-21 18:14:09 +00:00
1 parent 9b826b29a4
commit 9ee0d2fcf4
@@ -237,7 +237,7 @@ flags are present on RBD volumes.
 * `name`: The name of the Ceph pool is based on the `metadata.name` of the CephBlockPool CR. Some built-in Ceph pools
    require names that are incompatible with K8s resource names. These special pools can be configured
    by setting this `name` to override the name of the Ceph pool that is created instead of using the `metadata.name` for the pool.
-    Overriding only the following pool names is supported: `.nfs`, `.mgr`, and `.rgw.root`. See the example
+    Overriding only the following pool names is supported: `.nfs`, `.mgr`, `.nvmeof`, and `.rgw.root`. See the example
    [builtin mgr pool](https://github.com/rook/rook/blob/master/deploy/examples/pool-builtin-mgr.yaml).
 * `application`: The type of application set on the pool. By default, Ceph pools for CephBlockPools will be `rbd`,
    CephObjectStore pools will be `rgw`, and CephFilesystem pools will be `cephfs`.
@@ -4571,17 +4571,6 @@ int
 </tr>
 <tr>
 <td>
-<code>pool</code><br/>
-<em>
-string
-</em>
-</td>
-<td>
-<p>Pool is the RADOS pool where NVMe-oF configuration is stored</p>
-</td>
-</tr>
-<tr>
-<td>
 <code>group</code><br/>
 <em>
 string
@@ -10869,17 +10858,6 @@ int
 </tr>
 <tr>
 <td>
-<code>pool</code><br/>
-<em>
-string
-</em>
-</td>
-<td>
-<p>Pool is the RADOS pool where NVMe-oF configuration is stored</p>
-</td>
-</tr>
-<tr>
-<td>
 <code>group</code><br/>
 <em>
 string
@@ -28,59 +28,47 @@ This guide assumes a Rook cluster as explained in the [Quickstart Guide](../../G

 - **Ceph Version**: Ceph v20 (Tentacle) or later

-## Step 1: Create a Ceph Block Pool
-
-Before creating the NVMe-oF gateway, you need to create a CephBlockPool that will be used by the gateway:
-
-```yaml
-apiVersion: ceph.rook.io/v1
-kind: CephBlockPool
-metadata:
-  name: nvmeof
-  namespace: rook-ceph
-spec:
-  failureDomain: host
-  replicated:
-    size: 3
-```
-
-Create the pool:
-
-```console
-kubectl create -f deploy/examples/csi/nvmeof/nvmeof-pool.yaml
-```
-
-## Step 2: Create the NVMe-oF Gateway
+## Step 1: Create the NVMe-oF Gateway

 The `CephNVMeOFGateway` CRD manages the NVMe-oF gateway infrastructure. The operator will automatically create the following resources:

 - **Service**: One per gateway instance for service discovery
 - **Deployment**: One per gateway instance running the NVMe-oF gateway daemon

-Create the gateway:
+The gateway also requires an internal `.nvmeof` pool for its state. This pool is defined as a
+CephBlockPool CR with `spec.name: .nvmeof`.
+
+Create the gateway and the `.nvmeof` pool:

 ```yaml
 apiVersion: ceph.rook.io/v1
+kind: CephBlockPool
+metadata:
+  name: builtin-nvmeof
+  namespace: rook-ceph
+spec:
+  name: .nvmeof
+  failureDomain: host
+  replicated:
+    size: 3
+---
+apiVersion: ceph.rook.io/v1
 kind: CephNVMeOFGateway
 metadata:
  name: nvmeof
  namespace: rook-ceph
 spec:
-  # Container image for the NVMe-oF gateway daemon
-  image: quay.io/ceph/nvmeof:1.5
-  # Pool name that will be used by the NVMe-oF gateway
-  pool: nvmeof
  # ANA (Asymmetric Namespace Access) group name
  group: group-a
-  # Number of gateway instances to run
-  instances: 1
+  # Number of gateway instances to run (2 recommended for HA)
+  instances: 2
  hostNetwork: false
 ```

 Apply the gateway configuration:

 ```console
-kubectl create -f deploy/examples/nvmeof-test.yaml
+kubectl create -f deploy/examples/nvmeof.yaml
 ```

 Verify the gateway is running:
@@ -96,7 +84,7 @@ NAME                                         READY   STATUS    RESTARTS   AGE
 rook-ceph-nvmeof-nvmeof-a-85844ff6b8-4r8gj   1/1     Running   0          91s
 ```

-## Step 3: Deploy the NVMe-oF CSI Driver via CSI Operator
+## Step 2: Deploy the NVMe-oF CSI Driver via CSI Operator

 The NVMe-oF CSI driver is deployed via the ceph-csi operator.

@@ -120,6 +108,26 @@ rook-ceph.nvmeof.csi.ceph.com-ctrlplugin-d9d77fb7c-kkl28   5/5     Running   0
 rook-ceph.nvmeof.csi.ceph.com-nodeplugin-xvt5g              2/2     Running   0          60s
 ```

+## Step 3: Create the Data Pool
+
+Create a CephBlockPool that will be used by the StorageClass to store data:
+
+```yaml
+apiVersion: ceph.rook.io/v1
+kind: CephBlockPool
+metadata:
+  name: nvmeof
+  namespace: rook-ceph
+spec:
+  failureDomain: host
+  replicated:
+    size: 3
+```
+
+```console
+kubectl create -f deploy/examples/csi/nvmeof/nvmeof-pool.yaml
+```
+
 ## Step 4: Create the StorageClass

 Create a StorageClass that uses the NVMe-oF CSI driver.
@@ -292,22 +300,9 @@ sudo mount /dev/nvmeXnY /mnt/nvmeof

 ## High Availability

-For production deployments, configure multiple gateway instances for high availability:
-
-1. **Increase Gateway Instances**: Set `instances: 2` or higher in the `CephNVMeOFGateway` spec
-2. **Update StorageClass Listeners**: Add all gateway deployment hostnames to the `listeners` array
-3. **Load Balancing**: Each gateway instance has its own Service; list all of them to support multipath/HA
-
-Example with multiple instances:
-
-```yaml
-spec:
-  instances: 2
-  # ... other settings
-```
-
-Then update the StorageClass `listeners` to include all gateway
-hostnames:
+The example (`nvmeof.yaml`) configures `instances: 2` for high availability.
+When running multiple gateway instances, update the StorageClass `listeners` to include all gateway
+deployment hostnames for multipath/HA:

 ```yaml
 listeners: |
@@ -375,10 +370,10 @@ kubectl delete storageclass ceph-nvmeof
 # Delete the NVMe-oF CSI operator resources
 kubectl delete -f deploy/examples/csi/nvmeof/csi-operator-nvmeof.yaml

-# Delete the NVMe-oF gateway
-kubectl delete -f deploy/examples/nvmeof-test.yaml
+# Delete the NVMe-oF gateway and its metadata pool
+kubectl delete -f deploy/examples/nvmeof.yaml

-# Delete the block pool (optional)
+# Delete the data pool (optional)
 kubectl delete -f deploy/examples/csi/nvmeof/nvmeof-pool.yaml
 ```

@@ -543,6 +543,7 @@ spec:
                    - .rgw.root
                    - .nfs
                    - .mgr
+                    - .nvmeof
                  type: string
                parameters:
                  additionalProperties:
@@ -12402,10 +12403,6 @@ spec:
                        type: object
                      type: array
                  type: object
-                pool:
-                  description: Pool is the RADOS pool where NVMe-oF configuration is stored
-                  minLength: 1
-                  type: string
                ports:
                  description: Ports configuration for the NVMe-oF gateway
                  properties:
@@ -12499,7 +12496,6 @@ spec:
              required:
                - group
                - instances
-                - pool
              type: object
            status:
              description: NVMeOFGatewayStatus represents the status of Ceph NVMe-oF Gateway
@@ -545,6 +545,7 @@ spec:
                    - .rgw.root
                    - .nfs
                    - .mgr
+                    - .nvmeof
                  type: string
                parameters:
                  additionalProperties:
@@ -12394,10 +12395,6 @@ spec:
                        type: object
                      type: array
                  type: object
-                pool:
-                  description: Pool is the RADOS pool where NVMe-oF configuration is stored
-                  minLength: 1
-                  type: string
                ports:
                  description: Ports configuration for the NVMe-oF gateway
                  properties:
@@ -12491,7 +12488,6 @@ spec:
              required:
                - group
                - instances
-                - pool
              type: object
            status:
              description: NVMeOFGatewayStatus represents the status of Ceph NVMe-oF Gateway
@@ -1,5 +1,19 @@
 # This example is for Ceph v20 and above only
+# This is a test configuration for CI environments only.
+# For production deployments, use nvmeof.yaml instead.

+apiVersion: ceph.rook.io/v1
+kind: CephBlockPool
+metadata:
+  name: builtin-nvmeof
+  namespace: rook-ceph # namespace:cluster
+spec:
+  name: .nvmeof
+  failureDomain: osd
+  replicated:
+    size: 1
+    requireSafeReplicaSize: false
+---
 apiVersion: ceph.rook.io/v1
 kind: CephNVMeOFGateway
 metadata:
@@ -10,7 +24,6 @@ spec:
  # (ceph config get mgr mgr/cephadm/container_image_nvmeof).
  # Set the image property to override the default:
  # image: quay.io/ceph/nvmeof:<version>
-  pool: nvmeof
  group: group-a
  instances: 1
  hostNetwork: false
@@ -0,0 +1,26 @@
+# This example is for Ceph v20 and above only
+
+apiVersion: ceph.rook.io/v1
+kind: CephBlockPool
+metadata:
+  name: builtin-nvmeof
+  namespace: rook-ceph # namespace:cluster
+spec:
+  name: .nvmeof
+  failureDomain: host
+  replicated:
+    size: 3
+---
+apiVersion: ceph.rook.io/v1
+kind: CephNVMeOFGateway
+metadata:
+  name: nvmeof
+  namespace: rook-ceph # namespace:cluster
+spec:
+  # image is optional. If not specified, the image is fetched from the Ceph config store
+  # (ceph config get mgr mgr/cephadm/container_image_nvmeof).
+  # Set the image property to override the default:
+  # image: quay.io/ceph/nvmeof:<version>
+  group: group-a
+  instances: 2
+  hostNetwork: false
@@ -1111,7 +1111,7 @@ type PoolSpec struct {
 // allowed pool names that can be specified.
 type NamedBlockPoolSpec struct {
 	// The desired name of the pool if different from the CephBlockPool CR name.
-	// +kubebuilder:validation:Enum=.rgw.root;.nfs;.mgr
+	// +kubebuilder:validation:Enum=.rgw.root;.nfs;.mgr;.nvmeof
 	// +optional
 	Name string `json:"name,omitempty"`
 	// The core pool configuration
@@ -4238,10 +4238,6 @@ type NVMeOFGatewaySpec struct {
 	// +kubebuilder:validation:Minimum=1
 	Instances int `json:"instances"`

-	// Pool is the RADOS pool where NVMe-oF configuration is stored
-	// +kubebuilder:validation:MinLength=1
-	Pool string `json:"pool"`
-
 	// Group is the gateway group name for high availability (ANA group)
 	// +kubebuilder:validation:MinLength=1
 	Group string `json:"group"`
@@ -55,6 +55,10 @@ import (
 const (
 	packageName    = "ceph-nvmeof-gateway"
 	controllerName = packageName + "-controller"
+
+	// nvmeofPoolName is the required Ceph pool for NVMe-oF gateway state.
+	// This matches the pool that Ceph NVMe-oF creates automatically.
+	nvmeofPoolName = ".nvmeof"
 )

 var logger = capnslog.NewPackageLogger("github.com/rook/rook", packageName)
@@ -473,14 +477,13 @@ func getNVMeOFGatewayConfig(poolName, podName, podIP, anaGroup string, userConfi
 }

 func (r *ReconcileCephNVMeOFGateway) generateConfigMap(nvmeof *cephv1.CephNVMeOFGateway, daemonID string) (*v1.ConfigMap, error) {
-	poolName := nvmeof.Spec.Pool
 	anaGroup := nvmeof.Spec.Group
 	podName := instanceName(nvmeof, daemonID)
 	// Use placeholder that will be replaced at runtime with actual pod IP
 	// The init container will replace @@POD_IP@@ with the actual pod IP
 	podIP := "@@POD_IP@@"

-	configContent, err := getNVMeOFGatewayConfig(poolName, podName, podIP, anaGroup, nvmeof.Spec.NVMeOFConfig)
+	configContent, err := getNVMeOFGatewayConfig(nvmeofPoolName, podName, podIP, anaGroup, nvmeof.Spec.NVMeOFConfig)
 	if err != nil {
 		return nil, errors.Wrap(err, "failed to generate nvmeof config")
 	}
@@ -540,10 +543,6 @@ func validateGateway(g *cephv1.CephNVMeOFGateway) error {
 		return errors.New("gateway group name is required")
 	}

-	if g.Spec.Pool == "" {
-		return errors.New("pool name is required")
-	}
-
 	return nil
 }

@@ -119,7 +119,6 @@ func TestCephNVMeOFGatewayController(t *testing.T) {
 				Finalizers: []string{"cephnvmeofgateway.ceph.rook.io"},
 			},
 			Spec: cephv1.NVMeOFGatewaySpec{
-				Pool:      "nvmeof",
 				Group:     "group-a",
 				Instances: 1,
 			},
@@ -461,7 +460,6 @@ func TestNVMeOFKeyRotation(t *testing.T) {
 			Namespace: namespace,
 		},
 		Spec: cephv1.NVMeOFGatewaySpec{
-			Pool:      "nvmeof",
 			Group:     "group-a",
 			Instances: 1,
 		},
@@ -696,7 +694,6 @@ func TestNVMeOFConfigMapGeneration(t *testing.T) {
 			Namespace: "rook-ceph",
 		},
 		Spec: cephv1.NVMeOFGatewaySpec{
-			Pool:      "pool-a",
 			Group:     "group-a",
 			Instances: 1,
 			NVMeOFConfig: map[string]map[string]string{
@@ -721,5 +718,5 @@ func TestNVMeOFConfigMapGeneration(t *testing.T) {
 	assert.Equal(t, "@@POD_IP@@", cfg.Section("gateway").Key("addr").String())
 	assert.Equal(t, "group-a", cfg.Section("gateway").Key("group").String())
 	assert.Equal(t, "5511", cfg.Section("gateway").Key("port").String())
-	assert.Equal(t, "pool-a", cfg.Section("ceph").Key("pool").String())
+	assert.Equal(t, nvmeofPoolName, cfg.Section("ceph").Key("pool").String())
 }
@@ -299,7 +299,6 @@ func (r *ReconcileCephNVMeOFGateway) createCephConfigInitContainer(nvmeof *cephv
 	script := connectionConfigScript

 	gatewayName := instanceName(nvmeof, daemonID)
-	poolName := nvmeof.Spec.Pool
 	anaGroup := nvmeof.Spec.Group

 	// Build environment variables using Rook's helper functions
@@ -312,7 +311,7 @@ func (r *ReconcileCephNVMeOFGateway) createCephConfigInitContainer(nvmeof *cephv
 		},
 		v1.EnvVar{
 			Name:  "POOL_NAME",
-			Value: poolName,
+			Value: nvmeofPoolName,
 		},
 		v1.EnvVar{
 			Name:  "ANA_GROUP",
@@ -43,8 +43,16 @@ import (

 func newDeploymentSpecTest(t *testing.T) (*ReconcileCephNVMeOFGateway, string) {
 	clientset := optest.New(t, 1)
+	executor := &exectest.MockExecutor{
+		MockExecuteCommandWithTimeout: func(timeout time.Duration, command string, arg ...string) (string, error) {
+			if command == "ceph" && arg[0] == "config" && arg[1] == "get" {
+				return "quay.io/ceph/nvmeof:1.5", nil
+			}
+			return "", nil
+		},
+	}
 	c := &clusterd.Context{
-		Executor:      &exectest.MockExecutor{},
+		Executor:      executor,
 		RookClientset: rookclient.NewSimpleClientset(),
 		Clientset:     clientset,
 	}
@@ -59,7 +67,6 @@ func newDeploymentSpecTest(t *testing.T) (*ReconcileCephNVMeOFGateway, string) {
 			TypeMeta: controllerTypeMeta,
 			Spec: cephv1.NVMeOFGatewaySpec{
 				Image:     "quay.io/ceph/nvmeof:1.5",
-				Pool:      "nvmeof",
 				Group:     "group-a",
 				Instances: 1,
 			},
@@ -97,7 +104,6 @@ func TestDeploymentSpec(t *testing.T) {
 			},
 			Spec: cephv1.NVMeOFGatewaySpec{
 				Image:     "quay.io/ceph/nvmeof:1.5",
-				Pool:      "nvmeof",
 				Group:     "group-a",
 				Instances: 3,
 				Resources: v1.ResourceRequirements{
@@ -149,8 +155,6 @@ func TestDeploymentSpec(t *testing.T) {
 				Namespace: "rook-ceph-test-ns",
 			},
 			Spec: cephv1.NVMeOFGatewaySpec{
-				Image:     "quay.io/ceph/nvmeof:1.5",
-				Pool:      "nvmeof",
 				Group:     "group-a",
 				Instances: 3,
 				Resources: v1.ResourceRequirements{
@@ -198,7 +202,6 @@ func TestDeploymentSpec(t *testing.T) {
 			},
 			Spec: cephv1.NVMeOFGatewaySpec{
 				Image:     "quay.io/ceph/nvmeof:1.5",
-				Pool:      "nvmeof",
 				Group:     "group-a",
 				Instances: 3,
 			},
@@ -228,7 +231,6 @@ func TestDeploymentSpec(t *testing.T) {
 			},
 			Spec: cephv1.NVMeOFGatewaySpec{
 				Image:     "quay.io/ceph/nvmeof:1.5",
-				Pool:      "nvmeof",
 				Group:     "group-a",
 				Instances: 3,
 				Placement: cephv1.Placement{
@@ -271,7 +273,6 @@ func TestDeploymentSpec(t *testing.T) {
 		assert.Empty(t, d.Spec.Template.Spec.Affinity.PodAntiAffinity.PreferredDuringSchedulingIgnoredDuringExecution)
 		assert.Equal(t, "topology.kubernetes.io/zone", d.Spec.Template.Spec.Affinity.PodAntiAffinity.RequiredDuringSchedulingIgnoredDuringExecution[0].TopologyKey)
 		assert.Equal(t, map[string]string{"app": "custom-app"}, d.Spec.Template.Spec.Affinity.PodAntiAffinity.RequiredDuringSchedulingIgnoredDuringExecution[0].LabelSelector.MatchLabels)
-
 		assert.Len(t, d.Spec.Template.Spec.TopologySpreadConstraints, 1)
 		assert.Equal(t, int32(2), d.Spec.Template.Spec.TopologySpreadConstraints[0].MaxSkew)
 		assert.Equal(t, "topology.kubernetes.io/zone", d.Spec.Template.Spec.TopologySpreadConstraints[0].TopologyKey)
@@ -287,7 +288,6 @@ func TestDeploymentSpec(t *testing.T) {
 			},
 			Spec: cephv1.NVMeOFGatewaySpec{
 				Image:     "quay.io/ceph/nvmeof:1.5",
-				Pool:      "nvmeof",
 				Group:     "group-a",
 				Instances: 1,
 			},
@@ -316,7 +316,6 @@ func TestDeploymentSpec(t *testing.T) {
 			},
 			Spec: cephv1.NVMeOFGatewaySpec{
 				Image:        "quay.io/ceph/nvmeof:1.5",
-				Pool:         "nvmeof",
 				Group:        "group-a",
 				Instances:    1,
 				ConfigMapRef: "custom-configmap",
@@ -350,7 +349,6 @@ func TestDeploymentSpec(t *testing.T) {
 			},
 			Spec: cephv1.NVMeOFGatewaySpec{
 				Image:     "quay.io/ceph/nvmeof:1.5",
-				Pool:      "nvmeof",
 				Group:     "group-a",
 				Instances: 1,
 			},
@@ -370,7 +368,7 @@ func TestDeploymentSpec(t *testing.T) {
 		assert.Contains(t, initCont.Args, "--keyring=/etc/ceph/keyring")

 		assertEnvVar(t, initCont.Env, "GATEWAY_NAME", instanceName(nvmeof, "0"))
-		assertEnvVar(t, initCont.Env, "POOL_NAME", nvmeof.Spec.Pool)
+		assertEnvVar(t, initCont.Env, "POOL_NAME", nvmeofPoolName)
 		assertEnvVar(t, initCont.Env, "ANA_GROUP", nvmeof.Spec.Group)
 		assertEnvVarPresent(t, initCont.Env, "POD_IP")

@@ -391,7 +389,6 @@ func TestDeploymentSpec(t *testing.T) {
 			},
 			Spec: cephv1.NVMeOFGatewaySpec{
 				Image:     "quay.io/ceph/nvmeof:1.5",
-				Pool:      "nvmeof",
 				Group:     "group-a",
 				Instances: 1,
 			},
@@ -433,19 +430,12 @@ func TestDeploymentSpec(t *testing.T) {
 				Namespace: "rook-ceph-test-ns",
 			},
 			Spec: cephv1.NVMeOFGatewaySpec{
-				Pool:      "nvmeof",
 				Group:     "group-a",
 				Instances: 1,
 			},
 		}

 		r, configHash := newDeploymentSpecTest(t)
-		executor := &exectest.MockExecutor{
-			MockExecuteCommandWithTimeout: func(timeout time.Duration, command string, arg ...string) (string, error) {
-				return "quay.io/ceph/nvmeof:1.5", nil
-			},
-		}
-		r.context.Executor = executor

 		configMapName := fmt.Sprintf("rook-ceph-nvmeof-%s-config", nvmeof.Name)
 		d, err := r.makeDeployment(nvmeof, "0", configMapName, configHash)
@@ -464,7 +454,6 @@ func TestDeploymentSpec(t *testing.T) {
 				Namespace: "rook-ceph-test-ns",
 			},
 			Spec: cephv1.NVMeOFGatewaySpec{
-				Pool:      "nvmeof",
 				Group:     "group-a",
 				Instances: 1,
 			},
@@ -492,7 +481,6 @@ func TestDeploymentSpec(t *testing.T) {
 			},
 			Spec: cephv1.NVMeOFGatewaySpec{
 				Image:     "quay.io/ceph/nvmeof:1.5",
-				Pool:      "nvmeof",
 				Group:     "group-a",
 				Instances: 1,
 			},
@@ -513,7 +501,6 @@ func TestDeploymentSpec(t *testing.T) {
 			},
 			Spec: cephv1.NVMeOFGatewaySpec{
 				Image:     "quay.io/ceph/nvmeof:1.5",
-				Pool:      "nvmeof",
 				Group:     "group-a",
 				Instances: 1,
 			},
@@ -533,8 +520,6 @@ func TestDeploymentSpec(t *testing.T) {
 				Namespace: "rook-ceph-test-ns",
 			},
 			Spec: cephv1.NVMeOFGatewaySpec{
-				Image:     "quay.io/ceph/nvmeof:1.5",
-				Pool:      "nvmeof",
 				Group:     "group-a",
 				Instances: 1,
 			},
@@ -582,8 +567,6 @@ func TestDeploymentSpec(t *testing.T) {
 				Namespace: "rook-ceph-test-ns",
 			},
 			Spec: cephv1.NVMeOFGatewaySpec{
-				Image:     "quay.io/ceph/nvmeof:1.5",
-				Pool:      "nvmeof",
 				Group:     "group-a",
 				Instances: 1,
 				Ports: &cephv1.NVMeOFGatewayPorts{
@@ -1056,6 +1056,7 @@ function test_csi_nvmeof_workload {
  wait_for cephblockpool nvmeof rook-ceph 600

  kubectl create -f "${REPO_DIR}/deploy/examples/nvmeof-test.yaml"
+  wait_for cephblockpool builtin-nvmeof rook-ceph 600
  wait_for cephnvmeofgateway nvmeof rook-ceph 600
  timeout 300 bash <<EOF
 until kubectl -n rook-ceph get pod --no-headers 2>/dev/null | awk '/rook-ceph-nvmeof-nvmeof-a-/ && \$3 == "Running" {f=1} END {exit !f}'; do