INSIGHTS
Infrastructure & Systems

CNCF CKA: Persistent Volumes and Storage Classes

In this article
  1. Separate ephemeral container storage from persistent application data
  2. Use PersistentVolumes to represent storage resources
  3. Let applications request storage with PersistentVolumeClaims
  4. Use StorageClasses to describe provisioning policy
  5. Understand dynamic provisioning and binding modes
  6. Choose reclaim policy with data lifecycle in mind
  7. Use CSI as the integration boundary for storage drivers
  8. Plan expansion, snapshots, and backups separately
  9. Troubleshoot storage in the order Kubernetes uses it

Containers are disposable, but application data often is not. Kubernetes therefore separates workload lifecycle from persistent storage through PersistentVolumes, PersistentVolumeClaims, StorageClasses, and the Container Storage Interface. These abstractions let applications request storage without hard-coding every vendor-specific disk detail into the Pod specification.

The current CKA gives storage 10% of the exam. The percentage may look smaller than troubleshooting or networking, but storage problems can prevent workloads from scheduling entirely. Administrators need to understand binding, dynamic provisioning, access modes, reclaim behavior, and the difference between making a volume available and protecting the data on it.

Separate ephemeral container storage from persistent application data

A container filesystem is tied to the container lifecycle. Replacing the container can discard changes that were written only into that writable layer. Pods can also use ephemeral volumes whose lifetime is tied to the Pod. Those mechanisms are useful for caches, scratch space, and temporary working data.

Persistent application state needs a lifecycle independent of one Pod. Databases, content repositories, message systems, and other stateful workloads often require storage that remains available after a Pod is rescheduled or recreated.

The broader differences among block, file, and object storage still matter. Kubernetes can abstract provisioning, but the application’s access pattern still determines which underlying storage technology is appropriate.

Use PersistentVolumes to represent storage resources

A PersistentVolume, or PV, represents storage available to the cluster. It has capacity, access modes, a reclaim policy, and information about how the storage is provided. A PV can be created manually by an administrator or dynamically through a StorageClass.

The PV object is not the data itself. It is the Kubernetes representation of a storage resource. The actual bytes may live on a cloud disk, SAN volume, network filesystem, or another backend exposed through a compatible driver.

That distinction becomes important during recovery. Deleting a Kubernetes object and deleting the underlying storage are related actions, but they are not always the same action. Reclaim policy determines what Kubernetes attempts after a claim is released.

Let applications request storage with PersistentVolumeClaims

A PersistentVolumeClaim, or PVC, is a user’s request for storage. The claim specifies requirements such as capacity, access mode, and optionally a StorageClass. Kubernetes finds or provisions a compatible PV and binds it to the claim.

Pods then reference the claim rather than the vendor-specific volume directly. This separation lets application manifests describe the storage they need while administrators control how those requests are satisfied in the environment.

When a Pod is Pending because of storage, inspect the PVC status. An unbound claim can indicate missing capacity, incompatible access modes, a missing StorageClass, provisioning failure, or a topology constraint that cannot be satisfied.

Use StorageClasses to describe provisioning policy

A StorageClass describes a category of storage that administrators make available. It specifies a provisioner and can define parameters, reclaim policy, volume binding mode, expansion support, and other implementation-specific settings.

Different classes may represent high-performance SSD storage, cheaper capacity-oriented storage, encrypted storage, or storage with different backup or replication expectations. Kubernetes does not define what names such as “gold” or “fast” mean; the organization does.

Applications should request a class according to service requirements rather than by guessing at provider implementation details. Platform teams should document performance, durability, cost, topology, and recovery expectations for each class.

Understand dynamic provisioning and binding modes

Dynamic provisioning lets Kubernetes request new storage automatically when a PVC references a StorageClass. That removes the need for administrators to pre-create one PV for every application claim.

Volume binding mode affects when provisioning and binding happen. Immediate binding can provision storage as soon as the PVC appears. WaitForFirstConsumer delays binding until Kubernetes has more information about where the consuming Pod can be scheduled, which helps with topology-constrained storage such as zone-specific disks.

This is an important scheduling interaction. A storage decision made too early can place a volume in one zone while the workload can only run in another. Delayed binding lets the scheduler consider both compute and storage placement together.

Choose reclaim policy with data lifecycle in mind

StorageClasses and PVs can use reclaim policies such as Delete or Retain. Delete allows the provisioned backing storage to be removed when the claim lifecycle ends, while Retain preserves the storage for manual recovery or reuse steps.

Neither setting is universally correct. Temporary environments may benefit from automatic cleanup. Critical data may require retention, backups, or approval before deletion. The policy should reflect business recovery requirements rather than administrator convenience.

Teams should also understand application-level deletion. A database can delete records while the volume remains intact, and a volume snapshot cannot always replace transaction-aware backup procedures. Kubernetes storage lifecycle is only one part of data protection.

Persistent volumes declare access modes that describe how they can be mounted, such as ReadWriteOnce, ReadOnlyMany, or ReadWriteMany. Exact behavior depends on the driver and storage backend. Administrators should not assume that a requested mode is supported simply because Kubernetes recognizes the field.

StatefulSets often pair each replica with its own PVC because many databases expect exclusive writable storage per instance. Shared file workloads may require ReadWriteMany storage. Block devices may offer different performance characteristics than network filesystems.

Choose the access pattern based on application semantics first, then select a storage implementation that can satisfy it reliably.

Use CSI as the integration boundary for storage drivers

The Container Storage Interface allows storage vendors and platform providers to integrate provisioning, attachment, mounting, snapshots, and expansion without embedding every driver into Kubernetes core. This modular approach lets storage implementations evolve independently of the main Kubernetes release.

CSI introduces its own operational components. Controller plugins may handle provisioning and attachment while node plugins handle mounting on individual nodes. If one part is unhealthy, PVCs can remain Pending or Pods can become stuck during mount operations.

Administrators should know where CSI controller and node components run and how to inspect their logs. A storage incident may be a backend outage, a permissions problem, a topology conflict, or a driver issue rather than a Kubernetes API problem.

Plan expansion, snapshots, and backups separately

Some StorageClasses allow volume expansion. A PVC can request a larger capacity, and the driver may expand the backing volume and filesystem. Expansion generally supports growing a volume, not shrinking it, and application behavior during resizing depends on the filesystem and driver.

Snapshots can create point-in-time copies when the CSI driver and snapshot APIs support them. Snapshots are useful for recovery and cloning, but they should not automatically be treated as a complete backup strategy. A snapshot stored in the same failure domain or account may not survive a broader incident.

The same principle behind storage backup design applies: define retention, independent failure domains, restore objectives, and regular recovery testing.

Troubleshoot storage in the order Kubernetes uses it

Start with the PVC. Is it Bound? If not, inspect events for provisioning or class errors. If it is bound, inspect the PV, StorageClass, and topology. Then inspect the Pod events for attach or mount failures. Check the CSI controller and node plugin logs when the Kubernetes objects look correct.

Also verify node-level prerequisites and provider limits. A cloud disk may have attachment limits, a node may lack required packages, or network storage may be blocked by firewall rules. The Pod often surfaces only the final mount failure, not the root cause.

Storage topology is one of the easiest operational details to miss. A dynamically provisioned volume may exist only in one availability zone or on one class of node. If the scheduler places the Pod somewhere that cannot access the volume, the workload can remain Pending or fail attachment. StorageClasses that use WaitForFirstConsumer help delay provisioning until Kubernetes knows enough about the Pod’s placement constraints to choose compatible storage.

Volume attachment and mount are separate stages. A cloud disk may first need to attach to the selected node, then the node plugin mounts the filesystem into the Pod. Failure messages can therefore refer to attachment limits, device discovery, filesystem checks, mount options, or permissions. Reading the exact event usually reveals which phase failed.

Stateful applications also need application-consistent recovery. A raw storage snapshot taken while a database is processing writes may be crash-consistent but not transaction-consistent. Databases often provide backup or snapshot coordination features that should be used before or alongside infrastructure snapshots. The storage layer protects blocks; the application still understands its own consistency model.

PersistentVolumeClaims are namespaced, while PersistentVolumes are cluster-scoped. That difference affects administration and RBAC. Application teams commonly manage claims in their namespace, while platform administrators govern StorageClasses and the cluster-wide storage inventory. Clear role separation prevents application manifests from depending on backend details they should not control.

VolumeMode can expose storage as a filesystem or as a raw block device when the driver and workload support it. Filesystem mode is common because Kubernetes and the node handle mounting. Raw block mode gives an application direct access to the device and can be useful for specialized systems, but it places more responsibility on the workload.

Default StorageClasses deserve attention. A PVC that omits storageClassName may bind to the default class, which can be convenient but can also hide cost or performance assumptions. Organizations with several storage tiers should document the default and require explicit classes for workloads with meaningful durability or latency requirements.

Reclaim policy interacts with deletion protection and recovery procedures. A Retain policy preserves the backing storage after a claim is deleted, but an administrator must then decide how to inspect, sanitize, rebind, or eventually remove it. Retain is not automatic backup; it is a deliberate pause in the cleanup lifecycle.

Capacity monitoring should cover both claims and the underlying storage system. A PVC may request enough capacity while the filesystem inside it approaches full usage. Thin-provisioned backends can also report large logical volumes while physical pools are nearing exhaustion. Monitor application-visible free space and backend pool health rather than trusting one number.

Performance troubleshooting should consider IOPS, throughput, latency, queue depth, and access pattern. A volume can be correctly mounted but too slow for the application. Moving a database from local SSD to a network class with higher latency may turn an application problem into what looks like CPU or timeout instability.

Storage encryption has several layers. A cloud disk may be encrypted at rest by the provider, the filesystem may use additional encryption, and application-level encryption may protect selected fields. Know where keys are managed and what happens during node or cluster recovery. A volume that is restored without access to its encryption key is not actually recoverable.

Backup restore testing should include the Kubernetes objects required to attach the data to a new workload. Restoring a disk image is only part of the procedure; administrators may need a compatible StorageClass, PV definition, PVC, Secret, and application configuration before the data is usable.

StatefulSet volumeClaimTemplates create predictable claims for each replica, but deleting a StatefulSet does not necessarily delete its claims. That behavior helps protect data from accidental workload deletion, yet it also means old claims can accumulate. Cleanup procedures should distinguish intentionally retained state from abandoned storage.

AccessMode does not automatically enforce every application-level concurrency rule. A backend may technically support multi-attach while the filesystem or database cannot tolerate simultaneous writers. Storage semantics must be validated end to end, not inferred from one Kubernetes field.

Before changing storage classes in production, test migration and rollback. Kubernetes does not provide a universal in-place mechanism for moving every workload from one class to another. Many migrations require snapshots, replication, application-level copy, or a new PVC followed by controlled cutover.

Storage permissions can fail after a successful mount. The application process may run as a non-root UID that cannot write the mounted filesystem. SecurityContext fields such as fsGroup can influence ownership behavior, while some CSI drivers implement their own mount-permission logic. Distinguish a mount failure from an application-level permission failure.

Deletion finalizers and protection mechanisms can keep PVs or PVCs in Terminating while references still exist. Before removing finalizers manually, identify what controller is expected to complete cleanup. Force-removing protection can leave leaked cloud disks or detach state that the platform no longer tracks correctly.

Cost governance belongs in storage policy. High-performance classes can be several times more expensive than capacity-oriented storage, and forgotten PVCs may continue generating charges after applications are removed. Label claims with ownership and environment, report unattached or unused volumes, and make cleanup part of application decommissioning rather than a separate infrastructure scavenger hunt.

Storage documentation should record the expected recovery point and recovery time for each class. A fast volume with no tested backup may be less suitable for critical data than a slower class with strong replication and recovery procedures. Kubernetes can provision storage automatically, but it cannot infer how much data loss the business can tolerate.

Test node replacement for stateful workloads. A volume that works only because it is already attached to one long-lived node may expose hidden permissions, topology, or driver problems when the Pod moves. Planned relocation is one of the best ways to prove that persistence is truly independent of a particular worker machine.

For administrators in the CNCF ecosystem, the durable storage model is a chain: the application requests storage with a PVC, a PV satisfies the request, a StorageClass defines provisioning policy, and a CSI driver connects Kubernetes to the actual backend. Understanding that chain makes most storage incidents much easier to isolate.

Filed under Infrastructure & Systems