Creating PersistentVolumeClaims using Azure Files CSI and an existing file share

With the release of Kubernetes 1.21 support in AKS, the move away from in-tree storage drivers is complete. All future versions of AKS will rely on the CSI drivers as a managed component of the cluster.

While there's some good public documentation (Azure Files CSI drivers and Azure Disk CSI drivers for files and managed disks, respectively) and the documentation on GitHub for Azure Files and Disks is great as a reference, I recently hit a scenario that wasn't covered or addressed anywhere and required some time to prove out.

Hopefully this write-up will help what may be a gap in documentation and stop someone from fighting with the Azure CSI drivers.

Scenario

In this scenario, we have an AKS cluster in subscription A that needs to use an Azure File Share created on a storage account in subscription B. We previously accomplished this through the use of a custom StorageClass configured to use our storage account and existing file share, and created a PersistentVolumeClaim using the storage class. Potentially using something like the following YAML:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: custom-storage-class
provisioner: file.csi.azure.com
parameters:
  resourceGroup: storage-account-rg
  storageAccount: mystorageaccount
  secretName: my-storage-secret
  shareName: my-file-share
reclaimPolicy: Retain
volumeBindingMode: Immediate
allowVolumeExpansion: true
mountOptions:
  - dir_mode=0640
  - file_mode=0640
  - uid=0
  - gid=0
  - mfsymlinks
  - cache=strict
  - nosharesock
  - actimeo=30
---
apiVersion: v1
kind: Secret
metadata:
  name: my-storage-secret
type: Opaque
stringData:
  azurestorageaccountname: mystorageaccount
  azurestorageaccountkey: mystorageaccountaccesskey
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: custom-files-pvc
spec:
  resources:
    requests:
      storage: 10Gi 
  storageClassName: custom-storage-class
  accessModes:
    - ReadWriteMany

The above snippet will create a custom StorageClass and a PVC that uses it. Unfortunately, if that storage account is in a separate subscription than our cluster, we'll run into some errors when we attempt to use the PVC (and potentially see errors in the output of a describe) related to permissions as well as the cluster potentially not finding the storage account in the cluster's subscription.

What do we do?

We're pretty close to having the correct YAML for our StorageClass already - there's a few missing components though that would get us across the finish line.

The Azure Files CSI driver documentation isn't really clear on what's required to get our StorageClass working for a storage account in a different subscription. Fortunately for us, the documentation on GitHub and the code examples are a little bit better. There's also a great (at this time, open) issue on this same topic and a comment from Andy Zhang on how we can get it working in the current driver implementation!

Andy references the Bring Your Own Storage Account portion of the end-to-end usage docs. That docs section points to the YAML at https://github.com/kubernetes-sigs/azurefile-csi-driver/blob/master/deploy/example/storageclass-azurefile-secret.yaml which has a load of new entries under the parameters section of the StorageClass.

Andy notes that the addition of those parameters along with a shareName parameter (if you're using a pre-existing file share) will accomplish our goal. If we don't care to use the existing file share and only want the storage account, we can drop the shareName attribute entirely.

Putting all of that into an updated YAML, we get the following:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: custom-storage-class
provisioner: file.csi.azure.com
parameters:
  shareName: my-file-share
  csi.storage.k8s.io/provisioniner-secret-name: my-storage-secret
  csi.storage.k8s.io/provisioniner-secret-namespace: default
  csi.storage.k8s.io/node-stage-secret-name: my-storage-secret
  csi.storage.k8s.io/node-stage-secret-namespace: default
  csi.storage.k8s.io/controller-expand-secret-name: my-storage-secret
  csi.storage.k8s.io/controller-expand-secret-namespace: default
reclaimPolicy: Retain
volumeBindingMode: Immediate
allowVolumeExpansion: true
mountOptions:
  - dir_mode=0640
  - file_mode=0640
  - uid=0
  - gid=0
  - mfsymlinks
  - cache=strict
  - nosharesock
  - actimeo=30
---
apiVersion: v1
kind: Secret
metadata:
  name: my-storage-secret
type: Opaque
stringData:
  azurestorageaccountname: mystorageaccount
  azurestorageaccountkey: mystorageaccountaccesskey
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: custom-files-pvc
spec:
  resources:
    requests:
      storage: 10Gi 
  storageClassName: custom-storage-class
  accessModes:
    - ReadWriteMany

With that YAML applied, our StorageClass will use the existing storage account and file share for the newly-created custom-files-pvc created in the default namespace.