Skip to content
This repository was archived by the owner on Jun 22, 2026. It is now read-only.
This repository was archived by the owner on Jun 22, 2026. It is now read-only.

Allow multiple volumes with managed EFS for a single service #2921

Open

Description

@efekarakus

We should enable more than one storage.volumes with managed EFS for a single service.

Ask

Original ask: #1559 (comment)

Today users are limited to a single EFS access point per service:

storage:
  volumes:
    frontend:
      efs:
        uid: 123
        gid: 123
      path: /var/data

If they try to use the same EFS filesystem for multiple volumes like this:

storage:
  volumes:
    v1:
      efs:
        uid: 123
        gid: 123
      path: /etc/mount1
    v2:
      efs:
        uid: 456
        gid: 456
      path: /etc/mount2

We return the following error:

validate "storage": cannot specify more than one managed volume per service

Programming model

  1. In order to support multiple access points for a single service we need to transform the following resource into a list:
    {{- if .Storage.ManagedVolumeInfo}}
    AccessPoint:
  2. We need to assign a new RootDirectory.Path for the additional access points. Today, we hard-code it to the service name like "frontend". The proposal is to do <svcName>-<volumeName>.
    RootDirectory:
    Path: !Sub '/{{.Storage.ManagedVolumeInfo.DirName}}'
  3. Update the IAM policy in the TaskRole to grant access to multiple access points:
    {{- if .Storage.ManagedVolumeInfo}}
    - PolicyName: 'GrantAccessCopilotManagedEFS'
    PolicyDocument:

Activity

  1. added
    type/featureIssues that are new feature requests.
    type/requestIssues that are created by customers.
    size/MWe should be able to deliver roughly 1 medium issue in a sprint.
    on Oct 14, 2021
  2. FraserThompson commented on May 31, 2022

    @FraserThompson

    Just in case this helps prioritize this... Multiple access points is essential for running CMSs like Drupal (probably others) which save user files in multiple locations. For example:

    Public filesystem in /opt/drupal/web/sites/default/files
    Private fileystem /opt/drupal/private

    A single access point couldn't meet this use case. It's possible with externally managed filesystems, but having all this managed by Copilot would be much more convenient.

  3. efekarakus commented on Jun 3, 2022

    @efekarakus
    ContributorAuthor

    Hi @FraserThompson ! out of curiosity wouldn't this scenario be satisfied with:

    storage:
      volumes:
        frontend:
          efs: true
          path: /opt/drupal
    

    Or am I misunderstanding a step?

  4. FraserThompson commented on Jun 8, 2022

    @FraserThompson

    Hi @efekarakus ! This would work to persist files, yes, but as far as I can tell it would go against best practices.

    The other contents of /opt/drupal come from the Docker image and should only exist in ephemeral storage because they're not stateful. The only stateful data is contained in those two directories.

    Ideally you could put all files you want persisted into one directory and make that an EFS volume, but Drupal doesn't offer full control over where these files can be.

    Also there are other factors, like we might want to share just the public files with an NGINX sidecar so it can serve static assets. We might have another sidecar running Solr for search, and this will also need to persist the search index somewhere. I don't think it would make sense for all of these to persist their data to the same place.

    For reference the AWS example for Drupal on ECS uses one filesystem with multiple access points for each directory they want to persist. Unless I'm misunderstanding something (which is entirely possible) I don't think we can do this right now with Copilot.

  5. hardeepjethwani commented on Feb 3, 2023

    @hardeepjethwani

    We should enable more than one storage.volumes with managed EFS for a single service.

    Ask

    Original ask: #1559 (comment)

    Today users are limited to a single EFS access point per service:

    storage:
      volumes:
        frontend:
          efs:
            uid: 123
            gid: 123
          path: /var/data

    If they try to use the same EFS filesystem for multiple volumes like this:

    storage:
      volumes:
        v1:
          efs:
            uid: 123
            gid: 123
          path: /etc/mount1
        v2:
          efs:
            uid: 456
            gid: 456
          path: /etc/mount2

    We return the following error:

    validate "storage": cannot specify more than one managed volume per service
    

    Programming model

    1. In order to support multiple access points for a single service we need to transform the following resource into a list:
      {{- if .Storage.ManagedVolumeInfo}}
      AccessPoint:
    2. We need to assign a new RootDirectory.Path for the additional access points. Today, we hard-code it to the service name like "frontend". The proposal is to do <svcName>-<volumeName>.
      RootDirectory:
      Path: !Sub '/{{.Storage.ManagedVolumeInfo.DirName}}'
    3. Update the IAM policy in the TaskRole to grant access to multiple access points:
      {{- if .Storage.ManagedVolumeInfo}}
      - PolicyName: 'GrantAccessCopilotManagedEFS'
      PolicyDocument:

    Hi @efekarakus , Do we have any update on this issue resolution?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    size/MWe should be able to deliver roughly 1 medium issue in a sprint.type/featureIssues that are new feature requests.type/requestIssues that are created by customers.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions