Documentation

CLI Commands

mapfs provides a set of CLI commands for users to interact with the filesystem:

List all available commands

  # list all available commands
  $ mapfs

  # help message
  $ mapfs --help
  $ mapfs -h

  # version
  $ mapfs --version
  $ mapfs -v

  # help message for a specific command
  $ mapfs <command> --help
  
  # e.g. help message for "mount" command
  $ mapfs mount
  
  # e.g. help message for "umount" command
  $ mapfs umount

  # e.g. help message for "add" command
  $ mapfs add

Add/Remove one volume

Adding one volume means creating a mapping between a cloud storage and a local volume.

Removing one volume means deleting the mapping between a cloud storage and a local volume.

Usage:

  $ mapfs add <VolumeName> <CloudVendor> [CloudOptions] [CacheOptions]

<VolumeName>:
Up to 253 characters. May contain only lowercase letters, digits, '-' and '.'.
Must start and end with a lowercase letter or digit.

<CloudVendor>: aws | s3, google | gcs, aliyun, wasabi, cloudflare | r2, oracle | oci, s3compatible, azure

CloudOptions (required, depends on <CloudVendor>):

  aws | s3 | gcs | google | aliyun | wasabi:
      --access-key-id,     -i <AccessKeyID>
      --secret-access-key, -k <SecretAccessKey>
      --bucket-name,       -b <BucketName>
      --region,            -r <Region>

  oracle | oci:
      --access-key-id,     -i <AccessKeyID>
      --secret-access-key, -k <SecretAccessKey>
      --bucket-name,       -b <BucketName>
      --region,            -r <Region>
      --oci-namespace,     -n <OCI-Namespace>

  cloudflare | r2:
      --access-key-id,     -i <AccessKeyID>
      --secret-access-key, -k <SecretAccessKey>
      --bucket-name,       -b <BucketName>
      --account-id,        -u <AccountID>

  s3compatible:
      --access-key-id,     -i <AccessKeyID>
      --secret-access-key, -k <SecretAccessKey>
      --bucket-name,       -b <BucketName>
      --endpoint,          -e <Endpoint>

  azure:
      --connection-string, -c <ConnectionString>
      --container-name,    -t <ContainerName>

CacheOptions (optional):

  --cache-dir, -d <CacheDir>
      the directory to hold cache data.
      default: /var/cache/mapfs/<VolumeName>
      e.g., --cache-dir /data/MyVolume, -d /data/MyVolume

  --cache-size, -z <CacheSizeInGB>
      the maximum GB for the cache.
      default: 100, maximum 100000 (100TB)
      e.g., --cache-size 10, -z 10

  --cache-timeout, -T <CacheTimeoutInSeconds>
      the cache expire time in seconds. "0" means never expire.
      default: 0
      e.g., --cache-timeout 300, -T 300

Example:

  $ mapfs add my-volume aws -i <AccessKeyID> -k <SecretAccessKey> -b my-bucket -r us-east-1

  $ mapfs add my-volume oracle --access-key-id <AccessKeyID> --secret-access-key <SecretAccessKey> --bucket-name my-bucket --region us-ashburn-1 --oci-namespace mynamespace

List all configured volumes:

  $ mapfs list

Remove one volume:

Note: Removing one volume will delete the mapping between the cloud storage and the local volume, and delete the local cache data of this volume, but won't delete any data in the cloud storage.

  # mapfs remove <VolumeName>
  # for example
  $ mapfs remove my-volume

Configure Cache parameters

By default, the cache directory relies on /opt/mapfs/cache directory. And the default data cache size is 100GB.

Note:
1, Cache size is in GB.
2, Shrinking cache size is not supported.
3, Changing cache directory is not supported.

Configure cache size or timeout for an existing volume:

  # mapfs configure <VolumeName> cache-size=<GB>
  # mapfs configure <VolumeName> cache-timeout=<seconds>

  # for example
  $ mapfs configure s3_volume cache-size=10
  $ mapfs configure s3_volume cache-timeout=3600

Note:before configuring, the volume must be in the Unmount status

Mount/Unmount one volume

After mouting one configured volume, users can access the cloud storage through the local mount point.

Note:
1, "iouring" mount option is recommended for extremely high IO performance scenarios with small size read requests.
for example, when the read request size is 4KB or even smaller, users can enable "iouring" to improve IO performance.

2, "mount" and "umount" commands require root privileges.

  # mount one volume
  # mapfs mount <VolumeName> <MountPoint> [iouring]
  # for example
  $ mapfs mount s3_volume /mnt/s3_volume

  # sometimes, users may want to enable "fuse over io uring" to improve IOPS, then they can add "iouring" option when mounting the volume
  $ mapfs mount s3_volume /mnt/s3_volume iouring

  # after mouting, you can access the cloud storage through the local mount point
  $ ls /mnt/s3_volume
  $ cat /mnt/s3_volume/file.txt
  $ mkdir /mnt/s3_volume/new_folder

Unmounting one volume will disconnect the mapping between the cloud storage and the local mount point, but won't delete any data in cloud storage.

  # unmount one volume
  # sudo mapfs umount <VolumeName>, or
  # sudo umount <MountPoint> [-f]
  # for example
  $ sudo mapfs umount s3_volume
  $ sudo umount /mnt/s3_volume
  $ sudo umount /mnt/s3_volume -f

Warm up cache

  $ mapfs load <filename>

Get volume runtime status

Displays the IO Status, Cache Hit/Miss Statistics information for all the mounted volumes.

  # get runtime status for all mounted volumes
  $ mapfs status