Configuration options#

CUNO_OPTIONS#

Overview#

Setting the environment variable CUNO_OPTIONS can be used to further configure cunoFS. The -o option on the cuno command can also be used to specify these options.

Options available for CUNO_OPTIONS:

off

Disable cunoFS access.

+static

Enable interception of statically compiled binaries. (default)

-static

Disable interception of statically compiled binaries.

+closekeepcache=[N]

Keep recently closed virtual files cached for N*0.1 seconds. This helps the performance of applications like Apache which frequently open/close/reopen files. (Disabled by default, if N not specified then 1 second by default, i.e. N=10)

+cloud

Enable cloud access using both xx:// and /cuno/xx formats. (default)

-cloud

Disable cloud access. If followed by +urifile, then only xx:// formats are enabled.

+urifile=strict

Enable cloud access using strict URI formats such as xx://.

+urifile

Slight variations of URI format xx:// are also accepted, e.g., /xx://, /xx:/ , xx:/. (default)

+urifile=any

/xx:/ is recognized anywhere in the full given path to a file. Everything before the xx:/ is ignored. e.g. /my/path/s3:/bucket/object is interpreted as s3://bucket/object.

-urifile

Disable cloud access using URIs such as xx:// and other variations.

+uricompat=[binary1:binary2:…]

When a paired URI is detected, and if the binary is a member of this colon-separated list, then URI remapping is enforced, instead of the binary’s own URI handling scheme. Can also be wildcard * to mean all binaries.

-uricompat

Disable URI remapping scheme for all binaries.

-forceuri

URI remapping is enforced for all URIs, not just paired URIs.

+hardlink

Enable hard link functionality. (default ONLY when POSIX File Access is enabled)

-hardlink

Disable hard link functionality.

+cloudroot=[directory]

Define a custom directory path prefix that will be used instead of ‘/cuno’ to introduce cloud locations. For how this relates to FlexMount and the effects of setting cloudroot when mounting a cunoFS Mount, see the The cloudroot option section. Using a tilde, ‘~’, in the path is unsupported. (default: /cuno)

-cloudrootover

cunoFS Direct Interception will prioritise the cunoFS Mount at the path specified by the cloudroot option. This means that any intercepted calls to a location within the cloudroot path will be handled by the cunoFS Mount at that location. This disables FlexMount mode, equivalent to ‘cloudrootover=off’. (default)

+cloudrootover

cunoFS Direct Interception will prioritise itself over anything at the path specified by cloudroot, but only if the input path exactly matches the text in cloudroot, equivalent to ‘cloudrootover=exact’. Use this to enable FlexMount behavior at the cloudroot.

cloudrootover=[off,exact,resolve,exe,<path-to-cunoFS-mount]

These settings affect what happens to when cunoFS Direct Interception (i.e. when cunoFS CLI is launched or LD_PRELOAD is set) encounters a path which already exists on the local filesystem, and affects the FlexMount behaviour if that local path refers to a cunoFS Mount. See the descriptions of each individual setting to learn more about what they do. (default: off)

cloudrootover=off

cunoFS Direct Interception will prioritise local directories, including the cunoFS Mount at the path specified by the cloudroot option. This means that any intercepted calls to a location within the cloudroot path will be handled by the cunoFS Mount at that location. This disables FlexMount mode. (default)

cloudrootover=exact

cunoFS Direct Interception will prioritise itself over local directories at the path specified by cloudroot, but only if the input path exactly matches the text in cloudroot. This also disables cunoFS Direct Interception’s static binary translator and cloud binary interception, so static binaries and binaries in object storage are passed through to the cunoFS Mount at the cloudroot. Use this to enable FlexMount behavior at the cloudroot.

cloudrootover=resolve

cunoFS Direct Interception will prioritise itself over anything that resolves to the same local path as the path specified by cloudroot. This also disables cunoFS Direct Interception’s static binary translator, so static binaries and binaries in object storage are passed through to the cunoFS Mount at the cloudroot. Use this to enable FlexMount behavior at the cloudroot, in a more resilient way.

cloudrootover=exe

cunoFS Direct Interception of static binaries and binaries on object storage will be re-enabled. Must be used in conjunction with a FlexMount-enabling cloudrootover option. This also enables ‘cloudrootover=resolve’.

cloudrootover=<path-to-cunoFS-mount>

The <path-to-cunoFS-mount> only needs to specified if an explicit ‘–root’ option was used when creating the cunoFS Mount. In this case, assuming the ‘–root’ value was set to a bucket or a directory on object storage (e.g. ‘cuno mount “$HOME/mybucket” –root /cuno/s3/mybucket/foo’), then you must set cloudrootover to the mount point AND not set cloudroot when launching cunoFS CLI to use the mount as FlexMount (e.g. ‘cuno -o cloudrootover=”$HOME/mybucket”’). This also implies ‘cloudrootover=exact’. If a cloudroot option was specified when creating the cunoFS Mount, or the –root option was used when creating the mount and refers to anywhere that isn’t inside the default cunoFS Mount cloudroot (‘/cuno’), then refer to the User Guide section The cloudroot option. Using a tilde, ‘~’, in the path is unsupported.

-cloudrootoverunload

We prefer that certain shells and debuggers (e.g. sh, tsh and gdb) disable/unload cunoFS Direct Interception in FlexMount-enabled modes for stability. Note that Direct Interception is re-enabled for any programs that they launch. This option disables this unloading behaviour. Warning: this may break things when you run these shells.

+cloudrootoverunload

We prefer that certain shells and debuggers (e.g. sh, tsh and gdb) disable cunoFS Direct Interception in cloudrootover enabled modes for stability. This option disables cunoFS Direct Interception (enables un-loading) when cloudrootover is enabled. (default)

cloudrootoverunload=[binary1:binary2:…]

Colon-separated list of paths to binaries. We prefer that certain shells and debuggers (e.g. sh, tsh and gdb) disable cunoFS Direct Interception in FlexMount-enabled modes for stability. This option disables cunoFS Direct Interception (enables un-loading) when FlexMount is enabled and one of binaries specified is launched. (default: bin/sh:bin/tsh:bin/gdb)

cloudrootoverunload=+<path-to-binary>

Appends <path-to-binary> to the list of ‘cloudrootoverunload’ binaries. This option will disable cunoFS Direct Interception (enable un-loading) in FlexMount-enabled modes when the specified binary is launched.

-cachehome

Disable cunoFS high performance metadata caching.

+cachehome=[directory]

Enable cunoFS high performance metadata caching and optionally set a semicolon-delimited list of directories to consider for caching (default: /cunodb;/dev/shm). If none of these directories exist, fall back to /tmp/cache/cuno. (default)

retry-multiplier=[decimal]

Scaling factor for the number of retries for non-fatal failed requests made to the server, e.g. CUNO_OPTIONS="retry-multiplier=1.5" increases the number of retries by 50%. If not set, the default multiplier value is set to 1. Accepted values are from 0 (disables retries) to 10000.

uid=[integer]

Define the default user ownership of files and directories within a bucket. Supply the user identifier in the numerical representation derived from id --user <USERNAME>.

gid=[integer]

Define the default group ownership of files and directories within a bucket. Supply the group identifier in the numerical representation derived from id --group <USERNAME>.

filemode=[octal]

Define the default file access permission bits of files within a bucket. Supply the octal (numeric) representation of the permissions you want to apply.

dirmode=[octal]

Define the default file access permission bits of directories within a bucket. Supply the octal (numeric) representation of the permissions you want to apply.

reset

Resetting discards any options from previous CUNO_OPTIONS assignments within the current shell. Subcommands only see the current CUNO_OPTIONS.

Note

Individual options must be separated by spaces. Don’t forget to enclose multiple options in quotes (" or '), or to escape spaces.

The static option#

To use cunoFS with static binaries, the parent process needs to be intercepted in static mode. Interception of static binaries is enabled by default; to disable it requires setting both CUNO_OPTIONS="-static" and starting a new cunoFS wrapped shell.

The uricompat option#

cunoFS can override the handling of URI-style paths in applications that attempt to provide internal handling of such URIs. Users may provide the +uricompat option to add similar patching for further applications.

To specify any applications to be patched, specify them as a colon-separated list:

+uricompat=<executable 1>:<exectuable 2>:< ... >

To limit the cases in which this patching occurs, you can filter interceptions to only occur when certain command-line arguments match specified keywords. This may be useful for certain code interpreters/executors (see the example below). To do this, in place of an executable name, use a slash-separated list of the name, a 1-based index of the argument to be matched (or any using *), and a value to match against:

+uricompat=<name of binary>/<index of argument to be matched>/<argument value to match>

Note

The +uricompat option only works on URIs that have been paired.

If you are working with a public access bucket, you will need to run the cuno creds pair command (see Pair containers and credentials) against it for the +uricompat to take effect.

Applications with patched URI handling#

By default, cunoFS will attempt to override the internal URI handling of popular tools, such as:

  • rsync

  • ffmpeg

  • tar

And some genomics tools:

  • samtools

  • igv

  • fastQC

This means that when a URI-style path (corresponding to a paired bucket or container) is passed to any of the applications above, cunoFS will prevent the application from handling the path in a special way.

For example, fmpeg has special handling for some protocols which they specify in a similar URI format to cunoFS URI cloud paths (e.g. ftp://example.foo). As a result, cunoFS needs to prevent ffmpeg from failing (when given a path like s3://bucket/file) because s3, az and gs are not protocols it supports.

Example#

To override a Java application’s handling of URI-style paths, we may use:

export CUNO_OPTIONS="+uricompat=java/2/appname.jar:java/*/org.name.main.Class"

This will intercept any invocation of java with appname.jar as its second argument, e.g. java -jar appname.jar s3://bucketname/file.bam. It will also intercept invocations of Java where any argument is equal to org.name.main.Class, e.g. java org.name.main.Class s3://bucketname/file.bam.

The cloudroot option#

You may define a custom directory path prefix to replace the default /cuno path (see :ref:`` ). Once defined, the path specified will be made available at the root of the file system. For example, if you configure the following:

export CUNO_OPTIONS='+cloudroot=my-object-storage'

You will then be able to list your configured S3 buckets with:

ls /my-object-storage/s3/<bucket>

Warning

We strongly recommend that the cloudroot path does not exist locally, because it can confuse cunoFS ability to distinguish whether a path refers to the local file system or to the cloud.

If you specify a cloudroot which conflicts with an existing directory, then it must not contain directories or files named s3, az or gs.

Note

In order for the “cd to cloud” functionality to see this option, you will need to spawn a new shell after CUNO_OPTIONS.

FlexMount usage#

Setting a cloudroot during the mounting process can affect the options required to be set when using it as a FlexMount.

[Recommended option] If cloudroot was not set when creating the mount (e.g. cuno mount ~/my-object-storage), then cloudroot needs to be set as the path to the mountpoint when launching cunoFS CLI in order to use that mount as a FlexMount (e.g. cuno -o cloudrootover -o cloudroot="<full-path-to-mountpoint>").

In some circumstances, you may need to set a cloudroot when creating the mount (for example, if a /cuno directory already exists on your systems). In these cases, we strongly recommend migrating or removing any such physical directory so that you can continue to use the default settings. If this is not possible, there are a number of options available.

[Recommended option if cloudroot must be changed] If a cloudroot needs to be set when creating the mount, then you should set the --root option to the same cloudroot to get expected behaviour when using cunoFS FlexMount (e.g. cuno -o cloudroot=/altcloudroot mount $HOME/my-object-storage --root /altcloudroot). In this case, continue to use the same options when launching cuno (e.g. cuno -o cloudrootover -o cloudroot="<full-path-to-mountpoint>") to access the mount as a FlexMount.

In very limited circumstances, it is not possible to choose a cloudroot that can also be used as the --root of the mount point (for example, if the cloudroot is chosen to be somewhere that is not mountable). In this case, we would advise you to choose a different cloudroot, one that can be mounted (such as a position within your user home directory e.g. cuno -o cloudroot=$HOME/mycloudroot mount $HOME/my-object-storage --root $HOME/mycloudroot). If this is not possible, then there are two options:

-. [Highly discouraged] If a cloudroot needs to be set when creating the mount, and the --root option is set to a path within the cloudroot (e.g. cuno -o cloudroot=/altcloudroot mount $HOME/my-object-storage --root /altcloudroot/s3/mybucket), then, when using the FlexMount later, one must set the ‘cloudrootover’ path as so: cuno -o cloudroot=/altcloudroot -o cloudrootover=$HOME/my-object-storage. Important: always check that the mount point exposes the cloud at the same point when used directly as a cunoFS Mount (so outside of a cunoFS CLI instance) and when accessed as a FlexMount.

Warning

cunoFS behaviour will be broken and dangerous if you do not set the correct options when launching cunoFS CLI to this type of FlexMount. For example, in this case if the default parameters for launching a FlexMount are used, namely cuno -o cloudrootover -o cloudroot=$HOME/my-object-storage, then applications that are intercepted directly will see $HOME/my-object-storage as the cloudroot (so the immediate children are az gs s3) whereas any applications that fall through to the cunoFS Mount (static binaries, cloud binaries, SUID binaries, Snap apps, AppImage apps, and Flatpak apps) will see $HOME/my-object-storage as the bucket or directory specified as the --root. This could cause a loss of data, accidentally expose data in the wrong place or other misbehaviour when running scripts.

-. [HIGHLY discouraged] If a cloudroot needs to be set when creating the mount, and it is not possible for ‘–root’ to be set to a path within the cloudroot (i.e. cuno -o cloudroot=/altroot mount $HOME/my-object-storage --root /), then, when using the FlexMount later, one must set the cloudroot as so: cuno -o cloudrootover -o cloudroot=$HOME/my-object-storage/my/cloud/root.

Warning

cunoFS behaviour will be broken and dangerous if you do not set the correct options when launching cunoFS CLI to this type of FlexMount. For example, in this case if the default parameters for launching a FlexMount are used, namely cuno -o cloudrootover -o cloudroot=$HOME/my-object-storage, then applications that are intercepted directly will see $HOME/my-object-storage as the cloudroot (so the immediate children are az gs s3) whereas any applications that fall through to the cunoFS Mount (static binaries, cloud binaries, SUID binaries, Snap apps, AppImage apps, and Flatpak apps) will see $HOME/my-object-storage as / - the root of your local filesystem. This could cause a loss of data, accidentally expose data in the wrong place or other misbehaviour when running scripts.

Ownership and Permissions#

Core File Access#

In Core File Access mode, and for files in object storage uploaded using tools other than cunoFS, we have some dynamic defaults set for ownership and permissions. In these circumstances, the owner of cloud objects is always reported to be the current user, the directory mode is reported as 0777, and the file mode is reported as 0666.

The defaults can be overridden by using the uid, gid, filemode and dirmode options within the CUNO_OPTIONS environment variable. By doing so, an administrator can set the default UID/GID and access mode permissions that apply to all files and directories of cloud storage accessed by cunoFS. This might be considered in circumstances where an application starts as one user, but mid-process switches context to run as another; services like web-servers often function in this way.

To demonstrate this in action, you could run the following:

$ cuno -o 'uid=0 gid=0 filemode=0444 dirmode=0447' \
  mount --root s3://examplebucket/directory/ /mnt/s3-bucket

$ ls -l /mnt/s3-bucket
dr--r--rwx  1 root root 0 Jan  1  1970 directory
-r--r--r--  1 root root 0 Nov 10 11:16 file

While the default UID/GID and access mode permissions can be set ahead of time, they cannot be later modified.

POSIX File Access#

To persist and modify file system metadata, POSIX File Access must be enabled. This allows the virtual files presented by cunoFS to have their system metadata modified by tools such as chown, chmod, setfacl, and touch.

Note

The system metadata is stored as objects within your object storage system within hidden subdirectories where the data resides. The name of that subdirectory is a period followed by a unicode U+FBF80 character.

From within cunoFS you will not see that directory or objects inside of it, as cunoFS filters them from view; however, accessing your object storage using other tools, they will be visible.

Non-cunoFS access which renames, moves or copies objects with cunoFS file attributes, will result in those objects losing their system metadata. You will need to use cunoFS to manage those files and preserve their attributes.

The functionality below requires upcoming features to function as described.

Enabling for Direct Interception#

There are two ways to enable POSIX File Access when using Direct Interception via cunoFS CLI:

  1. Using cuno creds setposix.

Note

The following permissions are required to use the cuno creds setposix command:

  • S3-compatible: s3:PutBucketTagging and s3:GetBucketTagging

  • Google Cloud Storage: storage.buckets.update and storage.buckets.get IAM permissions

  • Azure Blob Storage: Only the container owner may set labels

This setting toggles POSIX mode on for the bucket and takes precedence over the CUNO_POSIX environment variable.

cuno creds setposix s3://examplebucket true

Valid options are:

  • true - enable posix in enforcing mode, preventing unauthorized access.

  • false - disable posix

  • permissive - enable posix in permissive mode, where object attributes like uid, gid, mode, timestamps are stored but user access privilege is not checked

  • metadata - enable posix in enforcing mode and store posix attributes as object metadata

  • permissive-with-metadata - enable posix in permissive mode and store posix attributes as object metadata

  • unset - remove posix configuration from the bucket

The cuno creds setposix command writes a tag to the bucket’s metadata on the bucket store. The setting set by this command applies to all users using the bucket.

Warning

Some S3-compatible stores may not support setting custom metadata flags required for this feature to function.

To unset the bucket-level setting and allow controlling POSIX mode using either the CUNO_POSIX environment variable or the --posix command line flag, use cuno creds setposix with a setting of unset:

cuno creds setposix s3://examplebucket unset
  1. If POSIX mode is not set at the bucket level, it may be enabled by setting the environment variable CUNO_POSIX to 1.

Enabling for a cunoFS Mount#

If a bucket doesn’t have POSIX mode already set, you can use the option --posix when mounting a cunoFS Mount to enable fine-grained control over ownership and permissions. If a bucket’s POSIX mode tag is set, the --posix flag is ignored and the bucket’s setting will apply (see Ownership and Permissions).

Usage examples#

Assuming a mount location of /mnt/s3-bucket:

  1. Change the permissions of the object:

$ chmod 0770 /mnt/s3-bucket/file
$ ls -l /mnt/s3-bucket/file
-rwxrwx---  1 root  root 0 Nov 10 11:16 file
  1. Change the owner of the object:

$ chown alice /mnt/s3-bucket/file
$ ls -l /mnt/s3-bucket/file
-rwxrwx---  1 alice root 0 Nov 10 11:16 file
  1. Add an additional access control entry to the object:

$ setfacl --modify bob:rwx /mnt/s3-bucket/file
$ ls -l /mnt/s3-bucket/file
-rwxrwx---+ 1 alice root 0 Nov 10 11:16 file
$ getfacl /mnt/s3-bucket/file
# file: file
# owner: alice
# group: root
user::rwx
user:bob:rwx
group::rwx
other::---
  1. Alter the modify time:

$ echo "Hello" > /mnt/s3-bucket/file
$ ls -l /mnt/s3-bucket/file
-rwxrwx---+ 1 alice root 6 Nov 10 11:32 file
  1. Alter the change time

$ chmod +x /mnt/s3-bucket/file
$ stat /mnt/s3-bucket/file
File: /mnt/s3-bucket/file
Size: 6               Blocks: 1          IO Block: 65536  regular file
Device: 49h/73d Inode: 4           Links: 1
Access: (0770/-rwxrwx---)  Uid: ( 1009/   alice)   Gid: ( 0/    root)
Access: 2021-11-10 11:32:59.185000000 +0000
Modify: 2021-11-10 11:32:59.185000000 +0000
Change: 2021-11-10 11:47:21.846000000 +0000

Other Environment Variables#

CUNO_CREDENTIALS#

The environment variable CUNO_CREDENTIALS allows you to customise the location of the cunoFS credentials store, and is elaborated upon in Credential management.

Proxy server tunneling#

cunoFS supports HTTP and HTTPS proxying access through to your cloud object storage.

Provide the environment variable http_proxy or https_proxy with your gateway and cunoFS will tunnel all transactions through your designated proxy.

Verbose Debug Output#

There are rare occasions where cunoFS will output to the command line, and this will generally indicate a fault. All messages will be prefixed with cuno: and these will always be sent a to the standard error stream (stderr).

The options available for CUNO_LOG are cumulative and build on the previous level:

error

At this level, only critical failures are output. (Default)

warning

At this level, errors and warnings are output.

debug

At this level, information relevant to debugging will be output.

trace

At this level, a verbose logging of all steps will be output.

For example, a directory listing with the trace level set, can output similar to the below:

$ CUNO_LOG=trace ls s3://example-bucket
cuno: [TRACE][01/02/21 08:39:07.39][Thread: 3817873920] Trying bucket store at: /home/user/.config/cuno/creds/...
cuno: [TRACE][01/02/21 08:39:07.40][Thread: 3817873920] Loading S3Client
cuno: [DEBUG][01/02/21 08:39:07.40][Thread: 3817873920] Not On EC2
cuno: [TRACE][01/02/21 08:39:07.40][Thread: 3817873920] Attempting to find CaCerts.
cuno: [DEBUG][01/02/21 08:39:07.40][Thread: 3817873920] Using Certs: /etc/ssl/certs/ca-certificates.crt
cuno: [TRACE][01/02/21 08:39:07.40][Thread: 3817873920] Attempting to load PGMAN
cuno: [DEBUG][01/02/21 08:39:07.40][Thread: 3817873920] Using custom Endpoint: http://127.0.0.1:8080
cuno: [DEBUG][01/02/21 08:39:07.40][Thread: 3817873920] Using custom pathstyle

There is a separate option: access that enables detailed logging of the file access performed by cunoFS, which includes read, write, open, close, seek and similiar. access can be combined with the other log levels to log both file access and the desired logs. For example:

$ CUNO_LOG=trace,access ls s3://example-bucket
cuno: [DEBUG][07/17/23 13:59:03.287][P/T: 296181:2897806400] Not On EC2
cuno: [DEBUG][07/17/23 13:59:03.287][P/T: 296181:2897806400] Using Certs: /etc/ssl/certs/ca-certificates.crt
cuno: [ACCESS][07/17/23 13:59:03.355][P/T: 296181:2897806400] s3://example-bucket/file write() 12 @ 0