Configuration options#
CUNO_OPTIONS#
Overview#
Setting the environment variable CUNO_OPTIONS can be used to further configure cunoFS. The -o option on the cuno command can also be used to specify these options.
Options available for CUNO_OPTIONS:
off |
Disable cunoFS access. |
+static |
Enable interception of statically compiled binaries. (default) |
-static |
Disable interception of statically compiled binaries. |
+closekeepcache=[N] |
Keep recently closed virtual files cached for N*0.1 seconds. This helps the performance of applications like Apache which frequently open/close/reopen files. (Disabled by default, if |
+cloud |
Enable cloud access using both |
-cloud |
Disable cloud access. If followed by +urifile, then only |
+urifile=strict |
Enable cloud access using strict URI formats such as |
+urifile |
Slight variations of URI format |
+urifile=any |
|
-urifile |
Disable cloud access using URIs such as |
+uricompat=[binary1:binary2:…] |
When a paired URI is detected, and if the binary is a member of this colon-separated list, then URI remapping is enforced, instead of the binary’s own URI handling scheme. Can also be wildcard |
-uricompat |
Disable URI remapping scheme for all binaries. |
-forceuri |
URI remapping is enforced for all URIs, not just paired URIs. |
+hardlink |
Enable hard link functionality. (default ONLY when POSIX File Access is enabled) |
-hardlink |
Disable hard link functionality. |
+cloudroot=[directory] |
Define a custom directory path prefix that will be used instead of ‘/cuno’ to introduce cloud locations. For how this relates to FlexMount and the effects of setting cloudroot when mounting a cunoFS Mount, see the The cloudroot option section. Using a tilde, ‘~’, in the path is unsupported. (default: /cuno) |
-cloudrootover |
cunoFS Direct Interception will prioritise the cunoFS Mount at the path specified by the cloudroot option. This means that any intercepted calls to a location within the cloudroot path will be handled by the cunoFS Mount at that location. This disables FlexMount mode, equivalent to ‘cloudrootover=off’. (default) |
+cloudrootover |
cunoFS Direct Interception will prioritise itself over anything at the path specified by cloudroot, but only if the input path exactly matches the text in cloudroot, equivalent to ‘cloudrootover=exact’. Use this to enable FlexMount behavior at the cloudroot. |
cloudrootover=[off,exact,resolve,exe,<path-to-cunoFS-mount] |
These settings affect what happens to when cunoFS Direct Interception (i.e. when cunoFS CLI is launched or LD_PRELOAD is set) encounters a path which already exists on the local filesystem, and affects the FlexMount behaviour if that local path refers to a cunoFS Mount. See the descriptions of each individual setting to learn more about what they do. (default: off) |
cloudrootover=off |
cunoFS Direct Interception will prioritise local directories, including the cunoFS Mount at the path specified by the cloudroot option. This means that any intercepted calls to a location within the cloudroot path will be handled by the cunoFS Mount at that location. This disables FlexMount mode. (default) |
cloudrootover=exact |
cunoFS Direct Interception will prioritise itself over local directories at the path specified by cloudroot, but only if the input path exactly matches the text in cloudroot. This also disables cunoFS Direct Interception’s static binary translator and cloud binary interception, so static binaries and binaries in object storage are passed through to the cunoFS Mount at the cloudroot. Use this to enable FlexMount behavior at the cloudroot. |
cloudrootover=resolve |
cunoFS Direct Interception will prioritise itself over anything that resolves to the same local path as the path specified by cloudroot. This also disables cunoFS Direct Interception’s static binary translator, so static binaries and binaries in object storage are passed through to the cunoFS Mount at the cloudroot. Use this to enable FlexMount behavior at the cloudroot, in a more resilient way. |
cloudrootover=exe |
cunoFS Direct Interception of static binaries and binaries on object storage will be re-enabled. Must be used in conjunction with a FlexMount-enabling cloudrootover option. This also enables ‘cloudrootover=resolve’. |
cloudrootover=<path-to-cunoFS-mount> |
The <path-to-cunoFS-mount> only needs to specified if an explicit ‘–root’ option was used when creating the cunoFS Mount. In this case, assuming the ‘–root’ value was set to a bucket or a directory on object storage (e.g. ‘cuno mount “$HOME/mybucket” –root /cuno/s3/mybucket/foo’), then you must set cloudrootover to the mount point AND not set cloudroot when launching cunoFS CLI to use the mount as FlexMount (e.g. ‘cuno -o cloudrootover=”$HOME/mybucket”’). This also implies ‘cloudrootover=exact’. If a cloudroot option was specified when creating the cunoFS Mount, or the –root option was used when creating the mount and refers to anywhere that isn’t inside the default cunoFS Mount cloudroot (‘/cuno’), then refer to the User Guide section The cloudroot option. Using a tilde, ‘~’, in the path is unsupported. |
-cloudrootoverunload |
We prefer that certain shells and debuggers (e.g. sh, tsh and gdb) disable/unload cunoFS Direct Interception in FlexMount-enabled modes for stability. Note that Direct Interception is re-enabled for any programs that they launch. This option disables this unloading behaviour. Warning: this may break things when you run these shells. |
+cloudrootoverunload |
We prefer that certain shells and debuggers (e.g. sh, tsh and gdb) disable cunoFS Direct Interception in cloudrootover enabled modes for stability. This option disables cunoFS Direct Interception (enables un-loading) when cloudrootover is enabled. (default) |
cloudrootoverunload=[binary1:binary2:…] |
Colon-separated list of paths to binaries. We prefer that certain shells and debuggers (e.g. sh, tsh and gdb) disable cunoFS Direct Interception in FlexMount-enabled modes for stability. This option disables cunoFS Direct Interception (enables un-loading) when FlexMount is enabled and one of binaries specified is launched. (default: bin/sh:bin/tsh:bin/gdb) |
cloudrootoverunload=+<path-to-binary> |
Appends <path-to-binary> to the list of ‘cloudrootoverunload’ binaries. This option will disable cunoFS Direct Interception (enable un-loading) in FlexMount-enabled modes when the specified binary is launched. |
-cachehome |
Disable cunoFS high performance metadata caching. |
+cachehome=[directory] |
Enable cunoFS high performance metadata caching and optionally set a semicolon-delimited list of directories to consider for caching (default: |
retry-multiplier=[decimal] |
Scaling factor for the number of retries for non-fatal failed requests made to the server, e.g. |
uid=[integer] |
Define the default user ownership of files and directories within a bucket. Supply the user identifier in the numerical representation derived from |
gid=[integer] |
Define the default group ownership of files and directories within a bucket. Supply the group identifier in the numerical representation derived from |
filemode=[octal] |
Define the default file access permission bits of files within a bucket. Supply the octal (numeric) representation of the permissions you want to apply. |
dirmode=[octal] |
Define the default file access permission bits of directories within a bucket. Supply the octal (numeric) representation of the permissions you want to apply. |
reset |
Resetting discards any options from previous CUNO_OPTIONS assignments within the current shell. Subcommands only see the current CUNO_OPTIONS. |
Note
Individual options must be separated by spaces. Don’t forget to enclose multiple options in quotes (" or '), or to escape spaces.
The static option#
To use cunoFS with static binaries, the parent process needs to be intercepted in static mode. Interception of static binaries is enabled by default; to disable it requires setting both CUNO_OPTIONS="-static" and starting a new cunoFS wrapped shell.
The uricompat option#
cunoFS can override the handling of URI-style paths in applications that attempt to provide internal handling of such URIs. Users may provide the +uricompat option to add similar patching for further applications.
To specify any applications to be patched, specify them as a colon-separated list:
+uricompat=<executable 1>:<exectuable 2>:< ... >
To limit the cases in which this patching occurs, you can filter interceptions to only occur when certain command-line arguments match specified keywords. This may be useful for certain code interpreters/executors (see the example below). To do this, in place of an executable name, use a slash-separated list of the name, a 1-based index of the argument to be matched (or any using *), and a value to match against:
+uricompat=<name of binary>/<index of argument to be matched>/<argument value to match>
Note
The +uricompat option only works on URIs that have been paired.
If you are working with a public access bucket, you will need to run the cuno creds pair command (see Pair containers and credentials) against it for the +uricompat to take effect.
Applications with patched URI handling#
By default, cunoFS will attempt to override the internal URI handling of popular tools, such as:
rsyncffmpegtar
And some genomics tools:
samtoolsigvfastQC
This means that when a URI-style path (corresponding to a paired bucket or container) is passed to any of the applications above, cunoFS will prevent the application from handling the path in a special way.
For example, fmpeg has special handling for some protocols which they specify in a similar URI format to cunoFS URI cloud paths (e.g. ftp://example.foo). As a result, cunoFS needs to prevent ffmpeg from failing (when given a path like s3://bucket/file) because s3, az and gs are not protocols it supports.
Example#
To override a Java application’s handling of URI-style paths, we may use:
export CUNO_OPTIONS="+uricompat=java/2/appname.jar:java/*/org.name.main.Class"
This will intercept any invocation of java with appname.jar as its second argument, e.g. java -jar appname.jar s3://bucketname/file.bam. It will also intercept invocations of Java where any argument is equal to org.name.main.Class, e.g. java org.name.main.Class s3://bucketname/file.bam.
The cloudroot option#
You may define a custom directory path prefix to replace the default /cuno path (see :ref:`` ). Once defined, the path specified will be made available at the root of the file system. For example, if you configure the following:
export CUNO_OPTIONS='+cloudroot=my-object-storage'
You will then be able to list your configured S3 buckets with:
ls /my-object-storage/s3/<bucket>
Warning
We strongly recommend that the cloudroot path does not exist locally, because it can confuse cunoFS ability to distinguish whether a path refers to the local file system or to the cloud.
If you specify a cloudroot which conflicts with an existing directory, then it must not contain directories or files named s3, az or gs.
Note
In order for the “cd to cloud” functionality to see this option, you will need to spawn a new shell after CUNO_OPTIONS.
FlexMount usage#
Setting a cloudroot during the mounting process can affect the options required to be set when using it as a FlexMount.
[Recommended option] If cloudroot was not set when creating the mount (e.g. cuno mount ~/my-object-storage), then cloudroot needs to be set as the path to the mountpoint when launching cunoFS CLI in order to use that mount as a FlexMount (e.g. cuno -o cloudrootover -o cloudroot="<full-path-to-mountpoint>").
In some circumstances, you may need to set a cloudroot when creating the mount (for example, if a /cuno directory already exists on your systems). In these cases, we strongly recommend migrating or removing any such physical directory so that you can continue to use the default settings. If this is not possible, there are a number of options available.
[Recommended option if cloudroot must be changed] If a cloudroot needs to be set when creating the mount, then you should set the --root option to the same cloudroot to get expected behaviour when using cunoFS FlexMount (e.g. cuno -o cloudroot=/altcloudroot mount $HOME/my-object-storage --root /altcloudroot). In this case, continue to use the same options when launching cuno (e.g. cuno -o cloudrootover -o cloudroot="<full-path-to-mountpoint>") to access the mount as a FlexMount.
In very limited circumstances, it is not possible to choose a cloudroot that can also be used as the --root of the mount point (for example, if the cloudroot is chosen to be somewhere that is not mountable). In this case, we would advise you to choose a different cloudroot, one that can be mounted (such as a position within your user home directory e.g. cuno -o cloudroot=$HOME/mycloudroot mount $HOME/my-object-storage --root $HOME/mycloudroot). If this is not possible, then there are two options:
-. [Highly discouraged] If a cloudroot needs to be set when creating the mount, and the --root option is set to a path within the cloudroot (e.g. cuno -o cloudroot=/altcloudroot mount $HOME/my-object-storage --root /altcloudroot/s3/mybucket), then, when using the FlexMount later, one must set the ‘cloudrootover’ path as so: cuno -o cloudroot=/altcloudroot -o cloudrootover=$HOME/my-object-storage. Important: always check that the mount point exposes the cloud at the same point when used directly as a cunoFS Mount (so outside of a cunoFS CLI instance) and when accessed as a FlexMount.
Warning
cunoFS behaviour will be broken and dangerous if you do not set the correct options when launching cunoFS CLI to this type of FlexMount. For example, in this case if the default parameters for launching a FlexMount are used, namely cuno -o cloudrootover -o cloudroot=$HOME/my-object-storage, then applications that are intercepted directly will see $HOME/my-object-storage as the cloudroot (so the immediate children are az gs s3) whereas any applications that fall through to the cunoFS Mount (static binaries, cloud binaries, SUID binaries, Snap apps, AppImage apps, and Flatpak apps) will see $HOME/my-object-storage as the bucket or directory specified as the --root. This could cause a loss of data, accidentally expose data in the wrong place or other misbehaviour when running scripts.
-. [HIGHLY discouraged] If a cloudroot needs to be set when creating the mount, and it is not possible for ‘–root’ to be set to a path within the cloudroot (i.e. cuno -o cloudroot=/altroot mount $HOME/my-object-storage --root /), then, when using the FlexMount later, one must set the cloudroot as so: cuno -o cloudrootover -o cloudroot=$HOME/my-object-storage/my/cloud/root.
Warning
cunoFS behaviour will be broken and dangerous if you do not set the correct options when launching cunoFS CLI to this type of FlexMount. For example, in this case if the default parameters for launching a FlexMount are used, namely cuno -o cloudrootover -o cloudroot=$HOME/my-object-storage, then applications that are intercepted directly will see $HOME/my-object-storage as the cloudroot (so the immediate children are az gs s3) whereas any applications that fall through to the cunoFS Mount (static binaries, cloud binaries, SUID binaries, Snap apps, AppImage apps, and Flatpak apps) will see $HOME/my-object-storage as / - the root of your local filesystem. This could cause a loss of data, accidentally expose data in the wrong place or other misbehaviour when running scripts.
Ownership and Permissions#
Core File Access#
In Core File Access mode, and for files in object storage uploaded using tools other than cunoFS, we have some dynamic defaults set for ownership and permissions. In these circumstances, the owner of cloud objects is always reported to be the current user, the directory mode is reported as 0777, and the file mode is reported as 0666.
The defaults can be overridden by using the uid, gid, filemode and dirmode options within the CUNO_OPTIONS environment variable. By doing so, an administrator can set the default UID/GID and access mode permissions that apply to all files and directories of cloud storage accessed by cunoFS. This might be considered in circumstances where an application starts as one user, but mid-process switches context to run as another; services like web-servers often function in this way.
To demonstrate this in action, you could run the following:
$ cuno -o 'uid=0 gid=0 filemode=0444 dirmode=0447' \
mount --root s3://examplebucket/directory/ /mnt/s3-bucket
$ ls -l /mnt/s3-bucket
dr--r--rwx 1 root root 0 Jan 1 1970 directory
-r--r--r-- 1 root root 0 Nov 10 11:16 file
While the default UID/GID and access mode permissions can be set ahead of time, they cannot be later modified.
POSIX File Access#
To persist and modify file system metadata, POSIX File Access must be enabled. This allows the virtual files presented by cunoFS to have their system metadata modified by tools such as chown, chmod, setfacl, and touch.
Note
The system metadata is stored as objects within your object storage system within hidden subdirectories where the data resides. The name of that subdirectory is a period followed by a unicode U+FBF80 character.
From within cunoFS you will not see that directory or objects inside of it, as cunoFS filters them from view; however, accessing your object storage using other tools, they will be visible.
Non-cunoFS access which renames, moves or copies objects with cunoFS file attributes, will result in those objects losing their system metadata. You will need to use cunoFS to manage those files and preserve their attributes.
The functionality below requires upcoming features to function as described.
Enabling for Direct Interception#
There are two ways to enable POSIX File Access when using Direct Interception via cunoFS CLI:
Using
cuno creds setposix.
Note
The following permissions are required to use the
cuno creds setposixcommand:
S3-compatible:
s3:PutBucketTaggingands3:GetBucketTaggingGoogle Cloud Storage:
storage.buckets.updateandstorage.buckets.getIAM permissionsAzure Blob Storage: Only the container owner may set labels
This setting toggles POSIX mode on for the bucket and takes precedence over the
CUNO_POSIXenvironment variable.cuno creds setposix s3://examplebucket trueValid options are:
true - enable posix in enforcing mode, preventing unauthorized access.
false - disable posix
permissive - enable posix in permissive mode, where object attributes like uid, gid, mode, timestamps are stored but user access privilege is not checked
metadata - enable posix in enforcing mode and store posix attributes as object metadata
permissive-with-metadata - enable posix in permissive mode and store posix attributes as object metadata
unset - remove posix configuration from the bucket
The
cuno creds setposixcommand writes a tag to the bucket’s metadata on the bucket store. The setting set by this command applies to all users using the bucket.Warning
Some S3-compatible stores may not support setting custom metadata flags required for this feature to function.
To unset the bucket-level setting and allow controlling POSIX mode using either the
CUNO_POSIXenvironment variable or the--posixcommand line flag, usecuno creds setposixwith a setting of unset:cuno creds setposix s3://examplebucket unset
If POSIX mode is not set at the bucket level, it may be enabled by setting the environment variable
CUNO_POSIXto 1.
Enabling for a cunoFS Mount#
If a bucket doesn’t have POSIX mode already set, you can use the option --posix when mounting a cunoFS Mount to enable fine-grained control over ownership and permissions. If a bucket’s POSIX mode tag is set, the --posix flag is ignored and the bucket’s setting will apply (see Ownership and Permissions).
Usage examples#
Assuming a mount location of /mnt/s3-bucket:
Change the permissions of the object:
$ chmod 0770 /mnt/s3-bucket/file $ ls -l /mnt/s3-bucket/file -rwxrwx--- 1 root root 0 Nov 10 11:16 file
Change the owner of the object:
$ chown alice /mnt/s3-bucket/file $ ls -l /mnt/s3-bucket/file -rwxrwx--- 1 alice root 0 Nov 10 11:16 file
Add an additional access control entry to the object:
$ setfacl --modify bob:rwx /mnt/s3-bucket/file $ ls -l /mnt/s3-bucket/file -rwxrwx---+ 1 alice root 0 Nov 10 11:16 file $ getfacl /mnt/s3-bucket/file # file: file # owner: alice # group: root user::rwx user:bob:rwx group::rwx other::---
Alter the modify time:
$ echo "Hello" > /mnt/s3-bucket/file $ ls -l /mnt/s3-bucket/file -rwxrwx---+ 1 alice root 6 Nov 10 11:32 file
Alter the change time
$ chmod +x /mnt/s3-bucket/file $ stat /mnt/s3-bucket/file File: /mnt/s3-bucket/file Size: 6 Blocks: 1 IO Block: 65536 regular file Device: 49h/73d Inode: 4 Links: 1 Access: (0770/-rwxrwx---) Uid: ( 1009/ alice) Gid: ( 0/ root) Access: 2021-11-10 11:32:59.185000000 +0000 Modify: 2021-11-10 11:32:59.185000000 +0000 Change: 2021-11-10 11:47:21.846000000 +0000
Other Environment Variables#
CUNO_CREDENTIALS#
The environment variable CUNO_CREDENTIALS allows you to customise the location of the cunoFS credentials store, and is elaborated upon in Credential management.
Proxy server tunneling#
cunoFS supports HTTP and HTTPS proxying access through to your cloud object storage.
Provide the environment variable http_proxy or https_proxy with your gateway and cunoFS will tunnel all transactions through your designated proxy.
Verbose Debug Output#
There are rare occasions where cunoFS will output to the command line, and this will generally indicate a fault.
All messages will be prefixed with cuno: and these will always be sent a to the standard error stream (stderr).
The options available for CUNO_LOG are cumulative and build on the previous level:
errorAt this level, only critical failures are output. (Default)
warningAt this level, errors and warnings are output.
debugAt this level, information relevant to debugging will be output.
traceAt this level, a verbose logging of all steps will be output.
For example, a directory listing with the trace level set, can output similar to the below:
$ CUNO_LOG=trace ls s3://example-bucket
cuno: [TRACE][01/02/21 08:39:07.39][Thread: 3817873920] Trying bucket store at: /home/user/.config/cuno/creds/...
cuno: [TRACE][01/02/21 08:39:07.40][Thread: 3817873920] Loading S3Client
cuno: [DEBUG][01/02/21 08:39:07.40][Thread: 3817873920] Not On EC2
cuno: [TRACE][01/02/21 08:39:07.40][Thread: 3817873920] Attempting to find CaCerts.
cuno: [DEBUG][01/02/21 08:39:07.40][Thread: 3817873920] Using Certs: /etc/ssl/certs/ca-certificates.crt
cuno: [TRACE][01/02/21 08:39:07.40][Thread: 3817873920] Attempting to load PGMAN
cuno: [DEBUG][01/02/21 08:39:07.40][Thread: 3817873920] Using custom Endpoint: http://127.0.0.1:8080
cuno: [DEBUG][01/02/21 08:39:07.40][Thread: 3817873920] Using custom pathstyle
There is a separate option: access that enables detailed logging of the file access performed by cunoFS, which includes read, write, open, close, seek and similiar.
access can be combined with the other log levels to log both file access and the desired logs. For example:
$ CUNO_LOG=trace,access ls s3://example-bucket
cuno: [DEBUG][07/17/23 13:59:03.287][P/T: 296181:2897806400] Not On EC2
cuno: [DEBUG][07/17/23 13:59:03.287][P/T: 296181:2897806400] Using Certs: /etc/ssl/certs/ca-certificates.crt
cuno: [ACCESS][07/17/23 13:59:03.355][P/T: 296181:2897806400] s3://example-bucket/file write() 12 @ 0