Basic loading#

cunoFS may be loaded in a number of ways, depending on user requirements and environment.

For program calls to be routed through cunoFS, they must use

Directly Intercepted using the cuno command line or LD_PRELOAD environment variable;
or the program must be pointed to a cunoFS Mount (a custom FUSE mount) set up using cuno mount.

There is also cunoFS FlexMount, which makes use of both Direct Interception and a cunoFS Mount as a fallback.

Direct Interception with cunoFS CLI#

When running in Direct Interception mode, cunoFS will insert itself between your applications and the operating system. It will dynamically redirect relevant filesystem calls made by the application through the API of your object storage solution. As a result, our mapping between filesystem concepts and object storage gives any application instant access to objects as files.

Note

This includes applications you’ve written yourself - just treat the paths to object storage as local filesystem paths, and cunoFS handles the rest.

Direct Interception offers the highest performance access to object storage that cunoFS provides.

Warning

Direct interception does not currently support SUID binaries, or certain packaged apps like Snap, AppImage, or Flatpak applications. Future updates are planned to address this.

This mode is best suited for situations in which installed app compatibility is not a concern - for example, when setting up a fixed workflow with cunoFS that can be tested and verified before putting it into production.

Direct Interception can be enabled per-session or per-command. Enable it for a session by calling cuno on the command line:

cuno

This will launch a new interactive “wrapped” shell with cunoFS acting in Direct Interception mode. The shell itself has cunoFS intercepting its calls, so every application launched from within it can be intercepted as well as the command line arguments being used.

Note

When the cunoFS CLI is used to launch a new shell, the primary purpose is to start a new shell (whether that’s bash, zsh, or whatever else) with LD_PRELOAD set to point at cuno.so. There is no cunoFS shell binary - it only wraps existing shells with cunoFS pre-loaded.

To enable direct interception of a single command, use:

cuno run bash -c "<your command and arguments>"

Note

Always execute one-time commands this way to maintain support for wildcard expansion (*) .

Note

How it works

Direct interception uses the LD_PRELOAD environment variable so that cunoFS can capture and redirect storage access library and system calls through object storage APIs. If a static binary is intercepted, a JIT ELF binary translator will redirect relevant calls when the binary is loaded into memory.

Advantages and Disadvantages#

Direct Interception - Advantages and Disadvantages :header-rows: 1 :widths: 1 1#
Advantages	Disadvantages
Speed: Dynamic interception offers the best performance.	Compatibility: Direct interception does not currently support SUID binaries, Snaps, AppArmor, or Flatpak applications.
Set up time: This is the default; all you have to do is launch a cunoFS shell with the `cuno` command line utility. No FUSE mount set up is required.	Activation: In some cases, it is difficult to enable direct interception or keep it enabled (because of environment variable clobbering or lack of privileges to set use LD_PRELOAD).
_	Non-technical users: If cunoFS is being used by many people in an organization, teaching people how to use the cunoFS shell and diagnose if it’s working correctly may be less desirable than setting up a cunoFS Mount on their behalf.

Usage#

When Direct Interception is enabled using cuno or cuno run <command>, object storage is accessed either in path or URI formats:

$ cuno
(cuno) $ ls s3://<bucket>/<path>
(cuno) $ ls /cuno/s3/<bucket>/<path>

See Transparent access to object storage for more information, options and examples of using Direct Interception with the cunoFS CLI.

Fully supported shells#

bash and zsh are fully supported when using cunoFS CLI. This includes support for:

tab auto-completion of remote paths;
wildcard expansion (*) of filesystem and remote paths.

In either of these fully-supported shells, the prompt will be prefixed with (cuno) to indicate that Direct Interception is enabled and that you are using a cunoFS CLI shell, like so:

(cuno) user@host:~$

Note

To select between the two shells, cunoFS detects if either is present in one of the following, in order:

the shell used to launch cunoFS;
the user’s preferred login shell;
all installed shells.

If neither bash nor zsh are found, then the first shell in this list is used. If no shell is set, then /bin/sh is used.

To launch a specific shell with cunoFS enabled, use cuno run:

cuno run <shell>

cunoFS Mount#

cunoFS Mount allows you to mount an object storage path in a directory within the local file system hierarchy. This allows you and any other user of the mount to access object storage as if it were just another directory.

cunoFS Mount uses Linux FUSE (Filesystem in Userspace) v3 to mount an object storage path in a directory within the file system hierarchy. Due to the nature of FUSE file systems, cunoFS Mount is usually less performant than Direct Interception using cunoFS CLI. Consider using cunoFS CLI or the userspace library (described in User-mode library) if speed is the primary objective.

Note

The --posix option requires that a FUSE package be installed on the system.

Advantages#

Compatibility: use a cunoFS Mount when extensively using cunoFS with a variety of operating systems, applications or when compatibility is a particular concern.
Simple usage: after a mount is set up, scripts only need to be changed to point at it, without any further changes to the workflow. No environment variables need to be set, and no cunoFS-wrapped shell has to be launched.
Greater admin control: a centralised cunoFS Mount can be set up by an admin, without giving users any credentials. This allows the admin to abstract away the concepts of object storage entirely from users.
When launching the cuno binary is not possible (limited cases, e.g. an automation engine without the required features).

Disadvantages#

You cannot use URI paths like “s3://bucket1/foo” directly when using a cunoFS mount. To convert such a path to be usable, you need to replace the URI prefix (s3://) with the path to your mount.

Usage and options#

cuno [cuno-options] mount [mount-options] <mount-path>

There are various options that can be specified, including

cunoFS Mount options
cunoFS options for the mount’s cunoFS subsystem
FUSE options

Mount options#

On the right hand-side of the mount verb, you can specify options that are specific to the mount operation.

cuno mount [option] ...

Note

These options must come AFTER the verb mount.

cunoFS Mount options#
Option	Description
`--root <cloud path>`	Specifies the root object storage path to be mounted. For example, to mount a single bucket, you would use `--root s3://<bucket>`.
`--posix`	Enables setting and enforcing access permissions, symbolic and hard links, users, groups, etc. Can be used with default permissions. Implicitly sets `-o allow_other`. This option is ideally used with a mount that is only on a single bucket, e.g. `cuno mount --root s3://<bucket> --posix <mount-path>`. You should also be using `cuno creds setposix s3://examplebucket true` to enable POSIX handling of the bucket consistently when Direct Interception or a FlexMount is used.
`<FUSE option>`	Options that are specific to the FUSE mount operation. These options are passed on to `fum` ( fusermount3). Not to be confused with cunoFS subsystem options which use the same `-o` syntax. See below for some of these. See FUSE options.
`--no-allow-root`	Do not allow root to access the mount (allowed by default). Disables support for SUID binaries, Snap, AppArmor, and Flatpak applications.
`--auto-restart`	Automatically restarts cunoFS Mount if problems occur during execution.
`--mkdir`	Automatically creates the mount point directory if it does not exist.
`--debug`	Enables debug output (same as `cuno mount -o debug`).
`--dev-logs`	Enabled debug logging to `/tmp/fuse.logs`
`--verbose`	Enables verbose output.

Options to configure the mount’s cunoFS subsystem#

On the left hand-side of the mount verb, you can specify options that are specific to the cunoFS subsystem.

cuno -o <cunoFS option> mount ...

export CUNO_OPTIONS="<cunoFS option>"

Some relevant cunoFS options are given below. Refer to Configuration options for more information.

Note

These options must come BEFORE the verb mount.

cunoFS options relevant to mount#
cunoFS option	Description
`cachehome=<directory>` (default: `/cunodb;/dev/shm;/tmp/cache/cuno`)	This option allows you to set a semicolon-delimited list of directories to consider for caching. This is for the internal metadata cache of the cunoFS process behind the mount. Note that the FUSE cache is separate from this cache. Example: cuno -o cachehome=/mnt/cache;/cunodb;/dev/shm mount <mount-path>
`uid=<integer>`	Define the default user ownership of files and directories within a bucket. These are the UIDs that the cunoFS subsystem will feed to the FUSE mount for non-POSIX (core file access) buckets. Not recommended for most use cases, but may be useful if mounting non-POSIX buckets in the same mount as POSIX buckets. Ignored if the bucket has an enabled POSIX tag. Example: cuno -o uid=$(id -u <username>) mount <mount-path> To enforce these settings, use the FUSE mount option `-o default_permissions`.
`gid=<integer>`	Define the default group ownership of files and directories within a bucket. These are the GIDs that the cunoFS subsystem will feed to the FUSE mount for non-POSIX (core file access) buckets. Not recommended for most use cases, prefer to use POSIX file access. Not recommended for most use cases, but may be useful if mounting non-POSIX buckets in the same mount as POSIX buckets. Ignored if the bucket has an enabled POSIX tag. Example: cuno -o gid=$(id -g <username>) mount <mount-path> To enforce these settings, use the FUSE mount option `-o default_permissions`.
`filemode=<octal>`	Define the default file access permission bits of files within a bucket. Supply the octal (numeric) representation of the permissions you want to apply. These are the permissions that the cunoFS subsystem will feed to the FUSE mount for non-POSIX (core file access) buckets. Not recommended for most use cases, but may be useful if mounting non-POSIX buckets in the same mount as POSIX buckets. Ignored if the bucket has an enabled POSIX tag. For example, to set the default file access permission bits to 0770 (`-rwxrwx---`), use the following command: cuno -o filemode=0770 mount <mount-path> To enforce these settings, use the FUSE mount option `-o default_permissions`.
`dirmode=<octal>`	Define the default file access permission bits of directories within a bucket. Supply the octal (numeric) representation of the permissions you want to apply. These are the permissions that the cunoFS subsystem will feed to the FUSE mount for non-POSIX (core file access) buckets. Not recommended for most use cases, but may be useful if mounting non-POSIX buckets in the same mount as POSIX buckets. Ignored if the bucket has an enabled POSIX tag. For example, to set the default file access permission bits to 0770 (`drwxrwx---`), use the following command: cuno -o filemode=0770 mount <mount-path> To enforce these settings, use the FUSE mount option `-o default_permissions`.

FUSE options#

On the right hand-side of the mount verb, you can also specify options that are specific to the FUSE mount operation. These options are passed on to fum ( fusermount3).

cuno mount [FUSE option] ...

Note

These options must come AFTER the verb mount.

Some of these are provided using the same -o syntax as the cuno -o options, but they are not the same.

FUSE mount options#
FUSE option	Description
`-o allow_root`	Allows root access to the mount; required for SUID permissions.
`-o allow_other`	Allows other users to access the mount; requires `user_allow_other` in `/etc/fuse.conf`.
`-o default_permissions`	Enable permission checking by the kernel. To use Enforced POSIX, use `--posix`, which will set this internally.
`-o auto_unmount`	Automatically unmounts the mount when the process terminates.
`-o ro`	Mounts the file system read-only. To achieve a read-only mount through cunoFS, you can alternatively use `cuno mount --posix` and change the permissions on dirs/files in the mount manually to read-only using `chmod`. See here for more on standard mount options like `ro`, `rw`, etc.
`-o rw` (default)	Mounts the file system read-write.
`-o exec` (default)	Allow execution of binaries on the file system.
`-o noexec`	Disallow execution of binaries on the file system.
`-o clone_fd`	Uses a separate fuse device fd for each thread (may improve performance).
`-o max_idle_threads`	The maximum number of idle worker threads allowed (default: 10).
`-s`	Run in single-threaded mode.
`-f`	Run in the foreground.
`-o debug`	Enable debug output (implies `-f`).
FUSE kernel default attributes
`-o uid=N`	Sets the file owner of all mounted files/dirs to the specified user ID. These are the UIDs that the FUSE mount will expose to the user. This will ignore any cuonFS POSIX settings including the bucket tag. To enforce these settings, use the FUSE mount option -o default_permissions.
`-o gid=N`	Sets the file group of all mounted files/dirs to the specified group ID. These are the GIDs that the FUSE mount will expose to the user. This will ignore any cuonFS POSIX settings including the bucket tag. To enforce these settings, use the FUSE mount option -o default_permissions.
`-o umask=M`	Sets the file permissions (octal) of all mounted files/dirs. These are the permissions that the FUSE mount will expose to the user. This will ignore any cuonFS POSIX settings including the bucket tag. To enforce these settings, use the FUSE mount option -o default_permissions.
FUSE kernel cache configuration
`-o kernel_cache`	This method caches file data across `open`; i.e. disables flushing of the file data cache on every `open`. Without this option (and neither `direct_io`), data is cached before the next `open` so a `read` syscall may not initiate a `read` operation.
`-o auto_cache`	Enables automatic flushing of the data cache on open(). The cache is only flushed if the modification based on modification times (off).
`-o entry_timeout=T`	Sets the kernel cache timeout for names (1.0s).
`-o negative_timeout=T`	Sets the kernel cache timeout for a negative lookup (file not found) (0.0s).
`-o attr_timeout=T`	Sets the kernel cache timeout for attributes (1.0s).
`-o ac_attr_timeout=T`	Sets the kernel cache timeout for checking if `-o auto_cache` should flush file data on `open`.

cunoFS Mount commands#

Once a cunoFS Mount is set up, you can use the following commands to manage it.

cunoFS Mount commands#
Command	Description
`--list`	Lists your active cunoFS Mounts. You can also the use the Linux utility `mount` directly. Example usage: cuno mount --list
`--unmount` / `-u`	Unmounts a cunoFS Mount from the specified mount path. You can also use the Linux utility `umount` directly. Example usage: cuno mount --unmount <path to mount>
`--unmount-kill` / `-U`	Unmounts a cunoFS Mount from the specified mount path and kills the fum process. You can also use the Linux utility `umount` directly. Example usage: cuno mount --unmount-kill <path to mount>

Mount on boot#

You can add mount commands to /etc/fstab to automount on start up.

cunoFS FlexMount#

cunoFS FlexMount is for when the speed of direct interception is preferred whenever it is possible to use, but wider-support for different contexts and types of applications is also required.

FlexMount is used by setting up a cunoFS Mount and then accessing the cloud using cunoFS CLI always “through” the mount path. CunoFS will recognise that the path is a mount and use direct interception where possible for faster access. It will naturally fall back to the cunoFS Mount for anything that cannot be directly intercepted.

Usage#

A FlexMount is set up as follows:

First, set up a cunoFS Mount, with a cloudroot set:

$ mkdir "$HOME/my-object-storage"
$ cuno mount "$HOME/my-object-storage"

Each time a cunoFS CLI is launched, use the parameters --flex ="<full path to mountpoint>", for example:
```
$ cuno --flex "$HOME/my-object-storage"
(cuno) $ ls $HOME/object_storage_mount/s3/<bucket>/<path>
```
Warning

You cannot use a tilde ~ in your CUNO_OPTIONS or cuno -o CLI options when setting up a FlexMount as this is something that the shell needs to resolve. You may still use it for your cuno mount commands, and subsequent FlexMount operations (such as ls ~/my-object-storage).

The same FlexMount can be re-used across multiple cunoFS wrapped shells.

The -flex option can also be used with cuno run to run a single command/script with cunoFS enabled, for example:

cuno run --flex "$HOME/my-object-storage" bash -c "touch $HOME/object_storage_mount/s3/<bucket>/newfile"

The -flex option is synonymous with the -o cloudrootover=exact -o cloudroot="<mountpoint>" option, which is used to tell cunoFS to intercept paths that exactly match the cloudroot setting, and to handle them using the cunoFS Mount.

When using cunoFS Direct Inteception in FlexMount mode, most “local” paths are intercepted but not actioned upon, because they can be handled by the local file system. The cloudrootover setting is telling cunoFS to intercept local paths that match the cloudroot setting, and to prioritise itself handling them over the mount. This means that whenever a path is recognised as the cloudroot, it can be more efficiently handled in user-space by the Direct Inteception/LD_PRELOAD library. That path recognition can be done in two ways:

(default) Exact string matching (exact) will match the cloudroot setting exactly. This is faster, and will end up relying on the FUSE mount whenver the paths don’t match the cloudroot - for example with symbolic links located outside the mount pointing into the mount.
Resolved path matching (resolve) will resolve the full path given in a file system call including symbolic links to check if the file is ultimately located inside the mount point. This requires more calls for every path-based file system call, so is slower when many files are being accessed. It is useful when symbolic links are used to point into the mount, and other cases where the path ultimately is inside the mount. This can be more efficient in some special cases (few files, large transfers, complex relationships between files) as cuno.so will intercept more calls without requiring them to go through the slower FUSE mount.

To use resolve mode use the parameters -o cloudrootover=resolve -o cloudroot="<full path to mountpoint>", for example:
```
$ cuno -o cloudrootover=resolve -o cloudroot="$HOME/my-object-storage"
(cuno) $ ls $HOME/object_storage_mount/s3/<bucket>/<path>
```

Advantages#

Speed: where interception is possible, cunoFS in FlexMount mode will be as fast as cunoFS
Support: support for all POSIX applications, as anything that cannot be directly intercepted falls through to the cunoFS Mount.

Disadvantages#

Set up time: a cunoFS Mount needs to be set up. Consider setting up the mount at boot.
Launch is more complicated: each time a cunoFS shell is launched it must be configured to use the mount. This can be worked around by setting up a cunoFS mount on boot, and setting an alias to launch a cunoFS shell with the correct parameters.

Basic loading

Contents

Basic loading#

Direct Interception with cunoFS CLI#

Advantages and Disadvantages#

Usage#

Fully supported shells#

cunoFS Mount#

Advantages#

Disadvantages#

Usage and options#

Mount options#

Options to configure the mount’s cunoFS subsystem#

FUSE options#

cunoFS Mount commands#

Mount on boot#

cunoFS FlexMount#

Usage#

Advantages#

Disadvantages#