Skip to content

Gateways on Oak#

What are Oak Gateways?#

Oak Gateways are well-resourced servers that allow you to access your Oak storage. Oak is pre-configured on popular on-campus computing resources like Sherlock and SCG. If you're not using either of those systems to manage your storage, you'll need to use one of the gateway options listed below to connect to Oak.

Oak's internal network consists of multiple servers and storage devices, connected using advanced Infiniband fiber-optic technology to create a single filesystem. This system presents a challenge as Infiniband doesn't use the typical IPv4/6 protocol. To bridge this gap between our Infiniband network and IPv4/6 users, we employ gateways.

Gateways, capable of speaking both Infiniband and IPv4/6, function as an intermediary. With a 200Gb/s connection to Oak and a 25Gb/s Ethernet connection to the Stanford Research Network, they enable data transfer across network fabrics. Whether you're using a Data Transfer Node, Globus, or mounting Oak with NFS or SMB, gateways facilitate the process.

Types of Oak Gateways#

Oak Globus Endpoint#

Cost: Free for Oak users#
Example Use Case: Transferring large datasets in/out of Oak from non-Stanford locations or sharing files with collaborators outside of Stanford.#

Globus is a cloud-based data transfer and sharing platform widely used in research computing. Unlike the DTN (which requires command-line tools or an SFTP client), Globus provides a browser-based interface for moving large datasets reliably between endpoints, including other universities, national computing facilities, and your own laptop via Globus Connect Personal. It also supports sharing folders with outside collaborators who have a Globus account, without requiring them to have an Oak account.

Please see our Globus page for detailed instructions on how to connect and transfer files.

Oak Data Transfer Node (DTN)#

Cost: Free for Oak users#
Example Use Case: Using rsync to send or receive data between your Oak and your computer.#

Authorized Oak users can use the Oak Data Transfer Node (DTN). Similar to the Sherlock DTN, the following protocols are available: rsync, sftp, scp, sshfs, and bbcp.

The Oak DTN is available from anywhere and requires two-factor authentication using Duo (Kerberos is not required).

Rsync over SSH#

Rsync is widely used for backups and mirroring and as an improved copy command for everyday use. Please see the example below which would copy a directory named my_dir to your Oak's group directory (on Linux or macOS):

 rsync -rltP my_dir $SUNETID@dtn.oak.stanford.edu:/oak/stanford/groups/$GROUP

Known rsync Limitations

Due to the way Oak exposes file transfer tools on the DTNs, shell globbing (*) is not supported in remote paths. Instead, use a combination of --include and --exclude options to selectively transfer files. For example, to copy all files ending in .jpg:

✅ Do this:

rsync -rltP --include='*.jpg' --exclude='*' $SUNETID@dtn.oak.stanford.edu:/oak/stanford/groups/bprogers/images/ .

❌ Not this:

rsync -rltP $SUNETID@dtn.oak.stanford.edu:/oak/stanford/groups/bprogers/*.jpg images/ .

rsync on the DTN doesn't support all features like preservation of ACLs and permissions. It currently behaves pretty much like scp in that aspect.

rclone#

rclone is an actively developed command-line tool for copying and syncing files between local storage and a long list of remote storage systems. Its SFTP backend works with the Oak DTN, which makes it a good fit for one-off transfers, scheduled syncs, and (via rclone mount) mounting Oak as a filesystem.

This configuration depends on public key authentication

The key_file setting below assumes you have already set up public key authentication on the Oak DTN. Without it, every rclone invocation prompts for your SUNet ID password and a Duo approval, which makes unattended and scripted transfers impractical.

Keep in mind that public keys unused for 45 days are automatically removed from the DTN, and re-adding one requires Duo. A scheduled rclone job that sits idle longer than that will start failing until you re-install your key.

Add the following to your rclone config file, which lives at ~/.config/rclone/rclone.conf on Linux and macOS. Adjust key_file to point at your own private key.

[oak]
type = sftp
host = dtn.oak.stanford.edu
key_file = ~/.ssh/id_ecdsa
shell_type = unix
chunk_size = 255
md5sum_command = none
sha1sum_command = sha1sum

A few notes on why those settings are needed:

  • shell_type = unix tells rclone what kind of shell the remote offers. The DTN doesn't provide an interactive shell, so rclone can't detect this on its own.
  • md5sum_command = none disables MD5 checksums, which the DTN doesn't expose. sha1sum_command = sha1sum points rclone at the SHA-1 tool that is available, so rclone check and hash-based comparisons still work.
  • chunk_size = 255 is 255 KiB (bare numbers are interpreted as KiB), up from rclone's 32 KiB default. This sits just under the largest packet size the DTN's SFTP server accepts and noticeably improves throughput.
  • rclone defaults to your local username. If that differs from your SUNet ID, add a user = $SUNETID line.

You can also build this config interactively by running rclone config and choosing the sftp backend.

To confirm everything is wired up, list the subdirectories of your group directory. This exercises the key-based login, the connection to the DTN, and path resolution in one step:

rclone lsd oak:/oak/stanford/groups/$GROUP

Prefer lsd or lsf over ls for a quick look

rclone ls recurses by default, so pointing it at a large group directory will walk the entire tree. rclone lsd (directories) and rclone lsf (directory contents) do not recurse unless you pass -R. If you do want ls, bound it with --max-depth 1.

Once configured, refer to Oak as oak: followed by an absolute path:

# List a directory
rclone lsf oak:/oak/stanford/groups/$GROUP

# Copy a local directory up to Oak, showing progress
rclone copy -P ./my_dir oak:/oak/stanford/groups/$GROUP/my_dir

# Copy back down from Oak
rclone copy -P oak:/oak/stanford/groups/$GROUP/my_dir ./my_dir

# Verify that source and destination match
rclone check ./my_dir oak:/oak/stanford/groups/$GROUP/my_dir

rclone sync deletes files

rclone sync makes the destination match the source exactly, which means it removes files at the destination that are not present at the source. Always dry-run it first with --dry-run before pointing it at an Oak directory.

To mount Oak as a filesystem instead, which is the closest equivalent to the SSHFS approach described below:

mkdir ~/Desktop/Oak
rclone mount oak:/oak/stanford/groups/$GROUP ~/Desktop/Oak --vfs-cache-mode writes

This requires FUSE on Linux or macFUSE on macOS. Press Ctrl-C in the terminal running rclone mount to unmount.

SFTP#

University IT provides free licenses for some SFTP clients: Fetch for Mac and SecureFX for Windows. Below is an example of using Fetch on Mac. Replace $SUNETID with your SUNet ID and enter your SUNet ID password. The Initial Folder field is optional.

If you want to connect using SFTP from a Linux machine, just type:

sftp $SUNETID@dtn.oak.stanford.edu

When using SFTP, your folders and files inherit permissions from the parent folder which make them writable by all members of your group by default.

SCP (Secure Copy)#

Example of use on Linux or Mac:

scp archive.zip $SUNETID@dtn.oak.stanford.edu:/oak/stanford/groups/$GROUP
Note: when using SCP, your folders and files won't necessarily inherit the parent folder permissions, even without -p.

SSHFS#

SSHFS is maintained, but not actively developed

The SSHFS project continues to publish bug-fix and security releases: version 3.7.6, released in May 2026, addressed two security vulnerabilities. Its own development status notes, however, that the project has no regular contributors and that its maintainers only have the capacity to address high-impact issues. If you mount Oak with SSHFS, please keep your installation current and watch for security releases. For a tool with a larger development community behind it, see rclone above.

To mount Oak on macOS using SSHFS:

Replace $SUNETID and $GROUP respectively to fit your needs. 

mkdir ~/Desktop/Oak
sshfs $SUNETID@dtn.oak.stanford.edu:/oak/stanford/groups/$GROUP ~/Desktop/Oak -o cache=no -o nolocalcaches -o volname=oak-sshfs -o defer_permissions

When you're done, we recommend to properly un-mount Oak:

umount ~/Desktop/Oak

How do I avoid a Duo prompt every time I connect to the DTN?#

Here are two ways you can avoid using a Duo Prompt:

  1. Set up Public Key Authentication on the Oak DTN

  2. Edit the SSH options your client will use for the Oak DTN in your ~/.ssh/config file on your local machine following the template below. This will enable a persistent ControlMaster which will create a tunnel on your first login, and will re-use the same tunnel on subsequent connections. 

   Host oak-dtn dtn.oak.stanford.edu
   Hostname dtn.oak.stanford.edu
   ControlMaster auto
   ControlPath ~/.ssh/%r@%h:%p
   ControlPersist yes

Oak SMB Gateway#

Cost: Monthly fee required. For latest rates, please see our rates page.#
Example Use Case: Your lab wants to mount Oak storage as a network drive using Windows Explorer or macOS finder.#

SMB Gateways are a popular choice, allowing you to mount your Oak space as a network volume on Windows Explorer, macOS, or Linux. They are also a great way to provide a massive storage target for instruments that generate large amounts of data (sequencers, microscopes, spectrometers, MRI/CT scanners, etc.).

Connecting to your SMB Gateway#

Please see our SMB Gateways page for detailed instructions on how to connect.

Oak NFSv4 Gateways#

Cost: Monthly fee required. For latest rates, please see our rates page.#
Example Use Case: Your lab wants to mount Oak storage on an internal server.#

NFS Gateways enable you to mount Oak on your own servers, workstations, and instrumentation.

Oak NFS gateways have a very specific configuration, which clients must support:

  • NFS version 4.0 or later (ideally 4.1 or 4.2)

  • Kerberos security (sec=krb5). Specifically, sec=sys is NOT supported.

  • All client devices must have a valid host Keytab

  • All users accessing Oak must have an active Kerberos credential.

Why do we have such strict requirements?

Older NFS (NFS 3.x, and NFS 4 with sec=sys) delegates at least some of the access control responsibilities to the client. In order to allow access by clients we do not control, we have to limit NFS to a configuration that ensures all access-control decisions are handled within Oak. That means NFS 4, with sec=krb5p. Because of that requirement, all access must be from users who are using Kerberos credentials.

Please contact us at srcc-support@stanford.edu for more info about NFSv4 Oak gateways.

For instructions on mounting NFSv4 Gateways, please see our NFSv4 Gateways page.

© Stanford University.   Stanford, California 94305.