Frequently Asked Questions#

Access#

Can I Access Oak from Sherlock?#

Oak storage is available from all nodes on Sherlock under /oak. Like Sherlock’s /scratch, Oak is based on the Lustre parallel filesystem and is connected to Sherlock through a low-latency Infiniband network.

Can I Archive files to Oak from Sherlock?#

The mpiFileUtils utilities are designed to copy files in parallel so you can quickly archives terabytes of data from scratch to Oak. The example below shows how to launch screen and launch a job that uses the dcp tool to copy a large directory:

[login_node ~]$ screen
[login_node ~]$ module load system mpifileutils
[login_node ~]$ srun -p dev -n 2 dcp $SCRATCH/dir $OAK/scratch_archive/

If you’re a Sherlock owner, you may want to replace -p dev with -p your_partition and increase the number of MPI tasks (-n) to copy even faster!

Can I Access Oak from SCG?#

As of January 2020, all SCG labs (in /labs) and projects (in /projects) are hosted on Oak. When you get access to a lab or project in SCG, you also get access to the corresponding space in Oak. SCG is connected to Oak using a pair of 100 Gigabit Ethernet Lustre routers.

Groups who directly rent Oak storage will be able to access that Oak storage under /oak, so long as SCG users are in the corresponding workgroup. The groups used to manage SCG labs and projects space are separate from the workgroups used to manage directly-rented Oak space.

The $OAK environment variable is not set on SCG.

I'm an administrator of my Oak workgroup but can't access my Oak folder? What's going on?#

Access to Oak is managed by Stanford Workgroup Manager. There are two levels of membership available in Stanford Workgroup Manager: Member and Administrator. Members are people who have been granted access to the Oak space. Workgroup administrators manage who is in the member list and can add or remove people as needed. Being a workgroup administrator does not in and of itself grant you access to Oak, however, administrators may add themselves as members if they need to manage access to an Oak space and access it themselves.

Note: The administrator role in Workgroup Manager does not grant you any root permissions or special privileges on Oak. It is solely an authority to manage members of a given workgroup to grant or revoke access to a given Oak space. The PI who owns a space is the sole Workgroup administrator by default. They can add a lab Manager or trusted individual as an administrator of their workgroup if they want to delegate access control of their Oak space to that individual.

Moving Data#

Can I access Oak from my desktop/laptop?#

Yes, please see the Oak Gateways page.

Can I mount Oak on my own computer at Stanford?#

Yes, the Oak team can deploy specific NFSv4 or [SMB][oak-smb-gateway] gateways to mount your Oak directory on your Windows, macOS, or Linux-based desktop or server. It is mandatory to use SUNet IDs to connect and Kerberos authentication is required to access Oak. Private NFS and SMB gateways are a paid add-on service for Oak spaces. For the latest gateway pricing information, please see our rates page.

Backup#

Is Oak backed up?#

While its hardware configuration is quite robust, Oak does not provide local or remote data backup, and should be considered as a single copy. The SRCC is currently evaluating options for adding automatic remote backups (to cloud storage). Such a service will incur additional costs to the PI.

Quotas#

How are Oak quotas measured?#

Quotas on Oak are measured in two ways: Volume (TB) and the number of files and folders stored (inodes). There is a fixed ratio of 1.5 Million inodes for every 10TB of volume.

One way to think of this is like an elevator’s weight and occupancy limit. Weight is like the volume measurement, and occupancy is like inodes. If you put many smaller people into an elevator, you may not hit the weight limit, but you will hit the occupancy limit - it’s the same basic idea with inodes and volume. Many small files tend to lead to groups who hit inode quotas well before volume quotas. There’s nothing inherently wrong with this, and it’s just a result of different data workflows creating unique constraints. When you hit an inode quota, there are a few actions that you can choose from:

• Purchase more Oak space to increase your quota.
• Use a tool like tar to pack many files into a single archive file.
  • Using tar helps to reduce inode counts and is helpful for groups that may not have the budget for quota increases, don't want to delete data, and don't need to immediately access older, disused directories that may have thousands or tens of thousands of small files.
  • You must have adequate volume available in your Oak quota to use this option. If you aren't sure of your quota utilization, check your Oak Dashboard.
• Delete unnecessary items to free up space.

News & Alerts#

How can I stay up-to-date on Oak?#

We highly recommend that all Oak users join the SRCC Slack workspace (SUNet ID Required) and follow the #oak-announce channel to stay abreast of any system updates, maintainence, or outages. #oak-users is best for sharing best practices and asking questions among a welcoming community of other Oak users.

Email notification for system maintenance and issues#

Oak administrators maintain a mailing list called oak-announce. This very low traffic list is used for official announcements such as planned maintenance. Oak users are automatically subscribed to this list by default. For more information, please see the following page:

Oak-announce List Information