High Performance Computing

Users of INL HPC resources are responsible for their data. To protect data against loss, a user must understand how INL HPC resources retain data on INL HPC systems. INL HPC uses a centralized storage system where the protected user home directory is available across all INL HPC clusters. INL HPC also manages an off-site data storage system for backup, disaster recovery, and archival purposes.

File storage is either user-centric:

• /home/<username>
• /scratch/<username>

or project-centric:

• /projects/<project-name>

User-Centric Data

User accounts are locked and access to files is lost when the user account expires or a contractual arrangement with INL ends. User accounts expire on an annual basis for external users (see User Type and Account Status Policy) but can be renewed for as long as the user is approved to use INL resources.  Users will be warned about account expiration before the expiration date.

Files in /home/<username> will be retained for 90 days after a user account is disabled. After the 90-day period ends, the files will be removed from /home/<username> and archived and stored on the off-site storage system. The archived user data will be retained for one year, after which we reserve the right to delete files and reclaim space. Users may request copies of their files during the expired and archived time periods by submitting an HPC support ticket.

Files in /scratch/<username> are automatically purged every 90 days to prevent the file system from filling up. The 90-day purge can be shortened if needed to free up scratch storage space.

Project-Centric Data

INL HPC users may request to have a project directory created on INL HPC storage to establish a shared storage space for team collaboration. Users can submit an INL HPC support ticket to request to create an INL HPC project directory. Project directory owners may request access for additional users by submitting an INL HPC support ticket.

Project directory owners must be INL HPC users with enabled HPC accounts. If the project directory owner’s account becomes disabled for any reason, a replacement owner must be identified and assigned. If no replacement owner can be identified, INL HPC reserves the right to delete project directories and the files they contain to reclaim space.  A notification will be sent to all members of the project group before files are deleted.

Data Backup

INL provides HPC data storage resources, including offsite backup systems, that ensure the integrity and availability of user data. HPC data storage systems create both routine snapshots and offsite backups on the following schedule:

• Hourly snapshots are created daily between 0600 and 1900 hours and are kept for eight days
• Weekly snapshots are created every Friday at 2100 and are kept for 31 days
• Monthly snapshots are created the last Friday of the month at 2200 and kept for 17 weeks

This allows recovery of stored information including data unintentionally deleted provided the error is detected within the data retention window.

Temporary storage space (/scratch and /tmp directories) is not backed up. Users should copy needed files to a persistent location after analysis is completed. Scratch files are automatically purged after 90 days.