Individuals requesting an INL HPC account are required to have an INL collaborator act as a sponsor. The sponsor helps ensure appropriate use and coordinates the approvals required for granting access to INL computer systems. They should have some level of familiarity with the work the requestor wants to accomplish using INL computing resources.

For INL employees, the account sponsor is the employee’s line manager. External requestors may select an account sponsor as part of the request process. The account sponsor should be:

• A person with whom the requestor has a collaborative relationship.
• A U.S. citizen.
• An INL employee.

For external requestors who do not have an INL collaborator that meets the above requirements, instructions on providing additional details in the account request are provided in the registration system.

The following summarizes INL HPC use policies, procedures, and security rules that apply to individual end users of INL HPC resources, which in total makes up the INL HPC Appropriate Use Policy. Users are responsible for ensuring that these policies, procedures, and security rules are followed. Users must understand and explicitly agree to abide by INL’s HPC Appropriate Use Policy to be granted access to the systems.

The DOE Office of Nuclear Energy (DOE-NE) is a primary sponsor for INL’s HPC resources through the Nuclear Science User Facility (NSUF) Program. Therefore, the priority for these systems is nuclear energy research, development, and demonstration. This includes activities that support DOE-NE’s R&D programs, organizations performing work associated with DOE through grants and awards, and openly published research of benefit to the nuclear energy community. Additionally, access to INL HPC computing resources are available to users that support INL’s non-nuclear energy research activities as well as education and workforce development. All access requires an appropriate justification.

Further information on all INL HPC policies and practices can be found on the INL HPC Home page.

HPC User Accountability

Each HPC user is accountable for their actions. Violations of policy, procedure, and security rules may result in applicable administrative sanctions or legal actions against the violator.

HPC Resource Use

INL HPC resources are to be used only for activities authorized by the U.S. Department of Energy (DOE) or the INL Advanced Scientific Computing Director.

The use of INL HPC resources should be consistent with the intended usage documented on the account request submission.  Any changes in a user’s intended use from what was approved must be requested and approved in advance by emailing ncrc@inl.gov.  For example, if an HPC account request states that the intended usage is density function theory computations, but the research focus changes and astrophysics simulations are needed, that would need to be reported and approved in advance.

Users must not use INL HPC resources to support illegal, fraudulent, or malicious activities. Users must not use any INL HPC resources to facilitate any transaction that would violate U.S. export control regulations.

The United States DOE and the Management and Operating Contractor of INL make no express or implied warranty with respect to the use of INL HPC resources. Neither DOE nor the Management and Operating Contractor of INL shall be liable in the event of any HPC system failure or loss of data.

Intent to Publish

I will use best efforts to publish the results from my use of the INL HPC Resources in an open scientific journal or significant industry technical journal or conference proceedings. I will acknowledge use of the INL HPC Resources in the publication and notify the INL of any publications that result from my use of the computing resources.

HPC Use by Foreign Nationals

INL complies with U.S. export control policies and regulations. HPC use by foreign nationals is generally permitted regardless of whether access to INL HPC resources is from the United States or abroad. However, the Office of Foreign Assets Control (OFAC) of the U.S. Department of the Treasury administers and enforces economic and trade sanctions based on U.S. foreign policy and national security goals and prohibits use of HPC resources by individuals and companies on the OFAC sanctioned list.  In alignment with this policy, INL will not permit access to HPC resources to citizens of – or companies/individuals physically located in – countries listed on OFAC Website. This access restriction also applies to companies owned or controlled by, or acting for or on behalf of, the listed countries.

Usernames and Passwords

A user identifier (username) and an associated password are required of all INL HPC users. Individuals who have an INL-assigned user identifier are responsible for protecting the associated password. Passwords must be changed on a regular basis per HPC Password Policy or at INL’s request. Password renewal notifications are sent to users when the password is about to expire. Passwords not changed in the allotted timeframe will result in the user’s account being disabled. All passwords must conform to the INL HPC guidelines. Passwords must not be shared with any other person and must be changed as soon as possible after an unacceptable exposure, suspected compromise, or at the direction of INL personnel. These requirements apply equally to any two-factor authentication is provisioned by INL staff for HPC access.

Multifactor Tokens

HPC users connecting to INL HPC resources from offsite will be issued a multifactor token generation device, which will be in the form of software installed on a smartphone, or a physical hardware token. Only under special circumstances is the physical token made available. When account entitlement ends, the HPC user’s token will be disabled. Physical tokens remain the property of INL and must be returned upon completion of approved activities. Two-factor PINs and tokens are not to be shared with any other individual or transferred to another person. If a physical token is no longer required, it must be returned to INL.

Account Usage

Users are not permitted to share accounts, passwords, PINs, or tokens with others. If a user is found in violation of this, they will have their account terminated immediately.

Notification

Users must immediately notify hpcsupport@inl.gov promptly if they become aware that any of their accounts used to access INL HPC resources have been compromised. Upon actual or suspected loss, disclosure, or compromise of the multifactor authentication physical or virtual token and associated password, users must immediately notify hpcsupport@inl.gov.

Users must promptly inform INL of any changes in contact information or affiliation.

Multiple Accounts

HPC users with multiple affiliations may, under very limited and controlled circumstances, be permitted to have multiple accounts on INL HPC resources. The intent of this policy is to allow individuals with multiple affiliations the opportunity to manage proprietary data or licensed code access, enabling physical separation of said data or codes. Copying of proprietary data or licensed codes between accounts is strictly forbidden. If other, non-restricted data needs to be copied between accounts, written permission for the originating account sponsor must be obtained in advance.

Account Renewal

INL HPC policy requires external users to renew their accounts annually. As part of the renewal process, users must provide project summaries describing their work that involved use of INL HPC resources. The account renewal request is effectively a user’s proposal to continue to access INL HPC resources and as such, should clearly communicate intended use and potential research impact. Failure to submit account renewal in the allotted timeframe will result in the account being disabled until the required information is provided.

Software and Data

INL HPC resources are operated as research systems and should only be used to access and store data related to research. These research systems are categorized as moderate per FIPS-199 and protected to the NIST 800-53 moderate security control baseline.

INL HPC resources control data access via username and password authentication for network access and UNIX directory and file permissions for data storage. Network access and data storage systems provide no explicit encryption.  HPC home directories are accessible by the directory owner only; system protections ensure that home directories cannot be shared.  Project directories are accessible only by the directory owner and others designated in written communication with HPC staff.

HPC users are responsible for protecting data files and acknowledge and understand that INL’s HPC security control implementation is sufficient for research data access and storage. Users recognize that files stored in temporary, or scratch, storage areas might not have the same level of data protection as files stored in home or project directories.

HPC users must ensure, when using HPC resources, that all software is acquired and used according to appropriate licensing. Possession, use, or transmission of illegally obtained software on HPC resources is prohibited. HPC users shall not copy, store, or transfer copyrighted software or data using HPC resources, except as expressly permitted by the copyright owner. In certain cases, HPC staff will require proof of end-user license or access approval.

THE USE OF INL HPC RESOURCES TO STORE, MANIPULATE, OR REMOTELY ACCESS CLASSIFIED INFORMATION IS EXPRESSLY PROHIBITED.

Data Retention

INL reserves the right to remove any data at any time and/or transfer data to other individuals (such as principal investigators working on the same or a similar project) after a user account is deleted or a user no longer has a business association with INL.

Although INL takes steps to ensure the integrity of stored data, INL does not guarantee that data files are protected against destruction. INL uses standard enterprise data storage systems with features such as snapshots and remote replication but is not liable for data loss due to major system failures or catastrophic events. HPC users are strongly encouraged to read the INL HPC Data Protection Policy and the INL HPC Retention and Backup Policy and to make backup copies of all critical data and important software.

Deviations from Authorized Privileges Not Allowed

HPC users may not deviate from the terms of this INL HPC Appropriate Use Policy in any way, including, but not limited to, the following prohibitions:

• Unauthorized Access:HPC users are prohibited from attempting to send or receive messages or access information by unauthorized means, such as imitating another system, impersonating another user or other person, misusing legal user credentials (usernames, passwords, etc.), or causing a system component to function incorrectly.
• Altering Authorized Access:HPC users are prohibited from changing or circumventing access controls to allow the user or others to perform actions outside authorized privileges.
• Reconstruction of Information or Software:HPC users are prohibited from reconstructing or re-creating information or software outside authorized privileges.
• Data Modification or Destruction:HPC users are prohibited from taking actions that intentionally modify or delete information or programs outside authorized privileges.
• Malicious Software:HPC users are prohibited from intentionally introducing or using malicious software, including, but not limited to, computer viruses, Trojan horses, or worms.
• Denial of Service Actions:HPC users are prohibited from using INL HPC resources to interfere with any service availability, either at INL or at other sites.
• Pornography: HPC users are prohibited from using INL HPC resources to access, upload, download, store, transmit, create, or otherwise use sexually explicit or pornographic material.
• Harassment:HPC users are prohibited from engaging in offensive or harassing actions toward another individual or organization.
• Cryptocurrency: HPC users are prohibited from any cryptocurrency mining. Additionally, any cryptocurrency transaction support, including clearing and validating, is explicitly prohibited.

Monitoring and Privacy

HPC users have no explicit or implicit expectation of privacy. INL retains the right to actively monitor all HPC resources and activities on INL systems and networks, and to access any file without prior knowledge or consent of HPC users, senders, or recipients.  INL may retain copies of any network traffic, computer files, or messages indefinitely without user’s prior knowledge or consent. INL may, at its discretion, share information gathered through monitoring with the Department of Energy, other incident response organizations, and local, state, federal, and international law enforcement organizations.

INL personnel and HPC users are required to address, safeguard against, and report misuse, abuse, and criminal activities. Misuse of INL HPC resources can lead to temporary or permanent disabling of accounts, administrative sanctions, and/or legal actions.

Patentable Inventions

U.S. Government funds support INL HPC resources and the use of HPC resources by users. Absent any statutory provision or express waiver of intellectual property rights by the U.S. Government, the U.S. Government owns any patentable inventions that may be conceived or first actually reduced to practice through use of the INL HPC. If the use of the INL HPC resources is funded by a U.S. Government research grant, cooperative agreement, or other U.S. Government contract the intellectual property terms (if any) of that grant, agreement, or contract will govern ownership of such intellectual property. If an HPC user is employed by a federal government agency, National Laboratory, University, or private entity, the intellectual property terms (if any) of such employment will govern the use of INL HPC resources.

I will disclose, to the U.S. Government and the INL Contactor, any invention conceived as a part of the work at a INL HPC and will protect the invention until a patent application can be filed. I understand that the U.S. Government retains rights to practice and have others practice the invention and may own the invention.

Commercial for-profit use of INL HPC resources is permissible for activities authorized by the U.S. Department of Energy, provided the work is:

• Conducted in accordance with INL HPC policies and procedures
• Benefits DOE and/or INL missions
• Approved by the INL Advanced Scientific Computing director
• Consistent with the intended usage documented on the account request submission.

Except for documented exceptions, results of all commercial for-profit work must be made available to the public and openly published. Commercial use that is not open to public disclosure will only be considered and approved after the requesting entity enters into a contractual agreement with INL.

INL HPC staff members regularly communicate with users via email for several reasons, including:

• Announcements and updates regarding the status of INL HPC systems
• Announcements regarding outages of INL HPC systems
• Notifications regarding training sessions and educational opportunities
• Warnings regarding password expiration
• Warnings regarding an approaching account expiration date
• Notification that an account has been archived.

These communications are provided at various intervals, depending on the topic, and are sent to all users with enabled accounts.

Data protection and privacy is a foundational part of cybersecurity in the INL High Performance Computing environment. The purpose of this document is to summarize current HPC data protection and management policies and practices.

Directory Permissions

Access to INL HPC resources is controlled via username and password authentication for both network login access to systems as well as UNIX file system permissions for data storage. Network and data storage systems provide no explicit encryption. HPC home directories are accessible by the directory owner and INL HPC system administrators; immutable system protections ensure that home directories cannot be shared with other users. Project data directories are accessible to those designated by the project owner through written communication with HPC staff.

HPC users are responsible for protecting data files. They acknowledge and understand that INL’s HPC security control implementation is sufficient for data access and storage. Users recognize that files stored in temporary, or scratch, storage areas do not have the same level of data protection as files stored in home or project directories.

INL’s HPC Department does not generate, manage, own or share a user’s research data. Individual users are responsible for the collection, management and sharing of their research data consistent with their own or their sponsor’s data management policies.

Federal Information Security Management Act Compliance

INL HPC systems protect user data in accordance with the Federal Information Security Modernization Act as implemented under DOE Order 205.1C, Department of Energy Cybersecurity Program, which includes compliance with NIST SP 800-53 and NIST SP 800-37, and otherwise in accordance with applicable law cited in DOE Order 205.1C.

INL HPC resources are operated as research systems and should only be used to access and store data related to research and/or education. These research systems are categorized as “moderate” per FIPS-199 and protected to the NIST 800-53 moderate security control baseline.

System Administration Staff

In order to operate and maintain INL HPC resources, system administration staff members have administration-level privileges on all HPC systems, and therefore have access to user and project directories for routine operational support. System administration staff members are U.S. citizens. They follow INL cyber security plans and work under an HPC-specific Department of Energy authority to operate.

HPC Use by Foreign Nationals

INL complies with U.S. export control policies and regulations. HPC use by foreign nationals is generally permitted, given the appropriate approvals are granted from the International Access Program. However, the Office of Foreign Assets Control (OFAC) of the U.S. Department of the Treasury administers and enforces economic and trade sanctions based on U.S. foreign policy and national security goals and prohibits the use of HPC resources by individuals and companies on the OFAC sanctioned list. In alignment with this policy, INL will not permit access to HPC resources to citizens of or companies/individuals physically located in countries listed on OFAC Website. This access restriction also applies to companies owned or controlled by, or acting for or on behalf of, the listed countries.

Users of INL HPC resources are responsible for their data. To protect data against loss, a user must understand how INL HPC resources retain data on INL HPC systems. INL HPC uses a centralized storage system where the protected user home directory is available across all INL HPC clusters. INL HPC also manages an off-site data storage system for backup, disaster recovery, and archival purposes.

File storage is either user-centric:

• /home/<username>
• /scratch/<username>

or project-centric:

• /projects/<project-name>

User-Centric Data

User accounts are locked and access to files is lost when the user account expires or a contractual arrangement with INL ends. User accounts expire on an annual basis for external users (see User Type and Account Status Policy) but can be renewed for as long as the user is approved to use INL resources.  Users will be warned about account expiration before the expiration date.

Files in /home/<username> will be retained for 90 days after a user account is disabled. After the 90-day period ends, the files will be removed from /home/<username> and archived and stored on the off-site storage system. The archived user data will be retained for one year, after which we reserve the right to delete files and reclaim space. Users may request copies of their files during the expired and archived time periods by submitting an HPC support ticket.

Files in /scratch/<username> are automatically purged every 90 days to prevent the file system from filling up. The 90-day purge can be shortened if needed to free up scratch storage space.

Project-Centric Data

INL HPC users may request to have a project directory created on INL HPC storage to establish a shared storage space for team collaboration. Users can submit an INL HPC support ticket to request to create an INL HPC project directory. Project directory owners may request access for additional users by submitting an INL HPC support ticket.

Project directory owners must be INL HPC users with enabled HPC accounts. If the project directory owner’s account becomes disabled for any reason, a replacement owner must be identified and assigned. If no replacement owner can be identified, INL HPC reserves the right to delete project directories and the files they contain to reclaim space.  A notification will be sent to all members of the project group before files are deleted.

Data Backup

INL provides HPC data storage resources, including offsite backup systems, that ensure the integrity and availability of user data. HPC data storage systems create both routine snapshots and offsite backups on the following schedule:

• Hourly snapshots are created daily between 0600 and 1900 hours and are kept for eight days
• Weekly snapshots are created every Friday at 2100 and are kept for 31 days
• Monthly snapshots are created the last Friday of the month at 2200 and kept for 17 weeks

This allows recovery of stored information including data unintentionally deleted provided the error is detected within the data retention window.

Temporary storage space (/scratch and /tmp directories) is not backed up. Users should copy needed files to a persistent location after analysis is completed. Scratch files are automatically purged after 90 days.

INL HPC passwords must conform to a minimum set of password requirements, including complexity and length. Never share passwords with another person. The HPC team will never ask for an HPC password.

Basic password requirements are as follows:

 Some prohibited password options:

• Must not contain the user ID.
• Must not contain common English dictionary word, spelled forward or backwards (except words of three or fewer characters).
• Must not employ common names.
• Must not contain a commonly used number associated with the user (e.g., SSN or license number).
• Must not contain simple patterns of letters or numbers.

Note: The HPC password is different than the passcode for two-factor authentication. When interacting with the RSA SecurID token server (https://cybele.inl.gov:7004/console-selfservice), HPC OnDemand (https://hpcondemand.inl.gov), or SSH to hpclogin.inl.gov, the user is prompted for a passcode, which is the HPC RSA SecurID PIN + token code. Your HPC password is used for all other HPC systems once you are connected to the HPC network.

The HPC job scheduler utilizes fair-share metrics for prioritizing HPC jobs. The scheduler maintains high overall system utilization while assuring that mission critical work runs first.

Under special circumstances and upon request, the Advanced Scientific Computing director can authorize certain users or jobs to be elevated in priority for short, pre-determined periods of time. To request special consideration, submit an HPC support ticket with details and justification to hpcsupport@inl.gov.

There are several user types on INL’s HPC system, and a user can be in one of several states depending on their level of activity and actions they take regarding their account.

User Type

A “user account” is associated with an individual user of INL HPC resources. Users are granted user accounts at their request, if eligible, and under the condition of agreement to all HPC policies, including the Appropriate Use Policy. User accounts expire as outlined in the table below or when the contractual arrangement between the user’s employer and INL ends, whichever is sooner. Users are responsible for moving their data before their account is archived.

There are three HPC user account types, listed below. The user’s affiliation determines the account type.

Account States

1. Enabled

A user account is Enabled once a request is received, processed, and approved by INL Staff. Users are notified once a request is approved, and their account is Enabled.

When a user account is enabled:

• The user will be able to log in.
• Data will be accessible in /home/<username>.
• Users will receive email announcements and updates about INL HPC systems.
• Users will receive warnings about password expiration.
• Users will receive warnings as their account expiration date approaches.

2. Disabled

A user account moves from enabled to disabled when:

• User fails to complete HPC password renewal.
• User HPC account reaches its end, and an account renewal is not completed.
• For non-U.S. citizen users, the International Access Program plan reaches its end, and no renewal is completed.

When a user account is disabled:

• User will not be able to log in.
• User will not receive email announcements and updates about INL HPC systems.

3. Archived

User accounts are archived when any of these conditions are true:

• The user’s account becomes invalid for any reason (e.g., change of employer, end of contractual arrangement between INL and user employer).
• The relationship between the user and either their employer or INL is terminated.
• For non-U.S. citizen users, the International Access Program plan expires or is closed out.
• User account has been in disabled state for 30 days or more.

When a user account is archived:

• Data will not be accessible in /home/<username>.
• User will not receive email announcements and updates about INL HPC systems.
• Data will be retained in long-term storage for a period of 12 months.