<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:\7B49\7EBF;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Aptos;}
@font-face
{font-family:"\@\7B49\7EBF";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:12.0pt;
font-family:"Aptos",sans-serif;
mso-ligatures:standardcontextual;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#467886;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Aptos",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.25in 1.0in 1.25in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:487595767;
mso-list-template-ids:1150723206;}
@list l1
{mso-list-id:2110849094;
mso-list-type:hybrid;
mso-list-template-ids:1375209998 67698711 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l1:level1
{mso-level-number-format:alpha-lower;
mso-level-text:"%1\)";
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l1:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l1:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#467886" vlink="#96607D" style="word-wrap:break-word">
<div class="WordSection1">
<p>Dear LONI HPC users,<o:p></o:p></p>
<p>We are pleased to announce that the LONI QB-4 cluster can now be accessed in the friendly user mode. You can log in with your current LONI HPC username and password using
<strong><span style="font-family:"Aptos",sans-serif">qbd.loni.org</span></strong> as the hostname. Please bear in mind the QB-4 cluster is still in the deployment process, thus the "friendly user mode", which means that the hardware/software configuration and
policy may change without advance notice. <o:p></o:p></p>
<p>Before you submit jobs on QB-4, please make sure that you review the user documentation here:<o:p></o:p></p>
<p><a href="http://hpc.loni.org/docs/guides.php?system=Qb4" target="_blank">http://hpc.loni.org/docs/guides.php?system=Qb4</a><o:p></o:p></p>
<p>In summary:<o:p></o:p></p>
<ol start="1" type="1">
<li style="mso-list:l0 level1 lfo1">QB-4 uses two-factor authentication for increased security. Please refer to the information here for how to set up and log in with two-factor authentication on QB-4:
<a href="http://hpc.loni.org/docs/systems/twofa.php" target="_blank">http://hpc.loni.org/docs/systems/twofa.php</a><o:p></o:p></li><li style="mso-list:l0 level1 lfo1">The /work and /project filesystems for QB-4 are the same as they are for QB-3. The /home filesystem is new.<o:p></o:p></li><li style="mso-list:l0 level1 lfo1">If you already have a LONI HPC allocation, it should work on QB-4 as well. There are 64 cores on each QB-4 compute node, so please be mindful of the size and duration of your jobs when running on QB-4 otherwise you may find
your compute allocation running out quickly. <o:p></o:p></li><li style="mso-list:l0 level1 lfo1">QB-4 uses the same workload and resource manager, Slurm, as QB-3. Here's a quick guide:<o:p></o:p></li></ol>
<ol start="1" type="a">
<ol start="1" type="a">
<li style="mso-list:l1 level2 lfo2">For GPU nodes, there are two partitions/queues on QB-4. The “gpu2” partition is for nodes with 2 A100 GPUs each (52 nodes in total), which “gpu4” is for nodes with 4 A100 GPUs each (10 nodes in total). When submitting jobs
to these queues, one must use the “--gres=gpu:<# of GPUs>” flag to specify the number of GPUs per node. For instance, the command below launches an MPI GPU application with 2 nodes and 2 GPUs per node:<o:p></o:p></li></ol>
</ol>
<p style="margin-left:99.0pt">srun -N2 -n128 -p gpu2 --gres=gpu:2 -A <allocation name> ./my_gpu_app
<br>
<br>
How to submit GPU jobs:<br>
<a href="http://hpc.loni.org/docs/slurm.php#gpu" target="_blank">http://hpc.loni.org/docs/slurm.php#gpu</a><o:p></o:p></p>
<ol start="1" type="a">
<ol start="2" type="a">
<li style="mso-list:l1 level2 lfo2">For MPI applications that are compiled with the default MPI on QB-4 (Intel MPI 2021.5.1 with Intel compiler 2021.5.0), please use the "srun" command to launch MPI applications. For instance, the command below launches the
MPI application "my_mpmi_app" with 2 nodes, 2 MPI processes per node and 32 threads per MPI process:<br>
<br>
srun -N2 -n4 -c32 ./my_mpi_app -p <partition name> -A <allocation name><br>
<br>
, where "-N<number of nodes>" indicates the number of nodes, "-n<number of processes>" the total number of MPI processes and "-c<number of threads>" the number of threads per MPI process.<o:p></o:p></li><li style="mso-list:l1 level2 lfo2">To submit a batch job, use the "sbatch" command.<o:p></o:p></li><li style="mso-list:l1 level2 lfo2">To request an interactive session, use the "salloc" command
<strong><span style="font-family:"Aptos",sans-serif">from the login nodes qbd1 or qbd2</span></strong>. For instance, the command below requests an interactive session with one full node for 4 hours in the checkpt partition/queue:<br>
<br>
salloc -N1 -n64 -t 4:00:00 -p checkpt -A <allocation name><o:p></o:p></li></ol>
</ol>
<p>Detailed information about Slurm can be found in the user guide.<o:p></o:p></p>
<p>Please send feedback and report issues to <a href="mailto:sys-help@loni.org">sys-help@loni.org</a>. Thank you!<o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:10.5pt;font-family:"Helvetica",sans-serif;color:#333333;background:white">Feng Chen, PhD</span></b><span style="font-size:10.5pt;font-family:"Helvetica",sans-serif;color:#333333"><br>
<span style="background:white">Manager of User Services</span><br>
<span style="background:white">High Performance Computing</span><br>
<span style="background:white">Louisiana State University</span><br>
<span style="background:white">329 Frey Computing Services Center, Baton Rouge, LA 70803</span><br>
<span style="background:white">office 225-578-2924 </span><br>
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><a href="mailto:fchen14@lsu.edu"><span style="font-size:10.5pt;font-family:"Helvetica",sans-serif;color:#451D7C;background:white">fchen14@lsu.edu</span></a>
<o:p></o:p></span></p>
</div>
</body>
</html>