Stampede Forum

Mathematica in GUI mode

2016-09-24T19:42:36Z

Hi all,
Does anyone know how to run Mathematica in GUI mode?
Thanks
Ashu

RE: Account being deactivated

2016-06-03T21:55:56Z

Hi Hongpin,

It looks like you are no longer on an active allocation. Your PI needs to add you to an active allocation in order for you to get access to Stampede again. Feel free to submit a ticket (https://portal.xsede.org/help-desk) if you need assistance.

Thanks,
Maytal

Account being deactivated

2016-06-03T18:24:11Z

Hi,
I registered as a XSEDE user last year, but did not use it in the past year.
Right now, I am trying to log on stampede and get refused all the time.

Error message:
ssh: connect to host login.xsede.org port 22: Connection refused

I would like to know how to reactivate my account for stampede?

BTW, I can log on XSEDE portal with passcode.

Thanks for helping.

--Hongbin

how to allocate more memory

2016-05-26T03:07:00Z

Hi

what switch of sbatch can be used to allocate more memory. I am using normal queue with N=1 and n=1 to test my program and its exiting with bad allocation.

Regards

RE: Can't use 256 nodes for symmetric computing

2016-01-27T22:46:34Z

Hi David,

For timely answers to technical questions like this, you should submit a ticket to help@xsede.org. But, I should be able to help you here.

With the upgrade to Intel 15 and Intel MPI 5, the behavior of mpiexec.hydra (which starts the mpi job) changed. There is an internal buffer that processes the arguments for each executable, and the handling of this buffer has changed. This error doesn't occur when using Intel MPI 4.

To work around this buffer overflow, I've created a version of ibrun.symm that does not set up the MIC environment using the mpiexec.hydra task starter. This reduces the number of characters for each excutable command and avoids the buffer overflow issue.

However, using this version of ibrun.symm starts the MIC executable with the incorrect environment. So, a wrapper script must be used also. I've attached the modified version of ibrun.symm and the wrapper script to this response.

To use this version of ibrun.symm, replace your current run line with the following:

./ibrun.symm.no_env -m "./mic_env_run ./arc_mpi.exe.mic" -c "./arc_mpi.exe"

If you continue to have problems with this, please submit a ticket.

Thanks,

John

Can't use 256 nodes for symmetric computing

2016-01-25T16:29:53Z

I am having a problem submitting symmetric computing (CPU and MIC) jobs to Stampede's normal-mic queue. The normal-mic queue currently allows a job to use a maximum of 256 nodes.

When my job script requests 256 CPU-MIC nodes, my job fails. The output includes messages that the argument list is too long. My job script requests 16 CPU MPI tasks and 15 MIC MPI tasks per node.

I changed the application so that the CPU and MIC versions don't use command line arguments. However, the same error messages appear. What does "argument list is too long" mean when the application does not have any command line arguments? How can I get past this error and use all 256 CPU-MIC nodes?

The program runs correctly when requesting from 1 to 128 CPU-MIC nodes. Also, the program runs correctly when requesting from 1 to 128 CPU-MIC-MIC nodes on the normal-2mic queue (which allows a job to use a maximum of 128 nodes).

I've attached my job script and the output file with the error messages for reference.

Thank you.

RE: ibrun command line inputs / python-managed sequential parallel jobs

2016-01-11T23:04:42Z

Update:
'ibrun -o 0 -n %i %s'
solves this problem.
where %i and %s are the number of processors and the command string provided later. The -o option specifies the offset for the processors, and in this context the offset is always 0.

I will be updating our open-source code (SU2) so that this is used automatically on TACC machines in case someone else needs to use our code on XSEDE.

ibrun command line inputs / python-managed sequential parallel jobs

2015-12-17T04:22:33Z

I have a python script (which works well on other systems) that launches a number of parallel jobs sequentially - any recommendations on launching slurm jobs from within python scripts? Especially regarding doing it in the right way for the stampede system?

It is an optimization process, where the optimizer is serial and the function calls are parallel, so it's not feasible to launch each parallel simulation manually.

I have tried 'ibrun -n %i %s' where the inputs are provided later in the script; however this produced an error about not providing a -o command.

Has anyone tried something similar to this, and/or can recommend the correct syntax?
This process previously worked with 'srun -n %i %s' on a different system, so I think it is just the ibrun inputs I need help with.

Error while loading shared libraries.

2015-07-06T14:07:44Z

Hello,

I'm trying to run a LAMMPS simulation with the following job script:

1#!/bin/bash
2
3#SBATCH -J SiCyl
4#SBATCH -o outSiCyl_8c1n.txt
5#SBATCH -p normal
6#SBATCH -N 1
7#SBATCH -n 8
8#SBATCH -t 12:00:00
9
10
11module load openkim/1.2.2
12module load voro++/0.4.5
13module load lammps
14ibrun /opt/apps/intel13/mvapich2_1_9/lammps/2014_01_10/bin/lmp_stampede < /home1/03036/[username]/8cores_1node_070215/in.SiCyl

However, when I run the job, I get the error

error while loading shared libraries: /admin/build/admin/rpms/stampede/BUILD/openkim-api-v1.2.2/KIM_API/libkim.so: cannot open shared object file: No such file or directory

Could someone please tell me what caused this error and how to fix it? Has the path to the lmp_stampede executable changed?

Thank you.

RE: missing shared libraries

2015-06-05T22:18:39Z

Update. When I compile my program with mpicxx the error goes away. With gcc or icpc the error shows up.

missing shared libraries

2015-06-05T21:36:29Z

Hi everyone,

I am trying to get mpi up and running with my experiment platform and I'm almost there, but I get a weird message in my STDERR.

******************************************************
There are missing shared libraries
******************************************************

It looks like my experiments are running fine, but I know better than to ignore error messages. I don't think this message is coming from my experiment setup, and I was wondering if it was native to TACC or ibrun?

I thought the error message might be coming from ibrun, but I've been able to turn on verbosity with ibrun. Anyone have any ideas?

scp timeout from stampede to local pc

2015-04-23T16:27:20Z

Hi,

I am trying to copy some files ~1G from stampede to local Ubuntu. Its getting timed out.
I am just using:
scp file-name user@remote-ip:/home/user

Do I need to do something else, any config ?

Any suggestions are appreciated.

-Thanks
Bharat

Monitor processor and memory loads

2014-09-26T20:21:08Z

I am trying to fine tune a few things in my code and was wondering if there is any way to look at the processor and memory loads on a node (something like top). I can't run an interactive job on the developement nodes because I need to run for longer than two hours to check everything.

If anyone knows of a way to peck at a node I would appreciate it.

Greg

Managing serial programs on Stampede

2014-06-25T16:54:39Z

What's the recommended way of running serial programs on Stampede? That is, if you have a program that's intrinsically single-threaded/single-processed, what's the best way of submitting Stampede jobs using it?

I know that there is a "serial" queue for jobs requesting just a single processor, but how would you go about managing a large number of the single-process runs? (E.g. if you're running the intrinsically-serial program with a large number of different inputs, or under a range of different parameters.) Is there an approved method for managing distribution of multiple serial runs across multiple nodes of a single job, or even just the 16 processors of a single node from the main queues? Are ad-hoc bash scripts the way to go, or is there a better way to go about doing it?

Thanks.

BLAS and LAPACK

2014-06-24T22:29:41Z

I am trying to run JAGS through R (rjags package) on Stampede. JAGS relays on BLAS and LAPACK and I was able to install it in my home directory using MKL. However, to use MKL JAGS has to be compiled as a static object which means it can't been used from R. Are there stand alone BLAS and LAPACK libraries on Stampede I could link to instead of MKL? If not are there recommended versions of the BLAS and LAPACK libraries I can download and install myself?

Thank you,
Greg

RE: ibrun fails

2014-06-09T21:29:30Z

Stefan Lemvig Glimberg:

Hi Haoyuan,

I did only what you explained you also did. So if it still is not working I am afraid I cannot help

/Stefan

That's okay. I'll keep trying~

RE: ibrun fails

2014-06-09T21:06:31Z

Hi Haoyuan,

I did only what you explained you also did. So if it still is not working I am afraid I cannot help

/Stefan

RE: ibrun fails

2014-06-09T20:21:14Z

Stefan Lemvig Glimberg:

I solved this now. Move the .ssh folder, then log out and in again and a new .ssh folder is created =)

/Stefan

Hi Stefan,

Could you please tell me what EXACTLY you did to solve this? I encounted the same problem (on gordon, not stampede though) and I tried to move my /.ssh folder to a new directory /.ssh2. Then I logged out and in again. It asked for my password and it did created a new /.ssh folder with different keys. However when I submitted my job again I got the same error as before.

Many thanks!

Haoyuan

Optimal Job Size for the SLURM Scheduler?

2014-03-21T15:12:38Z

I developed my experiment setup on Kraken and ported to Stampede a month and a half ago, and for the first month it was great. In the last week or so, however, my queue throughput has tanked pretty severely. On Kraken, I think the scheduler favored large jobs, except if you could make them small enough to run as "backfill," and I managed to split my experiments up into small enough chunks that it was frequently running as backfill. This brings me to my question: what is the optimal job size? Is it set up the same as Kraken to where it favors larger jobs, but will backfill? If so, how small do jobs need to be to run as backfill? Waiting 4 hours for 32-core-for-20-minute jobs to go through is getting a little old.

Thanks!

RE: Using ssh keys to login

2013-11-06T16:21:44Z

I can recommend the use of an ssh-agent, it helps tremendously. Also for large file transfers I will also recommend globus online.

A quick google search for ssh-agent how to:
http://superuser.com/questions/8077/how-do-i-set-up-ssh-so-i-dont-have-to-type-my-password

And globus online:
https://www.globusonline.org/

Regards
Yaakoub