mailRe: Using multi-processor for model_free


Others Months | Index by Date | Thread Index
>>   [Date Prev] [Date Next] [Thread Prev] [Thread Next]

Header


Content

Posted by Troels Emtekær Linnet on September 14, 2016 - 15:35:
Hi Sam.

To tackle this problem, I would advice to create another bug.
Creation and closing of a bug "leaves trails", which maybe will help
another person, when googling for the same problem.

To help you, can you do a "relax -i" on both computers?
That give some indication about package versions and computer setup.

The first thing we need to establish, is that mpirun is working.
We have to test the installation without relax.

Can you have a look at:
http://wiki.nmr-relax.com/OpenMPI

Try the different things like:
lscpu
mpirun --report-bindings -np 11 echo "hello world"
mpirun --report-bindings -np 4 relax --multi='mpi4py'

When we are confident about this, then we will try make a small test script
for relax.

Please try these things at both computers, and provide 2 files with
commands and output.

Then attach it to the bug report.





2016-09-14 6:40 GMT+02:00 Mahdi, Sam <sam.mahdi.846@xxxxxxxxxxx>:

Hi Troels,

So I saw the tutorial you put, and the main problem was I had not
specified my data was only for the Nitrogen spins. After applying the spin
column, my data loaded and relax ran model free with no problem. I have a
script that starts and runs relax and model free all automatic, if you wish
I can send it via email to you and you can upload it to the tutorial wiki
page. So I can successfully run model-free in script mode for a
uni-processor.
The problem now with the multi-processor is that the script won't load. In
the bug page I uploaded a screenshot where I had input the 'mpirun -np 4
../relax --multi='mpi4py' command, however I had no output. I checked
processes running in the background, and saw that there was indeed 4
processess running in the background (1 master and 3 slaves) for relax; but
there was no output, so I was unable to load any data, or create a pipe or
anything. This was only on the Fedora 24 computer, not the Fedora 20. On
the Fedora 20 computer, I was able to successfully open relax on a multi
processor platform. I can send the screenshots and the relax -i for both
computers again. I don't know why it doesn't work the fedora 24. Do you
know what could be causing this?

Thanks again in advance

On Tue, Sep 13, 2016 at 9:32 PM, Troels Emtekær Linnet <
tlinnet@xxxxxxxxxxxxx> wrote:

Hi Sam

Can you send the mail again and include the maillist?

Best Troels


Den tirsdag den 13. september 2016 skrev Mahdi, Sam <
sam.mahdi.846@xxxxxxxxxxx>:

HI Troels,

So I saw the tutorial you put, and the main problem was I had not
specified my data was only for the Nitrogen spins. After applying the spin
column, my data loaded and relax ran model free with no problem. I have a
script that starts and runs relax and model free all automatic, if you 
wish
I can send it via email to you and you can upload it to the tutorial wiki
page. So I can successfully run model-free in script mode for a
uni-processor.
The problem now with the multi-processor is that the script won't load.
In the bug page I uploaded a screenshot where I had input the 'mpirun -np 
4
../relax --multi='mpi4py' command, however I had no output. I checked
processes running in the background, and saw that there was indeed 4
processess running in the background (1 master and 3 slaves) for relax; 
but
there was no output, so I was unable to load any data, or create a pipe or
anything. This was only on the Fedora 24 computer, not the Fedora 20. On
the Fedora 20 computer, I was able to successfully open relax on a multi
processor platform. I can send the screenshots and the relax -i for both
computers again. I don't know why it doesn't work the fedora 24. Do you
know what could be causing this?

Thanks again in advance
Sam

On Tue, Sep 13, 2016 at 1:01 PM, Troels Emtekær Linnet <
tlinnet@xxxxxxxxxxxxx> wrote:

Hi Sam.

I closed the 2 bug reports as invalid.

The data is not labelled correct.
But this can be corrected.

Please see this tutorial I wrote:
http://wiki.nmr-relax.com/Tutorial_for_model-free_analysis_sam_mahdi

I hope this give some guidance.

If you experience any new problems, please feel free to ask!!

What you experience, will probably be the same for many.
Your feedback is valuable for the development.

Please wait with using mpirun and multiple processors, before you are
absolutely sure
that it will run on 1 processor.

Bugfixing when using multiple processors is a nightmare....

Best
Troels

2016-09-12 17:36 GMT+02:00 Mahdi, Sam <sam.mahdi.846@xxxxxxxxxxx>:

Hi Troels,

I just created another bug report. I simply copy pasted the email, and
uploaded the script files there.

Sincerely,
Sam

On Mon, Sep 12, 2016 at 5:14 AM, Troels Emtekær Linnet <
tlinnet@xxxxxxxxxxxxx> wrote:

Hi Sam.

Can you produce another bug report.

Please don't attach files to these mails as it will strain  the
mailinglists.

Cheers
Troels


Den lørdag den 10. september 2016 skrev Mahdi, Sam <
sam.mahdi.846@xxxxxxxxxxx>:

Hi Troels,

Additional question that I had, if you could also look into this as
well on Tuesday please. I have decided to try to write a script to 
automate
this whole process (since I won't be using the gui to do model free), 
and
I've come across a problem. I can successfully open up relax using 
openmpi,
and can load the pdb file, and assign all the spins and isotopes; 
however,
it appears it will only load one data file (the very first one I'll 
have
inputed in the script). I don't know if there is a problem with how I 
wrote
my script. Not only will it not load the rest of my data sets, it 
won't
actually run dAuvergne's protocol either, it'll just load the data 
set and
exit out of the program. Attached is the script I wrote for relax.

This is the output once relax has loaded

script = 'model_free_sample_script.py'
------------------------------------------------------------
----------------------------------------
from time import asctime, localtime
from auto_analyses.dauvergne_protocol import dAuvergne_protocol
DIFF_MODEL=['local_tm','sphere','prolate','oblate','ellipsoi
d','final']
MF_MODELS=['m0','m1','m2','m3','m4','m5','m6','m7','m8','m9']
LOCAL_TM_MODELS=['tm0','tm1','tm2','tm3','tm4','tm5','tm6','
tm7','tm7','tm8','tm9']
GRID_INC=11
MIN_ALGOR='newton'
MC_NUM=500
CONV_LOOP=True
pipe_bundle="mf(%s)"%asctime(localtime())
name="origin-"+pipe_bundle
pipe.create(name,'mf',bundle=pipe_bundle)
structure.read_pdb('2d9j.pdb',set_mol_name='hRGS7')
structure.load_spins('@N',ave_pos=True)
structure.load_spins('@NE1',ave_pos=True)
structure.load_spins('@H',ave_pos=True)
structure.load_spins('@HE1',ave_pos=True)
spin.isotope('15N',spin_id='@N*')
spin.isotope('1H',spin_id='@H*')
relax_data.read(ri_id='R1_Agnes',ri_type='R1',frq=599.642*1e6,
file='R1_Agnes',res_num_col=1,data_col=2,error_col=3)
relax_data.read(ri_id='R2_Agnes',ri_type='R2',frq=599.642*1e6,
file='R2_Agnes',res_num_col=1,data_col=2,error_col=3)
relax_data.read(ri_id='ssNOE_Agnes',ri_type='NOE',frq=599.642*1e6,
file='ssNOE_Agnes',res_num_col=1,data_col=2,error_col=3)
relax_data.read(ri_id='R1_NMRFAM',ri_type='R1',frq=799.642*1e6,
file='R1_NMRFAM',res_num_col=1,data_col=2,error_col=3)
relax_data.read(ri_id='R2_NMRFAM',ri_type='R2',frq=799.642*1e6,
file='R2_NMRFAM',res_num_col=1,data_col=2,error_col=3)
relax_data.read(ri_id='ssNOE_NMRFAM',ri_type='NOE',frq=799.642*1e6,
file='ssNOE_NMRFAM',res_num_col=1,data_col=2,error_col=3)
interatom.define(spin_id1='@N',spin_id2='@H', direct_bond=True)
interatom.define(spin_id1='@NE1',spin_id2='@HE1', direct_bond=True)
interatom.set_dist(spin_id1='@N*',spin_id2='@H*',ave_dist=1.
02*1e-10)
interatom.unit_vectors()
value.set(-172*1e-6,'csa',spin_id='@N*')
dAuvergne_protocol(pipe_name=name,pipe_bundle=pipe_bundle,di
ff_model=DIFF_MODEL,mf_models=MF_MODELS,local_tm_models=LOCA
L_TM_MODELS,grid_inc=GRID_INC,min_algor=MIN_ALGOR,mc_sim_num
=MC_NUM,conv_loop=CONV_LOOP)

So it indicates that my script has loaded. However, after it loads
the spins from the pdb file, this is what happens after my first data 
set
has been loaded:

relax> relax_data.read(ri_id='R1_Agnes', ri_type='R1',
frq=599642000.0, file='R1_Agnes', dir=None, spin_id_col=None,
mol_name_col=None, res_num_col=1, res_name_col=None, 
spin_num_col=None,
spin_name_col=None, data_col=2, error_col=3, sep=None, spin_id=None)
Opening the file 'R1_Agnes' for reading.
RelaxWarning: The sequence data in the line ['Residue', 'R1',
'Error'] is invalid, the residue number data 'Residue' is invalid.
RelaxWarning: The sequence data in the line ['1'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['2'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['3'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['4'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['5'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['6'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['7'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['8'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['9'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['10'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['11'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['16'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['17'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['18'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['21'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['22'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['23'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['26'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['27'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['28'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['31'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['40'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['46'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['58'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['61'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['62'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['63'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['73'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['76'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['79'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['81'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['82'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['85'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['94'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['97'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['99'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['106'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['115'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['121'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['126'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['127'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['134'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['135'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['136'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['137'] is invalid, the
data is missing.
RelaxWarning: The sequence data in the line ['139'] is invalid, the
data is missing.

RelaxError: The spin ID '#hRGS7:12' corresponds to multiple spins,
including '#hRGS7:12@N' and '#hRGS7:12@H'.
crowlab: [~/relax-4.0.2]>

As you can see, I have all 6 data sets set to load, but only the
very first one appears to do so, and after it loads, it just exits 
out of
relax. Again, I don't know if this is a problem with how I wrote the
script. The Relax_script1 is the one that I load up to run the whole 
thing.
The model free script.py is just the script it reads once relax has 
opened
up.  Again, I can see all the spins are properly loaded, and the 
isotopes
are set. It just everything after the first data set that doesn't 
load.
Thanks again in advance.

Sincerely,
Sam

On Thu, Sep 8, 2016 at 10:15 AM, Mahdi, Sam <
sam.mahdi.846@xxxxxxxxxxx> wrote:

Hi Troels,

Thank you so much. If there is any extra info you need please let
me know.

Sincerely,
Sam

On Thu, Sep 8, 2016 at 9:12 AM, Troels Emtekær Linnet <
tlinnet@xxxxxxxxxxxxx> wrote:

Hi Sam.

I will have some time on Tuesday, and then I will look at it.

Best
Troels


Den onsdag den 7. september 2016 skrev Mahdi, Sam <
sam.mahdi.846@xxxxxxxxxxx>:

Hello Troels,

I  uploaded all the files, and even added in the entire output
that i recieved using model free in script mode. I didn't know if 
all the
files uploaded need to have that link, so only the initial files 
that were
uploaded it, have it.
Thank you in advance for your help!
Sincerely,
Sam

On Wed, Sep 7, 2016 at 12:41 AM, Troels Emtekær Linnet <
tlinnet@xxxxxxxxxxxxx> wrote:

Hi Sam.

You should be able to upload more files after the initial upload.
In the comment thread, please also make a link to this
discussion.

https://mail.gna.org/public/relax-users/2016-09/threads.html
#00001

Best
Troels



2016-09-06 19:10 GMT+02:00 Mahdi, Sam <sam.mahdi.846@xxxxxxxxxxx
:

Thank you for your reply. When I come to upload my data though,
I see there are only 4 available slots I can upload my data. I 
have a total
of 6 data files however, that need to be uploaded (3 of each 
frequency). I
also need to upload the relax -i of 2 different computers, and 
the script
file I've been using for a total of 9 files that need to be 
uploaded. Is
there a way to increase the amount I can upload, or can I upload 
more after
the initial submission?

On Mon, Sep 5, 2016 at 2:46 AM, Troels Emtekær Linnet <
tlinnet@xxxxxxxxxxxxx> wrote:

Hi Sam.

To solve this problem, it would be easier to have access to
some of your data.
Can you upload to: https://gna.org/bugs/?group=relax

Take each of your data files, and delete all data, except 2
spins.
Also provide your script file, or a description of which
button you press in the GUI.

Please also provide information about your system with:
relax -i

Then I will make a tutorial for you. To be added here:
http://wiki.nmr-relax.com/Category:Tutorials

If there is a problem in relax, I will write a systemtest
which will solve the problem.
And the problem will never return.

If this a user error, the tutorial should help to prevent
this, and would be the first step before
adding/modifying the manual.

Regarding using mpirun.
Have a look at this page. Maybe it helps.
http://wiki.nmr-relax.com/OpenMPI


Cheers.


2016-09-03 4:13 GMT+02:00 Mahdi, Sam <
sam.mahdi.846@xxxxxxxxxxx>:

Hello everyone,

So I was able to set up and run the dauvergne_protocol
successfully by
using the script in the wiki. The problem I have come across
now is the
program doesn't seem to read my data. Using the gui interface
I was able to
successfully load my data and run it. When I upload my data
using the
script command:
relax_data.read(ri_id='R1_Agnes',ri_type='R1',frq=599.642*1e
6,
file='R1_Agnes',res_num_col=1,data_col=2,error_col=3)

The output file simply gives errors for amino acids I don't
have data for:
RelaxWarning: The sequence data in the line ['1'] is invalid,
the data is
missing.

This is fine as relax just ignores these values and continues
its
calculations. I only receive this warning for values I don't
have data for.
This is the same thing I got when using the gui interface
(the gui however
showed my values that I have data for and the residue it
corresponds to,
using the script I don't receive such an output, I don't know
whether this
is normal or not). However, since I don't get this warning
for every amino
acid, I assume this means it has read the values for the
other amino acids.
All of my data is the same, relax warnings only pop up for
amino acids that
I don't have data for. The problem is, when I enter the
dAuvergne protocol,
I get the protocol working, it starts running local_tm
however it appears
none of my data has been uploaded:
RelaxWarning: The spin '#hRGS7:2@N' has been deselected
because of missing
relaxation data
RelaxWarning: The spin '#hRGS7:3@N' has been deselected
because of missing
relaxation data

And I get that warning for every single amino acid. From the
output, it
appears to have read the file since it knows exactly which
amino acids I
don't have data for, but I don't know why when it comes to
running the
protocol, it tells me I havn't inputed any data. I have typed
everything
directly according to the script from the wiki. From running
the protoco,
it appears everything has properly been uploaded, structure
data, magnetic
dipole interactions, csa, the data pipe, the analysis
variables, the python
module imports, and setting up the spins from the pdb file.
It appears the
only error is from loading the actual relaxation data.

On a completely unrelated side note, I have been attempting
to run relax on
multiple processors. I have tried 2 different computers, both
fedora linux.
I have mpi4py and openmpi downloaded on both. On one, I can
get relax on
multiple cores working (havn't been able to successfully run
it due to
being unable to upload any data properly). On the other
however, I type in
the mpirun -np --multi='mpi4py' script, but I get no output.
I can see that
it's running in the background (top command), but nothing
pops up, no text
command, nothing. I typed the same mpirun with the --gui, but
that opened
up nothing. On a uni-processor (typing in the exact same
command without
indicating how many cores i.e. no -np --multi='mpi4py') it
works just fine,
so I don't think its my openmpi that's an issue. I don't know
whether this
is an issue with my mpi4py or a personal computer issue
(since on the other
computer relax runs just fine on multiple cores).

Sincerely,
Sam

P.S. when I do enter the top command to see what's running.
My master shows
mpirun, and the 3 slaves display python when I put -np 4, so
I know
something is running in the background. I have 8 cores.

On Wed, Aug 31, 2016 at 6:49 PM, Mahdi, Sam <
sam.mahdi.846@xxxxxxxxxxx>
wrote:

Hello everyone,

I am attempting to run relax on a multi-processor mode. I
have been able
to successfully set-up relax to operate in a
multi-processor mode by using
the mpirun -np #ofprocessors /location/of/relax
--multi='mpi4py'

The problem I encounter is when using the --tee log
dauvergne_protocol.py
command. I receive this error
RelaxError: the script file 'dauvergne_protocol.py' does
not exist

I located the script file and tried to direct to it's path
mpirun /usr/local/Relax/relax-2.2.5/relax --multi='mpi4py'
--tee log
/usr/local/Relax/relax-2.2.5/a
uto-analyses/dauvergne_protocol.py

But i received this error
RelaxError: the script file '/usr/local/Relax/relax-2.2.5/
auto-analyses/dauvergne_protocol.py' does not exist.

Even though I have the script, it doesn't seem to be able
to locate it.

On a side note, in the manual, one dash doesn't actually
run the command.
I.e. in the manual it displays -multi='mpi4py' . What it
should be is
--multi='mpi4py' . The same goes for -tee. It should be
--tee.

Sincerely,
Sam

_______________________________________________
relax (http://www.nmr-relax.com)

This is the relax-users mailing list
relax-users@xxxxxxx

To unsubscribe from this list, get a password
reminder, or change your subscription options,
visit the list information page at
https://mail.gna.org/listinfo/relax-users














Related Messages


Powered by MHonArc, Updated Mon Sep 19 17:20:12 2016