mailRe: Using multi-processor for model_free


Others Months | Index by Date | Thread Index
>>   [Date Prev] [Date Next] [Thread Prev] [Thread Next]

Header


Content

Posted by Mahdi, Sam on September 10, 2016 - 02:05:
Hi Troels,

Additional question that I had, if you could also look into this as well on
Tuesday please. I have decided to try to write a script to automate this
whole process (since I won't be using the gui to do model free), and I've
come across a problem. I can successfully open up relax using openmpi, and
can load the pdb file, and assign all the spins and isotopes; however, it
appears it will only load one data file (the very first one I'll have
inputed in the script). I don't know if there is a problem with how I wrote
my script. Not only will it not load the rest of my data sets, it won't
actually run dAuvergne's protocol either, it'll just load the data set and
exit out of the program. Attached is the script I wrote for relax.

This is the output once relax has loaded

script = 'model_free_sample_script.py'
----------------------------------------------------------------------------------------------------
from time import asctime, localtime
from auto_analyses.dauvergne_protocol import dAuvergne_protocol
DIFF_MODEL=['local_tm','sphere','prolate','oblate','ellipsoid','final']
MF_MODELS=['m0','m1','m2','m3','m4','m5','m6','m7','m8','m9']
LOCAL_TM_MODELS=['tm0','tm1','tm2','tm3','tm4','tm5','tm6','tm7','tm7','tm8','tm9']
GRID_INC=11
MIN_ALGOR='newton'
MC_NUM=500
CONV_LOOP=True
pipe_bundle="mf(%s)"%asctime(localtime())
name="origin-"+pipe_bundle
pipe.create(name,'mf',bundle=pipe_bundle)
structure.read_pdb('2d9j.pdb',set_mol_name='hRGS7')
structure.load_spins('@N',ave_pos=True)
structure.load_spins('@NE1',ave_pos=True)
structure.load_spins('@H',ave_pos=True)
structure.load_spins('@HE1',ave_pos=True)
spin.isotope('15N',spin_id='@N*')
spin.isotope('1H',spin_id='@H*')
relax_data.read(ri_id='R1_Agnes',ri_type='R1',frq=599.642*1e6,
file='R1_Agnes',res_num_col=1,data_col=2,error_col=3)
relax_data.read(ri_id='R2_Agnes',ri_type='R2',frq=599.642*1e6,
file='R2_Agnes',res_num_col=1,data_col=2,error_col=3)
relax_data.read(ri_id='ssNOE_Agnes',ri_type='NOE',frq=599.642*1e6,
file='ssNOE_Agnes',res_num_col=1,data_col=2,error_col=3)
relax_data.read(ri_id='R1_NMRFAM',ri_type='R1',frq=799.642*1e6,
file='R1_NMRFAM',res_num_col=1,data_col=2,error_col=3)
relax_data.read(ri_id='R2_NMRFAM',ri_type='R2',frq=799.642*1e6,
file='R2_NMRFAM',res_num_col=1,data_col=2,error_col=3)
relax_data.read(ri_id='ssNOE_NMRFAM',ri_type='NOE',frq=799.642*1e6,
file='ssNOE_NMRFAM',res_num_col=1,data_col=2,error_col=3)
interatom.define(spin_id1='@N',spin_id2='@H', direct_bond=True)
interatom.define(spin_id1='@NE1',spin_id2='@HE1', direct_bond=True)
interatom.set_dist(spin_id1='@N*',spin_id2='@H*',ave_dist=1.02*1e-10)
interatom.unit_vectors()
value.set(-172*1e-6,'csa',spin_id='@N*')
dAuvergne_protocol(pipe_name=name,pipe_bundle=pipe_bundle,diff_model=DIFF_MODEL,mf_models=MF_MODELS,local_tm_models=LOCAL_TM_MODELS,grid_inc=GRID_INC,min_algor=MIN_ALGOR,mc_sim_num=MC_NUM,conv_loop=CONV_LOOP)

So it indicates that my script has loaded. However, after it loads the
spins from the pdb file, this is what happens after my first data set has
been loaded:

relax> relax_data.read(ri_id='R1_Agnes', ri_type='R1', frq=599642000.0,
file='R1_Agnes', dir=None, spin_id_col=None, mol_name_col=None,
res_num_col=1, res_name_col=None, spin_num_col=None, spin_name_col=None,
data_col=2, error_col=3, sep=None, spin_id=None)
Opening the file 'R1_Agnes' for reading.
RelaxWarning: The sequence data in the line ['Residue', 'R1', 'Error'] is
invalid, the residue number data 'Residue' is invalid.
RelaxWarning: The sequence data in the line ['1'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['2'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['3'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['4'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['5'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['6'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['7'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['8'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['9'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['10'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['11'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['16'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['17'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['18'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['21'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['22'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['23'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['26'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['27'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['28'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['31'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['40'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['46'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['58'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['61'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['62'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['63'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['73'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['76'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['79'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['81'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['82'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['85'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['94'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['97'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['99'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['106'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['115'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['121'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['126'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['127'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['134'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['135'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['136'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['137'] is invalid, the data is
missing.
RelaxWarning: The sequence data in the line ['139'] is invalid, the data is
missing.

RelaxError: The spin ID '#hRGS7:12' corresponds to multiple spins,
including '#hRGS7:12@N' and '#hRGS7:12@H'.
crowlab: [~/relax-4.0.2]>

As you can see, I have all 6 data sets set to load, but only the very first
one appears to do so, and after it loads, it just exits out of relax.
Again, I don't know if this is a problem with how I wrote the script. The
Relax_script1 is the one that I load up to run the whole thing. The model
free script.py is just the script it reads once relax has opened up.
Again, I can see all the spins are properly loaded, and the isotopes are
set. It just everything after the first data set that doesn't load. Thanks
again in advance.

Sincerely,
Sam

On Thu, Sep 8, 2016 at 10:15 AM, Mahdi, Sam <sam.mahdi.846@xxxxxxxxxxx>
wrote:

Hi Troels,

Thank you so much. If there is any extra info you need please let me know.

Sincerely,
Sam

On Thu, Sep 8, 2016 at 9:12 AM, Troels Emtekær Linnet <
tlinnet@xxxxxxxxxxxxx> wrote:

Hi Sam.

I will have some time on Tuesday, and then I will look at it.

Best
Troels


Den onsdag den 7. september 2016 skrev Mahdi, Sam <
sam.mahdi.846@xxxxxxxxxxx>:

Hello Troels,

I  uploaded all the files, and even added in the entire output that i
recieved using model free in script mode. I didn't know if all the files
uploaded need to have that link, so only the initial files that were
uploaded it, have it.
Thank you in advance for your help!
Sincerely,
Sam

On Wed, Sep 7, 2016 at 12:41 AM, Troels Emtekær Linnet <
tlinnet@xxxxxxxxxxxxx> wrote:

Hi Sam.

You should be able to upload more files after the initial upload.
In the comment thread, please also make a link to this discussion.

https://mail.gna.org/public/relax-users/2016-09/threads.html#00001

Best
Troels



2016-09-06 19:10 GMT+02:00 Mahdi, Sam <sam.mahdi.846@xxxxxxxxxxx>:

Thank you for your reply. When I come to upload my data though, I see
there are only 4 available slots I can upload my data. I have a total 
of 6
data files however, that need to be uploaded (3 of each frequency). I 
also
need to upload the relax -i of 2 different computers, and the script 
file
I've been using for a total of 9 files that need to be uploaded. Is 
there a
way to increase the amount I can upload, or can I upload more after the
initial submission?

On Mon, Sep 5, 2016 at 2:46 AM, Troels Emtekær Linnet <
tlinnet@xxxxxxxxxxxxx> wrote:

Hi Sam.

To solve this problem, it would be easier to have access to some of
your data.
Can you upload to: https://gna.org/bugs/?group=relax

Take each of your data files, and delete all data, except 2 spins.
Also provide your script file, or a description of which button you
press in the GUI.

Please also provide information about your system with:
relax -i

Then I will make a tutorial for you. To be added here:
http://wiki.nmr-relax.com/Category:Tutorials

If there is a problem in relax, I will write a systemtest which will
solve the problem.
And the problem will never return.

If this a user error, the tutorial should help to prevent this, and
would be the first step before
adding/modifying the manual.

Regarding using mpirun.
Have a look at this page. Maybe it helps.
http://wiki.nmr-relax.com/OpenMPI


Cheers.


2016-09-03 4:13 GMT+02:00 Mahdi, Sam <sam.mahdi.846@xxxxxxxxxxx>:

Hello everyone,

So I was able to set up and run the dauvergne_protocol successfully
by
using the script in the wiki. The problem I have come across now is
the
program doesn't seem to read my data. Using the gui interface I was
able to
successfully load my data and run it. When I upload my data using the
script command:
relax_data.read(ri_id='R1_Agnes',ri_type='R1',frq=599.642*1e6,
file='R1_Agnes',res_num_col=1,data_col=2,error_col=3)

The output file simply gives errors for amino acids I don't have
data for:
RelaxWarning: The sequence data in the line ['1'] is invalid, the
data is
missing.

This is fine as relax just ignores these values and continues its
calculations. I only receive this warning for values I don't have
data for.
This is the same thing I got when using the gui interface (the gui
however
showed my values that I have data for and the residue it corresponds
to,
using the script I don't receive such an output, I don't know
whether this
is normal or not). However, since I don't get this warning for every
amino
acid, I assume this means it has read the values for the other amino
acids.
All of my data is the same, relax warnings only pop up for amino
acids that
I don't have data for. The problem is, when I enter the dAuvergne
protocol,
I get the protocol working, it starts running local_tm however it
appears
none of my data has been uploaded:
RelaxWarning: The spin '#hRGS7:2@N' has been deselected because of
missing
relaxation data
RelaxWarning: The spin '#hRGS7:3@N' has been deselected because of
missing
relaxation data

And I get that warning for every single amino acid. From the output,
it
appears to have read the file since it knows exactly which amino
acids I
don't have data for, but I don't know why when it comes to running
the
protocol, it tells me I havn't inputed any data. I have typed
everything
directly according to the script from the wiki. From running the
protoco,
it appears everything has properly been uploaded, structure data,
magnetic
dipole interactions, csa, the data pipe, the analysis variables, the
python
module imports, and setting up the spins from the pdb file. It
appears the
only error is from loading the actual relaxation data.

On a completely unrelated side note, I have been attempting to run
relax on
multiple processors. I have tried 2 different computers, both fedora
linux.
I have mpi4py and openmpi downloaded on both. On one, I can get
relax on
multiple cores working (havn't been able to successfully run it due
to
being unable to upload any data properly). On the other however, I
type in
the mpirun -np --multi='mpi4py' script, but I get no output. I can
see that
it's running in the background (top command), but nothing pops up,
no text
command, nothing. I typed the same mpirun with the --gui, but that
opened
up nothing. On a uni-processor (typing in the exact same command
without
indicating how many cores i.e. no -np --multi='mpi4py') it works
just fine,
so I don't think its my openmpi that's an issue. I don't know
whether this
is an issue with my mpi4py or a personal computer issue (since on
the other
computer relax runs just fine on multiple cores).

Sincerely,
Sam

P.S. when I do enter the top command to see what's running. My
master shows
mpirun, and the 3 slaves display python when I put -np 4, so I know
something is running in the background. I have 8 cores.

On Wed, Aug 31, 2016 at 6:49 PM, Mahdi, Sam <
sam.mahdi.846@xxxxxxxxxxx>
wrote:

Hello everyone,

I am attempting to run relax on a multi-processor mode. I have
been able
to successfully set-up relax to operate in a multi-processor mode
by using
the mpirun -np #ofprocessors /location/of/relax --multi='mpi4py'

The problem I encounter is when using the --tee log
dauvergne_protocol.py
command. I receive this error
RelaxError: the script file 'dauvergne_protocol.py' does not exist

I located the script file and tried to direct to it's path
mpirun /usr/local/Relax/relax-2.2.5/relax --multi='mpi4py' --tee
log
/usr/local/Relax/relax-2.2.5/auto-analyses/dauvergne_protocol.py

But i received this error
RelaxError: the script file '/usr/local/Relax/relax-2.2.5/
auto-analyses/dauvergne_protocol.py' does not exist.

Even though I have the script, it doesn't seem to be able to
locate it.

On a side note, in the manual, one dash doesn't actually run the
command.
I.e. in the manual it displays -multi='mpi4py' . What it should be
is
--multi='mpi4py' . The same goes for -tee. It should be --tee.

Sincerely,
Sam

_______________________________________________
relax (http://www.nmr-relax.com)

This is the relax-users mailing list
relax-users@xxxxxxx

To unsubscribe from this list, get a password
reminder, or change your subscription options,
visit the list information page at
https://mail.gna.org/listinfo/relax-users









Related Messages


Powered by MHonArc, Updated Mon Sep 12 14:20:08 2016