Hi Paul,
Thank you for the code contribution! First of all, I have to
apologise for the length of this email. I have tried to be as
comprehensive as possible.
I have attached the ZIP file to the relax task #7712
(https://gna.org/task/?7712). Note that I have added the [relax-devel
att gna dot org] mailing list to the CC list to bring this out into
the public - it will be permanently archived in multiple locations
(see http://news.gmane.org/gmane.science.nmr.relax.devel,
http://www.mail-archive.com/relax-devel@xxxxxxx/,
https://mail.gna.org/public/relax-devel/,
http://marc.info/?l=relax-devel&r=1&w=2). This is necessary as relax
is an open source project where, without exception, all development
occurs in public. It is supported by a number of developers, though
many are now inactive
(https://gna.org/project/memberlist.php?group=relax). Anyone is free
to become a relax developer and advance the project for their own
needs/papers (http://www.nmr-relax.com/manual/relax_development.html).
Below I will address each of your points. But first, and most
importantly, we need to address the copyright issue. For the code to
be added to relax, each person who has contributed code to the
fitting_main.py file and the original MATLAB scripts needs to
communicate in public, by having a message permanently archived on the
[relax-devel att gna dot org] mailing list, that they agree to have
their code licenced under the Gnu Public Licence version 3 or higher
(GPL3+). This does not prevent the author placing their code under a
different copyright licence. But this is only their code, and the
code which was released under the GPL3 licence can never be retracted.
For details, see http://gplv3.fsf.org/,
http://www.gnu.org/licenses/gpl-3.0.html, or the docs/COPYING file
that comes with relax. I will add a copyright statement to each file
produced, and it will look like the start of
http://svn.gna.org/viewcvs/relax/trunk/test_suite/system_tests/consistency_tests.py?view=markup.
There will be one line for each author. This consists of the years
the author wrote code, the full author name, and maybe a contact email
address. This would also include all authors of the original MATLAB
scripts.
For reference, an archived message thread on this development is:
http://thread.gmane.org/gmane.science.nmr.relax.devel/3790
Also for reference is the history of relaxation dispersion in relax.
Development is happening in a relax branch rather than the main
development line. Therefore it will not be included in an official
relax release until the branch is completed and merged back. The
branch was started by Sébastien Morin who contributed much of the
infrastructure back in 2009. I recently ported it to the newest relax
code base and significantly expanded its capabilities for my own
needs. Troels Linnet is also working on this branch for his own paper
and needs, and is advancing towards becoming a full relax developer.
Now, to the mail:
On 9 July 2013 08:16, Paul Schanda <paul.schanda@xxxxxx> wrote:
Hi Edward,
As promised, I am sending you in the attachment the python functions that we
are using for fitting CPMG data, that was written essentially by Mathilde
Lescanne and Dominique Marion, based on my MATLAB scripts.
Thank you. As discussed at the IBS in Grenoble two weeks ago, this
could one day also be incorporated into the NESSY project
(http://home.gna.org/nessy/). However that project is in a frozen
state so it would require a significant commitment by a dedicated
developer for this to happen.
Also, the code is in a state whereby it can be directly copied and
pasted into relax modules. Only small modifications will be
necessary. This includes adding docstrings
(http://www.python.org/dev/peps/pep-0257/) using the Epydoc format
(http://epydoc.sourceforge.net/,
http://epydoc.sourceforge.net/epytext.html) for the relax API
documentation (http://www.nmr-relax.com/api/). I will also add lots
of comments as the relax coding style requires ~15 to 30% of comment
lines, simply to make it easier for non-coders to understand the code
(http://www.nmr-relax.com/manual/Comments.html). I may ask you,
Mathilde, about these comments later.
In principle, the fit program should directly work as it is, and generate
plots automatically. (using Matplotlib! I would strongly recommend that
relax also uses matplotlib. Creating plots exclusively outside of python is
a pain in the neck, and only useful for writing a paper, which is the step
that one does after having looked at tons of figures...)
As I mentioned, Troels Linnet is also interested in Matplotlib as he
has used that in his own dispersion scripts. Matplotlib support would
be quite useful and is welcome. But it is also limited compared to
the capabilities of Grace. But the main point with matplotlib support
would be that it be added as an optional dependency. For the
automated relaxation dispersion analysis in relax, matplotlib plots
can be created when the Python package is present. We should probably
discuss this in a different thread.
You will find five different versions of fitting CPMG data. In fact we only
use two of them, namely an explicit numerical Bloch-McConnell treatment that
works for perfect 180deg pulses, and a Maple-derived solution of the
Bloch-McConnell treatment. The latter is numerically exact and in practice
the most useful version. It is being used by some of the Kay-(ex-)postdocs.
These, I assume, are the functions returned by MakeFunction() when the
a argument is set to 1 and 5 respectively. Would you have the primary
references for these two with DOI numbers? This is important for the
relax documentation (both the user manual and in the user function
documentation).
The script allows to fit multiple residues and multiple fields at the same
time. This is all controlled via a Excel spreadsheet, where the user can put
the information. In relax you would do this differently, in a GUI, but the
logic would stay pretty much the same. The script uses a python package for
handling Excel files; you may need to install it.
For relax, the details in the Excel spreadsheet would either be placed
directly into a script (for the script UI) or manually input via the
GUI. relax supports all of these options, though in the automated
analysis it is run a little differently. You directly define spin
clusters. Then all spin clusters as well as free spins are optimised
to all selected dispersion models simultaneously. You are however
free to create a non-automated, custom analysis in a script whereby
you can do whatever you wish. relax is designed with flexibility in
mind and allows you to do anything you wish (and if that doesn't work,
then it should be considered as a bug).
If you could incorporate this into relax it would be great.
Once the copyright issues are sorted out and there are responses from
all involved, I could then incorporate this quite quickly into relax.
It will only be a few hours of work - mainly due to the documentation.
So having as much detail as possible about these numerical models
makes my life much easier. See the user manual (you need to compile
it yourself) and relax_disp.select_model user function for how this
documentation is currently set up.
I would also
greatly appreciate if you could help us getting relax to run. Our interest
in sending this stuff to you is that we would like to have an environment
where the data handling (input, output) is done properly, but which
basically does the fitting that we would like to do (Bloch-McConnell).
For the installing the relaxation dispersion branch, Troels has
created a very useful tutorial:
http://wiki.nmr-relax.com/index.php?title=Installation_linux
This includes instructions for building the relax user manual (this
requires LaTeX). Note that this wiki is still in a state of being set
up, so user creation and page editing is not yet fully functional.
As said, it would be really great to have the program create plots
automatically. I don't want to use xmgrace for generating plots, or at least
not if I just want to get a quick overview of the fits. Could you please
have a look how complicated it would be to include matplotlib-plotting into
relax?
The complication is only deciding at which level this would be added.
For example the R2eff/R1rho plots are currently created by the
relax_disp.plot_disp_curves user function after a given dispersion
model has been optimised for all spin clusters and free spins (and
Monte Carlo simulations have been performed). This could simply be
modified by adding an argument for selecting the graph type (set to
either 'grace' or 'matplotlib'). If this is added, then in the
auto-analysis I would enable both. For this, ideas can be taken from
the fitting_main.py script, from Troels' scripts, and from NESSY's
matplotlib plotting.
On another level, there are the grace.view and grace.write user
functions. These are used for plotting all of the other parameters.
If there is interest, then this code could be copied and modified to
the matplotlib.view and matplotlib.write user functions.
Troels, what do you think?
Note that, although Grace and OpenDX are the current ways of viewing
2D and 3D plots, the relax design is not exclusive to these softwares
and allows other plotting capabilities to be relatively easily added
by anyone with interest. Matplotlib, Gnuplot, SigmaPlot, Octave
(which uses Gnuplot), MATLAB, and Mathematica have all been considered
at some point as a plotting output from relax. Though most
users/developers found Grace sufficient so these were never
implemented due to a lack of interest.
Please let me know if you need any further information.
We'll see what is needed as things progress.
Cheers,
Edward
best -
Paul
--
Paul Schanda, Ph.D.
Biomolecular NMR group
Institut de Biologie Structurale Jean-Pierre Ebel (IBS)
41, rue Jules Horowitz
F-38027 Grenoble
France
+33 438 78 95 55
paul.schanda@xxxxxx
http://www.ibs.fr/groups/biomolecular-nmr-spectroscopy?lang=en
Numerical integration of the Bloch-McConnell equations for the relaxation dispersion analysis in relax.