On 7 May 2013 14:18, Troels Emtekær Linnet <
tlinnet@xxxxxxxxx> wrote:
> Hi Edward.
>
> Thanks for the lengthy explanation, and I hope that I can honor your effort
> in explaining. :-)
>
> I would be interested to get these things to work in relax, which we most
> often use in our lab:
> - off-resonance T1rho
> - CPMG
> -- Fast (Meiboom)
> -- slow/intermediate (Richard-Carver)
> -- very slow (Tollinger/Kay equation)
>
> So I will go for the Tollinger, since that is the "easiest" next to the
> fast.
>
> I would need to do the code development at my Windows machine, and
> I checked out the relax-disp branch yesterday.
> Should I/How do I compile with scons under windows?
> Or should I install the pre-compiled Windows binaries, and replace the
> files?
>
> Best
> Troels
>
> Troels Emtekær Linnet
> Ved kløvermarken 9,
1.th
> 2300 København S
> Mobil:
+45 60210234
>
>
> 2013/5/7 Edward d'Auvergne <
edward@xxxxxxxxxxxxx>
>>
>> Hi again,
>>
>> Just for reference in the mailing list archives, the sub-thread has
>> appeared as a new thread at
>>
http://thread.gmane.org/gmane.science.nmr.relax.devel/3835.
>>
>> Regards,
>>
>> Edward
>>
>>
>>
>>
>> On 7 May 2013 11:36, Edward d'Auvergne <
edward@xxxxxxxxxxxxx> wrote:
>> > Hi Troels,
>> >
>> > This sub-thread (which will appear at
>> >
http://thread.gmane.org/gmane.science.nmr.relax.devel/3833) will
>> > hopefully be a mini-tutorial covering the development of the
>> > relax_disp branch. Before you can be accepted as a relax developer
>> > with commit access to the source code repository, you should first
>> > submit changes as patches. This takes longer initially, but it allows
>> > the other relax developers to see how you code and if you are
>> > following the coding conventions as described in the development
>> > chapter of the relax manual
>> > (
http://www.nmr-relax.com/manual/relax_development.html). I can give
>> > you feedback as you go as to how to improve the code to fit into
>> > relax. We, the relax developers, will after a few patches have a
>> > private vote to accept you as a relax developer. This is standard
>> > practice in an open source project. The full procedure for becoming a
>> > developer is detailed in the 'Committers' section of the manual
>> > (
http://www.nmr-relax.com/manual/Committers.html). The PDF version of
>> > the manual is easier to read
>> > (
http://download.gna.org/relax/manual/relax.pdf). Patches can be
>> > posted to the patch tracker (
https://gna.org/patch/?group=relax).
>> >
>> > relax development begins and ends with the test suite. The idea is
>> > that, before any code is present, a relax system test must be created.
>> > This allows you to develop the ideas for how the UI should work with
>> > the analysis - i.e. which new user functions will need to be created
>> > and which ones will need to be expanded. A script is added to
>> > test_suite/system_tests/scripts/relax_disp/ and then a test added to
>> > test_suite/system_tests/relax_disp.py which executes the script and
>> > then checks the data and results. For example see the script
>> > 'test_suite/system_tests/scripts/relax_disp/hansen_data.py' and the
>> > function test_hansen_cpmg_data_fast_2site() in the file
>> > 'test_suite/system_tests/relax_disp.py'. This is obviously not
>> > complete as only the script is executed - the results are not yet
>> > checked (as we do not know what the result for the optimised model
>> > should be yet). This individual test can be executed with the
>> > command:
>> >
>> > $ relax -s Relax_disp.test_hansen_cpmg_data_fast_2site
>> >
>> > This test, as well as the other Relax_disp tests, were created by
>> > Sebastien Morin when he started the development of the relax_disp
>> > branch. I have renamed everything since he added it, and will
>> > probably do so again soon. It is best to develop for the script UI
>> > first - the GUI will later be modified around the graphical versions
>> > of the user functions, or directly accessing the back end of the user
>> > function. Due to the advanced state of the relax_disp branch, you
>> > probably do not need to worry about new user functions. This may be
>> > needed if you would like to expand the analysis to new types of data
>> > (for example off-resonance R1rho where R1 data need to be measured and
>> > used in the analysis, H/D exchange, etc.).
>> >
>> > The test suite is one area which can be expanded to handle the
>> > different CPMG models. The testing is currently not very extensive.
>> > For example before a new dispersion model is added to relax, it would
>> > be good if synthetic data were to be created in an external program (a
>> > Python script, Matlab, Mathematica, Maxima, etc.). It is very
>> > important that relax is not used to create the data. Synthetic data
>> > is very important for making sure that relax obtains the correct
>> > result, as you know what the result should be. With measured data you
>> > can never really know what the true result is - this is the entire
>> > point of the mathematical field of modelling (this field makes that of
>> > NMR look very, very small). Synthetic data is also useful for double
>> > checking results against other relaxation dispersion software (for
>> > reference: NESSY -
http://home.gna.org/nessy/; CPMGFit -
>> >
http://www.palmer.hs.columbia.edu/software/cpmgfit.html; ShereKhan -
>> >
http://sherekhan.bionmr.org/; CATIA -
>> >
http://www.biochem.ucl.ac.uk/hansen/catia/). Data could also be taken
>> > from Art Palmer's CPMGFit manual
>> > (
http://www.palmer.hs.columbia.edu/software/cpmgfit_manual.html).
>> > This would need to be converted into peak intensities in a peak list
>> > file, but that is easy enough by simply picking random I0 values for
>> > the exponential curves. The data could be passed quickly through each
>> > of the models of the CPMGFit program and results noted. Then the
>> > results would be added to the checks of different relax system tests.
>> >
>> > Each different data set used in the testing process should be located
>> > in its own directory in test_suite/shared_data/dispersion/. That
>> > directory can include the data and all scripts used to generate the
>> > data and, for reference, it can also contain subdirectories for
>> > holding the input and output for different programs (as long as the
>> > files are not too big).
>> >
>> > The current state of the branch is that all of the user functions are
>> > pretty close to complete. The user function consists of a front end
>> > definition in user_functions/, and a backend either in pipe_control/
>> > or specific_analyses/. The relaxation dispersion target function
>> > setup for optimisation is close to complete. You can see this in the
>> > minimise() method of the specific_analyses/relax_disp/__init__.py
>> > file, and then the __init__() method of the class in
>> > target_functions/relax_disp.py. As you will see in the model_loop()
>> > method of the specific_analyses/relax_disp/__init__.py code,
>> > clustering of spin systems is already part of this design - everything
>> > handles a group of spins assuming the same parameter values. One
>> > missing feature that I might work on soon is the handling of missing
>> > input data, as this affects my current work. This is a problem
>> > currently caught by the
>> > test_suite/shared_data/dispersion/Hansen/relax_disp.py script, as
>> > residue :71 is missing data at one field strength. But once the
>> > dispersion tests have been expanded, this can be tested properly by
>> > deleting data for single points on the exponential curves, deleting
>> > entire exponential curves (or dispersion points for the two-point
>> > analysis type), or all data from a single spectrometer field strength
>> > for a single spin.
>> >
>> > So I would suggest that you pick one of the dispersion models you are
>> > interested in and try to implement that. I am working on the Luz and
>> > Meiboom, 1963 model, but all of the other models are safe to work on.
>> > Just say which you are interested in so that we don't both change the
>> > same code. The system test data would come first. The formula can be
>> > taken, a set of parameters for 2-3 spins chosen, and a simple script
>> > written to generate the R2eff data, importantly at multiple magnetic
>> > field strengths. That data can then be converted into a generic peak
>> > list for different time periods on a basic 2-parameter exponential
>> > curve. See the 'File formats' section of the
>> > spectrum.read_intensities user function docstring, for example by
>> > typing help(spectrum.read_intensities) in the prompt UI. In the same
>> > script the creation of input files for other programs could be added,
>> > possibly at a later stage, and the data quickly run through CPMGFit,
>> > for example, for a sanity check.
>> >
>> > If you do test the other programs, you may encounter a severe bug in
>> > one of their models. No software is bug free. In such a case, we
>> > should communicate with the authors in private and they can decide
>> > what to do. You can see that I did this with Art Palmer's Modelfree
>> > program at
>> >
http://biochemistry.hs.columbia.edu/labs/palmer/software/modelfree.html.
>> > Versions 4.16 and 4.20 consist of patches that I send to Art to fix
>> > compilation issues and other bugs (I pointed out the grid search
>> > problem due to the singular matrix failure of the Levenberg-Marquardt
>> > algorithm and Art made that change himself).
>> >
>> > Once some data has been created and files attached to the patch
>> > tracker (
https://gna.org/patch/?group=relax), then the relax script
>> > can be written and added to
>> > test_suite/system_tests/scripts/relax_disp/. The best way would
>> > probably be for one of the current scripts to be copied (by me to
>> > start with) in the repository and then you make small changes to it
>> > and send the patches created with:
>> >
>> > $ svn diff > patch
>> >
>> > Then the script execution and data and parameter checking code can be
>> > added to test_suite/system_tests/relax_disp.py - again you can look at
>> > the other methods in that file and create a new one by copying how an
>> > old method operates. In that system test you would check that the
>> > original parameters have been found.
>> >
>> > At this stage, the test should run fine up to the grid_search user
>> > function, and then fail (or possibly at the relax_disp.select_model
>> > user function call in the script depending on whether you use the
>> > auto-analysis code in auto_analyses.relax_disp or not). This is the
>> > point where the model can be implemented. Then you would take the
>> > following steps:
>> >
>> > - Add a description of the new model with the equation and reference
>> > to the user_functions.relax_disp module.
>> >
>> > - Add the model and its parameters to the _select_model() method of
>> > the specific_analyses/relax_disp/__init__.py file.
>> >
>> > - Add any new parameter definitions to the top of the
>> > specific_analyses/relax_disp/__init__.py file in the __init__() method
>> > as needed. If new parameters are needed, then there are various
>> > places in the specific_analyses.relax_disp package where support will
>> > be needed, mainly in the specific_analyses.relax_disp.parameters
>> > module.
>> >
>> > - Create a new module in the lib.dispersion package for the model
>> > function. This module will eventually hold the model function, the
>> > gradient (each partial derivative with respect to each parameter would
>> > be in a different function), and the Hessian (the matrix of second
>> > partial derivatives). Having the gradient and Hessian will allow for
>> > the more powerful optimisation algorithms to be used.
>> >
>> > - Add a new method to target_functions/relax_disp.py which uses the
>> > new code in lib.dispersion to calculate R2eff values, combine this
>> > with the chi2 function, and return the chi-squared value (see the
>> > current func_LM63() method for how to do this).
>> >
>> > - Finally, see if the system test passes. If not, then it is time to
>> > debug.
>> >
>> > During these steps, the unit test part of the test suite can be used
>> > to make sure that individual functions and methods behave correctly.
>> > This is useful as users will always find a way to break your code.
>> > Once the system test passes, then you will know that the
>> > implementation is complete and fully functional.
>> >
>> >
>> > If your interest is in the numerical integration of the
>> > Bloch-McConnell equations, then the procedure might be slightly
>> > different. We would have to discuss this in more detail, with paper
>> > references and the necessary equations. But I think that all of this
>> > can be handled in a module of the lib.dispersion package, and the rest
>> > of the above detailed procedure would be the same. I hope this post
>> > wasn't too long for you!
>> >
>> > Regards,
>> >
>> > Edward
>> >
>> >
>> >
>> >
>> > On 6 May 2013 21:14, Troels Emtekær Linnet <
tlinnet@xxxxxxxxx> wrote:
>> >> Hi Edward.
>> >>
>> >> When you have completed your ideas of change to the
>> >> disp branch, could you send me a notits?
>> >>
>> >> And maybe a script file, how to launch the code?
>> >>
>> >> Then I could try to figure out where I should extend new code.
>> >>
>> >> Best
>> >> Troels
>> >>
>> >>
>> >> _______________________________________________
>> >> relax (
http://www.nmr-relax.com)
>> >>
>> >> This is the relax-devel mailing list
>> >>
relax-devel@xxxxxxx
>> >>
>> >> To unsubscribe from this list, get a password
>> >> reminder, or change your subscription options,
>> >> visit the list information page at
>> >>
https://mail.gna.org/listinfo/relax-devel
>> >>
>
>
Re: Development of the relax-disp branch.