Re: Idea to fix systemtest? -- June 16, 2014 - 08:38

Hi,

Using manual bisection, I found the problem originates with r23946
(http://thread.gmane.org/gmane.science.nmr.relax.scm/21696,
http://svn.gna.org/viewcvs/relax/branches/disp_spin_speed/target_functions/relax_disp.py?view=diff&r1=23945&r2=23946&pathrev=23946).
The problem is clearly as follows.  The relax_disp.plot_disp_curves
user function backend obtains part of its data via the lines:

"""
            # Back calculate R2eff data for the second sets of plots.
            back_calc =
specific_analyses.relax_disp.optimisation.back_calc_r2eff(spin=spin,
spin_id=spin_id, cpmg_frqs=cpmg_frqs_new,
spin_lock_nu1=spin_lock_nu1_new)
"""

This is now different, as the back_calc pure array structure now
returned is different from the list of lists structures where the last
dimension of the dispersion points was of different length for each
{ei, si, mi, oi}.  But look a few lines before the error line:

"""
                    # Loop over the dispersion points.
                    for di in range(len(back_calc[ei][0][mi][oi])):
                        # Skip invalid points (values of 1e100).
                        if back_calc[ei][0][mi][oi][di] > 1e50:
                            continue

                        # The X point.
                        if exp_type in EXP_TYPE_LIST_CPMG:
                            point = cpmg_frqs_new[ei][mi][oi][di]
"""

Here the loop is assuming that the number of dispersion points is
given by the length of the last back_calc element!  You have changed
this so it is no longer the case, hence the IndexError in the
cpmg_frqs_new structure.  So there are a few options for you:

  - Find a different way of looping over di here.  The
specific_analyses.relax_disp.data.loop_point() function might work.

  - Modify the specific_analyses.relax_disp.optimisation.back_calc_r2eff()
function so that the back_calc data structure it obtains from the
target function is the correct form.  I.e. you convert it into an
array-type in the target function, but here you need to convert it
back to a list of lists.

  - Create the get_back_calc() method in the dispersion target
function class, as mentioned in
http://thread.gmane.org/gmane.science.nmr.relax.scm/21692/focus=6238.
This would perform an array-type to list-of-lists conversion, the
exact reverse what you now do in __init__().  This is by far the most
elegant solution for the API, as then your array-type data structure
is only know to the target function class - nothing else touches it.

The last option should only be a few lines using either the numpy
tolist() function or looping and the list append() method, and it is
very easy to set up.  It has the added benefit in that all of these
problems of accessing back_calc will just go away, and it will avoid
breakages in the future.

Regards,

Edward


On 15 June 2014 23:02, Troels Emtekær Linnet <tlinnet@xxxxxxxxxxxxx> wrote:
> Hi Ed.
>
> This now fails:
>  Relax_disp.test_bug_21460_disp_cluster_fail
>
> "/Users/tlinnet/software/disp_spin_speed/specific_analyses/relax_disp/data.py",
> line 1863, in plot_disp_curves
>     point = cpmg_frqs_new[ei][mi][oi][di]
> IndexError: list index out of range
>
> Do you have an idea?
>
> Best
> Troels
>
>
> -------------------
> ---------------------
> - The 'R2eff' model -
> ---------------------
>
>
> relax> pipe.copy(pipe_from='origin - relax_disp (Thu Jan  2 13:46:44
> 2014)', pipe_to='R2eff - relax_disp (Thu Jan  2 13:46:44 2014)',
> bundle_to='relax_disp (Thu Jan  2 13:46:44 2014)')
>
> relax> pipe.switch(pipe_name='R2eff - relax_disp (Thu Jan  2 13:46:44
> 2014)')
>
> relax> relax_disp.select_model(model='R2eff')
> The model for determining the R2eff/R1rho values from peak intensities.
>
> relax> calc(verbosity=1)
> Calculating the R2eff/R1rho values for fixed relaxation time period data.
> Spin ':459@N'.
>
>
> Results writing
> ===============
>
>
> relax>
> relax_disp.plot_disp_curves(dir='/var/folders/ww/1jkhkh315x57jglgxnr9g24w0000gp/T/tmppeiu5o/R2eff',
> num_points=1000, extend=500.0, force=True)
> Opening the file
> '/var/folders/ww/1jkhkh315x57jglgxnr9g24w0000gp/T/tmppeiu5o/R2eff/disp_459_N.agr'
> for writing.
> Opening the file
> '/var/folders/ww/1jkhkh315x57jglgxnr9g24w0000gp/T/tmppeiu5o/R2eff/grace2images.py'
> for writing.
>
> relax>
> relax_disp.write_disp_curves(dir='/var/folders/ww/1jkhkh315x57jglgxnr9g24w0000gp/T/tmppeiu5o/R2eff',
> force=True)
> Opening the file
> '/var/folders/ww/1jkhkh315x57jglgxnr9g24w0000gp/T/tmppeiu5o/R2eff/disp_459_N.out'
> for writing.
>
> relax> value.write(param='r2eff', file='r2eff.out',
> dir='/var/folders/ww/1jkhkh315x57jglgxnr9g24w0000gp/T/tmppeiu5o/R2eff',
> scaling=1.0, comment=None, bc=False, force=True)
> Opening the file
> '/var/folders/ww/1jkhkh315x57jglgxnr9g24w0000gp/T/tmppeiu5o/R2eff/r2eff.out'
> for writing.
>
> relax> grace.write(x_data_type='res_num', y_data_type='r2eff',
> spin_id=None, plot_data='value', file='r2eff.agr',
> dir='/var/folders/ww/1jkhkh315x57jglgxnr9g24w0000gp/T/tmppeiu5o/R2eff',
> force=True, norm=False)
> Opening the file
> '/var/folders/ww/1jkhkh315x57jglgxnr9g24w0000gp/T/tmppeiu5o/R2eff/r2eff.agr'
> for writing.
>
> relax> results.write(file='results',
> dir='/var/folders/ww/1jkhkh315x57jglgxnr9g24w0000gp/T/tmppeiu5o/R2eff',
> compress_type=1, force=True)
> Opening the file
> '/var/folders/ww/1jkhkh315x57jglgxnr9g24w0000gp/T/tmppeiu5o/R2eff/results.bz2'
> for writing.
>
>
>
> ----------------------
> - The 'No Rex' model -
> ----------------------
>
>
> relax> pipe.copy(pipe_from='origin - relax_disp (Thu Jan  2 13:46:44
> 2014)', pipe_to='No Rex - relax_disp (Thu Jan  2 13:46:44 2014)',
> bundle_to='relax_disp (Thu Jan  2 13:46:44 2014)')
>
> relax> pipe.switch(pipe_name='No Rex - relax_disp (Thu Jan  2 13:46:44
> 2014)')
>
> relax> relax_disp.select_model(model='No Rex')
> The model for no chemical exchange relaxation.
>
> relax> value.copy(pipe_from='R2eff - relax_disp (Thu Jan  2 13:46:44
> 2014)', pipe_to='No Rex - relax_disp (Thu Jan  2 13:46:44 2014)',
> param='r2eff')
>
>
> Optimisation
> ============
>
>
> Nesting and model equivalence checks
> ------------------------------------
>
> No model nesting or model equivalence detected.
>
> relax> grid_search(lower=None, upper=None, inc=3, constraints=True,
> verbosity=1)
>
>
> Fitting to the spin block [':459@N']
> ------------------------------------
>
> Unconstrained grid search size: 9 (constraints may decrease this size).
>
>
> Grid search
> ~~~~~~~~~~~
>
> Searching through 9 grid nodes.
> k: 0        xk: [            0.5,             0.5]    fk: 436.862516468
>
> k: 3        xk: [            0.5,            1.25]    fk: 406.49758716
>
>
> Optimised parameter values:
> r2 (SQ CPMG - 0.00059977 MHz)         5.000000000000000
> r2 (SQ CPMG - 599.77212354 MHz)        12.500000000000000
>
> relax> minimise(min_algor='simplex', line_search=None, hessian_mod=None,
> hessian_type=None, func_tol=1e-05, grad_tol=None, max_iter=1000,
> constraints=True, scaling=True, verbosity=1)
>
>
> Fitting to the spin block [':459@N']
> ------------------------------------
>
>
>
> Logarithmic barrier function
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> k: 0        xk: [            0.5,            1.25]    fk: 406.49758716
>
> Entering sub-algorithm.
>  Simplex minimisation
>  ~~~~~~~~~~~~~~~~~~~~
>
> k: 0        xk: [            0.5,            1.25]    fk: 406.443271114
>
>
> Parameter values: [0.0021207261241890383, 0.89458126560725759]
> Function value:   1.2065517126305849
>  Iterations:       43
> Function calls:   83
> Gradient calls:   0
>  Hessian calls:    0
> Warning:          None
>
> k: 1        xk: [ 0.002120726124,    0.8945812656]    fk: 1.20356618386
>
> Entering sub-algorithm.
>  Simplex minimisation
> ~~~~~~~~~~~~~~~~~~~~
>
> k: 0        xk: [ 0.002120726124,    0.8945812656]    fk: 1.20336591206
>
>
> Parameter values: [0.00022841489930163758, 0.89459438262210345]
>  Function value:   1.1986979999438134
> Iterations:       23
> Function calls:   48
>  Gradient calls:   0
> Hessian calls:    0
> Warning:          None
>
> k: 2        xk: [0.0002284148993,    0.8945943826]    fk: 1.19844299599
>
> Entering sub-algorithm.
>  Simplex minimisation
>  ~~~~~~~~~~~~~~~~~~~~
>
> k: 0        xk: [0.0002284148993,    0.8945943826]    fk: 1.19844554603
>
>
> Parameter values: [0.00022841489930163758, 0.89459438262210345]
> Function value:   1.1984455460281873
>  Iterations:       9
> Function calls:   21
> Gradient calls:   0
>  Hessian calls:    0
> Warning:          None
>
> k: 3        xk: [0.0002284148993,    0.8945943826]    fk: 1.19844895966
>
> Entering sub-algorithm.
>  Simplex minimisation
> ~~~~~~~~~~~~~~~~~~~~
>
> k: 0        xk: [0.0002284148993,    0.8945943826]    fk: 1.19844302149
>
>
> Parameter values: [0.00022841489930163758, 0.89459438262210345]
>  Function value:   1.1984430214890309
> Iterations:       9
> Function calls:   21
>  Gradient calls:   0
> Hessian calls:    0
> Warning:          None
>
>
> Parameter values: [0.00022841489930163758, 0.89459438262210345]
> Function value:   1.1984429959886356
> Iterations:       84
> Function calls:   173
> Gradient calls:   0
> Hessian calls:    0
> Warning:          None
>
>
> Optimised parameter values:
> r2 (SQ CPMG - 0.00059977 MHz)         0.002284148993016
> r2 (SQ CPMG - 599.77212354 MHz)         8.945943826221034
>
> relax> eliminate(function=None, args=None)
>
>
> Results writing
> ===============
>
>
> relax>
> relax_disp.plot_disp_curves(dir='/var/folders/ww/1jkhkh315x57jglgxnr9g24w0000gp/T/tmppeiu5o/No
> Rex', num_points=1000, extend=500.0, force=True)
> Opening the file
> '/var/folders/ww/1jkhkh315x57jglgxnr9g24w0000gp/T/tmppeiu5o/No
> Rex/disp_459_N.agr' for writing.
> Traceback (most recent call last):
>   File
> "/Users/tlinnet/software/disp_spin_speed/test_suite/system_tests/relax_disp.py",
> line 926, in test_bug_21460_disp_cluster_fail
>     relax_disp.Relax_disp(pipe_name="origin - relax_disp (Thu Jan  2
> 13:46:44 2014)", pipe_bundle="relax_disp (Thu Jan  2 13:46:44 2014)",
> results_dir=self.tmpdir, models=['R2eff', 'No Rex', 'CR72', 'NS CPMG 2-site
> expanded'], grid_inc=3, mc_sim_num=5, modsel='AIC', pre_run_dir=None,
> insignificance=1.0, numeric_only=False, mc_sim_all_models=False,
> eliminate=True)
>   File
> "/Users/tlinnet/software/disp_spin_speed/auto_analyses/relax_disp.py", line
> 124, in __init__
>     self.run()
>   File
> "/Users/tlinnet/software/disp_spin_speed/auto_analyses/relax_disp.py", line
> 494, in run
>     self.write_results(path=path, model=model)
>   File
> "/Users/tlinnet/software/disp_spin_speed/auto_analyses/relax_disp.py", line
> 541, in write_results
>     self.interpreter.relax_disp.plot_disp_curves(dir=path, force=True)
>   File "/Users/tlinnet/software/disp_spin_speed/prompt/uf_objects.py", line
> 223, in __call__
>     self._backend(*new_args, **uf_kargs)
>   File
> "/Users/tlinnet/software/disp_spin_speed/specific_analyses/relax_disp/data.py",
> line 1863, in plot_disp_curves
>     point = cpmg_frqs_new[ei][mi][oi][di]
> IndexError: list index out of range
>
> ----------------------------------------------------------------------
> Ran 1 test in 0.121s
> _______________________________________________
> relax (http://www.nmr-relax.com)
>
> This is the relax-devel mailing list
> relax-devel@xxxxxxx
>
> To unsubscribe from this list, get a password
> reminder, or change your subscription options,
> visit the list information page at
> https://mail.gna.org/listinfo/relax-devel