Author: bugman
Date: Tue Jul 22 18:56:51 2014
New Revision: 24655
URL: http://svn.gna.org/viewcvs/relax?rev=24655&view=rev
Log:
Merged revisions 23760-23770,23787-23789,23791-23806,23813-23855 via svnmerge
from
svn+ssh://bugman@xxxxxxxxxxx/svn/relax/branches/disp_spin_speed
................
r23760 | tlinnet | 2014-06-10 01:01:49 +0200 (Tue, 10 Jun 2014) | 5 lines
Moved the calculation of pA and kex out off all loopes.
This was done by having two special 1/0 spin structure arrays.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23761 | tlinnet | 2014-06-10 01:01:51 +0200 (Tue, 10 Jun 2014) | 3 lines
Removed dw_frq_a numpy array, as it was not necessary.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23762 | tlinnet | 2014-06-10 01:01:53 +0200 (Tue, 10 Jun 2014) | 6 lines
Removed all looping over spin and spectrometer frequency.
This is the last loop!
Wuhu.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23763 | tlinnet | 2014-06-10 01:01:55 +0200 (Tue, 10 Jun 2014) | 3 lines
Reordered arrays for beauty of code.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23764 | tlinnet | 2014-06-10 01:01:57 +0200 (Tue, 10 Jun 2014) | 3 lines
Made the back_calc array be initiated as copy of the values array.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23765 | tlinnet | 2014-06-10 01:01:59 +0200 (Tue, 10 Jun 2014) | 3 lines
Small edit to profiling script, to help bug finding.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23766 | tlinnet | 2014-06-10 01:02:00 +0200 (Tue, 10 Jun 2014) | 3 lines
Fixed that arrays are correctly initiated with one or zero values.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23767 | tlinnet | 2014-06-10 01:02:02 +0200 (Tue, 10 Jun 2014) | 5 lines
Very important fix, for only replacing part of data array which have Nan
values.
Before, all values were replaced, which was wrong.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23768 | tlinnet | 2014-06-10 01:02:04 +0200 (Tue, 10 Jun 2014) | 8 lines
Needed to increase the relative tolerance when testing if pA array is 1.
Now system test Relax_disp.test_hansen_cpmg_data_missing_auto_analysis
passes.
Also added some comments lines, to prepare for mask replace of values.
For example if only some of etapos values should be replaced.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23769 | tlinnet | 2014-06-10 01:02:06 +0200 (Tue, 10 Jun 2014) | 3 lines
Restored profiling script to normal.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23770 | tlinnet | 2014-06-10 01:21:14 +0200 (Tue, 10 Jun 2014) | 3 lines
Made the logic and comments much clearer about how to reshape, expand axis,
and tile numpy arrays.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23787 | tlinnet | 2014-06-10 16:51:30 +0200 (Tue, 10 Jun 2014) | 3 lines
Removed line "# -*- coding: utf-8 -*-" which was automatically inserted by
new python code editor.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23788 | tlinnet | 2014-06-10 16:51:33 +0200 (Tue, 10 Jun 2014) | 10 lines
Implemented a masked array search for where "missing" array is equal 1.
This makes it possible to replace all values with this mask, from the value
array.
This eliminates the last loops over the missing values.
It took over 4 hours to figure out, that the mask should be called with
mask.mask,
to return the same fulls structure,
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23789 | tlinnet | 2014-06-10 16:51:35 +0200 (Tue, 10 Jun 2014) | 3 lines
Yet another small improvement for the profiling script.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23791 | tlinnet | 2014-06-10 18:40:55 +0200 (Tue, 10 Jun 2014) | 5 lines
Removed the multi dimensional structure of pA.
pA is not multi-dimensional, and can just be multiplied with numpy arrays.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23792 | tlinnet | 2014-06-10 18:40:59 +0200 (Tue, 10 Jun 2014) | 3 lines
Fix for testing of pA in lib function, when pA is just float.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23793 | tlinnet | 2014-06-10 18:41:01 +0200 (Tue, 10 Jun 2014) | 3 lines
Modified unit tests, so pA is sent to target function as float instead of
array.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23794 | tlinnet | 2014-06-10 18:41:03 +0200 (Tue, 10 Jun 2014) | 5 lines
Removed the multi dimensional structure of kex.
kex is not multi-dimensional, and can just be multiplied with numpy arrays.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23795 | tlinnet | 2014-06-10 18:41:05 +0200 (Tue, 10 Jun 2014) | 3 lines
Fix for testing of kex in lib function, when kex is just float.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23796 | tlinnet | 2014-06-10 18:41:06 +0200 (Tue, 10 Jun 2014) | 3 lines
Modified unit tests, so kex is sent to target function as float instead of
array.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23797 | tlinnet | 2014-06-10 18:41:08 +0200 (Tue, 10 Jun 2014) | 10 lines
Important fix for replacing values if eta_pos > 700 is violated.
This fixes systemtest: Relax_disp.test_sod1wt_t25_to_cr72, which failed
after making kex to a numpy float.
The trick is to make a numpy mask which stores the position where to
replace the values.
Then replace the values just before last return.
This makes sure, that not all values are changed.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23798 | tlinnet | 2014-06-10 18:41:10 +0200 (Tue, 10 Jun 2014) | 5 lines
Increased the kex speed to 1e7 in clustered unit tests cases.
This is to demonstrate where there will be no excange.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23799 | tlinnet | 2014-06-10 18:54:28 +0200 (Tue, 10 Jun 2014) | 3 lines
Added a multi-dimensional numpy array chi2 value calculation function.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23800 | tlinnet | 2014-06-10 18:54:31 +0200 (Tue, 10 Jun 2014) | 3 lines
Called the newly created chi2 function to calculate for multi dimensional
numpy arrays.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23801 | tlinnet | 2014-06-10 19:11:07 +0200 (Tue, 10 Jun 2014) | 5 lines
Renamed chi2_ND to chi2_rankN.
This is a better name for representing multiple axis calculation.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23802 | tlinnet | 2014-06-10 20:58:15 +0200 (Tue, 10 Jun 2014) | 5 lines
Made special ei, si, mi, and oi numpy structure array.
This is for rapid speed-up of numpy array creation in target function.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23803 | tlinnet | 2014-06-10 20:58:19 +0200 (Tue, 10 Jun 2014) | 3 lines
Code clean-up in lib function.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23804 | tlinnet | 2014-06-10 20:58:22 +0200 (Tue, 10 Jun 2014) | 3 lines
Replace np.tile with tile.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23805 | tlinnet | 2014-06-10 20:58:24 +0200 (Tue, 10 Jun 2014) | 3 lines
Replaced self.spins_a with self.disp_struct.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23806 | tlinnet | 2014-06-10 21:06:12 +0200 (Tue, 10 Jun 2014) | 3 lines
Removed un-used self structures in __init__of class.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23813 | tlinnet | 2014-06-11 08:28:25 +0200 (Wed, 11 Jun 2014) | 3 lines
Made iinitialisation structures for dw.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23814 | tlinnet | 2014-06-11 08:28:27 +0200 (Wed, 11 Jun 2014) | 3 lines
Initial try to reshape dw faster.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23815 | tlinnet | 2014-06-11 08:40:38 +0200 (Wed, 11 Jun 2014) | 5 lines
Switched to use self.ei, self.si, self.mi, self.oi, self.di.
This is for better reading of code.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23816 | tlinnet | 2014-06-11 08:40:40 +0200 (Wed, 11 Jun 2014) | 3 lines
Comment out the sys.exit(), which would make the code fail for wrong
calculation of dw.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23817 | tlinnet | 2014-06-11 08:46:00 +0200 (Wed, 11 Jun 2014) | 7 lines
Copied profiling script for CPMG model CR72 to R1rho DPL94 model.
The framework of the script will be the same, but the data a little
different.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23818 | tlinnet | 2014-06-11 10:08:02 +0200 (Wed, 11 Jun 2014) | 3 lines
Started converting profiling script to DPL94.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23819 | tlinnet | 2014-06-11 10:08:05 +0200 (Wed, 11 Jun 2014) | 5 lines
Replaced self.(ei,si,mi,oi,di) with self.(NE,NS,NM,NO,ND).
These numbers represents the maximum number of dimensions, instead of index.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23820 | tlinnet | 2014-06-11 10:08:06 +0200 (Wed, 11 Jun 2014) | 3 lines
Added the ei index, when creating the first dw_mask.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23821 | tlinnet | 2014-06-11 10:08:08 +0200 (Wed, 11 Jun 2014) | 3 lines
Reordered how the structures dw init structures are created.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23822 | tlinnet | 2014-06-11 10:08:10 +0200 (Wed, 11 Jun 2014) | 3 lines
Clearing the dw_struct before calculation.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23823 | tlinnet | 2014-06-11 10:16:27 +0200 (Wed, 11 Jun 2014) | 8 lines
Started using the new way of constructing dw.
This is for running system tests.
Note, somewhere in the dw array, the frequencies will be different between
the two implementations.
But apparently, this does not matter.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23824 | tlinnet | 2014-06-11 10:46:42 +0200 (Wed, 11 Jun 2014) | 3 lines
Inserted temporary method to switch for profiling.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23825 | tlinnet | 2014-06-11 10:46:44 +0200 (Wed, 11 Jun 2014) | 3 lines
First try to speed-up the old dw structure calculation.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23826 | tlinnet | 2014-06-11 13:12:12 +0200 (Wed, 11 Jun 2014) | 3 lines
Simplified calculation.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23827 | tlinnet | 2014-06-11 13:12:14 +0200 (Wed, 11 Jun 2014) | 3 lines
Yet another try to implement a fast dw structure method.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23828 | tlinnet | 2014-06-11 13:12:16 +0200 (Wed, 11 Jun 2014) | 6 lines
Implemented the fastest way to calculate the dw structure.
This uses the numpy ufunc multiply.outer function to create the outer
array, and then multiply
with the frqs_structure.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23829 | tlinnet | 2014-06-11 13:12:18 +0200 (Wed, 11 Jun 2014) | 3 lines
Renamed dw temporary structure to generic structure.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23830 | tlinnet | 2014-06-11 13:12:20 +0200 (Wed, 11 Jun 2014) | 3 lines
Restructured the calculation of R20A and R20B to the most efficient way.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23831 | tlinnet | 2014-06-11 13:12:21 +0200 (Wed, 11 Jun 2014) | 3 lines
Made the lib/cr72.py to a numpy multi dimensional numpy array calculation.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23832 | tlinnet | 2014-06-11 14:05:35 +0200 (Wed, 11 Jun 2014) | 5 lines
Changed the catching when dw is zero, to use masked array.
Implemented backwards compability with unit tests.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23833 | tlinnet | 2014-06-11 14:05:37 +0200 (Wed, 11 Jun 2014) | 5 lines
Bugfix for testing if kex is zero.
It was tested if kex was equal 1.0.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23834 | tlinnet | 2014-06-11 14:54:37 +0200 (Wed, 11 Jun 2014) | 3 lines
Implemented masked replacement if fact is less that 1.0.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23835 | tlinnet | 2014-06-11 14:55:31 +0200 (Wed, 11 Jun 2014) | 1 line
Replaced isnan mask with function that catches all invalid values.
................
r23836 | tlinnet | 2014-06-11 14:55:32 +0200 (Wed, 11 Jun 2014) | 3 lines
Clean up of comment lines.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23837 | tlinnet | 2014-06-11 14:55:34 +0200 (Wed, 11 Jun 2014) | 8 lines
Removed the masked replacement if fact is less than 1.0.
This is very strange, but otherwise system test:
Relax_disp.test_hansen_cpmg_data_missing_auto_analysis
would fail.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23838 | tlinnet | 2014-06-11 15:09:01 +0200 (Wed, 11 Jun 2014) | 5 lines
Removed the slow allclose() function to test if R20A and R20B is equal.
It is MUCH faster to just subtract and check sum is not 0.0.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23839 | tlinnet | 2014-06-11 15:25:32 +0200 (Wed, 11 Jun 2014) | 3 lines
Replaced the temporary variable R2eff with back_calc, and used numpy
subtract to speed up.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23840 | tlinnet | 2014-06-11 15:25:34 +0200 (Wed, 11 Jun 2014) | 5 lines
Made the lib function into a pure numpy array calculation.
This requires, that r20a, r20b and dw has same dimension as the dispersion
points.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23841 | tlinnet | 2014-06-11 15:25:36 +0200 (Wed, 11 Jun 2014) | 3 lines
Changes too unit tests, so data is sent to target function in numpy array
format.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23842 | tlinnet | 2014-06-11 16:19:10 +0200 (Wed, 11 Jun 2014) | 3 lines
Removed an the creation of a unnessary structure bu using numpy multiply.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23843 | tlinnet | 2014-06-11 16:19:12 +0200 (Wed, 11 Jun 2014) | 3 lines
Moved the mask which finds where to replace values into the __init__
function.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23844 | tlinnet | 2014-06-11 16:43:17 +0200 (Wed, 11 Jun 2014) | 7 lines
Replaced the multiplcation with the disp structure, to a replacement with a
mask.
My timings are varying at my computer, so results are within the very same
timing.
But for reading the code, having two masks, is essential better.
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
................
r23845 | tlinnet | 2014-06-11 17:09:23 +0200 (Wed, 11 Jun 2014) | 14 lines
Merged revisions 23743 via svnmerge from
svn+ssh://tlinnet@xxxxxxxxxxx/svn/relax/trunk
........
r23743 | tlinnet | 2014-06-08 22:42:55 +0200 (Sun, 08 Jun 2014) | 6 lines
Updated profiling text for CR72 model.
Now it is tested for 3 fields.
This is related to:
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion
models for Clustered analysis.
........
................
r23846 | tlinnet | 2014-06-11 17:14:09 +0200 (Wed, 11 Jun 2014) | 25 lines
Reverted r23845-r23844 as this was using a wrong commit message.
The command used was:
svn merge -r23845:23844 .
........
r23845 | tlinnet | 2014-06-11 17:09:23 +0200 (Wed, 11 Jun 2014) | 14
lines
Merged revisions 23743 via svnmerge from
[... 121 lines stripped ...]
r24655 - in /trunk: ./ lib/dispersion/ target_functions/ test_suite/shared_data/dispersion/profiling/ test_suite/unit_tests/...