Author: bugman
Date: Thu Oct 1 20:08:08 2015
New Revision: 27918
URL: http://svn.gna.org/viewcvs/relax?rev=27918&view=rev
Log:
Merged revisions
23241-23245,23251-23253,23256-23258,23280-23283,23309,23321-23322 via
svnmerge from
svn+ssh://bugman@xxxxxxxxxxx/svn/relax/branches/frame_order_cleanup
........
r23241 | bugman | 2014-05-19 15:10:15 +0200 (Mon, 19 May 2014) | 5 lines
Increased the precision of all the data in the CaM frame order test data
generation base script.
These have all been converted from float16 to float64 numpy types.
........
r23242 | bugman | 2014-05-19 16:13:24 +0200 (Mon, 19 May 2014) | 5 lines
Fix for the RDC error setting in the CaM frame order test data generation
base script.
The rdc_err data structure is located in the interatomic data containers,
no the spin containers.
........
r23243 | bugman | 2014-05-19 16:54:19 +0200 (Mon, 19 May 2014) | 4 lines
Modification of the structure loading part of the CaM frame order data
generation base script.
The structures are now only loaded if the DIST_PDB flag is set, as they are
only used for generating
the 3D distribution of structures. This saves a lot of time and computer
memory.
........
r23244 | bugman | 2014-05-19 18:11:12 +0200 (Mon, 19 May 2014) | 9 lines
Huge speed up of the CaM frame order test data generation base script.
By using multidimensional numpy arrays to store the atomic positions and XH
unit vectors of all
spins, and performing the rotations on these structures using
numpy.tensordot(), the calculations
are now a factor of 10 times faster. The progress meter had to be changed
to show every 1000 rather
than 100 iterations.
The rotations of the positions and vectors are now performed sequentially,
accidentally fixing a bug
with the double motion models (i.e. the 'double rotor' model).
........
r23245 | bugman | 2014-05-19 20:10:33 +0200 (Mon, 19 May 2014) | 5 lines
Modified the CaM frame order test data generation base script to conserve
computer RAM.
The XH vector and atomic position data structures for all N rotations are
now of the numpy.float32
rather than numpy.float64 type. The main change is to calculate the
averaged RDCs and averaged PCSs
separately, deleting the N-sized data structures once the data files are
written.
........
r23251 | bugman | 2014-05-20 11:09:21 +0200 (Tue, 20 May 2014) | 21 lines
Complete redesign of the CaM frame order data generation base script for
speed and memory savings.
Although the rotated XH bond vector and atomic position code was very fast,
the amount of memory
needed to store these in the spin containers and interatomic data
containers was huge when N > 1e6.
The subsequent rdc.back_calc and pcs.back_calc user function calls would
also take far too long.
Therefore the base script has been redesigned. The _create_distribution()
method has been split
into four: _calculate_pcs(), _calculate_rdc(), _create_distribution(), and
_pipe_setup().
The _pipe_setup() method is called first to set up the data pipe with all
required data. Then the
_calculate_rdc() and _calculate_pcs() methods, and finally
_create_distribution() if the DIST_PDB
flag is set.
The calls to the rdc.back_calc and pcs.back_calc user functions have been
eliminated. Instead the
_calculate_rdc() and _calculate_pcs() methods calculate the averaged RDC
and PCS themselves as numpy
array structures. Rather than storing the huge rotated vectors and atomic
positions data
structures, the RDCs and PCSs are summed. These are then divided by self.N
at the end to average
the values.
Compared to the old code, when N is set to 20 million the RAM usage drops
from ~20 GB to ~65 MB.
The total run time is also decreased on one system from a few days to a few
hours (an order or two
of magnitude).
........
r23252 | bugman | 2014-05-20 11:48:11 +0200 (Tue, 20 May 2014) | 7 lines
Large increase in accuracy of the RDC and PCS averaging.
This is for the CaM frame order test data generation base script. By
storing the RDCs and PCSs in
numpy.float128 arrays, a 64-bit system is required, and then using
numpy.average(), the average
value can be calculated with a much higher accuracy. As N becomes larger,
the averaging introduces
greater and greater amounts of truncation artifacts. So this change
alleviates this. However a lot
more RAM is now required.
........
r23253 | bugman | 2014-05-20 11:48:13 +0200 (Tue, 20 May 2014) | 4 lines
Changed the progress meter updating for the CaM frame order test data
generation base script.
The spinner was far too fast, updating every 5 increments, and is now
updated every 250. And the
total number is now only printed every 10,000 increments.
........
r23256 | bugman | 2014-05-20 13:47:33 +0200 (Tue, 20 May 2014) | 20 lines
Reverted r23252 as this uses far, far too much RAM, and there is a better
solution.
The command used was:
svn merge -r23252:r23251 .
.....
r23252 | bugman | 2014-05-20 11:48:11 +0200 (Tue, 20 May 2014) | 7 lines
Changed paths:
M
/branches/frame_order_cleanup/test_suite/shared_data/frame_order/cam/generate_base.py
Large increase in accuracy of the RDC and PCS averaging.
This is for the CaM frame order test data generation base script. By
storing the RDCs and PCSs in
numpy.float128 arrays, a 64-bit system is required, and then using
numpy.average(), the average
value can be calculated with a much higher accuracy. As N becomes
larger, the averaging introduces
greater and greater amounts of truncation artifacts. So this change
alleviates this. However a lot
more RAM is now required.
.....
........
r23257 | bugman | 2014-05-20 13:47:59 +0200 (Tue, 20 May 2014) | 3 lines
Improvements to the progress meter for the CaM frame order test data
generation base script.
Commas are now printed between the thousands and the numbers are now right
justified.
........
r23258 | bugman | 2014-05-20 13:57:31 +0200 (Tue, 20 May 2014) | 9 lines
Large increase in accuracy of the RDC and PCS averaging.
This is for the CaM frame order test data generation base script. By
summing the RDCs and PCSs into
1D numpy.float128 arrays (for this, a 64-bit system is required), and then
dividing by N at the end,
the average value can be calculated with a much higher accuracy. As N
becomes larger, the
numerical averaging introduces greater and greater amounts of truncation
artifacts. So this change
alleviates this.
This change is what r23252 should have been.
........
r23280 | bugman | 2014-05-21 09:24:28 +0200 (Wed, 21 May 2014) | 5 lines
Fix for the RDC and PCS averaging in the CaM frame order test data
generation base script.
For the double rotor model, or any multiple motional mode model, the
averaging was incorrect.
Instead of dividing by N, the values should be divided by N**M, where M is
the number of motional
modes.
........
r23281 | bugman | 2014-05-21 09:24:31 +0200 (Wed, 21 May 2014) | 9 lines
Huge increase in precision for the CaM frame order free rotor model test
data.
The higher precision is because the number structures in the distribution
is now twenty million
rather than one million, and the much higher precision numpy.float128
averaging of the updated data
generation base script has been used.
This data should allow for a much better estimate of the beta and gamma
average domain position
parameter values for the free rotor models which are affected by the
collapse of the alpha parameter
to zero.
........
r23282 | bugman | 2014-05-21 09:24:34 +0200 (Wed, 21 May 2014) | 5 lines
Huge increase in precision for the CaM frame order double rotor model test
data.
The higher precision is because the number structures in the distribution
is now over twenty million
(4500**2) rather than a quarter of a million (500**2). And the much higher
precision numpy.float128
averaging of the updated data generation base script has been used.
........
r23283 | bugman | 2014-05-21 09:59:51 +0200 (Wed, 21 May 2014) | 3 lines
Fix for the constraint deactivation in the frame order minimisation when no
constraints are present.
This problem was introduced at r23211.
........
r23309 | bugman | 2014-05-21 17:06:17 +0200 (Wed, 21 May 2014) | 4 lines
Deletion of the frame_order.quad_int user function.
This no longer has a purpose, as the incredibly slow quadratic integration
code has been removed
from relax.
........
r23321 | bugman | 2014-05-22 08:58:54 +0200 (Thu, 22 May 2014) | 4 lines
Huge increase in precision for the CaM frame order rotor model test data.
The higher precision is because the number structures in the distribution
is now 20 million rather
than 166,666, and the numpy.float128 data averaging has been used.
........
r23322 | bugman | 2014-05-22 08:59:00 +0200 (Thu, 22 May 2014) | 4 lines
Large increase in precision for the 2nd CaM frame order rotor model test
data set.
The higher precision is because the number structures in the distribution
is now 20 million rather
than 1,000,001 and the numpy.float128 data averaging has been used.
........
Added:
trunk/test_suite/shared_data/frame_order/cam/free_rotor/axis.pdb
- copied unchanged from r23283,
branches/frame_order_cleanup/test_suite/shared_data/frame_order/cam/free_rotor/axis.pdb
trunk/test_suite/shared_data/frame_order/cam/free_rotor/generate_distribution.bz2
- copied unchanged from r23283,
branches/frame_order_cleanup/test_suite/shared_data/frame_order/cam/free_rotor/generate_distribution.bz2
trunk/test_suite/shared_data/frame_order/cam/rotor/generate_distribution.bz2
- copied unchanged from r23322,
branches/frame_order_cleanup/test_suite/shared_data/frame_order/cam/rotor/generate_distribution.bz2
trunk/test_suite/shared_data/frame_order/cam/rotor2/generate_distribution.bz2
- copied unchanged from r23322,
branches/frame_order_cleanup/test_suite/shared_data/frame_order/cam/rotor2/generate_distribution.bz2
trunk/test_suite/shared_data/frame_order/cam/rotor2/rotations.bz2
- copied unchanged from r23322,
branches/frame_order_cleanup/test_suite/shared_data/frame_order/cam/rotor2/rotations.bz2
Modified:
trunk/ (props changed)
trunk/auto_analyses/frame_order.py
trunk/specific_analyses/frame_order/api.py
trunk/specific_analyses/frame_order/optimisation.py
trunk/specific_analyses/frame_order/uf.py
trunk/target_functions/frame_order.py
trunk/test_suite/shared_data/frame_order/cam/double_rotor/frame_order_matrix
trunk/test_suite/shared_data/frame_order/cam/double_rotor/generate_distribution.log.bz2
trunk/test_suite/shared_data/frame_order/cam/double_rotor/generate_distribution.py
trunk/test_suite/shared_data/frame_order/cam/double_rotor/pcs_dy.txt
trunk/test_suite/shared_data/frame_order/cam/double_rotor/pcs_dy_subset.txt
trunk/test_suite/shared_data/frame_order/cam/double_rotor/pcs_er.txt
trunk/test_suite/shared_data/frame_order/cam/double_rotor/pcs_er_subset.txt
trunk/test_suite/shared_data/frame_order/cam/double_rotor/pcs_tb.txt
trunk/test_suite/shared_data/frame_order/cam/double_rotor/pcs_tb_subset.txt
trunk/test_suite/shared_data/frame_order/cam/double_rotor/pcs_tm.txt
trunk/test_suite/shared_data/frame_order/cam/double_rotor/pcs_tm_subset.txt
trunk/test_suite/shared_data/frame_order/cam/double_rotor/rdc_dy.txt
trunk/test_suite/shared_data/frame_order/cam/double_rotor/rdc_er.txt
trunk/test_suite/shared_data/frame_order/cam/double_rotor/rdc_tb.txt
trunk/test_suite/shared_data/frame_order/cam/double_rotor/rdc_tm.txt
trunk/test_suite/shared_data/frame_order/cam/free_rotor/frame_order.py
trunk/test_suite/shared_data/frame_order/cam/free_rotor/frame_order_matrix
trunk/test_suite/shared_data/frame_order/cam/free_rotor/generate_distribution.log.bz2
trunk/test_suite/shared_data/frame_order/cam/free_rotor/generate_distribution.py
trunk/test_suite/shared_data/frame_order/cam/free_rotor/pcs_dy.txt
trunk/test_suite/shared_data/frame_order/cam/free_rotor/pcs_dy_subset.txt
trunk/test_suite/shared_data/frame_order/cam/free_rotor/pcs_er.txt
trunk/test_suite/shared_data/frame_order/cam/free_rotor/pcs_er_subset.txt
trunk/test_suite/shared_data/frame_order/cam/free_rotor/pcs_tb.txt
trunk/test_suite/shared_data/frame_order/cam/free_rotor/pcs_tb_subset.txt
trunk/test_suite/shared_data/frame_order/cam/free_rotor/pcs_tm.txt
trunk/test_suite/shared_data/frame_order/cam/free_rotor/pcs_tm_subset.txt
trunk/test_suite/shared_data/frame_order/cam/free_rotor/rdc_dy.txt
trunk/test_suite/shared_data/frame_order/cam/free_rotor/rdc_er.txt
trunk/test_suite/shared_data/frame_order/cam/free_rotor/rdc_tb.txt
trunk/test_suite/shared_data/frame_order/cam/free_rotor/rdc_tm.txt
trunk/test_suite/shared_data/frame_order/cam/free_rotor2/frame_order.py
trunk/test_suite/shared_data/frame_order/cam/free_rotor_missing_data/frame_order.py
trunk/test_suite/shared_data/frame_order/cam/generate_base.py
trunk/test_suite/shared_data/frame_order/cam/iso_cone/frame_order.py
trunk/test_suite/shared_data/frame_order/cam/iso_cone_free_rotor/frame_order.py
trunk/test_suite/shared_data/frame_order/cam/iso_cone_free_rotor2/frame_order.py
trunk/test_suite/shared_data/frame_order/cam/iso_cone_torsionless/frame_order.py
trunk/test_suite/shared_data/frame_order/cam/pseudo_ellipse/frame_order.py
trunk/test_suite/shared_data/frame_order/cam/pseudo_ellipse/multi_processor_test.py
trunk/test_suite/shared_data/frame_order/cam/pseudo_ellipse2/frame_order.py
trunk/test_suite/shared_data/frame_order/cam/pseudo_ellipse_free_rotor/frame_order.py
trunk/test_suite/shared_data/frame_order/cam/pseudo_ellipse_torsionless/frame_order.py
trunk/test_suite/shared_data/frame_order/cam/rigid/frame_order.py
trunk/test_suite/shared_data/frame_order/cam/rotor/axis.pdb
trunk/test_suite/shared_data/frame_order/cam/rotor/frame_order.py
trunk/test_suite/shared_data/frame_order/cam/rotor/frame_order_matrix
trunk/test_suite/shared_data/frame_order/cam/rotor/generate_distribution.log.bz2
trunk/test_suite/shared_data/frame_order/cam/rotor/generate_distribution.py
trunk/test_suite/shared_data/frame_order/cam/rotor/pcs_dy.txt
trunk/test_suite/shared_data/frame_order/cam/rotor/pcs_dy_subset.txt
trunk/test_suite/shared_data/frame_order/cam/rotor/pcs_er.txt
trunk/test_suite/shared_data/frame_order/cam/rotor/pcs_er_subset.txt
trunk/test_suite/shared_data/frame_order/cam/rotor/pcs_tb.txt
trunk/test_suite/shared_data/frame_order/cam/rotor/pcs_tb_subset.txt
trunk/test_suite/shared_data/frame_order/cam/rotor/pcs_tm.txt
trunk/test_suite/shared_data/frame_order/cam/rotor/pcs_tm_subset.txt
trunk/test_suite/shared_data/frame_order/cam/rotor/rdc_dy.txt
trunk/test_suite/shared_data/frame_order/cam/rotor/rdc_er.txt
trunk/test_suite/shared_data/frame_order/cam/rotor/rdc_tb.txt
trunk/test_suite/shared_data/frame_order/cam/rotor/rdc_tm.txt
trunk/test_suite/shared_data/frame_order/cam/rotor2/axis.pdb
trunk/test_suite/shared_data/frame_order/cam/rotor2/frame_order.py
trunk/test_suite/shared_data/frame_order/cam/rotor2/frame_order_matrix
trunk/test_suite/shared_data/frame_order/cam/rotor2/generate_distribution.log.bz2
trunk/test_suite/shared_data/frame_order/cam/rotor2/generate_distribution.py
trunk/test_suite/shared_data/frame_order/cam/rotor2/pcs_dy.txt
trunk/test_suite/shared_data/frame_order/cam/rotor2/pcs_dy_subset.txt
trunk/test_suite/shared_data/frame_order/cam/rotor2/pcs_er.txt
trunk/test_suite/shared_data/frame_order/cam/rotor2/pcs_er_subset.txt
trunk/test_suite/shared_data/frame_order/cam/rotor2/pcs_only/frame_order.py
trunk/test_suite/shared_data/frame_order/cam/rotor2/pcs_tb.txt
trunk/test_suite/shared_data/frame_order/cam/rotor2/pcs_tb_subset.txt
trunk/test_suite/shared_data/frame_order/cam/rotor2/pcs_tm.txt
trunk/test_suite/shared_data/frame_order/cam/rotor2/pcs_tm_subset.txt
trunk/test_suite/shared_data/frame_order/cam/rotor2/rdc_dy.txt
trunk/test_suite/shared_data/frame_order/cam/rotor2/rdc_er.txt
trunk/test_suite/shared_data/frame_order/cam/rotor2/rdc_tb.txt
trunk/test_suite/shared_data/frame_order/cam/rotor2/rdc_tm.txt
trunk/test_suite/shared_data/frame_order/cam/rotor_2_state/frame_order.py
trunk/user_functions/frame_order.py
[This mail would be too long, it was shortened to contain the URLs only.]
Modified: trunk/auto_analyses/frame_order.py
URL:
http://svn.gna.org/viewcvs/relax/trunk/auto_analyses/frame_order.py?rev=27918&r1=27917&r2=27918&view=diff
Modified: trunk/specific_analyses/frame_order/api.py
URL:
http://svn.gna.org/viewcvs/relax/trunk/specific_analyses/frame_order/api.py?rev=27918&r1=27917&r2=27918&view=diff
Modified: trunk/specific_analyses/frame_order/optimisation.py
URL:
http://svn.gna.org/viewcvs/relax/trunk/specific_analyses/frame_order/optimisation.py?rev=27918&r1=27917&r2=27918&view=diff
Modified: trunk/specific_analyses/frame_order/uf.py
URL:
http://svn.gna.org/viewcvs/relax/trunk/specific_analyses/frame_order/uf.py?rev=27918&r1=27917&r2=27918&view=diff
Modified: trunk/target_functions/frame_order.py
URL:
http://svn.gna.org/viewcvs/relax/trunk/target_functions/frame_order.py?rev=27918&r1=27917&r2=27918&view=diff
Modified:
trunk/test_suite/shared_data/frame_order/cam/double_rotor/frame_order_matrix
URL:
http://svn.gna.org/viewcvs/relax/trunk/test_suite/shared_data/frame_order/cam/double_rotor/frame_order_matrix?rev=27918&r1=27917&r2=27918&view=diff
Modified:
trunk/test_suite/shared_data/frame_order/cam/double_rotor/generate_distribution.log.bz2
URL:
http://svn.gna.org/viewcvs/relax/trunk/test_suite/shared_data/frame_order/cam/double_rotor/generate_distribution.log.bz2?rev=27918&r1=27917&r2=27918&view=diff
Modified:
trunk/test_suite/shared_data/frame_order/cam/double_rotor/generate_distribution.py
URL:
http://svn.gna.org/viewcvs/relax/trunk/test_suite/shared_data/frame_order/cam/double_rotor/generate_distribution.py?rev=27918&r1=27917&r2=27918&view=diff
Modified: trunk/test_suite/shared_data/frame_order/cam/double_rotor/pcs_dy.txt
URL:
http://svn.gna.org/viewcvs/relax/trunk/test_suite/shared_data/frame_order/cam/double_rotor/pcs_dy.txt?rev=27918&r1=27917&r2=27918&view=diff
Modified:
trunk/test_suite/shared_data/frame_order/cam/double_rotor/pcs_dy_subset.txt
URL:
http://svn.gna.org/viewcvs/relax/trunk/test_suite/shared_data/frame_order/cam/double_rotor/pcs_dy_subset.txt?rev=27918&r1=27917&r2=27918&view=diff
Modified: trunk/test_suite/shared_data/frame_order/cam/double_rotor/pcs_er.txt
URL:
http://svn.gna.org/viewcvs/relax/trunk/test_suite/shared_data/frame_order/cam/double_rotor/pcs_er.txt?rev=27918&r1=27917&r2=27918&view=diff
Modified:
trunk/test_suite/shared_data/frame_order/cam/double_rotor/pcs_er_subset.txt
URL:
http://svn.gna.org/viewcvs/relax/trunk/test_suite/shared_data/frame_order/cam/double_rotor/pcs_er_subset.txt?rev=27918&r1=27917&r2=27918&view=diff
Modified: trunk/test_suite/shared_data/frame_order/cam/double_rotor/pcs_tb.txt
URL:
http://svn.gna.org/viewcvs/relax/trunk/test_suite/shared_data/frame_order/cam/double_rotor/pcs_tb.txt?rev=27918&r1=27917&r2=27918&view=diff
Modified:
trunk/test_suite/shared_data/frame_order/cam/double_rotor/pcs_tb_subset.txt
URL:
http://svn.gna.org/viewcvs/relax/trunk/test_suite/shared_data/frame_order/cam/double_rotor/pcs_tb_subset.txt?rev=27918&r1=27917&r2=27918&view=diff
Modified: trunk/test_suite/shared_data/frame_order/cam/double_rotor/pcs_tm.txt
URL:
http://svn.gna.org/viewcvs/relax/trunk/test_suite/shared_data/frame_order/cam/double_rotor/pcs_tm.txt?rev=27918&r1=27917&r2=27918&view=diff
[... 213 lines stripped ...]
r27918 - in /trunk: ./ auto_analyses/ specific_analyses/frame_order/ target_functions/ test_suite/shared_data/frame_order/ca...