Can one profile the systemtests ? Best Troels
2014-06-13 10:56 GMT+02:00 Edward d'Auvergne <[email protected]>: > For me the system tests are not much faster compared to trunk. Now at > r23912, the disp_spin_speed branch takes ~216 seconds for all > Relax_disp system tests. But for the trunk it is ~233 seconds. From > your profiling script it's pretty clear though that there are insane > speed ups happening, but it's not reflected so much in the system > tests. It might be interesting to see what is causing these system > tests to take so long - it's clear that it's not the target functions. > > Regards, > > Edward > > > On 13 June 2014 10:46, Troels Emtekær Linnet <[email protected]> > wrote: > > Yeahaa! > > > > I realised, that if I wanted to have the systemtests go faster, > > I should kill this one. > > > > It is nice to see, that systemtest suite just runs faster and faster, the > > more models > > i kill. > > > > > > > > 2014-06-13 10:37 GMT+02:00 Edward d'Auvergne <[email protected]>: > >> > >> Now this code might be too fast for the user - they may not be fast > >> enough to even see it in the log files ;) > >> > >> Regards, > >> > >> Edward > >> > >> > >> On 13 June 2014 10:10, <[email protected]> wrote: > >> > Author: tlinnet > >> > Date: Fri Jun 13 10:10:04 2014 > >> > New Revision: 23908 > >> > > >> > URL: http://svn.gna.org/viewcvs/relax?rev=23908&view=rev > >> > Log: > >> > Made the NOREX model af faster numpy array calculation. > >> > > >> > Task #7807 (https://gna.org/task/index.php?7807): Speed-up of > dispersion > >> > models for Clustered analysis. > >> > > >> > Modified: > >> > branches/disp_spin_speed/target_functions/relax_disp.py > >> > > >> > Modified: branches/disp_spin_speed/target_functions/relax_disp.py > >> > URL: > >> > > http://svn.gna.org/viewcvs/relax/branches/disp_spin_speed/target_functions/relax_disp.py?rev=23908&r1=23907&r2=23908&view=diff > >> > > >> > > ============================================================================== > >> > --- branches/disp_spin_speed/target_functions/relax_disp.py > >> > (original) > >> > +++ branches/disp_spin_speed/target_functions/relax_disp.py Fri > Jun > >> > 13 10:10:04 2014 > >> > @@ -394,8 +394,11 @@ > >> > if model == MODEL_NS_MMQ_3SITE_LINEAR: > >> > self.func = self.func_ns_mmq_3site_linear > >> > > >> > + > >> > # Setup special numpy array structures, for higher > dimensional > >> > computation. > >> > - if model in [MODEL_B14, MODEL_B14_FULL, MODEL_CR72, > >> > MODEL_CR72_FULL, MODEL_DPL94, MODEL_TAP03, MODEL_TP02, MODEL_TSMFK01]: > >> > + test_models = [MODEL_B14, MODEL_B14_FULL, MODEL_CR72, > >> > MODEL_CR72_FULL, MODEL_DPL94, MODEL_TAP03, MODEL_TP02, MODEL_TSMFK01] > >> > + > >> > + if model in test_models + [MODEL_NOREX]: > >> > # Get the shape of back_calc structure. > >> > # If using just one field, or having the same number of > >> > dispersion points, the shape would extend to that number. > >> > # Shape has to be: [ei][si][mi][oi]. > >> > @@ -443,8 +446,11 @@ > >> > self.r20b_struct = deepcopy(zeros_a) > >> > self.dw_struct = deepcopy(zeros_a) > >> > > >> > - # Extract the frequencies to numpy array. > >> > - self.frqs_a = multiply.outer( > >> > asarray(self.frqs).reshape(self.NE, self.NS, self.NM), > self.no_nd_struct ) > >> > + # Setup special numpy array structures, for higher > dimensional > >> > computation. > >> > + if model in test_models + [MODEL_NOREX]: > >> > + if model in test_models: > >> > + # Extract the frequencies to numpy array. > >> > + self.frqs_a = multiply.outer( > >> > asarray(self.frqs).reshape(self.NE, self.NS, self.NM), > self.no_nd_struct ) > >> > > >> > if model in MODEL_LIST_CPMG_FULL: > >> > self.cpmg_frqs_a = deepcopy(ones_a) > >> > @@ -476,7 +482,7 @@ > >> > # Extract number of dispersion points. > >> > num_disp_points = > >> > self.num_disp_points[ei][si][mi][oi] > >> > > >> > - if model in MODEL_LIST_CPMG_FULL: > >> > + if model in MODEL_LIST_CPMG_FULL and > model > >> > != MODEL_NOREX: > >> > # Extract cpmg_frqs and > num_disp_points > >> > from lists. > >> > > >> > self.cpmg_frqs_a[ei][si][mi][oi][:num_disp_points] = > >> > self.cpmg_frqs[ei][mi][oi] > >> > > >> > self.num_disp_points_a[ei][si][mi][oi][:num_disp_points] = > >> > self.num_disp_points[ei][si][mi][oi] > >> > @@ -497,7 +503,7 @@ > >> > self.power_a[ei][si][mi][oi][di] > = > >> > int(round(self.cpmg_frqs[ei][mi][0][di] * self.relax_times[ei][mi])) > >> > > self.tau_cpmg_a[ei][si][mi][oi][di] > >> > = 0.25 / self.cpmg_frqs[ei][mi][0][di] > >> > # For R1rho data. > >> > - if model in MODEL_LIST_R1RHO_FULL: > >> > + if model in MODEL_LIST_R1RHO_FULL and > >> > model != MODEL_NOREX: > >> > > >> > self.disp_struct[ei][si][mi][oi][di] = 1.0 > >> > > >> > # Extract the frequencies to > numpy > >> > array. > >> > @@ -511,11 +517,12 @@ > >> > else: > >> > > >> > self.num_disp_points_a[ei][si][mi][oi][di] = 0 > >> > > >> > + if model in test_models: > >> > + # Pre calculate frqs structure > >> > + self.frqs_struct = self.frqs_a * self.disp_struct > >> > + > >> > # Make copy of values structure. > >> > self.back_calc_a = deepcopy(self.values_a) > >> > - > >> > - # Pre calculate frqs structure > >> > - self.frqs_struct = self.frqs_a * self.disp_struct > >> > > >> > # Find the numpy mask, which tells where values should be > >> > replaced. > >> > self.mask_replace_blank = masked_equal(missing_a, 1.0) > >> > @@ -1465,34 +1472,16 @@ > >> > # Unpack the parameter values. > >> > R20 = params > >> > > >> > - # Initialise. > >> > - chi2_sum = 0.0 > >> > - > >> > - # Loop over the experiment types. > >> > - for ei in range(self.num_exp): > >> > - # Loop over the spins. > >> > - for si in range(self.num_spins): > >> > - # Loop over the spectrometer frequencies. > >> > - for mi in range(self.num_frq): > >> > - # The R20 index. > >> > - r20_index = mi + si*self.num_frq > >> > - > >> > - # Loop over the offsets. > >> > - for oi in range(self.num_offsets[ei][si][mi]): > >> > - # The R2eff values as R20 values. > >> > - for di in > >> > range(self.num_disp_points[ei][si][mi][oi]): > >> > - self.back_calc[ei][si][mi][oi][di] = > >> > R20[r20_index] > >> > - > >> > - # For all missing data points, set the > >> > back-calculated value to the measured values so that it has no effect > on the > >> > chi-squared value. > >> > - for di in > >> > range(self.num_disp_points[ei][si][mi][oi]): > >> > - if self.missing[ei][si][mi][oi][di]: > >> > - self.back_calc[ei][si][mi][oi][di] = > >> > self.values[ei][si][mi][oi][di] > >> > - > >> > - # Calculate and return the chi-squared value. > >> > - chi2_sum += chi2(self.values[ei][si][mi][oi], > >> > self.back_calc[ei][si][mi][oi], self.errors[ei][si][mi][oi]) > >> > + # Reshape R20 to per experiment, spin and frequency. > >> > + self.back_calc_a[:] = multiply.outer( R20.reshape(self.NE, > >> > self.NS, self.NM), self.no_nd_struct ) > >> > + > >> > + ## For all missing data points, set the back-calculated value > >> > to the measured values so that it has no effect on the chi-squared > value. > >> > + if self.has_missing: > >> > + # Replace with values. > >> > + self.back_calc_a[self.mask_replace_blank.mask] = > >> > self.values_a[self.mask_replace_blank.mask] > >> > > >> > # Return the total chi-squared value. > >> > - return chi2_sum > >> > + return chi2_rankN(self.values_a, self.back_calc_a, > >> > self.errors_a) > >> > > >> > > >> > def func_ns_cpmg_2site_3D(self, params): > >> > > >> > > >> > _______________________________________________ > >> > relax (http://www.nmr-relax.com) > >> > > >> > This is the relax-commits mailing list > >> > [email protected] > >> > > >> > To unsubscribe from this list, get a password > >> > reminder, or change your subscription options, > >> > visit the list information page at > >> > https://mail.gna.org/listinfo/relax-commits > >> > >> _______________________________________________ > >> relax (http://www.nmr-relax.com) > >> > >> This is the relax-devel mailing list > >> [email protected] > >> > >> > >> To unsubscribe from this list, get a password > >> reminder, or change your subscription options, > >> visit the list information page at > >> https://mail.gna.org/listinfo/relax-devel > > > > > _______________________________________________ relax (http://www.nmr-relax.com) This is the relax-devel mailing list [email protected] To unsubscribe from this list, get a password reminder, or change your subscription options, visit the list information page at https://mail.gna.org/listinfo/relax-devel
