Of course. Just set the profile flag in the 'relax' file to True. It can be difficult to work out what is happening with the system tests. But also note that it is probably not a worthwhile target speeding up the parts that slow the system tests down. These are often the user functions for setting up the analysis, and the analysis setup is only a tiny fraction of the time the user waits for. The user functions for setting up, which are by far the majority, are not optimised in any way as I've never seen a need to work on that. The only advantage would be to drop the total system test time down by a large percentage.
Regards, Edward On 13 June 2014 11:07, Troels Emtekær Linnet <[email protected]> wrote: > Can one profile the systemtests ? > > Best > Troels > > > 2014-06-13 10:56 GMT+02:00 Edward d'Auvergne <[email protected]>: > >> For me the system tests are not much faster compared to trunk. Now at >> r23912, the disp_spin_speed branch takes ~216 seconds for all >> Relax_disp system tests. But for the trunk it is ~233 seconds. From >> your profiling script it's pretty clear though that there are insane >> speed ups happening, but it's not reflected so much in the system >> tests. It might be interesting to see what is causing these system >> tests to take so long - it's clear that it's not the target functions. >> >> Regards, >> >> Edward >> >> >> On 13 June 2014 10:46, Troels Emtekær Linnet <[email protected]> >> wrote: >> > Yeahaa! >> > >> > I realised, that if I wanted to have the systemtests go faster, >> > I should kill this one. >> > >> > It is nice to see, that systemtest suite just runs faster and faster, >> > the >> > more models >> > i kill. >> > >> > >> > >> > 2014-06-13 10:37 GMT+02:00 Edward d'Auvergne <[email protected]>: >> >> >> >> Now this code might be too fast for the user - they may not be fast >> >> enough to even see it in the log files ;) >> >> >> >> Regards, >> >> >> >> Edward >> >> >> >> >> >> On 13 June 2014 10:10, <[email protected]> wrote: >> >> > Author: tlinnet >> >> > Date: Fri Jun 13 10:10:04 2014 >> >> > New Revision: 23908 >> >> > >> >> > URL: http://svn.gna.org/viewcvs/relax?rev=23908&view=rev >> >> > Log: >> >> > Made the NOREX model af faster numpy array calculation. >> >> > >> >> > Task #7807 (https://gna.org/task/index.php?7807): Speed-up of >> >> > dispersion >> >> > models for Clustered analysis. >> >> > >> >> > Modified: >> >> > branches/disp_spin_speed/target_functions/relax_disp.py >> >> > >> >> > Modified: branches/disp_spin_speed/target_functions/relax_disp.py >> >> > URL: >> >> > >> >> > http://svn.gna.org/viewcvs/relax/branches/disp_spin_speed/target_functions/relax_disp.py?rev=23908&r1=23907&r2=23908&view=diff >> >> > >> >> > >> >> > ============================================================================== >> >> > --- branches/disp_spin_speed/target_functions/relax_disp.py >> >> > (original) >> >> > +++ branches/disp_spin_speed/target_functions/relax_disp.py Fri >> >> > Jun >> >> > 13 10:10:04 2014 >> >> > @@ -394,8 +394,11 @@ >> >> > if model == MODEL_NS_MMQ_3SITE_LINEAR: >> >> > self.func = self.func_ns_mmq_3site_linear >> >> > >> >> > + >> >> > # Setup special numpy array structures, for higher >> >> > dimensional >> >> > computation. >> >> > - if model in [MODEL_B14, MODEL_B14_FULL, MODEL_CR72, >> >> > MODEL_CR72_FULL, MODEL_DPL94, MODEL_TAP03, MODEL_TP02, >> >> > MODEL_TSMFK01]: >> >> > + test_models = [MODEL_B14, MODEL_B14_FULL, MODEL_CR72, >> >> > MODEL_CR72_FULL, MODEL_DPL94, MODEL_TAP03, MODEL_TP02, MODEL_TSMFK01] >> >> > + >> >> > + if model in test_models + [MODEL_NOREX]: >> >> > # Get the shape of back_calc structure. >> >> > # If using just one field, or having the same number of >> >> > dispersion points, the shape would extend to that number. >> >> > # Shape has to be: [ei][si][mi][oi]. >> >> > @@ -443,8 +446,11 @@ >> >> > self.r20b_struct = deepcopy(zeros_a) >> >> > self.dw_struct = deepcopy(zeros_a) >> >> > >> >> > - # Extract the frequencies to numpy array. >> >> > - self.frqs_a = multiply.outer( >> >> > asarray(self.frqs).reshape(self.NE, self.NS, self.NM), >> >> > self.no_nd_struct ) >> >> > + # Setup special numpy array structures, for higher >> >> > dimensional >> >> > computation. >> >> > + if model in test_models + [MODEL_NOREX]: >> >> > + if model in test_models: >> >> > + # Extract the frequencies to numpy array. >> >> > + self.frqs_a = multiply.outer( >> >> > asarray(self.frqs).reshape(self.NE, self.NS, self.NM), >> >> > self.no_nd_struct ) >> >> > >> >> > if model in MODEL_LIST_CPMG_FULL: >> >> > self.cpmg_frqs_a = deepcopy(ones_a) >> >> > @@ -476,7 +482,7 @@ >> >> > # Extract number of dispersion points. >> >> > num_disp_points = >> >> > self.num_disp_points[ei][si][mi][oi] >> >> > >> >> > - if model in MODEL_LIST_CPMG_FULL: >> >> > + if model in MODEL_LIST_CPMG_FULL and >> >> > model >> >> > != MODEL_NOREX: >> >> > # Extract cpmg_frqs and >> >> > num_disp_points >> >> > from lists. >> >> > >> >> > self.cpmg_frqs_a[ei][si][mi][oi][:num_disp_points] = >> >> > self.cpmg_frqs[ei][mi][oi] >> >> > >> >> > self.num_disp_points_a[ei][si][mi][oi][:num_disp_points] = >> >> > self.num_disp_points[ei][si][mi][oi] >> >> > @@ -497,7 +503,7 @@ >> >> > self.power_a[ei][si][mi][oi][di] >> >> > = >> >> > int(round(self.cpmg_frqs[ei][mi][0][di] * self.relax_times[ei][mi])) >> >> > >> >> > self.tau_cpmg_a[ei][si][mi][oi][di] >> >> > = 0.25 / self.cpmg_frqs[ei][mi][0][di] >> >> > # For R1rho data. >> >> > - if model in MODEL_LIST_R1RHO_FULL: >> >> > + if model in MODEL_LIST_R1RHO_FULL >> >> > and >> >> > model != MODEL_NOREX: >> >> > >> >> > self.disp_struct[ei][si][mi][oi][di] = 1.0 >> >> > >> >> > # Extract the frequencies to >> >> > numpy >> >> > array. >> >> > @@ -511,11 +517,12 @@ >> >> > else: >> >> > >> >> > self.num_disp_points_a[ei][si][mi][oi][di] = 0 >> >> > >> >> > + if model in test_models: >> >> > + # Pre calculate frqs structure >> >> > + self.frqs_struct = self.frqs_a * self.disp_struct >> >> > + >> >> > # Make copy of values structure. >> >> > self.back_calc_a = deepcopy(self.values_a) >> >> > - >> >> > - # Pre calculate frqs structure >> >> > - self.frqs_struct = self.frqs_a * self.disp_struct >> >> > >> >> > # Find the numpy mask, which tells where values should >> >> > be >> >> > replaced. >> >> > self.mask_replace_blank = masked_equal(missing_a, 1.0) >> >> > @@ -1465,34 +1472,16 @@ >> >> > # Unpack the parameter values. >> >> > R20 = params >> >> > >> >> > - # Initialise. >> >> > - chi2_sum = 0.0 >> >> > - >> >> > - # Loop over the experiment types. >> >> > - for ei in range(self.num_exp): >> >> > - # Loop over the spins. >> >> > - for si in range(self.num_spins): >> >> > - # Loop over the spectrometer frequencies. >> >> > - for mi in range(self.num_frq): >> >> > - # The R20 index. >> >> > - r20_index = mi + si*self.num_frq >> >> > - >> >> > - # Loop over the offsets. >> >> > - for oi in range(self.num_offsets[ei][si][mi]): >> >> > - # The R2eff values as R20 values. >> >> > - for di in >> >> > range(self.num_disp_points[ei][si][mi][oi]): >> >> > - self.back_calc[ei][si][mi][oi][di] = >> >> > R20[r20_index] >> >> > - >> >> > - # For all missing data points, set the >> >> > back-calculated value to the measured values so that it has no effect >> >> > on the >> >> > chi-squared value. >> >> > - for di in >> >> > range(self.num_disp_points[ei][si][mi][oi]): >> >> > - if self.missing[ei][si][mi][oi][di]: >> >> > - self.back_calc[ei][si][mi][oi][di] = >> >> > self.values[ei][si][mi][oi][di] >> >> > - >> >> > - # Calculate and return the chi-squared >> >> > value. >> >> > - chi2_sum += >> >> > chi2(self.values[ei][si][mi][oi], >> >> > self.back_calc[ei][si][mi][oi], self.errors[ei][si][mi][oi]) >> >> > + # Reshape R20 to per experiment, spin and frequency. >> >> > + self.back_calc_a[:] = multiply.outer( R20.reshape(self.NE, >> >> > self.NS, self.NM), self.no_nd_struct ) >> >> > + >> >> > + ## For all missing data points, set the back-calculated >> >> > value >> >> > to the measured values so that it has no effect on the chi-squared >> >> > value. >> >> > + if self.has_missing: >> >> > + # Replace with values. >> >> > + self.back_calc_a[self.mask_replace_blank.mask] = >> >> > self.values_a[self.mask_replace_blank.mask] >> >> > >> >> > # Return the total chi-squared value. >> >> > - return chi2_sum >> >> > + return chi2_rankN(self.values_a, self.back_calc_a, >> >> > self.errors_a) >> >> > >> >> > >> >> > def func_ns_cpmg_2site_3D(self, params): >> >> > >> >> > >> >> > _______________________________________________ >> >> > relax (http://www.nmr-relax.com) >> >> > >> >> > This is the relax-commits mailing list >> >> > [email protected] >> >> > >> >> > To unsubscribe from this list, get a password >> >> > reminder, or change your subscription options, >> >> > visit the list information page at >> >> > https://mail.gna.org/listinfo/relax-commits >> >> >> >> _______________________________________________ >> >> relax (http://www.nmr-relax.com) >> >> >> >> This is the relax-devel mailing list >> >> [email protected] >> >> >> >> >> >> To unsubscribe from this list, get a password >> >> reminder, or change your subscription options, >> >> visit the list information page at >> >> https://mail.gna.org/listinfo/relax-devel >> > >> > > > _______________________________________________ relax (http://www.nmr-relax.com) This is the relax-devel mailing list [email protected] To unsubscribe from this list, get a password reminder, or change your subscription options, visit the list information page at https://mail.gna.org/listinfo/relax-devel
