Re: [Dbix-class] Why is this so slow?

2010-06-08 Thread Nigel Metheringham
On 8 Jun 2010, at 16:50, Nigel Metheringham wrote: > > You may be able to approach the speed of pure DBI if you use > DBIx::Class::ResultClass::HashRefInflator Thinking about this, if you are restricting the column set you are using, you are strangling your objects in a nasty way anyhow, so you

Re: [Dbix-class] Why is this so slow?

2010-06-08 Thread Nigel Metheringham
A loop over 100 million rows with multiple object inflation for each row is going to suck a serious amount of CPU time. [Even more horribly, if you are using some databases - for example DBD::Pg - then you are almost certainly not using a cursor or chunk at a time approach but instead sucking the

Re: [Dbix-class] Why is this so slow?

2010-06-08 Thread Alex J. G. Burzyński
Hi Chris, On 08/06/10 16:27, Chris Cole wrote: > On 08/06/10 15:05, "Alex J. G. Burzyński" wrote: >> Hi, >> >> On 08/06/10 14:09, Chris Cole wrote: >>> Given the schema provided at the end, why is this code so slow? >>> >> >> >> As you are creating an object $hit (NgsMappings with it's relationshi

Re: [Dbix-class] Why is this so slow?

2010-06-08 Thread Chris Cole
On 08/06/10 15:53, Nigel Metheringham wrote: On 8 Jun 2010, at 15:17, Chris Cole wrote: Which when run in an SQL client takes<1 sec to run and via 'normal' DBI the full code still runs in ~1.5 min. So, there's nothing wrong with the SQL. There seems to be something up with DBIC. This is DBIC

Re: [Dbix-class] Why is this so slow?

2010-06-08 Thread Chris Cole
On 08/06/10 15:05, "Alex J. G. Burzyński" wrote: Hi, On 08/06/10 14:09, Chris Cole wrote: Given the schema provided at the end, why is this code so slow? sub getCounts ($) { my $self = shift; my $dataset = shift; croak "ERROR - dataset name is required\n" unless ($dataset); m

Re: [Dbix-class] Why is this so slow?

2010-06-08 Thread Nigel Metheringham
On 8 Jun 2010, at 15:17, Chris Cole wrote: > > Which when run in an SQL client takes <1 sec to run and via 'normal' DBI the > full code still runs in ~1.5 min. So, there's nothing wrong with the SQL. > There seems to be something up with DBIC. This is DBIC v0.08122 BTW. How many rows are there

Re: [Dbix-class] Why is this so slow?

2010-06-08 Thread Chris Cole
Thanks for the reply. The SQL being executed is: SELECT me.mp_strand, mp_ref_id.rs_name, me.mp_start, me.mp_end, me.mp_freq FROM ngs_mappings me JOIN ngs_map_exps mp_me_id ON mp_me_id.me_id = me.mp_me_id JOIN ngs_ref_seqs mp_ref_id ON mp_ref_id.rs_id = me.mp_ref_id WHERE ( me_sample LIKE 'WT_t%_r

Re: [Dbix-class] Why is this so slow?

2010-06-08 Thread Ronald J Kimball
On Tue, Jun 8, 2010 at 9:09 AM, Chris Cole wrote: > > # make SQL search both technical replicates at the same time > $dataset =~ s/t\d/t%/; > my $rs = $self->resultset('NgsMappings')->search( > { > 'me_sample' => {'like', $dataset}, > }, > { > columns => [qw/mp

Re: [Dbix-class] Why is this so slow?

2010-06-08 Thread Alex J. G. Burzyński
Hi, On 08/06/10 14:09, Chris Cole wrote: > Given the schema provided at the end, why is this code so slow? > > sub getCounts ($) { >my $self = shift; >my $dataset = shift; > >croak "ERROR - dataset name is required\n" unless ($dataset); > >my $nHits; > ># make SQL search both t

[Dbix-class] Why is this so slow?

2010-06-08 Thread Chris Cole
Given the schema provided at the end, why is this code so slow? sub getCounts ($) { my $self = shift; my $dataset = shift; croak "ERROR - dataset name is required\n" unless ($dataset); my $nHits; # make SQL search both technical replicates at the same time $dataset =~ s/t\d/t