Re: [PHP] Exporting large data from mysql to html using php

2011-11-03 Thread Ashley Sheridan
On Wed, 2011-11-02 at 19:47 -0400, Jason Pruim wrote: > Jason Pruim > li...@pruimphotography.com > > > > On Oct 31, 2011, at 7:52 PM, Ashley Sheridan wrote: > > > On Mon, 2011-10-31 at 19:29 -0400, Jason Pruim wrote: > >> > >> Jason Pruim > >> li...@pruimphotography.com > >> > >> > >> > >>

Re: [PHP] Exporting large data from mysql to html using php

2011-11-02 Thread Jason Pruim
Jason Pruim li...@pruimphotography.com On Oct 31, 2011, at 7:52 PM, Ashley Sheridan wrote: > On Mon, 2011-10-31 at 19:29 -0400, Jason Pruim wrote: >> >> Jason Pruim >> li...@pruimphotography.com >> >> >> >> On Oct 31, 2011, at 7:11 PM, Jim Lucas wrote: >> >> > On 10/24/2011 5:50 PM, Jason

Re: [PHP] Exporting large data from mysql to html using php

2011-10-31 Thread Ashley Sheridan
On Mon, 2011-10-31 at 19:29 -0400, Jason Pruim wrote: > Jason Pruim > li...@pruimphotography.com > > > > On Oct 31, 2011, at 7:11 PM, Jim Lucas wrote: > > > On 10/24/2011 5:50 PM, Jason Pruim wrote: > >> Now that I've managed to list 3 separate programming languages and > >> somewhat tie it b

Re: [PHP] Exporting large data from mysql to html using php

2011-10-31 Thread Jason Pruim
Jason Pruim li...@pruimphotography.com On Oct 31, 2011, at 7:11 PM, Jim Lucas wrote: > On 10/24/2011 5:50 PM, Jason Pruim wrote: >> Now that I've managed to list 3 separate programming languages and somewhat >> tie it back into php here's the question... >> >> I have about 89 million records

Re: [PHP] Exporting large data from mysql to html using php

2011-10-31 Thread Jim Lucas
On 10/24/2011 5:50 PM, Jason Pruim wrote: > Now that I've managed to list 3 separate programming languages and somewhat > tie it back into php here's the question... > > I have about 89 million records in mysql... the initial load of the page > takes 2 to 3 minutes, I am using pagination, so I h

Re: [PHP] Exporting large data from mysql to html using php

2011-10-27 Thread Jason Pruim
Jason Pruim li...@pruimphotography.com On Oct 27, 2011, at 4:08 PM, Nathan Nobbe wrote: > On Mon, Oct 24, 2011 at 6:50 PM, Jason Pruim > wrote: > Now that I've managed to list 3 separate programming languages and somewhat > tie it back into php here's the question... > > I have about 89 mi

Re: [PHP] Exporting large data from mysql to html using php

2011-10-27 Thread Jason Pruim
Jason Pruim li...@pruimphotography.com On Oct 27, 2011, at 2:44 PM, Tommy Pham wrote: > On Wed, Oct 26, 2011 at 5:47 PM, Jason Pruim > wrote: > > Jason Pruim > li...@pruimphotography.com > > The server that's running it is a home computer with a VPS installed... It's > not my dev environm

Re: [PHP] Exporting large data from mysql to html using php

2011-10-27 Thread Jim Giner
> > Good luck, that's a LOT of reading. I'd estimate that's about 3k+ pages > of > reading. :) > > >> > >> > >> > Regards, >> > Tommy >> >> > nice to see someone else is finally getting the point that I'm been making. -- PHP General Mailing List (http://www.php.net/) To unsu

Re: [PHP] Exporting large data from mysql to html using php

2011-10-27 Thread Nathan Nobbe
On Mon, Oct 24, 2011 at 6:50 PM, Jason Pruim wrote: > Now that I've managed to list 3 separate programming languages and somewhat > tie it back into php here's the question... > > I have about 89 million records in mysql... the initial load of the page > takes 2 to 3 minutes, I am using pagination

Re: [PHP] Exporting large data from mysql to html using php

2011-10-27 Thread Tommy Pham
On Wed, Oct 26, 2011 at 5:47 PM, Jason Pruim wrote: > > Jason Pruim > li...@pruimphotography.com > > The server that's running it is a home computer with a VPS installed... > It's not my dev environment :) > > Home computer used for a production environment? Wow.. I'm speechless. > The informat

Re: [PHP] Exporting large data from mysql to html using php

2011-10-27 Thread David Robley
Jim Giner wrote: > "David Robley" wrote in message > news:49.50.34068.1b567...@pb1.pair.com... >> >> Consider running EXPLAIN on all your queries to see if there is something >> Mysql thinks could be done to improve performance. >> > > Why do so many responders seem to think the problem here is

Re: [PHP] Exporting large data from mysql to html using php

2011-10-27 Thread Lester Caine
Jim Giner wrote: Your boss wants to give access to phone numbers to the public in general? Then what? Glad mine's unlisted. Is it? Does it start 518248 ? I often forget to get a phone number when a parcel has to go by carrier, and paypal does not include that info, but only rarely does one no

Re: [PHP] Exporting large data from mysql to html using php

2011-10-26 Thread Jason Pruim
Jason Pruim li...@pruimphotography.com On Oct 26, 2011, at 9:09 PM, Jim Giner wrote: > Your boss wants to give access to phone numbers to the public in general? > Then what? > > Glad mine's unlisted. There's no identifying information on the phone numbers... Simply just the phone number...

Re: [PHP] Exporting large data from mysql to html using php

2011-10-26 Thread Jim Giner
Your boss wants to give access to phone numbers to the public in general? Then what? Glad mine's unlisted. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP] Exporting large data from mysql to html using php

2011-10-26 Thread Jason Pruim
Jason Pruim li...@pruimphotography.com On Oct 26, 2011, at 6:23 AM, Tommy Pham wrote: > On Wed, Oct 26, 2011 at 1:40 AM, Lester Caine wrote: > >> Tommy Pham wrote: >> >>> I wonder ... The real question is what's the purpose of the DB? Is it for >>> OLAP >>> or OLTP? ;) >>> As for dealing w

Re: [PHP] Exporting large data from mysql to html using php

2011-10-26 Thread Tommy Pham
On Wed, Oct 26, 2011 at 4:14 AM, Lester Caine wrote: > Tommy Pham wrote: > >> >>Many of my customers have coming up on 20 years of data available. >> There has >>been a debate on transferring historic data to a separate database, but >>having it available is not causing a problem, exc

Re: [PHP] Exporting large data from mysql to html using php

2011-10-26 Thread Lester Caine
Tommy Pham wrote: Many of my customers have coming up on 20 years of data available. There has been a debate on transferring historic data to a separate database, but having it available is not causing a problem, except for some counts and larger search actions, and being able to

Re: [PHP] Exporting large data from mysql to html using php

2011-10-26 Thread Tommy Pham
On Wed, Oct 26, 2011 at 3:23 AM, Tommy Pham wrote: > On Wed, Oct 26, 2011 at 1:40 AM, Lester Caine wrote: > >> Tommy Pham wrote: >> >>> I wonder ... The real question is what's the purpose of the DB? Is it >>> for OLAP >>> or OLTP? ;) >>> As for dealing with DB having millions of rows, you're c

Re: [PHP] Exporting large data from mysql to html using php

2011-10-26 Thread Tommy Pham
On Wed, Oct 26, 2011 at 1:40 AM, Lester Caine wrote: > Tommy Pham wrote: > >> I wonder ... The real question is what's the purpose of the DB? Is it for >> OLAP >> or OLTP? ;) >> As for dealing with DB having millions of rows, you're crossing over into >> DBA area. >> > > Many of my customers hav

Re: [PHP] Exporting large data from mysql to html using php

2011-10-26 Thread Lester Caine
Tommy Pham wrote: I wonder ... The real question is what's the purpose of the DB? Is it for OLAP or OLTP? ;) As for dealing with DB having millions of rows, you're crossing over into DBA area. Many of my customers have coming up on 20 years of data available. There has been a debate on trans

Re: [PHP] Exporting large data from mysql to html using php

2011-10-26 Thread Tommy Pham
On Wed, Oct 26, 2011 at 12:52 AM, Lester Caine wrote: > Tommy Pham wrote: > >> It turns out the issue was actually in the pagination... I'm reworking the >>> > whole thing and stream lining it... But in the pagination that I found >>> on >>> > the internet it used a "SELECT COUNT(*) WHERE state

Re: [PHP] Exporting large data from mysql to html using php

2011-10-26 Thread Lester Caine
Tommy Pham wrote: It turns out the issue was actually in the pagination... I'm reworking the > whole thing and stream lining it... But in the pagination that I found on > the internet it used a "SELECT COUNT(*) WHERE state='{$state}'"; and the > COUNT was killing the time... Once that was remo

Re: [PHP] Exporting large data from mysql to html using php

2011-10-25 Thread Tommy Pham
On Tue, Oct 25, 2011 at 7:06 PM, Jason Pruim wrote: > > It turns out the issue was actually in the pagination... I'm reworking the > whole thing and stream lining it... But in the pagination that I found on > the internet it used a "SELECT COUNT(*) WHERE state='{$state}'"; and the > COUNT was kill

Re: [PHP] Exporting large data from mysql to html using php

2011-10-25 Thread Jason Pruim
Jason Pruim li...@pruimphotography.com On Oct 25, 2011, at 10:10 PM, Jim Giner wrote: > > - Original Message - From: "Jason Pruim" > To: "Jim Giner" > Cc: > Sent: Tuesday, October 25, 2011 10:06 PM > Subject: Re: [PHP] Exporting large data f

Re: [PHP] Exporting large data from mysql to html using php

2011-10-25 Thread Jim Giner
- Original Message - From: "Jason Pruim" To: "Jim Giner" Cc: Sent: Tuesday, October 25, 2011 10:06 PM Subject: Re: [PHP] Exporting large data from mysql to html using php It turns out the issue was actually in the pagination... I'm reworking the whole t

Re: [PHP] Exporting large data from mysql to html using php

2011-10-25 Thread Jason Pruim
Jason Pruim li...@pruimphotography.com On Oct 25, 2011, at 9:58 PM, Jim Giner wrote: > "David Robley" wrote in message > news:49.50.34068.1b567...@pb1.pair.com... >> >> Consider running EXPLAIN on all your queries to see if there is something >> Mysql thinks could be done to improve perform

Re: [PHP] Exporting large data from mysql to html using php

2011-10-25 Thread David Robley
Jason Pruim wrote: > > Jason Pruim > li...@pruimphotography.com > > > > On Oct 25, 2011, at 10:51 AM, Jim Giner wrote: > >> I disagree. It's not about "tuning the queries", it is more about the >> appl. design that currently thinks it SHOULD do such huge queries. >> >> My approach would be

Re: [PHP] Exporting large data from mysql to html using php

2011-10-25 Thread Jim Giner
"David Robley" wrote in message news:49.50.34068.1b567...@pb1.pair.com... > > Consider running EXPLAIN on all your queries to see if there is something > Mysql thinks could be done to improve performance. > Why do so many responders seem to think the problem here is in the preparation of the qu

Re: [PHP] Exporting large data from mysql to html using php

2011-10-25 Thread Bastien
On 2011-10-25, at 6:56 PM, Jason Pruim wrote: > > Jason Pruim > li...@pruimphotography.com > > > > On Oct 25, 2011, at 6:35 PM, Jim Giner wrote: > >> Again why even do a detail query? Nobody is going to examine pages and >> pages and etc. >> Do a summary qry if u just need a count -

Re: [PHP] Exporting large data from mysql to html using php

2011-10-25 Thread Jason Pruim
Jason Pruim li...@pruimphotography.com On Oct 25, 2011, at 6:35 PM, Jim Giner wrote: > Again why even do a detail query? Nobody is going to examine pages and > pages and etc. > Do a summary qry if u just need a count - no pagination there > jg The bosses wanted them to be able to page t

Re: [PHP] Exporting large data from mysql to html using php

2011-10-25 Thread Jim Giner
Again why even do a detail query? Nobody is going to examine pages and pages and etc. Do a summary qry if u just need a count - no pagination there jg On Oct 25, 2011, at 6:26 PM, Jason Pruim wrote: > > Jason Pruim > li...@pruimphotography.com > > > > On Oct 25, 2011, at 10:51 AM, Jim

Re: [PHP] Exporting large data from mysql to html using php

2011-10-25 Thread Jason Pruim
Jason Pruim li...@pruimphotography.com On Oct 25, 2011, at 10:51 AM, Jim Giner wrote: > I disagree. It's not about "tuning the queries", it is more about the appl. > design that currently thinks it SHOULD do such huge queries. > > My approach would be to prompt the user for filtering criter

Re: [PHP] Exporting large data from mysql to html using php

2011-10-25 Thread Jim Giner
I disagree. It's not about "tuning the queries", it is more about the appl. design that currently thinks it SHOULD do such huge queries. My approach would be to prompt the user for filtering criteria that automatically would reduce the result set size. Although at this time I believe the OP m

Re: [PHP] Exporting large data from mysql to html using php

2011-10-24 Thread tamouse mailing lists
On Mon, Oct 24, 2011 at 7:50 PM, Jason Pruim wrote: > I have about 89 million records in mysql... the initial load of the page > takes 2 to 3 minutes, I am using pagination, so I have LIMIT's on the SQL > query's... But they just aren't going fast enough... > > What I would like to do, is pull t

Re: [PHP] Exporting large data from mysql to html using php

2011-10-24 Thread Bastien
On 2011-10-24, at 11:14 PM, Jason Pruim wrote: > > Jason Pruim > li...@pruimphotography.com > > > > On Oct 24, 2011, at 9:20 PM, Bastien wrote: > >> >> >> On 2011-10-24, at 8:50 PM, Jason Pruim wrote: >> >>> Now that I've managed to list 3 separate programming languages and somewhat >>

Re: [PHP] Exporting large data from mysql to html using php

2011-10-24 Thread Bastien
On 2011-10-24, at 11:26 PM, "Jim Giner" wrote: > Yes - but - we're talking about a user-app that the OP is trying to provide > 89M records to. Sure - "some" users might have need of looking at even as > much as a million records IF they were researching something that needed it. > But - for

Re: [PHP] Exporting large data from mysql to html using php

2011-10-24 Thread Jim Giner
Yes - but - we're talking about a user-app that the OP is trying to provide 89M records to. Sure - "some" users might have need of looking at even as much as a million records IF they were researching something that needed it. But - for the 'general' user of an app - I cannot see a need to be p

Re: [PHP] Exporting large data from mysql to html using php

2011-10-24 Thread Jason Pruim
Jason Pruim li...@pruimphotography.com On Oct 24, 2011, at 9:20 PM, Bastien wrote: > > > On 2011-10-24, at 8:50 PM, Jason Pruim wrote: > >> Now that I've managed to list 3 separate programming languages and somewhat >> tie it back into php here's the question... >> >> I have about 89 mil

Re: [PHP] Exporting large data from mysql to html using php

2011-10-24 Thread Jason Pruim
On Oct 24, 2011, at 10:44 PM, Jim Giner wrote: > Why would any user need to have access to 89M records? They don't need access to it to edit it... Just to be able to view it... Also, it will be expanding in the future to include alot more data. Jason Pruim li...@pruimphotography.com -- PH

Re: [PHP] Exporting large data from mysql to html using php

2011-10-24 Thread Bastien
On 2011-10-24, at 10:44 PM, "Jim Giner" wrote: > Why would any user need to have access to 89M records? > > > > -- > PHP General Mailing List (http://www.php.net/) > To unsubscribe, visit: http://www.php.net/unsub.php > History or audit trail data? I can think of lots. I know of an app wit

Re: [PHP] Exporting large data from mysql to html using php

2011-10-24 Thread Jim Giner
Why would any user need to have access to 89M records? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP] Exporting large data from mysql to html using php

2011-10-24 Thread Bastien
On 2011-10-24, at 8:50 PM, Jason Pruim wrote: > Now that I've managed to list 3 separate programming languages and somewhat > tie it back into php here's the question... > > I have about 89 million records in mysql... the initial load of the page > takes 2 to 3 minutes, I am using pagination

Re: [PHP] Exporting large data from mysql to html using php

2011-10-24 Thread Shiplu Mokaddim
2-3 minutes is long enough. I think your SELECT query and MySQL schema has something that can be improved. Sent from a handheld device. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php

[PHP] Exporting large data from mysql to html using php

2011-10-24 Thread Jason Pruim
Now that I've managed to list 3 separate programming languages and somewhat tie it back into php here's the question... I have about 89 million records in mysql... the initial load of the page takes 2 to 3 minutes, I am using pagination, so I have LIMIT's on the SQL query's... But they just are