Re: Model find() to return object; instead of array (?)
On Apr 4, 11:41 pm, Baz [EMAIL PROTECTED] wrote: Just throwing in my two cents, but I've had frequent problems with phpMyAdmin exceeding timeout limits (and once even memory limits) on a shared hosts... So, trying to compare phpMyAdmin with what you're trying to do isn't really fair. All it does are imports and exports (in terms of large processes). I haven't checked phpMyAdmin lately. But, as I remember, the export to SQL on phpMyAdmin is on-the-fly meaning that it is been sent to the browser in record by record. Of late, I have also seen programmers choose the direct export in SQL--instead of choosing it to zip/bzip---so that they don't get memory errors. However, you *should* be fine with just pumping up both the allocated RAM and exec. time for PHP, if you truly want to have this done through CakePHP. But what you're describing is neither a limitation for CakePHP or PHP itself. You just have a butt load of data to deal with and memory issues are going to be a problem with a butt load of data. I don't know why most of the experts are overlooking the situation. Some long time ago, when I said that there is no need for PHP 4 support, everyone was overlooking; now fairly everyone is realizing; similarly, I hope this situation will be realized sooner or later. The export data option is a common feature needed for any webapp--whether created with Cake or vanilla PHP. -- ?php echo 'Just another PHP saint'; ? Email: rrjanbiah-at-Y!comBlog: http://rajeshanbiah.blogspot.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Model find() to return object; instead of array (?)
On Apr 3, 10:40 pm, Chris Hartjes [EMAIL PROTECTED] wrote: On Thu, Apr 3, 2008 at 1:13 PM, R. Rajesh Jeba Anbiah [EMAIL PROTECTED] wrote: Cake's default way of dumping array in controller and iterating in view file, works only for the small data. But, on a typical project that requires exporting of data, it doesn't work as it runs out of memory. This needs fetching the record one-by-one like I mentioned above. I understand. Sounds to me like something that exports data like this is not suited to be a action within a controller. Maybe you should look into making this a Cake shell task, as that would let you use all the Cake goodness and be able to run from the command line. Shell is not the direct solution as it's exporting of data for admin and users. Either that or you'll have to run queries using LIMIT and OFFSET to grab what you need one chunk at a time. I don't know any scripting language that handles sucking in 600,000 records via a web page. Think about phpMyAdmin? BTW, whenever you reply here with your remarks, I thought of asking do you really use CakePHP? SCNR For a guy who calls himself Just another PHP saint, that's quite a bold statement. Good thing my life doesn't revolve around worrying about what ignorant people think about me. LOL. I'm not ignorant about you;-) Yes, I do use CakePHP. Otherwise I'd be wasting my time here. Don't blame me for failing to understand what your problem is. Maybe you should look into how you're asking your questions. Yup, good advice. -- ?php echo 'Just another PHP saint'; ? Email: rrjanbiah-at-Y!comBlog: http://rajeshanbiah.blogspot.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Model find() to return object; instead of array (?)
On Apr 3, 11:12 pm, Christian Winther [EMAIL PROTECTED] wrote: I use cake way for documents and sql data way above 100 megabytes in size My guess would be your setup is wrongly configured - rather than pointing to php or cake as the problem. Obviously, you're overlooking the problem. If you really need the resource, just do the mysql_select() and everything yourself, it will use your default mysql connection - and should work out of the box aswell. The reason why I posted it is that, I vaguely remember that it's been discussed and had solution. -- ?php echo 'Just another PHP saint'; ? Email: rrjanbiah-at-Y!comBlog: http://rajeshanbiah.blogspot.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Model find() to return object; instead of array (?)
On Apr 4, 4:08 am, Grant Cox [EMAIL PROTECTED] wrote: We have some large reports for our application - often involving ~250K rows from 8 different tables. The problem is, if your reporting needs to do any kind of collating or comparison, as ours does, you really can't avoid having all the data loaded somewhere. You may be able to do much of it in your database which should stop PHP from running out of memory, but it'll have to be used somewhere. If you can get the db to do everything, everyone's a winner. e.g. http://forums.mysql.com/read.php?79,11324,13062#msg-13062 If you can't get the db to do everything (because the logic isn't all in the db, or it's too complex to do so, or because, because etc.), if you generate your output in chunks (standard batch processing afaic) and dump it into a temporary table - then you can do as above and again: everyone's a winner :). Then you only need to send the file to the user, or redirect the user to the right location. For a standard csv dump you can reduce your php processing time and memory requirements to practically 0 in this way. hth, AD --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Model find() to return object; instead of array (?)
I suspect what you're looking for is the extremely un-Cake
DboSource::rawQuery, which will return you a resource depending on
your database connection type.
$conn = ConnectionManager::getDataSource('default');
$res = $conn-rawQuery($sql);
Or if you want a bit more help from cake, $conn-execute($sql);
mysql_fetch_row($res)... etc.
Take a look at cake/libs/model/datasources/dbo_source.php if you need
more info.
This is pretty much the most un Cake way you could possibly do
anything, as the other guys have already pointed out, but it should
sort of achieve what you're after. Bear in mind that Cake is not
hibernate, and can't really be, you will never get the sort of fancy
lazy fetching stuff that keeps memory usage down, but that doesn't
really matter, because it's a framework for making web pages - some
thing it does really well - and web pages tend not to require
gigabytes of memory, hence Cake very sensibly doesn't go through all
those other hoops that same an j2ee ORM would.
Simon
http://www.simonellistonball.com/
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups Cake
PHP group.
To post to this group, send email to cake-php@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/cake-php?hl=en
-~--~~~~--~~--~--~---
Re: Model find() to return object; instead of array (?)
On Apr 4, 4:51 pm, AD7six [EMAIL PROTECTED] wrote: On Apr 4, 4:08 am, Grant Cox [EMAIL PROTECTED] wrote: We have some large reports for our application - often involving ~250K rows from 8 different tables. The problem is, if your reporting needs to do any kind of collating or comparison, as ours does, you really can't avoid having all the data loaded somewhere. You may be able to do much of it in your database which should stop PHP from running out of memory, but it'll have to be used somewhere. If you can get the db to do everything, everyone's a winner. e.g.http://forums.mysql.com/read.php?79,11324,13062#msg-13062 If you can't get the db to do everything (because the logic isn't all in the db, or it's too complex to do so, or because, because etc.), if you generate your output in chunks (standard batch processing afaic) and dump it into a temporary table - then you can do as above and again: everyone's a winner :). Then you only need to send the file to the user, or redirect the user to the right location. For a standard csv dump you can reduce your php processing time and memory requirements to practically 0 in this way. But, that gonna increase the disk space requirements:-( I considered all known options; but record-by-record dumping is the only workable approach as far as I see. -- ?php echo 'Just another PHP saint'; ? Email: rrjanbiah-at-Y!comBlog: http://rajeshanbiah.blogspot.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Model find() to return object; instead of array (?)
On Apr 4, 2:41 pm, R. Rajesh Jeba Anbiah [EMAIL PROTECTED] wrote: On Apr 4, 4:51 pm, AD7six [EMAIL PROTECTED] wrote: On Apr 4, 4:08 am, Grant Cox [EMAIL PROTECTED] wrote: We have some large reports for our application - often involving ~250K rows from 8 different tables. The problem is, if your reporting needs to do any kind of collating or comparison, as ours does, you really can't avoid having all the data loaded somewhere. You may be able to do much of it in your database which should stop PHP from running out of memory, but it'll have to be used somewhere. If you can get the db to do everything, everyone's a winner. e.g.http://forums.mysql.com/read.php?79,11324,13062#msg-13062 If you can't get the db to do everything (because the logic isn't all in the db, or it's too complex to do so, or because, because etc.), if you generate your output in chunks (standard batch processing afaic) and dump it into a temporary table - then you can do as above and again: everyone's a winner :). Then you only need to send the file to the user, or redirect the user to the right location. For a standard csv dump you can reduce your php processing time and memory requirements to practically 0 in this way. But, that gonna increase the disk space requirements:-( I Holy crap, it'll mean you'll need a keyboard to do this aswell! The list never ends :D ps: cache. deleting cache, temporary files, etc. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Model find() to return object; instead of array (?)
Either that or you'll have to run queries using LIMIT and OFFSET to grab what you need one chunk at a time. I don't know any scripting language that handles sucking in 600,000 records via a web page. Think about phpMyAdmin? PHPMyAdmin makes the DB do all the work. When you upload a .sql file you are just uploading a file and then telling MySQL the location and it does all the importing. Same thing with CSV exports. And I've had plenty of timeouts even with that. If your reports are really that heavy duty you're probably better off doing them in batches or from another environment altogether (such is have CakePHP trigger a report to be run by a Python script). That said, you can probably heavily optimize your data in the DB, PHP shouldn't have to deal with all the records in the first place. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Model find() to return object; instead of array (?)
On Fri, Apr 4, 2008 at 10:44 AM, jonknee [EMAIL PROTECTED] wrote: Either that or you'll have to run queries using LIMIT and OFFSET to grab what you need one chunk at a time. I don't know any scripting language that handles sucking in 600,000 records via a web page. Think about phpMyAdmin? 1) phpMyAdmin is not a scripting language 2) Notice I've said sucking in 600,000 records not searching through 600,000 records looking for something. There is a huge difference in the two. I use phpMyAdmin every day for work and I can't help notice that by default it limits the amount of data you can pull back. 3) This discussion is veering very far away from CakePHP. -- Chris Hartjes Internet Loudmouth Motto for 2008: Moving from herding elephants to handling snakes... @TheKeyBoard: http://www.littlehart.net/atthekeyboard --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Model find() to return object; instead of array (?)
On Fri, Apr 4, 2008 at 10:49 AM, Chris Hartjes [EMAIL PROTECTED] wrote: On Fri, Apr 4, 2008 at 10:44 AM, jonknee [EMAIL PROTECTED] wrote: Either that or you'll have to run queries using LIMIT and OFFSET to grab what you need one chunk at a time. I don't know any scripting language that handles sucking in 600,000 records via a web page. Think about phpMyAdmin? 1) phpMyAdmin is not a scripting language In all fairness, the suggestion, I think, was that phpMyAdmin is an example of an application written using a scripting language that ... Getting back to the OP's question: use a regular query and iterate over the result just like you would in a regular PHP script. You lose a bunch of Cake's functionality but that's an easy trade-off in a situation like this. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Model find() to return object; instead of array (?)
On Apr 4, 6:04 pm, AD7six [EMAIL PROTECTED] wrote: On Apr 4, 2:41 pm, R. Rajesh Jeba Anbiah [EMAIL PROTECTED] wrote: On Apr 4, 4:51 pm, AD7six [EMAIL PROTECTED] wrote: On Apr 4, 4:08 am, Grant Cox [EMAIL PROTECTED] wrote: We have some large reports for our application - often involving ~250K rows from 8 different tables. The problem is, if your reporting needs to do any kind of collating or comparison, as ours does, you really can't avoid having all the data loaded somewhere. You may be able to do much of it in your database which should stop PHP from running out of memory, but it'll have to be used somewhere. If you can get the db to do everything, everyone's a winner. e.g.http://forums.mysql.com/read.php?79,11324,13062#msg-13062 If you can't get the db to do everything (because the logic isn't all in the db, or it's too complex to do so, or because, because etc.), if you generate your output in chunks (standard batch processing afaic) and dump it into a temporary table - then you can do as above and again: everyone's a winner :). Then you only need to send the file to the user, or redirect the user to the right location. For a standard csv dump you can reduce your php processing time and memory requirements to practically 0 in this way. But, that gonna increase the disk space requirements:-( I Holy crap, it'll mean you'll need a keyboard to do this aswell! The list never ends :D ps: cache. deleting cache, temporary files, etc. LOL. BTW, for now, I have a temporary workaround. As per the parseExtension(), only the csv needs the record by record iteration; so passing the model object to csv view file and invoking find() with limit and offset seems to handle the situation. But, obviously, it's getting crazy for this export module. -- ?php echo 'Just another PHP saint'; ? Email: rrjanbiah-at-Y!comBlog: http://rajeshanbiah.blogspot.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Model find() to return object; instead of array (?)
On Fri, Apr 4, 2008 at 12:02 PM, R. Rajesh Jeba Anbiah [EMAIL PROTECTED] wrote: I'm talking about handling exporting module with CakePHP. It's relevant when you develop a project/application for clients. I understand, I just think that CakePHP is probably not suited to do that sort of thing on the scale you require. As good as CakePHP is, there are some problems it cannot easily solve. -- Chris Hartjes Internet Loudmouth Motto for 2008: Moving from herding elephants to handling snakes... @TheKeyBoard: http://www.littlehart.net/atthekeyboard --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Model find() to return object; instead of array (?)
On Apr 4, 7:49 pm, Chris Hartjes [EMAIL PROTECTED] wrote: On Fri, Apr 4, 2008 at 10:44 AM, jonknee [EMAIL PROTECTED] wrote: Either that or you'll have to run queries using LIMIT and OFFSET to grab what you need one chunk at a time. I don't know any scripting language that handles sucking in 600,000 records via a web page. Think about phpMyAdmin? 1) phpMyAdmin is not a scripting language 2) Notice I've said sucking in 600,000 records not searching through 600,000 records looking for something. There is a huge difference in the two. I use phpMyAdmin every day for work and I can't help notice that by default it limits the amount of data you can pull back. 3) This discussion is veering very far away from CakePHP. I'm talking about handling exporting module with CakePHP. It's relevant when you develop a project/application for clients. -- ?php echo 'Just another PHP saint'; ? Email: rrjanbiah-at-Y!comBlog: http://rajeshanbiah.blogspot.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Model find() to return object; instead of array (?)
Just throwing in my two cents, but I've had frequent problems with phpMyAdmin exceeding timeout limits (and once even memory limits) on a shared hosts... So, trying to compare phpMyAdmin with what you're trying to do isn't really fair. All it does are imports and exports (in terms of large processes). However, you *should* be fine with just pumping up both the allocated RAM and exec. time for PHP, if you truly want to have this done through CakePHP. But what you're describing is neither a limitation for CakePHP or PHP itself. You just have a butt load of data to deal with and memory issues are going to be a problem with a butt load of data. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Model find() to return object; instead of array (?)
My two cents as well, for 600 000+ records (even less) mysqldump is best ... Concerning phpmyAdmin, the frequent problems I have is on importing ..so again mysql in shell solves this .. On Apr 4, 8:41 pm, Baz [EMAIL PROTECTED] wrote: Just throwing in my two cents, but I've had frequent problems with phpMyAdmin exceeding timeout limits (and once even memory limits) on a shared hosts... So, trying to compare phpMyAdmin with what you're trying to do isn't really fair. All it does are imports and exports (in terms of large processes). However, you *should* be fine with just pumping up both the allocated RAM and exec. time for PHP, if you truly want to have this done through CakePHP. But what you're describing is neither a limitation for CakePHP or PHP itself. You just have a butt load of data to deal with and memory issues are going to be a problem with a butt load of data. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Model find() to return object; instead of array (?)
I'm not sure of any method - but why do you think that all your rows as arrays will use up more memory than the same amount of data wrapped in objects? Is it just a pass by reference vs pass by copy issue? At the heart of it the dbo_source.php file takes the data from the database query as an array. You could then build some object up out of this - but I'm not sure that it will really help any memory issues. On Apr 3, 3:41 pm, R. Rajesh Jeba Anbiah [EMAIL PROTECTED] wrote: It's very crucial to have the find() or similar returns DB object instead of full array--as for any data export module, we cannot dump the whole array which would result in memory error. IIRC, someone provided how to do that in CakePHP, but I lost the link; but, when I google, the closest I get now ishttp://jimmyzimmerman.com/blog/2007/05/why-i-prefer-ruby-on-rails-ove... Is there anyone who bookmarked such previous hack or can provide any hint? TIA -- ?php echo 'Just another PHP saint'; ? Email: rrjanbiah-at-Y!comBlog:http://rajeshanbiah.blogspot.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Model find() to return object; instead of array (?)
Like it is written in one of the comments on the page you link to. Cake does not return objects partly because it does not fit well with the PHP-language. In ruby, everyhing is an object. Every character, every number is its own object. So it is very natural to use objects for the model-data to follow the language conventions. Doing the same in PHP would be fighting the language every step of the way. I understand your desire to have lazy data retieval and access to methods on your data. I want that to from time to time (specifically when I have been developing in a more OO language for a while and come back to PHP). As on that page I would also suggest you look at some of the bahaviors in the Bakery that can help with binding and unbinding data not needed for the current request. Another thing you can make use of for exports is paginate. Yes really :) If you are dumping 600 000 rows at once you could probably run out of memory. By running the export in a few passes are appending the exported file (or wherever the data is going) each time you will use less memory. (sure you can also paginate manually but what fun would that be?) I often run big-memory and long-time data management from the php-cli using cron or other trigger. /Martin On Apr 3, 7:41 am, R. Rajesh Jeba Anbiah [EMAIL PROTECTED] wrote: It's very crucial to have the find() or similar returns DB object instead of full array--as for any data export module, we cannot dump the whole array which would result in memory error. IIRC, someone provided how to do that in CakePHP, but I lost the link; but, when I google, the closest I get now ishttp://jimmyzimmerman.com/blog/2007/05/why-i-prefer-ruby-on-rails-ove... Is there anyone who bookmarked such previous hack or can provide any hint? TIA -- ?php echo 'Just another PHP saint'; ? Email: rrjanbiah-at-Y!com Blog:http://rajeshanbiah.blogspot.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Model find() to return object; instead of array (?)
I think this is the link your a looking for: http://bin.cakephp.org/saved/1237 On Thu, Apr 3, 2008 at 4:24 AM, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Like it is written in one of the comments on the page you link to. Cake does not return objects partly because it does not fit well with the PHP-language. In ruby, everyhing is an object. Every character, every number is its own object. So it is very natural to use objects for the model-data to follow the language conventions. Doing the same in PHP would be fighting the language every step of the way. I understand your desire to have lazy data retieval and access to methods on your data. I want that to from time to time (specifically when I have been developing in a more OO language for a while and come back to PHP). As on that page I would also suggest you look at some of the bahaviors in the Bakery that can help with binding and unbinding data not needed for the current request. Another thing you can make use of for exports is paginate. Yes really :) If you are dumping 600 000 rows at once you could probably run out of memory. By running the export in a few passes are appending the exported file (or wherever the data is going) each time you will use less memory. (sure you can also paginate manually but what fun would that be?) I often run big-memory and long-time data management from the php-cli using cron or other trigger. /Martin On Apr 3, 7:41 am, R. Rajesh Jeba Anbiah [EMAIL PROTECTED] wrote: It's very crucial to have the find() or similar returns DB object instead of full array--as for any data export module, we cannot dump the whole array which would result in memory error. IIRC, someone provided how to do that in CakePHP, but I lost the link; but, when I google, the closest I get now ishttp:// jimmyzimmerman.com/blog/2007/05/why-i-prefer-ruby-on-rails-ove... Is there anyone who bookmarked such previous hack or can provide any hint? TIA -- ?php echo 'Just another PHP saint'; ? Email: rrjanbiah-at-Y!comBlog:http://rajeshanbiah.blogspot.com/ -- Matias Lespiau http://www.gignus.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Model find() to return object; instead of array (?)
On Apr 3, 11:51 am, Grant Cox [EMAIL PROTECTED] wrote:
I'm not sure of any method - but why do you think that all your rows
as arrays will use up more memory than the same amount of data wrapped
in objects? Is it just a pass by reference vs pass by copy issue?
snip
I'm sorry, I goofed it up with the term object; I actually meant
resource. Cake is currently dealing like this:
?php
$result = mysql_query('SELECT * FROM foo');
while ($row = mysql_fetch_assoc($result)) {
$array[] = $row['foo'];
}
//---
// And, in view
foreach($array as $x) {
echo $x;
}
?
I want something like:
?php
$result = mysql_query('SELECT * FROM foo');
//---
// And, in view
while ($row = mysql_fetch_assoc($result)) {
echo $row['foo'];
}
?
--
?php echo 'Just another PHP saint'; ?
Email: rrjanbiah-at-Y!comBlog: http://rajeshanbiah.blogspot.com/
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups Cake
PHP group.
To post to this group, send email to cake-php@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/cake-php?hl=en
-~--~~~~--~~--~--~---
Re: Model find() to return object; instead of array (?)
On Apr 3, 12:24 pm, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: snip As on that page I would also suggest you look at some of the bahaviors in the Bakery that can help with binding and unbinding data not needed for the current request. Thanks for your ideas; but none of them are helpful as it's about exporting of data Another thing you can make use of for exports is paginate. Yes really :) If you are dumping 600 000 rows at once you could probably run out of memory. By running the export in a few passes are appending the exported file (or wherever the data is going) each time you will use less memory. (sure you can also paginate manually but what fun would that be?) Hmm molding our code for the framework! Anyway, good thought. I often run big-memory and long-time data management from the php-cli using cron or other trigger. The case, I'm talking about is not exclusive of cli php -- ?php echo 'Just another PHP saint'; ? Email: rrjanbiah-at-Y!comBlog: http://rajeshanbiah.blogspot.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Model find() to return object; instead of array (?)
On Apr 3, 3:48 pm, Matias Lespiau [EMAIL PROTECTED] wrote: I think this is the link your a looking for:http://bin.cakephp.org/saved/1237 I'm sorry, I actually meant resource; not object. -- ?php echo 'Just another PHP saint'; ? Email: rrjanbiah-at-Y!comBlog: http://rajeshanbiah.blogspot.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
RE: Model find() to return object; instead of array (?)
Why on earth would you do that?
-Original Message-
From: cake-php@googlegroups.com [mailto:[EMAIL PROTECTED] On Behalf Of R.
Rajesh Jeba Anbiah
Sent: 3. april 2008 14:49
To: Cake PHP
Subject: Re: Model find() to return object; instead of array (?)
On Apr 3, 11:51 am, Grant Cox [EMAIL PROTECTED] wrote:
I'm not sure of any method - but why do you think that all your rows
as arrays will use up more memory than the same amount of data wrapped
in objects? Is it just a pass by reference vs pass by copy issue?
snip
I'm sorry, I goofed it up with the term object; I actually meant
resource. Cake is currently dealing like this:
?php
$result = mysql_query('SELECT * FROM foo');
while ($row = mysql_fetch_assoc($result)) {
$array[] = $row['foo'];
}
//---
// And, in view
foreach($array as $x) {
echo $x;
}
?
I want something like:
?php
$result = mysql_query('SELECT * FROM foo');
//---
// And, in view
while ($row = mysql_fetch_assoc($result)) {
echo $row['foo'];
}
?
--
?php echo 'Just another PHP saint'; ?
Email: rrjanbiah-at-Y!comBlog: http://rajeshanbiah.blogspot.com/
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups Cake
PHP group.
To post to this group, send email to cake-php@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/cake-php?hl=en
-~--~~~~--~~--~--~---
RE: Model find() to return object; instead of array (?)
It would kill all the cakephp filters and magic at best
-Original Message-
From: cake-php@googlegroups.com [mailto:[EMAIL PROTECTED] On Behalf Of R.
Rajesh Jeba Anbiah
Sent: 3. april 2008 14:49
To: Cake PHP
Subject: Re: Model find() to return object; instead of array (?)
On Apr 3, 11:51 am, Grant Cox [EMAIL PROTECTED] wrote:
I'm not sure of any method - but why do you think that all your rows
as arrays will use up more memory than the same amount of data wrapped
in objects? Is it just a pass by reference vs pass by copy issue?
snip
I'm sorry, I goofed it up with the term object; I actually meant
resource. Cake is currently dealing like this:
?php
$result = mysql_query('SELECT * FROM foo');
while ($row = mysql_fetch_assoc($result)) {
$array[] = $row['foo'];
}
//---
// And, in view
foreach($array as $x) {
echo $x;
}
?
I want something like:
?php
$result = mysql_query('SELECT * FROM foo');
//---
// And, in view
while ($row = mysql_fetch_assoc($result)) {
echo $row['foo'];
}
?
--
?php echo 'Just another PHP saint'; ?
Email: rrjanbiah-at-Y!comBlog: http://rajeshanbiah.blogspot.com/
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups Cake
PHP group.
To post to this group, send email to cake-php@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/cake-php?hl=en
-~--~~~~--~~--~--~---
Re: Model find() to return object; instead of array (?)
On Apr 3, 5:58 pm, Christian Winther [EMAIL PROTECTED] wrote: Why on earth would you do that? snip Read the thread above.. the array dumping doesn't work for all the cases--especially on exporting the data. -- ?php echo 'Just another PHP saint'; ? Email: rrjanbiah-at-Y!comBlog: http://rajeshanbiah.blogspot.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Model find() to return object; instead of array (?)
On Thu, Apr 3, 2008 at 9:46 AM, R. Rajesh Jeba Anbiah [EMAIL PROTECTED] wrote: On Apr 3, 5:58 pm, Christian Winther [EMAIL PROTECTED] wrote: Why on earth would you do that? snip Read the thread above.. the array dumping doesn't work for all the cases--especially on exporting the data. If you insist on fighting against Cake's conventions, or insist on not adjusting your application to the realities of Cake's conventions, you are facing an uphill battle. Maybe I'm stupid, but please enlighten me on how having Cake return result sets as an object instead of an array set will help you. I am not understanding how the objects will take up less memory than an array when you have 600,000+ records. -- Chris Hartjes Internet Loudmouth Motto for 2008: Moving from herding elephants to handling snakes... @TheKeyBoard: http://www.littlehart.net/atthekeyboard --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Model find() to return object; instead of array (?)
On Apr 3, 6:51 pm, Chris Hartjes [EMAIL PROTECTED] wrote: On Thu, Apr 3, 2008 at 9:46 AM, R. Rajesh Jeba Anbiah [EMAIL PROTECTED] wrote: On Apr 3, 5:58 pm, Christian Winther [EMAIL PROTECTED] wrote: Why on earth would you do that? snip Read the thread above.. the array dumping doesn't work for all the cases--especially on exporting the data. If you insist on fighting against Cake's conventions, or insist on not adjusting your application to the realities of Cake's conventions, you are facing an uphill battle. Maybe I'm stupid, but please enlighten me on how having Cake return result sets as an object instead of an array set will help you. I am not understanding how the objects will take up less memory than an array when you have 600,000+ records. Like I mentioned above in *this* thread, I wrongly used the term object--but I actually meant resource. Cake's default way of dumping array in controller and iterating in view file, works only for the small data. But, on a typical project that requires exporting of data, it doesn't work as it runs out of memory. This needs fetching the record one-by-one like I mentioned above. BTW, whenever you reply here with your remarks, I thought of asking do you really use CakePHP? SCNR -- ?php echo 'Just another PHP saint'; ? Email: rrjanbiah-at-Y!comBlog: http://rajeshanbiah.blogspot.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Model find() to return object; instead of array (?)
On Thu, Apr 3, 2008 at 2:13 PM, R. Rajesh Jeba Anbiah [EMAIL PROTECTED] wrote: BTW, whenever you reply here with your remarks, I thought of asking do you really use CakePHP? SCNR Chris helps out a lot in the CakePHP community (written articles, mailing list, etc). He has that grumpy attitude, sure, but he knows CakePHP quite well. Your comment is only asking for trouble and not the way to motivate people to reply to your question (except to throw some back). - Gonzalo --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Model find() to return object; instead of array (?)
On Thu, Apr 3, 2008 at 1:13 PM, R. Rajesh Jeba Anbiah [EMAIL PROTECTED] wrote: Cake's default way of dumping array in controller and iterating in view file, works only for the small data. But, on a typical project that requires exporting of data, it doesn't work as it runs out of memory. This needs fetching the record one-by-one like I mentioned above. I understand. Sounds to me like something that exports data like this is not suited to be a action within a controller. Maybe you should look into making this a Cake shell task, as that would let you use all the Cake goodness and be able to run from the command line. Either that or you'll have to run queries using LIMIT and OFFSET to grab what you need one chunk at a time. I don't know any scripting language that handles sucking in 600,000 records via a web page. BTW, whenever you reply here with your remarks, I thought of asking do you really use CakePHP? SCNR For a guy who calls himself Just another PHP saint, that's quite a bold statement. Good thing my life doesn't revolve around worrying about what ignorant people think about me. Yes, I do use CakePHP. Otherwise I'd be wasting my time here. Don't blame me for failing to understand what your problem is. Maybe you should look into how you're asking your questions. -- Chris Hartjes Internet Loudmouth Motto for 2008: Moving from herding elephants to handling snakes... @TheKeyBoard: http://www.littlehart.net/atthekeyboard --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Model find() to return object; instead of array (?)
On Thu, Apr 3, 2008 at 2:12 PM, Christian Winther [EMAIL PROTECTED] wrote: I use cake way for documents and sql data way above 100 megabytes in size I was thinking that unless those records are really small, calling in 60 records via a select is going to be really difficult to do without either running out of memory or execution time. I'd do it in chunks if it were me. -- Chris Hartjes Internet Loudmouth Motto for 2008: Moving from herding elephants to handling snakes... @TheKeyBoard: http://www.littlehart.net/atthekeyboard --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
RE: Model find() to return object; instead of array (?)
I use cake way for documents and sql data way above 100 megabytes in size My guess would be your setup is wrongly configured - rather than pointing to php or cake as the problem. If you really need the resource, just do the mysql_select() and everything yourself, it will use your default mysql connection - and should work out of the box aswell. -Original Message- From: cake-php@googlegroups.com [mailto:[EMAIL PROTECTED] On Behalf Of R. Rajesh Jeba Anbiah Sent: 3. april 2008 19:14 To: Cake PHP Subject: Re: Model find() to return object; instead of array (?) On Apr 3, 6:51 pm, Chris Hartjes [EMAIL PROTECTED] wrote: On Thu, Apr 3, 2008 at 9:46 AM, R. Rajesh Jeba Anbiah [EMAIL PROTECTED] wrote: On Apr 3, 5:58 pm, Christian Winther [EMAIL PROTECTED] wrote: Why on earth would you do that? snip Read the thread above.. the array dumping doesn't work for all the cases--especially on exporting the data. If you insist on fighting against Cake's conventions, or insist on not adjusting your application to the realities of Cake's conventions, you are facing an uphill battle. Maybe I'm stupid, but please enlighten me on how having Cake return result sets as an object instead of an array set will help you. I am not understanding how the objects will take up less memory than an array when you have 600,000+ records. Like I mentioned above in *this* thread, I wrongly used the term object--but I actually meant resource. Cake's default way of dumping array in controller and iterating in view file, works only for the small data. But, on a typical project that requires exporting of data, it doesn't work as it runs out of memory. This needs fetching the record one-by-one like I mentioned above. BTW, whenever you reply here with your remarks, I thought of asking do you really use CakePHP? SCNR -- ?php echo 'Just another PHP saint'; ? Email: rrjanbiah-at-Y!comBlog: http://rajeshanbiah.blogspot.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
RE: Model find() to return object; instead of array (?)
Yeah, I'm using that technique now, but before I cakeified it - the other solution worked rather well too - just the prove the point that its probably a local issue on his setup rather than a limit on cakephp / php -Original Message- From: cake-php@googlegroups.com [mailto:[EMAIL PROTECTED] On Behalf Of Chris Hartjes Sent: 3. april 2008 20:20 To: cake-php@googlegroups.com Subject: Re: Model find() to return object; instead of array (?) On Thu, Apr 3, 2008 at 2:12 PM, Christian Winther [EMAIL PROTECTED] wrote: I use cake way for documents and sql data way above 100 megabytes in size I was thinking that unless those records are really small, calling in 60 records via a select is going to be really difficult to do without either running out of memory or execution time. I'd do it in chunks if it were me. -- Chris Hartjes Internet Loudmouth Motto for 2008: Moving from herding elephants to handling snakes... @TheKeyBoard: http://www.littlehart.net/atthekeyboard --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Model find() to return object; instead of array (?)
We have some large reports for our application - often involving ~250K rows from 8 different tables. The problem is, if your reporting needs to do any kind of collating or comparison, as ours does, you really can't avoid having all the data loaded somewhere. You may be able to do much of it in your database which should stop PHP from running out of memory, but it'll have to be used somewhere. Anyway, my point is that ram is cheap, and if you're doing those kinds of reports the easy solution is just to ensure that you have enough memory available. We set up a second apache instance where each process is allowed a gig of ram, and have a proxy rewrite rule to ensure the reporting requests (of which there are very few) get routed to this second instance. I'm sure we'll eventually find a better way to do it, but this was a simple solution that's holding up fine. On Apr 4, 5:28 am, Christian Winther [EMAIL PROTECTED] wrote: Yeah, I'm using that technique now, but before I cakeified it - the other solution worked rather well too - just the prove the point that its probably a local issue on his setup rather than a limit on cakephp / php -Original Message- From: cake-php@googlegroups.com [mailto:[EMAIL PROTECTED] On Behalf Of Chris Hartjes Sent: 3. april 2008 20:20 To: cake-php@googlegroups.com Subject: Re: Model find() to return object; instead of array (?) On Thu, Apr 3, 2008 at 2:12 PM, Christian Winther [EMAIL PROTECTED] wrote: I use cake way for documents and sql data way above 100 megabytes in size I was thinking that unless those records are really small, calling in 60 records via a select is going to be really difficult to do without either running out of memory or execution time. I'd do it in chunks if it were me. -- Chris Hartjes Internet Loudmouth Motto for 2008: Moving from herding elephants to handling snakes... @TheKeyBoard:http://www.littlehart.net/atthekeyboard --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
