[zeta-dev] Review Request: foo test
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/534/ --- Review request for zetacomponents. Summary --- Test review, please ignore. Diffs - Diff: https://reviews.apache.org/r/534/diff Testing --- Foo test Thanks, Tony
[zeta-dev] Bug with Webdav + Bugfix
Hi all. I wanted to open a Bug at the issue tracker but at some reasons i didn't receive any registration mail. So I'm wirting here. I hope that someone can help me to post my issue and the fix in a correct way or better do it for me. The Problem: On my work with the Webdav component i figured out that every php script using with component crashs with file uploads larger than 1GB. After some server tweaking it was clear that there must be a reason in the component code. I traced the code and found a line like this $properties['body'] = $body . Then it was clear. A string with 1GB was copied into an array and crashes the php module with this operation. The workaround: To avoid large memory usage I went over to use the php tempory storage. So the maximum amout of used ram is at my configuration 2MB and the rest is streamed to disk. To make working with it easier I wrote a wrapper clase and placed it on every part that is reading or writeing the document body. Luckily this is only neccessary at Webdav/src/transport.php and all of my own files. As the wrapper could also be used with other components its located at Base. There is one new file: Base/src/streambuffer.php And two modified: Webdav/src/transport.php Base/src/base_autoload.php A patch file for base and one for webdav are attached. I hope they have the right format. Regards Chriss HDPnet GmbH Erwin-Rohde-Str. 18 69120 Heidelberg Geschaeftsfuehrer: Marc Hermann Registergericht: Mannheim HRB 337012 Sitz: Heidelberg Umsatzsteuer ID Nr.: DE 211 257 470 www.hdpnet.de Diese E-Mail enthaelt vertrauliche und/oder rechtlich geschuetzte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtuemlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail ist nicht gestattet.diff Base/src/base_autoload.php --- Base/src/base_autoload.php 2011-03-30 09:31:11.0 +0200 +++ Base/src/base_autoload.php 2011-03-30 13:19:06.0 +0200 @@ -59,5 +59,6 @@ 'ezcBaseMetaDataTarballReader'= 'Base/metadata/tarball.php', 'ezcBasePersistable' = 'Base/interfaces/persistable.php', 'ezcBaseRepositoryDirectory' = 'Base/structs/repository_directory.php', + 'ezcStreamBuffer' = 'Base/streambuffer.php', ); ? new Base/src/streambuffer.php +++ Base/src/streambuffer.php @@ -0,0 +1,191 @@ +?php +/** + * File containing the ezcStreamBuffer class. + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * License); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + * + * @package Base + * @version //autogentag// + * @license http://www.apache.org/licenses/LICENSE-2.0 Apache License, Version 2.0 + */ +/** + * Class that handles large data streams without memory conflicts + * + * This class uses internaly the temporary memory from php. It stores the specified amount of data + * in the memory of the system (RAM). If there is more data, it will be stored on the persistent memory (HDD) + * The user can pass data as string and receive it as string. Also the user can pass streams to copy data from or to. + * + * @version //autogentag// + * @package Base + */ +class ezcStreamBuffer{ + + private $handle = null; + + /** +* +* Constructor +* +* If no value is provided then 2MB of data will be stored in memory and the rest on hard drive. +* +* @param int $maxValue - Amount of data in MB that will resist directly in memory +* @return ezcStreamBuffer +* +* @author Christian Weber we...@hdpnet.de +*/ + public function __construct($maxValue = 2){ + + $maxMBs = $maxValue * 1024 * 1024; + + $this-handle = fopen(php://temp/maxmemory:$maxMBs, 'r+'); + if(FALSE === $this-handle)$this-handle = null; + } + + /** +* +* Destructor +* +* Frees the memory and closes the stream handle +* +* @return void +* +* @author Christian Weber we...@hdpnet.de +*/ +
Re: [zeta-dev] Bug with Webdav + Bugfix
Hi Christian, On Wed, Mar 30, 2011 at 2:40 PM, Christian Weber webe...@hdpmail.de wrote: Hi all. I wanted to open a Bug at the issue tracker but at some reasons i didn't receive any registration mail. This is weird. I filed an issue on your behalf : - https://issues.apache.org/jira/browse/ZETACOMP-70 However I did not attach your patches to avoid any legal issue. I would recommend you to upload the patches yourself (once you can login into JIRA) and to grant license to ASF for inclusion in our code base. Best Regards, -- Jérôme Renard http://39web.fr | http://jrenard.info | http://twitter.com/jeromerenard
Re: [zeta-dev] Bug with Webdav + Bugfix
Hi Christian, first of all, thanks for your contributions to Zeta. On 03/30/2011 02:40 PM, Christian Weber wrote: I wanted to open a Bug at the issue tracker but at some reasons i didn't receive any registration mail. So I'm wirting here. I hope that someone can help me to post my issue and the fix in a correct way or better do it for me. The Problem: On my work with the Webdav component i figured out that every php script using with component crashs with file uploads larger than 1GB. After some server tweaking it was clear that there must be a reason in the component code. I traced the code and found a line like this $properties['body'] = $body . Then it was clear. A string with 1GB was copied into an array and crashes the php module with this operation. The workaround: To avoid large memory usage I went over to use the php tempory storage. So the maximum amout of used ram is at my configuration 2MB and the rest is streamed to disk. To make working with it easier I wrote a wrapper clase and placed it on every part that is reading or writeing the document body. Luckily this is only neccessary at Webdav/src/transport.php and all of my own files. As the wrapper could also be used with other components its located at Base. There is one new file: Base/src/streambuffer.php And two modified: Webdav/src/transport.php Base/src/base_autoload.php A patch file for base and one for webdav are attached. I hope they have the right format. This issue was on my internal todo for some time now. Thanks a lot for looking into it. The idea of your fix looks generally correct to me. However, we cannot simply change the content of the request body, since it would break BC. Instead we basically have 2 choices for implementing handling of large files: 1. Adding an option to switch on stream based body handling. 2. Abstracting body extraction into a dedicated object to allow dependency injection for handling of large bodies. I'm not yet sure, which way to go. We should discuss this on the list. In addition, some of your code does not comply to our coding guidelines [1]. Would you mind updating it? Cheers, Toby [1] http://incubator.apache.org/zetacomponents/community/implementation.html -- Tobias Schlitthttp://schlitt.infoGPG Key: 0xC462BC14 Want to hire me? Need quality assurance?http://qafoo.com eZ Components are Zeta Components now! http://bit.ly/9S7zbn
Re: [zeta-dev] Bug with Webdav + Bugfix
Hi Jerome, On 03/30/2011 10:00 PM, Jerome Renard wrote: On Wed, Mar 30, 2011 at 5:55 PM, Tobias Schlitt tob...@schlitt.info wrote: The idea of your fix looks generally correct to me. However, we cannot simply change the content of the request body, since it would break BC. Instead we basically have 2 choices for implementing handling of large files: 1. Adding an option to switch on stream based body handling. 2. Abstracting body extraction into a dedicated object to allow dependency injection for handling of large bodies. I'm not yet sure, which way to go. We should discuss this on the list. Option #1 should be enough I think. This idea of using a stream is really interesting. I hardly see any use case for option #2 actually. I think we should actually do a mixture, like setting a body extractor object in the options, which defaults to setting the body as the veriable contents. Regards, Toby -- Tobias Schlitthttp://schlitt.infoGPG Key: 0xC462BC14 Want to hire me? Need quality assurance?http://qafoo.com eZ Components are Zeta Components now! http://bit.ly/9S7zbn