Hi all.
I wanted to open a Bug at the issue tracker but at some reasons i didn't
receive any registration mail.
So I'm wirting here. I hope that someone can help me to post my issue
and the fix in a correct way or better do it for me.
The Problem:
On my work with the Webdav component i figured out that every php script
using with component crashs with file uploads larger than 1GB. After
some server tweaking it was clear that there must be a reason in the
component code.
I traced the code and found a line like this " $properties['body'] =
$body ".
Then it was clear. A string with 1GB was copied into an array and
crashes the php module with this operation.
The workaround:
To avoid large memory usage I went over to use the php tempory storage.
So the maximum amout of used ram is at my configuration 2MB and the rest
is streamed to disk. To make working with it easier I wrote a wrapper
clase and placed it on every part that is reading or writeing the
document body. Luckily this is only neccessary at
Webdav/src/transport.php and all of my own files.
As the wrapper could also be used with other components its located at Base.
There is one new file: Base/src/streambuffer.php
And two modified: Webdav/src/transport.php
Base/src/base_autoload.php
A patch file for base and one for webdav are attached. I hope they have
the right format.
Regards Chriss
HDPnet GmbH
Erwin-Rohde-Str. 18
69120 Heidelberg
Geschaeftsfuehrer: Marc Hermann
Registergericht: Mannheim HRB 337012
Sitz: Heidelberg
Umsatzsteuer ID Nr.: DE 211 257 470
www.hdpnet.de
Diese E-Mail enthaelt vertrauliche und/oder rechtlich geschuetzte
Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail
irrtuemlich erhalten haben, informieren Sie bitte sofort den Absender und
vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte
Weitergabe dieser Mail ist nicht gestattet.
diff Base/src/base_autoload.php
--- Base/src/base_autoload.php 2011-03-30 09:31:11.000000000 +0200
+++ Base/src/base_autoload.php 2011-03-30 13:19:06.000000000 +0200
@@ -59,5 +59,6 @@
'ezcBaseMetaDataTarballReader' =>
'Base/metadata/tarball.php',
'ezcBasePersistable' =>
'Base/interfaces/persistable.php',
'ezcBaseRepositoryDirectory' =>
'Base/structs/repository_directory.php',
+ 'ezcStreamBuffer' =>
'Base/streambuffer.php',
);
?>
new Base/src/streambuffer.php
+++ Base/src/streambuffer.php
@@ -0,0 +1,191 @@
+<?php
+/**
+ * File containing the ezcStreamBuffer class.
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ * @package Base
+ * @version //autogentag//
+ * @license http://www.apache.org/licenses/LICENSE-2.0 Apache License, Version
2.0
+ */
+/**
+ * Class that handles large data streams without memory conflicts
+ *
+ * This class uses internaly the temporary memory from php. It stores the
specified amount of data
+ * in the memory of the system (RAM). If there is more data, it will be stored
on the persistent memory (HDD)
+ * The user can pass data as string and receive it as string. Also the user
can pass streams to copy data from or to.
+ *
+ * @version //autogentag//
+ * @package Base
+ */
+class ezcStreamBuffer{
+
+ private $handle = null;
+
+ /**
+ *
+ * Constructor
+ *
+ * If no value is provided then 2MB of data will be stored in memory
and the rest on hard drive.
+ *
+ * @param int $maxValue - Amount of data in MB that will resist
directly in memory
+ * @return ezcStreamBuffer
+ *
+ * @author Christian Weber <we...@hdpnet.de>
+ */
+ public function __construct($maxValue = 2){
+
+ $maxMBs = $maxValue * 1024 * 1024;
+
+ $this->handle = fopen("php://temp/maxmemory:$maxMBs", 'r+');
+ if(FALSE === $this->handle)$this->handle = null;
+ }
+
+ /**
+ *
+ * Destructor
+ *
+ * Frees the memory and closes the stream handle
+ *
+ * @return void
+ *
+ * @author Christian Weber <we...@hdpnet.de>
+ */
+ public function __destruct(){
+ if(null == $this->handle);
+ else {
+ fclose($this->handle);
+ }
+ }
+
+ /**
+ *
+ * Stores the data of the given stream in the temporary memory
+ *
+ * @param resource $stream - Stream to read data from
+ * @return void
+ *
+ * @author Christian Weber <we...@hdpnet.de>
+ */
+ public function readFromStream($stream = null){
+
+ if(null == $stream || null == $this->handle)return;
+
+ rewind($this->handle);
+ rewind($stream);
+
+ while ( $data = fread($stream,1024) )
+ {
+ if(FALSE === $data)break;
+ fwrite($this->handle,$data);
+ }
+
+ }
+
+ /**
+ *
+ * Stores the data from the given string
+ *
+ * @param string $data
+ * @param int $bytes
+ * @return void
+ *
+ * @author Christian Weber <we...@hdpnet.de>
+ */
+ public function readFromString($data = '',$bytes = 0){
+ if(null == $this->handle)return;
+
+ rewind($this->handle);
+ fwrite($this->handle,$data,$bytes);
+ }
+
+ /**
+ *
+ * Writes the stored data to the given stream.
+ *
+ * Existing data in the target stream will be overwritten
+ *
+ * @param resource $stream
+ * @return void
+ *
+ * @author Christian Weber <we...@hdpnet.de>
+ */
+ public function writeToStream($stream = null){
+ if(null == $stream || null == $this->handle)return;
+
+ rewind($this->handle);
+ rewind($stream);
+ while ( $data = fread($this->handle,1024) )
+ {
+ if(FALSE === $data)break;
+ fwrite($stream,$data,1024);
+ }
+ }
+
+ /**
+ *
+ * Returens the stored data as string
+ *
+ * @return string
+ *
+ * @author Christian Weber <we...@hdpnet.de>
+ */
+ public function getAsString(){
+ if(null == $this->handle)return false;
+ $return = '';
+
+ rewind($this->handle);
+ while ( $data = fread($this->handle,1024) )
+ {
+ $return .= $data;
+ }
+
+ return $return;
+ }
+
+ /**
+ *
+ * Returns the internal used stream handle
+ *
+ * @return resource
+ *
+ * @author Christian Weber <we...@hdpnet.de>
+ */
+ public function getStream(){
+ if(null == $this->handle)return false;
+
+ rewind($this->handle);
+ return $this->handle;
+ }
+
+ /**
+ *
+ * Closes the current stream and replace it with the given stream
+ *
+ * @param resource $stream
+ * @return void
+ *
+ * @author Christian Weber <we...@hdpnet.de>
+ */
+ public function setStream($stream){
+ if(null == $this->handle);
+ else
fclose($this->handle);
+
+ $this->handle = $stream;
+ }
+}diff Webdav/src/transport.php
--- Webdav/src/transport.php 2011-03-30 09:32:04.000000000 +0200
+++ Webdav/src/transport.php 2011-03-30 13:11:56.000000000 +0200
@@ -189,7 +189,10 @@
{
$body = $this->retrieveBody();
$path = $this->retrievePath( $uri );
-
+
+ if('PUT' == $_SERVER['REQUEST_METHOD']);
+ else
$body = $body->getAsString();
+
if ( isset( self::$parsingMap[$_SERVER['REQUEST_METHOD']] ) )
{
try
@@ -364,20 +367,15 @@
* usefull to be overriden during inheritence to filter the body of
* missbehaving WebDAV clients.
*
- * @return string The request body.
+ * @return ezcStreamBuffer The request body.
*/
protected function retrieveBody()
{
- $body = '';
+ $body = new ezcStreamBuffer();
$in = fopen( 'php://input', 'r' );
- while ( $data = fread( $in, 1024 ) )
- {
- // This line is untestable, since it reads from STDIN and during
- // testing there is no input to read.
- // @codeCoverageIgnoreStart
- $body .= $data;
- }
+ $body->setStream($in);
+
// @codeCoverageIgnoreEnd
return $body;
}
@@ -549,7 +547,19 @@
header( $output->status );
// Content-Length header automatically send
- echo $output->body;
+ if($output->body instanceof ezcStreamBuffer){
+
+ $data = false;
+ $stream = $output->body->getStream();
+
+ while ( $data = fread($stream,1024) )
+ {
+ if(FALSE === $data)break;
+ echo $data;
+ }
+ }else{
+ echo $output->body;
+ }
}
/*
