Hello,
I have found a bug in my code: When Range: variable was present the server
response code must be 206 for deciding the entire file size. If it is 200
(in the case of PHP for instance) then the web server content-length is the
size for the entire file. This was causing the cache issue.
The second one: In my function below,
unsigned int __fastcall ReverseProxyClientClass::compressFullStream(TStream
*stream, __int64 FRequestRangeStart, __int64 contentLength)
contentLength was not being taken into consideration. Here is a function I
wrote for ICSZlibHigh which takes that into consideration:
procedure ZlibCompressStreamEx(InStream, OutStream: TStream;
NumLevel: Integer; StreamType : TZStreamType; offset : int64;
length : int64);
const
//64 KB buffer
BufSize = 65536;
var
strm : z_stream;
InBuf, OutBuf : PAnsiChar;
UseInBuf, UseOutBuf : boolean;
LastOutCount : integer;
Finished : boolean;
Cancel: boolean;
Totcount: int64;
toRead: integer;
exitLoop: boolean;
procedure WriteOut;
begin
if UseOutBuf then
begin
if LastOutCount > 0 then OutStream.Write(OutBuf^, LastOutCount -
strm.avail_out);
strm.avail_out := BufSize;
strm.next_out := OutBuf;
end
else
begin
if (strm.avail_out = 0) then ExpandStream(OutStream, OutStream.Size +
BufSize);
OutStream.Seek(LastOutCount - strm.avail_out, soFromCurrent);
strm.next_out := DMAOfStream(OutStream, strm.avail_out);
//because we can't really know how much resize is increasing!
end;
LastOutCount := strm.avail_out;
end;
begin
FillChar(strm, sizeof(strm), 0);
InBuf := nil;
OutBuf := nil;
LastOutCount := 0;
Totcount := 0;
exitLoop := false;
InStream.Seek(offset, soFromBeginning);
strm.next_in := DMAOfStream(InStream, strm.avail_in);
UseInBuf := strm.next_in = nil;
if UseInBuf then
GetMem(InBuf, BufSize);
UseOutBuf := not (CanResizeDMAStream(OutStream));
if UseOutBuf then GetMem(OutBuf, BufSize);
ZlibCCheck(deflateInitEx(strm, NumLevel, StreamType)); { V6.01 }
try
repeat
if strm.avail_in = 0 then
begin
if UseInBuf then
begin
toRead := BufSize;
if totCount + BufSize > length then
begin
toRead := length - totCount;
exitLoop := true;
end;
strm.avail_in := InStream.Read(InBuf^, toRead);
strm.next_in := InBuf;
end;
if strm.avail_in = 0 then break;
if exitLoop then break;
end;
if strm.avail_out = 0 then WriteOut;
ZlibCCheck(deflate(strm, Z_NO_FLUSH));
inc(Totcount, strm.avail_in); { V6.01 keep track of data read }
until false;
repeat
if strm.avail_out = 0 then WriteOut;
Finished := ZlibCCheck(deflate(strm, Z_FINISH)) = Z_STREAM_END;
WriteOut;
until Finished;
if not UseOutBuf then
begin
//truncate when using direct output
OutStream.Size := OutStream.Position;
end;
//adjust position of the input stream
if UseInBuf then
//seek back when unused data
InStream.Seek(-strm.avail_in, soFromCurrent)
else
//simple seek
InStream.Seek(strm.total_in, soFromCurrent);
ZlibCCheck(deflateEnd(strm));
finally
if InBuf <> nil then FreeMem(InBuf);
if OutBuf <> nil then FreeMem(OutBuf);
end;
end;
Could TeamICS please add this to the SVN?
Best Regards,
SZ
On Mon, May 2, 2011 at 18:24, Fastream Technologies <ga...@fastream.com>wrote:
> Hello,
>
> I noticed contentlength is not taken into consideration in the
> compressFullStream() function. We want to sponsor a new function in ICS
> Zlib... for this for $50. I can pay with Paypal. I have spoken with Francois
> and we will donate the code to ICS. I suspect there may be another issue
> with our application which I will try to isolate.
>
> Best Regards,
>
> SZ
> On Mon, May 2, 2011 at 16:30, Fastream Technologies <ga...@fastream.com>wrote:
>
>> This is my function:
>>
>> void __fastcall ReverseProxyClientClass::prepareCompressedStream(void)
>> {
>> if(!compressedStream)
>> compressedStream = new TMemoryStream();
>> else
>> compressedStream->Clear();
>> }
>>
>> //---------------------------------------------------------------------------
>> unsigned int __fastcall
>> ReverseProxyClientClass::compressFullStream(TStream *stream, __int64
>> FRequestRangeStart, __int64 contentLength)
>> {
>> prepareCompressedStream();
>> if(!stream->Size)
>> return 0;
>> stream->Seek(FRequestRangeStart, soBeginning);
>> try
>> {
>> ZlibCompressStreamEx(stream, compressedStream,
>> (TCompressionLevel)compressionType, zsGZip, true);
>> }
>> catch(...)
>> {
>> return 0;
>> }
>> return compressedStream->Size;
>> }
>>
>> //---------------------------------------------------------------------------
>>
>> Do you guys see anything wrong with this code? Seems that some internal
>> ZLib buffer is causing this problem to me. Please help as I have been
>> working on this for weeks now...
>>
>> SZ
>> On Mon, May 2, 2011 at 16:16, Fastream Technologies
>> <ga...@fastream.com>wrote:
>>
>>> Hello,
>>>
>>> When we proxy our PHP home page at http://www.iqproxyserver.com, only
>>> with GZip compression on proxy, after 1 or 2 days of running it starts to
>>> produce pages with 7 copies of the same HTML!
>>>
>>> The original page size is 23829 Bytes. When compressed correctly it
>>> outputs 6356 Bytes. When this issue happens, it is 142974 (uncompressed 7
>>> pages) -> 9573Bytes (compressed 7 pages).
>>> The web server is IIS7 + PHP 5.3.6 (non-thread-safe). Seems to me that
>>> issue is in our application but have no clue where! Any idea?
>>>
>>> Best Regards,
>>>
>>> SZ
>>>
>>
>
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
