Thilina Gunarathne wrote:
Wow Wow... pretty impressive.. Way to go guys... Keep up the good work..
It'll be great if we can give it a try to benchmark our stuff against
WCF ,Xfire , etc too...
Have you already implemented the Base64 conversion to happen
automatically when the binary is not optimized.. It's critical to have
it before we claim about the XOP/MTOM feature.
Base64 is a TODO. There are some other issues reported in Jira as well.
Samisa...
~Thilina
On 5/13/06, *Kapila Dissanayake* <[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>> wrote:
Hi ,
I did another modification to improve the messege transfer speed.
http_transport_utils.c was modified. (Buffer size was changed from
1024 to 1024 * 1024)
Now the transfer speed has been changed significantly. Please see
the table below.
The attached file includes the svn diff of the change.
Thank You
Kapila
(File Size) vs (Time Taken in milli seconds) to send binary
attachments
Environment
----------
Processor - Intel Celeron M 1.40GHz
RAM - 256 MB
OS - Fedora Core 4.
Java Server - axis2 Java
C Server - axis2
mtom C and java test clients were used
http_transport_utils.c was modified. (Buffer size was changed from
1024 to 1024 * 1024)
Time Taken in milli seconds
File Size (kb) Java Server Java Server
C Server C
Server C Server C Server C Server C
Server C Server
C Server
Java Client (ms) C Client (ms) C
Client
(ms) Java Client (ms)
*data_handler.c Buffer Size 1024 *
*data_handler.c 'Buffer Size 1024 X 1024 * *Using stat*
*http_transport_utils.c 'Buffer Size 1024 X 1024 * * * *Buffer
Size '1024 * *Buffer Size '1024 X 1024 * *Using stat*
*http_transport_utils.c 'Buffer Size 1024 X 1024 *
----------------- ----------------- -----------------
----------------- ----------------- -----------------
----------------- ----------------- -----------------
--------------------------------- -----------------
----------------- ----------------- -----------------
---------------------------------
13.8 147 158 73 1884 78
76 188 143 136 136
28.2 515 488 78 1912 78
78 222 413 364 387
37.8 538 694 131 1917 81
81 232 474 377 396
132 571 807 212 2177 102
83 263 420 381 432
154 547 672 239 2215 117
84 347 389 383 517
165 688 683 397 2380 205
85 364 400 393 602
561 1340 1024 995 2537 777
100 637 899 775 766
762 1766 1336 1903 2890 1458
111 1248 1218
1051 1055
1162 3464 1970 4859 3602 3405
135 1702
1718 1712 1092
1761 4666 3274 10928 8028 7927
188 2780 3249
2691 2596
2081 6777 2993 15066 11008 10864
219 3595
3281 2859 3185
4241 23038 Could not transfer (Java client Broken)
52853 33351 32957 519 16234
115777 7991 6499
On 5/12/06, *Samisa Abeysinghe* <[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>> wrote:
BTW,
If we are going to use chunking at some point in time, then
the data
handler logic for reading file too has to change to keep track
of the
amount of chunks already read, and current position of read
pointer.
There is no provision for such reading in the code as of now,
neither in
the data handler nor in the mime_output that uses the data
handler API.
So there is considerable amount of work on the part of the
data hander
and the places that uses the API of data handler to read to
turn on
chunking as far as I can see.
Thanks,
Samisa...
Samisa Abeysinghe wrote:
> Nabeel wrote:
>
>> Kapila,
>> With your latest changes you always read the file in one go
>> (i.e. not in chunks) and you don't need a loop. The
attached compact
>> code does the same thing.
>>
>> Suggestion:
>> When we need to attach *very* large file, we may not be
keep the
>> whole file in memory. IMO, this where chunk reading is
really useful.
>> If you are to implement this, you'll have to change the
code from
>> data_handler -> mime_body_part -> mime_output to om_output
so that
>> you'll keep only chunk_size amount of bytes in memory at
any given time.
>
>
> Chunking is a very good idea, in fact that is the perfect
solution for
> large attachments.
> However, AFAIK, we have chunking capability only on the
client side
> transport, not on the server side transport.
> Hence, we got to implement chunking capability into simple
axis server
> transport first. (May be we can raise a Jira on this). Or we
have to
> see if we can piggyback on Apache for chunking.
> Secondly, we got to research and find out the ideal chunk
size. As we
> have seen through the results sent by Kapila, too small a
chunk size
> makes too large file transmission very slow and too large
chunk size
> makes small file transmission very slow. Best is to not use
chunking
> for small files and use chunking, with a reasonable chunk
size, for
> large files. So we have several parameters to figure out.
> i. Max size we can read a file in one go without chunking.
> ii. When chunking is turned on, ideal chunk size.
> As I mentioned earlier, we need some research around this to
decide on
> the figures.
>
> Thanks,
> Samisa...
>
>>
>> -Nabeel
>>
>> Samisa Abeysinghe wrote:
>>
>>> Cool, so there sure is an improvement when using stat on
read side
>>> of data handler. Hence lets use this solution.
>>> Still we need to figure out what goes wrong with large
files beyond 1M
>>>
>>> Samisa...
>>>
>>> Kapila Dissanayake wrote:
>>>
>>>> Hi
>>>>
>>>> I did some modification to the data handler. Insted of
using fixed
>>>> buffer size to read files
>>>> 'stat' is used.
>>>>
>>>> Please see the perfomance details.
>>>>
>>>> Thank You
>>>>
>>>> Kapila
>>>>
>>>>
>>>>
Environment ---------- Processor -
Intel
>>>> Celeron M 1.40GHz
>>>> RAM - 256 MB OS - Fedora Core 4. Java
Server -
>>>> axis2 Java C Server - axis2 mtom C and
java test
>>>> clients were used (in axis2 c data_handler.c,
different buffer
>>>> sizes were used to read files)
>>>> Time Taken in milli seconds
>>>> File Size (kb) Java Server (ms) Java Server
>>>> (ms) C Server (ms) C Server (ms)
>>>> Java Client (ms) C Client (ms) C
>>>> Client (ms) Java Client (ms)
>>>> *Buffer Size 1024
* *Buffer
>>>> Size 1024 X 1024 * *Using stat* * * *Buffer Size
'1024
>>>> * *Buffer Size '1024 X 1024 * *Using stat *
>>>> ----------------- -----------------
-----------------
>>>> ----------------- ----------------- -----------------
>>>> ----------------- ----------------- -----------------
>>>> ----------------- -----------------
-----------------
>>>> -----------------
>>>> 13.8 147 158 73 1884
>>>> 78 188 143 136
>>>> 28.2 515 488 78 1912
>>>> 78 222 413 364
>>>> 37.8 538 694 131 1917
>>>> 81 232 474 377
>>>> 132 571 807 212 2177
>>>> 102 263 420 381
>>>> 154 547 672 239 2215
>>>> 117 347 389 383
>>>> 165 688 683 397 2380
>>>> 205 364 400 393
>>>> 561 1340 1024 995 2537
>>>> 777 637 899 775
>>>> 762 1766 1336 1903 2890
>>>> 1458 1248 1218 1051
>>>> 1162 3464 1970 4859 3602
>>>> 3405 1702 1718 1712
>>>> 1761 4666 3274 10928 8028
>>>> 7927 2780 3249 2691
>>>> 2081 6777 2993 15066 11008
>>>> 10864 3595 3281 2859
>>>> 4241 23038 Could not transfer (Java
client
>>>> Broken) 52853 33351 32957 16234
>>>> 115777 7991
>>>>
>>>>
>>>>
>>>>
>>>> On 5/10/06, *Samisa Abeysinghe* <
[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>
>>>> <mailto: [EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>>> wrote:
>>>>
>>>> Well, from this I am certain that we should not
change the
>>>> buffer size
>>>> to 1024*1024, rather keep it at 1024.
>>>> Please try with stat, using a dynamic buffer size
based on the
>>>> file
>>>> size, my gut feel is that it is going to be the best
solution.
>>>>
>>>> Thanks,
>>>> Samisa...
>>>>
>>>> Kapila Dissanayake wrote:
>>>>
>>>> >
>>>> > Hi,
>>>> >
>>>> > I did some testing agian with to measure the binary
file
>>>> transfer
>>>> > times. In axis2 c data_handler.c, different buffer
sizes
>>>> (1024 and
>>>> > 1024*1024) were used to read files.
>>>> >
>>>> > I think using fixed size buffers to read binary
files in axis2-C
>>>> has
>>>> > been affected in some cases to slow down the transfer.
>>>> >
>>>> > Thanks
>>>> >
>>>> > Kapila
>>>> >
>>>> > Please see the following test results
>>>> >
>>>> > Environment
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > ----------
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > Processor - Intel Celeron M 1.40GHz
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > RAM - 256 MB
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > OS - Fedora Core 4.
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > Java Server - axis2 Java
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > C Server - axis2
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > mtom C and java test clients were used
...
[Message clipped]
--
"May the SourcE be with u"
http://webservices.apache.org/~thilina/
<http://webservices.apache.org/%7Ethilina/>
http://thilinag.blogspot.com/
http://www.bloglines.com/blog/Thilina
