[jira] [Updated] (THRIFT-1240) TBinarySerializer.php invalid serialization due to TBufferTransport not flushing last chunk of data into TMemoryBuffer
[
https://issues.apache.org/jira/browse/THRIFT-1240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marimuthu Ponnambalam updated THRIFT-1240:
--
Attachment: TBinarySerializer.php
Patch: Fixed for invalid serialization
> TBinarySerializer.php invalid serialization due to TBufferTransport not
> flushing last chunk of data into TMemoryBuffer
> --
>
> Key: THRIFT-1240
> URL: https://issues.apache.org/jira/browse/THRIFT-1240
> Project: Thrift
> Issue Type: Bug
> Components: PHP - Library
>Affects Versions: 0.6.1
> Environment: Cent OS and Mac OS 10.5
>Reporter: Marimuthu Ponnambalam
>Priority: Critical
> Labels: TBinarySerializer, invalid, serialization
> Fix For: 0.6.1
>
> Attachments: TBinarySerializer.php, simple.thrift, testsimple.php
>
>
> Simple Serialization errors out, due to the TBufferTransport not flushing the
> data into TMemoryBuffer after $object->write($protocol) in TBinarySerializer.
> Please find the error output of my test program below. It fixed after I added
> the following line on the serialize method in TBinarySerializer.
> $protocol->getTransport()->flush();
> This because of TBinaryProtocolAccelerated internally wraps TMemoryBuffer
> within TBufferTransport in its constructor.
> TBufferTransport write only if the wbuf_ is full (512 bytes) into
> TMemoryBuffer. Hence data smaller than 512 will not get written into
> TMemoryBuffer. While getting serialized data from TMemoryBuffer it might not
> have the last chunk of data which is less than 512 bytes.
> This outputs invalid serialized data
> TestSchema$ php testsimple.php
> Serialized .
> mari
> Fatal error: Uncaught exception 'TTransportException' with message
> 'TMemoryBuffer: Could not read 512 bytes from buffer.' in
> /Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/transport/TMemoryBuffer.php
> on line 58
> TTransportException: TMemoryBuffer: Could not read 512 bytes from buffer. in
> /Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/transport/TMemoryBuffer.php
> on line 58
> Call Stack:
> 0.0003 636976 1. {main}()
> /Users/mponnambalam/work/eclipse/workspace/TestSchema/testsimple.php:0
> 0.00431509728 2. TBinarySerializer::deserialize()
> /Users/mponnambalam/work/eclipse/workspace/TestSchema/testsimple.php:47
> 0.00431511808 3. thrift_protocol_read_binary()
> /Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/protocol/TBinarySerializer.php:62
> thrift_protoco_read_binary disabled ( I commented extension in
> thrift_protocol.ini)
>
> TestSchema$ php testsimple.php
> Serialized .
> Fatal error: Uncaught exception 'TTransportException' with message
> 'TMemoryBuffer: Could not read 1 bytes from buffer.' in
> /Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/transport/TMemoryBuffer.php
> on line 58
> TTransportException: TMemoryBuffer: Could not read 1 bytes from buffer. in
> /Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/transport/TMemoryBuffer.php
> on line 58
> Call Stack:
> 0.0003 636976 1. {main}()
> /Users/mponnambalam/work/eclipse/workspace/TestSchema/testsimple.php:0
> 0.00411509944 2. TBinarySerializer::deserialize()
> /Users/mponnambalam/work/eclipse/workspace/TestSchema/testsimple.php:47
> 0.00411512248 3. Person->read()
> /Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/protocol/TBinarySerializer.php:66
> 0.00411512472 4. TBinaryProtocol->readFieldBegin()
> /Users/mponnambalam/work/eclipse/workspace/TestSchema/gen-php/simple/simple_types.php:45
> 0.00411512472 5. TBinaryProtocol->readByte()
> /Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/protocol/TBinaryProtocol.php:231
> 0.00411512520 6. TBufferedTransport->readAll()
> /Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/protocol/TBinaryProtocol.php:283
> 0.00411512568 7. TTransport->readAll()
> /Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/transport/TBufferedTransport.php:109
> 0.00411512744 8. TMemoryBuffer->read()
> /Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/transport/TTransport.php:87
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (THRIFT-1240) TBinarySerializer.php invalid serialization due to TBufferTransport not flushing last chunk of data into TMemoryBuffer
[
https://issues.apache.org/jira/browse/THRIFT-1240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marimuthu Ponnambalam updated THRIFT-1240:
--
Attachment: testsimple.php
Test file to for simple IDL
> TBinarySerializer.php invalid serialization due to TBufferTransport not
> flushing last chunk of data into TMemoryBuffer
> --
>
> Key: THRIFT-1240
> URL: https://issues.apache.org/jira/browse/THRIFT-1240
> Project: Thrift
> Issue Type: Bug
> Components: PHP - Library
>Affects Versions: 0.6.1
> Environment: Cent OS and Mac OS 10.5
>Reporter: Marimuthu Ponnambalam
>Priority: Critical
> Labels: TBinarySerializer, invalid, serialization
> Fix For: 0.6.1
>
> Attachments: simple.thrift, testsimple.php
>
>
> Simple Serialization errors out, due to the TBufferTransport not flushing the
> data into TMemoryBuffer after $object->write($protocol) in TBinarySerializer.
> Please find the error output of my test program below. It fixed after I added
> the following line on the serialize method in TBinarySerializer.
> $protocol->getTransport()->flush();
> This because of TBinaryProtocolAccelerated internally wraps TMemoryBuffer
> within TBufferTransport in its constructor.
> TBufferTransport write only if the wbuf_ is full (512 bytes) into
> TMemoryBuffer. Hence data smaller than 512 will not get written into
> TMemoryBuffer. While getting serialized data from TMemoryBuffer it might not
> have the last chunk of data which is less than 512 bytes.
> This outputs invalid serialized data
> TestSchema$ php testsimple.php
> Serialized .
> mari
> Fatal error: Uncaught exception 'TTransportException' with message
> 'TMemoryBuffer: Could not read 512 bytes from buffer.' in
> /Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/transport/TMemoryBuffer.php
> on line 58
> TTransportException: TMemoryBuffer: Could not read 512 bytes from buffer. in
> /Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/transport/TMemoryBuffer.php
> on line 58
> Call Stack:
> 0.0003 636976 1. {main}()
> /Users/mponnambalam/work/eclipse/workspace/TestSchema/testsimple.php:0
> 0.00431509728 2. TBinarySerializer::deserialize()
> /Users/mponnambalam/work/eclipse/workspace/TestSchema/testsimple.php:47
> 0.00431511808 3. thrift_protocol_read_binary()
> /Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/protocol/TBinarySerializer.php:62
> thrift_protoco_read_binary disabled ( I commented extension in
> thrift_protocol.ini)
>
> TestSchema$ php testsimple.php
> Serialized .
> Fatal error: Uncaught exception 'TTransportException' with message
> 'TMemoryBuffer: Could not read 1 bytes from buffer.' in
> /Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/transport/TMemoryBuffer.php
> on line 58
> TTransportException: TMemoryBuffer: Could not read 1 bytes from buffer. in
> /Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/transport/TMemoryBuffer.php
> on line 58
> Call Stack:
> 0.0003 636976 1. {main}()
> /Users/mponnambalam/work/eclipse/workspace/TestSchema/testsimple.php:0
> 0.00411509944 2. TBinarySerializer::deserialize()
> /Users/mponnambalam/work/eclipse/workspace/TestSchema/testsimple.php:47
> 0.00411512248 3. Person->read()
> /Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/protocol/TBinarySerializer.php:66
> 0.00411512472 4. TBinaryProtocol->readFieldBegin()
> /Users/mponnambalam/work/eclipse/workspace/TestSchema/gen-php/simple/simple_types.php:45
> 0.00411512472 5. TBinaryProtocol->readByte()
> /Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/protocol/TBinaryProtocol.php:231
> 0.00411512520 6. TBufferedTransport->readAll()
> /Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/protocol/TBinaryProtocol.php:283
> 0.00411512568 7. TTransport->readAll()
> /Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/transport/TBufferedTransport.php:109
> 0.00411512744 8. TMemoryBuffer->read()
> /Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/transport/TTransport.php:87
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (THRIFT-1240) TBinarySerializer.php invalid serialization due to TBufferTransport not flushing last chunk of data into TMemoryBuffer
[
https://issues.apache.org/jira/browse/THRIFT-1240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marimuthu Ponnambalam updated THRIFT-1240:
--
Attachment: simple.thrift
Thrift IDL for simple test
> TBinarySerializer.php invalid serialization due to TBufferTransport not
> flushing last chunk of data into TMemoryBuffer
> --
>
> Key: THRIFT-1240
> URL: https://issues.apache.org/jira/browse/THRIFT-1240
> Project: Thrift
> Issue Type: Bug
> Components: PHP - Library
>Affects Versions: 0.6.1
> Environment: Cent OS and Mac OS 10.5
>Reporter: Marimuthu Ponnambalam
>Priority: Critical
> Labels: TBinarySerializer, invalid, serialization
> Fix For: 0.6.1
>
> Attachments: simple.thrift
>
>
> Simple Serialization errors out, due to the TBufferTransport not flushing the
> data into TMemoryBuffer after $object->write($protocol) in TBinarySerializer.
> Please find the error output of my test program below. It fixed after I added
> the following line on the serialize method in TBinarySerializer.
> $protocol->getTransport()->flush();
> This because of TBinaryProtocolAccelerated internally wraps TMemoryBuffer
> within TBufferTransport in its constructor.
> TBufferTransport write only if the wbuf_ is full (512 bytes) into
> TMemoryBuffer. Hence data smaller than 512 will not get written into
> TMemoryBuffer. While getting serialized data from TMemoryBuffer it might not
> have the last chunk of data which is less than 512 bytes.
> This outputs invalid serialized data
> TestSchema$ php testsimple.php
> Serialized .
> mari
> Fatal error: Uncaught exception 'TTransportException' with message
> 'TMemoryBuffer: Could not read 512 bytes from buffer.' in
> /Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/transport/TMemoryBuffer.php
> on line 58
> TTransportException: TMemoryBuffer: Could not read 512 bytes from buffer. in
> /Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/transport/TMemoryBuffer.php
> on line 58
> Call Stack:
> 0.0003 636976 1. {main}()
> /Users/mponnambalam/work/eclipse/workspace/TestSchema/testsimple.php:0
> 0.00431509728 2. TBinarySerializer::deserialize()
> /Users/mponnambalam/work/eclipse/workspace/TestSchema/testsimple.php:47
> 0.00431511808 3. thrift_protocol_read_binary()
> /Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/protocol/TBinarySerializer.php:62
> thrift_protoco_read_binary disabled ( I commented extension in
> thrift_protocol.ini)
>
> TestSchema$ php testsimple.php
> Serialized .
> Fatal error: Uncaught exception 'TTransportException' with message
> 'TMemoryBuffer: Could not read 1 bytes from buffer.' in
> /Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/transport/TMemoryBuffer.php
> on line 58
> TTransportException: TMemoryBuffer: Could not read 1 bytes from buffer. in
> /Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/transport/TMemoryBuffer.php
> on line 58
> Call Stack:
> 0.0003 636976 1. {main}()
> /Users/mponnambalam/work/eclipse/workspace/TestSchema/testsimple.php:0
> 0.00411509944 2. TBinarySerializer::deserialize()
> /Users/mponnambalam/work/eclipse/workspace/TestSchema/testsimple.php:47
> 0.00411512248 3. Person->read()
> /Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/protocol/TBinarySerializer.php:66
> 0.00411512472 4. TBinaryProtocol->readFieldBegin()
> /Users/mponnambalam/work/eclipse/workspace/TestSchema/gen-php/simple/simple_types.php:45
> 0.00411512472 5. TBinaryProtocol->readByte()
> /Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/protocol/TBinaryProtocol.php:231
> 0.00411512520 6. TBufferedTransport->readAll()
> /Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/protocol/TBinaryProtocol.php:283
> 0.00411512568 7. TTransport->readAll()
> /Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/transport/TBufferedTransport.php:109
> 0.00411512744 8. TMemoryBuffer->read()
> /Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/transport/TTransport.php:87
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (THRIFT-1240) TBinarySerializer.php invalid serialization due to TBufferTransport not flushing last chunk of data into TMemoryBuffer
TBinarySerializer.php invalid serialization due to TBufferTransport not
flushing last chunk of data into TMemoryBuffer
--
Key: THRIFT-1240
URL: https://issues.apache.org/jira/browse/THRIFT-1240
Project: Thrift
Issue Type: Bug
Components: PHP - Library
Affects Versions: 0.6.1
Environment: Cent OS and Mac OS 10.5
Reporter: Marimuthu Ponnambalam
Priority: Critical
Fix For: 0.6.1
Simple Serialization errors out, due to the TBufferTransport not flushing the
data into TMemoryBuffer after $object->write($protocol) in TBinarySerializer.
Please find the error output of my test program below. It fixed after I added
the following line on the serialize method in TBinarySerializer.
$protocol->getTransport()->flush();
This because of TBinaryProtocolAccelerated internally wraps TMemoryBuffer
within TBufferTransport in its constructor.
TBufferTransport write only if the wbuf_ is full (512 bytes) into
TMemoryBuffer. Hence data smaller than 512 will not get written into
TMemoryBuffer. While getting serialized data from TMemoryBuffer it might not
have the last chunk of data which is less than 512 bytes.
This outputs invalid serialized data
TestSchema$ php testsimple.php
Serialized .
mari
Fatal error: Uncaught exception 'TTransportException' with message
'TMemoryBuffer: Could not read 512 bytes from buffer.' in
/Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/transport/TMemoryBuffer.php
on line 58
TTransportException: TMemoryBuffer: Could not read 512 bytes from buffer. in
/Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/transport/TMemoryBuffer.php
on line 58
Call Stack:
0.0003 636976 1. {main}()
/Users/mponnambalam/work/eclipse/workspace/TestSchema/testsimple.php:0
0.00431509728 2. TBinarySerializer::deserialize()
/Users/mponnambalam/work/eclipse/workspace/TestSchema/testsimple.php:47
0.00431511808 3. thrift_protocol_read_binary()
/Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/protocol/TBinarySerializer.php:62
thrift_protoco_read_binary disabled ( I commented extension in
thrift_protocol.ini)
TestSchema$ php testsimple.php
Serialized .
Fatal error: Uncaught exception 'TTransportException' with message
'TMemoryBuffer: Could not read 1 bytes from buffer.' in
/Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/transport/TMemoryBuffer.php
on line 58
TTransportException: TMemoryBuffer: Could not read 1 bytes from buffer. in
/Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/transport/TMemoryBuffer.php
on line 58
Call Stack:
0.0003 636976 1. {main}()
/Users/mponnambalam/work/eclipse/workspace/TestSchema/testsimple.php:0
0.00411509944 2. TBinarySerializer::deserialize()
/Users/mponnambalam/work/eclipse/workspace/TestSchema/testsimple.php:47
0.00411512248 3. Person->read()
/Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/protocol/TBinarySerializer.php:66
0.00411512472 4. TBinaryProtocol->readFieldBegin()
/Users/mponnambalam/work/eclipse/workspace/TestSchema/gen-php/simple/simple_types.php:45
0.00411512472 5. TBinaryProtocol->readByte()
/Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/protocol/TBinaryProtocol.php:231
0.00411512520 6. TBufferedTransport->readAll()
/Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/protocol/TBinaryProtocol.php:283
0.00411512568 7. TTransport->readAll()
/Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/transport/TBufferedTransport.php:109
0.00411512744 8. TMemoryBuffer->read()
/Users/mponnambalam/work/eclipse/workspace/thrift0.6.1/lib/php/src/transport/TTransport.php:87
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (THRIFT-1239) TupleProtocol- An extremely compact, temporary protocol
[ https://issues.apache.org/jira/browse/THRIFT-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Armaan Sarkar updated THRIFT-1239: -- Attachment: tuple_generator.patch A patch that generates the TupleSchemeFactory, TupleScheme and the appropriate read and write methods. > TupleProtocol- An extremely compact, temporary protocol > --- > > Key: THRIFT-1239 > URL: https://issues.apache.org/jira/browse/THRIFT-1239 > Project: Thrift > Issue Type: New Feature > Components: Java - Compiler >Reporter: Armaan Sarkar >Priority: Minor > Attachments: tuple_generator.patch, tuple_protocol.patch > > > Currently, protocols are built to be pretty robust to 'schema' changes. This > is done by sending metadata about when a struct or a field will start/end, > the number of fields to expect and the types of each field, etc. However, > there are cases when the recipient knows all of this, even before it receives > this metadata. In these cases, sending the metadata unnecessarily eats up > bandwidth. The TupleProtocol rectifies this by sending and receiving only the > value of each field in a specified order. The only metadata passed is about > variable information such as the size of a container or which optional fields > are set. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (THRIFT-1239) TupleProtocol- An extremely compact, temporary protocol
[ https://issues.apache.org/jira/browse/THRIFT-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Armaan Sarkar updated THRIFT-1239: -- Attachment: tuple_protocol.patch A patch that includes the actual TupleProtocol class and an associated test file. > TupleProtocol- An extremely compact, temporary protocol > --- > > Key: THRIFT-1239 > URL: https://issues.apache.org/jira/browse/THRIFT-1239 > Project: Thrift > Issue Type: New Feature > Components: Java - Compiler >Reporter: Armaan Sarkar >Priority: Minor > Attachments: tuple_generator.patch, tuple_protocol.patch > > > Currently, protocols are built to be pretty robust to 'schema' changes. This > is done by sending metadata about when a struct or a field will start/end, > the number of fields to expect and the types of each field, etc. However, > there are cases when the recipient knows all of this, even before it receives > this metadata. In these cases, sending the metadata unnecessarily eats up > bandwidth. The TupleProtocol rectifies this by sending and receiving only the > value of each field in a specified order. The only metadata passed is about > variable information such as the size of a container or which optional fields > are set. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-916) gcc warnings in c++ header files
[ https://issues.apache.org/jira/browse/THRIFT-916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13068033#comment-13068033 ] Christian Lavoie commented on THRIFT-916: - Works for me on both Mac OS X and my usual Debian machine. Can we commit a patch that turns on -Wall -Wextra -pedantic and -Werror, assuming we take out -Werror for releases? > gcc warnings in c++ header files > > > Key: THRIFT-916 > URL: https://issues.apache.org/jira/browse/THRIFT-916 > Project: Thrift > Issue Type: Improvement > Components: C++ - Library >Affects Versions: 0.4 >Reporter: Piotr Bartosiewicz >Assignee: Roger Meier > Attachments: DebianLenny_error.log, > THRIFT-916_Wall_Wextra__NOWARNINGS_DebianLenny.patch, > THRIFT-916_Wall_Wextra_pedantic_Wno-long-long_Wno-variadic-macros_Wno-overflow_NOWARNINGS_DebianLenny.patch, > THRIFT-916_Wall_pedantic__noerrors.patch, THRIFT-916_fix_no-overflow.patch, > make-k.log, v1-enable-wall-werror.patch, v1-fix-format-strings.patch, > v1-fix-java-style-and-warnings.patch, v1-fix-more-warnings.patch, > v1-fix-random-silly-warnings.patch, v1-long-long-to-int64_t.patch, > v2-enable-wall-werror.patch, v2-fix-all-warnings.patch, > v3-enable-wall-werror.patch, v3-fix-all-warnings.patch, > v4-enable-wall-werror.patch, v4-fix-all-warnings.patch > > > I tried to add an extra gcc warning options in my project but a number of > warnings was emmited from thrift headers. > Following the boost library recommendations, code should be clean with "-Wall > -Wextra -pedantic" options. > Currently only with -Wall thrift has no warnings. > Other options worth considering (but require more work) are -Wconversion > -Wold-style-cast > So the task is to improve the thrift headers. > I suggest also update all the thrift makefiles so the thrift sources an tests > will indicate the faulty code. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (THRIFT-1239) TupleProtocol- An extremely compact, temporary protocol
[ https://issues.apache.org/jira/browse/THRIFT-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13068006#comment-13068006 ] Armaan Sarkar edited comment on THRIFT-1239 at 7/19/11 9:56 PM: This issue depends on the pluggable serializer because the TupleProtocol relies on changes in the read/write methods of an object. was (Author: armaansarkar): The TupleProtocol relies on changes in the read/write methods of an object > TupleProtocol- An extremely compact, temporary protocol > --- > > Key: THRIFT-1239 > URL: https://issues.apache.org/jira/browse/THRIFT-1239 > Project: Thrift > Issue Type: New Feature > Components: Java - Compiler >Reporter: Armaan Sarkar >Priority: Minor > > Currently, protocols are built to be pretty robust to 'schema' changes. This > is done by sending metadata about when a struct or a field will start/end, > the number of fields to expect and the types of each field, etc. However, > there are cases when the recipient knows all of this, even before it receives > this metadata. In these cases, sending the metadata unnecessarily eats up > bandwidth. The TupleProtocol rectifies this by sending and receiving only the > value of each field in a specified order. The only metadata passed is about > variable information such as the size of a container or which optional fields > are set. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (THRIFT-769) Pluggable Serializers
[ https://issues.apache.org/jira/browse/THRIFT-769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Armaan Sarkar updated THRIFT-769: - Attachment: pluggable_serializer.patch Adds the pluggable serializer from scratch > Pluggable Serializers > - > > Key: THRIFT-769 > URL: https://issues.apache.org/jira/browse/THRIFT-769 > Project: Thrift > Issue Type: New Feature > Components: Java - Compiler, Java - Library >Reporter: Bryan Duxbury >Assignee: Bryan Duxbury > Labels: gsoc2011, mentor > Attachments: pluggable_serializer.patch, thrift_scheme.patch, > thrift_scheme_update1.patch > > > Thrift is already designed to have modular Protocols, which define the > various elements of binary encoding. However, the way that our code is > generated, we're also implicitly defining every protocol to have a > *structure* like that of the Binary protocol. > This has served us well so far with it's simplicity, and other protocols > (like Compact and JSON) have been able to squeeze into this model > successfully, though at a noticeable cost. For instance, Compact protocol has > to keep its own internal state in a stack to know what the last field id we > wrote was so that it can write compact deltas. The JSON protocol is even more > complex, keeping a truly monstrous amount of state around in order to be able > to do the right thing at the right time. All this state tracking comes with > cost, both in complexity and in actual runtime. > I think that there's a solution to this problem - making the serialization > code itself pluggable. That is, rather than there being a single "write" and > "read" method for every protocol, we could let Protocols require a certain > kind of Serializer to interact with them. Binary protocol could use the > "default tagged" serializer, which would look like what we have today. JSON > would probably use a custom one that basically let it write out strings and > nothing else. Compact would also likely use a custom serializer that kept all > the needed state on the system stack, rather than in a user-code stack. Dare > I say it, but it's possible that if we did it right, we could even serialize > Thrift structs with Avro's serialization format! > The upshot of this change would be to make many protocols faster and simpler, > while also allowing us to implement a broader set of protocols that don't fit > with the traditional Thrift-style protocols. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (THRIFT-1239) TupleProtocol- An extremely compact, temporary protocol
TupleProtocol- An extremely compact, temporary protocol --- Key: THRIFT-1239 URL: https://issues.apache.org/jira/browse/THRIFT-1239 Project: Thrift Issue Type: New Feature Components: Java - Compiler Reporter: Armaan Sarkar Priority: Minor Currently, protocols are built to be pretty robust to 'schema' changes. This is done by sending metadata about when a struct or a field will start/end, the number of fields to expect and the types of each field, etc. However, there are cases when the recipient knows all of this, even before it receives this metadata. In these cases, sending the metadata unnecessarily eats up bandwidth. The TupleProtocol rectifies this by sending and receiving only the value of each field in a specified order. The only metadata passed is about variable information such as the size of a container or which optional fields are set. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1233) Remove unused include in generated C++ code
[ https://issues.apache.org/jira/browse/THRIFT-1233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13067747#comment-13067747 ] Diwaker Gupta commented on THRIFT-1233: --- @alexandre: Can you provide some more details about your build (include path etc?). I've built Thrift compiled C++ code with cob_style enabled and have not seen this issue. > Remove unused include in generated C++ code > --- > > Key: THRIFT-1233 > URL: https://issues.apache.org/jira/browse/THRIFT-1233 > Project: Thrift > Issue Type: Improvement > Components: C++ - Compiler > Environment: Ubuntu 11.04, latest trunk. >Reporter: Diwaker Gupta >Assignee: Jake Farrell >Priority: Trivial > Fix For: 0.7 > > Attachments: THRIFT-1233.patch > > > When using cob_style with the CPP generator, the compiler adds an include for > TTransportUtils. This include doesn't seem to be used anywhere and creates an > unnecessary dependency. It also complicates compiling Thrift generated code > on other platforms. > Patch available for review here: > https://reviews.apache.org/r/1034/ -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1233) Remove unused include in generated C++ code
[ https://issues.apache.org/jira/browse/THRIFT-1233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13067741#comment-13067741 ] Jake Farrell commented on THRIFT-1233: -- If you have a patch for this please attach it > Remove unused include in generated C++ code > --- > > Key: THRIFT-1233 > URL: https://issues.apache.org/jira/browse/THRIFT-1233 > Project: Thrift > Issue Type: Improvement > Components: C++ - Compiler > Environment: Ubuntu 11.04, latest trunk. >Reporter: Diwaker Gupta >Assignee: Jake Farrell >Priority: Trivial > Fix For: 0.7 > > Attachments: THRIFT-1233.patch > > > When using cob_style with the CPP generator, the compiler adds an include for > TTransportUtils. This include doesn't seem to be used anywhere and creates an > unnecessary dependency. It also complicates compiling Thrift generated code > on other platforms. > Patch available for review here: > https://reviews.apache.org/r/1034/ -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (THRIFT-1233) Remove unused include in generated C++ code
[ https://issues.apache.org/jira/browse/THRIFT-1233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] alexandre parenteau reopened THRIFT-1233: - Assignee: Jake Farrell (was: Diwaker Gupta) This change breaks the cob style compilation, here is an example below. The problem seems to be that TMemoryBuffer definition is not getting included. I'd suggest replace the original: #include by: #include Example: In file included from ...sources/thrift/gen-cpp/Imaging.cpp:6:0: ...sources/thrift/gen-cpp/Imaging.h:435:22: error: 'TMemoryBuffer' is not a member of 'apache::thrift::transport' ...sources/thrift/gen-cpp/Imaging.h:435:22: error: 'TMemoryBuffer' is not a member of 'apache::thrift::transport' ...sources/thrift/gen-cpp/Imaging.h:435:64: error: template argument 1 is invalid ...sources/thrift/gen-cpp/Imaging.h:436:22: error: 'TMemoryBuffer' is not a member of 'apache::thrift::transport' ...sources/thrift/gen-cpp/Imaging.h:436:22: error: 'TMemoryBuffer' is not a member of 'apache::thrift::transport' ...sources/thrift/gen-cpp/Imaging.h:436:64: error: template argument 1 is invalid ...sources/thrift/gen-cpp/Imaging.h: In constructor 'imaging::ImagingCobClient::ImagingCobClient(boost::shared_ptr, apache::thrift::protocol::TProtocolFactory*)': > Remove unused include in generated C++ code > --- > > Key: THRIFT-1233 > URL: https://issues.apache.org/jira/browse/THRIFT-1233 > Project: Thrift > Issue Type: Improvement > Components: C++ - Compiler > Environment: Ubuntu 11.04, latest trunk. >Reporter: Diwaker Gupta >Assignee: Jake Farrell >Priority: Trivial > Fix For: 0.7 > > Attachments: THRIFT-1233.patch > > > When using cob_style with the CPP generator, the compiler adds an include for > TTransportUtils. This include doesn't seem to be used anywhere and creates an > unnecessary dependency. It also complicates compiling Thrift generated code > on other platforms. > Patch available for review here: > https://reviews.apache.org/r/1034/ -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
