Re: [protobuf] Re: Getting IOException while deserializing a proto which contains a repeated JKL and JKL has a string type set ( works for repeated JKL and JKL has the same with int type set )
there's nothing wrong with those message definitions; you haven't told us
which platform and/or library you're using, but: my hunch would be that
you're transporting the data incorrectly in some way, meaning: corrupting
it in *your* code - or the code you're using to compress/decompress is
slightly wrong. Unfortunately we can't see any of that. But: speaking
generally: there's no reason it shouldn't work, so: we'd need more details
to provide useful help.
On Thu, 13 Dec 2018 at 18:52, Shibha Malik wrote:
>
>
> On Thursday, December 13, 2018 at 10:32:52 AM UTC-8, Shibha Malik wrote:
>>
>> Hi Team,
>>
>> i am getting IOException while deserializing a proto which contains a
>> repeated Data and Data has a string type set ( works for repeated Data and
>> Data has the same with int type set )
>>
>> I have a proto which has the following structure :
>>
>> message XYZData
>> {
>> ABCId abc_id = 1;
>> DEF def= 2;
>> google.protobuf.Timestamp ghi= 3;
>> repeated JKL jkl = 4;
>> MNO mno = 5;
>> }
>>
>> where JKL is
>> message JKL
>> {
>> Culprit culprit = 1;
>> PQR pqr = 2;
>> }
>> where Culprit is
>> message Culprit
>> {
>> oneof culprit
>> {
>> int32 int_culprit = 1;
>> string string_culprit = 2;
>> }
>> }
>>
>> When i use the int_culprit and use GZIP compression. it serializes and
>> deserializes fine.
>> BUT when i use the string_culprit and use GZIP compression. it serializes
>> fine and deserialization gives me IOException.
>>
>>
>> PLEASE help us on this as this is blocking a major deliverable for us.
>>
>> Thanks,
>> Shibha
>>
>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to protobuf+unsubscr...@googlegroups.com.
> To post to this group, send email to protobuf@googlegroups.com.
> Visit this group at https://groups.google.com/group/protobuf.
> For more options, visit https://groups.google.com/d/optout.
>
--
Regards,
Marc
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
[protobuf] Re: Getting IOException while deserializing a proto which contains a repeated Data and Data has a string type set ( works for repeated Data and Data has the same with int type set )
FYI i m using proto3
On Thursday, December 13, 2018 at 10:32:52 AM UTC-8, Shibha Malik wrote:
>
> Hi Team,
>
> i am getting IOException while deserializing a proto which contains a
> repeated Data and Data has a string type set ( works for repeated Data and
> Data has the same with int type set )
>
> I have a proto which has the following structure :
>
> message XYZData
> {
> ABCId abc_id = 1;
> DEF def= 2;
> google.protobuf.Timestamp ghi= 3;
> repeated JKL jkl = 4;
> MNO mno = 5;
> }
>
> where JKL is
> message JKL
> {
> Culprit culprit = 1;
> PQR pqr = 2;
> }
> where Culprit is
> message Culprit
> {
> oneof culprit
> {
> int32 int_culprit = 1;
> string string_culprit = 2;
> }
> }
>
> When i use the int_culprit and use GZIP compression. it serializes and
> deserializes fine.
> BUT when i use the string_culprit and use GZIP compression. it serializes
> fine and deserialization gives me IOException.
>
>
> PLEASE help us on this as this is blocking a major deliverable for us.
>
> Thanks,
> Shibha
>
>
>
>
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
[protobuf] Re: Getting IOException while deserializing a proto which contains a repeated JKL and JKL has a string type set ( works for repeated JKL and JKL has the same with int type set )
On Thursday, December 13, 2018 at 10:32:52 AM UTC-8, Shibha Malik wrote:
>
> Hi Team,
>
> i am getting IOException while deserializing a proto which contains a
> repeated Data and Data has a string type set ( works for repeated Data and
> Data has the same with int type set )
>
> I have a proto which has the following structure :
>
> message XYZData
> {
> ABCId abc_id = 1;
> DEF def= 2;
> google.protobuf.Timestamp ghi= 3;
> repeated JKL jkl = 4;
> MNO mno = 5;
> }
>
> where JKL is
> message JKL
> {
> Culprit culprit = 1;
> PQR pqr = 2;
> }
> where Culprit is
> message Culprit
> {
> oneof culprit
> {
> int32 int_culprit = 1;
> string string_culprit = 2;
> }
> }
>
> When i use the int_culprit and use GZIP compression. it serializes and
> deserializes fine.
> BUT when i use the string_culprit and use GZIP compression. it serializes
> fine and deserialization gives me IOException.
>
>
> PLEASE help us on this as this is blocking a major deliverable for us.
>
> Thanks,
> Shibha
>
>
>
>
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
[protobuf] Getting IOException while deserializing a proto which contains a repeated Data and Data has a string type set ( works for repeated Data and Data has the same with int type set )
Hi Team,
i am getting IOException while deserializing a proto which contains a
repeated Data and Data has a string type set ( works for repeated Data and
Data has the same with int type set )
I have a proto which has the following structure :
message XYZData
{
ABCId abc_id = 1;
DEF def= 2;
google.protobuf.Timestamp ghi= 3;
repeated JKL jkl = 4;
MNO mno = 5;
}
where JKL is
message JKL
{
Culprit culprit = 1;
PQR pqr = 2;
}
where Culprit is
message Culprit
{
oneof culprit
{
int32 int_culprit = 1;
string string_culprit = 2;
}
}
When i use the int_culprit and use GZIP compression. it serializes and
deserializes fine.
BUT when i use the string_culprit and use GZIP compression. it serializes
fine and deserialization gives me IOException.
PLEASE help us on this as this is blocking a major deliverable for us.
Thanks,
Shibha
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
Re: [protobuf] Protobuf Binary Size
What architecture are you building for, and what level of optimization? On Wed, Dec 12, 2018 at 11:30 PM Roy Barda wrote: > Tried your approach, I have built my project statically against > protobuf-lite library (Release with the relevant compilation and linkage > flags) and reached a 7MB solution. > I wonder how you reached 135KB, thoughts? Is there anything else I should > do? > > Thanks, > Roy. > > On Fri, Dec 7, 2018 at 2:51 AM Adam Cozzette wrote: > >> OK, you might want to ask on the gRPC mailing list why streams can't be >> used with the lite runtime. Maybe there is a plan to eventually add support >> for that, especially since it's common for mobile apps to use lite protos. >> >> On Thu, Dec 6, 2018 at 7:01 AM Roy Barda wrote: >> >>> Yes, according to current gRPC implementation, stream can't be used with >>> LITE_RUNTIME configuration in the .proto file? >>> >>> Thanks, >>> Roy. >>> >>> On Tue, Dec 4, 2018 at 7:19 PM Adam Cozzette >>> wrote: >>> For one Android binary I worked on recently, the protobuf runtime used about 135 KB of space. This was for an ARM build, and I imagine the amount could vary quite a bit depending on the platform and what runtime code you end up depending on. I'm not familiar with that gRPC issue, so you may want to ask the gRPC folks about that. Does the "stream" modifier prevent you from setting optimize_for = LITE_RUNTIME in the .proto file? On Mon, Dec 3, 2018 at 11:05 PM Roy Barda wrote: > Thank you Adam for the detailed suggestion. > > 1. Can you please let me know what was the addition in KB/MB of > protobuf usage when you linked it statically to your Android code? > 2. Using libprotobuf-lite blocks me from using the "stream" modifier > (streaming feature) in gRPC, anyway to work around it? > > Appreciate your help, > Roy. > > On Tuesday, December 4, 2018 at 5:51:57 AM UTC+2, Adam Cozzette wrote: >> >> Do you plan to link statically or dynamically against the protobuf >> library? If you can link against it statically, I think you may find that >> the linker is able to remove a lot of unused code. For Android (which is >> an >> environment where we care a lot about code size), what we generally do >> is: >> - Use lite protos only (i.e. set option optimize_for = LITE_RUNTIME >> in every .proto file) >> - Link against libprotobuf-lite and not libprotobuf >> - Pass -ffunction-sections and -fdata-sections to the compiler and >> -Wl,-gc-sections to the linker to maximize the linker's ability to strip >> out unused code >> >> On Sun, Dec 2, 2018 at 10:34 PM Roy Barda wrote: >> >>> Hi, >>> >>> I have build protobuf on my Ubuntu environment and saw that the >>> libraries size are extremely big. >>> >>> 28 MB for libprotobuf. >>> 4 MB for libprotobuf-lite >>> >>> I'm aware of nanopb project but I need to use gRPC with protobuf and >>> there is no good, working and maintained solution for nanopb. >>> >>> I followed the regular build process when I built protobuf (with no >>> special configuration) >>> >>> Is there any way to reduce the protobuf library footprint? We are >>> looking for a 1MB solution. >>> >>> Thanks, >>> >>> Roy. >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Protocol Buffers" group. >>> To unsubscribe from this group and stop receiving emails from it, >>> send an email to protobuf+u...@googlegroups.com. >>> To post to this group, send email to prot...@googlegroups.com. >>> Visit this group at https://groups.google.com/group/protobuf. >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- > You received this message because you are subscribed to the Google > Groups "Protocol Buffers" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to protobuf+unsubscr...@googlegroups.com. > To post to this group, send email to protobuf@googlegroups.com. > Visit this group at https://groups.google.com/group/protobuf. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at https://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/d/optout.
