subject:"\[protobuf\] Re\: Trying to rewrite a protobuf message but changing a couple of values"

[protobuf] Re: Trying to rewrite a protobuf message but changing a couple of values

2023-09-20 Thread Joan Balagueró

Ok thanks, I was trying to avoid this, to compile and use the generated 
classes. Now we are currently processing the raw protobuf message as a map 
of UnknownFieldSet, and I was wondering if there was a choice to make this 
"rewrite" without the need to compile and use the class, just traversing 
the protobuf.

I will check and try.

Thanks,

Joan.

On Wednesday, September 20, 2023 at 12:30:26 PM UTC+2 Florian Enner wrote:

> As the very first step you should compile your schema and work with the 
> generated classes: 
> https://protobuf.dev/getting-started/javatutorial/#compiling-protocol-buffers
>
> Don't mess with the reflection API if you can avoid it. 
>
>
> On Wednesday, September 20, 2023 at 12:19:38 PM UTC+2 Joan Balagueró wrote:
>
>> Hi Florian,
>>
>> Not sure what I can do with this code. I only have a byte array 
>> representing the above protobuf, I don't have any object or message. At 
>> most I have a map with the list of the "UnknownFieldSet.Field" fields after 
>> parsing the byte array (this is java code).
>>
>> I'm relatively new to protobuf, I read a lot on the Internet trying to 
>> find a solution but I did not find anything.
>>
>> That's why I tried to traverse the map of fields and write them to the 
>> new byte array. It's the only solution I could think of, but I'm doing 
>> something wrong.
>>
>> Not sure if you can help me a bit more to solve this.
>>
>> Anyways thanks.
>>
>> Joan.
>>
>>
>>
>>
>>
>> On Wednesday, September 20, 2023 at 11:56:19 AM UTC+2 Florian Enner wrote:
>>
>>> Messages are serialized with a length delimiter, so changing the content 
>>> produces a mismatch and invalid message.
>>>
>>> Your schema has no affected repeated fields, so appending a delta should 
>>> work. I've never used the C# API, but here is some hopefully understandable 
>>> pseudo code:
>>>
>>> var delta = Request.newInstance();
>>> delta.getMutableMeta().getMutableCutOffTime()   
>>>   .setValue(value)
>>>   .setScale(TimeSpanScale.MINMAX)
>>> byte[] output = append(unmodifiedInputBytes, delta.toByteArray());
>>>
>>> If the server expects a length delimiter you'd need to update it to the 
>>> new combined length.
>>>
>>> - Florian
>>>
>>> On Wednesday, September 20, 2023 at 11:03:46 AM UTC+2 Joan Balagueró 
>>> wrote:
>>>
 Hi Florian,

 Thanks for your quick response. I'm stuck on this.

 1) It's not working. When I send the protobuf to the backend server 
 (it's not our api nor server) using the first method, I get a right 
 response. But using the second method I receive this error:
 ProtoBuf.ProtoException: Invalid wire-type; this usually means you have 
 over-written a file without truncating or setting the length; see 
 https://stackoverflow.com/q/2152978/23354
at ProtoBuf.ProtoReader.StartSubItem(ProtoReader reader) in 
 C:\code\protobuf-net\src\protobuf-net\ProtoReader.cs:line 637
at ProtoBuf.ProtoReader.ReadTypedObject(Object value, Int32 key, 
 ProtoReader reader, Type type) in 
 C:\code\protobuf-net\src\protobuf-net\ProtoReader.cs:line 584
at proto_40(Object , ProtoReader )
at ProtoBuf.Meta.TypeModel.DeserializeCore(ProtoReader reader, Type 
 type, Object value, Boolean noAutoCreate) in 
 C:\code\protobuf-net\src\protobuf-net\Meta\TypeModel.cs:line 722
at ProtoBuf.Meta.TypeModel.Deserialize(Stream source, Object value, 
 Type type, SerializationContext context) in 
 C:\code\protobuf-net\src\protobuf-net\Meta\TypeModel.cs:line 599
at 
 WebBeds.Connect.AspNetCore.Formatters.ProtobufInputFormatter.ReadRequestBodyAsync(InputFormatterContext

 context)
at 
 Microsoft.AspNetCore.Mvc.Formatters.InputFormatter.ReadAsync(InputFormatterContext

 context)
at 
 Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder.BindModelAsync(ModelBindingContext

 bindingContext)
at 
 Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder.BindModelAsync(ActionContext

 actionContext, IModelBinder modelBinder, IValueProvider valueProvider, 
 ParameterDescriptor parameter, ModelMetadata metadata, Object value, 
 Object 
 container)
at 
 Microsoft.AspNetCore.Mvc.Controllers.ControllerBinderDelegateProvider.<>c__DisplayClass0_0.d.MoveNext()
 --- End of stack trace from previous location ---
at 
 Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.g__Awaited|13_0(ControllerActionInvoker

 invoker, Task lastTask, State next, Scope scope, Object state, Boolean 
 isCompleted)
at 
 Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|26_0(ResourceInvoker

 invoker, Task lastTask, State next, Scope scope, Object state, Boolean 
 isCompleted

 2) This is the proto:

 The request with the 'Meta' element:

 message Request {
Meta Meta = 1;
repeated int32 Hotels

[protobuf] Re: Trying to rewrite a protobuf message but changing a couple of values

2023-09-20 Thread Florian Enner

As the very first step you should compile your schema and work with the 
generated classes: 
https://protobuf.dev/getting-started/javatutorial/#compiling-protocol-buffers

Don't mess with the reflection API if you can avoid it. 


On Wednesday, September 20, 2023 at 12:19:38 PM UTC+2 Joan Balagueró wrote:

> Hi Florian,
>
> Not sure what I can do with this code. I only have a byte array 
> representing the above protobuf, I don't have any object or message. At 
> most I have a map with the list of the "UnknownFieldSet.Field" fields after 
> parsing the byte array (this is java code).
>
> I'm relatively new to protobuf, I read a lot on the Internet trying to 
> find a solution but I did not find anything.
>
> That's why I tried to traverse the map of fields and write them to the new 
> byte array. It's the only solution I could think of, but I'm doing 
> something wrong.
>
> Not sure if you can help me a bit more to solve this.
>
> Anyways thanks.
>
> Joan.
>
>
>
>
>
> On Wednesday, September 20, 2023 at 11:56:19 AM UTC+2 Florian Enner wrote:
>
>> Messages are serialized with a length delimiter, so changing the content 
>> produces a mismatch and invalid message.
>>
>> Your schema has no affected repeated fields, so appending a delta should 
>> work. I've never used the C# API, but here is some hopefully understandable 
>> pseudo code:
>>
>> var delta = Request.newInstance();
>> delta.getMutableMeta().getMutableCutOffTime()   
>>   .setValue(value)
>>   .setScale(TimeSpanScale.MINMAX)
>> byte[] output = append(unmodifiedInputBytes, delta.toByteArray());
>>
>> If the server expects a length delimiter you'd need to update it to the 
>> new combined length.
>>
>> - Florian
>>
>> On Wednesday, September 20, 2023 at 11:03:46 AM UTC+2 Joan Balagueró 
>> wrote:
>>
>>> Hi Florian,
>>>
>>> Thanks for your quick response. I'm stuck on this.
>>>
>>> 1) It's not working. When I send the protobuf to the backend server 
>>> (it's not our api nor server) using the first method, I get a right 
>>> response. But using the second method I receive this error:
>>> ProtoBuf.ProtoException: Invalid wire-type; this usually means you have 
>>> over-written a file without truncating or setting the length; see 
>>> https://stackoverflow.com/q/2152978/23354
>>>at ProtoBuf.ProtoReader.StartSubItem(ProtoReader reader) in 
>>> C:\code\protobuf-net\src\protobuf-net\ProtoReader.cs:line 637
>>>at ProtoBuf.ProtoReader.ReadTypedObject(Object value, Int32 key, 
>>> ProtoReader reader, Type type) in 
>>> C:\code\protobuf-net\src\protobuf-net\ProtoReader.cs:line 584
>>>at proto_40(Object , ProtoReader )
>>>at ProtoBuf.Meta.TypeModel.DeserializeCore(ProtoReader reader, Type 
>>> type, Object value, Boolean noAutoCreate) in 
>>> C:\code\protobuf-net\src\protobuf-net\Meta\TypeModel.cs:line 722
>>>at ProtoBuf.Meta.TypeModel.Deserialize(Stream source, Object value, 
>>> Type type, SerializationContext context) in 
>>> C:\code\protobuf-net\src\protobuf-net\Meta\TypeModel.cs:line 599
>>>at 
>>> WebBeds.Connect.AspNetCore.Formatters.ProtobufInputFormatter.ReadRequestBodyAsync(InputFormatterContext
>>>  
>>> context)
>>>at 
>>> Microsoft.AspNetCore.Mvc.Formatters.InputFormatter.ReadAsync(InputFormatterContext
>>>  
>>> context)
>>>at 
>>> Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder.BindModelAsync(ModelBindingContext
>>>  
>>> bindingContext)
>>>at 
>>> Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder.BindModelAsync(ActionContext
>>>  
>>> actionContext, IModelBinder modelBinder, IValueProvider valueProvider, 
>>> ParameterDescriptor parameter, ModelMetadata metadata, Object value, Object 
>>> container)
>>>at 
>>> Microsoft.AspNetCore.Mvc.Controllers.ControllerBinderDelegateProvider.<>c__DisplayClass0_0.d.MoveNext()
>>> --- End of stack trace from previous location ---
>>>at 
>>> Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.g__Awaited|13_0(ControllerActionInvoker
>>>  
>>> invoker, Task lastTask, State next, Scope scope, Object state, Boolean 
>>> isCompleted)
>>>at 
>>> Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|26_0(ResourceInvoker
>>>  
>>> invoker, Task lastTask, State next, Scope scope, Object state, Boolean 
>>> isCompleted
>>>
>>>
>>> 2) This is the proto:
>>>
>>> The request with the 'Meta' element:
>>>
>>> message Request {
>>>Meta Meta = 1;
>>>repeated int32 Hotels = 2 [packed = false];
>>>Country Market = 3;
>>>repeated Room Rooms = 4;
>>>.bcl.DateTime CheckIn = 5;
>>>.bcl.DateTime CheckOut = 6;
>>>OptionalCriteria OptionalCriteria = 7;
>>> }
>>>
>>> The 'Meta' element that contains the 'CutOffTime' that we want to modify:
>>>
>>> message Meta {
>>>int32 Client = 1;
>>>int32 Brand = 2;
>>>bool UseCache = 3;
>>>.bcl.TimeSpan CutOffTime = 4;
>>>bool B2C = 5;
>>>Language Language = 6;
>>>Currency Currency = 7;
>>>bool IncludeProviderAuditData = 8;
>>>

[protobuf] Re: Trying to rewrite a protobuf message but changing a couple of values

2023-09-20 Thread Joan Balagueró

Hi Florian,

Not sure what I can do with this code. I only have a byte array 
representing the above protobuf, I don't have any object or message. At 
most I have a map with the list of the "UnknownFieldSet.Field" fields after 
parsing the byte array (this is java code).

I'm relatively new to protobuf, I read a lot on the Internet trying to find 
a solution but I did not find anything.

That's why I tried to traverse the map of fields and write them to the new 
byte array. It's the only solution I could think of, but I'm doing 
something wrong.

Not sure if you can help me a bit more to solve this.

Anyways thanks.

Joan.





On Wednesday, September 20, 2023 at 11:56:19 AM UTC+2 Florian Enner wrote:

> Messages are serialized with a length delimiter, so changing the content 
> produces a mismatch and invalid message.
>
> Your schema has no affected repeated fields, so appending a delta should 
> work. I've never used the C# API, but here is some hopefully understandable 
> pseudo code:
>
> var delta = Request.newInstance();
> delta.getMutableMeta().getMutableCutOffTime()   
>   .setValue(value)
>   .setScale(TimeSpanScale.MINMAX)
> byte[] output = append(unmodifiedInputBytes, delta.toByteArray());
>
> If the server expects a length delimiter you'd need to update it to the 
> new combined length.
>
> - Florian
>
> On Wednesday, September 20, 2023 at 11:03:46 AM UTC+2 Joan Balagueró wrote:
>
>> Hi Florian,
>>
>> Thanks for your quick response. I'm stuck on this.
>>
>> 1) It's not working. When I send the protobuf to the backend server (it's 
>> not our api nor server) using the first method, I get a right response. But 
>> using the second method I receive this error:
>> ProtoBuf.ProtoException: Invalid wire-type; this usually means you have 
>> over-written a file without truncating or setting the length; see 
>> https://stackoverflow.com/q/2152978/23354
>>at ProtoBuf.ProtoReader.StartSubItem(ProtoReader reader) in 
>> C:\code\protobuf-net\src\protobuf-net\ProtoReader.cs:line 637
>>at ProtoBuf.ProtoReader.ReadTypedObject(Object value, Int32 key, 
>> ProtoReader reader, Type type) in 
>> C:\code\protobuf-net\src\protobuf-net\ProtoReader.cs:line 584
>>at proto_40(Object , ProtoReader )
>>at ProtoBuf.Meta.TypeModel.DeserializeCore(ProtoReader reader, Type 
>> type, Object value, Boolean noAutoCreate) in 
>> C:\code\protobuf-net\src\protobuf-net\Meta\TypeModel.cs:line 722
>>at ProtoBuf.Meta.TypeModel.Deserialize(Stream source, Object value, 
>> Type type, SerializationContext context) in 
>> C:\code\protobuf-net\src\protobuf-net\Meta\TypeModel.cs:line 599
>>at 
>> WebBeds.Connect.AspNetCore.Formatters.ProtobufInputFormatter.ReadRequestBodyAsync(InputFormatterContext
>>  
>> context)
>>at 
>> Microsoft.AspNetCore.Mvc.Formatters.InputFormatter.ReadAsync(InputFormatterContext
>>  
>> context)
>>at 
>> Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder.BindModelAsync(ModelBindingContext
>>  
>> bindingContext)
>>at 
>> Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder.BindModelAsync(ActionContext
>>  
>> actionContext, IModelBinder modelBinder, IValueProvider valueProvider, 
>> ParameterDescriptor parameter, ModelMetadata metadata, Object value, Object 
>> container)
>>at 
>> Microsoft.AspNetCore.Mvc.Controllers.ControllerBinderDelegateProvider.<>c__DisplayClass0_0.d.MoveNext()
>> --- End of stack trace from previous location ---
>>at 
>> Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.g__Awaited|13_0(ControllerActionInvoker
>>  
>> invoker, Task lastTask, State next, Scope scope, Object state, Boolean 
>> isCompleted)
>>at 
>> Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|26_0(ResourceInvoker
>>  
>> invoker, Task lastTask, State next, Scope scope, Object state, Boolean 
>> isCompleted
>>
>>
>> 2) This is the proto:
>>
>> The request with the 'Meta' element:
>>
>> message Request {
>>Meta Meta = 1;
>>repeated int32 Hotels = 2 [packed = false];
>>Country Market = 3;
>>repeated Room Rooms = 4;
>>.bcl.DateTime CheckIn = 5;
>>.bcl.DateTime CheckOut = 6;
>>OptionalCriteria OptionalCriteria = 7;
>> }
>>
>> The 'Meta' element that contains the 'CutOffTime' that we want to modify:
>>
>> message Meta {
>>int32 Client = 1;
>>int32 Brand = 2;
>>bool UseCache = 3;
>>.bcl.TimeSpan CutOffTime = 4;
>>bool B2C = 5;
>>Language Language = 6;
>>Currency Currency = 7;
>>bool IncludeProviderAuditData = 8;
>>SalesChannel SalesChannel = 9;
>>string AgentId = 10;
>> }
>>
>> The 'CutOffTime':
>>
>> message TimeSpan {
>>   sint64 value = 1; // the size of the timespan (in units of the selected 
>> scale)
>>   TimeSpanScale scale = 2; // the scale of the timespan [default = DAYS]
>>   enum TimeSpanScale {
>> DAYS = 0;
>> HOURS = 1;
>> MINUTES = 2;
>> SECONDS = 3;
>> MILLISECONDS = 4;
>> TICKS = 5;
>>
>> MINMAX = 15; // dubious
>>   }
>> }

[protobuf] Re: Trying to rewrite a protobuf message but changing a couple of values

2023-09-20 Thread Florian Enner

Messages are serialized with a length delimiter, so changing the content 
produces a mismatch and invalid message.

Your schema has no affected repeated fields, so appending a delta should 
work. I've never used the C# API, but here is some hopefully understandable 
pseudo code:

var delta = Request.newInstance();
delta.getMutableMeta().getMutableCutOffTime()   
  .setValue(value)
  .setScale(TimeSpanScale.MINMAX)
byte[] output = append(unmodifiedInputBytes, delta.toByteArray());

If the server expects a length delimiter you'd need to update it to the new 
combined length.

- Florian

On Wednesday, September 20, 2023 at 11:03:46 AM UTC+2 Joan Balagueró wrote:

> Hi Florian,
>
> Thanks for your quick response. I'm stuck on this.
>
> 1) It's not working. When I send the protobuf to the backend server (it's 
> not our api nor server) using the first method, I get a right response. But 
> using the second method I receive this error:
> ProtoBuf.ProtoException: Invalid wire-type; this usually means you have 
> over-written a file without truncating or setting the length; see 
> https://stackoverflow.com/q/2152978/23354
>at ProtoBuf.ProtoReader.StartSubItem(ProtoReader reader) in 
> C:\code\protobuf-net\src\protobuf-net\ProtoReader.cs:line 637
>at ProtoBuf.ProtoReader.ReadTypedObject(Object value, Int32 key, 
> ProtoReader reader, Type type) in 
> C:\code\protobuf-net\src\protobuf-net\ProtoReader.cs:line 584
>at proto_40(Object , ProtoReader )
>at ProtoBuf.Meta.TypeModel.DeserializeCore(ProtoReader reader, Type 
> type, Object value, Boolean noAutoCreate) in 
> C:\code\protobuf-net\src\protobuf-net\Meta\TypeModel.cs:line 722
>at ProtoBuf.Meta.TypeModel.Deserialize(Stream source, Object value, 
> Type type, SerializationContext context) in 
> C:\code\protobuf-net\src\protobuf-net\Meta\TypeModel.cs:line 599
>at 
> WebBeds.Connect.AspNetCore.Formatters.ProtobufInputFormatter.ReadRequestBodyAsync(InputFormatterContext
>  
> context)
>at 
> Microsoft.AspNetCore.Mvc.Formatters.InputFormatter.ReadAsync(InputFormatterContext
>  
> context)
>at 
> Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder.BindModelAsync(ModelBindingContext
>  
> bindingContext)
>at 
> Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder.BindModelAsync(ActionContext
>  
> actionContext, IModelBinder modelBinder, IValueProvider valueProvider, 
> ParameterDescriptor parameter, ModelMetadata metadata, Object value, Object 
> container)
>at 
> Microsoft.AspNetCore.Mvc.Controllers.ControllerBinderDelegateProvider.<>c__DisplayClass0_0.d.MoveNext()
> --- End of stack trace from previous location ---
>at 
> Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.g__Awaited|13_0(ControllerActionInvoker
>  
> invoker, Task lastTask, State next, Scope scope, Object state, Boolean 
> isCompleted)
>at 
> Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|26_0(ResourceInvoker
>  
> invoker, Task lastTask, State next, Scope scope, Object state, Boolean 
> isCompleted
>
>
> 2) This is the proto:
>
> The request with the 'Meta' element:
>
> message Request {
>Meta Meta = 1;
>repeated int32 Hotels = 2 [packed = false];
>Country Market = 3;
>repeated Room Rooms = 4;
>.bcl.DateTime CheckIn = 5;
>.bcl.DateTime CheckOut = 6;
>OptionalCriteria OptionalCriteria = 7;
> }
>
> The 'Meta' element that contains the 'CutOffTime' that we want to modify:
>
> message Meta {
>int32 Client = 1;
>int32 Brand = 2;
>bool UseCache = 3;
>.bcl.TimeSpan CutOffTime = 4;
>bool B2C = 5;
>Language Language = 6;
>Currency Currency = 7;
>bool IncludeProviderAuditData = 8;
>SalesChannel SalesChannel = 9;
>string AgentId = 10;
> }
>
> The 'CutOffTime':
>
> message TimeSpan {
>   sint64 value = 1; // the size of the timespan (in units of the selected 
> scale)
>   TimeSpanScale scale = 2; // the scale of the timespan [default = DAYS]
>   enum TimeSpanScale {
> DAYS = 0;
> HOURS = 1;
> MINUTES = 2;
> SECONDS = 3;
> MILLISECONDS = 4;
> TICKS = 5;
>
> MINMAX = 15; // dubious
>   }
> }
>
> Thanks,
>
> Joan.
>
>
>
>
>
>
> On Wednesday, September 20, 2023 at 10:40:08 AM UTC+2 Florian Enner wrote:
>
>> 1) A "varint" is a "variable length integer". When you replace a large 
>> number with a small one, it's entirely possible to lose some bytes and 
>> still be valid. You need to check the actual output.
>>
>> 2) Can you provide the proto definition of the field you want to modify? 
>> Scalar fields get set to the last encountered value, so the easiest option 
>> may be to copy the original bytes and append a delta containing the 
>> differences.
>>  
>>
>>
>>
>> On Wednesday, September 20, 2023 at 10:19:07 AM UTC+2 Joan Balagueró 
>> wrote:
>>
>>> Hello,
>>>
>>> I have a protobuf message like this into a byte array:
>>>
>>> 1: { // META element
>>> 1: 2
>>> 2: 1
>>> 3: 1
>>>

[protobuf] Re: Trying to rewrite a protobuf message but changing a couple of values

2023-09-20 Thread Joan Balagueró

Hi Florian,

Thanks for your quick response. I'm stuck on this.

1) It's not working. When I send the protobuf to the backend server (it's 
not our api nor server) using the first method, I get a right response. But 
using the second method I receive this error:
ProtoBuf.ProtoException: Invalid wire-type; this usually means you have 
over-written a file without truncating or setting the length; see 
https://stackoverflow.com/q/2152978/23354
   at ProtoBuf.ProtoReader.StartSubItem(ProtoReader reader) in 
C:\code\protobuf-net\src\protobuf-net\ProtoReader.cs:line 637
   at ProtoBuf.ProtoReader.ReadTypedObject(Object value, Int32 key, 
ProtoReader reader, Type type) in 
C:\code\protobuf-net\src\protobuf-net\ProtoReader.cs:line 584
   at proto_40(Object , ProtoReader )
   at ProtoBuf.Meta.TypeModel.DeserializeCore(ProtoReader reader, Type 
type, Object value, Boolean noAutoCreate) in 
C:\code\protobuf-net\src\protobuf-net\Meta\TypeModel.cs:line 722
   at ProtoBuf.Meta.TypeModel.Deserialize(Stream source, Object value, Type 
type, SerializationContext context) in 
C:\code\protobuf-net\src\protobuf-net\Meta\TypeModel.cs:line 599
   at 
WebBeds.Connect.AspNetCore.Formatters.ProtobufInputFormatter.ReadRequestBodyAsync(InputFormatterContext

context)
   at 
Microsoft.AspNetCore.Mvc.Formatters.InputFormatter.ReadAsync(InputFormatterContext

context)
   at 
Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder.BindModelAsync(ModelBindingContext

bindingContext)
   at 
Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder.BindModelAsync(ActionContext

actionContext, IModelBinder modelBinder, IValueProvider valueProvider, 
ParameterDescriptor parameter, ModelMetadata metadata, Object value, Object 
container)
   at 
Microsoft.AspNetCore.Mvc.Controllers.ControllerBinderDelegateProvider.<>c__DisplayClass0_0.d.MoveNext()
--- End of stack trace from previous location ---
   at 
Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.g__Awaited|13_0(ControllerActionInvoker

invoker, Task lastTask, State next, Scope scope, Object state, Boolean 
isCompleted)
   at 
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|26_0(ResourceInvoker

invoker, Task lastTask, State next, Scope scope, Object state, Boolean 
isCompleted

2) This is the proto:

The request with the 'Meta' element:

message Request {
   Meta Meta = 1;
   repeated int32 Hotels = 2 [packed = false];
   Country Market = 3;
   repeated Room Rooms = 4;
   .bcl.DateTime CheckIn = 5;
   .bcl.DateTime CheckOut = 6;
   OptionalCriteria OptionalCriteria = 7;
}

The 'Meta' element that contains the 'CutOffTime' that we want to modify:

message Meta {
   int32 Client = 1;
   int32 Brand = 2;
   bool UseCache = 3;
   .bcl.TimeSpan CutOffTime = 4;
   bool B2C = 5;
   Language Language = 6;
   Currency Currency = 7;
   bool IncludeProviderAuditData = 8;
   SalesChannel SalesChannel = 9;
   string AgentId = 10;
}

The 'CutOffTime':

message TimeSpan {
  sint64 value = 1; // the size of the timespan (in units of the selected 
scale)
  TimeSpanScale scale = 2; // the scale of the timespan [default = DAYS]
  enum TimeSpanScale {
DAYS = 0;
HOURS = 1;
MINUTES = 2;
SECONDS = 3;
MILLISECONDS = 4;
TICKS = 5;

MINMAX = 15; // dubious
  }
}

Thanks,

Joan.

On Wednesday, September 20, 2023 at 10:40:08 AM UTC+2 Florian Enner wrote:

> 1) A "varint" is a "variable length integer". When you replace a large 
> number with a small one, it's entirely possible to lose some bytes and 
> still be valid. You need to check the actual output.
>
> 2) Can you provide the proto definition of the field you want to modify? 
> Scalar fields get set to the last encountered value, so the easiest option 
> may be to copy the original bytes and append a delta containing the 
> differences.
>  
>
>
>
> On Wednesday, September 20, 2023 at 10:19:07 AM UTC+2 Joan Balagueró wrote:
>
>> Hello,
>>
>> I have a protobuf message like this into a byte array:
>>
>> 1: { // META element
>> 1: 2
>> 2: 1
>> 3: 1
>> 4: {// CutOffTime element within META
>> 1: 10
>> 2: 3
>>}
>> 5: 1
>> 6: 40
>> }
>> 2: 9836 // HOTEL element
>> 3: 724 // MARKET element
>>
>>
>> We need to traverse this message and write it to a 'CodeOutputStream', 
>> but changing the values of the 'cutoff' element to, for example to '4: { 1: 
>> 7, 2: 4 }'. I'm not able to do it, I need some help.
>>
>> A basic algorithm that writes the same protobuf to another byte array but 
>> without changing anything works. Here I try the special case of 'META' (key 
>> = 1) that contains the 'cutoff' element.
>>
>> Map rootFields = 
>> UnknownFieldSet.parseFrom(document).asMap();
>>
>> for (Map.Entry entry : 
>> rootFields.entrySet()) {
>> if (entry.getKey() == 1) {
>> ByteString bs

[protobuf] Re: Trying to rewrite a protobuf message but changing a couple of values

2023-09-20 Thread Florian Enner

1) A "varint" is a "variable length integer". When you replace a large 
number with a small one, it's entirely possible to lose some bytes and 
still be valid. You need to check the actual output.

2) Can you provide the proto definition of the field you want to modify? 
Scalar fields get set to the last encountered value, so the easiest option 
may be to copy the original bytes and append a delta containing the 
differences.
 



On Wednesday, September 20, 2023 at 10:19:07 AM UTC+2 Joan Balagueró wrote:

> Hello,
>
> I have a protobuf message like this into a byte array:
>
> 1: { // META element
> 1: 2
> 2: 1
> 3: 1
> 4: {// CutOffTime element within META
> 1: 10
> 2: 3
>}
> 5: 1
> 6: 40
> }
> 2: 9836 // HOTEL element
> 3: 724 // MARKET element
>
>
> We need to traverse this message and write it to a 'CodeOutputStream', but 
> changing the values of the 'cutoff' element to, for example to '4: { 1: 7, 
> 2: 4 }'. I'm not able to do it, I need some help.
>
> A basic algorithm that writes the same protobuf to another byte array but 
> without changing anything works. Here I try the special case of 'META' (key 
> = 1) that contains the 'cutoff' element.
>
> Map rootFields = 
> UnknownFieldSet.parseFrom(document).asMap();
>
> for (Map.Entry entry : 
> rootFields.entrySet()) {
> if (entry.getKey() == 1) {
> ByteString bs = 
> entry.getValue().getLengthDelimitedList().get(0);
> output.writeByteArray(1, bs.toByteArray());
> }
> else {
> entry.getValue().writeTo(entry.getKey(), output);
> }
> }
>
>
> Now I try to go a bit further, trying to read the cuotff element, change 
> the values and rewrite them to the 'output'. And here is when I'm not able 
> to solve the problem. Below my try that does not work, it generates a byte 
> array
> of 69 bytes instead of 73 (I'm losing 4 bytes):
>
> Map rootFields = 
> UnknownFieldSet.parseFrom(document).asMap();
> 
> for (Map.Entry entry : 
> rootFields.entrySet()) {
> if (entry.getKey() == 1) {
> ByteString bs = 
> entry.getValue().getLengthDelimitedList().get(0);
> Map ufs = 
> UnknownFieldSet.parseFrom(bs).asMap();
>
> for (Map.Entry item : 
> ufs.entrySet()) {
> if (item.getKey() == 4) {
> Map cutoff = 
> UnknownFieldSet.parseFrom(item.getValue().getLengthDelimitedList().get(0)).asMap();
> cutoff.put(1, 
> UnknownFieldSet.Field.newBuilder().addVarint(7).build()).writeTo(1, output);
> cutoff.put(2, 
> UnknownFieldSet.Field.newBuilder().addVarint(4).build()).writeTo(1, output);
> }
> else {
> item.getValue().writeTo(item.getKey(), output);
> }
> }
> }
> else {
> entry.getValue().writeTo(entry.getKey(), output);
> }
> }
>
> Any help would be really appreciated.
>
> Thanks,
>
> Joan.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/protobuf/d7cf1a43-2613-4dd1-8e80-2fbbd2b5db77n%40googlegroups.com.

[protobuf] Re: Trying to rewrite a protobuf message but changing a couple of values

[protobuf] Re: Trying to rewrite a protobuf message but changing a couple of values

[protobuf] Re: Trying to rewrite a protobuf message but changing a couple of values

[protobuf] Re: Trying to rewrite a protobuf message but changing a couple of values

[protobuf] Re: Trying to rewrite a protobuf message but changing a couple of values

[protobuf] Re: Trying to rewrite a protobuf message but changing a couple of values

6 matches

Site Navigation

Mail list logo

Footer information