Hi Bhalchandra,
Do you have any open source code (e.g. forked thrift) to show how you
implemented it? The blog post stated what's the problem but really lack
information on the solution side:
1. How did you solve the problem?
2. In which language(s) did you implement the solution?
Without that information it's really not much we can do here.
Also just based on the problem you described, a simple solution would be
just to duplicate the struct definition and remove the fields you don't
need, for example:
// Original struct
struct Payload {
1: optional Type1 field1,
2: optional Type2 field2,
3: optional Type3 field3,
...
}
// Slim struct
struct SlimPayload {
1: optional Type1 field1,
// field2 removed because we don't care about it in this use case
3: optional SlimType3 field3,
...
}
But of course it's hard to keep SlimPayload in sync with the original
Payload so I can see there are some values to have some helpers to help
that, but as long as you don't make breaking changes into Payload "keep
them in sync" is a false problem.
On Mon, Jul 12, 2021 at 12:56 PM Bhalchandra Pandit
<kpan...@pinterest.com.invalid> wrote:
> Hi All,
> I work for Pinterest. I developed a technique for partial deserialization
> of Thrift that has been very useful in significantly improving efficiency
> of the data processing at Pinterest. I would like to contribute that
> feature to Apache Thrift. More details on this technique are available in
> this blog I recently wrote:
>
> https://medium.com/pinterest-engineering/improving-data-processing-efficiency-using-partial-deserialization-of-thrift-16bc3a4a38b4
>
> I would like to know if any of you are interested in helping with
> contributing this work to the main branch.
>
> Kumar
>
