Re: partial deserialization of Thrift

2021-07-12 Thread Duru Can Celasun
On Tue, 13 Jul 2021, at 01:07, Bhalchandra Pandit wrote: > Thanks for showing interest and for raising valid questions. > > I do not currently have a forked branch to show how it is done. However, I > can certainly make it happen. It will take me a couple of weeks or so to do > it. I will need to

Re: partial deserialization of Thrift

2021-07-12 Thread Bhalchandra Pandit
Thanks for showing interest and for raising valid questions. I do not currently have a forked branch to show how it is done. However, I can certainly make it happen. It will take me a couple of weeks or so to do it. I will need to get myself familiarized with the contribution process. To answer o

Re: partial deserialization of Thrift

2021-07-12 Thread Duru Can Celasun
This is definitely an interesting problem at scale and it'd be great to have a solution upstream. I second Yuxuan's questions. From the blog post it seems you have an implementation for Java, but it would be great to have at least one more. On Tue, 13 Jul 2021, at 00:00, Yuxuan Wang wrote: > H

Re: partial deserialization of Thrift

2021-07-12 Thread Yuxuan Wang
Hi Bhalchandra, Do you have any open source code (e.g. forked thrift) to show how you implemented it? The blog post stated what's the problem but really lack information on the solution side: 1. How did you solve the problem? 2. In which language(s) did you implement the solution? Without that

partial deserialization of Thrift

2021-07-12 Thread Bhalchandra Pandit
Hi All, I work for Pinterest. I developed a technique for partial deserialization of Thrift that has been very useful in significantly improving efficiency of the data processing at Pinterest. I would like to contribute that feature to Apache Thrift. More details on this technique are available in