Re: ARROW-1463: SubTask ARROW-1472: Design updated Value Vector hierarchy.

2017-10-03 Thread Bryan Cutler
Thanks for doing this Siddarth! It sounds like a good idea to post a WIP so we can all take a look and try out some things out. There's a lot going on so it's difficult to try to solve it all on paper, so maybe as we can start with something very rough and take it from there. I think we all agre

Re: ARROW-1463: SubTask ARROW-1472: Design updated Value Vector hierarchy.

2017-10-03 Thread Siddharth Teotia
Anyway, I will create a PR in some time for the WIP prototype. I think once people eyeball the code there, we may have a consensus. On Tue, Oct 3, 2017 at 3:53 PM, Siddharth Teotia wrote: > Li, > > This is exactly what I was referring to in my previous email. I think if > we have the opportunit

Re: ARROW-1463: SubTask ARROW-1472: Design updated Value Vector hierarchy.

2017-10-03 Thread Siddharth Teotia
Li, This is exactly what I was referring to in my previous email. I think if we have the opportunity (which we have now), we should see if complex template code like this can be removed unless it is absolutely necessary. This is why I was suggesting if we can just not have any code generated class

Re: ARROW-1463: SubTask ARROW-1472: Design updated Value Vector hierarchy.

2017-10-03 Thread Li Jin
Siddharth, Thanks for the update. Without really sit down and do the prototype, my opinions can be wrong. But, I think a lot of the complication are code like this: <#elseif minor.class == "Decimal"> public void get(int index, ${minor.class}Holder holder) { holder.start = index * ${type.width};

Re: ARROW-1463: SubTask ARROW-1472: Design updated Value Vector hierarchy.

2017-10-03 Thread Siddharth Teotia
I am in the middle of a simple prototype that has the basic implementation of BaseFixedWidthVector, FixedValueVectorsPrototype.java (template) to generate a simple IntVector using the proposal mentioned in the document. I have realized that even though the LOCs in existing templates are reduced by

ARROW-1463: SubTask ARROW-1472: Design updated Value Vector hierarchy.

2017-10-03 Thread Siddharth Teotia
Hi All, You should have received an invitation to edit the following document. Please feel free to add comments or additional content. https://docs.google.com/document/d/1rl0PK5OnbQAnFUrhd4bQPtP0u7930sBKKaiyggOY7t4/edit Let me know if the document is not editable. Thanks, Siddharth