Re: Java allocate buffer code

2019-03-28 Thread Hitesh
Created jira for it https://issues.apache.org/jira/browse/ARROW-5057 Thanks. Hitesh. On Thursday, March 28, 2019, 11:25:58 AM PDT, Siddharth Teotia wrote: Hitesh, I suggest you file a JIRA for the potential issue you are seeing and if possible raise a PR with a test case that you think i

Re: Java allocate buffer code

2019-03-28 Thread Siddharth Teotia
Hitesh, I suggest you file a JIRA for the potential issue you are seeing and if possible raise a PR with a test case that you think is broken with current code. Happy to discuss on Jira or PR. Thanks, Siddharth On Thu, Mar 28, 2019 at 11:20 AM Hitesh wrote: > Hi Siddharth: > > Here, I see a pr

Re: Java allocate buffer code

2019-03-28 Thread Hitesh
Hi Siddharth: Here, I see a problem in line#162, where its taking "bufferSize" to find the extra allocated bytes. It should be "valueCount*typeWidth + valueCount/8". Here is an example for that. Let's take 1000 ints. Then, valueCount = 1000 ints typWidth = 4 bytes validitiyBufferSize = 125 bytes

Re: Java allocate buffer code

2019-03-28 Thread Siddharth Teotia
Hitesh, Yes, if you see in the code, the sliced buffers have their reference counts bumped up before the compound buffer is released. Bumping up the reference counts of child/sliced buffers allows us to release the compound buffer safely. Does that make sense? Thanks, Siddharth On Wed, Mar 27, 2

Re: Java allocate buffer code

2019-03-27 Thread Hitesh
Hi Siddarth: Thanks. yes, I am referring compound buffer as an extra buffer. This we release and further can be reused? Let's take an example of 1000 ints. Then, it will need the following bytes. getValidityBufferSize: 125value bufferSize: 4000combinedSize: 4128combinedSizeWith2ThePower: 8192 Th

Re: Java allocate buffer code

2019-03-27 Thread Siddharth Teotia
Hi Hitesh, The code you referenced allocates data and validity buffers for a fixed width vector. It first determines the appropriate buffer size for a given value count and then allocates a compound buffer. The compound buffer is then sliced to get data and validity buffers and finally compound bu

Java allocate buffer code

2019-03-27 Thread Hitesh Khamesra
Hi All: I was looking following code to release extra allocated buffer. It seems it should be considering actualCount as "valueCount*typeWidth". Then it should calculate extra buffer and release it. Right now, it calculates based on actually allocated size and not justifying the intend. Any thou