Re: Add support for Decimal64

If the timeline is not tight, I can help with Java side implementation. IIRC, we already have have 16 byte and 32 byte 2's complement based decimal vector implementations in Java based off BigDecimal. Is this similar work for 4 and 8 byte implementations? I will have to refresh my memory of code

Re: [ANNOUNCE] New Arrow PMC chair: Wes McKinney

Congratulations, Wes On Fri, Oct 23, 2020, 4:40 PM Neal Richardson wrote: > Congratulations, Wes! > > On Fri, Oct 23, 2020 at 4:35 PM Jacques Nadeau wrote: > > > I am pleased to announce that we have a new PMC chair and VP as per our > > newly started tradition of rotating the chair once a

Re: Help with Java PR backlog

I can take a look as well. On Thu, Jun 11, 2020, 7:18 PM Fan Liya wrote: > I would like to help with the review. > I will spend some time on it late today. > > Best, > Liya Fan > > > On Fri, Jun 12, 2020 at 9:56 AM Wes McKinney wrote: > > > hi folks, > > > > There's a number of Java PRs that

Re: [ANNOUNCE] New Arrow committers: Ji Liu and Liya Fan

invited to be Arrow committers and they have both > > accepted. > > > > Welcome, and thank you for your contributions! > > > -- *Best Regards,* *SIDDHARTH TEOTIA* *2008C6PS540G* *BITS PILANI- GOA CAMPUS* *+91 87911 75932*

Re: [Java] PR Reviewers

cent Java PRs (I think there are a > few dealing directly trying to separate ArrowBuf from Netty which I believe > builds off work you contributed in the past. Those might be a good place > to start). > > On Sun, Feb 2, 2020 at 9:20 PM siddharth teotia > wrote: > >> Hi A

Re: [Java] PR Reviewers

Hi All, I can help review Java PRs. Thanks Sidd On Sun, Feb 2, 2020, 8:37 PM Micah Kornfield wrote: > OK, I think I've triaged the open Java PRs. Lets see how it goes. > > On Mon, Jan 27, 2020 at 11:13 PM Micah Kornfield > wrote: > > > Somewhat related, but are there any thoughts about

Re: ARROW-3191: Making ArrowBuf work with arbitrary memory and setting io.netty.tryReflectionSetAccessible to true for java builds

> > On Thu, May 2, 2019 at 8:43 PM Jacques Nadeau wrote: > > > I'm onboard with this change. > > > > On Fri, Apr 26, 2019 at 2:14 AM Siddharth Teotia > > wrote: > > > > > As part of working on this patch < > > > https://github.com/apache

Re: [DISCUSS][JAVA]Support Fast/Unsafe Vector APIs for Arrow

Looks like there are 2 PRs for this work -- https://github.com/apache/arrow/pull/4186 this PR adds new getUnsafe type APIs to ArrowBuf that don't do checkIndex() before calling PlatformDependent.get(memory address). So the access will go through vector.get() -> buffer.get() ->

Re: ARROW-3191: Status update: Making ArrowBuf work with arbitrary memory

now. I hope people have looked at my previous email for the change I had made to get the jdk >= 9 builds passing. On Thu, Apr 25, 2019 at 3:13 PM Siddharth Teotia wrote: > As part of working on this patch > <https://github.com/apache/arrow/pull/4151>, I ran into a problem with

Re: ARROW-3191: Making ArrowBuf work with arbitrary memory and setting io.netty.tryReflectionSetAccessible to true for java builds

ing this to the maintainers of io.netty.util.internal.ReflectionUtil WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release Thanks. On Thu, Apr 18, 2019 at 3:39 PM Siddharth Teotia

Re: ARROW-3191: Making ArrowBuf work with arbitrary memory

changes). As of now, the java modules build fine but I have to fix test failures. That is in progress. On Wed, Apr 17, 2019 at 6:41 AM Jacques Nadeau wrote: > Are there any other general comments here? If not, let's get this done and > merged. > > On Mon, Apr 15, 2019, 4:19 PM Sidd

Re: ARROW-3191: Making ArrowBuf work with arbitrary memory

the concept of the reader and writer index need to be > on ArrowBuf? It seems like something that could be added as an additional > decoration/wrapper when needed instead of being part of the core structure. > > On Sat, Apr 13, 2019 at 11:26 AM Siddharth Teotia > wrote: > > > Hi A

ARROW-3191: Making ArrowBuf work with arbitrary memory

Hi All, I have put a PR with WIP changes. All the major set of changes have been done to decouple the usage of ArrowBuf and reference management. The ArrowBuf interface is much simpler and clean now. I believe there would be several folks in the community interested in these changes so please

Re: [VOTE] Proposed change to Arrow Flight protocol: endpoint URIs

+1 (binding) On Tue, Apr 9, 2019 at 9:53 PM Kouhei Sutou wrote: > +1 (binding) > > In > "[VOTE] Proposed change to Arrow Flight protocol: endpoint URIs" on Mon, > 8 Apr 2019 20:36:26 +0200, > Antoine Pitrou wrote: > > > > > Hello, > > > > David Li has proposed to make the following change

Re: Java allocate buffer code

this code should release > (combinedSizeWith2ThePowerSize - combinedSize) = 4064 bytes. But, this is > not happening. > > let me know if this example helps. Do we have some other channel to talk? > > Thanks. > Hitesh. > > > > > > > On Thursday, March 28, 2

Re: Java allocate buffer code

u think about it? > Thanks.Hitesh. >On Wednesday, March 27, 2019, 12:23:47 PM PDT, Siddharth Teotia < > siddha...@dremio.com> wrote: > > Hi Hitesh, > > The code you referenced allocates data and validity buffers for a fixed > width vector. It first determines the appropriate buff

Re: Java allocate buffer code

Hi Hitesh, The code you referenced allocates data and validity buffers for a fixed width vector. It first determines the appropriate buffer size for a given value count and then allocates a compound buffer. The compound buffer is then sliced to get data and validity buffers and finally compound

Re: Arrow development sync call today 12pm Eastern / 17:00 UTC

Notes: Attendees: Sidd Wes Ravindra Arvind Shyam Bryan Francois Bryan: 1. Switching over to Java time from Joda time. At Dremio we need to assess the impact of these changes. Bryan will put a WIP PR soon. There has been a discussion about this on mailing list 2. The Gandiva microbenchmark test

[jira] [Created] (ARROW-3194) Fix setValueCount in spitAndTransfer for variable width vectors

Siddharth Teotia created ARROW-3194: --- Summary: Fix setValueCount in spitAndTransfer for variable width vectors Key: ARROW-3194 URL: https://issues.apache.org/jira/browse/ARROW-3194 Project: Apache

Arrow Sync

I have a clash this morning so won't be able to join the call.

Re: [VOTE] Accept donation of Gandiva to Apache Arrow

+1 On Thu, Aug 16, 2018 at 9:57 AM, Julian Hyde wrote: > +1 > On Thu, Aug 16, 2018 at 8:56 AM Wes McKinney wrote: > > > > Dear all, > > > > The developers of Gandiva, an LLVM-based vectorized expression > > evaluation engine for Arrow columnar memory, are proposing to donate > > the project to

Re: [JAVA] SIMD vectorized fill of ArrowBuf from Java primitive type array?

points to copying of data buffer. Further down in the file, you will see BitCopier to copy validity bits. On Mon, Jul 23, 2018 at 5:19 PM, Siddharth Teotia wrote: > Eric, you can take a look here > <https://github.com/dremio/dremio-oss/blob/master/sabot/kernel/src/main/java/com/dremio/sabo

Re: [JAVA] SIMD vectorized fill of ArrowBuf from Java primitive type array?

Eric, you can take a look here how we try to optimize the copy (validity and data) in/out of vectors. We try to start with word-wise copy (64 column values and thus 64

Re: [DISCUSS] Developing a standard memory layout for in-memory records / "row-oriented" data

I am wondering if this can be considered as an opportunity to implement support in Arrow for building high performance in-memory row stores for low latency and high throughput key based queries. In other words, we can design the in-memory record format keeping efficient RDMA reads as one of the

Re: Arrow sync at 12:00 US/Eastern today

I have a conflict so won't be able to join. On Wed, Jun 13, 2018, 5:46 AM Wes McKinney wrote: > As usual we will be meeting at https://meet.google.com/vtm-teks-phx >

Re: JDBC Adapter PR - 1759

Hi Atul, I will take a look today. Thanks, Sidd On Tue, May 29, 2018 at 2:45 AM, Atul Dambalkar wrote: > Hi Sid, Laurent, Uwe, > > Any idea when can someone take a look at the PR https://github.com/apache/ > arrow/pull/1759/. > > Laurent had given bunch of comments earlier and now we have

Re: Is there list writer in Java?

Hi Teddy, Yes UnionListWriter currently doesn't support writing decimals into list vector. Basically we are missing APIs like UnionListWriter.decimal() which will return a DecimalWriter(we already have this) and the latter can be used to write decimals in list. I'd suggest you to go ahead and

Re: Correct way to set NULL values in VarCharVector (Java API)?

Another option is to use the set() API that allows you to indicate whether the value is NULL or not using an isSet parameter (0 for NULL, 1 otherwise). This is similar to holder based APIs where you need to indicate in holder.isSet whether value is NULL or not.

Re: What do people think about a one day get together?

+1. I would love to attend. On Tue, Apr 3, 2018 at 4:18 PM, Kevin Moore wrote: > Sounds great. Quilt Data may be able to sponsor some of the refreshment > costs. > > > Kevin Moore > CEO, Quilt Data, Inc. > ke...@quiltdata.io | LinkedIn

Re: Arrow sync tomorrow: 12:00 US/Eastern, please review packaging thread

Got it: https://meet.google.com/vtm-teks-phx On Wed, Apr 4, 2018 at 8:48 AM, Siddharth Teotia <siddha...@dremio.com> wrote: > Can someone please send me the link to gcal? For some reason it has > vanished from my calendar. > > On Wed, Apr 4, 2018 at 7:49 AM, Li Jin <ice.xel

Re: Arrow sync tomorrow: 12:00 US/Eastern, please review packaging thread

Can someone please send me the link to gcal? For some reason it has vanished from my calendar. On Wed, Apr 4, 2018 at 7:49 AM, Li Jin wrote: > Sorry I have a conflict today so won't be able to join. > > Li > > On Wed, Apr 4, 2018 at 1:53 AM, Bhaskar Mookerji

Re: Trouble Updating Java artifacts

com> wrote: > > Hello, > > > > you need to first setup up Maven to know your Apache credentials: > http://www.apache.org/dev/publishing-maven-artifacts.html#dev-env > > > > I have taken care of the upload, please verify that the artifacts are > all up. > >

Trouble Updating Java artifacts

Hi All, I think the steps mentioned in RM doc for updating java artifacts are incomplete. I am getting the following error: Failed to deploy artifacts: Could not transfer artifact org.apache.arrow:arrow-java-root:pom:0.9.0 from/to apache.releases.https (

Re: Arrow Sync Call Started

, is now 1pm Eastern, or 1 hour > from now. In case there's some confusion, we may need to reschedule > and make sure we're all agreed on what time zone we're pinned to > > On Wed, Mar 21, 2018 at 12:02 PM, Siddharth Teotia <siddha...@dremio.com> > wrote: > > h

Arrow Sync Call Started

https://www.google.com/url?q=https%3A%2F%2Fmeet.google.com%2Fvtm-teks-phx

[jira] [Created] (ARROW-2329) [Website]: 0.9.0 release update

Siddharth Teotia created ARROW-2329: --- Summary: [Website]: 0.9.0 release update Key: ARROW-2329 URL: https://issues.apache.org/jira/browse/ARROW-2329 Project: Apache Arrow Issue Type: Task

Re: [VOTE] Release Apache Arrow 0.9.0 (RC2)

+1 Verified RC on Mac OS. Things look good. Thanks Sidd On Mar 19, 2018 12:31 AM, "Kouhei Sutou" wrote: +1 (binding), tested on Debian GNU/Linux sid with * GCC 7.3.0 * OpenJDK 9.0.4 * Ruby 2.5.0p0 * NodeJS 8.9.3 -- kou In

[jira] [Created] (ARROW-2294) Fix splitAndTransfer for variable width vector

Siddharth Teotia created ARROW-2294: --- Summary: Fix splitAndTransfer for variable width vector Key: ARROW-2294 URL: https://issues.apache.org/jira/browse/ARROW-2294 Project: Apache Arrow

Re: Working towards getting 0.9.0 release candidate up next week

y to help with this on Monday. > > - Wes > > On Thu, Mar 8, 2018 at 2:25 PM, Siddharth Teotia <siddha...@dremio.com> > wrote: > > All, > > > > I plan to get RC out over the weekend or early Monday. Is that fine with > > everybody? > > > > W

Re: Working towards getting 0.9.0 release candidate up next week

to see if > we can make an RC by Friday but if not we can defer to Monday/Tuesday > the following week (3/12 or 13). I will trim as much as possible out > of the current backlog to get things down to the essential > > - Wes > > On Sun, Feb 25, 2018 at 11:58 AM, Siddharth T

Arrow sync call today - March 7

I will be at Strata conference today and won't be able to join the call. Thanks Sidd

Re: Working towards getting 0.9.0 release candidate up next week

Sounds good. Thanks Sidd On Feb 24, 2018 6:24 PM, "Wes McKinney" <wesmck...@gmail.com> wrote: Hi Sidd, I think we have too many bugs to make an RC this coming week. I suggest we defer to the following week. Thanks Wes On Feb 24, 2018 7:09 PM, "Siddharth Teotia" &l

Working towards getting 0.9.0 release candidate up next week

Hi All, We currently have 10 issues in progress and PRs are available for 8 of them. In interest of getting a release candidate next week, I would request people to review PRs as soon as they can to help make progress and close out as many JIRAs as we can. There are 32 issues in TODO list and 25

Re: Allocating additional memory to the Java Vector objects

ation itself (which > I was thinking of doing explicitly in the code). Although it uses "reAlloc" > which allocates double the current size, for me it works - as I plan to > start with moderate initial capacity for the vectors. > > -Atul > > -----Original Message--

Re: Allocating additional memory to the Java Vector objects

Hi Atul, Currently there is no way for doing this. The only exposed method of expanding the vector buffer is reAlloc() and it allocates a new buffer of double the original capacity and copies the old contents into the new buffer. Thanks, Sidd On Fri, Feb 23, 2018 at 12:06 PM, Atul Dambalkar

[jira] [Created] (ARROW-2199) Follow up fixes for ARROW-2019. Ensure density driven capacity is never less than 1 and propagate density throughout the vector tree

Siddharth Teotia created ARROW-2199: --- Summary: Follow up fixes for ARROW-2019. Ensure density driven capacity is never less than 1 and propagate density throughout the vector tree Key: ARROW-2199 URL: https

[jira] [Created] (ARROW-2019) Control the memory allocated for inner vector in LIST

Siddharth Teotia created ARROW-2019: --- Summary: Control the memory allocated for inner vector in LIST Key: ARROW-2019 URL: https://issues.apache.org/jira/browse/ARROW-2019 Project: Apache Arrow

Re: [DRAFT] Apache Arrow board report

gt; > ## Health report: > > > > > > > > Project is very healthy with a growing developer and user community. > > > > > > > > ## PMC changes: > > > > > > > > - Currently 20 PMC members. > > > > - No

[jira] [Created] (ARROW-1946) Add APIs to decimal vector for writing big endian data

Siddharth Teotia created ARROW-1946: --- Summary: Add APIs to decimal vector for writing big endian data Key: ARROW-1946 URL: https://issues.apache.org/jira/browse/ARROW-1946 Project: Apache Arrow

[jira] [Created] (ARROW-1943) Handle setInitialCapacity() for deeply nested lists of lists

Siddharth Teotia created ARROW-1943: --- Summary: Handle setInitialCapacity() for deeply nested lists of lists Key: ARROW-1943 URL: https://issues.apache.org/jira/browse/ARROW-1943 Project: Apache

[jira] [Created] (ARROW-1939) Correct links in release 0.8 blog post

Siddharth Teotia created ARROW-1939: --- Summary: Correct links in release 0.8 blog post Key: ARROW-1939 URL: https://issues.apache.org/jira/browse/ARROW-1939 Project: Apache Arrow Issue Type

Re: arrow read/write examples in Java

>From Arrow 0.8, the second step "Grab the corresponding mutator and accessor objects by calls to getMutator(), getAccessor()" is not needed. In fact, it is not even there. On Tue, Dec 19, 2017 at 10:01 AM, Siddharth Teotia <siddha...@dremio.com> wrote: > Hi Animesh, &g

Re: arrow read/write examples in Java

Hi Animesh, Firstly I would like to suggest switching over to Arrow 0.8 release asap since you are writing JAVA programs and the API usage has changed drastically. The new APIs are much simpler with good javadocs and detailed internal comments. If you are writing stop-gap implementation then it

Re: Confirming Release Owners

Conda is done -- I updated arrow-cpp-feedstock and Uwe took care of parquet-cpp and pyarrow. On Sun, Dec 17, 2017 at 4:14 PM, Jacques Nadeau wrote: > Wes: Post to Dist, Upload Maven artifacts, Send Announce > Jacques: Update website/docs > Sidd (with Help from Uwe): Update

[jira] [Created] (ARROW-1922) Blog post on recent improvements/changes in JAVA Vectors

Siddharth Teotia created ARROW-1922: --- Summary: Blog post on recent improvements/changes in JAVA Vectors Key: ARROW-1922 URL: https://issues.apache.org/jira/browse/ARROW-1922 Project: Apache Arrow

[jira] [Created] (ARROW-1876) Transfer validity vector buffer data word at a time (currently we do byte at a time)

Siddharth Teotia created ARROW-1876: --- Summary: Transfer validity vector buffer data word at a time (currently we do byte at a time) Key: ARROW-1876 URL: https://issues.apache.org/jira/browse/ARROW-1876

General Suggestions and Request regarding recent JAVA changes.

Folks, Over the last couple of weeks, we have had several changes (both merged and in the pipeline) as follow-up work after ARROW-1463 was merged. I feel that refactoring suggestions are being proposed on-the-fly while the developer is already in progress with the code changes and it's too late

[jira] [Created] (ARROW-1813) Enforce checkstyle failure in JAVA build and fix all checkstyle

Siddharth Teotia created ARROW-1813: --- Summary: Enforce checkstyle failure in JAVA build and fix all checkstyle Key: ARROW-1813 URL: https://issues.apache.org/jira/browse/ARROW-1813 Project: Apache

Update on ARROW-1463 - Request for merging java-vector-refactor into master

Functional and Performance testing has been completed with Dremio. We have seen overall improvement in TPCH numbers. We had about 8000 regression tests and 12000 unit tests. I would like to start the process of merging java-vector-refactor branch into master. The branch has 2 patches with 95% of

[jira] [Created] (ARROW-1807) [JAVA] Reduce Heap Usage (Phase 3): consolidate buffers

Siddharth Teotia created ARROW-1807: --- Summary: [JAVA] Reduce Heap Usage (Phase 3): consolidate buffers Key: ARROW-1807 URL: https://issues.apache.org/jira/browse/ARROW-1807 Project: Apache Arrow

Re: [DISCUSS] readerIndex/writerIndex in Java vector refactor

ing Java vector refactor code and come cross > readerIndex/writerIndex on ArrowBuf. This issue has been brought up by > Siddharth > Teotia earlier but I might have missed the discussion so what to clarify. > > My understanding is that the current implementation in java refactor bran

Re: Arrow sync today

There was no meeting today. On Wed, Nov 1, 2017 at 10:14 AM, Li Jin wrote: > I wasn't able to join the chat room so not sure what's going on. Did we > have the meeting? > On Wed, Nov 1, 2017 at 12:55 PM Bryan Cutler wrote: > > > Sorry, I won't be able

Re: Arrow sync today

I have joined the meeting here -- https://meet.google.com/vtm-teks-phx I don't see anybody. Can someone please send out the correct link? On Wed, Nov 1, 2017 at 8:43 AM, Wes McKinney wrote: > I am not able to attend today’s Arrow sync. Others are free to meet and > relay

Re: Update on ARROW-1463, related subtasks and plan for testing and merging

; On Fri, Oct 13, 2017 at 5:42 PM, Siddharth Teotia <siddha...@dremio.com> > wrote: > > Li, > > > > I think there is some confusion. Are you suggesting merging into "java > > vector refactor" branch or the master? Is it fine to merge stuff on th

Re: Update on ARROW-1463, related subtasks and plan for testing and merging

people can comment and > discuss. > > Li > > > > > On Fri, Oct 13, 2017 at 2:37 PM, Siddharth Teotia <siddha...@dremio.com> > wrote: > > > I am not quite sure of the need to rename the vectors. Why do we need to > > rename? This would first require u

Re: Update on ARROW-1463, related subtasks and plan for testing and merging

vectors to new vectors (zero-copy, as the underlying buffers should be > transferred directly) > > Jacques also has a few comments, I don't know if they have been addressed. > > For other comments, I think we can add TODO and do it later. I think we can > merge this PR if we address

Re: Update on ARROW-1463, related subtasks and plan for testing and merging

been done. On the testing front, as far as the correctness is concerned, I have two failures in TestArrowFile and TestValueVector. I have added some more tests too. On Thu, Oct 12, 2017 at 2:18 PM, Siddharth Teotia <siddha...@dremio.com> wrote: > Yes, that is the intentio

Re: Update on ARROW-1463, related subtasks and plan for testing and merging

directly because Spark's use of Arrow is > very > > simple. > > > > > > > > On Thu, Oct 12, 2017 at 2:08 PM, Siddharth Teotia <siddha...@dremio.com> > > wrote: > > > >> Thanks Bryan and Li. > >> > >> Yes, the goal is

Re: Update on ARROW-1463, related subtasks and plan for testing and merging

- Spark Integration Tests: > > > > Bryan and I can help with integration test with Spark. I think the target > > timeline for Spark 2.3 release is some time in mid Nov (Bryan please > > correct me if I am wrong). > > > > I will take a look at the PR today. >

Update on ARROW-1463, related subtasks and plan for testing and merging

Hi All, I wanted to update everyone on state of this mini-project: - Requirements document and initial design proposal were sent out to the community for review and we have received some good feedback. All required docs are attached with corresponding JIRAs. - The initial

[jira] [Created] (ARROW-1655) Add Scale and Precision to ValueVectorTypes.tdd for Decimals

Siddharth Teotia created ARROW-1655: --- Summary: Add Scale and Precision to ValueVectorTypes.tdd for Decimals Key: ARROW-1655 URL: https://issues.apache.org/jira/browse/ARROW-1655 Project: Apache

Re: Arrow sync call tomorrow 4 October @ 16:00 UTC

I am out for doctor's appointment. I may have to miss it this time or I might join a bit late. On Oct 4, 2017 7:16 AM, "Wes McKinney" wrote: Heimir has offered to create a Hangout that should accommodate up to 25 participants:

Re: ARROW-1463: SubTask ARROW-1472: Design updated Value Vector hierarchy.

Anyway, I will create a PR in some time for the WIP prototype. I think once people eyeball the code there, we may have a consensus. On Tue, Oct 3, 2017 at 3:53 PM, Siddharth Teotia <siddha...@dremio.com> wrote: > Li, > > This is exactly what I was referring to in my previous

Re: ARROW-1463: SubTask ARROW-1472: Design updated Value Vector hierarchy.

gt; } > > > > Sorry I couldn't provide too much useful feedback without digging into the > template, but this is any general feeling about these templates - too many > "if" to types like "Interval" "Decimal" "Timestamp" > > >

Re: ARROW-1463: SubTask ARROW-1472: Design updated Value Vector hierarchy.

create a PR from this prototype code once it in reasonable shape for review but was wondering if people have any opinion. Thanks, Sidd On Tue, Oct 3, 2017 at 3:16 AM, Siddharth Teotia <siddha...@dremio.com> wrote: > Hi All, > > You should have received an invitation to edit the fol

Re: [ANNOUNCE] New Arrow committers: Phillip Cloud and Bryan Cutler

Congrats Philip and Bryan :) On Tue, Oct 3, 2017 at 11:57 AM, Holden Karau wrote: > Congrats to the both of you :) Really excited to see the Areow project > continue to grow :) > > On Tue, Oct 3, 2017 at 10:24 AM Julian Hyde wrote: > > > Congratulations

Re: ARROW-1463: Subtask ARROW-1471 Requirements for Value Vector Updates

PM, Li Jin <ice.xell...@gmail.com> wrote: > Siddharth, > > The requirement doc looks good to me. Thanks for putting this together. > > Li > > On Sun, Oct 1, 2017 at 1:20 PM, Siddharth Teotia <siddha...@dremio.com> > wrote: > > > Hi All, > > >

Re: ARROW-1463: Subtask ARROW-1471 Requirements for Value Vector Updates

lazy to cut and paste I'll try adding it again > > https://docs.google.com/document/d/1ysZ76zritBDwkeQz3C6- > vhQwD32jEXd1kUF4T936G1U/edit > > On Thu, Sep 21, 2017 at 2:18 PM, Siddharth Teotia <siddha...@dremio.com> > wrote: > > > Hi All, > > > > You should h

[jira] [Created] (ARROW-1621) Reduce Heap Usage per Vector

Siddharth Teotia created ARROW-1621: --- Summary: Reduce Heap Usage per Vector Key: ARROW-1621 URL: https://issues.apache.org/jira/browse/ARROW-1621 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-1618) See if the heap usage in vectors can be reduced.

Siddharth Teotia created ARROW-1618: --- Summary: See if the heap usage in vectors can be reduced. Key: ARROW-1618 URL: https://issues.apache.org/jira/browse/ARROW-1618 Project: Apache Arrow

ARROW-1463: Subtask ARROW-1471 Requirements for Value Vector Updates

Hi All, You should have received an invitation to edit the following document. Please feel free to add comments or additional content. https://docs.google.com/document/d/1ysZ76zritBDwkeQz3C6- vhQwD32jEXd1kUF4T936G1U/edit?usp=sharing Thanks, Siddharth

[jira] [Created] (ARROW-1553) Implement setInitialCapacity for MapWriter and pass on this capacity during lazy creation of child vectors

Siddharth Teotia created ARROW-1553: --- Summary: Implement setInitialCapacity for MapWriter and pass on this capacity during lazy creation of child vectors Key: ARROW-1553 URL: https://issues.apache.org/jira

[jira] [Created] (ARROW-1547) Fix 8x memory over-allocation in BitVector

Siddharth Teotia created ARROW-1547: --- Summary: Fix 8x memory over-allocation in BitVector Key: ARROW-1547 URL: https://issues.apache.org/jira/browse/ARROW-1547 Project: Apache Arrow Issue

[jira] [Created] (ARROW-1533) As part of buffer transfer (transferTo function), we should transfer the state for realloc

Siddharth Teotia created ARROW-1533: --- Summary: As part of buffer transfer (transferTo function), we should transfer the state for realloc Key: ARROW-1533 URL: https://issues.apache.org/jira/browse/ARROW-1533

Re: Travis CI problems

1c889f448b33d21f46). > > The flakiness started sometime in the last 3 days. > > > > If Laurent or someone with Java background could investigate the cause > > and either fix or disable entry in the Travis build matrix, that would > > be great. > > > &g

Travis CI problems

Is anyone else seeing the following failures in Travis CI build? I am seeing these problems for PR https://github.com/apache/arrow/pull/1052 I looked at the raw log and nothing seems to indicate problems w.r.t code changes.

ARROW-1463 subtask assignments - https://issues.apache.org/jira/browse/ARROW-1463

Hi All, I am wondering if anyone is interested in working on sub-tasks for ARROW-1463. Please feel free to grab the child JIRAs. Thanks, Siddharth

[jira] [Created] (ARROW-1478) clear should release the buffer only if the buffer is not NULL

SIDDHARTH TEOTIA created ARROW-1478: --- Summary: clear should release the buffer only if the buffer is not NULL Key: ARROW-1478 URL: https://issues.apache.org/jira/browse/ARROW-1478 Project: Apache

[jira] [Created] (ARROW-1467) Fix reset() and allocateNew() in Nullable Value Vectors template

SIDDHARTH TEOTIA created ARROW-1467: --- Summary: Fix reset() and allocateNew() in Nullable Value Vectors template Key: ARROW-1467 URL: https://issues.apache.org/jira/browse/ARROW-1467 Project: Apache

[jira] [Created] (ARROW-1444) BitVector.splitAndTransfer copies last byte incorrectly

SIDDHARTH TEOTIA created ARROW-1444: --- Summary: BitVector.splitAndTransfer copies last byte incorrectly Key: ARROW-1444 URL: https://issues.apache.org/jira/browse/ARROW-1444 Project: Apache Arrow

Re: Arrow 0.6.0 release planning and timeline

Reviewed https://github.com/apache/arrow/pull/915 for ARROW-1296 On Fri, Aug 4, 2017 at 11:45 AM, Siddharth Teotia <siddha...@dremio.com> wrote: > I will review it by EOD. > > On Fri, Aug 4, 2017 at 11:15 AM, Li Jin <ice.xell...@gmail.com> wrote: > >>

Re: Arrow 0.6.0 release planning and timeline

I will review it by EOD. On Fri, Aug 4, 2017 at 11:15 AM, Li Jin wrote: > On the Java side I have https://issues.apache.org/jira/browse/ARROW-1296, > which is small bug fix. > > If someone help review it would be great. Else if it doesn't get reviewed > by 0.6 rc cut, we

[jira] [Created] (ARROW-1310) Revert ARROW-886

SIDDHARTH TEOTIA created ARROW-1310: --- Summary: Revert ARROW-886 Key: ARROW-1310 URL: https://issues.apache.org/jira/browse/ARROW-1310 Project: Apache Arrow Issue Type: Bug

Re: AppVeyor build taking longer than usual?

our fork; > > on normal days Appveyor jobs will start up right away and finish in > > under an hour, e.g. https://ci.appveyor.com/project/wesm/arrow/history > > > > - Wes > > > > > > On Tue, Aug 1, 2017 at 1:06 AM, Siddharth Teotia <siddha...@dremio.com&g

AppVeyor build taking longer than usual?

Hi All, I created a PR 4 hours ago and the AppVeyor build still hasn't finished. The Travis CI build ran fine. Is this expected? I am surprised since for all my previous PRs, both checks used to get done in 1-2 hours. In fact, the 3 latest 3 PRs (923, 924, 925) are in a similar situation.

[jira] [Created] (ARROW-1300) Fix ListVector Tests

SIDDHARTH TEOTIA created ARROW-1300: --- Summary: Fix ListVector Tests Key: ARROW-1300 URL: https://issues.apache.org/jira/browse/ARROW-1300 Project: Apache Arrow Issue Type: Bug

Re: [Java] Strange reset() method in FixedValueVectors

Hi Li For FixedValueVectors.java template, the initial allocation will happen based on the value of allocationSizeInBytes. For example, for a 4 byte IntVector, this will be 16KB of memory which is equivalent to INITIAL_VALUE_ALLOCATION * ${type_width} in the code. So if the user invokes

[jira] [Created] (ARROW-1267) Handle zero length case in BitVector.splitAndTransfer

SIDDHARTH TEOTIA created ARROW-1267: --- Summary: Handle zero length case in BitVector.splitAndTransfer Key: ARROW-1267 URL: https://issues.apache.org/jira/browse/ARROW-1267 Project: Apache Arrow

[jira] [Created] (ARROW-1249) Expose the fillEmpties function from NullableVector.mutator

SIDDHARTH TEOTIA created ARROW-1249: --- Summary: Expose the fillEmpties function from NullableVector.mutator Key: ARROW-1249 URL: https://issues.apache.org/jira/browse/ARROW-1249 Project: Apache

[jira] [Created] (ARROW-1237) Expose the ability to set lastSet

SIDDHARTH TEOTIA created ARROW-1237: --- Summary: Expose the ability to set lastSet Key: ARROW-1237 URL: https://issues.apache.org/jira/browse/ARROW-1237 Project: Apache Arrow Issue Type

  1   2   >