Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/19601
Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19601
Sure, let me close this
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/19601
Hi, @kiszk . Can we close this for now? You can make another PR later if
you want.
---
-
To unsubscribe, e-mail:
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/19601
Hi, @kiszk . Is this still valid?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84253/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #84253 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84253/testReport)**
for PR 19601 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #84253 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84253/testReport)**
for PR 19601 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #84215 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84215/testReport)**
for PR 19601 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84215/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #84215 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84215/testReport)**
for PR 19601 at commit
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19601
Jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19601
@cloud-fan could you please review this?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84082/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #84082 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84082/testReport)**
for PR 19601 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #84082 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84082/testReport)**
for PR 19601 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84080/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #84080 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84080/testReport)**
for PR 19601 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #84080 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84080/testReport)**
for PR 19601 at commit
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19601
I see. Let us revisit this design later.
I would appreciate it if you would review this columnar cache reader with
simple primitive-type (non-nested) array.
---
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/19601
We'd need to change the `UnsafeArrayData` format too, to avoid data copying
when building the cache. BTW I think it's ok to release this columnar cache
reader without efficient complex type
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19601
There are some parts that relies on the format of `UnsafeArrayData`. I mean
that bit-by-bit copy of `UnsafeArrayData` is performed. Can we handle this copy
using the new format for an unsafe array?
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19601
This approach also works for nested array. I have this implementation in my
machine. For ease of review, I commit the version of only primitive type array
support. If you like it, I can commit the
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/19601
Does it work for non-top-level array type fields and nested array?
Generally I think this is not the right direction. The root cause is that,
table cache array format is not the arrow-style
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83934/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #83934 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83934/testReport)**
for PR 19601 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #83934 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83934/testReport)**
for PR 19601 at commit
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19601
Jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #83931 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83931/testReport)**
for PR 19601 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83931/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #83931 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83931/testReport)**
for PR 19601 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83930/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #83930 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83930/testReport)**
for PR 19601 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #83930 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83930/testReport)**
for PR 19601 at commit
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19601
@cloud-fan could you please review this again? I merged with the
`ColumnarArray`. As you suggested, the latest implementation does not change
`ColumnVector` and `ColumnarArray`.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83905/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #83905 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83905/testReport)**
for PR 19601 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #83905 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83905/testReport)**
for PR 19601 at commit
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19601
@cloud-fan could you please review this again? Now, this PR does not apply
any change to `ColumnVector` and `WritableColumnVector`.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83689/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #83689 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83689/testReport)**
for PR 19601 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #83689 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83689/testReport)**
for PR 19601 at commit
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19601
@cloud-fan could you please review this again since this version avoids to
override `ColumnVector.getArray` as you suggested?
cc @ueshin
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83465/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #83465 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83465/testReport)**
for PR 19601 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #83465 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83465/testReport)**
for PR 19601 at commit
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19601
Jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83463/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #83463 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83463/testReport)**
for PR 19601 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83460/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #83460 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83460/testReport)**
for PR 19601 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #83460 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83460/testReport)**
for PR 19601 at commit
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19601
My prototype for nested array can handle nested array by changing
`UnsafeArray.getArray` and its callee methods, and does not require to change
`ColumnVector`.
If the refactoring takes more
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/19601
can we hold it for a while? I'm thinking about ColumnVector refactoring and
see how to deal with nested data uniformly.
---
-
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19601
@cloud-fan could you please review this PR?
In my prototype, I succeeded to support a current nested array for table
cache by changing only UnsafeColumnVector.java.
For ease of review, I
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19601
@cloud-fan could you please review this PR?
In my prototype, I succeeded to support a current nested array for table
cache by changing only `UnsafeColumnVector.java`.
For ease of review,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83250/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #83250 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83250/testReport)**
for PR 19601 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #83250 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83250/testReport)**
for PR 19601 at commit
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19601
Jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83246/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #83246 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83246/testReport)**
for PR 19601 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #83246 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83246/testReport)**
for PR 19601 at commit
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19601
Jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83240/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19601
Jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83237/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19601
Jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83236/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19601
Jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #83223 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83223/testReport)**
for PR 19601 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83210/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #83210 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83210/testReport)**
for PR 19601 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83208/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19601
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #83208 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83208/testReport)**
for PR 19601 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #83210 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83210/testReport)**
for PR 19601 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19601
**[Test build #83208 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83208/testReport)**
for PR 19601 at commit
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19601
Jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19601
After I think about the choice for a while, I conclude that it is better to
add the new `WritableColumnVector` (i.e. `UnsafeColumnVector`) and to keep the
current `ColumnVector.Array`.
I think
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19601
For now, this implementation has an limitation only to support non-nested
array for ease of review.
---
-
To unsubscribe, e-mail:
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/19601
both ways work, just pick the simpler one. I'm concerned about how to
access the nested array, you can try both approaches and see which one can
solve the problem easier.
---
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19601
I agree with you that we need to improve the write path. It will be
addressed after improving the frequently-executed read path, as you suggested
before. It will be addressed by the following PR.
1 - 100 of 111 matches
Mail list logo
