[jira] [Commented] (SPARK-38146) UDAF fails with unsafe rows containing a TIMESTAMP_NTZ column

2022-02-08 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489207#comment-17489207 ] Bruce Robbins commented on SPARK-38146: --- This affects master only and has a simple

[jira] [Comment Edited] (SPARK-38146) UDAF fails with unsafe rows containing a TIMESTAMP_NTZ column

2022-02-08 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489207#comment-17489207 ] Bruce Robbins edited comment on SPARK-38146 at 2/9/22, 2:23 AM: --

[jira] [Updated] (SPARK-38146) UDAF fails with unsafe row buffer containing a TIMESTAMP_NTZ column

2022-02-09 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-38146: -- Summary: UDAF fails with unsafe row buffer containing a TIMESTAMP_NTZ column (was: UDAF fails

[jira] [Updated] (SPARK-38146) UDAF fails to aggregate TIMESTAMP_NTZ column

2022-02-09 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-38146: -- Summary: UDAF fails to aggregate TIMESTAMP_NTZ column (was: UDAF fails with unsafe row buffer

[jira] [Created] (SPARK-38221) Group by a stream of complex expressions fails

2022-02-15 Thread Bruce Robbins (Jira)
Bruce Robbins created SPARK-38221: - Summary: Group by a stream of complex expressions fails Key: SPARK-38221 URL: https://issues.apache.org/jira/browse/SPARK-38221 Project: Spark Issue Type:

[jira] [Commented] (SPARK-38221) Group by a stream of complex expressions fails

2022-02-15 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17492869#comment-17492869 ] Bruce Robbins commented on SPARK-38221: --- I think I have an idea what's going on. I

[jira] [Created] (SPARK-38308) Select of a stream of window expressions fails

2022-02-23 Thread Bruce Robbins (Jira)
Bruce Robbins created SPARK-38308: - Summary: Select of a stream of window expressions fails Key: SPARK-38308 URL: https://issues.apache.org/jira/browse/SPARK-38308 Project: Spark Issue Type:

[jira] [Commented] (SPARK-38308) Select of a stream of window expressions fails

2022-02-23 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17497035#comment-17497035 ] Bruce Robbins commented on SPARK-38308: --- The cause is similar issue to that of SPA

[jira] [Commented] (SPARK-38285) ClassCastException: GenericArrayData cannot be cast to InternalRow

2022-02-23 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17497111#comment-17497111 ] Bruce Robbins commented on SPARK-38285: --- Since {{eo.b}} is an array of sttructs, d

[jira] [Comment Edited] (SPARK-38285) ClassCastException: GenericArrayData cannot be cast to InternalRow

2022-02-23 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17497111#comment-17497111 ] Bruce Robbins edited comment on SPARK-38285 at 2/24/22, 2:01 AM: -

[jira] [Commented] (SPARK-38285) ClassCastException: GenericArrayData cannot be cast to InternalRow

2022-02-24 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17497662#comment-17497662 ] Bruce Robbins commented on SPARK-38285: --- I see your point. It appears to be cause

[jira] [Comment Edited] (SPARK-38285) ClassCastException: GenericArrayData cannot be cast to InternalRow

2022-02-24 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17497662#comment-17497662 ] Bruce Robbins edited comment on SPARK-38285 at 2/24/22, 7:19 PM: -

[jira] [Created] (SPARK-38528) NullPointerException when selecting a generator in a Stream of aggregate expressions

2022-03-11 Thread Bruce Robbins (Jira)
Bruce Robbins created SPARK-38528: - Summary: NullPointerException when selecting a generator in a Stream of aggregate expressions Key: SPARK-38528 URL: https://issues.apache.org/jira/browse/SPARK-38528

[jira] [Commented] (SPARK-38528) NullPointerException when selecting a generator in a Stream of aggregate expressions

2022-03-11 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17505149#comment-17505149 ] Bruce Robbins commented on SPARK-38528: --- This is a bug in {{ExtractGenerator}} in

[jira] [Updated] (SPARK-38308) Select of a stream of window expressions fails

2022-03-18 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-38308: -- Affects Version/s: 3.4.0 > Select of a stream of window expressions fails > --

[jira] [Created] (SPARK-24758) Create table wants to use /user/hive/warehouse in clean clone

2018-07-07 Thread Bruce Robbins (JIRA)
Bruce Robbins created SPARK-24758: - Summary: Create table wants to use /user/hive/warehouse in clean clone Key: SPARK-24758 URL: https://issues.apache.org/jira/browse/SPARK-24758 Project: Spark

[jira] [Commented] (SPARK-23629) Building streaming-kafka-0-8-assembly or streaming-flume-assembly adds incompatible jline jar to assembly

2018-07-07 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535944#comment-16535944 ] Bruce Robbins commented on SPARK-23629: --- Whatever was causing this, it is now gone

[jira] [Resolved] (SPARK-23629) Building streaming-kafka-0-8-assembly or streaming-flume-assembly adds incompatible jline jar to assembly

2018-07-07 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins resolved SPARK-23629. --- Resolution: Cannot Reproduce > Building streaming-kafka-0-8-assembly or streaming-flume-asse

[jira] [Created] (SPARK-24814) Relationship between catalog and datasources

2018-07-15 Thread Bruce Robbins (JIRA)
Bruce Robbins created SPARK-24814: - Summary: Relationship between catalog and datasources Key: SPARK-24814 URL: https://issues.apache.org/jira/browse/SPARK-24814 Project: Spark Issue Type: Ne

[jira] [Updated] (SPARK-24814) Relationship between catalog and datasources

2018-07-18 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-24814: -- Description: This is somewhat related, though not identical to, [~rdblue]'s SPIP on datasourc

[jira] [Commented] (SPARK-24814) Relationship between catalog and datasources

2018-07-23 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16553336#comment-16553336 ] Bruce Robbins commented on SPARK-24814: --- [~rdblue] Your parquet example is a compe

[jira] [Updated] (SPARK-24814) Relationship between catalog and datasources

2018-07-23 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-24814: -- Description: This is somewhat related, though not identical to, [~rdblue]'s SPIP on datasourc

[jira] [Created] (SPARK-24912) Broadcast join OutOfMemory stack trace obscures actual cause of OOM

2018-07-24 Thread Bruce Robbins (JIRA)
Bruce Robbins created SPARK-24912: - Summary: Broadcast join OutOfMemory stack trace obscures actual cause of OOM Key: SPARK-24912 URL: https://issues.apache.org/jira/browse/SPARK-24912 Project: Spark

[jira] [Updated] (SPARK-24912) Broadcast join OutOfMemory stack trace obscures actual cause of OOM

2018-07-24 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-24912: -- Priority: Minor (was: Major) > Broadcast join OutOfMemory stack trace obscures actual cause o

[jira] [Created] (SPARK-24914) totalSize is not a good estimate for broadcast joins

2018-07-24 Thread Bruce Robbins (JIRA)
Bruce Robbins created SPARK-24914: - Summary: totalSize is not a good estimate for broadcast joins Key: SPARK-24914 URL: https://issues.apache.org/jira/browse/SPARK-24914 Project: Spark Issue

[jira] [Updated] (SPARK-24914) totalSize is not a good estimate for broadcast joins

2018-07-24 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-24914: -- Description: When determining whether to do a broadcast join, Spark estimates the size of the

[jira] [Updated] (SPARK-24914) totalSize is not a good estimate for broadcast joins

2018-07-25 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-24914: -- Description: When determining whether to do a broadcast join, Spark estimates the size of the

[jira] [Commented] (SPARK-24914) totalSize is not a good estimate for broadcast joins

2018-07-25 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16555998#comment-16555998 ] Bruce Robbins commented on SPARK-24914: --- [~irashid] {quote} given HIVE-20079, can

[jira] [Commented] (SPARK-23207) Shuffle+Repartition on an DataFrame could lead to incorrect answers

2018-08-10 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16577041#comment-16577041 ] Bruce Robbins commented on SPARK-23207: --- I can help out here. I will make a PR for

[jira] [Created] (SPARK-25164) Parquet reader builds entire list of columns once for each column

2018-08-20 Thread Bruce Robbins (JIRA)
Bruce Robbins created SPARK-25164: - Summary: Parquet reader builds entire list of columns once for each column Key: SPARK-25164 URL: https://issues.apache.org/jira/browse/SPARK-25164 Project: Spark

[jira] [Updated] (SPARK-25164) Parquet reader builds entire list of columns once for each column

2018-08-20 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-25164: -- Description: {{VectorizedParquetRecordReader.initializeInternal}} loops through each column,

[jira] [Commented] (SPARK-25164) Parquet reader builds entire list of columns once for each column

2018-08-21 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588168#comment-16588168 ] Bruce Robbins commented on SPARK-25164: --- [~viirya] Sure. I will try to get somethi

[jira] [Commented] (SPARK-23207) Shuffle+Repartition on an DataFrame could lead to incorrect answers

2018-08-23 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16590847#comment-16590847 ] Bruce Robbins commented on SPARK-23207: --- Will we be back-porting this to 2.1, or d

[jira] [Commented] (SPARK-24316) Spark sql queries stall for column width more than 6k for parquet based table

2018-09-04 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16603453#comment-16603453 ] Bruce Robbins commented on SPARK-24316: --- This is likely SPARK-25164. > Spark sql

[jira] [Commented] (SPARK-24043) InterpretedPredicate.eval fails if expression tree contains Nondeterministic expressions

2018-04-23 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16449281#comment-16449281 ] Bruce Robbins commented on SPARK-24043: --- [~maropu] > Do I miss any precondition?

[jira] [Commented] (SPARK-24043) InterpretedPredicate.eval fails if expression tree contains Nondeterministic expressions

2018-04-23 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16449352#comment-16449352 ] Bruce Robbins commented on SPARK-24043: --- You're half-way there. When whole-stage co

[jira] [Commented] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

2018-04-25 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452790#comment-16452790 ] Bruce Robbins commented on SPARK-23715: --- [~cloud_fan] I'll give separate answers f

[jira] [Comment Edited] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

2018-04-25 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452790#comment-16452790 ] Bruce Robbins edited comment on SPARK-23715 at 4/25/18 10:00 PM: --

[jira] [Commented] (SPARK-23580) Interpreted mode fallback should be implemented for all expressions & projections

2018-04-26 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454918#comment-16454918 ] Bruce Robbins commented on SPARK-23580: --- Should SortPrefix also get this treatment?

[jira] [Commented] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

2018-04-28 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16457657#comment-16457657 ] Bruce Robbins commented on SPARK-23715: --- Maybe a configuration setting or differenc

[jira] [Commented] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

2018-04-28 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16457665#comment-16457665 ] Bruce Robbins commented on SPARK-23715: --- {quote}Which version did you use?{quote}

[jira] [Commented] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

2018-04-28 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16457674#comment-16457674 ] Bruce Robbins commented on SPARK-23715: --- Could be this: HIVE-14412 > from_utc_time

[jira] [Commented] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

2018-04-28 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16457754#comment-16457754 ] Bruce Robbins commented on SPARK-23715: --- I just downloaded and installed hive-2.3.3

[jira] [Commented] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

2018-04-28 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16457874#comment-16457874 ] Bruce Robbins commented on SPARK-23715: --- I might understand what's going on with Hi

[jira] [Commented] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

2018-04-28 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16457892#comment-16457892 ] Bruce Robbins commented on SPARK-23715: --- Still, I filed an Jira with Hive so they w

[jira] [Commented] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

2018-04-28 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16457896#comment-16457896 ] Bruce Robbins commented on SPARK-23715: --- [~hyukjin.kwon] Yes, I also built from sou

[jira] [Created] (SPARK-24119) Add interpreted execution to SortPrefix expression

2018-04-29 Thread Bruce Robbins (JIRA)
Bruce Robbins created SPARK-24119: - Summary: Add interpreted execution to SortPrefix expression Key: SPARK-24119 URL: https://issues.apache.org/jira/browse/SPARK-24119 Project: Spark Issue Ty

[jira] [Created] (SPARK-24142) Add interpreted execution to SortPrefix expression

2018-05-01 Thread Bruce Robbins (JIRA)
Bruce Robbins created SPARK-24142: - Summary: Add interpreted execution to SortPrefix expression Key: SPARK-24142 URL: https://issues.apache.org/jira/browse/SPARK-24142 Project: Spark Issue Ty

[jira] [Updated] (SPARK-24142) Add interpreted execution to SortPrefix expression

2018-05-01 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-24142: -- Affects Version/s: (was: 2.3.0) 2.4.0 > Add interpreted execution to

[jira] [Commented] (SPARK-24142) Add interpreted execution to SortPrefix expression

2018-05-01 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16460525#comment-16460525 ] Bruce Robbins commented on SPARK-24142: --- I opened another Jira on this a few days a

[jira] [Resolved] (SPARK-24142) Add interpreted execution to SortPrefix expression

2018-05-01 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins resolved SPARK-24142. --- Resolution: Duplicate > Add interpreted execution to SortPrefix expression >

[jira] [Commented] (SPARK-24119) Add interpreted execution to SortPrefix expression

2018-05-01 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16460527#comment-16460527 ] Bruce Robbins commented on SPARK-24119: --- [~maropu] Ahh... we crossed paths and I op

[jira] [Commented] (SPARK-24142) Add interpreted execution to SortPrefix expression

2018-05-01 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16460532#comment-16460532 ] Bruce Robbins commented on SPARK-24142: --- [~maropu] I don't seem to have the Jira au

[jira] [Commented] (SPARK-23936) High-order function: map_concat(map1, map2, ..., mapN) → map

2018-05-04 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16464245#comment-16464245 ] Bruce Robbins commented on SPARK-23936: --- [~ueshin] I have a question about map_con

[jira] [Commented] (SPARK-23936) High-order function: map_concat(map1, map2, ..., mapN) → map

2018-05-31 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496599#comment-16496599 ] Bruce Robbins commented on SPARK-23936: --- tl;dr version: Spark's Map type allows du

[jira] [Created] (SPARK-24633) arrays_zip function's code generator splits input processing incorrectly

2018-06-22 Thread Bruce Robbins (JIRA)
Bruce Robbins created SPARK-24633: - Summary: arrays_zip function's code generator splits input processing incorrectly Key: SPARK-24633 URL: https://issues.apache.org/jira/browse/SPARK-24633 Project: S

[jira] [Created] (SPARK-26450) Map of schema is built too frequently in some wide queries

2018-12-26 Thread Bruce Robbins (JIRA)
Bruce Robbins created SPARK-26450: - Summary: Map of schema is built too frequently in some wide queries Key: SPARK-26450 URL: https://issues.apache.org/jira/browse/SPARK-26450 Project: Spark

[jira] [Updated] (SPARK-26378) Queries of wide CSV/JSON data slowed after SPARK-26151

2018-12-26 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-26378: -- Summary: Queries of wide CSV/JSON data slowed after SPARK-26151 (was: Queries of wide CSV dat

[jira] [Updated] (SPARK-26378) Queries of wide CSV/JSON data slowed after SPARK-26151

2018-12-26 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-26378: -- Description: A recent change significantly slowed the queries of wide CSV tables. For example

[jira] [Commented] (SPARK-26450) Map of schema is built too frequently in some wide queries

2018-12-27 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16729647#comment-16729647 ] Bruce Robbins commented on SPARK-26450: --- I can attempt a patch later today. > Map

[jira] [Created] (SPARK-26496) Test "locality preferences of StateStoreAwareZippedRDD" frequently fails on High Sierra

2018-12-28 Thread Bruce Robbins (JIRA)
Bruce Robbins created SPARK-26496: - Summary: Test "locality preferences of StateStoreAwareZippedRDD" frequently fails on High Sierra Key: SPARK-26496 URL: https://issues.apache.org/jira/browse/SPARK-26496

[jira] [Commented] (SPARK-26680) StackOverflowError if Stream passed to groupBy

2019-01-21 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16748277#comment-16748277 ] Bruce Robbins commented on SPARK-26680: --- I will make a PR for this, but I would li

[jira] [Created] (SPARK-26680) StackOverflowError if Stream passed to groupBy

2019-01-21 Thread Bruce Robbins (JIRA)
Bruce Robbins created SPARK-26680: - Summary: StackOverflowError if Stream passed to groupBy Key: SPARK-26680 URL: https://issues.apache.org/jira/browse/SPARK-26680 Project: Spark Issue Type:

[jira] [Updated] (SPARK-26680) StackOverflowError if Stream passed to groupBy

2019-01-21 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-26680: -- Description: This Java code results in a StackOverflowError: {code:java} List groupByCols = ne

[jira] [Updated] (SPARK-26680) StackOverflowError if Stream passed to groupBy

2019-01-22 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-26680: -- Affects Version/s: 2.4.0 > StackOverflowError if Stream passed to groupBy > --

[jira] [Updated] (SPARK-26680) StackOverflowError if Stream passed to groupBy

2019-01-22 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-26680: -- Affects Version/s: 2.3.2 > StackOverflowError if Stream passed to groupBy > --

[jira] [Created] (SPARK-26707) Insert into table with single struct column fails

2019-01-23 Thread Bruce Robbins (JIRA)
Bruce Robbins created SPARK-26707: - Summary: Insert into table with single struct column fails Key: SPARK-26707 URL: https://issues.apache.org/jira/browse/SPARK-26707 Project: Spark Issue Typ

[jira] [Created] (SPARK-26711) JSON Schema inference takes 15 times longer

2019-01-23 Thread Bruce Robbins (JIRA)
Bruce Robbins created SPARK-26711: - Summary: JSON Schema inference takes 15 times longer Key: SPARK-26711 URL: https://issues.apache.org/jira/browse/SPARK-26711 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-26711) JSON Schema inference takes 15 times longer

2019-01-23 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16750704#comment-16750704 ] Bruce Robbins commented on SPARK-26711: --- ping [~maxgekk] [~hyukjin.kwon] > JSON S

[jira] [Updated] (SPARK-26711) JSON Schema inference takes 15 times longer

2019-01-23 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-26711: -- Description: I noticed that the first benchmark/case of JSONBenchmark ("JSON schema inferring

[jira] [Commented] (SPARK-26711) JSON Schema inference takes 15 times longer

2019-01-23 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16750741#comment-16750741 ] Bruce Robbins commented on SPARK-26711: --- [~hyukjin.kwon] inferTimestamp=: ~13 min

[jira] [Commented] (SPARK-26711) JSON Schema inference takes 15 times longer

2019-01-24 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16751655#comment-16751655 ] Bruce Robbins commented on SPARK-26711: --- Re: 7 minutes vs. 50 seconds: Looking at

[jira] [Commented] (SPARK-26711) JSON Schema inference takes 15 times longer

2019-01-25 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752304#comment-16752304 ] Bruce Robbins commented on SPARK-26711: --- [~hyukjin.kwon] Ok, that worked. I had in

[jira] [Commented] (SPARK-26708) Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan

2019-02-05 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16761289#comment-16761289 ] Bruce Robbins commented on SPARK-26708: --- How does one hit this issue? > Incorrect

[jira] [Comment Edited] (SPARK-26708) Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan

2019-02-05 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16761289#comment-16761289 ] Bruce Robbins edited comment on SPARK-26708 at 2/6/19 12:41 AM: --

[jira] [Commented] (SPARK-26804) Spark sql carries newline char from last csv column when imported

2019-02-08 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16763728#comment-16763728 ] Bruce Robbins commented on SPARK-26804: --- v2.4.0: Fails as described Tip of branch-

[jira] [Commented] (SPARK-26851) CachedRDDBuilder only partially implements double-checked locking

2019-02-08 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16763818#comment-16763818 ] Bruce Robbins commented on SPARK-26851: --- [~maropu] [~cloud_fan] I will let this J

[jira] [Created] (SPARK-26851) CachedRDDBuilder only partially implements double-checked locking

2019-02-08 Thread Bruce Robbins (JIRA)
Bruce Robbins created SPARK-26851: - Summary: CachedRDDBuilder only partially implements double-checked locking Key: SPARK-26851 URL: https://issues.apache.org/jira/browse/SPARK-26851 Project: Spark

[jira] [Comment Edited] (SPARK-26804) Spark sql carries newline char from last csv column when imported

2019-02-08 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16763728#comment-16763728 ] Bruce Robbins edited comment on SPARK-26804 at 2/8/19 10:13 PM: --

[jira] [Updated] (SPARK-26851) CachedRDDBuilder only partially implements double-checked locking

2019-02-08 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-26851: -- Labels: (was: con) > CachedRDDBuilder only partially implements double-checked locking > ---

[jira] [Updated] (SPARK-26851) CachedRDDBuilder only partially implements double-checked locking

2019-02-08 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-26851: -- Labels: con (was: ) > CachedRDDBuilder only partially implements double-checked locking > ---

[jira] [Updated] (SPARK-26851) CachedRDDBuilder only partially implements double-checked locking

2019-02-08 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-26851: -- Description: In CachedRDDBuilder, {{cachedColumnBuffers}} uses double-checked locking to lazi

[jira] [Commented] (SPARK-26804) Spark sql carries newline char from last csv column when imported

2019-02-09 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16764260#comment-16764260 ] Bruce Robbins commented on SPARK-26804: --- [~hipruthvi] It seems that neither 2.3 n

[jira] [Updated] (SPARK-26851) CachedRDDBuilder only partially implements double-checked locking

2019-02-12 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-26851: -- Description: In CachedRDDBuilder, {{cachedColumnBuffers}} uses double-checked locking to lazi

[jira] [Created] (SPARK-26990) Difference in handling of mixed-case partition columns after SPARK-26188

2019-02-25 Thread Bruce Robbins (JIRA)
Bruce Robbins created SPARK-26990: - Summary: Difference in handling of mixed-case partition columns after SPARK-26188 Key: SPARK-26990 URL: https://issues.apache.org/jira/browse/SPARK-26990 Project: S

[jira] [Updated] (SPARK-26990) Difference in handling of mixed-case partition column names after SPARK-26188

2019-02-25 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-26990: -- Summary: Difference in handling of mixed-case partition column names after SPARK-26188 (was:

[jira] [Updated] (SPARK-26990) Difference in handling of mixed-case partition column names after SPARK-26188

2019-02-28 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-26990: -- Fix Version/s: 3.0.0 > Difference in handling of mixed-case partition column names after SPARK

[jira] [Commented] (SPARK-23243) Shuffle+Repartition on an RDD could lead to incorrect answers

2018-09-08 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16608167#comment-16608167 ] Bruce Robbins commented on SPARK-23243: --- Any plans to back port this to 2.2? > Sh

[jira] [Commented] (SPARK-23243) Shuffle+Repartition on an RDD could lead to incorrect answers

2018-09-08 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16608169#comment-16608169 ] Bruce Robbins commented on SPARK-23243: --- BTW, I took a stab at back porting it to

[jira] [Commented] (SPARK-25164) Parquet reader builds entire list of columns once for each column

2018-09-11 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16611331#comment-16611331 ] Bruce Robbins commented on SPARK-25164: --- Thanks [~Tagar] for the feedback. I assum

[jira] [Commented] (SPARK-22036) BigDecimal multiplication sometimes returns null

2018-09-17 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16618104#comment-16618104 ] Bruce Robbins commented on SPARK-22036: --- [~mgaido] In this change, you modified ho

[jira] [Commented] (SPARK-25454) Division between operands with negative scale can cause precision loss

2018-09-18 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619911#comment-16619911 ] Bruce Robbins commented on SPARK-25454: --- Thanks [~mgaido], OK, so the way I under

[jira] [Commented] (SPARK-25164) Parquet reader builds entire list of columns once for each column

2018-09-19 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621125#comment-16621125 ] Bruce Robbins commented on SPARK-25164: --- {quote}I am thinking if it's feasible to

[jira] [Commented] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

2018-09-19 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621452#comment-16621452 ] Bruce Robbins commented on SPARK-23715: --- Hi [~rxin], Thanks for following up with

[jira] [Created] (SPARK-25643) Performance issues querying wide rows

2018-10-04 Thread Bruce Robbins (JIRA)
Bruce Robbins created SPARK-25643: - Summary: Performance issues querying wide rows Key: SPARK-25643 URL: https://issues.apache.org/jira/browse/SPARK-25643 Project: Spark Issue Type: Improveme

[jira] [Commented] (SPARK-25164) Parquet reader builds entire list of columns once for each column

2018-10-04 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16638803#comment-16638803 ] Bruce Robbins commented on SPARK-25164: --- [~Tagar] I've opened SPARK-25643 to keep

[jira] [Commented] (SPARK-25643) Performance issues querying wide rows

2018-10-15 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650866#comment-16650866 ] Bruce Robbins commented on SPARK-25643: --- [~viirya] Yes, in the case where I said "

[jira] [Comment Edited] (SPARK-25643) Performance issues querying wide rows

2018-10-15 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650866#comment-16650866 ] Bruce Robbins edited comment on SPARK-25643 at 10/15/18 10:08 PM:

[jira] [Commented] (SPARK-24758) Create table wants to use /user/hive/warehouse in clean clone

2018-10-31 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670665#comment-16670665 ] Bruce Robbins commented on SPARK-24758: --- This issue was introduced by commit  [b83

[jira] [Commented] (SPARK-23240) PythonWorkerFactory issues unhelpful message when pyspark.daemon produces bogus stdout

2018-02-10 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359568#comment-16359568 ] Bruce Robbins commented on SPARK-23240: --- A little background. A Spark installation

<    1   2   3   4   5   >