[jira] [Updated] (SPARK-24828) Incompatible parquet formats - java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary$PlainLongDictionary

[ https://issues.apache.org/jira/browse/SPARK-24828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-24828: Attachment: image-2018-07-18-13-57-21-148.png > Incompatible parquet formats -

[jira] [Comment Edited] (SPARK-24828) Incompatible parquet formats - java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary$PlainLongDictionary

[ https://issues.apache.org/jira/browse/SPARK-24828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547418#comment-16547418 ] Romeo Kienzer edited comment on SPARK-24828 at 7/18/18 5:53 AM: Dear

[jira] [Commented] (SPARK-24828) Incompatible parquet formats - java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary$PlainLongDictionary

[ https://issues.apache.org/jira/browse/SPARK-24828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547418#comment-16547418 ] Romeo Kienzer commented on SPARK-24828: --- Dear [~q79969786] - thanks for pointing this out.

[jira] [Assigned] (SPARK-24840) do not use dummy filter to switch codegen on/off

[ https://issues.apache.org/jira/browse/SPARK-24840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24840: Assignee: Wenchen Fan (was: Apache Spark) > do not use dummy filter to switch codegen

[jira] [Assigned] (SPARK-24840) do not use dummy filter to switch codegen on/off

[ https://issues.apache.org/jira/browse/SPARK-24840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24840: Assignee: Apache Spark (was: Wenchen Fan) > do not use dummy filter to switch codegen

[jira] [Commented] (SPARK-24840) do not use dummy filter to switch codegen on/off

[ https://issues.apache.org/jira/browse/SPARK-24840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547387#comment-16547387 ] Apache Spark commented on SPARK-24840: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Created] (SPARK-24840) do not use dummy filter to switch codegen on/off

Wenchen Fan created SPARK-24840: --- Summary: do not use dummy filter to switch codegen on/off Key: SPARK-24840 URL: https://issues.apache.org/jira/browse/SPARK-24840 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-24809) Serializing LongHashedRelation in executor may result in data error

[ https://issues.apache.org/jira/browse/SPARK-24809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547337#comment-16547337 ] zenglinxi edited comment on SPARK-24809 at 7/18/18 4:10 AM: [^Spark

[jira] [Comment Edited] (SPARK-24809) Serializing LongHashedRelation in executor may result in data error

[ https://issues.apache.org/jira/browse/SPARK-24809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547337#comment-16547337 ] zenglinxi edited comment on SPARK-24809 at 7/18/18 4:09 AM: [^Spark

[jira] [Commented] (SPARK-24768) Have a built-in AVRO data source implementation

[ https://issues.apache.org/jira/browse/SPARK-24768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547346#comment-16547346 ] Apache Spark commented on SPARK-24768: -- User 'ueshin' has created a pull request for this issue:

[jira] [Commented] (SPARK-24386) implement continuous processing coalesce(1)

[ https://issues.apache.org/jira/browse/SPARK-24386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547345#comment-16547345 ] Apache Spark commented on SPARK-24386: -- User 'ueshin' has created a pull request for this issue:

[jira] [Commented] (SPARK-24809) Serializing LongHashedRelation in executor may result in data error

[ https://issues.apache.org/jira/browse/SPARK-24809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547337#comment-16547337 ] zenglinxi commented on SPARK-24809: --- [^Spark LongHashedRelation serialization.svg] I think it's a

[jira] [Updated] (SPARK-24809) Serializing LongHashedRelation in executor may result in data error

[ https://issues.apache.org/jira/browse/SPARK-24809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zenglinxi updated SPARK-24809: -- Attachment: Spark LongHashedRelation serialization.svg > Serializing LongHashedRelation in executor

[jira] [Updated] (SPARK-24829) In Spark Thrift Server, CAST AS FLOAT inconsistent with spark-shell or spark-sql

[ https://issues.apache.org/jira/browse/SPARK-24829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zuotingbing updated SPARK-24829: Summary: In Spark Thrift Server, CAST AS FLOAT inconsistent with spark-shell or spark-sql (was:

[jira] [Updated] (SPARK-24829) In Spark Thrift Server, CAST AS FLOAT inconsistent with spark-shell or spark-sql

[ https://issues.apache.org/jira/browse/SPARK-24829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zuotingbing updated SPARK-24829: Attachment: (was: CAST-FLOAT.png) > In Spark Thrift Server, CAST AS FLOAT inconsistent with

[jira] [Updated] (SPARK-24829) In Spark Thrift Server, CAST AS FLOAT inconsistent with spark-shell or spark-sql

[ https://issues.apache.org/jira/browse/SPARK-24829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zuotingbing updated SPARK-24829: Attachment: 2018-07-18_11.png > In Spark Thrift Server, CAST AS FLOAT inconsistent with

[jira] [Updated] (SPARK-24829) In Spark Thrift Server, CAST AS FLOAT inconsistent with spark-shell or spark-sql

[ https://issues.apache.org/jira/browse/SPARK-24829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zuotingbing updated SPARK-24829: Attachment: 2018-07-18_110944.png > In Spark Thrift Server, CAST AS FLOAT inconsistent with

[jira] [Commented] (SPARK-24828) Incompatible parquet formats - java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary$PlainLongDictionary

[ https://issues.apache.org/jira/browse/SPARK-24828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547299#comment-16547299 ] Hyukjin Kwon commented on SPARK-24828: -- Thanks [~q79969786]. Yea, currently you can't mix the type.

[jira] [Resolved] (SPARK-23998) It may be better to add @transient to field 'taskMemoryManager' in class Task, for it is only be set and used in executor side

[ https://issues.apache.org/jira/browse/SPARK-23998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] eaton resolved SPARK-23998. --- Resolution: Won't Do > It may be better to add @transient to field 'taskMemoryManager' in class > Task,

[jira] [Commented] (SPARK-24828) Incompatible parquet formats - java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary$PlainLongDictionary

[ https://issues.apache.org/jira/browse/SPARK-24828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547274#comment-16547274 ] Yuming Wang commented on SPARK-24828: - You can't read different data with a schema. > Incompatible

[jira] [Comment Edited] (SPARK-24828) Incompatible parquet formats - java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary$PlainLongDictionary

[ https://issues.apache.org/jira/browse/SPARK-24828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547264#comment-16547264 ] Yuming Wang edited comment on SPARK-24828 at 7/18/18 1:27 AM: -- The

[jira] [Commented] (SPARK-24828) Incompatible parquet formats - java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary$PlainLongDictionary

[ https://issues.apache.org/jira/browse/SPARK-24828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547264#comment-16547264 ] Yuming Wang commented on SPARK-24828: - I'm working on > Incompatible parquet formats -

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547263#comment-16547263 ] Saisai Shao commented on SPARK-24615: - Sure, I will also add it as Xiangrui also suggested the same

[jira] [Resolved] (SPARK-24402) Optimize `In` expression when only one element in the collection or collection is empty

[ https://issues.apache.org/jira/browse/SPARK-24402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-24402. - Resolution: Resolved > Optimize `In` expression when only one element in the collection or >

[jira] [Assigned] (SPARK-24825) [K8S][TEST] Kubernetes integration tests don't trace the maven project dependency structure

[ https://issues.apache.org/jira/browse/SPARK-24825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shane knapp reassigned SPARK-24825: --- Assignee: Matt Cheah > [K8S][TEST] Kubernetes integration tests don't trace the maven

[jira] [Updated] (SPARK-24839) Incorrect drop of lit() column results in cross join

[ https://issues.apache.org/jira/browse/SPARK-24839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] marios iliofotou updated SPARK-24839: - Description: The problem shows up when joining a column that has constant (lit) value.

[jira] [Issue Comment Deleted] (SPARK-24839) Incorrect drop of lit() column results in cross join

[ https://issues.apache.org/jira/browse/SPARK-24839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] marios iliofotou updated SPARK-24839: - Comment: was deleted (was: Might be the closest issues related to this. ) > Incorrect

[jira] [Commented] (SPARK-24839) Incorrect drop of lit() column results in cross join

[ https://issues.apache.org/jira/browse/SPARK-24839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547246#comment-16547246 ] marios iliofotou commented on SPARK-24839: -- Might be the closest issues related to this.  >

[jira] [Updated] (SPARK-24839) Incorrect drop of lit() column results in cross join

[ https://issues.apache.org/jira/browse/SPARK-24839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] marios iliofotou updated SPARK-24839: - Description: The problem shows up when joining a column that has constant value. As

[jira] [Updated] (SPARK-24839) Incorrect drop of lit() column results in cross join

[ https://issues.apache.org/jira/browse/SPARK-24839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] marios iliofotou updated SPARK-24839: - Description: The problem shows up when joining a column that has constant value. As

[jira] [Updated] (SPARK-24839) Incorrect drop of lit() column results in cross join

[ https://issues.apache.org/jira/browse/SPARK-24839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] marios iliofotou updated SPARK-24839: - Description: The problem shows up when joining a column that has constant value. As

[jira] [Assigned] (SPARK-24825) [K8S][TEST] Kubernetes integration tests don't trace the maven project dependency structure

[ https://issues.apache.org/jira/browse/SPARK-24825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24825: Assignee: Apache Spark > [K8S][TEST] Kubernetes integration tests don't trace the maven

[jira] [Commented] (SPARK-24825) [K8S][TEST] Kubernetes integration tests don't trace the maven project dependency structure

[ https://issues.apache.org/jira/browse/SPARK-24825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547238#comment-16547238 ] Apache Spark commented on SPARK-24825: -- User 'mccheah' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24825) [K8S][TEST] Kubernetes integration tests don't trace the maven project dependency structure

[ https://issues.apache.org/jira/browse/SPARK-24825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24825: Assignee: (was: Apache Spark) > [K8S][TEST] Kubernetes integration tests don't trace

[jira] [Updated] (SPARK-24839) Incorrect drop of lit() column results in cross join

[ https://issues.apache.org/jira/browse/SPARK-24839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] marios iliofotou updated SPARK-24839: - Description: The problem shows up when joining a column that has constant value. As

[jira] [Updated] (SPARK-24839) Incorrect drop of lit() column results in cross join

[ https://issues.apache.org/jira/browse/SPARK-24839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] marios iliofotou updated SPARK-24839: - Description: The problem shows up when joining a column that has constant value. As

[jira] [Updated] (SPARK-24839) Incorrect drop of lit() column results in cross join

[ https://issues.apache.org/jira/browse/SPARK-24839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] marios iliofotou updated SPARK-24839: - Description: The problem shows up when joining a column that has constant value. As

[jira] [Updated] (SPARK-24839) Incorrect drop of lit() column results in cross join

[ https://issues.apache.org/jira/browse/SPARK-24839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] marios iliofotou updated SPARK-24839: - Summary: Incorrect drop of lit() column results in cross join (was: Incorrect drop of

[jira] [Updated] (SPARK-24839) Incorrect drop of lit column results in cross join.

[ https://issues.apache.org/jira/browse/SPARK-24839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] marios iliofotou updated SPARK-24839: - Description: {code} scala> val df1 = spark.createDataFrame(Seq((1, 2), (2,

[jira] [Updated] (SPARK-24839) Incorrect drop of lit column results in cross join.

[ https://issues.apache.org/jira/browse/SPARK-24839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] marios iliofotou updated SPARK-24839: - Description: {code:scala} scala> val df1 = spark.createDataFrame(Seq((1, 2), (2,

[jira] [Created] (SPARK-24839) Incorrect drop of lit column results in cross join.

marios iliofotou created SPARK-24839: Summary: Incorrect drop of lit column results in cross join. Key: SPARK-24839 URL: https://issues.apache.org/jira/browse/SPARK-24839 Project: Spark

[jira] [Comment Edited] (SPARK-24825) [K8S][TEST] Kubernetes integration tests don't trace the maven project dependency structure

[ https://issues.apache.org/jira/browse/SPARK-24825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547225#comment-16547225 ] Stavros Kontopoulos edited comment on SPARK-24825 at 7/17/18 11:43 PM:

[jira] [Comment Edited] (SPARK-24825) [K8S][TEST] Kubernetes integration tests don't trace the maven project dependency structure

[ https://issues.apache.org/jira/browse/SPARK-24825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547225#comment-16547225 ] Stavros Kontopoulos edited comment on SPARK-24825 at 7/17/18 11:43 PM:

[jira] [Comment Edited] (SPARK-24825) [K8S][TEST] Kubernetes integration tests don't trace the maven project dependency structure

[ https://issues.apache.org/jira/browse/SPARK-24825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547225#comment-16547225 ] Stavros Kontopoulos edited comment on SPARK-24825 at 7/17/18 11:38 PM:

[jira] [Commented] (SPARK-24825) [K8S][TEST] Kubernetes integration tests don't trace the maven project dependency structure

[ https://issues.apache.org/jira/browse/SPARK-24825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547225#comment-16547225 ] Stavros Kontopoulos commented on SPARK-24825: - Yes this seems to work:  ./build/mvn -T 1C

[jira] [Commented] (SPARK-24747) Make spark.ml.util.Instrumentation class more flexible

[ https://issues.apache.org/jira/browse/SPARK-24747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547200#comment-16547200 ] Apache Spark commented on SPARK-24747: -- User 'MrBago' has created a pull request for this issue:

[jira] [Commented] (SPARK-24825) [K8S][TEST] Kubernetes integration tests don't trace the maven project dependency structure

[ https://issues.apache.org/jira/browse/SPARK-24825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547189#comment-16547189 ] shane knapp commented on SPARK-24825: - [~mcheah] is working on a patch now...  > [K8S][TEST]

[jira] [Issue Comment Deleted] (SPARK-24825) [K8S][TEST] Kubernetes integration tests don't trace the maven project dependency structure

[ https://issues.apache.org/jira/browse/SPARK-24825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-24825: Comment: was deleted (was: [~srowen] I played with submodules but it didnt work

[jira] [Commented] (SPARK-24825) [K8S][TEST] Kubernetes integration tests don't trace the maven project dependency structure

[ https://issues.apache.org/jira/browse/SPARK-24825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547186#comment-16547186 ] Stavros Kontopoulos commented on SPARK-24825: - [~srowen] I played with submodules but it

[jira] [Commented] (SPARK-24825) [K8S][TEST] Kubernetes integration tests don't trace the maven project dependency structure

[ https://issues.apache.org/jira/browse/SPARK-24825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547142#comment-16547142 ] Sean Owen commented on SPARK-24825: --- To build and test only a child module, you can't just run tests

[jira] [Commented] (SPARK-24825) [K8S][TEST] Kubernetes integration tests don't trace the maven project dependency structure

[ https://issues.apache.org/jira/browse/SPARK-24825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547141#comment-16547141 ] shane knapp commented on SPARK-24825: - [~vanzin] [~joshrosen] any thoughts? > [K8S][TEST]

[jira] [Commented] (SPARK-24825) [K8S][TEST] Kubernetes integration tests don't trace the maven project dependency structure

[ https://issues.apache.org/jira/browse/SPARK-24825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547137#comment-16547137 ] Matt Cheah commented on SPARK-24825: We're looking into this now, this particular phase was built

[jira] [Comment Edited] (SPARK-24825) [K8S][TEST] Kubernetes integration tests don't trace the maven project dependency structure

[ https://issues.apache.org/jira/browse/SPARK-24825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547133#comment-16547133 ] Stavros Kontopoulos edited comment on SPARK-24825 at 7/17/18 9:37 PM:

[jira] [Comment Edited] (SPARK-24825) [K8S][TEST] Kubernetes integration tests don't trace the maven project dependency structure

[ https://issues.apache.org/jira/browse/SPARK-24825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547133#comment-16547133 ] Stavros Kontopoulos edited comment on SPARK-24825 at 7/17/18 9:37 PM:

[jira] [Comment Edited] (SPARK-24825) [K8S][TEST] Kubernetes integration tests don't trace the maven project dependency structure

[ https://issues.apache.org/jira/browse/SPARK-24825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547133#comment-16547133 ] Stavros Kontopoulos edited comment on SPARK-24825 at 7/17/18 9:36 PM:

[jira] [Commented] (SPARK-24825) [K8S][TEST] Kubernetes integration tests don't trace the maven project dependency structure

[ https://issues.apache.org/jira/browse/SPARK-24825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547133#comment-16547133 ] Stavros Kontopoulos commented on SPARK-24825: - You could remove .m2, downloads should be

[jira] [Comment Edited] (SPARK-24825) [K8S][TEST] Kubernetes integration tests don't trace the maven project dependency structure

[ https://issues.apache.org/jira/browse/SPARK-24825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547128#comment-16547128 ] Stavros Kontopoulos edited comment on SPARK-24825 at 7/17/18 9:34 PM:

[jira] [Commented] (SPARK-24825) [K8S][TEST] Kubernetes integration tests don't trace the maven project dependency structure

[ https://issues.apache.org/jira/browse/SPARK-24825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547132#comment-16547132 ] shane knapp commented on SPARK-24825: - yeah, `clean install` isn't probably a good long term

[jira] [Comment Edited] (SPARK-24825) [K8S][TEST] Kubernetes integration tests don't trace the maven project dependency structure

[ https://issues.apache.org/jira/browse/SPARK-24825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547128#comment-16547128 ] Stavros Kontopoulos edited comment on SPARK-24825 at 7/17/18 9:31 PM:

[jira] [Commented] (SPARK-24825) [K8S][TEST] Kubernetes integration tests don't trace the maven project dependency structure

[ https://issues.apache.org/jira/browse/SPARK-24825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547128#comment-16547128 ] Stavros Kontopoulos commented on SPARK-24825: - As I mentioned elsewhere one thing that

[jira] [Commented] (SPARK-24835) col function ignores drop

[ https://issues.apache.org/jira/browse/SPARK-24835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547123#comment-16547123 ] Liang-Chi Hsieh commented on SPARK-24835: - `drop` actually does to add a projection on top of

[jira] [Comment Edited] (SPARK-24835) col function ignores drop

[ https://issues.apache.org/jira/browse/SPARK-24835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547123#comment-16547123 ] Liang-Chi Hsieh edited comment on SPARK-24835 at 7/17/18 9:25 PM: --

[jira] [Resolved] (SPARK-24681) Cannot create a view from a table when a nested column name contains ':'

[ https://issues.apache.org/jira/browse/SPARK-24681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24681. - Resolution: Fixed Assignee: Takeshi Yamamuro Fix Version/s: 2.4.0 > Cannot create a

[jira] [Commented] (SPARK-24838) Support uncorrelated IN/EXISTS subqueries for more operators

[ https://issues.apache.org/jira/browse/SPARK-24838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547105#comment-16547105 ] Xiao Li commented on SPARK-24838: - cc [~dkbiswal] > Support uncorrelated IN/EXISTS subqueries for more

[jira] [Created] (SPARK-24838) Support uncorrelated IN/EXISTS subqueries for more operators

Qifan Pu created SPARK-24838: Summary: Support uncorrelated IN/EXISTS subqueries for more operators Key: SPARK-24838 URL: https://issues.apache.org/jira/browse/SPARK-24838 Project: Spark Issue

[jira] [Comment Edited] (SPARK-24825) [K8S][TEST] Kubernetes integration tests don't trace the maven project dependency structure

[ https://issues.apache.org/jira/browse/SPARK-24825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547061#comment-16547061 ] Stavros Kontopoulos edited comment on SPARK-24825 at 7/17/18 8:54 PM:

[jira] [Comment Edited] (SPARK-24825) [K8S][TEST] Kubernetes integration tests don't trace the maven project dependency structure

[ https://issues.apache.org/jira/browse/SPARK-24825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547061#comment-16547061 ] Stavros Kontopoulos edited comment on SPARK-24825 at 7/17/18 8:54 PM:

[jira] [Updated] (SPARK-23723) New encoding option for json datasource

[ https://issues.apache.org/jira/browse/SPARK-23723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-23723: Summary: New encoding option for json datasource (was: New charset option for json datasource) > New

[jira] [Commented] (SPARK-24825) [K8S][TEST] Kubernetes integration tests don't trace the maven project dependency structure

[ https://issues.apache.org/jira/browse/SPARK-24825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547084#comment-16547084 ] shane knapp commented on SPARK-24825: - we're trying to get things to work w/o uploading the jar, but

[jira] [Comment Edited] (SPARK-24825) [K8S][TEST] Kubernetes integration tests don't trace the maven project dependency structure

[ https://issues.apache.org/jira/browse/SPARK-24825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547061#comment-16547061 ] Stavros Kontopoulos edited comment on SPARK-24825 at 7/17/18 8:44 PM:

[jira] [Commented] (SPARK-24536) Query with nonsensical LIMIT hits AssertionError

[ https://issues.apache.org/jira/browse/SPARK-24536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547074#comment-16547074 ] Nihar Sheth commented on SPARK-24536: - I'd like to take a shot at this, if no one else is > Query

[jira] [Updated] (SPARK-24837) Add kafka as spark metrics sink

[ https://issues.apache.org/jira/browse/SPARK-24837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandish Kumar HN updated SPARK-24837: - Description: Sink spark metrics to kafka producer

[jira] [Updated] (SPARK-24837) Add kafka as spark metrics sink

[ https://issues.apache.org/jira/browse/SPARK-24837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandish Kumar HN updated SPARK-24837: - Summary: Add kafka as spark metrics sink (was: Add kafka as spark logs/metrics sink)

[jira] [Comment Edited] (SPARK-24825) [K8S][TEST] Kubernetes integration tests don't trace the maven project dependency structure

[ https://issues.apache.org/jira/browse/SPARK-24825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547061#comment-16547061 ] Stavros Kontopoulos edited comment on SPARK-24825 at 7/17/18 8:20 PM:

[jira] [Commented] (SPARK-24825) [K8S][TEST] Kubernetes integration tests don't trace the maven project dependency structure

[ https://issues.apache.org/jira/browse/SPARK-24825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547061#comment-16547061 ] Stavros Kontopoulos commented on SPARK-24825: - To reproduce it try build the test suite:

[jira] [Updated] (SPARK-24837) Add kafka as spark logs/metrics sink

[ https://issues.apache.org/jira/browse/SPARK-24837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandish Kumar HN updated SPARK-24837: - Description: Sink spark logs/metrics to kafka producer

[jira] [Created] (SPARK-24837) Add kafka as spark logs/metrics sink

Sandish Kumar HN created SPARK-24837: Summary: Add kafka as spark logs/metrics sink Key: SPARK-24837 URL: https://issues.apache.org/jira/browse/SPARK-24837 Project: Spark Issue Type: New

[jira] [Resolved] (SPARK-24747) Make spark.ml.util.Instrumentation class more flexible

[ https://issues.apache.org/jira/browse/SPARK-24747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-24747. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21719

[jira] [Commented] (SPARK-24644) Pyarrow exception while running pandas_udf on pyspark 2.3.1

[ https://issues.apache.org/jira/browse/SPARK-24644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547031#comment-16547031 ] Hichame El Khalfi commented on SPARK-24644: --- Indeed we were using an old version on pandas,

[jira] [Assigned] (SPARK-24836) New option - ignoreExtension

[ https://issues.apache.org/jira/browse/SPARK-24836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24836: Assignee: Apache Spark > New option - ignoreExtension > > >

[jira] [Assigned] (SPARK-24836) New option - ignoreExtension

[ https://issues.apache.org/jira/browse/SPARK-24836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24836: Assignee: (was: Apache Spark) > New option - ignoreExtension >

[jira] [Commented] (SPARK-24836) New option - ignoreExtension

[ https://issues.apache.org/jira/browse/SPARK-24836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547010#comment-16547010 ] Apache Spark commented on SPARK-24836: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Created] (SPARK-24836) New option - ignoreExtension

Maxim Gekk created SPARK-24836: -- Summary: New option - ignoreExtension Key: SPARK-24836 URL: https://issues.apache.org/jira/browse/SPARK-24836 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-24835) col function ignores drop

Michael Souder created SPARK-24835: -- Summary: col function ignores drop Key: SPARK-24835 URL: https://issues.apache.org/jira/browse/SPARK-24835 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-24801) Empty byte[] arrays in spark.network.sasl.SaslEncryption$EncryptedMessage can waste a lot of memory

[ https://issues.apache.org/jira/browse/SPARK-24801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16546981#comment-16546981 ] Misha Dmitriev commented on SPARK-24801: [~irashid] yes, I'll submit a change to lazily

[jira] [Resolved] (SPARK-24071) Micro-benchmark of Parquet Filter Pushdown

[ https://issues.apache.org/jira/browse/SPARK-24071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24071. - Resolution: Duplicate Fix Version/s: 2.4.0 > Micro-benchmark of Parquet Filter Pushdown >

[jira] [Resolved] (SPARK-24070) TPC-DS Performance Tests for Parquet 1.10.0 Upgrade

[ https://issues.apache.org/jira/browse/SPARK-24070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24070. - Resolution: Fixed Fix Version/s: 2.4.0 > TPC-DS Performance Tests for Parquet 1.10.0 Upgrade >

[jira] [Commented] (SPARK-24402) Optimize `In` expression when only one element in the collection or collection is empty

[ https://issues.apache.org/jira/browse/SPARK-24402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16546920#comment-16546920 ] Apache Spark commented on SPARK-24402: -- User 'dbtsai' has created a pull request for this issue:

[jira] [Commented] (SPARK-24801) Empty byte[] arrays in spark.network.sasl.SaslEncryption$EncryptedMessage can waste a lot of memory

[ https://issues.apache.org/jira/browse/SPARK-24801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16546916#comment-16546916 ] Imran Rashid commented on SPARK-24801: -- I don't think the messages themselves are actually empty,

[jira] [Commented] (SPARK-19680) Offsets out of range with no configured reset policy for partitions

[ https://issues.apache.org/jira/browse/SPARK-19680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16546891#comment-16546891 ] Gayathiri Duraikannu commented on SPARK-19680: -- Ours is a framework and multiple consumers

[jira] [Resolved] (SPARK-21590) Structured Streaming window start time should support negative values to adjust time zone

[ https://issues.apache.org/jira/browse/SPARK-21590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21590. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 18903

[jira] [Assigned] (SPARK-21590) Structured Streaming window start time should support negative values to adjust time zone

[ https://issues.apache.org/jira/browse/SPARK-21590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-21590: - Assignee: Kevin Zhang > Structured Streaming window start time should support negative values

[jira] [Commented] (SPARK-9850) Adaptive execution in Spark

[ https://issues.apache.org/jira/browse/SPARK-9850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16546840#comment-16546840 ] Michail Giannakopoulos commented on SPARK-9850: --- Hello [~yhuai]! Are people currently

[jira] [Assigned] (SPARK-24833) Allow specifying Kubernetes host name aliases in the pod specs

[ https://issues.apache.org/jira/browse/SPARK-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24833: Assignee: (was: Apache Spark) > Allow specifying Kubernetes host name aliases in the

[jira] [Assigned] (SPARK-24833) Allow specifying Kubernetes host name aliases in the pod specs

[ https://issues.apache.org/jira/browse/SPARK-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24833: Assignee: Apache Spark > Allow specifying Kubernetes host name aliases in the pod specs

[jira] [Commented] (SPARK-24833) Allow specifying Kubernetes host name aliases in the pod specs

[ https://issues.apache.org/jira/browse/SPARK-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16546798#comment-16546798 ] Apache Spark commented on SPARK-24833: -- User 'rvesse' has created a pull request for this issue:

[jira] [Resolved] (SPARK-24305) Avoid serialization of private fields in new collection expressions

[ https://issues.apache.org/jira/browse/SPARK-24305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-24305. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21352

[jira] [Assigned] (SPARK-24305) Avoid serialization of private fields in new collection expressions

[ https://issues.apache.org/jira/browse/SPARK-24305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-24305: --- Assignee: Marek Novotny > Avoid serialization of private fields in new collection

[jira] [Commented] (SPARK-24165) UDF within when().otherwise() raises NullPointerException

[ https://issues.apache.org/jira/browse/SPARK-24165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16546750#comment-16546750 ] Apache Spark commented on SPARK-24165: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24834) Utils#nanSafeCompare{Double,Float} functions do not differ from normal java double/float comparison

[ https://issues.apache.org/jira/browse/SPARK-24834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24834: Assignee: Apache Spark > Utils#nanSafeCompare{Double,Float} functions do not differ from

  1   2   >