[jira] [Commented] (SPARK-24467) VectorAssemblerEstimator

2018-07-03 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16532324#comment-16532324 ] Liang-Chi Hsieh commented on SPARK-24467: - It sounds good to me for the approach

[jira] [Commented] (SPARK-24422) Add JDK9+ in our Jenkins' build servers

2018-07-03 Thread vaquar khan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16532268#comment-16532268 ] vaquar khan commented on SPARK-24422: - I can't see any update on Jenkins and Java 9

[jira] [Commented] (SPARK-24421) sun.misc.Unsafe in JDK9+

2018-07-03 Thread vaquar khan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16532262#comment-16532262 ] vaquar khan commented on SPARK-24421: - 1) I would suggest to use "Java Dependency An

[jira] [Assigned] (SPARK-24681) Cannot create a view from a table when a nested column name contains ':'

2018-07-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24681: Assignee: (was: Apache Spark) > Cannot create a view from a table when a nested colum

[jira] [Assigned] (SPARK-24681) Cannot create a view from a table when a nested column name contains ':'

2018-07-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24681: Assignee: Apache Spark > Cannot create a view from a table when a nested column name cont

[jira] [Commented] (SPARK-24681) Cannot create a view from a table when a nested column name contains ':'

2018-07-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16532234#comment-16532234 ] Apache Spark commented on SPARK-24681: -- User 'maropu' has created a pull request fo

[jira] [Resolved] (SPARK-24732) Type coercion between MapTypes.

2018-07-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24732. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21703 [https://gi

[jira] [Assigned] (SPARK-24732) Type coercion between MapTypes.

2018-07-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-24732: Assignee: Takuya Ueshin > Type coercion between MapTypes. > -

[jira] [Assigned] (SPARK-24207) PrefixSpan: R API

2018-07-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24207: Assignee: (was: Apache Spark) > PrefixSpan: R API > - > >

[jira] [Assigned] (SPARK-24207) PrefixSpan: R API

2018-07-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24207: Assignee: Apache Spark > PrefixSpan: R API > - > > Key: S

[jira] [Commented] (SPARK-24207) PrefixSpan: R API

2018-07-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16532196#comment-16532196 ] Apache Spark commented on SPARK-24207: -- User 'huaxingao' has created a pull request

[jira] [Commented] (SPARK-24644) Pyarrow exception while running pandas_udf on pyspark 2.3.1

2018-07-03 Thread Hichame El Khalfi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16532178#comment-16532178 ] Hichame El Khalfi commented on SPARK-24644: --- Hello [~hyukjin.kwon] Thanks for

[jira] [Commented] (SPARK-24530) Sphinx doesn't render autodoc_docstring_signature correctly (with Python 2?) and pyspark.ml docs are broken

2018-07-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16532160#comment-16532160 ] Hyukjin Kwon commented on SPARK-24530: -- [~mengxr], I lowered the priority to {{Crit

[jira] [Updated] (SPARK-24530) Sphinx doesn't render autodoc_docstring_signature correctly (with Python 2?) and pyspark.ml docs are broken

2018-07-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-24530: - Priority: Critical (was: Blocker) > Sphinx doesn't render autodoc_docstring_signature correctly

[jira] [Created] (SPARK-24736) --py-files appears to pass non-local URL's into PYTHONPATH directly.

2018-07-03 Thread Jonathan A Weaver (JIRA)
Jonathan A Weaver created SPARK-24736: - Summary: --py-files appears to pass non-local URL's into PYTHONPATH directly. Key: SPARK-24736 URL: https://issues.apache.org/jira/browse/SPARK-24736 Projec

[jira] [Commented] (SPARK-24530) Sphinx doesn't render autodoc_docstring_signature correctly (with Python 2?) and pyspark.ml docs are broken

2018-07-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16532159#comment-16532159 ] Hyukjin Kwon commented on SPARK-24530: -- There's workaround for this. To cut it shor

[jira] [Commented] (SPARK-24535) Fix java version parsing in SparkR on Windows

2018-07-03 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16532145#comment-16532145 ] Saisai Shao commented on SPARK-24535: - Hi [~felixcheung] what's the current status o

[jira] [Commented] (SPARK-24530) Sphinx doesn't render autodoc_docstring_signature correctly (with Python 2?) and pyspark.ml docs are broken

2018-07-03 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16532143#comment-16532143 ] Saisai Shao commented on SPARK-24530: - Hi [~hyukjin.kwon] what is the current status

[jira] [Assigned] (SPARK-23698) Spark code contains numerous undefined names in Python 3

2018-07-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-23698: Assignee: cclauss > Spark code contains numerous undefined names in Python 3 > --

[jira] [Resolved] (SPARK-23698) Spark code contains numerous undefined names in Python 3

2018-07-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23698. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21702 [https://gi

[jira] [Resolved] (SPARK-24709) Inferring schema from JSON string literal

2018-07-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24709. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21686 [https://gi

[jira] [Assigned] (SPARK-24709) Inferring schema from JSON string literal

2018-07-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-24709: Assignee: Maxim Gekk > Inferring schema from JSON string literal > --

[jira] [Assigned] (SPARK-5152) Let metrics.properties file take an hdfs:// path

2018-07-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5152: --- Assignee: Apache Spark > Let metrics.properties file take an hdfs:// path >

[jira] [Assigned] (SPARK-5152) Let metrics.properties file take an hdfs:// path

2018-07-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5152: --- Assignee: (was: Apache Spark) > Let metrics.properties file take an hdfs:// path > -

[jira] [Commented] (SPARK-5152) Let metrics.properties file take an hdfs:// path

2018-07-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16532102#comment-16532102 ] Apache Spark commented on SPARK-5152: - User 'jzhuge' has created a pull request for t

[jira] [Resolved] (SPARK-9883) Distance to each cluster given a point (KMeansModel)

2018-07-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-9883. -- Resolution: Won't Fix > Distance to each cluster given a point (KMeansModel) >

[jira] [Resolved] (SPARK-7856) Scalable PCA implementation for tall and fat matrices

2018-07-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-7856. -- Resolution: Won't Fix See PR > Scalable PCA implementation for tall and fat matrices > ---

[jira] [Created] (SPARK-24735) Improve exception when mixing pandas_udf types

2018-07-03 Thread Bryan Cutler (JIRA)
Bryan Cutler created SPARK-24735: Summary: Improve exception when mixing pandas_udf types Key: SPARK-24735 URL: https://issues.apache.org/jira/browse/SPARK-24735 Project: Spark Issue Type: Im

[jira] [Commented] (SPARK-24579) SPIP: Standardize Optimized Data Exchange between Spark and DL/AI frameworks

2018-07-03 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531737#comment-16531737 ] Xiangrui Meng commented on SPARK-24579: --- [~sethah] [~kiszk] [~rxin] Please request

[jira] [Commented] (SPARK-20202) Remove references to org.spark-project.hive

2018-07-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531706#comment-16531706 ] Hyukjin Kwon commented on SPARK-20202: -- kindly ping [~owen.omalley] and [~rxin]. I

[jira] [Commented] (SPARK-24421) sun.misc.Unsafe in JDK9+

2018-07-03 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531705#comment-16531705 ] DB Tsai commented on SPARK-24421: - In JDK9+, `sun.misc.Unsafe` is private. We can either

[jira] [Resolved] (SPARK-24420) Upgrade ASM to 6.x to support JDK9+

2018-07-03 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-24420. - Resolution: Fixed Fix Version/s: 2.4.0 > Upgrade ASM to 6.x to support JDK9+ > --

[jira] [Issue Comment Deleted] (SPARK-24702) Unable to cast to calendar interval in spark sql.

2018-07-03 Thread Daniel Mateus Pires (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Mateus Pires updated SPARK-24702: Comment: was deleted (was: https://github.com/apache/spark/pull/21706) > Unable t

[jira] [Commented] (SPARK-24207) PrefixSpan: R API

2018-07-03 Thread Huaxin Gao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531498#comment-16531498 ] Huaxin Gao commented on SPARK-24207: I am working on this and will submit a PR soon.

[jira] [Commented] (SPARK-24702) Unable to cast to calendar interval in spark sql.

2018-07-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531481#comment-16531481 ] Apache Spark commented on SPARK-24702: -- User 'dmateusp' has created a pull request

[jira] [Assigned] (SPARK-24702) Unable to cast to calendar interval in spark sql.

2018-07-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24702: Assignee: Apache Spark > Unable to cast to calendar interval in spark sql. >

[jira] [Assigned] (SPARK-24702) Unable to cast to calendar interval in spark sql.

2018-07-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24702: Assignee: (was: Apache Spark) > Unable to cast to calendar interval in spark sql. > -

[jira] [Commented] (SPARK-24702) Unable to cast to calendar interval in spark sql.

2018-07-03 Thread Daniel Mateus Pires (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531480#comment-16531480 ] Daniel Mateus Pires commented on SPARK-24702: - https://github.com/apache/spa

[jira] [Commented] (SPARK-24702) Unable to cast to calendar interval in spark sql.

2018-07-03 Thread Daniel Mateus Pires (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531474#comment-16531474 ] Daniel Mateus Pires commented on SPARK-24702: - Got it working, I'll open a P

[jira] [Comment Edited] (SPARK-24702) Unable to cast to calendar interval in spark sql.

2018-07-03 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531388#comment-16531388 ] Takeshi Yamamuro edited comment on SPARK-24702 at 7/3/18 1:36 PM:

[jira] [Commented] (SPARK-24702) Unable to cast to calendar interval in spark sql.

2018-07-03 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531388#comment-16531388 ] Takeshi Yamamuro commented on SPARK-24702: -- In postgresql, we can do so; {code}

[jira] [Commented] (SPARK-24702) Unable to cast to calendar interval in spark sql.

2018-07-03 Thread Daniel Mateus Pires (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531379#comment-16531379 ] Daniel Mateus Pires commented on SPARK-24702: - The error is: DataType calen

[jira] [Assigned] (SPARK-24727) The cache 100 in CodeGenerator is too small for streaming

2018-07-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24727: Assignee: (was: Apache Spark) > The cache 100 in CodeGenerator is too small for strea

[jira] [Commented] (SPARK-24727) The cache 100 in CodeGenerator is too small for streaming

2018-07-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531376#comment-16531376 ] Apache Spark commented on SPARK-24727: -- User 'maropu' has created a pull request fo

[jira] [Assigned] (SPARK-24727) The cache 100 in CodeGenerator is too small for streaming

2018-07-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24727: Assignee: Apache Spark > The cache 100 in CodeGenerator is too small for streaming >

[jira] [Commented] (SPARK-24703) Unable to multiply calender interval with long/int

2018-07-03 Thread Daniel Mateus Pires (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531363#comment-16531363 ] Daniel Mateus Pires commented on SPARK-24703: - was able to reproduce in 2.4.

[jira] [Commented] (SPARK-24422) Add JDK9+ in our Jenkins' build servers

2018-07-03 Thread Andrew Korzhuev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531241#comment-16531241 ] Andrew Korzhuev commented on SPARK-24422: - Also `.travis.yml` needs to be fixed

[jira] [Comment Edited] (SPARK-24422) Add JDK9+ in our Jenkins' build servers

2018-07-03 Thread Andrew Korzhuev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531241#comment-16531241 ] Andrew Korzhuev edited comment on SPARK-24422 at 7/3/18 11:46 AM:

[jira] [Comment Edited] (SPARK-24421) sun.misc.Unsafe in JDK9+

2018-07-03 Thread Andrew Korzhuev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531235#comment-16531235 ] Andrew Korzhuev edited comment on SPARK-24421 at 7/3/18 11:44 AM:

[jira] [Commented] (SPARK-24421) sun.misc.Unsafe in JDK9+

2018-07-03 Thread Andrew Korzhuev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531235#comment-16531235 ] Andrew Korzhuev commented on SPARK-24421: - If I understand this correctly, then

[jira] [Commented] (SPARK-24734) Fix containsNull of Concat for array type.

2018-07-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531229#comment-16531229 ] Apache Spark commented on SPARK-24734: -- User 'ueshin' has created a pull request fo

[jira] [Assigned] (SPARK-24734) Fix containsNull of Concat for array type.

2018-07-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24734: Assignee: (was: Apache Spark) > Fix containsNull of Concat for array type. >

[jira] [Assigned] (SPARK-24734) Fix containsNull of Concat for array type.

2018-07-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24734: Assignee: Apache Spark > Fix containsNull of Concat for array type. > ---

[jira] [Comment Edited] (SPARK-24727) The cache 100 in CodeGenerator is too small for streaming

2018-07-03 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531225#comment-16531225 ] Wenchen Fan edited comment on SPARK-24727 at 7/3/18 11:29 AM:

[jira] [Commented] (SPARK-24727) The cache 100 in CodeGenerator is too small for streaming

2018-07-03 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531225#comment-16531225 ] Wenchen Fan commented on SPARK-24727: - BTW this needs to be a static SQL. the CodeG

[jira] [Commented] (SPARK-24727) The cache 100 in CodeGenerator is too small for streaming

2018-07-03 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531224#comment-16531224 ] Wenchen Fan commented on SPARK-24727: - it's because it was hard to access SQLConf at

[jira] [Created] (SPARK-24734) Fix containsNull of Concat for array type.

2018-07-03 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-24734: - Summary: Fix containsNull of Concat for array type. Key: SPARK-24734 URL: https://issues.apache.org/jira/browse/SPARK-24734 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-24733) Dataframe saved to parquet can have different metadata then the resulting parquet file

2018-07-03 Thread David Herskovics (JIRA)
David Herskovics created SPARK-24733: Summary: Dataframe saved to parquet can have different metadata then the resulting parquet file Key: SPARK-24733 URL: https://issues.apache.org/jira/browse/SPARK-24733

[jira] [Commented] (SPARK-24727) The cache 100 in CodeGenerator is too small for streaming

2018-07-03 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531144#comment-16531144 ] Takeshi Yamamuro commented on SPARK-24727: -- I see. It makes some sense to me. A

[jira] [Commented] (SPARK-24727) The cache 100 in CodeGenerator is too small for streaming

2018-07-03 Thread ant_nebula (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531107#comment-16531107 ] ant_nebula commented on SPARK-24727: NO.Spark would do DAG schedule for each streami

[jira] [Commented] (SPARK-24729) Spark - stackoverflow error - org.apache.spark.sql.catalyst.plans.QueryPlan

2018-07-03 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531071#comment-16531071 ] Takeshi Yamamuro commented on SPARK-24729: -- Can you run on v2.3.1? > Spark - s

[jira] [Commented] (SPARK-24727) The cache 100 in CodeGenerator is too small for streaming

2018-07-03 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531058#comment-16531058 ] Takeshi Yamamuro commented on SPARK-24727: -- Your query in streaming changes eve

[jira] [Commented] (SPARK-24705) Spark.sql.adaptive.enabled=true is enabled and self-join query

2018-07-03 Thread cheng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531023#comment-16531023 ] cheng commented on SPARK-24705: --- It is currently closed. If I want to use this feature to

[jira] [Comment Edited] (SPARK-24681) Cannot create a view from a table when a nested column name contains ':'

2018-07-03 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531016#comment-16531016 ] Takeshi Yamamuro edited comment on SPARK-24681 at 7/3/18 8:37 AM:

[jira] [Commented] (SPARK-24681) Cannot create a view from a table when a nested column name contains ':'

2018-07-03 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531016#comment-16531016 ] Takeshi Yamamuro commented on SPARK-24681: -- I've looked over related code andI

[jira] [Commented] (SPARK-24705) Spark.sql.adaptive.enabled=true is enabled and self-join query

2018-07-03 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531007#comment-16531007 ] Takeshi Yamamuro commented on SPARK-24705: -- Why don't you turn off `spark.sql.a

[jira] [Commented] (SPARK-24732) Type coercion between MapTypes.

2018-07-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16530993#comment-16530993 ] Apache Spark commented on SPARK-24732: -- User 'ueshin' has created a pull request fo

[jira] [Assigned] (SPARK-24732) Type coercion between MapTypes.

2018-07-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24732: Assignee: Apache Spark > Type coercion between MapTypes. > --

[jira] [Assigned] (SPARK-24732) Type coercion between MapTypes.

2018-07-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24732: Assignee: (was: Apache Spark) > Type coercion between MapTypes. > ---

[jira] [Created] (SPARK-24732) Type coercion between MapTypes.

2018-07-03 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-24732: - Summary: Type coercion between MapTypes. Key: SPARK-24732 URL: https://issues.apache.org/jira/browse/SPARK-24732 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-24705) Spark.sql.adaptive.enabled=true is enabled and self-join query

2018-07-03 Thread cheng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16530978#comment-16530978 ] cheng commented on SPARK-24705: --- This is indeed the case, and it is often used in my busin

[jira] [Comment Edited] (SPARK-24535) Fix java version parsing in SparkR on Windows

2018-07-03 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16530963#comment-16530963 ] Felix Cheung edited comment on SPARK-24535 at 7/3/18 7:50 AM:

[jira] [Updated] (SPARK-24535) Fix java version parsing in SparkR on Windows

2018-07-03 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-24535: - Summary: Fix java version parsing in SparkR on Windows (was: Fix java version parsing in SparkR

[jira] [Commented] (SPARK-24535) Fix java version parsing in SparkR

2018-07-03 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16530963#comment-16530963 ] Felix Cheung commented on SPARK-24535: -- ok, I submitted a modified package to win-b

[jira] [Updated] (SPARK-24731) java.io.IOException: s3n://bucketname: 400 : Bad Request

2018-07-03 Thread sivakphani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivakphani updated SPARK-24731: --- Summary: java.io.IOException: s3n://bucketname: 400 : Bad Request (was: java.io.IOException: s3n:

[jira] [Created] (SPARK-24731) java.io.IOException: s3n://aail-twitter : 400 : Bad Request

2018-07-03 Thread sivakphani (JIRA)
sivakphani created SPARK-24731: -- Summary: java.io.IOException: s3n://aail-twitter : 400 : Bad Request Key: SPARK-24731 URL: https://issues.apache.org/jira/browse/SPARK-24731 Project: Spark Iss

[jira] [Commented] (SPARK-23698) Spark code contains numerous undefined names in Python 3

2018-07-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16530938#comment-16530938 ] Apache Spark commented on SPARK-23698: -- User 'cclauss' has created a pull request f