[jira] [Created] (SPARK-28510) Implement Spark's own GetFunctionsOperation

2019-07-24 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-28510: --- Summary: Implement Spark's own GetFunctionsOperation Key: SPARK-28510 URL: https://issues.apache.org/jira/browse/SPARK-28510 Project: Spark Issue Type: Sub-tas

[jira] [Commented] (SPARK-28377) Fully support correlation names in the FROM clause

2019-07-24 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16892414#comment-16892414 ] Takeshi Yamamuro commented on SPARK-28377: -- ur, I see. > Fully support correla

[jira] [Commented] (SPARK-28377) Fully support correlation names in the FROM clause

2019-07-24 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16892404#comment-16892404 ] Yuming Wang commented on SPARK-28377: - It’s SQL standard.  *Feature ID*: E051-08 >

[jira] [Commented] (SPARK-28497) Disallow upcasting complex data types to string type

2019-07-24 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16892369#comment-16892369 ] Dongjoon Hyun commented on SPARK-28497: --- Is this issue targeting `master` only, [~

[jira] [Closed] (SPARK-25590) kubernetes-model-2.0.0.jar masks default Spark logging config

2019-07-24 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-25590. - > kubernetes-model-2.0.0.jar masks default Spark logging config > --

[jira] [Commented] (SPARK-27442) ParquetFileFormat fails to read column named with invalid characters

2019-07-24 Thread angerszhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16892366#comment-16892366 ] angerszhu commented on SPARK-27442: --- auto writing data, change format to orc. orc can

[jira] [Commented] (SPARK-28152) Mapped ShortType to SMALLINT and FloatType to REAL for MsSqlServerDialect

2019-07-24 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16892362#comment-16892362 ] Dongjoon Hyun commented on SPARK-28152: --- This is merged to `branch-2.4` via https:

[jira] [Updated] (SPARK-28152) Mapped ShortType to SMALLINT and FloatType to REAL for MsSqlServerDialect

2019-07-24 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28152: -- Fix Version/s: 2.4.4 > Mapped ShortType to SMALLINT and FloatType to REAL for MsSqlServerDiale

[jira] [Commented] (SPARK-27442) ParquetFileFormat fails to read column named with invalid characters

2019-07-24 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16892358#comment-16892358 ] Hyukjin Kwon commented on SPARK-27442: -- simply rename it before writing out. > Par

[jira] [Commented] (SPARK-28377) Fully support correlation names in the FROM clause

2019-07-24 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16892351#comment-16892351 ] Takeshi Yamamuro commented on SPARK-28377: -- In the other systems, this column r

[jira] [Commented] (SPARK-27442) ParquetFileFormat fails to read column named with invalid characters

2019-07-24 Thread angerszhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16892348#comment-16892348 ] angerszhu commented on SPARK-27442: --- Meet same problem, can't write parquet file with

[jira] [Commented] (SPARK-27931) Accept 'on' and 'off' as input for boolean data type

2019-07-24 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16892324#comment-16892324 ] Yuming Wang commented on SPARK-27931: - This ticket has a low priority. > Accept 'on

[jira] [Commented] (SPARK-28377) Fully support correlation names in the FROM clause

2019-07-24 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16892323#comment-16892323 ] Yuming Wang commented on SPARK-28377: - [~younggyuchun] [~maropu] I think this ticket

[jira] [Updated] (SPARK-28496) Use branch name instead of tag during dry-run

2019-07-24 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28496: -- Fix Version/s: 2.4.4 > Use branch name instead of tag during dry-run > ---

[jira] [Resolved] (SPARK-28465) K8s integration tests fail due to missing ceph-nano image

2019-07-24 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-28465. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25222 [https:

[jira] [Assigned] (SPARK-28465) K8s integration tests fail due to missing ceph-nano image

2019-07-24 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-28465: -- Assignee: Stavros Kontopoulos > K8s integration tests fail due to missing ceph-nano i

[jira] [Commented] (SPARK-28509) K8S integration tests are failing

2019-07-24 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16892219#comment-16892219 ] shane knapp commented on SPARK-28509: - ok, all fixed and builds are passing on this

[jira] [Resolved] (SPARK-28509) K8S integration tests are failing

2019-07-24 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shane knapp resolved SPARK-28509. - Resolution: Fixed > K8S integration tests are failing > - > >

[jira] [Updated] (SPARK-28237) Idempotence checker for Idempotent batches in RuleExecutors

2019-07-24 Thread Yesheng Ma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesheng Ma updated SPARK-28237: --- Summary: Idempotence checker for Idempotent batches in RuleExecutors (was: Add a new batch strategy

[jira] [Resolved] (SPARK-28496) Use branch name instead of tag during dry-run

2019-07-24 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-28496. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25240 [https:

[jira] [Assigned] (SPARK-28496) Use branch name instead of tag during dry-run

2019-07-24 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-28496: -- Assignee: Dongjoon Hyun > Use branch name instead of tag during dry-run > ---

[jira] [Updated] (SPARK-28482) Data incomplete when using pandas udf in Python 3

2019-07-24 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28482: -- Summary: Data incomplete when using pandas udf in Python 3 (was: Data incomplete when using p

[jira] [Updated] (SPARK-28489) KafkaOffsetRangeCalculator.getRanges may drop offsets

2019-07-24 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-28489: - Affects Version/s: 2.4.0 2.4.1 2.4.2 > KafkaOffset

[jira] [Commented] (SPARK-28509) K8S integration tests are failing

2019-07-24 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16892165#comment-16892165 ] shane knapp commented on SPARK-28509: - the entire k8s config on worker-09 is complet

[jira] [Commented] (SPARK-28509) K8S integration tests are failing

2019-07-24 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16892142#comment-16892142 ] shane knapp commented on SPARK-28509: - ah, got it! it's research-jenkins-worker-09!

[jira] [Assigned] (SPARK-28509) K8S integration tests are failing

2019-07-24 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shane knapp reassigned SPARK-28509: --- Assignee: shane knapp > K8S integration tests are failing > ---

[jira] [Commented] (SPARK-28509) K8S integration tests are failing

2019-07-24 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16892137#comment-16892137 ] shane knapp commented on SPARK-28509: - as a precautionary step, on all ubuntu worker

[jira] [Updated] (SPARK-28489) KafkaOffsetRangeCalculator.getRanges may drop offsets

2019-07-24 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-28489: - Description: KafkaOffsetRangeCalculator.getRanges may drop offsets due to round off errors.  

[jira] [Commented] (SPARK-16548) java.io.CharConversionException: Invalid UTF-32 character prevents me from querying my data

2019-07-24 Thread ShivaKumar SS (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16892098#comment-16892098 ] ShivaKumar SS commented on SPARK-16548: --- Hello guys,  I am using Spark 2.3.0 and

[jira] [Commented] (SPARK-28509) K8S integration tests are failing

2019-07-24 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16892079#comment-16892079 ] Marcelo Vanzin commented on SPARK-28509: [~shaneknapp] in case this is an infra

[jira] [Created] (SPARK-28509) K8S integration tests are failing

2019-07-24 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-28509: -- Summary: K8S integration tests are failing Key: SPARK-28509 URL: https://issues.apache.org/jira/browse/SPARK-28509 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-28293) Implement Spark's own GetTableTypesOperation

2019-07-24 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-28293. - Resolution: Fixed Assignee: Yuming Wang Fix Version/s: 3.0.0 > Implement Spark's own Get

[jira] [Created] (SPARK-28508) Support for range frame+row frame in the same query

2019-07-24 Thread Dylan Guedes (JIRA)
Dylan Guedes created SPARK-28508: Summary: Support for range frame+row frame in the same query Key: SPARK-28508 URL: https://issues.apache.org/jira/browse/SPARK-28508 Project: Spark Issue Typ

[jira] [Commented] (SPARK-27812) kubernetes client import non-daemon thread which block jvm exit.

2019-07-24 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16892037#comment-16892037 ] Erik Erlandson commented on SPARK-27812: Agreed with [~skonto] that downgrading

[jira] [Created] (SPARK-28507) remove deprecated API context(self, sqlContext) from pyspark/ml/util.py

2019-07-24 Thread Huaxin Gao (JIRA)
Huaxin Gao created SPARK-28507: -- Summary: remove deprecated API context(self, sqlContext) from pyspark/ml/util.py Key: SPARK-28507 URL: https://issues.apache.org/jira/browse/SPARK-28507 Project: Spark

[jira] [Updated] (SPARK-28506) not handling usage of group function and window function at some conditions

2019-07-24 Thread Dylan Guedes (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dylan Guedes updated SPARK-28506: - Summary: not handling usage of group function and window function at some conditions (was: now

[jira] [Created] (SPARK-28506) now handling usage of group function and window function at some conditions

2019-07-24 Thread Dylan Guedes (JIRA)
Dylan Guedes created SPARK-28506: Summary: now handling usage of group function and window function at some conditions Key: SPARK-28506 URL: https://issues.apache.org/jira/browse/SPARK-28506 Project:

[jira] [Updated] (SPARK-20856) support statement using nested joins

2019-07-24 Thread Dylan Guedes (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dylan Guedes updated SPARK-20856: - Description: While DB2, ORACLE etc support a join expressed as follows, SPARK SQL does not. Not

[jira] [Created] (SPARK-28505) Add data source option for omitting partitioned columns when saving to file

2019-07-24 Thread Juarez Rudsatz (JIRA)
Juarez Rudsatz created SPARK-28505: -- Summary: Add data source option for omitting partitioned columns when saving to file Key: SPARK-28505 URL: https://issues.apache.org/jira/browse/SPARK-28505 Proje

[jira] [Created] (SPARK-28504) Web UI does not always honor spark.ui.proxyBase and X-Forwarded-Context

2019-07-24 Thread Kyle Winkelman (JIRA)
Kyle Winkelman created SPARK-28504: -- Summary: Web UI does not always honor spark.ui.proxyBase and X-Forwarded-Context Key: SPARK-28504 URL: https://issues.apache.org/jira/browse/SPARK-28504 Project:

[jira] [Updated] (SPARK-28502) Error with struct conversion while using pandas_udf

2019-07-24 Thread Nasir Ali (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nasir Ali updated SPARK-28502: -- Description: What I am trying to do: Group data based on time intervals (e.g., 15 days window) and pe

[jira] [Commented] (SPARK-28503) Return null result on cast an out-of-range value to a numeric type

2019-07-24 Thread Gengliang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16891982#comment-16891982 ] Gengliang Wang commented on SPARK-28503: I am working on this. > Return null re

[jira] [Created] (SPARK-28503) Return null result on cast an out-of-range value to a numeric type

2019-07-24 Thread Gengliang Wang (JIRA)
Gengliang Wang created SPARK-28503: -- Summary: Return null result on cast an out-of-range value to a numeric type Key: SPARK-28503 URL: https://issues.apache.org/jira/browse/SPARK-28503 Project: Spark

[jira] [Updated] (SPARK-28502) Error with struct conversion while using pandas_udf

2019-07-24 Thread Nasir Ali (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nasir Ali updated SPARK-28502: -- Description: What I am trying to do: Group data based on time intervals (e.g., 15 days window) and pe

[jira] [Resolved] (SPARK-25590) kubernetes-model-2.0.0.jar masks default Spark logging config

2019-07-24 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-25590. Resolution: Duplicate > kubernetes-model-2.0.0.jar masks default Spark logging config > --

[jira] [Created] (SPARK-28502) Error with struct conversion while using pandas_udf

2019-07-24 Thread Nasir Ali (JIRA)
Nasir Ali created SPARK-28502: - Summary: Error with struct conversion while using pandas_udf Key: SPARK-28502 URL: https://issues.apache.org/jira/browse/SPARK-28502 Project: Spark Issue Type: Que

[jira] [Updated] (SPARK-28483) Canceling a spark job using barrier mode but barrier tasks do not exit

2019-07-24 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-28483: --- Summary: Canceling a spark job using barrier mode but barrier tasks do not exit (was: Canceling a s

[jira] [Updated] (SPARK-28091) Extend Spark metrics system with user-defined metrics using executor plugins

2019-07-24 Thread Luca Canali (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luca Canali updated SPARK-28091: Summary: Extend Spark metrics system with user-defined metrics using executor plugins (was: Exten

[jira] [Updated] (SPARK-28091) Extend Spark metrics system with executor plugin metrics

2019-07-24 Thread Luca Canali (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luca Canali updated SPARK-28091: Description: This proposes to improve Spark instrumentation by adding a hook for user-defined met

[jira] [Commented] (SPARK-28500) adds support for `filter` clause

2019-07-24 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16891930#comment-16891930 ] Yuming Wang commented on SPARK-28500: - ok. thank you. > adds support for `filter` c

[jira] [Updated] (SPARK-25382) Remove ImageSchema.readImages in 3.0

2019-07-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25382: -- Docs Text: In Spark 3.0.0, the deprecated ImageSchema class and its readImages methods have been remov

[jira] [Commented] (SPARK-27931) Accept 'on' and 'off' as input for boolean data type

2019-07-24 Thread YoungGyu Chun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16891867#comment-16891867 ] YoungGyu Chun commented on SPARK-27931: --- [~yumwang] [~maropu] Does SPARK-27764 it

[jira] [Commented] (SPARK-28500) adds support for `filter` clause

2019-07-24 Thread Dylan Guedes (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16891833#comment-16891833 ] Dylan Guedes commented on SPARK-28500: -- Hmm you are correct, there's some overlappi

[jira] [Commented] (SPARK-28500) adds support for `filter` clause

2019-07-24 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16891832#comment-16891832 ] Yuming Wang commented on SPARK-28500: - Duplicate with SPARK-27986? > adds support f

[jira] [Issue Comment Deleted] (SPARK-27986) Support Aggregate Expressions

2019-07-24 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27986: Comment: was deleted (was: Duplicate with SPARK-27986?) > Support Aggregate Expressions > ---

[jira] [Commented] (SPARK-27986) Support Aggregate Expressions

2019-07-24 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16891831#comment-16891831 ] Yuming Wang commented on SPARK-27986: - Duplicate with SPARK-27986? > Support Aggreg

[jira] [Created] (SPARK-28501) frame bound must be a literal

2019-07-24 Thread Dylan Guedes (JIRA)
Dylan Guedes created SPARK-28501: Summary: frame bound must be a literal Key: SPARK-28501 URL: https://issues.apache.org/jira/browse/SPARK-28501 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-28065) ntile only accepting positive (>0) values

2019-07-24 Thread Dylan Guedes (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dylan Guedes updated SPARK-28065: - Description: Currently, Spark does not accept null as an input for `ntile`, or zero, however Po

[jira] [Updated] (SPARK-28065) ntile only accepting posivile (>0) values

2019-07-24 Thread Dylan Guedes (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dylan Guedes updated SPARK-28065: - Summary: ntile only accepting posivile (>0) values (was: ntile does not accept NULL as input)

[jira] [Updated] (SPARK-28065) ntile only accepting positive (>0) values

2019-07-24 Thread Dylan Guedes (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dylan Guedes updated SPARK-28065: - Summary: ntile only accepting positive (>0) values (was: ntile only accepting posivile (>0) val

[jira] [Created] (SPARK-28500) adds support for `filter` clause

2019-07-24 Thread Dylan Guedes (JIRA)
Dylan Guedes created SPARK-28500: Summary: adds support for `filter` clause Key: SPARK-28500 URL: https://issues.apache.org/jira/browse/SPARK-28500 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-18356) KMeans should cache RDD before training

2019-07-24 Thread Konstantin Sk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16891759#comment-16891759 ] Konstantin Sk commented on SPARK-18356: --- Hey, I have a spark version 2.4.3 and I s

[jira] [Closed] (SPARK-8165) sqlContext.createDataFrame(dataWithoutHeader, csvSchema) type conversion error after .cache

2019-07-24 Thread David Berger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Berger closed SPARK-8165. --- > sqlContext.createDataFrame(dataWithoutHeader, csvSchema) type conversion > error after .cache > --

[jira] [Updated] (SPARK-28499) Optimize MinMaxScaler

2019-07-24 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-28499: - Description: current impl of MinMaxScaler has some small places to be optimized: 1, avoid call

[jira] [Created] (SPARK-28499) Optimize MinMaxScaler

2019-07-24 Thread zhengruifeng (JIRA)
zhengruifeng created SPARK-28499: Summary: Optimize MinMaxScaler Key: SPARK-28499 URL: https://issues.apache.org/jira/browse/SPARK-28499 Project: Spark Issue Type: Improvement Compo

[jira] [Created] (SPARK-28498) always create a fresh copy of the SparkSession before each test

2019-07-24 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-28498: --- Summary: always create a fresh copy of the SparkSession before each test Key: SPARK-28498 URL: https://issues.apache.org/jira/browse/SPARK-28498 Project: Spark

[jira] [Updated] (SPARK-28482) Data incomplete when using pandas udf in pyspark

2019-07-24 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28482: -- Affects Version/s: 2.3.3 > Data incomplete when using pandas udf in pyspark >

[jira] [Commented] (SPARK-28482) Data incomplete when using pandas udf in pyspark

2019-07-24 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16891692#comment-16891692 ] Dongjoon Hyun commented on SPARK-28482: --- Thank you for the confirmation, [~jiangyu

[jira] [Created] (SPARK-28497) Disallow upcasting complex data types to string type

2019-07-24 Thread Gengliang Wang (JIRA)
Gengliang Wang created SPARK-28497: -- Summary: Disallow upcasting complex data types to string type Key: SPARK-28497 URL: https://issues.apache.org/jira/browse/SPARK-28497 Project: Spark Issu

[jira] [Commented] (SPARK-28494) Expose CalendarIntervalType and CalendarInterval in Spark

2019-07-24 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16891669#comment-16891669 ] Hyukjin Kwon commented on SPARK-28494: -- Nope but seems like the argument is that we

[jira] [Commented] (SPARK-18805) InternalMapWithStateDStream make java.lang.StackOverflowError

2019-07-24 Thread Cosmin Ciobanu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16891667#comment-16891667 ] Cosmin Ciobanu commented on SPARK-18805: This issue just occurred in one of our

[jira] [Issue Comment Deleted] (SPARK-28424) Support typed interval expression

2019-07-24 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28424: Comment: was deleted (was: I'm working on.) > Support typed interval expression > ---

[jira] [Updated] (SPARK-28424) Support typed interval expression

2019-07-24 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28424: Summary: Support typed interval expression (was: Improve interval input) > Support typed interva

[jira] [Created] (SPARK-28496) Use branch name instead of tag during dry-run

2019-07-24 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-28496: - Summary: Use branch name instead of tag during dry-run Key: SPARK-28496 URL: https://issues.apache.org/jira/browse/SPARK-28496 Project: Spark Issue Type: I

[jira] [Created] (SPARK-28495) AssignableCast: A new type coercion following store assignment rules of ANSI SQL

2019-07-24 Thread Gengliang Wang (JIRA)
Gengliang Wang created SPARK-28495: -- Summary: AssignableCast: A new type coercion following store assignment rules of ANSI SQL Key: SPARK-28495 URL: https://issues.apache.org/jira/browse/SPARK-28495