[jira] [Commented] (SPARK-29707) Make PartitionFilters and PushedFilters abbreviate configurable in metadata

2019-10-31 Thread Hu Fuwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964655#comment-16964655 ] Hu Fuwang commented on SPARK-29707: --- I am working on this. > Make PartitionFilters an

[jira] [Created] (SPARK-29710) Seeing offsets not resetting even when reset policy is configured explicitly

2019-10-31 Thread Shyam (Jira)
Shyam created SPARK-29710: - Summary: Seeing offsets not resetting even when reset policy is configured explicitly Key: SPARK-29710 URL: https://issues.apache.org/jira/browse/SPARK-29710 Project: Spark

[jira] [Created] (SPARK-29709) structured streaming The offset in the checkpoint is suddenly reset to the earliest

2019-10-31 Thread test (Jira)
test created SPARK-29709: Summary: structured streaming The offset in the checkpoint is suddenly reset to the earliest Key: SPARK-29709 URL: https://issues.apache.org/jira/browse/SPARK-29709 Project: Spark

[jira] [Comment Edited] (SPARK-29630) Not allowed to create a permanent view by referencing a temporary view in EXISTS

2019-10-31 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964626#comment-16964626 ] Terry Kim edited comment on SPARK-29630 at 11/1/19 5:50 AM:

[jira] [Commented] (SPARK-29630) Not allowed to create a permanent view by referencing a temporary view in EXISTS

2019-10-31 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964626#comment-16964626 ] Terry Kim commented on SPARK-29630: --- In the above example, the EXISTS clause becomes a

[jira] [Updated] (SPARK-29698) Support grouping function with multiple arguments

2019-10-31 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-29698: - Description: In PgSQL, grouping() can have multiple arguments, but Spark grouping() must

[jira] [Updated] (SPARK-29697) Support bit string types/literals

2019-10-31 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-29697: - Description: In PgSQL, there are bit types and literals; {code} postgres=# create table

[jira] [Updated] (SPARK-29700) Support nested grouping sets

2019-10-31 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-29700: - Description: PgSQL can process nested grouping sets, but Spark cannot; {code:java} postg

[jira] [Updated] (SPARK-29705) Support more expressive forms in GroupingSets/Cube/Rollup

2019-10-31 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-29705: - Description: See a doc for the form: https://www.postgresql.org/docs/current/sql-select.

[jira] [Updated] (SPARK-29704) Support the combinations of grouping operations

2019-10-31 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-29704: - Description: PgSQL can accept a query below with the combinations of grouping operations

[jira] [Created] (SPARK-29708) Different answers in aggregates of multiple grouping sets

2019-10-31 Thread Takeshi Yamamuro (Jira)
Takeshi Yamamuro created SPARK-29708: Summary: Different answers in aggregates of multiple grouping sets Key: SPARK-29708 URL: https://issues.apache.org/jira/browse/SPARK-29708 Project: Spark

[jira] [Updated] (SPARK-29701) Different answers when empty input given in GROUPING SETS

2019-10-31 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-29701: - Description: A query below with an empty input seems to have different answers between P

[jira] [Created] (SPARK-29707) Make PartitionFilters and PushedFilters abbreviate configurable in metadata

2019-10-31 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-29707: --- Summary: Make PartitionFilters and PushedFilters abbreviate configurable in metadata Key: SPARK-29707 URL: https://issues.apache.org/jira/browse/SPARK-29707 Project: Sp

[jira] [Updated] (SPARK-29707) Make PartitionFilters and PushedFilters abbreviate configurable in metadata

2019-10-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-29707: Attachment: screenshot-1.png > Make PartitionFilters and PushedFilters abbreviate configurable in

[jira] [Updated] (SPARK-29707) Make PartitionFilters and PushedFilters abbreviate configurable in metadata

2019-10-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-29707: Description: !screenshot-1.png! It lost some key information. Related code: https://github.com

[jira] [Created] (SPARK-29706) Support an empty grouping expression

2019-10-31 Thread Takeshi Yamamuro (Jira)
Takeshi Yamamuro created SPARK-29706: Summary: Support an empty grouping expression Key: SPARK-29706 URL: https://issues.apache.org/jira/browse/SPARK-29706 Project: Spark Issue Type: Sub-

[jira] [Updated] (SPARK-29700) Support nested grouping sets

2019-10-31 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-29700: - Description: PgSQL can process nested grouping sets, but Spark cannot; {code:java} postg

[jira] [Updated] (SPARK-29704) Support the combinations of grouping operations

2019-10-31 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-29704: - Description: PgSQL can accept a query below with the combinations of grouping operations

[jira] [Updated] (SPARK-29704) Support the combinations of grouping operations

2019-10-31 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-29704: - Description: PgSQL can accept a query below with the combinations of grouping operations

[jira] [Updated] (SPARK-29705) Support more expressive forms in GroupingSets/Cube/Rollup

2019-10-31 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-29705: - Description: {code:java} postgres=# create table gstest2 (a integer, b integer, c intege

[jira] [Created] (SPARK-29705) Support more expressive forms in GroupingSets/Cube/Rollup

2019-10-31 Thread Takeshi Yamamuro (Jira)
Takeshi Yamamuro created SPARK-29705: Summary: Support more expressive forms in GroupingSets/Cube/Rollup Key: SPARK-29705 URL: https://issues.apache.org/jira/browse/SPARK-29705 Project: Spark

[jira] [Assigned] (SPARK-24203) Make executor's bindAddress configurable

2019-10-31 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-24203: --- Assignee: Nishchal Venkataramana > Make executor's bindAddress configurable > -

[jira] [Created] (SPARK-29704) Support the combinations of grouping operations

2019-10-31 Thread Takeshi Yamamuro (Jira)
Takeshi Yamamuro created SPARK-29704: Summary: Support the combinations of grouping operations Key: SPARK-29704 URL: https://issues.apache.org/jira/browse/SPARK-29704 Project: Spark Issue

[jira] [Reopened] (SPARK-24203) Make executor's bindAddress configurable

2019-10-31 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reopened SPARK-24203: - > Make executor's bindAddress configurable > > > Ke

[jira] [Commented] (SPARK-29670) Make executor's bindAddress configurable

2019-10-31 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964599#comment-16964599 ] DB Tsai commented on SPARK-29670: - This is a duplication of SPARK-29670 > Make executor

[jira] [Resolved] (SPARK-29670) Make executor's bindAddress configurable

2019-10-31 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-29670. - Resolution: Duplicate > Make executor's bindAddress configurable > -

[jira] [Created] (SPARK-29703) Support grouping() in GROUP BY without GroupingSets/Cube/Rollup

2019-10-31 Thread Takeshi Yamamuro (Jira)
Takeshi Yamamuro created SPARK-29703: Summary: Support grouping() in GROUP BY without GroupingSets/Cube/Rollup Key: SPARK-29703 URL: https://issues.apache.org/jira/browse/SPARK-29703 Project: Spar

[jira] [Created] (SPARK-29702) Resolve group-by columns with functional dependencies

2019-10-31 Thread Takeshi Yamamuro (Jira)
Takeshi Yamamuro created SPARK-29702: Summary: Resolve group-by columns with functional dependencies Key: SPARK-29702 URL: https://issues.apache.org/jira/browse/SPARK-29702 Project: Spark

[jira] [Created] (SPARK-29701) Different answers when empty input given in GROUPING SETS

2019-10-31 Thread Takeshi Yamamuro (Jira)
Takeshi Yamamuro created SPARK-29701: Summary: Different answers when empty input given in GROUPING SETS Key: SPARK-29701 URL: https://issues.apache.org/jira/browse/SPARK-29701 Project: Spark

[jira] [Created] (SPARK-29700) Support nested grouping sets

2019-10-31 Thread Takeshi Yamamuro (Jira)
Takeshi Yamamuro created SPARK-29700: Summary: Support nested grouping sets Key: SPARK-29700 URL: https://issues.apache.org/jira/browse/SPARK-29700 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-29699) Different answers in nested aggregates with window functions

2019-10-31 Thread Takeshi Yamamuro (Jira)
Takeshi Yamamuro created SPARK-29699: Summary: Different answers in nested aggregates with window functions Key: SPARK-29699 URL: https://issues.apache.org/jira/browse/SPARK-29699 Project: Spark

[jira] [Created] (SPARK-29698) Support grouping function with multiple arguments

2019-10-31 Thread Takeshi Yamamuro (Jira)
Takeshi Yamamuro created SPARK-29698: Summary: Support grouping function with multiple arguments Key: SPARK-29698 URL: https://issues.apache.org/jira/browse/SPARK-29698 Project: Spark Iss

[jira] [Resolved] (SPARK-29686) LinearSVC should persist instances if needed

2019-10-31 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-29686. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26344 [https://gi

[jira] [Created] (SPARK-29697) Support bit string types/literals

2019-10-31 Thread Takeshi Yamamuro (Jira)
Takeshi Yamamuro created SPARK-29697: Summary: Support bit string types/literals Key: SPARK-29697 URL: https://issues.apache.org/jira/browse/SPARK-29697 Project: Spark Issue Type: Sub-tas

[jira] [Comment Edited] (SPARK-27763) Port test cases from PostgreSQL to Spark SQL

2019-10-31 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964583#comment-16964583 ] Takeshi Yamamuro edited comment on SPARK-27763 at 11/1/19 3:54 AM: ---

[jira] [Commented] (SPARK-27763) Port test cases from PostgreSQL to Spark SQL

2019-10-31 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964583#comment-16964583 ] Takeshi Yamamuro commented on SPARK-27763: -- Thanks for the check, Hyukjin! I've

[jira] [Created] (SPARK-29696) Add groupingsets.sql

2019-10-31 Thread Takeshi Yamamuro (Jira)
Takeshi Yamamuro created SPARK-29696: Summary: Add groupingsets.sql Key: SPARK-29696 URL: https://issues.apache.org/jira/browse/SPARK-29696 Project: Spark Issue Type: Sub-task C

[jira] [Resolved] (SPARK-29676) ALTER TABLE (RENAME PARTITION) should look up catalog/table like v2 commands

2019-10-31 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-29676. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26350 [https://gith

[jira] [Assigned] (SPARK-29676) ALTER TABLE (RENAME PARTITION) should look up catalog/table like v2 commands

2019-10-31 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-29676: --- Assignee: Huaxin Gao > ALTER TABLE (RENAME PARTITION) should look up catalog/table like v2

[jira] [Resolved] (SPARK-29664) Column.getItem behavior is not consistent with Scala version

2019-10-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-29664. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26351 [https://gi

[jira] [Assigned] (SPARK-29664) Column.getItem behavior is not consistent with Scala version

2019-10-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-29664: Assignee: Terry Kim > Column.getItem behavior is not consistent with Scala version >

[jira] [Commented] (SPARK-29694) Execute UDF only once when there are multiple identical UDF usages

2019-10-31 Thread Xuedong Luan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964571#comment-16964571 ] Xuedong Luan commented on SPARK-29694: -- hi [~yumwang] I will work on this Jira >

[jira] [Updated] (SPARK-29657) Iterator spill supporting radix sort with null prefix

2019-10-31 Thread dzcxzl (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dzcxzl updated SPARK-29657: --- Issue Type: Bug (was: Improvement) > Iterator spill supporting radix sort with null prefix > --

[jira] [Commented] (SPARK-29695) ALTER TABLE (SerDe properties) should look up catalog/table like v2 commands

2019-10-31 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964559#comment-16964559 ] Huaxin Gao commented on SPARK-29695: I will work on this > ALTER TABLE (SerDe prope

[jira] [Created] (SPARK-29695) ALTER TABLE (SerDe properties) should look up catalog/table like v2 commands

2019-10-31 Thread Huaxin Gao (Jira)
Huaxin Gao created SPARK-29695: -- Summary: ALTER TABLE (SerDe properties) should look up catalog/table like v2 commands Key: SPARK-29695 URL: https://issues.apache.org/jira/browse/SPARK-29695 Project: Spa

[jira] [Created] (SPARK-29694) Execute UDF only once when there are multiple identical UDF usages

2019-10-31 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-29694: --- Summary: Execute UDF only once when there are multiple identical UDF usages Key: SPARK-29694 URL: https://issues.apache.org/jira/browse/SPARK-29694 Project: Spark

[jira] [Commented] (SPARK-23643) XORShiftRandom.hashSeed allocates unnecessary memory

2019-10-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964529#comment-16964529 ] Dongjoon Hyun commented on SPARK-23643: --- This causes the UI test result difference

[jira] [Updated] (SPARK-23643) XORShiftRandom.hashSeed allocates unnecessary memory

2019-10-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23643: -- Priority: Major (was: Trivial) > XORShiftRandom.hashSeed allocates unnecessary memory > -

[jira] [Updated] (SPARK-23643) XORShiftRandom.hashSeed allocates unnecessary memory

2019-10-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23643: -- Labels: release-notes (was: ) > XORShiftRandom.hashSeed allocates unnecessary memory > --

[jira] [Commented] (SPARK-23643) XORShiftRandom.hashSeed allocates unnecessary memory

2019-10-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964528#comment-16964528 ] Dongjoon Hyun commented on SPARK-23643: --- I added `release-note` label because this

[jira] [Commented] (SPARK-29693) Bucket map join if the one's bucket number is the multiple of the other

2019-10-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964491#comment-16964491 ] Yuming Wang commented on SPARK-29693: - cc [~gwang3] > Bucket map join if the one's

[jira] [Created] (SPARK-29693) Bucket map join if the one's bucket number is the multiple of the other

2019-10-31 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-29693: --- Summary: Bucket map join if the one's bucket number is the multiple of the other Key: SPARK-29693 URL: https://issues.apache.org/jira/browse/SPARK-29693 Project: Spark

[jira] [Commented] (SPARK-29693) Bucket map join if the one's bucket number is the multiple of the other

2019-10-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964490#comment-16964490 ] Yuming Wang commented on SPARK-29693: - https://data-flair.training/blogs/bucket-map-

[jira] [Commented] (SPARK-29625) Spark Structure Streaming Kafka Wrong Reset Offset twice

2019-10-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964489#comment-16964489 ] Hyukjin Kwon commented on SPARK-29625: -- It needs investigation. Can you share the c

[jira] [Commented] (SPARK-12806) Support SQL expressions extracting values from VectorUDT

2019-10-31 Thread John Bauer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-12806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964481#comment-16964481 ] John Bauer commented on SPARK-12806: Also, when using PyArrow to convert a Spark Dat

[jira] [Commented] (SPARK-12806) Support SQL expressions extracting values from VectorUDT

2019-10-31 Thread John Bauer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-12806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964475#comment-16964475 ] John Bauer commented on SPARK-12806: This is still a problem. For example, classifi

[jira] [Resolved] (SPARK-29687) Fix jdbc metrics counter type to long

2019-10-31 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-29687. -- Fix Version/s: 3.0.0 Assignee: ulysses you Resolution: Fixed Resolved

[jira] [Created] (SPARK-29692) SparkContext.defaultParallism should reflect resource limits when resource limits are set

2019-10-31 Thread Bago Amirbekian (Jira)
Bago Amirbekian created SPARK-29692: --- Summary: SparkContext.defaultParallism should reflect resource limits when resource limits are set Key: SPARK-29692 URL: https://issues.apache.org/jira/browse/SPARK-29692

[jira] [Updated] (SPARK-29691) Estimator fit method fails to copy params (in PySpark)

2019-10-31 Thread John Bauer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Bauer updated SPARK-29691: --- Description: Estimator `fit` method is supposed to copy a dictionary of params, overwriting the est

[jira] [Updated] (SPARK-29691) Estimator fit method fails to copy params (in PySpark)

2019-10-31 Thread John Bauer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Bauer updated SPARK-29691: --- Description: Estimator `fit` method (implemented in Params) is supposed to copy a dictionary of par

[jira] [Updated] (SPARK-29691) Estimator fit method fails to copy params (in PySpark)

2019-10-31 Thread John Bauer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Bauer updated SPARK-29691: --- Description: Estimator `fit` method (implemented in Params) is supposed to copy a dictionary of par

[jira] [Created] (SPARK-29691) Estimator fit method fails to copy params (in PySpark)

2019-10-31 Thread John Bauer (Jira)
John Bauer created SPARK-29691: -- Summary: Estimator fit method fails to copy params (in PySpark) Key: SPARK-29691 URL: https://issues.apache.org/jira/browse/SPARK-29691 Project: Spark Issue Type

[jira] [Comment Edited] (SPARK-29415) Stage Level Sched: Add base ResourceProfile and Request classes

2019-10-31 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964360#comment-16964360 ] Thomas Graves edited comment on SPARK-29415 at 10/31/19 8:03 PM: -

[jira] [Comment Edited] (SPARK-29415) Stage Level Sched: Add base ResourceProfile and Request classes

2019-10-31 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964360#comment-16964360 ] Thomas Graves edited comment on SPARK-29415 at 10/31/19 7:59 PM: -

[jira] [Commented] (SPARK-29415) Stage Level Sched: Add base ResourceProfile and Request classes

2019-10-31 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964360#comment-16964360 ] Thomas Graves commented on SPARK-29415: --- More details: TaskResourceRequest - this

[jira] [Commented] (SPARK-29673) upgrade jenkins pypy to PyPy3.6 v7.2.0

2019-10-31 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964359#comment-16964359 ] Shane Knapp commented on SPARK-29673: - pypy3.6-7.2.0-linux_x86_64-portable has been

[jira] [Commented] (SPARK-22579) BlockManager.getRemoteValues and BlockManager.getRemoteBytes should be implemented using streaming

2019-10-31 Thread Imran Rashid (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964281#comment-16964281 ] Imran Rashid commented on SPARK-22579: -- Sorry I had not noticed this issue before.

[jira] [Commented] (SPARK-29625) Spark Structure Streaming Kafka Wrong Reset Offset twice

2019-10-31 Thread Sandish Kumar HN (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964249#comment-16964249 ] Sandish Kumar HN commented on SPARK-29625: -- [~hyukjin.kwon] it is happening ran

[jira] [Commented] (SPARK-25923) SparkR UT Failure (checking CRAN incoming feasibility)

2019-10-31 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964247#comment-16964247 ] L. C. Hsieh commented on SPARK-25923: - Got reply back now. It should be fixed now.

[jira] [Commented] (SPARK-25923) SparkR UT Failure (checking CRAN incoming feasibility)

2019-10-31 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964236#comment-16964236 ] L. C. Hsieh commented on SPARK-25923: - Noticed that and asked help from CRAN two hou

[jira] [Commented] (SPARK-25923) SparkR UT Failure (checking CRAN incoming feasibility)

2019-10-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964232#comment-16964232 ] Sean R. Owen commented on SPARK-25923: -- [~viirya][~hyukjin.kwon][~dongjoon] Looks l

[jira] [Updated] (SPARK-29690) Spark Shell - Clear imports

2019-10-31 Thread dinesh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dinesh updated SPARK-29690: --- Description: I 'm facing below problem with Spark Shell. So, in a shell session - # I imported following - 

[jira] [Updated] (SPARK-29690) Spark Shell - Clear imports

2019-10-31 Thread dinesh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dinesh updated SPARK-29690: --- Description: I 'm facing below problem with Spark Shell. So, in a shell session - # I imported following - 

[jira] [Updated] (SPARK-29690) Spark Shell - Clear imports

2019-10-31 Thread dinesh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dinesh updated SPARK-29690: --- Description: I 'm facing below problem with Spark Shell. So, in a shell session - # I imported following - 

[jira] [Created] (SPARK-29690) Spark Shell - Clear imports

2019-10-31 Thread dinesh (Jira)
dinesh created SPARK-29690: -- Summary: Spark Shell - Clear imports Key: SPARK-29690 URL: https://issues.apache.org/jira/browse/SPARK-29690 Project: Spark Issue Type: Bug Components: Spark

[jira] [Updated] (SPARK-29644) ShortType is wrongly set as Int in JDBCUtils.scala

2019-10-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-29644: - Issue Type: Bug (was: New Feature) > ShortType is wrongly set as Int in JDBCUtils.scala > -

[jira] [Resolved] (SPARK-29675) Add exception when isolationLevel is Illegal

2019-10-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29675. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26334 [https://

[jira] [Assigned] (SPARK-29675) Add exception when isolationLevel is Illegal

2019-10-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29675: - Assignee: ulysses you > Add exception when isolationLevel is Illegal >

[jira] [Commented] (SPARK-29682) Failure when resolving conflicting references in Join:

2019-10-31 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964158#comment-16964158 ] Terry Kim commented on SPARK-29682: --- Sure, I will look into this. Thanks for pinging m

[jira] [Commented] (SPARK-29630) Not allowed to create a permanent view by referencing a temporary view in EXISTS

2019-10-31 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964157#comment-16964157 ] Terry Kim commented on SPARK-29630: --- Yes, I will take a look. > Not allowed to create

[jira] [Commented] (SPARK-29604) SessionState is initialized with isolated classloader for Hive if spark.sql.hive.metastore.jars is being set

2019-10-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964137#comment-16964137 ] Dongjoon Hyun commented on SPARK-29604: --- Thank you for keeping working on this. Ye

[jira] [Assigned] (SPARK-29277) DataSourceV2: Add early filter and projection pushdown

2019-10-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29277: - Assignee: Ryan Blue > DataSourceV2: Add early filter and projection pushdown >

[jira] [Resolved] (SPARK-29277) DataSourceV2: Add early filter and projection pushdown

2019-10-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29277. --- Resolution: Fixed Issue resolved by pull request 26341 [https://github.com/apache/spark/pull

[jira] [Updated] (SPARK-29689) [WEB-UI] When task failed during reading shuffle data or other failure, enable show total shuffle read size

2019-10-31 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29689: Description: As shown in the attachment, if task failed during reading shuffle data or because of executo

[jira] [Updated] (SPARK-29689) [WEB-UI] When task failed during reading shuffle data or other failure, enable show total shuffle read size

2019-10-31 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29689: Description: If task failed during reading shuffle data or because of executor loss, its shuffle read siz

[jira] [Updated] (SPARK-29689) [WEB-UI] When task failed during reading shuffle data or other failure, enable show total shuffle read size

2019-10-31 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29689: Attachment: screenshot-1.png > [WEB-UI] When task failed during reading shuffle data or other failure, >

[jira] [Updated] (SPARK-29689) [WEB-UI] When task failed during reading shuffle data or other failure, enable show total shuffle read size

2019-10-31 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29689: Summary: [WEB-UI] When task failed during reading shuffle data or other failure, enable show total shuffle

[jira] [Created] (SPARK-29689) [UI] When task failed during reading shuffle data or other failure, enable show total shuffle read size

2019-10-31 Thread feiwang (Jira)
feiwang created SPARK-29689: --- Summary: [UI] When task failed during reading shuffle data or other failure, enable show total shuffle read size Key: SPARK-29689 URL: https://issues.apache.org/jira/browse/SPARK-29689

[jira] [Created] (SPARK-29688) Support average with interval type values

2019-10-31 Thread Kent Yao (Jira)
Kent Yao created SPARK-29688: Summary: Support average with interval type values Key: SPARK-29688 URL: https://issues.apache.org/jira/browse/SPARK-29688 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-29687) Fix jdbc metrics counter type to long

2019-10-31 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964015#comment-16964015 ] ulysses you commented on SPARK-29687: - Sorry for a misstake. > Fix jdbc metrics cou

[jira] [Comment Edited] (SPARK-29687) Fix jdbc metrics counter type to long

2019-10-31 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964011#comment-16964011 ] ulysses you edited comment on SPARK-29687 at 10/31/19 1:33 PM: ---

[jira] [Commented] (SPARK-29687) Fix jdbc metrics counter type to long

2019-10-31 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964011#comment-16964011 ] ulysses you commented on SPARK-29687: - You can just see the pr [26334|[https://githu

[jira] [Comment Edited] (SPARK-29687) Fix jdbc metrics counter type to long

2019-10-31 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964011#comment-16964011 ] ulysses you edited comment on SPARK-29687 at 10/31/19 1:31 PM: ---

[jira] [Commented] (SPARK-29687) Fix jdbc metrics counter type to long

2019-10-31 Thread jobit mathew (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963993#comment-16963993 ] jobit mathew commented on SPARK-29687: -- Hi can you give some details about this var

[jira] [Updated] (SPARK-29687) Fix jdbc metrics counter type to long

2019-10-31 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-29687: Affects Version/s: (was: 2.4.4) 3.0.0 > Fix jdbc metrics counter type t

[jira] [Created] (SPARK-29687) Fix jdbc metrics counter type to long

2019-10-31 Thread ulysses you (Jira)
ulysses you created SPARK-29687: --- Summary: Fix jdbc metrics counter type to long Key: SPARK-29687 URL: https://issues.apache.org/jira/browse/SPARK-29687 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-29630) Not allowed to create a permanent view by referencing a temporary view in EXISTS

2019-10-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963858#comment-16963858 ] Wenchen Fan commented on SPARK-29630: - Yea this should be disallowed. We store view

[jira] [Commented] (SPARK-29682) Failure when resolving conflicting references in Join:

2019-10-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963850#comment-16963850 ] Wenchen Fan commented on SPARK-29682: - [~imback82] do you want to look into it? > F

[jira] [Updated] (SPARK-29685) Spark SQL also better to show the column details while doing SELECT * from table, like sparkshell and spark beeline

2019-10-31 Thread jobit mathew (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jobit mathew updated SPARK-29685: - Issue Type: Improvement (was: Bug) > Spark SQL also better to show the column details while doi

[jira] [Updated] (SPARK-29685) Spark SQL also better to show the column details while doing SELECT * from table, like sparkshell and spark beeline

2019-10-31 Thread jobit mathew (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jobit mathew updated SPARK-29685: - Description: Spark SQL also better to show the column details in top while doing SELECT * from

  1   2   >