Dhaniram Kshirsagar created SPARK-23761:
---
Summary: Dataframe filter(udf) followed by groupby in pyspark
throws a casting error
Key: SPARK-23761
URL: https://issues.apache.org/jira/browse/SPARK-23761
[
https://issues.apache.org/jira/browse/SPARK-23666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-23666.
-
Resolution: Fixed
Assignee: Takeshi Yamamuro
Fix Version/s: 2.4.0
> Undeterminist
[
https://issues.apache.org/jira/browse/SPARK-23234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler resolved SPARK-23234.
--
Resolution: Duplicate
> ML python test failure due to default outputCol
> -
[
https://issues.apache.org/jira/browse/SPARK-23244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407510#comment-16407510
]
Bryan Cutler commented on SPARK-23244:
--
Just to clarify, the PySpark save/load is ju
[
https://issues.apache.org/jira/browse/SPARK-23244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler resolved SPARK-23244.
--
Resolution: Duplicate
> Incorrect handling of default values when deserializing python wrappers
[
https://issues.apache.org/jira/browse/SPARK-23244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407507#comment-16407507
]
Bryan Cutler commented on SPARK-23244:
--
I looked into this and it is a little bit di
[
https://issues.apache.org/jira/browse/SPARK-23760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407466#comment-16407466
]
Apache Spark commented on SPARK-23760:
--
User 'rednaxelafx' has created a pull reques
[
https://issues.apache.org/jira/browse/SPARK-23760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-23760:
Assignee: (was: Apache Spark)
> CodegenContext.withSubExprEliminationExprs should save
[
https://issues.apache.org/jira/browse/SPARK-23760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-23760:
Assignee: Apache Spark
> CodegenContext.withSubExprEliminationExprs should save/restore CS
Kris Mok created SPARK-23760:
Summary: CodegenContext.withSubExprEliminationExprs should
save/restore CSE state correctly
Key: SPARK-23760
URL: https://issues.apache.org/jira/browse/SPARK-23760
Project: S
[
https://issues.apache.org/jira/browse/SPARK-19208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407451#comment-16407451
]
Teng Peng edited comment on SPARK-19208 at 3/21/18 4:44 AM:
[
[
https://issues.apache.org/jira/browse/SPARK-19208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407451#comment-16407451
]
Teng Peng commented on SPARK-19208:
---
[~timhunter] Has the Jira ticket been opened? I be
[
https://issues.apache.org/jira/browse/SPARK-23519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16389150#comment-16389150
]
Franck Tago edited comment on SPARK-23519 at 3/21/18 3:29 AM:
-
[
https://issues.apache.org/jira/browse/SPARK-23519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Franck Tago updated SPARK-23519:
Description:
1- create and populate a hive table . I did this in a hive cli session .[ not
that t
[
https://issues.apache.org/jira/browse/SPARK-23513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407355#comment-16407355
]
abel-sun commented on SPARK-23513:
--
Can you provide some more error message![~Fray]
> j
[
https://issues.apache.org/jira/browse/SPARK-20709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407334#comment-16407334
]
KaiXinXIaoLei commented on SPARK-20709:
---
[~ffbin] [~srowen] i also meet this proble
[
https://issues.apache.org/jira/browse/SPARK-23751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407302#comment-16407302
]
Weichen Xu commented on SPARK-23751:
I will work on this. :)
> Kolmogorov-Smirnoff t
[
https://issues.apache.org/jira/browse/SPARK-23455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joseph K. Bradley updated SPARK-23455:
--
Shepherd: Joseph K. Bradley
> Default Params in ML should be saved separately
> ---
[
https://issues.apache.org/jira/browse/SPARK-23759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407288#comment-16407288
]
Apache Spark commented on SPARK-23759:
--
User 'felixalbani' has created a pull reques
[
https://issues.apache.org/jira/browse/SPARK-23759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-23759:
Assignee: Apache Spark
> Unable to bind Spark2 history server to specific host name / IP
>
[
https://issues.apache.org/jira/browse/SPARK-23759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-23759:
Assignee: (was: Apache Spark)
> Unable to bind Spark2 history server to specific host
[
https://issues.apache.org/jira/browse/SPARK-23749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuming Wang updated SPARK-23749:
Description:
{noformat}
18/03/15 22:34:46 WARN Hive: Failed to register all functions.
org.apache.h
[
https://issues.apache.org/jira/browse/SPARK-23750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407248#comment-16407248
]
Apache Spark commented on SPARK-23750:
--
User 'ioana-delaney' has created a pull requ
[
https://issues.apache.org/jira/browse/SPARK-23750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-23750:
Assignee: Apache Spark
> [Performance] Inner Join Elimination based on Informational RI co
[
https://issues.apache.org/jira/browse/SPARK-23750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-23750:
Assignee: (was: Apache Spark)
> [Performance] Inner Join Elimination based on Informat
[
https://issues.apache.org/jira/browse/SPARK-23534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407247#comment-16407247
]
Darek commented on SPARK-23534:
---
https://github.com/Azure/azure-storage-java 7.0 will only
[
https://issues.apache.org/jira/browse/SPARK-20697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Madav updated SPARK-20697:
---
Priority: Critical (was: Major)
> MSCK REPAIR TABLE resets the Storage Information for bucke
[
https://issues.apache.org/jira/browse/SPARK-20697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Madav updated SPARK-20697:
---
Affects Version/s: 2.2.0
2.2.1
2.3.0
> MSCK REP
[
https://issues.apache.org/jira/browse/SPARK-23686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407205#comment-16407205
]
Joseph K. Bradley commented on SPARK-23686:
---
This will be useful! Synced offli
[
https://issues.apache.org/jira/browse/SPARK-10884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407163#comment-16407163
]
Joseph K. Bradley commented on SPARK-10884:
---
I know a lot of people are watchin
Felix created SPARK-23759:
-
Summary: Unable to bind Spark2 history server to specific host
name / IP
Key: SPARK-23759
URL: https://issues.apache.org/jira/browse/SPARK-23759
Project: Spark
Issue Type
[
https://issues.apache.org/jira/browse/SPARK-23739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407112#comment-16407112
]
Marco Gaido commented on SPARK-23739:
-
Can you provide some more info about how you a
[
https://issues.apache.org/jira/browse/SPARK-18813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407108#comment-16407108
]
Joseph K. Bradley commented on SPARK-18813:
---
I just linked the roadmap for 2.4
[
https://issues.apache.org/jira/browse/SPARK-23758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joseph K. Bradley updated SPARK-23758:
--
Description:
h1. Roadmap process
This roadmap is a master list for MLlib improvements
Joseph K. Bradley created SPARK-23758:
-
Summary: MLlib 2.4 Roadmap
Key: SPARK-23758
URL: https://issues.apache.org/jira/browse/SPARK-23758
Project: Spark
Issue Type: New Feature
[
https://issues.apache.org/jira/browse/SPARK-23690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joseph K. Bradley updated SPARK-23690:
--
Shepherd: Joseph K. Bradley
> VectorAssembler should have handleInvalid to handle colum
[
https://issues.apache.org/jira/browse/SPARK-23500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li resolved SPARK-23500.
-
Resolution: Fixed
Assignee: Henry Robinson
Fix Version/s: 2.4.0
> Filters on named_struct
Ioana Delaney created SPARK-23757:
-
Summary: [Performance] Star schema detection improvements
Key: SPARK-23757
URL: https://issues.apache.org/jira/browse/SPARK-23757
Project: Spark
Issue Type
[
https://issues.apache.org/jira/browse/SPARK-19842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406909#comment-16406909
]
Ioana Delaney commented on SPARK-19842:
---
I opened several performance JIRAs to show
Ioana Delaney created SPARK-23756:
-
Summary: [Performance] Redundant join elimination
Key: SPARK-23756
URL: https://issues.apache.org/jira/browse/SPARK-23756
Project: Spark
Issue Type: Sub-ta
Ioana Delaney created SPARK-23755:
-
Summary: [Performance] Distinct elimination
Key: SPARK-23755
URL: https://issues.apache.org/jira/browse/SPARK-23755
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-23754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin updated SPARK-23754:
---
Description:
Reproduce:
{code:java}
df = spark.range(0, 1000)
from pyspark.sql.functions import udf
def foo(
[
https://issues.apache.org/jira/browse/SPARK-23754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin updated SPARK-23754:
---
Description:
{code:java}
df = spark.range(0, 1000)
from pyspark.sql.functions import udf
def foo(x):
rai
Li Jin created SPARK-23754:
--
Summary: StopIterator exception in Python UDF results in partial
result
Key: SPARK-23754
URL: https://issues.apache.org/jira/browse/SPARK-23754
Project: Spark
Issue Typ
[
https://issues.apache.org/jira/browse/SPARK-6190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406875#comment-16406875
]
Matthew Porter commented on SPARK-6190:
---
Experiencing similar frustrations to Brian,
[
https://issues.apache.org/jira/browse/SPARK-23737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-23737.
--
Resolution: Duplicate
> Scala API documentation leads to nonexistent pages for sources
> --
[
https://issues.apache.org/jira/browse/SPARK-23574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-23574.
-
Resolution: Fixed
Assignee: Jose Torres
Fix Version/s: 2.4.0
> SinglePartition in
Ioana Delaney created SPARK-23753:
-
Summary: [Performance] Group By Push Down through Join
Key: SPARK-23753
URL: https://issues.apache.org/jira/browse/SPARK-23753
Project: Spark
Issue Type: S
[
https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bruce Robbins updated SPARK-23715:
--
Description:
This produces the expected answer:
{noformat}
df.select(from_utc_timestamp(lit("20
[
https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406836#comment-16406836
]
Bruce Robbins commented on SPARK-23715:
---
A fix to this requires some ugly hacking o
Ioana Delaney created SPARK-23752:
-
Summary: [Performance] Existential Subquery to Inner Join
Key: SPARK-23752
URL: https://issues.apache.org/jira/browse/SPARK-23752
Project: Spark
Issue Type
[
https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bruce Robbins updated SPARK-23715:
--
Description:
This produces the expected answer:
{noformat}
df.select(from_utc_timestamp(lit("20
[
https://issues.apache.org/jira/browse/SPARK-23519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Franck Tago updated SPARK-23519:
Component/s: SQL
> Create View Commands Fails with The view output (col1,col1) contains
> duplica
[
https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bruce Robbins updated SPARK-23715:
--
Description:
This produces the expected answer:
{noformat}
df.select(from_utc_timestamp(lit("20
[
https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bruce Robbins updated SPARK-23715:
--
Description:
This produces the expected answer:
{noformat}
df.select(from_utc_timestamp(lit("20
Joseph K. Bradley created SPARK-23751:
-
Summary: Kolmogorov-Smirnoff test Python API in pyspark.ml
Key: SPARK-23751
URL: https://issues.apache.org/jira/browse/SPARK-23751
Project: Spark
I
[
https://issues.apache.org/jira/browse/SPARK-21898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joseph K. Bradley resolved SPARK-21898.
---
Resolution: Fixed
Fix Version/s: 2.4.0
Issue resolved by pull request 19108
[h
[
https://issues.apache.org/jira/browse/SPARK-21898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joseph K. Bradley reassigned SPARK-21898:
-
Assignee: Weichen Xu
> Feature parity for KolmogorovSmirnovTest in MLlib
> -
[
https://issues.apache.org/jira/browse/SPARK-23499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406796#comment-16406796
]
Pascal GILLET commented on SPARK-23499:
---
[~susanxhuynh] Certainly, none of the prop
Ioana Delaney created SPARK-23750:
-
Summary: [Performance] Inner Join Elimination based on
Informational RI constraints
Key: SPARK-23750
URL: https://issues.apache.org/jira/browse/SPARK-23750
Project:
[
https://issues.apache.org/jira/browse/SPARK-23749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-23749:
Assignee: (was: Apache Spark)
> Avoid Hive.get() to compatible with different Hive met
[
https://issues.apache.org/jira/browse/SPARK-23749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406724#comment-16406724
]
Apache Spark commented on SPARK-23749:
--
User 'wangyum' has created a pull request fo
[
https://issues.apache.org/jira/browse/SPARK-23749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-23749:
Assignee: Apache Spark
> Avoid Hive.get() to compatible with different Hive metastore
> --
Yuming Wang created SPARK-23749:
---
Summary: Avoid Hive.get() to compatible with different Hive
metastore
Key: SPARK-23749
URL: https://issues.apache.org/jira/browse/SPARK-23749
Project: Spark
I
Jose Torres created SPARK-23748:
---
Summary: Support select from temp tables
Key: SPARK-23748
URL: https://issues.apache.org/jira/browse/SPARK-23748
Project: Spark
Issue Type: Sub-task
Jose Torres created SPARK-23747:
---
Summary: Add EpochCoordinator unit tests
Key: SPARK-23747
URL: https://issues.apache.org/jira/browse/SPARK-23747
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-23737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406591#comment-16406591
]
Alexander Bessonov commented on SPARK-23737:
Oh, thanks. Linked them.
> Scal
Izhar Ahmed created SPARK-23746:
---
Summary: HashMap UserDefinedType giving cast exception in Spark
1.6.2 while implementing UDAF
Key: SPARK-23746
URL: https://issues.apache.org/jira/browse/SPARK-23746
Pr
[
https://issues.apache.org/jira/browse/SPARK-23542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-23542:
Assignee: Apache Spark
> The exists action shoule be further optimized in logical plan
> -
[
https://issues.apache.org/jira/browse/SPARK-23542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406172#comment-16406172
]
Apache Spark commented on SPARK-23542:
--
User 'KaiXinXiaoLei' has created a pull requ
[
https://issues.apache.org/jira/browse/SPARK-23542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-23542:
Assignee: (was: Apache Spark)
> The exists action shoule be further optimized in logic
[
https://issues.apache.org/jira/browse/SPARK-23542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
KaiXinXIaoLei updated SPARK-23542:
--
Description:
The optimized logical plan of query '*select * from tt1 where exists (select *
f
[
https://issues.apache.org/jira/browse/SPARK-23513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406164#comment-16406164
]
Narsireddy AVula commented on SPARK-23513:
--
Seems provided information is not su
[
https://issues.apache.org/jira/browse/SPARK-16745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406153#comment-16406153
]
Sujit Kumar Mahapatra commented on SPARK-16745:
---
+1. Getting similar issue
[
https://issues.apache.org/jira/browse/SPARK-16872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406128#comment-16406128
]
zhengruifeng commented on SPARK-16872:
--
I think both 1) a new GNB estimator and 2) c
[
https://issues.apache.org/jira/browse/SPARK-23745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-23745:
Assignee: Apache Spark
> Remove the directories of the “hive.downloaded.resources.dir” whe
[
https://issues.apache.org/jira/browse/SPARK-23745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-23745:
Assignee: (was: Apache Spark)
> Remove the directories of the “hive.downloaded.resourc
[
https://issues.apache.org/jira/browse/SPARK-23745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406050#comment-16406050
]
Apache Spark commented on SPARK-23745:
--
User 'zuotingbing' has created a pull reques
[
https://issues.apache.org/jira/browse/SPARK-23745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zuotingbing updated SPARK-23745:
Description:
!2018-03-20_164832.png!
when start the HiveThriftServer2, we create some directorie
[
https://issues.apache.org/jira/browse/SPARK-23745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zuotingbing updated SPARK-23745:
Description:
!2018-03-20_164832.png!
when start the HiveThriftServer2, we create some directori
[
https://issues.apache.org/jira/browse/SPARK-23745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zuotingbing updated SPARK-23745:
Description:
when start the HiveThriftServer2, we create some directories for
hive.downloaded.
[
https://issues.apache.org/jira/browse/SPARK-23745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zuotingbing updated SPARK-23745:
Attachment: 2018-03-20_164832.png
> Remove the directories of the “hive.downloaded.resources.dir” w
zuotingbing created SPARK-23745:
---
Summary: Remove the directories of the
“hive.downloaded.resources.dir” when HiveThriftServer2 stopped
Key: SPARK-23745
URL: https://issues.apache.org/jira/browse/SPARK-23745
[
https://issues.apache.org/jira/browse/SPARK-23691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-23691:
-
Fix Version/s: 2.3.1
> Use sql_conf util in PySpark tests where possible
> --
[
https://issues.apache.org/jira/browse/SPARK-20964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405962#comment-16405962
]
Alex Ott edited comment on SPARK-20964 at 3/20/18 8:36 AM:
---
Jus
[
https://issues.apache.org/jira/browse/SPARK-20964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405962#comment-16405962
]
Alex Ott edited comment on SPARK-20964 at 3/20/18 8:35 AM:
---
Jus
[
https://issues.apache.org/jira/browse/SPARK-20964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405962#comment-16405962
]
Alex Ott commented on SPARK-20964:
--
Just want to add another example of query that is re
87 matches
Mail list logo
