[
https://issues.apache.org/jira/browse/SPARK-28180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
M. Le Bihan updated SPARK-28180:
Description:
I am converting an _RDD_ spark program to a _Dataset_ one.
Once, it was converting a
[
https://issues.apache.org/jira/browse/SPARK-28180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
M. Le Bihan updated SPARK-28180:
Description:
I am converting an _RDD_ spark program to a _Dataset_ one.
Once, it was converting a
[
https://issues.apache.org/jira/browse/SPARK-28180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
M. Le Bihan updated SPARK-28180:
Description:
I am converting an _RDD_ spark program to a _Dataset_ one.
Once, it was converting a
M. Le Bihan created SPARK-28180:
---
Summary: Encoding CSV to Pojo works with Encoders.bean on RDD but
fail on asserts when attemtping it from a Dataset
Key: SPARK-28180
URL: https://issues.apache.org/jira/browse/SPARK
[
https://issues.apache.org/jira/browse/SPARK-28161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873813#comment-16873813
]
Hyukjin Kwon commented on SPARK-28161:
--
Can you show the full error messages? Also,
[
https://issues.apache.org/jira/browse/SPARK-28161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-28161.
--
Resolution: Invalid
> Can't build Spark 2.4.3 based on Ubuntu and Oracle Java 8 SDK v212
> ---
[
https://issues.apache.org/jira/browse/SPARK-28175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873804#comment-16873804
]
Shan Zhao commented on SPARK-28175:
---
We have another cluster with spark 2.3, and appar
[
https://issues.apache.org/jira/browse/SPARK-28175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873801#comment-16873801
]
Hyukjin Kwon commented on SPARK-28175:
--
hm, then was it not tested in Spark 2.3?
>
[
https://issues.apache.org/jira/browse/SPARK-28175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873800#comment-16873800
]
Shan Zhao commented on SPARK-28175:
---
No since only 2.0.0 is available in our cluster
[
https://issues.apache.org/jira/browse/SPARK-28152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-28152.
--
Resolution: Duplicate
> ShortType and FloatTypes are not correctly mapped to right JDBC types
[
https://issues.apache.org/jira/browse/SPARK-28170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-28170:
-
Component/s: PySpark
> DenseVector .toArray() and .values documentation do not specify they are
[
https://issues.apache.org/jira/browse/SPARK-28179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-28179:
Assignee: Apache Spark
> Avoid hard-coded config: spark.sql.globalTempDatabase
>
[
https://issues.apache.org/jira/browse/SPARK-28179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-28179:
Assignee: (was: Apache Spark)
> Avoid hard-coded config: spark.sql.globalTempDatabase
Yuming Wang created SPARK-28179:
---
Summary: Avoid hard-coded config: spark.sql.globalTempDatabase
Key: SPARK-28179
URL: https://issues.apache.org/jira/browse/SPARK-28179
Project: Spark
Issue Typ
[
https://issues.apache.org/jira/browse/SPARK-28168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-28168:
-
Description: Hadoop 3.2 support was added. We might have to add Hadoop-3.2
in release-build.sh.
[
https://issues.apache.org/jira/browse/SPARK-28168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-28168:
-
Description: Hadoop-3.2 release process in o release-build.sh. (was: Add
Hadoop-3.2 to release-
[
https://issues.apache.org/jira/browse/SPARK-28171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuming Wang updated SPARK-28171:
Summary: Difference of interval type conversion between SparkSQL and
PostgreSQL (was: Correct int
[
https://issues.apache.org/jira/browse/SPARK-28171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuming Wang updated SPARK-28171:
Issue Type: Sub-task (was: Bug)
Parent: SPARK-27764
> Correct interval type conversion be
[
https://issues.apache.org/jira/browse/SPARK-28172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-28172.
--
Resolution: Duplicate
> pyspark DataFrame equality operator
>
[
https://issues.apache.org/jira/browse/SPARK-28172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873764#comment-16873764
]
Hyukjin Kwon commented on SPARK-28172:
--
{code}
>>> df1 == df2
True
{cod
[
https://issues.apache.org/jira/browse/SPARK-28175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-28175:
-
Labels: (was: version)
> Exception when using userClassPathFirst
> ---
[
https://issues.apache.org/jira/browse/SPARK-28175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873762#comment-16873762
]
Hyukjin Kwon commented on SPARK-28175:
--
Are you able to test this against higher ve
[
https://issues.apache.org/jira/browse/SPARK-27369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xingbo Jiang resolved SPARK-27369.
--
Resolution: Done
> Standalone worker can load resource conf and discover resources
> -
[
https://issues.apache.org/jira/browse/SPARK-28178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Zhuge updated SPARK-28178:
---
Description:
Support multiple catalogs in the following InsertInto use cases:
* DataFrameWriter.ins
John Zhuge created SPARK-28178:
--
Summary: DataSourceV2: DataFrameWriter.insertInfo
Key: SPARK-28178
URL: https://issues.apache.org/jira/browse/SPARK-28178
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-27845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Zhuge updated SPARK-27845:
---
Description:
Support multiple catalogs in the following InsertInto use cases:
* INSERT INTO [TABLE]
[
https://issues.apache.org/jira/browse/SPARK-27845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Zhuge updated SPARK-27845:
---
Description:
Support multiple catalogs in the following use cases:
* INSERT INTO [TABLE] catalog.db
[
https://issues.apache.org/jira/browse/SPARK-28036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873696#comment-16873696
]
Shivu Sondur commented on SPARK-28036:
--
[~yumwang]
You can close this jira right?.
[
https://issues.apache.org/jira/browse/SPARK-28177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-28177:
Assignee: (was: Apache Spark)
> Adjust post shuffle partition number in adaptive exec
[
https://issues.apache.org/jira/browse/SPARK-28177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-28177:
Assignee: Apache Spark
> Adjust post shuffle partition number in adaptive execution
> ---
Carson Wang created SPARK-28177:
---
Summary: Adjust post shuffle partition number in adaptive execution
Key: SPARK-28177
URL: https://issues.apache.org/jira/browse/SPARK-28177
Project: Spark
Issu
[
https://issues.apache.org/jira/browse/SPARK-27292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873670#comment-16873670
]
Padmakar commented on SPARK-27292:
--
Hi, am facing a similar issue on my 10 node cluster
Josh Rosen created SPARK-28176:
--
Summary: Add Dataset.collect(PartialFunction) method for parity
with RDD API
Key: SPARK-28176
URL: https://issues.apache.org/jira/browse/SPARK-28176
Project: Spark
Shan Zhao created SPARK-28175:
-
Summary: Exception when using userClassPathFirst
Key: SPARK-28175
URL: https://issues.apache.org/jira/browse/SPARK-28175
Project: Spark
Issue Type: Bug
C
[
https://issues.apache.org/jira/browse/SPARK-27992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler resolved SPARK-27992.
--
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 24834
[https://gi
[
https://issues.apache.org/jira/browse/SPARK-27992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler reassigned SPARK-27992:
Assignee: Bryan Cutler
> PySpark socket server should sync with JVM connection thread fut
[
https://issues.apache.org/jira/browse/SPARK-28174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-28174:
--
Description:
This issue updates Kafka dependency to 2.3.0 to bring the following 9
client-sid
[
https://issues.apache.org/jira/browse/SPARK-28174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-28174:
--
Description:
This issue updates Kafka dependency to 2.3.0 to bring the following 9
client-sid
[
https://issues.apache.org/jira/browse/SPARK-28174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-28174:
--
Description:
This issue updates Kafka dependency to 2.3.0 to bring the following 10
client-si
[
https://issues.apache.org/jira/browse/SPARK-28174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-28174:
Assignee: (was: Apache Spark)
> Upgrade to Kafka 2.3.0
> --
>
>
[
https://issues.apache.org/jira/browse/SPARK-28174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-28174:
Assignee: Apache Spark
> Upgrade to Kafka 2.3.0
> --
>
>
[
https://issues.apache.org/jira/browse/SPARK-28165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873614#comment-16873614
]
Imran Rashid commented on SPARK-28165:
--
btw if anybody wants to investigate this mo
Dongjoon Hyun created SPARK-28174:
-
Summary: Upgrade to Kafka 2.3.0
Key: SPARK-28174
URL: https://issues.apache.org/jira/browse/SPARK-28174
Project: Spark
Issue Type: Improvement
Co
[
https://issues.apache.org/jira/browse/SPARK-28157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-28157:
--
Affects Version/s: 2.3.2
2.3.3
> Make SHS clear KVStore LogInfo for the
[
https://issues.apache.org/jira/browse/SPARK-28157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
DB Tsai resolved SPARK-28157.
-
Resolution: Resolved
Assignee: Dongjoon Hyun
Fix Version/s: 3.0.0
> Make SHS clear KVSto
[
https://issues.apache.org/jira/browse/SPARK-28164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen updated SPARK-28164:
--
Priority: Minor (was: Major)
> usage description does not match with shell scripts
>
[
https://issues.apache.org/jira/browse/SPARK-28164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen reassigned SPARK-28164:
-
Assignee: Shivu Sondur
> usage description does not match with shell scripts
>
[
https://issues.apache.org/jira/browse/SPARK-28164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-28164.
---
Resolution: Fixed
Fix Version/s: 2.4.4
2.3.4
3.0.0
Issu
[
https://issues.apache.org/jira/browse/SPARK-28173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873422#comment-16873422
]
Gabor Somogyi commented on SPARK-28173:
---
Will file the appropriate PR when Kafka p
[
https://issues.apache.org/jira/browse/SPARK-28173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873420#comment-16873420
]
Gabor Somogyi commented on SPARK-28173:
---
cc [~viktorsomogyi]
> Add Kafka delegati
Gabor Somogyi created SPARK-28173:
-
Summary: Add Kafka delegation token proxy user support
Key: SPARK-28173
URL: https://issues.apache.org/jira/browse/SPARK-28173
Project: Spark
Issue Type: I
[
https://issues.apache.org/jira/browse/SPARK-28005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Imran Rashid reassigned SPARK-28005:
Assignee: Gabor Somogyi
> SparkRackResolver should not log for resolving empty list
> ---
[
https://issues.apache.org/jira/browse/SPARK-28005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Imran Rashid resolved SPARK-28005.
--
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 24935
[https://gi
Hugo created SPARK-28172:
Summary: pyspark DataFrame equality operator
Key: SPARK-28172
URL: https://issues.apache.org/jira/browse/SPARK-28172
Project: Spark
Issue Type: Improvement
Compone
[
https://issues.apache.org/jira/browse/SPARK-28171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873318#comment-16873318
]
Yuming Wang commented on SPARK-28171:
-
{code:sql}
postgres=# select substr(version()
Zhu, Lipeng created SPARK-28171:
---
Summary: Correct interval type conversion behavior
Key: SPARK-28171
URL: https://issues.apache.org/jira/browse/SPARK-28171
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-28164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873275#comment-16873275
]
Shivu Sondur commented on SPARK-28164:
--
[~hannankan]
i updated the usage message a
[
https://issues.apache.org/jira/browse/SPARK-28164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-28164:
Assignee: Apache Spark
> usage description does not match with shell scripts
> --
[
https://issues.apache.org/jira/browse/SPARK-28164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-28164:
Assignee: (was: Apache Spark)
> usage description does not match with shell scripts
>
[
https://issues.apache.org/jira/browse/SPARK-28164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873272#comment-16873272
]
Apache Spark commented on SPARK-28164:
--
User 'shivusondur' has created a pull reque
Sivam Pasupathipillai created SPARK-28170:
-
Summary: DenseVector .toArray() and .values documentation do not
specify they are aliases
Key: SPARK-28170
URL: https://issues.apache.org/jira/browse/SPARK-28170
[
https://issues.apache.org/jira/browse/SPARK-26683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amar Agrawal updated SPARK-26683:
-
Fix Version/s: 3.0.0
> Incorrect value of "internal.metrics.input.recordsRead" when reading from
[
https://issues.apache.org/jira/browse/SPARK-28091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873161#comment-16873161
]
Steve Loughran commented on SPARK-28091:
IO metrics.
hmmm. Maybe, but there's a
t;{color:#d04437}dt{color}",
> if I use query :
> {code:java}
> select * from default.test where dt=20190625 or (dt = 20190626 and id in
> (1,2,3) )
> {code}
> In this case, Spark will resolve or condition as one expression, and since
> this {color:#33}expr {color}has refer
> if I use query :
> {code:java}
> select * from default.test where dt=20190625 or (dt = 20190626 and id in
> (1,2,3) )
> {code}
> In this case, Spark will resolve or condition as one expression, and since
> this {color:#33}expr {color}has reference of "{color:#FF}i
04437}default.test{color}, his partition col
is "{color:#d04437}dt{color}",
if I use query :
{code:java}
select * from default.test where dt=20190625 or (dt = 20190626 and id in
(1,2,3) )
{code}
In this case, Spark will resolve or condition as one expression, and since this
{color:#333
[
https://issues.apache.org/jira/browse/SPARK-28091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873041#comment-16873041
]
Luca Canali commented on SPARK-28091:
-
Indeed toString() is quite useful, thanks.
+
angerszhu created SPARK-28169:
-
Summary: Spark can’t push down predicate for OR expression
Key: SPARK-28169
URL: https://issues.apache.org/jira/browse/SPARK-28169
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-27100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
DB Tsai resolved SPARK-27100.
-
Resolution: Fixed
Fix Version/s: 2.4.4
Issue resolved by pull request 24957
[https://github.com/a
69 matches
Mail list logo
