[
https://issues.apache.org/jira/browse/SPARK-25175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593194#comment-16593194
]
Chenxiao Mao edited comment on SPARK-25175 at 8/27/18 6:56 AM:
---
[
https://issues.apache.org/jira/browse/SPARK-25175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593185#comment-16593185
]
Chenxiao Mao edited comment on SPARK-25175 at 8/27/18 6:45 AM:
---
[
https://issues.apache.org/jira/browse/SPARK-25175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593200#comment-16593200
]
Chenxiao Mao commented on SPARK-25175:
--
Also here is the similar investigation I di
[
https://issues.apache.org/jira/browse/SPARK-25175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chenxiao Mao reopened SPARK-25175:
--
> Case-insensitive field resolution when reading from ORC
> --
[
https://issues.apache.org/jira/browse/SPARK-25175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593194#comment-16593194
]
Chenxiao Mao commented on SPARK-25175:
--
[~dongjoon] [~yucai] Here is a brief summar
[
https://issues.apache.org/jira/browse/SPARK-25175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593185#comment-16593185
]
Chenxiao Mao commented on SPARK-25175:
--
Thorough investigation about ORC tables
{co
[
https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yucai updated SPARK-25206:
--
Description:
In current Spark 2.3.1, below query returns wrong data silently.
{code:java}
spark.range(10).writ
[
https://issues.apache.org/jira/browse/SPARK-25248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593176#comment-16593176
]
Apache Spark commented on SPARK-25248:
--
User 'mengxr' has created a pull request fo
Xiangrui Meng created SPARK-25248:
-
Summary: Audit barrier APIs for Spark 2.4
Key: SPARK-25248
URL: https://issues.apache.org/jira/browse/SPARK-25248
Project: Spark
Issue Type: Story
Xiangrui Meng created SPARK-25247:
-
Summary: Make RDDBarrier configurable
Key: SPARK-25247
URL: https://issues.apache.org/jira/browse/SPARK-25247
Project: Spark
Issue Type: Story
Co
[
https://issues.apache.org/jira/browse/SPARK-25175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593156#comment-16593156
]
Dongjoon Hyun commented on SPARK-25175:
---
Thanks, [~yucai]. I'm highly interested i
[
https://issues.apache.org/jira/browse/SPARK-25175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-25175.
---
Resolution: Cannot Reproduce
I followed the same direction given by SPARK-25132, but I canno
[
https://issues.apache.org/jira/browse/SPARK-25175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593152#comment-16593152
]
yucai commented on SPARK-25175:
---
I pinged [~seancxmao] offline, he will give more details.
[
https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yucai updated SPARK-25206:
--
Summary: wrong records are returned when Hive metastore schema and parquet
schema are in different letter case
[
https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yucai updated SPARK-25206:
--
Description:
In current Spark 2.3.1, below query returns wrong data silently.
{code:java}
spark.range(10).writ
[
https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yucai updated SPARK-25206:
--
Description:
In current Spark 2.3.1, below query returns wrong data silently.
{code:java}
spark.range(10).writ
[
https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yucai updated SPARK-25206:
--
Summary: data issue when Hive metastore schema and parquet schema are in
different letter cases (was: data is
[
https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yucai updated SPARK-25206:
--
Summary: data issue when Hive metastore schema and parquet schema have
different letter case (was: data issue
[
https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yucai updated SPARK-25206:
--
Description:
In current Spark 2.3.1, below query returns wrong data silently.
{code:java}
spark.range(10).writ
[
https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yucai updated SPARK-25206:
--
Summary: data issue when (was: data issue because wrong column is
pushdown for parquet)
> data issue when
[
https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yucai updated SPARK-25206:
--
Description:
In current Spark 2.3.1, below query returns wrong data silently.
{code:java}
spark.range(10).writ
[
https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yucai updated SPARK-25206:
--
Description:
In current Spark 2.3.1, below query returns wrong data silently.
{code:java}
spark.range(10).writ
[
https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yucai updated SPARK-25206:
--
Description:
In current Spark 2.3.1, below query returns wrong data silently.
{code:java}
spark.range(10).writ
[
https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593126#comment-16593126
]
yucai edited comment on SPARK-25206 at 8/27/18 2:27 AM:
[~dongjo
[
https://issues.apache.org/jira/browse/SPARK-25207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593128#comment-16593128
]
Dongjoon Hyun commented on SPARK-25207:
---
[~yucai]. My bad. Please ignore that. It
[
https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593126#comment-16593126
]
yucai commented on SPARK-25206:
---
[~dongjoon], because of the below root cause
{quote}Spark
[
https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yucai updated SPARK-25206:
--
Description:
In current Spark 2.3.1, below query returns wrong data silently.
{code:java}
spark.range(10).writ
[
https://issues.apache.org/jira/browse/SPARK-25221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Saisai Shao updated SPARK-25221:
Target Version/s: (was: 2.3.2, 2.4.0)
> [DEPLOY] Consistent trailing whitespace treatment of con
[
https://issues.apache.org/jira/browse/SPARK-25221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593124#comment-16593124
]
Saisai Shao commented on SPARK-25221:
-
I'm going to remove the target version, I don
[
https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yucai updated SPARK-25206:
--
Summary: data issue because wrong column is pushdown for parquet (was:
Wrong data may be returned for Parquet
[
https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593117#comment-16593117
]
Hyukjin Kwon commented on SPARK-25206:
--
[~yucai], mind fixing the JIRA title?
> Wr
[
https://issues.apache.org/jira/browse/SPARK-25207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593110#comment-16593110
]
yucai commented on SPARK-25207:
---
[~dongjoon] , sorry if I am confusing you.
This bug i
[
https://issues.apache.org/jira/browse/SPARK-25207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593108#comment-16593108
]
Dongjoon Hyun commented on SPARK-25207:
---
According to the PR, this seems to be a n
[
https://issues.apache.org/jira/browse/SPARK-25207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-25207:
--
Attachment: image.png
> Case-insensitve field resolution for filter pushdown when reading Parq
[
https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593102#comment-16593102
]
yucai commented on SPARK-25206:
---
I am OK with "known correctness bug in 2.3" way, just rai
[
https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593100#comment-16593100
]
yucai commented on SPARK-25206:
---
[~smilegator] , sure, I will add tests.
If we don't b
[
https://issues.apache.org/jira/browse/SPARK-25236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593098#comment-16593098
]
holdenk commented on SPARK-25236:
-
Probably. The only thing would be probably wanting to
[
https://issues.apache.org/jira/browse/SPARK-25236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593097#comment-16593097
]
Liang-Chi Hsieh commented on SPARK-25236:
-
hmm, maybe dumb question, can't we us
[
https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593096#comment-16593096
]
Wenchen Fan commented on SPARK-25206:
-
I'm fine to mark it as a known correctness bu
[
https://issues.apache.org/jira/browse/SPARK-24826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593085#comment-16593085
]
Michail Giannakopoulos edited comment on SPARK-24826 at 8/27/18 12:53 AM:
[
https://issues.apache.org/jira/browse/SPARK-24826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593085#comment-16593085
]
Michail Giannakopoulos commented on SPARK-24826:
[~dongjoon] I will and
[
https://issues.apache.org/jira/browse/SPARK-19355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593080#comment-16593080
]
Apache Spark commented on SPARK-19355:
--
User 'viirya' has created a pull request fo
[
https://issues.apache.org/jira/browse/SPARK-25207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-25207:
--
Description:
Currently, filter pushdown will not work if Parquet schema and Hive metastore
sc
[
https://issues.apache.org/jira/browse/SPARK-24766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-24766:
--
Labels: Parquet (was: )
> CreateHiveTableAsSelect and InsertIntoHiveDir won't generate decima
[
https://issues.apache.org/jira/browse/SPARK-24826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593067#comment-16593067
]
Dongjoon Hyun commented on SPARK-24826:
---
Hi, [~miccagiann]. Could you try that in
[
https://issues.apache.org/jira/browse/SPARK-25132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-25132:
--
Labels: Parquet (was: )
> Case-insensitive field resolution when reading from Parquet
> -
[
https://issues.apache.org/jira/browse/SPARK-25135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-25135:
--
Labels: Parquet correctness (was: correctness)
> insert datasource table may all null when se
[
https://issues.apache.org/jira/browse/SPARK-25207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-25207:
--
Labels: Parquet (was: )
> Case-insensitve field resolution for filter pushdown when reading P
[
https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-25206:
--
Labels: Parquet correctness (was: correctness)
> Wrong data may be returned for Parquet
> ---
[
https://issues.apache.org/jira/browse/SPARK-25135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593066#comment-16593066
]
Dongjoon Hyun commented on SPARK-25135:
---
[~yumwang]. Could you update your PR acco
[
https://issues.apache.org/jira/browse/SPARK-25135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-25135:
--
Description:
This happens on parquet.
How to reproduce in parquet.
{code:scala}
val path = "/
[
https://issues.apache.org/jira/browse/SPARK-25135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-25135:
--
Description:
How to reproduce:
{code:scala}
val path = "/tmp/spark/parquet"
val cnt = 30
spark
[
https://issues.apache.org/jira/browse/SPARK-25135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-25135:
--
Summary: insert datasource table may all null when select from view on
parquet (was: insert d
[
https://issues.apache.org/jira/browse/SPARK-25091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593061#comment-16593061
]
Dongjoon Hyun commented on SPARK-25091:
---
Hi, [~Chao Fang]. Could you remove `Spark
[
https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593041#comment-16593041
]
Xiao Li edited comment on SPARK-25206 at 8/26/18 10:45 PM:
---
Cu
[
https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593041#comment-16593041
]
Xiao Li commented on SPARK-25206:
-
Previously, we do not have a good test coverage when
[
https://issues.apache.org/jira/browse/SPARK-25246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593040#comment-16593040
]
shahid commented on SPARK-25246:
I am working on it :)
> When the spark.eventLog.compre
shahid created SPARK-25246:
--
Summary: When the spark.eventLog.compress is enabled, the
Application is not showing in the History server UI ('incomplete application'
page), initially.
Key: SPARK-25246
URL: https://issues
[
https://issues.apache.org/jira/browse/SPARK-25245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-25245:
Assignee: Apache Spark
> Explain regarding limiting modification on "spark.sql.shuffle.pa
[
https://issues.apache.org/jira/browse/SPARK-25245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593034#comment-16593034
]
Apache Spark commented on SPARK-25245:
--
User 'HeartSaVioR' has created a pull reque
[
https://issues.apache.org/jira/browse/SPARK-25245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-25245:
Assignee: (was: Apache Spark)
> Explain regarding limiting modification on "spark.sql
Jungtaek Lim created SPARK-25245:
Summary: Explain regarding limiting modification on
"spark.sql.shuffle.partitions" for structured streaming
Key: SPARK-25245
URL: https://issues.apache.org/jira/browse/SPARK-25245
[
https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593024#comment-16593024
]
Dongjoon Hyun commented on SPARK-25206:
---
Hi, [~yucai], [~cloud_fan], [~smilegator]
[
https://issues.apache.org/jira/browse/SPARK-25175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593013#comment-16593013
]
Dongjoon Hyun commented on SPARK-25175:
---
[~seancxmao]. If there is no example, we
[
https://issues.apache.org/jira/browse/SPARK-25244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anton Daitche updated SPARK-25244:
--
Description:
The setting `spark.sql.session.timeZone` is respected by PySpark when
converting
Anton Daitche created SPARK-25244:
-
Summary: [Python] Setting `spark.sql.session.timeZone` only
partially respected
Key: SPARK-25244
URL: https://issues.apache.org/jira/browse/SPARK-25244
Project: Spa
[
https://issues.apache.org/jira/browse/SPARK-25244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anton Daitche updated SPARK-25244:
--
Description:
The setting `spark.sql.session.timeZone` is respected by PySpark when
converting
[
https://issues.apache.org/jira/browse/SPARK-25243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-25243:
Assignee: Apache Spark
> Use FailureSafeParser in from_json
> ---
[
https://issues.apache.org/jira/browse/SPARK-25243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16592940#comment-16592940
]
Apache Spark commented on SPARK-25243:
--
User 'MaxGekk' has created a pull request f
[
https://issues.apache.org/jira/browse/SPARK-25243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-25243:
Assignee: (was: Apache Spark)
> Use FailureSafeParser in from_json
>
Maxim Gekk created SPARK-25243:
--
Summary: Use FailureSafeParser in from_json
Key: SPARK-25243
URL: https://issues.apache.org/jira/browse/SPARK-25243
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-23707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-23707.
--
Resolution: Cannot Reproduce
> Don't need shuffle exchange with single partition for 'spark.ra
[
https://issues.apache.org/jira/browse/SPARK-25013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-25013.
--
Resolution: Won't Fix
I wouldn't add this into Spark for now unless there's strong request fro
[
https://issues.apache.org/jira/browse/SPARK-10697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-10697:
Assignee: Apache Spark
> Lift Calculation in Association Rule mining
> --
[
https://issues.apache.org/jira/browse/SPARK-10697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-10697:
Assignee: (was: Apache Spark)
> Lift Calculation in Association Rule mining
> ---
[
https://issues.apache.org/jira/browse/SPARK-10697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16592899#comment-16592899
]
Apache Spark commented on SPARK-10697:
--
User 'mgaido91' has created a pull request
[
https://issues.apache.org/jira/browse/SPARK-23792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-23792.
---
Resolution: Fixed
Fix Version/s: 2.4.0
Issue resolved by pull request 20901
[https://github.c
[
https://issues.apache.org/jira/browse/SPARK-23792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen reassigned SPARK-23792:
-
Assignee: A Bradbury
> Documentation improvements for datetime functions
>
[
https://issues.apache.org/jira/browse/SPARK-25080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-25080.
--
Resolution: Cannot Reproduce
> NPE in HiveShim$.toCatalystDecimal(HiveShim.scala:110)
> --
[
https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16592893#comment-16592893
]
Hyukjin Kwon commented on SPARK-25206:
--
Please fix the JIRA title to reflect more p
[
https://issues.apache.org/jira/browse/SPARK-25135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16592890#comment-16592890
]
Yuming Wang commented on SPARK-25135:
-
Another serious case:
{code:scala}
withTem
[
https://issues.apache.org/jira/browse/SPARK-23698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16592848#comment-16592848
]
Apache Spark commented on SPARK-23698:
--
User 'HyukjinKwon' has created a pull reque
82 matches
Mail list logo
