[
https://issues.apache.org/jira/browse/SPARK-24399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16493121#comment-16493121
]
Takeshi Yamamuro commented on SPARK-24399:
--
I checked on master and branch-2.3 and this issue
[
https://issues.apache.org/jira/browse/SPARK-24401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jorge Machado updated SPARK-24401:
--
Attachment: (was: testDF.parquet)
> Aggreate on Decimal Types does not work
>
[
https://issues.apache.org/jira/browse/SPARK-24401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jorge Machado resolved SPARK-24401.
---
Resolution: Fixed
Was already Fixed on 2.3.0
> Aggreate on Decimal Types does not work
>
[
https://issues.apache.org/jira/browse/SPARK-24401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jorge Machado updated SPARK-24401:
--
Affects Version/s: (was: 2.3.0)
Fix Version/s: 2.3.0
> Aggreate on Decimal Types
[
https://issues.apache.org/jira/browse/SPARK-24401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16493118#comment-16493118
]
Jorge Machado commented on SPARK-24401:
---
Yes, I just tried my code and it works on spark 2.3.0
>
[
https://issues.apache.org/jira/browse/SPARK-24401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jorge Machado updated SPARK-24401:
--
Description:
Hi,
I think I found a really ugly bug in spark when performing aggregations
[
https://issues.apache.org/jira/browse/SPARK-24401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jorge Machado updated SPARK-24401:
--
Description:
Hi,
I think I found a really ugly bug in spark when performing aggregations
[
https://issues.apache.org/jira/browse/SPARK-24406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pratyush Sharma updated SPARK-24406:
Summary: Exposing custom spark scala ml transformers in pyspark (was:
Exposing custom
Pratyush Sharma created SPARK-24406:
---
Summary: Exposing custom spark scala transformers in pyspark
Key: SPARK-24406
URL: https://issues.apache.org/jira/browse/SPARK-24406
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-24404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liangchang Zhu updated SPARK-24404:
---
Description:
In CP mode, based on PR #21353 #21332 #21293 and the latest master
[
https://issues.apache.org/jira/browse/SPARK-24404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liangchang Zhu updated SPARK-24404:
---
Summary: Increase currentEpoch when meet a EpochMarker in
ContinuousQueuedDataReader.next()
[
https://issues.apache.org/jira/browse/SPARK-24404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liangchang Zhu updated SPARK-24404:
---
Description:
In CP mode, ContinuousQueuedDataReader.next() will be invoked by
[
https://issues.apache.org/jira/browse/SPARK-6190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16493061#comment-16493061
]
James Gan commented on SPARK-6190:
--
Thank, [~irashid]. It's great to see that your test shows most
[
https://issues.apache.org/jira/browse/SPARK-24377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-24377:
Assignee: Saisai Shao
> Make --py-files work in non pyspark application
>
[
https://issues.apache.org/jira/browse/SPARK-24377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-24377.
--
Resolution: Fixed
Fix Version/s: 2.4.0
Issue resolved by pull request 21420
[
https://issues.apache.org/jira/browse/SPARK-19613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-19613.
--
Resolution: Fixed
Assignee: Dongjoon Hyun
Fix Version/s: 2.4.0
[
https://issues.apache.org/jira/browse/SPARK-24401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16493015#comment-16493015
]
Takeshi Yamamuro commented on SPARK-24401:
--
I didn't narrow down and I'm not sure that this is
[
https://issues.apache.org/jira/browse/SPARK-24365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gengliang Wang updated SPARK-24365:
---
Summary: Add data source write benchmark (was: Add Parquet write benchmark)
> Add data
[
https://issues.apache.org/jira/browse/SPARK-24365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gengliang Wang updated SPARK-24365:
---
Description: Add data source write benchmark. So that it would be easier to
measure the
[
https://issues.apache.org/jira/browse/SPARK-24356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492960#comment-16492960
]
Edwina Lu commented on SPARK-24356:
---
Thanks! This is interesting, and could help with some of our
[
https://issues.apache.org/jira/browse/SPARK-24356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492931#comment-16492931
]
Imran Rashid commented on SPARK-24356:
--
Yeah Misha has a change ready -- I sent him a message about
Deepansh created SPARK-24405:
Summary: parameter for worker timeout in python
Key: SPARK-24405
URL: https://issues.apache.org/jira/browse/SPARK-24405
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-24405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepansh updated SPARK-24405:
-
Description:
Currently, the python worker timeout is set to 6 ms in code and cannot be
changed. I
[
https://issues.apache.org/jira/browse/SPARK-6190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492930#comment-16492930
]
Imran Rashid commented on SPARK-6190:
-
[~jamgan] yes, this particular task was done a while ago.
[
https://issues.apache.org/jira/browse/SPARK-24405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepansh updated SPARK-24405:
-
Summary: parameter for python worker timeout (was: parameter for worker
timeout in python)
>
[
https://issues.apache.org/jira/browse/SPARK-19613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-19613:
--
Summary: Flaky test: StateStoreRDDSuite (was: Flaky test:
StateStoreRDDSuite.versioning and
[
https://issues.apache.org/jira/browse/SPARK-19613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492804#comment-16492804
]
Apache Spark commented on SPARK-19613:
--
User 'dongjoon-hyun' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-19613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-19613:
Assignee: Apache Spark
> Flaky test: StateStoreRDDSuite.versioning and immutability
>
[
https://issues.apache.org/jira/browse/SPARK-19613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-19613:
Assignee: (was: Apache Spark)
> Flaky test: StateStoreRDDSuite.versioning and
[
https://issues.apache.org/jira/browse/SPARK-24404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492794#comment-16492794
]
Apache Spark commented on SPARK-24404:
--
User 'LiangchangZ' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-24404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24404:
Assignee: (was: Apache Spark)
> Increase currentEpoch when meet a EpochMarker in
>
[
https://issues.apache.org/jira/browse/SPARK-24404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24404:
Assignee: Apache Spark
> Increase currentEpoch when meet a EpochMarker in
>
[
https://issues.apache.org/jira/browse/SPARK-19613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun reopened SPARK-19613:
---
> Flaky test: StateStoreRDDSuite.versioning and immutability
>
[
https://issues.apache.org/jira/browse/SPARK-24404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liangchang Zhu updated SPARK-24404:
---
Description: In CP mode, ContinuousQueuedDataReader.next() will be invoked
by
Liangchang Zhu created SPARK-24404:
--
Summary: Increase currentEpoch when meet a EpochMarker in
ContinuousQueuedDataReader.next() in CP mode
Key: SPARK-24404
URL:
[
https://issues.apache.org/jira/browse/SPARK-24401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492781#comment-16492781
]
Marco Gaido commented on SPARK-24401:
-
I followed the repro steps using the file you attached and I
[
https://issues.apache.org/jira/browse/SPARK-24369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24369:
Assignee: Apache Spark
> A bug when having multiple distinct aggregations
>
[
https://issues.apache.org/jira/browse/SPARK-24369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24369:
Assignee: (was: Apache Spark)
> A bug when having multiple distinct aggregations
>
[
https://issues.apache.org/jira/browse/SPARK-24369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492767#comment-16492767
]
Apache Spark commented on SPARK-24369:
--
User 'maropu' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-24389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marco Gaido resolved SPARK-24389.
-
Resolution: Cannot Reproduce
> describe() can't work on column that name contain dots
>
[
https://issues.apache.org/jira/browse/SPARK-24375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492745#comment-16492745
]
Wenchen Fan commented on SPARK-24375:
-
For the PySpark side, we don't need to care about the
[
https://issues.apache.org/jira/browse/SPARK-24353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492682#comment-16492682
]
Stavros Kontopoulos commented on SPARK-24353:
-
[~liyinan926] I added the design doc in the
[
https://issues.apache.org/jira/browse/SPARK-24353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stavros Kontopoulos updated SPARK-24353:
Description:
Spark on K8s allows to place driver/executor pods on specific k8s
[
https://issues.apache.org/jira/browse/SPARK-24353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stavros Kontopoulos updated SPARK-24353:
Description:
Spark on K8s allows to place driver/executor pods on specific k8s
[
https://issues.apache.org/jira/browse/SPARK-24401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jorge Machado updated SPARK-24401:
--
Description:
Hi,
I think I found a really ugly bug in spark when performing aggregations
[
https://issues.apache.org/jira/browse/SPARK-22947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492624#comment-16492624
]
Tomasz Gawęda commented on SPARK-22947:
---
[~icexelloss] Hi, just wondering - will you continue to
Deepansh created SPARK-24403:
Summary: reuse r worker
Key: SPARK-24403
URL: https://issues.apache.org/jira/browse/SPARK-24403
Project: Spark
Issue Type: Improvement
Components: SparkR
[
https://issues.apache.org/jira/browse/SPARK-24387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492563#comment-16492563
]
Rui Li commented on SPARK-24387:
Instead of let HeartbeatReceiver tell TaskScheduler the executor is
[
https://issues.apache.org/jira/browse/SPARK-24401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jorge Machado updated SPARK-24401:
--
Description:
Hi,
I think I found a really ugly bug in spark when performing aggregations
[
https://issues.apache.org/jira/browse/SPARK-24381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Miles Yucht resolved SPARK-24381.
-
Resolution: Fixed
Fix Version/s: 2.3.2
This PR has merged. In adding these tests we
[
https://issues.apache.org/jira/browse/SPARK-24402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
DB Tsai reassigned SPARK-24402:
---
Assignee: DB Tsai
> Optimize `In` expression when only one element in the collection or
>
[
https://issues.apache.org/jira/browse/SPARK-24402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24402:
Assignee: Apache Spark
> Optimize `In` expression when only one element in the collection
[
https://issues.apache.org/jira/browse/SPARK-24402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24402:
Assignee: (was: Apache Spark)
> Optimize `In` expression when only one element in the
[
https://issues.apache.org/jira/browse/SPARK-24402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492352#comment-16492352
]
Apache Spark commented on SPARK-24402:
--
User 'dbtsai' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-24402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
DB Tsai updated SPARK-24402:
Description:
Two new rules in the logical plan optimizers are added.
# When there is only one element in
DB Tsai created SPARK-24402:
---
Summary: Optimize `In` expression when only one element in the
collection or collection is empty
Key: SPARK-24402
URL: https://issues.apache.org/jira/browse/SPARK-24402
[
https://issues.apache.org/jira/browse/SPARK-24359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492341#comment-16492341
]
Felix Cheung edited comment on SPARK-24359 at 5/28/18 7:10 AM:
---
re
[
https://issues.apache.org/jira/browse/SPARK-24359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492341#comment-16492341
]
Felix Cheung commented on SPARK-24359:
--
re repo/release/version - again to be clear, what Shivaram
[
https://issues.apache.org/jira/browse/SPARK-21024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492333#comment-16492333
]
Takeshi Yamamuro commented on SPARK-21024:
--
No, feel free to take over if someone has good
[
https://issues.apache.org/jira/browse/SPARK-21024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492331#comment-16492331
]
Vadim commented on SPARK-21024:
---
Any news regarding this issue?
> CSV parse mode handles Univocity parser
[
https://issues.apache.org/jira/browse/SPARK-24401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jorge Machado updated SPARK-24401:
--
Attachment: testDF.parquet
> Aggreate on Decimal Types does not work
>
[
https://issues.apache.org/jira/browse/SPARK-24401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jorge Machado updated SPARK-24401:
--
Attachment: (was: testDF.parquet)
> Aggreate on Decimal Types does not work
>
[
https://issues.apache.org/jira/browse/SPARK-24401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492326#comment-16492326
]
Jorge Machado commented on SPARK-24401:
---
Probably is worth to mention that my dataset is coming
Jorge Machado created SPARK-24401:
-
Summary: Aggreate on Decimal Types does not work
Key: SPARK-24401
URL: https://issues.apache.org/jira/browse/SPARK-24401
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-24401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jorge Machado updated SPARK-24401:
--
Attachment: testDF.parquet
> Aggreate on Decimal Types does not work
>
[
https://issues.apache.org/jira/browse/SPARK-6190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492320#comment-16492320
]
James Gan commented on SPARK-6190:
--
[~irashid], is the fix in an official release now? I saw the patch
[
https://issues.apache.org/jira/browse/SPARK-24371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
DB Tsai updated SPARK-24371:
Description:
Implemented *{{isInCollection}}* in DataFrame API for both Scala and Java, so
users can do
[
https://issues.apache.org/jira/browse/SPARK-24371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
DB Tsai updated SPARK-24371:
Description:
Implemented *{{isInCollection}}* in DataFrame API for both Scala and Java, so
users can do
[
https://issues.apache.org/jira/browse/SPARK-24371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
DB Tsai updated SPARK-24371:
Description:
Implemented *{{isinSet}}* in DataFrame API for both Scala and Java, so users
can do
{code}
[
https://issues.apache.org/jira/browse/SPARK-24371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
DB Tsai updated SPARK-24371:
Summary: Added isInCollection in DataFrame API for Scala and Java. (was:
Added isinSet in DataFrame API
[
https://issues.apache.org/jira/browse/SPARK-24400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492291#comment-16492291
]
Supreeth Sharma commented on SPARK-24400:
-
cc [~hyukjin.kwon]
> Issue with spark while accessing
71 matches
Mail list logo
