[
https://issues.apache.org/jira/browse/SPARK-25411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wang, Gang updated SPARK-25411:
---
Attachment: range partition design doc.pdf
> Implement range partition in Spark
> --
[
https://issues.apache.org/jira/browse/SPARK-25411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wang, Gang updated SPARK-25411:
---
Attachment: (was: range partition design doc.pdf)
> Implement range partition in Spark
> ---
[
https://issues.apache.org/jira/browse/SPARK-17570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16642730#comment-16642730
]
Wang, Gang commented on SPARK-17570:
Any update?
> Avoid Hash and Exchange in Sort
[
https://issues.apache.org/jira/browse/SPARK-25411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16661610#comment-16661610
]
Wang, Gang commented on SPARK-25411:
[~cloud_fan] How do you think of this feature?
Wang, Gang created SPARK-25401:
--
Summary: Reorder the required ordering to match the table's output
ordering for bucket join
Key: SPARK-25401
URL: https://issues.apache.org/jira/browse/SPARK-25401
Projec
Wang, Gang created SPARK-25411:
--
Summary: Implement range partition in Spark
Key: SPARK-25411
URL: https://issues.apache.org/jira/browse/SPARK-25411
Project: Spark
Issue Type: New Feature
[
https://issues.apache.org/jira/browse/SPARK-25411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wang, Gang updated SPARK-25411:
---
Description:
In our PROD environment, there are some partitioned fact tables, which are all
quite h
[
https://issues.apache.org/jira/browse/SPARK-25411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wang, Gang updated SPARK-25411:
---
Description:
In our product environment, there are some partitioned fact tables, which are
all quit
[
https://issues.apache.org/jira/browse/SPARK-25411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16627145#comment-16627145
]
Wang, Gang commented on SPARK-25411:
Add a design doc, please help to review.
> Imp
[
https://issues.apache.org/jira/browse/SPARK-25411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wang, Gang updated SPARK-25411:
---
Attachment: range partition design doc.pdf
> Implement range partition in Spark
> --
Wang, Gang created SPARK-23373:
--
Summary: Can not execute "count distinct" queries on parquet
formatted table
Key: SPARK-23373
URL: https://issues.apache.org/jira/browse/SPARK-23373
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-23373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wang, Gang updated SPARK-23373:
---
Issue Type: Bug (was: New Feature)
> Can not execute "count distinct" queries on parquet formatted t
[
https://issues.apache.org/jira/browse/SPARK-23373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wang, Gang updated SPARK-23373:
---
Description:
I failed to run sql "select count(distinct n_name) from nation", table nation
is format
[
https://issues.apache.org/jira/browse/SPARK-23373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wang, Gang updated SPARK-23373:
---
Description:
I failed to run sql "select count(distinct n_name) from nation", table nation
is format
[
https://issues.apache.org/jira/browse/SPARK-23373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358355#comment-16358355
]
Wang, Gang commented on SPARK-23373:
Yes. Seems related to my test environment.
Whil
[
https://issues.apache.org/jira/browse/SPARK-23373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358355#comment-16358355
]
Wang, Gang edited comment on SPARK-23373 at 2/9/18 1:01 PM:
Y
[
https://issues.apache.org/jira/browse/SPARK-23839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16421906#comment-16421906
]
Wang, Gang commented on SPARK-23839:
Good point! Currently, Spark just take data size
[
https://issues.apache.org/jira/browse/SPARK-25401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16712873#comment-16712873
]
Wang, Gang commented on SPARK-25401:
Yeah. I think so.
And please make sure the ou
Wang, Gang created SPARK-26375:
--
Summary: Rule PruneFileSourcePartitions should be fired before any
other rules based on data size
Key: SPARK-26375
URL: https://issues.apache.org/jira/browse/SPARK-26375
[
https://issues.apache.org/jira/browse/SPARK-26375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wang, Gang updated SPARK-26375:
---
Summary: Rule PruneFileSourcePartitions should be fired before any other
rules based on table statis
[
https://issues.apache.org/jira/browse/SPARK-26375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wang, Gang updated SPARK-26375:
---
Description: In catalyst, some optimize rules are base on table statistics,
like rule ReorderJoin, i
[
https://issues.apache.org/jira/browse/SPARK-26375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wang, Gang resolved SPARK-26375.
Resolution: Not A Problem
> Rule PruneFileSourcePartitions should be fired before any other rules
[
https://issues.apache.org/jira/browse/SPARK-26375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16724827#comment-16724827
]
Wang, Gang commented on SPARK-26375:
Should be okay, filter on partition columns is
Wang, Gang created SPARK-26672:
--
Summary: SinglePartition should not satisfies
HashClusteredDistribution/OrderedDistribution
Key: SPARK-26672
URL: https://issues.apache.org/jira/browse/SPARK-26672
Projec
[
https://issues.apache.org/jira/browse/SPARK-26672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wang, Gang updated SPARK-26672:
---
Description:
If we are loading data to a *bucketed table* TEST_TABLE of which bucket number
is not
[
https://issues.apache.org/jira/browse/SPARK-26672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wang, Gang updated SPARK-26672:
---
Summary: SinglePartition may not satisfies
HashClusteredDistribution/OrderedDistribution (was: Sing
[
https://issues.apache.org/jira/browse/SPARK-26672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wang, Gang closed SPARK-26672.
--
> SinglePartition may not satisfies
> HashClusteredDistribution/OrderedDistribution
> ---
[
https://issues.apache.org/jira/browse/SPARK-26672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wang, Gang resolved SPARK-26672.
Resolution: Not A Problem
This is only a bug in our inner Spark version. It's ok in community vers
[
https://issues.apache.org/jira/browse/SPARK-25411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885028#comment-16885028
]
Wang, Gang commented on SPARK-25411:
I referred to ORACLE DDLs, quite close to PG.
Wang, Gang created SPARK-13004:
--
Summary: Support Non-Volatile Data and Operations
Key: SPARK-13004
URL: https://issues.apache.org/jira/browse/SPARK-13004
Project: Spark
Issue Type: Epic
[
https://issues.apache.org/jira/browse/SPARK-13004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wang, Gang updated SPARK-13004:
---
Description:
Based on our experiments, the SerDe-like operations have some significant
negative perf
[
https://issues.apache.org/jira/browse/SPARK-13004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wang, Gang updated SPARK-13004:
---
Description:
Based on our experiments, the SerDe-like operations have some significant
negative perf
[
https://issues.apache.org/jira/browse/SPARK-13004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wang, Gang updated SPARK-13004:
---
Description:
Based on our experiments, the SerDe-like operations have some significant
negative perf
[
https://issues.apache.org/jira/browse/SPARK-13004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15118288#comment-15118288
]
Wang, Gang commented on SPARK-13004:
Yes, That is one of prototype for concept of pro
[
https://issues.apache.org/jira/browse/SPARK-13004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15118288#comment-15118288
]
Wang, Gang edited comment on SPARK-13004 at 1/27/16 12:44 AM:
-
[
https://issues.apache.org/jira/browse/SPARK-13004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15118468#comment-15118468
]
Wang, Gang commented on SPARK-13004:
Yes, That is one of our prototypes for concept o
[
https://issues.apache.org/jira/browse/SPARK-13004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wang, Gang updated SPARK-13004:
---
Comment: was deleted
(was: Yes, That is one of our prototypes for concept of proof. We are preparing
[
https://issues.apache.org/jira/browse/SPARK-13004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15118468#comment-15118468
]
Wang, Gang edited comment on SPARK-13004 at 1/27/16 5:34 AM:
-
[
https://issues.apache.org/jira/browse/SPARK-13004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wang, Gang closed SPARK-13004.
--
Resolution: Later
Preparing actionable items so close it temporarily as Sean Owen suggested.
> Support
[
https://issues.apache.org/jira/browse/SPARK-13004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15119970#comment-15119970
]
Wang, Gang commented on SPARK-13004:
I have closed it according to your advice, Thank
Wang, Gang created SPARK-29189:
--
Summary: Add an option to ignore block locations when listing file
Key: SPARK-29189
URL: https://issues.apache.org/jira/browse/SPARK-29189
Project: Spark
Issue T
[
https://issues.apache.org/jira/browse/SPARK-29189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wang, Gang updated SPARK-29189:
---
Description:
In our PROD env, we have a pure Spark cluster, I think this is also pretty
common, whe
Wang, Gang created SPARK-32464:
--
Summary: Support skew handling on join with one side that has no
query stage
Key: SPARK-32464
URL: https://issues.apache.org/jira/browse/SPARK-32464
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-32464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wang, Gang updated SPARK-32464:
---
Summary: Support skew handling on join that has one side with no query
stage (was: Support skew han
[
https://issues.apache.org/jira/browse/SPARK-32464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166071#comment-17166071
]
Wang, Gang commented on SPARK-32464:
A PR [https://github.com/apache/spark/pull/2926
45 matches
Mail list logo
