[GitHub] spark issue #14439: [SPARK-16714][SPARK-16735][SPARK-16646] array, map, grea...

2016-08-03 Thread yhuai
Github user yhuai commented on the issue:

https://github.com/apache/spark/pull/14439
  
OK. I am merging this PR to master and branch 2.0.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #14439: [SPARK-16714][SPARK-16735][SPARK-16646] array, map, grea...

2016-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14439
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63177/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #14439: [SPARK-16714][SPARK-16735][SPARK-16646] array, map, grea...

2016-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14439
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #14439: [SPARK-16714][SPARK-16735][SPARK-16646] array, map, grea...

2016-08-03 Thread SparkQA
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14439
  
**[Test build #63177 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63177/consoleFull)**
 for PR 14439 at commit 
[`9f1e642`](https://github.com/apache/spark/commit/9f1e6421d973390bb91fb79e681c0633091b1698).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #14439: [SPARK-16714][SPARK-16735][SPARK-16646] array, map, grea...

2016-08-03 Thread SparkQA
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14439
  
**[Test build #63177 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63177/consoleFull)**
 for PR 14439 at commit 
[`9f1e642`](https://github.com/apache/spark/commit/9f1e6421d973390bb91fb79e681c0633091b1698).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #14439: [SPARK-16714][SPARK-16735][SPARK-16646] array, map, grea...

2016-08-03 Thread yhuai
Github user yhuai commented on the issue:

https://github.com/apache/spark/pull/14439
  
@cloud-fan Thanks for the fix. The new logic looks good. I will merge it 
once jenkins pass.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #14439: [SPARK-16714][SPARK-16735][SPARK-16646] array, map, grea...

2016-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14439
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #14439: [SPARK-16714][SPARK-16735][SPARK-16646] array, map, grea...

2016-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14439
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63111/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #14439: [SPARK-16714][SPARK-16735][SPARK-16646] array, map, grea...

2016-08-02 Thread SparkQA
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14439
  
**[Test build #63111 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63111/consoleFull)**
 for PR 14439 at commit 
[`9def789`](https://github.com/apache/spark/commit/9def789e609f62af3b90a91e4ab6df377a76b50d).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #14439: [SPARK-16714][SPARK-16735][SPARK-16646] array, map, grea...

2016-08-02 Thread SparkQA
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14439
  
**[Test build #63111 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63111/consoleFull)**
 for PR 14439 at commit 
[`9def789`](https://github.com/apache/spark/commit/9def789e609f62af3b90a91e4ab6df377a76b50d).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #14439: [SPARK-16714][SPARK-16735][SPARK-16646] array, map, grea...

2016-08-02 Thread cloud-fan
Github user cloud-fan commented on the issue:

https://github.com/apache/spark/pull/14439
  
@yhuai , I checked the decimal truncation logic in hive, hive will truncate 
`decimal(76, 38)` to `decimal(38, 0)`, which makes more senses than ours, as 
keeping the integral part can make the result more accurate.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #14439: [SPARK-16714][SPARK-16735][SPARK-16646] array, map, grea...

2016-08-01 Thread yhuai
Github user yhuai commented on the issue:

https://github.com/apache/spark/pull/14439
  
It will be good to summarize the behaviors of other systems in the 
description. Let's also explain the behavioral change of this pr in the 
description. So, others can understand its implication. 

Also, I am wondering if we can change the behavior of 
`DecimalPrecision.widerDecimalType`. Right now, `widerDecimalType` will 
truncate the integral part, which is not intuitive. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #14439: [SPARK-16714][SPARK-16735][SPARK-16646] array, map, grea...

2016-08-01 Thread rxin
Github user rxin commented on the issue:

https://github.com/apache/spark/pull/14439
  
@yhuai do you have a concrete suggestion other than being careful here?



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #14439: [SPARK-16714][SPARK-16735][SPARK-16646] array, map, grea...

2016-08-01 Thread yhuai
Github user yhuai commented on the issue:

https://github.com/apache/spark/pull/14439
  
Let's be careful at here. I am not sure we can just use 
`DecimalPrecision.widerDecimalType`, which produces `Decimal(38, 38)` when we 
have one decimal with the type of `Decimal(38, 0)` and another one with the 
type of `Decimal(38, 38)`.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #14439: [SPARK-16714][SPARK-16735][SPARK-16646] array, map, grea...

2016-08-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14439
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63080/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #14439: [SPARK-16714][SPARK-16735][SPARK-16646] array, map, grea...

2016-08-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14439
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #14439: [SPARK-16714][SPARK-16735][SPARK-16646] array, map, grea...

2016-08-01 Thread SparkQA
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14439
  
**[Test build #63080 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63080/consoleFull)**
 for PR 14439 at commit 
[`d48590e`](https://github.com/apache/spark/commit/d48590e3f1c3276dc030518b2b9581d33b46c4a2).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #14439: [SPARK-16714][SPARK-16735][SPARK-16646] array, map, grea...

2016-08-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14439
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #14439: [SPARK-16714][SPARK-16735][SPARK-16646] array, map, grea...

2016-08-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/14439
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63079/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #14439: [SPARK-16714][SPARK-16735][SPARK-16646] array, map, grea...

2016-08-01 Thread SparkQA
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14439
  
**[Test build #63079 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63079/consoleFull)**
 for PR 14439 at commit 
[`95f0866`](https://github.com/apache/spark/commit/95f0866b9ca82e117b7454a1d72706618f7db4f4).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #14439: [SPARK-16714][SPARK-16735][SPARK-16646] array, map, grea...

2016-08-01 Thread SparkQA
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14439
  
**[Test build #63080 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63080/consoleFull)**
 for PR 14439 at commit 
[`d48590e`](https://github.com/apache/spark/commit/d48590e3f1c3276dc030518b2b9581d33b46c4a2).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #14439: [SPARK-16714][SPARK-16735][SPARK-16646] array, map, grea...

2016-08-01 Thread SparkQA
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/14439
  
**[Test build #63079 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63079/consoleFull)**
 for PR 14439 at commit 
[`95f0866`](https://github.com/apache/spark/commit/95f0866b9ca82e117b7454a1d72706618f7db4f4).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #14439: [SPARK-16714][SPARK-16735][SPARK-16646] array, map, grea...

2016-08-01 Thread cloud-fan
Github user cloud-fan commented on the issue:

https://github.com/apache/spark/pull/14439
  
This is a quick fix for both master and 2.0 branch. After this we can adopt 
https://github.com/apache/spark/pull/14389 to make the code of master branch 
cleaner.

cc @petermaxlee @rxin @yhuai 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org