[GitHub] spark pull request: [SPARK-8583] [SPARK-5482] [BUILD] Refactor pyt...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6967#issuecomment-115976114 [Test build #35892 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35892/console) for PR 6967 at commit [`34c98d2`](https://github.com/apache/spark/commit/34c98d2bac769e1e57f5f4a5a13e94145402efd2). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class Module(object):` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8583] [SPARK-5482] [BUILD] Refactor pyt...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6967#issuecomment-115976126 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6380] Resolution of equi-join key in po...
Github user bryanjj commented on the pull request: https://github.com/apache/spark/pull/6141#issuecomment-115977592 ok sounds good, I'll close it --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6380] Resolution of equi-join key in po...
GitHub user bryanjj reopened a pull request: https://github.com/apache/spark/pull/6141 [SPARK-6380] Resolution of equi-join key in post-join projection create a new method in LogicalPlan for resolving ambiguous references and wrote an implentation for Project. handles cases like: df1.join(df2, df1(key) === df2(key)).select(key) You can merge this pull request into a Git repository by running: $ git pull https://github.com/bryanjj/spark spark-6380-equi-join-key Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/6141.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #6141 commit 09131bda73dcfc4b17b201aa4a788a4bce2d3c60 Author: Bryan Jacobs brya...@gmail.com Date: 2015-05-14T07:15:09Z [SPARK-6380] Resolution of equi-join key in post-join projection create a new method in LogicalPlan for resolving ambiguous references and wrote an implentation for Project. handles cases like: df1.join(df2, df1(key) === df2(key)).select(key) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6910] [SQL] [WIP] Support for pushing p...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6921#issuecomment-115997708 [Test build #35893 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35893/console) for PR 6921 at commit [`afeb1ba`](https://github.com/apache/spark/commit/afeb1ba4540f2bf9dadf482e8f5a341b2b571b70). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6910] [SQL] [WIP] Support for pushing p...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6921#issuecomment-115997709 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8560][UI] The Executors page will have ...
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/6950#issuecomment-115999390 @JoshRosen is a better person to check this out. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8657] [YARN] Simplify method addResourc...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7053#issuecomment-115975997 [Test build #967 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/967/console) for PR 7053 at commit [`75b457e`](https://github.com/apache/spark/commit/75b457ecfebd379221f26f60277c0346081017e5). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class Sha2(left: Expression, right: Expression)` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8657] [YARN] Simplify method addResourc...
Github user litao-buptsse commented on the pull request: https://github.com/apache/spark/pull/7053#issuecomment-115977907 Some unit tests failed. I will review my PR code again. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-6548 Adding stddev to DataFrame function...
Github user JihongMA commented on the pull request: https://github.com/apache/spark/pull/6297#issuecomment-115977926 while preparing the code change to address review comments. I noticed the fix for SPARK-8359 is causing issue with decimal type, I put a comment there on that JIRA and hoping them to fix it as one of the test case I added for testing decimal type is failing due to the fix. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8649] [BUILD] Mapr repository is not de...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7054#issuecomment-115984108 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8649] [BUILD] Mapr repository is not de...
GitHub user tszym opened a pull request: https://github.com/apache/spark/pull/7054 [SPARK-8649] [BUILD] Mapr repository is not defined properly The previous commiter on this part was @pwendell The previous url gives 404, the new one seems to be OK. This patch is added under the Apache License 2.0. The JIRA link: https://issues.apache.org/jira/browse/SPARK-8649 You can merge this pull request into a Git repository by running: $ git pull https://github.com/tszym/spark SPARK-8649 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/7054.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #7054 commit bfda9c44407ebf24954d04eacb1b24b278123e06 Author: Thomas Szymanski deve...@tszymanski.com Date: 2015-06-27T08:11:22Z [SPARK-8649] [BUILD] Mapr repository is not defined properly --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6910] [SQL] [WIP] Support for pushing p...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6921#issuecomment-115997080 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7944][SPARK-8013] Remove most of the Sp...
Github user dragos commented on the pull request: https://github.com/apache/spark/pull/6903#issuecomment-115999679 @ScrapCodes @mateiz Anything else I can do to get this in? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6380] Resolution of equi-join key in po...
Github user bryanjj closed the pull request at: https://github.com/apache/spark/pull/6141 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6380] Resolution of equi-join key in po...
Github user bryanjj closed the pull request at: https://github.com/apache/spark/pull/6141 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8657] [YARN] Fail to upload conf archiv...
Github user litao-buptsse commented on the pull request: https://github.com/apache/spark/pull/7042#issuecomment-115978366 Scala style tests failed. I will view my code again. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3629] [YARN] [DOCS]: Improvement of the...
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/6924#issuecomment-115971724 I've eyeballed the change again and looks like it's consistent with everyone's intentions. I'll merge after checking that docs build after this doc-only change. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3629] [YARN] [DOCS]: Improvement of the...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/6924 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8657] [YARN] Fail to upload conf archiv...
Github user litao-buptsse commented on the pull request: https://github.com/apache/spark/pull/7042#issuecomment-115979188 OK. I will try to make a version same as master #7053 for branch-1.4 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8669] [SQL] Fix crash with BINARY (ENUM...
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/7048#issuecomment-115984726 Thanks for fixing this! `ENUM` is not specified in the Parquet format spec, but according to parquet-mr 1.7.0, it is only used for converting Avro, ProtoBuf, and Thrift files. Double checked that in all cases `ENUM` is mapped to UTF8 string. So I think it's OK to always map `ENUM` to `StringType` regardless the value of `assumeBinaryIsString`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6910] [SQL] [WIP] Support for pushing p...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6921#issuecomment-115997423 [Test build #35893 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35893/consoleFull) for PR 6921 at commit [`afeb1ba`](https://github.com/apache/spark/commit/afeb1ba4540f2bf9dadf482e8f5a341b2b571b70). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6910] [SQL] [WIP] Support for pushing p...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6921#issuecomment-115997110 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6263][MLLIB] Python MLlib API missing i...
Github user Lewuathe commented on the pull request: https://github.com/apache/spark/pull/5707#issuecomment-115997683 @jkbradley I'm so sorry for keeping bothering you, but could you check it again? Thank you. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8677][SQL] Fix non-terminating decimal ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7056#issuecomment-116005594 [Test build #35894 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35894/consoleFull) for PR 7056 at commit [`34d7419`](https://github.com/apache/spark/commit/34d741951d7e878107dc948ce4d8b7a6d1e7e661). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8677][SQL] Fix non-terminating decimal ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7056#issuecomment-116040685 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8677][SQL] Fix non-terminating decimal ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7056#issuecomment-116040653 [Test build #35894 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35894/console) for PR 7056 at commit [`34d7419`](https://github.com/apache/spark/commit/34d741951d7e878107dc948ce4d8b7a6d1e7e661). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8657] [YARN] Simplify method addResourc...
Github user litao-buptsse commented on the pull request: https://github.com/apache/spark/pull/7053#issuecomment-116045509 @srowen I think maybe the first fs param in addResource() is needed, especially it's useful when doing unit test. I see the ClientDistributedCacheManagerSuite usually pass a mock fs to the addResource(). So I think to keep the first fs param might be better, although it's sometimes redundant. ClientDistributedCacheManagerSuite unit tests: https://github.com/apache/spark/blob/master/yarn/src/test/scala/org/apache/spark/deploy/yarn/ClientDistributedCacheManagerSuite.scala My failed unit tests: https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/967/testReport/org.apache.spark.deploy.yarn/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5205][Streaming]:Inconsistent behaviour...
Github user uncleGen closed the pull request at: https://github.com/apache/spark/pull/4135 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8657] [YARN] Fail to upload conf archiv...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7055#issuecomment-116003341 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8657] [YARN] Fail to upload conf archiv...
Github user litao-buptsse closed the pull request at: https://github.com/apache/spark/pull/7042 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8677][SQL] Fix non-terminating decimal ...
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/7056 [SPARK-8677][SQL] Fix non-terminating decimal expansion for decimal divide operation JIRA: https://issues.apache.org/jira/browse/SPARK-8677 You can merge this pull request into a Git repository by running: $ git pull https://github.com/viirya/spark-1 fix_decimal3 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/7056.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #7056 commit 34d741951d7e878107dc948ce4d8b7a6d1e7e661 Author: Liang-Chi Hsieh vii...@gmail.com Date: 2015-06-27T10:43:22Z Fix Non-terminating decimal expansion for decimal divide operation. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8657] [YARN] Fail to upload conf archiv...
GitHub user litao-buptsse opened a pull request: https://github.com/apache/spark/pull/7055 [SPARK-8657] [YARN] Fail to upload conf archive to viewfs Fail to upload conf archive to viewfs in spark-1.4 JIRA Link: https://issues.apache.org/jira/browse/SPARK-8657 You can merge this pull request into a Git repository by running: $ git pull https://github.com/litao-buptsse/spark SPARK-8657-for-branch-1.4 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/7055.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #7055 commit cbae84e72cc8c1949e96b0d17c7ff38ca6da7281 Author: Tao Li li...@sogou-inc.com Date: 2015-06-27T10:36:12Z [SPARK-8657] [YARN] Fail to upload conf archive to viewfs. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8677][SQL] Fix non-terminating decimal ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7056#issuecomment-116004905 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8677][SQL] Fix non-terminating decimal ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7056#issuecomment-116004882 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8657] [YARN] Fail to upload conf archiv...
Github user litao-buptsse commented on the pull request: https://github.com/apache/spark/pull/7042#issuecomment-116004053 I create a clean new PR #7055 (ajust scala code style) for this issue and I will close this PR. @srowen you can merge this PR to branch-1.4 temporaryly. In future, I will create another PR like #7053 to simplify and correct method addResource() for master and branch-1.4 at the same time.(For #7053, some units can't pass, I will try to fix it) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8679][SQL] remove dummy java class Base...
Github user davies commented on the pull request: https://github.com/apache/spark/pull/7059#issuecomment-116109310 @cloud-fan Thanks for looking into this, We have some offline discussion last week, because of the data source API, we have to make InternalRow can be casted into Row for performance, if we don't change the data source API much. So, we still need to make InternalRow as child of Row, so abort the #6869. I like the idea of removing BaseMutableRow, will merging these into #7003, if you don't mind. Just came back from retreat, will update that PR soon. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8610] [SQL] Separate Row and InternalRo...
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/7003#discussion_r33414717
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/InternalRow.scala ---
@@ -19,14 +19,54 @@ package org.apache.spark.sql.catalyst
import org.apache.spark.sql.Row
import org.apache.spark.sql.catalyst.expressions._
+import org.apache.spark.unsafe.types.UTF8String
/**
* An abstract class for row used internal in Spark SQL, which only
contain the columns as
* internal types.
*/
abstract class InternalRow extends Row {
+
+ // default implementation for codegen (for a Row which does not have
those types)
+ override def getBoolean(i: Int): Boolean = throw new
UnsupportedOperationException
+ override def getByte(i: Int): Byte = throw new
UnsupportedOperationException
+ override def getShort(i: Int): Short = throw new
UnsupportedOperationException
+ override def getInt(i: Int): Int = throw new
UnsupportedOperationException
+ override def getLong(i: Int): Long = throw new
UnsupportedOperationException
+ override def getFloat(i: Int): Float = throw new
UnsupportedOperationException
+ override def getDouble(i: Int): Double = throw new
UnsupportedOperationException
+
+ // This is only use for test
+ override def getString(i: Int): String = getAs[UTF8String](i).toString
+
+ // These expensive API should not be used internally.
+ override def getDecimal(i: Int): java.math.BigDecimal = throw new
UnsupportedOperationException
+ override def getDate(i: Int): java.sql.Date = throw new
UnsupportedOperationException
+ override def getTimestamp(i: Int): java.sql.Timestamp = throw new
UnsupportedOperationException
+ override def getSeq[T](i: Int): Seq[T] = throw new
UnsupportedOperationException
+ override def getList[T](i: Int): java.util.List[T] = throw new
UnsupportedOperationException
+ override def getMap[K, V](i: Int): scala.collection.Map[K, V] =
+throw new UnsupportedOperationException
+ override def getJavaMap[K, V](i: Int): java.util.Map[K, V] =
+throw new UnsupportedOperationException
+ override def getStruct(i: Int): InternalRow = throw new
UnsupportedOperationException
+ override def getAs[T](fieldName: String): T = throw new
UnsupportedOperationException
+ override def getValuesMap[T](fieldNames: Seq[String]): Map[String, T] =
--- End diff --
will do
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8679] [PySpark] [MLlib] Default values ...
GitHub user MechCoder opened a pull request:
https://github.com/apache/spark/pull/7058
[SPARK-8679] [PySpark] [MLlib] Default values in Pipeline API should be
immutable
It might be dangerous to have a mutable as value for default param.
e.g
def func(example, f={}):
f[example] = 1
return f
func(2)
{2: 1}
func(3)
{2:1, 3:1}
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/MechCoder/spark pipeline_api_playground
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/7058.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #7058
commit 95f7ff294ece95affe4f59efe7d7727120121a41
Author: MechCoder manojkumarsivaraj...@gmail.com
Date: 2015-06-27T16:41:47Z
[SPARK-8679] [PySpark] [MLlib] Default values in Pipeline API should be
immutable
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8679][SQL] remove dummy java class Base...
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/7059 [SPARK-8679][SQL] remove dummy java class BaseRow and BaseMutableRow we can use `abstract class` instead of `trait` for `MutableRow`, and remove the dummy java row classes. This PR also contains some small cleanup. I keep primitive type order as boolean, byte, short, int, long, float, double, is this conventional? You can merge this pull request into a Git repository by running: $ git pull https://github.com/cloud-fan/spark step1 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/7059.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #7059 commit f7b530898cd9d8f6a8d2a2bcd905eb273f7de3e4 Author: Wenchen Fan cloud0...@outlook.com Date: 2015-06-27T16:44:14Z remove dummy java class --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8679][SQL] remove dummy java class Base...
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/7059#issuecomment-116101231 cc @rxin @davies I can take over the following internal/external row splitting job if @davies is too busy recently. Some thoughts about future plans: * create `GenericInternalRow` and apply it to the right place. Based on https://github.com/apache/spark/pull/7003 * make `Row` and `InternalRow` siblings instead of parent and child.(may break this into several parts after digging into) * remove unnecessary functions from `InternalRow`. * improve the access control of internal row related stuff, so that we can hide them outside spark sql module, and only expose `Row`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8510] [CORE] [PYSPARK] NumPy arrays and...
Github user paberline commented on the pull request: https://github.com/apache/spark/pull/6995#issuecomment-116092604 I want to work on this some more to incorporate the improvement suggestion I have received. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8638] [SQL] Window Function Performance...
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/7057 [SPARK-8638] [SQL] Window Function Performance Improvements ## Description Performance improvements for Spark Window functions. This PR will also serve as the basis for moving away from Hive UDAFs to Spark UDAFs. See JIRA tickets SPARK-8638 and SPARK-7712 for more information. The original work including some benchmarking code for the running case can be here: https://github.com/hvanhovell/spark-window ## Improvements * Much better performance (10x) in running cases (e.g. BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) and UNBOUDED FOLLOWING cases. The current implementation in spark uses a sliding window approach in these cases. This means that an aggregate is maintained for every row, so space usage is N (N being the number of rows). This also means that all these aggregates all need to be updated separately, this takes N*(N-1)/2 updates. The running case differs from the Sliding case because we are only adding data to an aggregate function (no reset is required), we only need to maintain one aggregate (like in the UNBOUNDED PRECEDING AND UNBOUNDED case), update the aggregate for each row, and get the aggregate value after each update. This is what the new implementation does. This approach only uses 1 buffer, and only requires N updates; I am currently working on data with window sizes of 500-1000 doing running sums and this saves a lot of time. The CURRENT ROW AND UNBOUNDED FOLLOWING case als o uses this approach and the fact that aggregate operations are communitative, there is one twist though it will process the input buffer in reverse. * Fewer comparisons in the sliding case. The current implementation determines frame boundaries for every input row. The new implementation makes more use of the fact that the window is sorted, maintains the boundaries, and only moves them when the current row order changes. This is a minor improvement. * A single Window node is able to process all types of Frames for the same Partitioning/Ordering. This saves a little time/memory spent buffering and managing partitions. This will be enabled in a follow-up PR. * A lot of the staging code is moved from the execution phase to the initialization phase. Minor performance improvement, and improves readability of the execution code. You can merge this pull request into a Git repository by running: $ git pull https://github.com/hvanhovell/spark SPARK-8638 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/7057.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #7057 commit f161920218c880e4f2804b19db129936d0d34d61 Author: Herman van Hovell hvanhov...@questtec.nl Date: 2015-06-27T15:39:53Z Major overhaul of Window operator. commit 22e51d39c833d0e69a788ad41b8bd790688b9bd9 Author: Herman van Hovell hvanhov...@questtec.nl Date: 2015-06-27T15:54:00Z Fixed aggregation and range betweens with unbounded test - two different window frames were compared. commit ad7820c6ac04f188e8a6239d314b680c8a6b4551 Author: Herman van Hovell hvanhov...@questtec.nl Date: 2015-06-27T16:00:23Z Disabled Tests 42 43 because tiny numerical differences in answers. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8510] [CORE] [PYSPARK] NumPy arrays and...
Github user paberline closed the pull request at: https://github.com/apache/spark/pull/6995 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8591][CORE]Block failed to unroll to me...
Github user lianhuiwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/6990#discussion_r33413900
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala
---
@@ -833,8 +833,10 @@ private[spark] class BlockManager(
logDebug(Put block %s locally took %s.format(blockId,
Utils.getUsedTimeMs(startTimeMs)))
// Either we're storing bytes and we asynchronously started
replication, or we're storing
-// values and need to serialize and replicate them now:
-if (putLevel.replication 1) {
+// values and need to serialize and replicate them now.
+// Should not replicate the block if its StorageLevel is
StorageLevel.NONE or
+// putting it to local is failed.
+if (!putBlockInfo.isFailed putLevel.replication 1) {
--- End diff --
i think @squito said is correct. now the problem is when block is failed in
local and replication factor needs 2, but now how we choose to deal with this
situation, there are three ways:
one is making replication factor to 1 and continue to replication to remote
node.
two is stop to replication to remote node and its replication factor is 0.
three is continuing to replication to two remote node and its replication
factor is 2.
so i think the three maybe a good solution. how about your opinions?
@andrewor14 @squito
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8638] [SQL] Window Function Performance...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7057#issuecomment-116114231 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8679][SQL] remove dummy java class Base...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7059#issuecomment-116114228 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8679] [PySpark] [MLlib] Default values ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7058#issuecomment-116114229 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8669] [SQL] Fix crash with BINARY (ENUM...
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/7048#discussion_r33414842
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/parquet/CatalystSchemaConverter.scala
---
@@ -178,8 +178,8 @@ private[parquet] class CatalystSchemaConverter(
case BINARY =
field.getOriginalType match {
case UTF8 = StringType
- case null if assumeBinaryIsString = StringType
- case null = BinaryType
+ case null | ENUM if assumeBinaryIsString = StringType
+ case null | ENUM = BinaryType
--- End diff --
Let's revert changes to these two lines and change the first `case` branch
to:
```scala
case UTF8 | ENUM = StringType
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8679][SQL] remove dummy java class Base...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7059#issuecomment-116114520 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8679] [PySpark] [MLlib] Default values ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7058#issuecomment-116114519 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8679] [PySpark] [MLlib] Default values ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7058#issuecomment-116114714 [Test build #35896 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35896/consoleFull) for PR 7058 at commit [`95f7ff2`](https://github.com/apache/spark/commit/95f7ff294ece95affe4f59efe7d7727120121a41). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8679][SQL] remove dummy java class Base...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7059#issuecomment-116115057 [Test build #35895 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35895/consoleFull) for PR 7059 at commit [`f7b5308`](https://github.com/apache/spark/commit/f7b530898cd9d8f6a8d2a2bcd905eb273f7de3e4). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8638] [SQL] Window Function Performance...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7057#issuecomment-116115100 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8638] [SQL] Window Function Performance...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7057#issuecomment-116115023 Jenkins, this is ok to test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8638] [SQL] Window Function Performance...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7057#issuecomment-116115098 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8638] [SQL] Window Function Performance...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7057#issuecomment-116116301 [Test build #35897 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35897/consoleFull) for PR 7057 at commit [`ad7820c`](https://github.com/apache/spark/commit/ad7820c6ac04f188e8a6239d314b680c8a6b4551). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8679] [PySpark] [MLlib] Default values ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7058#issuecomment-116121629 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8679] [PySpark] [MLlib] Default values ...
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7058#issuecomment-116121669 jenkins test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8679] [PySpark] [MLlib] Default values ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7058#issuecomment-116121626 [Test build #35896 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35896/console) for PR 7058 at commit [`95f7ff2`](https://github.com/apache/spark/commit/95f7ff294ece95affe4f59efe7d7727120121a41). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8679] [PySpark] [MLlib] Default values ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7058#issuecomment-116121945 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8679] [PySpark] [MLlib] Default values ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7058#issuecomment-116121931 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8679] [PySpark] [MLlib] Default values ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7058#issuecomment-116122319 [Test build #35898 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35898/consoleFull) for PR 7058 at commit [`95f7ff2`](https://github.com/apache/spark/commit/95f7ff294ece95affe4f59efe7d7727120121a41). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8681] fixed wrong ordering of columns i...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7060#issuecomment-116138794 [Test build #35899 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35899/consoleFull) for PR 7060 at commit [`d96da7e`](https://github.com/apache/spark/commit/d96da7e955574b12587e10d69d6008115ebc6352). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-746][CORE][WIP] Added Avro Serializatio...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7004#issuecomment-116173510 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-746][CORE][WIP] Added Avro Serializatio...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7004#issuecomment-116173506 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-746][CORE][WIP] Added Avro Serializatio...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7004#issuecomment-116173172 Jenkins, this is ok to test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7845] [BUILD] Bumping default Hadoop ve...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7062#issuecomment-116181658 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7845] [BUILD] Bumping default Hadoop ve...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7062#issuecomment-116181663 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8528] Expose SparkContext.applicationId...
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/6936#discussion_r33416662 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -315,6 +315,14 @@ class SparkContext(config: SparkConf) extends Logging with ExecutorAllocationCli _dagScheduler = ds } + /** + * A unique identifier for the Spark application. + * Its format depends on the scheduler implementation. + * (i.e. + * in case of local spark app something like u'local-1433865536131' --- End diff -- Minor nit: the `u'` is going to be confusing to Scala users. I'd just go with regular quoted strings here. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8048] Partitionning of an RDD with 0 pa...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7020#issuecomment-116147553 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8048] Partitionning of an RDD with 0 pa...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7020#issuecomment-116147548 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8048] Partitionning of an RDD with 0 pa...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7020#issuecomment-116147483 Jenkins, this is ok to test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8610] [SQL] Separate Row and InternalRo...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7003#issuecomment-116148896 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8610] [SQL] Separate Row and InternalRo...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7003#issuecomment-116148941 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8066, SPARK-8067] [hive] Add support fo...
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/7026#discussion_r33416949
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/client/VersionsSuite.scala ---
@@ -170,5 +180,12 @@ class VersionsSuite extends SparkFunSuite with Logging
{
false,
false)
}
+
+test(s$version: create index and reset) {
+ client.runSqlHive(CREATE TABLE indexed_table (key INT))
+ client.runSqlHive(CREATE INDEX index_1 ON TABLE indexed_table(key)
+
+as 'COMPACT' WITH DEFERRED REBUILD)
+ client.reset()
+}
--- End diff --
I am always wondering if index actually works in Hive. Anyone really uses
it?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8583] [SPARK-5482] [BUILD] Refactor pyt...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6967#issuecomment-116164101 [Test build #35904 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35904/consoleFull) for PR 6967 at commit [`f578d6d`](https://github.com/apache/spark/commit/f578d6da50cd09a3ad13dbe6dcc6b22870f227e5). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8528] Expose SparkContext.applicationId...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6936#issuecomment-116164305 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8528] Expose SparkContext.applicationId...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6936#issuecomment-116164301 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8528] Expose SparkContext.applicationId...
Github user smartkiwi commented on the pull request: https://github.com/apache/spark/pull/6936#issuecomment-116164420 Python doctests are rather limited in expressiveness, but efficient in keeping examples bug free in documentation. Anywhere - here is another patch that works in both py2.6/2.7 and py3.4 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8528] Expose SparkContext.applicationId...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6936#issuecomment-116164424 [Test build #35905 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35905/consoleFull) for PR 6936 at commit [`870338b`](https://github.com/apache/spark/commit/870338b04922e253c43b88662086d5b8ed3a2c65). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-746][CORE][WIP] Added Avro Serializatio...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7004#issuecomment-116164570 Also, to clarify: is this primarily intended to improve the performance of programs written against the Spark Core API? For Spark SQL + DataFrames, I think the [spark-avro](https://github.com/databricks/spark-avro) library will convert the Avro records into Spark SQL's internal Row representation, which should be more efficient to serialize and shuffle. I'd be curious to know whether you could see most of these benefits for simpler workflows by using Dataframes and leaving the serialization up to that. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8649] [BUILD] Mapr repository is not de...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7054#issuecomment-116164685 Jenkins, this is ok to test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8649] [BUILD] Mapr repository is not de...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7054#issuecomment-116164794 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8649] [BUILD] Mapr repository is not de...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7054#issuecomment-116164800 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8649] [BUILD] Mapr repository is not de...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7054#issuecomment-116164855 [Test build #35906 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35906/consoleFull) for PR 7054 at commit [`bfda9c4`](https://github.com/apache/spark/commit/bfda9c44407ebf24954d04eacb1b24b278123e06). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-746][CORE][WIP] Added Avro Serializatio...
Github user JDrit commented on the pull request: https://github.com/apache/spark/pull/7004#issuecomment-116165233 This was intended to be a performance increase for spark-core with RDDs. This works separately from spark-avro. I realize that dataframes could also be used but the goal was to make it easier for users who just want to use RDDs. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8683] [SPARK] Depend on mockito-core in...
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/7061 [SPARK-8683] [SPARK] Depend on mockito-core instead of mockito-all Spark's tests currently depend on `mockito-all`, which bundles Hamcrest and Objenesis classes. Instead, it should depend on `mockito-core`, which declares those libraries as Maven dependencies. This is necessary in order to fix a dependency conflict that leads to a NoSuchMethodError when using certain Hamcrest matchers. See https://github.com/mockito/mockito/wiki/Declaring-mockito-dependency for more details. You can merge this pull request into a Git repository by running: $ git pull https://github.com/JoshRosen/spark mockito-core-instead-of-all Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/7061.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #7061 commit 70eccbef850d2b3a142c6446b922a6eaabba49dd Author: Josh Rosen joshro...@databricks.com Date: 2015-06-27T23:14:09Z Depend on mockito-core instead of mockito-all. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8683] [BUILD] Depend on mockito-core in...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7061#issuecomment-116165349 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8683] [BUILD] Depend on mockito-core in...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7061#issuecomment-116165354 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8683] [BUILD] Depend on mockito-core in...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7061#issuecomment-116165427 [Test build #35907 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35907/consoleFull) for PR 7061 at commit [`70eccbe`](https://github.com/apache/spark/commit/70eccbef850d2b3a142c6446b922a6eaabba49dd). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8610] [SQL] Separate Row and InternalRo...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7003#issuecomment-116166766 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8610] [SQL] Separate Row and InternalRo...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7003#issuecomment-116166759 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8610] [SQL] Separate Row and InternalRo...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7003#issuecomment-116166791 [Test build #35908 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35908/consoleFull) for PR 7003 at commit [`87b13cf`](https://github.com/apache/spark/commit/87b13cffdf8e3abb07aed85356d23e2e3ab974ce). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8679][SQL] remove dummy java class Base...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7059#issuecomment-116132934 [Test build #35895 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35895/console) for PR 7059 at commit [`f7b5308`](https://github.com/apache/spark/commit/f7b530898cd9d8f6a8d2a2bcd905eb273f7de3e4). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8679][SQL] remove dummy java class Base...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7059#issuecomment-116132999 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8679] [PySpark] [MLlib] Default values ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7058#issuecomment-116132214 [Test build #35898 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35898/console) for PR 7058 at commit [`95f7ff2`](https://github.com/apache/spark/commit/95f7ff294ece95affe4f59efe7d7727120121a41). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8681] fixed wrong ordering of columns i...
GitHub user brkyvz opened a pull request: https://github.com/apache/spark/pull/7060 [SPARK-8681] fixed wrong ordering of columns in crosstab I specifically randomized the test. What crosstab does is equivalent to a countByKey, therefore if this test fails again for any reason, we will know that we hit a corner case or something. cc @rxin @marmbrus You can merge this pull request into a Git repository by running: $ git pull https://github.com/brkyvz/spark crosstab-fixes Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/7060.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #7060 commit d96da7e955574b12587e10d69d6008115ebc6352 Author: Burak Yavuz brk...@gmail.com Date: 2015-06-27T20:35:02Z fixed wrong ordering of columns in crosstab --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-8623. Hadoop RDDs fail to properly seria...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7050#issuecomment-116146384 Saw confirmation on JIRA that this patch fixes the issue, so I'm going to merge to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8610] [SQL] Separate Row and InternalRo...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7003#issuecomment-116149511 [Test build #35901 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35901/console) for PR 7003 at commit [`bd4e99c`](https://github.com/apache/spark/commit/bd4e99ce79414d47b868a1ccc5ac9f47da4bb569). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
