[GitHub] spark pull request: [SPARK-4145] Web UI job pages
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3009#issuecomment-63935148 [Test build #23714 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23714/consoleFull) for PR 3009 at commit [`b89c258`](https://github.com/apache/spark/commit/b89c2587efb52ab3f5d8fc1a60fbd5a4f9c07510). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4526][MLLIB]GradientDescent get a wrong...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3399#issuecomment-63935157 [Test build #23713 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23713/consoleFull) for PR 3399 at commit [`42f113f`](https://github.com/apache/spark/commit/42f113fa8d73ecc09e3c18b7cb400c25584a2176). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4487][SQL] Fix attribute reference reso...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3363#issuecomment-63934907 [Test build #23707 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23707/consoleFull) for PR 3363 at commit [`fd314f3`](https://github.com/apache/spark/commit/fd314f3ecedb4e2cdc987a663afe63c6f0b9a181). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class RandomForestModel(JavaModelWrapper):` * `class RandomForest(object):` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4487][SQL] Fix attribute reference reso...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3363#issuecomment-63934914 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23707/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4526][MLLIB]GradientDescent get a wrong...
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/3399#issuecomment-63934659 AmplabJenkins retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4526][MLLIB]GradientDescent get a wrong...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3399#issuecomment-63934535 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23711/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4145] Web UI job pages
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3009#issuecomment-63934214 Argh, not again! That's what I get for playing whackamole with individual test suites without running all of them... I've spotted the cause behind this latest test failure and I'm fixing it now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4145] Web UI job pages
Github user kayousterhout commented on the pull request: https://github.com/apache/spark/pull/3009#issuecomment-63934171 This skipped thing looks great -- I withdraw my -0.5 (which I didn't realize meant this couldn't get merged into 1.2...didn't realize code voting was different than release voting) and am fine to merge this in! Did not do another detailed look at this code since it seems like Andrew had a close look. Thanks for all of the hard work on this Josh! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-4532: Fix bug in detection of Hive in Sp...
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/3398#issuecomment-63934015 @zzcclp You also need to add `-Phive`. `-Phive` implies `-Phive-0.13.1`, but not vice versa. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4526][MLLIB]GradientDescent get a wrong...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3399#issuecomment-63933817 [Test build #23712 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23712/consoleFull) for PR 3399 at commit [`42f113f`](https://github.com/apache/spark/commit/42f113fa8d73ecc09e3c18b7cb400c25584a2176). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4453][SPARK-4213][SQL] Additional test ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/#issuecomment-63933624 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23704/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4453][SPARK-4213][SQL] Additional test ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/#issuecomment-63933618 [Test build #23704 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23704/consoleFull) for PR at commit [`9016933`](https://github.com/apache/spark/commit/901693330b38156e4b14f850ec18b179c6ccbb31). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4526][MLLIB]GradientDescent get a wrong...
GitHub user witgo opened a pull request: https://github.com/apache/spark/pull/3399 [SPARK-4526][MLLIB]GradientDescent get a wrong gradient value according to the gradient formula. This is caused by the miniBatchSize parameter.The number of `RDD.sample` returns is not fixed. cc @mengxr You can merge this pull request into a Git repository by running: $ git pull https://github.com/witgo/spark GradientDescent Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/3399.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3399 commit 606b27a1a6c1e5a1e4c51d01d1f6da9f6ed31524 Author: GuoQiang Li Date: 2014-11-21T06:34:50Z GradientDescent get a wrong gradient value according to the gradient formula, which is caused by the miniBatchSize parameter. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4145] Web UI job pages
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3009#issuecomment-63933196 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23705/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4145] Web UI job pages
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3009#issuecomment-63933193 [Test build #23705 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23705/consoleFull) for PR 3009 at commit [`ff804cd`](https://github.com/apache/spark/commit/ff804cd3699218fccdd191743af7ff855a0235f1). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-4532: Fix bug in detection of Hive in Sp...
Github user zzcclp commented on the pull request: https://github.com/apache/spark/pull/3398#issuecomment-63932145 @pwendell, I use command as follows: mvn help:evaluate -Dexpression=project.activeProfiles -pl sql/hive -Phadoop-2.3 -Phive-0.13.1 -Phive-thriftserver -Pyarn -Dyarn.version=2.3.0-cdh5.1.2 -Dhadoop.version=2.3.0-cdh5.1.2 2>/dev/null | grep -v "INFO" | fgrep --count "hive" or mvn help:evaluate -Dexpression=project.activeProfiles -pl sql/hive -Phadoop-2.3 -Phive-0.13.1 -Phive-thriftserver -Pyarn -Dyarn.version=2.3.0-cdh5.1.2 -Dhadoop.version=2.3.0-cdh5.1.2 2>/dev/null | grep -v "INFO" | fgrep --count "hive-0.13.1" it still return 0, if use command as follows: mvn help:evaluate -Dexpression=project.activeProfiles -Phadoop-2.3 -Phive-0.13.1 -Phive-thriftserver -Pyarn -Dyarn.version=2.3.0-cdh5.1.2 -Dhadoop.version=2.3.0-cdh5.1.2 2>/dev/null | grep -v "INFO" | fgrep --count "hive-0.13.1" it return 1, is something wrong with it? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-4532: Fix bug in detection of Hive in Sp...
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/3398#issuecomment-63931788 Thanks @liancheng --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-4532: Fix bug in detection of Hive in Sp...
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/3398#issuecomment-63931758 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4531] [MLlib] cache serialized java obj...
Github user davies commented on the pull request: https://github.com/apache/spark/pull/3397#issuecomment-63931688 How about we call .cache() at the begging of iterations? Right now, we show a warning. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-4532: Fix bug in detection of Hive in Sp...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3398#issuecomment-63931573 [Test build #23710 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23710/consoleFull) for PR 3398 at commit [`8a58279`](https://github.com/apache/spark/commit/8a582797763e6836477df6da349074c05b395a76). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4483][SQL]Optimization about reduce mem...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3375#issuecomment-63931410 [Test build #23709 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23709/consoleFull) for PR 3375 at commit [`a676de6`](https://github.com/apache/spark/commit/a676de66396123e55720b9d537374ec038ce7237). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4483][SQL]Optimization about reduce mem...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3375#issuecomment-63931414 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23709/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4439] [MLlib] add python api for random...
Github user davies closed the pull request at: https://github.com/apache/spark/pull/3320 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4477] [PySpark] remove numpy from RDDSa...
Github user davies closed the pull request at: https://github.com/apache/spark/pull/3351 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Fix bug in detection of Hive in Spark 1.2
GitHub user pwendell opened a pull request: https://github.com/apache/spark/pull/3398 Fix bug in detection of Hive in Spark 1.2 Because the Hive profile is no longer defined in the root pom, we need to check specifically in the sql/hive pom when we perform the check in make-distribtion.sh. You can merge this pull request into a Git repository by running: $ git pull https://github.com/pwendell/spark make-distribution Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/3398.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3398 commit 8a582797763e6836477df6da349074c05b395a76 Author: Patrick Wendell Date: 2014-11-21T06:53:04Z Fix bug in detection of Hive in Spark 1.2 Because the Hive profile is no longer defined in the root pom, we need to check specifically in the sql/hive pom when we perform the check in make-distribtion.sh. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: add Sphinx as a dependency of building docs
Github user davies closed the pull request at: https://github.com/apache/spark/pull/3388 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4483][SQL]Optimization about reduce mem...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3375#issuecomment-63931128 [Test build #23709 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23709/consoleFull) for PR 3375 at commit [`a676de6`](https://github.com/apache/spark/commit/a676de66396123e55720b9d537374ec038ce7237). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4531] [MLlib] cache serialized java obj...
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/3397#issuecomment-63931044 It might be good to cache for decision tree too since it makes a couple of passes through the original RDD (before it creates the TreePoint RDD). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4531] [MLlib] cache serialized java obj...
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/3397#issuecomment-63931012 @davies Could we cache with MEMORY_AND_DISK? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4529] [SQL] support view with column al...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3396#issuecomment-63930866 [Test build #23703 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23703/consoleFull) for PR 3396 at commit [`4d001d0`](https://github.com/apache/spark/commit/4d001d0c99fe3e2b5399236f30e6b4994f5dc0ad). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4531] [MLlib] cache serialized java obj...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3397#issuecomment-63930850 [Test build #23708 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23708/consoleFull) for PR 3397 at commit [`f1063e1`](https://github.com/apache/spark/commit/f1063e150f5e8a8ec4b654d708786d221295f96a). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4529] [SQL] support view with column al...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3396#issuecomment-63930868 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23703/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4531] [MLlib] cache serialized java obj...
GitHub user davies opened a pull request: https://github.com/apache/spark/pull/3397 [SPARK-4531] [MLlib] cache serialized java object The Pyrolite is pretty slow (comparing to the adhoc serializer in 1.1), it cause much performance regression in 1.2, because we cache the serialized Python object in JVM, deserialize them into Java object in each step. This PR change to cache the deserialized JavaRDD instead of PythonRDD to avoid the deserialization of Pyrolite. It should have similar memory usage as before, but much faster. You can merge this pull request into a Git repository by running: $ git pull https://github.com/davies/spark cache Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/3397.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3397 commit f1063e150f5e8a8ec4b654d708786d221295f96a Author: Davies Liu Date: 2014-11-21T06:41:54Z cache serialized java object --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4487][SQL] Fix attribute reference reso...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3363#issuecomment-63930368 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23706/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4487][SQL] Fix attribute reference reso...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3363#issuecomment-63929946 [Test build #23707 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23707/consoleFull) for PR 3363 at commit [`fd314f3`](https://github.com/apache/spark/commit/fd314f3ecedb4e2cdc987a663afe63c6f0b9a181). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4487][SQL] Fix attribute reference reso...
Github user sarutak commented on the pull request: https://github.com/apache/spark/pull/3363#issuecomment-63929467 @chenghao-intel , @marmbrus Thanks for your comment! Now I've just fixed that. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4453][SPARK-4213][SQL] Additional test ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/#issuecomment-63929039 [Test build #23704 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23704/consoleFull) for PR at commit [`9016933`](https://github.com/apache/spark/commit/901693330b38156e4b14f850ec18b179c6ccbb31). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4145] Web UI job pages
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3009#issuecomment-63929025 [Test build #23705 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23705/consoleFull) for PR 3009 at commit [`ff804cd`](https://github.com/apache/spark/commit/ff804cd3699218fccdd191743af7ff855a0235f1). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4145] Web UI job pages
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3009#issuecomment-63928641 Ah, spotted the problem: I forgot to remove the line that wrote the `Stage Ids` JSON field, so this was mistakenly causing the read path to treat data written in the new format as though it was written using the old one. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4145] Web UI job pages
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/3009#issuecomment-63928198
@pwendell Yep, it looks like a legitimate failure in ReplayListenerSuite:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23698/testReport/
I'm digging in now to understand the error message. It looks like it's
failing this assertion:
```
val originalEvents = sc.eventLogger.get.loggedEvents
val replayedEvents = eventMonster.loggedEvents
originalEvents.zip(replayedEvents).foreach { case (e1, e2) => assert(e1
=== e2) }
```
I wonder if this is due to that `StageInfo.equals()` issue that I mentioned
earlier.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4145] Web UI job pages
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/3009#issuecomment-63927868 @JoshRosen I believe this is failing tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4529] [SQL] support view with column al...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3396#issuecomment-63926798 [Test build #23703 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23703/consoleFull) for PR 3396 at commit [`4d001d0`](https://github.com/apache/spark/commit/4d001d0c99fe3e2b5399236f30e6b4994f5dc0ad). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4529] [SQL] support view with column al...
GitHub user adrian-wang opened a pull request:
https://github.com/apache/spark/pull/3396
[SPARK-4529] [SQL] support view with column alias
Support view definition like
CREATE VIEW view3(valoo)
TBLPROPERTIES ("fear" = "factor")
AS SELECT upper(value) FROM src WHERE key=86;
[valoo as the alias of upper(value)]. This is missing part of SPARK-4239,
for a fully view support.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/adrian-wang/spark viewcolumn
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/3396.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3396
commit 4d001d0c99fe3e2b5399236f30e6b4994f5dc0ad
Author: Daoyuan Wang
Date: 2014-11-21T05:30:29Z
support view with column alias
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4518][SPARK-4519][Streaming] Refactored...
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/3389#issuecomment-63926244 I found potential bugs and corner cases regarding the `ignoreThreshold` . If there is a batch with no files, then the `minModTime` will be stored as -1, and the `ignoreThreshold` will get calculated as -1, thus allowing all files to be accepted. Fixing and testing this. Will update when I am convinced that this is resolved. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4522][SQL] Parse schema with missing me...
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/3392#issuecomment-63925889 LGTM. It seems that this was already merged:) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3325][Streaming] Add a parameter to the...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3237#issuecomment-63922623 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23702/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3325][Streaming] Add a parameter to the...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3237#issuecomment-63922618 [Test build #23702 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23702/consoleFull) for PR 3237 at commit [`ec8a3af`](https://github.com/apache/spark/commit/ec8a3af8242fddf8d861a23e081cb861c3d6a092). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4487][SQL] Fix attribute reference reso...
Github user sarutak commented on a diff in the pull request: https://github.com/apache/spark/pull/3363#discussion_r20697518 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -181,7 +181,7 @@ class Analyzer(catalog: Catalog, registry: FunctionRegistry, caseSensitive: Bool // Add missing attributes and then project them away after the sort. Project(projectList, --- End diff -- I see. I'll modify that. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3325][Streaming] Add a parameter to the...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3237#issuecomment-63920844 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23701/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3325][Streaming] Add a parameter to the...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3237#issuecomment-63920843 [Test build #23701 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23701/consoleFull) for PR 3237 at commit [`26a70c0`](https://github.com/apache/spark/commit/26a70c0958a4d7223a3a4c2e098de5b0d6c0f1ea). * This patch **fails MiMa tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [WIP] [SPARK-4517] Improve memory efficiency o...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3394#issuecomment-63919624 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23700/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [WIP] [SPARK-4517] Improve memory efficiency o...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3394#issuecomment-63919619 [Test build #23700 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23700/consoleFull) for PR 3394 at commit [`e8aa918`](https://github.com/apache/spark/commit/e8aa918201349db045f2e8e8a09fb12b47c4e13c). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: add Sphinx as a dependency of building docs
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/3388#issuecomment-63918860 Thanks davies - I pulled this in --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4527][SQl]Add BroadcastNestedLoopJoin o...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3395#issuecomment-63918766 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4527][SQl]Add BroadcastNestedLoopJoin o...
GitHub user wangxiaojing opened a pull request: https://github.com/apache/spark/pull/3395 [SPARK-4527][SQl]Add BroadcastNestedLoopJoin operator selection testsuite In `JoinSuite` add BroadcastNestedLoopJoin operator selection testsuite You can merge this pull request into a Git repository by running: $ git pull https://github.com/wangxiaojing/spark SPARK-4527 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/3395.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3395 commit 53c39524703cec6e89886dd3b4d202fbb2141039 Author: wangxiaojing Date: 2014-11-21T03:03:38Z Add BroadcastNestedLoopJoin operator selection testsuite --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3325][Streaming] Add a parameter to the...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3237#issuecomment-63917639 [Test build #23702 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23702/consoleFull) for PR 3237 at commit [`ec8a3af`](https://github.com/apache/spark/commit/ec8a3af8242fddf8d861a23e081cb861c3d6a092). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3325][Streaming] Add a parameter to the...
Github user watermen commented on the pull request: https://github.com/apache/spark/pull/3237#issuecomment-63917535 @tdas I had moved to Spark 1.3. Many thanks to you for giving me so many information to help me. I'm new to contribute. @giwa thanks for the code snippet. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3325][Streaming] Add a parameter to the...
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/3237#issuecomment-63916003 And @giwa thanks for the code snippet. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3325][Streaming] Add a parameter to the...
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/3237#issuecomment-63915944 Aah, that is probably because the master branch has been marked for Spark 1.3, and so the filter needs to be moved to Spark 1.3. Could you try that. We cant make it to Spark 1.2 as of now, so this is fine. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Documentation: add description for repartition...
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/3390#issuecomment-63915538 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3325][Streaming] Add a parameter to the...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3237#issuecomment-63915216 [Test build #23701 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23701/consoleFull) for PR 3237 at commit [`26a70c0`](https://github.com/apache/spark/commit/26a70c0958a4d7223a3a4c2e098de5b0d6c0f1ea). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [WIP] [SPARK-4517] Improve memory efficiency o...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3394#issuecomment-63915227 [Test build #23700 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23700/consoleFull) for PR 3394 at commit [`e8aa918`](https://github.com/apache/spark/commit/e8aa918201349db045f2e8e8a09fb12b47c4e13c). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [WIP] [SPARK-4517] Improve memory efficiency o...
GitHub user davies opened a pull request: https://github.com/apache/spark/pull/3394 [WIP] [SPARK-4517] Improve memory efficiency of python broadcast TBD You can merge this pull request into a Git repository by running: $ git pull https://github.com/davies/spark by_pass_ser Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/3394.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3394 commit 5a35a5bc74cb4228e340cfec72d8668e93d2af76 Author: Davies Liu Date: 2014-11-21T01:42:36Z improve memory efficency of torrentbroadcast commit e8aa918201349db045f2e8e8a09fb12b47c4e13c Author: Davies Liu Date: 2014-11-21T01:56:12Z bugfix --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4485][SQL]Add BroadcastHashOuterJoin
Github user wangxiaojing commented on the pull request: https://github.com/apache/spark/pull/3362#issuecomment-63914461 @liancheng Add testsuite. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4145] Web UI job pages
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3009#issuecomment-63914127 [Test build #23698 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23698/consoleFull) for PR 3009 at commit [`6f17f3f`](https://github.com/apache/spark/commit/6f17f3f61102f5685d20cf42f79a049a5bbaad06). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4145] Web UI job pages
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3009#issuecomment-63914131 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23698/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4413][SQL] Parquet support through data...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3269#issuecomment-63914073 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23697/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4413][SQL] Parquet support through data...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3269#issuecomment-63914070 [Test build #23697 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23697/consoleFull) for PR 3269 at commit [`1dd75f1`](https://github.com/apache/spark/commit/1dd75f11208441bebf87c4315435587897685ab5). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class DefaultSource extends RelationProvider ` * `case class ParquetRelation2(path: String)(@transient val sqlContext: SQLContext)` * `abstract class CatalystScan extends BaseRelation ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4525] MesosSchedulerBackend.resourceOff...
Github user jongyoul commented on the pull request: https://github.com/apache/spark/pull/3393#issuecomment-63912860 @tnachen +1, Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4525] MesosSchedulerBackend.resourceOff...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3393#issuecomment-63912614 [Test build #23696 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23696/consoleFull) for PR 3393 at commit [`f20f1b3`](https://github.com/apache/spark/commit/f20f1b379f4405bbbe21315ffd8166827132fe64). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4525] MesosSchedulerBackend.resourceOff...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3393#issuecomment-63912616 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23696/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3891][SQL] Add array support to percent...
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/2802#issuecomment-63912580 @gvramana #3109 is merged, can you remove the unnecessary `TestHive.reset`, and see if that helps --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4485][SQL]Add BroadcastHashOuterJoin
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3362#issuecomment-63912271 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23699/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4485][SQL]Add BroadcastHashOuterJoin
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3362#issuecomment-63912265 [Test build #23699 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23699/consoleFull) for PR 3362 at commit [`3c23b42`](https://github.com/apache/spark/commit/3c23b420cb6b1d1d32fdd75adf94b1ad3a9bc868). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class BroadcastHashOuterJoin(` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4485][SQL]Add BroadcastHashOuterJoin
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3362#issuecomment-63912184 [Test build #23699 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23699/consoleFull) for PR 3362 at commit [`3c23b42`](https://github.com/apache/spark/commit/3c23b420cb6b1d1d32fdd75adf94b1ad3a9bc868). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: add jackson-core-asl 1.8.8 dependency
Github user devlatte closed the pull request at: https://github.com/apache/spark/pull/3379 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4048] Enhance and extend hadoop-provide...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2982#issuecomment-63911952 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23695/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4048] Enhance and extend hadoop-provide...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2982#issuecomment-63911945 [Test build #23695 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23695/consoleFull) for PR 2982 at commit [`322f882`](https://github.com/apache/spark/commit/322f882ce3de83f0a47a357f8209d08874c4d1d1). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4485][SQL]Add BroadcastHashOuterJoin
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/3362#issuecomment-63911728 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4431][MLlib] Implement efficient active...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3288#issuecomment-63910649 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23694/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4431][MLlib] Implement efficient active...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3288#issuecomment-63910641 [Test build #23694 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23694/consoleFull) for PR 3288 at commit [`1907ae1`](https://github.com/apache/spark/commit/1907ae122ac0f385e5c408b827bd438e209cd71e). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4508] [SQL] build native date type to c...
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/3381#issuecomment-63909766 This builds successfully locally, and the build error is very confusing, since I never changed anything related to that. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2261] Make event logger use a single fi...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1222#issuecomment-63909685 [Test build #23693 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23693/consoleFull) for PR 1222 at commit [`3f4500f`](https://github.com/apache/spark/commit/3f4500fe53b9dd0b5f1674d3664746c556ff9d2a). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2261] Make event logger use a single fi...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1222#issuecomment-63909689 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23693/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2261] Make event logger use a single fi...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1222#issuecomment-63908290 [Test build #23690 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23690/consoleFull) for PR 1222 at commit [`3f4500f`](https://github.com/apache/spark/commit/3f4500fe53b9dd0b5f1674d3664746c556ff9d2a). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4413][SQL] Parquet support through data...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3269#issuecomment-63908278 [Test build #23697 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23697/consoleFull) for PR 3269 at commit [`1dd75f1`](https://github.com/apache/spark/commit/1dd75f11208441bebf87c4315435587897685ab5). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2261] Make event logger use a single fi...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1222#issuecomment-63908298 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23690/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4145] Web UI job pages
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3009#issuecomment-63908292 [Test build #23698 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23698/consoleFull) for PR 3009 at commit [`6f17f3f`](https://github.com/apache/spark/commit/6f17f3f61102f5685d20cf42f79a049a5bbaad06). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4145] [WIP] Web UI job pages
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3009#issuecomment-63908091 Alright, I pushed that final cleanup commit. @andrewor14, want to take a final look on the JsonProtocol backwards-compatibility stuff? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2669] Localise hadoop configuration whe...
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/1574#issuecomment-63908038 @redbaron did you have a chance to look at the feedback and address the issues? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4525] MesosSchedulerBackend.resourceOff...
Github user tnachen commented on the pull request: https://github.com/apache/spark/pull/3393#issuecomment-63907940 Good catch, I think I didn't completely understand how TaskSchedulerImpl are using the offers and forgot not all acceptable offers are eventually used. Your PR LGTM, +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4244] [SQL] Support Hive Generic UDFs w...
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/3109#issuecomment-63907943 Thanks for explaining. Merged to master and 1.2. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4493][SQL] Don't pushdown Eq, NotEq, Lt...
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3367#issuecomment-63907746
[Test build #530 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/530/consoleFull)
for PR 3367 at commit
[`de7de28`](https://github.com/apache/spark/commit/de7de288e3e609feaee1d70b4cfbfcca624edec2).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `class LinearBinaryClassificationModel(LinearModel):`
* `class LogisticRegressionModel(LinearBinaryClassificationModel):`
* `class LogisticRegressionWithLBFGS(object):`
* `class SVMModel(LinearBinaryClassificationModel):`
* `class Rating(namedtuple("Rating", ["user", "product", "rating"])):`
* `class RDDRangeSampler(RDDSamplerBase):`
* `class SizeLimitedStream(object):`
* `class CompressedStream(object):`
* `class LargeObjectSerializer(Serializer):`
* `class CompressedSerializer(Serializer):`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4244] [SQL] Support Hive Generic UDFs w...
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/3109#discussion_r20690457
--- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUdfs.scala
---
@@ -162,9 +161,8 @@ private[hive] case class
HiveGenericUdf(functionClassName: String, children: Seq
(udfType != null && udfType.deterministic())
}
- override def foldable = {
-isUDFDeterministic && children.foldLeft(true)((prev, n) => prev &&
n.foldable)
- }
+ override def foldable =
+isUDFDeterministic &&
returnInspector.isInstanceOf[ConstantObjectInspector]
--- End diff --
The key change here is we need to get the folded result via Hive the method
`initializeAndFoldConstants` of UDF, not the `initialize` method, that's why I
made the change in L155-L156. UDF itself knows better how to constant fold the
computing if it's applicable, and the return value of
`initializeAndFoldConstants` tells us if it's can be or not and what the result
it is.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3697] Ignore event directories that can...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3391#issuecomment-63907601 [Test build #23688 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23688/consoleFull) for PR 3391 at commit [`5616fcd`](https://github.com/apache/spark/commit/5616fcd149e8485081ecd80c9d2cff326f2f8a2e). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3697] Ignore event directories that can...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3391#issuecomment-63907609 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23688/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4522][SQL] Parse schema with missing me...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3392#issuecomment-63907350 [Test build #23691 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23691/consoleFull) for PR 3392 at commit [`bcc6626`](https://github.com/apache/spark/commit/bcc6626c99361b4da8bca12bf45ceef8a49a3f45). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4522][SQL] Parse schema with missing me...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3392#issuecomment-63907351 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23691/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4477] [PySpark] remove numpy from RDDSa...
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/3351#issuecomment-63906978 Merged into master and branch-1.2. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2309][MLlib] Generalize the binary logi...
Github user dbtsai commented on the pull request: https://github.com/apache/spark/pull/1379#issuecomment-63906768 no, in the algorithm, I already model the problem http://www.slideshare.net/dbtsai/2014-0620-mlor-36132297/24 , so there will always be only (num_features + 1)(num_classes-1) parameters. Of course, you can chose any transformation to make it over-parameterize, see `Properties of softmax regression parameterization` session in wiki for detail. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
