[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/6910#issuecomment-114594279 Oops, missed the title. Looks like this was merged though - there is another PR for the bump to 1.6.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/6910#issuecomment-114379229 Merged to master and 1.4. But I think I missed the RC1 cutoff. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/6910 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/6910#issuecomment-114376812 dude, you didnt update the title and PR message. Its going to be super confusing for people later. Please update asap, and I can include it in 1.4.1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/6910#issuecomment-114270314 ah, looks like there are no flume python tests. Since this only touches flume code it doesn't have to run streaming tests. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/6910#issuecomment-114270274 It took only about 15 mins, so it seems to have run only a few tests or none at all. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/6910#issuecomment-114268388 @JoshRosen seems like this is not running pyspark tests. Do you know what the issue is? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6910#issuecomment-114256537 [Test build #35485 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35485/console) for PR 6910 at commit [`9875f7d`](https://github.com/apache/spark/commit/9875f7d0d2a7afe60512277f566496a0a34b9de1). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6910#issuecomment-114256562 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6910#issuecomment-114253575 [Test build #35485 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35485/consoleFull) for PR 6910 at commit [`9875f7d`](https://github.com/apache/spark/commit/9875f7d0d2a7afe60512277f566496a0a34b9de1). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6910#issuecomment-114253346 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6910#issuecomment-114253321 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/6910#issuecomment-114253086 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6910#issuecomment-114228515 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6910#issuecomment-114224409 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/6910#issuecomment-114224203 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/6910#issuecomment-114224144 Looks like github plugin fail --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6910#issuecomment-114223402 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6910#issuecomment-114219382 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6910#issuecomment-114219432 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/6910#discussion_r32882795 --- Diff: external/flume-sink/src/main/scala/org/apache/spark/streaming/flume/sink/SparkAvroCallbackHandler.scala --- @@ -53,7 +53,7 @@ private[flume] class SparkAvroCallbackHandler(val threads: Int, val channel: Cha // Since the new txn may not have the same sequence number we must guard against accidentally // committing a new transaction. To reduce the probability of that happening a random string is // prepended to the sequence number. Does not change for life of sink - private val seqBase = RandomStringUtils.randomAlphanumeric(8) + private val seqBase = UUID.randomUUID().toString.substring(0, 8) --- End diff -- Ah, this is a Type 4 random UUID so it only relies on random numbers and not something less random, such as the date / time or a MAC address or namespace, so this seems fine to me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/6910#issuecomment-113689433 Could you please make a separate JIRA and PR for bumping the version. These are two completely different issues. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6910#issuecomment-113687421 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6910#issuecomment-113687397 [Test build #35335 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35335/console) for PR 6910 at commit [`ca35eb0`](https://github.com/apache/spark/commit/ca35eb085a71a44e8e7e36d0e6a96b951727f0a1). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
Github user harishreedharan commented on a diff in the pull request: https://github.com/apache/spark/pull/6910#discussion_r32876281 --- Diff: external/flume-sink/src/main/scala/org/apache/spark/streaming/flume/sink/SparkAvroCallbackHandler.scala --- @@ -53,7 +53,7 @@ private[flume] class SparkAvroCallbackHandler(val threads: Int, val channel: Cha // Since the new txn may not have the same sequence number we must guard against accidentally // committing a new transaction. To reduce the probability of that happening a random string is // prepended to the sequence number. Does not change for life of sink - private val seqBase = RandomStringUtils.randomAlphanumeric(8) + private val seqBase = UUID.randomUUID().toString.substring(0, 8) --- End diff -- Last time, we tried it and we ended up getting the same string multiple times. I am not entirely sure why, but that was the reason we used RandomStringUtils in the first place. The idea of using the seqBase is to protect against the case where a sink restart can cause collisions. I think this should be random enough, else we can use the full UUID which I feel is adding too much overhead without a whole lot of gain. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/6910#discussion_r32876062 --- Diff: external/flume-sink/src/main/scala/org/apache/spark/streaming/flume/sink/SparkAvroCallbackHandler.scala --- @@ -53,7 +53,7 @@ private[flume] class SparkAvroCallbackHandler(val threads: Int, val channel: Cha // Since the new txn may not have the same sequence number we must guard against accidentally // committing a new transaction. To reduce the probability of that happening a random string is // prepended to the sequence number. Does not change for life of sink - private val seqBase = RandomStringUtils.randomAlphanumeric(8) + private val seqBase = UUID.randomUUID().toString.substring(0, 8) --- End diff -- Why not just use the Scala random string functionality instead? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/6910#discussion_r32875995 --- Diff: external/flume-sink/src/main/scala/org/apache/spark/streaming/flume/sink/SparkAvroCallbackHandler.scala --- @@ -53,7 +53,7 @@ private[flume] class SparkAvroCallbackHandler(val threads: Int, val channel: Cha // Since the new txn may not have the same sequence number we must guard against accidentally // committing a new transaction. To reduce the probability of that happening a random string is // prepended to the sequence number. Does not change for life of sink - private val seqBase = RandomStringUtils.randomAlphanumeric(8) + private val seqBase = UUID.randomUUID().toString.substring(0, 8) --- End diff -- Will this be sufficiently random? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6910#issuecomment-113664378 [Test build #35335 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35335/consoleFull) for PR 6910 at commit [`ca35eb0`](https://github.com/apache/spark/commit/ca35eb085a71a44e8e7e36d0e6a96b951727f0a1). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6910#issuecomment-113663947 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6910#issuecomment-113663931 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8483][Streaming] Remove commons-lang3 d...
GitHub user harishreedharan opened a pull request: https://github.com/apache/spark/pull/6910 [SPARK-8483][Streaming] Remove commons-lang3 dependency from Flume Si⦠â¦nk. Also bump Flume version to 1.6.0 You can merge this pull request into a Git repository by running: $ git pull https://github.com/harishreedharan/spark remove-commons-lang3 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/6910.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #6910 commit ca35eb085a71a44e8e7e36d0e6a96b951727f0a1 Author: Hari Shreedharan Date: 2015-06-19T22:42:40Z [SPARK-8483][Streaming] Remove commons-lang3 dependency from Flume Sink. Also bump Flume version to 1.6.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
