[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2016-02-08 Thread agsachin
Github user agsachin commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-181379130 @srowen Is anyone working on third-party package for twitter, I suggest to add FilterQuery support to existing api, and move twitter Api's out of spark

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-09-15 Thread sjbrunst
Github user sjbrunst closed the pull request at: https://github.com/apache/spark/pull/1717 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-08-20 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-133000700 You can add any third-party package with `--packages` or add it to your application as a dependency. I don't think Josh is saying it _will_ be deprecated, certainly not

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-08-20 Thread sjbrunst
Github user sjbrunst commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-132999140 @JoshRosen Do we really want to deprecate the Twitter APIs before there is a user-friendly way to use it in an external package? I think there is a large user base for

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-08-18 Thread dmvieira
Github user dmvieira commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-132340568 I'm starting a third-party package as suggested by @srowen and I hope you enjoy. Feel free to collaborate: https://github.com/dmvieira/spark-twitter-stream-receiver

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-08-17 Thread dmvieira
Github user dmvieira commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-131894055 So, why not improve it with this PR and then move it to a new project / package when we think about a better solution? We can create an issue or you can talk with

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-08-17 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-131902614 @dmvieira I'm not sure who you're addressing there, but if this isn't something that will go in Spark, there's no reason to discuss the change here, right? Make a new

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-08-17 Thread sjbrunst
Github user sjbrunst commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-131888477 @srowen I am willing to close this PR, but I agree with @dmvieira . It doesn't matter to me whether this functionality gets added inside Spark or as an external

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-08-17 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-131890174 Personally I would love to see the Twitter package moved from Spark itself into a separate project / package; the only reason that we have it is for legacy reasons.

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-08-17 Thread sjbrunst
Github user sjbrunst commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-131907646 What is the timeline on moving the Twitter package out of Spark and into a separate project? If it is going to be another year before that actually happens then it

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-08-17 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-131909451 Should we deprecate these Twitter APIs in order to encourage them to be split into a third-party package? --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-08-17 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-131881635 I don't think this PR is going forward, and should be closed. Do you mind closing this PR @sjbrunst ? You can see some other related efforts, but the impression I have

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-08-17 Thread dmvieira
Github user dmvieira commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-131885136 But without this path you're restricting a lot Twitter functionalities inside Spark and still supporting Twitter interface. Spark still maintain Twitter API interface

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-08-17 Thread dmvieira
Github user dmvieira commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-131880125 Hey guys, I need this patch too. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-07-14 Thread sjbrunst
Github user sjbrunst commented on a diff in the pull request: https://github.com/apache/spark/pull/1717#discussion_r34572915 --- Diff: external/twitter/src/main/scala/org/apache/spark/streaming/twitter/TwitterInputDStream.scala --- @@ -86,12 +97,18 @@ class TwitterReceiver(

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-121073750 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-07-13 Thread huitseeker
Github user huitseeker commented on a diff in the pull request: https://github.com/apache/spark/pull/1717#discussion_r34440188 --- Diff: external/twitter/src/main/scala/org/apache/spark/streaming/twitter/TwitterInputDStream.scala --- @@ -28,11 +28,16 @@ import

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-07-13 Thread huitseeker
Github user huitseeker commented on a diff in the pull request: https://github.com/apache/spark/pull/1717#discussion_r34441284 --- Diff: external/twitter/src/main/scala/org/apache/spark/streaming/twitter/TwitterInputDStream.scala --- @@ -86,12 +97,18 @@ class TwitterReceiver(

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-07-09 Thread sjbrunst
Github user sjbrunst commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-120095542 The discussion hasn't moved anywhere, as far as I know. I was waiting for @tdas to look at the latest changes. --- If your project is set up for it, you can reply to

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-07-09 Thread philcontrolf1
Github user philcontrolf1 commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-119880257 Hi all - this functionality is certainly something I'm interested in, but this discussion seems to have stalled somewhat. Is this still the main discussion for

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-07-09 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-120147119 To throw a monkey wrench into this discussion: do we really want to be maintaining a Twitter library inside of Spark itself or should we try to move the ongoing

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-96770284 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-72789158 [Test build #26709 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26709/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-72789165 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-72783897 [Test build #26709 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26709/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-72748803 [Test build #26676 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26676/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-72748809 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-72739435 [Test build #26676 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26676/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-01-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-68612972 [Test build #25017 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25017/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-01-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-68613000 [Test build #25017 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25017/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-01-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-68613002 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-01-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-68615337 [Test build #25018 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25018/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-01-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-68615610 [Test build #25019 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25019/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-01-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-68616927 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-01-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-68617147 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2015-01-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-68617146 [Test build #25019 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25019/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-11-06 Thread sjbrunst
Github user sjbrunst commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-61988263 Ping @tdas Have you had a chance to look at this yet? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-10-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-59415277 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21814/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-10-16 Thread sjbrunst
Github user sjbrunst commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-59415270 @tdas I've built the facade. I tried to add the language parameter, but it failed to compile saying that the language method does not exist. Apparently it hasn't been

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-10-16 Thread sjbrunst
Github user sjbrunst commented on a diff in the pull request: https://github.com/apache/spark/pull/1717#discussion_r18979675 --- Diff: external/twitter/src/main/scala/org/apache/spark/streaming/twitter/TwitterUtils.scala --- @@ -32,16 +32,62 @@ object TwitterUtils { *

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-10-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-59422448 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-10-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-59422438 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21814/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-10-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-59427005 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-10-14 Thread surendramarupudi
Github user surendramarupudi commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-59057521 @sjbrunst do you have any idea how to cast JavaDStreamStatus to JavaRDDStatus, I need to cast this to save retrieved tweets to mongoDB. --- If your project is

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-10-14 Thread sjbrunst
Github user sjbrunst commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-59079944 @surendramarupudi A DStream is a series of RDDs, so you can't cast a DStream to an RDD. Try using the ``DStream.foreachRDD(func)`` to apply ``func`` to each RDD in the

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-10-13 Thread sjbrunst
Github user sjbrunst commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-58916787 @surendramarupudi Are you using a released version of Spark or a copy of my repo? Either way, using a double[][] will not work. This change has not been pulled into

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-10-13 Thread surendramarupudi
Github user surendramarupudi commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-58950787 @sjbrunst I am using the copy of your repo. Could you please give me one example for TwitterUtils.createStream(ssc,filters,bounding_box) and how to use

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-10-13 Thread sjbrunst
Github user sjbrunst commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-58955712 @surendramarupudi When you create a BoundingBox, you give it latitudes and longitudes to define a rectangular area, and any tweet inside that rectangle may be returned.

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-10-13 Thread surendramarupudi
Github user surendramarupudi commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-58956675 @sjbrunst Thanks a lot for your clarification. I am able to get the geotagged tweets but I found one issue that is filtering based on keywords is not working,

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-10-12 Thread surendramarupudi
Github user surendramarupudi commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-58811881 Hi, I am trying to use this new location feature TwitterUtils.createStream(ssc,filters,double[][]) for this filters are working but filtering for location is

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-10-10 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-58725605 Sorry for taking so long to get back on this. I agree with @ezhulenev that the Twitter4J Authorization has already leaked into public API, so we are essentially tied with

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-10-08 Thread sjbrunst
Github user sjbrunst commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-58447539 @tdas Have you had any more thoughts on this? Which direction should we take here? --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-10-01 Thread sjbrunst
Github user sjbrunst commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57464410 @tdas That would be a better long-term solution, as I'm sure there will be some users down the road who want to use the other parts of FilterQuery. That way they won't

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-10-01 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57508615 @sjbrunst Seems like @ezhulenev has submitted #2618 to add the filterquery based stream. I think I prefer that over adding this explicit location support. I know that you

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-10-01 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57538997 I thought about this a little bit more. @sjbrunst That is a possibility. But I dont want to add two new sets of methods ( 2 x ( 4 + 4 ) new methods). Also, now that

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-10-01 Thread sjbrunst
Github user sjbrunst commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57552179 @tdas The way I understand it, we have two options here. The first is that we allow users to pass in their own FilterQuery object. That will minimize the number of

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-10-01 Thread ezhulenev
Github user ezhulenev commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57560702 Twitter4j Authorization already leaked into public API, so FilterQuery doesn't make things much worse. I personally don't like Twitter4j for too javaish API

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-10-01 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57575122 Do you mind adding closes #2098 to the description of your PR so that this automatically closes the other PR when merged? Thanks!! --- If your project is set up for

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-30 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57378916 Something has really messed up this PR!! There are 112 changed files, with 51 commits. Please fix this. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-30 Thread sjbrunst
Github user sjbrunst commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57396987 @tdas Should be fixed now. Should @ezhulenev 's changes be part of this PR? They're not part of location filtering, but they require similar changes. --- If

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57397139 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21073/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57397587 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21075/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-30 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57402267 @sjbrunst @ezhulenev Can you explain what is the point of count? If I understood Twitter4j API correctly, count sets the pagination size. How does that help here? Or is my

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57402782 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57402777 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21073/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57403336 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21075/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57403344 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-30 Thread ezhulenev
Github user ezhulenev commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57405752 @tdas count sets number of status updates from the past, that will be added to stream before switching to live mode --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-30 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57408771 Aah I looking at the wrong docs (twitter4j.Query rather than twitter4j.FilterQuery). Looking at FilterQuery, I find that it has many functionalities, tracking, etc. Not

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57157005 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20972/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57164020 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57164005 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20972/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-28 Thread sjbrunst
Github user sjbrunst commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57102225 @ezhulenev I added it in, but I'm having trouble getting the tests to run on my own computer. Also, I noticed you only added count to the Scala API. Shouldn't

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57102315 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20952/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57103889 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57103886 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20952/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57055491 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20918/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57055518 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20918/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57055519 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57055921 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20919/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-27 Thread sjbrunst
Github user sjbrunst commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57056007 @ezhulenev I've pulled in your changes and fixed a small scalastyle error. I agree that we should avoid having too many methods for all the parameter

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57056016 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20919/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57056019 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-27 Thread ezhulenev
Github user ezhulenev commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57059126 @sjbrunst you need to rollback your changes in TwitterAlgebirdCMD TwitterAlgebirdHLL (remove Nil for locations), and after that project will compile and I should

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-27 Thread sjbrunst
Github user sjbrunst commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57059353 @ezhulenev I've rolled back those changes now. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57059419 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20920/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57060929 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20920/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57060931 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-27 Thread ezhulenev
Github user ezhulenev commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57071591 @sjbrunst aargh, TwitterStreamSuite.scala:53 requred to add count parameter --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-26 Thread ezhulenev
Github user ezhulenev commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57012331 @sjbrunst I need to add count parameter to twitter streams, I merged your fork and made small changes. Unfortunately I can't submit PR to your repository. It would be

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-26 Thread ezhulenev-at-pellucid
Github user ezhulenev-at-pellucid commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57023036 @sjbrunst I tested my branch with standalone Spark 1.1.0 and it works fine, even without additional constructors, so I removed them. Looks like it's binary

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-24 Thread sjbrunst
Github user sjbrunst commented on a diff in the pull request: https://github.com/apache/spark/pull/1717#discussion_r17972016 --- Diff: external/twitter/src/main/scala/org/apache/spark/streaming/twitter/TwitterUtils.scala --- @@ -33,15 +33,38 @@ object TwitterUtils { *

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-24 Thread sjbrunst
Github user sjbrunst commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-56673018 @tdas The current version of TwitterUtils.scala only has new methods. The diff makes it look like I changed the original methods, but they are all there. The original

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-23 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-56578032 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-56578914 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20722/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-56588183 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20722/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-23 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-56596924 @sjbrunst this is again failing the binary compatibility check. I think its same problem once again, that you had fixed in the earlier version of the patch. Do not change

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-56385915 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20655/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-56394656 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20655/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-13 Thread sjbrunst
Github user sjbrunst commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-55511357 @tdas It's ready for another look! I added a BoundingBox class that can be used to pass in the coordinates, which should be much more intuitive. --- If your project is

  1   2   >