Github user sjbrunst closed the pull request at:
https://github.com/apache/spark/pull/1717
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user sjbrunst commented on the pull request:
https://github.com/apache/spark/pull/1717#issuecomment-132999140
@JoshRosen Do we really want to deprecate the Twitter APIs before there is
a user-friendly way to use it in an external package? I think there is a large
user base
Github user sjbrunst commented on the pull request:
https://github.com/apache/spark/pull/1717#issuecomment-131888477
@srowen I am willing to close this PR, but I agree with @dmvieira . It
doesn't matter to me whether this functionality gets added inside Spark or as
an external
Github user sjbrunst commented on the pull request:
https://github.com/apache/spark/pull/1717#issuecomment-131907646
What is the timeline on moving the Twitter package out of Spark and into a
separate project? If it is going to be another year before that actually
happens
Github user sjbrunst commented on a diff in the pull request:
https://github.com/apache/spark/pull/1717#discussion_r34572915
--- Diff:
external/twitter/src/main/scala/org/apache/spark/streaming/twitter/TwitterInputDStream.scala
---
@@ -86,12 +97,18 @@ class TwitterReceiver
Github user sjbrunst commented on the pull request:
https://github.com/apache/spark/pull/1717#issuecomment-120095542
The discussion hasn't moved anywhere, as far as I know. I was waiting for
@tdas to look at the latest changes.
---
If your project is set up for it, you can reply
Github user sjbrunst commented on the pull request:
https://github.com/apache/spark/pull/1717#issuecomment-61988263
Ping @tdas Have you had a chance to look at this yet?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user sjbrunst commented on the pull request:
https://github.com/apache/spark/pull/1717#issuecomment-59415270
@tdas I've built the facade. I tried to add the language parameter, but it
failed to compile saying that the language method does not exist. Apparently it
hasn't been
Github user sjbrunst commented on a diff in the pull request:
https://github.com/apache/spark/pull/1717#discussion_r18979675
--- Diff:
external/twitter/src/main/scala/org/apache/spark/streaming/twitter/TwitterUtils.scala
---
@@ -32,16 +32,62 @@ object TwitterUtils
Github user sjbrunst commented on the pull request:
https://github.com/apache/spark/pull/1717#issuecomment-59079944
@surendramarupudi A DStream is a series of RDDs, so you can't cast a
DStream to an RDD. Try using the ``DStream.foreachRDD(func)`` to apply ``func``
to each RDD
Github user sjbrunst commented on the pull request:
https://github.com/apache/spark/pull/1717#issuecomment-58916787
@surendramarupudi Are you using a released version of Spark or a copy of my
repo? Either way, using a double[][] will not work. This change has not been
pulled
Github user sjbrunst commented on the pull request:
https://github.com/apache/spark/pull/1717#issuecomment-58955712
@surendramarupudi When you create a BoundingBox, you give it latitudes and
longitudes to define a rectangular area, and any tweet inside that rectangle
may be returned
Github user sjbrunst commented on the pull request:
https://github.com/apache/spark/pull/1717#issuecomment-58447539
@tdas Have you had any more thoughts on this? Which direction should we
take here?
---
If your project is set up for it, you can reply to this email and have your
Github user sjbrunst commented on the pull request:
https://github.com/apache/spark/pull/1717#issuecomment-57464410
@tdas That would be a better long-term solution, as I'm sure there will be
some users down the road who want to use the other parts of FilterQuery. That
way they won't
Github user sjbrunst commented on the pull request:
https://github.com/apache/spark/pull/1717#issuecomment-57552179
@tdas The way I understand it, we have two options here. The first is that
we allow users to pass in their own FilterQuery object. That will minimize the
number
Github user sjbrunst commented on the pull request:
https://github.com/apache/spark/pull/1717#issuecomment-57396987
@tdas Should be fixed now.
Should @ezhulenev 's changes be part of this PR? They're not part of
location filtering, but they require similar changes
Github user sjbrunst commented on the pull request:
https://github.com/apache/spark/pull/1717#issuecomment-57102225
@ezhulenev I added it in, but I'm having trouble getting the tests to run
on my own computer.
Also, I noticed you only added count to the Scala API. Shouldn't
Github user sjbrunst commented on the pull request:
https://github.com/apache/spark/pull/1717#issuecomment-57056007
@ezhulenev I've pulled in your changes and fixed a small scalastyle error.
I agree that we should avoid having too many methods for all the parameter
Github user sjbrunst commented on the pull request:
https://github.com/apache/spark/pull/1717#issuecomment-57059353
@ezhulenev I've rolled back those changes now. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user sjbrunst commented on a diff in the pull request:
https://github.com/apache/spark/pull/1717#discussion_r17972016
--- Diff:
external/twitter/src/main/scala/org/apache/spark/streaming/twitter/TwitterUtils.scala
---
@@ -33,15 +33,38 @@ object TwitterUtils
Github user sjbrunst commented on the pull request:
https://github.com/apache/spark/pull/1717#issuecomment-56673018
@tdas The current version of TwitterUtils.scala only has new methods. The
diff makes it look like I changed the original methods, but they are all there.
The original
Github user sjbrunst commented on the pull request:
https://github.com/apache/spark/pull/1717#issuecomment-55511357
@tdas It's ready for another look! I added a BoundingBox class that can be
used to pass in the coordinates, which should be much more intuitive.
---
If your project
Github user sjbrunst commented on the pull request:
https://github.com/apache/spark/pull/1717#issuecomment-55355516
I have the new case class written, I just haven't tested it with an actual
stream yet. It should be ready sometime tomorrow or Saturday.
---
If your project is set up
Github user sjbrunst commented on a diff in the pull request:
https://github.com/apache/spark/pull/1717#discussion_r17217079
--- Diff:
external/twitter/src/main/scala/org/apache/spark/streaming/twitter/TwitterInputDStream.scala
---
@@ -42,6 +44,7 @@ class TwitterInputDStream
Github user sjbrunst commented on a diff in the pull request:
https://github.com/apache/spark/pull/1717#discussion_r17210353
--- Diff:
external/twitter/src/main/scala/org/apache/spark/streaming/twitter/TwitterInputDStream.scala
---
@@ -42,9 +44,19 @@ class TwitterInputDStream
Github user sjbrunst commented on a diff in the pull request:
https://github.com/apache/spark/pull/1717#discussion_r17210396
--- Diff:
external/twitter/src/main/scala/org/apache/spark/streaming/twitter/TwitterUtils.scala
---
@@ -33,15 +33,20 @@ object TwitterUtils
Github user sjbrunst commented on the pull request:
https://github.com/apache/spark/pull/1717#issuecomment-54474415
Unit tests fail because my changes are not completely binary compatible
yet. I'm having some trouble overloading the Scala version of the
`createStream` method. See my
Github user sjbrunst commented on a diff in the pull request:
https://github.com/apache/spark/pull/1717#discussion_r17091665
--- Diff:
external/twitter/src/main/scala/org/apache/spark/streaming/twitter/TwitterUtils.scala
---
@@ -33,15 +33,20 @@ object TwitterUtils
Github user sjbrunst commented on a diff in the pull request:
https://github.com/apache/spark/pull/1717#discussion_r16729438
--- Diff:
external/twitter/src/main/scala/org/apache/spark/streaming/twitter/TwitterInputDStream.scala
---
@@ -85,9 +89,14 @@ class TwitterReceiver
Github user sjbrunst commented on the pull request:
https://github.com/apache/spark/pull/1717#issuecomment-53475085
@tdas Thanks for the comments! I'll work on fixing the binary
compatibility, though I might not have it done until sometime next week since
I'm currently on vacation
30 matches
Mail list logo
