Oh right, that could have happened only after Spark 1.0.0. So let me clarify. At some point, you were able to access TwitterUtils from spark shell using Spark 1.0.0+ ? If yes, then what change in Spark caused it to not work any more?
TD On Mon, Jul 14, 2014 at 7:52 PM, Nicholas Chammas < nicholas.cham...@gmail.com> wrote: > If we're talking about the issue you captured in SPARK-2464 > <https://issues.apache.org/jira/browse/SPARK-2464>, then it was a newly > launched EC2 cluster on 1.0.1. > > > On Mon, Jul 14, 2014 at 10:48 PM, Tathagata Das < > tathagata.das1...@gmail.com> wrote: > >> Did you make any updates in Spark version recently, after which you >> noticed this problem? Because if you were using Spark 0.8 and below, then >> twitter would have worked in the Spark shell. In Spark 0.9, we moved those >> dependencies out of the core spark for those to update more freely without >> raising dependency-related concerns into the core of spark streaming. >> >> TD >> >> >> On Mon, Jul 14, 2014 at 6:29 PM, Nicholas Chammas < >> nicholas.cham...@gmail.com> wrote: >> >>> On Mon, Jul 14, 2014 at 6:52 PM, Tathagata Das < >>> tathagata.das1...@gmail.com> wrote: >>> >>>> The twitter functionality is not available through the shell. >>>> >>> >>> I've been processing Tweets live from the shell, though not for a long >>> time. That's how I uncovered the problem with the Twitter receiver not >>> deregistering, btw. >>> >>> Did I misunderstand your comment? >>> >>> Nick >>> >> >> >
