I firstly tried simply copy the codes below into spark application that has codes working with kafka streaming. No any result can be printed into system console, then I tried spark shell, and it works. I can’t figured out the reason.
Best regards, Cui Lin From: Tobias Pfeiffer <[email protected]<mailto:[email protected]>> Date: Tuesday, March 3, 2015 at 5:13 PM To: Cui Lin <[email protected]<mailto:[email protected]>> Cc: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: Spark sql results can't be printed out to system console from spark streaming application Hi, can you explain how you copied that into your *streaming* application? Like, how do you issue the SQL, what data do you operate on, how do you view the logs etc.? Tobias On Wed, Mar 4, 2015 at 8:55 AM, Cui Lin <[email protected]<mailto:[email protected]>> wrote: >Dear all, > >I found the below sample code can be printed out only in spark shell, but >when I moved them into my spark streaming application, nothing can be >printed out into system console. Can you explain why it happened? anything >related to new spark context? Thanks a lot! > > >val anotherPeopleRDD = sc_context.parallelize( > """{"name":"Yin","address":{"city":"Columbus","state":"Ohio"}}""" :: >Nil) > >anotherPeopleRDD.toArray().foreach(line => System.out.println(line)) > >val jsonMessage = sqlContext.jsonRDD(anotherPeopleRDD) > >jsonMessage.toArray().foreach(line => System.out.println(line)) > >jsonMessage.registerTempTable("people") > >val test: SchemaRDD = sqlContext.sql("select count(*) from people") > >test.toArray().foreach(line => System.out.println(line)) > > > > >Best regards, > >Cui Lin > > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected]<mailto:[email protected]> For additional commands, e-mail: [email protected]<mailto:[email protected]>
