Re: toDebugString - RDD Logical Plan

2019-04-20 Thread Dylan Guedes
Kanchan, the `toDebugString` looks unformatted because in some scenarios you need to parse it before (can't remember the reason, though). I suggest you to print the RDD Lineage using `print(rdd.toDebugString().decode("utf-8"))` instead (obs: this only occurs in Pyspark). About the other question,

Re: Not able to access Pyspark into Jupyter notebook

2018-04-11 Thread Dylan Guedes
Well... could you post the log or any errors that occurs? I used this pyspark jupyter notebook and it worked great. On Wed, Apr 11, 2018 at 12:36 AM, @Nandan@ wrote: > Hi Users, > >

Re: Multiple Kafka Spark Streaming Dataframe Join query

2018-03-14 Thread Dylan Guedes
gt; Aakash. > > On 15-Mar-2018 12:27 AM, "Dylan Guedes" <djmggue...@gmail.com> wrote: > >> Hi, >> >> I've been using the Kafka with pyspark since 2.1. >> >> On Wed, Mar 14, 2018 at 3:49 PM, Aakash Basu <aakash.spark@gmail.com> >&g

Re: Multiple Kafka Spark Streaming Dataframe Join query

2018-03-14 Thread Dylan Guedes
Hi, I've been using the Kafka with pyspark since 2.1. On Wed, Mar 14, 2018 at 3:49 PM, Aakash Basu wrote: > Hi, > > I'm yet to. > > Just want to know, when does Spark 2.3 with 0.10 Kafka Spark Package > allows Python? I read somewhere, as of now Scala and Java are