Re: mapWithState and DataFrames

2016-11-06 Thread Victor Shafran
ht now I stream json messages from Kafka, update their state, output > the updated state as json and compose a dataframe from it. > It seems inefficient both in terms of processing and storage (a long > string instead of a compact DF). > > Is there a way to manage state for DataFrame?

Re: Sharing RDDS across applications and users

2016-10-27 Thread Victor Shafran
>>> applications? >>> >>> It looks feasible as RDDs are immutable and so are registered tempTables >>> etc. >>> >>> Thanks >>> >>> >>> Dr Mich Talebzadeh >>> >>> >>> LinkedIn * >>> ht