Yes. Yes you can.
On Tue, Jul 17, 2018 at 11:42 AM, Sathi Chowdhury wrote:
> Hi,
> My question is about ability to integrate spark streaming with multiple
> clusters.Is it a supported use case. An example of that is that two topics
> owned by different group and they have their own kakka infra
this may work
val df_post= listCustomCols
.foldLeft(df_pre){(tempDF, listValue) =>
tempDF.withColumn(
listValue.name,
new Column(listValue.name.toString + funcUDF(listValue.name))
)
and outsource the renaming to an udf
or you can rename the
Hi,My question is about ability to integrate spark streaming with multiple
clusters.Is it a supported use case. An example of that is that two topics
owned by different group and they have their own kakka infra .Can i have two
dataframes as a result of spark.readstream listening to different
Is there a version of withColumn or withColumnRenamed that accept Column
instead of String? That way I can specify FQN in case when there is
duplicate column names.
I can Drop column based on Column type argument then why can't I rename
them based on same type argument.
Use case is, I have
