For some reason my pasted screenshots were removed when I sent the email
(at least that's how it appeared on my end). Repasting as text below.
The sequence you are referring to represents the list of column names to
fill. I am asking about filling a column which is of type list with an
empty list.
The sequence you are referring to represents the list of column names to
fill. I am asking about filling a column which is of type list with an
empty list.
Here is a quick example of what I am doing:
The output of the show and printSchema for the collectList df:
So, the last line which doesn'
It does support it, at least in 2.0.2 as I am running:
Here one example:
val parsedLines = stream_of_logs
.map(line => p.parseRecord_viaCSVParser(line))
.join(appsCateg,$"Application"===$"name","left_outer")
.drop("id")
.na.fill(0, Seq(“numeric_field1”,"numeric_field2"))
.na.fill("", Se
Hi there,
I have two dataframes that each have some columns which are of list type
(array generated by the collect_list function actually).
I need to outer join these two dfs, however by nature of an outer join I am
sometimes left with null values. Normally I would use df.na.fill(...),
however it
