Unsubscribe
Ma Dianjun
Incase you are specifically looking for a createDataframe method, you can use
sparkSession.createDataFrame(
Arrays.asList("apple","orange","banana").stream().map(RowFactory::create).collect(Collectors.toList()),
new StructType().add("fruits", "string")
).show();
Regards,
Vibhor Gupta
unsubscribe
Hello,
Now Spark UI does not show HashAggregateExec modes, could we add the
aggregate modes in SparkPlan? I think it's helpful when we analyze a very
complicated SparkPlan.
Am I right?
For example:
SELECT key2, sum(value2) as sum_value2
FROM ( SELECT id % 1 as key2, id as value2
Thanks a lot, Sebastian and Vibhor. You're right, I can call the
createDataset() also on the Spark session. Not sure how I missed that.
Cheers,
Martin
Am 2021-11-18 12:01, schrieb Vibhor Gupta:
You can try something like below. It creates a dataset and then
converts it into a dataframe.
You can try something like below. It creates a dataset and then converts it
into a dataframe.
sparkSession.createDataset(
Arrays.asList("apple","orange","banana"),
Encoders.STRING()
).toDF("fruits").show();
Regards,
Vibhor Gupta.
From: Sebastian Piu
You can call that on sparkSession to
On Thu, 18 Nov 2021, 10:48 , wrote:
> PS: The following works, but it seems rather awkward having to use the
> SQLContext here.
>
> SQLContext sqlContext = new SQLContext(sparkContext);
>
> Dataset data = sqlContext
> .createDataset(textList,
The most convenient way I'm aware of from Java is to use createDataset and
pass Encoder.String
That gives you a Dataset if you still want Dataset the you can
call .toDF on it
On Thu, 18 Nov 2021, 10:27 , wrote:
> Hello,
>
> I am struggling with a task that should be super simple: I would like
PS: The following works, but it seems rather awkward having to use the
SQLContext here.
SQLContext sqlContext = new SQLContext(sparkContext);
Dataset data = sqlContext
.createDataset(textList, Encoders.STRING())
.withColumnRenamed("value", "text");
Am 2021-11-18 11:26, schrieb
Hello,
I am struggling with a task that should be super simple: I would like to
create a Spark DF of Type Dataset with one column from a single
String (or from a one-element List of Strings). The column header should
be "text".
SparkContext.parallelize() does not work, because it returns
unsubscribe
11 matches
Mail list logo
