Thank you! Yes that's the way to go taking care of selecting them in the proper
order first. Added a select before the toDF and it does the trick.
From: Sunitha Kambhampati [mailto:skambha...@gmail.com]
Sent: Friday, March 18, 2016 5:46 PM
To: Fernandez, Andres
Cc: user@spark.apache.org
Subject:
Good morning. I have a dataframe and would like to group by on two fields, and
perform a sum aggregation on more than 500 fields, though I would like to keep
the same name for the 500 hundred fields (instead of sum(Field)). I do have the
field names in an array. Could anybody help with this
Worked perfectly. Thanks very much Silvio.
From: Silvio Fiorito [mailto:silvio.fior...@granturing.com]
Sent: Tuesday, March 01, 2016 2:14 PM
To: Fernandez, Andres; user@spark.apache.org
Subject: Re: Union Parquet, DataFrame
Just replied to your other email, but here’s the same thing:
Just do:
Good day colleagues. Quick question on Parquet and Dataframes. Right now I have
the 4 parquet files stored in HDFS under the same path:
/path/to/parquets/parquet1, /path/to/parquets/parquet2,
/path/to/parquets/parquet3, /path/to/parquets/parquet4…
I want to perform a union on all this parquet
Good day colleagues. Quick question on Parquet and Dataframes. Right now I have
the 4 parquet files stored in HDFS under the same path:
/path/to/parquets/parquet1, /path/to/parquets/parquet2,
/path/to/parquets/parquet3, /path/to/parquets/parquet4…
I want to perform a union on all this parquet
So far, still cannot find a way of running a small Scala script right after
executing the shell, and get the shell to remain open. Is there a way of doing
this?
Feels like a simple/naive question but really couldn’t find an answer.
From: Fernandez, Andres
Sent: Tuesday, January 26, 2016 2:53 PM
True thank you. Is there a way of having the shell not closed (how to avoid the
:quit statement). Thank you both.
Andres
From: Ewan Leith [mailto:ewan.le...@realitymine.com]
Sent: Tuesday, January 26, 2016 1:50 PM
To: Iulian Dragoș; Fernandez, Andres
Cc: user
Subject: RE: how to correctly run
