RE: Where is DataFrame.scala in 2.0?

[Gerhard Fiedler]

Thanks!

From: Herman van Hövell tot Westerflier [mailto:hvanhov...@questtec.nl]
Sent: Fri, Jun 03, 2016 10:05
To: Gerhard Fiedler <gfied...@algebraixdata.com>
Cc: dev@spark.apache.org
Subject: Re: Where is DataFrame.scala in 2.0?

Hi Gerhard,

DataFrame and DataSet have been merged in Spark 2.0. A DataFrame is now a 
DataSet that contains Row objects. We still maintain a type alias for 
DataFrame: 
https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/package.scala#L45

HTH

Kind regards,

Herman van Hövell tot Westerflier

2016-06-03 17:01 GMT+02:00 Gerhard Fiedler 
<gfied...@algebraixdata.com<mailto:gfied...@algebraixdata.com>>:
When I look at the sources in Github, I see DataFrame.scala at 
https://github.com/apache/spark/blob/branch-1.6/sql/core/src/main/scala/org/apache/spark/sql/DataFrame.scala
 in the 1.6 branch. But when I change the branch to branch-2.0 or master, I get 
a 404 error. I also can’t find the file in the directory listings, for example 
https://github.com/apache/spark/tree/branch-2.0/sql/core/src/main/scala/org/apache/spark/sql
 (for branch-2.0).

It seems that quite a few APIs use the DataFrame class, even in 2.0. Can 
someone please point me to its location, or otherwise explain why it is not 
there?

Thanks,
Gerhard

Re: Where is DataFrame.scala in 2.0?

[Michael Malak]

It's been reduced to a single line of code.
http://technicaltidbit.blogspot.com/2016/03/dataframedataset-swap-places-in-spark-20.html




  From: Gerhard Fiedler 
 To: "dev@spark.apache.org"  
 Sent: Friday, June 3, 2016 9:01 AM
 Subject: Where is DataFrame.scala in 2.0?
   
 When I look at the sources in Github, I see 
DataFrame.scala 
athttps://github.com/apache/spark/blob/branch-1.6/sql/core/src/main/scala/org/apache/spark/sql/DataFrame.scala
 in the 1.6 branch. But when I change the branch to branch-2.0 or master, I get 
a 404 error. I also can’t find the file in the directory listings, for example 
https://github.com/apache/spark/tree/branch-2.0/sql/core/src/main/scala/org/apache/spark/sql
 (for branch-2.0).    It seems that quite a few APIs use the DataFrame class, 
even in 2.0. Can someone please point me to its location, or otherwise explain 
why it is not there?    Thanks, Gerhard    

  

Re: Where is DataFrame.scala in 2.0?

[Herman van Hövell tot Westerflier]

Hi Gerhard,

DataFrame and DataSet have been merged in Spark 2.0. A DataFrame is now a
DataSet that contains Row objects. We still maintain a type alias for
DataFrame:
https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/package.scala#L45

HTH

Kind regards,

Herman van Hövell tot Westerflier

2016-06-03 17:01 GMT+02:00 Gerhard Fiedler :

> When I look at the sources in Github, I see DataFrame.scala at
> https://github.com/apache/spark/blob/branch-1.6/sql/core/src/main/scala/org/apache/spark/sql/DataFrame.scala
> in the 1.6 branch. But when I change the branch to branch-2.0 or master, I
> get a 404 error. I also can’t find the file in the directory listings, for
> example
> https://github.com/apache/spark/tree/branch-2.0/sql/core/src/main/scala/org/apache/spark/sql
> (for branch-2.0).
>
>
>
> It seems that quite a few APIs use the DataFrame class, even in 2.0. Can
> someone please point me to its location, or otherwise explain why it is not
> there?
>
>
>
> Thanks,
>
> Gerhard
>
>
>