Re: renaming SchemaRDD - DataFrame

2015-02-12 Thread vha14
/What-is-Semantic-Scholar-and-how-will-it-work -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/renaming-SchemaRDD-DataFrame-tp10271p10612.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com

Re: renaming SchemaRDD - DataFrame

2015-02-10 Thread Koert Kuipers
: Monday, January 26, 2015 4:01 PM Subject: Re: renaming SchemaRDD - DataFrame One thing potentially not clear from this e-mail, there will be a 1:1 correspondence where you can get an RDD to/from a DataFrame. On Mon, Jan 26, 2015 at 2:18

Re: renaming SchemaRDD - DataFrame

2015-02-10 Thread Reynold Xin
pwend...@gmail.com To: Reynold Xin r...@databricks.com Cc: dev@spark.apache.org dev@spark.apache.org Sent: Monday, January 26, 2015 4:01 PM Subject: Re: renaming SchemaRDD - DataFrame One thing potentially not clear from this e-mail, there will be a 1

Re: renaming SchemaRDD - DataFrame

2015-02-10 Thread Matei Zaharia
, January 26, 2015 4:01 PM Subject: Re: renaming SchemaRDD - DataFrame One thing potentially not clear from this e-mail, there will be a 1:1 correspondence where you can get an RDD to/from a DataFrame. On Mon, Jan 26, 2015 at 2:18 PM, Reynold Xin r...@databricks.com

Re: renaming SchemaRDD - DataFrame

2015-02-10 Thread Reynold Xin
://www.youtube.com/watch?v=YWppYPWznSQ From: Patrick Wendell pwend...@gmail.com To: Reynold Xin r...@databricks.com Cc: dev@spark.apache.org dev@spark.apache.org Sent: Monday, January 26, 2015 4:01 PM Subject: Re: renaming SchemaRDD - DataFrame

Re: renaming SchemaRDD - DataFrame

2015-02-10 Thread Koert Kuipers
: renaming SchemaRDD - DataFrame One thing potentially not clear from this e-mail, there will be a 1:1 correspondence where you can get an RDD to/from a DataFrame. On Mon, Jan 26, 2015 at 2:18 PM, Reynold Xin r...@databricks.com wrote: Hi, We

Re: renaming SchemaRDD - DataFrame

2015-02-01 Thread Evan Chan
: renaming SchemaRDD - DataFrame One thing potentially not clear from this e-mail, there will be a 1:1 correspondence where you can get an RDD to/from a DataFrame. On Mon, Jan 26, 2015 at 2:18 PM, Reynold Xin r...@databricks.com wrote: Hi, We are considering renaming

Re: renaming SchemaRDD - DataFrame

2015-01-29 Thread Evan Chan
: renaming SchemaRDD - DataFrame One thing potentially not clear from this e-mail, there will be a 1:1 correspondence where you can get an RDD to/from a DataFrame. On Mon, Jan 26, 2015 at 2:18 PM, Reynold Xin r...@databricks.com wrote

Re: renaming SchemaRDD - DataFrame

2015-01-29 Thread Cheng Lian
://www.youtube.com/watch?v=YWppYPWznSQ From: Patrick Wendell pwend...@gmail.com To: Reynold Xin r...@databricks.com Cc: dev@spark.apache.org dev@spark.apache.org Sent: Monday, January 26, 2015 4:01 PM Subject: Re: renaming SchemaRDD - DataFrame One thing potentially

Re: renaming SchemaRDD - DataFrame

2015-01-29 Thread Koert Kuipers
: Monday, January 26, 2015 4:01 PM Subject: Re: renaming SchemaRDD - DataFrame One thing potentially not clear from this e-mail, there will be a 1:1 correspondence where you can get an RDD to/from a DataFrame. On Mon, Jan 26

Re: renaming SchemaRDD - DataFrame

2015-01-29 Thread Cheng Lian
From: Patrick Wendell pwend...@gmail.com To: Reynold Xin r...@databricks.com Cc: dev@spark.apache.org dev@spark.apache.org Sent: Monday, January 26, 2015 4:01 PM Subject: Re: renaming SchemaRDD - DataFrame One thing potentially not clear from this e-mail

Re: renaming SchemaRDD - DataFrame

2015-01-28 Thread Reynold Xin
From: Patrick Wendell pwend...@gmail.com To: Reynold Xin r...@databricks.com Cc: dev@spark.apache.org dev@spark.apache.org Sent: Monday, January 26, 2015 4:01 PM Subject: Re: renaming SchemaRDD - DataFrame One thing potentially not clear from this e-mail

Re: renaming SchemaRDD - DataFrame

2015-01-28 Thread Evan Chan
@spark.apache.org Sent: Monday, January 26, 2015 4:01 PM Subject: Re: renaming SchemaRDD - DataFrame One thing potentially not clear from this e-mail, there will be a 1:1 correspondence where you can get an RDD to/from a DataFrame. On Mon, Jan 26, 2015 at 2:18 PM

Re: renaming SchemaRDD - DataFrame

2015-01-28 Thread Evan Chan
:-). https://www.youtube.com/watch?v=YWppYPWznSQ From: Patrick Wendell pwend...@gmail.com To: Reynold Xin r...@databricks.com Cc: dev@spark.apache.org dev@spark.apache.org Sent: Monday, January 26, 2015 4:01 PM Subject: Re: renaming SchemaRDD - DataFrame

Re: renaming SchemaRDD - DataFrame

2015-01-28 Thread Michael Armbrust
Subject: Re: renaming SchemaRDD - DataFrame One thing potentially not clear from this e-mail, there will be a 1:1 correspondence where you can get an RDD to/from a DataFrame. On Mon, Jan 26, 2015 at 2:18 PM, Reynold Xin r

Re: renaming SchemaRDD - DataFrame

2015-01-28 Thread Evan Chan
From: Patrick Wendell pwend...@gmail.com To: Reynold Xin r...@databricks.com Cc: dev@spark.apache.org dev@spark.apache.org Sent: Monday, January 26, 2015 4:01 PM Subject: Re: renaming SchemaRDD - DataFrame One thing potentially not clear from this e-mail

Re: renaming SchemaRDD - DataFrame

2015-01-27 Thread Reynold Xin
Subject: Re: renaming SchemaRDD - DataFrame One thing potentially not clear from this e-mail, there will be a 1:1 correspondence where you can get an RDD to/from a DataFrame. On Mon, Jan 26, 2015 at 2:18 PM, Reynold Xin r...@databricks.com wrote: Hi, We are considering renaming

Re: renaming SchemaRDD - DataFrame

2015-01-27 Thread Dirceu Semighini Filho
@spark.apache.org Sent: Monday, January 26, 2015 4:01 PM Subject: Re: renaming SchemaRDD - DataFrame One thing potentially not clear from this e-mail, there will be a 1:1 correspondence where you can get an RDD to/from a DataFrame. On Mon, Jan 26, 2015 at 2:18

Re: renaming SchemaRDD - DataFrame

2015-01-27 Thread Reynold Xin
26, 2015 4:01 PM Subject: Re: renaming SchemaRDD - DataFrame One thing potentially not clear from this e-mail, there will be a 1:1 correspondence where you can get an RDD to/from a DataFrame. On Mon, Jan 26, 2015 at 2:18 PM, Reynold Xin r

Re: renaming SchemaRDD - DataFrame

2015-01-27 Thread Koert Kuipers
pwend...@gmail.com To: Reynold Xin r...@databricks.com Cc: dev@spark.apache.org dev@spark.apache.org Sent: Monday, January 26, 2015 4:01 PM Subject: Re: renaming SchemaRDD - DataFrame One thing potentially not clear from this e-mail, there will be a 1:1 correspondence where you can

Re: renaming SchemaRDD - DataFrame

2015-01-27 Thread Dmitriy Lyubimov
It has been pretty evident for some time that's what it is, hasn't it? Yes that's a better name IMO. On Mon, Jan 26, 2015 at 2:18 PM, Reynold Xin r...@databricks.com wrote: Hi, We are considering renaming SchemaRDD - DataFrame in 1.3, and wanted to get the community's opinion. The context

Re: renaming SchemaRDD - DataFrame

2015-01-27 Thread Matei Zaharia
: Patrick Wendell pwend...@gmail.com To: Reynold Xin r...@databricks.com Cc: dev@spark.apache.org dev@spark.apache.org Sent: Monday, January 26, 2015 4:01 PM Subject: Re: renaming SchemaRDD - DataFrame One thing potentially not clear from this e-mail, there will be a 1:1 correspondence where

Re: renaming SchemaRDD - DataFrame

2015-01-27 Thread Evan R. Sparks
...@databricks.com Cc: dev@spark.apache.org dev@spark.apache.org Sent: Monday, January 26, 2015 4:01 PM Subject: Re: renaming SchemaRDD - DataFrame One thing potentially not clear from this e-mail, there will be a 1:1 correspondence where you can get an RDD to/from a DataFrame

Re: renaming SchemaRDD - DataFrame

2015-01-27 Thread Mark Hamstra
Cc: dev@spark.apache.org dev@spark.apache.org Sent: Monday, January 26, 2015 4:01 PM Subject: Re: renaming SchemaRDD - DataFrame One thing potentially not clear from this e-mail, there will be a 1:1 correspondence where you can get an RDD to/from a DataFrame. On Mon, Jan 26

Re: renaming SchemaRDD - DataFrame

2015-01-27 Thread Michael Malak
, January 27, 2015 9:55 AM Subject: Re: renaming SchemaRDD - DataFrame I'm +1 on this, although a little worried about unknowingly introducing SparkSQL dependencies every time someone wants to use this. It would be great if the interface can be abstract and the implementation (in this case, SparkSQL

Re: renaming SchemaRDD - DataFrame

2015-01-26 Thread Matei Zaharia
Wendell pwend...@gmail.com To: Reynold Xin r...@databricks.com Cc: dev@spark.apache.org dev@spark.apache.org Sent: Monday, January 26, 2015 4:01 PM Subject: Re: renaming SchemaRDD - DataFrame One thing potentially not clear from this e-mail, there will be a 1:1 correspondence where you can

Re: renaming SchemaRDD - DataFrame

2015-01-26 Thread Matei Zaharia
From: Patrick Wendell pwend...@gmail.com To: Reynold Xin r...@databricks.com Cc: dev@spark.apache.org dev@spark.apache.org Sent: Monday, January 26, 2015 4:01 PM Subject: Re: renaming SchemaRDD - DataFrame One thing potentially not clear from this e-mail, there will be a 1:1

Re: renaming SchemaRDD - DataFrame

2015-01-26 Thread Kushal Datta
@spark.apache.org Sent: Monday, January 26, 2015 4:01 PM Subject: Re: renaming SchemaRDD - DataFrame One thing potentially not clear from this e-mail, there will be a 1:1 correspondence where you can get an RDD to/from a DataFrame. On Mon, Jan 26, 2015 at 2:18 PM, Reynold Xin r

renaming SchemaRDD - DataFrame

2015-01-26 Thread Reynold Xin
Hi, We are considering renaming SchemaRDD - DataFrame in 1.3, and wanted to get the community's opinion. The context is that SchemaRDD is becoming a common data format used for bringing data into Spark from external systems, and used for various components of Spark, e.g. MLlib's new pipeline API

Re: renaming SchemaRDD - DataFrame

2015-01-26 Thread Koert Kuipers
From: Patrick Wendell pwend...@gmail.com To: Reynold Xin r...@databricks.com Cc: dev@spark.apache.org dev@spark.apache.org Sent: Monday, January 26, 2015 4:01 PM Subject: Re: renaming SchemaRDD - DataFrame One thing potentially not clear from this e-mail