[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-21 Thread AmplabJenkins 
Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11262#issuecomment-186986677
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-21 Thread AmplabJenkins 
Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11262#issuecomment-186986680
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/51651/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-21 Thread SparkQA 
Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11262#issuecomment-186986183
  
**[Test build #51651 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51651/consoleFull)**
 for PR 11262 at commit 
[`3a39625`](https://github.com/apache/spark/commit/3a3962590890a9dbb9f65ec2115f495ba242fe8e).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-21 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/11262


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-21 Thread rxin 
Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/11262#issuecomment-186980335
  
Thanks - merging in master.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-21 Thread SparkQA 
Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11262#issuecomment-186973763
  
**[Test build #2563 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2563/consoleFull)**
 for PR 11262 at commit 
[`3a39625`](https://github.com/apache/spark/commit/3a3962590890a9dbb9f65ec2115f495ba242fe8e).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-21 Thread AmplabJenkins 
Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11262#issuecomment-186971819
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-21 Thread AmplabJenkins 
Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11262#issuecomment-186971820
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/51648/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-21 Thread SparkQA 
Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11262#issuecomment-186971303
  
**[Test build #51648 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51648/consoleFull)**
 for PR 11262 at commit 
[`06a660e`](https://github.com/apache/spark/commit/06a660ee37a9babeb107a7dfdf7d63a43070330f).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-21 Thread SparkQA 
Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11262#issuecomment-186969215
  
**[Test build #51651 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51651/consoleFull)**
 for PR 11262 at commit 
[`3a39625`](https://github.com/apache/spark/commit/3a3962590890a9dbb9f65ec2115f495ba242fe8e).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-21 Thread HyukjinKwon 
Github user HyukjinKwon commented on the pull request:

https://github.com/apache/spark/pull/11262#issuecomment-186967397
  
retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-21 Thread AmplabJenkins 
Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11262#issuecomment-186960032
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-21 Thread AmplabJenkins 
Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11262#issuecomment-186960034
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/51649/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-21 Thread SparkQA 
Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11262#issuecomment-186957557
  
**[Test build #2563 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2563/consoleFull)**
 for PR 11262 at commit 
[`3a39625`](https://github.com/apache/spark/commit/3a3962590890a9dbb9f65ec2115f495ba242fe8e).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-21 Thread rxin 
Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/11262#issuecomment-186957384
  
LGTM pending tests.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-21 Thread HyukjinKwon 
Github user HyukjinKwon commented on the pull request:

https://github.com/apache/spark/pull/11262#issuecomment-186956637
  
Sure.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-21 Thread SparkQA 
Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11262#issuecomment-186954558
  
**[Test build #51648 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51648/consoleFull)**
 for PR 11262 at commit 
[`06a660e`](https://github.com/apache/spark/commit/06a660ee37a9babeb107a7dfdf7d63a43070330f).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-21 Thread rxin 
Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/11262#issuecomment-186954500
  
Can you also update the pr description?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-21 Thread rxin 
Github user rxin commented on a diff in the pull request:

https://github.com/apache/spark/pull/11262#discussion_r53578002
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala ---
@@ -345,6 +346,46 @@ class DataFrameReader private[sql](sqlContext: 
SQLContext) extends Logging {
   }
 
   /**
+   * Loads a CSV file and returns the result as a [[DataFrame]].
+   *
+   * This function goes through the input once to determine the input 
schema. If you know the
+   * schema in advance, use the version that specifies the schema to avoid 
the extra scan.
--- End diff --

"If you know the schema in advance, use the version that specifies the 
schema to avoid the extra scan."

There is no version that does that. Maybe just say "To avoid going through 
the entire data once, specify the schema explicitly using [[schema]]."


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-21 Thread HyukjinKwon 
Github user HyukjinKwon commented on a diff in the pull request:

https://github.com/apache/spark/pull/11262#discussion_r53577852
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala ---
@@ -345,6 +346,46 @@ class DataFrameReader private[sql](sqlContext: 
SQLContext) extends Logging {
   }
 
   /**
+   * Loads a CSV file and returns the result as a [[DataFrame]].
+   *
+   * This function goes through the input once to determine the input 
schema. If you know the
+   * schema in advance, use the version that specifies the schema to avoid 
the extra scan.
--- End diff --

Filed [here](https://issues.apache.org/jira/browse/SPARK-13425)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-19 Thread rxin 
Github user rxin commented on a diff in the pull request:

https://github.com/apache/spark/pull/11262#discussion_r53508815
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala ---
@@ -345,6 +346,46 @@ class DataFrameReader private[sql](sqlContext: 
SQLContext) extends Logging {
   }
 
   /**
+   * Loads a CSV file and returns the result as a [[DataFrame]].
+   *
+   * This function goes through the input once to determine the input 
schema. If you know the
+   * schema in advance, use the version that specifies the schema to avoid 
the extra scan.
--- End diff --

let's also create a 2.0-blocker jira for documenting all the csv options 
before we release 2.0.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-19 Thread rxin 
Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/11262#issuecomment-186374116
  
I'd keep only the csv for paths, and remove the rest. Also we should add a 
Python version, in which we can have a lot of options directly built-in.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-19 Thread rxin 
Github user rxin commented on a diff in the pull request:

https://github.com/apache/spark/pull/11262#discussion_r53508447
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala ---
@@ -345,6 +346,46 @@ class DataFrameReader private[sql](sqlContext: 
SQLContext) extends Logging {
   }
 
   /**
+   * Loads a CSV file and returns the result as a [[DataFrame]].
+   *
+   * This function goes through the input once to determine the input 
schema. If you know the
+   * schema in advance, use the version that specifies the schema to avoid 
the extra scan.
+   *
+   * @since 2.0.0
+   */
+  def csv(paths: String*): DataFrame = format("csv").load(paths : _*)
--- End diff --

I'd keep only the path one, and remove the rest.

Also you need to add the scala annotation to enable calling this in Java.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-19 Thread HyukjinKwon 
Github user HyukjinKwon commented on the pull request:

https://github.com/apache/spark/pull/11262#issuecomment-186173846
  
cc @rxin 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-19 Thread AmplabJenkins 
Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11262#issuecomment-186173761
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/51547/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-19 Thread AmplabJenkins 
Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11262#issuecomment-186173759
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-19 Thread SparkQA 
Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11262#issuecomment-186173240
  
**[Test build #51547 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51547/consoleFull)**
 for PR 11262 at commit 
[`b144354`](https://github.com/apache/spark/commit/b1443543d3952242fcd458d582703d2c563e274d).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-19 Thread SparkQA 
Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11262#issuecomment-186139322
  
**[Test build #51547 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51547/consoleFull)**
 for PR 11262 at commit 
[`b144354`](https://github.com/apache/spark/commit/b1443543d3952242fcd458d582703d2c563e274d).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-19 Thread AmplabJenkins 
Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11262#issuecomment-186124036
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-19 Thread AmplabJenkins 
Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11262#issuecomment-186124043
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/51537/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-19 Thread SparkQA 
Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11262#issuecomment-186123881
  
**[Test build #51537 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51537/consoleFull)**
 for PR 11262 at commit 
[`fd2507c`](https://github.com/apache/spark/commit/fd2507ccfa147e80dea78d8db2948e7afec530db).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-18 Thread SparkQA 
Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11262#issuecomment-186108498
  
**[Test build #51537 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51537/consoleFull)**
 for PR 11262 at commit 
[`fd2507c`](https://github.com/apache/spark/commit/fd2507ccfa147e80dea78d8db2948e7afec530db).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13381][SQL] Support for loading CSV wit...

2016-02-18 Thread HyukjinKwon 
GitHub user HyukjinKwon opened a pull request:

https://github.com/apache/spark/pull/11262

[SPARK-13381][SQL] Support for loading CSV with a single function call

https://issues.apache.org/jira/browse/SPARK-13381

This PR adds the support to load CSV data directly by a single call.

Three functions are added to load them.
1. Load csv data from `paths`
2. Load csv data from `RDD[String]`
3. Load csv data from `JavaRDD[String]`

Also, I corrected this to refer all paths rather than the first path in 
schema inference, which JSON datasource dose.

Several unitests were added for each functionality.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/HyukjinKwon/spark SPARK-13381

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/11262.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #11262


commit fd2507ccfa147e80dea78d8db2948e7afec530db
Author: hyukjinkwon 
Date:   2016-02-19T07:31:08Z

Add an interface to load CSV directly by a single function call.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org