[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/3470


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[scwf]

Github user scwf commented on the pull request:

https://github.com/apache/spark/pull/3470#issuecomment-67260582
  
Updated


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/3470#issuecomment-67259760
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24514/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/3470#issuecomment-67259754
  
  [Test build #24514 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24514/consoleFull)
 for   PR 3470 at commit 
[`ae78509`](https://github.com/apache/spark/commit/ae7850947363b82aec126a09793ebda284b71544).
 * This patch **passes all tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/3470#issuecomment-67252279
  
  [Test build #24514 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24514/consoleFull)
 for   PR 3470 at commit 
[`ae78509`](https://github.com/apache/spark/commit/ae7850947363b82aec126a09793ebda284b71544).
 * This patch merges cleanly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[marmbrus]

Github user marmbrus commented on the pull request:

https://github.com/apache/spark/pull/3470#issuecomment-67247835
  
Minor style comments otherwise LGTM.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[marmbrus]

Github user marmbrus commented on a diff in the pull request:

https://github.com/apache/spark/pull/3470#discussion_r21939291
  
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/ddl.scala ---
@@ -106,3 +106,24 @@ private[sql] case class CreateTableUsing(
 Seq.empty
   }
 }
+
+/**
+ * Builds a map in which keys are case insensitive
+ */
+object CaseInsensitiveMap {
+  def apply(kvs: Seq[(String, String)]) = new CaseInsensitiveMap(kvs.toMap)
--- End diff --

I would either do the work of converting the map in the factory method or 
not have a factory method.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[marmbrus]

Github user marmbrus commented on a diff in the pull request:

https://github.com/apache/spark/pull/3470#discussion_r21939209
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala ---
@@ -36,7 +36,10 @@ import 
org.apache.spark.sql.catalyst.expressions.{Expression, Attribute}
  */
 @DeveloperApi
 trait RelationProvider {
-  /** Returns a new base relation with the given parameters. */
+  /**
+   * Returns a new base relation with the given parameters.
+   * Note: the parameters'keywords are case insensitive.
--- End diff --

Space after `'` and add the phrase "and this insensitivity is enforced by 
the `Map` that is passed to the function."


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[marmbrus]

Github user marmbrus commented on a diff in the pull request:

https://github.com/apache/spark/pull/3470#discussion_r21939140
  
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/ddl.scala ---
@@ -106,3 +106,24 @@ private[sql] case class CreateTableUsing(
 Seq.empty
   }
 }
+
+/**
+ * Builds a map in which keys are case insensitive
+ */
+object CaseInsensitiveMap {
+  def apply(kvs: Seq[(String, String)]) = new CaseInsensitiveMap(kvs.toMap)
+}
+
+class CaseInsensitiveMap(_baseMap: Map[String, String]) extends 
Map[String, String] {
--- End diff --

Lets make both the class and the object `protected`.  Also we don't 
generally use the `_underscore` convention in Spark SQL.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[scwf]

Github user scwf commented on the pull request:

https://github.com/apache/spark/pull/3470#issuecomment-66858359
  
@marmbrus, updated, is this ok


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/3470#issuecomment-66742685
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24393/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/3470#issuecomment-66742683
  
  [Test build #24393 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24393/consoleFull)
 for   PR 3470 at commit 
[`8f4f585`](https://github.com/apache/spark/commit/8f4f585711a9b860db10f4db6ca92ac0522475f3).
 * This patch **passes all tests**.
 * This patch merges cleanly.
 * This patch adds the following public classes _(experimental)_:
  * `class CaseInsensitiveMap(_baseMap: Map[String, String]) extends 
Map[String, String] `



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/3470#issuecomment-66737945
  
  [Test build #24393 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24393/consoleFull)
 for   PR 3470 at commit 
[`8f4f585`](https://github.com/apache/spark/commit/8f4f585711a9b860db10f4db6ca92ac0522475f3).
 * This patch merges cleanly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[scwf]

Github user scwf commented on a diff in the pull request:

https://github.com/apache/spark/pull/3470#discussion_r21350936
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala ---
@@ -37,7 +37,7 @@ import 
org.apache.spark.sql.catalyst.expressions.{Expression, Attribute}
 @DeveloperApi
 trait RelationProvider {
   /** Returns a new base relation with the given parameters. */
-  def createRelation(sqlContext: SQLContext, parameters: Map[String, 
String]): BaseRelation
+  def createRelation(sqlContext: SQLContext, parameters: 
CaseInsensitiveMap): BaseRelation
--- End diff --

1 About ```spark.sql.caseSensitive```, my idea is that we do not need make 
attributes and table names case insensitive, just like hive. Configurable there 
make things complex.  
2 "You can still pass your CaseInsensitiveMap in and it will have the 
desired effect. Just don't change the function signature."
You mean like this?
```
class DefaultSource extends RelationProvider {
  /** Returns a new base relation with the given parameters. */
  override def createRelation(
  sqlContext: SQLContext,
  parameters: Map[String, String]): BaseRelation = {
val path = new CaseInsensitiveMap(parameters)
  .getOrElse("path", sys.error("'path' must be specified for parquet 
tables."))

ParquetRelation2(path)(sqlContext)
  }
}
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[marmbrus]

Github user marmbrus commented on a diff in the pull request:

https://github.com/apache/spark/pull/3470#discussion_r21341852
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala ---
@@ -37,7 +37,7 @@ import 
org.apache.spark.sql.catalyst.expressions.{Expression, Attribute}
 @DeveloperApi
 trait RelationProvider {
   /** Returns a new base relation with the given parameters. */
-  def createRelation(sqlContext: SQLContext, parameters: Map[String, 
String]): BaseRelation
+  def createRelation(sqlContext: SQLContext, parameters: 
CaseInsensitiveMap): BaseRelation
--- End diff --

`spark.sql.caseSensitive` is about identifiers (i.e.,  attributes and table 
names).  I'd say this is more analogous to keyword case insensitivity.  I don't 
know any database that doesn't treat `SELECT` and `select` the same so I'm not 
sure if that should be configurable.

You can still pass your `CaseInsensitiveMap ` in and it will have the 
desired effect.  Just don't change the function signature.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[scwf]

Github user scwf commented on a diff in the pull request:

https://github.com/apache/spark/pull/3470#discussion_r21201664
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala ---
@@ -37,7 +37,7 @@ import 
org.apache.spark.sql.catalyst.expressions.{Expression, Attribute}
 @DeveloperApi
 trait RelationProvider {
   /** Returns a new base relation with the given parameters. */
-  def createRelation(sqlContext: SQLContext, parameters: Map[String, 
String]): BaseRelation
+  def createRelation(sqlContext: SQLContext, parameters: 
CaseInsensitiveMap): BaseRelation
--- End diff --

Hmm, how about revert to ```Map``` here and make case insensitive 
configurable, so we can write like this:
```
private[sql] class DefaultSource extends RelationProvider {
  /** Returns a new base relation with the given parameters. */
  override def createRelation(
  sqlContext: SQLContext,
  parameters: Map[String, String]): BaseRelation = {
val caseSensitive = sqlContext.getConf("spark.sql.caseSensitive", 
"false").toBoolean
val paras =
if (caseSensitive) {
  parameters
} else {
  new CaseInsensitiveMap(parameters)
}
val fileName = paras.getOrElse("path", sys.error("Option 'path' not 
specified"))
val samplingRatio = 
paras.get("samplingRatio").map(_.toDouble).getOrElse(1.0)

JSONRelation(fileName, samplingRatio)(sqlContext)
  }
}
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[marmbrus]

Github user marmbrus commented on a diff in the pull request:

https://github.com/apache/spark/pull/3470#discussion_r21183110
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala ---
@@ -37,7 +37,7 @@ import 
org.apache.spark.sql.catalyst.expressions.{Expression, Attribute}
 @DeveloperApi
 trait RelationProvider {
   /** Returns a new base relation with the given parameters. */
--- End diff --

We need to document that parameters is case insensitive here.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[marmbrus]

Github user marmbrus commented on a diff in the pull request:

https://github.com/apache/spark/pull/3470#discussion_r21183040
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala ---
@@ -37,7 +37,7 @@ import 
org.apache.spark.sql.catalyst.expressions.{Expression, Attribute}
 @DeveloperApi
 trait RelationProvider {
   /** Returns a new base relation with the given parameters. */
-  def createRelation(sqlContext: SQLContext, parameters: Map[String, 
String]): BaseRelation
+  def createRelation(sqlContext: SQLContext, parameters: 
CaseInsensitiveMap): BaseRelation
--- End diff --

I don't know if we actually want to change the signature here. This is a 
public API, and while 1.2 hasn't been officially released, some libraries have 
already been published against the original signature.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/3470#issuecomment-65265133
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24048/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/3470#issuecomment-65265127
  
  [Test build #24048 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24048/consoleFull)
 for   PR 3470 at commit 
[`3c132ef`](https://github.com/apache/spark/commit/3c132efa9747448b73605e83594d4ae161b7b86e).
 * This patch **passes all tests**.
 * This patch merges cleanly.
 * This patch adds the following public classes _(experimental)_:
  * `class CaseInsensitiveMap(_baseMap: Map[String, String]) extends 
Map[String, String] `



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/3470#issuecomment-65253492
  
  [Test build #24048 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24048/consoleFull)
 for   PR 3470 at commit 
[`3c132ef`](https://github.com/apache/spark/commit/3c132efa9747448b73605e83594d4ae161b7b86e).
 * This patch merges cleanly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[scwf]

Github user scwf commented on the pull request:

https://github.com/apache/spark/pull/3470#issuecomment-65184315
  
Hi @liancheng, diff of options in datasource API and Hive table properties 
is there are some options very like Keywords and from users they want them case 
insensitive. After we make them case insensitive users can write ```PATH``` for 
```path``` and also dotted option names work.  


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[liancheng]

Github user liancheng commented on the pull request:

https://github.com/apache/spark/pull/3470#issuecomment-65182946
  
Actually I'm not sure whether making option names case insensitive a good 
idea. Semantically, these options are very similar to Hive table properties, 
which are case sensitive. This makes me think these options should be case 
sensitive at the very beginning. For simple options that look like keywords, 
case insensitive might make sense. But we probably want to add dotted option 
names like `partition.defaultName` in the future. Another reason is that, case 
insensitivity is always a source of bugs...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[rxin]

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/3470#issuecomment-65175457
  
That definitely seems like the better option to me. It makes the options 
universally lower case.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[marmbrus]

Github user marmbrus commented on the pull request:

https://github.com/apache/spark/pull/3470#issuecomment-65175244
  
Yeah, that's what I was thinking when I said case insensitive map.
On Dec 1, 2014 6:33 PM, "Reynold Xin"  wrote:

> How about moving the toLowerCase into the get funciton itself?
>
> —
> Reply to this email directly or view it on GitHub
> .
>


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[scwf]

Github user scwf commented on the pull request:

https://github.com/apache/spark/pull/3470#issuecomment-65175161
  
Yes, We can implement a  case insensitive map and in this map's ```get``` 
function we use toLowercase


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[rxin]

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/3470#issuecomment-65174831
  
How about moving the toLowerCase into the get funciton itself?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[marmbrus]

Github user marmbrus commented on the pull request:

https://github.com/apache/spark/pull/3470#issuecomment-65157328
  
I think this is probably a reasonable semantic to have given keywords are 
not generally case sensitive in SQL, but I think that we need to add some 
documentation to `DataSource` that says that this is the case.  You might even 
go so far as to provide a case insensitive map to avoid confusion when 
developers try to do lookups with keywords that contain capital letters.

thoughts @liancheng @rxin?

Also, given the current implementation, I would just statically lowercase 
samplingRatio instead of using toLower.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/3470#issuecomment-64519815
  
  [Test build #23877 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23877/consoleFull)
 for   PR 3470 at commit 
[`e244e8d`](https://github.com/apache/spark/commit/e244e8d40112991bb4a4bf6c73a3b8ee5bbcf05d).
 * This patch **passes all tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/3470#issuecomment-64519818
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23877/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/3470#issuecomment-64515778
  
  [Test build #23877 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23877/consoleFull)
 for   PR 3470 at commit 
[`e244e8d`](https://github.com/apache/spark/commit/e244e8d40112991bb4a4bf6c73a3b8ee5bbcf05d).
 * This patch merges cleanly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-4618][SQL] Make foreign DDL commands op...

[scwf]

GitHub user scwf opened a pull request:

https://github.com/apache/spark/pull/3470

[SPARK-4618][SQL] Make foreign DDL commands options case-insensitive

Using lowercase for ```options``` key to make it case-insensitive, we 
should use lower case to get value from parameters.
So flowing cmd work
```
  create temporary table normal_parquet
  USING org.apache.spark.sql.parquet
  OPTIONS (
PATH '/xxx/data'
  )
``` 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/scwf/spark ddl-ulcase

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/3470.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3470


commit e0cb0171a30dc8c0b56a05687a3f651436acded1
Author: wangfei 
Date:   2014-11-26T04:47:29Z

make options in-casesensitive

commit e244e8d40112991bb4a4bf6c73a3b8ee5bbcf05d
Author: wangfei 
Date:   2014-11-26T04:52:50Z

using lower case in json




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org