[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[bdwyer2]

Github user bdwyer2 commented on the issue:

https://github.com/apache/spark/pull/16247
  
@shivaram @felixcheung I'll close this PR so that one of you can take over 
in order to have it done in time for the RC.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[shivaram]

Github user shivaram commented on the issue:

https://github.com/apache/spark/pull/16247
  
@bdwyer2 Let us know if you have problems setting up the environment - If 
so me or @felixcheung can open a new PR that includes your changes (we can 
still assign the JIRA as your contribution)

The reason I ask is that it will be good to get this in before the RC3 cut 
as this helps the SparkR CRAN release etc.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[felixcheung]

Github user felixcheung commented on the issue:

https://github.com/apache/spark/pull/16247
  
yes, something like what's being used here 
https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/internal/SharedState.scala#L49



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[bdwyer2]

Github user bdwyer2 commented on the issue:

https://github.com/apache/spark/pull/16247
  
How would we access that value on the scala side? Would 
`sparkContext.hadoopConfiguration.get("spark.sql.warehouse.default.dir")` work?

I'm currently unable to compile Spark which makes experimenting with scala 
difficult.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[felixcheung]

Github user felixcheung commented on the issue:

https://github.com/apache/spark/pull/16247
  
But yes _other_ Spark config properties would be set by the user in 
sparkConfig parameter of sparkR.session method. We would just add to that like
```
sparkConfig[["spark.sql.warehouse.default.dir"]] <- tempdir()
```
without adding another parameter to sparkR.session()




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[felixcheung]

Github user felixcheung commented on the issue:

https://github.com/apache/spark/pull/16247
  
But yes other Spark config properties would be set by the user in 
sparkConfig parameter of sparkR.session method. We would just add to that 
without adding another parameter to sparkR.session




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[felixcheung]

Github user felixcheung commented on the issue:

https://github.com/apache/spark/pull/16247
  
I mean it as something we set to the SparkContext or SparkSession and not a 
parameter of sparkR.session().




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/16247
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70093/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/16247
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/16247
  
**[Test build #70093 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70093/consoleFull)**
 for PR 16247 at commit 
[`9142397`](https://github.com/apache/spark/commit/9142397b39ff5886cf51a4255591251241d103b8).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/16247
  
**[Test build #70093 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70093/consoleFull)**
 for PR 16247 at commit 
[`9142397`](https://github.com/apache/spark/commit/9142397b39ff5886cf51a4255591251241d103b8).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[shivaram]

Github user shivaram commented on the issue:

https://github.com/apache/spark/pull/16247
  
Yeah disabling Hive for the test is fine. @bdwyer2 Can you add the new 
config flag as well ? We can do one final pass of review after that


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[bdwyer2]

Github user bdwyer2 commented on the issue:

https://github.com/apache/spark/pull/16247
  
Would calling the test with this be an acceptable solution?
```R
sparkR.session(enableHiveSupport = FALSE)
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[felixcheung]

Github user felixcheung commented on the issue:

https://github.com/apache/spark/pull/16247
  
Possibly,  SPARK-16027 was just a hack, the root issue remains I think


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[HyukjinKwon]

Github user HyukjinKwon commented on the issue:

https://github.com/apache/spark/pull/16247
  
This test failure seems related with this PR. It seems because the previous 
Hive-enabled spark session is not closed properly between `test_saprkSQL.R` and 
`test_sparkR.R` here, in particular, I suspect the lock in derby via Hive 
client. It seems apparently related with SPARK-16027?

There was only single instance of Hive-enabled spark session there but the 
test here introduces another one. I manually tested for each after removing 
each other and it seems working fine.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[felixcheung]

Github user felixcheung commented on the issue:

https://github.com/apache/spark/pull/16247
  
re: test failure. it might be related to this change? the call stack is 
hidden, it should be trying to call into 
https://github.com/apache/spark/blob/master/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionState.scala
 and failed.

Is it possible that tmpdir() is not writeable on Jenkins?



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[cloud-fan]

Github user cloud-fan commented on the issue:

https://github.com/apache/spark/pull/16247
  
a new property for default warehouse LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/16247
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/16247
  
**[Test build #70047 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70047/consoleFull)**
 for PR 16247 at commit 
[`84a110e`](https://github.com/apache/spark/commit/84a110e138601173a3d4b42cedcdb8db597fa07d).
 * This patch **fails SparkR unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/16247
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70047/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/16247
  
**[Test build #70047 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70047/consoleFull)**
 for PR 16247 at commit 
[`84a110e`](https://github.com/apache/spark/commit/84a110e138601173a3d4b42cedcdb8db597fa07d).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/16247
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70040/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/16247
  
**[Test build #70040 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70040/consoleFull)**
 for PR 16247 at commit 
[`17b7bff`](https://github.com/apache/spark/commit/17b7bffcc8dbbe3c0d91b1ee9cccff56d038e8ba).
 * This patch **fails SparkR unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/16247
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/16247
  
**[Test build #70040 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70040/consoleFull)**
 for PR 16247 at commit 
[`17b7bff`](https://github.com/apache/spark/commit/17b7bffcc8dbbe3c0d91b1ee9cccff56d038e8ba).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[jodersky]

Github user jodersky commented on the issue:

https://github.com/apache/spark/pull/16247
  
jenkins, retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[bdwyer2]

Github user bdwyer2 commented on the issue:

https://github.com/apache/spark/pull/16247
  
I don't see how my last commit could of caused this
```
functions in sparkR.R: .
SparkSQL functions: Spark package found in SPARK_HOME: 
/home/jenkins/workspace/SparkPullRequestBuilder
Error in handleErrors(returnStatus, conn) : 
  java.lang.IllegalArgumentException: Error while instantiating 
'org.apache.spark.sql.hive.HiveSessionState':
at 
org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:981)
at 
org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
at 
org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
at 
org.apache.spark.sql.api.r.SQLUtils$$anonfun$setSparkContextSessionConf$2.apply(SQLUtils.scala:67)
at 
org.apache.spark.sql.api.r.SQLUtils$$anonfun$setSparkContextSessionConf$2.apply(SQLUtils.scala:66)
at 
scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:733)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at scala.collection.TraversableLike$WithF
Calls: test_package ... sparkR.session -> callJStatic -> invokeJava -> 
handleErrors
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/16247
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70035/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/16247
  
**[Test build #70035 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70035/consoleFull)**
 for PR 16247 at commit 
[`17b7bff`](https://github.com/apache/spark/commit/17b7bffcc8dbbe3c0d91b1ee9cccff56d038e8ba).
 * This patch **fails SparkR unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/16247
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/16247
  
**[Test build #70035 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70035/consoleFull)**
 for PR 16247 at commit 
[`17b7bff`](https://github.com/apache/spark/commit/17b7bffcc8dbbe3c0d91b1ee9cccff56d038e8ba).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/16247
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70032/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/16247
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/16247
  
**[Test build #70032 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70032/consoleFull)**
 for PR 16247 at commit 
[`cee3944`](https://github.com/apache/spark/commit/cee39446899387fa40c3e853cd53d3c0d043c4bd).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/16247
  
**[Test build #70032 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70032/consoleFull)**
 for PR 16247 at commit 
[`cee3944`](https://github.com/apache/spark/commit/cee39446899387fa40c3e853cd53d3c0d043c4bd).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[shivaram]

Github user shivaram commented on the issue:

https://github.com/apache/spark/pull/16247
  
@bdwyer2 The test case idea sounds good ! 

Regarding the conf naming for warehouse dir lets also check with 
contributors who are more familiar with SQL. 
cc @yhuai @cloud-fan 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[bdwyer2]

Github user bdwyer2 commented on the issue:

https://github.com/apache/spark/pull/16247
  
@shivaram I can create a test to verify the output of `list.files()` is the 
same before and after running `sparkR.session()`.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[felixcheung]

Github user felixcheung commented on the issue:

https://github.com/apache/spark/pull/16247
  
... that's actually what I meant with "we might want to pass the R 
tempdir() along as a property" <-- this would be a new property and not 
`spark.sql.warehouse.dir`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[shivaram]

Github user shivaram commented on the issue:

https://github.com/apache/spark/pull/16247
  
Or we could introduce a new property say spark.sql.default.warehouse and
set that to tmpdir()

On Dec 10, 2016 16:53, "Felix Cheung"  wrote:

> I think we should only change spark.sql.warehouse.dir when we are loading
> SparkR as a package. This should minimize changes in the case where we are
> running in cluster mode and so on.
>
> For that purpose, let's check for R interactive().
>
> Conceptually, we might want to pass the R tempdir() along as a property
> and then check again after config and hive config is applied, if it is
> still not set, take that property and set it to spark.sql.warehouse.dir.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> , or 
mute
> the thread
> 

> .
>



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[felixcheung]

Github user felixcheung commented on the issue:

https://github.com/apache/spark/pull/16247
  
I think we should only change `spark.sql.warehouse.dir` when we are loading 
SparkR as a package. This should minimize changes in the case where we are 
running in cluster mode and so on.

For that purpose, let's check for R `interactive()`.

Conceptually, we might want to pass the R `tempdir()` along as a property 
and then check again after config and hive config is applied, if it is still 
not set, take that property and set it to `spark.sql.warehouse.dir`.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/16247
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/16247
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69975/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/16247
  
**[Test build #69975 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69975/consoleFull)**
 for PR 16247 at commit 
[`79caf1c`](https://github.com/apache/spark/commit/79caf1c3cf94b0e97ab09106aca5bab7d9fbf1c9).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

[shivaram]

Github user shivaram commented on the issue:

https://github.com/apache/spark/pull/16247
  
@bdwyer2 One more thing: Is there a good way to test this ? 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org