Mailing lists matching spark.apache.org
commits spark.apache.orgdev spark.apache.org
issues spark.apache.org
reviews spark.apache.org
user spark.apache.org
[GitHub] spark pull request #14008: [SPARK-16281][SQL] Implement parse_url SQL functi...
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69350933
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/StringExpressionsSuite.scala
---
@@ -725,4 +725,43 @@ class StringExpressionsSuite extends SparkFunSuite
with ExpressionEvalHelper {
checkEvaluation(FindInSet(Literal("abf"), Literal("abc,b,ab,c,def")),
0)
checkEvaluation(FindInSet(Literal("ab,"), Literal("abc,b,ab,c,def")),
0)
}
+
+ test("ParseUrl") {
+def checkParseUrl(expected: String, urlStr: String, partToExtract:
String): Unit = {
+ checkEvaluation(
+ParseUrl(Literal.create(urlStr, StringType),
Literal.create(partToExtract, StringType)),
+expected)
+}
+def checkParseUrlWithKey(expected: String, urlStr: String,
+ partToExtract: String, key: String): Unit = {
+ checkEvaluation(
+ParseUrl(Literal.create(urlStr, StringType),
Literal.create(partToExtract, StringType),
+ Literal.create(key, StringType)), expected)
+ }
+
+checkParseUrl("spark.apache.org",
"http://spark.apache.org/path?query=1;, "HOST")
+checkParseUrl("/path", "http://spark.apache.org/path?query=1;, "PATH")
+checkParseUrl("query=1", "http://spark.apache.org/path?query=1;,
"QUERY")
+checkParseUrl("Ref", "http://spark.apache.org/path?query=1#Ref;, "REF")
+checkParseUrl("http", "http://spark.apache.org/path?query=1;,
"PROTOCOL")
+checkParseUrl("/path?query=1", "http://spark.apache.org/path?query=1;,
"FILE")
+checkParseUrl("spark.apache.org:8080",
"http://spark.apache.org:8080/path?query=1;, "AUTHORITY")
+checkParseUrl("jian", "http://j...@spark.apache.org/path?query=1;,
"USERINFO")
+checkParseUrlWithKey("1", "http://spark.apache.org/path?query=1;,
"QUERY", "query")
+
+// Null checking
+checkParseUrl(null, null, "HOST")
+checkParseUrl(null, "http://spark.apache.org/path?query=1;, null)
+checkParseUrl(null, null, null)
+checkParseUrl(null, "test", "HOST")
+checkParseUrl(null, "http://spark.apache.org/path?query=1;, "NO")
+checkParseUrlWithKey(null, "http://spark.apache.org/path?query=1;,
"HOST", "query")
+checkParseUrlWithKey(null, "http://spark.apache.org/path?query=1;,
"QUERY", "quer")
+checkParseUrlWithKey(null, "http://spark.apache.org/path?query=1;,
"QUERY", null)
--- End diff --
I'm not sure how to handle this kind of malformed queries. Hive makes
reasonable message.
```
hive> SELECT parse_url('https://?1=1;?','QUERY', '???');
FAILED: SemanticException [Error 10014]: Line 1:7 Wrong arguments ''???'':
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14008: [SPARK-16281][SQL] Implement parse_url SQL functi...
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69370542
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/StringExpressionsSuite.scala
---
@@ -725,4 +725,43 @@ class StringExpressionsSuite extends SparkFunSuite
with ExpressionEvalHelper {
checkEvaluation(FindInSet(Literal("abf"), Literal("abc,b,ab,c,def")),
0)
checkEvaluation(FindInSet(Literal("ab,"), Literal("abc,b,ab,c,def")),
0)
}
+
+ test("ParseUrl") {
+def checkParseUrl(expected: String, urlStr: String, partToExtract:
String): Unit = {
+ checkEvaluation(
+ParseUrl(Literal.create(urlStr, StringType),
Literal.create(partToExtract, StringType)),
+expected)
+}
+def checkParseUrlWithKey(expected: String, urlStr: String,
+ partToExtract: String, key: String): Unit = {
+ checkEvaluation(
+ParseUrl(Literal.create(urlStr, StringType),
Literal.create(partToExtract, StringType),
+ Literal.create(key, StringType)), expected)
+ }
+
+checkParseUrl("spark.apache.org",
"http://spark.apache.org/path?query=1;, "HOST")
+checkParseUrl("/path", "http://spark.apache.org/path?query=1;, "PATH")
+checkParseUrl("query=1", "http://spark.apache.org/path?query=1;,
"QUERY")
+checkParseUrl("Ref", "http://spark.apache.org/path?query=1#Ref;, "REF")
+checkParseUrl("http", "http://spark.apache.org/path?query=1;,
"PROTOCOL")
+checkParseUrl("/path?query=1", "http://spark.apache.org/path?query=1;,
"FILE")
+checkParseUrl("spark.apache.org:8080",
"http://spark.apache.org:8080/path?query=1;, "AUTHORITY")
+checkParseUrl("jian", "http://j...@spark.apache.org/path?query=1;,
"USERINFO")
+checkParseUrlWithKey("1", "http://spark.apache.org/path?query=1;,
"QUERY", "query")
+
+// Null checking
+checkParseUrl(null, null, "HOST")
+checkParseUrl(null, "http://spark.apache.org/path?query=1;, null)
+checkParseUrl(null, null, null)
+checkParseUrl(null, "test", "HOST")
+checkParseUrl(null, "http://spark.apache.org/path?query=1;, "NO")
+checkParseUrlWithKey(null, "http://spark.apache.org/path?query=1;,
"HOST", "query")
+checkParseUrlWithKey(null, "http://spark.apache.org/path?query=1;,
"QUERY", "quer")
+checkParseUrlWithKey(null, "http://spark.apache.org/path?query=1;,
"QUERY", null)
--- End diff --
Yes, you are right. Invalid `key` does this job. I'll fix this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Re: functools.partial as UserDefinedFunction
Hi, I've filed a JIRA (https://issues.apache.org/jira/browse/SPARK-6553) and suggested a fix (https://github.com/apache/spark/pull/5206). On 2015-03-25 19:49, Davies Liu wrote: It’s good to support functools.partial, could you file a JIRA for it? On Wednesday, March 25, 2015 at 5:42 AM, Karlson wrote: Hi all, passing a functools.partial-function as a UserDefinedFunction to DataFrame.select raises an AttributeException, because functools.partial does not have the attribute __name__. Is there any alternative to relying on __name__ in pyspark/sql/functions.py:126 ? - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org (mailto:dev-unsubscr...@spark.apache.org) For additional commands, e-mail: dev-h...@spark.apache.org (mailto:dev-h...@spark.apache.org) - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
[jira] [Created] (SPARK-36209) https://spark.apache.org/docs/latest/sql-programming-guide.html contains invalid link to Python doc
Dominik Gehl created SPARK-36209: Summary: https://spark.apache.org/docs/latest/sql-programming-guide.html contains invalid link to Python doc Key: SPARK-36209 URL: https://issues.apache.org/jira/browse/SPARK-36209 Project: Spark Issue Type: Bug Components: Documentation Affects Versions: 3.1.2 Environment: On https://spark.apache.org/docs/latest/sql-programming-guide.html, the link to the python doc points to https://spark.apache.org/docs/latest/api/python/pyspark.sql.html#pyspark.sql.DataFrame which returns a "Not found" Reporter: Dominik Gehl -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
Re: WELCOME to user@spark.apache.org
Hello, I would like to know how to parse XML files using Apache spark by java language. I am doing this for my senior project and I am a beginner in Apache Spark and I have just a little experience with spark. Thank you. On Apr 18, 2016, at 3:14 AM, user-h...@spark.apache.org wrote: > Hi! This is the ezmlm program. I'm managing the > user@spark.apache.org mailing list. > > Acknowledgment: I have added the address > > j.r.alhaj...@hotmail.com > > to the user mailing list. > > Welcome to user@spark.apache.org! > > Please save this message so that you know the address you are > subscribed under, in case you later want to unsubscribe or change your > subscription address. > > > --- Administrative commands for the user list --- > > I can handle administrative requests automatically. Please > do not send them to the list address! Instead, send > your message to the correct command address: > > To subscribe to the list, send a message to: > <user-subscr...@spark.apache.org> > > To remove your address from the list, send a message to: > <user-unsubscr...@spark.apache.org> > > Send mail to the following for info and FAQ for this list: > <user-i...@spark.apache.org> > <user-...@spark.apache.org> > > Similar addresses exist for the digest list: > <user-digest-subscr...@spark.apache.org> > <user-digest-unsubscr...@spark.apache.org> > > To get messages 123 through 145 (a maximum of 100 per request), mail: > <user-get.123_...@spark.apache.org> > > To get an index with subject and author for messages 123-456 , mail: > <user-index.123_...@spark.apache.org> > > They are always returned as sets of 100, max 2000 per request, > so you'll actually get 100-499. > > To receive all messages with the same subject as message 12345, > send a short message to: > <user-thread.12...@spark.apache.org> > > The messages should contain one line or word of text to avoid being > treated as sp@m, but I will ignore their content. > Only the ADDRESS you send to is important. > > You can start a subscription for an alternate address, > for example "john@host.domain", just add a hyphen and your > address (with '=' instead of '@') after the command word: >
[jira] [Comment Edited] (SPARK-24530) pyspark.ml doesn't generate class docs correctly
[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16511730#comment-16511730 ] Dongjoon Hyun edited comment on SPARK-24530 at 6/13/18 10:28 PM: - Hi, [~mengxr] . I got the following locally. It generated correctly. !image-2018-06-13-15-15-51-025.png! However, as you pointed out, I found that the following status. Some docs are broken. 2.1.x (O) [https://spark.apache.org/docs/2.1.0/api/python/pyspark.ml.html#pyspark.ml.classification.LogisticRegression] (O) [https://spark.apache.org/docs/2.1.1/api/python/pyspark.ml.html#pyspark.ml.classification.LogisticRegression] (X) [https://spark.apache.org/docs/2.1.2/api/python/pyspark.ml.html#pyspark.ml.classification.LogisticRegression] 2.2.x (O) [https://spark.apache.org/docs/2.2.0/api/python/pyspark.ml.html#pyspark.ml.classification.LogisticRegression] (X) [https://spark.apache.org/docs/2.2.1/api/python/pyspark.ml.html#pyspark.ml.classification.LogisticRegression] 2.3.x (O) [https://spark.apache.org/docs/2.3.0/api/python/pyspark.ml.html#pyspark.ml.classification.LogisticRegression] (X) [https://spark.apache.org/docs/2.3.1/api/python/pyspark.ml.html#pyspark.ml.classification.LogisticRegression] was (Author: dongjoon): Hi, [~mengxr] . I got the following locally. It generated correctly. !image-2018-06-13-15-15-51-025.png! However, as you pointed out, I found that the following status. Some latest docs are broken. 2.1.x (O) https://spark.apache.org/docs/2.1.0/api/python/pyspark.ml.html#pyspark.ml.classification.LogisticRegression (O) https://spark.apache.org/docs/2.1.1/api/python/pyspark.ml.html#pyspark.ml.classification.LogisticRegression (X) https://spark.apache.org/docs/2.1.2/api/python/pyspark.ml.html#pyspark.ml.classification.LogisticRegression 2.2.x (O) https://spark.apache.org/docs/2.2.0/api/python/pyspark.ml.html#pyspark.ml.classification.LogisticRegression (X) https://spark.apache.org/docs/2.2.1/api/python/pyspark.ml.html#pyspark.ml.classification.LogisticRegression 2.3.x (O) https://spark.apache.org/docs/2.3.0/api/python/pyspark.ml.html#pyspark.ml.classification.LogisticRegression (X) https://spark.apache.org/docs/2.3.1/api/python/pyspark.ml.html#pyspark.ml.classification.LogisticRegression > pyspark.ml doesn't generate class docs correctly > > > Key: SPARK-24530 > URL: https://issues.apache.org/jira/browse/SPARK-24530 > Project: Spark > Issue Type: Bug > Components: ML, PySpark >Affects Versions: 2.4.0 >Reporter: Xiangrui Meng >Priority: Blocker > Attachments: Screen Shot 2018-06-12 at 8.23.18 AM.png, Screen Shot > 2018-06-12 at 8.23.29 AM.png, image-2018-06-13-15-15-51-025.png, > pyspark-ml-doc-utuntu18.04-python2.7-sphinx-1.7.5.png > > > I generated python docs from master locally using `make html`. However, the > generated html doc doesn't render class docs correctly. I attached the > screenshot from Spark 2.3 docs and master docs generated on my local. Not > sure if this is because my local setup. > cc: [~dongjoon] Could you help verify? > > The followings are our released doc status. Some recent docs seems to be > broken. > *2.1.x* > (O) > [https://spark.apache.org/docs/2.1.0/api/python/pyspark.ml.html#pyspark.ml.classification.LogisticRegression] > (O) > [https://spark.apache.org/docs/2.1.1/api/python/pyspark.ml.html#pyspark.ml.classification.LogisticRegression] > (X) > [https://spark.apache.org/docs/2.1.2/api/python/pyspark.ml.html#pyspark.ml.classification.LogisticRegression] > *2.2.x* > (O) > [https://spark.apache.org/docs/2.2.0/api/python/pyspark.ml.html#pyspark.ml.classification.LogisticRegression] > (X) > [https://spark.apache.org/docs/2.2.1/api/python/pyspark.ml.html#pyspark.ml.classification.LogisticRegression] > *2.3.x* > (O) > [https://spark.apache.org/docs/2.3.0/api/python/pyspark.ml.html#pyspark.ml.classification.LogisticRegression] > (X) > [https://spark.apache.org/docs/2.3.1/api/python/pyspark.ml.html#pyspark.ml.classification.LogisticRegression] -- This message was sent by Atlassian JIRA (v7.6.3#76005) ----- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[1/2] spark-website git commit: Update sitemap and make it partly auto-generate; fix site URL
Repository: spark-website Updated Branches: refs/heads/asf-site 3529c3f19 -> 8d0a10ae0 http://git-wip-us.apache.org/repos/asf/spark-website/blob/8d0a10ae/sitemap.xml -- diff --git a/sitemap.xml b/sitemap.xml index c2b271e..ac3b8a0 100644 --- a/sitemap.xml +++ b/sitemap.xml @@ -1,1871 +1,151 @@ +--- +sitemap: false +--- http://www.sitemaps.org/schemas/sitemap/0.9; - xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; - xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 +xmlns="http://www.sitemaps.org/schemas/sitemap/0.9; +xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; +xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd;> - + http://spark.apache.org/ - 2015-01-22T00:27:22+00:00 daily - - - http://spark.apache.org/downloads.html - 2015-01-22T00:27:22+00:00 - weekly - - - http://spark.apache.org/sql/ - 2015-01-22T00:27:22+00:00 - weekly - - - http://spark.apache.org/streaming/ - 2015-01-22T00:27:22+00:00 - weekly - - - http://spark.apache.org/mllib/ - 2015-01-22T00:27:22+00:00 - weekly - - - http://spark.apache.org/graphx/ - 2015-01-22T00:27:22+00:00 - weekly - - - http://spark.apache.org/documentation.html - 2015-01-22T00:27:22+00:00 - weekly - - - http://spark.apache.org/docs/latest/ 1.0 - 2014-12-19T00:12:40+00:00 - weekly - - - http://spark.apache.org/examples.html - 2015-01-22T00:27:22+00:00 - weekly - - - http://spark.apache.org/community.html - 2015-01-22T00:27:22+00:00 - weekly - - - http://spark.apache.org/faq.html - 2015-01-22T00:27:22+00:00 - weekly - - - http://spark.apache.org/news/spark-summit-east-agenda-posted.html - 2015-01-22T00:27:22+00:00 - weekly - - - http://spark.apache.org/news/spark-1-2-0-released.html - 2015-01-22T00:27:22+00:00 - weekly - - - http://spark.apache.org/news/spark-1-1-1-released.html - 2015-01-22T00:27:22+00:00 - weekly - - - http://spark.apache.org/news/registration-open-for-spark-summit-east.html - 2015-01-22T00:27:22+00:00 - weekly + - http://spark.apache.org/news/index.html - 2015-01-22T00:27:22+00:00 + http://spark.apache.org/docs/latest/index.html daily - - - http://spark.apache.org/docs/latest/spark-standalone.html - 2014-12-19T00:12:40+00:00 - weekly - 1.0 - - - http://spark.apache.org/docs/latest/ec2-scripts.html - 2014-12-19T00:12:40+00:00 - weekly 1.0 http://spark.apache.org/docs/latest/quick-start.html - 2014-12-19T00:12:40+00:00 - weekly - 1.0 - - - http://spark.apache.org/releases/spark-release-1-2-0.html - 2015-01-22T00:27:22+00:00 - weekly - - - http://spark.apache.org/docs/latest/building-spark.html - 2014-12-19T00:12:40+00:00 - weekly + daily 1.0 - http://spark.apache.org/docs/latest/sql-programming-guide.html - 2014-12-19T00:12:40+00:00 - weekly + http://spark.apache.org/docs/latest/programming-guide.html + daily 1.0 http://spark.apache.org/docs/latest/streaming-programming-guide.html - 2014-12-19T00:12:40+00:00 - weekly - 1.0 - - - http://spark.apache.org/docs/latest/mllib-guide.html - 2015-01-15T02:38:52+00:00 - weekly - 1.0 - - - http://spark.apache.org/docs/latest/graphx-programming-guide.html - 2014-12-19T00:12:40+00:00 - weekly - 1.0 - - - http://spark.apache.org/docs/1.2.0/ - 2014-11-24T23:38:52+00:00 - weekly - 0.5 - - - http://spark.apache.org/docs/1.1.1/ - 2014-11-24T23:38:52+00:00 - weekly - 0.5 - - - http://spark.apache.org/docs/1.0.2/ - 2014-08-06T00:40:54+00:00 - weekly - 0.5 - - - http://spark.apache.org/docs/0.9.2/ - 2014-07-23T23:08:20+00:00 - weekly - 0.4 - - - http://spark.apache.org/docs/0.8.1/ - 2013-12-19T23:20:24+00:00 - weekly - 0.4 - - - http://spark.apache.org/docs/0.7.3/ - 2013-08-24T03:23:13+00:00 - weekly - 0.3 - - - http://spark.apache.org/docs/0.6.2/ - 2013-08-24T03:23:13+00:00 - weekly - 0.3 - - - http://spark.apache.org/screencasts/1-first-steps-with-spark.html - 2015-01-22T00:27:22+00:00 - weekly - - - http://spark.apache.org/screencasts/2-spark-documentation-overview.html - 2015-01-22T00:27:22+00:00 - weekly - - - http://spark.apache.org/screencasts/3-transformations-and-caching.html - 2015-01-22T00:27:22+00:00 - weekly - - - http://spark.apache.org/screencasts/4-a-standalone-job-in-spark.html - 2015-01-22T00:27:22+00:00 - weekly - - - http://spark.apache.org/research.html - 2015-01-22T00:27:22+00:00 - weekly - - - http://spark.apache.org/docs/latest/index.html - 2014-12-19T00:12:40+00:00 - weekly + daily 1.0 - http://spark.apache.org/docs/latest/programming-guide.html - 2014-12-19T00:12:40+00:00 - weekly + http://spark.apache.org/docs/latest/sql-programming-guide.html + daily 1.0 - http://spark.apache.org/docs/latest/bagel-programming-guide.html - 2014-12-19T00:12:40+00:00 - weekly + http
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.0.0-rc2 [deleted] 327ab1e24 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.0.0-rc7 [deleted] bc81ecfdc - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.0.0-rc6 [deleted] aab03f5f9 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.0.0-rc8 [deleted] 9abe0192a - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.0.1-rc1 [deleted] 2a9114542 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.0.0-rc10 [deleted] eafc6f6f4 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v0.9.2-rc1 [deleted] 5a3020615 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v0.9.1-rc3 [deleted] 931ac8a0e - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v0.9.1-rc1 [deleted] 20a2d247a - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v0.9.1-rc2 [deleted] 95d70498b - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.0.2-rc1 [deleted] 913e3de6d - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.1.0-snapshot1 [created] db4a0a5e8 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.1.0-snapshot2 [created] 631b798a5 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.1.0-rc1 [created] 1dc825d90 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.1.0-rc1 [deleted] 1dc825d90 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.1.0-rc1 [created] d6d19b581 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.1.0-rc1 [deleted] d6d19b581 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.1.0-rc1 [created] c0cb9d6b5 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.1.0-rc3 [created] 25596d1da - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.1.0-rc3 [deleted] 25596d1da - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.1.0-rc3 [created] c8886db83 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.1.0-rc4 [created] 5918ea4c9 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.1.0 [created] 2f9b2bd78 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.1.0-rc3 [deleted] c8886db83 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.1.0-snapshot1 [deleted] db4a0a5e8 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.1.0-rc2 [deleted] 446173ca1 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.1.0-snapshot2 [deleted] 631b798a5 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.1.0-rc1 [deleted] c0cb9d6b5 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Branches: refs/heads/branch-1.2 [created] 76386e1a2 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.1.0-rc4 [deleted] 5918ea4c9 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Branches: refs/heads/scala-2.11-prashant [deleted] fd849b0b4 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.1.1-rc1 [deleted] 490ffd35a - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.1.1-rc1 [created] 9cffba01e - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.1.1-rc1 [deleted] 9cffba01e - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.1.1-rc1 [created] d18bc77d3 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.1.1-rc1 [created] defa2f1a5 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.1.1-rc1 [deleted] d18bc77d3 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.1.0 [created] 2f9b2bd78 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/test [deleted] 70d037168 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/test [created] 70d037168 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.2.0-snapshot0 [created] bc0987579 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.2.0-snapshot1 [created] 38c1fbd96 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Branches: refs/heads/filestream-fix1 [deleted] 6b8d85b2b - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.1.1-rc1 [deleted] defa2f1a5 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.2.0-rc1 [created] db7f4a898 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.2.0-rc1 [deleted] db7f4a898 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.2.0-rc1 [created] 5247dd859 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.2.0-rc1 [deleted] cc2c05e4e - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.2.0-rc1 [created] 39c7d1c1f - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.2.0-rc1 [deleted] 39c7d1c1f - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.2.0-rc1 [created] 1056e9ec1 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.2.0-rc2 [created] 2b72c569a - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.2.0-rc2 [deleted] 2b72c569a - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.2.0 [created] 2b72c569a - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.2.0 [deleted] 2b72c569a - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.2.0 [deleted] 2b72c569a - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.2.0 [created] a428c446e - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Which part of the code deals with communication?
Hi all, Workers will exchange data in between, right? What classes are in charge of these actions? - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
one hot encoding
Do we have one-hot encoding in spark MLLib 1.1.1 or 1.2.0 ? It wasn't available in 1.1.0. Thanks. - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Hadoop Distributed Cache
Hi, As part of SparkContext.newAPIHadoopRDD(). Would Spark support an InputFormat that uses Hadoop’s distributed cache? Thanks, Máximo - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Where can I find the module diagram of SPARK?
Hi, Please help me with that. BR, Theodore Si - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
How to benchmark SPARK apps?
Hi all, What tools should I use to benchmark SPARK applications? BR, Theo - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Spark assembly for YARN/CDH5
Does anyone know if there Spark assemblies are created and available for download that have been built for CDH5 and YARN? Thanks, Philip - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Re: unsubscribe
Send it to user-unsubscr...@spark.apache.org. Read the community page https://spark.apache.org/community.html for more info Thanks Best Regards On Wed, Oct 29, 2014 at 3:32 AM, Ricky Thomas ri...@truedash.io wrote:
Kafka examples
Hi guys, The Kafka’s examples in master branch were canceled? Thanks -- Informativa sulla Privacy: http://www.unibs.it/node/8155 - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Interoperability between ScalaRDD, JavaRDD and PythonRDD
Hello, Is it possible to reuse RDD implementations written in Scala/Java with PySpark? Thanks, Nam - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.2.1-rc3 [created] b6eaf77d4 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.2.0-snapshot1 [deleted] 38c1fbd96 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.2.1-rc2 [deleted] b77f87673 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.2.0-snapshot0 [deleted] bc0987579 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.2.0-rc1 [deleted] 1056e9ec1 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.2.1-rc1 [deleted] 3e2d7d310 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.2.0-rc2 [deleted] a428c446e - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.2.1 [created] b6eaf77d4 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Re: Welcoming three new committers
Congratulations! -- Nan Zhu http://codingcat.me On Tuesday, February 3, 2015 at 8:08 PM, Xuefeng Wu wrote: Congratulations！well done. Yours, Xuefeng Wu 吴雪峰 敬上 On 2015年2月4日, at 上午6:34, Matei Zaharia matei.zaha...@gmail.com (mailto:matei.zaha...@gmail.com) wrote: Hi all, The PMC recently voted to add three new committers: Cheng Lian, Joseph Bradley and Sean Owen. All three have been major contributors to Spark in the past year: Cheng on Spark SQL, Joseph on MLlib, and Sean on ML and many pieces throughout Spark Core. Join me in welcoming them as committers! Matei - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org (mailto:dev-unsubscr...@spark.apache.org) For additional commands, e-mail: dev-h...@spark.apache.org (mailto:dev-h...@spark.apache.org) - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org (mailto:dev-unsubscr...@spark.apache.org) For additional commands, e-mail: dev-h...@spark.apache.org (mailto:dev-h...@spark.apache.org)
What will happen if the Driver exits abnormally?
As title. Is there some mechanism to recover to make the job can be completed? Any comments will be very appreciated. Best Regards, Anzhsoft - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Is there a separate mailing list for Spark Developers ?
d...@spark.apache.org http://apache-spark-developers-list.1001551.n3.nabble.com/ mentioned on http://spark.apache.org/community.html seems to be bouncing. Is there another one ?
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.2.1-rc1 [created] e87eb2b42 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.2.1-rc1 [deleted] e87eb2b42 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.2.1-rc1 [created] 3e2d7d310 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.2.1-rc2 [created] b77f87673 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Re: Can Spark benefit from Hive-like partitions?
Ah, well that is interesting. I'll experiment further tomorrow. Thank you for the info! - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.3.0-snapshot1 [created] d97bfc6f2 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.3.0-snapshot1 [deleted] 53068f56f - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.3.0-snapshot1 [created] 53068f56f - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.3.0-snapshot1 [created] c2e400103 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.3.0-snapshot1 [deleted] c2e400103 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
sparksql native jdbc driver
hey guys, In my understanding SparkSQL only supports JDBC connection through hive thrift server, is this correct? Thanks - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
1.3 release
Is there a reason why the prebuilt releases don't include current CDH distros and YARN support? Eric Friedman - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.3.0-rc3 [deleted] 6fb4af2fb - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.3.0-rc3 [created] 6fb4af2fb - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.3.0-rc3 [created] 4aaf48d46 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
Git Push Summary
Repository: spark Updated Tags: refs/tags/v1.3.0 [created] 4aaf48d46 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
