[GitHub] spark issue #17237: [SPARK-19852][PYSPARK][ML] Update Python API setHandleIn...

2017-03-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17237 **[Test build #74711 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74711/testReport)** for PR 17237 at commit

[GitHub] spark issue #17191: [SPARK-14471][SQL] Aliases in SELECT could be used in GR...

2017-03-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17191 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74712/ Test FAILed. ---

[GitHub] spark issue #17191: [SPARK-14471][SQL] Aliases in SELECT could be used in GR...

2017-03-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17191 **[Test build #74712 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74712/testReport)** for PR 17191 at commit

[GitHub] spark issue #17191: [SPARK-14471][SQL] Aliases in SELECT could be used in GR...

2017-03-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17191 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17319: [SPARK-19765][SPARK-18549][SPARK-19093][SPARK-19736][BAC...

2017-03-16 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17319 thanks, merging to 2.1! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17191: [SPARK-14471][SQL] Aliases in SELECT could be used in GR...

2017-03-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17191 **[Test build #74712 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74712/testReport)** for PR 17191 at commit

[GitHub] spark issue #17237: [SPARK-19852][PYSPARK][ML] Update Python API setHandleIn...

2017-03-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17237 **[Test build #74711 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74711/testReport)** for PR 17237 at commit

[GitHub] spark issue #17191: [SPARK-14471][SQL] Aliases in SELECT could be used in GR...

2017-03-16 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17191 It's kind of weird if we have both "group by ordinal" feature and this "group by alias" feature, shall we only pick one of them? --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #17088: [SPARK-19753][CORE] Un-register all shuffle output on a ...

2017-03-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17088 **[Test build #74710 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74710/testReport)** for PR 17088 at commit

[GitHub] spark issue #17179: [SPARK-19067][SS] Processing-time-based timeout in MapGr...

2017-03-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17179 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74698/ Test PASSed. ---

[GitHub] spark issue #17179: [SPARK-19067][SS] Processing-time-based timeout in MapGr...

2017-03-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17179 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17179: [SPARK-19067][SS] Processing-time-based timeout in MapGr...

2017-03-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17179 **[Test build #74698 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74698/testReport)** for PR 17179 at commit

[GitHub] spark pull request #16499: [SPARK-17204][CORE] Fix replicated off heap stora...

2017-03-16 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16499#discussion_r106573530 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -1048,7 +1065,7 @@ private[spark] class BlockManager( try

[GitHub] spark issue #17087: [SPARK-19372][SQL] Fix throwing a Java exception at df.f...

2017-03-16 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/17087 @marmbrus, could you please take a look at this PR for fixing a problem? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #16028: [SPARK-18518][ML] HasSolver supports override

2017-03-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16028 **[Test build #74709 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74709/testReport)** for PR 16028 at commit

[GitHub] spark issue #17328: [SPARK-19975][Python][SQL] Add map_keys and map_values f...

2017-03-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17328 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #17320: [SPARK-19967][SQL] Add from_json in FunctionRegistry

2017-03-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17320 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74701/ Test FAILed. ---

[GitHub] spark issue #17320: [SPARK-19967][SQL] Add from_json in FunctionRegistry

2017-03-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17320 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17320: [SPARK-19967][SQL] Add from_json in FunctionRegistry

2017-03-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17320 **[Test build #74701 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74701/testReport)** for PR 17320 at commit

[GitHub] spark pull request #17328: [SPARK-19975][Python][SQL] Add map_keys and map_v...

2017-03-16 Thread yongtang
GitHub user yongtang opened a pull request: https://github.com/apache/spark/pull/17328 [SPARK-19975][Python][SQL] Add map_keys and map_values functions to Python ## What changes were proposed in this pull request? This fix tries to address the issue in SPARK-19975 where we

[GitHub] spark issue #16171: [SPARK-18739][ML][PYSPARK] Classification and regression...

2017-03-16 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/16171 @jkbradley Could you please to make a pass? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #17014: [SPARK-18608][ML] Fix double-caching in ML algorithms

2017-03-16 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/17014 ping @hhbyyh ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #17179: [SPARK-19067][SS] Processing-time-based timeout in MapGr...

2017-03-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17179 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74700/ Test FAILed. ---

[GitHub] spark issue #17179: [SPARK-19067][SS] Processing-time-based timeout in MapGr...

2017-03-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17179 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17179: [SPARK-19067][SS] Processing-time-based timeout in MapGr...

2017-03-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17179 **[Test build #74700 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74700/testReport)** for PR 17179 at commit

[GitHub] spark issue #17095: [SPARK-19763][SQL]qualified external datasource table lo...

2017-03-16 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/17095 I suspect that this PR is the cause of consistent failures in the maven build, in the HiveCatalogedDDLSuite unit test:

[GitHub] spark issue #17216: [SPARK-19873][SS] Record num shuffle partitions in offse...

2017-03-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17216 **[Test build #74708 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74708/testReport)** for PR 17216 at commit

[GitHub] spark issue #17314: Fix for JIRA ticket SPARK-15790

2017-03-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17314 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #17314: Fix for JIRA ticket SPARK-15790

2017-03-16 Thread ehsun7b
Github user ehsun7b commented on the issue: https://github.com/apache/spark/pull/17314 I updated the title and provided more details about the change. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #17314: Fix for JIRA ticket SPARK-15790

2017-03-16 Thread ehsun7b
GitHub user ehsun7b reopened a pull request: https://github.com/apache/spark/pull/17314 Fix for JIRA ticket SPARK-15790 ## What changes were proposed in this pull request? This is the partial fix for this JIRA: https://issues.apache.org/jira/browse/SPARK-15790

[GitHub] spark issue #17216: [SPARK-19873][SS] Record num shuffle partitions in offse...

2017-03-16 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/17216 LGTM. Will merge after tests pass. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17179: [SPARK-19067][SS] Processing-time-based timeout in MapGr...

2017-03-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17179 **[Test build #74707 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74707/testReport)** for PR 17179 at commit

[GitHub] spark issue #17320: [SPARK-19967][SQL] Add from_json in FunctionRegistry

2017-03-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17320 **[Test build #74706 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74706/testReport)** for PR 17320 at commit

[GitHub] spark pull request #17287: [SPARK-19945][SQL]add test suite for SessionCatal...

2017-03-16 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17287#discussion_r106570569 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala --- @@ -27,41 +27,67 @@ import

[GitHub] spark pull request #17320: [SPARK-19967][SQL] Add from_json in FunctionRegis...

2017-03-16 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/17320#discussion_r106570477 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/JsonFunctionsSuite.scala --- @@ -220,4 +220,39 @@ class JsonFunctionsSuite extends QueryTest with

[GitHub] spark issue #16971: [SPARK-19573][SQL] Make NaN/null handling consistent in ...

2017-03-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16971 **[Test build #74705 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74705/testReport)** for PR 16971 at commit

[GitHub] spark pull request #17287: [SPARK-19945][SQL]add test suite for SessionCatal...

2017-03-16 Thread windpiger
Github user windpiger commented on a diff in the pull request: https://github.com/apache/spark/pull/17287#discussion_r106570185 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala --- @@ -27,41 +27,67 @@ import

[GitHub] spark pull request #17315: [SPARK-19949][SQL] unify bad record handling in C...

2017-03-16 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17315#discussion_r106570171 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JSONOptions.scala --- @@ -65,7 +65,7 @@ private[sql] class JSONOptions(

[GitHub] spark pull request #17287: [SPARK-19945][SQL]add test suite for SessionCatal...

2017-03-16 Thread windpiger
Github user windpiger commented on a diff in the pull request: https://github.com/apache/spark/pull/17287#discussion_r106569991 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala --- @@ -27,41 +27,67 @@ import

[GitHub] spark pull request #17287: [SPARK-19945][SQL]add test suite for SessionCatal...

2017-03-16 Thread windpiger
Github user windpiger commented on a diff in the pull request: https://github.com/apache/spark/pull/17287#discussion_r106569958 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala --- @@ -27,41 +27,67 @@ import

[GitHub] spark pull request #17179: [SPARK-19067][SS] Processing-time-based timeout i...

2017-03-16 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17179#discussion_r106569878 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/KeyedState.scala --- @@ -61,25 +65,50 @@ import

[GitHub] spark pull request #17179: [SPARK-19067][SS] Processing-time-based timeout i...

2017-03-16 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17179#discussion_r106569867 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/KeyedState.scala --- @@ -61,25 +65,50 @@ import

[GitHub] spark issue #17191: [SPARK-14471][SQL] Aliases in SELECT could be used in GR...

2017-03-16 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/17191 I'll add a config soon --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #17070: [SPARK-19721][SS] Good error message for version mismatc...

2017-03-16 Thread lw-lin
Github user lw-lin commented on the issue: https://github.com/apache/spark/pull/17070 @zsxwing sure, please see https://github.com/apache/spark/pull/17327 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #17327: [SPARK-19721][SS][BRANCH-2.1] Good error message for ver...

2017-03-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17327 **[Test build #74704 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74704/testReport)** for PR 17327 at commit

[GitHub] spark pull request #17327: [SPARK-19721][SS][BRANCH-2.1] Good error message ...

2017-03-16 Thread lw-lin
GitHub user lw-lin opened a pull request: https://github.com/apache/spark/pull/17327 [SPARK-19721][SS][BRANCH-2.1] Good error message for version mismatch in log files ## Problem There are several places where we write out version identifiers in various logs for

[GitHub] spark issue #17287: [SPARK-19945][SQL]add test suite for SessionCatalog with...

2017-03-16 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17287 a late LGTM :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #16971: [SPARK-19573][SQL] Make NaN/null handling consistent in ...

2017-03-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16971 **[Test build #74703 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74703/testReport)** for PR 16971 at commit

[GitHub] spark pull request #16971: [SPARK-19573][SQL] Make NaN/null handling consist...

2017-03-16 Thread zhengruifeng
Github user zhengruifeng commented on a diff in the pull request: https://github.com/apache/spark/pull/16971#discussion_r106569149 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/QuantileSummariesSuite.scala --- @@ -55,7 +55,7 @@ class

[GitHub] spark issue #17166: [SPARK-19820] [core] Allow reason to be specified for ta...

2017-03-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17166 **[Test build #74702 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74702/testReport)** for PR 17166 at commit

[GitHub] spark issue #17166: [SPARK-19820] [core] Allow reason to be specified for ta...

2017-03-16 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/17166 Test failure seems unrelated. jenkins retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #16971: [SPARK-19573][SQL] Make NaN/null handling consist...

2017-03-16 Thread zhengruifeng
Github user zhengruifeng commented on a diff in the pull request: https://github.com/apache/spark/pull/16971#discussion_r106569064 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/ApproximatePercentile.scala --- @@ -245,7 +245,7 @@ object

[GitHub] spark pull request #17322: [SPARK-19987][SQL] Pass all filters into FileInde...

2017-03-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17322 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #17287: [SPARK-19945][SQL]add test suite for SessionCatal...

2017-03-16 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17287#discussion_r106568913 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala --- @@ -27,41 +27,67 @@ import

[GitHub] spark pull request #17287: [SPARK-19945][SQL]add test suite for SessionCatal...

2017-03-16 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17287#discussion_r106568853 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala --- @@ -27,41 +27,67 @@ import

[GitHub] spark issue #17322: [SPARK-19987][SQL] Pass all filters into FileIndex

2017-03-16 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17322 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #17287: [SPARK-19945][SQL]add test suite for SessionCatal...

2017-03-16 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17287#discussion_r106568674 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala --- @@ -27,41 +27,67 @@ import

[GitHub] spark issue #17322: [SPARK-19987][SQL] Pass all filters into FileIndex

2017-03-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17322 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17322: [SPARK-19987][SQL] Pass all filters into FileIndex

2017-03-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17322 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74696/ Test PASSed. ---

[GitHub] spark issue #17166: [SPARK-19820] [core] Allow reason to be specified for ta...

2017-03-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17166 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74697/ Test FAILed. ---

[GitHub] spark issue #17322: [SPARK-19987][SQL] Pass all filters into FileIndex

2017-03-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17322 **[Test build #74696 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74696/testReport)** for PR 17322 at commit

[GitHub] spark issue #17166: [SPARK-19820] [core] Allow reason to be specified for ta...

2017-03-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17166 **[Test build #74697 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74697/testReport)** for PR 17166 at commit

[GitHub] spark issue #17166: [SPARK-19820] [core] Allow reason to be specified for ta...

2017-03-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17166 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #17179: [SPARK-19067][SS] Processing-time-based timeout i...

2017-03-16 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17179#discussion_r106568435 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/KeyedState.scala --- @@ -61,25 +65,50 @@ import

[GitHub] spark pull request #17179: [SPARK-19067][SS] Processing-time-based timeout i...

2017-03-16 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17179#discussion_r106568427 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/streaming/KeyedStateTimeout.java --- @@ -0,0 +1,45 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #17179: [SPARK-19067][SS] Processing-time-based timeout i...

2017-03-16 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17179#discussion_r106568386 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/KeyedState.scala --- @@ -92,27 +121,33 @@ import

[GitHub] spark pull request #17179: [SPARK-19067][SS] Processing-time-based timeout i...

2017-03-16 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17179#discussion_r106568279 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/KeyedStateImpl.scala --- @@ -60,6 +82,45 @@ private[sql] class

[GitHub] spark pull request #17192: [SPARK-19849][SQL] Support ArrayType in to_json t...

2017-03-16 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/17192#discussion_r106567525 --- Diff: python/pyspark/sql/functions.py --- @@ -1802,10 +1802,10 @@ def from_json(col, schema, options={}): @since(2.1) def to_json(col,

[GitHub] spark pull request #17192: [SPARK-19849][SQL] Support ArrayType in to_json t...

2017-03-16 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/17192#discussion_r106567845 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonGenerator.scala --- @@ -37,6 +37,11 @@ private[sql] class JacksonGenerator(

[GitHub] spark pull request #17192: [SPARK-19849][SQL] Support ArrayType in to_json t...

2017-03-16 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/17192#discussion_r106566917 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -589,52 +591,69 @@ case class StructToJson(

[GitHub] spark pull request #17192: [SPARK-19849][SQL] Support ArrayType in to_json t...

2017-03-16 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/17192#discussion_r106567643 --- Diff: python/pyspark/sql/functions.py --- @@ -1802,10 +1802,10 @@ def from_json(col, schema, options={}): @since(2.1) def to_json(col,

[GitHub] spark pull request #17192: [SPARK-19849][SQL] Support ArrayType in to_json t...

2017-03-16 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/17192#discussion_r106567410 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala --- @@ -422,7 +422,7 @@ object FunctionRegistry {

[GitHub] spark pull request #17320: [SPARK-19967][SQL] Add from_json in FunctionRegis...

2017-03-16 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/17320#discussion_r106568036 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/JsonFunctionsSuite.scala --- @@ -220,4 +220,39 @@ class JsonFunctionsSuite extends QueryTest with

[GitHub] spark pull request #17192: [SPARK-19849][SQL] Support ArrayType in to_json t...

2017-03-16 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/17192#discussion_r106567583 --- Diff: R/pkg/R/functions.R --- @@ -1795,10 +1795,10 @@ setMethod("to_date", #' to_json #' -#' Converts a column containing a

[GitHub] spark issue #17320: [SPARK-19967][SQL] Add from_json in FunctionRegistry

2017-03-16 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/17320 okay, I'll test more. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #17320: [SPARK-19967][SQL] Add from_json in FunctionRegistry

2017-03-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17320 **[Test build #74701 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74701/testReport)** for PR 17320 at commit

[GitHub] spark issue #17088: [SPARK-19753][CORE] Un-register all shuffle output on a ...

2017-03-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17088 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74694/ Test FAILed. ---

[GitHub] spark issue #17088: [SPARK-19753][CORE] Un-register all shuffle output on a ...

2017-03-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17088 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #17179: [SPARK-19067][SS] Processing-time-based timeout i...

2017-03-16 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17179#discussion_r106567105 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FlatMapGroupsWithStateExec.scala --- @@ -0,0 +1,270 @@ +/* + * Licensed to

[GitHub] spark issue #17088: [SPARK-19753][CORE] Un-register all shuffle output on a ...

2017-03-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17088 **[Test build #74694 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74694/testReport)** for PR 17088 at commit

[GitHub] spark issue #17313: [SPARK-19974][Block Manager] in-memory LRU for partition...

2017-03-16 Thread jianran
Github user jianran commented on the issue: https://github.com/apache/spark/pull/17313 I am sorry,you are right, the second assertion is ok; but what do you think about the assertion comment:_Do a get() on rdd_0_2 so that it is the most recently used item_, what is the purpose of the

[GitHub] spark issue #17130: [SPARK-19791] [ML] Add doc and example for fpgrowth

2017-03-16 Thread hhbyyh
Github user hhbyyh commented on the issue: https://github.com/apache/spark/pull/17130 Please hold on merging this until https://github.com/apache/spark/pull/17321 is resolved. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request #17320: [SPARK-19967][SQL] Add from_json in FunctionRegis...

2017-03-16 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/17320#discussion_r106566646 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -634,7 +661,12 @@ case class StructToJson(

[GitHub] spark pull request #17320: [SPARK-19967][SQL] Add from_json in FunctionRegis...

2017-03-16 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/17320#discussion_r106566472 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/JsonFunctionsSuite.scala --- @@ -202,12 +202,12 @@ class JsonFunctionsSuite extends QueryTest with

[GitHub] spark pull request #17179: [SPARK-19067][SS] Processing-time-based timeout i...

2017-03-16 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17179#discussion_r106566487 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStore.scala --- @@ -50,6 +50,8 @@ trait StateStore { /** Get the

[GitHub] spark pull request #17315: [SPARK-19949][SQL] unify bad record handling in C...

2017-03-16 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17315#discussion_r106566183 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala --- @@ -391,9 +288,9 @@ class JacksonParser(

[GitHub] spark pull request #17179: [SPARK-19067][SS] Processing-time-based timeout i...

2017-03-16 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17179#discussion_r106566096 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/KeyValueGroupedDataset.scala --- @@ -361,18 +435,20 @@ class KeyValueGroupedDataset[K, V] private[sql](

[GitHub] spark pull request #17179: [SPARK-19067][SS] Processing-time-based timeout i...

2017-03-16 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17179#discussion_r106566078 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/KeyValueGroupedDataset.scala --- @@ -284,6 +322,38 @@ class KeyValueGroupedDataset[K, V] private[sql](

[GitHub] spark pull request #17179: [SPARK-19067][SS] Processing-time-based timeout i...

2017-03-16 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17179#discussion_r106566062 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/KeyValueGroupedDataset.scala --- @@ -249,6 +250,43 @@ class KeyValueGroupedDataset[K, V] private[sql](

[GitHub] spark pull request #17179: [SPARK-19067][SS] Processing-time-based timeout i...

2017-03-16 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17179#discussion_r106566088 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/KeyValueGroupedDataset.scala --- @@ -298,12 +368,14 @@ class KeyValueGroupedDataset[K, V] private[sql](

[GitHub] spark issue #17179: [SPARK-19067][SS] Processing-time-based timeout in MapGr...

2017-03-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17179 **[Test build #74700 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74700/testReport)** for PR 17179 at commit

[GitHub] spark issue #17307: [SPARK-13369] Make number of consecutive fetch failures ...

2017-03-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17307 **[Test build #74699 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74699/testReport)** for PR 17307 at commit

[GitHub] spark issue #17307: [SPARK-13369] Make number of consecutive fetch failures ...

2017-03-16 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/17307 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17307: [SPARK-13369] Make number of consecutive fetch failures ...

2017-03-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17307 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17307: [SPARK-13369] Make number of consecutive fetch failures ...

2017-03-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17307 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74692/ Test FAILed. ---

[GitHub] spark issue #17307: [SPARK-13369] Make number of consecutive fetch failures ...

2017-03-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17307 **[Test build #74692 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74692/testReport)** for PR 17307 at commit

[GitHub] spark issue #17322: [SPARK-19987][SQL] Pass all filters into FileIndex

2017-03-16 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17322 LGTM pending Jenkins --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-16 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15363 It sounds like all the comments have been addressed. LGTM except one minor comment. cc @sameeragarwal @cloud-fan @hvanhovell --- If your project is set up for it, you can reply to

[GitHub] spark issue #17179: [SPARK-19067][SS] Processing-time-based timeout in MapGr...

2017-03-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17179 **[Test build #74698 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74698/testReport)** for PR 17179 at commit

[GitHub] spark pull request #15363: [SPARK-17791][SQL] Join reordering using star sch...

2017-03-16 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15363#discussion_r106562261 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/patterns.scala --- @@ -167,8 +167,8 @@ object ExtractFiltersAndInnerJoins

<    1   2   3   4   5   6   >