[jira] [Resolved] (SPARK-15072) Remove SparkSession.withHiveSupport

2016-05-05 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15072. --- Resolution: Fixed > Remove SparkSession.withHiveSupport > --- > >

[jira] [Resolved] (SPARK-14896) Deprecate HiveContext in Python

2016-05-04 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-14896. --- Resolution: Fixed Fix Version/s: 2.0.0 > Deprecate HiveContext in Python > ---

[jira] [Updated] (SPARK-15037) Use SparkSession instead of SQLContext in testsuites

2016-05-04 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15037: -- Assignee: Sandeep Singh > Use SparkSession instead of SQLContext in testsuites > --

[jira] [Updated] (SPARK-15045) Remove dead code in TaskMemoryManager.cleanUpAllAllocatedMemory for pageTable

2016-05-04 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15045: -- Priority: Major (was: Trivial) > Remove dead code in TaskMemoryManager.cleanUpAllAllocatedMemory for p

[jira] [Resolved] (SPARK-15116) In REPL we should create SparkSession first and get SparkContext from it

2016-05-04 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15116. --- Resolution: Fixed Fix Version/s: 2.0.0 > In REPL we should create SparkSession first and get S

[jira] [Resolved] (SPARK-15031) Use SparkSession in Scala/Python/Java example.

2016-05-04 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15031. --- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > Use SparkSession in

[jira] [Updated] (SPARK-13001) Coarse-grained Mesos scheduler should reject offers for longer period of time when reached max cores

2016-05-04 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13001: -- Assignee: Sebastien Rainville > Coarse-grained Mesos scheduler should reject offers for longer period o

[jira] [Resolved] (SPARK-13001) Coarse-grained Mesos scheduler should reject offers for longer period of time when reached max cores

2016-05-04 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13001. --- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > Coarse-grained Meso

[jira] [Resolved] (SPARK-15121) Improve logging of external shuffle handler

2016-05-04 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15121. --- Resolution: Fixed Assignee: Thomas Graves Fix Version/s: 2.0.0 Target Ver

[jira] [Updated] (SPARK-15121) Improve logging of external shuffle handler

2016-05-04 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15121: -- Priority: Minor (was: Major) > Improve logging of external shuffle handler > -

[jira] [Resolved] (SPARK-12299) Remove history serving functionality from standalone Master

2016-05-04 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12299. --- Resolution: Fixed Assignee: Bryan Cutler Fix Version/s: 2.0.0 > Remove history servin

[jira] [Resolved] (SPARK-15126) RuntimeConfig.set should return Unit rather than RuntimeConfig itself

2016-05-04 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15126. --- Resolution: Fixed > RuntimeConfig.set should return Unit rather than RuntimeConfig itself > -

[jira] [Commented] (SPARK-13269) Expose more executor stats in stable status API

2016-05-03 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270067#comment-15270067 ] Andrew Or commented on SPARK-13269: --- Oops actually this was already done in SPARK-14069

[jira] [Resolved] (SPARK-13269) Expose more executor stats in stable status API

2016-05-03 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13269. --- Resolution: Fixed Assignee: Wenchen Fan Fix Version/s: 2.0.0 > Expose more executor s

[jira] [Updated] (SPARK-14645) non local Python resource doesn't work with Mesos cluster mode

2016-05-03 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-14645: -- Assignee: Timothy Chen > non local Python resource doesn't work with Mesos cluster mode > -

[jira] [Resolved] (SPARK-14414) Make error messages consistent across DDLs

2016-05-03 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-14414. --- Resolution: Fixed Fix Version/s: 2.0.0 > Make error messages consistent across DDLs >

[jira] [Resolved] (SPARK-15097) Import fails for someDataset.sqlContext.implicits._

2016-05-03 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15097. --- Resolution: Fixed Assignee: Koert Kuipers Fix Version/s: 2.0.0 Target Ver

[jira] [Resolved] (SPARK-15084) Use builder pattern to create SparkSession in PySpark

2016-05-03 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15084. --- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > Use builder pattern

[jira] [Resolved] (SPARK-14645) non local Python resource doesn't work with Mesos cluster mode

2016-05-03 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-14645. --- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > non local Python re

[jira] [Updated] (SPARK-14422) Improve handling of optional configs in SQLConf

2016-05-03 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-14422: -- Assignee: Sandeep Singh > Improve handling of optional configs in SQLConf > ---

[jira] [Resolved] (SPARK-14422) Improve handling of optional configs in SQLConf

2016-05-03 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-14422. --- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > Improve handling of

[jira] [Updated] (SPARK-15072) Remove SparkSession.withHiveSupport

2016-05-03 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15072: -- Assignee: Sandeep Singh > Remove SparkSession.withHiveSupport > --- > >

[jira] [Updated] (SPARK-15084) Use builder pattern to create SparkSession in PySpark

2016-05-03 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15084: -- Assignee: Dongjoon Hyun > Use builder pattern to create SparkSession in PySpark > -

[jira] [Assigned] (SPARK-15073) Make SparkSession constructors private

2016-05-03 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or reassigned SPARK-15073: - Assignee: Andrew Or > Make SparkSession constructors private > -

[jira] [Updated] (SPARK-15037) Use SparkSession instead of SQLContext in testsuites

2016-05-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15037: -- Summary: Use SparkSession instead of SQLContext in testsuites (was: Use SparkSession instread of SQLCo

[jira] [Resolved] (SPARK-15068) Use proper metastore warehouse path

2016-05-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15068. --- Resolution: Duplicate > Use proper metastore warehouse path > --- > >

[jira] [Created] (SPARK-15068) Use proper metastore warehouse path

2016-05-02 Thread Andrew Or (JIRA)
Andrew Or created SPARK-15068: - Summary: Use proper metastore warehouse path Key: SPARK-15068 URL: https://issues.apache.org/jira/browse/SPARK-15068 Project: Spark Issue Type: Bug Compo

[jira] [Updated] (SPARK-15068) Use proper metastore warehouse path

2016-05-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15068: -- Description: Today, we use "" empty string without Hive, and "/user/hive/warehouse" with Hive, which is

[jira] [Closed] (SPARK-14895) SparkSession Python API

2016-04-29 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-14895. - Resolution: Duplicate > SparkSession Python API > --- > > Key: SPARK-

[jira] [Resolved] (SPARK-15019) Propagate all Spark Confs to HiveConf created in HiveClientImpl

2016-04-29 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15019. --- Resolution: Fixed Fix Version/s: 2.0.0 > Propagate all Spark Confs to HiveConf created in Hive

[jira] [Updated] (SPARK-14896) Deprecate HiveContext in Python

2016-04-29 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-14896: -- Summary: Deprecate HiveContext in Python (was: Remove HiveContext in Python) > Deprecate HiveContext

[jira] [Resolved] (SPARK-14673) Remove HiveContext

2016-04-29 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-14673. --- Resolution: Fixed Fix Version/s: 2.0.0 > Remove HiveContext > -- > >

[jira] [Reopened] (SPARK-14896) Remove HiveContext in Python

2016-04-29 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or reopened SPARK-14896: --- > Remove HiveContext in Python > > > Key: SPARK-14896 >

[jira] [Closed] (SPARK-14896) Remove HiveContext in Python

2016-04-29 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-14896. - Resolution: Won't Fix > Remove HiveContext in Python > > > K

[jira] [Created] (SPARK-15012) Simplify configuration API further

2016-04-29 Thread Andrew Or (JIRA)
Andrew Or created SPARK-15012: - Summary: Simplify configuration API further Key: SPARK-15012 URL: https://issues.apache.org/jira/browse/SPARK-15012 Project: Spark Issue Type: Sub-task C

[jira] [Resolved] (SPARK-14988) Implement catalog and conf API in Python SparkSession

2016-04-29 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-14988. --- Resolution: Fixed Fix Version/s: 2.0.0 > Implement catalog and conf API in Python SparkSession

[jira] [Updated] (SPARK-14988) Implement catalog and conf API in Python SparkSession

2016-04-28 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-14988: -- Description: This is like implementing SPARK-13477 and SPARK-13487 in the Python SparkSession. > Imple

[jira] [Created] (SPARK-14988) Implement catalog and conf API in Python SparkSession

2016-04-28 Thread Andrew Or (JIRA)
Andrew Or created SPARK-14988: - Summary: Implement catalog and conf API in Python SparkSession Key: SPARK-14988 URL: https://issues.apache.org/jira/browse/SPARK-14988 Project: Spark Issue Type: S

[jira] [Commented] (SPARK-14014) Replace existing analysis.Catalog with SessionCatalog

2016-04-27 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15261177#comment-15261177 ] Andrew Or commented on SPARK-14014: --- Pretty sure this was fixed. :) > Replace existing

[jira] [Resolved] (SPARK-14014) Replace existing analysis.Catalog with SessionCatalog

2016-04-27 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-14014. --- Resolution: Fixed Fix Version/s: 2.0.0 > Replace existing analysis.Catalog with SessionCatalog

[jira] [Commented] (SPARK-14915) Tasks that fail due to CommitDeniedException (a side-effect of speculation) can cause job to never complete

2016-04-27 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15260623#comment-15260623 ] Andrew Or commented on SPARK-14915: --- I haven't looked into the scheduler code in detail

[jira] [Assigned] (SPARK-14945) Python SparkSession API

2016-04-26 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or reassigned SPARK-14945: - Assignee: Andrew Or > Python SparkSession API > --- > > Key:

[jira] [Created] (SPARK-14945) Python SparkSession API

2016-04-26 Thread Andrew Or (JIRA)
Andrew Or created SPARK-14945: - Summary: Python SparkSession API Key: SPARK-14945 URL: https://issues.apache.org/jira/browse/SPARK-14945 Project: Spark Issue Type: Sub-task Components:

[jira] [Created] (SPARK-14940) Move ExternalCatalog to own file

2016-04-26 Thread Andrew Or (JIRA)
Andrew Or created SPARK-14940: - Summary: Move ExternalCatalog to own file Key: SPARK-14940 URL: https://issues.apache.org/jira/browse/SPARK-14940 Project: Spark Issue Type: Bug Componen

[jira] [Created] (SPARK-14904) Add back HiveContext in compatibility package

2016-04-25 Thread Andrew Or (JIRA)
Andrew Or created SPARK-14904: - Summary: Add back HiveContext in compatibility package Key: SPARK-14904 URL: https://issues.apache.org/jira/browse/SPARK-14904 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-14902) Expose user-facing RuntimeConfig in SparkSession

2016-04-25 Thread Andrew Or (JIRA)
Andrew Or created SPARK-14902: - Summary: Expose user-facing RuntimeConfig in SparkSession Key: SPARK-14902 URL: https://issues.apache.org/jira/browse/SPARK-14902 Project: Spark Issue Type: Sub-ta

[jira] [Resolved] (SPARK-14721) Remove the HiveContext class

2016-04-25 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-14721. --- Resolution: Fixed Fix Version/s: 2.0.0 > Remove the HiveContext class > --

[jira] [Created] (SPARK-14896) Remove HiveContext in Python

2016-04-25 Thread Andrew Or (JIRA)
Andrew Or created SPARK-14896: - Summary: Remove HiveContext in Python Key: SPARK-14896 URL: https://issues.apache.org/jira/browse/SPARK-14896 Project: Spark Issue Type: Sub-task Compone

[jira] [Created] (SPARK-14895) SparkSession Python API

2016-04-25 Thread Andrew Or (JIRA)
Andrew Or created SPARK-14895: - Summary: SparkSession Python API Key: SPARK-14895 URL: https://issues.apache.org/jira/browse/SPARK-14895 Project: Spark Issue Type: New Feature Component

[jira] [Created] (SPARK-14893) Re-enable HiveSparkSubmitSuite SPARK-8489 test after HiveContext is removed

2016-04-25 Thread Andrew Or (JIRA)
Andrew Or created SPARK-14893: - Summary: Re-enable HiveSparkSubmitSuite SPARK-8489 test after HiveContext is removed Key: SPARK-14893 URL: https://issues.apache.org/jira/browse/SPARK-14893 Project: Spark

[jira] [Created] (SPARK-14861) Replace internal usages of SQLContext with SparkSession

2016-04-22 Thread Andrew Or (JIRA)
Andrew Or created SPARK-14861: - Summary: Replace internal usages of SQLContext with SparkSession Key: SPARK-14861 URL: https://issues.apache.org/jira/browse/SPARK-14861 Project: Spark Issue Type:

[jira] [Created] (SPARK-14828) Start SparkSession in REPL instead of SQLContext

2016-04-21 Thread Andrew Or (JIRA)
Andrew Or created SPARK-14828: - Summary: Start SparkSession in REPL instead of SQLContext Key: SPARK-14828 URL: https://issues.apache.org/jira/browse/SPARK-14828 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-14823) Fix all references to HiveContext in comments and docs

2016-04-21 Thread Andrew Or (JIRA)
Andrew Or created SPARK-14823: - Summary: Fix all references to HiveContext in comments and docs Key: SPARK-14823 URL: https://issues.apache.org/jira/browse/SPARK-14823 Project: Spark Issue Type:

[jira] [Created] (SPARK-14824) Rename object HiveContext to something else

2016-04-21 Thread Andrew Or (JIRA)
Andrew Or created SPARK-14824: - Summary: Rename object HiveContext to something else Key: SPARK-14824 URL: https://issues.apache.org/jira/browse/SPARK-14824 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-14721) Remove the HiveContext class

2016-04-21 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-14721: -- Summary: Remove the HiveContext class (was: Actually remove the HiveContext file itself) > Remove the

[jira] [Commented] (SPARK-14789) Move FunctionResourceLoader into SessionState

2016-04-21 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15252393#comment-15252393 ] Andrew Or commented on SPARK-14789: --- Fixed in https://github.com/apache/spark/pull/1255

[jira] [Resolved] (SPARK-14789) Move FunctionResourceLoader into SessionState

2016-04-21 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-14789. --- Resolution: Fixed Fix Version/s: 2.0.0 > Move FunctionResourceLoader into SessionState > -

[jira] [Created] (SPARK-14789) Move FunctionResourceLoader into SessionState

2016-04-20 Thread Andrew Or (JIRA)
Andrew Or created SPARK-14789: - Summary: Move FunctionResourceLoader into SessionState Key: SPARK-14789 URL: https://issues.apache.org/jira/browse/SPARK-14789 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-14721) Actually remove the HiveContext file itself

2016-04-18 Thread Andrew Or (JIRA)
Andrew Or created SPARK-14721: - Summary: Actually remove the HiveContext file itself Key: SPARK-14721 URL: https://issues.apache.org/jira/browse/SPARK-14721 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-14720) Move the rest of HiveContext to HiveSessionState

2016-04-18 Thread Andrew Or (JIRA)
Andrew Or created SPARK-14720: - Summary: Move the rest of HiveContext to HiveSessionState Key: SPARK-14720 URL: https://issues.apache.org/jira/browse/SPARK-14720 Project: Spark Issue Type: Sub-ta

[jira] [Commented] (SPARK-14647) Group SQLContext/HiveContext state into PersistentState

2016-04-17 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15245009#comment-15245009 ] Andrew Or commented on SPARK-14647: --- I've reverted it for now. > Group SQLContext/Hive

[jira] [Created] (SPARK-14674) Move HiveContext.hiveconf to HiveSessionState

2016-04-15 Thread Andrew Or (JIRA)
Andrew Or created SPARK-14674: - Summary: Move HiveContext.hiveconf to HiveSessionState Key: SPARK-14674 URL: https://issues.apache.org/jira/browse/SPARK-14674 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-14647) Group SQLContext/HiveContext state into PersistentState

2016-04-15 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-14647: -- Issue Type: Sub-task (was: Bug) Parent: SPARK-14673 > Group SQLContext/HiveContext state into

[jira] [Updated] (SPARK-14672) Move HiveContext analyze logic to AnalyzeTable

2016-04-15 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-14672: -- Issue Type: Sub-task (was: Bug) Parent: SPARK-14673 > Move HiveContext analyze logic to Analyz

[jira] [Updated] (SPARK-14668) Move current_database to sql/core

2016-04-15 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-14668: -- Parent Issue: SPARK-14673 (was: SPARK-14118) > Move current_database to sql/core > ---

[jira] [Created] (SPARK-14673) Remove HiveContext

2016-04-15 Thread Andrew Or (JIRA)
Andrew Or created SPARK-14673: - Summary: Remove HiveContext Key: SPARK-14673 URL: https://issues.apache.org/jira/browse/SPARK-14673 Project: Spark Issue Type: Bug Components: SQL Af

[jira] [Created] (SPARK-14672) Move HiveContext analyze logic to AnalyzeTable

2016-04-15 Thread Andrew Or (JIRA)
Andrew Or created SPARK-14672: - Summary: Move HiveContext analyze logic to AnalyzeTable Key: SPARK-14672 URL: https://issues.apache.org/jira/browse/SPARK-14672 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-14670) Allow updating SQLMetrics on driver

2016-04-15 Thread Andrew Or (JIRA)
Andrew Or created SPARK-14670: - Summary: Allow updating SQLMetrics on driver Key: SPARK-14670 URL: https://issues.apache.org/jira/browse/SPARK-14670 Project: Spark Issue Type: Bug Compo

[jira] [Created] (SPARK-14647) Group SQLContext/HiveContext state into PersistentState

2016-04-14 Thread Andrew Or (JIRA)
Andrew Or created SPARK-14647: - Summary: Group SQLContext/HiveContext state into PersistentState Key: SPARK-14647 URL: https://issues.apache.org/jira/browse/SPARK-14647 Project: Spark Issue Type:

[jira] [Updated] (SPARK-14130) [Table related commands] Alter column

2016-04-14 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-14130: -- Assignee: Yin Huai (was: Andrew Or) > [Table related commands] Alter column >

[jira] [Resolved] (SPARK-14592) Create table like

2016-04-14 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-14592. --- Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 2.0.0 Target V

[jira] [Resolved] (SPARK-14558) In ClosureCleaner, clean the outer pointer if it's a REPL line object

2016-04-14 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-14558. --- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > In ClosureCleaner,

[jira] [Resolved] (SPARK-14617) Remove deprecated APIs in TaskMetrics

2016-04-14 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-14617. --- Resolution: Fixed > Remove deprecated APIs in TaskMetrics > - > >

[jira] [Resolved] (SPARK-14619) Track internal accumulators (metrics) by stage attempt rather than stage

2016-04-14 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-14619. --- Resolution: Fixed > Track internal accumulators (metrics) by stage attempt rather than stage > --

[jira] [Resolved] (SPARK-12133) Support dynamic allocation in Spark Streaming

2016-04-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12133. --- Resolution: Fixed Fix Version/s: 2.0.0 > Support dynamic allocation in Spark Streaming > -

[jira] [Updated] (SPARK-14445) Show columns/partitions

2016-04-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-14445: -- Assignee: Dilip Biswal > Show columns/partitions > --- > > Key: SPA

[jira] [Updated] (SPARK-14445) Support native execution of SHOW COLUMNS and SHOW PARTITIONS command

2016-04-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-14445: -- Issue Type: Sub-task (was: Improvement) Parent: SPARK-14118 > Support native execution of SHOW

[jira] [Updated] (SPARK-14445) Show columns/partitions

2016-04-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-14445: -- Summary: Show columns/partitions (was: show columns/partitions) > Show columns/partitions > --

[jira] [Updated] (SPARK-14445) show columns/partitions

2016-04-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-14445: -- Summary: show columns/partitions (was: Support native execution of SHOW COLUMNS and SHOW PARTITIONS c

[jira] [Updated] (SPARK-14499) Add tests to make sure drop partitions of an external table will not delete data

2016-04-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-14499: -- Assignee: Xiao Li > Add tests to make sure drop partitions of an external table will not delete > data

[jira] [Assigned] (SPARK-14441) Consolidate DDL tests

2016-04-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or reassigned SPARK-14441: - Assignee: Andrew Or > Consolidate DDL tests > - > > Key: SPA

[jira] [Reopened] (SPARK-14414) Make error messages consistent across DDLs

2016-04-12 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or reopened SPARK-14414: --- > Make error messages consistent across DDLs > -- > >

[jira] [Updated] (SPARK-14357) Tasks that fail due to CommitDeniedException (a side-effect of speculation) can cause job failure

2016-04-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-14357: -- Assignee: Jason Moore > Tasks that fail due to CommitDeniedException (a side-effect of speculation) >

[jira] [Updated] (SPARK-14357) Tasks that fail due to CommitDeniedException (a side-effect of speculation) can cause job failure

2016-04-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-14357: -- Target Version/s: 1.5.2, 1.6.2, 2.0.0 Fix Version/s: 1.5.2 2.0.0

[jira] [Resolved] (SPARK-14455) ReceiverTracker#allocatedExecutors throw NPE for receiver-less streaming application

2016-04-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-14455. --- Resolution: Fixed Assignee: Saisai Shao Fix Version/s: 2.0.0 Target Versi

[jira] [Resolved] (SPARK-14506) HiveClientImpl's toHiveTable misses a table property for external tables

2016-04-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-14506. --- Resolution: Fixed Assignee: Yin Huai Fix Version/s: 2.0.0 > HiveClientImpl's toHiveTa

[jira] [Resolved] (SPARK-14468) Always enable OutputCommitCoordinator

2016-04-07 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-14468. --- Resolution: Fixed Fix Version/s: 1.5.2 2.0.0

[jira] [Assigned] (SPARK-14388) Create Table

2016-04-07 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or reassigned SPARK-14388: - Assignee: Andrew Or > Create Table > > > Key: SPARK-14388 >

[jira] [Issue Comment Deleted] (SPARK-14410) SessionCatalog needs to check function existence

2016-04-07 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-14410: -- Comment: was deleted (was: User 'rekhajoshm' has created a pull request for this issue: https://github.

[jira] [Created] (SPARK-14468) Always enable OutputCommitCoordinator

2016-04-07 Thread Andrew Or (JIRA)
Andrew Or created SPARK-14468: - Summary: Always enable OutputCommitCoordinator Key: SPARK-14468 URL: https://issues.apache.org/jira/browse/SPARK-14468 Project: Spark Issue Type: Bug Com

[jira] [Assigned] (SPARK-14130) [Table related commands] Alter column

2016-04-06 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or reassigned SPARK-14130: - Assignee: Andrew Or > [Table related commands] Alter column > --

[jira] [Resolved] (SPARK-13112) CoarsedExecutorBackend register to driver should wait Executor was ready

2016-04-06 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13112. --- Resolution: Fixed Assignee: Shixiong Zhu Fix Version/s: 2.0.0 Target Vers

[jira] [Resolved] (SPARK-14444) Add a new scalastyle `NoScalaDoc` to prevent ScalaDoc-style multiline comments

2016-04-06 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-1. --- Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 2.0.0 Target Ver

[jira] [Resolved] (SPARK-14424) spark-class and related (spark-shell, etc.) no longer work with sbt build as documented

2016-04-06 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-14424. --- Resolution: Fixed Assignee: holdenk Fix Version/s: 2.0.0 Target Version/s

[jira] [Updated] (SPARK-14391) Flaky Test org.apache.spark.launcher.LauncherServerSuite.testCommunication

2016-04-06 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-14391: -- Fix Version/s: (was: 1.6.2) > Flaky Test org.apache.spark.launcher.LauncherServerSuite.testCommunic

[jira] [Updated] (SPARK-14391) Flaky Test org.apache.spark.launcher.LauncherServerSuite.testCommunication

2016-04-06 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-14391: -- Target Version/s: 2.0.0 (was: 1.6.2, 2.0.0) > Flaky Test org.apache.spark.launcher.LauncherServerSuite

[jira] [Resolved] (SPARK-14391) Flaky Test org.apache.spark.launcher.LauncherServerSuite.testCommunication

2016-04-06 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-14391. --- Resolution: Fixed Assignee: Marcelo Vanzin Fix Version/s: 2.0.0

[jira] [Updated] (SPARK-14127) [Table related commands] Describe table

2016-04-06 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-14127: -- Assignee: (was: Andrew Or) > [Table related commands] Describe table >

[jira] [Resolved] (SPARK-14384) Remove alterFunction from SessionCatalog

2016-04-06 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-14384. --- Resolution: Fixed Fix Version/s: 2.0.0 > Remove alterFunction from SessionCatalog > --

[jira] [Updated] (SPARK-14441) Consolidate DDL tests

2016-04-06 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-14441: -- Description: Today we have DDLSuite, DDLCommandSuite, HiveDDLCommandSuite. It's confusing whether a tes

<    1   2   3   4   5   6   7   8   9   10   >