[jira] [Resolved] (SPARK-12699) R driver process should start in a clean state

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-12699. -- Resolution: Incomplete > R driver process should start in a clean state >

[jira] [Resolved] (SPARK-15877) DataSource executed twice when using ORDER BY

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-15877. -- Resolution: Incomplete > DataSource executed twice when using ORDER BY >

[jira] [Resolved] (SPARK-15712) Proper temp table support

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-15712. -- Resolution: Incomplete > Proper temp table support > - > >

[jira] [Resolved] (SPARK-18469) Cannot make MLlib model predictions in Spark streaming with checkpointing

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-18469. -- Resolution: Incomplete > Cannot make MLlib model predictions in Spark streaming with

[jira] [Resolved] (SPARK-18014) Filters are incorrectly being grouped together when there is processing in between

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-18014. -- Resolution: Incomplete > Filters are incorrectly being grouped together when there is

[jira] [Resolved] (SPARK-16484) Incremental Cardinality estimation operations with Hyperloglog

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-16484. -- Resolution: Incomplete > Incremental Cardinality estimation operations with Hyperloglog >

[jira] [Resolved] (SPARK-13145) checkAnswer in SQL query suites should tolerate small float number error

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-13145. -- Resolution: Incomplete > checkAnswer in SQL query suites should tolerate small float number

[jira] [Resolved] (SPARK-8697) MatchIterator not serializable exception in RegexTokenizer

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-8697. - Resolution: Incomplete > MatchIterator not serializable exception in RegexTokenizer >

[jira] [Resolved] (SPARK-11465) Support multiple eigenvectors in power iteration clustering

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-11465. -- Resolution: Incomplete > Support multiple eigenvectors in power iteration clustering >

[jira] [Resolved] (SPARK-17911) Scheduler does not need messageScheduler for ResubmitFailedStages

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-17911. -- Resolution: Incomplete > Scheduler does not need messageScheduler for ResubmitFailedStages >

[jira] [Resolved] (SPARK-13283) Spark doesn't escape column names when creating table on JDBC

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-13283. -- Resolution: Incomplete > Spark doesn't escape column names when creating table on JDBC >

[jira] [Resolved] (SPARK-7494) spark.ml Model should call copyValues in construction

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-7494. - Resolution: Incomplete > spark.ml Model should call copyValues in construction >

[jira] [Resolved] (SPARK-12262) describe extended doesn't return table on detail info tabled stored as PARQUET format

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-12262. -- Resolution: Incomplete > describe extended doesn't return table on detail info tabled stored

[jira] [Resolved] (SPARK-15855) dataframe.R example fails with "java.io.IOException: No input paths specified in job"

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-15855. -- Resolution: Incomplete > dataframe.R example fails with "java.io.IOException: No input paths

[jira] [Resolved] (SPARK-10376) Once/When YARN permits it, only use POST for kill action

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-10376. -- Resolution: Incomplete > Once/When YARN permits it, only use POST for kill action >

[jira] [Resolved] (SPARK-12825) Spark-submit Jar URL loading fails on redirect

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-12825. -- Resolution: Incomplete > Spark-submit Jar URL loading fails on redirect >

[jira] [Resolved] (SPARK-10950) ApplicationHistoryInfo to include spark version; History Server to report incompatibility with later versions

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-10950. -- Resolution: Incomplete > ApplicationHistoryInfo to include spark version; History Server to

[jira] [Resolved] (SPARK-10583) Correctness test for Multilayer Perceptron using Weka Reference

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-10583. -- Resolution: Incomplete > Correctness test for Multilayer Perceptron using Weka Reference >

[jira] [Resolved] (SPARK-16362) Suport ArrayType and StructType in vectorization Parquet reader

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-16362. -- Resolution: Incomplete > Suport ArrayType and StructType in vectorization Parquet reader >

[jira] [Resolved] (SPARK-9744) Add RDD method to map with lag and lead

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-9744. - Resolution: Incomplete > Add RDD method to map with lag and lead >

[jira] [Resolved] (SPARK-18083) Locality Sensitive Hashing (LSH) - BitSampling

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-18083. -- Resolution: Incomplete > Locality Sensitive Hashing (LSH) - BitSampling >

[jira] [Resolved] (SPARK-17550) DataFrameWriter.partitionBy() should throw exception if column is not present in Dataframe

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-17550. -- Resolution: Incomplete > DataFrameWriter.partitionBy() should throw exception if column is

[jira] [Resolved] (SPARK-18225) job will miss when driver removed by master in spark streaming

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-18225. -- Resolution: Incomplete > job will miss when driver removed by master in spark streaming >

[jira] [Resolved] (SPARK-11400) BroadcastNestedLoopJoin should support LeftSemi join

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-11400. -- Resolution: Incomplete > BroadcastNestedLoopJoin should support LeftSemi join >

[jira] [Resolved] (SPARK-15831) Kryo 2.21 TreeMap serialization bug causes random job failures with RDDs of HBase puts

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-15831. -- Resolution: Incomplete > Kryo 2.21 TreeMap serialization bug causes random job failures with

[jira] [Resolved] (SPARK-8517) Improve the organization and style of MLlib's user guide

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-8517. - Resolution: Incomplete > Improve the organization and style of MLlib's user guide >

[jira] [Resolved] (SPARK-12493) Can't open "details" span of ExecutionsPage in IE11

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-12493. -- Resolution: Incomplete > Can't open "details" span of ExecutionsPage in IE11 >

[jira] [Resolved] (SPARK-7002) Persist on RDD fails the second time if the action is called on a child RDD without showing a FAILED message

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-7002. - Resolution: Incomplete > Persist on RDD fails the second time if the action is called on a child

[jira] [Resolved] (SPARK-15090) Spark Hive thriftserver can get 413 errors in Kerberos+AD deployments

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-15090. -- Resolution: Incomplete > Spark Hive thriftserver can get 413 errors in Kerberos+AD

[jira] [Resolved] (SPARK-13934) SqlParser.parseTableIdentifier cannot recognize table name start with scientific notation

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-13934. -- Resolution: Incomplete > SqlParser.parseTableIdentifier cannot recognize table name start

[jira] [Resolved] (SPARK-13317) SPARK_LOCAL_IP does not bind to public IP on Slaves

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-13317. -- Resolution: Incomplete > SPARK_LOCAL_IP does not bind to public IP on Slaves >

[jira] [Resolved] (SPARK-13585) addPyFile behavior change between 1.6 and before

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-13585. -- Resolution: Incomplete > addPyFile behavior change between 1.6 and before >

[jira] [Resolved] (SPARK-11666) Find the best `k` by cutting bisecting k-means cluster tree without recomputation

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-11666. -- Resolution: Incomplete > Find the best `k` by cutting bisecting k-means cluster tree without

[jira] [Resolved] (SPARK-10803) Allow users to write and query Parquet user-defined key-value metadata directly

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-10803. -- Resolution: Incomplete > Allow users to write and query Parquet user-defined key-value

[jira] [Resolved] (SPARK-13605) Bean encoder cannot handle nonbean properties - no way to Encode nonbean Java objects with columns

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-13605. -- Resolution: Incomplete > Bean encoder cannot handle nonbean properties - no way to Encode

[jira] [Resolved] (SPARK-14297) DAG visualization cropped in SparkUI

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-14297. -- Resolution: Incomplete > DAG visualization cropped in SparkUI >

[jira] [Resolved] (SPARK-13437) Add InternalColumn

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-13437. -- Resolution: Incomplete > Add InternalColumn > -- > > Key:

[jira] [Resolved] (SPARK-16424) Add support for Structured Streaming to the ML Pipeline API

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-16424. -- Resolution: Incomplete > Add support for Structured Streaming to the ML Pipeline API >

[jira] [Resolved] (SPARK-8113) SQL module test cleanup

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-8113. - Resolution: Incomplete > SQL module test cleanup > --- > >

[jira] [Resolved] (SPARK-10945) GraphX computes Pagerank with NaN (with some datasets)

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-10945. -- Resolution: Incomplete > GraphX computes Pagerank with NaN (with some datasets) >

[jira] [Resolved] (SPARK-10406) Document spark on yarn distributed cache symlink functionality

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-10406. -- Resolution: Incomplete > Document spark on yarn distributed cache symlink functionality >

[jira] [Resolved] (SPARK-13212) Provide a way to unregister data sources from a SQLContext

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-13212. -- Resolution: Incomplete > Provide a way to unregister data sources from a SQLContext >

[jira] [Resolved] (SPARK-10333) Add user guide for linear-methods.md columns

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-10333. -- Resolution: Incomplete > Add user guide for linear-methods.md columns >

[jira] [Resolved] (SPARK-15027) ALS.train should use DataFrame instead of RDD

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-15027. -- Resolution: Incomplete > ALS.train should use DataFrame instead of RDD >

[jira] [Resolved] (SPARK-13623) Relaxed mode for querying Dataframes, so columns that don't exist or have an incompatible schema return null rather than error

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-13623. -- Resolution: Incomplete > Relaxed mode for querying Dataframes, so columns that don't exist or

[jira] [Resolved] (SPARK-10638) spark streaming stop gracefully keeps the spark context

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-10638. -- Resolution: Incomplete > spark streaming stop gracefully keeps the spark context >

[jira] [Resolved] (SPARK-11664) Add methods to get bisecting k-means cluster structure

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-11664. -- Resolution: Incomplete > Add methods to get bisecting k-means cluster structure >

[jira] [Resolved] (SPARK-7871) Improve the outputPartitioning for HashOuterJoin(full outer join)

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-7871. - Resolution: Incomplete > Improve the outputPartitioning for HashOuterJoin(full outer join) >

[jira] [Resolved] (SPARK-7126) For spark.ml Classifiers, automatically index labels if they are not yet indexed

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-7126. - Resolution: Incomplete > For spark.ml Classifiers, automatically index labels if they are not

[jira] [Resolved] (SPARK-17815) Report committed offsets

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-17815. -- Resolution: Incomplete > Report committed offsets > > >

[jira] [Resolved] (SPARK-15872) Dataset of Array of Custom case class throws MissingRequirementError

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-15872. -- Resolution: Incomplete > Dataset of Array of Custom case class throws MissingRequirementError

[jira] [Resolved] (SPARK-11308) Change spark streaming's job scheduler logic to ensuer guaranteed order of batch processing

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-11308. -- Resolution: Incomplete > Change spark streaming's job scheduler logic to ensuer guaranteed

[jira] [Resolved] (SPARK-12141) Use Jackson to serialize all events when writing event log

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-12141. -- Resolution: Incomplete > Use Jackson to serialize all events when writing event log >

[jira] [Resolved] (SPARK-17606) New batches are not created when there are 1000 created after restarting streaming from checkpoint.

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-17606. -- Resolution: Incomplete > New batches are not created when there are 1000 created after

[jira] [Resolved] (SPARK-18095) There is a display problem in spark UI storage tab when rdd was persisted in multiple replicas

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-18095. -- Resolution: Incomplete > There is a display problem in spark UI storage tab when rdd was

[jira] [Resolved] (SPARK-10898) Setting spark.streaming.concurrentJobs causes blocks to be deleted before read

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-10898. -- Resolution: Incomplete > Setting spark.streaming.concurrentJobs causes blocks to be deleted

[jira] [Resolved] (SPARK-15666) Join on two tables generated from a same table throwing query analyzer issue

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-15666. -- Resolution: Incomplete > Join on two tables generated from a same table throwing query

[jira] [Resolved] (SPARK-11152) Streaming UI: Input sizes are 0 for makeup batches started from a checkpoint

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-11152. -- Resolution: Incomplete > Streaming UI: Input sizes are 0 for makeup batches started from a

[jira] [Resolved] (SPARK-9315) SparkR DataFrame improvements to be more R-friendly

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-9315. - Resolution: Incomplete > SparkR DataFrame improvements to be more R-friendly >

[jira] [Resolved] (SPARK-17856) JVM Crash during tests: pyspark.mllib.linalg.distributed

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-17856. -- Resolution: Incomplete > JVM Crash during tests: pyspark.mllib.linalg.distributed >

[jira] [Resolved] (SPARK-10010) Decide on a name for generating Linear/LogisticRegressionSummary on test set data

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-10010. -- Resolution: Incomplete > Decide on a name for generating Linear/LogisticRegressionSummary on

[jira] [Resolved] (SPARK-14032) Eliminate Unnecessary Distinct/Aggregate

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-14032. -- Resolution: Incomplete > Eliminate Unnecessary Distinct/Aggregate >

[jira] [Resolved] (SPARK-15119) DecisionTreeParams.minInfoGain does not have a validator

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-15119. -- Resolution: Incomplete > DecisionTreeParams.minInfoGain does not have a validator >

[jira] [Resolved] (SPARK-10370) After a stages map outputs are registered, all running attempts should be marked as zombies

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-10370. -- Resolution: Incomplete > After a stages map outputs are registered, all running attempts

[jira] [Resolved] (SPARK-8663) Dirver will be hang if there is a job submit during SparkContex stop Interval

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-8663. - Resolution: Incomplete > Dirver will be hang if there is a job submit during SparkContex stop

[jira] [Resolved] (SPARK-15729) Clarify that saveAs*File doesn't make sense with local FS in cluster context

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-15729. -- Resolution: Incomplete > Clarify that saveAs*File doesn't make sense with local FS in cluster

[jira] [Resolved] (SPARK-17221) Build File-based Test Cases for Using Join and Left-Semi Join

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-17221. -- Resolution: Incomplete > Build File-based Test Cases for Using Join and Left-Semi Join >

[jira] [Resolved] (SPARK-14593) Make currentVars work with splitExpressions to enable whole stage codegen for large input columns

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-14593. -- Resolution: Incomplete > Make currentVars work with splitExpressions to enable whole stage

[jira] [Resolved] (SPARK-15970) WARNing message related to persisting table to Hive Megastore while Spark SQL is running in-memory catalog mode

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-15970. -- Resolution: Incomplete > WARNing message related to persisting table to Hive Megastore while

[jira] [Resolved] (SPARK-12163) FPGrowth unusable on some datasets without extensive tweaking of the support threshold

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-12163. -- Resolution: Incomplete > FPGrowth unusable on some datasets without extensive tweaking of the

[jira] [Resolved] (SPARK-10910) spark.{executor,driver}.userClassPathFirst don't work for kryo (probably others)

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-10910. -- Resolution: Incomplete > spark.{executor,driver}.userClassPathFirst don't work for kryo

[jira] [Resolved] (SPARK-11458) add word count example for Dataset

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-11458. -- Resolution: Incomplete > add word count example for Dataset >

[jira] [Resolved] (SPARK-10347) Investigate the usage of normalizePath()

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-10347. -- Resolution: Incomplete > Investigate the usage of normalizePath() >

[jira] [Resolved] (SPARK-9108) Expose Kryo serializer buffer size

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-9108. - Resolution: Incomplete > Expose Kryo serializer buffer size >

[jira] [Resolved] (SPARK-16657) Replace children by innerChildren in InsertIntoHadoopFsRelationCommand and CreateHiveTableAsSelectCommand

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-16657. -- Resolution: Incomplete > Replace children by innerChildren in

[jira] [Resolved] (SPARK-9879) OOM in LIMIT clause with large number

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-9879. - Resolution: Incomplete > OOM in LIMIT clause with large number >

[jira] [Resolved] (SPARK-16937) Confusing behaviors when View and Temp View sharing the same names

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-16937. -- Resolution: Incomplete > Confusing behaviors when View and Temp View sharing the same names >

[jira] [Resolved] (SPARK-18096) Spark on have - 'Update' save mode

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-18096. -- Resolution: Incomplete > Spark on have - 'Update' save mode >

[jira] [Resolved] (SPARK-6920) Be more explicit about references to "executor" and "task" in Spark on Mesos

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-6920. - Resolution: Incomplete > Be more explicit about references to "executor" and "task" in Spark on

[jira] [Resolved] (SPARK-15673) Indefinite hanging issue with combination of cache, sort and unionAll

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-15673. -- Resolution: Incomplete > Indefinite hanging issue with combination of cache, sort and

[jira] [Resolved] (SPARK-12277) Use sparkIMain to compile and interpret string throw java.lang.ClassNotFoundException.

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-12277. -- Resolution: Incomplete > Use sparkIMain to compile and interpret string throw >

[jira] [Resolved] (SPARK-14784) Build SQL for EXISTS/IN subquery

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-14784. -- Resolution: Incomplete > Build SQL for EXISTS/IN subquery >

[jira] [Resolved] (SPARK-13800) Hive conf will be modified on multi-beeline connect to thriftserver

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-13800. -- Resolution: Incomplete > Hive conf will be modified on multi-beeline connect to thriftserver

[jira] [Resolved] (SPARK-13978) [GSoC 2016] Build monitoring UI and related infrastructure for Spark SQL and structured streaming

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-13978. -- Resolution: Incomplete > [GSoC 2016] Build monitoring UI and related infrastructure for Spark

[jira] [Resolved] (SPARK-10407) Possible Stack-overflow using InheritableThreadLocal nested-properties for SparkContext.localProperties

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-10407. -- Resolution: Incomplete > Possible Stack-overflow using InheritableThreadLocal

[jira] [Resolved] (SPARK-7960) Serialization problem when multiple receivers are specified in a loop

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-7960. - Resolution: Incomplete > Serialization problem when multiple receivers are specified in a loop >

[jira] [Resolved] (SPARK-11535) StringIndexer should handle empty String specially

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-11535. -- Resolution: Incomplete > StringIndexer should handle empty String specially >

[jira] [Resolved] (SPARK-17319) Move addJar from HiveSessionState to HiveSharedState

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-17319. -- Resolution: Incomplete > Move addJar from HiveSessionState to HiveSharedState >

[jira] [Resolved] (SPARK-11238) SparkR: Documentation change for merge function

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-11238. -- Resolution: Incomplete > SparkR: Documentation change for merge function >

[jira] [Resolved] (SPARK-16682) pyspark 1.6.0 not handling multiple level import when the necessary files are zipped

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-16682. -- Resolution: Incomplete > pyspark 1.6.0 not handling multiple level import when the necessary

[jira] [Resolved] (SPARK-13729) Reimplement the planning tests on SimpleTextRelation

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-13729. -- Resolution: Incomplete > Reimplement the planning tests on SimpleTextRelation >

[jira] [Resolved] (SPARK-10897) Custom job/stage names

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-10897. -- Resolution: Incomplete > Custom job/stage names > -- > >

[jira] [Resolved] (SPARK-17781) datetime is serialized as double inside dapply()

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-17781. -- Resolution: Incomplete > datetime is serialized as double inside dapply() >

[jira] [Resolved] (SPARK-14533) RowMatrix.computeCovariance inaccurate when values are very large

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-14533. -- Resolution: Incomplete > RowMatrix.computeCovariance inaccurate when values are very large >

[jira] [Resolved] (SPARK-10870) Criteo Display Advertising Challenge

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-10870. -- Resolution: Incomplete > Criteo Display Advertising Challenge >

[jira] [Resolved] (SPARK-8502) One character switches into uppercase, causing failures [serialization? shuffle?]

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-8502. - Resolution: Incomplete > One character switches into uppercase, causing failures [serialization?

[jira] [Resolved] (SPARK-16001) request that spark history server write a log entry whenever it (1) tries cleaning old event logs and (2) has found and deleted old event logs

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-16001. -- Resolution: Incomplete > request that spark history server write a log entry whenever it (1)

[jira] [Resolved] (SPARK-11160) CloudPickeSerializer conflicts with xmlrunner

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-11160. -- Resolution: Incomplete > CloudPickeSerializer conflicts with xmlrunner >

[jira] [Resolved] (SPARK-12185) Add Histogram support to Spark SQL/DataFrames

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-12185. -- Resolution: Incomplete > Add Histogram support to Spark SQL/DataFrames >

[jira] [Resolved] (SPARK-15907) Issue Exception when Not Enough Input Columns for Dynamic Partitioning

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-15907. -- Resolution: Incomplete > Issue Exception when Not Enough Input Columns for Dynamic

<    5   6   7   8   9   10   11   12   13   14   >