[jira] [Resolved] (SPARK-24998) spark-sql will scan the same table repeatedly when doing multi-insert

[ https://issues.apache.org/jira/browse/SPARK-24998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24998. -- Resolution: Incomplete > spark-sql will scan the same table repeatedly when doing multi-insert

[jira] [Resolved] (SPARK-13998) HashingTF should extend UnaryTransformer

[ https://issues.apache.org/jira/browse/SPARK-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-13998. -- Resolution: Incomplete > HashingTF should extend UnaryTransformer > --

[jira] [Resolved] (SPARK-22402) Allow fetcher URIs to be downloaded to specific locations relative to Mesos Sandbox

[ https://issues.apache.org/jira/browse/SPARK-22402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-22402. -- Resolution: Incomplete > Allow fetcher URIs to be downloaded to specific locations relative to

[jira] [Resolved] (SPARK-24264) [Structured Streaming] Remove 'mergeSchema' option from Parquet source configuration

[ https://issues.apache.org/jira/browse/SPARK-24264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24264. -- Resolution: Incomplete > [Structured Streaming] Remove 'mergeSchema' option from Parquet sourc

[jira] [Resolved] (SPARK-24554) Add MapType Support for Arrow in PySpark

[ https://issues.apache.org/jira/browse/SPARK-24554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24554. -- Resolution: Incomplete > Add MapType Support for Arrow in PySpark > --

[jira] [Resolved] (SPARK-24011) Cache rdd's immediate parent ShuffleDependencies to accelerate getShuffleDependencies()

[ https://issues.apache.org/jira/browse/SPARK-24011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24011. -- Resolution: Incomplete > Cache rdd's immediate parent ShuffleDependencies to accelerate > get

[jira] [Resolved] (SPARK-10795) FileNotFoundException while deploying pyspark job on cluster

[ https://issues.apache.org/jira/browse/SPARK-10795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-10795. -- Resolution: Incomplete > FileNotFoundException while deploying pyspark job on cluster > --

[jira] [Resolved] (SPARK-23531) When explain, plan's output should include attribute type info

[ https://issues.apache.org/jira/browse/SPARK-23531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23531. -- Resolution: Incomplete > When explain, plan's output should include attribute type info >

[jira] [Resolved] (SPARK-5158) Allow for keytab-based HDFS security in Standalone mode

[ https://issues.apache.org/jira/browse/SPARK-5158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-5158. - Resolution: Incomplete > Allow for keytab-based HDFS security in Standalone mode > --

[jira] [Resolved] (SPARK-24189) Spark Strcutured Streaming not working with the Kafka Transactions

[ https://issues.apache.org/jira/browse/SPARK-24189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24189. -- Resolution: Incomplete > Spark Strcutured Streaming not working with the Kafka Transactions >

[jira] [Resolved] (SPARK-25171) After restart, StreamingContext is replaying the last successful micro-batch right before the stop

[ https://issues.apache.org/jira/browse/SPARK-25171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25171. -- Resolution: Incomplete > After restart, StreamingContext is replaying the last successful micr

[jira] [Resolved] (SPARK-24280) Speed up indexing of files in object stores by using listFiles(path, recursive=true)

[ https://issues.apache.org/jira/browse/SPARK-24280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24280. -- Resolution: Incomplete > Speed up indexing of files in object stores by using listFiles(path,

[jira] [Resolved] (SPARK-22869) 64KB JVM bytecode limit problem with filter

[ https://issues.apache.org/jira/browse/SPARK-22869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-22869. -- Resolution: Incomplete > 64KB JVM bytecode limit problem with filter > ---

[jira] [Resolved] (SPARK-23954) Converting spark dataframe containing int64 fields to R dataframes leads to impredictable errors.

[ https://issues.apache.org/jira/browse/SPARK-23954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23954. -- Resolution: Incomplete > Converting spark dataframe containing int64 fields to R dataframes le

[jira] [Resolved] (SPARK-14834) Force adding doc for new api in pyspark with @since annotation

[ https://issues.apache.org/jira/browse/SPARK-14834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-14834. -- Resolution: Incomplete > Force adding doc for new api in pyspark with @since annotation >

[jira] [Resolved] (SPARK-21722) Enable timezone-aware timestamp type when creating Pandas DataFrame.

[ https://issues.apache.org/jira/browse/SPARK-21722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-21722. -- Resolution: Incomplete > Enable timezone-aware timestamp type when creating Pandas DataFrame.

[jira] [Resolved] (SPARK-23968) allow reading JSON that is composed of pure maps

[ https://issues.apache.org/jira/browse/SPARK-23968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23968. -- Resolution: Incomplete > allow reading JSON that is composed of pure maps > --

[jira] [Resolved] (SPARK-22926) Respect table-level conf compression codec `Compression` in multiple scenarios

[ https://issues.apache.org/jira/browse/SPARK-22926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-22926. -- Resolution: Incomplete > Respect table-level conf compression codec `Compression` in multiple

[jira] [Resolved] (SPARK-21389) ALS recommendForAll optimization uses Native BLAS

[ https://issues.apache.org/jira/browse/SPARK-21389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-21389. -- Resolution: Incomplete > ALS recommendForAll optimization uses Native BLAS > -

[jira] [Resolved] (SPARK-24735) Improve exception when mixing up pandas_udf types

[ https://issues.apache.org/jira/browse/SPARK-24735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24735. -- Resolution: Incomplete > Improve exception when mixing up pandas_udf types > -

[jira] [Resolved] (SPARK-8799) OneVsRestModel should extend ClassificationModel

[ https://issues.apache.org/jira/browse/SPARK-8799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-8799. - Resolution: Incomplete > OneVsRestModel should extend ClassificationModel > -

[jira] [Resolved] (SPARK-24693) Row order preservation for operations on MLlib IndexedRowMatrix

[ https://issues.apache.org/jira/browse/SPARK-24693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24693. -- Resolution: Incomplete > Row order preservation for operations on MLlib IndexedRowMatrix > ---

[jira] [Resolved] (SPARK-22035) the value of statistical logicalPlan.stats.sizeInBytes which is not expected

[ https://issues.apache.org/jira/browse/SPARK-22035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-22035. -- Resolution: Incomplete > the value of statistical logicalPlan.stats.sizeInBytes which is not e

[jira] [Resolved] (SPARK-9140) Replace TimeTracker by Stopwatch

[ https://issues.apache.org/jira/browse/SPARK-9140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-9140. - Resolution: Incomplete > Replace TimeTracker by Stopwatch > > >

[jira] [Resolved] (SPARK-22600) Fix 64kb limit for deeply nested expressions under wholestage codegen

[ https://issues.apache.org/jira/browse/SPARK-22600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-22600. -- Resolution: Incomplete > Fix 64kb limit for deeply nested expressions under wholestage codegen

[jira] [Resolved] (SPARK-24258) SPIP: Improve PySpark support for ML Matrix and Vector types

[ https://issues.apache.org/jira/browse/SPARK-24258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24258. -- Resolution: Incomplete > SPIP: Improve PySpark support for ML Matrix and Vector types > --

[jira] [Resolved] (SPARK-25198) org.apache.spark.sql.catalyst.parser.ParseException: DataType json is not supported.

[ https://issues.apache.org/jira/browse/SPARK-25198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25198. -- Resolution: Incomplete > org.apache.spark.sql.catalyst.parser.ParseException: DataType json is

[jira] [Resolved] (SPARK-23952) remove type parameter in DataReaderFactory

[ https://issues.apache.org/jira/browse/SPARK-23952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23952. -- Resolution: Incomplete > remove type parameter in DataReaderFactory >

[jira] [Resolved] (SPARK-23879) Introduce MemoryBlock API instead of Platform API with Object

[ https://issues.apache.org/jira/browse/SPARK-23879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23879. -- Resolution: Incomplete > Introduce MemoryBlock API instead of Platform API with Object >

[jira] [Resolved] (SPARK-23995) initial job has not accept any resources and executor keep exit

[ https://issues.apache.org/jira/browse/SPARK-23995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23995. -- Resolution: Incomplete > initial job has not accept any resources and executor keep exit > ---

[jira] [Resolved] (SPARK-2620) case class cannot be used as key for reduce

[ https://issues.apache.org/jira/browse/SPARK-2620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-2620. - Resolution: Incomplete > case class cannot be used as key for reduce > --

[jira] [Resolved] (SPARK-23543) Automatic Module creation fails in Java 9

[ https://issues.apache.org/jira/browse/SPARK-23543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23543. -- Resolution: Incomplete > Automatic Module creation fails in Java 9 > -

[jira] [Resolved] (SPARK-18649) sc.textFile(my_file).collect() raises socket.timeout on large files

[ https://issues.apache.org/jira/browse/SPARK-18649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-18649. -- Resolution: Incomplete > sc.textFile(my_file).collect() raises socket.timeout on large files >

[jira] [Resolved] (SPARK-18082) Locality Sensitive Hashing (LSH) - SignRandomProjection

[ https://issues.apache.org/jira/browse/SPARK-18082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-18082. -- Resolution: Incomplete > Locality Sensitive Hashing (LSH) - SignRandomProjection > ---

[jira] [Resolved] (SPARK-23171) Reduce the time costs of the rule runs that do not change the plans

[ https://issues.apache.org/jira/browse/SPARK-23171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23171. -- Resolution: Incomplete > Reduce the time costs of the rule runs that do not change the plans

[jira] [Resolved] (SPARK-24631) Cannot up cast column from bigint to smallint as it may truncate

[ https://issues.apache.org/jira/browse/SPARK-24631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24631. -- Resolution: Incomplete > Cannot up cast column from bigint to smallint as it may truncate > --

[jira] [Resolved] (SPARK-24218) Allow Configuration of DynamoDbEndpointUrl in KinesisReceiver

[ https://issues.apache.org/jira/browse/SPARK-24218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24218. -- Resolution: Incomplete > Allow Configuration of DynamoDbEndpointUrl in KinesisReceiver > -

[jira] [Resolved] (SPARK-24362) SUM function precision issue

[ https://issues.apache.org/jira/browse/SPARK-24362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24362. -- Resolution: Incomplete > SUM function precision issue > > >

[jira] [Resolved] (SPARK-25593) JDBC write Impala, `truncate` true option in Overwrite mode for JDBC DataFrameWriter is dropping and creating the table instead of truncating.

[ https://issues.apache.org/jira/browse/SPARK-25593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25593. -- Resolution: Incomplete > JDBC write Impala, `truncate` true option in Overwrite mode for JDBC

[jira] [Resolved] (SPARK-25316) Spark error - ERROR ContextCleaner: Error cleaning broadcast 22, Exception thrown in awaitResult:

[ https://issues.apache.org/jira/browse/SPARK-25316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25316. -- Resolution: Incomplete > Spark error - ERROR ContextCleaner: Error cleaning broadcast 22, Exc

[jira] [Resolved] (SPARK-23673) PySpark dayofweek does not conform with ISO 8601

[ https://issues.apache.org/jira/browse/SPARK-23673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23673. -- Resolution: Incomplete > PySpark dayofweek does not conform with ISO 8601 > --

[jira] [Resolved] (SPARK-23056) parse_url regression when switched to using java.net.URI instead of java.net.URL

[ https://issues.apache.org/jira/browse/SPARK-23056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23056. -- Resolution: Incomplete > parse_url regression when switched to using java.net.URI instead of

[jira] [Resolved] (SPARK-22743) Consolidate logic for handling spark.driver.memoryOverhead and spark.executor.memoryOverhead

[ https://issues.apache.org/jira/browse/SPARK-22743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-22743. -- Resolution: Incomplete > Consolidate logic for handling spark.driver.memoryOverhead and > spa

[jira] [Resolved] (SPARK-22741) Add global aggregate for typed aggregation

[ https://issues.apache.org/jira/browse/SPARK-22741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-22741. -- Resolution: Incomplete > Add global aggregate for typed aggregation >

[jira] [Resolved] (SPARK-22748) Error in query: grouping_id() can only be used with GroupingSets/Cube/Rollup;

[ https://issues.apache.org/jira/browse/SPARK-22748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-22748. -- Resolution: Incomplete > Error in query: grouping_id() can only be used with GroupingSets/Cube

[jira] [Resolved] (SPARK-20915) lpad/rpad with empty pad string different from MySQL

[ https://issues.apache.org/jira/browse/SPARK-20915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-20915. -- Resolution: Incomplete > lpad/rpad with empty pad string different from MySQL > --

[jira] [Resolved] (SPARK-19680) Offsets out of range with no configured reset policy for partitions

[ https://issues.apache.org/jira/browse/SPARK-19680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-19680. -- Resolution: Incomplete > Offsets out of range with no configured reset policy for partitions >

[jira] [Resolved] (SPARK-11136) Warm-start support for ML estimator

[ https://issues.apache.org/jira/browse/SPARK-11136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-11136. -- Resolution: Incomplete > Warm-start support for ML estimator > ---

[jira] [Resolved] (SPARK-24394) Nodes in decision tree sometimes have negative impurity values

[ https://issues.apache.org/jira/browse/SPARK-24394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24394. -- Resolution: Incomplete > Nodes in decision tree sometimes have negative impurity values >

[jira] [Resolved] (SPARK-23745) Remove the directories of the “hive.downloaded.resources.dir” when HiveThriftServer2 stopped

[ https://issues.apache.org/jira/browse/SPARK-23745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23745. -- Resolution: Incomplete > Remove the directories of the “hive.downloaded.resources.dir” when >

[jira] [Resolved] (SPARK-25329) Support passing Kerberos configuration information

[ https://issues.apache.org/jira/browse/SPARK-25329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25329. -- Resolution: Incomplete > Support passing Kerberos configuration information >

[jira] [Resolved] (SPARK-24469) Support collations in Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-24469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24469. -- Resolution: Incomplete > Support collations in Spark SQL > --- > >

[jira] [Resolved] (SPARK-23650) Slow SparkR udf (dapply)

[ https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23650. -- Resolution: Incomplete > Slow SparkR udf (dapply) > > >

[jira] [Resolved] (SPARK-24269) Infer nullability rather than declaring all columns as nullable

[ https://issues.apache.org/jira/browse/SPARK-24269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24269. -- Resolution: Incomplete > Infer nullability rather than declaring all columns as nullable > ---

[jira] [Resolved] (SPARK-23536) Update each Data frame row with a random value

[ https://issues.apache.org/jira/browse/SPARK-23536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23536. -- Resolution: Incomplete > Update each Data frame row with a random value >

[jira] [Resolved] (SPARK-23221) Fix KafkaContinuousSourceStressForDontFailOnDataLossSuite to run with enough cores

[ https://issues.apache.org/jira/browse/SPARK-23221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23221. -- Resolution: Incomplete > Fix KafkaContinuousSourceStressForDontFailOnDataLossSuite to run with

[jira] [Resolved] (SPARK-23258) Should not split Arrow record batches based on row count

[ https://issues.apache.org/jira/browse/SPARK-23258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23258. -- Resolution: Incomplete > Should not split Arrow record batches based on row count > --

[jira] [Resolved] (SPARK-22055) Port release scripts

[ https://issues.apache.org/jira/browse/SPARK-22055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-22055. -- Resolution: Incomplete > Port release scripts > > > Key:

[jira] [Resolved] (SPARK-25397) SparkSession.conf fails when given default value with Python 3

[ https://issues.apache.org/jira/browse/SPARK-25397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25397. -- Resolution: Incomplete > SparkSession.conf fails when given default value with Python 3 >

[jira] [Resolved] (SPARK-20592) Alter table concatenate is not working as expected.

[ https://issues.apache.org/jira/browse/SPARK-20592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-20592. -- Resolution: Incomplete > Alter table concatenate is not working as expected. > ---

[jira] [Resolved] (SPARK-22943) OneHotEncoder supports manual specification of categorySizes

[ https://issues.apache.org/jira/browse/SPARK-22943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-22943. -- Resolution: Incomplete > OneHotEncoder supports manual specification of categorySizes > --

[jira] [Resolved] (SPARK-25109) spark python should retry reading another datanode if the first one fails to connect

[ https://issues.apache.org/jira/browse/SPARK-25109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25109. -- Resolution: Incomplete > spark python should retry reading another datanode if the first one f

[jira] [Resolved] (SPARK-22887) ML test for StructuredStreaming: spark.ml.fpm

[ https://issues.apache.org/jira/browse/SPARK-22887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-22887. -- Resolution: Incomplete > ML test for StructuredStreaming: spark.ml.fpm > -

[jira] [Resolved] (SPARK-24616) Need to retreive free memory on command prompt on DSE cluster

[ https://issues.apache.org/jira/browse/SPARK-24616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24616. -- Resolution: Incomplete > Need to retreive free memory on command prompt on DSE cluster > -

[jira] [Resolved] (SPARK-22723) Add support for other data types and add mode info to ImageSchema

[ https://issues.apache.org/jira/browse/SPARK-22723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-22723. -- Resolution: Incomplete > Add support for other data types and add mode info to ImageSchema >

[jira] [Resolved] (SPARK-24512) SparkSQL ThriftServer port (ie 10015) supports TLSv1.0

[ https://issues.apache.org/jira/browse/SPARK-24512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24512. -- Resolution: Incomplete > SparkSQL ThriftServer port (ie 10015) supports TLSv1.0 >

[jira] [Resolved] (SPARK-17570) Avoid Hash and Exchange in Sort Merge join if bucketing factor is multiple for tables

[ https://issues.apache.org/jira/browse/SPARK-17570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-17570. -- Resolution: Incomplete > Avoid Hash and Exchange in Sort Merge join if bucketing factor is mul

[jira] [Resolved] (SPARK-24756) Incorrect Statistics

[ https://issues.apache.org/jira/browse/SPARK-24756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24756. -- Resolution: Incomplete > Incorrect Statistics > > > Key:

[jira] [Resolved] (SPARK-23074) Dataframe-ified zipwithindex

[ https://issues.apache.org/jira/browse/SPARK-23074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23074. -- Resolution: Incomplete > Dataframe-ified zipwithindex > > >

[jira] [Resolved] (SPARK-24922) Iterative rdd union + reduceByKey operations on small dataset leads to "No space left on device" error on account of lot of shuffle spill.

[ https://issues.apache.org/jira/browse/SPARK-24922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24922. -- Resolution: Incomplete > Iterative rdd union + reduceByKey operations on small dataset leads t

[jira] [Resolved] (SPARK-25537) spark.pyspark.driver.python when set in code doesnt work

[ https://issues.apache.org/jira/browse/SPARK-25537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25537. -- Resolution: Incomplete > spark.pyspark.driver.python when set in code doesnt work > --

[jira] [Resolved] (SPARK-23058) Show create table can't show non printable field delim

[ https://issues.apache.org/jira/browse/SPARK-23058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23058. -- Resolution: Incomplete > Show create table can't show non printable field delim >

[jira] [Resolved] (SPARK-22005) CrossValidator, TrainValidationSplit dump sub models to disk when fitting: Python API

[ https://issues.apache.org/jira/browse/SPARK-22005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-22005. -- Resolution: Incomplete > CrossValidator, TrainValidationSplit dump sub models to disk when fi

[jira] [Resolved] (SPARK-16483) Unifying struct fields and columns

[ https://issues.apache.org/jira/browse/SPARK-16483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-16483. -- Resolution: Incomplete > Unifying struct fields and columns >

[jira] [Resolved] (SPARK-24358) createDataFrame in Python 3 should be able to infer bytes type as Binary type

[ https://issues.apache.org/jira/browse/SPARK-24358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24358. -- Resolution: Incomplete > createDataFrame in Python 3 should be able to infer bytes type as Bin

[jira] [Resolved] (SPARK-22461) Move Spark ML model summaries into a dedicated package

[ https://issues.apache.org/jira/browse/SPARK-22461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-22461. -- Resolution: Incomplete > Move Spark ML model summaries into a dedicated package >

[jira] [Resolved] (SPARK-21166) Automated ML persistence

[ https://issues.apache.org/jira/browse/SPARK-21166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-21166. -- Resolution: Incomplete > Automated ML persistence > > >

[jira] [Resolved] (SPARK-23255) Add user guide and examples for DataFrame image reading functions

[ https://issues.apache.org/jira/browse/SPARK-23255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23255. -- Resolution: Incomplete > Add user guide and examples for DataFrame image reading functions > -

[jira] [Resolved] (SPARK-23777) Missing DAG arrows between stages

[ https://issues.apache.org/jira/browse/SPARK-23777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23777. -- Resolution: Incomplete > Missing DAG arrows between stages > -

[jira] [Resolved] (SPARK-23227) Add user guide entry for collecting sub models for cross-validation classes

[ https://issues.apache.org/jira/browse/SPARK-23227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23227. -- Resolution: Incomplete > Add user guide entry for collecting sub models for cross-validation c

[jira] [Resolved] (SPARK-24406) Exposing custom spark scala ml transformers in pyspark

[ https://issues.apache.org/jira/browse/SPARK-24406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24406. -- Resolution: Incomplete > Exposing custom spark scala ml transformers in pyspark > ---

[jira] [Resolved] (SPARK-24354) Adding support for quoteMode in Spark's build in CSV DataFrameWriter

[ https://issues.apache.org/jira/browse/SPARK-24354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24354. -- Resolution: Incomplete > Adding support for quoteMode in Spark's build in CSV DataFrameWriter

[jira] [Resolved] (SPARK-19903) Watermark metadata is lost when using resolved attributes

[ https://issues.apache.org/jira/browse/SPARK-19903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-19903. -- Resolution: Incomplete > Watermark metadata is lost when using resolved attributes > -

[jira] [Resolved] (SPARK-25340) Pushes down Sample beneath deterministic Project

[ https://issues.apache.org/jira/browse/SPARK-25340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25340. -- Resolution: Incomplete > Pushes down Sample beneath deterministic Project > --

[jira] [Resolved] (SPARK-24405) parameter for python worker timeout

[ https://issues.apache.org/jira/browse/SPARK-24405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24405. -- Resolution: Incomplete > parameter for python worker timeout > ---

[jira] [Resolved] (SPARK-17694) convert DataFrame to DataSet should check columns match

[ https://issues.apache.org/jira/browse/SPARK-17694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-17694. -- Resolution: Incomplete > convert DataFrame to DataSet should check columns match > ---

[jira] [Resolved] (SPARK-23744) Memory leak in ReadableChannelFileRegion

[ https://issues.apache.org/jira/browse/SPARK-23744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23744. -- Resolution: Incomplete > Memory leak in ReadableChannelFileRegion > --

[jira] [Resolved] (SPARK-14585) Provide accessor methods for Pipeline stages

[ https://issues.apache.org/jira/browse/SPARK-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-14585. -- Resolution: Incomplete > Provide accessor methods for Pipeline stages > --

[jira] [Resolved] (SPARK-25377) spark sql dataframe cache is invalid

[ https://issues.apache.org/jira/browse/SPARK-25377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25377. -- Resolution: Incomplete > spark sql dataframe cache is invalid > --

[jira] [Resolved] (SPARK-25361) Support for Kinesis Client Library 2.0

[ https://issues.apache.org/jira/browse/SPARK-25361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25361. -- Resolution: Incomplete > Support for Kinesis Client Library 2.0 >

[jira] [Resolved] (SPARK-23073) Fix incorrect R doc page header for generated sql functions

[ https://issues.apache.org/jira/browse/SPARK-23073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23073. -- Resolution: Incomplete > Fix incorrect R doc page header for generated sql functions > ---

[jira] [Resolved] (SPARK-16707) TransportClientFactory.createClient may throw NPE

[ https://issues.apache.org/jira/browse/SPARK-16707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-16707. -- Resolution: Incomplete > TransportClientFactory.createClient may throw NPE > -

[jira] [Resolved] (SPARK-25244) [Python] Setting `spark.sql.session.timeZone` only partially respected

[ https://issues.apache.org/jira/browse/SPARK-25244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25244. -- Resolution: Incomplete > [Python] Setting `spark.sql.session.timeZone` only partially respecte

[jira] [Resolved] (SPARK-23983) Disable X-Frame-Options from Spark UI response headers if explicitly configured

[ https://issues.apache.org/jira/browse/SPARK-23983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23983. -- Resolution: Incomplete > Disable X-Frame-Options from Spark UI response headers if explicitly

[jira] [Resolved] (SPARK-12449) Pushing down arbitrary logical plans to data sources

[ https://issues.apache.org/jira/browse/SPARK-12449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-12449. -- Resolution: Incomplete > Pushing down arbitrary logical plans to data sources > --

[jira] [Resolved] (SPARK-21972) Allow users to control input data persistence in ML Estimators via a handlePersistence ml.Param

[ https://issues.apache.org/jira/browse/SPARK-21972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-21972. -- Resolution: Incomplete > Allow users to control input data persistence in ML Estimators via a

[jira] [Resolved] (SPARK-20598) Iterative checkpoints do not get removed from HDFS

[ https://issues.apache.org/jira/browse/SPARK-20598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-20598. -- Resolution: Incomplete > Iterative checkpoints do not get removed from HDFS >

[jira] [Resolved] (SPARK-24260) Support for multi-statement SQL in SparkSession.sql API

[ https://issues.apache.org/jira/browse/SPARK-24260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24260. -- Resolution: Incomplete > Support for multi-statement SQL in SparkSession.sql API > ---

[jira] [Resolved] (SPARK-22954) ANALYZE TABLE fails with NoSuchTableException for temporary tables (but should have reported "not supported on views")

[ https://issues.apache.org/jira/browse/SPARK-22954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-22954. -- Resolution: Incomplete > ANALYZE TABLE fails with NoSuchTableException for temporary tables (b

[jira] [Resolved] (SPARK-24390) confusion of columns in projection after WITH ROLLUP

[ https://issues.apache.org/jira/browse/SPARK-24390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24390. -- Resolution: Incomplete > confusion of columns in projection after WITH ROLLUP > --

<    1   2   3   4   5   6   >