[jira] [Updated] (SPARK-1921) Allow duplicate jar files among the app jar and secondary jars in yarn-cluster mode

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-1921: Labels: bulk-closed (was: ) > Allow duplicate jar files among the app jar and secondary jars in

[jira] [Updated] (SPARK-3251) Clarify learning interfaces

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-3251: Labels: bulk-closed (was: ) > Clarify learning interfaces > > >

[jira] [Updated] (SPARK-4229) Create hadoop configuration in a consistent way

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-4229: Labels: bulk-closed (was: ) > Create hadoop configuration in a consistent way >

[jira] [Updated] (SPARK-3717) DecisionTree, RandomForest: Partition by feature

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-3717: Labels: bulk-closed (was: ) > DecisionTree, RandomForest: Partition by feature >

[jira] [Resolved] (SPARK-6619) Improve Jar caching on executors

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-6619. - Resolution: Incomplete > Improve Jar caching on executors > > >

[jira] [Resolved] (SPARK-3601) Kryo NPE for output operations on Avro complex Objects even after registering.

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-3601. - Resolution: Incomplete > Kryo NPE for output operations on Avro complex Objects even after

[jira] [Resolved] (SPARK-5431) SparkSubmitSuite and DriverSuite hang indefinitely if Master fails

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-5431. - Resolution: Incomplete > SparkSubmitSuite and DriverSuite hang indefinitely if Master fails >

[jira] [Resolved] (SPARK-4206) BlockManager warnings in local mode: "Block $blockId already exists on this machine; not re-adding it

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-4206. - Resolution: Incomplete > BlockManager warnings in local mode: "Block $blockId already exists on

[jira] [Resolved] (SPARK-4716) Avoid shuffle when all-to-all operation has single input and output partition

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-4716. - Resolution: Incomplete > Avoid shuffle when all-to-all operation has single input and output

[jira] [Resolved] (SPARK-1823) ExternalAppendOnlyMap can still OOM if one key is very large

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-1823. - Resolution: Incomplete > ExternalAppendOnlyMap can still OOM if one key is very large >

[jira] [Resolved] (SPARK-5079) Detect failed jobs / batches in Spark Streaming unit tests

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-5079. - Resolution: Incomplete > Detect failed jobs / batches in Spark Streaming unit tests >

[jira] [Resolved] (SPARK-4911) Report the inputs and outputs of Spark jobs so that external systems can track data lineage

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-4911. - Resolution: Incomplete > Report the inputs and outputs of Spark jobs so that external systems

[jira] [Resolved] (SPARK-5043) Implement updated Receiver API

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-5043. - Resolution: Incomplete > Implement updated Receiver API > -- > >

[jira] [Resolved] (SPARK-4488) Add control over map-side aggregation

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-4488. - Resolution: Incomplete > Add control over map-side aggregation >

[jira] [Resolved] (SPARK-5713) Support python serialization for RandomForest

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-5713. - Resolution: Incomplete > Support python serialization for RandomForest >

[jira] [Resolved] (SPARK-6798) Fix Date serialization in SparkR

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-6798. - Resolution: Incomplete > Fix Date serialization in SparkR > > >

[jira] [Resolved] (SPARK-6497) Class is not registered: scala.reflect.ManifestFactory$$anon$9

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-6497. - Resolution: Incomplete > Class is not registered: scala.reflect.ManifestFactory$$anon$9 >

[jira] [Resolved] (SPARK-2913) Spark's log4j.properties should always appear ahead of Hadoop's on classpath

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-2913. - Resolution: Incomplete > Spark's log4j.properties should always appear ahead of Hadoop's on

[jira] [Resolved] (SPARK-3115) Improve task broadcast latency for small tasks

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-3115. - Resolution: Incomplete > Improve task broadcast latency for small tasks >

[jira] [Resolved] (SPARK-6815) Support accumulators in R

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-6815. - Resolution: Incomplete > Support accumulators in R > - > >

[jira] [Resolved] (SPARK-2253) [Core] Disable partial aggregation automatically when reduction factor is low

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-2253. - Resolution: Incomplete > [Core] Disable partial aggregation automatically when reduction factor

[jira] [Resolved] (SPARK-4545) If first Spark Streaming batch fails, it waits 10x batch duration before stopping

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-4545. - Resolution: Incomplete > If first Spark Streaming batch fails, it waits 10x batch duration

[jira] [Resolved] (SPARK-1107) Add shutdown hook on executor stop to stop running tasks

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-1107. - Resolution: Incomplete > Add shutdown hook on executor stop to stop running tasks >

[jira] [Resolved] (SPARK-636) Add mechanism to run system management/configuration tasks on all workers

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-636. Resolution: Incomplete > Add mechanism to run system management/configuration tasks on all workers

[jira] [Resolved] (SPARK-2280) Java & Scala reference docs should describe function reference behavior.

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-2280. - Resolution: Incomplete > Java & Scala reference docs should describe function reference

[jira] [Resolved] (SPARK-2545) Add a diagnosis mode for closures to figure out what they're bringing in

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-2545. - Resolution: Incomplete > Add a diagnosis mode for closures to figure out what they're bringing

[jira] [Resolved] (SPARK-4500) Improve exact stratified sampling implementation

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-4500. - Resolution: Incomplete > Improve exact stratified sampling implementation >

[jira] [Resolved] (SPARK-5488) SPARK_LOCAL_IP not read by mesos scheduler

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-5488. - Resolution: Incomplete > SPARK_LOCAL_IP not read by mesos scheduler >

[jira] [Resolved] (SPARK-3916) recognize appended data in textFileStream()

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-3916. - Resolution: Incomplete > recognize appended data in textFileStream() >

[jira] [Resolved] (SPARK-5272) Refactor NaiveBayes to support discrete and continuous labels,features

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-5272. - Resolution: Incomplete > Refactor NaiveBayes to support discrete and continuous labels,features

[jira] [Resolved] (SPARK-5077) Map output statuses can still exceed spark.akka.frameSize

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-5077. - Resolution: Incomplete > Map output statuses can still exceed spark.akka.frameSize >

[jira] [Resolved] (SPARK-6026) Eliminate the bypassMergeThreshold parameter and associated hash-ish shuffle within the Sort shuffle code

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-6026. - Resolution: Incomplete > Eliminate the bypassMergeThreshold parameter and associated hash-ish

[jira] [Resolved] (SPARK-4144) Support incremental model training of Naive Bayes classifier

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-4144. - Resolution: Incomplete > Support incremental model training of Naive Bayes classifier >

[jira] [Resolved] (SPARK-2408) RDD.map(func) dependencies issue after checkpoint & count

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-2408. - Resolution: Incomplete > RDD.map(func) dependencies issue after checkpoint & count >

[jira] [Resolved] (SPARK-4698) Data-locality aware Partitioners

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-4698. - Resolution: Incomplete > Data-locality aware Partitioners > > >

[jira] [Resolved] (SPARK-3835) Spark applications that are killed should show up as "KILLED" or "CANCELLED" in the Spark UI

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-3835. - Resolution: Incomplete > Spark applications that are killed should show up as "KILLED" or

[jira] [Resolved] (SPARK-4489) JavaPairRDD.collectAsMap from checkpoint RDD may fail with ClassCastException

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-4489. - Resolution: Incomplete > JavaPairRDD.collectAsMap from checkpoint RDD may fail with

[jira] [Resolved] (SPARK-5091) Hooks for PySpark tasks

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-5091. - Resolution: Incomplete > Hooks for PySpark tasks > --- > >

[jira] [Resolved] (SPARK-6312) ChiSqTest should check for too few counts

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-6312. - Resolution: Incomplete > ChiSqTest should check for too few counts >

[jira] [Resolved] (SPARK-3306) Addition of external resource dependency in executors

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-3306. - Resolution: Incomplete > Addition of external resource dependency in executors >

[jira] [Resolved] (SPARK-5748) Improve Vectors.sqdist implementation

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-5748. - Resolution: Incomplete > Improve Vectors.sqdist implementation >

[jira] [Resolved] (SPARK-2581) complete or withdraw visitedStages optimization in DAGScheduler’s stageDependsOn

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-2581. - Resolution: Incomplete > complete or withdraw visitedStages optimization in DAGScheduler’s >

[jira] [Resolved] (SPARK-5490) KMeans costs can be incorrect if tasks need to be rerun

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-5490. - Resolution: Incomplete > KMeans costs can be incorrect if tasks need to be rerun >

[jira] [Resolved] (SPARK-5142) Possibly data may be ruined in Spark Streaming's WAL mechanism.

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-5142. - Resolution: Incomplete > Possibly data may be ruined in Spark Streaming's WAL mechanism. >

[jira] [Resolved] (SPARK-5480) GraphX pageRank: java.lang.ArrayIndexOutOfBoundsException:

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-5480. - Resolution: Incomplete > GraphX pageRank: java.lang.ArrayIndexOutOfBoundsException: >

[jira] [Resolved] (SPARK-4885) Enable fetched blocks to exceed 2 GB

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-4885. - Resolution: Incomplete > Enable fetched blocks to exceed 2 GB >

[jira] [Resolved] (SPARK-1921) Allow duplicate jar files among the app jar and secondary jars in yarn-cluster mode

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-1921. - Resolution: Incomplete > Allow duplicate jar files among the app jar and secondary jars in >

[jira] [Resolved] (SPARK-3380) DecisionTree: overflow and precision in aggregation

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-3380. - Resolution: Incomplete > DecisionTree: overflow and precision in aggregation >

[jira] [Resolved] (SPARK-3153) shuffle will run out of space when disks have different free space

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-3153. - Resolution: Incomplete > shuffle will run out of space when disks have different free space >

[jira] [Resolved] (SPARK-4868) Twitter DStream.map() throws "Task not serializable"

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-4868. - Resolution: Incomplete > Twitter DStream.map() throws "Task not serializable" >

[jira] [Resolved] (SPARK-4653) DAGScheduler refactoring and cleanup

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-4653. - Resolution: Incomplete > DAGScheduler refactoring and cleanup >

[jira] [Resolved] (SPARK-3717) DecisionTree, RandomForest: Partition by feature

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-3717. - Resolution: Incomplete > DecisionTree, RandomForest: Partition by feature >

[jira] [Resolved] (SPARK-5506) java.lang.ClassCastException using lambda expressions in combination of spark and Servlet

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-5506. - Resolution: Incomplete > java.lang.ClassCastException using lambda expressions in combination of

[jira] [Resolved] (SPARK-5497) start-all script not working properly on Standalone HA cluster (with Zookeeper)

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-5497. - Resolution: Incomplete > start-all script not working properly on Standalone HA cluster (with >

[jira] [Resolved] (SPARK-5685) Show warning when users open text files compressed with non-splittable algorithms like gzip

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-5685. - Resolution: Incomplete > Show warning when users open text files compressed with non-splittable

[jira] [Resolved] (SPARK-5045) Update FlumePollingReceiver to use updated Receiver API

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-5045. - Resolution: Incomplete > Update FlumePollingReceiver to use updated Receiver API >

[jira] [Resolved] (SPARK-6462) UpdateStateByKey should allow inner join of new with old keys

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-6462. - Resolution: Incomplete > UpdateStateByKey should allow inner join of new with old keys >

[jira] [Resolved] (SPARK-4540) Improve Executor ID Logging

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-4540. - Resolution: Incomplete > Improve Executor ID Logging > --- > >

[jira] [Resolved] (SPARK-3134) Update block locations asynchronously in TorrentBroadcast

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-3134. - Resolution: Incomplete > Update block locations asynchronously in TorrentBroadcast >

[jira] [Resolved] (SPARK-5674) Spark Job Explain Plan Proof of Concept

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-5674. - Resolution: Incomplete > Spark Job Explain Plan Proof of Concept >

[jira] [Resolved] (SPARK-6165) Aggregate and reduce should be able to work with very large number of tasks.

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-6165. - Resolution: Incomplete > Aggregate and reduce should be able to work with very large number of

[jira] [Resolved] (SPARK-3631) Add docs for checkpoint usage

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-3631. - Resolution: Incomplete > Add docs for checkpoint usage > - > >

[jira] [Resolved] (SPARK-6378) srcAttr in graph.triplets don't update when the size of graph is huge

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-6378. - Resolution: Incomplete > srcAttr in graph.triplets don't update when the size of graph is huge >

[jira] [Resolved] (SPARK-6415) Spark Streaming fail-fast: Stop scheduling jobs when a batch fails, and kills the app

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-6415. - Resolution: Incomplete > Spark Streaming fail-fast: Stop scheduling jobs when a batch fails, and

[jira] [Resolved] (SPARK-6148) cachedDataSourceTables may store outdated metadata if the table is updated from another HiveContext

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-6148. - Resolution: Incomplete > cachedDataSourceTables may store outdated metadata if the table is

[jira] [Resolved] (SPARK-6637) Test lambda weighting in implicit ALS

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-6637. - Resolution: Incomplete > Test lambda weighting in implicit ALS >

[jira] [Resolved] (SPARK-5104) Distributed Representations of Sentences and Documents

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-5104. - Resolution: Incomplete > Distributed Representations of Sentences and Documents >

[jira] [Resolved] (SPARK-3750) Log ulimit settings at warning if they are too low

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-3750. - Resolution: Incomplete > Log ulimit settings at warning if they are too low >

[jira] [Resolved] (SPARK-3504) KMeans optimization: track distances and unmoved cluster centers across iterations

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-3504. - Resolution: Incomplete > KMeans optimization: track distances and unmoved cluster centers across

[jira] [Resolved] (SPARK-799) Windows versions of the deploy scripts

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-799. Resolution: Incomplete > Windows versions of the deploy scripts >

[jira] [Resolved] (SPARK-4684) Add a script to run JDBC server on Windows

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-4684. - Resolution: Incomplete > Add a script to run JDBC server on Windows >

[jira] [Resolved] (SPARK-5575) Artificial neural networks for MLlib deep learning

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-5575. - Resolution: Incomplete > Artificial neural networks for MLlib deep learning >

[jira] [Resolved] (SPARK-5915) Spillable should check every N bytes rather than every 32 elements

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-5915. - Resolution: Incomplete > Spillable should check every N bytes rather than every 32 elements >

[jira] [Resolved] (SPARK-5372) Change the default storage level of window operators

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-5372. - Resolution: Incomplete > Change the default storage level of window operators >

[jira] [Resolved] (SPARK-6808) Checkpointing after zipPartitions results in NODE_LOCAL execution

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-6808. - Resolution: Incomplete > Checkpointing after zipPartitions results in NODE_LOCAL execution >

[jira] [Resolved] (SPARK-1863) Allowing user jars to take precedence over Spark jars does not work as expected

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-1863. - Resolution: Incomplete > Allowing user jars to take precedence over Spark jars does not work as

[jira] [Resolved] (SPARK-2610) When spark.serializer is set as org.apache.spark.serializer.KryoSerializer, importing a method causes multiple spark applications creations

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-2610. - Resolution: Incomplete > When spark.serializer is set as

[jira] [Resolved] (SPARK-6160) ChiSqSelector should keep test statistic info

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-6160. - Resolution: Incomplete > ChiSqSelector should keep test statistic info >

[jira] [Resolved] (SPARK-5046) Update KinesisReceiver to use updated Receiver API

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-5046. - Resolution: Incomplete > Update KinesisReceiver to use updated Receiver API >

[jira] [Resolved] (SPARK-5150) Strange implicit resolution behavior in Spark REPL

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-5150. - Resolution: Incomplete > Strange implicit resolution behavior in Spark REPL >

[jira] [Resolved] (SPARK-3244) Add fate sharing across related files in Jenkins

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-3244. - Resolution: Incomplete > Add fate sharing across related files in Jenkins >

[jira] [Resolved] (SPARK-1272) Don't fail job if some local directories are buggy

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-1272. - Resolution: Incomplete > Don't fail job if some local directories are buggy >

[jira] [Resolved] (SPARK-4524) Add documentation on packaging Python dependencies / installing them on clusters

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-4524. - Resolution: Incomplete > Add documentation on packaging Python dependencies / installing them on

[jira] [Resolved] (SPARK-6208) executor-memory does not work when using local cluster

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-6208. - Resolution: Incomplete > executor-memory does not work when using local cluster >

[jira] [Resolved] (SPARK-3735) Sending the factor directly or AtA based on the cost in ALS

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-3735. - Resolution: Incomplete > Sending the factor directly or AtA based on the cost in ALS >

[jira] [Resolved] (SPARK-2296) Refactor util.JsonProtocol for evolvability

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-2296. - Resolution: Incomplete > Refactor util.JsonProtocol for evolvability >

[jira] [Resolved] (SPARK-5918) Spark Thrift server reports metadata for VARCHAR column as STRING in result set schema

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-5918. - Resolution: Incomplete > Spark Thrift server reports metadata for VARCHAR column as STRING in

[jira] [Resolved] (SPARK-2723) Block Manager should catch exceptions in putValues

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-2723. - Resolution: Incomplete > Block Manager should catch exceptions in putValues >

[jira] [Resolved] (SPARK-5042) Updated Receiver API to make it easier to write reliable receivers that ack source

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-5042. - Resolution: Incomplete > Updated Receiver API to make it easier to write reliable receivers that

[jira] [Resolved] (SPARK-4679) Race condition in querying the Spark UI JSON endpoint when Jetty context handlers are added and removed

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-4679. - Resolution: Incomplete > Race condition in querying the Spark UI JSON endpoint when Jetty

[jira] [Resolved] (SPARK-5392) Shuffle spill size is shown as negative

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-5392. - Resolution: Incomplete > Shuffle spill size is shown as negative >

[jira] [Resolved] (SPARK-4112) Have a reserved copy of Sorter/SortDataFormat

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-4112. - Resolution: Incomplete > Have a reserved copy of Sorter/SortDataFormat >

[jira] [Resolved] (SPARK-6760) Sketch algorithms for SQL/DataFrames

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-6760. - Resolution: Incomplete > Sketch algorithms for SQL/DataFrames >

[jira] [Resolved] (SPARK-5646) Record output metrics for cache

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-5646. - Resolution: Incomplete > Record output metrics for cache > --- > >

[jira] [Resolved] (SPARK-6183) Skip bad workers when re-launching executors

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-6183. - Resolution: Incomplete > Skip bad workers when re-launching executors >

[jira] [Resolved] (SPARK-765) Test suite should run Spark example programs

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-765. Resolution: Incomplete > Test suite should run Spark example programs >

[jira] [Resolved] (SPARK-4940) Support more evenly distributing cores for Mesos mode

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-4940. - Resolution: Incomplete > Support more evenly distributing cores for Mesos mode >

[jira] [Resolved] (SPARK-3031) Create JsonSerializable and move JSON serialization from JsonProtocol into each class

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-3031. - Resolution: Incomplete > Create JsonSerializable and move JSON serialization from JsonProtocol

[jira] [Resolved] (SPARK-5225) Support coalesed Input Metrics from different sources

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-5225. - Resolution: Incomplete > Support coalesed Input Metrics from different sources >

[jira] [Resolved] (SPARK-1910) Add onBlockComplete API to receiver

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-1910. - Resolution: Incomplete > Add onBlockComplete API to receiver >

  1   2   3   4   5   6   7   8   9   10   >