[jira] [Commented] (SPARK-4832) some other processes might take the daemon pid

[ https://issues.apache.org/jira/browse/SPARK-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243830#comment-14243830 ] Apache Spark commented on SPARK-4832: - User 'WangTaoTheTonic' has created a pull reque

[jira] [Created] (SPARK-4832) some other processes might take the daemon pid

WangTaoTheTonic created SPARK-4832: -- Summary: some other processes might take the daemon pid Key: SPARK-4832 URL: https://issues.apache.org/jira/browse/SPARK-4832 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-4829) eliminate expressions calculation in count expression

[ https://issues.apache.org/jira/browse/SPARK-4829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4829. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 3676 [https:/

[jira] [Resolved] (SPARK-4742) The name of Parquet File generated by AppendingParquetOutputFormat should be zero padded

[ https://issues.apache.org/jira/browse/SPARK-4742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4742. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 3602 [https:/

[jira] [Resolved] (SPARK-4825) CTAS fails to resolve when created using saveAsTable

[ https://issues.apache.org/jira/browse/SPARK-4825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4825. - Resolution: Fixed Fix Version/s: 1.2.1 Target Version/s: 1.2.1 (was: 1.2.

[jira] [Resolved] (SPARK-4828) sum and avg over empty table should return null

[ https://issues.apache.org/jira/browse/SPARK-4828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4828. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 3675 [https:/

[jira] [Resolved] (SPARK-4293) Make Cast be able to handle complex types.

[ https://issues.apache.org/jira/browse/SPARK-4293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4293. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 3150 [https:/

[jira] [Resolved] (SPARK-4639) Pass maxIterations in as a parameter in Analyzer

[ https://issues.apache.org/jira/browse/SPARK-4639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4639. - Resolution: Fixed Issue resolved by pull request 3499 [https://github.com/apache/spark/pul

[jira] [Resolved] (SPARK-4662) Whitelist more Hive unittest

[ https://issues.apache.org/jira/browse/SPARK-4662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4662. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 3522 [https:/

[jira] [Resolved] (SPARK-4713) SchemaRDD.unpersist() should not raise exception if it is not cached.

[ https://issues.apache.org/jira/browse/SPARK-4713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4713. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 3572 [https:/

[jira] [Commented] (SPARK-4817) [streaming]Print the specified number of data and handle all of the elements in RDD

[ https://issues.apache.org/jira/browse/SPARK-4817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243683#comment-14243683 ] 宿荣全 commented on SPARK-4817: [~srowen] I‘m sorry that didn't describe the problem clearly. If

[jira] [Resolved] (SPARK-3636) It is not friendly to interrupt a Job when user passes different storageLevels to a RDD

[ https://issues.apache.org/jira/browse/SPARK-3636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3636. -- Resolution: Won't Fix Target Version/s: (was: 1.1.1) PR discussion says this is WontFix > I

[jira] [Resolved] (SPARK-3663) Document SPARK_LOG_DIR and SPARK_PID_DIR

[ https://issues.apache.org/jira/browse/SPARK-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3663. -- Resolution: Fixed Fix Version/s: 1.3.0 The PR was merged, though looks like not for 1.2: https://

[jira] [Resolved] (SPARK-3689) FileLogger should create new instance of FileSystem regardless of it's scheme

[ https://issues.apache.org/jira/browse/SPARK-3689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3689. -- Resolution: Not a Problem Target Version/s: (was: 1.1.1, 1.2.0) The PR says it's not a probl

[jira] [Resolved] (SPARK-3712) add a new UpdateDStream to update a rdd dynamically

[ https://issues.apache.org/jira/browse/SPARK-3712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3712. -- Resolution: Won't Fix Target Version/s: (was: 1.1.1, 1.2.0) Withdrawn in the PR by the propo

[jira] [Resolved] (SPARK-3719) Spark UI: "complete/failed stages" is better to show the total number of stages

[ https://issues.apache.org/jira/browse/SPARK-3719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3719. -- Resolution: Duplicate Target Version/s: (was: 1.1.1, 1.2.0) Apparently resolved by the very

[jira] [Resolved] (SPARK-3433) Mima false-positives with @DeveloperAPI and @Experimental annotations

[ https://issues.apache.org/jira/browse/SPARK-3433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3433. -- Resolution: Fixed Fix Version/s: 1.2.0 The PR was https://github.com/apache/spark/pull/2285 and i

[jira] [Resolved] (SPARK-3548) Display cache hit ratio on WebUI

[ https://issues.apache.org/jira/browse/SPARK-3548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3548. -- Resolution: Won't Fix I think this particular suggestion is WontFix since the idea of a hit ratio was d

[jira] [Resolved] (SPARK-3201) Yarn Client do not support the "-X" java opts

[ https://issues.apache.org/jira/browse/SPARK-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3201. -- Resolution: Won't Fix Looks like it was abandoned by the OP, possibly in favor of SPARK-1953 or SPARK-1

[jira] [Resolved] (SPARK-3229) spark.shuffle.safetyFraction and spark.storage.safetyFraction is not documented

[ https://issues.apache.org/jira/browse/SPARK-3229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3229. -- Resolution: Won't Fix Another one that says it's WontFix in the PR. > spark.shuffle.safetyFraction and

[jira] [Commented] (SPARK-2426) Quadratic Minimization for MLlib ALS

[ https://issues.apache.org/jira/browse/SPARK-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243456#comment-14243456 ] Debasish Das commented on SPARK-2426: - [~akopich] I got good MAP results on recommenda

[jira] [Resolved] (SPARK-3352) Rename Flume Polling stream to Pull Based stream

[ https://issues.apache.org/jira/browse/SPARK-3352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3352. -- Resolution: Won't Fix According to the PR, this is WontFix. > Rename Flume Polling stream to Pull Based

[jira] [Commented] (SPARK-3358) PySpark worker fork()ing performance regression in m3.* / PVM instances

[ https://issues.apache.org/jira/browse/SPARK-3358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243454#comment-14243454 ] Sean Owen commented on SPARK-3358: -- Is this then resolved by one of https://github.com/ap

[jira] [Resolved] (SPARK-3124) Jar version conflict in the assembly package

[ https://issues.apache.org/jira/browse/SPARK-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3124. -- Resolution: Fixed I'm a little unclear on the outcome, but in master, running {{mvn -Phive -Phive-thri

[jira] [Commented] (SPARK-2426) Quadratic Minimization for MLlib ALS

[ https://issues.apache.org/jira/browse/SPARK-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243398#comment-14243398 ] Valeriy Avanesov commented on SPARK-2426: - > what's the normalization constraint ?

[jira] [Resolved] (SPARK-3038) delete history server logs when there are too many logs

[ https://issues.apache.org/jira/browse/SPARK-3038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3038. -- Resolution: Won't Fix Fix Version/s: (was: 1.2.0) PR says this is WontFix. > delete history

[jira] [Resolved] (SPARK-3099) Staging Directory is never deleted when we run job with YARN Client Mode

[ https://issues.apache.org/jira/browse/SPARK-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3099. -- Resolution: Won't Fix PR discussion says this was obsoleted by SPARK-2933. > Staging Directory is never

[jira] [Resolved] (SPARK-2947) DAGScheduler resubmit the stage into an infinite loop

[ https://issues.apache.org/jira/browse/SPARK-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2947. -- Resolution: Duplicate Fix Version/s: (was: 1.0.3) (was: 1.2.0) Discuss

[jira] [Resolved] (SPARK-2872) Fix conflict between code and doc in YarnClientSchedulerBackend

[ https://issues.apache.org/jira/browse/SPARK-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2872. -- Resolution: Won't Fix Looks like this was obsoleted by subsequent changes to how YARN parses configurat

[jira] [Resolved] (SPARK-2896) Remove redundant error notification by results.put() when fetching from remote

[ https://issues.apache.org/jira/browse/SPARK-2896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2896. -- Resolution: Won't Fix Another WontFix, given PR discussion. > Remove redundant error notification by re

[jira] [Updated] (SPARK-4180) SparkContext constructor should throw exception if another SparkContext is already running

[ https://issues.apache.org/jira/browse/SPARK-4180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4180: -- Labels: backport-needed (was: ) > SparkContext constructor should throw exception if another SparkConte

[jira] [Resolved] (SPARK-2710) Build SchemaRDD from a JdbcRDD with MetaData (no hard-coded case class)

[ https://issues.apache.org/jira/browse/SPARK-2710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2710. -- Resolution: Won't Fix PR discussion says that this should become an external library, given the new ext

[jira] [Resolved] (SPARK-2715) ExternalAppendOnlyMap adds max limit of times and max limit of disk bytes written for spilling

[ https://issues.apache.org/jira/browse/SPARK-2715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2715. -- Resolution: Won't Fix PR discussion says it is a WontFix. > ExternalAppendOnlyMap adds max limit of tim

[jira] [Commented] (SPARK-3247) Improved support for external data sources

[ https://issues.apache.org/jira/browse/SPARK-3247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243253#comment-14243253 ] Matei Zaharia commented on SPARK-3247: -- For those looking to learn about the interfac

[jira] [Commented] (SPARK-2750) Add Https support for Web UI

[ https://issues.apache.org/jira/browse/SPARK-2750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243250#comment-14243250 ] Sean Owen commented on SPARK-2750: -- Shall this be rolled into SPARK-3883? both have an op

[jira] [Commented] (SPARK-2770) Rename spark-ganglia-lgpl to ganglia-lgpl

[ https://issues.apache.org/jira/browse/SPARK-2770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243244#comment-14243244 ] Sean Owen commented on SPARK-2770: -- Is this still active? the PR was attempted but got me

[jira] [Commented] (SPARK-2604) Spark Application hangs on yarn in edge case scenario of executor memory requirement

[ https://issues.apache.org/jira/browse/SPARK-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243221#comment-14243221 ] Sean Owen commented on SPARK-2604: -- PR comments suggest this was fixed by SPARK-2140? ht

[jira] [Resolved] (SPARK-2671) BlockObjectWriter should create parent directory when the directory doesn't exist

[ https://issues.apache.org/jira/browse/SPARK-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2671. -- Resolution: Won't Fix This is another where the PR discussion indicates this is WontFix. > BlockObjectW

[jira] [Resolved] (SPARK-2542) Exit Code Class should be renamed and placed package properly

[ https://issues.apache.org/jira/browse/SPARK-2542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2542. -- Resolution: Won't Fix PR discussion says this is WontFix. > Exit Code Class should be renamed and place

[jira] [Commented] (SPARK-4823) rowSimilarities

[ https://issues.apache.org/jira/browse/SPARK-4823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243207#comment-14243207 ] Debasish Das commented on SPARK-4823: - Even for matrix factorization userFactors are u

[jira] [Resolved] (SPARK-2402) DiskBlockObjectWriter should update the initial position when reusing this object

[ https://issues.apache.org/jira/browse/SPARK-2402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2402. -- Resolution: Won't Fix There was disagreement about whether to merge this change, but looks like it was

[jira] [Resolved] (SPARK-2368) Improve io.netty related handlers and clients in network.netty

[ https://issues.apache.org/jira/browse/SPARK-2368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2368. -- Resolution: Won't Fix PR discussion says that the OP abandoned this change as it was covered by other c

[jira] [Commented] (SPARK-2016) rdd in-memory storage UI becomes unresponsive when the number of RDD partitions is large

[ https://issues.apache.org/jira/browse/SPARK-2016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243153#comment-14243153 ] Andrew Or commented on SPARK-2016: -- This was filed before SPARK-2316 (https://github.com/

[jira] [Resolved] (SPARK-2381) streaming receiver crashed,but seems nothing happened

[ https://issues.apache.org/jira/browse/SPARK-2381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2381. -- Resolution: Won't Fix PR comments didn't receive follow-up changes either, so per comments here, looks

[jira] [Commented] (SPARK-2426) Quadratic Minimization for MLlib ALS

[ https://issues.apache.org/jira/browse/SPARK-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243149#comment-14243149 ] Debasish Das commented on SPARK-2426: - [~mengxr] as per our discussion, QuadraticMinim

[jira] [Commented] (SPARK-2016) rdd in-memory storage UI becomes unresponsive when the number of RDD partitions is large

[ https://issues.apache.org/jira/browse/SPARK-2016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243147#comment-14243147 ] Reynold Xin commented on SPARK-2016: cc [~andrewor14] can you comment on this? > rdd

[jira] [Resolved] (SPARK-2127) Use application specific folders to dump metrics via CsvSink

[ https://issues.apache.org/jira/browse/SPARK-2127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2127. -- Resolution: Duplicate The PR that closes SPARK-3377 also closed the PR for this JIRA, and it looks like

[jira] [Commented] (SPARK-4823) rowSimilarities

[ https://issues.apache.org/jira/browse/SPARK-4823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243133#comment-14243133 ] Reza Zadeh commented on SPARK-4823: --- Given that we're talking about RowMatrices, computi

[jira] [Resolved] (SPARK-2171) Groovy bindings for Spark

[ https://issues.apache.org/jira/browse/SPARK-2171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2171. -- Resolution: Won't Fix PR discussion says this will live in another project / repo > Groovy bindings for

[jira] [Resolved] (SPARK-2180) Basic HAVING clauses support for HiveQL

[ https://issues.apache.org/jira/browse/SPARK-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2180. -- Resolution: Fixed Fix Version/s: 1.1.0 Looks like this was merged for 1.1.0: https://github.com/

[jira] [Resolved] (SPARK-2193) Improve tasks‘ preferred locality by sorting tasks partial ordering

[ https://issues.apache.org/jira/browse/SPARK-2193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2193. -- Resolution: Won't Fix Last word appears to be that this was obviated by SPARK-2294 and https://github.c

[jira] [Resolved] (SPARK-2201) Improve FlumeInputDStream's stability and make it scalable

[ https://issues.apache.org/jira/browse/SPARK-2201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2201. -- Resolution: Won't Fix I hope I understood this right, but the PR discussion seemed to end with suggesti

[jira] [Resolved] (SPARK-2227) Support "dfs" command

[ https://issues.apache.org/jira/browse/SPARK-2227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2227. -- Resolution: Fixed Fix Version/s: 1.1.0 Looks like this was in fact merged in https://github.com/

[jira] [Commented] (SPARK-2016) rdd in-memory storage UI becomes unresponsive when the number of RDD partitions is large

[ https://issues.apache.org/jira/browse/SPARK-2016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243109#comment-14243109 ] Sean Owen commented on SPARK-2016: -- Is this and SPARK-2017 now subsumed by SPARK-3644? th

[jira] [Commented] (SPARK-2069) MIMA false positives (umbrella)

[ https://issues.apache.org/jira/browse/SPARK-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243105#comment-14243105 ] Sean Owen commented on SPARK-2069: -- May we resolve this? the sub-tasks are all Resolved.

[jira] [Closed] (SPARK-1880) Eliminate unnecessary job executions.

[ https://issues.apache.org/jira/browse/SPARK-1880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-1880. -- Resolution: Won't Fix > Eliminate unnecessary job executions. > - >

[jira] [Commented] (SPARK-1880) Eliminate unnecessary job executions.

[ https://issues.apache.org/jira/browse/SPARK-1880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243103#comment-14243103 ] Sean Owen commented on SPARK-1880: -- Is this now a WontFix? The PR refers to this being su

[jira] [Commented] (SPARK-4746) integration tests should be separated from faster unit tests

[ https://issues.apache.org/jira/browse/SPARK-4746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243101#comment-14243101 ] Ryan Williams commented on SPARK-4746: -- I don't have any experience with test tags, b

[jira] [Resolved] (SPARK-1888) enhance MEMORY_AND_DISK mode by dropping blocks in parallel

[ https://issues.apache.org/jira/browse/SPARK-1888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1888. -- Resolution: Duplicate >From the end of the PR discussion, it looks like this was continued in >SPARK-30

[jira] [Resolved] (SPARK-1700) PythonRDD leaks socket descriptors during cancellation

[ https://issues.apache.org/jira/browse/SPARK-1700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1700. -- Resolution: Fixed Fix Version/s: 1.0.0 The PR was https://github.com/apache/spark/pull/623 and sa

[jira] [Resolved] (SPARK-1771) CoarseGrainedSchedulerBackend is not resilient to Akka restarts

[ https://issues.apache.org/jira/browse/SPARK-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1771. -- Resolution: Won't Fix The PR says this was abandoned in favor of SPARK-4004 > CoarseGrainedSchedulerBac

[jira] [Commented] (SPARK-2426) Quadratic Minimization for MLlib ALS

[ https://issues.apache.org/jira/browse/SPARK-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243087#comment-14243087 ] Debasish Das commented on SPARK-2426: - I meant \sum_j ( r_ij - w_i*h_j)^2...what's the

[jira] [Commented] (SPARK-1532) provide option for more restrictive firewall rule in ec2/spark_ec2.py

[ https://issues.apache.org/jira/browse/SPARK-1532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243086#comment-14243086 ] Sean Owen commented on SPARK-1532: -- [~foundart] Is this abandoned? your second PR was rea

[jira] [Resolved] (SPARK-1559) Add conf dir to CLASSPATH in compute-classpath.sh dependent on whether SPARK_CONF_DIR is set

[ https://issues.apache.org/jira/browse/SPARK-1559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1559. -- Resolution: Duplicate The PR discussion suggests it was duplicated by the PR for SPARK-2058. > Add conf

[jira] [Resolved] (SPARK-1581) Allow One Flume Avro RPC Server for Each Worker rather than Just One Worker

[ https://issues.apache.org/jira/browse/SPARK-1581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1581. -- Resolution: Won't Fix No follow-up from OP explaining the change, and so the PR was closed already. > A

[jira] [Updated] (SPARK-2253) [Core] Disable partial aggregation automatically when reduction factor is low

[ https://issues.apache.org/jira/browse/SPARK-2253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2253: --- Summary: [Core] Disable partial aggregation automatically when reduction factor is low (was: Disable

[jira] [Resolved] (SPARK-1627) Support external aggregation in Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-1627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1627. -- Resolution: Won't Fix Fix Version/s: (was: 1.2.0) The discussion in https://github.com/apache

[jira] [Updated] (SPARK-1412) [SQL] Disable partial aggregation automatically when reduction factor is low

[ https://issues.apache.org/jira/browse/SPARK-1412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-1412: --- Summary: [SQL] Disable partial aggregation automatically when reduction factor is low (was: Disable p

[jira] [Updated] (SPARK-1412) Disable partial aggregation automatically when reduction factor is low

[ https://issues.apache.org/jira/browse/SPARK-1412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-1412: --- Fix Version/s: (was: 1.2.0) > Disable partial aggregation automatically when reduction factor is l

[jira] [Commented] (SPARK-1412) Disable partial aggregation automatically when reduction factor is low

[ https://issues.apache.org/jira/browse/SPARK-1412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243077#comment-14243077 ] Reynold Xin commented on SPARK-1412: I think we should still do it - it's just that th

[jira] [Commented] (SPARK-4831) Current directory always on classpath with spark-submit

[ https://issues.apache.org/jira/browse/SPARK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243074#comment-14243074 ] Daniel Darabos commented on SPARK-4831: --- bq. Is it perhaps finding and exploded dire

[jira] [Commented] (SPARK-1412) Disable partial aggregation automatically when reduction factor is low

[ https://issues.apache.org/jira/browse/SPARK-1412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243072#comment-14243072 ] Sean Owen commented on SPARK-1412: -- The PRs for SPARK-2253 and SPARK-1412 were abandoned.

[jira] [Commented] (SPARK-4823) rowSimilarities

[ https://issues.apache.org/jira/browse/SPARK-4823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243055#comment-14243055 ] Sean Owen commented on SPARK-4823: -- I don't think MapReduce matters here. You can compute

[jira] [Commented] (SPARK-4823) rowSimilarities

[ https://issues.apache.org/jira/browse/SPARK-4823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243048#comment-14243048 ] Debasish Das commented on SPARK-4823: - [~srowen] did you implement map-reduce row simi

[jira] [Commented] (SPARK-4455) Exclude dependency on hbase-annotations module

[ https://issues.apache.org/jira/browse/SPARK-4455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243044#comment-14243044 ] Sean Owen commented on SPARK-4455: -- FWIW this change also unfortunately causes compiler w

[jira] [Commented] (SPARK-4675) Find similar products and similar users in MatrixFactorizationModel

[ https://issues.apache.org/jira/browse/SPARK-4675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243026#comment-14243026 ] Debasish Das commented on SPARK-4675: - Is there a metric like MAP / AUC kind of measur

[jira] [Issue Comment Deleted] (SPARK-4728) Add exponential, log normal, and gamma distributions to data generator to MLlib

[ https://issues.apache.org/jira/browse/SPARK-4728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] RJ Nowling updated SPARK-4728: -- Comment: was deleted (was: I posted a PR for this issue: https://github.com/apache/spark/pull/3680) > A

[jira] [Commented] (SPARK-4728) Add exponential, log normal, and gamma distributions to data generator to MLlib

[ https://issues.apache.org/jira/browse/SPARK-4728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14242924#comment-14242924 ] RJ Nowling commented on SPARK-4728: --- I posted a PR for this issue: https://github.com/ap

[jira] [Commented] (SPARK-4728) Add exponential, log normal, and gamma distributions to data generator to MLlib

[ https://issues.apache.org/jira/browse/SPARK-4728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14242915#comment-14242915 ] Apache Spark commented on SPARK-4728: - User 'rnowling' has created a pull request for

[jira] [Commented] (SPARK-3533) Add saveAsTextFileByKey() method to RDDs

[ https://issues.apache.org/jira/browse/SPARK-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14242913#comment-14242913 ] Ilya Ganelin commented on SPARK-3533: - I am looking into a solution for this. > Add s

[jira] [Commented] (SPARK-4779) PySpark Shuffle Fails Looking for Files that Don't Exist when low on Memory

[ https://issues.apache.org/jira/browse/SPARK-4779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14242831#comment-14242831 ] Ilya Ganelin commented on SPARK-4779: - I've seen this issue on Scala as well. This hap

[jira] [Commented] (SPARK-4675) Find similar products and similar users in MatrixFactorizationModel

[ https://issues.apache.org/jira/browse/SPARK-4675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14242808#comment-14242808 ] Sean Owen commented on SPARK-4675: -- The lower dimensional space is of course smaller. Thi

[jira] [Resolved] (SPARK-4458) Skip compilation of tests classes when using make-distribution

[ https://issues.apache.org/jira/browse/SPARK-4458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-4458. -- Resolution: Won't Fix Given PR discussion, looks like a WontFix. > Skip compilation of tests classes wh

[jira] [Updated] (SPARK-3918) Forget Unpersist in RandomForest.scala(train Method)

[ https://issues.apache.org/jira/browse/SPARK-3918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-3918: - Target Version/s: 1.2.0 (was: 1.1.0) Affects Version/s: (was: 1.2.0) 1.1.0

[jira] [Resolved] (SPARK-3918) Forget Unpersist in RandomForest.scala(train Method)

[ https://issues.apache.org/jira/browse/SPARK-3918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3918. -- Resolution: Fixed Great, looks like this was in fact fixed for 1.2 then. > Forget Unpersist in RandomFo

[jira] [Resolved] (SPARK-3677) pom.xml and SparkBuild.scala are wrong : Scalastyle is never applyed to the sources under yarn/common

[ https://issues.apache.org/jira/browse/SPARK-3677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3677. -- Resolution: Not a Problem Target Version/s: (was: 1.1.2, 1.2.1) Obsoleted by the restructuri

[jira] [Reopened] (SPARK-2892) Socket Receiver does not stop when streaming context is stopped

[ https://issues.apache.org/jira/browse/SPARK-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-2892: -- OK. The issues may have a common cause but that can be deferred until the other JIRA is resolved. If it tur

[jira] [Commented] (SPARK-4831) Current directory always on classpath with spark-submit

[ https://issues.apache.org/jira/browse/SPARK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14242778#comment-14242778 ] Sean Owen commented on SPARK-4831: -- Hm, so I made a quick test, where I put a class {{Foo

[jira] [Commented] (SPARK-2980) Python support for chi-squared test

[ https://issues.apache.org/jira/browse/SPARK-2980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14242718#comment-14242718 ] Apache Spark commented on SPARK-2980: - User 'jbencook' has created a pull request for

[jira] [Created] (SPARK-4831) Current directory always on classpath with spark-submit

Daniel Darabos created SPARK-4831: - Summary: Current directory always on classpath with spark-submit Key: SPARK-4831 URL: https://issues.apache.org/jira/browse/SPARK-4831 Project: Spark Issue

[jira] [Commented] (SPARK-4740) Netty's network throughput is about 1/2 of NIO's in spark-perf sortByKey

[ https://issues.apache.org/jira/browse/SPARK-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14242656#comment-14242656 ] Zhang, Liye commented on SPARK-4740: Hi [~adav], I missed there is another patch from

[jira] [Commented] (SPARK-4814) Enable assertions in SBT, Maven tests / AssertionError from Hive's LazyBinaryInteger

[ https://issues.apache.org/jira/browse/SPARK-4814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14242611#comment-14242611 ] Cheng Lian commented on SPARK-4814: --- This assertion failure seems to be related to detai

[jira] [Resolved] (SPARK-4806) Update Streaming Programming Guide for Spark 1.2

[ https://issues.apache.org/jira/browse/SPARK-4806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-4806. -- Resolution: Done Target Version/s: 1.2.0, 1.3.0 (was: 1.2.0) > Update Streaming Progr

[jira] [Commented] (SPARK-3655) Support sorting of values in addition to keys (i.e. secondary sort)

[ https://issues.apache.org/jira/browse/SPARK-3655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14242547#comment-14242547 ] koert kuipers commented on SPARK-3655: -- i updated the pullreq to use Iterables instea

[jira] [Updated] (SPARK-4830) Spark Streaming Java Application : java.lang.ClassNotFoundException

[ https://issues.apache.org/jira/browse/SPARK-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mykhaylo Telizhyn updated SPARK-4830: - Summary: Spark Streaming Java Application : java.lang.ClassNotFoundException (was: Spark

[jira] [Updated] (SPARK-4830) Spark Java Application : java.lang.ClassNotFoundException

[ https://issues.apache.org/jira/browse/SPARK-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mykhaylo Telizhyn updated SPARK-4830: - Description: h4. Application Overview: We have Spark Streaming application that

[jira] [Updated] (SPARK-4830) Spark Java Application : java.lang.ClassNotFoundException

[ https://issues.apache.org/jira/browse/SPARK-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mykhaylo Telizhyn updated SPARK-4830: - Component/s: Streaming > Spark Java Application : java.lang.ClassNotFoundException > -

[jira] [Updated] (SPARK-4830) Spark Java Application : java.lang.ClassNotFoundException

[ https://issues.apache.org/jira/browse/SPARK-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mykhaylo Telizhyn updated SPARK-4830: - Description: h4. Application Overview: We have Spark Streaming application that

[jira] [Commented] (SPARK-4526) Gradient should be added batch computing interface

[ https://issues.apache.org/jira/browse/SPARK-4526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14242504#comment-14242504 ] Apache Spark commented on SPARK-4526: - User 'witgo' has created a pull request for thi

[jira] [Updated] (SPARK-4830) Spark Java Application : java.lang.ClassNotFoundException

[ https://issues.apache.org/jira/browse/SPARK-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mykhaylo Telizhyn updated SPARK-4830: - Description: We have Spark Streaming application that consumes messages from RabbitMQ and

  1   2   >