[jira] [Commented] (SPARK-12843) Spark should avoid scanning all partitions when limit is set

[ https://issues.apache.org/jira/browse/SPARK-12843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15112052#comment-15112052 ] Maciej Bryński commented on SPARK-12843: Let's assume that I have a big table. 20

[jira] [Created] (SPARK-12963) In cluster mode,spark_local_ip will cause driver exception:Service 'Driver' failed after 16 retries!

lichenglin created SPARK-12963: -- Summary: In cluster mode,spark_local_ip will cause driver exception:Service 'Driver' failed after 16 retries! Key: SPARK-12963 URL: https://issues.apache.org/jira/browse/SPARK-12963

[jira] [Assigned] (SPARK-12962) PySpark support covar_samp and covar_pop

[ https://issues.apache.org/jira/browse/SPARK-12962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12962: Assignee: Apache Spark > PySpark support covar_samp and covar_pop > --

[jira] [Assigned] (SPARK-12962) PySpark support covar_samp and covar_pop

[ https://issues.apache.org/jira/browse/SPARK-12962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12962: Assignee: (was: Apache Spark) > PySpark support covar_samp and covar_pop > ---

[jira] [Commented] (SPARK-12962) PySpark support covar_samp and covar_pop

[ https://issues.apache.org/jira/browse/SPARK-12962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15112010#comment-15112010 ] Apache Spark commented on SPARK-12962: -- User 'yanboliang' has created a pull request

[jira] [Created] (SPARK-12962) PySpark support covar_samp and covar_pop

Yanbo Liang created SPARK-12962: --- Summary: PySpark support covar_samp and covar_pop Key: SPARK-12962 URL: https://issues.apache.org/jira/browse/SPARK-12962 Project: Spark Issue Type: Improvemen

[jira] [Updated] (SPARK-12959) Writing Bucketed Data with Disabled Bucketing in SQLConf

[ https://issues.apache.org/jira/browse/SPARK-12959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-12959: Summary: Writing Bucketed Data with Disabled Bucketing in SQLConf (was: Silent switch to normal table writ

[jira] [Updated] (SPARK-12959) Silent switch to normal table writing when writing bucketed data with bucketing disabled

[ https://issues.apache.org/jira/browse/SPARK-12959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-12959: Description: When users turn off bucketing in SQLConf, writing bucketed data will not be affected. (was:

[jira] [Comment Edited] (SPARK-12843) Spark should avoid scanning all partitions when limit is set

[ https://issues.apache.org/jira/browse/SPARK-12843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111945#comment-15111945 ] dileep edited comment on SPARK-12843 at 1/22/16 6:04 AM: - Dear Ma

[jira] [Issue Comment Deleted] (SPARK-12843) Spark should avoid scanning all partitions when limit is set

[ https://issues.apache.org/jira/browse/SPARK-12843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dileep updated SPARK-12843: --- Comment: was deleted (was: Its not selecting entire records when I put Limit after doing caching. So you can

[jira] [Issue Comment Deleted] (SPARK-12843) Spark should avoid scanning all partitions when limit is set

[ https://issues.apache.org/jira/browse/SPARK-12843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dileep updated SPARK-12843: --- Comment: was deleted (was: Its a caching issue, while scanning the table need to cache the Data Frame, so fr

[jira] [Issue Comment Deleted] (SPARK-12843) Spark should avoid scanning all partitions when limit is set

[ https://issues.apache.org/jira/browse/SPARK-12843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dileep updated SPARK-12843: --- Comment: was deleted (was: I will look in to this issue) > Spark should avoid scanning all partitions when

[jira] [Issue Comment Deleted] (SPARK-12843) Spark should avoid scanning all partitions when limit is set

[ https://issues.apache.org/jira/browse/SPARK-12843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dileep updated SPARK-12843: --- Comment: was deleted (was: When I verified with 2 lakhs records, I am able to check the milliseconds differe

[jira] [Commented] (SPARK-12843) Spark should avoid scanning all partitions when limit is set

[ https://issues.apache.org/jira/browse/SPARK-12843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111945#comment-15111945 ] dileep commented on SPARK-12843: Can you elaborate it? > Spark should avoid scanning all

[jira] [Issue Comment Deleted] (SPARK-12843) Spark should avoid scanning all partitions when limit is set

[ https://issues.apache.org/jira/browse/SPARK-12843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dileep updated SPARK-12843: --- Comment: was deleted (was: Maciej Bryński Can you ellaborate it?) > Spark should avoid scanning all partiti

[jira] [Commented] (SPARK-12843) Spark should avoid scanning all partitions when limit is set

[ https://issues.apache.org/jira/browse/SPARK-12843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111944#comment-15111944 ] dileep commented on SPARK-12843: Maciej Bryński Can you ellaborate it? > Spark should a

[jira] [Resolved] (SPARK-12960) Some examples are missing support for python2

[ https://issues.apache.org/jira/browse/SPARK-12960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12960. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10872 [https://github.

[jira] [Commented] (SPARK-12961) Work around memory leak in Snappy library

[ https://issues.apache.org/jira/browse/SPARK-12961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111898#comment-15111898 ] Apache Spark commented on SPARK-12961: -- User 'viirya' has created a pull request for

[jira] [Assigned] (SPARK-12961) Work around memory leak in Snappy library

[ https://issues.apache.org/jira/browse/SPARK-12961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12961: Assignee: Apache Spark > Work around memory leak in Snappy library > -

[jira] [Assigned] (SPARK-12961) Work around memory leak in Snappy library

[ https://issues.apache.org/jira/browse/SPARK-12961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12961: Assignee: (was: Apache Spark) > Work around memory leak in Snappy library > --

[jira] [Created] (SPARK-12961) Work around memory leak in Snappy library

Josh Rosen created SPARK-12961: -- Summary: Work around memory leak in Snappy library Key: SPARK-12961 URL: https://issues.apache.org/jira/browse/SPARK-12961 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-6847) Stack overflow on updateStateByKey which followed by a dstream with checkpoint set

[ https://issues.apache.org/jira/browse/SPARK-6847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111849#comment-15111849 ] Jack Hu commented on SPARK-6847: Hi [~zsxwing] I just test a simple case with 1.6, it sti

[jira] [Updated] (SPARK-12747) Postgres JDBC ArrayType(DoubleType) 'Unable to find server array type'

[ https://issues.apache.org/jira/browse/SPARK-12747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-12747: Fix Version/s: 1.6.1 > Postgres JDBC ArrayType(DoubleType) 'Unable to find server array type' > ---

[jira] [Resolved] (SPARK-12747) Postgres JDBC ArrayType(DoubleType) 'Unable to find server array type'

[ https://issues.apache.org/jira/browse/SPARK-12747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-12747. - Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 2.0.0 > Postgres JDBC

[jira] [Assigned] (SPARK-12957) Derive and propagate data constrains in logical plan

[ https://issues.apache.org/jira/browse/SPARK-12957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12957: Assignee: (was: Apache Spark) > Derive and propagate data constrains in logical plan

[jira] [Commented] (SPARK-12957) Derive and propagate data constrains in logical plan

[ https://issues.apache.org/jira/browse/SPARK-12957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111831#comment-15111831 ] Apache Spark commented on SPARK-12957: -- User 'sameeragarwal' has created a pull requ

[jira] [Assigned] (SPARK-12957) Derive and propagate data constrains in logical plan

[ https://issues.apache.org/jira/browse/SPARK-12957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12957: Assignee: Apache Spark > Derive and propagate data constrains in logical plan > -

[jira] [Comment Edited] (SPARK-12850) Support bucket pruning (predicate pushdown for bucketed tables)

[ https://issues.apache.org/jira/browse/SPARK-12850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111825#comment-15111825 ] Xiao Li edited comment on SPARK-12850 at 1/22/16 2:45 AM: -- Sorry

[jira] [Commented] (SPARK-12850) Support bucket pruning (predicate pushdown for bucketed tables)

[ https://issues.apache.org/jira/browse/SPARK-12850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111825#comment-15111825 ] Xiao Li commented on SPARK-12850: - Sorry, I missed this message. Glad to take it. Will do

[jira] [Commented] (SPARK-5865) Add doc warnings for methods that return local data structures

[ https://issues.apache.org/jira/browse/SPARK-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111793#comment-15111793 ] Apache Spark commented on SPARK-5865: - User 'Wenpei' has created a pull request for th

[jira] [Assigned] (SPARK-5865) Add doc warnings for methods that return local data structures

[ https://issues.apache.org/jira/browse/SPARK-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5865: --- Assignee: (was: Apache Spark) > Add doc warnings for methods that return local data struc

[jira] [Assigned] (SPARK-5865) Add doc warnings for methods that return local data structures

[ https://issues.apache.org/jira/browse/SPARK-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5865: --- Assignee: Apache Spark > Add doc warnings for methods that return local data structures > ---

[jira] [Commented] (SPARK-12953) RDDRelation write set mode will be better to avoid error "pair.parquet already exists"

[ https://issues.apache.org/jira/browse/SPARK-12953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111750#comment-15111750 ] shijinkui commented on SPARK-12953: --- OK, get it. > RDDRelation write set mode will be

[jira] [Comment Edited] (SPARK-12140) Support Streaming UI in HistoryServer

[ https://issues.apache.org/jira/browse/SPARK-12140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111656#comment-15111656 ] Saisai Shao edited comment on SPARK-12140 at 1/22/16 1:29 AM: -

[jira] [Comment Edited] (SPARK-12140) Support Streaming UI in HistoryServer

[ https://issues.apache.org/jira/browse/SPARK-12140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111656#comment-15111656 ] Saisai Shao edited comment on SPARK-12140 at 1/22/16 1:29 AM: -

[jira] [Resolved] (SPARK-12908) Add tests to make sure that ml.classification.LogisticRegression returns meaningful result when labels are the same without intercept

[ https://issues.apache.org/jira/browse/SPARK-12908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-12908. - Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10862 [https://github.com/ap

[jira] [Assigned] (SPARK-12859) Names of input streams with receivers don't fit in Streaming page

[ https://issues.apache.org/jira/browse/SPARK-12859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12859: Assignee: (was: Apache Spark) > Names of input streams with receivers don't fit in Str

[jira] [Commented] (SPARK-12859) Names of input streams with receivers don't fit in Streaming page

[ https://issues.apache.org/jira/browse/SPARK-12859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111660#comment-15111660 ] Apache Spark commented on SPARK-12859: -- User 'ajbozarth' has created a pull request

[jira] [Assigned] (SPARK-12859) Names of input streams with receivers don't fit in Streaming page

[ https://issues.apache.org/jira/browse/SPARK-12859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12859: Assignee: Apache Spark > Names of input streams with receivers don't fit in Streaming page

[jira] [Commented] (SPARK-12140) Support Streaming UI in HistoryServer

[ https://issues.apache.org/jira/browse/SPARK-12140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111656#comment-15111656 ] Saisai Shao commented on SPARK-12140: - Hi guys, I though a bit on this feature, besid

[jira] [Commented] (SPARK-12946) The SQL page is empty

[ https://issues.apache.org/jira/browse/SPARK-12946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111645#comment-15111645 ] Alex Bozarth commented on SPARK-12946: -- These may be the same problem > The SQL pag

[jira] [Issue Comment Deleted] (SPARK-12946) The SQL page is empty

[ https://issues.apache.org/jira/browse/SPARK-12946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Bozarth updated SPARK-12946: - Comment: was deleted (was: These may be the same problem) > The SQL page is empty >

[jira] [Commented] (SPARK-12859) Names of input streams with receivers don't fit in Streaming page

[ https://issues.apache.org/jira/browse/SPARK-12859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111632#comment-15111632 ] Alex Bozarth commented on SPARK-12859: -- This is a easy one line css fix, I can open

[jira] [Assigned] (SPARK-12960) Some examples are missing support for python2

[ https://issues.apache.org/jira/browse/SPARK-12960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12960: Assignee: Apache Spark > Some examples are missing support for python2 > -

[jira] [Assigned] (SPARK-12960) Some examples are missing support for python2

[ https://issues.apache.org/jira/browse/SPARK-12960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12960: Assignee: (was: Apache Spark) > Some examples are missing support for python2 > --

[jira] [Commented] (SPARK-12960) Some examples are missing support for python2

[ https://issues.apache.org/jira/browse/SPARK-12960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111625#comment-15111625 ] Apache Spark commented on SPARK-12960: -- User 'markgrover' has created a pull request

[jira] [Created] (SPARK-12960) Some examples are missing support for python2

Mark Grover created SPARK-12960: --- Summary: Some examples are missing support for python2 Key: SPARK-12960 URL: https://issues.apache.org/jira/browse/SPARK-12960 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-4073) Parquet+Snappy can cause significant off-heap memory usage

[ https://issues.apache.org/jira/browse/SPARK-4073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111607#comment-15111607 ] Juliet Hougland commented on SPARK-4073: For those playing along at home-- the sol

[jira] [Commented] (SPARK-12946) The SQL page is empty

[ https://issues.apache.org/jira/browse/SPARK-12946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111598#comment-15111598 ] Alex Bozarth commented on SPARK-12946: -- Can you give more details on this? Such as w

[jira] [Commented] (SPARK-12957) Derive and propagate data constrains in logical plan

[ https://issues.apache.org/jira/browse/SPARK-12957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111597#comment-15111597 ] Xiao Li commented on SPARK-12957: - I have two related PRs that require a general null fil

[jira] [Commented] (SPARK-12957) Derive and propagate data constrains in logical plan

[ https://issues.apache.org/jira/browse/SPARK-12957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111583#comment-15111583 ] Xiao Li commented on SPARK-12957: - Are you saying the predicate transitivity? or for null

[jira] [Commented] (SPARK-9721) TreeTests.checkEqual should compare predictions on data

[ https://issues.apache.org/jira/browse/SPARK-9721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111579#comment-15111579 ] Seth Hendrickson commented on SPARK-9721: - I assume there is some motivation behin

[jira] [Commented] (SPARK-10498) Add requirements file for create dev python tools

[ https://issues.apache.org/jira/browse/SPARK-10498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111527#comment-15111527 ] Apache Spark commented on SPARK-10498: -- User 'holdenk' has created a pull request fo

[jira] [Assigned] (SPARK-10498) Add requirements file for create dev python tools

[ https://issues.apache.org/jira/browse/SPARK-10498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10498: Assignee: Apache Spark > Add requirements file for create dev python tools > -

[jira] [Assigned] (SPARK-10498) Add requirements file for create dev python tools

[ https://issues.apache.org/jira/browse/SPARK-10498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10498: Assignee: (was: Apache Spark) > Add requirements file for create dev python tools > --

[jira] [Commented] (SPARK-11045) Contributing Receiver based Low Level Kafka Consumer from Spark-Packages to Apache Spark Project

[ https://issues.apache.org/jira/browse/SPARK-11045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111398#comment-15111398 ] Cody Koeninger commented on SPARK-11045: There's already work being done on 0.9

[jira] [Assigned] (SPARK-12959) Silent switch to normal table writing when writing bucketed data with bucketing disabled

[ https://issues.apache.org/jira/browse/SPARK-12959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12959: Assignee: (was: Apache Spark) > Silent switch to normal table writing when writing buc

[jira] [Commented] (SPARK-12959) Silent switch to normal table writing when writing bucketed data with bucketing disabled

[ https://issues.apache.org/jira/browse/SPARK-12959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111379#comment-15111379 ] Apache Spark commented on SPARK-12959: -- User 'gatorsmile' has created a pull request

[jira] [Assigned] (SPARK-12959) Silent switch to normal table writing when writing bucketed data with bucketing disabled

[ https://issues.apache.org/jira/browse/SPARK-12959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12959: Assignee: Apache Spark > Silent switch to normal table writing when writing bucketed data

[jira] [Created] (SPARK-12959) Silent switch to normal table writing when writing bucketed data with bucketing disabled

Xiao Li created SPARK-12959: --- Summary: Silent switch to normal table writing when writing bucketed data with bucketing disabled Key: SPARK-12959 URL: https://issues.apache.org/jira/browse/SPARK-12959 Projec

[jira] [Commented] (SPARK-5865) Add doc warnings for methods that return local data structures

[ https://issues.apache.org/jira/browse/SPARK-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111317#comment-15111317 ] Tommy Yu commented on SPARK-5865: - I will work on this taks. Thanks. > Add doc warnings f

[jira] [Commented] (SPARK-10498) Add requirements file for create dev python tools

[ https://issues.apache.org/jira/browse/SPARK-10498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111306#comment-15111306 ] holdenk commented on SPARK-10498: - Sounds good - I'll give this a shot > Add requirement

[jira] [Commented] (SPARK-12684) Matrix.toString should take a format for how each cell should be printed

[ https://issues.apache.org/jira/browse/SPARK-12684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111304#comment-15111304 ] holdenk commented on SPARK-12684: - So I think this issue is probably related to https://

[jira] [Commented] (SPARK-11045) Contributing Receiver based Low Level Kafka Consumer from Spark-Packages to Apache Spark Project

[ https://issues.apache.org/jira/browse/SPARK-11045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111302#comment-15111302 ] Marko Bonaci commented on SPARK-11045: -- While both "sides" brought up some valid arg

[jira] [Commented] (SPARK-12731) PySpark docstring cleanup

[ https://issues.apache.org/jira/browse/SPARK-12731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111301#comment-15111301 ] holdenk commented on SPARK-12731: - So is this a thing we should consider pursuing or mayb

[jira] [Commented] (SPARK-11045) Contributing Receiver based Low Level Kafka Consumer from Spark-Packages to Apache Spark Project

[ https://issues.apache.org/jira/browse/SPARK-11045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111243#comment-15111243 ] Saisai Shao commented on SPARK-11045: - Hi [~dibbhatt], I'm afraid I could not agree w

[jira] [Commented] (SPARK-11045) Contributing Receiver based Low Level Kafka Consumer from Spark-Packages to Apache Spark Project

[ https://issues.apache.org/jira/browse/SPARK-11045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111206#comment-15111206 ] Dibyendu Bhattacharya commented on SPARK-11045: --- Thanks Dan for your commen

[jira] [Assigned] (SPARK-10911) Executors should System.exit on clean shutdown

[ https://issues.apache.org/jira/browse/SPARK-10911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10911: Assignee: Apache Spark (was: Zhuo Liu) > Executors should System.exit on clean shutdown >

[jira] [Assigned] (SPARK-10911) Executors should System.exit on clean shutdown

[ https://issues.apache.org/jira/browse/SPARK-10911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10911: Assignee: Zhuo Liu (was: Apache Spark) > Executors should System.exit on clean shutdown >

[jira] [Reopened] (SPARK-10911) Executors should System.exit on clean shutdown

[ https://issues.apache.org/jira/browse/SPARK-10911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuo Liu reopened SPARK-10911: -- Reopen since we got more discussion on this issue. > Executors should System.exit on clean shutdown >

[jira] [Commented] (SPARK-4289) Creating an instance of Hadoop Job fails in the Spark shell when toString() is called on the instance.

[ https://issues.apache.org/jira/browse/SPARK-4289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1555#comment-1555 ] Steven Pearson commented on SPARK-4289: --- You can wrap the job in another class and t

[jira] [Commented] (SPARK-4073) Parquet+Snappy can cause significant off-heap memory usage

[ https://issues.apache.org/jira/browse/SPARK-4073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1548#comment-1548 ] Juliet Hougland commented on SPARK-4073: I have run in to a related problem. I am

[jira] [Created] (SPARK-12958) Map accumulator in spark

Souri created SPARK-12958: - Summary: Map accumulator in spark Key: SPARK-12958 URL: https://issues.apache.org/jira/browse/SPARK-12958 Project: Spark Issue Type: Wish Components: Spark Core

[jira] [Created] (SPARK-12957) Derive and propagate data constrains in logical plan

Yin Huai created SPARK-12957: Summary: Derive and propagate data constrains in logical plan Key: SPARK-12957 URL: https://issues.apache.org/jira/browse/SPARK-12957 Project: Spark Issue Type: New

[jira] [Commented] (SPARK-5629) Add spark-ec2 action to return info about an existing cluster

[ https://issues.apache.org/jira/browse/SPARK-5629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111056#comment-15111056 ] Shivaram Venkataraman commented on SPARK-5629: -- Yes - though I think its bene

[jira] [Commented] (SPARK-11045) Contributing Receiver based Low Level Kafka Consumer from Spark-Packages to Apache Spark Project

[ https://issues.apache.org/jira/browse/SPARK-11045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111044#comment-15111044 ] Dan Dutrow commented on SPARK-11045: +1 to Dibyendu's comment that "Being at spark-pa

[jira] [Commented] (SPARK-12650) No means to specify Xmx settings for SparkSubmit in yarn-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-12650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111024#comment-15111024 ] Marcelo Vanzin commented on SPARK-12650: If it works it's a workaround; that's a

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15110994#comment-15110994 ] Mario Briggs commented on SPARK-12177: -- bq. If one uses the kafka v9 jar even when u

[jira] [Commented] (SPARK-1680) Clean up use of setExecutorEnvs in SparkConf

[ https://issues.apache.org/jira/browse/SPARK-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15110979#comment-15110979 ] Apache Spark commented on SPARK-1680: - User 'weineran' has created a pull request for

[jira] [Commented] (SPARK-12650) No means to specify Xmx settings for SparkSubmit in yarn-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-12650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15110964#comment-15110964 ] Sean Owen commented on SPARK-12650: --- [~vanzin] is that the intended way to set this? if

[jira] [Created] (SPARK-12956) add spark.yarn.hdfs.home.directory property

PJ Fanning created SPARK-12956: -- Summary: add spark.yarn.hdfs.home.directory property Key: SPARK-12956 URL: https://issues.apache.org/jira/browse/SPARK-12956 Project: Spark Issue Type: Improveme

[jira] [Created] (SPARK-12955) Spark-HiveSQL: It fail when is quering a nested structure

Gerardo Villarroel created SPARK-12955: -- Summary: Spark-HiveSQL: It fail when is quering a nested structure Key: SPARK-12955 URL: https://issues.apache.org/jira/browse/SPARK-12955 Project: Spark

[jira] [Commented] (SPARK-12941) Spark-SQL JDBC Oracle dialect fails to map string datatypes to Oracle VARCHAR datatype

[ https://issues.apache.org/jira/browse/SPARK-12941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15110906#comment-15110906 ] Jose Martinez Poblete commented on SPARK-12941: --- Thanks, let us know if thi

[jira] [Commented] (SPARK-12760) inaccurate description for difference between local vs cluster mode in closure handling

[ https://issues.apache.org/jira/browse/SPARK-12760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15110908#comment-15110908 ] Apache Spark commented on SPARK-12760: -- User 'mortada' has created a pull request fo

[jira] [Resolved] (SPARK-9282) Filter on Spark DataFrame with multiple columns

[ https://issues.apache.org/jira/browse/SPARK-9282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-9282. -- Resolution: Not A Problem > Filter on Spark DataFrame with multiple columns > --

[jira] [Commented] (SPARK-4878) driverPropsFetcher causes spurious Akka disassociate errors

[ https://issues.apache.org/jira/browse/SPARK-4878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15110865#comment-15110865 ] Sean Owen commented on SPARK-4878: -- I think this may be defunct anyway, but, the code in

[jira] [Resolved] (SPARK-4247) [SQL] use beeline execute "create table as" thriftserver is not use "hive" user ,but the new hdfs dir's owner is "hive"

[ https://issues.apache.org/jira/browse/SPARK-4247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-4247. -- Resolution: Not A Problem I think this is stale anyway, but I think this is a question about Hive and ho

[jira] [Resolved] (SPARK-4171) StreamingContext.actorStream throws serializationError

[ https://issues.apache.org/jira/browse/SPARK-4171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-4171. -- Resolution: Won't Fix I think this is obsolete now that the Akka actor bits are being removed. > Stream

[jira] [Resolved] (SPARK-6056) Unlimit offHeap memory use cause RM killing the container

[ https://issues.apache.org/jira/browse/SPARK-6056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6056. -- Resolution: Not A Problem > Unlimit offHeap memory use cause RM killing the container >

[jira] [Resolved] (SPARK-6137) G-Means clustering algorithm implementation

[ https://issues.apache.org/jira/browse/SPARK-6137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6137. -- Resolution: Won't Fix > G-Means clustering algorithm implementation > --

[jira] [Resolved] (SPARK-6034) DESCRIBE EXTENDED viewname is not supported for HiveContext

[ https://issues.apache.org/jira/browse/SPARK-6034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6034. -- Resolution: Won't Fix > DESCRIBE EXTENDED viewname is not supported for HiveContext > --

[jira] [Resolved] (SPARK-6009) IllegalArgumentException thrown by TimSort when SQL ORDER BY RAND ()

[ https://issues.apache.org/jira/browse/SPARK-6009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6009. -- Resolution: Duplicate > IllegalArgumentException thrown by TimSort when SQL ORDER BY RAND () > -

[jira] [Commented] (SPARK-5629) Add spark-ec2 action to return info about an existing cluster

[ https://issues.apache.org/jira/browse/SPARK-5629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15110853#comment-15110853 ] Sean Owen commented on SPARK-5629: -- Are all of the EC2 tickets becoming essentially "wont

[jira] [Resolved] (SPARK-5929) Pyspark: Register a pip requirements file with spark_context

[ https://issues.apache.org/jira/browse/SPARK-5929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5929. -- Resolution: Won't Fix > Pyspark: Register a pip requirements file with spark_context > -

[jira] [Resolved] (SPARK-5647) Output metrics do not show up for older hadoop versions (< 2.5)

[ https://issues.apache.org/jira/browse/SPARK-5647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5647. -- Resolution: Duplicate I think this is nearly moot for Spark 2.x, given that Hadoop support may get to 2

[jira] [Updated] (SPARK-12797) Aggregation without grouping keys

[ https://issues.apache.org/jira/browse/SPARK-12797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12797: -- Assignee: Davies Liu > Aggregation without grouping keys > - > >

[jira] [Updated] (SPARK-12953) RDDRelation write set mode will be better to avoid error "pair.parquet already exists"

[ https://issues.apache.org/jira/browse/SPARK-12953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12953: -- Priority: Minor (was: Major) Fix Version/s: (was: 1.6.1) Issue Type: Improvement (

[jira] [Updated] (SPARK-12945) ERROR LiveListenerBus: Listener JobProgressListener threw an exception

[ https://issues.apache.org/jira/browse/SPARK-12945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12945: -- Component/s: Web UI > ERROR LiveListenerBus: Listener JobProgressListener threw an exception >

[jira] [Updated] (SPARK-12946) The SQL page is empty

[ https://issues.apache.org/jira/browse/SPARK-12946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12946: -- Target Version/s: (was: 1.6.1) Fix Version/s: (was: 1.6.1) > The SQL page is empty > -

[jira] [Commented] (SPARK-12843) Spark should avoid scanning all partitions when limit is set

[ https://issues.apache.org/jira/browse/SPARK-12843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15110836#comment-15110836 ] Maciej Bryński commented on SPARK-12843: [~dileep] I think you miss the point of

  1   2   >