[jira] [Commented] (SPARK-17347) Encoder in Dataset example is incorrect on type

2016-08-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453216#comment-15453216 ] Apache Spark commented on SPARK-17347: -- User 'CodingCat' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17347) Encoder in Dataset example is incorrect on type

2016-08-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17347: Assignee: Apache Spark > Encoder in Dataset example is incorrect on type >

[jira] [Assigned] (SPARK-17347) Encoder in Dataset example is incorrect on type

2016-08-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17347: Assignee: (was: Apache Spark) > Encoder in Dataset example is incorrect on type >

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-08-31 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453324#comment-15453324 ] Cody Koeninger commented on SPARK-15406: If people want to use older versions of kafka, why not

[jira] [Updated] (SPARK-17099) Incorrect result when HAVING clause is added to group by query

2016-08-31 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17099: --- Labels: correctness (was: ) > Incorrect result when HAVING clause is added to group by query >

[jira] [Updated] (SPARK-16991) Full outer join followed by inner join produces wrong results

2016-08-31 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-16991: --- Labels: correctness (was: ) > Full outer join followed by inner join produces wrong results >

[jira] [Updated] (SPARK-15706) Wrong Answer when using IF NOT EXISTS in INSERT OVERWRITE for DYNAMIC PARTITION

2016-08-31 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-15706: --- Labels: correctness (was: ) > Wrong Answer when using IF NOT EXISTS in INSERT OVERWRITE for DYNAMIC

[jira] [Commented] (SPARK-17348) Incorrect results from subquery transformation

2016-08-31 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453516#comment-15453516 ] Herman van Hovell commented on SPARK-17348: --- This is an interesting one. TBH I have never seen

[jira] [Commented] (SPARK-17211) Broadcast join produces incorrect results on EMR with large driver memory

2016-08-31 Thread gurmukh singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453541#comment-15453541 ] gurmukh singh commented on SPARK-17211: --- Hi I can see this in Apache Spark 2.0 as well running

[jira] [Created] (SPARK-17347) Encoder in Dataset example is incorrect on type

2016-08-31 Thread Nan Zhu (JIRA)
Nan Zhu created SPARK-17347: --- Summary: Encoder in Dataset example is incorrect on type Key: SPARK-17347 URL: https://issues.apache.org/jira/browse/SPARK-17347 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-16490) Python mllib example for chi-squared feature selector

2016-08-31 Thread Ruben Janssen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453344#comment-15453344 ] Ruben Janssen commented on SPARK-16490: --- Hi [~holdenk], I updated the PR after your request, could

[jira] [Updated] (SPARK-17093) Roundtrip encoding of array<struct<>> fields is wrong when whole-stage codegen is disabled

2016-08-31 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17093: --- Labels: correctness (was: ) > Roundtrip encoding of array> fields is wrong when

[jira] [Updated] (SPARK-17060) Call inner join after outer join will miss rows with null values

2016-08-31 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17060: --- Labels: correctness join (was: join) > Call inner join after outer join will miss rows with null

[jira] [Updated] (SPARK-16633) lag/lead using constant input values does not return the default value when the offset row does not exist

2016-08-31 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-16633: --- Labels: correctness (was: ) > lag/lead using constant input values does not return the default

[jira] [Commented] (SPARK-17349) Update testthat package on Jenkins

2016-08-31 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453397#comment-15453397 ] Shivaram Venkataraman commented on SPARK-17349: --- [~shaneknapp] Could we upgrade the

[jira] [Comment Edited] (SPARK-4105) FAILED_TO_UNCOMPRESS(5) errors when fetching shuffle data with sort-based shuffle

2016-08-31 Thread Matthew Seal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453034#comment-15453034 ] Matthew Seal edited comment on SPARK-4105 at 8/31/16 9:42 PM: -- Producible on

[jira] [Updated] (SPARK-17061) Incorrect results returned following a join of two datasets and a map step where total number of columns >100

2016-08-31 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17061: --- Labels: correctness (was: ) > Incorrect results returned following a join of two datasets and a map

[jira] [Updated] (SPARK-16818) Exchange reuse incorrectly reuses scans over different sets of partitions

2016-08-31 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-16818: --- Labels: correctness (was: ) > Exchange reuse incorrectly reuses scans over different sets of

[jira] [Comment Edited] (SPARK-17211) Broadcast join produces incorrect results on EMR with large driver memory

2016-08-31 Thread gurmukh singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453541#comment-15453541 ] gurmukh singh edited comment on SPARK-17211 at 8/31/16 10:22 PM: - Hi I

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-08-31 Thread Robert Conrad (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453193#comment-15453193 ] Robert Conrad commented on SPARK-15406: --- Would that mean the solution to this would require Kafka

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-08-31 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453246#comment-15453246 ] Cody Koeninger commented on SPARK-15406: Yes. > Structured streaming support for consuming from

[jira] [Created] (SPARK-17348) Incorrect results from subquery transformation

2016-08-31 Thread Nattavut Sutyanyong (JIRA)
Nattavut Sutyanyong created SPARK-17348: --- Summary: Incorrect results from subquery transformation Key: SPARK-17348 URL: https://issues.apache.org/jira/browse/SPARK-17348 Project: Spark

[jira] [Comment Edited] (SPARK-17348) Incorrect results from subquery transformation

2016-08-31 Thread Nattavut Sutyanyong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453285#comment-15453285 ] Nattavut Sutyanyong edited comment on SPARK-17348 at 8/31/16 8:37 PM:

[jira] [Comment Edited] (SPARK-16490) Python mllib example for chi-squared feature selector

2016-08-31 Thread Ruben Janssen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453344#comment-15453344 ] Ruben Janssen edited comment on SPARK-16490 at 8/31/16 8:59 PM: Hi

[jira] [Updated] (SPARK-12586) Wrong answer with registerTempTable and union sql query

2016-08-31 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-12586: --- Description: The following sequence of sql(), registerTempTable() calls gets the wrong answer. The

[jira] [Comment Edited] (SPARK-17211) Broadcast join produces incorrect results on EMR with large driver memory

2016-08-31 Thread gurmukh singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453541#comment-15453541 ] gurmukh singh edited comment on SPARK-17211 at 8/31/16 10:19 PM: - Hi I

[jira] [Commented] (SPARK-16581) Making JVM backend calling functions public

2016-08-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453592#comment-15453592 ] Apache Spark commented on SPARK-16581: -- User 'shivaram' has created a pull request for this issue:

[jira] [Commented] (SPARK-17339) Fix SparkR tests on Windows

2016-08-31 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453238#comment-15453238 ] Shivaram Venkataraman commented on SPARK-17339: --- Looking at the code my current guess is

[jira] [Comment Edited] (SPARK-15570) Pregel functions fail when run multiple times in the same jvm using sequence of graphs

2016-08-31 Thread Shishir Kharel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453331#comment-15453331 ] Shishir Kharel edited comment on SPARK-15570 at 8/31/16 8:54 PM: - It

[jira] [Commented] (SPARK-17349) Update testthat package on Jenkins

2016-08-31 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453349#comment-15453349 ] Shivaram Venkataraman commented on SPARK-17349: --- cc [~hyukjin.kwon] [~felixcheung] >

[jira] [Created] (SPARK-17349) Update testthat package on Jenkins

2016-08-31 Thread Shivaram Venkataraman (JIRA)
Shivaram Venkataraman created SPARK-17349: - Summary: Update testthat package on Jenkins Key: SPARK-17349 URL: https://issues.apache.org/jira/browse/SPARK-17349 Project: Spark Issue

[jira] [Updated] (SPARK-17114) Adding a 'GROUP BY 1' where first column is literal results in wrong answer

2016-08-31 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17114: --- Labels: correctness (was: ) > Adding a 'GROUP BY 1' where first column is literal results in wrong

[jira] [Updated] (SPARK-10169) Evaluating AggregateFunction1 (old code path) may return wrong answers when grouping expressions are used as arguments of aggregate functions

2016-08-31 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-10169: --- Labels: correctness (was: ) > Evaluating AggregateFunction1 (old code path) may return wrong

[jira] [Updated] (SPARK-7965) Wrong answers for queries with multiple window specs in the same expression

2016-08-31 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-7965: -- Labels: correctness (was: ) > Wrong answers for queries with multiple window specs in the same

[jira] [Updated] (SPARK-13221) GroupingSets Returns an Incorrect Results

2016-08-31 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-13221: --- Labels: correctness (was: ) > GroupingSets Returns an Incorrect Results >

[jira] [Updated] (SPARK-11883) New Parquet reader generate wrong result

2016-08-31 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-11883: --- Labels: correctness (was: ) > New Parquet reader generate wrong result >

[jira] [Updated] (SPARK-11949) Query on DataFrame from cube gives wrong results

2016-08-31 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-11949: --- Labels: correctness dataframe sql (was: dataframe sql) > Query on DataFrame from cube gives wrong

[jira] [Updated] (SPARK-17228) Not infer/propagate non-deterministic constraints

2016-08-31 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17228: --- Labels: correctness (was: ) > Not infer/propagate non-deterministic constraints >

[jira] [Updated] (SPARK-16837) TimeWindow incorrectly drops slideDuration in constructors

2016-08-31 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-16837: --- Labels: correctness (was: ) > TimeWindow incorrectly drops slideDuration in constructors >

[jira] [Updated] (SPARK-17244) Joins should not pushdown non-deterministic conditions

2016-08-31 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17244: --- Labels: correctness (was: ) > Joins should not pushdown non-deterministic conditions >

[jira] [Comment Edited] (SPARK-17211) Broadcast join produces incorrect results on EMR with large driver memory

2016-08-31 Thread gurmukh singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453541#comment-15453541 ] gurmukh singh edited comment on SPARK-17211 at 8/31/16 10:24 PM: - Hi I

[jira] [Commented] (SPARK-17147) Spark Streaming Kafka 0.10 Consumer Can't Handle Non-consecutive Offsets

2016-08-31 Thread Robert Conrad (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453229#comment-15453229 ] Robert Conrad commented on SPARK-17147: --- [~graphex] you're absolutely right about the seek, but

[jira] [Commented] (SPARK-7445) StringIndexer should handle binary labels properly

2016-08-31 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453318#comment-15453318 ] yuhao yang commented on SPARK-7445: --- Yuhao on business trip from Aug 31th to Sep 2nd. Email response

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-08-31 Thread Robert Conrad (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453334#comment-15453334 ] Robert Conrad commented on SPARK-15406: --- Not sure about others but my use-case is to consume a

[jira] [Updated] (SPARK-17120) Analyzer incorrectly optimizes plan to empty LocalRelation

2016-08-31 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17120: --- Labels: correctness (was: ) > Analyzer incorrectly optimizes plan to empty LocalRelation >

[jira] [Updated] (SPARK-16994) Filter and limit are illegally permuted.

2016-08-31 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-16994: --- Labels: correctness (was: ) > Filter and limit are illegally permuted. >

[jira] [Commented] (SPARK-17316) Don't block StandaloneSchedulerBackend.executorRemoved

2016-08-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453471#comment-15453471 ] Apache Spark commented on SPARK-17316: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Commented] (SPARK-17342) Style of event timeline is broken

2016-08-31 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453235#comment-15453235 ] Dongjoon Hyun commented on SPARK-17342: --- Oops. Thank you for reporting and going to fix that,

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-08-31 Thread Ofir Manor (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453289#comment-15453289 ] Ofir Manor commented on SPARK-15406: Cody, why do you think Structured Streaming support for Kafka

[jira] [Commented] (SPARK-15570) Pregel functions fail when run multiple times in the same jvm using sequence of graphs

2016-08-31 Thread Shishir Kharel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453331#comment-15453331 ] Shishir Kharel commented on SPARK-15570: It looks like the problem does not exist with Spark 2.0.

[jira] [Updated] (SPARK-12030) Incorrect results when aggregate joined data

2016-08-31 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-12030: --- Labels: correctness (was: ) > Incorrect results when aggregate joined data >

[jira] [Updated] (SPARK-17348) Incorrect results from subquery transformation

2016-08-31 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17348: --- Labels: correctness (was: ) > Incorrect results from subquery transformation >

[jira] [Updated] (SPARK-17347) Encoder in Dataset example has incorrect type

2016-08-31 Thread Nan Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu updated SPARK-17347: Summary: Encoder in Dataset example has incorrect type (was: Encoder in Dataset example is incorrect on

[jira] [Commented] (SPARK-7445) StringIndexer should handle binary labels properly

2016-08-31 Thread Ruben Janssen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453317#comment-15453317 ] Ruben Janssen commented on SPARK-7445: -- I agree, could we then close this JIRA if [~mengxr] agrees?

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-08-31 Thread Ofir Manor (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453412#comment-15453412 ] Ofir Manor commented on SPARK-15406: For me - structured streaming is currently all about real window

[jira] [Resolved] (SPARK-17349) Update testthat package on Jenkins

2016-08-31 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shane knapp resolved SPARK-17349. - Resolution: Fixed done! > Update testthat package on Jenkins >

[jira] [Commented] (SPARK-14234) Executor crashes for TaskRunner thread interruption

2016-08-31 Thread Barry Becker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453414#comment-15453414 ] Barry Becker commented on SPARK-14234: -- Is it a lot of work to backport this fix 1.6.3? We have an

[jira] [Updated] (SPARK-17349) Update testthat package on Jenkins

2016-08-31 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-17349: -- Assignee: shane knapp > Update testthat package on Jenkins >

[jira] [Updated] (SPARK-16721) Lead/lag needs to respect nulls

2016-08-31 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-16721: --- Labels: correctness (was: ) > Lead/lag needs to respect nulls > >

[jira] [Commented] (SPARK-17339) Fix SparkR tests on Windows

2016-08-31 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453139#comment-15453139 ] Shivaram Venkataraman commented on SPARK-17339: --- We are discussing automating Windows

[jira] [Commented] (SPARK-17348) Incorrect results from subquery transformation

2016-08-31 Thread Nattavut Sutyanyong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453285#comment-15453285 ] Nattavut Sutyanyong commented on SPARK-17348: - The root cause is in the Analysis phase where

[jira] [Updated] (SPARK-17326) Tests with HiveContext in SparkR being skipped always

2016-08-31 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-17326: -- Assignee: Hyukjin Kwon > Tests with HiveContext in SparkR being skipped always

[jira] [Resolved] (SPARK-17326) Tests with HiveContext in SparkR being skipped always

2016-08-31 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-17326. --- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue

[jira] [Updated] (SPARK-6851) Wrong answers for self joins of converted parquet relations

2016-08-31 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-6851: -- Labels: correctness (was: ) > Wrong answers for self joins of converted parquet relations >

[jira] [Commented] (SPARK-17195) Dealing with JDBC column nullability when it is not reliable

2016-08-31 Thread Jason Moore (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453801#comment-15453801 ] Jason Moore commented on SPARK-17195: - That's right, and I totally agree that's where the fix needs

[jira] [Commented] (SPARK-17341) Can't read Parquet data with fields containing periods "."

2016-08-31 Thread Don Drake (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453806#comment-15453806 ] Don Drake commented on SPARK-17341: --- I just downloaded the nightly build from 8/31/2016 and gave it a

[jira] [Assigned] (SPARK-17351) Refactor JDBCRDD to expose JDBC -> SparkSQL conversion functionality

2016-08-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17351: Assignee: Apache Spark (was: Josh Rosen) > Refactor JDBCRDD to expose JDBC -> SparkSQL

[jira] [Created] (SPARK-17351) Refactor JDBCRDD to expose JDBC -> SparkSQL conversion functionality

2016-08-31 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-17351: -- Summary: Refactor JDBCRDD to expose JDBC -> SparkSQL conversion functionality Key: SPARK-17351 URL: https://issues.apache.org/jira/browse/SPARK-17351 Project: Spark

[jira] [Commented] (SPARK-17351) Refactor JDBCRDD to expose JDBC -> SparkSQL conversion functionality

2016-08-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453896#comment-15453896 ] Apache Spark commented on SPARK-17351: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17351) Refactor JDBCRDD to expose JDBC -> SparkSQL conversion functionality

2016-08-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17351: Assignee: Josh Rosen (was: Apache Spark) > Refactor JDBCRDD to expose JDBC -> SparkSQL

[jira] [Updated] (SPARK-17309) ALTER VIEW should throw exception if view not exist

2016-08-31 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-17309: Fix Version/s: 2.01 > ALTER VIEW should throw exception if view not exist >

[jira] [Updated] (SPARK-17323) ALTER VIEW AS should keep the previous table properties, comment, create_time, etc.

2016-08-31 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-17323: Fix Version/s: 2.0.1 > ALTER VIEW AS should keep the previous table properties, comment, >

[jira] [Updated] (SPARK-17180) Unable to Alter the Temporary View Using ALTER VIEW command

2016-08-31 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-17180: Fix Version/s: 2.0.1 > Unable to Alter the Temporary View Using ALTER VIEW command >

[jira] [Commented] (SPARK-5992) Locality Sensitive Hashing (LSH) for MLlib

2016-08-31 Thread Yun Ni (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453980#comment-15453980 ] Yun Ni commented on SPARK-5992: --- Thanks very much for reviewing, Joseph! Based on your comments, I have

[jira] [Commented] (SPARK-17211) Broadcast join produces incorrect results on EMR with large driver memory

2016-08-31 Thread gurmukh singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15454147#comment-15454147 ] gurmukh singh commented on SPARK-17211: --- yes, outside of EMR. On a node you will have OS,

[jira] [Updated] (SPARK-16942) CREATE TABLE LIKE generates External table when source table is an External Hive Serde table

2016-08-31 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-16942: Description: When the table type of source table is an EXTERNAL Hive serde table, {{CREATE TABLE LIKE}}

[jira] [Created] (SPARK-17352) Executor computing time can be negative-number because of calculation error

2016-08-31 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-17352: -- Summary: Executor computing time can be negative-number because of calculation error Key: SPARK-17352 URL: https://issues.apache.org/jira/browse/SPARK-17352

[jira] [Commented] (SPARK-17318) Fix flaky test: o.a.s.repl.ReplSuite replicating blocks of object with class defined in repl

2016-08-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453793#comment-15453793 ] Apache Spark commented on SPARK-17318: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17318) Fix flaky test: o.a.s.repl.ReplSuite replicating blocks of object with class defined in repl

2016-08-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17318: Assignee: Shixiong Zhu (was: Apache Spark) > Fix flaky test: o.a.s.repl.ReplSuite

[jira] [Assigned] (SPARK-17318) Fix flaky test: o.a.s.repl.ReplSuite replicating blocks of object with class defined in repl

2016-08-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17318: Assignee: Apache Spark (was: Shixiong Zhu) > Fix flaky test: o.a.s.repl.ReplSuite

[jira] [Comment Edited] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-08-31 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453789#comment-15453789 ] Cody Koeninger edited comment on SPARK-15406 at 9/1/16 2:26 AM: There's a

[jira] [Assigned] (SPARK-17353) CREATE TABLE LIKE statements when Source is a VIEW

2016-08-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17353: Assignee: Apache Spark > CREATE TABLE LIKE statements when Source is a VIEW >

[jira] [Updated] (SPARK-17319) Move addJar from HiveSessionState to HiveSharedState

2016-08-31 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-17319: Summary: Move addJar from HiveSessionState to HiveSharedState (was: Move addJar from HiveSessionState to

[jira] [Resolved] (SPARK-17241) SparkR spark.glm should have configurable regularization parameter

2016-08-31 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-17241. --- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request

[jira] [Updated] (SPARK-17241) SparkR spark.glm should have configurable regularization parameter

2016-08-31 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-17241: -- Assignee: Xin Ren > SparkR spark.glm should have configurable regularization

[jira] [Commented] (SPARK-17342) Style of event timeline is broken

2016-08-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453694#comment-15453694 ] Apache Spark commented on SPARK-17342: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17342) Style of event timeline is broken

2016-08-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17342: Assignee: (was: Apache Spark) > Style of event timeline is broken >

[jira] [Created] (SPARK-17350) Disable default use of KryoSerializer in Thrift Server

2016-08-31 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-17350: -- Summary: Disable default use of KryoSerializer in Thrift Server Key: SPARK-17350 URL: https://issues.apache.org/jira/browse/SPARK-17350 Project: Spark Issue

[jira] [Updated] (SPARK-17180) Unable to Alter the Temporary View Using ALTER VIEW command

2016-08-31 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-17180: Assignee: Wenchen Fan > Unable to Alter the Temporary View Using ALTER VIEW command >

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-08-31 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15454043#comment-15454043 ] Reynold Xin commented on SPARK-15406: - +1. > Structured streaming support for consuming from Kafka >

[jira] [Updated] (SPARK-17319) Move addJar from HiveSessionState to HiveSharedState

2016-08-31 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-17319: Description: aThe added jar is shared by all the sessions, because SparkContext does not support

[jira] [Updated] (SPARK-17319) Move addJar from HiveSessionState to HiveSharedState

2016-08-31 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-17319: Description: aThe added jar is shared by all the sessions, because SparkContext does not support

[jira] [Assigned] (SPARK-17342) Style of event timeline is broken

2016-08-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17342: Assignee: Apache Spark > Style of event timeline is broken >

[jira] [Commented] (SPARK-17341) Can't read Parquet data with fields containing periods "."

2016-08-31 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453775#comment-15453775 ] Hyukjin Kwon commented on SPARK-17341: -- Ah, the issue itself seems not duplicated but the fix should

[jira] [Commented] (SPARK-17147) Spark Streaming Kafka 0.10 Consumer Can't Handle Non-consecutive Offsets

2016-08-31 Thread Sean McKibben (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453884#comment-15453884 ] Sean McKibben commented on SPARK-17147: --- I think Kafka's log compaction's design is still intended

[jira] [Comment Edited] (SPARK-17147) Spark Streaming Kafka 0.10 Consumer Can't Handle Non-consecutive Offsets

2016-08-31 Thread Sean McKibben (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453884#comment-15453884 ] Sean McKibben edited comment on SPARK-17147 at 9/1/16 1:08 AM: --- I think

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-08-31 Thread Frederick Reiss (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453749#comment-15453749 ] Frederick Reiss commented on SPARK-15406: - WRT Kafka 0.8: I'm under the impression that there is

[jira] [Commented] (SPARK-17349) Update testthat package on Jenkins

2016-08-31 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453784#comment-15453784 ] Hyukjin Kwon commented on SPARK-17349: -- Cool! > Update testthat package on Jenkins >

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-08-31 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453789#comment-15453789 ] Cody Koeninger commented on SPARK-15406: There's a big difference between continuing to publish

[jira] [Assigned] (SPARK-17350) Disable default use of KryoSerializer in Thrift Server

2016-08-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17350: Assignee: Josh Rosen (was: Apache Spark) > Disable default use of KryoSerializer in

  1   2   3   >