[jira] [Comment Edited] (SPARK-18187) CompactibleFileStreamLog should not rely on "compactInterval" to detect a compaction batch

[ https://issues.apache.org/jira/browse/SPARK-18187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646800#comment-15646800 ] Genmao Yu edited comment on SPARK-18187 at 11/8/16 7:53 AM: [

[jira] [Commented] (SPARK-18187) CompactibleFileStreamLog should not rely on "compactInterval" to detect a compaction batch

[ https://issues.apache.org/jira/browse/SPARK-18187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646800#comment-15646800 ] Genmao Yu commented on SPARK-18187: --- [~marmbrus] +1 to your +think the configuration

[jira] [Commented] (SPARK-18353) spark.rpc.askTimeout defalut value is not 120s

[ https://issues.apache.org/jira/browse/SPARK-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646795#comment-15646795 ] Jason Pan commented on SPARK-18353: --- in org.apache.spark.deploy.Client, there is one li

[jira] [Updated] (SPARK-18353) spark.rpc.askTimeout defalut value is not 120s

[ https://issues.apache.org/jira/browse/SPARK-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Pan updated SPARK-18353: -- Description: in http://spark.apache.org/docs/latest/configuration.html spark.rpc.askTimeout 120s

[jira] [Created] (SPARK-18353) spark.rpc.askTimeout defalut value is not 120s

Jason Pan created SPARK-18353: - Summary: spark.rpc.askTimeout defalut value is not 120s Key: SPARK-18353 URL: https://issues.apache.org/jira/browse/SPARK-18353 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-16496) Add wholetext as option for reading text in SQL.

[ https://issues.apache.org/jira/browse/SPARK-16496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-16496: Target Version/s: 2.2.0 > Add wholetext as option for reading text in SQL. > --

[jira] [Reopened] (SPARK-17969) I think it's user unfriendly to process standard json file with DataFrame

[ https://issues.apache.org/jira/browse/SPARK-17969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reopened SPARK-17969: - > I think it's user unfriendly to process standard json file with DataFrame > --

[jira] [Closed] (SPARK-10840) SparkSQL doesn't work well with JSON

[ https://issues.apache.org/jira/browse/SPARK-10840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-10840. --- Resolution: Duplicate > SparkSQL doesn't work well with JSON > >

[jira] [Closed] (SPARK-17969) I think it's user unfriendly to process standard json file with DataFrame

[ https://issues.apache.org/jira/browse/SPARK-17969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-17969. --- Resolution: Duplicate > I think it's user unfriendly to process standard json file with DataFrame >

[jira] [Reopened] (SPARK-7366) Support multi-line JSON objects

[ https://issues.apache.org/jira/browse/SPARK-7366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reopened SPARK-7366: > Support multi-line JSON objects > --- > > Key: SPARK-7366

[jira] [Updated] (SPARK-18352) Parse normal, multi-line JSON files (not just JSON Lines)

[ https://issues.apache.org/jira/browse/SPARK-18352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18352: Summary: Parse normal, multi-line JSON files (not just JSON Lines) (was: Parse normal JSON files (

[jira] [Closed] (SPARK-7366) Support multi-line JSON objects

[ https://issues.apache.org/jira/browse/SPARK-7366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-7366. -- Resolution: Duplicate > Support multi-line JSON objects > --- > >

[jira] [Closed] (SPARK-7366) Support multi-line JSON objects

[ https://issues.apache.org/jira/browse/SPARK-7366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-7366. -- Resolution: Fixed I'm closing this in favor of https://issues.apache.org/jira/browse/SPARK-18352 In rea

[jira] [Created] (SPARK-18352) Parse normal JSON files (not just JSON Lines)

Reynold Xin created SPARK-18352: --- Summary: Parse normal JSON files (not just JSON Lines) Key: SPARK-18352 URL: https://issues.apache.org/jira/browse/SPARK-18352 Project: Spark Issue Type: New F

[jira] [Resolved] (SPARK-18351) from_json and to_json for parsing JSON for string columns

[ https://issues.apache.org/jira/browse/SPARK-18351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18351. - Resolution: Fixed Fix Version/s: 2.1.0 > from_json and to_json for parsing JSON for string

[jira] [Updated] (SPARK-18295) Match up to_json to from_json in null safety

[ https://issues.apache.org/jira/browse/SPARK-18295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18295: Assignee: Hyukjin Kwon > Match up to_json to from_json in null safety > ---

[jira] [Updated] (SPARK-18295) Match up to_json to from_json in null safety

[ https://issues.apache.org/jira/browse/SPARK-18295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18295: Issue Type: Sub-task (was: Bug) Parent: SPARK-18351 > Match up to_json to from_json in nul

[jira] [Updated] (SPARK-17764) to_json function for parsing Structs to json Strings

[ https://issues.apache.org/jira/browse/SPARK-17764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17764: Issue Type: Sub-task (was: Improvement) Parent: SPARK-18351 > to_json function for parsing

[jira] [Created] (SPARK-18351) from_json and to_json for parsing JSON for string columns

Reynold Xin created SPARK-18351: --- Summary: from_json and to_json for parsing JSON for string columns Key: SPARK-18351 URL: https://issues.apache.org/jira/browse/SPARK-18351 Project: Spark Issue

[jira] [Updated] (SPARK-18260) from_json can throw a better exception when it can't find the column or be nullSafe

[ https://issues.apache.org/jira/browse/SPARK-18260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18260: Issue Type: Sub-task (was: Bug) Parent: SPARK-18351 > from_json can throw a better excepti

[jira] [Updated] (SPARK-17699) from_json function for parsing json Strings into Structs

[ https://issues.apache.org/jira/browse/SPARK-17699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17699: Issue Type: Sub-task (was: New Feature) Parent: SPARK-18351 > from_json function for parsi

[jira] [Commented] (SPARK-18350) Support session local timezone

[ https://issues.apache.org/jira/browse/SPARK-18350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646704#comment-15646704 ] Reynold Xin commented on SPARK-18350: - I'm guessing the easiest way to do this is to

[jira] [Created] (SPARK-18350) Support session local timezone

Reynold Xin created SPARK-18350: --- Summary: Support session local timezone Key: SPARK-18350 URL: https://issues.apache.org/jira/browse/SPARK-18350 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-16545) Structured Streaming : foreachSink creates the Physical Plan multiple times per TriggerInterval

[ https://issues.apache.org/jira/browse/SPARK-16545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646665#comment-15646665 ] Mario Briggs commented on SPARK-16545: -- [~lwlin] I agree with the PR discussion. I a

[jira] [Created] (SPARK-18349) Update R API documentation on ml model summary

Felix Cheung created SPARK-18349: Summary: Update R API documentation on ml model summary Key: SPARK-18349 URL: https://issues.apache.org/jira/browse/SPARK-18349 Project: Spark Issue Type: Bu

[jira] [Created] (SPARK-18348) Improve tree ensemble model summary

Felix Cheung created SPARK-18348: Summary: Improve tree ensemble model summary Key: SPARK-18348 URL: https://issues.apache.org/jira/browse/SPARK-18348 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-18347) Infra for R - need qpdf on Jenkins

[ https://issues.apache.org/jira/browse/SPARK-18347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-18347: - Description: As a part of working on building R package (https://issues.apache.org/jira/browse/S

[jira] [Updated] (SPARK-18347) Infra for R - need qpdf on Jenkins

[ https://issues.apache.org/jira/browse/SPARK-18347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-18347: - Description: As a part of working on building R package (https://issues.apache.org/jira/browse/S

[jira] [Created] (SPARK-18347) Infra for R - need qpdf on Jenkins

Felix Cheung created SPARK-18347: Summary: Infra for R - need qpdf on Jenkins Key: SPARK-18347 URL: https://issues.apache.org/jira/browse/SPARK-18347 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-18264) Build and package R vignettes

[ https://issues.apache.org/jira/browse/SPARK-18264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-18264: - Summary: Build and package R vignettes (was: Package R vignettes) > Build and package R vignette

[jira] [Commented] (SPARK-18332) SparkR 2.1 QA: Programming guide update and migration guide

[ https://issues.apache.org/jira/browse/SPARK-18332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646598#comment-15646598 ] Felix Cheung commented on SPARK-18332: -- The R vignettes is a R-specific thing that i

[jira] [Comment Edited] (SPARK-18332) SparkR 2.1 QA: Programming guide update and migration guide

[ https://issues.apache.org/jira/browse/SPARK-18332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646598#comment-15646598 ] Felix Cheung edited comment on SPARK-18332 at 11/8/16 5:46 AM:

[jira] [Assigned] (SPARK-18345) Structured Streaming quick examples fails with default configuration

[ https://issues.apache.org/jira/browse/SPARK-18345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18345: Assignee: Apache Spark > Structured Streaming quick examples fails with default configurat

[jira] [Commented] (SPARK-18345) Structured Streaming quick examples fails with default configuration

[ https://issues.apache.org/jira/browse/SPARK-18345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646594#comment-15646594 ] Apache Spark commented on SPARK-18345: -- User 'oza' has created a pull request for th

[jira] [Issue Comment Deleted] (SPARK-18332) SparkR 2.1 QA: Programming guide update and migration guide

[ https://issues.apache.org/jira/browse/SPARK-18332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-18332: - Comment: was deleted (was: link https://issues.apache.org/jira/browse/SPARK-18279 https://issues.

[jira] [Assigned] (SPARK-18345) Structured Streaming quick examples fails with default configuration

[ https://issues.apache.org/jira/browse/SPARK-18345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18345: Assignee: (was: Apache Spark) > Structured Streaming quick examples fails with default

[jira] [Commented] (SPARK-18332) SparkR 2.1 QA: Programming guide update and migration guide

[ https://issues.apache.org/jira/browse/SPARK-18332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646592#comment-15646592 ] Felix Cheung commented on SPARK-18332: -- link https://issues.apache.org/jira/browse/S

[jira] [Commented] (SPARK-16892) flatten function to get flat array (or map) column from array of array (or array of map) column

[ https://issues.apache.org/jira/browse/SPARK-16892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646584#comment-15646584 ] Kapil Singh commented on SPARK-16892: - It's not for flattening Rows. It's for flatten

[jira] [Commented] (SPARK-18346) TRUNCATE TABLE should fail if no partition is matched for the given non-partial partition spec

[ https://issues.apache.org/jira/browse/SPARK-18346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646578#comment-15646578 ] Apache Spark commented on SPARK-18346: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-18346) TRUNCATE TABLE should fail if no partition is matched for the given non-partial partition spec

[ https://issues.apache.org/jira/browse/SPARK-18346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18346: Assignee: Wenchen Fan (was: Apache Spark) > TRUNCATE TABLE should fail if no partition is

[jira] [Assigned] (SPARK-18346) TRUNCATE TABLE should fail if no partition is matched for the given non-partial partition spec

[ https://issues.apache.org/jira/browse/SPARK-18346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18346: Assignee: Apache Spark (was: Wenchen Fan) > TRUNCATE TABLE should fail if no partition is

[jira] [Commented] (SPARK-16894) take function for returning the first n elements of array column

[ https://issues.apache.org/jira/browse/SPARK-16894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646576#comment-15646576 ] Kapil Singh commented on SPARK-16894: - This is not about selecting first n elements/c

[jira] [Created] (SPARK-18346) TRUNCATE TABLE should fail if no partition is matched for the given non-partial partition spec

Wenchen Fan created SPARK-18346: --- Summary: TRUNCATE TABLE should fail if no partition is matched for the given non-partial partition spec Key: SPARK-18346 URL: https://issues.apache.org/jira/browse/SPARK-18346

[jira] [Commented] (SPARK-16894) take function for returning the first n elements of array column

[ https://issues.apache.org/jira/browse/SPARK-16894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646563#comment-15646563 ] Kapil Singh commented on SPARK-16894: - The use case is similar to scala collection's

[jira] [Commented] (SPARK-18345) Structured Streaming quick examples fails with default configuration

[ https://issues.apache.org/jira/browse/SPARK-18345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646551#comment-15646551 ] Tsuyoshi Ozawa commented on SPARK-18345: I would like to tackle this problem. I f

[jira] [Created] (SPARK-18345) Structured Streaming quick examples fails with default configuration

Tsuyoshi Ozawa created SPARK-18345: -- Summary: Structured Streaming quick examples fails with default configuration Key: SPARK-18345 URL: https://issues.apache.org/jira/browse/SPARK-18345 Project: Spa

[jira] [Created] (SPARK-18344) TRUNCATE TABLE should fail if no partition is matched for the given non-partial partition spec

Wenchen Fan created SPARK-18344: --- Summary: TRUNCATE TABLE should fail if no partition is matched for the given non-partial partition spec Key: SPARK-18344 URL: https://issues.apache.org/jira/browse/SPARK-18344

[jira] [Comment Edited] (SPARK-18055) Dataset.flatMap can't work with types from customized jar

[ https://issues.apache.org/jira/browse/SPARK-18055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646477#comment-15646477 ] Song Jun edited comment on SPARK-18055 at 11/8/16 5:04 AM: --- [~d

[jira] [Created] (SPARK-18343) FileSystem$Statistics$StatisticsDataReferenceCleaner hangs on s3 write

Luke Miner created SPARK-18343: -- Summary: FileSystem$Statistics$StatisticsDataReferenceCleaner hangs on s3 write Key: SPARK-18343 URL: https://issues.apache.org/jira/browse/SPARK-18343 Project: Spark

[jira] [Comment Edited] (SPARK-18055) Dataset.flatMap can't work with types from customized jar

[ https://issues.apache.org/jira/browse/SPARK-18055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646477#comment-15646477 ] Song Jun edited comment on SPARK-18055 at 11/8/16 4:51 AM: --- [~d

[jira] [Commented] (SPARK-18342) HDFSBackedStateStore can fail to rename files causing snapshotting and recovery to fail

[ https://issues.apache.org/jira/browse/SPARK-18342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646484#comment-15646484 ] Apache Spark commented on SPARK-18342: -- User 'brkyvz' has created a pull request for

[jira] [Assigned] (SPARK-18342) HDFSBackedStateStore can fail to rename files causing snapshotting and recovery to fail

[ https://issues.apache.org/jira/browse/SPARK-18342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18342: Assignee: (was: Apache Spark) > HDFSBackedStateStore can fail to rename files causing

[jira] [Assigned] (SPARK-18342) HDFSBackedStateStore can fail to rename files causing snapshotting and recovery to fail

[ https://issues.apache.org/jira/browse/SPARK-18342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18342: Assignee: Apache Spark > HDFSBackedStateStore can fail to rename files causing snapshottin

[jira] [Commented] (SPARK-18055) Dataset.flatMap can't work with types from customized jar

[ https://issues.apache.org/jira/browse/SPARK-18055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646477#comment-15646477 ] Song Jun commented on SPARK-18055: -- [~davies] I can't reproduce it on the master branch

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646453#comment-15646453 ] Harish commented on SPARK-17463: I was able to figure out the issue, its not related to

[jira] [Comment Edited] (SPARK-16892) flatten function to get flat array (or map) column from array of array (or array of map) column

[ https://issues.apache.org/jira/browse/SPARK-16892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646443#comment-15646443 ] Jayadevan M edited comment on SPARK-16892 at 11/8/16 4:21 AM: -

[jira] [Commented] (SPARK-16892) flatten function to get flat array (or map) column from array of array (or array of map) column

[ https://issues.apache.org/jira/browse/SPARK-16892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646443#comment-15646443 ] Jayadevan M commented on SPARK-16892: - I hope you can use flatMap for this scenario.

[jira] [Created] (SPARK-18342) HDFSBackedStateStore can fail to rename files causing snapshotting and recovery to fail

Burak Yavuz created SPARK-18342: --- Summary: HDFSBackedStateStore can fail to rename files causing snapshotting and recovery to fail Key: SPARK-18342 URL: https://issues.apache.org/jira/browse/SPARK-18342

[jira] [Created] (SPARK-18341) Eliminate use of SingularMatrixException in WeightedLeastSquares logic

Joseph K. Bradley created SPARK-18341: - Summary: Eliminate use of SingularMatrixException in WeightedLeastSquares logic Key: SPARK-18341 URL: https://issues.apache.org/jira/browse/SPARK-18341 Proj

[jira] [Commented] (SPARK-18298) HistoryServer use GMT time all time

[ https://issues.apache.org/jira/browse/SPARK-18298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646333#comment-15646333 ] Song Jun commented on SPARK-18298: -- [~WangTao] I test it,and the ui should show the use

[jira] [Commented] (SPARK-18298) HistoryServer use GMT time all time

[ https://issues.apache.org/jira/browse/SPARK-18298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646270#comment-15646270 ] Tao Wang commented on SPARK-18298: -- [~ajbozarth] Thanks for your attention. I think it'

[jira] [Commented] (SPARK-18298) HistoryServer use GMT time all time

[ https://issues.apache.org/jira/browse/SPARK-18298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646269#comment-15646269 ] Tao Wang commented on SPARK-18298: -- [~ajbozarth] Thanks for your attention. I think it'

[jira] [Resolved] (SPARK-16575) partition calculation mismatch with sc.binaryFiles

[ https://issues.apache.org/jira/browse/SPARK-16575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-16575. - Resolution: Fixed Assignee: Tarun Kumar Fix Version/s: 2.1.0 > partition calculat

[jira] [Commented] (SPARK-18263) Configuring spark.kryo.registrator programmatically doesn't take effect

[ https://issues.apache.org/jira/browse/SPARK-18263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646224#comment-15646224 ] inred commented on SPARK-18263: --- config at builder take effect val spark = SparkSession

[jira] [Resolved] (SPARK-18263) Configuring spark.kryo.registrator programmatically doesn't take effect

[ https://issues.apache.org/jira/browse/SPARK-18263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] inred resolved SPARK-18263. --- Resolution: Fixed > Configuring spark.kryo.registrator programmatically doesn't take effect > ---

[jira] [Closed] (SPARK-18263) Configuring spark.kryo.registrator programmatically doesn't take effect

[ https://issues.apache.org/jira/browse/SPARK-18263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] inred closed SPARK-18263. - > Configuring spark.kryo.registrator programmatically doesn't take effect > -

[jira] [Updated] (SPARK-18339) Don't push down current_timestamp for filters in StructuredStreaming

[ https://issues.apache.org/jira/browse/SPARK-18339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-18339: - Target Version/s: 2.2.0 > Don't push down current_timestamp for filters in StructuredStre

[jira] [Resolved] (SPARK-18217) Disallow creating permanent views based on temporary views or UDFs

[ https://issues.apache.org/jira/browse/SPARK-18217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18217. - Resolution: Fixed Fix Version/s: 2.1.0 > Disallow creating permanent views based on tempor

[jira] [Updated] (SPARK-16609) Single function for parsing timestamps/dates

[ https://issues.apache.org/jira/browse/SPARK-16609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-16609: Target Version/s: 2.2.0 (was: 2.1.0) > Single function for parsing timestamps/dates >

[jira] [Assigned] (SPARK-18298) HistoryServer use GMT time all time

[ https://issues.apache.org/jira/browse/SPARK-18298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18298: Assignee: (was: Apache Spark) > HistoryServer use GMT time all time >

[jira] [Commented] (SPARK-18298) HistoryServer use GMT time all time

[ https://issues.apache.org/jira/browse/SPARK-18298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646209#comment-15646209 ] Apache Spark commented on SPARK-18298: -- User 'windpiger' has created a pull request

[jira] [Assigned] (SPARK-18298) HistoryServer use GMT time all time

[ https://issues.apache.org/jira/browse/SPARK-18298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18298: Assignee: Apache Spark > HistoryServer use GMT time all time > ---

[jira] [Updated] (SPARK-18340) Inconsistent error messages in launching scripts and hanging in sparkr script for wrong options

[ https://issues.apache.org/jira/browse/SPARK-18340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-18340: - Description: It seems there are some problems with handling wrong options as below: *{{spark-sub

[jira] [Created] (SPARK-18340) Inconsistent error messages in launching scripts and hanging in sparkr script for wrong options

Hyukjin Kwon created SPARK-18340: Summary: Inconsistent error messages in launching scripts and hanging in sparkr script for wrong options Key: SPARK-18340 URL: https://issues.apache.org/jira/browse/SPARK-18340

[jira] [Updated] (SPARK-17019) Expose off-heap memory usage in various places

[ https://issues.apache.org/jira/browse/SPARK-17019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17019: Target Version/s: 2.2.0 (was: 2.1.0) > Expose off-heap memory usage in various places > --

[jira] [Updated] (SPARK-16317) Add file filtering interface for FileFormat

[ https://issues.apache.org/jira/browse/SPARK-16317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-16317: Target Version/s: 2.2.0 (was: 2.1.0) > Add file filtering interface for FileFormat > -

[jira] [Resolved] (SPARK-18261) Add statistics to MemorySink for joining

[ https://issues.apache.org/jira/browse/SPARK-18261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18261. - Resolution: Fixed Assignee: Liwei Lin Fix Version/s: 2.1.0 > Add statistics to Me

[jira] [Resolved] (SPARK-18086) Regression: Hive variables no longer work in Spark 2.0

[ https://issues.apache.org/jira/browse/SPARK-18086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18086. - Resolution: Fixed Assignee: Ryan Blue Fix Version/s: 2.1.0 > Regression: Hive var

[jira] [Created] (SPARK-18339) Don't push down current_timestamp for filters in StructuredStreaming

Burak Yavuz created SPARK-18339: --- Summary: Don't push down current_timestamp for filters in StructuredStreaming Key: SPARK-18339 URL: https://issues.apache.org/jira/browse/SPARK-18339 Project: Spark

[jira] [Resolved] (SPARK-18295) Match up to_json to from_json in null safety

[ https://issues.apache.org/jira/browse/SPARK-18295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-18295. -- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 15792 [htt

[jira] [Commented] (SPARK-18318) ML, Graph 2.1 QA: API: New Scala APIs, docs

[ https://issues.apache.org/jira/browse/SPARK-18318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646011#comment-15646011 ] Yanbo Liang commented on SPARK-18318: - I'm interested in contributing this task. Than

[jira] [Commented] (SPARK-18316) Spark MLlib, GraphX 2.1 QA umbrella

[ https://issues.apache.org/jira/browse/SPARK-18316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646006#comment-15646006 ] Yanbo Liang commented on SPARK-18316: - Typo? Here should be 2.1 rather than 2.0? > S

[jira] [Resolved] (SPARK-18236) Reduce memory usage of Spark UI and HistoryServer by reducing duplicate objects

[ https://issues.apache.org/jira/browse/SPARK-18236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-18236. Resolution: Fixed Fix Version/s: 2.2.0 Merged into master (2.2.0). > Reduce memory usage of

[jira] [Updated] (SPARK-18338) ObjectHashAggregateSuite fails under Maven builds

[ https://issues.apache.org/jira/browse/SPARK-18338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-18338: --- Description: Test case initialization order under Maven and SBT are different. Maven always creates

[jira] [Commented] (SPARK-18338) ObjectHashAggregateSuite fails under Maven builds

[ https://issues.apache.org/jira/browse/SPARK-18338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15645865#comment-15645865 ] Apache Spark commented on SPARK-18338: -- User 'liancheng' has created a pull request

[jira] [Assigned] (SPARK-18338) ObjectHashAggregateSuite fails under Maven builds

[ https://issues.apache.org/jira/browse/SPARK-18338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18338: Assignee: Apache Spark (was: Cheng Lian) > ObjectHashAggregateSuite fails under Maven bui

[jira] [Assigned] (SPARK-18338) ObjectHashAggregateSuite fails under Maven builds

[ https://issues.apache.org/jira/browse/SPARK-18338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18338: Assignee: Cheng Lian (was: Apache Spark) > ObjectHashAggregateSuite fails under Maven bui

[jira] [Created] (SPARK-18338) ObjectHashAggregateSuite fails under Maven builds

Cheng Lian created SPARK-18338: -- Summary: ObjectHashAggregateSuite fails under Maven builds Key: SPARK-18338 URL: https://issues.apache.org/jira/browse/SPARK-18338 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-17490) Optimize SerializeFromObject for primitive array

[ https://issues.apache.org/jira/browse/SPARK-17490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-17490. --- Resolution: Fixed Assignee: Kazuaki Ishizaki Fix Version/s: 2.1.0 > O

[jira] [Comment Edited] (SPARK-8614) Row order preservation for operations on MLlib IndexedRowMatrix

[ https://issues.apache.org/jira/browse/SPARK-8614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15645652#comment-15645652 ] Mark Tygert edited comment on SPARK-8614 at 11/7/16 10:36 PM: --

[jira] [Commented] (SPARK-18337) Memory Sink should be able to recover from checkpoints in Complete OutputMode

[ https://issues.apache.org/jira/browse/SPARK-18337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15645673#comment-15645673 ] Apache Spark commented on SPARK-18337: -- User 'brkyvz' has created a pull request for

[jira] [Assigned] (SPARK-18337) Memory Sink should be able to recover from checkpoints in Complete OutputMode

[ https://issues.apache.org/jira/browse/SPARK-18337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18337: Assignee: (was: Apache Spark) > Memory Sink should be able to recover from checkpoints

[jira] [Assigned] (SPARK-18337) Memory Sink should be able to recover from checkpoints in Complete OutputMode

[ https://issues.apache.org/jira/browse/SPARK-18337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18337: Assignee: Apache Spark > Memory Sink should be able to recover from checkpoints in Complet

[jira] [Created] (SPARK-18337) Memory Sink should be able to recover from checkpoints in Complete OutputMode

Burak Yavuz created SPARK-18337: --- Summary: Memory Sink should be able to recover from checkpoints in Complete OutputMode Key: SPARK-18337 URL: https://issues.apache.org/jira/browse/SPARK-18337 Project:

[jira] [Commented] (SPARK-8614) Row order preservation for operations on MLlib IndexedRowMatrix

[ https://issues.apache.org/jira/browse/SPARK-8614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15645652#comment-15645652 ] Mark Tygert commented on SPARK-8614: This remains a big issue, rendering the results p

[jira] [Updated] (SPARK-17993) Spark prints an avalanche of warning messages from Parquet when reading parquet files written by older versions of Parquet-mr

[ https://issues.apache.org/jira/browse/SPARK-17993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Allman updated SPARK-17993: --- Summary: Spark prints an avalanche of warning messages from Parquet when reading parquet file

[jira] [Assigned] (SPARK-18334) MinHash should use binary hash distance

[ https://issues.apache.org/jira/browse/SPARK-18334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18334: Assignee: (was: Apache Spark) > MinHash should use binary hash distance >

[jira] [Commented] (SPARK-18334) MinHash should use binary hash distance

[ https://issues.apache.org/jira/browse/SPARK-18334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15645540#comment-15645540 ] Apache Spark commented on SPARK-18334: -- User 'Yunni' has created a pull request for

[jira] [Assigned] (SPARK-18334) MinHash should use binary hash distance

[ https://issues.apache.org/jira/browse/SPARK-18334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18334: Assignee: Apache Spark > MinHash should use binary hash distance > ---

[jira] [Commented] (SPARK-17993) Spark spews a slew of harmless but annoying warning messages from Parquet when reading parquet files written by older versions of Parquet-mr

[ https://issues.apache.org/jira/browse/SPARK-17993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15645495#comment-15645495 ] Michael Allman commented on SPARK-17993: Thank you for your input, Keith. I agree

  1   2   3   4   >