[jira] [Comment Edited] (SPARK-15420) Repartition and sort before Parquet writes

[ https://issues.apache.org/jira/browse/SPARK-15420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627859#comment-15627859 ] Reynold Xin edited comment on SPARK-15420 at 11/2/16 5:57 AM: -- Ryan I looked

[jira] [Commented] (SPARK-15420) Repartition and sort before Parquet writes

[ https://issues.apache.org/jira/browse/SPARK-15420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627859#comment-15627859 ] Reynold Xin commented on SPARK-15420: - Ryan I looked at this just now (sorry not looking earlier). I

[jira] [Commented] (SPARK-18133) Python ML Pipeline Example has syntax errors

[ https://issues.apache.org/jira/browse/SPARK-18133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627856#comment-15627856 ] Apache Spark commented on SPARK-18133: -- User 'jagadeesanas2' has created a pull request for this

[jira] [Updated] (SPARK-15420) Repartition and sort before Parquet writes

[ https://issues.apache.org/jira/browse/SPARK-15420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-15420: Target Version/s: 2.2.0 (was: 2.1.0) > Repartition and sort before Parquet writes >

[jira] [Commented] (SPARK-17822) JVMObjectTracker.objMap may leak JVM objects

[ https://issues.apache.org/jira/browse/SPARK-17822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627846#comment-15627846 ] Felix Cheung commented on SPARK-17822: -- I don't have a good handle on what actually is the problem.

[jira] [Commented] (SPARK-18212) Flaky test: org.apache.spark.sql.kafka010.KafkaSourceSuite.assign from specific offsets

[ https://issues.apache.org/jira/browse/SPARK-18212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627838#comment-15627838 ] Cody Koeninger commented on SPARK-18212: So here's a heavily excerpted version of what I see

[jira] [Updated] (SPARK-17868) Do not use bitmasks during parsing and analysis of CUBE/ROLLUP/GROUPING SETS

[ https://issues.apache.org/jira/browse/SPARK-17868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17868: Target Version/s: 2.2.0 (was: 2.1.0) > Do not use bitmasks during parsing and analysis of

[jira] [Closed] (SPARK-17402) separate the management of temp views and metastore tables/views in SessionCatalog

[ https://issues.apache.org/jira/browse/SPARK-17402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-17402. --- Resolution: Fixed > separate the management of temp views and metastore tables/views in >

[jira] [Commented] (SPARK-18193) queueStream not updated if rddQueue.add after create queueStream in Java

[ https://issues.apache.org/jira/browse/SPARK-18193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627774#comment-15627774 ] Hubert Kang commented on SPARK-18193: - Thanks Sean. While it's inconsistent with that in

[jira] [Resolved] (SPARK-17992) HiveClient.getPartitionsByFilter throws an exception for some unsupported filters when hive.metastore.try.direct.sql=false

[ https://issues.apache.org/jira/browse/SPARK-17992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17992. - Resolution: Fixed Assignee: Michael Allman Fix Version/s: 2.1.0 >

[jira] [Commented] (SPARK-17838) Strict type checking for arguments with a better messages across APIs.

[ https://issues.apache.org/jira/browse/SPARK-17838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627770#comment-15627770 ] Felix Cheung commented on SPARK-17838: -- merged to master. this should be very safe to go in

[jira] [Resolved] (SPARK-17838) Strict type checking for arguments with a better messages across APIs.

[ https://issues.apache.org/jira/browse/SPARK-17838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-17838. -- Resolution: Fixed Assignee: Hyukjin Kwon Fix Version/s: 2.2.0

[jira] [Commented] (SPARK-7755) MetadataCache.refresh does not take into account _SUCCESS

[ https://issues.apache.org/jira/browse/SPARK-7755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627754#comment-15627754 ] Hyukjin Kwon commented on SPARK-7755: - Hi [~liancheng], didn't we remove

[jira] [Commented] (SPARK-18073) Migrate wiki to spark.apache.org web site

[ https://issues.apache.org/jira/browse/SPARK-18073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627731#comment-15627731 ] Siddharth Ahuja commented on SPARK-18073: - Hi [~srowen], I would be happy to work on this one if

[jira] [Closed] (SPARK-6825) Data sources implementation to support `sequenceFile`

[ https://issues.apache.org/jira/browse/SPARK-6825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-6825. -- Resolution: Won't Fix I'm marking this as won't fix for now. It's unclear how the interface would look

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627715#comment-15627715 ] Xiao Li commented on SPARK-18209: - Ok, will do it. Thanks! > More robust view canonicalization without

[jira] [Commented] (SPARK-6825) Data sources implementation to support `sequenceFile`

[ https://issues.apache.org/jira/browse/SPARK-6825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627714#comment-15627714 ] Hyukjin Kwon commented on SPARK-6825: - Hi [~shivaram], do we still need this? If so, I can maybe give

[jira] [Created] (SPARK-18217) Disallow creating permanent views based on temporary views

Reynold Xin created SPARK-18217: --- Summary: Disallow creating permanent views based on temporary views Key: SPARK-18217 URL: https://issues.apache.org/jira/browse/SPARK-18217 Project: Spark

[jira] [Assigned] (SPARK-18217) Disallow creating permanent views based on temporary views

[ https://issues.apache.org/jira/browse/SPARK-18217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-18217: --- Assignee: Xiao Li > Disallow creating permanent views based on temporary views >

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627710#comment-15627710 ] Reynold Xin commented on SPARK-18209: - Actually I'd consider it a "bug" and fix this bug first in

[jira] [Commented] (SPARK-16808) History Server main page does not honor APPLICATION_WEB_PROXY_BASE

[ https://issues.apache.org/jira/browse/SPARK-16808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627706#comment-15627706 ] Vinayak Joshi commented on SPARK-16808: --- The same issue also affects the case where

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627702#comment-15627702 ] Xiao Li commented on SPARK-18209: - True. {code} Seq((1, (1, 1))).toDF().createTempView("temp_jt")

[jira] [Comment Edited] (SPARK-4549) Support BigInt -> Decimal in convertToCatalyst in SparkSQL

[ https://issues.apache.org/jira/browse/SPARK-4549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627695#comment-15627695 ] Hyukjin Kwon edited comment on SPARK-4549 at 11/2/16 4:41 AM: -- Could we maybe

[jira] [Commented] (SPARK-4549) Support BigInt -> Decimal in convertToCatalyst in SparkSQL

[ https://issues.apache.org/jira/browse/SPARK-4549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627695#comment-15627695 ] Hyukjin Kwon commented on SPARK-4549: - Could we maybe close this for now if no one can't explain when

[jira] [Commented] (SPARK-17967) Support for list or other types as an option for datasources

[ https://issues.apache.org/jira/browse/SPARK-17967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627698#comment-15627698 ] Reynold Xin commented on SPARK-17967: - +1 on json arrays. > Support for list or other types as an

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627679#comment-15627679 ] Reynold Xin commented on SPARK-18209: - Yes, both global and local temp views. The issue with temp

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627673#comment-15627673 ] Xiao Li commented on SPARK-18209: - Without SQL expansion, also need to block the global temp view usage

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627664#comment-15627664 ] Reynold Xin commented on SPARK-18209: - Yup - we will have to disallow it, for good reasons actually.

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627662#comment-15627662 ] Xiao Li commented on SPARK-18209: - Yeah, I just posted the example to show it. I mentioned [~vssrinath]

[jira] [Commented] (SPARK-18133) Python ML Pipeline Example has syntax errors

[ https://issues.apache.org/jira/browse/SPARK-18133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627648#comment-15627648 ] Nirmal Fernando commented on SPARK-18133: - Thanks All. > Python ML Pipeline Example has syntax

[jira] [Resolved] (SPARK-18216) Make Column.expr public

[ https://issues.apache.org/jira/browse/SPARK-18216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18216. - Resolution: Fixed Fix Version/s: 2.1.0 > Make Column.expr public >

[jira] [Commented] (SPARK-18133) Python ML Pipeline Example has syntax errors

[ https://issues.apache.org/jira/browse/SPARK-18133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627633#comment-15627633 ] Apache Spark commented on SPARK-18133: -- User 'jagadeesanas2' has created a pull request for this

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627628#comment-15627628 ] Reynold Xin commented on SPARK-18209: - That's what [~vssrinath] pointed out isn't that? Good point

[jira] [Commented] (SPARK-17816) Json serialzation of accumulators are failing with ConcurrentModificationException

[ https://issues.apache.org/jira/browse/SPARK-17816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627622#comment-15627622 ] Jonathan Alvarado commented on SPARK-17816: --- Can I assume that I can disregard this error for

[jira] [Comment Edited] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627595#comment-15627595 ] Xiao Li edited comment on SPARK-18209 at 11/2/16 4:00 AM: -- If we do not qualify

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627599#comment-15627599 ] Xiao Li commented on SPARK-18209: - Please hold on until we finalize the design. > More robust view

[jira] [Assigned] (SPARK-17895) Improve documentation of "rowsBetween" and "rangeBetween"

[ https://issues.apache.org/jira/browse/SPARK-17895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17895: Assignee: (was: Apache Spark) > Improve documentation of "rowsBetween" and

[jira] [Assigned] (SPARK-17895) Improve documentation of "rowsBetween" and "rangeBetween"

[ https://issues.apache.org/jira/browse/SPARK-17895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17895: Assignee: Apache Spark > Improve documentation of "rowsBetween" and "rangeBetween" >

[jira] [Commented] (SPARK-17895) Improve documentation of "rowsBetween" and "rangeBetween"

[ https://issues.apache.org/jira/browse/SPARK-17895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627597#comment-15627597 ] Apache Spark commented on SPARK-17895: -- User 'david-weiluo-ren' has created a pull request for this

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627595#comment-15627595 ] Xiao Li commented on SPARK-18209: - If we do not qualify the table/persistent view name when we create a

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627588#comment-15627588 ] Xiao Li commented on SPARK-18209: - {code} sql("CREATE VIEW jtv1 AS SELECT * FROM jt WHERE id > 3")

[jira] [Updated] (SPARK-18198) Highlight code snippets for Streaming integretion docs

[ https://issues.apache.org/jira/browse/SPARK-18198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liwei Lin updated SPARK-18198: -- Component/s: (was: SQL) Structured Streaming > Highlight code snippets for

[jira] [Commented] (SPARK-16545) Structured Streaming : foreachSink creates the Physical Plan multiple times per TriggerInterval

[ https://issues.apache.org/jira/browse/SPARK-16545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627554#comment-15627554 ] Liwei Lin commented on SPARK-16545: --- hi [~mariobriggs], per discussion on the PR, would you mind

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627518#comment-15627518 ] Dongjoon Hyun commented on SPARK-18209: --- Thank you. Now, I understand. I have been wondering the

[jira] [Comment Edited] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627518#comment-15627518 ] Dongjoon Hyun edited comment on SPARK-18209 at 11/2/16 3:21 AM: Thank

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627502#comment-15627502 ] Reynold Xin commented on SPARK-18209: - The pr is still useful -- the only thing that made it very

[jira] [Commented] (SPARK-18206) Log instrumentation in MPC, NB, LDA, AFT, GLR, Isotonic, LinReg

[ https://issues.apache.org/jira/browse/SPARK-18206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627497#comment-15627497 ] Apache Spark commented on SPARK-18206: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-18206) Log instrumentation in MPC, NB, LDA, AFT, GLR, Isotonic, LinReg

[ https://issues.apache.org/jira/browse/SPARK-18206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18206: Assignee: zhengruifeng (was: Apache Spark) > Log instrumentation in MPC, NB, LDA, AFT,

[jira] [Assigned] (SPARK-18206) Log instrumentation in MPC, NB, LDA, AFT, GLR, Isotonic, LinReg

[ https://issues.apache.org/jira/browse/SPARK-18206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18206: Assignee: Apache Spark (was: zhengruifeng) > Log instrumentation in MPC, NB, LDA, AFT,

[jira] [Commented] (SPARK-18107) Insert overwrite statement runs much slower in spark-sql than it does in hive-client

[ https://issues.apache.org/jira/browse/SPARK-18107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627426#comment-15627426 ] Apache Spark commented on SPARK-18107: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-17879) Don't compact metadata logs constantly into a single compacted file

[ https://issues.apache.org/jira/browse/SPARK-17879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627421#comment-15627421 ] Burak Yavuz commented on SPARK-17879: - We should be doing the second. What you said makes sense, we

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627407#comment-15627407 ] Dongjoon Hyun commented on SPARK-18209: --- BTW, [~rxin]. I didn't notice that the following was the

[jira] [Commented] (SPARK-17982) Spark 2.0.0 CREATE VIEW statement fails :: java.lang.RuntimeException: Failed to analyze the canonicalized SQL. It is possible there is a bug in Spark.

[ https://issues.apache.org/jira/browse/SPARK-17982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627406#comment-15627406 ] Franck Tago commented on SPARK-17982: - Wanted to mention that I was able to successfully verify my

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627367#comment-15627367 ] Jiang Xingbo commented on SPARK-18209: -- I'm working on this, thanks! > More robust view

[jira] [Commented] (SPARK-18167) Flaky test when hive partition pruning is enabled

[ https://issues.apache.org/jira/browse/SPARK-18167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627343#comment-15627343 ] Apache Spark commented on SPARK-18167: -- User 'ericl' has created a pull request for this issue:

[jira] [Updated] (SPARK-17937) Clarify Kafka offset semantics for Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-17937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-17937: - Component/s: Structured Streaming > Clarify Kafka offset semantics for Structured

[jira] [Updated] (SPARK-18057) Update structured streaming kafka from 10.0.1 to 10.1.0

[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-18057: - Component/s: Structured Streaming > Update structured streaming kafka from 10.0.1 to

[jira] [Updated] (SPARK-17343) Prerequisites for Kafka 0.8 support in Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-17343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-17343: - Component/s: (was: DStreams) Structured Streaming > Prerequisites

[jira] [Updated] (SPARK-17837) Disaster recovery of offsets from WAL

[ https://issues.apache.org/jira/browse/SPARK-17837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-17837: - Component/s: Structured Streaming > Disaster recovery of offsets from WAL >

[jira] [Updated] (SPARK-17834) Fetch the earliest offsets manually in KafkaSource instead of counting on KafkaConsumer

[ https://issues.apache.org/jira/browse/SPARK-17834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-17834: - Component/s: (was: SQL) Structured Streaming > Fetch the earliest

[jira] [Updated] (SPARK-17346) Kafka 0.10 support in Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-17346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-17346: - Component/s: (was: DStreams) Structured Streaming > Kafka 0.10

[jira] [Updated] (SPARK-17345) Prerequisites for Kafka 0.10 support in Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-17345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-17345: - Component/s: (was: DStreams) Structured Streaming > Prerequisites

[jira] [Closed] (SPARK-18201) add toDense and toSparse into Matrix trait, like Vector design

[ https://issues.apache.org/jira/browse/SPARK-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu closed SPARK-18201. -- Resolution: Duplicate It will fix in this PR https://github.com/apache/spark/pull/15628 > add toDense

[jira] [Updated] (SPARK-17815) Report committed offsets

[ https://issues.apache.org/jira/browse/SPARK-17815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-17815: - Component/s: (was: SQL) Structured Streaming > Report committed

[jira] [Updated] (SPARK-17812) More granular control of starting offsets (assign)

[ https://issues.apache.org/jira/browse/SPARK-17812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-17812: - Component/s: (was: SQL) Structured Streaming > More granular

[jira] [Updated] (SPARK-17813) Maximum data per trigger

[ https://issues.apache.org/jira/browse/SPARK-17813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-17813: - Component/s: (was: SQL) Structured Streaming > Maximum data per

[jira] [Updated] (SPARK-17344) Kafka 0.8 support for Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-17344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-17344: - Component/s: (was: DStreams) Structured Streaming > Kafka 0.8

[jira] [Closed] (SPARK-17345) Prerequisites for Kafka 0.10 support in Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-17345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust closed SPARK-17345. Resolution: Fixed > Prerequisites for Kafka 0.10 support in Structured Streaming >

[jira] [Updated] (SPARK-15406) Structured streaming support for consuming from Kafka

[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-15406: - Component/s: Structured Streaming > Structured streaming support for consuming from

[jira] [Updated] (SPARK-17183) put hive serde table schema to table properties like data source table

[ https://issues.apache.org/jira/browse/SPARK-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-17183: - Priority: Blocker (was: Major) > put hive serde table schema to table properties like data source table

[jira] [Resolved] (SPARK-18025) Port streaming to use the commit protocol API

[ https://issues.apache.org/jira/browse/SPARK-18025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-18025. -- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 15710

[jira] [Updated] (SPARK-18192) Support all file formats in structured streaming

[ https://issues.apache.org/jira/browse/SPARK-18192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18192: Component/s: Structured Streaming > Support all file formats in structured streaming >

[jira] [Updated] (SPARK-18025) Port streaming to use the commit protocol API

[ https://issues.apache.org/jira/browse/SPARK-18025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-18025: - Component/s: Structured Streaming > Port streaming to use the commit protocol API >

[jira] [Closed] (SPARK-18215) Make Column.expr public

[ https://issues.apache.org/jira/browse/SPARK-18215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-18215. --- Resolution: Duplicate Target Version/s: (was: 2.1.0) > Make Column.expr public >

[jira] [Assigned] (SPARK-18216) Make Column.expr public

[ https://issues.apache.org/jira/browse/SPARK-18216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reassigned SPARK-18216: --- Assignee: Reynold Xin > Make Column.expr public > --- > >

[jira] [Assigned] (SPARK-18215) Make Column.expr public

[ https://issues.apache.org/jira/browse/SPARK-18215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reassigned SPARK-18215: --- Assignee: Reynold Xin > Make Column.expr public > --- > >

[jira] [Assigned] (SPARK-18216) Make Column.expr public

[ https://issues.apache.org/jira/browse/SPARK-18216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18216: Assignee: (was: Apache Spark) > Make Column.expr public > --- > >

[jira] [Assigned] (SPARK-18216) Make Column.expr public

[ https://issues.apache.org/jira/browse/SPARK-18216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18216: Assignee: Apache Spark > Make Column.expr public > --- > >

[jira] [Commented] (SPARK-18216) Make Column.expr public

[ https://issues.apache.org/jira/browse/SPARK-18216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627220#comment-15627220 ] Apache Spark commented on SPARK-18216: -- User 'rxin' has created a pull request for this issue:

[jira] [Created] (SPARK-18215) Make Column.expr public

Reynold Xin created SPARK-18215: --- Summary: Make Column.expr public Key: SPARK-18215 URL: https://issues.apache.org/jira/browse/SPARK-18215 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-18216) Make Column.expr public

Reynold Xin created SPARK-18216: --- Summary: Make Column.expr public Key: SPARK-18216 URL: https://issues.apache.org/jira/browse/SPARK-18216 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-18214) Simplify RuntimeReplaceable type coercion

[ https://issues.apache.org/jira/browse/SPARK-18214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627202#comment-15627202 ] Apache Spark commented on SPARK-18214: -- User 'rxin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-18214) Simplify RuntimeReplaceable type coercion

[ https://issues.apache.org/jira/browse/SPARK-18214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18214: Assignee: Reynold Xin (was: Apache Spark) > Simplify RuntimeReplaceable type coercion >

[jira] [Assigned] (SPARK-18214) Simplify RuntimeReplaceable type coercion

[ https://issues.apache.org/jira/browse/SPARK-18214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18214: Assignee: Apache Spark (was: Reynold Xin) > Simplify RuntimeReplaceable type coercion >

[jira] [Created] (SPARK-18214) Simplify RuntimeReplaceable type coercion

Reynold Xin created SPARK-18214: --- Summary: Simplify RuntimeReplaceable type coercion Key: SPARK-18214 URL: https://issues.apache.org/jira/browse/SPARK-18214 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-16845) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB

[ https://issues.apache.org/jira/browse/SPARK-16845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15626664#comment-15626664 ] Don Drake edited comment on SPARK-16845 at 11/2/16 12:32 AM: - I've been

[jira] [Updated] (SPARK-15581) MLlib 2.1 Roadmap

[ https://issues.apache.org/jira/browse/SPARK-15581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-15581: -- Fix Version/s: 2.1.0 > MLlib 2.1 Roadmap > - > > Key:

[jira] [Closed] (SPARK-15581) MLlib 2.1 Roadmap

[ https://issues.apache.org/jira/browse/SPARK-15581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley closed SPARK-15581. - Resolution: Done > MLlib 2.1 Roadmap > - > > Key:

[jira] [Commented] (SPARK-15581) MLlib 2.1 Roadmap

[ https://issues.apache.org/jira/browse/SPARK-15581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627180#comment-15627180 ] Joseph K. Bradley commented on SPARK-15581: --- Well, the 2.1 code freeze has come up quickly,

[jira] [Updated] (SPARK-16578) Configurable hostname for RBackend

[ https://issues.apache.org/jira/browse/SPARK-16578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-16578: -- Target Version/s: 2.2.0 (was: 2.1.0) > Configurable hostname for RBackend >

[jira] [Resolved] (SPARK-16411) Add textFile API to structured streaming.

[ https://issues.apache.org/jira/browse/SPARK-16411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-16411. -- Resolution: Fixed Assignee: Prashant Sharma Fix Version/s: 2.1.0 > Add

[jira] [Resolved] (SPARK-15944) Make spark.ml package backward compatible with spark.mllib vectors

[ https://issues.apache.org/jira/browse/SPARK-15944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-15944. --- Resolution: Fixed Fix Version/s: 2.1.0 > Make spark.ml package backward

[jira] [Resolved] (SPARK-16000) Make model loading backward compatible with saved models using old vector columns

[ https://issues.apache.org/jira/browse/SPARK-16000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-16000. --- Resolution: Fixed Fix Version/s: 2.1.0 > Make model loading backward

[jira] [Commented] (SPARK-16000) Make model loading backward compatible with saved models using old vector columns

[ https://issues.apache.org/jira/browse/SPARK-16000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627151#comment-15627151 ] Joseph K. Bradley commented on SPARK-16000: --- I just checked through, and the PRs cover all

[jira] [Commented] (SPARK-16738) Queryable state for Spark State Store

[ https://issues.apache.org/jira/browse/SPARK-16738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627148#comment-15627148 ] Michael Armbrust commented on SPARK-16738: -- You can already query the state store today, if you

[jira] [Updated] (SPARK-18187) CompactibleFileStreamLog should not rely on "compactInterval" to detect a compaction batch

[ https://issues.apache.org/jira/browse/SPARK-18187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-18187: - Priority: Critical (was: Major) > CompactibleFileStreamLog should not rely on

[jira] [Commented] (SPARK-18187) CompactibleFileStreamLog should not rely on "compactInterval" to detect a compaction batch

[ https://issues.apache.org/jira/browse/SPARK-18187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627134#comment-15627134 ] Michael Armbrust commented on SPARK-18187: -- I think the configuration should only be used when

[jira] [Updated] (SPARK-16240) model loading backward compatibility for ml.clustering.LDA

[ https://issues.apache.org/jira/browse/SPARK-16240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-16240: -- Issue Type: Improvement (was: Bug) > model loading backward compatibility for

[jira] [Commented] (SPARK-16454) Consider adding a per-batch transform for structured streaming

[ https://issues.apache.org/jira/browse/SPARK-16454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627125#comment-15627125 ] Michael Armbrust commented on SPARK-16454: -- What specifically is missing from the {{foreach}}

[jira] [Commented] (SPARK-15867) TABLESAMPLE BUCKET semantics don't match Hive's

[ https://issues.apache.org/jira/browse/SPARK-15867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627120#comment-15627120 ] Tejas Patil commented on SPARK-15867: - Yes. I am interested in this support. > TABLESAMPLE BUCKET

  1   2   3   4   >