[jira] [Commented] (SPARK-12677) Lazy file discovery for parquet

[ https://issues.apache.org/jira/browse/SPARK-12677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15628995#comment-15628995 ] Tiago Albineli Motta commented on SPARK-12677: -- It doenst launch another job

[jira] [Commented] (SPARK-18227) Parquet file stream sink create a hidden directory "_spark_metadata" cause the DataFrame read failed

[ https://issues.apache.org/jira/browse/SPARK-18227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15628997#comment-15628997 ] Lantao Jin commented on SPARK-18227: hadoop fs -ls hdfs:///path/out Found 3 items -rw

[jira] [Updated] (SPARK-18189) task not serializable with groupByKey() + mapGroups() + map

[ https://issues.apache.org/jira/browse/SPARK-18189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18189: -- Assignee: Ergin Seyfe > task not serializable with groupByKey() + mapGroups() + map > -

[jira] [Comment Edited] (SPARK-18227) Parquet file stream sink create a hidden directory "_spark_metadata" cause the DataFrame read failed

[ https://issues.apache.org/jira/browse/SPARK-18227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15628997#comment-15628997 ] Lantao Jin edited comment on SPARK-18227 at 11/2/16 1:39 PM: -

[jira] [Updated] (SPARK-17764) to_json function for parsing Structs to json Strings

[ https://issues.apache.org/jira/browse/SPARK-17764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17764: -- Assignee: Hyukjin Kwon > to_json function for parsing Structs to json Strings > ---

[jira] [Updated] (SPARK-18025) Port streaming to use the commit protocol API

[ https://issues.apache.org/jira/browse/SPARK-18025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18025: -- Assignee: Reynold Xin > Port streaming to use the commit protocol API > ---

[jira] [Commented] (SPARK-18227) Parquet file stream sink create a hidden directory "_spark_metadata" cause the DataFrame read failed

[ https://issues.apache.org/jira/browse/SPARK-18227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629047#comment-15629047 ] Lantao Jin commented on SPARK-18227: I also have an idea that add a "spark.parquet.me

[jira] [Updated] (SPARK-18227) Parquet file stream sink create a hidden directory "_spark_metadata" cause the DataFrame read from directory failed

[ https://issues.apache.org/jira/browse/SPARK-18227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lantao Jin updated SPARK-18227: --- Summary: Parquet file stream sink create a hidden directory "_spark_metadata" cause the DataFrame rea

[jira] [Commented] (SPARK-17938) Backpressure rate not adjusting

[ https://issues.apache.org/jira/browse/SPARK-17938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629049#comment-15629049 ] Fabien LD commented on SPARK-17938: --- For us, backpressure works fine with: - spark 2.0.

[jira] [Commented] (SPARK-17775) pyspark: take(num) failed, but collect() worked for big dataset

[ https://issues.apache.org/jira/browse/SPARK-17775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629076#comment-15629076 ] Oleh Koval commented on SPARK-17775: Seems to be the same issue as [SPARK-12261] > p

[jira] [Commented] (SPARK-12677) Lazy file discovery for parquet

[ https://issues.apache.org/jira/browse/SPARK-12677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629114#comment-15629114 ] Hyukjin Kwon commented on SPARK-12677: -- Ah, I see. IMHO, it might not be an issue as

[jira] [Comment Edited] (SPARK-12677) Lazy file discovery for parquet

[ https://issues.apache.org/jira/browse/SPARK-12677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629114#comment-15629114 ] Hyukjin Kwon edited comment on SPARK-12677 at 11/2/16 2:23 PM:

[jira] [Commented] (SPARK-18207) class "org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection" grows beyond 64 KB

[ https://issues.apache.org/jira/browse/SPARK-18207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629140#comment-15629140 ] Don Drake commented on SPARK-18207: --- I opened it based on [~lwlin]'s suggestion in the

[jira] [Commented] (SPARK-18211) Spark SQL ignores split.size

[ https://issues.apache.org/jira/browse/SPARK-18211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629156#comment-15629156 ] lostinoverflow commented on SPARK-18211: Thank you for the prompt response. Do yo

[jira] [Commented] (SPARK-18207) class "org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection" grows beyond 64 KB

[ https://issues.apache.org/jira/browse/SPARK-18207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629154#comment-15629154 ] Sean Owen commented on SPARK-18207: --- Can you note the difference here? if he's just say

[jira] [Commented] (SPARK-12261) pyspark crash for large dataset

[ https://issues.apache.org/jira/browse/SPARK-12261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629165#comment-15629165 ] Oleh Koval commented on SPARK-12261: Hey guys, Having the same issue with Spark 1.6.1

[jira] [Commented] (SPARK-12261) pyspark crash for large dataset

[ https://issues.apache.org/jira/browse/SPARK-12261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629182#comment-15629182 ] Shea Parkes commented on SPARK-12261: - I'm still maintaining the two-line bandaid to

[jira] [Commented] (SPARK-18211) Spark SQL ignores split.size

[ https://issues.apache.org/jira/browse/SPARK-18211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629171#comment-15629171 ] Sean Owen commented on SPARK-18211: --- Actually, I take that back. I wonder if this is th

[jira] [Commented] (SPARK-18207) class "org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection" grows beyond 64 KB

[ https://issues.apache.org/jira/browse/SPARK-18207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629177#comment-15629177 ] Don Drake commented on SPARK-18207: --- The difference with my case versus the other test

[jira] [Assigned] (SPARK-18212) Flaky test: org.apache.spark.sql.kafka010.KafkaSourceSuite.assign from specific offsets

[ https://issues.apache.org/jira/browse/SPARK-18212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18212: Assignee: Apache Spark > Flaky test: org.apache.spark.sql.kafka010.KafkaSourceSuite.assign

[jira] [Assigned] (SPARK-18212) Flaky test: org.apache.spark.sql.kafka010.KafkaSourceSuite.assign from specific offsets

[ https://issues.apache.org/jira/browse/SPARK-18212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18212: Assignee: (was: Apache Spark) > Flaky test: org.apache.spark.sql.kafka010.KafkaSourceS

[jira] [Commented] (SPARK-18212) Flaky test: org.apache.spark.sql.kafka010.KafkaSourceSuite.assign from specific offsets

[ https://issues.apache.org/jira/browse/SPARK-18212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629256#comment-15629256 ] Apache Spark commented on SPARK-18212: -- User 'koeninger' has created a pull request

[jira] [Comment Edited] (SPARK-12261) pyspark crash for large dataset

[ https://issues.apache.org/jira/browse/SPARK-12261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629165#comment-15629165 ] Oleh Koval edited comment on SPARK-12261 at 11/2/16 3:15 PM: -

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629273#comment-15629273 ] Nattavut Sutyanyong commented on SPARK-18209: - The challenge in Spark is the

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629282#comment-15629282 ] Nattavut Sutyanyong commented on SPARK-18209: - Sorry if this is a naive quest

[jira] [Updated] (SPARK-17822) JVMObjectTracker.objMap may leak JVM objects

[ https://issues.apache.org/jira/browse/SPARK-17822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-17822: - Attachment: screenshot-1.png > JVMObjectTracker.objMap may leak JVM objects > ---

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629296#comment-15629296 ] Herman van Hovell commented on SPARK-18209: --- It might be even easier to store t

[jira] [Updated] (SPARK-17822) JVMObjectTracker.objMap may leak JVM objects

[ https://issues.apache.org/jira/browse/SPARK-17822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-17822: - Description: JVMObjectTracker.objMap is used to track JVM objects for SparkR. However, we observed that

[jira] [Commented] (SPARK-17822) JVMObjectTracker.objMap may leak JVM objects

[ https://issues.apache.org/jira/browse/SPARK-17822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629313#comment-15629313 ] Yin Huai commented on SPARK-17822: -- Basically, the problem that I have observed is a lon

[jira] [Created] (SPARK-18228) Enhance visibility of Spark wiki

Michael Allman created SPARK-18228: -- Summary: Enhance visibility of Spark wiki Key: SPARK-18228 URL: https://issues.apache.org/jira/browse/SPARK-18228 Project: Spark Issue Type: Documentatio

[jira] [Commented] (SPARK-13417) SQL subquery support

[ https://issues.apache.org/jira/browse/SPARK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629328#comment-15629328 ] Nattavut Sutyanyong commented on SPARK-13417: - I'd like to share that there i

[jira] [Commented] (SPARK-17938) Backpressure rate not adjusting

[ https://issues.apache.org/jira/browse/SPARK-17938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629332#comment-15629332 ] Cody Koeninger commented on SPARK-17938: Direct stream isn't a receiver, receiver

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629339#comment-15629339 ] Nattavut Sutyanyong commented on SPARK-18209: - Agreed. > More robust view ca

[jira] [Updated] (SPARK-13127) Upgrade Parquet to 1.9 (Fixes parquet sorting)

[ https://issues.apache.org/jira/browse/SPARK-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Allman updated SPARK-13127: --- Priority: Major (was: Minor) > Upgrade Parquet to 1.9 (Fixes parquet sorting) >

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629368#comment-15629368 ] Nattavut Sutyanyong commented on SPARK-18209: - I could think of another alter

[jira] [Commented] (SPARK-17348) Incorrect results from subquery transformation

[ https://issues.apache.org/jira/browse/SPARK-17348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629385#comment-15629385 ] Nattavut Sutyanyong commented on SPARK-17348: - I propose to use this JIRA to

[jira] [Commented] (SPARK-18228) Enhance visibility of Spark wiki

[ https://issues.apache.org/jira/browse/SPARK-18228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629357#comment-15629357 ] Sean Owen commented on SPARK-18228: --- Sounds good. I'm thinking Documentation? it alread

[jira] [Commented] (SPARK-18211) Spark SQL ignores split.size

[ https://issues.apache.org/jira/browse/SPARK-18211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629446#comment-15629446 ] lostinoverflow commented on SPARK-18211: It could be although I am not sure. I tr

[jira] [Created] (SPARK-18229) Row.toSeq causes java.io.NotSerializableException

Daniel Haviv created SPARK-18229: Summary: Row.toSeq causes java.io.NotSerializableException Key: SPARK-18229 URL: https://issues.apache.org/jira/browse/SPARK-18229 Project: Spark Issue Type:

[jira] [Commented] (SPARK-18229) Row.toSeq causes java.io.NotSerializableException

[ https://issues.apache.org/jira/browse/SPARK-18229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629519#comment-15629519 ] Sean Owen commented on SPARK-18229: --- String.toSeq gives you a Scala WrappedString, whic

[jira] [Created] (SPARK-18230) MatrixFactorizationModel.recommendProducts throws NoSuchElement exception when the user does not exist

Mikael Ståldal created SPARK-18230: -- Summary: MatrixFactorizationModel.recommendProducts throws NoSuchElement exception when the user does not exist Key: SPARK-18230 URL: https://issues.apache.org/jira/browse/SPA

[jira] [Commented] (SPARK-18207) class "org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection" grows beyond 64 KB

[ https://issues.apache.org/jira/browse/SPARK-18207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629522#comment-15629522 ] Kazuaki Ishizaki commented on SPARK-18207: -- I created a smaller program to repro

[jira] [Updated] (SPARK-18230) MatrixFactorizationModel.recommendProducts throws NoSuchElement exception when the user does not exist

[ https://issues.apache.org/jira/browse/SPARK-18230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18230: -- Priority: Minor (was: Major) Issue Type: Improvement (was: Bug) Agree, seems like a legitimate

[jira] [Closed] (SPARK-18229) Row.toSeq causes java.io.NotSerializableException

[ https://issues.apache.org/jira/browse/SPARK-18229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Haviv closed SPARK-18229. Resolution: Not A Problem > Row.toSeq causes java.io.NotSerializableException > ---

[jira] [Commented] (SPARK-14008) Cleanup/Extend the Vectorized Parquet Reader

[ https://issues.apache.org/jira/browse/SPARK-14008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629572#comment-15629572 ] Sameer Agarwal commented on SPARK-14008: Thanks [~hyukjin.kwon], marked this as d

[jira] [Created] (SPARK-18231) Optimise SizeEstimator implementation

Adam Roberts created SPARK-18231: Summary: Optimise SizeEstimator implementation Key: SPARK-18231 URL: https://issues.apache.org/jira/browse/SPARK-18231 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-13127) Upgrade Parquet to 1.9 (Fixes parquet sorting)

[ https://issues.apache.org/jira/browse/SPARK-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Allman updated SPARK-13127: --- Affects Version/s: (was: 1.6.0) 2.0.0 2.0.1

[jira] [Resolved] (SPARK-14008) Cleanup/Extend the Vectorized Parquet Reader

[ https://issues.apache.org/jira/browse/SPARK-14008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sameer Agarwal resolved SPARK-14008. Resolution: Done > Cleanup/Extend the Vectorized Parquet Reader > -

[jira] [Resolved] (SPARK-17683) Support ArrayType in Literal.apply

[ https://issues.apache.org/jira/browse/SPARK-17683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17683. - Resolution: Fixed Fix Version/s: 2.1.0 This was merged -- the patch accepts Array[_] (not

[jira] [Resolved] (SPARK-17895) Improve documentation of "rowsBetween" and "rangeBetween"

[ https://issues.apache.org/jira/browse/SPARK-17895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17895. - Resolution: Fixed Assignee: Weiluo Ren Fix Version/s: 2.1.0 > Improve documentati

[jira] [Updated] (SPARK-14393) values generated by non-deterministic functions shouldn't change after coalesce or union

[ https://issues.apache.org/jira/browse/SPARK-14393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-14393: Labels: correctness releasenotes (was: correctness) > values generated by non-deterministic functi

[jira] [Updated] (SPARK-14393) values generated by non-deterministic functions shouldn't change after coalesce or union

[ https://issues.apache.org/jira/browse/SPARK-14393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-14393: Summary: values generated by non-deterministic functions shouldn't change after coalesce or union

[jira] [Resolved] (SPARK-14393) values generated by non-deterministic functions shouldn't change after coalesce or union

[ https://issues.apache.org/jira/browse/SPARK-14393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-14393. - Resolution: Fixed Fix Version/s: 2.1.0 > values generated by non-deterministic functions s

[jira] [Resolved] (SPARK-13417) SQL subquery support

[ https://issues.apache.org/jira/browse/SPARK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-13417. - Resolution: Fixed Fix Version/s: 2.1.0 > SQL subquery support > > >

[jira] [Commented] (SPARK-13417) SQL subquery support

[ https://issues.apache.org/jira/browse/SPARK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15630019#comment-15630019 ] Reynold Xin commented on SPARK-13417: - Yup good to track bugs elsewhere. > SQL subq

[jira] [Updated] (SPARK-18217) Disallow creating permanent views based on temporary views or UDFs

[ https://issues.apache.org/jira/browse/SPARK-18217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18217: Description: See the discussion in the parent ticket SPARK-18209. It doesn't really make sense to c

[jira] [Commented] (SPARK-14241) Output of monotonically_increasing_id lacks stable relation with rows of DataFrame

[ https://issues.apache.org/jira/browse/SPARK-14241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15630024#comment-15630024 ] Xiangrui Meng commented on SPARK-14241: --- This bug should be fixed in 2.0 already si

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15630023#comment-15630023 ] Reynold Xin commented on SPARK-18209: - Yea it's definitely much harder to make logica

[jira] [Updated] (SPARK-18217) Disallow creating permanent views based on temporary views or UDFs

[ https://issues.apache.org/jira/browse/SPARK-18217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18217: Summary: Disallow creating permanent views based on temporary views or UDFs (was: Disallow creatin

[jira] [Updated] (SPARK-18111) Wrong ApproximatePercentile answer when multiple records have the minimum value

[ https://issues.apache.org/jira/browse/SPARK-18111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18111: Fix Version/s: 2.0.3 > Wrong ApproximatePercentile answer when multiple records have the minimum >

[jira] [Assigned] (SPARK-18086) Regression: Hive variables no longer work in Spark 2.0

[ https://issues.apache.org/jira/browse/SPARK-18086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18086: Assignee: Apache Spark > Regression: Hive variables no longer work in Spark 2.0 >

[jira] [Assigned] (SPARK-18086) Regression: Hive variables no longer work in Spark 2.0

[ https://issues.apache.org/jira/browse/SPARK-18086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18086: Assignee: (was: Apache Spark) > Regression: Hive variables no longer work in Spark 2.0

[jira] [Commented] (SPARK-18086) Regression: Hive variables no longer work in Spark 2.0

[ https://issues.apache.org/jira/browse/SPARK-18086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15630027#comment-15630027 ] Apache Spark commented on SPARK-18086: -- User 'rdblue' has created a pull request for

[jira] [Resolved] (SPARK-18160) spark.files & spark.jars should not be passed to driver in yarn mode

[ https://issues.apache.org/jira/browse/SPARK-18160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-18160. Resolution: Fixed Assignee: Jeff Zhang Fix Version/s: 2.1.0

[jira] [Commented] (SPARK-18086) Regression: Hive variables no longer work in Spark 2.0

[ https://issues.apache.org/jira/browse/SPARK-18086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15630045#comment-15630045 ] Ryan Blue commented on SPARK-18086: --- [~rxin], I think the fix for this should go into 2

[jira] [Updated] (SPARK-17058) Add maven snapshots-and-staging profile to build/test against staging artifacts

[ https://issues.apache.org/jira/browse/SPARK-17058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17058: Fix Version/s: 2.1.0 > Add maven snapshots-and-staging profile to build/test against staging > art

[jira] [Commented] (SPARK-18086) Regression: Hive variables no longer work in Spark 2.0

[ https://issues.apache.org/jira/browse/SPARK-18086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15630057#comment-15630057 ] Reynold Xin commented on SPARK-18086: - [~rdblue] There are two separate issues here I

[jira] [Resolved] (SPARK-17058) Add maven snapshots-and-staging profile to build/test against staging artifacts

[ https://issues.apache.org/jira/browse/SPARK-17058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17058. - Resolution: Fixed Assignee: Steve Loughran Fix Version/s: 2.2.0 > Add maven snaps

[jira] [Updated] (SPARK-14241) Output of monotonically_increasing_id lacks stable relation with rows of DataFrame

[ https://issues.apache.org/jira/browse/SPARK-14241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-14241: -- Fix Version/s: 2.0.0 > Output of monotonically_increasing_id lacks stable relation with rows of

[jira] [Comment Edited] (SPARK-14241) Output of monotonically_increasing_id lacks stable relation with rows of DataFrame

[ https://issues.apache.org/jira/browse/SPARK-14241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15630024#comment-15630024 ] Xiangrui Meng edited comment on SPARK-14241 at 11/2/16 7:05 PM: ---

[jira] [Resolved] (SPARK-14241) Output of monotonically_increasing_id lacks stable relation with rows of DataFrame

[ https://issues.apache.org/jira/browse/SPARK-14241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-14241. --- Resolution: Fixed > Output of monotonically_increasing_id lacks stable relation with rows of

[jira] [Commented] (SPARK-18086) Regression: Hive variables no longer work in Spark 2.0

[ https://issues.apache.org/jira/browse/SPARK-18086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15630132#comment-15630132 ] Ryan Blue commented on SPARK-18086: --- Hive variables are set on the Hive SessionState an

[jira] [Commented] (SPARK-16808) History Server main page does not honor APPLICATION_WEB_PROXY_BASE

[ https://issues.apache.org/jira/browse/SPARK-16808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15630168#comment-15630168 ] Apache Spark commented on SPARK-16808: -- User 'jantes' has created a pull request for

[jira] [Assigned] (SPARK-16808) History Server main page does not honor APPLICATION_WEB_PROXY_BASE

[ https://issues.apache.org/jira/browse/SPARK-16808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16808: Assignee: (was: Apache Spark) > History Server main page does not honor APPLICATION_WE

[jira] [Assigned] (SPARK-16808) History Server main page does not honor APPLICATION_WEB_PROXY_BASE

[ https://issues.apache.org/jira/browse/SPARK-16808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16808: Assignee: Apache Spark > History Server main page does not honor APPLICATION_WEB_PROXY_BAS

[jira] [Commented] (SPARK-11879) Checkpoint support for DataFrame/Dataset

[ https://issues.apache.org/jira/browse/SPARK-11879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15630537#comment-15630537 ] Cheng Lian commented on SPARK-11879: Sorry that I didn't notice this ticket while wor

[jira] [Updated] (SPARK-11879) Checkpoint support for DataFrame/Dataset

[ https://issues.apache.org/jira/browse/SPARK-11879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-11879: Assignee: Cheng Lian > Checkpoint support for DataFrame/Dataset > -

[jira] [Resolved] (SPARK-11879) Checkpoint support for DataFrame/Dataset

[ https://issues.apache.org/jira/browse/SPARK-11879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-11879. Resolution: Duplicate > Checkpoint support for DataFrame/Dataset >

[jira] [Updated] (SPARK-11879) Checkpoint support for DataFrame/Dataset

[ https://issues.apache.org/jira/browse/SPARK-11879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-11879: Fix Version/s: 2.1.0 > Checkpoint support for DataFrame/Dataset > -

[jira] [Updated] (SPARK-17972) Query planning slows down dramatically for large query plans even when sub-trees are cached

[ https://issues.apache.org/jira/browse/SPARK-17972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-17972: --- Description: The following Spark shell snippet creates a series of query plans that grow exponential

[jira] [Closed] (SPARK-4549) Support BigInt -> Decimal in convertToCatalyst in SparkSQL

[ https://issues.apache.org/jira/browse/SPARK-4549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-4549. - Resolution: Incomplete > Support BigInt -> Decimal in convertToCatalyst in SparkSQL >

[jira] [Commented] (SPARK-18212) Flaky test: org.apache.spark.sql.kafka010.KafkaSourceSuite.assign from specific offsets

[ https://issues.apache.org/jira/browse/SPARK-18212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15630567#comment-15630567 ] Shixiong Zhu commented on SPARK-18212: -- [~c...@koeninger.org] Sounds good to me. >

[jira] [Created] (SPARK-18232) Support Mesos CNI

Michael Gummelt created SPARK-18232: --- Summary: Support Mesos CNI Key: SPARK-18232 URL: https://issues.apache.org/jira/browse/SPARK-18232 Project: Spark Issue Type: Improvement Com

[jira] [Created] (SPARK-18233) Failed to deserialize the task

Davies Liu created SPARK-18233: -- Summary: Failed to deserialize the task Key: SPARK-18233 URL: https://issues.apache.org/jira/browse/SPARK-18233 Project: Spark Issue Type: Bug Report

[jira] [Commented] (SPARK-16726) Improve `Union/Intersect/Except` error messages on incompatible types

[ https://issues.apache.org/jira/browse/SPARK-16726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15630597#comment-15630597 ] Nicholas Chammas commented on SPARK-16726: -- I just hit this error in 2.0.1 and i

[jira] [Commented] (SPARK-18226) SparkR displaying vector columns in incorrect way

[ https://issues.apache.org/jira/browse/SPARK-18226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15630623#comment-15630623 ] Felix Cheung commented on SPARK-18226: -- Thanks, this is actually the issue outlined

[jira] [Commented] (SPARK-18131) Support returning Vector/Dense Vector from backend

[ https://issues.apache.org/jira/browse/SPARK-18131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15630625#comment-15630625 ] Felix Cheung commented on SPARK-18131: -- See https://issues.apache.org/jira/browse/SP

[jira] [Commented] (SPARK-16726) Improve `Union/Intersect/Except` error messages on incompatible types

[ https://issues.apache.org/jira/browse/SPARK-16726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15630657#comment-15630657 ] Dongjoon Hyun commented on SPARK-16726: --- You're welcome. Thank you, [~nchammas]! >

[jira] [Commented] (SPARK-18086) Regression: Hive variables no longer work in Spark 2.0

[ https://issues.apache.org/jira/browse/SPARK-18086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15630718#comment-15630718 ] Reynold Xin commented on SPARK-18086: - The thing is that we don't really propagate Hi

[jira] [Commented] (SPARK-18230) MatrixFactorizationModel.recommendProducts throws NoSuchElement exception when the user does not exist

[ https://issues.apache.org/jira/browse/SPARK-18230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15630741#comment-15630741 ] yuhao yang commented on SPARK-18230: Perhaps we can use Double.NaN for the case, just

[jira] [Created] (SPARK-18234) Update mode

Michael Armbrust created SPARK-18234: Summary: Update mode Key: SPARK-18234 URL: https://issues.apache.org/jira/browse/SPARK-18234 Project: Spark Issue Type: New Feature Compone

[jira] [Updated] (SPARK-18234) Update mode in structured streaming

[ https://issues.apache.org/jira/browse/SPARK-18234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18234: Summary: Update mode in structured streaming (was: Update mode) > Update mode in structured stream

[jira] [Assigned] (SPARK-18232) Support Mesos CNI

[ https://issues.apache.org/jira/browse/SPARK-18232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18232: Assignee: Apache Spark > Support Mesos CNI > - > > Key: SP

[jira] [Assigned] (SPARK-18232) Support Mesos CNI

[ https://issues.apache.org/jira/browse/SPARK-18232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18232: Assignee: (was: Apache Spark) > Support Mesos CNI > - > >

[jira] [Commented] (SPARK-18232) Support Mesos CNI

[ https://issues.apache.org/jira/browse/SPARK-18232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15630768#comment-15630768 ] Apache Spark commented on SPARK-18232: -- User 'mgummelt' has created a pull request f

[jira] [Commented] (SPARK-18200) GraphX Invalid initial capacity when running triangleCount

[ https://issues.apache.org/jira/browse/SPARK-18200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15630798#comment-15630798 ] Dongjoon Hyun commented on SPARK-18200: --- Hi, [~dennyglee]. It's due to `OpenHashSet

[jira] [Assigned] (SPARK-18200) GraphX Invalid initial capacity when running triangleCount

[ https://issues.apache.org/jira/browse/SPARK-18200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18200: Assignee: (was: Apache Spark) > GraphX Invalid initial capacity when running triangleC

[jira] [Commented] (SPARK-18200) GraphX Invalid initial capacity when running triangleCount

[ https://issues.apache.org/jira/browse/SPARK-18200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15630814#comment-15630814 ] Apache Spark commented on SPARK-18200: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Assigned] (SPARK-18200) GraphX Invalid initial capacity when running triangleCount

[ https://issues.apache.org/jira/browse/SPARK-18200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18200: Assignee: Apache Spark > GraphX Invalid initial capacity when running triangleCount >

[jira] [Commented] (SPARK-18200) GraphX Invalid initial capacity when running triangleCount

[ https://issues.apache.org/jira/browse/SPARK-18200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15630821#comment-15630821 ] Dongjoon Hyun commented on SPARK-18200: --- Actually, there is a node whose don't have

<    1   2   3   >