[jira] [Commented] (SPARK-6364) hashCode and equals for Matrices

[ https://issues.apache.org/jira/browse/SPARK-6364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366720#comment-14366720 ] Apache Spark commented on SPARK-6364: - User 'MechCoder' has created a pull request for

[jira] [Commented] (SPARK-6394) cleanup BlockManager companion object and improve the getCacheLocs method in DAGScheduler

[ https://issues.apache.org/jira/browse/SPARK-6394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366622#comment-14366622 ] Apache Spark commented on SPARK-6394: - User 'cloud-fan' has created a pull request for

[jira] [Created] (SPARK-6394) cleanup BlockManager companion object and improve the getCacheLocs method in DAGScheduler

Wenchen Fan created SPARK-6394: -- Summary: cleanup BlockManager companion object and improve the getCacheLocs method in DAGScheduler Key: SPARK-6394 URL: https://issues.apache.org/jira/browse/SPARK-6394 P

[jira] [Updated] (SPARK-6145) ORDER BY fails to resolve nested fields

[ https://issues.apache.org/jira/browse/SPARK-6145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6145: Target Version/s: 1.3.1 (was: 1.3.0) > ORDER BY fails to resolve nested fields > --

[jira] [Commented] (SPARK-6200) Support dialect in SQL

[ https://issues.apache.org/jira/browse/SPARK-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366575#comment-14366575 ] Michael Armbrust commented on SPARK-6200: - Thanks for taking the time to explain y

[jira] [Commented] (SPARK-6200) Support dialect in SQL

[ https://issues.apache.org/jira/browse/SPARK-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366569#comment-14366569 ] haiyang commented on SPARK-6200: Thank you for your comments. The core idea of this imple

[jira] [Commented] (SPARK-6356) Support the ROLLUP/CUBE/GROUPING SETS/grouping() in SQLContext

[ https://issues.apache.org/jira/browse/SPARK-6356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366551#comment-14366551 ] Apache Spark commented on SPARK-6356: - User 'watermen' has created a pull request for

[jira] [Resolved] (SPARK-6247) Certain self joins cannot be analyzed

[ https://issues.apache.org/jira/browse/SPARK-6247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-6247. - Resolution: Fixed Fix Version/s: 1.3.1 Issue resolved by pull request 5062 [https:/

[jira] [Resolved] (SPARK-5651) Support 'create db.table' in HiveContext

[ https://issues.apache.org/jira/browse/SPARK-5651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-5651. - Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 4427 [https:/

[jira] [Resolved] (SPARK-5404) Statistic of Logical Plan is too aggresive

[ https://issues.apache.org/jira/browse/SPARK-5404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-5404. - Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 4914 [https:/

[jira] [Commented] (SPARK-6392) [SQL]class not found exception thows when `add jar` use spark cli

[ https://issues.apache.org/jira/browse/SPARK-6392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366506#comment-14366506 ] Apache Spark commented on SPARK-6392: - User 'jeanlyn' has created a pull request for t

[jira] [Created] (SPARK-6393) Extra RPC to the AM during killExecutor invocation

Sandy Ryza created SPARK-6393: - Summary: Extra RPC to the AM during killExecutor invocation Key: SPARK-6393 URL: https://issues.apache.org/jira/browse/SPARK-6393 Project: Spark Issue Type: Improv

[jira] [Resolved] (SPARK-5908) Hive udtf with single alias should be resolved correctly

[ https://issues.apache.org/jira/browse/SPARK-5908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-5908. - Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 4692 [https:/

[jira] [Updated] (SPARK-6383) Few examples on Dataframe operation give compiler errors

[ https://issues.apache.org/jira/browse/SPARK-6383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-6383: --- Fix Version/s: (was: 1.3.0) 1.3.1 1.4.0 > Few examples on Da

[jira] [Updated] (SPARK-6383) Few examples on Dataframe operation give compiler errors

[ https://issues.apache.org/jira/browse/SPARK-6383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-6383: --- Labels: DataFrame (was: ) > Few examples on Dataframe operation give compiler errors > -

[jira] [Resolved] (SPARK-6383) Few examples on Dataframe operation give compiler errors

[ https://issues.apache.org/jira/browse/SPARK-6383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-6383. Resolution: Fixed Fix Version/s: 1.3.0 > Few examples on Dataframe operation give compiler er

[jira] [Created] (SPARK-6392) [SQL]class not found exception thows when `add jar` use spark cli

jeanlyn created SPARK-6392: -- Summary: [SQL]class not found exception thows when `add jar` use spark cli Key: SPARK-6392 URL: https://issues.apache.org/jira/browse/SPARK-6392 Project: Spark Issue T

[jira] [Updated] (SPARK-6366) In Python API, the default save mode for save and saveAsTable should be "error" instead of "append".

[ https://issues.apache.org/jira/browse/SPARK-6366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6366: -- Fix Version/s: (was: 1.3.0) 1.3.1 > In Python API, the default save mode for save

[jira] [Resolved] (SPARK-6366) In Python API, the default save mode for save and saveAsTable should be "error" instead of "append".

[ https://issues.apache.org/jira/browse/SPARK-6366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-6366. --- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 5053 [https://github.com/

[jira] [Closed] (SPARK-6204) GenerateProjection's equals should check length equality

[ https://issues.apache.org/jira/browse/SPARK-6204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust closed SPARK-6204. --- Resolution: Won't Fix > GenerateProjection's equals should check length equality > ---

[jira] [Commented] (SPARK-6330) newParquetRelation gets incorrect FileSystem

[ https://issues.apache.org/jira/browse/SPARK-6330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366394#comment-14366394 ] Apache Spark commented on SPARK-6330: - User 'ypcat' has created a pull request for thi

[jira] [Updated] (SPARK-6391) Update Tachyon version compatibility documentation

[ https://issues.apache.org/jira/browse/SPARK-6391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haoyuan Li updated SPARK-6391: -- Affects Version/s: 1.3.0 > Update Tachyon version compatibility documentation >

[jira] [Updated] (SPARK-6391) Update Tachyon version compatibility documentation

[ https://issues.apache.org/jira/browse/SPARK-6391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haoyuan Li updated SPARK-6391: -- Fix Version/s: 1.4.0 > Update Tachyon version compatibility documentation >

[jira] [Updated] (SPARK-6391) Update Tachyon version compatibility documentation

[ https://issues.apache.org/jira/browse/SPARK-6391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haoyuan Li updated SPARK-6391: -- Component/s: (was: Spark Core) Documentation > Update Tachyon version compatibility

[jira] [Created] (SPARK-6391) Update Tachyon version compatibility documentation

Calvin Jia created SPARK-6391: - Summary: Update Tachyon version compatibility documentation Key: SPARK-6391 URL: https://issues.apache.org/jira/browse/SPARK-6391 Project: Spark Issue Type: Docume

[jira] [Commented] (SPARK-6248) LocalRelation needs to implement statistics

[ https://issues.apache.org/jira/browse/SPARK-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366333#comment-14366333 ] Yin Huai commented on SPARK-6248: - https://github.com/apache/spark/pull/5062 will also fix

[jira] [Updated] (SPARK-6248) LocalRelation needs to implement statistics

[ https://issues.apache.org/jira/browse/SPARK-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-6248: Target Version/s: 1.3.1 (was: 1.4.0) > LocalRelation needs to implement statistics > --

[jira] [Updated] (SPARK-6248) LocalRelation needs to implement statistics

[ https://issues.apache.org/jira/browse/SPARK-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-6248: Description: We need to implement statistics for LocalRelation. Otherwise, we cannot join a LocalRelation w

[jira] [Commented] (SPARK-6154) Support Kafka, JDBC in Scala 2.11

[ https://issues.apache.org/jira/browse/SPARK-6154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366295#comment-14366295 ] Olivier Toupin commented on SPARK-6154: --- I made an unperfect fix for hive-thriftserv

[jira] [Commented] (SPARK-6250) Types are now reserved words in DDL parser.

[ https://issues.apache.org/jira/browse/SPARK-6250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366284#comment-14366284 ] Yin Huai commented on SPARK-6250: - [~nitay] Can you try https://github.com/apache/spark/pu

[jira] [Commented] (SPARK-6250) Types are now reserved words in DDL parser.

[ https://issues.apache.org/jira/browse/SPARK-6250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366269#comment-14366269 ] Apache Spark commented on SPARK-6250: - User 'yhuai' has created a pull request for thi

[jira] [Commented] (SPARK-5654) Integrate SparkR into Apache Spark

[ https://issues.apache.org/jira/browse/SPARK-5654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366237#comment-14366237 ] Apache Spark commented on SPARK-5654: - User 'davies' has created a pull request for th

[jira] [Issue Comment Deleted] (SPARK-6116) Making DataFrame API non-experimental

[ https://issues.apache.org/jira/browse/SPARK-6116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-6116: --- Comment: was deleted (was: User 'azagrebin' has created a pull request for this issue: https://github.

[jira] [Commented] (SPARK-5955) Add checkpointInterval to ALS

[ https://issues.apache.org/jira/browse/SPARK-5955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366127#comment-14366127 ] Apache Spark commented on SPARK-5955: - User 'mengxr' has created a pull request for th

[jira] [Updated] (SPARK-6370) RDD sampling with replacement intermittently yields incorrect number of samples

[ https://issues.apache.org/jira/browse/SPARK-6370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marko Bonaci updated SPARK-6370: Priority: Minor (was: Major) Issue Type: Documentation (was: Bug) > RDD sampling with replac

[jira] [Commented] (SPARK-5954) Add topByKey to pair RDDs

[ https://issues.apache.org/jira/browse/SPARK-5954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366081#comment-14366081 ] Apache Spark commented on SPARK-5954: - User 'coderxiang' has created a pull request fo

[jira] [Assigned] (SPARK-5955) Add checkpointInterval to ALS

[ https://issues.apache.org/jira/browse/SPARK-5955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-5955: Assignee: Xiangrui Meng > Add checkpointInterval to ALS > - > >

[jira] [Commented] (SPARK-6334) spark-local dir not getting cleared during ALS

[ https://issues.apache.org/jira/browse/SPARK-6334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366021#comment-14366021 ] Xiangrui Meng commented on SPARK-6334: -- Couple suggestions before SPARK-5955 is imple

[jira] [Commented] (SPARK-4416) Support Mesos framework authentication

[ https://issues.apache.org/jira/browse/SPARK-4416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366011#comment-14366011 ] Adam B commented on SPARK-4416: --- Can close as duplicate of SPARK-6284 > Support Mesos frame

[jira] [Commented] (SPARK-6192) Enhance MLlib's Python API (GSoC 2015)

[ https://issues.apache.org/jira/browse/SPARK-6192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365918#comment-14365918 ] Xiangrui Meng commented on SPARK-6192: -- [~MechCoder] Please be a little (but not too)

[jira] [Commented] (SPARK-5836) Highlight in Spark documentation that by default Spark does not delete its temporary files

[ https://issues.apache.org/jira/browse/SPARK-5836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365897#comment-14365897 ] Apache Spark commented on SPARK-5836: - User 'ilganeli' has created a pull request for

[jira] [Commented] (SPARK-5750) Document that ordering of elements in shuffled partitions is not deterministic across runs

[ https://issues.apache.org/jira/browse/SPARK-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365895#comment-14365895 ] Apache Spark commented on SPARK-5750: - User 'ilganeli' has created a pull request for

[jira] [Commented] (SPARK-3441) Explain in docs that repartitionAndSortWithinPartitions enacts Hadoop style shuffle

[ https://issues.apache.org/jira/browse/SPARK-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365896#comment-14365896 ] Apache Spark commented on SPARK-3441: - User 'ilganeli' has created a pull request for

[jira] [Created] (SPARK-6390) Add MatrixUDT in PySpark

Xiangrui Meng created SPARK-6390: Summary: Add MatrixUDT in PySpark Key: SPARK-6390 URL: https://issues.apache.org/jira/browse/SPARK-6390 Project: Spark Issue Type: New Feature Comp

[jira] [Updated] (SPARK-6390) Add MatrixUDT in PySpark

[ https://issues.apache.org/jira/browse/SPARK-6390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6390: - Component/s: PySpark > Add MatrixUDT in PySpark > > > Key

[jira] [Resolved] (SPARK-6226) Support model save/load in Python's KMeans

[ https://issues.apache.org/jira/browse/SPARK-6226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-6226. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5049 [https://githu

[jira] [Updated] (SPARK-6336) LBFGS should document what convergenceTol means

[ https://issues.apache.org/jira/browse/SPARK-6336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6336: - Target Version/s: 1.4.0, 1.3.1 (was: 1.4.0) > LBFGS should document what convergenceTol means > -

[jira] [Updated] (SPARK-6336) LBFGS should document what convergenceTol means

[ https://issues.apache.org/jira/browse/SPARK-6336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6336: - Assignee: Kai Sasaki > LBFGS should document what convergenceTol means > -

[jira] [Resolved] (SPARK-6336) LBFGS should document what convergenceTol means

[ https://issues.apache.org/jira/browse/SPARK-6336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-6336. -- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 5033 [https://githu

[jira] [Updated] (SPARK-6336) LBFGS should document what convergenceTol means

[ https://issues.apache.org/jira/browse/SPARK-6336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6336: - Fix Version/s: (was: 1.3.0) 1.3.1 1.4.0 > LBFGS should d

[jira] [Commented] (SPARK-6117) describe function for summary statistics

[ https://issues.apache.org/jira/browse/SPARK-6117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365781#comment-14365781 ] Apache Spark commented on SPARK-6117: - User 'azagrebin' has created a pull request for

[jira] [Closed] (SPARK-6358) Spark-submit error when using PYSPARK_PYTHON enviromnental variable

[ https://issues.apache.org/jira/browse/SPARK-6358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dustin davidson closed SPARK-6358. -- Resolution: Invalid Found a work around. The issue was with group permissions on the '/home' d

[jira] [Commented] (SPARK-6388) Spark 1.3 + Hadoop 2.6 Can't work on Java 8_40

[ https://issues.apache.org/jira/browse/SPARK-6388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365758#comment-14365758 ] Michael Malak commented on SPARK-6388: -- Isn't it Hadoop 2.7 that is supposed to provi

[jira] [Commented] (SPARK-6250) Types are now reserved words in DDL parser.

[ https://issues.apache.org/jira/browse/SPARK-6250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365735#comment-14365735 ] Yin Huai commented on SPARK-6250: - Update: Seems this problem affects struct fields when a

[jira] [Commented] (SPARK-6363) make scala 2.11 default language

[ https://issues.apache.org/jira/browse/SPARK-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365726#comment-14365726 ] Jianshi Huang commented on SPARK-6363: -- My two cents. The only module that's not wor

[jira] [Closed] (SPARK-3351) Yarn YarnRMClientImpl.shutdown can be called before register - NPE

[ https://issues.apache.org/jira/browse/SPARK-3351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves closed SPARK-3351. Resolution: Invalid > Yarn YarnRMClientImpl.shutdown can be called before register - NPE > -

[jira] [Commented] (SPARK-3351) Yarn YarnRMClientImpl.shutdown can be called before register - NPE

[ https://issues.apache.org/jira/browse/SPARK-3351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365607#comment-14365607 ] Thomas Graves commented on SPARK-3351: -- ok lets close this and reopen if we see it ag

[jira] [Commented] (SPARK-4590) Early investigation of parameter server

[ https://issues.apache.org/jira/browse/SPARK-4590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365581#comment-14365581 ] Reza Zadeh commented on SPARK-4590: --- Hi Qiping, We are waiting for IndexedRDD to be opti

[jira] [Commented] (SPARK-5313) Create simple framework for highlighting changes introduced in a PR

[ https://issues.apache.org/jira/browse/SPARK-5313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365561#comment-14365561 ] Apache Spark commented on SPARK-5313: - User 'brennonyork' has created a pull request f

[jira] [Resolved] (SPARK-6365) jetty-security needed for SPARK_PREPEND_CLASSES to work

[ https://issues.apache.org/jira/browse/SPARK-6365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid resolved SPARK-6365. - Resolution: Fixed Fix Version/s: 1.3.1 Issue resolved by pull request 5071 [https://github.

[jira] [Commented] (SPARK-6195) Specialized in-memory column type for decimal

[ https://issues.apache.org/jira/browse/SPARK-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365549#comment-14365549 ] Jianshi Huang commented on SPARK-6195: -- Like this optimization! :) Jianshi > Specia

[jira] [Commented] (SPARK-3351) Yarn YarnRMClientImpl.shutdown can be called before register - NPE

[ https://issues.apache.org/jira/browse/SPARK-3351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365547#comment-14365547 ] Steve Loughran commented on SPARK-3351: --- The NPE is certainly gone. I don't know abo

[jira] [Commented] (SPARK-6293) SQLContext.implicits should provide automatic conversion for RDD[Row]

[ https://issues.apache.org/jira/browse/SPARK-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365500#comment-14365500 ] Chen Song commented on SPARK-6293: -- Yeah, this is a problem. Maybe we can rebuild the sch

[jira] [Commented] (SPARK-6293) SQLContext.implicits should provide automatic conversion for RDD[Row]

[ https://issues.apache.org/jira/browse/SPARK-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365504#comment-14365504 ] Joseph K. Bradley commented on SPARK-6293: -- Hm, maybe I was wrong to say it belon

[jira] [Commented] (SPARK-6313) Fetch File Lock file creation doesnt work when Spark working dir is on a NFS mount

[ https://issues.apache.org/jira/browse/SPARK-6313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365452#comment-14365452 ] Josh Rosen commented on SPARK-6313: --- I've merged Nathan's patch into 1.4.0, 1.3.1, and 1

[jira] [Updated] (SPARK-6313) Fetch File Lock file creation doesnt work when Spark working dir is on a NFS mount

[ https://issues.apache.org/jira/browse/SPARK-6313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-6313: -- Assignee: Nathan McCarthy > Fetch File Lock file creation doesnt work when Spark working dir is on a NFS

[jira] [Resolved] (SPARK-6313) Fetch File Lock file creation doesnt work when Spark working dir is on a NFS mount

[ https://issues.apache.org/jira/browse/SPARK-6313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-6313. --- Resolution: Fixed Fix Version/s: 1.2.2 1.3.1 1.4.0 Issue

[jira] [Commented] (SPARK-3266) JavaDoubleRDD doesn't contain max()

[ https://issues.apache.org/jira/browse/SPARK-3266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365434#comment-14365434 ] Josh Rosen commented on SPARK-3266: --- After many months, I've finally fixed this issue th

[jira] [Commented] (SPARK-1834) NoSuchMethodError when invoking JavaPairRDD.reduce() in Java

[ https://issues.apache.org/jira/browse/SPARK-1834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365438#comment-14365438 ] Josh Rosen commented on SPARK-1834: --- Note that SPARK-3266 has now been fixed for 1.3.1 /

[jira] [Commented] (SPARK-6293) SQLContext.implicits should provide automatic conversion for RDD[Row]

[ https://issues.apache.org/jira/browse/SPARK-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365432#comment-14365432 ] Yin Huai commented on SPARK-6293: - [~josephkb] Actually, only a GenericRowWithSchema has a

[jira] [Commented] (SPARK-6382) withUDF(...) {...} for supporting temporary UDF definitions in the scope

[ https://issues.apache.org/jira/browse/SPARK-6382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365433#comment-14365433 ] Chen Song commented on SPARK-6382: -- If we don't change the global scope of UDF registrati

[jira] [Resolved] (SPARK-3266) JavaDoubleRDD doesn't contain max()

[ https://issues.apache.org/jira/browse/SPARK-3266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-3266. --- Resolution: Fixed Fix Version/s: 1.3.1 1.4.0 Issue resolved by pull request

[jira] [Commented] (SPARK-3351) Yarn YarnRMClientImpl.shutdown can be called before register - NPE

[ https://issues.apache.org/jira/browse/SPARK-3351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365425#comment-14365425 ] Thomas Graves commented on SPARK-3351: -- it was against yarn alpha which was hadoop 0.

[jira] [Commented] (SPARK-3351) Yarn YarnRMClientImpl.shutdown can be called before register - NPE

[ https://issues.apache.org/jira/browse/SPARK-3351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365419#comment-14365419 ] Steve Loughran commented on SPARK-3351: --- Thomas: what Yarn version was this against?

[jira] [Commented] (SPARK-6292) Add RDD methods to DataFrame to preserve schema

[ https://issues.apache.org/jira/browse/SPARK-6292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365404#comment-14365404 ] Chen Song commented on SPARK-6292: -- I got it. I think we can expand the RDDApi and create

[jira] [Commented] (SPARK-6292) Add RDD methods to DataFrame to preserve schema

[ https://issues.apache.org/jira/browse/SPARK-6292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365375#comment-14365375 ] Joseph K. Bradley commented on SPARK-6292: -- If you compare the APIs of DataFrame

[jira] [Commented] (SPARK-6292) Add RDD methods to DataFrame to preserve schema

[ https://issues.apache.org/jira/browse/SPARK-6292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365370#comment-14365370 ] Chen Song commented on SPARK-6292: -- Can you explain this JIRA more exactly? Or you can gi

[jira] [Commented] (SPARK-6250) Types are now reserved words in DDL parser.

[ https://issues.apache.org/jira/browse/SPARK-6250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365307#comment-14365307 ] Yin Huai commented on SPARK-6250: - I see. The problem is that the data type string returne

[jira] [Commented] (SPARK-6320) Adding new query plan strategy to SQLContext

[ https://issues.apache.org/jira/browse/SPARK-6320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365274#comment-14365274 ] Youssef Hatem commented on SPARK-6320: -- Thank you Michael for your response, Actuall

[jira] [Updated] (SPARK-6377) Set the number of shuffle partitions for Exchange operator automatically based on the size of input tables and the reduce-side operation.

[ https://issues.apache.org/jira/browse/SPARK-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-6377: Summary: Set the number of shuffle partitions for Exchange operator automatically based on the size of input

[jira] [Commented] (SPARK-6377) Set the number of shuffle partitions for Exchange operator automatically based on the size of input tables and the reduce-side operation.

[ https://issues.apache.org/jira/browse/SPARK-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365257#comment-14365257 ] Yin Huai commented on SPARK-6377: - I have updated the title to be more specific. > Set th

[jira] [Commented] (SPARK-6250) Types are now reserved words in DDL parser.

[ https://issues.apache.org/jira/browse/SPARK-6250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365248#comment-14365248 ] Nitay Joffe commented on SPARK-6250: [~yhuai] that query works, but that's because the

[jira] [Commented] (SPARK-6365) jetty-security needed for SPARK_PREPEND_CLASSES to work

[ https://issues.apache.org/jira/browse/SPARK-6365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365251#comment-14365251 ] Apache Spark commented on SPARK-6365: - User 'squito' has created a pull request for th

[jira] [Commented] (SPARK-6377) Set the number of shuffle partitions for Exchange operator automatically based on the size of input tables and the reduce-side operation.

[ https://issues.apache.org/jira/browse/SPARK-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365203#comment-14365203 ] Yin Huai commented on SPARK-6377: - [~sowen] I's related to that one. But, the scope of thi

[jira] [Commented] (SPARK-6388) Spark 1.3 + Hadoop 2.6 Can't work on Java 8_40

[ https://issues.apache.org/jira/browse/SPARK-6388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365211#comment-14365211 ] John commented on SPARK-6388: - Thanks, I will try it later > Spark 1.3 + Hadoop 2.6 Can't wor

[jira] [Commented] (SPARK-6389) YARN app diagnostics report doesn't report NPEs

[ https://issues.apache.org/jira/browse/SPARK-6389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365168#comment-14365168 ] Apache Spark commented on SPARK-6389: - User 'steveloughran' has created a pull request

[jira] [Created] (SPARK-6389) YARN app diagnostics report doesn't report NPEs

Steve Loughran created SPARK-6389: - Summary: YARN app diagnostics report doesn't report NPEs Key: SPARK-6389 URL: https://issues.apache.org/jira/browse/SPARK-6389 Project: Spark Issue Type: B

[jira] [Commented] (SPARK-6388) Spark 1.3 + Hadoop 2.6 Can't work on Java 8_40

[ https://issues.apache.org/jira/browse/SPARK-6388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365158#comment-14365158 ] John commented on SPARK-6388: - OK. But I rollback my system to Java 7 environment, I don't kee

[jira] [Commented] (SPARK-6388) Spark 1.3 + Hadoop 2.6 Can't work on Java 8_40

[ https://issues.apache.org/jira/browse/SPARK-6388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365159#comment-14365159 ] Sean Owen commented on SPARK-6388: -- I build just like you do: {{mvn clean package -Pyarn

[jira] [Commented] (SPARK-6388) Spark 1.3 + Hadoop 2.6 Can't work on Java 8_40

[ https://issues.apache.org/jira/browse/SPARK-6388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365152#comment-14365152 ] Sean Owen commented on SPARK-6388: -- OK. Although I don't see a problem building and runni

[jira] [Commented] (SPARK-6388) Spark 1.3 + Hadoop 2.6 Can't work on Java 8_40

[ https://issues.apache.org/jira/browse/SPARK-6388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365148#comment-14365148 ] John commented on SPARK-6388: - Ok, it looks like my problem. I'll try it later. Sorry for open

[jira] [Closed] (SPARK-6388) Spark 1.3 + Hadoop 2.6 Can't work on Java 8_40

[ https://issues.apache.org/jira/browse/SPARK-6388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John closed SPARK-6388. --- Resolution: Not a Problem > Spark 1.3 + Hadoop 2.6 Can't work on Java 8_40 > -

[jira] [Updated] (SPARK-6388) Spark 1.3 + Hadoop 2.6 Can't work on Java 8_40

[ https://issues.apache.org/jira/browse/SPARK-6388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John updated SPARK-6388: Environment: 1. Linux version 3.16.0-30-generic (buildd@komainu) (gcc version 4.9.1 (Ubuntu 4.9.1-16ubuntu6) ) #40-

[jira] [Commented] (SPARK-6388) Spark 1.3 + Hadoop 2.6 Can't work on Java 8_40

[ https://issues.apache.org/jira/browse/SPARK-6388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365142#comment-14365142 ] Sean Owen commented on SPARK-6388: -- Hm, it's not clear what you mean here. You are just s

[jira] [Created] (SPARK-6388) Spark 1.3 + Hadoop 2.6 Can't work on Java 8_40

John created SPARK-6388: --- Summary: Spark 1.3 + Hadoop 2.6 Can't work on Java 8_40 Key: SPARK-6388 URL: https://issues.apache.org/jira/browse/SPARK-6388 Project: Spark Issue Type: Bug Componen

[jira] [Commented] (SPARK-6365) jetty-security needed for SPARK_PREPEND_CLASSES to work

[ https://issues.apache.org/jira/browse/SPARK-6365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365132#comment-14365132 ] Apache Spark commented on SPARK-6365: - User 'squito' has created a pull request for th

[jira] [Commented] (SPARK-3685) Spark's local dir should accept only local paths

[ https://issues.apache.org/jira/browse/SPARK-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365087#comment-14365087 ] Steve Loughran commented on SPARK-3685: --- YARN-1197 covers supporting resizing existi

[jira] [Commented] (SPARK-5563) LDA with online variational inference

[ https://issues.apache.org/jira/browse/SPARK-5563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365073#comment-14365073 ] Matthew Willson commented on SPARK-5563: No thank you for working on this! great s

[jira] [Commented] (SPARK-6370) RDD sampling with replacement intermittently yields incorrect number of samples

[ https://issues.apache.org/jira/browse/SPARK-6370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365067#comment-14365067 ] Sean Owen commented on SPARK-6370: -- Yes exactly, a pull request. See https://cwiki.apach

[jira] [Commented] (SPARK-6116) Making DataFrame API non-experimental

[ https://issues.apache.org/jira/browse/SPARK-6116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365062#comment-14365062 ] Apache Spark commented on SPARK-6116: - User 'azagrebin' has created a pull request for

  1   2   >