[jira] [Commented] (SPARK-18823) Assignation by column name variable not available or bug?

[ https://issues.apache.org/jira/browse/SPARK-18823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15820389#comment-15820389 ] Felix Cheung commented on SPARK-18823: -- Yap. I'll start on this shortly. > Assignat

[jira] [Commented] (SPARK-19179) spark.yarn.access.namenodes description is wrong

[ https://issues.apache.org/jira/browse/SPARK-19179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15820371#comment-15820371 ] Saisai Shao commented on SPARK-19179: - Thanks [~tgraves] to point out the left thing,

[jira] [Resolved] (SPARK-12076) countDistinct behaves inconsistently

[ https://issues.apache.org/jira/browse/SPARK-12076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-12076. -- Resolution: Cannot Reproduce I strongly think no one is going to reproduce this. I tried to ima

[jira] [Assigned] (SPARK-18693) BinaryClassificationEvaluator, RegressionEvaluator, and MulticlassClassificationEvaluator should use sample weight data

[ https://issues.apache.org/jira/browse/SPARK-18693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18693: Assignee: Apache Spark > BinaryClassificationEvaluator, RegressionEvaluator, and > Multic

[jira] [Assigned] (SPARK-18693) BinaryClassificationEvaluator, RegressionEvaluator, and MulticlassClassificationEvaluator should use sample weight data

[ https://issues.apache.org/jira/browse/SPARK-18693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18693: Assignee: (was: Apache Spark) > BinaryClassificationEvaluator, RegressionEvaluator, an

[jira] [Commented] (SPARK-18693) BinaryClassificationEvaluator, RegressionEvaluator, and MulticlassClassificationEvaluator should use sample weight data

[ https://issues.apache.org/jira/browse/SPARK-18693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15820273#comment-15820273 ] Apache Spark commented on SPARK-18693: -- User 'imatiach-msft' has created a pull requ

[jira] [Resolved] (SPARK-16848) Check schema validation for user-specified schema in jdbc and table APIs

[ https://issues.apache.org/jira/browse/SPARK-16848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-16848. - Resolution: Fixed Assignee: Hyukjin Kwon Fix Version/s: 2.2.0 > Check schema validation f

[jira] [Comment Edited] (SPARK-10078) Vector-free L-BFGS

[ https://issues.apache.org/jira/browse/SPARK-10078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819964#comment-15819964 ] Weichen Xu edited comment on SPARK-10078 at 1/12/17 4:59 AM: -

[jira] [Comment Edited] (SPARK-10078) Vector-free L-BFGS

[ https://issues.apache.org/jira/browse/SPARK-10078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819964#comment-15819964 ] Weichen Xu edited comment on SPARK-10078 at 1/12/17 4:58 AM: -

[jira] [Comment Edited] (SPARK-10078) Vector-free L-BFGS

[ https://issues.apache.org/jira/browse/SPARK-10078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15820180#comment-15820180 ] Weichen Xu edited comment on SPARK-10078 at 1/12/17 4:54 AM: -

[jira] [Comment Edited] (SPARK-10078) Vector-free L-BFGS

[ https://issues.apache.org/jira/browse/SPARK-10078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15820180#comment-15820180 ] Weichen Xu edited comment on SPARK-10078 at 1/12/17 4:54 AM: -

[jira] [Commented] (SPARK-10078) Vector-free L-BFGS

[ https://issues.apache.org/jira/browse/SPARK-10078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15820180#comment-15820180 ] Weichen Xu commented on SPARK-10078: As the detail problems I list above(I only list

[jira] [Comment Edited] (SPARK-10078) Vector-free L-BFGS

[ https://issues.apache.org/jira/browse/SPARK-10078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819851#comment-15819851 ] Weichen Xu edited comment on SPARK-10078 at 1/12/17 4:43 AM: -

[jira] [Comment Edited] (SPARK-19051) test_hivecontext (pyspark.sql.tests.HiveSparkSubmitTests) fails in python/run-tests

[ https://issues.apache.org/jira/browse/SPARK-19051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819957#comment-15819957 ] Hyukjin Kwon edited comment on SPARK-19051 at 1/12/17 4:36 AM:

[jira] [Updated] (SPARK-19188) Run spark in scala as script file, note not just REPL

[ https://issues.apache.org/jira/browse/SPARK-19188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhihao updated SPARK-19188: --- Description: Hi, I'm looking for the feature to run spark/scala in script file. The current spark-sh

[jira] [Updated] (SPARK-19188) Run spark in scala as script file, note not just REPL

[ https://issues.apache.org/jira/browse/SPARK-19188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhihao updated SPARK-19188: --- Description: Hi, I'm looking for the feature to run spark/scala in script file. The current spark-sh

[jira] [Created] (SPARK-19188) Run spark in scala as script file, note not just REPL

wangzhihao created SPARK-19188: -- Summary: Run spark in scala as script file, note not just REPL Key: SPARK-19188 URL: https://issues.apache.org/jira/browse/SPARK-19188 Project: Spark Issue Type:

[jira] [Updated] (SPARK-19133) SparkR glm Gamma family results in error

[ https://issues.apache.org/jira/browse/SPARK-19133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-19133: - Affects Version/s: 2.0.0 Target Version/s: 2.0.3, 2.1.1, 2.2.0 (was: 2.2.0) Fix Vers

[jira] [Created] (SPARK-19187) querying from parquet partitioned table throws FileNotFoundException when some partitions' hdfs locations do not exist

roncenzhao created SPARK-19187: -- Summary: querying from parquet partitioned table throws FileNotFoundException when some partitions' hdfs locations do not exist Key: SPARK-19187 URL: https://issues.apache.org/jira/br

[jira] [Created] (SPARK-19186) Hash symbol in middle of Sybase database table name causes Spark Exception

Adrian Schulewitz created SPARK-19186: - Summary: Hash symbol in middle of Sybase database table name causes Spark Exception Key: SPARK-19186 URL: https://issues.apache.org/jira/browse/SPARK-19186

[jira] [Comment Edited] (SPARK-10078) Vector-free L-BFGS

[ https://issues.apache.org/jira/browse/SPARK-10078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819964#comment-15819964 ] Weichen Xu edited comment on SPARK-10078 at 1/12/17 3:02 AM: -

[jira] [Comment Edited] (SPARK-10078) Vector-free L-BFGS

[ https://issues.apache.org/jira/browse/SPARK-10078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819964#comment-15819964 ] Weichen Xu edited comment on SPARK-10078 at 1/12/17 2:55 AM: -

[jira] [Commented] (SPARK-14901) java exception when showing join

[ https://issues.apache.org/jira/browse/SPARK-14901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819984#comment-15819984 ] Brent Elmer commented on SPARK-14901: - Netezza is an IBM product so there is no place

[jira] [Comment Edited] (SPARK-10078) Vector-free L-BFGS

[ https://issues.apache.org/jira/browse/SPARK-10078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819964#comment-15819964 ] Weichen Xu edited comment on SPARK-10078 at 1/12/17 2:48 AM: -

[jira] [Comment Edited] (SPARK-10078) Vector-free L-BFGS

[ https://issues.apache.org/jira/browse/SPARK-10078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819964#comment-15819964 ] Weichen Xu edited comment on SPARK-10078 at 1/12/17 2:45 AM: -

[jira] [Commented] (SPARK-10078) Vector-free L-BFGS

[ https://issues.apache.org/jira/browse/SPARK-10078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819964#comment-15819964 ] Weichen Xu commented on SPARK-10078: [~debasish83] But when we implement VF-LBFGS/VF-

[jira] [Closed] (SPARK-19007) Speedup and optimize the GradientBoostedTrees in the "data>memory" scene

[ https://issues.apache.org/jira/browse/SPARK-19007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangdenghui closed SPARK-19007. Resolution: Later > Speedup and optimize the GradientBoostedTrees in the "data>memory" scene >

[jira] [Commented] (SPARK-19051) test_hivecontext (pyspark.sql.tests.HiveSparkSubmitTests) fails in python/run-tests

[ https://issues.apache.org/jira/browse/SPARK-19051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819957#comment-15819957 ] Hyukjin Kwon commented on SPARK-19051: -- I just if this JIRA says a flaky test or a c

[jira] [Resolved] (SPARK-17923) dateFormat unexpected kwarg to df.write.csv

[ https://issues.apache.org/jira/browse/SPARK-17923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-17923. -- Resolution: Duplicate This should be fixed in 2.0.1 and 2.1.0. > dateFormat unexpected kwarg t

[jira] [Assigned] (SPARK-19184) Improve numerical stability for method tallSkinnyQR.

[ https://issues.apache.org/jira/browse/SPARK-19184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19184: Assignee: Apache Spark > Improve numerical stability for method tallSkinnyQR. > --

[jira] [Assigned] (SPARK-19184) Improve numerical stability for method tallSkinnyQR.

[ https://issues.apache.org/jira/browse/SPARK-19184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19184: Assignee: (was: Apache Spark) > Improve numerical stability for method tallSkinnyQR. >

[jira] [Commented] (SPARK-19184) Improve numerical stability for method tallSkinnyQR.

[ https://issues.apache.org/jira/browse/SPARK-19184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819941#comment-15819941 ] Apache Spark commented on SPARK-19184: -- User 'hl475' has created a pull request for

[jira] [Commented] (SPARK-15407) Floor division

[ https://issues.apache.org/jira/browse/SPARK-15407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819926#comment-15819926 ] Hyukjin Kwon commented on SPARK-15407: -- Hi [~hvanhovell], I just wonder if we should

[jira] [Resolved] (SPARK-15251) Cannot apply PythonUDF to aggregated column

[ https://issues.apache.org/jira/browse/SPARK-15251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-15251. -- Resolution: Cannot Reproduce {code} >>> def timesTwo(x): ... return x * 2 ... >>> sqlContex

[jira] [Commented] (SPARK-14901) java exception when showing join

[ https://issues.apache.org/jira/browse/SPARK-14901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819920#comment-15819920 ] Hyukjin Kwon commented on SPARK-14901: -- Would this be possible to provide a self-con

[jira] [Created] (SPARK-19185) ConcurrentModificationExceptions with CachedKafkaConsumers when Windowing

Kalvin Chau created SPARK-19185: --- Summary: ConcurrentModificationExceptions with CachedKafkaConsumers when Windowing Key: SPARK-19185 URL: https://issues.apache.org/jira/browse/SPARK-19185 Project: Spar

[jira] [Created] (SPARK-19184) Improve numerical stability for method tallSkinnyQR.

Huamin Li created SPARK-19184: - Summary: Improve numerical stability for method tallSkinnyQR. Key: SPARK-19184 URL: https://issues.apache.org/jira/browse/SPARK-19184 Project: Spark Issue Type: Im

[jira] [Commented] (SPARK-13303) Spark fails with pandas import error when pandas is not explicitly imported by user

[ https://issues.apache.org/jira/browse/SPARK-13303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819894#comment-15819894 ] Hyukjin Kwon commented on SPARK-13303: -- +1 > Spark fails with pandas import error w

[jira] [Commented] (SPARK-12717) pyspark broadcast fails when using multiple threads

[ https://issues.apache.org/jira/browse/SPARK-12717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819883#comment-15819883 ] Hyukjin Kwon commented on SPARK-12717: -- It still happens in the current master. > p

[jira] [Resolved] (SPARK-11428) Schema Merging Broken for Some Queries

[ https://issues.apache.org/jira/browse/SPARK-11428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-11428. -- Resolution: Duplicate I am pretty sure that it is a duplicate of SPARK-11103. Please reopen thi

[jira] [Resolved] (SPARK-8128) Schema Merging Broken: Dataframe Fails to Recognize Column in Schema

[ https://issues.apache.org/jira/browse/SPARK-8128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-8128. - Resolution: Duplicate I am pretty sure that it duplicates SPARK-11103. Please reopen this if anyon

[jira] [Commented] (SPARK-10078) Vector-free L-BFGS

[ https://issues.apache.org/jira/browse/SPARK-10078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819851#comment-15819851 ] Weichen Xu commented on SPARK-10078: [~debasish83] Can L-BFGS-B be distributed comput

[jira] [Commented] (SPARK-19164) Review of UserDefinedFunction._broadcast

[ https://issues.apache.org/jira/browse/SPARK-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819820#comment-15819820 ] Reynold Xin commented on SPARK-19164: - Which one should I review? I see that you open

[jira] [Commented] (SPARK-19115) SparkSQL unsupports the command " create external table if not exist new_tbl like old_tbl"

[ https://issues.apache.org/jira/browse/SPARK-19115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819716#comment-15819716 ] Xiaochen Ouyang commented on SPARK-19115: - May I ask you whether Spark supports t

[jira] [Assigned] (SPARK-19180) the offset of short is 4 in OffHeapColumnVector's putShorts

[ https://issues.apache.org/jira/browse/SPARK-19180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19180: Assignee: Apache Spark > the offset of short is 4 in OffHeapColumnVector's putShorts > ---

[jira] [Assigned] (SPARK-19180) the offset of short is 4 in OffHeapColumnVector's putShorts

[ https://issues.apache.org/jira/browse/SPARK-19180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19180: Assignee: (was: Apache Spark) > the offset of short is 4 in OffHeapColumnVector's putS

[jira] [Commented] (SPARK-19180) the offset of short is 4 in OffHeapColumnVector's putShorts

[ https://issues.apache.org/jira/browse/SPARK-19180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819634#comment-15819634 ] Apache Spark commented on SPARK-19180: -- User 'yucai' has created a pull request for

[jira] [Commented] (SPARK-19183) Add deleteWithJob hook to internal commit protocol API

[ https://issues.apache.org/jira/browse/SPARK-19183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819623#comment-15819623 ] Apache Spark commented on SPARK-19183: -- User 'ericl' has created a pull request for

[jira] [Assigned] (SPARK-19183) Add deleteWithJob hook to internal commit protocol API

[ https://issues.apache.org/jira/browse/SPARK-19183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19183: Assignee: (was: Apache Spark) > Add deleteWithJob hook to internal commit protocol API

[jira] [Assigned] (SPARK-19183) Add deleteWithJob hook to internal commit protocol API

[ https://issues.apache.org/jira/browse/SPARK-19183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19183: Assignee: Apache Spark > Add deleteWithJob hook to internal commit protocol API >

[jira] [Created] (SPARK-19183) Add deleteWithJob hook to internal commit protocol API

Eric Liang created SPARK-19183: -- Summary: Add deleteWithJob hook to internal commit protocol API Key: SPARK-19183 URL: https://issues.apache.org/jira/browse/SPARK-19183 Project: Spark Issue Type

[jira] [Updated] (SPARK-19182) Optimize the lock in StreamingJobProgressListener to not block UI when generating Streaming jobs

[ https://issues.apache.org/jira/browse/SPARK-19182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19182: - Summary: Optimize the lock in StreamingJobProgressListener to not block UI when generating Stream

[jira] [Updated] (SPARK-19182) Optimize the lock in StreamingJobProgressListener to not block when generating Streaming jobs

[ https://issues.apache.org/jira/browse/SPARK-19182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19182: - Description: When DStreamGraph is generating a job, it will hold a lock and block other APIs. Be

[jira] [Created] (SPARK-19182) Optimize the lock in StreamingJobProgressListener to not block when generating Streaming jobs

Shixiong Zhu created SPARK-19182: Summary: Optimize the lock in StreamingJobProgressListener to not block when generating Streaming jobs Key: SPARK-19182 URL: https://issues.apache.org/jira/browse/SPARK-19182

[jira] [Resolved] (SPARK-19132) Add test cases for row size estimation

[ https://issues.apache.org/jira/browse/SPARK-19132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-19132. - Resolution: Fixed Fix Version/s: 2.2.0 > Add test cases for row size estimation >

[jira] [Commented] (SPARK-18823) Assignation by column name variable not available or bug?

[ https://issues.apache.org/jira/browse/SPARK-18823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819446#comment-15819446 ] Shivaram Venkataraman commented on SPARK-18823: --- Yeah I think it makes sens

[jira] [Commented] (SPARK-19180) the offset of short is 4 in OffHeapColumnVector's putShorts

[ https://issues.apache.org/jira/browse/SPARK-19180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819400#comment-15819400 ] yucai commented on SPARK-19180: --- Hi Owen, Thanks a lot for comments, it is using unsafe AP

[jira] [Resolved] (SPARK-18801) Support resolve a nested view

[ https://issues.apache.org/jira/browse/SPARK-18801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-18801. --- Resolution: Fixed Assignee: Jiang Xingbo > Support resolve a nested view >

[jira] [Resolved] (SPARK-19130) SparkR should support setting and adding new column with singular value implicitly

[ https://issues.apache.org/jira/browse/SPARK-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-19130. --- Resolution: Fixed Assignee: Felix Cheung Fix Version/s: 2.2.0

[jira] [Commented] (SPARK-19177) SparkR Data Frame operation between columns elements

[ https://issues.apache.org/jira/browse/SPARK-19177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819260#comment-15819260 ] Shivaram Venkataraman commented on SPARK-19177: --- Thanks [~masip85] - Can yo

[jira] [Commented] (SPARK-19181) SparkListenerSuite.local metrics fails when average executorDeserializeTime is too short.

[ https://issues.apache.org/jira/browse/SPARK-19181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819186#comment-15819186 ] Jose Soltren commented on SPARK-19181: -- SPARK-2208 disabled a similar metric previou

[jira] [Created] (SPARK-19181) SparkListenerSuite.local metrics fails when average executorDeserializeTime is too short.

Jose Soltren created SPARK-19181: Summary: SparkListenerSuite.local metrics fails when average executorDeserializeTime is too short. Key: SPARK-19181 URL: https://issues.apache.org/jira/browse/SPARK-19181

[jira] [Assigned] (SPARK-9435) Java UDFs don't work with GROUP BY expressions

[ https://issues.apache.org/jira/browse/SPARK-9435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9435: --- Assignee: Apache Spark > Java UDFs don't work with GROUP BY expressions > ---

[jira] [Assigned] (SPARK-9435) Java UDFs don't work with GROUP BY expressions

[ https://issues.apache.org/jira/browse/SPARK-9435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9435: --- Assignee: (was: Apache Spark) > Java UDFs don't work with GROUP BY expressions >

[jira] [Commented] (SPARK-9435) Java UDFs don't work with GROUP BY expressions

[ https://issues.apache.org/jira/browse/SPARK-9435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819117#comment-15819117 ] Apache Spark commented on SPARK-9435: - User 'HyukjinKwon' has created a pull request f

[jira] [Commented] (SPARK-19180) the offset of short is 4 in OffHeapColumnVector's putShorts

[ https://issues.apache.org/jira/browse/SPARK-19180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819073#comment-15819073 ] Sean Owen commented on SPARK-19180: --- Are you sure? most stuff is int aligned in the JVM

[jira] [Commented] (SPARK-17136) Design optimizer interface for ML algorithms

[ https://issues.apache.org/jira/browse/SPARK-17136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819062#comment-15819062 ] Seth Hendrickson commented on SPARK-17136: -- I'm interested in working on this ta

[jira] [Created] (SPARK-19180) the offset of short is 4 in OffHeapColumnVector's putShorts

yucai created SPARK-19180: - Summary: the offset of short is 4 in OffHeapColumnVector's putShorts Key: SPARK-19180 URL: https://issues.apache.org/jira/browse/SPARK-19180 Project: Spark Issue Type: Bu

[jira] [Resolved] (SPARK-17568) Add spark-submit option for user to override ivy settings used to resolve packages/artifacts

[ https://issues.apache.org/jira/browse/SPARK-17568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-17568. Resolution: Fixed Assignee: Bryan Cutler Fix Version/s: 2.2.0 > Add spark-s

[jira] [Commented] (SPARK-18075) UDF doesn't work on non-local spark

[ https://issues.apache.org/jira/browse/SPARK-18075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15818849#comment-15818849 ] Dan commented on SPARK-18075: - If he is running into the same issue with spark-shell, which i

[jira] [Assigned] (SPARK-19152) DataFrameWriter.saveAsTable should work with hive format with append mode

[ https://issues.apache.org/jira/browse/SPARK-19152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19152: Assignee: Apache Spark > DataFrameWriter.saveAsTable should work with hive format with app

[jira] [Commented] (SPARK-19152) DataFrameWriter.saveAsTable should work with hive format with append mode

[ https://issues.apache.org/jira/browse/SPARK-19152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15818731#comment-15818731 ] Apache Spark commented on SPARK-19152: -- User 'windpiger' has created a pull request

[jira] [Assigned] (SPARK-19152) DataFrameWriter.saveAsTable should work with hive format with append mode

[ https://issues.apache.org/jira/browse/SPARK-19152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19152: Assignee: (was: Apache Spark) > DataFrameWriter.saveAsTable should work with hive form

[jira] [Commented] (SPARK-18075) UDF doesn't work on non-local spark

[ https://issues.apache.org/jira/browse/SPARK-18075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15818720#comment-15818720 ] Sean Owen commented on SPARK-18075: --- Yes, spark-shell is submitted the same way. If you

[jira] [Commented] (SPARK-19090) Dynamic Resource Allocation not respecting spark.executor.cores

[ https://issues.apache.org/jira/browse/SPARK-19090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15818716#comment-15818716 ] nirav patel commented on SPARK-19090: - [~q79969786] As I mentioned in previous commen

[jira] [Commented] (SPARK-17101) Provide consistent format identifiers for TextFileFormat and ParquetFileFormat

[ https://issues.apache.org/jira/browse/SPARK-17101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15818648#comment-15818648 ] Shuai Lin commented on SPARK-17101: --- Seems this issue has already been resolved by htt

[jira] [Commented] (SPARK-18075) UDF doesn't work on non-local spark

[ https://issues.apache.org/jira/browse/SPARK-18075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15818645#comment-15818645 ] Nick Orka commented on SPARK-18075: --- This is really cool conversation, but how about if

[jira] [Created] (SPARK-19179) spark.yarn.access.namenodes description is wrong

Thomas Graves created SPARK-19179: - Summary: spark.yarn.access.namenodes description is wrong Key: SPARK-19179 URL: https://issues.apache.org/jira/browse/SPARK-19179 Project: Spark Issue Type

[jira] [Commented] (SPARK-19169) columns changed orc table encouter 'IndexOutOfBoundsException' when read the old schema files

[ https://issues.apache.org/jira/browse/SPARK-19169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15818635#comment-15818635 ] roncenzhao commented on SPARK-19169: I have the two doubts: 1. In the method `HiveTab

[jira] [Resolved] (SPARK-19021) Generailize HDFSCredentialProvider to support non HDFS security FS

[ https://issues.apache.org/jira/browse/SPARK-19021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-19021. --- Resolution: Fixed Assignee: Saisai Shao Fix Version/s: 2.2.0 > Generailize HD

[jira] [Commented] (SPARK-19169) columns changed orc table encouter 'IndexOutOfBoundsException' when read the old schema files

[ https://issues.apache.org/jira/browse/SPARK-19169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15818599#comment-15818599 ] Sean Owen commented on SPARK-19169: --- It sounds like you're saying you read the data wit

[jira] [Commented] (SPARK-13198) sc.stop() does not clean up on driver, causes Java heap OOM.

[ https://issues.apache.org/jira/browse/SPARK-13198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15818594#comment-15818594 ] Sean Owen commented on SPARK-13198: --- I think that's up to you if you're interested in t

[jira] [Commented] (SPARK-18075) UDF doesn't work on non-local spark

[ https://issues.apache.org/jira/browse/SPARK-18075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15818591#comment-15818591 ] Sean Owen commented on SPARK-18075: --- It's possible in many cases already and always has

[jira] [Commented] (SPARK-19169) columns changed orc table encouter 'IndexOutOfBoundsException' when read the old schema files

[ https://issues.apache.org/jira/browse/SPARK-19169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15818552#comment-15818552 ] roncenzhao commented on SPARK-19169: I do not think this is a misusage of ORC. If we

[jira] [Commented] (SPARK-18075) UDF doesn't work on non-local spark

[ https://issues.apache.org/jira/browse/SPARK-18075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15818540#comment-15818540 ] Wenchen Fan commented on SPARK-18075: - Although it's not a bug, I think this could be

[jira] [Commented] (SPARK-18075) UDF doesn't work on non-local spark

[ https://issues.apache.org/jira/browse/SPARK-18075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15818526#comment-15818526 ] Michael David Pedersen commented on SPARK-18075: I'm encountering this pr

[jira] [Comment Edited] (SPARK-13198) sc.stop() does not clean up on driver, causes Java heap OOM.

[ https://issues.apache.org/jira/browse/SPARK-13198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15818498#comment-15818498 ] Dmytro Bielievtsov edited comment on SPARK-13198 at 1/11/17 2:38 PM: --

[jira] [Commented] (SPARK-13198) sc.stop() does not clean up on driver, causes Java heap OOM.

[ https://issues.apache.org/jira/browse/SPARK-13198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15818498#comment-15818498 ] Dmytro Bielievtsov commented on SPARK-13198: [~srowen] Looks like a growing n

[jira] [Commented] (SPARK-19132) Add test cases for row size estimation

[ https://issues.apache.org/jira/browse/SPARK-19132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15818492#comment-15818492 ] Apache Spark commented on SPARK-19132: -- User 'wzhfy' has created a pull request for

[jira] [Assigned] (SPARK-19132) Add test cases for row size estimation

[ https://issues.apache.org/jira/browse/SPARK-19132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19132: Assignee: Zhenhua Wang (was: Apache Spark) > Add test cases for row size estimation > ---

[jira] [Assigned] (SPARK-19178) convert string of large numbers to int should return null

[ https://issues.apache.org/jira/browse/SPARK-19178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19178: Assignee: Wenchen Fan (was: Apache Spark) > convert string of large numbers to int should

[jira] [Assigned] (SPARK-19132) Add test cases for row size estimation

[ https://issues.apache.org/jira/browse/SPARK-19132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19132: Assignee: Apache Spark (was: Zhenhua Wang) > Add test cases for row size estimation > ---

[jira] [Commented] (SPARK-19178) convert string of large numbers to int should return null

[ https://issues.apache.org/jira/browse/SPARK-19178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15818477#comment-15818477 ] Apache Spark commented on SPARK-19178: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-19178) convert string of large numbers to int should return null

[ https://issues.apache.org/jira/browse/SPARK-19178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19178: Assignee: Apache Spark (was: Wenchen Fan) > convert string of large numbers to int should

[jira] [Created] (SPARK-19178) convert string of large numbers to int should return null

Wenchen Fan created SPARK-19178: --- Summary: convert string of large numbers to int should return null Key: SPARK-19178 URL: https://issues.apache.org/jira/browse/SPARK-19178 Project: Spark Issue

[jira] [Commented] (SPARK-19151) DataFrameWriter.saveAsTable should work with hive format with overwrite mode

[ https://issues.apache.org/jira/browse/SPARK-19151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15818366#comment-15818366 ] Apache Spark commented on SPARK-19151: -- User 'windpiger' has created a pull request

[jira] [Assigned] (SPARK-19151) DataFrameWriter.saveAsTable should work with hive format with overwrite mode

[ https://issues.apache.org/jira/browse/SPARK-19151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19151: Assignee: Apache Spark > DataFrameWriter.saveAsTable should work with hive format with ove

[jira] [Assigned] (SPARK-19151) DataFrameWriter.saveAsTable should work with hive format with overwrite mode

[ https://issues.apache.org/jira/browse/SPARK-19151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19151: Assignee: (was: Apache Spark) > DataFrameWriter.saveAsTable should work with hive form

[jira] [Commented] (SPARK-19175) columns changed orc table encouter 'IndexOutOfBoundsException' when read the old schema files

[ https://issues.apache.org/jira/browse/SPARK-19175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15818344#comment-15818344 ] Sean Owen commented on SPARK-19175: --- No, continue on the JIRA I left open, SPARK-19169.

[jira] [Commented] (SPARK-19158) ml.R example fails in yarn-cluster mode due to lacks of e1071 package

[ https://issues.apache.org/jira/browse/SPARK-19158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15818300#comment-15818300 ] Apache Spark commented on SPARK-19158: -- User 'yanboliang' has created a pull request

  1   2   >