[jira] [Commented] (SPARK-18692) Test Java 8 unidoc build on Jenkins master builder

2017-03-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948514#comment-15948514 ] Apache Spark commented on SPARK-18692: -- User 'HyukjinKwon' has created a pull reques

[jira] [Assigned] (SPARK-18692) Test Java 8 unidoc build on Jenkins master builder

2017-03-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18692: Assignee: Apache Spark > Test Java 8 unidoc build on Jenkins master builder >

[jira] [Created] (SPARK-20155) CSV-files with quoted quotes can't be parsed, if delimiter followes quoted quote

2017-03-29 Thread Rick Moritz (JIRA)
Rick Moritz created SPARK-20155: --- Summary: CSV-files with quoted quotes can't be parsed, if delimiter followes quoted quote Key: SPARK-20155 URL: https://issues.apache.org/jira/browse/SPARK-20155 Projec

[jira] [Assigned] (SPARK-18692) Test Java 8 unidoc build on Jenkins master builder

2017-03-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18692: Assignee: (was: Apache Spark) > Test Java 8 unidoc build on Jenkins master builder > -

[jira] [Updated] (SPARK-20154) In web ui,http://ip:4040/executors/,the title 'Storage Memory' should modify 'Storage Memory used /total'

2017-03-29 Thread guoxiaolongzte (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guoxiaolongzte updated SPARK-20154: --- Attachment: Before the change.png After the change.png > In web ui,http://ip:

[jira] [Created] (SPARK-20154) In web ui,http://ip:4040/executors/,the title 'Storage Memory' should modify 'Storage Memory used /total'

2017-03-29 Thread guoxiaolongzte (JIRA)
guoxiaolongzte created SPARK-20154: -- Summary: In web ui,http://ip:4040/executors/,the title 'Storage Memory' should modify 'Storage Memory used /total' Key: SPARK-20154 URL: https://issues.apache.org/jira/browse

[jira] [Created] (SPARK-20153) Support Multiple aws credentials in order to access multiple Hive on S3 table in spark application

2017-03-29 Thread Franck Tago (JIRA)
Franck Tago created SPARK-20153: --- Summary: Support Multiple aws credentials in order to access multiple Hive on S3 table in spark application Key: SPARK-20153 URL: https://issues.apache.org/jira/browse/SPARK-20153

[jira] [Updated] (SPARK-20152) Time zone is not respected while parsing csv for timeStampFormat "MM-dd-yyyy'T'HH:mm:ss.SSSZZ"

2017-03-29 Thread Navya Krishnappa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navya Krishnappa updated SPARK-20152: - Description: When reading the below mentioned time value by specifying the "timestampFor

[jira] [Created] (SPARK-20152) Time zone is not respected while parsing csv for timeStampFormat "MM-dd-yyyy'T'HH:mm:ss.SSSZZ"

2017-03-29 Thread Navya Krishnappa (JIRA)
Navya Krishnappa created SPARK-20152: Summary: Time zone is not respected while parsing csv for timeStampFormat "MM-dd-'T'HH:mm:ss.SSSZZ" Key: SPARK-20152 URL: https://issues.apache.org/jira/browse/SPARK-2

[jira] [Assigned] (SPARK-20151) Account for partition pruning in scan metadataTime metrics

2017-03-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20151: Assignee: Reynold Xin (was: Apache Spark) > Account for partition pruning in scan metadat

[jira] [Commented] (SPARK-20151) Account for partition pruning in scan metadataTime metrics

2017-03-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948422#comment-15948422 ] Apache Spark commented on SPARK-20151: -- User 'rxin' has created a pull request for t

[jira] [Assigned] (SPARK-20151) Account for partition pruning in scan metadataTime metrics

2017-03-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20151: Assignee: Apache Spark (was: Reynold Xin) > Account for partition pruning in scan metadat

[jira] [Updated] (SPARK-20151) Account for partition pruning in scan metadataTime metrics

2017-03-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-20151: Summary: Account for partition pruning in scan metadataTime metrics (was: Take partition pruning t

[jira] [Created] (SPARK-20151) Take partition pruning timing into account in scan metadataTime metrics

2017-03-29 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-20151: --- Summary: Take partition pruning timing into account in scan metadataTime metrics Key: SPARK-20151 URL: https://issues.apache.org/jira/browse/SPARK-20151 Project: Spark

[jira] [Updated] (SPARK-20151) Account for partition pruning in scan metadataTime metrics

2017-03-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-20151: Description: After SPARK-20136, we report metadata timing metrics in scan operator. However, that

[jira] [Resolved] (SPARK-20148) Extend the file commit interface to allow subscribing to task commit messages

2017-03-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-20148. - Resolution: Fixed Assignee: Eric Liang Fix Version/s: 2.2.0 > Extend the file com

[jira] [Updated] (SPARK-20150) Add permsize statistics for worker memory which may be very useful for the memory usage assessment

2017-03-29 Thread Jinhua Fu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jinhua Fu updated SPARK-20150: -- Summary: Add permsize statistics for worker memory which may be very useful for the memory usage assess

[jira] [Commented] (SPARK-14492) Spark SQL 1.6.0 does not work with Hive version lower than 1.2.0; its not backwards compatible with earlier version

2017-03-29 Thread Chico Qi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948335#comment-15948335 ] Chico Qi commented on SPARK-14492: -- I had the same issue when I upgraded to Spark 2.1.0

[jira] [Created] (SPARK-20150) Can the spark add a mechanism for permsize statistics which may be very useful for the memory usage assessment

2017-03-29 Thread Jinhua Fu (JIRA)
Jinhua Fu created SPARK-20150: - Summary: Can the spark add a mechanism for permsize statistics which may be very useful for the memory usage assessment Key: SPARK-20150 URL: https://issues.apache.org/jira/browse/SPARK

[jira] [Resolved] (SPARK-20136) Add num files and metadata operation timing to scan metrics

2017-03-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-20136. - Resolution: Fixed Fix Version/s: 2.2.0 > Add num files and metadata operation timing to sc

[jira] [Resolved] (SPARK-20146) Column comment information is missing for Thrift Server's TableSchema

2017-03-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-20146. - Resolution: Fixed Assignee: Bo Meng Fix Version/s: 2.2.0 > Column comment informa

[jira] [Updated] (SPARK-20104) Don't estimate IsNull or IsNotNull predicates for non-leaf node

2017-03-29 Thread Zhenhua Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-20104: - Issue Type: Sub-task (was: Bug) Parent: SPARK-16026 > Don't estimate IsNull or IsNotNull

[jira] [Commented] (SPARK-18692) Test Java 8 unidoc build on Jenkins master builder

2017-03-29 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948248#comment-15948248 ] Hyukjin Kwon commented on SPARK-18692: -- Thank you for asking this. Let me give a sho

[jira] [Resolved] (SPARK-15427) Spark SQL doesn't support field case sensitive when load data use Phoenix

2017-03-29 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-15427. -- Resolution: Not A Problem {{SELECT * FROM $table WHERE 1=0}} seems now changable via dialect in

[jira] [Created] (SPARK-20149) Audit PySpark code base for 2.6 specific work arounds

2017-03-29 Thread holdenk (JIRA)
holdenk created SPARK-20149: --- Summary: Audit PySpark code base for 2.6 specific work arounds Key: SPARK-20149 URL: https://issues.apache.org/jira/browse/SPARK-20149 Project: Spark Issue Type: Sub-t

[jira] [Commented] (SPARK-14657) RFormula output wrong features when formula w/o intercept

2017-03-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948142#comment-15948142 ] Joseph K. Bradley commented on SPARK-14657: --- I'm going to remove the target ver

[jira] [Updated] (SPARK-14657) RFormula output wrong features when formula w/o intercept

2017-03-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-14657: -- Target Version/s: (was: 2.2.0) > RFormula output wrong features when formula w/o inte

[jira] [Updated] (SPARK-14657) RFormula output wrong features when formula w/o intercept

2017-03-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-14657: -- Shepherd: (was: Xiangrui Meng) > RFormula output wrong features when formula w/o inte

[jira] [Commented] (SPARK-20148) Extend the file commit interface to allow subscribing to task commit messages

2017-03-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948085#comment-15948085 ] Apache Spark commented on SPARK-20148: -- User 'ericl' has created a pull request for

[jira] [Assigned] (SPARK-20148) Extend the file commit interface to allow subscribing to task commit messages

2017-03-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20148: Assignee: (was: Apache Spark) > Extend the file commit interface to allow subscribing

[jira] [Assigned] (SPARK-20148) Extend the file commit interface to allow subscribing to task commit messages

2017-03-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20148: Assignee: Apache Spark > Extend the file commit interface to allow subscribing to task com

[jira] [Created] (SPARK-20148) Extend the file commit interface to allow subscribing to task commit messages

2017-03-29 Thread Eric Liang (JIRA)
Eric Liang created SPARK-20148: -- Summary: Extend the file commit interface to allow subscribing to task commit messages Key: SPARK-20148 URL: https://issues.apache.org/jira/browse/SPARK-20148 Project: Sp

[jira] [Updated] (SPARK-18958) SparkR should support toJSON on DataFrame

2017-03-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18958: -- Fix Version/s: 2.2.0 > SparkR should support toJSON on DataFrame >

[jira] [Updated] (SPARK-3723) DecisionTree, RandomForest: Add more instrumentation

2017-03-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-3723: - Target Version/s: (was: 2.2.0) > DecisionTree, RandomForest: Add more instrumentation >

[jira] [Updated] (SPARK-3723) DecisionTree, RandomForest: Add more instrumentation

2017-03-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-3723: - Component/s: (was: MLlib) ML > DecisionTree, RandomForest: Add more i

[jira] [Updated] (SPARK-3723) DecisionTree, RandomForest: Add more instrumentation

2017-03-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-3723: - Shepherd: Joseph K. Bradley > DecisionTree, RandomForest: Add more instrumentation > -

[jira] [Commented] (SPARK-18570) Consider supporting other R formula operators

2017-03-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948056#comment-15948056 ] Joseph K. Bradley commented on SPARK-18570: --- Is this still targeted for 2.2, or

[jira] [Commented] (SPARK-3181) Add Robust Regression Algorithm with Huber Estimator

2017-03-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948055#comment-15948055 ] Joseph K. Bradley commented on SPARK-3181: -- Is this still active, and should it b

[jira] [Updated] (SPARK-14659) OneHotEncoder support drop first category alphabetically in the encoded vector

2017-03-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-14659: -- Target Version/s: (was: 2.2.0) > OneHotEncoder support drop first category alphabetic

[jira] [Commented] (SPARK-14659) OneHotEncoder support drop first category alphabetically in the encoded vector

2017-03-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948049#comment-15948049 ] Joseph K. Bradley commented on SPARK-14659: --- [~actuaryzhang] I'm sorry I haven'

[jira] [Updated] (SPARK-18822) Support ML Pipeline in SparkR

2017-03-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18822: -- Target Version/s: (was: 2.2.0) > Support ML Pipeline in SparkR >

[jira] [Commented] (SPARK-18822) Support ML Pipeline in SparkR

2017-03-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948045#comment-15948045 ] Joseph K. Bradley commented on SPARK-18822: --- Since 2.2 will be cut soon (I pres

[jira] [Updated] (SPARK-20103) Spark structured steaming from kafka - last message processed again after resume from checkpoint

2017-03-29 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-20103: - Fix Version/s: 2.2.0 > Spark structured steaming from kafka - last message processed agai

[jira] [Commented] (SPARK-20103) Spark structured steaming from kafka - last message processed again after resume from checkpoint

2017-03-29 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948035#comment-15948035 ] Michael Armbrust commented on SPARK-20103: -- It is fixed in 2.2 but by [SPARK-198

[jira] [Updated] (SPARK-20103) Spark structured steaming from kafka - last message processed again after resume from checkpoint

2017-03-29 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-20103: - Description: When the application starts after a failure or a graceful shutdown, it is c

[jira] [Updated] (SPARK-20103) Spark structured steaming from kafka - last message processed again after resume from checkpoint

2017-03-29 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-20103: - Docs Text: (was: object StructuredStreaming { def main(args: Array[String]): Unit = {

[jira] [Resolved] (SPARK-20120) spark-sql CLI support silent mode

2017-03-29 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-20120. - Resolution: Fixed Assignee: Yuming Wang Fix Version/s: 2.2.0 > spark-sql CLI support sile

[jira] [Updated] (SPARK-20147) Cloning SessionState does not clone streaming query listeners

2017-03-29 Thread Kunal Khamar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Khamar updated SPARK-20147: - Description: Cloning session should clone StreamingQueryListeners registered on the StreamingQue

[jira] [Updated] (SPARK-20147) Cloning SessionState does not clone streaming query listeners

2017-03-29 Thread Kunal Khamar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Khamar updated SPARK-20147: - Description: Cloning session should clone StreamingQueryListeners registered on the StreamingQuer

[jira] [Created] (SPARK-20147) Cloning SessionState does not clone streaming query listeners

2017-03-29 Thread Kunal Khamar (JIRA)
Kunal Khamar created SPARK-20147: Summary: Cloning SessionState does not clone streaming query listeners Key: SPARK-20147 URL: https://issues.apache.org/jira/browse/SPARK-20147 Project: Spark

[jira] [Commented] (SPARK-19088) Optimize sequence type deserialization codegen

2017-03-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947972#comment-15947972 ] Apache Spark commented on SPARK-19088: -- User 'ueshin' has created a pull request for

[jira] [Updated] (SPARK-20144) spark.read.parquet no long maintains ordering of the data

2017-03-29 Thread Li Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Jin updated SPARK-20144: --- Summary: spark.read.parquet no long maintains ordering of the data (was: spark.read.parquet no long maintain

[jira] [Commented] (SPARK-20145) "SELECT * FROM range(1)" works, but "SELECT * FROM RANGE(1)" doesn't

2017-03-29 Thread sam elamin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947781#comment-15947781 ] sam elamin commented on SPARK-20145: if no one is picking this up, id love to take it

[jira] [Resolved] (SPARK-20009) Use user-friendly DDL formats for defining a schema in user-facing APIs

2017-03-29 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-20009. - Resolution: Fixed Assignee: Takeshi Yamamuro Fix Version/s: 2.2.0 > Use user-friendly DDL

[jira] [Updated] (SPARK-20009) Use user-friendly DDL formats for defining a schema in functions.from_json

2017-03-29 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-20009: Summary: Use user-friendly DDL formats for defining a schema in functions.from_json (was: Use user-friend

[jira] [Resolved] (SPARK-20048) Cloning SessionState does not clone query execution listeners

2017-03-29 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-20048. --- Resolution: Fixed Assignee: Kunal Khamar Fix Version/s: 2.2.0 > Cloni

[jira] [Commented] (SPARK-19999) Test failures in Spark Core due to java.nio.Bits.unaligned()

2017-03-29 Thread sam elamin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947761#comment-15947761 ] sam elamin commented on SPARK-1: can someone assign this to me, happy to take it

[jira] [Commented] (SPARK-19999) Test failures in Spark Core due to java.nio.Bits.unaligned()

2017-03-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947755#comment-15947755 ] Apache Spark commented on SPARK-1: -- User 'samelamin' has created a pull request

[jira] [Assigned] (SPARK-19999) Test failures in Spark Core due to java.nio.Bits.unaligned()

2017-03-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-1: Assignee: Apache Spark > Test failures in Spark Core due to java.nio.Bits.unaligned() > --

[jira] [Assigned] (SPARK-19999) Test failures in Spark Core due to java.nio.Bits.unaligned()

2017-03-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-1: Assignee: (was: Apache Spark) > Test failures in Spark Core due to java.nio.Bits.unali

[jira] [Comment Edited] (SPARK-16938) Cannot resolve column name after a join

2017-03-29 Thread sam elamin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947732#comment-15947732 ] sam elamin edited comment on SPARK-16938 at 3/29/17 7:20 PM: -

[jira] [Commented] (SPARK-16938) Cannot resolve column name after a join

2017-03-29 Thread sam elamin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947732#comment-15947732 ] sam elamin commented on SPARK-16938: [~cloud_fan] I am happy picking up this ticket,

[jira] [Commented] (SPARK-20145) "SELECT * FROM range(1)" works, but "SELECT * FROM RANGE(1)" doesn't

2017-03-29 Thread Bo Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947726#comment-15947726 ] Bo Meng commented on SPARK-20145: - >From the current code, I can see builtinFunctions is

[jira] [Resolved] (SPARK-19955) Update run-tests to support conda

2017-03-29 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk resolved SPARK-19955. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17355 [https://github.com/ap

[jira] [Commented] (SPARK-3577) Add task metric to report spill time

2017-03-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947678#comment-15947678 ] Apache Spark commented on SPARK-3577: - User 'sitalkedia' has created a pull request fo

[jira] [Assigned] (SPARK-19955) Update run-tests to support conda

2017-03-29 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk reassigned SPARK-19955: --- Assignee: holdenk > Update run-tests to support conda > - > >

[jira] [Commented] (SPARK-3577) Add task metric to report spill time

2017-03-29 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947669#comment-15947669 ] Sital Kedia commented on SPARK-3577: I am making a change to report correct spill data

[jira] [Assigned] (SPARK-20146) Column comment information is missing for Thrift Server's TableSchema

2017-03-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20146: Assignee: (was: Apache Spark) > Column comment information is missing for Thrift Serve

[jira] [Commented] (SPARK-20146) Column comment information is missing for Thrift Server's TableSchema

2017-03-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947665#comment-15947665 ] Apache Spark commented on SPARK-20146: -- User 'bomeng' has created a pull request for

[jira] [Assigned] (SPARK-20146) Column comment information is missing for Thrift Server's TableSchema

2017-03-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20146: Assignee: Apache Spark > Column comment information is missing for Thrift Server's TableSc

[jira] [Created] (SPARK-20146) Column comment information is missing for Thrift Server's TableSchema

2017-03-29 Thread Bo Meng (JIRA)
Bo Meng created SPARK-20146: --- Summary: Column comment information is missing for Thrift Server's TableSchema Key: SPARK-20146 URL: https://issues.apache.org/jira/browse/SPARK-20146 Project: Spark

[jira] [Commented] (SPARK-18692) Test Java 8 unidoc build on Jenkins master builder

2017-03-29 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947627#comment-15947627 ] Josh Rosen commented on SPARK-18692: We can't get the full Jekyll doc build running u

[jira] [Issue Comment Deleted] (SPARK-20132) Add documentation for column string functions

2017-03-29 Thread Michael Patterson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Patterson updated SPARK-20132: -- Comment: was deleted (was: I have a commit with the documentation: https://github.com/

[jira] [Assigned] (SPARK-20132) Add documentation for column string functions

2017-03-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20132: Assignee: (was: Apache Spark) > Add documentation for column string functions > --

[jira] [Commented] (SPARK-20132) Add documentation for column string functions

2017-03-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947538#comment-15947538 ] Apache Spark commented on SPARK-20132: -- User 'map222' has created a pull request for

[jira] [Assigned] (SPARK-20132) Add documentation for column string functions

2017-03-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20132: Assignee: Apache Spark > Add documentation for column string functions > -

[jira] [Resolved] (SPARK-20059) HbaseCredentialProvider uses wrong classloader

2017-03-29 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-20059. Resolution: Fixed Assignee: Saisai Shao Fix Version/s: 2.2.0

[jira] [Commented] (SPARK-16938) Cannot resolve column name after a join

2017-03-29 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947485#comment-15947485 ] Dongjoon Hyun commented on SPARK-16938: --- Sure, go ahead. I'm not working on this.

[jira] [Commented] (SPARK-16938) Cannot resolve column name after a join

2017-03-29 Thread sam elamin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947395#comment-15947395 ] sam elamin commented on SPARK-16938: [~dongjoon] I can pick this up if you dont mind,

[jira] [Created] (SPARK-20145) "SELECT * FROM range(1)" works, but "SELECT * FROM RANGE(1)" doesn't

2017-03-29 Thread Juliusz Sompolski (JIRA)
Juliusz Sompolski created SPARK-20145: - Summary: "SELECT * FROM range(1)" works, but "SELECT * FROM RANGE(1)" doesn't Key: SPARK-20145 URL: https://issues.apache.org/jira/browse/SPARK-20145 Projec

[jira] [Assigned] (SPARK-20143) DataType.fromJson should throw an exception with better message

2017-03-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20143: Assignee: (was: Apache Spark) > DataType.fromJson should throw an exception with bette

[jira] [Assigned] (SPARK-20143) DataType.fromJson should throw an exception with better message

2017-03-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20143: Assignee: Apache Spark > DataType.fromJson should throw an exception with better message >

[jira] [Commented] (SPARK-20143) DataType.fromJson should throw an exception with better message

2017-03-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947237#comment-15947237 ] Apache Spark commented on SPARK-20143: -- User 'HyukjinKwon' has created a pull reques

[jira] [Updated] (SPARK-20144) spark.read.parquet no long maintains the ordering the the data

2017-03-29 Thread Li Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Jin updated SPARK-20144: --- Description: Hi, We are trying to upgrade Spark from 1.6.3 to 2.0.2. One issue we found is when we read parq

[jira] [Updated] (SPARK-20144) spark.read.parquet no long maintains the ordering the the data

2017-03-29 Thread Li Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Jin updated SPARK-20144: --- Description: Hi, We are trying to upgrade Spark from 1.6.3 to 2.0.2. One issue we found is when we read parq

[jira] [Created] (SPARK-20144) spark.read.parquet no long maintains the ordering the the data

2017-03-29 Thread Li Jin (JIRA)
Li Jin created SPARK-20144: -- Summary: spark.read.parquet no long maintains the ordering the the data Key: SPARK-20144 URL: https://issues.apache.org/jira/browse/SPARK-20144 Project: Spark Issue Typ

[jira] [Created] (SPARK-20143) DataType.fromJson should throw an exception with better message

2017-03-29 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-20143: Summary: DataType.fromJson should throw an exception with better message Key: SPARK-20143 URL: https://issues.apache.org/jira/browse/SPARK-20143 Project: Spark

[jira] [Created] (SPARK-20142) Move RewriteDistinctAggregates later into query execution

2017-03-29 Thread Juliusz Sompolski (JIRA)
Juliusz Sompolski created SPARK-20142: - Summary: Move RewriteDistinctAggregates later into query execution Key: SPARK-20142 URL: https://issues.apache.org/jira/browse/SPARK-20142 Project: Spark

[jira] [Commented] (SPARK-14174) Accelerate KMeans via Mini-Batch EM

2017-03-29 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947114#comment-15947114 ] Nick Pentreath commented on SPARK-14174: The actual fix in the PR is pretty small

[jira] [Commented] (SPARK-20141) jdbc query gives ORA-00903

2017-03-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947106#comment-15947106 ] Sean Owen commented on SPARK-20141: --- That sounds like an Oracle error. There's no detai

[jira] [Commented] (SPARK-20140) Remove hardcoded kinesis retry wait and max retries

2017-03-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947099#comment-15947099 ] Apache Spark commented on SPARK-20140: -- User 'yssharma' has created a pull request f

[jira] [Assigned] (SPARK-20140) Remove hardcoded kinesis retry wait and max retries

2017-03-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20140: Assignee: Apache Spark > Remove hardcoded kinesis retry wait and max retries > ---

[jira] [Assigned] (SPARK-20140) Remove hardcoded kinesis retry wait and max retries

2017-03-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20140: Assignee: (was: Apache Spark) > Remove hardcoded kinesis retry wait and max retries >

[jira] [Updated] (SPARK-20141) jdbc query gives ORA-00903

2017-03-29 Thread sergio (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sergio updated SPARK-20141: --- Attachment: exception.png > jdbc query gives ORA-00903 > -- > > Key:

[jira] [Created] (SPARK-20141) jdbc query gives ORA-00903

2017-03-29 Thread sergio (JIRA)
sergio created SPARK-20141: -- Summary: jdbc query gives ORA-00903 Key: SPARK-20141 URL: https://issues.apache.org/jira/browse/SPARK-20141 Project: Spark Issue Type: Bug Components: Input/Ou

[jira] [Updated] (SPARK-20139) Spark UI reports partial success for completed stage while log shows all tasks are finished

2017-03-29 Thread Etti Gur (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Etti Gur updated SPARK-20139: - Description: Spark UI reports partial success for completed stage while log shows all tasks are finished

[jira] [Resolved] (SPARK-19556) Broadcast data is not encrypted when I/O encryption is on

2017-03-29 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-19556. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17295 [https://githu

[jira] [Comment Edited] (SPARK-12261) pyspark crash for large dataset

2017-03-29 Thread Tomas Pranckevicius (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15934218#comment-15934218 ] Tomas Pranckevicius edited comment on SPARK-12261 at 3/29/17 12:28 PM:

[jira] [Assigned] (SPARK-19556) Broadcast data is not encrypted when I/O encryption is on

2017-03-29 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-19556: --- Assignee: Marcelo Vanzin > Broadcast data is not encrypted when I/O encryption is on > -

[jira] [Commented] (SPARK-20140) Remove hardcoded kinesis retry wait and max retries

2017-03-29 Thread Yash Sharma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947015#comment-15947015 ] Yash Sharma commented on SPARK-20140: - Proposing : https://github.com/apache/spark/pu

  1   2   >