[jira] [Commented] (SPARK-20153) Support Multiple aws credentials in order to access multiple Hive on S3 table in spark application

2017-04-04 Thread Franck Tago (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956346#comment-15956346 ] Franck Tago commented on SPARK-20153: - ok thanks for the tips. It appears that EMR 5.4.0 also

[jira] [Assigned] (SPARK-20222) No Spark SQL UI when executing queries in Spark SQL CLI

2017-04-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20222: Assignee: (was: Apache Spark) > No Spark SQL UI when executing queries in Spark SQL

[jira] [Commented] (SPARK-20222) No Spark SQL UI when executing queries in Spark SQL CLI

2017-04-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956332#comment-15956332 ] Apache Spark commented on SPARK-20222: -- User 'carsonwang' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20222) No Spark SQL UI when executing queries in Spark SQL CLI

2017-04-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20222: Assignee: Apache Spark > No Spark SQL UI when executing queries in Spark SQL CLI >

[jira] [Updated] (SPARK-20218) '/applications/[app-id]/stages' in REST API,add description.

2017-04-04 Thread guoxiaolongzte (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guoxiaolongzte updated SPARK-20218: --- Description: 1. '/applications/[app-id]/stages' in rest api.status should add description

[jira] [Created] (SPARK-20222) No Spark SQL UI when executing queries in Spark SQL CLI

2017-04-04 Thread Carson Wang (JIRA)
Carson Wang created SPARK-20222: --- Summary: No Spark SQL UI when executing queries in Spark SQL CLI Key: SPARK-20222 URL: https://issues.apache.org/jira/browse/SPARK-20222 Project: Spark Issue

[jira] [Comment Edited] (SPARK-20207) Add ablity to exclude current row in WindowSpec

2017-04-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956316#comment-15956316 ] Sean Owen edited comment on SPARK-20207 at 4/5/17 5:15 AM: --- Out of curiosity,

[jira] [Commented] (SPARK-20207) Add ablity to exclude current row in WindowSpec

2017-04-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956316#comment-15956316 ] Sean Owen commented on SPARK-20207: --- Out of curiosity, it doesn't just work to make one element of the

[jira] [Created] (SPARK-20221) Port pyspark.mllib.linalg tests in pyspark/mllib/tests.py to pyspark.ml.linalg

2017-04-04 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-20221: - Summary: Port pyspark.mllib.linalg tests in pyspark/mllib/tests.py to pyspark.ml.linalg Key: SPARK-20221 URL: https://issues.apache.org/jira/browse/SPARK-20221

[jira] [Commented] (SPARK-12704) we may repartition a relation even it's not needed

2017-04-04 Thread yucai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956285#comment-15956285 ] yucai commented on SPARK-12704: --- When sort-merge-join and shuffled-hash-join, the sparkPlan looks like

[jira] [Assigned] (SPARK-20218) '/applications/[app-id]/stages' in REST API,add description.

2017-04-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20218: Assignee: Apache Spark > '/applications/[app-id]/stages' in REST API,add description. >

[jira] [Assigned] (SPARK-20218) '/applications/[app-id]/stages' in REST API,add description.

2017-04-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20218: Assignee: (was: Apache Spark) > '/applications/[app-id]/stages' in REST API,add

[jira] [Commented] (SPARK-20218) '/applications/[app-id]/stages' in REST API,add description.

2017-04-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956258#comment-15956258 ] Apache Spark commented on SPARK-20218: -- User 'guoxiaolongzte' has created a pull request for this

[jira] [Created] (SPARK-20220) Add thrift scheduling pool config in scheduling docs

2017-04-04 Thread Miklos Christine (JIRA)
Miklos Christine created SPARK-20220: Summary: Add thrift scheduling pool config in scheduling docs Key: SPARK-20220 URL: https://issues.apache.org/jira/browse/SPARK-20220 Project: Spark

[jira] [Assigned] (SPARK-20219) Schedule tasks based on size of input from ScheduledRDD

2017-04-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20219: Assignee: (was: Apache Spark) > Schedule tasks based on size of input from

[jira] [Assigned] (SPARK-20219) Schedule tasks based on size of input from ScheduledRDD

2017-04-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20219: Assignee: Apache Spark > Schedule tasks based on size of input from ScheduledRDD >

[jira] [Commented] (SPARK-20219) Schedule tasks based on size of input from ScheduledRDD

2017-04-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956245#comment-15956245 ] Apache Spark commented on SPARK-20219: -- User 'jinxing64' has created a pull request for this issue:

[jira] [Created] (SPARK-20219) Schedule tasks based on size of input from ScheduledRDD

2017-04-04 Thread jin xing (JIRA)
jin xing created SPARK-20219: Summary: Schedule tasks based on size of input from ScheduledRDD Key: SPARK-20219 URL: https://issues.apache.org/jira/browse/SPARK-20219 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-20214) pyspark.mllib SciPyTests test_serialize

2017-04-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20214: Assignee: Apache Spark > pyspark.mllib SciPyTests test_serialize >

[jira] [Commented] (SPARK-20214) pyspark.mllib SciPyTests test_serialize

2017-04-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956242#comment-15956242 ] Apache Spark commented on SPARK-20214: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20214) pyspark.mllib SciPyTests test_serialize

2017-04-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20214: Assignee: (was: Apache Spark) > pyspark.mllib SciPyTests test_serialize >

[jira] [Commented] (SPARK-20214) pyspark.mllib SciPyTests test_serialize

2017-04-04 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956237#comment-15956237 ] Liang-Chi Hsieh commented on SPARK-20214: - Confirmed that dok_matrix.tocsc() won't guarantee

[jira] [Commented] (SPARK-20135) spark thriftserver2: no job running but containers not release on yarn

2017-04-04 Thread bruce xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956235#comment-15956235 ] bruce xu commented on SPARK-20135: -- OK, Thanks. > spark thriftserver2: no job running but containers

[jira] [Created] (SPARK-20218) '/applications/[app-id]/stages' in REST API,add description.

2017-04-04 Thread guoxiaolongzte (JIRA)
guoxiaolongzte created SPARK-20218: -- Summary: '/applications/[app-id]/stages' in REST API,add description. Key: SPARK-20218 URL: https://issues.apache.org/jira/browse/SPARK-20218 Project: Spark

[jira] [Commented] (SPARK-18971) Netty issue may cause the shuffle client hang

2017-04-04 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956209#comment-15956209 ] Shixiong Zhu commented on SPARK-18971: -- It's not backported because there are two many changes in

[jira] [Updated] (SPARK-20184) performance regression for complex/long sql when enable whole stage codegen

2017-04-04 Thread Fei Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fei Wang updated SPARK-20184: - Summary: performance regression for complex/long sql when enable whole stage codegen (was: performance

[jira] [Updated] (SPARK-20184) performance regression for complex/long sql when enable codegen

2017-04-04 Thread Fei Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fei Wang updated SPARK-20184: - Description: The performance of following SQL get much worse in spark 2.x in contrast with codegen

[jira] [Updated] (SPARK-20184) performance regression for complex/long sql when enable codegen

2017-04-04 Thread Fei Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fei Wang updated SPARK-20184: - Description: The performance of following SQL get much worse in spark 2.x in contrast with codegen

[jira] [Commented] (SPARK-20207) Add ablity to exclude current row in WindowSpec

2017-04-04 Thread Satyajit varma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956134#comment-15956134 ] Satyajit varma commented on SPARK-20207: I can take up this ticket, will keep updating on the

[jira] [Resolved] (SPARK-20003) FPGrowthModel setMinConfidence should affect rules generation and transform

2017-04-04 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-20003. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17336

[jira] [Commented] (SPARK-20217) Executor should not fail stage if killed task throws non-interrupted exception

2017-04-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956093#comment-15956093 ] Apache Spark commented on SPARK-20217: -- User 'ericl' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20217) Executor should not fail stage if killed task throws non-interrupted exception

2017-04-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20217: Assignee: (was: Apache Spark) > Executor should not fail stage if killed task throws

[jira] [Assigned] (SPARK-20217) Executor should not fail stage if killed task throws non-interrupted exception

2017-04-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20217: Assignee: Apache Spark > Executor should not fail stage if killed task throws

[jira] [Resolved] (SPARK-20183) Add outlierRatio option to testOutliersWithSmallWeights

2017-04-04 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-20183. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17501

[jira] [Updated] (SPARK-20217) Executor should not fail stage if killed task throws non-interrupted exception

2017-04-04 Thread Eric Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Liang updated SPARK-20217: --- Description: This is reproducible as follows. Run the following, and then use

[jira] [Created] (SPARK-20217) Executor should not fail stage if killed task throws non-interrupted exception

2017-04-04 Thread Eric Liang (JIRA)
Eric Liang created SPARK-20217: -- Summary: Executor should not fail stage if killed task throws non-interrupted exception Key: SPARK-20217 URL: https://issues.apache.org/jira/browse/SPARK-20217 Project:

[jira] [Updated] (SPARK-19716) Dataset should allow by-name resolution for struct type elements in array

2017-04-04 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-19716: --- Fix Version/s: (was: 2.3.0) 2.2.0 > Dataset should allow by-name resolution

[jira] [Resolved] (SPARK-19716) Dataset should allow by-name resolution for struct type elements in array

2017-04-04 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-19716. Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 17398

[jira] [Assigned] (SPARK-19716) Dataset should allow by-name resolution for struct type elements in array

2017-04-04 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-19716: -- Assignee: Wenchen Fan > Dataset should allow by-name resolution for struct type elements in

[jira] [Commented] (SPARK-20216) Install pandoc on machine(s) used for packaging

2017-04-04 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956045#comment-15956045 ] Michael Armbrust commented on SPARK-20216: -- I think it all runs on

[jira] [Commented] (SPARK-20144) spark.read.parquet no long maintains ordering of the data

2017-04-04 Thread Li Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956030#comment-15956030 ] Li Jin commented on SPARK-20144: > When you save the sorted data into Parquet, only the data in

[jira] [Created] (SPARK-20216) Install pandoc on machine(s) used for packaging

2017-04-04 Thread holdenk (JIRA)
holdenk created SPARK-20216: --- Summary: Install pandoc on machine(s) used for packaging Key: SPARK-20216 URL: https://issues.apache.org/jira/browse/SPARK-20216 Project: Spark Issue Type: Bug

[jira] [Comment Edited] (SPARK-14492) Spark SQL 1.6.0 does not work with external Hive metastore version lower than 1.2.0; its not backwards compatible with earlier version

2017-04-04 Thread Sunil Rangwani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955510#comment-15955510 ] Sunil Rangwani edited comment on SPARK-14492 at 4/4/17 10:42 PM: -

[jira] [Created] (SPARK-20215) ReuseExchange is boken in SparkSQL

2017-04-04 Thread Zhan Zhang (JIRA)
Zhan Zhang created SPARK-20215: -- Summary: ReuseExchange is boken in SparkSQL Key: SPARK-20215 URL: https://issues.apache.org/jira/browse/SPARK-20215 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-20153) Support Multiple aws credentials in order to access multiple Hive on S3 table in spark application

2017-04-04 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955956#comment-15955956 ] Steve Loughran commented on SPARK-20153: I'm glad we are both in agreement about not using

[jira] [Created] (SPARK-20214) pyspark.mllib SciPyTests test_serialize

2017-04-04 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-20214: - Summary: pyspark.mllib SciPyTests test_serialize Key: SPARK-20214 URL: https://issues.apache.org/jira/browse/SPARK-20214 Project: Spark Issue

[jira] [Created] (SPARK-20213) DataFrameWriter operations do not show up in SQL tab

2017-04-04 Thread Ryan Blue (JIRA)
Ryan Blue created SPARK-20213: - Summary: DataFrameWriter operations do not show up in SQL tab Key: SPARK-20213 URL: https://issues.apache.org/jira/browse/SPARK-20213 Project: Spark Issue Type:

[jira] [Commented] (SPARK-18737) Serialization setting "spark.serializer" ignored in Spark 2.x

2017-04-04 Thread Prasanth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955747#comment-15955747 ] Prasanth commented on SPARK-18737: -- We are using 2.0.1. Can you tell us how to "disable the Kryo

[jira] [Commented] (SPARK-17495) Hive hash implementation

2017-04-04 Thread Dominic Ricard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955714#comment-15955714 ] Dominic Ricard commented on SPARK-17495: [~tejasp] We use murmur3 hash internally in some of our

[jira] [Commented] (SPARK-20203) Change default maxPatternLength value to Int.MaxValue in PrefixSpan

2017-04-04 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955705#comment-15955705 ] yuhao yang commented on SPARK-20203: [~Syrux] Since you got some experiences using the PrefixSpan,

[jira] [Commented] (SPARK-20153) Support Multiple aws credentials in order to access multiple Hive on S3 table in spark application

2017-04-04 Thread Franck Tago (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955695#comment-15955695 ] Franck Tago commented on SPARK-20153: - oh I would definitely not consider including the key into the

[jira] [Resolved] (SPARK-20204) remove SimpleCatalystConf and CatalystConf type alias

2017-04-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-20204. - Resolution: Fixed Fix Version/s: 2.2.0 > remove SimpleCatalystConf and CatalystConf type

[jira] [Resolved] (SPARK-20191) RackResolver not correctly being overridden in YARN tests

2017-04-04 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-20191. Resolution: Fixed Assignee: Marcelo Vanzin Fix Version/s: 2.1.2

[jira] [Commented] (SPARK-5158) Allow for keytab-based HDFS security in Standalone mode

2017-04-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955564#comment-15955564 ] Apache Spark commented on SPARK-5158: - User 'themodernlife' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10364) Support Parquet logical type TIMESTAMP_MILLIS

2017-04-04 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-10364: --- Assignee: Dilip Biswal > Support Parquet logical type TIMESTAMP_MILLIS >

[jira] [Commented] (SPARK-5158) Allow for keytab-based HDFS security in Standalone mode

2017-04-04 Thread Ian Hummel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955528#comment-15955528 ] Ian Hummel commented on SPARK-5158: --- At Bloomberg we've been working on a solution to this issue so we

[jira] [Comment Edited] (SPARK-5158) Allow for keytab-based HDFS security in Standalone mode

2017-04-04 Thread Ian Hummel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955528#comment-15955528 ] Ian Hummel edited comment on SPARK-5158 at 4/4/17 6:07 PM: --- At Bloomberg we've

[jira] [Comment Edited] (SPARK-14492) Spark SQL 1.6.0 does not work with external Hive metastore version lower than 1.2.0; its not backwards compatible with earlier version

2017-04-04 Thread Sunil Rangwani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955510#comment-15955510 ] Sunil Rangwani edited comment on SPARK-14492 at 4/4/17 6:06 PM: {quote}If

[jira] [Comment Edited] (SPARK-14492) Spark SQL 1.6.0 does not work with external Hive metastore version lower than 1.2.0; its not backwards compatible with earlier version

2017-04-04 Thread Sunil Rangwani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955510#comment-15955510 ] Sunil Rangwani edited comment on SPARK-14492 at 4/4/17 6:00 PM: {quote}If

[jira] [Comment Edited] (SPARK-14492) Spark SQL 1.6.0 does not work with external Hive metastore version lower than 1.2.0; its not backwards compatible with earlier version

2017-04-04 Thread Sunil Rangwani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955510#comment-15955510 ] Sunil Rangwani edited comment on SPARK-14492 at 4/4/17 5:58 PM: {quote}If

[jira] [Comment Edited] (SPARK-14492) Spark SQL 1.6.0 does not work with external Hive metastore version lower than 1.2.0; its not backwards compatible with earlier version

2017-04-04 Thread Sunil Rangwani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955510#comment-15955510 ] Sunil Rangwani edited comment on SPARK-14492 at 4/4/17 5:55 PM: {quote}If

[jira] [Commented] (SPARK-14492) Spark SQL 1.6.0 does not work with external Hive metastore version lower than 1.2.0; its not backwards compatible with earlier version

2017-04-04 Thread Sunil Rangwani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955510#comment-15955510 ] Sunil Rangwani commented on SPARK-14492: {quote}If this were true, that it doesn't work with

[jira] [Resolved] (SPARK-20161) Default log4j properties file should print thread-id in ConversionPattern

2017-04-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20161. --- Resolution: Won't Fix Target Version/s: (was: 2.2.0) > Default log4j properties file

[jira] [Created] (SPARK-20212) UDFs with Option[Primitive Type] don't work as expected

2017-04-04 Thread Marius Feteanu (JIRA)
Marius Feteanu created SPARK-20212: -- Summary: UDFs with Option[Primitive Type] don't work as expected Key: SPARK-20212 URL: https://issues.apache.org/jira/browse/SPARK-20212 Project: Spark

[jira] [Commented] (SPARK-14492) Spark SQL 1.6.0 does not work with external Hive metastore version lower than 1.2.0; its not backwards compatible with earlier version

2017-04-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955398#comment-15955398 ] Sean Owen commented on SPARK-14492: --- If this were true, that it doesn't work with 1.2.1 because of a

[jira] [Commented] (SPARK-20026) Document R GLM Tweedie family support in programming guide and code example

2017-04-04 Thread Wayne Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955342#comment-15955342 ] Wayne Zhang commented on SPARK-20026: - [~felixcheung] Yes, I will work on this. Thanks. > Document

[jira] [Updated] (SPARK-20204) remove SimpleCatalystConf and CatalystConf type alias

2017-04-04 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-20204: Summary: remove SimpleCatalystConf and CatalystConf type alias (was: separate SQLConf into

[jira] [Comment Edited] (SPARK-14492) Spark SQL 1.6.0 does not work with external Hive metastore version lower than 1.2.0; its not backwards compatible with earlier version

2017-04-04 Thread Sunil Rangwani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955315#comment-15955315 ] Sunil Rangwani edited comment on SPARK-14492 at 4/4/17 4:11 PM: {quote}

[jira] [Updated] (SPARK-14492) Spark SQL 1.6.0 does not work with external Hive metastore version lower than 1.2.0; its not backwards compatible with earlier version

2017-04-04 Thread Sunil Rangwani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Rangwani updated SPARK-14492: --- Summary: Spark SQL 1.6.0 does not work with external Hive metastore version lower than

[jira] [Commented] (SPARK-14492) Spark SQL 1.6.0 does not work with external Hive metastore version lower than 1.2.0; its not backwards compatible with earlier version

2017-04-04 Thread Sunil Rangwani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955317#comment-15955317 ] Sunil Rangwani commented on SPARK-14492: Appears to be a problem with version 2.1.0 too as

[jira] [Commented] (SPARK-14492) Spark SQL 1.6.0 does not work with Hive version lower than 1.2.0; its not backwards compatible with earlier version

2017-04-04 Thread Sunil Rangwani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955315#comment-15955315 ] Sunil Rangwani commented on SPARK-14492: {quote} How would updating your external Hive metastore

[jira] [Comment Edited] (SPARK-14492) Spark SQL 1.6.0 does not work with Hive version lower than 1.2.0; its not backwards compatible with earlier version

2017-04-04 Thread Sunil Rangwani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955284#comment-15955284 ] Sunil Rangwani edited comment on SPARK-14492 at 4/4/17 3:58 PM: [~srowen]

[jira] [Commented] (SPARK-14492) Spark SQL 1.6.0 does not work with Hive version lower than 1.2.0; its not backwards compatible with earlier version

2017-04-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955294#comment-15955294 ] Sean Owen commented on SPARK-14492: --- This isn't even consistent with the title of this JIRA. How would

[jira] [Comment Edited] (SPARK-14492) Spark SQL 1.6.0 does not work with Hive version lower than 1.2.0; its not backwards compatible with earlier version

2017-04-04 Thread Sunil Rangwani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955284#comment-15955284 ] Sunil Rangwani edited comment on SPARK-14492 at 4/4/17 3:55 PM: [~srowen]

[jira] [Comment Edited] (SPARK-14492) Spark SQL 1.6.0 does not work with Hive version lower than 1.2.0; its not backwards compatible with earlier version

2017-04-04 Thread Sunil Rangwani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955284#comment-15955284 ] Sunil Rangwani edited comment on SPARK-14492 at 4/4/17 3:51 PM: [~srowen]

[jira] [Comment Edited] (SPARK-14492) Spark SQL 1.6.0 does not work with Hive version lower than 1.2.0; its not backwards compatible with earlier version

2017-04-04 Thread Sunil Rangwani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955284#comment-15955284 ] Sunil Rangwani edited comment on SPARK-14492 at 4/4/17 3:51 PM: [~srowen]

[jira] [Commented] (SPARK-20211) `1 > 0.0001` throws Decimal scale (0) cannot be greater than precision (-2) exception

2017-04-04 Thread StanZhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955285#comment-15955285 ] StanZhai commented on SPARK-20211: -- A workaround is difficult for me, because of all of my SQL are

[jira] [Reopened] (SPARK-14492) Spark SQL 1.6.0 does not work with Hive version lower than 1.2.0; its not backwards compatible with earlier version

2017-04-04 Thread Sunil Rangwani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Rangwani reopened SPARK-14492: > Spark SQL 1.6.0 does not work with Hive version lower than 1.2.0; its not > backwards

[jira] [Commented] (SPARK-14492) Spark SQL 1.6.0 does not work with Hive version lower than 1.2.0; its not backwards compatible with earlier version

2017-04-04 Thread Sunil Rangwani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955284#comment-15955284 ] Sunil Rangwani commented on SPARK-14492: [~srowen] Please don't close with a comment "should

[jira] [Updated] (SPARK-20211) `1 > 0.0001` throws Decimal scale (0) cannot be greater than precision (-2) exception

2017-04-04 Thread StanZhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] StanZhai updated SPARK-20211: - Priority: Major (was: Minor) > `1 > 0.0001` throws Decimal scale (0) cannot be greater than precision

[jira] [Assigned] (SPARK-20211) `1 > 0.0001` throws Decimal scale (0) cannot be greater than precision (-2) exception

2017-04-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20211: Assignee: Apache Spark > `1 > 0.0001` throws Decimal scale (0) cannot be greater than

[jira] [Assigned] (SPARK-20211) `1 > 0.0001` throws Decimal scale (0) cannot be greater than precision (-2) exception

2017-04-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20211: Assignee: (was: Apache Spark) > `1 > 0.0001` throws Decimal scale (0) cannot be

[jira] [Commented] (SPARK-20211) `1 > 0.0001` throws Decimal scale (0) cannot be greater than precision (-2) exception

2017-04-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955243#comment-15955243 ] Apache Spark commented on SPARK-20211: -- User 'stanzhai' has created a pull request for this issue:

[jira] [Commented] (SPARK-20211) `1 > 0.0001` throws Decimal scale (0) cannot be greater than precision (-2) exception

2017-04-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955239#comment-15955239 ] Hyukjin Kwon commented on SPARK-20211: -- Please refer http://spark.apache.org/contributing.html

[jira] [Updated] (SPARK-20211) `1 > 0.0001` throws Decimal scale (0) cannot be greater than precision (-2) exception

2017-04-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-20211: - Priority: Minor (was: Critical) > `1 > 0.0001` throws Decimal scale (0) cannot be greater than

[jira] [Created] (SPARK-20211) `1 > 0.0001` throws Decimal scale (0) cannot be greater than precision (-2) exception

2017-04-04 Thread StanZhai (JIRA)
StanZhai created SPARK-20211: Summary: `1 > 0.0001` throws Decimal scale (0) cannot be greater than precision (-2) exception Key: SPARK-20211 URL: https://issues.apache.org/jira/browse/SPARK-20211

[jira] [Comment Edited] (SPARK-12606) Scala/Java compatibility issue Re: how to extend java transformer from Scala UnaryTransformer ?

2017-04-04 Thread Guillaume Dardelet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955191#comment-15955191 ] Guillaume Dardelet edited comment on SPARK-12606 at 4/4/17 2:27 PM: I

[jira] [Comment Edited] (SPARK-12606) Scala/Java compatibility issue Re: how to extend java transformer from Scala UnaryTransformer ?

2017-04-04 Thread Guillaume Dardelet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955191#comment-15955191 ] Guillaume Dardelet edited comment on SPARK-12606 at 4/4/17 2:27 PM: I

[jira] [Commented] (SPARK-12606) Scala/Java compatibility issue Re: how to extend java transformer from Scala UnaryTransformer ?

2017-04-04 Thread Guillaume Dardelet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955191#comment-15955191 ] Guillaume Dardelet commented on SPARK-12606: I had the same issue in Scala and I solved it by

[jira] [Commented] (SPARK-20202) Remove references to org.spark-project.hive

2017-04-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955188#comment-15955188 ] Sean Owen commented on SPARK-20202: --- [~marmbrus] as release manager of the moment, I suggest we

[jira] [Commented] (SPARK-20210) Scala tests aborted in Spark SQL on ppc64le

2017-04-04 Thread Sonia Garudi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955032#comment-15955032 ] Sonia Garudi commented on SPARK-20210: -- [~srowen] , its not a flaky test. Although I am not sure

[jira] [Closed] (SPARK-14492) Spark SQL 1.6.0 does not work with Hive version lower than 1.2.0; its not backwards compatible with earlier version

2017-04-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen closed SPARK-14492. - > Spark SQL 1.6.0 does not work with Hive version lower than 1.2.0; its not > backwards compatible with

[jira] [Resolved] (SPARK-14492) Spark SQL 1.6.0 does not work with Hive version lower than 1.2.0; its not backwards compatible with earlier version

2017-04-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-14492. --- Resolution: Not A Problem [~sunil.rangwani] please don't reopen this unless you have new

[jira] [Issue Comment Deleted] (SPARK-14492) Spark SQL 1.6.0 does not work with Hive version lower than 1.2.0; its not backwards compatible with earlier version

2017-04-04 Thread Sunil Rangwani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Rangwani updated SPARK-14492: --- Comment: was deleted (was: It was closed with the comment "should work" Needs looking into)

[jira] [Reopened] (SPARK-14492) Spark SQL 1.6.0 does not work with Hive version lower than 1.2.0; its not backwards compatible with earlier version

2017-04-04 Thread Sunil Rangwani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Rangwani reopened SPARK-14492: It was closed with the comment "should work" Needs looking into > Spark SQL 1.6.0 does not

[jira] [Resolved] (SPARK-20144) spark.read.parquet no long maintains ordering of the data

2017-04-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20144. --- Resolution: Not A Problem > spark.read.parquet no long maintains ordering of the data >

[jira] [Resolved] (SPARK-20185) csv decompressed incorrectly with extention other than 'gz'

2017-04-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20185. --- Resolution: Not A Problem > csv decompressed incorrectly with extention other than 'gz' >

[jira] [Resolved] (SPARK-20198) Remove the inconsistency in table/function name conventions in SparkSession.Catalog APIs

2017-04-04 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20198. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17518

[jira] [Commented] (SPARK-17742) Spark Launcher does not get failed state in Listener

2017-04-04 Thread Daan Van den Nest (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15954988#comment-15954988 ] Daan Van den Nest commented on SPARK-17742: --- Hi [~srowen], unfortunately I'm a java dev, so

[jira] [Closed] (SPARK-20206) spark.ui.killEnabled=false property doesn't reflect on task/stages

2017-04-04 Thread srinivasan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] srinivasan closed SPARK-20206. -- Resolution: Not A Bug > spark.ui.killEnabled=false property doesn't reflect on task/stages >

  1   2   >