[jira] [Comment Edited] (SPARK-16498) move hive hack for data source table into HiveExternalCatalog

[ https://issues.apache.org/jira/browse/SPARK-16498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15430072#comment-15430072 ] Yin Huai edited comment on SPARK-16498 at 8/22/16 5:26 AM: --- This comment is for

[jira] [Commented] (SPARK-16498) move hive hack for data source table into HiveExternalCatalog

[ https://issues.apache.org/jira/browse/SPARK-16498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15430072#comment-15430072 ] Yin Huai commented on SPARK-16498: -- After this change, {{hive.default.rcfile.serde}} will not be read

[jira] [Updated] (SPARK-16498) move hive hack for data source table into HiveExternalCatalog

[ https://issues.apache.org/jira/browse/SPARK-16498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-16498: - Labels: release_notes releasenotes (was: ) > move hive hack for data source table into

[jira] [Resolved] (SPARK-16498) move hive hack for data source table into HiveExternalCatalog

[ https://issues.apache.org/jira/browse/SPARK-16498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-16498. -- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14155

[jira] [Created] (SPARK-17179) Consider improving partition pruning in HiveMetastoreCatalog

Rajesh Balamohan created SPARK-17179: Summary: Consider improving partition pruning in HiveMetastoreCatalog Key: SPARK-17179 URL: https://issues.apache.org/jira/browse/SPARK-17179 Project: Spark

[jira] [Assigned] (SPARK-16896) Loading csv with duplicate column names

[ https://issues.apache.org/jira/browse/SPARK-16896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16896: Assignee: (was: Apache Spark) > Loading csv with duplicate column names >

[jira] [Commented] (SPARK-16896) Loading csv with duplicate column names

[ https://issues.apache.org/jira/browse/SPARK-16896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15430040#comment-15430040 ] Apache Spark commented on SPARK-16896: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-16896) Loading csv with duplicate column names

[ https://issues.apache.org/jira/browse/SPARK-16896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16896: Assignee: Apache Spark > Loading csv with duplicate column names >

[jira] [Commented] (SPARK-17086) QuantileDiscretizer throws InvalidArgumentException (parameter splits given invalid value) on valid data

[ https://issues.apache.org/jira/browse/SPARK-17086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15430009#comment-15430009 ] Yanbo Liang commented on SPARK-17086: - We should not throw exception in this case. If the number of

[jira] [Commented] (SPARK-17172) pyspak hiveContext can not create UDF: Py4JJavaError: An error occurred while calling None.org.apache.spark.sql.hive.HiveContext.

[ https://issues.apache.org/jira/browse/SPARK-17172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15430004#comment-15430004 ] Andrew Davidson commented on SPARK-17172: - Hi Sean I do not think it is the same error. In the

[jira] [Created] (SPARK-17178) Allow to set sparkr shell command through --conf

Jeff Zhang created SPARK-17178: -- Summary: Allow to set sparkr shell command through --conf Key: SPARK-17178 URL: https://issues.apache.org/jira/browse/SPARK-17178 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-17086) QuantileDiscretizer throws InvalidArgumentException (parameter splits given invalid value) on valid data

[ https://issues.apache.org/jira/browse/SPARK-17086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429951#comment-15429951 ] Qian Huang edited comment on SPARK-17086 at 8/22/16 1:48 AM: - I also agree on

[jira] [Commented] (SPARK-17086) QuantileDiscretizer throws InvalidArgumentException (parameter splits given invalid value) on valid data

[ https://issues.apache.org/jira/browse/SPARK-17086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429951#comment-15429951 ] Qian Huang commented on SPARK-17086: I also agree on that it does not make sense if we put the same

[jira] [Commented] (SPARK-16896) Loading csv with duplicate column names

[ https://issues.apache.org/jira/browse/SPARK-16896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429950#comment-15429950 ] Hyukjin Kwon commented on SPARK-16896: -- Yup, then I will work on this and submit a PR within few

[jira] [Commented] (SPARK-17168) CSV with header is incorrectly read if file is partitioned

[ https://issues.apache.org/jira/browse/SPARK-17168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429947#comment-15429947 ] Hyukjin Kwon commented on SPARK-17168: -- Thanks for cc me [~maropu]! I also tend to agree CSV files

[jira] [Comment Edited] (SPARK-16578) Configurable hostname for RBackend

[ https://issues.apache.org/jira/browse/SPARK-16578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429942#comment-15429942 ] Xiangrui Meng edited comment on SPARK-16578 at 8/22/16 12:47 AM: -

[jira] [Comment Edited] (SPARK-16578) Configurable hostname for RBackend

[ https://issues.apache.org/jira/browse/SPARK-16578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429942#comment-15429942 ] Xiangrui Meng edited comment on SPARK-16578 at 8/22/16 12:46 AM: -

[jira] [Commented] (SPARK-16578) Configurable hostname for RBackend

[ https://issues.apache.org/jira/browse/SPARK-16578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429942#comment-15429942 ] Xiangrui Meng commented on SPARK-16578: --- [~shivaram] I had an offline discussion with [~junyangq]

[jira] [Commented] (SPARK-17134) Use level 2 BLAS operations in LogisticAggregator

[ https://issues.apache.org/jira/browse/SPARK-17134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429941#comment-15429941 ] Qian Huang commented on SPARK-17134: Thank you. I will do it. > Use level 2 BLAS operations in

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429928#comment-15429928 ] Robert Conrad commented on SPARK-15406: --- Bump for input from [~tdas] > Structured streaming

[jira] [Commented] (SPARK-9041) Support reading of delimited text files as DataFrames

[ https://issues.apache.org/jira/browse/SPARK-9041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429920#comment-15429920 ] Dongjoon Hyun commented on SPARK-9041: -- Thank you, [~rxin] > Support reading of delimited text files

[jira] [Closed] (SPARK-9041) Support reading of delimited text files as DataFrames

[ https://issues.apache.org/jira/browse/SPARK-9041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-9041. -- Resolution: Fixed Fix Version/s: 2.0.0 This is fixed in spark 2.0 with the CSV functionality.

[jira] [Commented] (SPARK-9041) Support reading of delimited text files as DataFrames

[ https://issues.apache.org/jira/browse/SPARK-9041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429914#comment-15429914 ] Dongjoon Hyun commented on SPARK-9041: -- Hi, [~Vishal B] and [~srowen]. Is it possible to close or

[jira] [Commented] (SPARK-7493) ALTER TABLE statement

[ https://issues.apache.org/jira/browse/SPARK-7493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429910#comment-15429910 ] Dongjoon Hyun commented on SPARK-7493: -- Also, this seems to be supported in 1.6.2 for non-timestamp

[jira] [Commented] (SPARK-7493) ALTER TABLE statement

[ https://issues.apache.org/jira/browse/SPARK-7493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429905#comment-15429905 ] Dongjoon Hyun commented on SPARK-7493: -- Hi, [~ssemichev]. Just FYI, Spark 2.0 supports this now.

[jira] [Assigned] (SPARK-17177) Make grouping columns accessible from RelationalGroupedDataset

[ https://issues.apache.org/jira/browse/SPARK-17177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17177: Assignee: (was: Apache Spark) > Make grouping columns accessible from

[jira] [Assigned] (SPARK-17177) Make grouping columns accessible from RelationalGroupedDataset

[ https://issues.apache.org/jira/browse/SPARK-17177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17177: Assignee: Apache Spark > Make grouping columns accessible from RelationalGroupedDataset >

[jira] [Commented] (SPARK-17177) Make grouping columns accessible from RelationalGroupedDataset

[ https://issues.apache.org/jira/browse/SPARK-17177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429903#comment-15429903 ] Apache Spark commented on SPARK-17177: -- User 'NarineK' has created a pull request for this issue:

[jira] [Updated] (SPARK-17177) Make grouping columns accessible from RelationalGroupedDataset

[ https://issues.apache.org/jira/browse/SPARK-17177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Narine Kokhlikyan updated SPARK-17177: -- Component/s: SQL > Make grouping columns accessible from RelationalGroupedDataset >

[jira] [Created] (SPARK-17177) Make grouping columns accessible from RelationalGroupedDataset

Narine Kokhlikyan created SPARK-17177: - Summary: Make grouping columns accessible from RelationalGroupedDataset Key: SPARK-17177 URL: https://issues.apache.org/jira/browse/SPARK-17177 Project:

[jira] [Assigned] (SPARK-6832) Handle partial reads in SparkR JVM to worker communication

[ https://issues.apache.org/jira/browse/SPARK-6832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6832: --- Assignee: (was: Apache Spark) > Handle partial reads in SparkR JVM to worker

[jira] [Commented] (SPARK-6832) Handle partial reads in SparkR JVM to worker communication

[ https://issues.apache.org/jira/browse/SPARK-6832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429895#comment-15429895 ] Apache Spark commented on SPARK-6832: - User 'krishnakalyan3' has created a pull request for this

[jira] [Assigned] (SPARK-6832) Handle partial reads in SparkR JVM to worker communication

[ https://issues.apache.org/jira/browse/SPARK-6832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6832: --- Assignee: Apache Spark > Handle partial reads in SparkR JVM to worker communication >

[jira] [Resolved] (SPARK-17098) "SELECT COUNT(NULL) OVER ()" throws UnsupportedOperationException during analysis

[ https://issues.apache.org/jira/browse/SPARK-17098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-17098. --- Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 2.1.0

[jira] [Commented] (SPARK-16581) Making JVM backend calling functions public

[ https://issues.apache.org/jira/browse/SPARK-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429858#comment-15429858 ] Shivaram Venkataraman commented on SPARK-16581: --- These are good points -- The way I see it

[jira] [Assigned] (SPARK-16577) Add check-cran script to Jenkins

[ https://issues.apache.org/jira/browse/SPARK-16577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman reassigned SPARK-16577: - Assignee: Shivaram Venkataraman > Add check-cran script to Jenkins >

[jira] [Commented] (SPARK-16577) Add check-cran script to Jenkins

[ https://issues.apache.org/jira/browse/SPARK-16577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429850#comment-15429850 ] Shivaram Venkataraman commented on SPARK-16577: --- I just tested this last week on the

[jira] [Commented] (SPARK-17147) Spark Streaming Kafka 0.10 Consumer Can't Handle Non-consecutive Offsets

[ https://issues.apache.org/jira/browse/SPARK-17147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429846#comment-15429846 ] Robert Conrad commented on SPARK-17147: --- Nope, I started the project with kafka 0.10 and spark

[jira] [Commented] (SPARK-17094) provide simplified API for ML pipeline

[ https://issues.apache.org/jira/browse/SPARK-17094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429797#comment-15429797 ] Jacek Laskowski commented on SPARK-17094: - Are there any other pipelines? Please explain if you

[jira] [Updated] (SPARK-16578) Configurable hostname for RBackend

[ https://issues.apache.org/jira/browse/SPARK-16578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16578: -- Assignee: Junyang Qian > Configurable hostname for RBackend >

[jira] [Commented] (SPARK-17147) Spark Streaming Kafka 0.10 Consumer Can't Handle Non-consecutive Offsets

[ https://issues.apache.org/jira/browse/SPARK-17147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429742#comment-15429742 ] Cody Koeninger commented on SPARK-17147: Have you successfully used the 0.8 consumer with

[jira] [Commented] (SPARK-17175) Add a expert formula to aggregationDepth of SharedParam

[ https://issues.apache.org/jira/browse/SPARK-17175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429736#comment-15429736 ] Sean Owen commented on SPARK-17175: --- OK, can you clarify what this change would entail? I'm not sure if

[jira] [Commented] (SPARK-17024) Weird behaviour of the DataFrame when a column name contains dots.

[ https://issues.apache.org/jira/browse/SPARK-17024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429734#comment-15429734 ] Iaroslav Zeigerman commented on SPARK-17024: no reproduce on master. thanks a lot

[jira] [Closed] (SPARK-17024) Weird behaviour of the DataFrame when a column name contains dots.

[ https://issues.apache.org/jira/browse/SPARK-17024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Iaroslav Zeigerman closed SPARK-17024. -- Resolution: Cannot Reproduce > Weird behaviour of the DataFrame when a column name

[jira] [Commented] (SPARK-17175) Add a expert formula to aggregationDepth of SharedParam

[ https://issues.apache.org/jira/browse/SPARK-17175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429707#comment-15429707 ] Qian Huang commented on SPARK-17175: It needs to determine via some tests, you can see the details in

[jira] [Commented] (SPARK-16914) NodeManager crash when spark are registering executor infomartion into leveldb

[ https://issues.apache.org/jira/browse/SPARK-16914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429706#comment-15429706 ] Daniel Haviv commented on SPARK-16914: -- What if the path was ok on NM startup but became bad

[jira] [Comment Edited] (SPARK-17024) Weird behaviour of the DataFrame when a column name contains dots.

[ https://issues.apache.org/jira/browse/SPARK-17024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429659#comment-15429659 ] Hyukjin Kwon edited comment on SPARK-17024 at 8/21/16 11:26 AM: I can't

[jira] [Comment Edited] (SPARK-17024) Weird behaviour of the DataFrame when a column name contains dots.

[ https://issues.apache.org/jira/browse/SPARK-17024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429659#comment-15429659 ] Hyukjin Kwon edited comment on SPARK-17024 at 8/21/16 11:25 AM: I can't

[jira] [Commented] (SPARK-17172) pyspak hiveContext can not create UDF: Py4JJavaError: An error occurred while calling None.org.apache.spark.sql.hive.HiveContext.

[ https://issues.apache.org/jira/browse/SPARK-17172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429680#comment-15429680 ] Sean Owen commented on SPARK-17172: --- Yeah, is it the same error? the actual error doesn't seem to be

[jira] [Updated] (SPARK-17002) Document that spark.ssl.protocol. is required for SSL

[ https://issues.apache.org/jira/browse/SPARK-17002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17002: -- Assignee: Miao Wang > Document that spark.ssl.protocol. is required for SSL >

[jira] [Updated] (SPARK-17002) Document that spark.ssl.protocol. is required for SSL

[ https://issues.apache.org/jira/browse/SPARK-17002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17002: -- Priority: Trivial (was: Minor) > Document that spark.ssl.protocol. is required for SSL >

[jira] [Resolved] (SPARK-17002) Document that spark.ssl.protocol. is required for SSL

[ https://issues.apache.org/jira/browse/SPARK-17002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17002. --- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14674

[jira] [Commented] (SPARK-17024) Weird behaviour of the DataFrame when a column name contains dots.

[ https://issues.apache.org/jira/browse/SPARK-17024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429672#comment-15429672 ] Hyukjin Kwon commented on SPARK-17024: -- It seems I can't reproduce the example in the JIRA

[jira] [Comment Edited] (SPARK-17024) Weird behaviour of the DataFrame when a column name contains dots.

[ https://issues.apache.org/jira/browse/SPARK-17024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429659#comment-15429659 ] Hyukjin Kwon edited comment on SPARK-17024 at 8/21/16 10:23 AM: I can't

[jira] [Commented] (SPARK-17024) Weird behaviour of the DataFrame when a column name contains dots.

[ https://issues.apache.org/jira/browse/SPARK-17024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429659#comment-15429659 ] Hyukjin Kwon commented on SPARK-17024: -- I can't reproduce this against current master and suspect a

[jira] [Commented] (SPARK-17175) Add a expert formula to aggregationDepth of SharedParam

[ https://issues.apache.org/jira/browse/SPARK-17175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429658#comment-15429658 ] Sean Owen commented on SPARK-17175: --- What's an expert formula? > Add a expert formula to

[jira] [Commented] (SPARK-17168) CSV with header is incorrectly read if file is partitioned

[ https://issues.apache.org/jira/browse/SPARK-17168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429635#comment-15429635 ] Dongjoon Hyun commented on SPARK-17168: --- It's interesting. Usually, CSV files are not made by this

[jira] [Assigned] (SPARK-17176) Task are sorted by "Index" in Stage Page.

[ https://issues.apache.org/jira/browse/SPARK-17176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17176: Assignee: Apache Spark > Task are sorted by "Index" in Stage Page. >

[jira] [Assigned] (SPARK-17176) Task are sorted by "Index" in Stage Page.

[ https://issues.apache.org/jira/browse/SPARK-17176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17176: Assignee: (was: Apache Spark) > Task are sorted by "Index" in Stage Page. >

[jira] [Commented] (SPARK-17176) Task are sorted by "Index" in Stage Page.

[ https://issues.apache.org/jira/browse/SPARK-17176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429617#comment-15429617 ] Apache Spark commented on SPARK-17176: -- User 'cenyuhai' has created a pull request for this issue:

[jira] [Created] (SPARK-17176) Task are sorted by "Index" in Stage Page.

cen yuhai created SPARK-17176: - Summary: Task are sorted by "Index" in Stage Page. Key: SPARK-17176 URL: https://issues.apache.org/jira/browse/SPARK-17176 Project: Spark Issue Type: Improvement