[jira] [Commented] (SPARK-28338) spark.read.format("csv") treat empty string as null if csv file don't have quotes in data

[ https://issues.apache.org/jira/browse/SPARK-28338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882920#comment-16882920 ] Jayadevan M commented on SPARK-28338: - Hi [~maxgekk] Consider a specific example of SAP/ SQL server

[jira] [Updated] (SPARK-28338) spark.read.format("csv") treat empty string as null if csv file don't have quotes in data

[ https://issues.apache.org/jira/browse/SPARK-28338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayadevan M updated SPARK-28338: Description: The csv input file +cat sample.csv+ Name,Lastname,Age abc,,32 pqr,xxx,30  

[jira] [Updated] (SPARK-28338) spark.read.format("csv") treat empty string as null if csv file don't have quotes in data

[ https://issues.apache.org/jira/browse/SPARK-28338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayadevan M updated SPARK-28338: Summary: spark.read.format("csv") treat empty string as null if csv file don't have quotes in

[jira] [Updated] (SPARK-28338) spark.read.format("csv") treat empty string as null if csv file don't quotes in data

[ https://issues.apache.org/jira/browse/SPARK-28338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayadevan M updated SPARK-28338: Summary: spark.read.format("csv") treat empty string as null if csv file don't quotes in data

[jira] [Created] (SPARK-28338) spark.read.format("csv") treat empty string as null if csv file don't quotes in columns

Jayadevan M created SPARK-28338: --- Summary: spark.read.format("csv") treat empty string as null if csv file don't quotes in columns Key: SPARK-28338 URL: https://issues.apache.org/jira/browse/SPARK-28338

[jira] [Commented] (SPARK-19473) Several DataFrame Methods still fail with dot in column names

[ https://issues.apache.org/jira/browse/SPARK-19473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858239#comment-15858239 ] Jayadevan M commented on SPARK-19473: - I think this is a duplicate jira -

[jira] [Commented] (SPARK-17557) SQL query on parquet table java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary

[ https://issues.apache.org/jira/browse/SPARK-17557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15830247#comment-15830247 ] Jayadevan M commented on SPARK-17557: - Hi [~epahomov] I tried to replicate this issue using below

[jira] [Commented] (SPARK-19059) Unable to retrieve data from a parquet table whose name starts with underscore

[ https://issues.apache.org/jira/browse/SPARK-19059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828593#comment-15828593 ] Jayadevan M commented on SPARK-19059: - @uncleGen @Eric Liang @cloud-fan Could you please review this

[jira] [Commented] (SPARK-18120) QueryExecutionListener method doesnt' get executed for DataFrameWriter methods

[ https://issues.apache.org/jira/browse/SPARK-18120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15705744#comment-15705744 ] Jayadevan M commented on SPARK-18120: - [~thomastechs],[~salilsurendran][~rxin] Hi All, I am working

[jira] [Commented] (SPARK-18558) spark-csv: infer data type for mixed integer/null columns causes exception

[ https://issues.apache.org/jira/browse/SPARK-18558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15701508#comment-15701508 ] Jayadevan M commented on SPARK-18558: - [~PeterRose] I tried to replicate this issue, But did not get

[jira] [Commented] (SPARK-18097) Can't drop a table from Hive if the schema is corrupt

[ https://issues.apache.org/jira/browse/SPARK-18097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15654507#comment-15654507 ] Jayadevan M commented on SPARK-18097: - Hi [~davies] Can you give steps to reproduce this issue? How

[jira] [Commented] (SPARK-18251) DataSet API | RuntimeException: Null value appeared in non-nullable field when holding Option Case Class

[ https://issues.apache.org/jira/browse/SPARK-18251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15651228#comment-15651228 ] Jayadevan M commented on SPARK-18251: - Hi [~aniket] I tried to run your code. I am able to run

[jira] [Commented] (SPARK-18352) Parse normal, multi-line JSON files (not just JSON Lines)

[ https://issues.apache.org/jira/browse/SPARK-18352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15649803#comment-15649803 ] Jayadevan M commented on SPARK-18352: - [~rxin] Are you looking a new api like

[jira] [Commented] (SPARK-17691) Add aggregate function to collect list with maximum number of elements

[ https://issues.apache.org/jira/browse/SPARK-17691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15647445#comment-15647445 ] Jayadevan M commented on SPARK-17691: - I am interested to take look into this. > Add aggregate

[jira] [Comment Edited] (SPARK-16892) flatten function to get flat array (or map) column from array of array (or array of map) column

[ https://issues.apache.org/jira/browse/SPARK-16892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15646443#comment-15646443 ] Jayadevan M edited comment on SPARK-16892 at 11/8/16 4:21 AM: -- I hope you

[jira] [Commented] (SPARK-16892) flatten function to get flat array (or map) column from array of array (or array of map) column

[ https://issues.apache.org/jira/browse/SPARK-16892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15646443#comment-15646443 ] Jayadevan M commented on SPARK-16892: - I hope you can use flatMap for this scenario. For example var

[jira] [Commented] (SPARK-16894) take function for returning the first n elements of array column

[ https://issues.apache.org/jira/browse/SPARK-16894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15643081#comment-15643081 ] Jayadevan M commented on SPARK-16894: - [~kapilsingh5050] Can you tell me the use case for this >

[jira] [Commented] (SPARK-13342) Cannot run INSERT statements in Spark

[ https://issues.apache.org/jira/browse/SPARK-13342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15292874#comment-15292874 ] Jayadevan M commented on SPARK-13342: - Hi neo I tried to test this in 1.5.0. But I am not able to

[jira] [Issue Comment Deleted] (SPARK-14698) CREATE FUNCTION cloud not add function to hive metastore

[ https://issues.apache.org/jira/browse/SPARK-14698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayadevan M updated SPARK-14698: Comment: was deleted (was: Thanks [~poseidon]. I will post updates) > CREATE FUNCTION cloud not

[jira] [Commented] (SPARK-14698) CREATE FUNCTION cloud not add function to hive metastore

[ https://issues.apache.org/jira/browse/SPARK-14698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15291030#comment-15291030 ] Jayadevan M commented on SPARK-14698: - Thanks [~poseidon]. I will post updates > CREATE FUNCTION

[jira] [Issue Comment Deleted] (SPARK-14698) CREATE FUNCTION cloud not add function to hive metastore

[ https://issues.apache.org/jira/browse/SPARK-14698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayadevan M updated SPARK-14698: Comment: was deleted (was: Thanks [~poseidon]. I will post updates) > CREATE FUNCTION cloud not

[jira] [Commented] (SPARK-14698) CREATE FUNCTION cloud not add function to hive metastore

[ https://issues.apache.org/jira/browse/SPARK-14698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15291029#comment-15291029 ] Jayadevan M commented on SPARK-14698: - Thanks [~poseidon]. I will post updates > CREATE FUNCTION

[jira] [Commented] (SPARK-14698) CREATE FUNCTION cloud not add function to hive metastore

[ https://issues.apache.org/jira/browse/SPARK-14698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15274160#comment-15274160 ] Jayadevan M commented on SPARK-14698: - [~poseidon] I would like to work this issue. Can you give the

[jira] [Commented] (SPARK-12563) "No suitable driver" when calling JdbcUtils.saveTable in isolation

[ https://issues.apache.org/jira/browse/SPARK-12563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15274115#comment-15274115 ] Jayadevan M commented on SPARK-12563: - Ok. I think we can close this issue. > "No suitable driver"

[jira] [Commented] (SPARK-13441) NullPointerException when either HADOOP_CONF_DIR or YARN_CONF_DIR is not readable

[ https://issues.apache.org/jira/browse/SPARK-13441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159125#comment-15159125 ] Jayadevan M commented on SPARK-13441: - I think this approach is fine > NullPointerException when

[jira] [Commented] (SPARK-13441) NullPointerException when either HADOOP_CONF_DIR or YARN_CONF_DIR is not readable

[ https://issues.apache.org/jira/browse/SPARK-13441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15158645#comment-15158645 ] Jayadevan M commented on SPARK-13441: - Are you planning to create PR for this ?, Otherwise I will

[jira] [Commented] (SPARK-13441) NullPointerException when either HADOOP_CONF_DIR or YARN_CONF_DIR is not readable

[ https://issues.apache.org/jira/browse/SPARK-13441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15158458#comment-15158458 ] Jayadevan M commented on SPARK-13441: - @Terence Yim So, are you expecting any valid exception say "No

[jira] [Commented] (SPARK-12941) Spark-SQL JDBC Oracle dialect fails to map string datatypes to Oracle VARCHAR datatype

[ https://issues.apache.org/jira/browse/SPARK-12941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15157405#comment-15157405 ] Jayadevan M commented on SPARK-12941: - Working with [~thomastechs] for creating oracle docker suite

[jira] [Commented] (SPARK-13311) prettyString of IN is not good

[ https://issues.apache.org/jira/browse/SPARK-13311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148318#comment-15148318 ] Jayadevan M commented on SPARK-13311: - [~davies] I would like to work on this issue. Could you tell

[jira] [Commented] (SPARK-13268) SQL Timestamp stored as GMT but toString returns GMT-08:00

[ https://issues.apache.org/jira/browse/SPARK-13268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15142267#comment-15142267 ] Jayadevan M commented on SPARK-13268: - @Ilya Ganelin Can you tell how you import ZonedDateTime in

[jira] [Created] (SPARK-13197) When trying to select from the data frame which contains the columns with . in it, it is throwing exception.

Jayadevan M created SPARK-13197: --- Summary: When trying to select from the data frame which contains the columns with . in it, it is throwing exception. Key: SPARK-13197 URL:

[jira] [Issue Comment Deleted] (SPARK-12982) SQLContext: temporary table registration does not accept valid identifier

[ https://issues.apache.org/jira/browse/SPARK-12982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayadevan M updated SPARK-12982: Comment: was deleted (was: yes, was able to replicate.) > SQLContext: temporary table

[jira] [Commented] (SPARK-12982) SQLContext: temporary table registration does not accept valid identifier

[ https://issues.apache.org/jira/browse/SPARK-12982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15123389#comment-15123389 ] Jayadevan M commented on SPARK-12982: - yes, was able to replicate. > SQLContext: temporary table

[jira] [Commented] (SPARK-12982) SQLContext: temporary table registration does not accept valid identifier

[ https://issues.apache.org/jira/browse/SPARK-12982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15123388#comment-15123388 ] Jayadevan M commented on SPARK-12982: - yes, was able to replicate. > SQLContext: temporary table

[jira] [Comment Edited] (SPARK-12982) SQLContext: temporary table registration does not accept valid identifier

[ https://issues.apache.org/jira/browse/SPARK-12982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15123023#comment-15123023 ] Jayadevan M edited comment on SPARK-12982 at 1/29/16 9:03 AM: -- I tried

[jira] [Commented] (SPARK-12982) SQLContext: temporary table registration does not accept valid identifier

[ https://issues.apache.org/jira/browse/SPARK-12982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15123023#comment-15123023 ] Jayadevan M commented on SPARK-12982: - I tried your scenario in spark 1.6.0. But I can register table

[jira] [Commented] (SPARK-12987) Drop fails when columns contain dots

[ https://issues.apache.org/jira/browse/SPARK-12987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120922#comment-15120922 ] Jayadevan M commented on SPARK-12987: - I think the exception is occurred during select method

[jira] [Commented] (SPARK-12941) Spark-SQL JDBC Oracle dialect fails to map string datatypes to Oracle VARCHAR datatype

[ https://issues.apache.org/jira/browse/SPARK-12941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115682#comment-15115682 ] Jayadevan M commented on SPARK-12941: - Working on JdbcDialect.scala > Spark-SQL JDBC Oracle dialect

[jira] [Commented] (SPARK-12941) Spark-SQL JDBC Oracle dialect fails to map string datatypes to Oracle VARCHAR datatype

[ https://issues.apache.org/jira/browse/SPARK-12941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110112#comment-15110112 ] Jayadevan M commented on SPARK-12941: - [~jpoblete] Would like to work on this. > Spark-SQL JDBC

[jira] [Commented] (SPARK-12911) Cacheing a dataframe causes array comparisons to fail (in filter / where) after 1.6

[ https://issues.apache.org/jira/browse/SPARK-12911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15108795#comment-15108795 ] Jayadevan M commented on SPARK-12911: - [~jesse.english] I just try following statments in

[jira] [Commented] (SPARK-11137) Make StreamingContext.stop() exception-safe

[ https://issues.apache.org/jira/browse/SPARK-11137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15105523#comment-15105523 ] Jayadevan M commented on SPARK-11137: - [~thomastechs] working on the fix > Make

[jira] [Commented] (SPARK-12262) describe extended doesn't return table on detail info tabled stored as PARQUET format

[ https://issues.apache.org/jira/browse/SPARK-12262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15098027#comment-15098027 ] Jayadevan M commented on SPARK-12262: - Yea Agreed. This workaround is working fine. But why FORMATTED

[jira] [Commented] (SPARK-12825) Spark-submit Jar URL loading fails on redirect

[ https://issues.apache.org/jira/browse/SPARK-12825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15098139#comment-15098139 ] Jayadevan M commented on SPARK-12825: - Can you tell the full command. > Spark-submit Jar URL

[jira] [Commented] (SPARK-11139) Make SparkContext.stop() exception-safe

[ https://issues.apache.org/jira/browse/SPARK-11139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15085135#comment-15085135 ] Jayadevan M commented on SPARK-11139: - [~felixcheung] Is this issue resolved ? If yes can you tell

[jira] [Commented] (SPARK-12563) "No suitable driver" when calling JdbcUtils.saveTable in isolation

[ https://issues.apache.org/jira/browse/SPARK-12563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15081050#comment-15081050 ] Jayadevan M commented on SPARK-12563: - [~sonya] I would like to work this. I think we need use

[jira] [Commented] (SPARK-12622) spark-submit fails on executors when jar has a space in it

[ https://issues.apache.org/jira/browse/SPARK-12622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15082175#comment-15082175 ] Jayadevan M commented on SPARK-12622: - [~abridgett]@Adrian Bridgett do you think is this a valid

[jira] [Commented] (SPARK-12589) result row size is wrong in UnsafeRowParquetRecordReader

[ https://issues.apache.org/jira/browse/SPARK-12589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15080768#comment-15080768 ] Jayadevan M commented on SPARK-12589: - @ Wenchen Fan Can you please elaborate more ? How can I

[jira] [Commented] (SPARK-12533) hiveContext.table() throws the wrong exception

[ https://issues.apache.org/jira/browse/SPARK-12533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15076327#comment-15076327 ] Jayadevan M commented on SPARK-12533: - Thomas, Updated my thoughts in the pull request. Pls check. >

[jira] [Commented] (SPARK-12262) describe extended doesn't return table on detail info tabled stored as PARQUET format

[ https://issues.apache.org/jira/browse/SPARK-12262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061863#comment-15061863 ] Jayadevan M commented on SPARK-12262: - I was able to replicate this issue. I would like to work on