[jira] [Created] (SPARK-41084) Support Bit manipulation function SHIFTRIGHT

2022-11-09 Thread Vinod KC (Jira)
Vinod KC created SPARK-41084: Summary: Support Bit manipulation function SHIFTRIGHT Key: SPARK-41084 URL: https://issues.apache.org/jira/browse/SPARK-41084 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-41083) Support Bit manipulation function SHIFTLEFT

2022-11-09 Thread Vinod KC (Jira)
Vinod KC created SPARK-41083: Summary: Support Bit manipulation function SHIFTLEFT Key: SPARK-41083 URL: https://issues.apache.org/jira/browse/SPARK-41083 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-41082) Support Bit manipulation function ROTATERIGHT

2022-11-09 Thread Vinod KC (Jira)
Vinod KC created SPARK-41082: Summary: Support Bit manipulation function ROTATERIGHT Key: SPARK-41082 URL: https://issues.apache.org/jira/browse/SPARK-41082 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-41081) Support Bit manipulation function ROTATELEFT

2022-11-09 Thread Vinod KC (Jira)
Vinod KC created SPARK-41081: Summary: Support Bit manipulation function ROTATELEFT Key: SPARK-41081 URL: https://issues.apache.org/jira/browse/SPARK-41081 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-41079) Support Bit manipulation functions perform bitwise operations

2022-11-09 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-41079: - Description: Support Bit manipulation functions perform bitwise operations  COUNTSET  ROTATELEFT

[jira] [Updated] (SPARK-41079) Support Bit manipulation functions perform bitwise operations

2022-11-09 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-41079: - Description: Support Bit manipulation functions to perform bitwise operations  COUNTSET  ROTATELEFT

[jira] [Created] (SPARK-41080) Support Bit manipulation function SETBIT

2022-11-09 Thread Vinod KC (Jira)
Vinod KC created SPARK-41080: Summary: Support Bit manipulation function SETBIT Key: SPARK-41080 URL: https://issues.apache.org/jira/browse/SPARK-41080 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-41079) Support Bit manipulation functions perform bitwise operations

2022-11-09 Thread Vinod KC (Jira)
Vinod KC created SPARK-41079: Summary: Support Bit manipulation functions perform bitwise operations Key: SPARK-41079 URL: https://issues.apache.org/jira/browse/SPARK-41079 Project: Spark Issue

[jira] (SPARK-40686) Support data masking and redacting built-in functions

2022-10-28 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40686 ] Vinod KC deleted comment on SPARK-40686: -- was (Author: vinodkc): I'm working on these 6 sub-tasks > Support data masking and redacting built-in functions >

[jira] [Updated] (SPARK-40686) Support data masking and redacting built-in functions

2022-10-28 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-40686: - Description: Support built-in data masking and redacting functions  (was: Support built-in data

[jira] [Commented] (SPARK-40686) Support data masking built-in functions

2022-10-28 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17625819#comment-17625819 ] Vinod KC commented on SPARK-40686: -- [~dtenedor] , Yes, please add them here. Let us change the

[jira] [Created] (SPARK-40945) Support built-in function to truncate numbers

2022-10-27 Thread Vinod KC (Jira)
Vinod KC created SPARK-40945: Summary: Support built-in function to truncate numbers Key: SPARK-40945 URL: https://issues.apache.org/jira/browse/SPARK-40945 Project: Spark Issue Type:

[jira] [Updated] (SPARK-40686) Support data masking built-in functions

2022-10-08 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-40686: - Summary: Support data masking built-in functions (was: Support data Masking built-in Functions) >

[jira] [Commented] (SPARK-40691) Support data masking built-in function 'mask_show_last_n'

2022-10-08 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17614508#comment-17614508 ] Vinod KC commented on SPARK-40691: -- I'm working on this sub task > Support data masking built-in

[jira] [Commented] (SPARK-40692) Support data masking built-in function 'mask_hash'

2022-10-08 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17614509#comment-17614509 ] Vinod KC commented on SPARK-40692: -- I'm working on this sub task > Support data masking built-in

[jira] [Commented] (SPARK-40690) Support data masking built-in function 'mask_show_first_n'

2022-10-08 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17614507#comment-17614507 ] Vinod KC commented on SPARK-40690: -- I'm working on this sub task > Support data masking built-in

[jira] [Commented] (SPARK-40689) Support data masking built-in function 'mask_last_n'

2022-10-08 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17614506#comment-17614506 ] Vinod KC commented on SPARK-40689: -- I'm working on this sub task > Support data masking built-in

[jira] [Commented] (SPARK-40688) Support data masking built-in function 'mask_first_n'

2022-10-08 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17614505#comment-17614505 ] Vinod KC commented on SPARK-40688: -- I'm working on this sub task > Support data masking built-in

[jira] [Updated] (SPARK-40689) Support data masking built-in function 'mask_last_n'

2022-10-06 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-40689: - Description: Support data masking built-in function *mask_last_n* Return a masked version of str with

[jira] [Comment Edited] (SPARK-40686) Support data Masking built-in Functions

2022-10-06 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17613819#comment-17613819 ] Vinod KC edited comment on SPARK-40686 at 10/7/22 12:34 AM: I'm working on

[jira] [Updated] (SPARK-40688) Support data masking built-in function 'mask_first_n'

2022-10-06 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-40688: - Description: Support data masking built-in function  *mask_first_n* Return a masked version of str

[jira] [Updated] (SPARK-40687) Support data masking built-in Function 'mask'

2022-10-06 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-40687: - Description: Support data masking built-in Function  *mask* Return a masked version of str. By

[jira] [Updated] (SPARK-40687) Support data masking built-in Function 'mask'

2022-10-06 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-40687: - Description: Support data masking built-in Function  *mask* Return a masked version of str. By

[jira] [Created] (SPARK-40692) Support data masking built-in function 'mask_hash'

2022-10-06 Thread Vinod KC (Jira)
Vinod KC created SPARK-40692: Summary: Support data masking built-in function 'mask_hash' Key: SPARK-40692 URL: https://issues.apache.org/jira/browse/SPARK-40692 Project: Spark Issue Type:

[jira] [Created] (SPARK-40691) Support data masking built-in function 'mask_show_last_n'

2022-10-06 Thread Vinod KC (Jira)
Vinod KC created SPARK-40691: Summary: Support data masking built-in function 'mask_show_last_n' Key: SPARK-40691 URL: https://issues.apache.org/jira/browse/SPARK-40691 Project: Spark Issue

[jira] [Commented] (SPARK-40686) Support data Masking built-in Functions

2022-10-06 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17613819#comment-17613819 ] Vinod KC commented on SPARK-40686: -- Im working on these sub-tasks > Support data Masking built-in

[jira] [Created] (SPARK-40690) Support data masking built-in function 'mask_show_first_n'

2022-10-06 Thread Vinod KC (Jira)
Vinod KC created SPARK-40690: Summary: Support data masking built-in function 'mask_show_first_n' Key: SPARK-40690 URL: https://issues.apache.org/jira/browse/SPARK-40690 Project: Spark Issue

[jira] [Created] (SPARK-40689) Support data masking built-in function 'mask_last_n'

2022-10-06 Thread Vinod KC (Jira)
Vinod KC created SPARK-40689: Summary: Support data masking built-in function 'mask_last_n' Key: SPARK-40689 URL: https://issues.apache.org/jira/browse/SPARK-40689 Project: Spark Issue Type:

[jira] [Updated] (SPARK-40687) Support data masking built-in Function 'mask'

2022-10-06 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-40687: - Description: Support data masking built-in Function  *mask* (was: Support data Masking built-in

[jira] [Updated] (SPARK-40687) Support data masking built-in Function - Mask

2022-10-06 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-40687: - Parent: SPARK-40686 Issue Type: Sub-task (was: Improvement) > Support data masking built-in

[jira] [Updated] (SPARK-40687) Support data masking built-in Function 'mask'

2022-10-06 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-40687: - Summary: Support data masking built-in Function 'mask' (was: Support data masking built-in Function -

[jira] [Created] (SPARK-40688) Support data masking built-in function 'mask_first_n'

2022-10-06 Thread Vinod KC (Jira)
Vinod KC created SPARK-40688: Summary: Support data masking built-in function 'mask_first_n' Key: SPARK-40688 URL: https://issues.apache.org/jira/browse/SPARK-40688 Project: Spark Issue Type:

[jira] [Created] (SPARK-40687) Support data masking built-in Function - Mask

2022-10-06 Thread Vinod KC (Jira)
Vinod KC created SPARK-40687: Summary: Support data masking built-in Function - Mask Key: SPARK-40687 URL: https://issues.apache.org/jira/browse/SPARK-40687 Project: Spark Issue Type:

[jira] [Created] (SPARK-40686) Support data Masking built-in Functions

2022-10-06 Thread Vinod KC (Jira)
Vinod KC created SPARK-40686: Summary: Support data Masking built-in Functions Key: SPARK-40686 URL: https://issues.apache.org/jira/browse/SPARK-40686 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-25177) When dataframe decimal type column having scale higher than 6, 0 values are shown in scientific notation

2022-04-27 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17528877#comment-17528877 ] Vinod KC edited comment on SPARK-25177 at 4/27/22 4:29 PM: --- In case, if anyone

[jira] [Commented] (SPARK-25177) When dataframe decimal type column having scale higher than 6, 0 values are shown in scientific notation

2022-04-27 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17528877#comment-17528877 ] Vinod KC commented on SPARK-25177: -- In case, if anyone looking for a workaround to convert  0 in

[jira] [Updated] (SPARK-35747) Avoid printing full Exception stack trace, if HBase/Kafka/Hive services are not running in a secure cluster

2021-06-17 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-35747: - Description: In a secure Yarn cluster, even though HBase or Kafka, or Hive services are not used in

[jira] [Updated] (SPARK-35747) Avoid printing full Exception stack trace, if HBase/Kafka/Hive services are not running in a secure cluster

2021-06-17 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-35747: - Description: In a secure Yarn cluster, even though HBase or Kafka, or Hive services are not used in

[jira] [Updated] (SPARK-35747) Avoid printing full Exception stack trace, if HBase/Kafka/Hive services are not running in a secure cluster

2021-06-17 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-35747: - Summary: Avoid printing full Exception stack trace, if HBase/Kafka/Hive services are not running in a

[jira] [Updated] (SPARK-35747) Avoid printing full Exception stack trace, if HBase service is not running in a secure cluster

2021-06-12 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-35747: - Description: In a secure Yarn cluster where HBase service is down, even if the spark application is

[jira] [Created] (SPARK-35747) Avoid printing full Exception stack trace, if HBase service is not running in a secure cluster

2021-06-12 Thread Vinod KC (Jira)
Vinod KC created SPARK-35747: Summary: Avoid printing full Exception stack trace, if HBase service is not running in a secure cluster Key: SPARK-35747 URL: https://issues.apache.org/jira/browse/SPARK-35747

[jira] [Commented] (SPARK-35517) Critical Vulnerabilities: jackson-databind 2.4.0 shipped with htrace-core4-4.1.0-incubating.jar

2021-05-25 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351134#comment-17351134 ] Vinod KC commented on SPARK-35517: -- [~ldeflandre], In Spark 3.2.0, SPARK-34784 upgraded,

[jira] [Created] (SPARK-35513) Upgrade joda-time to 2.10.10

2021-05-25 Thread Vinod KC (Jira)
Vinod KC created SPARK-35513: Summary: Upgrade joda-time to 2.10.10 Key: SPARK-35513 URL: https://issues.apache.org/jira/browse/SPARK-35513 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-35490) Update json4s to 3.7.0-M11

2021-05-25 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-35490: - Summary: Update json4s to 3.7.0-M11 (was: Update json4s to 3.7.0-M16) > Update json4s to 3.7.0-M11 >

[jira] [Created] (SPARK-35492) Upgrade Apache HttpCore from 4.4.12 to 4.4.14

2021-05-23 Thread Vinod KC (Jira)
Vinod KC created SPARK-35492: Summary: Upgrade Apache HttpCore from 4.4.12 to 4.4.14 Key: SPARK-35492 URL: https://issues.apache.org/jira/browse/SPARK-35492 Project: Spark Issue Type:

[jira] [Created] (SPARK-35490) Update json4s to 3.7.0-M16

2021-05-22 Thread Vinod KC (Jira)
Vinod KC created SPARK-35490: Summary: Update json4s to 3.7.0-M16 Key: SPARK-35490 URL: https://issues.apache.org/jira/browse/SPARK-35490 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-35488) Upgrade ASM to 7.3.1

2021-05-22 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-35488: - Priority: Minor (was: Major) > Upgrade ASM to 7.3.1 > > > Key:

[jira] [Created] (SPARK-35488) Upgrade ASM to 7.3.1

2021-05-22 Thread Vinod KC (Jira)
Vinod KC created SPARK-35488: Summary: Upgrade ASM to 7.3.1 Key: SPARK-35488 URL: https://issues.apache.org/jira/browse/SPARK-35488 Project: Spark Issue Type: Improvement Components:

[jira] [Commented] (SPARK-26825) Spark Structure Streaming job failing when submitted in cluster mode

2020-11-10 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17229719#comment-17229719 ] Vinod KC commented on SPARK-26825: -- As a workaround, we can set a {{checkpointLocation to stream.}}  

[jira] [Commented] (SPARK-32938) Spark can not cast long value from Kafka

2020-09-22 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17200028#comment-17200028 ] Vinod KC commented on SPARK-32938: -- [~maseiler],  Can you please test with this example?  {code:java}

[jira] [Commented] (SPARK-32635) When pyspark.sql.functions.lit() function is used with dataframe cache, it returns wrong result

2020-08-21 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181800#comment-17181800 ] Vinod KC commented on SPARK-32635: -- [~sascha.baumanns], updated code section. Thanks > When

[jira] [Updated] (SPARK-32635) When pyspark.sql.functions.lit() function is used with dataframe cache, it returns wrong result

2020-08-21 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-32635: - Description: When pyspark.sql.functions.lit() function is used with dataframe cache, it returns wrong

[jira] [Comment Edited] (SPARK-20880) When spark SQL is used with Avro-backed HIVE tables, NPE from org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.supportedCategories.

2020-08-19 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17180307#comment-17180307 ] Vinod KC edited comment on SPARK-20880 at 8/19/20, 6:55 AM:

[jira] [Comment Edited] (SPARK-20880) When spark SQL is used with Avro-backed HIVE tables, NPE from org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.supportedCategories.

2020-08-19 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17180307#comment-17180307 ] Vinod KC edited comment on SPARK-20880 at 8/19/20, 6:54 AM: SPARK-29981  has

[jira] [Comment Edited] (SPARK-20880) When spark SQL is used with Avro-backed HIVE tables, NPE from org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.supportedCategories.

2020-08-19 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17180307#comment-17180307 ] Vinod KC edited comment on SPARK-20880 at 8/19/20, 6:53 AM: SPARK-29981  has

[jira] [Commented] (SPARK-20880) When spark SQL is used with Avro-backed HIVE tables, NPE from org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.supportedCategories.

2020-08-19 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17180307#comment-17180307 ] Vinod KC commented on SPARK-20880: -- SPARK-29981 has added support to use hive-2.3 in Spark 3,  with

[jira] [Updated] (SPARK-20880) When spark SQL is used with Avro-backed HIVE tables, NPE from org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.supportedCategories.

2020-08-19 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-20880: - Description: When spark SQL is used with Avro-backed HIVE tables, intermittently getting NPE from

[jira] [Updated] (SPARK-20880) When spark SQL is used with Avro-backed HIVE tables, NPE from org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.supportedCategories.

2020-08-19 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-20880: - Description: When spark SQL is used with Avro-backed HIVE tables, intermittently getting NPE from

[jira] [Updated] (SPARK-32635) When pyspark.sql.functions.lit() function is used with dataframe cache, it returns wrong result

2020-08-17 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-32635: - Description: When pyspark.sql.functions.lit() function is used with dataframe cache, it returns wrong

[jira] [Created] (SPARK-32635) When pyspark.sql.functions.lit() function is used with dataframe cache, it returns wrong result

2020-08-17 Thread Vinod KC (Jira)
Vinod KC created SPARK-32635: Summary: When pyspark.sql.functions.lit() function is used with dataframe cache, it returns wrong result Key: SPARK-32635 URL: https://issues.apache.org/jira/browse/SPARK-32635

[jira] [Comment Edited] (SPARK-28287) Convert and port 'udaf.sql' into UDF test base

2019-07-11 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882743#comment-16882743 ] Vinod KC edited comment on SPARK-28287 at 7/11/19 8:31 AM: --- [~skonto]I'm

[jira] [Commented] (SPARK-28287) Convert and port 'udaf.sql' into UDF test base

2019-07-11 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882743#comment-16882743 ] Vinod KC commented on SPARK-28287: -- [~skonto]I'm sorry, without checking your comment I've raised a PR.

[jira] [Updated] (SPARK-25301) When a view uses an UDF from a non default database, Spark analyser throws AnalysisException

2018-09-04 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-25301: - Description: When a hive view uses an UDF from a non default database, Spark analyser throws

[jira] [Updated] (SPARK-25301) When a view uses an UDF from a non default database, Spark analyser throws AnalysisException

2018-09-04 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-25301: - Description: When a hive view uses an UDF from a non default database, Spark analyser throws

[jira] [Updated] (SPARK-25301) When a view uses an UDF from a non default database, Spark analyser throws AnalysisException

2018-09-02 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-25301: - Description: When a hive view uses an UDF from a non default database, Spark analyser throws

[jira] [Created] (SPARK-25301) When a view uses an UDF from a non default database, Spark analyser throws AnalysisException

2018-08-31 Thread Vinod KC (JIRA)
Vinod KC created SPARK-25301: Summary: When a view uses an UDF from a non default database, Spark analyser throws AnalysisException Key: SPARK-25301 URL: https://issues.apache.org/jira/browse/SPARK-25301

[jira] [Updated] (SPARK-25177) When dataframe decimal type column having scale higher than 6, 0 values are shown in scientific notation

2018-08-21 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-25177: - Description: If scale of decimal type is > 6 , 0 value will be shown in scientific notation and hence,

[jira] [Updated] (SPARK-25177) When dataframe decimal type column having scale higher than 6, 0 values are shown in scientific notation

2018-08-21 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-25177: - Description: If scale of decimal type is > 6 , 0 value will be shown in scientific notation and hence,

[jira] [Updated] (SPARK-25177) When dataframe decimal type column having scale higher than 6, 0 values are shown in scientific notation

2018-08-21 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-25177: - Description: If scale of decimal type is > 6 , 0 value will be shown in scientific notation and hence,

[jira] [Updated] (SPARK-25177) When dataframe decimal type column having scale higher than 6, 0 values are shown in scientific notation

2018-08-21 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-25177: - Summary: When dataframe decimal type column having scale higher than 6, 0 values are shown in

[jira] [Created] (SPARK-25177) When dataframe decimal type column having a scale higher than 6, 0 values are shown in scientific notation

2018-08-21 Thread Vinod KC (JIRA)
Vinod KC created SPARK-25177: Summary: When dataframe decimal type column having a scale higher than 6, 0 values are shown in scientific notation Key: SPARK-25177 URL:

[jira] [Created] (SPARK-25137) NumberFormatException` when starting spark-shell from Mac terminal

2018-08-16 Thread Vinod KC (JIRA)
Vinod KC created SPARK-25137: Summary: NumberFormatException` when starting spark-shell from Mac terminal Key: SPARK-25137 URL: https://issues.apache.org/jira/browse/SPARK-25137 Project: Spark

[jira] [Commented] (SPARK-21756) Add JSON option to allow unquoted control characters

2017-08-20 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134515#comment-16134515 ] Vinod KC commented on SPARK-21756: -- https://github.com/apache/spark/pull/19008 > Add JSON option to

[jira] [Commented] (SPARK-21756) Add JSON option to allow unquoted control characters

2017-08-20 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134484#comment-16134484 ] Vinod KC commented on SPARK-21756: -- [~bryanch] I'm planning to work on this . Can you please share your

[jira] [Commented] (SPARK-21684) df.write double escaping all the already escaped characters except the first one

2017-08-16 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128960#comment-16128960 ] Vinod KC commented on SPARK-21684: -- [~taransaini43] Can try with option("quoteAll", "true") ? Eg :

[jira] [Updated] (SPARK-21665) Need to close resources after use

2017-08-08 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-21665: - Description: Some methods in Core - SparkSubmitArguments.scala, Spark-launcher -

[jira] [Commented] (SPARK-21665) Need to close resources after use

2017-08-08 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118074#comment-16118074 ] Vinod KC commented on SPARK-21665: -- [~sowen] Can you please review this PR :

[jira] [Created] (SPARK-21665) Need to close resources after use

2017-08-08 Thread Vinod KC (JIRA)
Vinod KC created SPARK-21665: Summary: Need to close resources after use Key: SPARK-21665 URL: https://issues.apache.org/jira/browse/SPARK-21665 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-21588) SQLContext.getConf(key, null) should return null, but it throws NPE

2017-08-05 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115625#comment-16115625 ] Vinod KC commented on SPARK-21588: -- https://github.com/apache/spark/pull/18852 >

[jira] [Commented] (SPARK-21390) Dataset filter api inconsistency

2017-08-01 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108880#comment-16108880 ] Vinod KC commented on SPARK-21390: -- [~kiszk]case class is compiled differently in the spark shell . For

[jira] [Commented] (SPARK-21588) SQLContext.getConf(key, null) should return null, but it throws NPE

2017-08-01 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108622#comment-16108622 ] Vinod KC commented on SPARK-21588: -- [~brkyvz] Can you share sample code and NPE stack trace? >

[jira] [Commented] (SPARK-21478) Unpersist a DF also unpersists related DFs

2017-07-29 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106203#comment-16106203 ] Vinod KC commented on SPARK-21478: -- https://github.com/apache/spark/pull/18771 > Unpersist a DF also

[jira] [Updated] (SPARK-20880) When spark SQL is used with Avro-backed HIVE tables, NPE from org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.supportedCategories.

2017-05-25 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-20880: - Description: When spark SQL is used with Avro-backed HIVE tables, intermittently getting NPE from

[jira] [Created] (SPARK-20880) When spark SQL is used with Avro-backed HIVE tables, NPE from org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.supportedCategories.

2017-05-25 Thread Vinod KC (JIRA)
Vinod KC created SPARK-20880: Summary: When spark SQL is used with Avro-backed HIVE tables, NPE from org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.supportedCategories. Key: SPARK-20880 URL:

[jira] [Commented] (SPARK-10631) Add missing API doc in pyspark.mllib.linalg.Vector

2015-09-16 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14746994#comment-14746994 ] Vinod KC commented on SPARK-10631: -- I'm working on this > Add missing API doc in

[jira] [Created] (SPARK-10575) Wrap RDD.takeSample with scope

2015-09-12 Thread Vinod KC (JIRA)
Vinod KC created SPARK-10575: Summary: Wrap RDD.takeSample with scope Key: SPARK-10575 URL: https://issues.apache.org/jira/browse/SPARK-10575 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-10468) Verify schema before Dataframe select API call

2015-09-06 Thread Vinod KC (JIRA)
Vinod KC created SPARK-10468: Summary: Verify schema before Dataframe select API call Key: SPARK-10468 URL: https://issues.apache.org/jira/browse/SPARK-10468 Project: Spark Issue Type:

[jira] [Commented] (SPARK-10414) DenseMatrix gives different hashcode even though equals returns true

2015-09-04 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14731757#comment-14731757 ] Vinod KC commented on SPARK-10414: -- Thanks Got the JIRA id

[jira] [Commented] (SPARK-10414) DenseMatrix gives different hashcode even though equals returns true

2015-09-04 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14731745#comment-14731745 ] Vinod KC commented on SPARK-10414: -- [~josephkb] Could you please share me that existing JIRA id to

[jira] [Commented] (SPARK-10199) Avoid using reflections for parquet model save

2015-09-04 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14731753#comment-14731753 ] Vinod KC commented on SPARK-10199: -- [~mengxr] Thanks for the suggestion. Shall I close the PR? > Avoid

[jira] [Created] (SPARK-10414) DenseMatrix gives different hashcode even though equals returns true

2015-09-02 Thread Vinod KC (JIRA)
Vinod KC created SPARK-10414: Summary: DenseMatrix gives different hashcode even though equals returns true Key: SPARK-10414 URL: https://issues.apache.org/jira/browse/SPARK-10414 Project: Spark

[jira] [Created] (SPARK-10430) AccumulableInfo and RDDOperationScope violates hashCode + equals contract

2015-09-02 Thread Vinod KC (JIRA)
Vinod KC created SPARK-10430: Summary: AccumulableInfo and RDDOperationScope violates hashCode + equals contract Key: SPARK-10430 URL: https://issues.apache.org/jira/browse/SPARK-10430 Project: Spark

[jira] [Commented] (SPARK-10199) Avoid using reflections for parquet model save

2015-09-02 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14728427#comment-14728427 ] Vinod KC commented on SPARK-10199: -- [~mengxr] could you please check the above micro-benchmarks and give

[jira] [Comment Edited] (SPARK-10199) Avoid using reflections for parquet model save

2015-09-01 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14724459#comment-14724459 ] Vinod KC edited comment on SPARK-10199 at 9/1/15 6:57 AM: -- [~mengxr] 1) I

[jira] [Comment Edited] (SPARK-10199) Avoid using reflections for parquet model save

2015-09-01 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14725211#comment-14725211 ] Vinod KC edited comment on SPARK-10199 at 9/1/15 2:15 PM: -- [~mengxr] I've

[jira] [Commented] (SPARK-10199) Avoid using reflections for parquet model save

2015-09-01 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14725211#comment-14725211 ] Vinod KC commented on SPARK-10199: -- [~mengxr] I've measured the overhead of reflexion in save/load

[jira] [Commented] (SPARK-10199) Avoid using reflections for parquet model save

2015-08-31 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14724459#comment-14724459 ] Vinod KC commented on SPARK-10199: -- [~mengxr] 1) I measured only schema inference part. Now, I will

[jira] [Commented] (SPARK-10199) Avoid using reflections for parquet model save

2015-08-30 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14721491#comment-14721491 ] Vinod KC commented on SPARK-10199: -- [~fliang] , As you suggested, 1) I've made

[jira] [Commented] (SPARK-10202) Specify schema during KMeansModel.save to avoid reflection

2015-08-25 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14712456#comment-14712456 ] Vinod KC commented on SPARK-10202: -- I'm working on this Specify schema during

[jira] [Commented] (SPARK-10203) Specify schema during GLMClassificationModel.save to avoid reflection

2015-08-25 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14712458#comment-14712458 ] Vinod KC commented on SPARK-10203: -- I'm working on this Specify schema during

<    1   2   3   >