[jira] [Created] (SPARK-31017) Test for shuffle requests packaging with different size and numBlocks limit

wuyi created SPARK-31017: Summary: Test for shuffle requests packaging with different size and numBlocks limit Key: SPARK-31017 URL: https://issues.apache.org/jira/browse/SPARK-31017 Project: Spark

[jira] [Updated] (SPARK-31016) [DEPLOY] Pack the user jars when submitting Spark Application

[ https://issues.apache.org/jira/browse/SPARK-31016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-31016: Description: Nowadays, Spark only pack the jars under $SPARK_HOME/jars. How about packing the user jars

[jira] [Created] (SPARK-31016) [DEPLOY] Pack the user jars when submitting Spark Application

feiwang created SPARK-31016: --- Summary: [DEPLOY] Pack the user jars when submitting Spark Application Key: SPARK-31016 URL: https://issues.apache.org/jira/browse/SPARK-31016 Project: Spark Issue

[jira] [Updated] (SPARK-16872) Impl Gaussian Naive Bayes Classifier

[ https://issues.apache.org/jira/browse/SPARK-16872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-16872: - Fix Version/s: (was: 3.1.0) 3.0.0 > Impl Gaussian Naive Bayes Classifier

[jira] [Updated] (SPARK-31009) Support json_object_keys function

[ https://issues.apache.org/jira/browse/SPARK-31009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh Raushan updated SPARK-31009: --- Description: This function will return all the keys from outer json object.   PostgreSQL 

[jira] [Commented] (SPARK-31009) Support json_object_keys function

[ https://issues.apache.org/jira/browse/SPARK-31009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049914#comment-17049914 ] Rakesh Raushan commented on SPARK-31009: [~hyukjin.kwon] I updated the description. PostgreSQL

[jira] [Updated] (SPARK-31009) Support json_object_keys function

[ https://issues.apache.org/jira/browse/SPARK-31009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh Raushan updated SPARK-31009: --- Description: This function will return all the keys from outer json object.   PostgreSQL

[jira] [Commented] (SPARK-30980) Issue not resolved of Caught Hive MetaException attempting to get partition metadata by filter from Hive

[ https://issues.apache.org/jira/browse/SPARK-30980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049911#comment-17049911 ] Pradyumn Agrawal commented on SPARK-30980: -- Issue is I have table dual partitioned on timestamp

[jira] [Resolved] (SPARK-30948) Sharing the same external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-30948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30948. -- Resolution: Invalid Please ask questions into mailing list

[jira] [Commented] (SPARK-30951) Potential data loss for legacy applications after switch to proleptic Gregorian calendar

[ https://issues.apache.org/jira/browse/SPARK-30951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049894#comment-17049894 ] Hyukjin Kwon commented on SPARK-30951: -- FYI [~cloud_fan], [~maxgekk], [~XuanYuan] > Potential data

[jira] [Resolved] (SPARK-30952) Grouped pandas_udf crashed when a group returned an empty DataFrame

[ https://issues.apache.org/jira/browse/SPARK-30952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30952. -- Resolution: Cannot Reproduce Seems fixed in the Spark master, or upper version combinations

[jira] [Commented] (SPARK-30957) Null-safe variant of Dataset.join(Dataset[_], Seq[String])

[ https://issues.apache.org/jira/browse/SPARK-30957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049890#comment-17049890 ] Hyukjin Kwon commented on SPARK-30957: -- I currently don't think this is particularly useful. We

[jira] [Resolved] (SPARK-30957) Null-safe variant of Dataset.join(Dataset[_], Seq[String])

[ https://issues.apache.org/jira/browse/SPARK-30957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30957. -- Resolution: Won't Fix > Null-safe variant of Dataset.join(Dataset[_], Seq[String]) >

[jira] [Resolved] (SPARK-30959) How to write using JDBC driver to SQL Server / Azure DWH to column of BINARY type?

[ https://issues.apache.org/jira/browse/SPARK-30959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30959. -- Resolution: Invalid [~Ceridan] please ask questions to mailing list or stackoverflow (see

[jira] [Updated] (SPARK-30961) Arrow enabled: to_pandas with date column fails

[ https://issues.apache.org/jira/browse/SPARK-30961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30961: - Target Version/s: (was: 2.4.6) > Arrow enabled: to_pandas with date column fails >

[jira] [Commented] (SPARK-30965) Support C ++ library to load Spark MLlib model

[ https://issues.apache.org/jira/browse/SPARK-30965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049886#comment-17049886 ] Hyukjin Kwon commented on SPARK-30965: -- [~dxwang] can you fill the JIRA description? > Support C

[jira] [Commented] (SPARK-30967) Achieve LAST_ACCESS_TIME column update in TBLS table of hive metastore on hive table access through pyspark

[ https://issues.apache.org/jira/browse/SPARK-30967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049885#comment-17049885 ] Hyukjin Kwon commented on SPARK-30967: -- Questions should better ask to mailing list (see

[jira] [Resolved] (SPARK-30967) Achieve LAST_ACCESS_TIME column update in TBLS table of hive metastore on hive table access through pyspark

[ https://issues.apache.org/jira/browse/SPARK-30967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30967. -- Resolution: Invalid > Achieve LAST_ACCESS_TIME column update in TBLS table of hive metastore

[jira] [Resolved] (SPARK-30974) org.apache.spark.sql.AnalysisException: expression 'default.udfvalidation.`empname`' is neither present in the group by, nor is it an aggregate function.

[ https://issues.apache.org/jira/browse/SPARK-30974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30974. -- Resolution: Invalid > org.apache.spark.sql.AnalysisException: expression >

[jira] [Updated] (SPARK-30989) TABLE.COLUMN reference doesn't work with new columns created by UDF

[ https://issues.apache.org/jira/browse/SPARK-30989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30989: - Component/s: (was: Spark Core) SQL > TABLE.COLUMN reference doesn't work

[jira] [Comment Edited] (SPARK-31011) Failed to register signal handler for PWR

[ https://issues.apache.org/jira/browse/SPARK-31011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049874#comment-17049874 ] Hyukjin Kwon edited comment on SPARK-31011 at 3/3/20 4:13 AM: -- [~gsomogyi],

[jira] [Resolved] (SPARK-30980) Issue not resolved of Caught Hive MetaException attempting to get partition metadata by filter from Hive

[ https://issues.apache.org/jira/browse/SPARK-30980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30980. -- Resolution: Cannot Reproduce > Issue not resolved of Caught Hive MetaException attempting to

[jira] [Commented] (SPARK-30980) Issue not resolved of Caught Hive MetaException attempting to get partition metadata by filter from Hive

[ https://issues.apache.org/jira/browse/SPARK-30980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049881#comment-17049881 ] Hyukjin Kwon commented on SPARK-30980: -- Please just don't copy and paste the error message. Could

[jira] [Commented] (SPARK-31011) Failed to register signal handler for PWR

[ https://issues.apache.org/jira/browse/SPARK-31011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049880#comment-17049880 ] Jungtaek Lim commented on SPARK-31011: -- According to the wikipedia, SIGPWR is NOT listed in the

[jira] [Comment Edited] (SPARK-31011) Failed to register signal handler for PWR

[ https://issues.apache.org/jira/browse/SPARK-31011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049880#comment-17049880 ] Jungtaek Lim edited comment on SPARK-31011 at 3/3/20 4:11 AM: --

[jira] [Commented] (SPARK-31009) Support json_object_keys function

[ https://issues.apache.org/jira/browse/SPARK-31009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049877#comment-17049877 ] Hyukjin Kwon commented on SPARK-31009: -- [~rakson] can you show other references from other DBMSs? I

[jira] [Resolved] (SPARK-30890) Arrange version info of history

[ https://issues.apache.org/jira/browse/SPARK-30890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30890. -- Resolution: Duplicate > Arrange version info of history > --- > >

[jira] [Commented] (SPARK-30890) Arrange version info of history

[ https://issues.apache.org/jira/browse/SPARK-30890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049875#comment-17049875 ] jiaan.geng commented on SPARK-30890: This is duplicate with 

[jira] [Commented] (SPARK-31011) Failed to register signal handler for PWR

[ https://issues.apache.org/jira/browse/SPARK-31011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049874#comment-17049874 ] Hyukjin Kwon commented on SPARK-31011: -- [~gsomogyi], can you show the comments you used for

[jira] [Created] (SPARK-31015) Star(*) expression fails when used with fully qualified column names for v2 tables

Terry Kim created SPARK-31015: - Summary: Star(*) expression fails when used with fully qualified column names for v2 tables Key: SPARK-31015 URL: https://issues.apache.org/jira/browse/SPARK-31015

[jira] [Commented] (SPARK-30890) Arrange version info of history

[ https://issues.apache.org/jira/browse/SPARK-30890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049871#comment-17049871 ] Hyukjin Kwon commented on SPARK-30890: -- [~beliefer] can you set the parent JIRA properly? >

[jira] [Created] (SPARK-31014) InMemoryStore: CountingRemoveIfForEach misses to remove key from parentToChildrenMap

Jungtaek Lim created SPARK-31014: Summary: InMemoryStore: CountingRemoveIfForEach misses to remove key from parentToChildrenMap Key: SPARK-31014 URL: https://issues.apache.org/jira/browse/SPARK-31014

[jira] [Commented] (SPARK-29969) parse_url function result in incorrect result

[ https://issues.apache.org/jira/browse/SPARK-29969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049806#comment-17049806 ] YoungGyu Chun commented on SPARK-29969: --- Thank you [~dongjoon] [~xiaoxigua] [~hyukjin.kwon] >

[jira] [Resolved] (SPARK-30991) Refactor AQE readers and RDDs

[ https://issues.apache.org/jira/browse/SPARK-30991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-30991. - Fix Version/s: 3.0.0 Resolution: Fixed > Refactor AQE readers and RDDs >

[jira] [Assigned] (SPARK-30991) Refactor AQE readers and RDDs

[ https://issues.apache.org/jira/browse/SPARK-30991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-30991: --- Assignee: Wei Xue > Refactor AQE readers and RDDs > - > >

[jira] [Resolved] (SPARK-31003) Fix incorrect use of assume() in tests

[ https://issues.apache.org/jira/browse/SPARK-31003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31003. --- Fix Version/s: 2.4.6 3.0.0 Resolution: Fixed This is resolved via

[jira] [Updated] (SPARK-31003) Fix incorrect use of assume() in tests

[ https://issues.apache.org/jira/browse/SPARK-31003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31003: -- Issue Type: Bug (was: Improvement) > Fix incorrect use of assume() in tests >

[jira] [Updated] (SPARK-30993) GenerateUnsafeRowJoiner corrupts the value if the datatype is UDF and its sql type has fixed length

[ https://issues.apache.org/jira/browse/SPARK-30993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30993: -- Affects Version/s: 2.3.0 2.3.1 2.3.2

[jira] [Updated] (SPARK-30447) Constant propagation nullability issue

[ https://issues.apache.org/jira/browse/SPARK-30447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30447: -- Affects Version/s: 2.3.0 2.3.1 2.3.2

[jira] [Comment Edited] (SPARK-27530) FetchFailedException: Received a zero-size buffer for block shuffle

[ https://issues.apache.org/jira/browse/SPARK-27530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049703#comment-17049703 ] Gabriel Church edited comment on SPARK-27530 at 3/2/20 10:15 PM: - This

[jira] [Updated] (SPARK-30082) Zeros are being treated as NaNs

[ https://issues.apache.org/jira/browse/SPARK-30082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30082: -- Affects Version/s: 2.0.2 2.1.3 2.2.3

[jira] [Comment Edited] (SPARK-27530) FetchFailedException: Received a zero-size buffer for block shuffle

[ https://issues.apache.org/jira/browse/SPARK-27530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049703#comment-17049703 ] Gabriel Church edited comment on SPARK-27530 at 3/2/20 10:14 PM: - This

[jira] [Commented] (SPARK-27530) FetchFailedException: Received a zero-size buffer for block shuffle

[ https://issues.apache.org/jira/browse/SPARK-27530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049703#comment-17049703 ] Gabriel Church commented on SPARK-27530: This is an annoying bug that seems to be related to

[jira] [Comment Edited] (SPARK-27530) FetchFailedException: Received a zero-size buffer for block shuffle

[ https://issues.apache.org/jira/browse/SPARK-27530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049703#comment-17049703 ] Gabriel Church edited comment on SPARK-27530 at 3/2/20 10:13 PM: - This

[jira] [Updated] (SPARK-29918) RecordBinaryComparator should check endianness when compared by long

[ https://issues.apache.org/jira/browse/SPARK-29918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29918: -- Affects Version/s: 2.4.0 2.4.1 2.4.2

[jira] [Updated] (SPARK-29743) sample should set needCopyResult to true if its child is

[ https://issues.apache.org/jira/browse/SPARK-29743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29743: -- Affects Version/s: 2.3.0 > sample should set needCopyResult to true if its child is >

[jira] [Updated] (SPARK-29503) MapObjects doesn't copy Unsafe data when nested under Safe data

[ https://issues.apache.org/jira/browse/SPARK-29503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29503: -- Affects Version/s: 2.2.3 2.3.4 2.4.5 >

[jira] [Updated] (SPARK-29042) Sampling-based RDD with unordered input should be INDETERMINATE

[ https://issues.apache.org/jira/browse/SPARK-29042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29042: -- Affects Version/s: 2.0.2 > Sampling-based RDD with unordered input should be INDETERMINATE >

[jira] [Updated] (SPARK-29042) Sampling-based RDD with unordered input should be INDETERMINATE

[ https://issues.apache.org/jira/browse/SPARK-29042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29042: -- Affects Version/s: 2.1.3 > Sampling-based RDD with unordered input should be INDETERMINATE >

[jira] [Updated] (SPARK-29042) Sampling-based RDD with unordered input should be INDETERMINATE

[ https://issues.apache.org/jira/browse/SPARK-29042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29042: -- Affects Version/s: 2.3.4 > Sampling-based RDD with unordered input should be INDETERMINATE >

[jira] [Updated] (SPARK-29042) Sampling-based RDD with unordered input should be INDETERMINATE

[ https://issues.apache.org/jira/browse/SPARK-29042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29042: -- Affects Version/s: 2.2.3 > Sampling-based RDD with unordered input should be INDETERMINATE >

[jira] [Commented] (SPARK-30931) ML 3.0 QA: API: Python API coverage

[ https://issues.apache.org/jira/browse/SPARK-30931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049675#comment-17049675 ] Huaxin Gao commented on SPARK-30931: Opened https://issues.apache.org/jira/browse/SPARK-31012 to fix

[jira] [Comment Edited] (SPARK-30929) ML, GraphX 3.0 QA: API: New Scala APIs, docs

[ https://issues.apache.org/jira/browse/SPARK-30929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049673#comment-17049673 ] Huaxin Gao edited comment on SPARK-30929 at 3/2/20 9:38 PM: Have checked

[jira] [Commented] (SPARK-30929) ML, GraphX 3.0 QA: API: New Scala APIs, docs

[ https://issues.apache.org/jira/browse/SPARK-30929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049673#comment-17049673 ] Huaxin Gao commented on SPARK-30929: Have checked documentation. Opened the following Jira for doc

[jira] [Commented] (SPARK-28375) Enforce idempotence on the PullupCorrelatedPredicates optimizer rule

[ https://issues.apache.org/jira/browse/SPARK-28375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049667#comment-17049667 ] Dongjoon Hyun commented on SPARK-28375: --- Although we don't backport this, I updated this issue

[jira] [Updated] (SPARK-28375) Enforce idempotence on the PullupCorrelatedPredicates optimizer rule

[ https://issues.apache.org/jira/browse/SPARK-28375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28375: -- Affects Version/s: 2.2.0 2.3.0 2.4.0 > Enforce

[jira] [Updated] (SPARK-28375) Enforce idempotence on the PullupCorrelatedPredicates optimizer rule

[ https://issues.apache.org/jira/browse/SPARK-28375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28375: -- Issue Type: Bug (was: Improvement) > Enforce idempotence on the PullupCorrelatedPredicates

[jira] [Commented] (SPARK-28344) fail the query if detect ambiguous self join

[ https://issues.apache.org/jira/browse/SPARK-28344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049661#comment-17049661 ] Dongjoon Hyun commented on SPARK-28344: --- I updated `Affected Versions` because SPARK-10892 is

[jira] [Updated] (SPARK-28344) fail the query if detect ambiguous self join

[ https://issues.apache.org/jira/browse/SPARK-28344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28344: -- Affects Version/s: 1.4.1 1.5.2 1.6.3

[jira] [Updated] (SPARK-10892) Join with Data Frame returns wrong results

[ https://issues.apache.org/jira/browse/SPARK-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-10892: -- Affects Version/s: 2.4.5 > Join with Data Frame returns wrong results >

[jira] [Updated] (SPARK-27907) HiveUDAF should return NULL in case of 0 rows

[ https://issues.apache.org/jira/browse/SPARK-27907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27907: -- Affects Version/s: 2.3.4 > HiveUDAF should return NULL in case of 0 rows >

[jira] [Updated] (SPARK-27798) ConvertToLocalRelation should tolerate expression reusing output object

[ https://issues.apache.org/jira/browse/SPARK-27798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27798: -- Affects Version/s: 2.0.2 2.1.3 2.2.3 >

[jira] [Updated] (SPARK-27798) ConvertToLocalRelation should tolerate expression reusing output object

[ https://issues.apache.org/jira/browse/SPARK-27798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27798: -- Affects Version/s: 1.6.3 > ConvertToLocalRelation should tolerate expression reusing output

[jira] [Updated] (SPARK-27798) ConvertToLocalRelation should tolerate expression reusing output object

[ https://issues.apache.org/jira/browse/SPARK-27798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27798: -- Summary: ConvertToLocalRelation should tolerate expression reusing output object (was:

[jira] [Updated] (SPARK-27619) MapType should be prohibited in hash expressions

[ https://issues.apache.org/jira/browse/SPARK-27619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27619: -- Affects Version/s: 2.3.0 > MapType should be prohibited in hash expressions >

[jira] [Updated] (SPARK-27494) Null keys/values don't work in Kafka source v2

[ https://issues.apache.org/jira/browse/SPARK-27494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27494: -- Affects Version/s: 2.4.0 > Null keys/values don't work in Kafka source v2 >

[jira] [Updated] (SPARK-27406) UnsafeArrayData serialization breaks when two machines have different Oops size

[ https://issues.apache.org/jira/browse/SPARK-27406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27406: -- Affects Version/s: 2.4.0 > UnsafeArrayData serialization breaks when two machines have

[jira] [Updated] (SPARK-27275) Potential corruption in EncryptedMessage.transferTo

[ https://issues.apache.org/jira/browse/SPARK-27275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27275: -- Affects Version/s: 2.2.0 2.3.0 > Potential corruption in

[jira] [Updated] (SPARK-27216) Upgrade RoaringBitmap to 0.7.45 to fix Kryo unsafe ser/dser issue

[ https://issues.apache.org/jira/browse/SPARK-27216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27216: -- Affects Version/s: 2.0.0 2.1.0 2.2.0 > Upgrade

[jira] [Updated] (SPARK-27160) Incorrect Literal Casting of DecimalType in OrcFilters

[ https://issues.apache.org/jira/browse/SPARK-27160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27160: -- Affects Version/s: 2.3.0 > Incorrect Literal Casting of DecimalType in OrcFilters >

[jira] [Updated] (SPARK-27097) Avoid embedding platform-dependent offsets literally in whole-stage generated code

[ https://issues.apache.org/jira/browse/SPARK-27097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27097: -- Affects Version/s: 2.0.0 2.1.3 2.2.3

[jira] [Updated] (SPARK-26873) FileFormatWriter creates inconsistent MR job IDs

[ https://issues.apache.org/jira/browse/SPARK-26873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26873: -- Affects Version/s: 2.1.0 > FileFormatWriter creates inconsistent MR job IDs >

[jira] [Updated] (SPARK-26873) FileFormatWriter creates inconsistent MR job IDs

[ https://issues.apache.org/jira/browse/SPARK-26873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26873: -- Affects Version/s: 2.2.0 > FileFormatWriter creates inconsistent MR job IDs >

[jira] [Created] (SPARK-31013) InMemoryStore: improve removeAllByIndexValues over natural key index

Gengliang Wang created SPARK-31013: -- Summary: InMemoryStore: improve removeAllByIndexValues over natural key index Key: SPARK-31013 URL: https://issues.apache.org/jira/browse/SPARK-31013 Project:

[jira] [Updated] (SPARK-27685) `union` doesn't promote non-nullable columns of struct to nullable

[ https://issues.apache.org/jira/browse/SPARK-27685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27685: -- Affects Version/s: 2.0.2 > `union` doesn't promote non-nullable columns of struct to nullable

[jira] [Updated] (SPARK-26812) PushProjectionThroughUnion nullability issue

[ https://issues.apache.org/jira/browse/SPARK-26812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26812: -- Affects Version/s: 2.0.2 > PushProjectionThroughUnion nullability issue >

[jira] [Updated] (SPARK-27685) `union` doesn't promote non-nullable columns of struct to nullable

[ https://issues.apache.org/jira/browse/SPARK-27685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27685: -- Affects Version/s: 2.1.3 > `union` doesn't promote non-nullable columns of struct to nullable

[jira] [Updated] (SPARK-26812) PushProjectionThroughUnion nullability issue

[ https://issues.apache.org/jira/browse/SPARK-26812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26812: -- Affects Version/s: 2.1.3 > PushProjectionThroughUnion nullability issue >

[jira] [Updated] (SPARK-26812) PushProjectionThroughUnion nullability issue

[ https://issues.apache.org/jira/browse/SPARK-26812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26812: -- Affects Version/s: 2.2.3 > PushProjectionThroughUnion nullability issue >

[jira] [Updated] (SPARK-27685) `union` doesn't promote non-nullable columns of struct to nullable

[ https://issues.apache.org/jira/browse/SPARK-27685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27685: -- Affects Version/s: 2.2.3 2.3.4 > `union` doesn't promote non-nullable

[jira] [Updated] (SPARK-26812) PushProjectionThroughUnion nullability issue

[ https://issues.apache.org/jira/browse/SPARK-26812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26812: -- Affects Version/s: 2.3.4 > PushProjectionThroughUnion nullability issue >

[jira] [Updated] (SPARK-23523) Incorrect result caused by the rule OptimizeMetadataOnlyQuery

[ https://issues.apache.org/jira/browse/SPARK-23523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23523: -- Labels: correctness (was: ) > Incorrect result caused by the rule OptimizeMetadataOnlyQuery

[jira] [Updated] (SPARK-26709) OptimizeMetadataOnlyQuery does not correctly handle the files with zero record

[ https://issues.apache.org/jira/browse/SPARK-26709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26709: -- Affects Version/s: 2.1.0 > OptimizeMetadataOnlyQuery does not correctly handle the files with

[jira] [Updated] (SPARK-23523) Incorrect result caused by the rule OptimizeMetadataOnlyQuery

[ https://issues.apache.org/jira/browse/SPARK-23523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23523: -- Affects Version/s: 2.1.0 > Incorrect result caused by the rule OptimizeMetadataOnlyQuery >

[jira] [Created] (SPARK-31012) Update ML 3.0 docs

Huaxin Gao created SPARK-31012: -- Summary: Update ML 3.0 docs Key: SPARK-31012 URL: https://issues.apache.org/jira/browse/SPARK-31012 Project: Spark Issue Type: Improvement Components:

[jira] [Updated] (SPARK-26682) Task attempt ID collision causes lost data

[ https://issues.apache.org/jira/browse/SPARK-26682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26682: -- Affects Version/s: 2.1.0 > Task attempt ID collision causes lost data >

[jira] [Updated] (SPARK-26572) Join on distinct column with monotonically_increasing_id produces wrong output

[ https://issues.apache.org/jira/browse/SPARK-26572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26572: -- Affects Version/s: 2.1.3 > Join on distinct column with monotonically_increasing_id produces

[jira] [Updated] (SPARK-26572) Join on distinct column with monotonically_increasing_id produces wrong output

[ https://issues.apache.org/jira/browse/SPARK-26572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26572: -- Affects Version/s: 2.0.2 > Join on distinct column with monotonically_increasing_id produces

[jira] [Updated] (SPARK-30993) GenerateUnsafeRowJoiner corrupts the value if the datatype is UDF and its sql type has fixed length

[ https://issues.apache.org/jira/browse/SPARK-30993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30993: -- Labels: correctness (was: ) > GenerateUnsafeRowJoiner corrupts the value if the datatype is

[jira] [Resolved] (SPARK-30986) Structured Streaming: mapGroupsWithState UDT serialization does not work

[ https://issues.apache.org/jira/browse/SPARK-30986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30986. --- Resolution: Duplicate > Structured Streaming: mapGroupsWithState UDT serialization does not

[jira] [Resolved] (SPARK-30993) GenerateUnsafeRowJoiner corrupts the value if the datatype is UDF and its sql type has fixed length

[ https://issues.apache.org/jira/browse/SPARK-30993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30993. --- Fix Version/s: 3.0.0 Assignee: Jungtaek Lim Resolution: Fixed This is

[jira] [Resolved] (SPARK-30969) Remove resource coordination support from Standalone

[ https://issues.apache.org/jira/browse/SPARK-30969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-30969. -- Fix Version/s: 3.0.0 Resolution: Fixed Fixed by

[jira] [Commented] (SPARK-30982) List All the removed APIs of Spark SQL and Core

[ https://issues.apache.org/jira/browse/SPARK-30982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049385#comment-17049385 ] Xiao Li commented on SPARK-30982: - {code:java} org.apache.spark.sql.sources.And

[jira] [Commented] (SPARK-28427) Support more Postgres JSON functions

[ https://issues.apache.org/jira/browse/SPARK-28427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049367#comment-17049367 ] Rakesh Raushan commented on SPARK-28427: I think we should add some of Postgres JSON functions

[jira] [Resolved] (SPARK-30813) Matrices.sprand mistakes in comments

[ https://issues.apache.org/jira/browse/SPARK-30813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-30813. -- Fix Version/s: 3.0.0 2.4.6 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-30813) Matrices.sprand mistakes in comments

[ https://issues.apache.org/jira/browse/SPARK-30813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-30813: Assignee: Xiaochang Wu > Matrices.sprand mistakes in comments >

[jira] [Commented] (SPARK-31006) Mark Spark streaming as deprecated and add warnings.

[ https://issues.apache.org/jira/browse/SPARK-31006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049256#comment-17049256 ] Sean R. Owen commented on SPARK-31006: -- Why do you believe it's deprecated? What is the Bug here?

[jira] [Commented] (SPARK-31011) Failed to register signal handler for PWR

[ https://issues.apache.org/jira/browse/SPARK-31011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049210#comment-17049210 ] Gabor Somogyi commented on SPARK-31011: --- [~holden] since you've added this recently any idea?  

[jira] [Created] (SPARK-31011) Failed to register signal handler for PWR

Gabor Somogyi created SPARK-31011: - Summary: Failed to register signal handler for PWR Key: SPARK-31011 URL: https://issues.apache.org/jira/browse/SPARK-31011 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-31010) forbid untyped scala UDF API by default

[ https://issues.apache.org/jira/browse/SPARK-31010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-31010: Summary: forbid untyped scala UDF API by default (was: forbid typed scala UDF API by default) >

  1   2   >