[jira] [Updated] (SPARK-34772) RebaseDateTime loadRebaseRecords should use Spark classloader instead of context
[ https://issues.apache.org/jira/browse/SPARK-34772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-34772: Description: We will use date formatter in `HiveShim` that convert `date` to `string`, if we set `spark.sql.legacy.timeParserPolicy=LEGACY` the `RebaseDateTime` code will be invoked. At that moment, in custom hive metastore version case, the context class loader is `IsolatedClientLoader` then such error msg is thrown: {code:java} java.lang.IllegalArgumentException: argument "src" is null at com.fasterxml.jackson.databind.ObjectMapper._assertNotNull(ObjectMapper.java:4413) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3157) at com.fasterxml.jackson.module.scala.ScalaObjectMapper.readValue(ScalaObjectMapper.scala:187) at com.fasterxml.jackson.module.scala.ScalaObjectMapper.readValue$(ScalaObjectMapper.scala:186) at org.apache.spark.sql.catalyst.util.RebaseDateTime$$anon$1.readValue(RebaseDateTime.scala:267) at org.apache.spark.sql.catalyst.util.RebaseDateTime$.loadRebaseRecords(RebaseDateTime.scala:269) at org.apache.spark.sql.catalyst.util.RebaseDateTime$.(RebaseDateTime.scala:291) at org.apache.spark.sql.catalyst.util.RebaseDateTime$.(RebaseDateTime.scala) at org.apache.spark.sql.catalyst.util.DateTimeUtils$.toJavaDate(DateTimeUtils.scala:109) at org.apache.spark.sql.catalyst.util.LegacyDateFormatter.format(DateFormatter.scala:95) at org.apache.spark.sql.catalyst.util.LegacyDateFormatter.format$(DateFormatter.scala:94) at org.apache.spark.sql.catalyst.util.LegacySimpleDateFormatter.format(DateFormatter.scala:138) at org.apache.spark.sql.hive.client.Shim_v0_13$ExtractableLiteral$1$.unapply(HiveShim.scala:661) at org.apache.spark.sql.hive.client.Shim_v0_13.convert$1(HiveShim.scala:785) at org.apache.spark.sql.hive.client.Shim_v0_13.$anonfun$convertFilters$4(HiveShim.scala:826) {code} > RebaseDateTime loadRebaseRecords should use Spark classloader instead of > context > > > Key: SPARK-34772 > URL: https://issues.apache.org/jira/browse/SPARK-34772 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.2.0, 3.1.1 >Reporter: ulysses you >Priority: Minor > > We will use date formatter in `HiveShim` that convert `date` to `string`, if > we set `spark.sql.legacy.timeParserPolicy=LEGACY` the `RebaseDateTime` code > will be invoked. At that moment, in custom hive metastore version case, the > context class loader is `IsolatedClientLoader` then such error msg is thrown: > {code:java} > java.lang.IllegalArgumentException: argument "src" is null > at > com.fasterxml.jackson.databind.ObjectMapper._assertNotNull(ObjectMapper.java:4413) > at > com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3157) > at > com.fasterxml.jackson.module.scala.ScalaObjectMapper.readValue(ScalaObjectMapper.scala:187) > at > com.fasterxml.jackson.module.scala.ScalaObjectMapper.readValue$(ScalaObjectMapper.scala:186) > at > org.apache.spark.sql.catalyst.util.RebaseDateTime$$anon$1.readValue(RebaseDateTime.scala:267) > at > org.apache.spark.sql.catalyst.util.RebaseDateTime$.loadRebaseRecords(RebaseDateTime.scala:269) > at > org.apache.spark.sql.catalyst.util.RebaseDateTime$.(RebaseDateTime.scala:291) > at > org.apache.spark.sql.catalyst.util.RebaseDateTime$.(RebaseDateTime.scala) > at > org.apache.spark.sql.catalyst.util.DateTimeUtils$.toJavaDate(DateTimeUtils.scala:109) > at > org.apache.spark.sql.catalyst.util.LegacyDateFormatter.format(DateFormatter.scala:95) > at > org.apache.spark.sql.catalyst.util.LegacyDateFormatter.format$(DateFormatter.scala:94) > at > org.apache.spark.sql.catalyst.util.LegacySimpleDateFormatter.format(DateFormatter.scala:138) > at > org.apache.spark.sql.hive.client.Shim_v0_13$ExtractableLiteral$1$.unapply(HiveShim.scala:661) > at org.apache.spark.sql.hive.client.Shim_v0_13.convert$1(HiveShim.scala:785) > at > org.apache.spark.sql.hive.client.Shim_v0_13.$anonfun$convertFilters$4(HiveShim.scala:826) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-34772) RebaseDateTime loadRebaseRecords should use Spark classloader instead of context
[ https://issues.apache.org/jira/browse/SPARK-34772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-34772: Description: We will use date formatter in `HiveShim` that convert `date` to `string`, if we set `spark.sql.legacy.timeParserPolicy=LEGACY` the `RebaseDateTime` code will be invoked. At that moment, if `RebaseDateTime` is initialized the first time then context class loader is `IsolatedClientLoader`. Such error msg is thrown: {code:java} java.lang.IllegalArgumentException: argument "src" is null at com.fasterxml.jackson.databind.ObjectMapper._assertNotNull(ObjectMapper.java:4413) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3157) at com.fasterxml.jackson.module.scala.ScalaObjectMapper.readValue(ScalaObjectMapper.scala:187) at com.fasterxml.jackson.module.scala.ScalaObjectMapper.readValue$(ScalaObjectMapper.scala:186) at org.apache.spark.sql.catalyst.util.RebaseDateTime$$anon$1.readValue(RebaseDateTime.scala:267) at org.apache.spark.sql.catalyst.util.RebaseDateTime$.loadRebaseRecords(RebaseDateTime.scala:269) at org.apache.spark.sql.catalyst.util.RebaseDateTime$.(RebaseDateTime.scala:291) at org.apache.spark.sql.catalyst.util.RebaseDateTime$.(RebaseDateTime.scala) at org.apache.spark.sql.catalyst.util.DateTimeUtils$.toJavaDate(DateTimeUtils.scala:109) at org.apache.spark.sql.catalyst.util.LegacyDateFormatter.format(DateFormatter.scala:95) at org.apache.spark.sql.catalyst.util.LegacyDateFormatter.format$(DateFormatter.scala:94) at org.apache.spark.sql.catalyst.util.LegacySimpleDateFormatter.format(DateFormatter.scala:138) at org.apache.spark.sql.hive.client.Shim_v0_13$ExtractableLiteral$1$.unapply(HiveShim.scala:661) at org.apache.spark.sql.hive.client.Shim_v0_13.convert$1(HiveShim.scala:785) at org.apache.spark.sql.hive.client.Shim_v0_13.$anonfun$convertFilters$4(HiveShim.scala:826) {code} was: We will use date formatter in `HiveShim` that convert `date` to `string`, if we set `spark.sql.legacy.timeParserPolicy=LEGACY` the `RebaseDateTime` code will be invoked. At that moment, if `RebaseDateTime` is initialzed the first time then context class loader is `IsolatedClientLoader`. Such error msg is thrown: {code:java} java.lang.IllegalArgumentException: argument "src" is null at com.fasterxml.jackson.databind.ObjectMapper._assertNotNull(ObjectMapper.java:4413) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3157) at com.fasterxml.jackson.module.scala.ScalaObjectMapper.readValue(ScalaObjectMapper.scala:187) at com.fasterxml.jackson.module.scala.ScalaObjectMapper.readValue$(ScalaObjectMapper.scala:186) at org.apache.spark.sql.catalyst.util.RebaseDateTime$$anon$1.readValue(RebaseDateTime.scala:267) at org.apache.spark.sql.catalyst.util.RebaseDateTime$.loadRebaseRecords(RebaseDateTime.scala:269) at org.apache.spark.sql.catalyst.util.RebaseDateTime$.(RebaseDateTime.scala:291) at org.apache.spark.sql.catalyst.util.RebaseDateTime$.(RebaseDateTime.scala) at org.apache.spark.sql.catalyst.util.DateTimeUtils$.toJavaDate(DateTimeUtils.scala:109) at org.apache.spark.sql.catalyst.util.LegacyDateFormatter.format(DateFormatter.scala:95) at org.apache.spark.sql.catalyst.util.LegacyDateFormatter.format$(DateFormatter.scala:94) at org.apache.spark.sql.catalyst.util.LegacySimpleDateFormatter.format(DateFormatter.scala:138) at org.apache.spark.sql.hive.client.Shim_v0_13$ExtractableLiteral$1$.unapply(HiveShim.scala:661) at org.apache.spark.sql.hive.client.Shim_v0_13.convert$1(HiveShim.scala:785) at org.apache.spark.sql.hive.client.Shim_v0_13.$anonfun$convertFilters$4(HiveShim.scala:826) {code} > RebaseDateTime loadRebaseRecords should use Spark classloader instead of > context > > > Key: SPARK-34772 > URL: https://issues.apache.org/jira/browse/SPARK-34772 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.2.0, 3.1.1 >Reporter: ulysses you >Priority: Minor > > We will use date formatter in `HiveShim` that convert `date` to `string`, if > we set `spark.sql.legacy.timeParserPolicy=LEGACY` the `RebaseDateTime` code > will be invoked. At that moment, if `RebaseDateTime` is initialized the first > time then context class loader is `IsolatedClientLoader`. Such error msg is > thrown: > {code:java} > java.lang.IllegalArgumentException: argument "src" is null > at > com.fasterxml.jackson.databind.ObjectMapper._assertNotNull(ObjectMapper.java:4413) > at > com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3157) > at > com.fasterxml.jackson.module.scala.ScalaObjectMapper.readValue(ScalaObjectMapper.scala:187) > at > com.fasterxml.jackson.module.scala.ScalaObjectM
[jira] [Updated] (SPARK-34772) RebaseDateTime loadRebaseRecords should use Spark classloader instead of context
[ https://issues.apache.org/jira/browse/SPARK-34772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-34772: Description: We will use date formatter in `HiveShim` that convert `date` to `string`, if we set `spark.sql.legacy.timeParserPolicy=LEGACY` the `RebaseDateTime` code will be invoked. At that moment, if `RebaseDateTime` is initialzed the first time then context class loader is `IsolatedClientLoader`. Such error msg is thrown: {code:java} java.lang.IllegalArgumentException: argument "src" is null at com.fasterxml.jackson.databind.ObjectMapper._assertNotNull(ObjectMapper.java:4413) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3157) at com.fasterxml.jackson.module.scala.ScalaObjectMapper.readValue(ScalaObjectMapper.scala:187) at com.fasterxml.jackson.module.scala.ScalaObjectMapper.readValue$(ScalaObjectMapper.scala:186) at org.apache.spark.sql.catalyst.util.RebaseDateTime$$anon$1.readValue(RebaseDateTime.scala:267) at org.apache.spark.sql.catalyst.util.RebaseDateTime$.loadRebaseRecords(RebaseDateTime.scala:269) at org.apache.spark.sql.catalyst.util.RebaseDateTime$.(RebaseDateTime.scala:291) at org.apache.spark.sql.catalyst.util.RebaseDateTime$.(RebaseDateTime.scala) at org.apache.spark.sql.catalyst.util.DateTimeUtils$.toJavaDate(DateTimeUtils.scala:109) at org.apache.spark.sql.catalyst.util.LegacyDateFormatter.format(DateFormatter.scala:95) at org.apache.spark.sql.catalyst.util.LegacyDateFormatter.format$(DateFormatter.scala:94) at org.apache.spark.sql.catalyst.util.LegacySimpleDateFormatter.format(DateFormatter.scala:138) at org.apache.spark.sql.hive.client.Shim_v0_13$ExtractableLiteral$1$.unapply(HiveShim.scala:661) at org.apache.spark.sql.hive.client.Shim_v0_13.convert$1(HiveShim.scala:785) at org.apache.spark.sql.hive.client.Shim_v0_13.$anonfun$convertFilters$4(HiveShim.scala:826) {code} was: We will use date formatter in `HiveShim` that convert `date` to `string`, if we set `spark.sql.legacy.timeParserPolicy=LEGACY` the `RebaseDateTime` code will be invoked. At that moment, in custom hive metastore version case, the context class loader is `IsolatedClientLoader` then such error msg is thrown: {code:java} java.lang.IllegalArgumentException: argument "src" is null at com.fasterxml.jackson.databind.ObjectMapper._assertNotNull(ObjectMapper.java:4413) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3157) at com.fasterxml.jackson.module.scala.ScalaObjectMapper.readValue(ScalaObjectMapper.scala:187) at com.fasterxml.jackson.module.scala.ScalaObjectMapper.readValue$(ScalaObjectMapper.scala:186) at org.apache.spark.sql.catalyst.util.RebaseDateTime$$anon$1.readValue(RebaseDateTime.scala:267) at org.apache.spark.sql.catalyst.util.RebaseDateTime$.loadRebaseRecords(RebaseDateTime.scala:269) at org.apache.spark.sql.catalyst.util.RebaseDateTime$.(RebaseDateTime.scala:291) at org.apache.spark.sql.catalyst.util.RebaseDateTime$.(RebaseDateTime.scala) at org.apache.spark.sql.catalyst.util.DateTimeUtils$.toJavaDate(DateTimeUtils.scala:109) at org.apache.spark.sql.catalyst.util.LegacyDateFormatter.format(DateFormatter.scala:95) at org.apache.spark.sql.catalyst.util.LegacyDateFormatter.format$(DateFormatter.scala:94) at org.apache.spark.sql.catalyst.util.LegacySimpleDateFormatter.format(DateFormatter.scala:138) at org.apache.spark.sql.hive.client.Shim_v0_13$ExtractableLiteral$1$.unapply(HiveShim.scala:661) at org.apache.spark.sql.hive.client.Shim_v0_13.convert$1(HiveShim.scala:785) at org.apache.spark.sql.hive.client.Shim_v0_13.$anonfun$convertFilters$4(HiveShim.scala:826) {code} > RebaseDateTime loadRebaseRecords should use Spark classloader instead of > context > > > Key: SPARK-34772 > URL: https://issues.apache.org/jira/browse/SPARK-34772 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.2.0, 3.1.1 >Reporter: ulysses you >Priority: Minor > > We will use date formatter in `HiveShim` that convert `date` to `string`, if > we set `spark.sql.legacy.timeParserPolicy=LEGACY` the `RebaseDateTime` code > will be invoked. At that moment, if `RebaseDateTime` is initialzed the first > time then context class loader is `IsolatedClientLoader`. Such error msg is > thrown: > {code:java} > java.lang.IllegalArgumentException: argument "src" is null > at > com.fasterxml.jackson.databind.ObjectMapper._assertNotNull(ObjectMapper.java:4413) > at > com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3157) > at > com.fasterxml.jackson.module.scala.ScalaObjectMapper.readValue(ScalaObjectMapper.scala:187) > at > com.fasterxml.jackson.module.scala.ScalaObjectMapper.re
[jira] [Updated] (SPARK-34772) RebaseDateTime loadRebaseRecords should use Spark classloader instead of context
[ https://issues.apache.org/jira/browse/SPARK-34772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-34772: Description: With custom `spark.sql.hive.metastore.version` and `spark.sql.hive.metastore.jars`. Spark would use date formatter in `HiveShim` that convert `date` to `string`, if we set `spark.sql.legacy.timeParserPolicy=LEGACY` the `RebaseDateTime` code will be invoked. At that moment, if `RebaseDateTime` is initialized the first time then context class loader is `IsolatedClientLoader`. Such error msg is thrown: {code:java} java.lang.IllegalArgumentException: argument "src" is null at com.fasterxml.jackson.databind.ObjectMapper._assertNotNull(ObjectMapper.java:4413) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3157) at com.fasterxml.jackson.module.scala.ScalaObjectMapper.readValue(ScalaObjectMapper.scala:187) at com.fasterxml.jackson.module.scala.ScalaObjectMapper.readValue$(ScalaObjectMapper.scala:186) at org.apache.spark.sql.catalyst.util.RebaseDateTime$$anon$1.readValue(RebaseDateTime.scala:267) at org.apache.spark.sql.catalyst.util.RebaseDateTime$.loadRebaseRecords(RebaseDateTime.scala:269) at org.apache.spark.sql.catalyst.util.RebaseDateTime$.(RebaseDateTime.scala:291) at org.apache.spark.sql.catalyst.util.RebaseDateTime$.(RebaseDateTime.scala) at org.apache.spark.sql.catalyst.util.DateTimeUtils$.toJavaDate(DateTimeUtils.scala:109) at org.apache.spark.sql.catalyst.util.LegacyDateFormatter.format(DateFormatter.scala:95) at org.apache.spark.sql.catalyst.util.LegacyDateFormatter.format$(DateFormatter.scala:94) at org.apache.spark.sql.catalyst.util.LegacySimpleDateFormatter.format(DateFormatter.scala:138) at org.apache.spark.sql.hive.client.Shim_v0_13$ExtractableLiteral$1$.unapply(HiveShim.scala:661) at org.apache.spark.sql.hive.client.Shim_v0_13.convert$1(HiveShim.scala:785) at org.apache.spark.sql.hive.client.Shim_v0_13.$anonfun$convertFilters$4(HiveShim.scala:826) {code} was: We will use date formatter in `HiveShim` that convert `date` to `string`, if we set `spark.sql.legacy.timeParserPolicy=LEGACY` the `RebaseDateTime` code will be invoked. At that moment, if `RebaseDateTime` is initialized the first time then context class loader is `IsolatedClientLoader`. Such error msg is thrown: {code:java} java.lang.IllegalArgumentException: argument "src" is null at com.fasterxml.jackson.databind.ObjectMapper._assertNotNull(ObjectMapper.java:4413) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3157) at com.fasterxml.jackson.module.scala.ScalaObjectMapper.readValue(ScalaObjectMapper.scala:187) at com.fasterxml.jackson.module.scala.ScalaObjectMapper.readValue$(ScalaObjectMapper.scala:186) at org.apache.spark.sql.catalyst.util.RebaseDateTime$$anon$1.readValue(RebaseDateTime.scala:267) at org.apache.spark.sql.catalyst.util.RebaseDateTime$.loadRebaseRecords(RebaseDateTime.scala:269) at org.apache.spark.sql.catalyst.util.RebaseDateTime$.(RebaseDateTime.scala:291) at org.apache.spark.sql.catalyst.util.RebaseDateTime$.(RebaseDateTime.scala) at org.apache.spark.sql.catalyst.util.DateTimeUtils$.toJavaDate(DateTimeUtils.scala:109) at org.apache.spark.sql.catalyst.util.LegacyDateFormatter.format(DateFormatter.scala:95) at org.apache.spark.sql.catalyst.util.LegacyDateFormatter.format$(DateFormatter.scala:94) at org.apache.spark.sql.catalyst.util.LegacySimpleDateFormatter.format(DateFormatter.scala:138) at org.apache.spark.sql.hive.client.Shim_v0_13$ExtractableLiteral$1$.unapply(HiveShim.scala:661) at org.apache.spark.sql.hive.client.Shim_v0_13.convert$1(HiveShim.scala:785) at org.apache.spark.sql.hive.client.Shim_v0_13.$anonfun$convertFilters$4(HiveShim.scala:826) {code} > RebaseDateTime loadRebaseRecords should use Spark classloader instead of > context > > > Key: SPARK-34772 > URL: https://issues.apache.org/jira/browse/SPARK-34772 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.2.0, 3.1.1 >Reporter: ulysses you >Priority: Minor > > With custom `spark.sql.hive.metastore.version` and > `spark.sql.hive.metastore.jars`. > Spark would use date formatter in `HiveShim` that convert `date` to `string`, > if we set `spark.sql.legacy.timeParserPolicy=LEGACY` the `RebaseDateTime` > code will be invoked. At that moment, if `RebaseDateTime` is initialized the > first time then context class loader is `IsolatedClientLoader`. Such error > msg is thrown: > {code:java} > java.lang.IllegalArgumentException: argument "src" is null > at > com.fasterxml.jackson.databind.ObjectMapper._assertNotNull(ObjectMapper.java:4413) > at > com.fasterxml.jackson.databind.ObjectMapper.readValu
[jira] [Updated] (SPARK-34772) RebaseDateTime loadRebaseRecords should use Spark classloader instead of context
[ https://issues.apache.org/jira/browse/SPARK-34772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-34772: Description: With custom `spark.sql.hive.metastore.version` and `spark.sql.hive.metastore.jars`. Spark would use date formatter in `HiveShim` that convert `date` to `string`, if we set `spark.sql.legacy.timeParserPolicy=LEGACY` the `RebaseDateTime` code will be invoked. At that moment, if `RebaseDateTime` is initialized the first time then context class loader is `IsolatedClientLoader`. Such error msg would throw: {code:java} java.lang.IllegalArgumentException: argument "src" is null at com.fasterxml.jackson.databind.ObjectMapper._assertNotNull(ObjectMapper.java:4413) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3157) at com.fasterxml.jackson.module.scala.ScalaObjectMapper.readValue(ScalaObjectMapper.scala:187) at com.fasterxml.jackson.module.scala.ScalaObjectMapper.readValue$(ScalaObjectMapper.scala:186) at org.apache.spark.sql.catalyst.util.RebaseDateTime$$anon$1.readValue(RebaseDateTime.scala:267) at org.apache.spark.sql.catalyst.util.RebaseDateTime$.loadRebaseRecords(RebaseDateTime.scala:269) at org.apache.spark.sql.catalyst.util.RebaseDateTime$.(RebaseDateTime.scala:291) at org.apache.spark.sql.catalyst.util.RebaseDateTime$.(RebaseDateTime.scala) at org.apache.spark.sql.catalyst.util.DateTimeUtils$.toJavaDate(DateTimeUtils.scala:109) at org.apache.spark.sql.catalyst.util.LegacyDateFormatter.format(DateFormatter.scala:95) at org.apache.spark.sql.catalyst.util.LegacyDateFormatter.format$(DateFormatter.scala:94) at org.apache.spark.sql.catalyst.util.LegacySimpleDateFormatter.format(DateFormatter.scala:138) at org.apache.spark.sql.hive.client.Shim_v0_13$ExtractableLiteral$1$.unapply(HiveShim.scala:661) at org.apache.spark.sql.hive.client.Shim_v0_13.convert$1(HiveShim.scala:785) at org.apache.spark.sql.hive.client.Shim_v0_13.$anonfun$convertFilters$4(HiveShim.scala:826) {code} was: With custom `spark.sql.hive.metastore.version` and `spark.sql.hive.metastore.jars`. Spark would use date formatter in `HiveShim` that convert `date` to `string`, if we set `spark.sql.legacy.timeParserPolicy=LEGACY` the `RebaseDateTime` code will be invoked. At that moment, if `RebaseDateTime` is initialized the first time then context class loader is `IsolatedClientLoader`. Such error msg is thrown: {code:java} java.lang.IllegalArgumentException: argument "src" is null at com.fasterxml.jackson.databind.ObjectMapper._assertNotNull(ObjectMapper.java:4413) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3157) at com.fasterxml.jackson.module.scala.ScalaObjectMapper.readValue(ScalaObjectMapper.scala:187) at com.fasterxml.jackson.module.scala.ScalaObjectMapper.readValue$(ScalaObjectMapper.scala:186) at org.apache.spark.sql.catalyst.util.RebaseDateTime$$anon$1.readValue(RebaseDateTime.scala:267) at org.apache.spark.sql.catalyst.util.RebaseDateTime$.loadRebaseRecords(RebaseDateTime.scala:269) at org.apache.spark.sql.catalyst.util.RebaseDateTime$.(RebaseDateTime.scala:291) at org.apache.spark.sql.catalyst.util.RebaseDateTime$.(RebaseDateTime.scala) at org.apache.spark.sql.catalyst.util.DateTimeUtils$.toJavaDate(DateTimeUtils.scala:109) at org.apache.spark.sql.catalyst.util.LegacyDateFormatter.format(DateFormatter.scala:95) at org.apache.spark.sql.catalyst.util.LegacyDateFormatter.format$(DateFormatter.scala:94) at org.apache.spark.sql.catalyst.util.LegacySimpleDateFormatter.format(DateFormatter.scala:138) at org.apache.spark.sql.hive.client.Shim_v0_13$ExtractableLiteral$1$.unapply(HiveShim.scala:661) at org.apache.spark.sql.hive.client.Shim_v0_13.convert$1(HiveShim.scala:785) at org.apache.spark.sql.hive.client.Shim_v0_13.$anonfun$convertFilters$4(HiveShim.scala:826) {code} > RebaseDateTime loadRebaseRecords should use Spark classloader instead of > context > > > Key: SPARK-34772 > URL: https://issues.apache.org/jira/browse/SPARK-34772 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.2.0, 3.1.1 >Reporter: ulysses you >Priority: Minor > > With custom `spark.sql.hive.metastore.version` and > `spark.sql.hive.metastore.jars`. > Spark would use date formatter in `HiveShim` that convert `date` to `string`, > if we set `spark.sql.legacy.timeParserPolicy=LEGACY` the `RebaseDateTime` > code will be invoked. At that moment, if `RebaseDateTime` is initialized the > first time then context class loader is `IsolatedClientLoader`. Such error > msg would throw: > {code:java} > java.lang.IllegalArgumentException: argument "src" is null > at > com.fasterxml.jackson.databind.ObjectMapper._asser