[jira] [Commented] (HIVE-26255) Drop custom code for String deduplication inside URI class
[ https://issues.apache.org/jira/browse/HIVE-26255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17540827#comment-17540827 ] Peter Vary commented on HIVE-26255: --- [~slachiewicz]: Which version of Java have you tried? We have a jira for upgrading to java 11: HIVE-22415 > Drop custom code for String deduplication inside URI class > -- > > Key: HIVE-26255 > URL: https://issues.apache.org/jira/browse/HIVE-26255 > Project: Hive > Issue Type: Improvement >Reporter: Sylwester Lachiewicz >Priority: Minor > > We have class > [https://github.com/apache/hive/blob/master/common/src/java/org/apache/hadoop/hive/common/StringInternUtils.java] > to do String.intern() inside the URI class via reflection. > Instead, we can recommend using newer java and enabling String deduplication > https://localcoder.org/is-string-deduplication-feature-of-the-g1-garbage-collector-enabled-by-default > -XX:+UseG1GC -XX:+UseStringDeduplication > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (HIVE-26255) Drop custom code for String deduplication inside URI class
[ https://issues.apache.org/jira/browse/HIVE-26255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17540829#comment-17540829 ] Peter Vary commented on HIVE-26255: --- Maybe related to the Java upgrade > Drop custom code for String deduplication inside URI class > -- > > Key: HIVE-26255 > URL: https://issues.apache.org/jira/browse/HIVE-26255 > Project: Hive > Issue Type: Improvement >Reporter: Sylwester Lachiewicz >Priority: Minor > > We have class > [https://github.com/apache/hive/blob/master/common/src/java/org/apache/hadoop/hive/common/StringInternUtils.java] > to do String.intern() inside the URI class via reflection. > Instead, we can recommend using newer java and enabling String deduplication > https://localcoder.org/is-string-deduplication-feature-of-the-g1-garbage-collector-enabled-by-default > -XX:+UseG1GC -XX:+UseStringDeduplication > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (HIVE-26255) Drop custom code for String deduplication inside URI class
[
https://issues.apache.org/jira/browse/HIVE-26255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17540506#comment-17540506
]
Sylwester Lachiewicz commented on HIVE-26255:
-
Also, if we use newer Java - few tests will fail
{noformat}
[ERROR]
org.apache.hadoop.hive.hbase.TestHBaseSerDe.testHBaseSerDeWithAvroExternalSchema
Time elapsed: 0.001 s <<< ERROR!
java.lang.NoClassDefFoundError: Could not initialize class
org.apache.hadoop.hive.common.StringInternUtils
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:330)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:322)
at
org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.createObjectInspectorWorker(AvroObjectInspectorGenerator.java:108)
at
org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.createObjectInspectorWorker(AvroObjectInspectorGenerator.java:129)
at
org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.createObjectInspectorWorker(AvroObjectInspectorGenerator.java:104)
at
org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.createObjectInspectorWorker(AvroObjectInspectorGenerator.java:121)
at
org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.createObjectInspectorWorker(AvroObjectInspectorGenerator.java:104)
at
org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.createObjectInspector(AvroObjectInspectorGenerator.java:83)
at
org.apache.hadoop.hive.serde2.avro.AvroObjectInspectorGenerator.(AvroObjectInspectorGenerator.java:56)
at
org.apache.hadoop.hive.hbase.HBaseSerDeHelper.generateAvroStructFromSchema(HBaseSerDeHelper.java:500)
at
org.apache.hadoop.hive.hbase.HBaseSerDeHelper.generateAvroStructFromClass(HBaseSerDeHelper.java:474)
at
org.apache.hadoop.hive.hbase.HBaseSerDeHelper.generateColumnStruct(HBaseSerDeHelper.java:446)
at
org.apache.hadoop.hive.hbase.HBaseSerDeHelper.generateColumnTypes(HBaseSerDeHelper.java:342)
at
org.apache.hadoop.hive.hbase.ColumnMappings.toTypesString(ColumnMappings.java:84)
at
org.apache.hadoop.hive.hbase.HBaseSerDeParameters.(HBaseSerDeParameters.java:94)
at org.apache.hadoop.hive.hbase.HBaseSerDe.initialize(HBaseSerDe.java:118)
at
org.apache.hadoop.hive.hbase.TestHBaseSerDe.testHBaseSerDeWithAvroExternalSchema(TestHBaseSerDe.java:1346)
at
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
at java.base/java.lang.reflect.Method.invoke(Method.java:577)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at
org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
at
org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:43)
at
java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
at
java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
at
java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1845)
at
java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
at
java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
at
java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
at
java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
at
java.base/java.
