[ https://issues.apache.org/jira/browse/FLINK-32760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17751715#comment-17751715 ]
dongwoo.kim commented on FLINK-32760: ------------------------------------- Thanks for your help. I'll soon share the pr > Version Conflict in flink-sql-connector-hive for shaded.parquet prefix > packages > ------------------------------------------------------------------------------- > > Key: FLINK-32760 > URL: https://issues.apache.org/jira/browse/FLINK-32760 > Project: Flink > Issue Type: Bug > Components: Connectors / Hive > Affects Versions: 1.17.1 > Reporter: dongwoo.kim > Priority: Major > Attachments: image-2023-08-05-14-50-47-806.png > > > h2. Summary > In https://issues.apache.org/jira/browse/FLINK-23074 it seems like shading > parquet dependency from *hive-exec* is done. > But I think this is not enough and causing errors like below when I try to > read parquet file using sql-gateway which requires both *flink-parquet* and > *flink-sql-connector-hive* dependencies. > !image-2023-08-05-14-50-47-806.png|width=1392,height=909! > > h2. {color:#172b4d}Cause{color} > {color:#172b4d}Parquet dependency not only includes *org.apache.parquet* but > also *shaded.parquet* prefix dependencies. > ([ref|https://github.com/apache/parquet-mr/blob/515734c373f69b5250e8b63eb3d1c973da893b63/pom.xml#L72]){color} > {color:#172b4d}So we need to shade both.{color} > {color:#172b4d}- flink-parquet depends on Parquet 1.12.3 with shaded Thrift > 0.16.0 (prefix: {{{}shaded.parquet{}}}){color} > {color:#172b4d}- flink-sql-connector-hive depends on hive-exec 3.1.3 with > Parquet 1.10.0 and shaded Thrift 0.9.3 (prefix: {{{}shaded.parquet{}}}){color} > {color:#172b4d}- Code compiled against Thrift 0.16.0 attempts to run against > 0.9.3, causing the error.{color} > h2. {color:#172b4d}Proposed solution{color} > Adding new shading rule to flink-sql-connector-hive project. > I have confirmed that adding this rule could resolve the above error. > {code:xml} > <relocation> > <pattern>shaded.parquet</pattern> > <shadedPattern>shaded.parquet.flink.hive.shaded</shadedPattern> > </relocation>{code} > > I would be happy to implement it if the proposal is accepted. Thanks > -- This message was sent by Atlassian Jira (v8.20.10#820010)