[jira] [Commented] (SPARK-26432) Not able to connect Hbase 2.1 service Getting NoSuchMethodException while trying to obtain token from Hbase 2.1 service.
[ https://issues.apache.org/jira/browse/SPARK-26432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16732299#comment-16732299 ] Sujith commented on SPARK-26432: Test description is been updated. let me know for any suggestions or input. thnks all. > Not able to connect Hbase 2.1 service Getting NoSuchMethodException while > trying to obtain token from Hbase 2.1 service. > > > Key: SPARK-26432 > URL: https://issues.apache.org/jira/browse/SPARK-26432 > Project: Spark > Issue Type: Improvement > Components: Spark Core >Affects Versions: 2.3.2, 2.4.0 >Reporter: Sujith >Priority: Major > Attachments: hbase-dep-obtaintok.png > > > Getting NoSuchMethodException : > org.apache.hadoop.hbase.security.token.TokenUtil(org.apache.hadoop.conf.Configuration) > while trying connect hbase 2.1 service from spark. > This is mainly happening because in spark uses a deprecated hbase api > public static Token obtainToken(Configuration > conf) > for obtaining the token and the same has been removed from hbase 2.1 version. > Test steps: > Steps to test Spark-Hbase connection > 1. Create 2 tables in hbase shell > >Launch hbase shell > >Enter commands to create tables and load data > create 'table1','cf' > put 'table1','row1','cf:cid','20' > create 'table2','cf' > put 'table2','row1','cf:cid','30' > > >Show values command > get 'table1','row1','cf:cid' will diplay value as 20 > get 'table2','row1','cf:cid' will diplay value as 30 > > > 2.Run SparkHbasetoHbase class in testSpark.jar using spark-submit > spark-submit --master yarn-cluster --class > com.mrs.example.spark.SparkHbasetoHbase --conf > "spark.yarn.security.credentials.hbase.enabled"="true" --conf > "spark.security.credentials.hbase.enabled"="true" --keytab > /opt/client/user.keytab --principal sen testSpark.jar > The SparkHbasetoHbase class will update the value of table2 with sum of > values of table1 & table2. > table2 = table1+table2 > > 3.Verify the result in hbase shell > Expected Result: The value of table2 should be 50. > get 'table1','row1','cf:cid' will diplay value as 50 > Actual Result : Not updating the value as an error will be thrown when spark > tries to connect with hbase service. > Attached the snapshot of error logs below for more details -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26432) Not able to connect Hbase 2.1 service Getting NoSuchMethodException while trying to obtain token from Hbase 2.1 service.
[ https://issues.apache.org/jira/browse/SPARK-26432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16732281#comment-16732281 ] Sujith commented on SPARK-26432: sorry for the late response due to holidays :), raised a PR please let me know for any suggestions. thanks > Not able to connect Hbase 2.1 service Getting NoSuchMethodException while > trying to obtain token from Hbase 2.1 service. > > > Key: SPARK-26432 > URL: https://issues.apache.org/jira/browse/SPARK-26432 > Project: Spark > Issue Type: Improvement > Components: Spark Core >Affects Versions: 2.3.2, 2.4.0 >Reporter: Sujith >Priority: Major > Attachments: hbase-dep-obtaintok.png > > > Getting NoSuchMethodException : > org.apache.hadoop.hbase.security.token.TokenUtil(org.apache.hadoop.conf.Configuration) > while trying connect hbase 2.1 service from spark. > This is mainly happening because in spark uses a deprecated hbase api > public static Token obtainToken(Configuration > conf) > for obtaining the token and the same has been removed from hbase 2.1 version. > > Attached the snapshot of error logs -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26432) Not able to connect Hbase 2.1 service Getting NoSuchMethodException while trying to obtain token from Hbase 2.1 service.
[ https://issues.apache.org/jira/browse/SPARK-26432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16728488#comment-16728488 ] Dongjoon Hyun commented on SPARK-26432: --- Thanks! > Not able to connect Hbase 2.1 service Getting NoSuchMethodException while > trying to obtain token from Hbase 2.1 service. > > > Key: SPARK-26432 > URL: https://issues.apache.org/jira/browse/SPARK-26432 > Project: Spark > Issue Type: Improvement > Components: Spark Core >Affects Versions: 2.3.2, 2.4.0 >Reporter: Sujith >Priority: Major > Attachments: hbase-dep-obtaintok.png > > > Getting NoSuchMethodException : > org.apache.hadoop.hbase.security.token.TokenUtil(org.apache.hadoop.conf.Configuration) > while trying connect hbase 2.1 service from spark. > This is mainly happening because in spark uses a deprecated hbase api > public static Token obtainToken(Configuration > conf) > for obtaining the token and the same has been removed from hbase 2.1 version. > > Attached the snapshot of error logs -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26432) Not able to connect Hbase 2.1 service Getting NoSuchMethodException while trying to obtain token from Hbase 2.1 service.
[ https://issues.apache.org/jira/browse/SPARK-26432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16728463#comment-16728463 ] Sujith commented on SPARK-26432: Thanks for the suggestions. I will update the description, this issue has been reported by our customer who were trying to connect spark with Hbase 2.1 version. - HBase 2.1 is the only one broken among HBase versions? Could you link the Apache HBase issue which removes that API here? _From Hbase 2.0 this particular deprecated API obtainToken(conf) API is been removed. https://issues.apache.org/jira/browse/HBASE-14713_ - Is it enough to make `HBaseDelegationTokenProvider` support HBase 2.1 - . _we already had a consistent API obtainToken(Connection con) available from older versions of hbase for obtaining the token. if we use this consistent API we can avoid break while using hbase upgraded versions ._ I will raise a PR for handling this issue soon where i can include more details for this issue. > Not able to connect Hbase 2.1 service Getting NoSuchMethodException while > trying to obtain token from Hbase 2.1 service. > > > Key: SPARK-26432 > URL: https://issues.apache.org/jira/browse/SPARK-26432 > Project: Spark > Issue Type: Improvement > Components: Spark Core >Affects Versions: 2.3.2, 2.4.0 >Reporter: Sujith >Priority: Major > Attachments: hbase-dep-obtaintok.png > > > Getting NoSuchMethodException : > org.apache.hadoop.hbase.security.token.TokenUtil(org.apache.hadoop.conf.Configuration) > while trying connect hbase 2.1 service from spark. > This is mainly happening because in spark uses a deprecated hbase api > public static Token obtainToken(Configuration > conf) > for obtaining the token and the same has been removed from hbase 2.1 version. > > Attached the snapshot of error logs -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26432) Not able to connect Hbase 2.1 service Getting NoSuchMethodException while trying to obtain token from Hbase 2.1 service.
[ https://issues.apache.org/jira/browse/SPARK-26432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16728121#comment-16728121 ] Dongjoon Hyun commented on SPARK-26432: --- Thank you for reporting, [~S71955]. If you update the JIRA description with a reproducible example, that would be helpful. I have two questions. - HBase 2.1 is the only one broken among HBase versions? Could you link the Apache HBase issue which removes that API here? - Is it enough to make `HBaseDelegationTokenProvider` support HBase 2.1? > Not able to connect Hbase 2.1 service Getting NoSuchMethodException while > trying to obtain token from Hbase 2.1 service. > > > Key: SPARK-26432 > URL: https://issues.apache.org/jira/browse/SPARK-26432 > Project: Spark > Issue Type: Improvement > Components: Spark Core >Affects Versions: 2.3.2, 2.4.0 >Reporter: Sujith >Priority: Major > Attachments: hbase-dep-obtaintok.png > > > Getting NoSuchMethodException : > org.apache.hadoop.hbase.security.token.TokenUtil(org.apache.hadoop.conf.Configuration) > while trying connect hbase 2.1 service from spark. > This is mainly happening because in spark uses a deprecated hbase api > public static Token obtainToken(Configuration > conf) > for obtaining the token and the same has been removed from hbase 2.1 version. > > Attached the snapshot of error logs -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
