I think it is fine to just use the original hadoop jars in HBase-2.0.1 to communicate with HDFS-2.8.4 or above?
The async wal has hacked into the internal of DFSClient so it will be easily broken when HDFS upgraded. I can take a look at the 2.8.4 problem but for 3.x, there is no production ready release yet so there is no plan to fix it yet. 2018-07-03 8:59 GMT+08:00 Sean Busbey <[email protected]>: > That's just a warning. Checking on HDFS-11644, it's only present in > Hadoop 2.9+ so seeing a lack of it with HDFS in 2.8.4 is expected. > (Presuming you are deploying on top of HDFS and not e.g. > LocalFileSystem.) > > Are there any ERROR messages in the regionservers or the master logs? > Could you post them somewhere and provide a link here? > > On Mon, Jul 2, 2018 at 5:11 PM, Andrey Elenskiy > <[email protected]> wrote: > > It's now stuck at Master Initializing and regionservers are complaining > > with: > > > > 18/07/02 21:12:20 WARN util.CommonFSUtils: Your Hadoop installation does > > not include the StreamCapabilities class from HDFS-11644, so we will skip > > checking if any FSDataOutputStreams actually support hflush/hsync. If you > > are running on top of HDFS this probably just means you have an older > > version and this can be ignored. If you are running on top of an > alternate > > FileSystem implementation you should manually verify that hflush and > hsync > > are implemented; otherwise you risk data loss and hard to diagnose errors > > when our assumptions are violated. > > > > I'm guessing hbase 2.0.1 on top of 2.8.4 hasn't been ironed out > completely > > yet (at least not with stock hadoop jars) unless I'm missing something. > > > > On Mon, Jul 2, 2018 at 3:02 PM, Mich Talebzadeh < > [email protected]> > > wrote: > > > >> You are lucky that HBASE 2.0.1 worked with Hadoop 2.8 > >> > >> I tried HBASE 2.0.1 with Hadoop 3.1 and there was endless problems with > the > >> Region server crashing because WAL file system issue. > >> > >> thread - Hbase hbase-2.0.1, region server does not start on Hadoop 3.1 > >> > >> Decided to roll back to Hbase 1.2.6 that works with Hadoop 3.1 > >> > >> HTH > >> > >> Dr Mich Talebzadeh > >> > >> > >> > >> LinkedIn * https://www.linkedin.com/profile/view?id= > >> AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > >> <https://www.linkedin.com/profile/view?id= > AAEAAAAWh2gBxianrbJd6zP6AcPCCd > >> OABUrV8Pw>* > >> > >> > >> > >> http://talebzadehmich.wordpress.com > >> > >> > >> *Disclaimer:* Use it at your own risk. Any and all responsibility for > any > >> loss, damage or destruction of data or any other property which may > arise > >> from relying on this email's technical content is explicitly disclaimed. > >> The author will in no case be liable for any monetary damages arising > from > >> such loss, damage or destruction. > >> > >> > >> > >> > >> On Mon, 2 Jul 2018 at 22:43, Andrey Elenskiy > >> <[email protected]> wrote: > >> > >> > <property> > >> > <name>hbase.wal.provider</name> > >> > <value>filesystem</value> > >> > </property> > >> > > >> > Seems to fix it, but would be nice to actually try the fanout wal with > >> > hadoop 2.8.4. > >> > > >> > On Mon, Jul 2, 2018 at 1:03 PM, Andrey Elenskiy < > >> > [email protected]> > >> > wrote: > >> > > >> > > Hello, we are running HBase 2.0.1 with official Hadoop 2.8.4 jars > and > >> > > hadoop 2.8.4 client (http://central.maven.org/ > >> maven2/org/apache/hadoop/ > >> > > hadoop-client/2.8.4/). Got the following exception on regionserver > >> which > >> > > brings it down: > >> > > > >> > > 18/07/02 18:51:06 WARN concurrent.DefaultPromise: An exception was > >> > thrown by org.apache.hadoop.hbase.io > >> > .asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete() > >> > > java.lang.Error: Couldn't properly initialize access to HDFS > internals. > >> > Please update your WAL Provider to not make use of the 'asyncfs' > >> provider. > >> > See HBASE-16110 for more information. > >> > > at org.apache.hadoop.hbase.io > >> > .asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.<clinit>( > >> FanOutOneBlockAsyncDFSOutputSaslHelper.java:268) > >> > > at org.apache.hadoop.hbase.io > >> > .asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize( > >> FanOutOneBlockAsyncDFSOutputHelper.java:661) > >> > > at org.apache.hadoop.hbase.io > >> > .asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300( > >> FanOutOneBlockAsyncDFSOutputHelper.java:118) > >> > > at org.apache.hadoop.hbase.io > >> > .asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete( > >> FanOutOneBlockAsyncDFSOutputHelper.java:720) > >> > > at org.apache.hadoop.hbase.io > >> > .asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete( > >> FanOutOneBlockAsyncDFSOutputHelper.java:715) > >> > > at > >> > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise. > >> notifyListener0(DefaultPromise.java:507) > >> > > at > >> > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise. > >> notifyListeners0(DefaultPromise.java:500) > >> > > at > >> > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise. > >> notifyListenersNow(DefaultPromise.java:479) > >> > > at > >> > org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise. > >> notifyListeners(DefaultPromise.java:420) > >> > > at > >> > org.apache.hbase.thirdparty.io.netty.util.concurrent. > >> DefaultPromise.trySuccess(DefaultPromise.java:104) > >> > > at > >> > org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise. > >> trySuccess(DefaultChannelPromise.java:82) > >> > > at > >> > org.apache.hbase.thirdparty.io.netty.channel.epoll. > AbstractEpollChannel$ > >> AbstractEpollUnsafe.fulfillConnectPromise( > AbstractEpollChannel.java:638) > >> > > at > >> > org.apache.hbase.thirdparty.io.netty.channel.epoll. > AbstractEpollChannel$ > >> AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:676) > >> > > at > >> > org.apache.hbase.thirdparty.io.netty.channel.epoll. > AbstractEpollChannel$ > >> AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:552) > >> > > at > >> > org.apache.hbase.thirdparty.io.netty.channel.epoll. > >> EpollEventLoop.processReady(EpollEventLoop.java:394) > >> > > at > >> > org.apache.hbase.thirdparty.io.netty.channel.epoll. > EpollEventLoop.run( > >> EpollEventLoop.java:304) > >> > > at > >> > org.apache.hbase.thirdparty.io.netty.util.concurrent. > >> SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) > >> > > at > >> > org.apache.hbase.thirdparty.io.netty.util.concurrent. > >> DefaultThreadFactory$DefaultRunnableDecorator.run( > >> DefaultThreadFactory.java:138) > >> > > at java.lang.Thread.run(Thread.java:748) > >> > > Caused by: java.lang.NoSuchMethodException: > >> > org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryption > >> Key(org.apache.hadoop.fs.FileEncryptionInfo) > >> > > at java.lang.Class.getDeclaredMethod(Class.java:2130) > >> > > at org.apache.hadoop.hbase.io > >> > .asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper. > >> createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSa > >> slHelper.java:232) > >> > > at org.apache.hadoop.hbase.io > >> > .asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.<clinit>( > >> FanOutOneBlockAsyncDFSOutputSaslHelper.java:262) > >> > > ... 18 more > >> > > > >> > > FYI, we don't have encryption enabled. Let me know if you need more > >> info > >> > > about our setup. > >> > > > >> > > >> >
