adamresson opened a new issue #428: GCS Hadoop Connector can't recover from Tablet Server failure. URL: https://github.com/apache/accumulo/issues/428 I was attempting to use the GCS Connector (https://github.com/GoogleCloudPlatform/bigdata-interop/tree/master/gcs) to back Accumulo on GCP. All pretty straight forward (just pointed instance.volumes to my bucket `gs://<bucketname>/accumulo` One of my Tablet Servers OOMed, and when I try to recover it, I end up getting the following error: ``` Failed to initiate log sort gs://<bucketname>/accumulo/wal/accumulo-gcs-w-1+9997/aa166493-637e-48e8-a9a6-3655dfb59a6c java.lang.IllegalStateException: Don't know how to recover a lease for com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem at org.apache.accumulo.server.master.recovery.HadoopLogCloser.close(HadoopLogCloser.java:70) at org.apache.accumulo.master.recovery.RecoveryManager$LogSortTask.run(RecoveryManager.java:96) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.accumulo.fate.util.LoggingRunnable.run(LoggingRunnable.java:35) at java.lang.Thread.run(Thread.java:748) ``` I traced that back to HadoopLogCloser, and since the GoogleHadoopFileSystem is a FileSystem, not a `DistributedFileSystem`, `LocalFileSystem`, `RawLocalFileSystem`, it bottoms out here: https://github.com/apache/accumulo/blob/5d216d414cc9ce9e79b161f12bf4c92a430c803d/server/base/src/main/java/org/apache/accumulo/server/master/recovery/HadoopLogCloser.java#L71 I'm not sure what to do from here. I was also told that Azure should work with accumulo no problem, but it looks like the NativeAzureFileSystem similarly doesn't implement any of these interfaces, so I would presume it would hit the same issue.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
