[jira] [Created] (HBASE-27889) Possible NPE while getting cluster status from master
Rajeshbabu Chintaguntla created HBASE-27889:
---
Summary: Possible NPE while getting cluster status from master
Key: HBASE-27889
URL: https://issues.apache.org/jira/browse/HBASE-27889
Project: HBase
Issue Type: Bug
Components: master
Reporter: Rajeshbabu Chintaguntla
Assignee: Rajeshbabu Chintaguntla
{noformat}
2023-05-23 13:31:33,840 ERROR
[RpcServer.default.FPBQ.Fifo.handler=395,queue=35,port=16000] ipc.RpcServer:
Unexpected throwable object
java.lang.NullPointerException
at
org.apache.hadoop.hbase.shaded.protobuf.generated.ClusterStatusProtos$ServerTask$Builder.setStatus(ClusterStatusProtos.java:14120)
at
org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toServerTask(ProtobufUtil.java:3565)
at
org.apache.hadoop.hbase.ClusterMetricsBuilder.lambda$toClusterStatus$4(ClusterMetricsBuilder.java:80)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at
java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at
java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
at
org.apache.hadoop.hbase.ClusterMetricsBuilder.toClusterStatus(ClusterMetricsBuilder.java:80)
at
org.apache.hadoop.hbase.master.MasterRpcServices.getClusterStatus(MasterRpcServices.java:980)
at
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:102)
at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82)
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-27888) Record readBlock message in log when it takes too long time
chaijunjie created HBASE-27888: -- Summary: Record readBlock message in log when it takes too long time Key: HBASE-27888 URL: https://issues.apache.org/jira/browse/HBASE-27888 Project: HBase Issue Type: Improvement Components: HFile Affects Versions: 2.4.17, 2.5.3, 2.4.15 Reporter: chaijunjie -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27887) IntegrationTestImportTsv doesn't respect -Dfs.defaultFS to override FS
Pratyush Bhatt created HBASE-27887:
--
Summary: IntegrationTestImportTsv doesn't respect -Dfs.defaultFS
to override FS
Key: HBASE-27887
URL: https://issues.apache.org/jira/browse/HBASE-27887
Project: HBase
Issue Type: Bug
Components: compatibility
Reporter: Pratyush Bhatt
Tried modifying the command using -Dfs.defaultFS to run the integration tests
on top of Ozone:
{noformat}
hbase org.apache.hadoop.hbase.mapreduce.IntegrationTestImportTsv
-Dfs.defaultFS=ofs://ozone1
-Dhbase.fs.tmp.dir=ofs://ozone1/vol1/bucket1/hbase/bulkload{noformat}
Still getting:
{noformat}
2023-05-24 19:16:12,683|INFO|MainThread|machine.py:203 -
run()||GUID=b47165ac-c898-410f-bbf9-eaaca57cb34f|Exception in thread "main"
java.lang.IllegalArgumentException: Wrong FS:
ofs://ozone1/vol1/bucket1/hbase/bulkload/partitions_b5cae401-0fb6-4322-b2bd-892eed7786ef,
expected: hdfs://ns1{noformat}
Full trace:
{noformat}
1.cdh7.1.8.p3.40935426/bin/../lib/hadoop/lib/native
2023-05-24 19:16:07,662|INFO|MainThread|machine.py:203 -
run()||GUID=b47165ac-c898-410f-bbf9-eaaca57cb34f|23/05/24 19:16:07 INFO
zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
2023-05-24 19:16:07,662|INFO|MainThread|machine.py:203 -
run()||GUID=b47165ac-c898-410f-bbf9-eaaca57cb34f|23/05/24 19:16:07 INFO
zookeeper.ZooKeeper: Client environment:java.compiler=
2023-05-24 19:16:07,662|INFO|MainThread|machine.py:203 -
run()||GUID=b47165ac-c898-410f-bbf9-eaaca57cb34f|23/05/24 19:16:07 INFO
zookeeper.ZooKeeper: Client environment:os.name=Linux
2023-05-24 19:16:07,662|INFO|MainThread|machine.py:203 -
run()||GUID=b47165ac-c898-410f-bbf9-eaaca57cb34f|23/05/24 19:16:07 INFO
zookeeper.ZooKeeper: Client environment:os.arch=amd64
2023-05-24 19:16:07,662|INFO|MainThread|machine.py:203 -
run()||GUID=b47165ac-c898-410f-bbf9-eaaca57cb34f|23/05/24 19:16:07 INFO
zookeeper.ZooKeeper: Client environment:os.version=5.4.0-135-generic
2023-05-24 19:16:07,663|INFO|MainThread|machine.py:203 -
run()||GUID=b47165ac-c898-410f-bbf9-eaaca57cb34f|23/05/24 19:16:07 INFO
zookeeper.ZooKeeper: Client environment:user.name=hrt_qa
2023-05-24 19:16:07,663|INFO|MainThread|machine.py:203 -
run()||GUID=b47165ac-c898-410f-bbf9-eaaca57cb34f|23/05/24 19:16:07 INFO
zookeeper.ZooKeeper: Client environment:user.home=/home/hrt_qa
2023-05-24 19:16:07,663|INFO|MainThread|machine.py:203 -
run()||GUID=b47165ac-c898-410f-bbf9-eaaca57cb34f|23/05/24 19:16:07 INFO
zookeeper.ZooKeeper: Client environment:user.dir=/hwqe/hadoopqe
2023-05-24 19:16:07,663|INFO|MainThread|machine.py:203 -
run()||GUID=b47165ac-c898-410f-bbf9-eaaca57cb34f|23/05/24 19:16:07 INFO
zookeeper.ZooKeeper: Client environment:os.memory.free=96MB
2023-05-24 19:16:07,663|INFO|MainThread|machine.py:203 -
run()||GUID=b47165ac-c898-410f-bbf9-eaaca57cb34f|23/05/24 19:16:07 INFO
zookeeper.ZooKeeper: Client environment:os.memory.max=228MB
2023-05-24 19:16:07,663|INFO|MainThread|machine.py:203 -
run()||GUID=b47165ac-c898-410f-bbf9-eaaca57cb34f|23/05/24 19:16:07 INFO
zookeeper.ZooKeeper: Client environment:os.memory.total=145MB
2023-05-24 19:16:07,666|INFO|MainThread|machine.py:203 -
run()||GUID=b47165ac-c898-410f-bbf9-eaaca57cb34f|23/05/24 19:16:07 INFO
zookeeper.ZooKeeper: Initiating client connection,
connectString=ozn-lease19-4.ozn-lease19.root.hwx.site:2181,ozn-lease19-1.ozn-lease19.root.hwx.site:2181,ozn-lease19-2.ozn-lease19.root.hwx.site:2181
sessionTimeout=3
watcher=org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$16/1316897284@50feaef5
2023-05-24 19:16:07,673|INFO|MainThread|machine.py:203 -
run()||GUID=b47165ac-c898-410f-bbf9-eaaca57cb34f|23/05/24 19:16:07 INFO
common.X509Util: Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to
disable client-initiated TLS renegotiation
2023-05-24 19:16:07,683|INFO|MainThread|machine.py:203 -
run()||GUID=b47165ac-c898-410f-bbf9-eaaca57cb34f|23/05/24 19:16:07 INFO
zookeeper.ClientCnxnSocket: jute.maxbuffer value is 4194304 Bytes
2023-05-24 19:16:07,692|INFO|MainThread|machine.py:203 -
run()||GUID=b47165ac-c898-410f-bbf9-eaaca57cb34f|23/05/24 19:16:07 INFO
zookeeper.ClientCnxn: zookeeper.request.timeout value is 0. feature enabled=
2023-05-24 19:16:07,712|INFO|MainThread|machine.py:203 -
run()||GUID=b47165ac-c898-410f-bbf9-eaaca57cb34f|23/05/24 19:16:07 INFO
zookeeper.Login: Client successfully logged in.
2023-05-24 19:16:07,714|INFO|MainThread|machine.py:203 -
run()||GUID=b47165ac-c898-410f-bbf9-eaaca57cb34f|23/05/24 19:16:07 INFO
zookeeper.Login: TGT refresh thread started.
2023-05-24 19:16:07,742|INFO|MainThread|machine.py:203 -
run()||GUID=b47165ac-c898-410f-bbf9-eaaca57cb34f|23/05/24 19:16:07 INFO
zookeeper.Login: TGT valid starting at:Wed May 24 18:49:50 UTC 2023
2023-05-24 19:16:07,743|INFO|MainThread|machine.py:203 -
run()||GUID=b47165ac-c898-410f-bbf9-eaaca57cb34f|23/05/24 19:16:07 INFO
zookeeper.Login: TGT
[jira] [Created] (HBASE-27886) wal.AbstractProtobufLogWriter: Failed to write trailer, non-fatal, continuing
Jepson created HBASE-27886: -- Summary: wal.AbstractProtobufLogWriter: Failed to write trailer, non-fatal, continuing Key: HBASE-27886 URL: https://issues.apache.org/jira/browse/HBASE-27886 Project: HBase Issue Type: Bug Components: wal Affects Versions: 2.2.2 Reporter: Jepson 2023-05-25 03:33:20,002 WARN [AsyncFSWAL-0] wal.AsyncFSWAL: sync failed java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:424) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:513) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.sync(AsyncProtobufLogWriter.java:144) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:352) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.consume(AsyncFSWAL.java:539) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2023-05-25 03:33:20,050 INFO [regionserver/bdp04:16020.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/bdp04,16020,1684954879778/bdp04%2C16020%2C1684954879778.1684956631604 with entries=179, filesize=1.50 MB; new WAL /hbase/WALs/bdp04,16020,1684954879778/bdp04%2C16020%2C1684954879778.168495683 2023-05-25 03:33:20,050 WARN [Close-WAL-Writer-8] wal.AbstractProtobufLogWriter: Failed to write trailer, non-fatal, continuing... java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:424) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:513) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.lambda$writeWALTrailerAndMagic$3(AsyncProtobufLogWriter.java:220) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.write(AsyncProtobufLogWriter.java:176) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALTrailerAndMagic(AsyncProtobufLogWriter.java:211) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.writeWALTrailer(AbstractProtobufLogWriter.java:235) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:153) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$6(AsyncFSWAL.java:646) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2023-05-25 03:33:20,050 WARN [Close-WAL-Writer-8] wal.AsyncProtobufLogWriter: normal close failed, try recover java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:521) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.close(FanOutOneBlockAsyncDFSOutput.java:569) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:154) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$6(AsyncFSWAL.java:646) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-27876) Only generate SBOM when releasing
[ https://issues.apache.org/jira/browse/HBASE-27876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-27876. --- Fix Version/s: 2.6.0 3.0.0-alpha-4 2.5.5 2.4.18 Hadoop Flags: Reviewed Resolution: Fixed Pushed to branch-2.4+. Thanks [~yamasakisua] for contributing! > Only generate SBOM when releasing > - > > Key: HBASE-27876 > URL: https://issues.apache.org/jira/browse/HBASE-27876 > Project: HBase > Issue Type: Improvement > Components: build, pom >Reporter: Duo Zhang >Assignee: Shuhei Yamasaki >Priority: Minor > Fix For: 2.6.0, 3.0.0-alpha-4, 2.5.5, 2.4.18 > > > The CycloneDX generation slows down the build so we'd better only generate it > when releasing, to speed up the building. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27885) expose metaCacheHits in MetricsConnection
ruanhui created HBASE-27885: --- Summary: expose metaCacheHits in MetricsConnection Key: HBASE-27885 URL: https://issues.apache.org/jira/browse/HBASE-27885 Project: HBase Issue Type: New Feature Components: Client Affects Versions: 3.0.0-alpha-3 Reporter: ruanhui Assignee: ruanhui Fix For: 3.0.0-alpha-4 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27884) [hbase-filesystem] Use log4j2 instead of log4j for logging
Peter Somogyi created HBASE-27884: - Summary: [hbase-filesystem] Use log4j2 instead of log4j for logging Key: HBASE-27884 URL: https://issues.apache.org/jira/browse/HBASE-27884 Project: HBase Issue Type: Task Components: Filesystem Integration Reporter: Peter Somogyi Move to log4j2 in hbase-filesystem. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27883) [hbase-connectors] Use log4j2 instead of log4j for logging
Peter Somogyi created HBASE-27883: - Summary: [hbase-connectors] Use log4j2 instead of log4j for logging Key: HBASE-27883 URL: https://issues.apache.org/jira/browse/HBASE-27883 Project: HBase Issue Type: Task Components: hbase-connectors Reporter: Peter Somogyi Fix For: hbase-connectors-1.1.0 Move to log4j2 in hbase-connectors. -- This message was sent by Atlassian Jira (v8.20.10#820010)
