Hi dev, Thanks everyone for supporting this!
> How will we avoid any conflicting commits that may go into “master" between steps 2 and 4? The conflicts are easy to resolve. I just tried merging the non-(gRPC zero-copy) commits from the master branch to a master-tmp branch [1]. There are 4 sections of commits in [2]: # Section 1: Pre-3.1.3 commits to be reworked This section only has one JIRA RATIS-2129 (do not acquire the RaftLog readLock for read-only operations). I plan to reopen the JIRA and re-work it. The reason is that the locking mechanism is not clear in RATIS-2129 with the existing code. We should make the locking mechanism obvious so it is easy to answer why the RaftLog readLock is not required. # Section 2: Pre-3.1.3 commits should be merged to the new master A few pre-3.1.3 non-(gRPC zero-copy) commits which somehow were missing in our previous releases. # Section 3: Post-3.1.3 commits should be merged to the new master The post-3.1.3 non-(gRPC zero-copy) commits should be merged to the new master branch. # Section 4: gRPC zero-copy commits should NOT be merged to the new master The gRPC zero-copy commits be excluded from the master branch. The master-tmp branch [1] was created by copy-and-pasting the cherry-pick in the order in [2]. [3] and [4] are the commits respectively in the master branch and the release-3.1.3 branch after RATIS-1994, where RATIS-1994 is the last matching commit of these two branches. [2] was generated by comparing [3] and [4]. Please review the master-tmp branch [1]. Thank you in advance. Tsz-Wo [1] https://github.com/szetszwo/ratis/tree/master-tmp [2] attachment: new-master_cherry-pick.sh.txt [3] attachment: master.sh.txt (all commits in master after RATIS-1994) [3] attachment: 3.1.3.sh.txt (all commits in release-3.1.3 after RATIS-1994) On Wed, Mar 26, 2025 at 8:15 AM Arpit Agarwal <[email protected]> wrote: > +1 - the main branch should remain healthy. > > How will we avoid any conflicting commits that may go into “master" between > steps 2 and 4? > > > On Mar 24, 2025 at 1:48:26 PM, Tsz Wo Sze <[email protected]> wrote: > > > Hi Ratis dev, > > > > The gRPC zero-copy feature [1] is still under development. However, > since > > the changes of the incomplete feature are merged to the master branch, > the > > gRPC > > zero-copy features keep causing test failures in our CI. It becomes very > > inconvenient for other developers. Therefore, we suggest moving the > > zero-copy feature to a dev branch and removing it from the master branch. > > > > How to do it? > > 1. Create a tmp branch using the 3.1.3 release tag. > > 2. Merge the non-(gRPC zero-copy) commits from the master branch to it. > > 3. Copy the current master branch to a new gRPC zero-copy branch > > 4. Force push (i.e. overwrite) the tmp branch as the master branch. > > > > What do you think? > > > > Tsz-Wo > > [1] gRPC zero-copy: https://issues.apache.org/jira/browse/RATIS-1931 > > >
echo This script $0 is NOT for running directly. exit 1 ########################################## # Section 1: Pre-3.1.3 commits to be re-worked git cherry-pick 781d61d37411b374f104eb0806e1e2c4090fb35e #RATIS-2129. Low replication performance because LogAppender is often blocked by RaftLog's readLock. (#1141) ########################################## # Section 2: Pre-3.1.3 commits should be merged to the new master git cherry-pick a0c4255d5154a9972d1a6d4aba49fff5d96ea4b3 #RATIS-2167. Add default value in TermIndex (#1161) git cherry-pick da3f90bc3a839bfb61b4d518b04eff93e1ee9071 #RATIS-2169. mvnw test: Corrupted STDOUT by directly writing to native stream in forked JVM 1 (#1162) git cherry-pick e96ed1a33840385446f4e647864a169467da5ab7 #RATIS-2168. Support custom gRPC services. (#1169) # Conflicts: GrpcConfigKeys, GrpcServicesImpl, MiniRaftClusterWithGrpc git cherry-pick 237b7c013b96821844d70879dbb6c662814a3be1 #RATIS-2182. Fix some Objects.requireNonNull messages (#1172) ########################################## # Section 3: Post-3.1.3 commits should be merged to the new master git cherry-pick 72c920c6e8e4bd24b254ad9e56514e1fa9a66283 #RATIS-2237 Add getFollowerMatchIndices interface for DivisionInfo # fix commit message git cherry-pick 8353a017fe6545fbfb74960ecb3a0f4396c478d2 #RATIS-2239. Publish build scans to develocity.apache.org (#1212) git cherry-pick 345641fb4ce434197981982b08f8d78f412753fb #RATIS-2240. updatePurgeIndex doesn't need to hold RaftLogBase.writeLock (#1214) git cherry-pick 2664ac8e50bf52202b581555134cbdf33c057603 #RATIS-2244. Reduce the number of log messages during bootstrap (#1217) git cherry-pick af8482c31f72e58224e7bda2e7114b4b61f9f636 #RATIS-2247. Bump GitHub Actions runner to ubuntu-24.04 (#1219) git cherry-pick bbe20b3ba76ea3b75b3a4fe690cc2beb067d62be #RATIS-2248. Rename CI workflows to .yaml (#1220) git cherry-pick a9ebdb636a9c8bb5e874c04309ac7379919e2d7a #RATIS-2249. Remove unused shellcheck.sh (#1221) git cherry-pick 663a44b5fe39ceb08f9f2e94a0b85c26da302c54 #RATIS-2245. Ratis should wait for all apply transaction futures before taking snapshot and group remove (#1218) #Conflict: StateMachineShutdownTests:L75 git cherry-pick fde8c7d4f37f37c61d5e270cf069a8a604ea0d6e #RATIS-2242 change consistency criteria of heartbeat during appendLog (#1215) #Conflict: RaftServerImpl, RaftServerProxy # fix commit message git cherry-pick 6238fe30fd2e54cdd6ec1a66edc383d033c7b47a #RATIS-2252. Replace JUnitRunListener with JUnit5TestExecutionListener (#1228) git cherry-pick e604110af2a31708e047858db45f4d47498495df #RATIS-2217. Automatically re-try flaky tests in CI (#1229) #Conflict: TestMultiRaftGroup, LeaderElectionTests git cherry-pick a477d6d289982a20d479f58fdc6a9daf835b1162 #RATIS-2254. Replace junit 4 Timeout with junit 5. (#1230) #Conflict: BaseTest git cherry-pick 59f199232e3ba268b436e48b2ff43367a28ac4f7 #RATIS-2022. Replace some Guava usage with builtins (#1231) git cherry-pick 09ff1e1f8e3c03bae476b1d9f3e8224bcfdbd491 #RATIS-2256. Update Maven to 3.9.9 (#1234) git cherry-pick d740e513555a3c4bbf26cab061bca5ff63bb1859 #RATIS-2255. Migrate ratis-grpc to junit 5. (#1233) #Conflict: MiniRaftClusterWithGrpc: no change git cherry-pick fab8a92dfb6732d4912b6dfcc6006fbc5212be7e #RATIS-2261. Mark TestRaftSnapshotWithGrpc as flaky git cherry-pick 3947cf38c5d749cd2dc6d1b6e12c0c46824d0897 #RATIS-2262. Mark RaftBasicTests#testStateMachineMetrics as flaky git cherry-pick 0bc5aad71ff735e923b2492edbde9a803c682078 #RATIS-2260. AtomicFileOutputStream should truncate temp file (#1237) git cherry-pick f00d764933a9793cc2d8168910151e8ce84ea907 #RATIS-2263. Bump sonar-maven-plugin to 5.0.0.4389 (#1238) git cherry-pick ac35188a4fb285842a05224141e3879aaf71e621 #RATIS-2258. Caching TermIndex objects (#1239) #Missing RaftTestUtil.gc(); git cherry-pick 7723c2b141e8c8be121c214926c2cc71145765bd #RATIS-2266. Use WeakValueCache instead of Guava cache in RaftId. (#1240) git cherry-pick 93ab53c45af8e10d7a7b88e79bc34621fca6e8e2 #RATIS-2268. A minor documatention fix for method getSuccessors. (#1242) git cherry-pick 72c17652f0bd8446f59fd13ed9e295c278e1fb95 #RATIS-2269. Remove all import java.util.* statements. (#1243) #Conflict: SegmentedRaftLogWorker ########################################## # Section 4: gRPC zero-copy commits should NOT be merged to the new master #RATIS-2228. Refactor the offered map in LogAppenderBase.nextAppendEntriesRequest (#1201) #RATIS-2227. LogEntryProto leak in SegmentedRaftLog (#1199) #RATIS-2184. Improve TestRaftWithGrpc test stability (#1177) #RATIS-2225. RaftClientRequest leak in RaftServerImpl. (#1198) #RATIS-2220. Skip further tests after leak detected (#1193) #RATIS-2173. Fix zero-copy bugs for non-gRPC cases. (#1167) #RATIS-2164. LeakDetector has a race condition. (#1163) #RATIS-2151. TestRaftWithGrpc keeps failing with zero-copy. (#1164) #RATIS-2120. Bump version after 3.1.0 release (#1118) #RATIS-2114. Corruption due to SegmentedRaftLogWorker queue LogEntry without reference counter (#1113) #RATIS-2094. Avoid corruptions from TransactionContext's stateMachineLogEntry and stateMachineContext. (#1106) #RATIS-2093. Decouple metadata and configuration entries from appendEntries buffer for stateMachineCache' (#1096) #RATIS-2096. Add a conf to enable/disable zero copy. (#1099) #RATIS-2092. Add metrics to expose number of zero-copy unclosed messages (#1095) #RATIS-2077. Timedout StateMachine retainRead is released twice (#1081) #Revert "RATIS-2080. Reuse LeaderElection executor. (#1082)" #RATIS-2080. Reuse LeaderElection executor. (#1082) #RATIS-2059. Missing reference count when putting log entries to cache on follower. (#1067) #RATIS-1979. Allow StateMachine.read to return a ReferentCountedObject (#1062) #RATIS-2026. LogAppender to consume log entries with reference count (#1049) #RATIS-2028. Refactor RaftLog to supply log as ReferenceCountedObject (#1045) #RATIS-2020. Refactor TransactionContext to supply LogEntryProto via a ReferenceCountedObject (#1042) #RATIS-1997. Refactor StateMachine interface to use ReferenceCountedObject (#1036) #RATIS-1978. Add tests assertions to verify all zero-copy messages are released properly (#1023) #RATIS-2018. Zero-copy buffers are not released - 2nd chunk (#1032) #RATIS-2007. Zero-copy buffers are not released (#1027) #RATIS-1934. Support Zero-Copy in GrpcServerProtocolService (#1014) #RATIS-2009. ReferenceCount should work for all LogEntry types. (#1021) #RATIS-1990. Refactor appendEntries processing to support reference count (#1011) #RATIS-1925. Support Zero-Copy in GrpcClientProtocolService (#1007)
echo This script $0 is NOT for running directly. exit 1 ########################################## git cherry-pick ac35188a4fb285842a05224141e3879aaf71e621 #RATIS-2258. Caching TermIndex objects (#1239) git cherry-pick f00d764933a9793cc2d8168910151e8ce84ea907 #RATIS-2263. Bump sonar-maven-plugin to 5.0.0.4389 (#1238) git cherry-pick 0bc5aad71ff735e923b2492edbde9a803c682078 #RATIS-2260. AtomicFileOutputStream should truncate temp file (#1237) git cherry-pick 3947cf38c5d749cd2dc6d1b6e12c0c46824d0897 #RATIS-2262. Mark RaftBasicTests#testStateMachineMetrics as flaky git cherry-pick fab8a92dfb6732d4912b6dfcc6006fbc5212be7e #RATIS-2261. Mark TestRaftSnapshotWithGrpc as flaky git cherry-pick d740e513555a3c4bbf26cab061bca5ff63bb1859 #RATIS-2255. Migrate ratis-grpc to junit 5. (#1233) git cherry-pick 09ff1e1f8e3c03bae476b1d9f3e8224bcfdbd491 #RATIS-2256. Update Maven to 3.9.9 (#1234) git cherry-pick 0091c1ca4975efefd1c137877c5b5fb08fb662b4 #RATIS-2124. Remove the use of org.junit.Rule. (#1232) git cherry-pick 59f199232e3ba268b436e48b2ff43367a28ac4f7 #RATIS-2022. Replace some Guava usage with builtins (#1231) git cherry-pick a477d6d289982a20d479f58fdc6a9daf835b1162 #RATIS-2254. Replace junit 4 Timeout with junit 5. (#1230) git cherry-pick e604110af2a31708e047858db45f4d47498495df #RATIS-2217. Automatically re-try flaky tests in CI (#1229) git cherry-pick 6238fe30fd2e54cdd6ec1a66edc383d033c7b47a #RATIS-2252. Replace JUnitRunListener with JUnit5TestExecutionListener (#1228) git cherry-pick fde8c7d4f37f37c61d5e270cf069a8a604ea0d6e #RATIS-2242 change consistency criteria of heartbeat during appendLog (#1215) git cherry-pick 663a44b5fe39ceb08f9f2e94a0b85c26da302c54 #RATIS-2245. Ratis should wait for all apply transaction futures before taking snapshot and group remove (#1218) git cherry-pick a9ebdb636a9c8bb5e874c04309ac7379919e2d7a #RATIS-2249. Remove unused shellcheck.sh (#1221) git cherry-pick bbe20b3ba76ea3b75b3a4fe690cc2beb067d62be #RATIS-2248. Rename CI workflows to .yaml (#1220) git cherry-pick af8482c31f72e58224e7bda2e7114b4b61f9f636 #RATIS-2247. Bump GitHub Actions runner to ubuntu-24.04 (#1219) git cherry-pick 2664ac8e50bf52202b581555134cbdf33c057603 #RATIS-2244. Reduce the number of log messages during bootstrap (#1217) git cherry-pick 345641fb4ce434197981982b08f8d78f412753fb #RATIS-2240. updatePurgeIndex doesn't need to hold RaftLogBase.writeLock (#1214) git cherry-pick 8353a017fe6545fbfb74960ecb3a0f4396c478d2 #RATIS-2239. Publish build scans to develocity.apache.org (#1212) git cherry-pick 72c920c6e8e4bd24b254ad9e56514e1fa9a66283 #RATIS-2237 Add getFollowerMatchIndices interface for DivisionInfo git cherry-pick bdde3ae9a20c0546f0ca268e5e9d94efd8f784a5 #RATIS-2236 Fixed bug where manual triggerSnapshot would never finish (#1207) git cherry-pick 17c965292133114d6169feb19f32670e2b122032 #RATIS-2215. Bump maven-remote-resources-plugin to 3.3.0 (#1208) git cherry-pick 9b7440123fe543bb0d0fba1a6dc97da40d0f681a #RATIS-2235. Allow only one thread to perform appendLog (#1206) git cherry-pick 5e6cc9de3fd10c4435839035087cd9d13f4b254d #RATIS-2233. make NOPROGRESS timeout configurable (#1204) git cherry-pick 6f34fb10beeb5a633650f17e30e1a321fef43c5b #RATIS-2229. Do not print the same conf values multiple times. (#1200) #RATIS-2228. Refactor the offered map in LogAppenderBase.nextAppendEntriesRequest (#1201) git cherry-pick 539e80405ec6d9d767e09e3c84da4778cf704a33 #RATIS-2234. Remove lock race between heartbeat and append log channels (#1205) git cherry-pick 053683fa483d338af01aea55f33665907b27e14a #RATIS-2232. Improve log for debugging on RaftLog / TransactionManager (#1203) git cherry-pick 320b2072bf5161d3546cad331daacbf686cd8e16 #RATIS-2231. Bump ratis-thirdparty to 1.0.8 (#1202) git cherry-pick 26912d91bbc6bd430c9618eee05f75106889f417 #RATIS-2200. Make Ratis example log at INFO level (#1189) git cherry-pick 117d03abeea7092f47c29654acd5aee8cf4a9488 #RATIS-2197. Clean remote stream to resolve direct memory leak (#1179) #RATIS-2227. LogEntryProto leak in SegmentedRaftLog (#1199) git cherry-pick c2ae86420dfc62ec88453b70a8e2155414e3e586 #RATIS-2226. Enable Develocity local build cache (#1196) git cherry-pick 6fd8ae1293044973c2cd22ba4800cea35ebf9f34 #RATIS-2210. Reduce duplication in CI workflow (#1195) #RATIS-2184. Improve TestRaftWithGrpc test stability (#1177) #RATIS-2225. RaftClientRequest leak in RaftServerImpl. (#1198) git cherry-pick de2fec6baaaa7598d533d48ee873558d307eebe9 #RATIS-2219. Remove duplicate test case from TestRaftWithGrpc (#1192) #RATIS-2220. Skip further tests after leak detected (#1193) git cherry-pick 551b8c94b78b8e8611c6c77794617d12682a9140 #RATIS-2222. Remove copy-rename-maven-plugin usage (#1194) git cherry-pick 92c4f72f1549fde810a612601429ab52d155d1a5 #RATIS-212. Fix typo in RaftAsyncTests (#1197) git cherry-pick 318c8970774f16d6cac12faaf82404d3e8086d91 #RATIS-2216. Bump maven-shade-plugin to 3.6.0 (#1191) git cherry-pick a0eada8b33c0418b967669ef2100e4ea005815b0 #RATIS-2211. publish-mvn fails with: ... did not assign a file to the build artifact (#1190) git cherry-pick 703189ed6c3c4671730367f92c062f8418de623f #RATIS-2206. Jars in Maven repo and binary tarball are not the same (#1188) git cherry-pick e9755a63b1392bc2d665c1ca2b5e566e65d8b175 #RATIS-2204. Avoid downloads from repository.apache.org (#1187) git cherry-pick a4826e2393a529cbe60d4e76084f2d5bf28f594b #RATIS-2185. Improve gRPC log messages debugability. (#1186) git cherry-pick 3efec8d78e143939234748e2d614dbcf5fbba467 #RATIS-2201. Bump ratis-thirdparty from 1.0.6 to 1.0.7 (#1185) git cherry-pick ccbe9a62ebd7eec51cd4c8b4b4a0d81c673529c6 #RATIS-2194. FileLock didn't unlock properly (#1183) git cherry-pick 04396ca780d857863e81a7bf48ae52fa9cbec2b2 #RATIS-2191. Add ResourceLeakDetector to Netty tests (#1180) git cherry-pick 06affdf69973c6fea57df5e601c735f3c68fc634 #RATIS-2189. Use ByteBufAllocator#ioBuffer in NettyDataStreamUtils (#1178) git cherry-pick 13b8cdd0ccdd9ae2578600fbc454315b339da4a1 #RATIS-2186. Raft log should not purge index lower than the log start index (#1175) git cherry-pick 26c1f04ccae5fd176b9f6071fe6be4b1604a05ef #RATIS-2177. Purge should delete segmentLog from small to large according to logIndex (#1174) git cherry-pick e75a0d51deab734bdaff4a9a743995c9feb242bc #RATIS-2183. Detect staled snapshot request. (#1173) git cherry-pick 237b7c013b96821844d70879dbb6c662814a3be1 #RATIS-2182. Fix some Objects.requireNonNull messages (#1172) git cherry-pick bf15765a58ac1476a13239d812df0c0b172acb22 #RATIS-2178. Add a log message whenever deleting a RaftLog segment. (#1170) git cherry-pick 3a51121adaf2145e4ec020f4c24858f9f03745d2 #RATIS-2179. Fix NPE in RaftServerImpl.getLogInfo. (#1171) git cherry-pick d08ba81c26bd5e3ab862be3649c7d5ec541a9b04 #RATIS-2174. Move future.join outside the lock (#1168) #RATIS-2173. Fix zero-copy bugs for non-gRPC cases. (#1167) git cherry-pick e96ed1a33840385446f4e647864a169467da5ab7 #RATIS-2168. Support custom gRPC services. (#1169) #RATIS-2164. LeakDetector has a race condition. (#1163) git cherry-pick 62ae6d9f068ce01dd467a6fa83fb74dad9c2c8d8 #RATIS-2172. RaftServer may lose FollowerState (#1166) #RATIS-2151. TestRaftWithGrpc keeps failing with zero-copy. (#1164) git cherry-pick 09f6b392b3fb5738a1a8a8efa771b05bc1a543dd #RATIS-2166. Update nextChunkIndex when snapshot is ALREADY_INSTALLED (#1159) git cherry-pick de4e24d4c026719d59d055933147b7c80ee1f702 #RATIS-2165. Do not include RaftClientRequest Message in ReadIndexRequest (#1160) git cherry-pick da3f90bc3a839bfb61b4d518b04eff93e1ee9071 #RATIS-2169. mvnw test: Corrupted STDOUT by directly writing to native stream in forked JVM 1 (#1162) git cherry-pick a0c4255d5154a9972d1a6d4aba49fff5d96ea4b3 #RATIS-2167. Add default value in TermIndex (#1161) git cherry-pick e3d6736c3c4072d100bfe8e1d871705c18348897 #RATIS-2162. When closing leaderState, if the logAppender thread sends a snapshot, a deadlock may occur. (#1154) git cherry-pick acb4d1a0957b3268123215ef9cfb5d101213f847 #RATIS-2160. MetricRegistriesLoader should not print INFO message in CLI. (#1152) git cherry-pick 8a5009914e7783552cdcd3ca3995c5ec5648e625 #RATIS-2158. Let the snapshot sender and receiver use a new digester each time (#1151) git cherry-pick 6451f9b3ee3784620f6fb8984bf305f4df11a185 #RATIS-2155. Add a builder for RatisShell. (#1150) git cherry-pick 94f3fef161fab04bbd79024cf27edb47fe925210 #RATIS-2157 Enhance make_rc.sh for non-first rc at release time #1149 git cherry-pick 0c9e299d0bde858fde9d34caf3cf31076637c788 #RATIS-2147. Md5 mismatch when snapshot install (#1142) git cherry-pick 37386c4b0e3defcb8eeb0a9286706245b7100cf0 #RATIS-2150. No need for manual assembly:single execution when mvn deploy #1144 git cherry-pick 9ac832bc1dccbb437addcb4c36feae360e7b4538 #RATIS-2152. GrpcLogAppender stucks while sending an installSnapshot notification request (#1146) git cherry-pick e6fe8fca4e6aa9d58c8bdcdba2b5fb1b7825ffb2 #RATIS-2154. The old leader may send appendEntries after term changed. (#1148) git cherry-pick 19184326a00535d28f8c09dfcd37852c3a8b1266 #RATIS-2153. ratis-version.properties missing from src bundle (#1147) git cherry-pick 3e38e6da1053c1b522d51123b8ac88ea15e60f94 #RATIS-2149.Do not perform leader election if the current RaftServer has not started yet (#1143) git cherry-pick d1e76cf2b0f5ce31a6e802d57e01537b7e920612 #RATIS-2148. Snapshot transfer may cause followers to trigger reloadStateMachine incorrectly (#1145) git cherry-pick 8f5159db4cade67b96c7b9c8589e7c0cdba571e0 #RATIS-2146. Fixed possible issues caused by concurrent deletion and election when member changes (#1140) git cherry-pick 781d61d37411b374f104eb0806e1e2c4090fb35e #RATIS-2129. Low replication performance because LogAppender is often blocked by RaftLog's readLock. (#1141) git cherry-pick dfed1012983d1d7b5fb2c408e19b8661cbe000b4 #RATIS-2145. Follower hangs until the next trigger to take a snapshot. (#1139) git cherry-pick 520ecab157c9c6aff87ed5c5978c08f98cd4ec6c #Revert "RATIS-2099. Cache TermIndexImpl instead of using anonymous class (#1100)" git cherry-pick da5d508caffc4ca90b0ab962b5105785b9774daa #Revert "RATIS-2101. Move TermIndex.PRIVATE_CACHE to Util.CACHE (#1103)" git cherry-pick c266cad48fd6e11d1cfba924b3451ada512a71c1 #RATIS-2113. Use consistent method names and parameter types in RaftUtils. (#1135) git cherry-pick 17a408f951c422294c220e565e46e7220c27377f #RATIS-2140. Thread wait when installing snapshot. (#1137) git cherry-pick 24fb55363e94f171ab8e1d736bdaf3745f16719f #RATIS-2144. SegmentedRaftLogWorker should close the stream before releasing the buffer. (#1138) git cherry-pick 287a4ac7c467c5a5132c0e85bc83923e27a0f064 #RATIS-2137. Fix LogAppenderDefault in handling of INCONSISTENCY. (#1136) git cherry-pick 8d4c4ae3082f24e22aab454ec144f384ec6e1b1a #RATIS-2139. Fix checkstyle:ParameterNumber in GrpcService. (#1134) git cherry-pick 217fdb082c18debf4b987b2f8536a3bf1d9e92bd #RATIS-2138 Remove uncessary error log in JVMPauseMonitor When GC is serious (#1133) git cherry-pick 69ec8f43621e30c7bddb032e8b2c18533a86053e #RATIS-2135. The leader keeps sending inconsistent entries repeatedly to followers. (#1132) git cherry-pick 177097cd31a103543680b0a063774d6f3ab86a0f #RATIS-2115. Fix flaky RetryCacheTests and InstallSnapshotFromLeaderTests. (#1129) git cherry-pick ebeab934472c42da5344729f35a7758baa596a13 #RATIS-2136 Fixed the issue that RatisServer could not start after frequent restarts (#1131) git cherry-pick 4ba463789df7876837c728ce6477fd18b787fed9 #RATIS-2134. `logMetadata` could miss appending the `metadataEntry` with the lastCommitIndex (#1130) git cherry-pick a793e6c45722e43f295519b56bf99ce876cdb251 #RATIS-2104. Double shutdown in TestLeaderInstallSnapshot. (#1108) git cherry-pick 99925e3224341b95bd195d728a220d41ba3d924a #RATIS-2133. Ignore http proxy for ratis grpc client & server (#1128) git cherry-pick 011dc0addd55e755e3b59282891d2dc06d283c36 #RATIS-2126. Optimize NotLeaderException message (#1124) git cherry-pick fb4968f1738172615d9c21f1673e49599dc13766 #RATIS-2131. Surround with [] only if hostName is a IPv6 string (#1125) git cherry-pick 7825b4f032653efc0cf75ff7e650f8055707f9d1 #RATIS-1071. NettyClientRpc supports sendRequestAsync. Contributed by Rui Wang and Tsz-Wo Nicholas Sze. (#1122) git cherry-pick 9bb6953217186d68362ae68492d7080874431158 #RATIS-2125. javadoc jars should not include non-doc items (#1123) git cherry-pick d10167839cf04728b9fd0439add8db64eefe32a4 #RATIS-1840. Avoid including build timestamp in artifacts (#1114) git cherry-pick 32745c3c4ecb9777a6d44e703134ef1181ad2dd2 #RATIS-2116. Fix the issue where RaftServerImpl.appendEntries may be blocked indefinitely (#1116) git cherry-pick eeaf6a463fa9bb59fbbf747c8bc042590c300f5d #RATIS-2121. Set javac --release when compiling with JDK 9+ (#1119) git cherry-pick 1cb6908a573a8948369c9eb0a52fa62c7ae63de3 #RATIS-2118. Improve assembly process (#1117) #RATIS-2120. Bump version after 3.1.0 release (#1118) git cherry-pick ee9e14f54a5053e0ee48f22773f410931b0ee9df #RATIS-2117. No need for manual assembly:single execution (#1115) git cherry-pick 19ead3912fc4017362645cd64015f9ae0bd6ad6a #RATIS-2110. Publish SBOM artifacts (#1110) #RATIS-2114. Corruption due to SegmentedRaftLogWorker queue LogEntry without reference counter (#1113) git cherry-pick 95ea26c29e457ff0f8aee7ff7106536c42c4dc52 #RATIS-2095. Extract common logic of ratis-shell to RaftUtils for reuse (#1098) git cherry-pick 25a41e36f4c94b9a3e37ca9c27feeb893206cb79 #[RATIS-2111] Reinitialize should load the latest snapshot (#1111) git cherry-pick e540e46b8b341c8df1687f030cafd5cbe51d4600 #RATIS-2112. Improve repeat-test workflow (#1112) git cherry-pick dac27e4fd3df042fcf50ed549404e4d82708ed2b #RATIS-2109. RaftLogBase#updateCommitIndex should return true only if commitIndex is increased (#1109) git cherry-pick bf13033547dcb4a27938f9e5f0a923de47a59f45 #RATIS-2106. Add configuration reference for RaftClient (#1107) #RATIS-2094. Avoid corruptions from TransactionContext's stateMachineLogEntry and stateMachineContext. (#1106) git cherry-pick 924a0cdf43557d67209805627bd6b1ec941776f0 #RATIS-2084. Follower reply ALREADY_INSTALLED when install old snapshots from leader (#1091) git cherry-pick bd4ab145d6b1a72d8ed84eab918cda4e61f08681 #RATIS-2102. AsyncApi#send() is not handling retry and reply correctly for replication levels higher than MAJORITY (#1104) git cherry-pick dd75ffbfb94dcd624be0e75171e5c54a12b029b2 #RATIS-2089. Add CommitInfoProto in NotReplicatedException (#1105) git cherry-pick 640749bb39a435eef9dac4a9fec2cdc75b6a939e #RATIS-2100. The `closeFuture` never completed while closing from the `NEW` state (#1102) git cherry-pick 93eb32a8620fdd4e5119592ef32bc50590810c7b #RATIS-2101. Move TermIndex.PRIVATE_CACHE to Util.CACHE (#1103) #RATIS-2093. Decouple metadata and configuration entries from appendEntries buffer for stateMachineCache' (#1096) #RATIS-2096. Add a conf to enable/disable zero copy. (#1099) git cherry-pick 1782cd9f171519c8fd9006f24cbfe6fab91a1daf #RATIS-2099. Cache TermIndexImpl instead of using anonymous class (#1100) git cherry-pick 64ae88f017cb0fd5649bd465027bd524296ada11 #RATIS-2097 Add a conf to enable/disable Metadata entries. (#1101) #RATIS-2092. Add metrics to expose number of zero-copy unclosed messages (#1095) git cherry-pick 16f2508d0b6606970d0a1accf7591461e6ec55df #RATIS-2098. RaftLogTruncateTests may fail. (#1097) git cherry-pick 8c7c444ca2453831bbaf1e5a5f4cb7ba76b232ff #RATIS-2081. Comment that we skip checkstyle on test directory on purpose (#1084) git cherry-pick 9cfd1bf71f523262928cbe09685696ae71dab8e7 #RATIS-2091. Fix other checkstyle issues in test-directory (#1094) git cherry-pick c81902a094344d953f2fc5f5d7ed08dc7c171cdf #RATIS-2066. RaftServerProxy supports close callback (#1072) git cherry-pick 192ce4880866bc05aa09fd3fec3431dbb9ab8e8c #RATIS-2090. Bump ratis-thirdparty to 1.0.6 (#1093) git cherry-pick 29bba5943ae9c7b5d538da62de683edc2eb9d8f0 #RATIS-2088. Fix name and imports related check in test-directory (#1092) git cherry-pick 599835dbab1e5dac2aaa4571f7c54d38c181b50f #RATIS-2083. Timer thread leak when log level is DEBUG (#1086) git cherry-pick 8bc09bbe57fb4e58f8a1367969b5538a34b4ce2b #RATIS-2085. Compile Ratis with JDK 17/21 in CI (#1087) #RATIS-2077. Timedout StateMachine retainRead is released twice (#1081) git cherry-pick c10e7bc8dce8ebea031592b56f90af4cd40235fd #RATIS-2076. Intermittent timeout in TestRaftWithNetty#testWithLoad. (#1090) #Revert "RATIS-2080. Reuse LeaderElection executor. (#1082)" git cherry-pick 0814b896bed41cc3b5e3d99e4e678fed67909874 #RATIS-2086. Autolink Ozone issues in PRs (#1088) git cherry-pick da9eee0fbd05ed5a8ede417ad098e679ef3254bb #RATIS-2087. Remove workaround for GitHub reverse DNS issue (#1089) #RATIS-2080. Reuse LeaderElection executor. (#1082) git cherry-pick e1765016323735124e3aaa5cee44688d49d48791 #RATIS-2078. Correct timeout unit for tests (#1083) git cherry-pick ac05d64f3f846d56658442896a39f4f5fd7495a2 #RATIS-2074. Intermittent fork timeout in TestRaftWithNetty#testBasicAppendEntriesKillLeader. (#1080) git cherry-pick 3888520717402da8638b85ae17f1ab494ecd2872 #RATIS-2067. Enhance content for concept in ratis-docs. (#1073) git cherry-pick 839f8fb265983b5ce14cf2cb3650a70d342ffc04 #RATIS-2073. Enable Surefire process checkers (#1079) git cherry-pick 58671923e78a10d5c6a7d49a06e2d95690d5e9bb #RATIS-2072. Limit timeout of CI workflow jobs (#1078) git cherry-pick 6fc42220bc8e39cf8f242f3205abcd4dce8bbe44 #RATIS-2071. Bump Bouncy Castle to 1.78.1 (#1076) git cherry-pick cbd9793a0e315276d370dbfc5dd6a75ccf67f454 #RATIS-2070. Skip push build for dependabot (#1077) git cherry-pick 51244e42cabb98f7ac9ca144e311222d9c6e5a45 #RATIS-2068. Avoid logging raw StateMachine data body. (#1074) git cherry-pick 4e3eee7c9f603d89b701921c3ab877134b342f0e #RATIS-2069. RaftMetaConf command sets incorrect peerAddress pattern. (#1075) git cherry-pick 1fe59e07dff9b02e46d150de8b46ed2d4fc3abce #RATIS-2065. Avoid the out-of-heap memory OOM phenomenon of frequent creation and deletion of Raft group scenarios. (#1071) git cherry-pick b0ec1143c49e61525baf5da935a3509a59246883 #RATIS-2061. Fix setCloseThreshold parameter in RaftServerConfigKeys (#1070) git cherry-pick 7f1b1b6be29fd0b8675e30be511b03f6342ad7e6 #RATIS-2060 Avoid unexpected shutdown of RaftServerProxy during membership change (#1069) git cherry-pick 8704cd23d3b38a90780ef34a7ed5f7c417ea1207 #RATIS-2055. Move notifyTermIndexUpdated after leader.checkReady (#1068) #RATIS-2059. Missing reference count when putting log entries to cache on follower. (#1067) git cherry-pick cf9d2292a9f53792a7765da41e3b4b903c0ec1d9 #RATIS-2057. Add back JavaUtils.attemptUntilTrue(..). (#1066) git cherry-pick 8913a999dd79cda58d18e17f4dd7971dda7070b7 #RATIS-2056. Update purge index when snapshot installed. (#1065) git cherry-pick 49b4006015699625c445d70e2113fd5904b4590d #RATIS-1504. Add timeout handling to DataStreamManagement#checkSuccessRemoteWrite. (#1064) git cherry-pick 8638a2ffa802de576aae546c22160da3d9339a8f #Ratis-2040. Fix RaftPeerId generated by command of "raftMetaConf" to use real PeerId (#1060) git cherry-pick bc6221b32fff9022cebb02bb243aea9fff35e290 #RATIS-1519. When DataStreamManagement#read an exception occurs, remove DataStream (#596) git cherry-pick f40424422b692349b5496ee30e24335c8186093b #RATIS-2051. Fix Sonar bugs for Ratis. (#1059) #RATIS-1979. Allow StateMachine.read to return a ReferentCountedObject (#1062) git cherry-pick a99e709fee3b922338936c6173e307ef221ef165 #RATIS-2052. Add docs for triggering snapshot when removing raftGroup parameter. (#1061) #RATIS-2026. LogAppender to consume log entries with reference count (#1049) git cherry-pick 14dbbf3d7cf8c78788fe7be93abf100c7dd52580 #RATIS-2050. Add creationGap param to snapshot management API (#1058) git cherry-pick 54c2b1f7f82c5cb66fa6b652a29ff9cef22bf4b1 #RATIS-2049. Suppress Sonar warning for volatile objects (#1057) git cherry-pick 0f631fef459b1263f266fa443fdb26cfbc2f0a75 #RATIS-2036. Avoid trigger snapshot when removing raftGroup (#1055) git cherry-pick be497d5c472038242f8493ef5e56b56b7ce69ef1 #RATIS-2048. Fix SimpleStateMachineStorage potential NPE exception (#1056) git cherry-pick c3d615b4f28b92e1d271b84752f1c9b0fc23341c #RATIS-2047. Avoid unnecessary warn log when creating raft group (#1054) git cherry-pick 72c060d9cc524045ce55e830db070c5514f23ed4 #RATIS-2045. SnapshotInstallationHandler doesn't notify follower when snapshotIndex is -1 and firstAvailableLogIndex is 0 (#1053) git cherry-pick 44a4f1fb73a073589a4563d492f6acfc68ce4a5d #RATIS-2044. Fix ReadIndex loss caused by data race in AppendEntriesListeners (#1052) git cherry-pick 422cb9d4cb3f6be8e3169797d11a3c2e666c138f #RATIS-2043. Change toString() of RaftConfigurationImpl (#1051) #RATIS-2028. Refactor RaftLog to supply log as ReferenceCountedObject (#1045) git cherry-pick 99a833c29eb28203cc6107ede26318ff86e636e1 #Ratis-2031. Add peer info to response of GroupInfoCommand CLI (#1047) git cherry-pick df436c73f66f192a2fb9f8e43500196e4e3a7cec #RATIS-2035. Refactor streaming code for Read. (#1046) git cherry-pick 1bd451046b2d33fe063712d7e811262bef580a4e #RATIS-2038. IllegalStateException when ServerState is closed before start. (#1048) #RATIS-2020. Refactor TransactionContext to supply LogEntryProto via a ReferenceCountedObject (#1042) git cherry-pick 044b111d991a619a7c2edcef235e28a7c623f090 #RATIS-2027. Ratis Streaming: Remote Stream copy data to heap. (#1044) git cherry-pick 67b76509d1d5b0037b278f1808bb0f5ecd7aceed #RATIS-2021. Migrate ratis-test tests to Junit 5 - Part 2. (#1043) git cherry-pick 16b6536e220ccbc8f9228b4fccfab8eb823ee83e #RATIS-2025. Move out assert and proto methods from RaftServerImpl. (#1041) git cherry-pick 1d99921ef4bf676029cf231faf2ca79eff59f44b #RATIS-2024. Refactor appendEntries code. (#1040) #RATIS-1997. Refactor StateMachine interface to use ReferenceCountedObject (#1036) git cherry-pick 4f5612fd89f4ff63a8f529d1517edc663cf84ec2 #RATIS-2023. Remove duplicate RefCountingMap (#1039) git cherry-pick e44de26623a6b65d9d66eb782df70ffd98fab364 #RATIS-1976. Migrate ratis-test tests to Junit 5 - Part 1 (#1038) git cherry-pick a2cb49e8cb64c30c1f7f7adb82f87477501f522d #RATIS-1975. Migrate ratis-metrics-dropwizard3 tests to Junit 5. (#1037) #RATIS-1978. Add tests assertions to verify all zero-copy messages are released properly (#1023) git cherry-pick d2f1c57fe60e3abe1bbfa1820297f9d60736bb42 #RATIS-2019 Fixed abnormal exit of StateMachineUpdater (#1033) git cherry-pick cf897f02957bce7ab587a895143405c914adbf91 #RATIS-1973. Migrate ratis-examples tests to Junit 5. (#1035) #RATIS-2018. Zero-copy buffers are not released - 2nd chunk (#1032) git cherry-pick 9cd74a21da9c007168e90b78a4ea2ee83dfdab49 #RATIS-1974. Migrate ratis-metrics-default tests to Junit 5 (#1034) git cherry-pick 035cd968f934f4a8f9d9ccec082f86b0e6437aab #RATIS-2016. Correct NotificationInstallSnapshot's index (#1030) git cherry-pick 7d587ec03c3d6e592a1fd544d93094fdeb229c45 #RATIS-2017: checkAndEvictCache must be done in writeLock. (#1031) git cherry-pick 5560718aba3ec9cb1447f7305ba4e23b2567effa #RATIS-2011. When a log entry is truncated, remove TransactionContext. (#1029) git cherry-pick a18d08c89cd92e52a6c5c075fed3df59012ce4b6 #RATIS-2015. Node.js 16 actions are deprecated (#1028) #RATIS-2007. Zero-copy buffers are not released (#1027) git cherry-pick 79923c2d79c70dcf642c2c4bd24cfbf2501d0090 #RATIS-2008. Follower should recognize candidate if the candidate is the same peer as the current valid leader (#1024) git cherry-pick ea1341b05dbaedb7b9775fa870522bf1de052a0e #RATIS-2012. Client should not retry after close. (#1025) git cherry-pick 0e247578bf46cd0ffdc9863da3ff77ba21753a37 #RATIS-1972. Add junit 5 dependencies in ratis-server. (#1003) git cherry-pick c1e0ec284aa88078db63a8b3cc1a90231144eff3 #RATIS-2013. OrderedAsync retry results an IllegalStateException in GrpcClientProtocolService. (#1026) git cherry-pick 561cd6d48ab95ff72bbfd4a89c9074581d2ad1bf #RATIS-2010. Remove dependency on spotbugs-annotations (#1022) #RATIS-1934. Support Zero-Copy in GrpcServerProtocolService (#1014) #RATIS-2009. ReferenceCount should work for all LogEntry types. (#1021) git cherry-pick fc5c7256096c6f5b85308b47a1b2e5242ded4d03 #RATIS-2004. Fix EQ_COMPARETO_USE_OBJECT_EQUALS in LogSegment (#1020) git cherry-pick 82f7b3b851211e35056ebf592a90829649568cdb #RATIS-2003. Fix IT_NO_SUCH_ELEMENT in InstallSnapshotRequests (#1016) #RATIS-1990. Refactor appendEntries processing to support reference count (#1011) git cherry-pick 14eb0bf057c89954a05e7d5c9915f71074591d47 #RATIS-2002. Fix findbugs warnings in LogSegmentPath (#1017) git cherry-pick 859a41074c9d896563c95d1451a44267276d72be #RATIS-2006. Fix DM_EXIT in filestore Client (#1019) git cherry-pick eae0d18eb3a33da5ddf4a20a79cd24c95c8c9e7a #RATIS-2005. Fix URF_UNREAD_FIELD in MetricServerCallListener (#1018) git cherry-pick 24482561f98e9f9b0555dffbddd92111af303724 #RATIS-2001. TransactionContext can be wrongly reused. (#1015) git cherry-pick 6671ef6839e55469333f2f88935fe2d924e6ae5b #RATIS-1999. Move RaftServerJmxAdapter out from RaftServerImpl. (#1010) git cherry-pick 863c2e8a069337f1dd1ae3f2520d118a443fa612 #RATIS-1987. Intermittent NPE in TestGrpcZeroCopy. (#1013) git cherry-pick 206205810db1cab29987b2331dac205aafd62497 #RATIS-1989. Intermittent timeout in TestStreamObserverWithTimeout (#1012) git cherry-pick 59dfd35491df2346de80a685365b14da52f6bc1f #RATIS-1998. Add watch request metrics (#1009) #RATIS-1925. Support Zero-Copy in GrpcClientProtocolService (#1007) ########################################## # last matching commit 5e8ef17e7a17042c99875d398c9e9b9cee8a3ac4 #RATIS-1994. AsyncApi send() to support optional replication level. (#1006)
echo This script $0 is NOT for running directly. exit 1 ########################################## git cherry-pick a84a924f4d698b40961b6e943b6b4f2b1258090c #Change version for the version 3.1.3 -rc2 git cherry-pick 24882c34321effa498b1c7936953ecda63e96cf3 #fix cherry-pick generated warnings git cherry-pick dd8486aafafc80846ce68a92dca16b52fa162659 #RATIS-2236 Fixed bug where manual triggerSnapshot would never finish (#1207) git cherry-pick a6da70640d8c9ae6364d7074a240cf18d042e666 #RATIS-2215. Bump maven-remote-resources-plugin to 3.3.0 (#1208) git cherry-pick 17ca6f41d0a577de2ecb452368c1a38b0c63d8b7 #RATIS-2235. Allow only one thread to perform appendLog (#1206) git cherry-pick cf893f64906df82908fcc43aed2d575e52f7a174 #RATIS-2233. make NOPROGRESS timeout configurable (#1204) git cherry-pick 3aa6e177418cb99e83760bf81df9a1fdd365431c #RATIS-2229. Do not print the same conf values multiple times. (#1200) git cherry-pick 337df17c7ea27fbaac9f5f82f8557dc815830d7c #RATIS-2234. Remove lock race between heartbeat and append log channels (#1205) git cherry-pick 2c4e354f133a44b971837ea33b5f89d62302cb63 #RATIS-2232. Improve log for debugging on RaftLog / TransactionManager (#1203) git cherry-pick 6cc5be003bf852fd25c7ddfc6d8db9b9555acca2 #RATIS-2231. Bump ratis-thirdparty to 1.0.8 (#1202) git cherry-pick a96927db927fa57ce918c63d97c680c6bdf38b8c #RATIS-2200. Make Ratis example log at INFO level (#1189) git cherry-pick 37617f8f3bd70c6c446f6caaa7a9755e067a4db4 #RATIS-2197. Clean remote stream to resolve direct memory leak (#1179) git cherry-pick 29b4960be986f8061a9d058fd571027dc16a83c8 #RATIS-2226. Enable Develocity local build cache (#1196) git cherry-pick 6faeb7522748880f3b2eabbb3ce031dd78141dbf #RATIS-2210. Reduce duplication in CI workflow (#1195) git cherry-pick 20ebed514880e45e48fe312a5c0e3be4c0ab7a6f #RATIS-2219. Remove duplicate test case from TestRaftWithGrpc (#1192) git cherry-pick 2dbeaf593eb8c0837688a81efebfcb15c17a90a2 #RATIS-2222. Remove copy-rename-maven-plugin usage (#1194) git cherry-pick 4f82123a38531fc22ff58c72f91c49275a9f506e #RATIS-212. Fix typo in RaftAsyncTests (#1197) git cherry-pick 3b0f0ab2eeff868b994294ede8790e56e6dac06f #RATIS-2216. Bump maven-shade-plugin to 3.6.0 (#1191) git cherry-pick 7dcc4d0d058b32a50575dfb7351bc37f8707e5df #RATIS-2211. publish-mvn fails with: ... did not assign a file to the build artifact (#1190) git cherry-pick f73a1ea37a46d00a8835d9c1ade8940848779b79 #RATIS-2206. Jars in Maven repo and binary tarball are not the same (#1188) git cherry-pick 9b186389654e21d417b86970e3116f3926fa1332 #RATIS-2204. Avoid downloads from repository.apache.org (#1187) git cherry-pick e49e7f80de17db0e6f1457f836e4d5a310fd2e12 #RATIS-2185. Improve gRPC log messages debugability. (#1186) git cherry-pick eb3d5040976af097c709099878c67c70f82746a5 #RATIS-2201. Bump ratis-thirdparty from 1.0.6 to 1.0.7 (#1185) git cherry-pick 268bd3c49e046ccae88267d07834b160a96db315 #RATIS-2194. FileLock didn't unlock properly (#1183) git cherry-pick fa5a06159f132d6440735ea1812eb3ee2121e02a #RATIS-2191. Add ResourceLeakDetector to Netty tests (#1180) git cherry-pick b80329fce2a76f2c3caa6a5cff7295efa07e9b3c #RATIS-2189. Use ByteBufAllocator#ioBuffer in NettyDataStreamUtils (#1178) git cherry-pick 1571cc73e36cec1a918321ec766fb745761a06d3 #RATIS-2186. Raft log should not purge index lower than the log start index (#1175) git cherry-pick 4a9ec4e4f75d9d964dd3a6c9d995209cd092aef7 #RATIS-2177. Purge should delete segmentLog from small to large according to logIndex (#1174) git cherry-pick ebc9c3e6599af8805bd456e14abd024b0a6ddb86 #Change version for the version 3.1.2 -rc1 git cherry-pick 7c3942d1d09bdfa6a062b06e4cef6cf01c4a2e1f #RATIS-2183. Detect staled snapshot request. (#1173) git cherry-pick 2981c6b158458a90ebad066415e43d07769afe7e #RATIS-2178. Add a log message whenever deleting a RaftLog segment. (#1170) git cherry-pick 263ea69b2ff295355dc7f2074bf7cf69a820e8b2 #RATIS-2179. Fix NPE in RaftServerImpl.getLogInfo. (#1171) git cherry-pick 7f10888a4150fac9426f3862c2b25e6301039465 #RATIS-2174. Move future.join outside the lock (#1168) git cherry-pick fbb9b5a663cc5e7a2ae2c8fab04d507f77189df7 #RATIS-2172. RaftServer may lose FollowerState (#1166) git cherry-pick 536419c50365e1765fe303e3452a967aae4cd1dd #RATIS-2166. Update nextChunkIndex when snapshot is ALREADY_INSTALLED (#1159) git cherry-pick 5d1a5ea93c6b79963f80e2b520cdbfde62e1efcc #RATIS-2165. Do not include RaftClientRequest Message in ReadIndexRequest (#1160) git cherry-pick d8482f1f0a72420e50aeeb2401a669d43bbfbf6d #RATIS-2162. When closing leaderState, if the logAppender thread sends a snapshot, a deadlock may occur. (#1154) git cherry-pick 3021c02440907d62f39fa752ab0edd06482bb785 #RATIS-2160. MetricRegistriesLoader should not print INFO message in CLI. (#1152) git cherry-pick 45a30d890451a44ec918fdee2732c5fff80ea17c #Change version for the version 3.1.1 -rc3 git cherry-pick d7709da87cf00bcada0797a5f2445ef442f6c149 #RATIS-2158. Let the snapshot sender and receiver use a new digester each time (#1151) git cherry-pick fd260d8fdc630db85f8bb308ea8275029ca45427 #RATIS-2155. Add a builder for RatisShell. (#1150) git cherry-pick 313c51a13dc665df624dd93bae4f0c65cfb24c4f #RATIS-2157 Enhance make_rc.sh for non-first rc at release time #1149 git cherry-pick 5a470a2b23275a10b532761554ee9a9f72a564be #RATIS-2147. Md5 mismatch when snapshot install (#1142) git cherry-pick 356ad8a27ffead990a4fc6108ffc6df8f3d2c58b #RATIS-2150. No need for manual assembly:single execution when mvn deploy #1144 git cherry-pick f23f9b7c10b3d96efc53a478e811d5c94f9d8a20 #RATIS-2152. GrpcLogAppender stucks while sending an installSnapshot notification request (#1146) git cherry-pick 5578be7fb07e52cfa90e2979fd251ae78badfc62 #RATIS-2154. The old leader may send appendEntries after term changed. (#1148) git cherry-pick 54a991623ec34d8386cda18ec2c2951ccb08a70b #RATIS-2148. Snapshot transfer may cause followers to trigger reloadStateMachine incorrectly (#1145) git cherry-pick 1e10b7186fa7f4bea695cab01c82579af8fee66b #RATIS-2149.Do not perform leader election if the current RaftServer has not started yet (#1143) git cherry-pick 342688de97ad8b125366bd324aaa3c6a7a72bc5f #RATIS-2153. ratis-version.properties missing from src bundle (#1147) git cherry-pick fc61d9650d31932c61efed334854134667d1c69b #Change version for the version 3.1.1 -rc2 git cherry-pick 211278e5677799deb6a6e342a7b835783d78ff01 #RATIS-2146. Fixed possible issues caused by concurrent deletion and election when member changes (#1140) git cherry-pick 5b8a74fc2a0e4f6c1428a565e41324322d358531 #RATIS-2145. Follower hangs until the next trigger to take a snapshot. (#1139) git cherry-pick d621c14efccf4ccd851b99a659e15251df8796e3 #Revert "RATIS-2099. Cache TermIndexImpl instead of using anonymous class (#1100)" git cherry-pick 6aa7349e918a445521a032064bd0c4542dac4498 #Revert "RATIS-2101. Move TermIndex.PRIVATE_CACHE to Util.CACHE (#1103)" git cherry-pick cdfd4f10d7febbd320c1104f2b173c5ac9b4264b #Revert "RATIS-1983. Refactor client request processing to support reference count. (#998)" git cherry-pick fc89b040535c1f457fe5d57e6fdcb6cab9a8e4ac #RATIS-2113. Use consistent method names and parameter types in RaftUtils. (#1135) git cherry-pick 2e7cb458ca6a10b4c38cafca7e8eee8a8e7fcef1 #RATIS-2140. Thread wait when installing snapshot. (#1137) git cherry-pick 557675f12b08c046d9256102478e02ac762feba0 #RATIS-2144. SegmentedRaftLogWorker should close the stream before releasing the buffer. (#1138) git cherry-pick 099d23f258153917d2ba63f0d72f38504eb5fd10 #RATIS-2137. Fix LogAppenderDefault in handling of INCONSISTENCY. (#1136) git cherry-pick 94692f3cb4e1a8e89b6e1a186835c5a12a9f5588 #RATIS-2139. Fix checkstyle:ParameterNumber in GrpcService. (#1134) git cherry-pick d3fa63b5e7014993769b176500f83724f8833acd #RATIS-2138 Remove uncessary error log in JVMPauseMonitor When GC is serious (#1133) git cherry-pick a78e6e2df822eea2bb534fdca3c55c1e77465232 #RATIS-2135. The leader keeps sending inconsistent entries repeatedly to followers. (#1132) git cherry-pick 58327e26ce691058e1efb6bb15e2c374aaefc957 #RATIS-2115. Fix flaky RetryCacheTests and InstallSnapshotFromLeaderTests. (#1129) git cherry-pick 5946796d5753555f6cfe292fad7a5568a367160c #RATIS-2136 Fixed the issue that RatisServer could not start after frequent restarts (#1131) git cherry-pick 49f1247a0d5b55e4aa3105eb4816e1ddfc10342a #RATIS-2134. `logMetadata` could miss appending the `metadataEntry` with the lastCommitIndex (#1130) git cherry-pick bd154742c099ff276958e5f873f108a411b436af #RATIS-2104. Double shutdown in TestLeaderInstallSnapshot. (#1108) git cherry-pick 1ff10882bb25142ec8865370c68b8fa617ac0141 #RATIS-2133. Ignore http proxy for ratis grpc client & server (#1128) git cherry-pick 443436455226a70d43a25b61292023e408a1c261 #RATIS-2126. Optimize NotLeaderException message (#1124) git cherry-pick 6a6caa2709d239a205bc400f9af9e3bbc18a6a68 #RATIS-2131. Surround with [] only if hostName is a IPv6 string (#1125) git cherry-pick 53b80c6dc767895805a2376f3a651506a92d0b10 #RATIS-1071. NettyClientRpc supports sendRequestAsync. Contributed by Rui Wang and Tsz-Wo Nicholas Sze. (#1122) git cherry-pick 054cfb707de0976e4dae55005d0b1934242dc983 #RATIS-2125. javadoc jars should not include non-doc items (#1123) git cherry-pick cfb95777fc908f266c880c0fe2a2ad1d2cae750e #RATIS-1840. Avoid including build timestamp in artifacts (#1114) git cherry-pick 6390a28bdf1d2c454d49a11dca117e5bbc482f54 #RATIS-2116. Fix the issue where RaftServerImpl.appendEntries may be blocked indefinitely (#1116) git cherry-pick 76fb1a316af4d717a35b8fe7feeb2eefbb1bfc71 #RATIS-2121. Set javac --release when compiling with JDK 9+ (#1119) git cherry-pick 8383845ddb77c64888bf8987fa30eb29a89461c1 #RATIS-2118. Improve assembly process (#1117) git cherry-pick 54e6fee7c1de75edad9258f9bb40e33c3e2d8976 #RATIS-2117. No need for manual assembly:single execution (#1115) git cherry-pick f2ab7d51dcb31d745a4a4edd3ccfbc378931dd51 #RATIS-2110. Publish SBOM artifacts (#1110) git cherry-pick ce6838a5d595e8f1488bef41951b63daf6e1ffd6 #RATIS-2095. Extract common logic of ratis-shell to RaftUtils for reuse (#1098) git cherry-pick 7159c5943aeb56652ce4e69d5a0ceaa2d4389106 #./mvnw versions:set -DnewVersion=3.1.1-SNAPSHOT git cherry-pick de2e777e19d48bd67fcba1538007940abde87170 #RATIS-2111 Reinitialize should load the latest snapshot (#1111) git cherry-pick e7e0c7931fa6774d9adcd59ec018ec7cf5428d9c #RATIS-2112. Improve repeat-test workflow (#1112) git cherry-pick e66b5afc355310aae639116b9cabcac791a5c5de #RATIS-2109. RaftLogBase#updateCommitIndex should return true only if commitIndex is increased (#1109) git cherry-pick e3e7430ed6410ca20fd4b5cac05d35d039755bb4 #RATIS-2106. Add configuration reference for RaftClient (#1107) git cherry-pick 10c362b761dbac0930546424ee2bf77c11b524a5 #RATIS-2084. Follower reply ALREADY_INSTALLED when install old snapshots from leader (#1091) git cherry-pick 0b44fe2dac9ab66ebf3797b797b7bce6bbbb4fd6 #RATIS-2102. AsyncApi#send() is not handling retry and reply correctly for replication levels higher than MAJORITY (#1104) git cherry-pick d425686df6b71b13a02d9ab7cf48476ae14a35d1 #RATIS-2089. Add CommitInfoProto in NotReplicatedException (#1105) git cherry-pick 42cfa9328fd93ff4d6fa5be949e9ecf7835485bb #RATIS-2100. The `closeFuture` never completed while closing from the `NEW` state (#1102) git cherry-pick b067e3d23832a0adf28ecac2f6bb7b7d28838482 #RATIS-2101. Move TermIndex.PRIVATE_CACHE to Util.CACHE (#1103) git cherry-pick 428ce4ae3d5a0349f3425cb85ef1a3d38dea24b1 #RATIS-2099. Cache TermIndexImpl instead of using anonymous class (#1100) git cherry-pick 9533415f08224e83f4595c5abf2846f40c6473d0 #RATIS-2097 Add a conf to enable/disable Metadata entries. (#1101) git cherry-pick 41286c7782d8603983675f0eed1aca46fa2ed683 #RATIS-2098. RaftLogTruncateTests may fail. (#1097) git cherry-pick c57112d6b443d2d8cae09bdfdac235ea3eb3c188 #RATIS-2081. Comment that we skip checkstyle on test directory on purpose (#1084) git cherry-pick 7e02718f8227d580f094b1f1e3555ee8f91b2806 #RATIS-2091. Fix other checkstyle issues in test-directory (#1094) git cherry-pick ec9ff902b714d909ee7d6b2ccad1e2c05836a226 #RATIS-2066. RaftServerProxy supports close callback (#1072) git cherry-pick ae9f34490a7184c1107cbc563e10f3181c4d27c7 #RATIS-2090. Bump ratis-thirdparty to 1.0.6 (#1093) git cherry-pick e9a9b1681decea15133b5c21dc86bae7d7f309da #RATIS-2088. Fix name and imports related check in test-directory (#1092) git cherry-pick 7b85a65e29b5b363f1d3edc0156cef866542a907 #RATIS-2083. Timer thread leak when log level is DEBUG (#1086) git cherry-pick 052a9ad87c31f825b5cf194621a8471847e6dc72 #RATIS-2085. Compile Ratis with JDK 17/21 in CI (#1087) git cherry-pick 2e0d697e1f6f05c33dcc91e0fe39e6a800d45426 #RATIS-2076. Intermittent timeout in TestRaftWithNetty#testWithLoad. (#1090) git cherry-pick 265c4fa921527d85e0ed1b730792d2e3571aa534 #RATIS-2086. Autolink Ozone issues in PRs (#1088) git cherry-pick e1842618418990c2a69b9b509e3cf4929f492307 #RATIS-2087. Remove workaround for GitHub reverse DNS issue (#1089) git cherry-pick 41c59be51e77f8efe9822234a9946f711c9886a3 #RATIS-2078. Correct timeout unit for tests (#1083) git cherry-pick fbe30bdd7df77a3f3e35992e9d7cf278bbfbadfb #RATIS-2074. Intermittent fork timeout in TestRaftWithNetty#testBasicAppendEntriesKillLeader. (#1080) git cherry-pick a1bf9ec4d612583c7fb149639f8a12c4720669c9 #RATIS-2067. Enhance content for concept in ratis-docs. (#1073) git cherry-pick bd1a0e932ba9596ca766184e8d878b86d788db9f #RATIS-2073. Enable Surefire process checkers (#1079) git cherry-pick 7df98fbe575678fd2a7ab02a99cfb94a8f8c1575 #RATIS-2072. Limit timeout of CI workflow jobs (#1078) git cherry-pick 158a19673cee57df041528f1c0feabd1eb97c277 #RATIS-2071. Bump Bouncy Castle to 1.78.1 (#1076) git cherry-pick 0cb1fdc35c95417505816ed49cdc6ad1cac91c09 #RATIS-2070. Skip push build for dependabot (#1077) git cherry-pick 070361ca1d3372f35bf49a1fdca7cd89e1e252be #RATIS-2068. Avoid logging raw StateMachine data body. (#1074) git cherry-pick a20405d015c04cb5151758574eebce29d4c1b664 #RATIS-2069. RaftMetaConf command sets incorrect peerAddress pattern. (#1075) git cherry-pick 7e9e2489165459f0e99391bd5fba7e509a679f62 #RATIS-2065. Avoid the out-of-heap memory OOM phenomenon of frequent creation and deletion of Raft group scenarios. (#1071) git cherry-pick 222a59351992df54b1e2b7a0a53f934b07a89bce #RATIS-2061. Fix setCloseThreshold parameter in RaftServerConfigKeys (#1070) git cherry-pick 6ed3b71ddd4be494725ae42f59ec5a33a8aaae60 #RATIS-2060 Avoid unexpected shutdown of RaftServerProxy during membership change (#1069) git cherry-pick 14f3a617680b20144edb7574523eb27e934c4b27 #RATIS-2055. Move notifyTermIndexUpdated after leader.checkReady (#1068) git cherry-pick 84dbe1fc40e78aabc0b63ba30bbe1f77a25b33ba #RATIS-2057. Add back JavaUtils.attemptUntilTrue(..). (#1066) git cherry-pick b9d0075c01015bf11b5628111690255fc2b3fb37 #RATIS-2056. Update purge index when snapshot installed. (#1065) git cherry-pick 67a0c6b827c9407aba8c38893283e0dd7922962f #RATIS-1504. Add timeout handling to DataStreamManagement#checkSuccessRemoteWrite. (#1064) git cherry-pick 018d2021e440a84743094276d2a7d5d0b3aff3de #Ratis-2040. Fix RaftPeerId generated by command of "raftMetaConf" to use real PeerId (#1060) git cherry-pick caa35dd0cfcfe9ff648312a8e9684084d2b67900 #RATIS-1519. When DataStreamManagement#read an exception occurs, remove DataStream (#596) git cherry-pick 20ed3c72859b02ae2de4be8a5404528c8efaf567 #RATIS-2051. Fix Sonar bugs for Ratis. (#1059) git cherry-pick 68b9969e843ea2068425a7c342cd5231add657a5 #RATIS-2052. Add docs for triggering snapshot when removing raftGroup parameter. (#1061) git cherry-pick 8e130f828b32178c1780a259443701cb17114ce4 #RATIS-2050. Add creationGap param to snapshot management API (#1058) git cherry-pick fe4d8d14ec7af90a67e7508e54a11875a8ba9ab0 #RATIS-2049. Suppress Sonar warning for volatile objects (#1057) git cherry-pick 2630b6a5a09d8b9ed7e641e9d8f01082051dbedd #RATIS-2036. Avoid trigger snapshot when removing raftGroup (#1055) git cherry-pick 24cf8fa5028cbbee3638feedc8b365c63beeeb1a #RATIS-2048. Fix SimpleStateMachineStorage potential NPE exception (#1056) git cherry-pick 93c3453e6f52dbc299d41a8e283660afc01f1f88 #RATIS-2047. Avoid unnecessary warn log when creating raft group (#1054) git cherry-pick 7760eec695d79d4363f980e31f76c8842a140568 #RATIS-2045. SnapshotInstallationHandler doesn't notify follower when snapshotIndex is -1 and firstAvailableLogIndex is 0 (#1053) git cherry-pick e1acb4bdd3014d1788232734199df177e1441662 #RATIS-2044. Fix ReadIndex loss caused by data race in AppendEntriesListeners (#1052) git cherry-pick d15bd1e4f37db5b3495ba2f22d3a38289ef8aa5e #RATIS-2043. Change toString() of RaftConfigurationImpl (#1051) git cherry-pick 57616b0fd7664872b74ffddb7cd66f3ef2131941 #Ratis-2031. Add peer info to response of GroupInfoCommand CLI (#1047) git cherry-pick f3b264c6b6ef49c4107176b7da4211a60ad2e3f2 #RATIS-2035. Refactor streaming code for Read. (#1046) git cherry-pick ea624d8ecd8c479237c537cbb7c947f2d2106d4f #RATIS-2038. IllegalStateException when ServerState is closed before start. (#1048) git cherry-pick 8bfbf2c46c92afffdb701581d7c961527de21929 #RATIS-2027. Ratis Streaming: Remote Stream copy data to heap. (#1044) git cherry-pick 30baa5a650d3d0b3b0dc33f60a0e995eee4e59df #RATIS-2021. Migrate ratis-test tests to Junit 5 - Part 2. (#1043) git cherry-pick 5e306ed742411f7ee3e4d4329a1c8a060112260f #RATIS-2025. Move out assert and proto methods from RaftServerImpl. (#1041) git cherry-pick d83e24493a89f435d37a6e9e24ccd58b26730112 #RATIS-2024. Refactor appendEntries code. (#1040) git cherry-pick f728173ecccbba0a8c85481b01d6099cc11fe8c6 #RATIS-2023. Remove duplicate RefCountingMap (#1039) git cherry-pick dfa30fe92b4b7ea4939c141da60d4ff6e9d3e753 #RATIS-1976. Migrate ratis-test tests to Junit 5 - Part 1 (#1038) git cherry-pick 016018f8d783b6e1eaa3e08a5cdb5606a92ef150 #RATIS-1975. Migrate ratis-metrics-dropwizard3 tests to Junit 5. (#1037) git cherry-pick 2b30cb832b296115e4b1eb7408884529e6417031 #RATIS-2019 Fixed abnormal exit of StateMachineUpdater (#1033) git cherry-pick 2074376cf7d27f8e858fe177bdee53d438185b5b #RATIS-1973. Migrate ratis-examples tests to Junit 5. (#1035) git cherry-pick 355d3f54e3a4c616d82faea7ecbd8de14a23af15 #RATIS-1974. Migrate ratis-metrics-default tests to Junit 5 (#1034) git cherry-pick 42bd89c9e00ee440198f03db7ed82c44acec39b3 #RATIS-2016. Correct NotificationInstallSnapshot's index (#1030) git cherry-pick 0a855e2a19863aae0e3dd6f64cd39eb0174623e5 #RATIS-2017: checkAndEvictCache must be done in writeLock. (#1031) git cherry-pick e7c6453afdaed3b8bc19ca358538c3ce87362b83 #RATIS-2011. When a log entry is truncated, remove TransactionContext. (#1029) git cherry-pick 3b74fe59de552259952118c8bebbeb73fdb84a7f #RATIS-2015. Node.js 16 actions are deprecated (#1028) git cherry-pick 6ba781ea70bd9860988bbb8ec918d47ad82d616d #RATIS-2008. Follower should recognize candidate if the candidate is the same peer as the current valid leader (#1024) git cherry-pick 50c2673db7aefd09886f8547efa8ec00f9e8056a #RATIS-2012. Client should not retry after close. (#1025) git cherry-pick 6fb56165b1abefc16c116dcae8c2a60ba919e690 #RATIS-1972. Add junit 5 dependencies in ratis-server. (#1003) git cherry-pick fb02f5422700333ecc9406737c0e8b01210e2623 #RATIS-2013. OrderedAsync retry results an IllegalStateException in GrpcClientProtocolService. (#1026) git cherry-pick ee98f91401ccda9f735287456a7765c22944bb85 #RATIS-2010. Remove dependency on spotbugs-annotations (#1022) git cherry-pick 03accc7b34890cde94c66291fcacad6456815863 #RATIS-2004. Fix EQ_COMPARETO_USE_OBJECT_EQUALS in LogSegment (#1020) git cherry-pick 45620814d2cec1159aa0bbb3dcd8591253b68cc3 #RATIS-2003. Fix IT_NO_SUCH_ELEMENT in InstallSnapshotRequests (#1016) git cherry-pick fc09dc8dc0853a6aea8834000733e744098ee46c #RATIS-2002. Fix findbugs warnings in LogSegmentPath (#1017) git cherry-pick eafc8e4f72914307f76d52dad040441998280709 #RATIS-2006. Fix DM_EXIT in filestore Client (#1019) git cherry-pick a11b0c1ab173bba812d40da08ae73be9a2769deb #RATIS-2005. Fix URF_UNREAD_FIELD in MetricServerCallListener (#1018) git cherry-pick 1ec1bd98c7baa14164f273f388df9dca676bbf02 #RATIS-2001. TransactionContext can be wrongly reused. (#1015) git cherry-pick e60c9c7c511bf578185e76d3a29c9c8a8620c639 #RATIS-1999. Move RaftServerJmxAdapter out from RaftServerImpl. (#1010) git cherry-pick 0bfd8d35bde6ddea76ffd8027ee3aa460208f8a4 #RATIS-1987. Intermittent NPE in TestGrpcZeroCopy. (#1013) git cherry-pick 6d9d0b0733ad9042a11b2ced8cf706384491da0a #RATIS-1989. Intermittent timeout in TestStreamObserverWithTimeout (#1012) git cherry-pick b4350aaef7e859239a8124ae08dda9b1f73bfa14 #RATIS-1998. Add watch request metrics (#1009) ########################################## # last matching commit 5e8ef17e7a17042c99875d398c9e9b9cee8a3ac4 #RATIS-1994. AsyncApi send() to support optional replication level. (#1006)
