Anonymitaet commented on a change in pull request #10874: URL: https://github.com/apache/pulsar/pull/10874#discussion_r648767686
########## File path: site2/website/release-notes.md ########## @@ -1,6 +1,419 @@ ## Apache Pulsar Release Notes +### 2.8.0 — 2021-06-12 <a id=“2.8.0”></a> + +#### PIPs +- [PIP 45] Pluggable metadata interface [#9148](https://github.com/apache/pulsar/pull/9148) [#9221](https://github.com/apache/pulsar/pull/9221) [#9240](https://github.com/apache/pulsar/pull/9240) [#9273](https://github.com/apache/pulsar/pull/9273) [#9274](https://github.com/apache/pulsar/pull/9274) [#9338](https://github.com/apache/pulsar/pull/9338) [#9346](https://github.com/apache/pulsar/pull/9346) [#9351](https://github.com/apache/pulsar/pull/9351) [#9412](https://github.com/apache/pulsar/pull/9412) [#9485](https://github.com/apache/pulsar/pull/9485) [#9586](https://github.com/apache/pulsar/pull/9586) [#9967](https://github.com/apache/pulsar/pull/9967) [#9973](https://github.com/apache/pulsar/pull/9973) [#10391](https://github.com/apache/pulsar/pull/10391) [#10457](https://github.com/apache/pulsar/pull/10457) [#10532](https://github.com/apache/pulsar/pull/10532) [#10545](https://github.com/apache/pulsar/pull/10545) [#10647](https://github.com/apache/pulsar/pull/10647) +- [PIP 50] Package management service [#8637](https://github.com/apache/pulsar/pull/8637) [#8680](https://github.com/apache/pulsar/pull/8680) [#8744](https://github.com/apache/pulsar/pull/8744) [#8764](https://github.com/apache/pulsar/pull/8764) [#8816](https://github.com/apache/pulsar/pull/8816) [#8817](https://github.com/apache/pulsar/pull/8817) [#8858](https://github.com/apache/pulsar/pull/8858) [#8861](https://github.com/apache/pulsar/pull/8861) [#8893](https://github.com/apache/pulsar/pull/8893) [#8907](https://github.com/apache/pulsar/pull/8907) +- [PIP 68] Exclusive producer [#8685](https://github.com/apache/pulsar/pull/8685) [#8992](https://github.com/apache/pulsar/pull/8992) [#9554](https://github.com/apache/pulsar/pull/9554) [#9600](https://github.com/apache/pulsar/pull/9600) +- [PIP 70] Lightweight broker entry metadata [#8618](https://github.com/apache/pulsar/pull/8618) [#9067](https://github.com/apache/pulsar/pull/9067) [#9088](https://github.com/apache/pulsar/pull/9088) [#9091](https://github.com/apache/pulsar/pull/9091) +- [PIP 71] Pulsar SQL migrate SchemaHandle to Presto decoder [#8422](https://github.com/apache/pulsar/pull/8422) +- [PIP 74] Client memory limits [#8965](https://github.com/apache/pulsar/pull/8965) [#9676](https://github.com/apache/pulsar/pull/9676) +- [PIP 75] Perform serialization/deserialization with LightProto [#9046](https://github.com/apache/pulsar/pull/9046) +- [PIP 76] Streaming offloader [#9096](https://github.com/apache/pulsar/pull/9096) +- [PIP 82] Tenant and namespace level rate limiting [#10008](https://github.com/apache/pulsar/pull/10008) [#10201](https://github.com/apache/pulsar/pull/10201) [#10204](https://github.com/apache/pulsar/pull/10204) [#10218](https://github.com/apache/pulsar/pull/10218) +- [PIP 83] Message consumption with pooled buffer [#10184](https://github.com/apache/pulsar/pull/10184) +- [PIP 85] Support get reader schema for a message in Java Client [#10476](https://github.com/apache/pulsar/pull/10476) + +#### Transactions +- Support pending ack state persistence [#8881](https://github.com/apache/pulsar/pull/8881) +- Stable position and low watermark for the transaction buffer [#9195](https://github.com/apache/pulsar/pull/9195) +- Transaction timeout support [#9229](https://github.com/apache/pulsar/pull/9229) +- Transaction coordinator retry to complete the transaction [#9236](https://github.com/apache/pulsar/pull/9236) +- Fix race condition when appending transaction log [#9238](https://github.com/apache/pulsar/pull/9238) +- Transaction buffer snapshot [#9490](https://github.com/apache/pulsar/pull/9490) +- Add metrics for transaction coordinator [#9706](https://github.com/apache/pulsar/pull/9706) +- Clean the useless transaction individual acknowledgements based on low watermark [#9722](https://github.com/apache/pulsar/pull/9722) +- Fix memory leak when delete transaction marker [#9751](https://github.com/apache/pulsar/pull/9751) +- Check the transaction state at the client side [#9776](https://github.com/apache/pulsar/pull/9776) +- Clean aborted transactions for the transaction buffer [#9974](https://github.com/apache/pulsar/pull/9974) +- Fix transaction coordinator retry to end transaction [#10131](https://github.com/apache/pulsar/pull/10131) +- Fix NPE when open a new transaction [#10139](https://github.com/apache/pulsar/pull/10139) +- Fix transaction log failed to recover [#10146](https://github.com/apache/pulsar/pull/10146) +- Fix transaction coordinator recover timeout [#10162](https://github.com/apache/pulsar/pull/10162) +- Handling committing and aborting state when recovering transaction coordinator [#10179](https://github.com/apache/pulsar/pull/10179) +- Fix NPE in case of enableTransactionCoordinator=false [#10182](https://github.com/apache/pulsar/pull/10182) +- Fix transaction buffer client timeout [#10206](https://github.com/apache/pulsar/pull/10206) +- Fix recover max local id issue for the transaction coordinator [#10224](https://github.com/apache/pulsar/pull/10224) +- Support using transactions on standalone [#10238](https://github.com/apache/pulsar/pull/10238) +- Fix transaction buffer lookup issue [#10257](https://github.com/apache/pulsar/pull/10257) +- Fix transaction timeout issue at the client side [#10323](https://github.com/apache/pulsar/pull/10323) +- Fix transaction client reconnect issue after transaction coordinator unloaded [#10327](https://github.com/apache/pulsar/pull/10327) +- Fix transaction timeout not canceled after expired [#10366](https://github.com/apache/pulsar/pull/10366) +- Transaction log low watermark optimization [#10422](https://github.com/apache/pulsar/pull/10422) +- Fix transaction buffer client channel not active problem [#10407](https://github.com/apache/pulsar/pull/10407) +- Admin API for getting transaction coordinator stats [#10639](https://github.com/apache/pulsar/pull/10639) +- Admin API for getting transaction in transaction buffer stats [#10642](https://github.com/apache/pulsar/pull/10642) +- Admin API for getting transaction in pending ack stats [#10648](https://github.com/apache/pulsar/pull/10648) +- Admin API for getting transaction buffer stats and pending ack stats [#10650](https://github.com/apache/pulsar/pull/10650) +- Admin API for getting transaction coordinator internal stats [#10653](https://github.com/apache/pulsar/pull/10653) +- Setup transaction metadata with metadata store [#10677](https://github.com/apache/pulsar/pull/10677) +- Fix issue with acknowledge messages from multiple subscriptions of a topic [#10689](https://github.com/apache/pulsar/pull/10689) +- Admin API for getting transaction metadata [#10690](https://github.com/apache/pulsar/pull/10690) +- Admin API for getting slow transactions [#10701](https://github.com/apache/pulsar/pull/10701) +- Fix transaction log handle managed ledger WriteFail state [#10711](https://github.com/apache/pulsar/pull/10711) +- Admin API for getting pending ack internal stats [#10725](https://github.com/apache/pulsar/pull/10725) +- Fix transaction ack delete marker position when don't have transaction ack [#10741](https://github.com/apache/pulsar/pull/10741) +- Fix transaction pending ack generate managedLedgerStats fail exception [#10797](https://github.com/apache/pulsar/pull/10797) +- Use zookeeper prometheus metric provider to export zookeeper metrics [#10803](https://github.com/apache/pulsar/pull/10803) + +#### Security +- Optional auth method name header in http authentication [#6799](https://github.com/apache/pulsar/pull/6799) +- Topics level authorization support [#7523](https://github.com/apache/pulsar/pull/7523) +- Authorization for setting topic/subscription auto-creation and subscription expire time on namespace [#7900](https://github.com/apache/pulsar/pull/7900) +- Allow serializable stream-provider field into AuthenticationTls [#10020](https://github.com/apache/pulsar/pull/10020) +- Add configuration for running OWASP Dependency Check for all modules [#10288](https://github.com/apache/pulsar/pull/10288) +- Update default TLS protocols to TLSv1.3,TLSv1.2 for the broker and proxy [#10598](https://github.com/apache/pulsar/pull/10598) + +#### Broker +- Share EventLoopGroup between Broker and BookKeeper client [#2603](https://github.com/apache/pulsar/pull/2603) +- Dispatch batch messages according to consumer permits [7266](https://github.com/apache/pulsar/pull/7266) +- Improve the max pending publish buffer mechanism [7406](https://github.com/apache/pulsar/pull/7406) +- Allow disabling HTTP TRACE/TRACK verbs [#7907](https://github.com/apache/pulsar/pull/7907) +- Fix DispatchRateLimiter does not take effect [#8611](https://github.com/apache/pulsar/pull/8611) +- Validate namespace isolation policy regex before updating [#8804](https://github.com/apache/pulsar/pull/8804) +- Perform automatically cert refresh for pulsar-admin [#8831](https://github.com/apache/pulsar/pull/8831) +- Fix Updating rack info dynamically [#8844](https://github.com/apache/pulsar/pull/8844) +- Fix deadlock when checking message expiration [#8877](https://github.com/apache/pulsar/pull/8877) +- Support limit max topics per namespace [#8942](https://github.com/apache/pulsar/pull/8942) +- Make ledger rollover check task internally [#8946](https://github.com/apache/pulsar/pull/8946) +- Clean up topic that failed to unload from the cache [#8968](https://github.com/apache/pulsar/pull/8968) +- Support get broker response for the message acknowledgement [#8996](https://github.com/apache/pulsar/pull/8996) +- Support message index for a topic [#9039](https://github.com/apache/pulsar/pull/9039) +- Supporting limit max topics per namespace by namespace policy [#9042](https://github.com/apache/pulsar/pull/9042) +- Streaming dipsatcher support [#9056](https://github.com/apache/pulsar/pull/9056) +- Make Netty acceptor threadPool size configurable [#9061](https://github.com/apache/pulsar/pull/9061) +- Fix deadlock when unloading namespace bundles [#9116](https://github.com/apache/pulsar/pull/9116) +- Fixed checking for maxTopicsPerNamespace [#9121](https://github.com/apache/pulsar/pull/9121) +- Change the service URL to a not required param when create a cluster [#9127](https://github.com/apache/pulsar/pull/9127) +- Support setting replicator dispatch rate policy at the topic level [#9175](https://github.com/apache/pulsar/pull/9175) +- Fix max topic for namespace does not work [#9193](https://github.com/apache/pulsar/pull/9193) +- Fix memory leak of the managed ledger interceptor [#9194](https://github.com/apache/pulsar/pull/9194) +- Fix max consumers per topic cannot be disabled at the namespace level [#9214](https://github.com/apache/pulsar/pull/9214) +- Support schema rest api for V1 topics [#9218](https://github.com/apache/pulsar/pull/9218) +- Fix peek message metadata while enable broker entry metadata [#9255](https://github.com/apache/pulsar/pull/9255) +- Support fetching metadata from entry data in publish callback [#9257](https://github.com/apache/pulsar/pull/9257) +- Wait for the async broker port listener close operations to complete at shutdown [#9308](https://github.com/apache/pulsar/pull/9308) +- Support script based mapping network topology [#9363](https://github.com/apache/pulsar/pull/9363) +- Make managed ledger storage configurable [#9397](https://github.com/apache/pulsar/pull/9397) +- Support setting enabled subscription types [#9401](https://github.com/apache/pulsar/pull/9401) +- Fixed NPE and cache invalidation in leader election [#9460](https://github.com/apache/pulsar/pull/9460) +- Fix exception when get optional field for Protobuf message [#9468](https://github.com/apache/pulsar/pull/9468) +- Ignore replicated subscription configurations from the client when disabled by broker [#9523](https://github.com/apache/pulsar/pull/9523) +- Support expose producer metrics through prometheus endpoint [#9541](https://github.com/apache/pulsar/pull/9541) +- Fix NPE that occurs in PersistentStickyKeyDispatcherMultipleConsumers when debug log enabled [#9587](https://github.com/apache/pulsar/pull/9587) +- Make LocalPolicies immutable to avoid concurrent modify inconsistent [#9598](https://github.com/apache/pulsar/pull/9598) +- Fix writing/encoding issue of GenericJsonRecord [#9608](https://github.com/apache/pulsar/pull/9608) +- Expose the native record for struct schema [#9614](https://github.com/apache/pulsar/pull/9614) +- Add metrics for producer throttling [#9649](https://github.com/apache/pulsar/pull/9649) +- Fix MaxUnackedMessagesPerConsumer cannot be changed dynamically and cannot be disabled [#9657](https://github.com/apache/pulsar/pull/9657) +- Added support for force deleting tenant [#9677](https://github.com/apache/pulsar/pull/9677) +- Fix managed ledger not found exception when force delete namespace [#9691](https://github.com/apache/pulsar/pull/9691) +- Reduce CPU consumption of metrics creation [#9735](https://github.com/apache/pulsar/pull/9735) +- Ensure read-lock is not continuously held on a section while iterating over concurrent maps [#9787](https://github.com/apache/pulsar/pull/9787) +- Add pending read subscription metrics to stats-internal [#9788](https://github.com/apache/pulsar/pull/9788) +- Allow broker to discover and unblock stuck subscription [#9789](https://github.com/apache/pulsar/pull/9789) +- Support disabled the tenants/namespaces force deletion in the broker.conf [#9819](https://github.com/apache/pulsar/pull/9819) +- Add metrics for the connections of the broker [#9876](https://github.com/apache/pulsar/pull/9876) +- Make readModifyUpdate in MetadataCacheImpl thread-safe [#9900](https://github.com/apache/pulsar/pull/9900) +- Optimize NamespaceBundle.toString() which is on the hot path [#9976](https://github.com/apache/pulsar/pull/9976) +- Support set compaction threshold in broker.conf [#9989](https://github.com/apache/pulsar/pull/9989) +- Support set properties for a namespace [#10015](https://github.com/apache/pulsar/pull/10015) +- Fix cannot cleanup expired data after managed-ledger restart [#10087](https://github.com/apache/pulsar/pull/10087) +- [Default configuration] Enable sticky read by default [#10090](https://github.com/apache/pulsar/pull/10090) +- Add time based backlog quota [#10093](https://github.com/apache/pulsar/pull/10093) +- Fix IllegalStateException in PersistentReplicator [#10098](https://github.com/apache/pulsar/pull/10098) +- Support set max consumers per subscription for non-persistent dispatcher [#10121](https://github.com/apache/pulsar/pull/10121) +- Limit the number of producers/consumers that can connect per topic for each IP address [#10188](https://github.com/apache/pulsar/pull/10188) +- Close namespace clients when PulsarService is closed [#10196](https://github.com/apache/pulsar/pull/10196) +- Shutdown Broker gracefully, but forcefully after brokerShutdownTimeoutMs [#10199](https://github.com/apache/pulsar/pull/10199) +- Update the authentication data when an authentication refresh happens [#10203](https://github.com/apache/pulsar/pull/10203) +- Add preciseTopicPublishRateLimiterEnable to broker.conf [#10216](https://github.com/apache/pulsar/pull/10216) +- Fix entry cache size to be a negative value [#10225](https://github.com/apache/pulsar/pull/10225) +- Fix replicated subscriptions related LightProto issues [#10247](https://github.com/apache/pulsar/pull/10247) +- Pause the replicated subscriptions snapshots when there is no real traffic [#10292](https://github.com/apache/pulsar/pull/10292) +- Fix the inconsistency of advertised address [#10312](https://github.com/apache/pulsar/pull/10312) +- Support listenerName for HttpLookupService [#10319](https://github.com/apache/pulsar/pull/10319) +- Support truncate topic [#10326](https://github.com/apache/pulsar/pull/10326) +- Fix authorization error if partition number of partitioned topic is updated [10333](https://github.com/apache/pulsar/pull/10333) +- Fix managed ledger name that transaction log used [#10334](https://github.com/apache/pulsar/pull/10334) +- Catch topic policy not hit exception in handleSubscribe [#10341](https://github.com/apache/pulsar/pull/10341) +- Fix ConcurrentModificationException when attempting to update local broker data [#10347](https://github.com/apache/pulsar/pull/10347) +- Support seek to separate messageId/time for multiple topic reader [#10348](https://github.com/apache/pulsar/pull/10348) +- Resource locks should automatically revalidate after a metadata session is re-established [#10351](https://github.com/apache/pulsar/pull/10351) +- Add authentication data for the remote cluster [#10357](https://github.com/apache/pulsar/pull/10357) +- Support array type claims in JWT [#10375](https://github.com/apache/pulsar/pull/10375) +- Optimize locks in AuthenticationAthenz [#10381](https://github.com/apache/pulsar/pull/10381) +- Prevent carrying state of PositionImplRecyclable when recycled [#10404](https://github.com/apache/pulsar/pull/10404) +- Dispatch messages to consumer with permits [#10417](https://github.com/apache/pulsar/pull/10417) +- Fix NPE in unblock stuck subscription task when dispatcher has not created [#10430](https://github.com/apache/pulsar/pull/10430) +- Fix topic loading fails without any error when replicator init fails [#10432](https://github.com/apache/pulsar/pull/10432) +- Set timeout to unblock web-threads on update partition API [#10447](https://github.com/apache/pulsar/pull/10447) +- Fix CPU 100% when deleting namespace [#10454](https://github.com/apache/pulsar/pull/10454) +- Remove residual info after forcibly deleting the namespace [#10465](https://github.com/apache/pulsar/pull/10465) +- Fix potential memory leak of TopicPolicies [#10466](https://github.com/apache/pulsar/pull/10466) +- Fix publish callback's entry data is null during ledger rollover [#10467](https://github.com/apache/pulsar/pull/10467) +- Fix readModifyUpdate should return the new value in the future for metadata cache [#10474](https://github.com/apache/pulsar/pull/10474) +- Fix partitioned system topic check issue [#10529](https://github.com/apache/pulsar/pull/10529) +- Removed AspectJ based metrics for ZooKeeper [#10533](https://github.com/apache/pulsar/pull/10533) +- Allow to create MetadataCache with custom serde [#10543](https://github.com/apache/pulsar/pull/10543) +- Fix ack receipt version check issue [#10551](https://github.com/apache/pulsar/pull/10551) +- Expose average message size metrics for a topic [#10553](https://github.com/apache/pulsar/pull/10553) +- Fixed missed ZK caching when fetching list of namespaces for a tenant [#10594](https://github.com/apache/pulsar/pull/10594) +- Setup pulsar cluster with MetadataStore [#10600](https://github.com/apache/pulsar/pull/10600) +- Setup initial namespaces with MetadataStore [#10612](https://github.com/apache/pulsar/pull/10612) +- Convert bundle split into an HTTP async operation [#10619](https://github.com/apache/pulsar/pull/10619) +- Add metrics for non-contiguous deleted messages range [#10638](https://github.com/apache/pulsar/pull/10638) +- Fix consumer related topic stats only available while consumer or reader are connected [#10644](https://github.com/apache/pulsar/pull/10644) +- Allow to configure the number of BK client worker threads [#10649](https://github.com/apache/pulsar/pull/10649) +- Fix ConcurrentOpenLongPairRangeSet remove all ranges [#10656](https://github.com/apache/pulsar/pull/10656) +- Ensure all the ReadHandle gets properly closed on cache invalidation [#10659](https://github.com/apache/pulsar/pull/10659) +- Avoid the context switch when managedLedgerNewEntriesCheckDelayInMillis=0 [#10660](https://github.com/apache/pulsar/pull/10660) +- Allow to configure busy-wait in broker and client [#10661](https://github.com/apache/pulsar/pull/10661) +- Use single Jwt parser instance [#10664](https://github.com/apache/pulsar/pull/10664) +- Fix issues in advanceNonDurableCursors [#10667](https://github.com/apache/pulsar/pull/10667) +- LockManager use concurrent hash map to handle locks notifiications [#10680](https://github.com/apache/pulsar/pull/10680) +- Avoid making copies of internal maps when iterating [#10691](https://github.com/apache/pulsar/pull/10691) +- Fix issue where StackOverflowError occurs when trying to redeliver a large number of already acked messages [#10696](https://github.com/apache/pulsar/pull/10696) +- Fix NPE when filtering read entries [#10704](https://github.com/apache/pulsar/pull/10704) +- Fix deadlock when enable managed ledger interceptor [#10706](https://github.com/apache/pulsar/pull/10706) Review comment: ```suggestion - Fix deadlock when enabling managed ledger interceptor [#10706](https://github.com/apache/pulsar/pull/10706) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
