[GitHub] [pulsar] sijie added a comment to the discussion: ManagedLedger should streamline the read requests

2022-12-09 Thread GitBox
GitHub user sijie added a comment to the discussion: ManagedLedger should streamline the read requests @MarvinCai Are you working on this issue already? GitHub link: https://github.com/apache/pulsar/discussions/18852#discussioncomment-4352609 This is an automatically sent email for dev@

[GitHub] [pulsar] sijie added a comment to the discussion: ManagedLedger should streamline the read requests

2022-12-09 Thread GitBox
GitHub user sijie added a comment to the discussion: ManagedLedger should streamline the read requests @vicaya thank you for your feedback. @MarvinCai are you willing to give it a try? GitHub link: https://github.com/apache/pulsar/discussions/18852#discussioncomment-4352607 This is an

[GitHub] [pulsar] sijie added a comment to the discussion: ManagedLedger should streamline the read requests

2022-12-09 Thread GitBox
GitHub user sijie added a comment to the discussion: ManagedLedger should streamline the read requests @MarvinCai yes. I think it is worth pushing this logic to BK to provide a `StreamingReadHandle` over the BK `ReadHandle`. so that the read ahead logic can be reused for both bookkeeper read

[GitHub] [pulsar] sijie added a comment to the discussion: ManagedLedger should streamline the read requests

2022-12-09 Thread GitBox
GitHub user sijie added a comment to the discussion: ManagedLedger should streamline the read requests /cc @jiazhai @eolivelli in this thread. so they can provide some more thoughts around this if it is worth adding this readahead logic to BK read handle. GitHub link: https://github.com/apac

[GitHub] [pulsar] sijie added a comment to the discussion: ManagedLedger should streamline the read requests

2022-12-09 Thread GitBox
GitHub user sijie added a comment to the discussion: ManagedLedger should streamline the read requests I have seen in a production deployment the consumer can never catch up if bookie's avg read latency is 10+ms bad due to the disk and other workloads running on same machine. the problem can