[GitHub] [pulsar] rdhabalia commented on issue #4400: PIP 37: [pulsar-client] support large message size
rdhabalia commented on issue #4400: PIP 37: [pulsar-client] support large message size URL: https://github.com/apache/pulsar/pull/4400#issuecomment-596827783 we have many customers need this feature so, can someone please review the PR and can we merge the PR soon. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rdhabalia commented on issue #4400: PIP 37: [pulsar-client] support large message size
rdhabalia commented on issue #4400: PIP 37: [pulsar-client] support large message size URL: https://github.com/apache/pulsar/pull/4400#issuecomment-579949686 we have many users waiting for this feature. Can you please review it. @merlimat @sijie @jiazhai @nkurihar @jai1 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rdhabalia commented on issue #4400: PIP 37: [pulsar-client] support large message size
rdhabalia commented on issue #4400: PIP 37: [pulsar-client] support large message size URL: https://github.com/apache/pulsar/pull/4400#issuecomment-546039490 I think rebasing this PR takes lot of time as txn and other changes have changed a lot after this PR. @merlimat can you please review it as multiple users need this feature and #4247 and txn is not useful for multiple reasons. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rdhabalia commented on issue #4400: PIP 37: [pulsar-client] support large message size
rdhabalia commented on issue #4400: PIP 37: [pulsar-client] support large message size URL: https://github.com/apache/pulsar/pull/4400#issuecomment-545645477 @jiazhai sorry for late response as I had missed your comment. > After PIP36 (#4247 ), pulsar could support message size large than 5MB. That's incorrect. #4247 is not a useful feature and that could be used for a specific usecase but we can not use for large scale and multi tenant system where same bookie is serving topics from different tenants. For example, If client wants to publish message with 100MB and if broker allows to store one message with 100MB into bookie then bookie will max out in 1 or 2 messages and it can directly impact to other tenants for publish and draining backlog. Also, it will be hard to force throttling for one message with 100MB payload size. I have also described this scenario into the PIP. Right now, we have multiple users who are chunking the payload at producer side and merging at consumer side and they need this feature to avoid such complexity. So, we need this feature available for multiple users. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rdhabalia commented on issue #4400: PIP 37: [pulsar-client] support large message size
rdhabalia commented on issue #4400: PIP 37: [pulsar-client] support large message size URL: https://github.com/apache/pulsar/pull/4400#issuecomment-497168596 @jiazhai >> There are some PRs on going for PIP-36, will it partly solve this issue? No, right now, pulsar can only support up to 5MB message size and PIP-36 doesn't support large message but it allows broker to configure max message size from 0 to 5MB. we need it for customers who want to publish large message into large-scale specialized pulsar instance for which txn might not be feasible option. it's similar like chunking in kafka mentioned [here](https://medium.com/workday-engineering/large-message-handling-with-kafka-chunking-vs-external-store-33b0fc4ccf14). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
