Re: error comes when set topology.acker.executors as 0
We are in a big company, and the infra group have changed the storm source code , at the same time it's not easy to drive the group to update it. Our project is a legacy system written by others, and we don't want to reconstruct it. I let the acker number down, and add the batch size, data in kafka can be proceessed now. But as the data grows, the same program may reproduce 2016-08-18 9:18 GMT+08:00 Manu Zhang : > Hi, > > That is pretty old version of Storm. > I'd recommend to upgrade to Storm 0.9.6 or 0.10.x. > > Thanks, > Manu > > On Wed, Aug 17, 2016 at 2:43 PM 杨浩 wrote: > > > - 0.9.1 > > - more than 250M B/s some time > > > > The flow is growing bigger and bigger and we use offline program to > update > > the realtime result. We don't care so much about real-time and stability, > > we want the program to process the high throughput. > > > > 2016-08-17 12:16 GMT+08:00 Manu Zhang : > > > > > Hi Hao, > > > > > > A few questions > > > > > > - Which version of storm are you using ? > > > - What's the current throughput number ? > > > - What's your expected throughput number ? > > > > > > You may try larger batch size. > > > > > > Thanks, > > > Manu > > > > > > On Wed, Aug 17, 2016 at 11:10 AM 杨浩 wrote: > > > > > > > thanks for that, is there any other method to improve the throuput > when > > > > using trident > > > > > > > > 2016-08-17 11:05 GMT+08:00 Manu Zhang : > > > > > > > > > Hi, > > > > > > > > > > Trident must have ackers to work. You can't set > > > topology.acker.executors > > > > as > > > > > 0. > > > > > > > > > > Thanks, > > > > > Manu > > > > > > > > > > On Wed, Aug 17, 2016 at 10:50 AM 杨浩 wrote: > > > > > > > > > > > I want to improve the throughput of Storm, and I > > > > > > set topology.acker.executors as 0, then program runs into error. > I > > > > write > > > > > > storm using trident, please help to solve the program > > > > > > [image: 内嵌图片 1] > > > > > > > > > > > > > > > > > > > > >
Re: error comes when set topology.acker.executors as 0
- 0.9.1 - more than 250M B/s some time The flow is growing bigger and bigger and we use offline program to update the realtime result. We don't care so much about real-time and stability, we want the program to process the high throughput. 2016-08-17 12:16 GMT+08:00 Manu Zhang : > Hi Hao, > > A few questions > > - Which version of storm are you using ? > - What's the current throughput number ? > - What's your expected throughput number ? > > You may try larger batch size. > > Thanks, > Manu > > On Wed, Aug 17, 2016 at 11:10 AM 杨浩 wrote: > > > thanks for that, is there any other method to improve the throuput when > > using trident > > > > 2016-08-17 11:05 GMT+08:00 Manu Zhang : > > > > > Hi, > > > > > > Trident must have ackers to work. You can't set > topology.acker.executors > > as > > > 0. > > > > > > Thanks, > > > Manu > > > > > > On Wed, Aug 17, 2016 at 10:50 AM 杨浩 wrote: > > > > > > > I want to improve the throughput of Storm, and I > > > > set topology.acker.executors as 0, then program runs into error. I > > write > > > > storm using trident, please help to solve the program > > > > [image: 内嵌图片 1] > > > > > > > > > >
Re: error comes when set topology.acker.executors as 0
thanks for that, is there any other method to improve the throuput when using trident 2016-08-17 11:05 GMT+08:00 Manu Zhang : > Hi, > > Trident must have ackers to work. You can't set topology.acker.executors as > 0. > > Thanks, > Manu > > On Wed, Aug 17, 2016 at 10:50 AM 杨浩 wrote: > > > I want to improve the throughput of Storm, and I > > set topology.acker.executors as 0, then program runs into error. I write > > storm using trident, please help to solve the program > > [image: 内嵌图片 1] > > >
error comes when set topology.acker.executors as 0
I want to improve the throughput of Storm, and I set topology.acker.executors as 0, then program runs into error. I write storm using trident, please help to solve the program [image: 内嵌图片 1]
