Re: TLP tools for stress testing and building test clusters in AWS
Thanks Anthony for going that proverbial extra mile to cover people in different time zones too. I believe other people will find your talk as helpful as we did. Regards On Wed, 17 Apr 2019 at 10:08, Anthony Grasso wrote: > > Hi Stefan and devs, > > I have set up a zoom link for the TLP tool set intro that will be on in an > hours time (17 April 2019 @ 11:00AM AEST): https://zoom.us/j/272648772 > > This link is open so if anyone else wishes to join they are welcome to do > so. I will be covering the same topics Jon is covering in his meeting > tomorrow. > > Regards, > Anthony > > > On Wed, 17 Apr 2019 at 08:29, Anthony Grasso > wrote: > > > Hi Stefan, > > > > Thanks for sending the invite out! > > > > Just wondering what do you think of the idea of having a Zoom meeting that > > anyone can join? This way anyone else interested can join us as well. I can > > set that up if you like? > > > > Cheers, > > Anthony > > > > On Tue, 16 Apr 2019 at 21:24, Stefan Miklosovic < > > stefan.mikloso...@instaclustr.com> wrote: > > > >> Hi Anthony, > >> > >> sounds good. I ve sent you Hangouts meeting invitation privately. > >> > >> Regards > >> > >> On Tue, 16 Apr 2019 at 14:53, Anthony Grasso > >> wrote: > >> > > >> > Hi Stefan, > >> > > >> > I have been working with Jon on developing the tool set. I can do a Zoom > >> > call tomorrow (Wednesday) at 11am AEST if that works for you? We can go > >> > through all the same information that Jon is going to go through in his > >> > call. Note that I am in the same timezone as you, so if tomorrow > >> morning is > >> > no good we can always do the afternoon. > >> > > >> > Cheers, > >> > Anthony > >> > > >> > > >> > On Sat, 13 Apr 2019 at 22:38, Stefan Miklosovic < > >> > stefan.mikloso...@instaclustr.com> wrote: > >> > > >> > > Hi Jon, > >> > > > >> > > I would like be on that call too but I am off on Thursday. > >> > > > >> > > I am from Australia so 5pm London time is ours 2am next day so your > >> > > Wednesday morning is my Thursday night. Wednesday early morning so > >> > > your Tuesday morning and London's afternoon would be the best. > >> > > > >> > > Recording the thing would be definitely helpful too. > >> > > > >> > > On Sat, 13 Apr 2019 at 07:45, Jon Haddad wrote: > >> > > > > >> > > > I'd be more than happy to hop on a call next week to give you both > >> > > > (and anyone else interested) a tour of our dev tools. Maybe > >> something > >> > > > early morning on my end, which should be your evening, could work? > >> > > > > >> > > > I can set up a Zoom conference to get everyone acquainted. We can > >> > > > record and post it for any who can't make it. > >> > > > > >> > > > I'm thinking Tuesday, Wednesday, or Thursday morning, 9AM Pacific > >> (5pm > >> > > > London)? If anyone's interested please reply with what dates work. > >> > > > I'll be sure to post the details back here with the zoom link in > >> case > >> > > > anyone wants to join that didn't get a chance to reply, as well as a > >> > > > link to the recorded call. > >> > > > > >> > > > Jon > >> > > > > >> > > > On Fri, Apr 12, 2019 at 10:41 AM Benedict Elliott Smith > >> > > > wrote: > >> > > > > > >> > > > > +1 > >> > > > > > >> > > > > I’m also just as excited to see some standardised workloads and > >> test > >> > > bed. At the moment we’re benefiting from some large contributors > >> doing > >> > > their own proprietary performance testing, which is super valuable and > >> > > something we’ve lacked before. But I’m also keen to see some more > >> > > representative workloads that are reproducible by anybody in the > >> community > >> > > take shape. > >> > > > > > >> > > > > > >> > > > > > On 12 Apr 2019, at 18:09, Aleksey Yeshchenko > >> > > wrote: > >> > > > > > > >> > > > > > Hey Jon, > >> > > > > > > >> > > > > > This sounds exciting and pretty useful, thanks. > >> > > > > > > >> > > > > > Looking forward to using tlp-stress for validating 15066 > >> performance. > >> > > > > > > >> > > > > > We should touch base some time next week to pick a > >> comprehensive set > >> > > of workloads and versions, perhaps? > >> > > > > > > >> > > > > > > >> > > > > >> On 12 Apr 2019, at 16:34, Jon Haddad > >> wrote: > >> > > > > >> > >> > > > > >> I don't want to derail the discussion about Stabilizing > >> Internode > >> > > > > >> Messaging, so I'm starting this as a separate thread. There > >> was a > >> > > > > >> comment that Josh made [1] about doing performance testing > >> with real > >> > > > > >> clusters as well as a lot of microbenchmarks, and I'm 100% in > >> > > support > >> > > > > >> of this. We've been working on some tooling at TLP for the > >> last > >> > > > > >> several months to make this a lot easier. One of the goals > >> has been > >> > > > > >> to help improve the 4.0 testing process. > >> > > > > >> > >> > > > > >> The first tool we have is tlp-stress [2]. It's designed with > >> a "get > >> > > > > >> started in 5 minutes" mindset. My goal was to ship a stress >
Re: TLP tools for stress testing and building test clusters in AWS
> I have set up a zoom link for the TLP tool set intro that will be on in an > hours time (17 April 2019 @ 11:00AM AEST): https://zoom.us/j/272648772 > Super rad - thanks for setting this up, Anthony! - To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
Re: TLP tools for stress testing and building test clusters in AWS
Hi Stefan and devs, I have set up a zoom link for the TLP tool set intro that will be on in an hours time (17 April 2019 @ 11:00AM AEST): https://zoom.us/j/272648772 This link is open so if anyone else wishes to join they are welcome to do so. I will be covering the same topics Jon is covering in his meeting tomorrow. Regards, Anthony On Wed, 17 Apr 2019 at 08:29, Anthony Grasso wrote: > Hi Stefan, > > Thanks for sending the invite out! > > Just wondering what do you think of the idea of having a Zoom meeting that > anyone can join? This way anyone else interested can join us as well. I can > set that up if you like? > > Cheers, > Anthony > > On Tue, 16 Apr 2019 at 21:24, Stefan Miklosovic < > stefan.mikloso...@instaclustr.com> wrote: > >> Hi Anthony, >> >> sounds good. I ve sent you Hangouts meeting invitation privately. >> >> Regards >> >> On Tue, 16 Apr 2019 at 14:53, Anthony Grasso >> wrote: >> > >> > Hi Stefan, >> > >> > I have been working with Jon on developing the tool set. I can do a Zoom >> > call tomorrow (Wednesday) at 11am AEST if that works for you? We can go >> > through all the same information that Jon is going to go through in his >> > call. Note that I am in the same timezone as you, so if tomorrow >> morning is >> > no good we can always do the afternoon. >> > >> > Cheers, >> > Anthony >> > >> > >> > On Sat, 13 Apr 2019 at 22:38, Stefan Miklosovic < >> > stefan.mikloso...@instaclustr.com> wrote: >> > >> > > Hi Jon, >> > > >> > > I would like be on that call too but I am off on Thursday. >> > > >> > > I am from Australia so 5pm London time is ours 2am next day so your >> > > Wednesday morning is my Thursday night. Wednesday early morning so >> > > your Tuesday morning and London's afternoon would be the best. >> > > >> > > Recording the thing would be definitely helpful too. >> > > >> > > On Sat, 13 Apr 2019 at 07:45, Jon Haddad wrote: >> > > > >> > > > I'd be more than happy to hop on a call next week to give you both >> > > > (and anyone else interested) a tour of our dev tools. Maybe >> something >> > > > early morning on my end, which should be your evening, could work? >> > > > >> > > > I can set up a Zoom conference to get everyone acquainted. We can >> > > > record and post it for any who can't make it. >> > > > >> > > > I'm thinking Tuesday, Wednesday, or Thursday morning, 9AM Pacific >> (5pm >> > > > London)? If anyone's interested please reply with what dates work. >> > > > I'll be sure to post the details back here with the zoom link in >> case >> > > > anyone wants to join that didn't get a chance to reply, as well as a >> > > > link to the recorded call. >> > > > >> > > > Jon >> > > > >> > > > On Fri, Apr 12, 2019 at 10:41 AM Benedict Elliott Smith >> > > > wrote: >> > > > > >> > > > > +1 >> > > > > >> > > > > I’m also just as excited to see some standardised workloads and >> test >> > > bed. At the moment we’re benefiting from some large contributors >> doing >> > > their own proprietary performance testing, which is super valuable and >> > > something we’ve lacked before. But I’m also keen to see some more >> > > representative workloads that are reproducible by anybody in the >> community >> > > take shape. >> > > > > >> > > > > >> > > > > > On 12 Apr 2019, at 18:09, Aleksey Yeshchenko >> > > wrote: >> > > > > > >> > > > > > Hey Jon, >> > > > > > >> > > > > > This sounds exciting and pretty useful, thanks. >> > > > > > >> > > > > > Looking forward to using tlp-stress for validating 15066 >> performance. >> > > > > > >> > > > > > We should touch base some time next week to pick a >> comprehensive set >> > > of workloads and versions, perhaps? >> > > > > > >> > > > > > >> > > > > >> On 12 Apr 2019, at 16:34, Jon Haddad >> wrote: >> > > > > >> >> > > > > >> I don't want to derail the discussion about Stabilizing >> Internode >> > > > > >> Messaging, so I'm starting this as a separate thread. There >> was a >> > > > > >> comment that Josh made [1] about doing performance testing >> with real >> > > > > >> clusters as well as a lot of microbenchmarks, and I'm 100% in >> > > support >> > > > > >> of this. We've been working on some tooling at TLP for the >> last >> > > > > >> several months to make this a lot easier. One of the goals >> has been >> > > > > >> to help improve the 4.0 testing process. >> > > > > >> >> > > > > >> The first tool we have is tlp-stress [2]. It's designed with >> a "get >> > > > > >> started in 5 minutes" mindset. My goal was to ship a stress >> tool >> > > that >> > > > > >> ships with real workloads out of the box that can be easily >> tweaked, >> > > > > >> similar to how fio allows you to design a disk workload and >> tweak it >> > > > > >> with paramaters. Included are stress workloads that stress >> LWTs >> > > (two >> > > > > >> different types), materialized views, counters, time series, >> and >> > > > > >> key-value workloads. Each workload can be modified easily to >> change >> > > > > >> compaction strategies,
Re: TLP tools for stress testing and building test clusters in AWS
The one I sent out is open, no separate invite required. On Tue, Apr 16, 2019 at 3:47 PM Dinesh Joshi wrote: > > I'm slightly confused. The zoom meeting mentioned in this thread is only open > to who have registered interest here? If so, can someone please add me? > > Dinesh > > > On Apr 16, 2019, at 3:29 PM, Anthony Grasso > > wrote: > > > > Hi Stefan, > > > > Thanks for sending the invite out! > > > > Just wondering what do you think of the idea of having a Zoom meeting that > > anyone can join? This way anyone else interested can join us as well. I can > > set that up if you like? > > > > Cheers, > > Anthony > > > > On Tue, 16 Apr 2019 at 21:24, Stefan Miklosovic < > > stefan.mikloso...@instaclustr.com> wrote: > > > >> Hi Anthony, > >> > >> sounds good. I ve sent you Hangouts meeting invitation privately. > >> > >> Regards > >> > >> On Tue, 16 Apr 2019 at 14:53, Anthony Grasso > >> wrote: > >>> > >>> Hi Stefan, > >>> > >>> I have been working with Jon on developing the tool set. I can do a Zoom > >>> call tomorrow (Wednesday) at 11am AEST if that works for you? We can go > >>> through all the same information that Jon is going to go through in his > >>> call. Note that I am in the same timezone as you, so if tomorrow morning > >> is > >>> no good we can always do the afternoon. > >>> > >>> Cheers, > >>> Anthony > >>> > >>> > >>> On Sat, 13 Apr 2019 at 22:38, Stefan Miklosovic < > >>> stefan.mikloso...@instaclustr.com> wrote: > >>> > Hi Jon, > > I would like be on that call too but I am off on Thursday. > > I am from Australia so 5pm London time is ours 2am next day so your > Wednesday morning is my Thursday night. Wednesday early morning so > your Tuesday morning and London's afternoon would be the best. > > Recording the thing would be definitely helpful too. > > On Sat, 13 Apr 2019 at 07:45, Jon Haddad wrote: > > > > I'd be more than happy to hop on a call next week to give you both > > (and anyone else interested) a tour of our dev tools. Maybe > >> something > > early morning on my end, which should be your evening, could work? > > > > I can set up a Zoom conference to get everyone acquainted. We can > > record and post it for any who can't make it. > > > > I'm thinking Tuesday, Wednesday, or Thursday morning, 9AM Pacific > >> (5pm > > London)? If anyone's interested please reply with what dates work. > > I'll be sure to post the details back here with the zoom link in case > > anyone wants to join that didn't get a chance to reply, as well as a > > link to the recorded call. > > > > Jon > > > > On Fri, Apr 12, 2019 at 10:41 AM Benedict Elliott Smith > > wrote: > >> > >> +1 > >> > >> I’m also just as excited to see some standardised workloads and > >> test > bed. At the moment we’re benefiting from some large contributors doing > their own proprietary performance testing, which is super valuable and > something we’ve lacked before. But I’m also keen to see some more > representative workloads that are reproducible by anybody in the > >> community > take shape. > >> > >> > >>> On 12 Apr 2019, at 18:09, Aleksey Yeshchenko > wrote: > >>> > >>> Hey Jon, > >>> > >>> This sounds exciting and pretty useful, thanks. > >>> > >>> Looking forward to using tlp-stress for validating 15066 > >> performance. > >>> > >>> We should touch base some time next week to pick a comprehensive > >> set > of workloads and versions, perhaps? > >>> > >>> > On 12 Apr 2019, at 16:34, Jon Haddad wrote: > > I don't want to derail the discussion about Stabilizing > >> Internode > Messaging, so I'm starting this as a separate thread. There > >> was a > comment that Josh made [1] about doing performance testing with > >> real > clusters as well as a lot of microbenchmarks, and I'm 100% in > support > of this. We've been working on some tooling at TLP for the last > several months to make this a lot easier. One of the goals has > >> been > to help improve the 4.0 testing process. > > The first tool we have is tlp-stress [2]. It's designed with a > >> "get > started in 5 minutes" mindset. My goal was to ship a stress > >> tool > that > ships with real workloads out of the box that can be easily > >> tweaked, > similar to how fio allows you to design a disk workload and > >> tweak it > with paramaters. Included are stress workloads that stress LWTs > (two > different types), materialized views, counters, time series, and > key-value workloads. Each workload can be modified easily to > >> change > compaction strategies, concurrent operations, number of > >> partitions. > We can run workloads for a set
Re: TLP tools for stress testing and building test clusters in AWS
I'm slightly confused. The zoom meeting mentioned in this thread is only open to who have registered interest here? If so, can someone please add me? Dinesh > On Apr 16, 2019, at 3:29 PM, Anthony Grasso wrote: > > Hi Stefan, > > Thanks for sending the invite out! > > Just wondering what do you think of the idea of having a Zoom meeting that > anyone can join? This way anyone else interested can join us as well. I can > set that up if you like? > > Cheers, > Anthony > > On Tue, 16 Apr 2019 at 21:24, Stefan Miklosovic < > stefan.mikloso...@instaclustr.com> wrote: > >> Hi Anthony, >> >> sounds good. I ve sent you Hangouts meeting invitation privately. >> >> Regards >> >> On Tue, 16 Apr 2019 at 14:53, Anthony Grasso >> wrote: >>> >>> Hi Stefan, >>> >>> I have been working with Jon on developing the tool set. I can do a Zoom >>> call tomorrow (Wednesday) at 11am AEST if that works for you? We can go >>> through all the same information that Jon is going to go through in his >>> call. Note that I am in the same timezone as you, so if tomorrow morning >> is >>> no good we can always do the afternoon. >>> >>> Cheers, >>> Anthony >>> >>> >>> On Sat, 13 Apr 2019 at 22:38, Stefan Miklosovic < >>> stefan.mikloso...@instaclustr.com> wrote: >>> Hi Jon, I would like be on that call too but I am off on Thursday. I am from Australia so 5pm London time is ours 2am next day so your Wednesday morning is my Thursday night. Wednesday early morning so your Tuesday morning and London's afternoon would be the best. Recording the thing would be definitely helpful too. On Sat, 13 Apr 2019 at 07:45, Jon Haddad wrote: > > I'd be more than happy to hop on a call next week to give you both > (and anyone else interested) a tour of our dev tools. Maybe >> something > early morning on my end, which should be your evening, could work? > > I can set up a Zoom conference to get everyone acquainted. We can > record and post it for any who can't make it. > > I'm thinking Tuesday, Wednesday, or Thursday morning, 9AM Pacific >> (5pm > London)? If anyone's interested please reply with what dates work. > I'll be sure to post the details back here with the zoom link in case > anyone wants to join that didn't get a chance to reply, as well as a > link to the recorded call. > > Jon > > On Fri, Apr 12, 2019 at 10:41 AM Benedict Elliott Smith > wrote: >> >> +1 >> >> I’m also just as excited to see some standardised workloads and >> test bed. At the moment we’re benefiting from some large contributors doing their own proprietary performance testing, which is super valuable and something we’ve lacked before. But I’m also keen to see some more representative workloads that are reproducible by anybody in the >> community take shape. >> >> >>> On 12 Apr 2019, at 18:09, Aleksey Yeshchenko wrote: >>> >>> Hey Jon, >>> >>> This sounds exciting and pretty useful, thanks. >>> >>> Looking forward to using tlp-stress for validating 15066 >> performance. >>> >>> We should touch base some time next week to pick a comprehensive >> set of workloads and versions, perhaps? >>> >>> On 12 Apr 2019, at 16:34, Jon Haddad wrote: I don't want to derail the discussion about Stabilizing >> Internode Messaging, so I'm starting this as a separate thread. There >> was a comment that Josh made [1] about doing performance testing with >> real clusters as well as a lot of microbenchmarks, and I'm 100% in support of this. We've been working on some tooling at TLP for the last several months to make this a lot easier. One of the goals has >> been to help improve the 4.0 testing process. The first tool we have is tlp-stress [2]. It's designed with a >> "get started in 5 minutes" mindset. My goal was to ship a stress >> tool that ships with real workloads out of the box that can be easily >> tweaked, similar to how fio allows you to design a disk workload and >> tweak it with paramaters. Included are stress workloads that stress LWTs (two different types), materialized views, counters, time series, and key-value workloads. Each workload can be modified easily to >> change compaction strategies, concurrent operations, number of >> partitions. We can run workloads for a set number of iterations or a custom duration. We've used this *extensively* at TLP to help our customers and most of our blog posts that discuss performance use it as >> well. It exports data to both a CSV format and auto sets up >> prometheus for metrics collection / aggregation. As an example, we were able >> to
Re: TLP tools for stress testing and building test clusters in AWS
Hi Stefan, Thanks for sending the invite out! Just wondering what do you think of the idea of having a Zoom meeting that anyone can join? This way anyone else interested can join us as well. I can set that up if you like? Cheers, Anthony On Tue, 16 Apr 2019 at 21:24, Stefan Miklosovic < stefan.mikloso...@instaclustr.com> wrote: > Hi Anthony, > > sounds good. I ve sent you Hangouts meeting invitation privately. > > Regards > > On Tue, 16 Apr 2019 at 14:53, Anthony Grasso > wrote: > > > > Hi Stefan, > > > > I have been working with Jon on developing the tool set. I can do a Zoom > > call tomorrow (Wednesday) at 11am AEST if that works for you? We can go > > through all the same information that Jon is going to go through in his > > call. Note that I am in the same timezone as you, so if tomorrow morning > is > > no good we can always do the afternoon. > > > > Cheers, > > Anthony > > > > > > On Sat, 13 Apr 2019 at 22:38, Stefan Miklosovic < > > stefan.mikloso...@instaclustr.com> wrote: > > > > > Hi Jon, > > > > > > I would like be on that call too but I am off on Thursday. > > > > > > I am from Australia so 5pm London time is ours 2am next day so your > > > Wednesday morning is my Thursday night. Wednesday early morning so > > > your Tuesday morning and London's afternoon would be the best. > > > > > > Recording the thing would be definitely helpful too. > > > > > > On Sat, 13 Apr 2019 at 07:45, Jon Haddad wrote: > > > > > > > > I'd be more than happy to hop on a call next week to give you both > > > > (and anyone else interested) a tour of our dev tools. Maybe > something > > > > early morning on my end, which should be your evening, could work? > > > > > > > > I can set up a Zoom conference to get everyone acquainted. We can > > > > record and post it for any who can't make it. > > > > > > > > I'm thinking Tuesday, Wednesday, or Thursday morning, 9AM Pacific > (5pm > > > > London)? If anyone's interested please reply with what dates work. > > > > I'll be sure to post the details back here with the zoom link in case > > > > anyone wants to join that didn't get a chance to reply, as well as a > > > > link to the recorded call. > > > > > > > > Jon > > > > > > > > On Fri, Apr 12, 2019 at 10:41 AM Benedict Elliott Smith > > > > wrote: > > > > > > > > > > +1 > > > > > > > > > > I’m also just as excited to see some standardised workloads and > test > > > bed. At the moment we’re benefiting from some large contributors doing > > > their own proprietary performance testing, which is super valuable and > > > something we’ve lacked before. But I’m also keen to see some more > > > representative workloads that are reproducible by anybody in the > community > > > take shape. > > > > > > > > > > > > > > > > On 12 Apr 2019, at 18:09, Aleksey Yeshchenko > > > wrote: > > > > > > > > > > > > Hey Jon, > > > > > > > > > > > > This sounds exciting and pretty useful, thanks. > > > > > > > > > > > > Looking forward to using tlp-stress for validating 15066 > performance. > > > > > > > > > > > > We should touch base some time next week to pick a comprehensive > set > > > of workloads and versions, perhaps? > > > > > > > > > > > > > > > > > >> On 12 Apr 2019, at 16:34, Jon Haddad wrote: > > > > > >> > > > > > >> I don't want to derail the discussion about Stabilizing > Internode > > > > > >> Messaging, so I'm starting this as a separate thread. There > was a > > > > > >> comment that Josh made [1] about doing performance testing with > real > > > > > >> clusters as well as a lot of microbenchmarks, and I'm 100% in > > > support > > > > > >> of this. We've been working on some tooling at TLP for the last > > > > > >> several months to make this a lot easier. One of the goals has > been > > > > > >> to help improve the 4.0 testing process. > > > > > >> > > > > > >> The first tool we have is tlp-stress [2]. It's designed with a > "get > > > > > >> started in 5 minutes" mindset. My goal was to ship a stress > tool > > > that > > > > > >> ships with real workloads out of the box that can be easily > tweaked, > > > > > >> similar to how fio allows you to design a disk workload and > tweak it > > > > > >> with paramaters. Included are stress workloads that stress LWTs > > > (two > > > > > >> different types), materialized views, counters, time series, and > > > > > >> key-value workloads. Each workload can be modified easily to > change > > > > > >> compaction strategies, concurrent operations, number of > partitions. > > > > > >> We can run workloads for a set number of iterations or a custom > > > > > >> duration. We've used this *extensively* at TLP to help our > > > customers > > > > > >> and most of our blog posts that discuss performance use it as > well. > > > > > >> It exports data to both a CSV format and auto sets up > prometheus for > > > > > >> metrics collection / aggregation. As an example, we were able > to > > > > > >> determine that the compression length set on the paxos tables > > > imposes > > >
Re: TLP tools for stress testing and building test clusters in AWS
Yes, sorry about that. Wednesday morning 9am PT On Tue, Apr 16, 2019 at 3:26 AM Benedict Elliott Smith wrote: > Just to confirm, this is on Wednesday? > > > On 15 Apr 2019, at 22:38, Jon Haddad wrote: > > > > Hey all, > > > > I've set up a Zoom call for 9AM Pacific time. Everyone's welcome to > join. > > > > https://zoom.us/j/189920888 > > > > Looking forward to a good discussion on how we can all pitch in on > > getting 4.0 out the door. > > > > Jon > > > > On Sat, Apr 13, 2019 at 9:14 AM Jonathan Koppenhofer > > wrote: > >> > >> Wednesday would work for me. > >> > >> We use and (slightly) contribute to tlp tools. We are platform testing > and > >> beginning 4.0 testing ourselves, so an in person overview would be > great! > >> > >> On Sat, Apr 13, 2019, 8:48 AM Aleksey Yeshchenko > > >> wrote: > >> > >>> Wednesday and Thursday, either, at 9 AM pacific WFM. > >>> > On 13 Apr 2019, at 13:31, Stefan Miklosovic < > >>> stefan.mikloso...@instaclustr.com> wrote: > > Hi Jon, > > I would like be on that call too but I am off on Thursday. > > I am from Australia so 5pm London time is ours 2am next day so your > Wednesday morning is my Thursday night. Wednesday early morning so > your Tuesday morning and London's afternoon would be the best. > > Recording the thing would be definitely helpful too. > > On Sat, 13 Apr 2019 at 07:45, Jon Haddad wrote: > > > > I'd be more than happy to hop on a call next week to give you both > > (and anyone else interested) a tour of our dev tools. Maybe > something > > early morning on my end, which should be your evening, could work? > > > > I can set up a Zoom conference to get everyone acquainted. We can > > record and post it for any who can't make it. > > > > I'm thinking Tuesday, Wednesday, or Thursday morning, 9AM Pacific > (5pm > > London)? If anyone's interested please reply with what dates work. > > I'll be sure to post the details back here with the zoom link in case > > anyone wants to join that didn't get a chance to reply, as well as a > > link to the recorded call. > > > > Jon > > > > On Fri, Apr 12, 2019 at 10:41 AM Benedict Elliott Smith > > wrote: > >> > >> +1 > >> > >> I’m also just as excited to see some standardised workloads and test > >>> bed. At the moment we’re benefiting from some large contributors doing > >>> their own proprietary performance testing, which is super valuable and > >>> something we’ve lacked before. But I’m also keen to see some more > >>> representative workloads that are reproducible by anybody in the > community > >>> take shape. > >> > >> > >>> On 12 Apr 2019, at 18:09, Aleksey Yeshchenko > >>> wrote: > >>> > >>> Hey Jon, > >>> > >>> This sounds exciting and pretty useful, thanks. > >>> > >>> Looking forward to using tlp-stress for validating 15066 > performance. > >>> > >>> We should touch base some time next week to pick a comprehensive > set > >>> of workloads and versions, perhaps? > >>> > >>> > On 12 Apr 2019, at 16:34, Jon Haddad wrote: > > I don't want to derail the discussion about Stabilizing Internode > Messaging, so I'm starting this as a separate thread. There was a > comment that Josh made [1] about doing performance testing with > real > clusters as well as a lot of microbenchmarks, and I'm 100% in > support > of this. We've been working on some tooling at TLP for the last > several months to make this a lot easier. One of the goals has > been > to help improve the 4.0 testing process. > > The first tool we have is tlp-stress [2]. It's designed with a > "get > started in 5 minutes" mindset. My goal was to ship a stress tool > >>> that > ships with real workloads out of the box that can be easily > tweaked, > similar to how fio allows you to design a disk workload and tweak > it > with paramaters. Included are stress workloads that stress LWTs > (two > different types), materialized views, counters, time series, and > key-value workloads. Each workload can be modified easily to > change > compaction strategies, concurrent operations, number of > partitions. > We can run workloads for a set number of iterations or a custom > duration. We've used this *extensively* at TLP to help our > customers > and most of our blog posts that discuss performance use it as > well. > It exports data to both a CSV format and auto sets up prometheus > for > metrics collection / aggregation. As an example, we were able to > determine that the compression length set on the paxos tables > imposes > a significant overhead when using the Locking LWT workload, which > simulates locking and
Re: Jira Updates
I have no useful feedback on the changes (haven't had a chance to test
them, probably won't soon) - just wanted to say thanks for taking point on
this Benedict.
On Tue, Apr 16, 2019 at 6:23 AM Benedict Elliott Smith
wrote:
> Some exciting news from the Jira changes (maybe).
>
> We’re done! We’ve even achieved the stretch goals.
>
> Does anyone have any further suggestions from the new workflow, after
> having tried it out for a bit? Speak up while we have the easy capability
> to make changes!
>
> - Somebody has proposed making Change Category a multi-select list, which
> would require recreating this. Does anybody have an opinion on this?
> - Would anybody prefer that one of the new fields be recreated as a
> project-specific select-list, like Platform and Impacts (see below), so we
> can manage the options?
>
> Gavin McDonald kindly set us up with two plugins. With the ScriptRunner
> plugin, I’ve setup scripted listeners to maintain the Authors field (based
> on Assignee) and Priority field (based on updates to Severity), as well as
> initialising both of these for all issues. Try it out - it works
> surprisingly well, but please let me know ASAP if there are any bugs.
>
> The Authors management may be slightly unintuitive, in that if assignee is
> updated we always overwrite Authors to the value of the assignee. We could
> maybe do something more sophisticated by checking what the prior value of
> Assignee was, but this looked very fiddly. I think in the vast majority of
> circumstances this will work as we expect; if there are multiple authors
> actively working I expect the assignee not to change; if the assignee is
> cleared we probably want to clear the Authors field; and if it is set to a
> new value it’s probably the new author. If somebody has suggestions for a
> better approach, please chime in.
>
> The other plugin provided us with project-specific select lists - which
> we’ve used for Platform and Impacts. These can be modified by any project
> admin, so we can maintain them ourselves. Impacts and Platform have a
> default value of None and All respectively, to avoid UX ugliness with this
> multi-select list; don’t ask me why this makes them prettier, but it does.
> I had originally proposed using these values so that we could force the
> user to specify a value, but opted to not make the UX ugly, while
> supporting project ownership of the options. Opinions on this decision
> welcome.
>
> For posterity, here are the ScriptRunner scripts I used and setup:
>
>
> = LISTENER TO SET PRIORITY WHEN SEVERITY UPDATED
>
> import com.atlassian.jira.component.ComponentAccessor
> import com.atlassian.jira.event.type.EventDispatchOption
> import com.atlassian.jira.issue.MutableIssue
> import com.atlassian.jira.util.ImportUtils
> import com.atlassian.jira.user.ApplicationUser;
> import com.atlassian.jira.issue.util.DefaultIssueChangeHolder
> import com.atlassian.jira.issue.util.IssueChangeHolder
> import com.atlassian.jira.issue.priority.Priority
> import com.atlassian.jira.issue.context.IssueContext
> import com.atlassian.jira.issue.customfields.option.Option
> import com.atlassian.jira.issue.customfields.option.Options
> import com.atlassian.jira.issue.customfields.manager.OptionsManager
>
> def issue = event.issue as MutableIssue
> def severityChange = null !=
> event?.getChangeLog()?.getRelated("ChildChangeItem")?.find {it.field ==
> "Severity"}
> if (issue.issueType.name == "Bug" && severityChange) {
> def issueManager = ComponentAccessor.getIssueManager()
> def customFieldManager = ComponentAccessor.getCustomFieldManager()
> def cf =
> customFieldManager.getCustomFieldObject("customfield_12313820" );
>
> def constantsManager = ComponentAccessor.getConstantsManager()
> Priority pUrgent = constantsManager.getPriorities().find {it.name ==
> "Urgent"}
> Priority pHigh = constantsManager.getPriorities().find {it.name ==
> "High"}
> Priority pNormal = constantsManager.getPriorities().find {it.name ==
> "Normal"}
> Priority pLow = constantsManager.getPriorities().find {it.name ==
> "Low"}
>
> def optManager = ComponentAccessor.getOptionsManager();
> Options severities =
> optManager.getOptions(cf.getRelevantConfig(IssueContext.GLOBAL))
> Option sCritical = severities.getOptionForValue("Critical", null)
> Option sNormal = severities.getOptionForValue("Normal", null)
> Option sLow = severities.getOptionForValue("Low", null)
>
> def sdUser = ComponentAccessor.getUserManager().getUserByKey('robot')
> def priority = issue.getPriority()
> def severity = issue.getCustomFieldValue(cf)
>
> if (severity != null) {
> def newPriority = priority;
> if (severity == sCritical) {
> newPriority = pUrgent
> } else if (severity == sNormal) {
> newPriority = pNormal
> } else if (severity == sLow) {
> newPriority = pLow
> }
> if (newPriority != priority) {
Re: Multi-column relations not allowed on partition key/s
Hi Abhishek, Sorry for the slow response. I would assume the main reason is simply that nobody has implemented the functionality. However, there might be some ideological opposition as well. This query is impossible to implement as efficiently on the server as it is on the client. It shouldn’t be too tricky to implement. If you’re interested in doing so, perhaps others can chime in with their views on its acceptability. I personally don’t see an issue with it, though it would be nice if we could submit warnings back with query responses that the query is inefficient - but that’s a separate feature request. In an ideal world, we would support an approach where a token-aware client could take a list parameter, then group the list by token and submit separate queries to the server. It might be that we would need to provide extra information to the client on preparing such a query. This might be a burden on clients, but would offer an efficient option without loss of ergonomics to the user, while also supporting the inefficient option where the client driver does not currently support the feature. Note, none of this would probably qualify for any community investment on review or integration until 4.0 is released. > On 22 Mar 2019, at 18:21, Abhishek Maloo wrote: > > Hello All, > I am trying to migrate some thrift "multiget(multiple partitions)" > operations to CQL. > My Schema is - > CREATE TABLE table1 (key1 int, key2 int, col1 int, val int,* primary > key((key1, key2), col1)*) > It has a *compound partition key* - (k1,k2) > > While converting the multiget I came up with this query - > *select * from table1 where (k1, k2) IN ((1,1), (2,2));* > > Cassandra threw an exception - *"InvalidRequest: Error from server: > code=2200 [Invalid query] message="Multi-column relations can only be > applied to clustering columns but was applied to: key1"* > > What is the rationale behind not supporting Multi-column relation for > partition keys only. I understand that we want to discourage the use of > multiget. > > Apparently below are valid cassandra queries - > *select * from table1 where k1 IN (1,2) and k2 = 1;* > *select * from table1 where k1 = 1 and k2 IN (1,2);* > > Thanks > -Abhishek - To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
Re: TLP tools for stress testing and building test clusters in AWS
Just to confirm, this is on Wednesday? > On 15 Apr 2019, at 22:38, Jon Haddad wrote: > > Hey all, > > I've set up a Zoom call for 9AM Pacific time. Everyone's welcome to join. > > https://zoom.us/j/189920888 > > Looking forward to a good discussion on how we can all pitch in on > getting 4.0 out the door. > > Jon > > On Sat, Apr 13, 2019 at 9:14 AM Jonathan Koppenhofer > wrote: >> >> Wednesday would work for me. >> >> We use and (slightly) contribute to tlp tools. We are platform testing and >> beginning 4.0 testing ourselves, so an in person overview would be great! >> >> On Sat, Apr 13, 2019, 8:48 AM Aleksey Yeshchenko >> wrote: >> >>> Wednesday and Thursday, either, at 9 AM pacific WFM. >>> On 13 Apr 2019, at 13:31, Stefan Miklosovic < >>> stefan.mikloso...@instaclustr.com> wrote: Hi Jon, I would like be on that call too but I am off on Thursday. I am from Australia so 5pm London time is ours 2am next day so your Wednesday morning is my Thursday night. Wednesday early morning so your Tuesday morning and London's afternoon would be the best. Recording the thing would be definitely helpful too. On Sat, 13 Apr 2019 at 07:45, Jon Haddad wrote: > > I'd be more than happy to hop on a call next week to give you both > (and anyone else interested) a tour of our dev tools. Maybe something > early morning on my end, which should be your evening, could work? > > I can set up a Zoom conference to get everyone acquainted. We can > record and post it for any who can't make it. > > I'm thinking Tuesday, Wednesday, or Thursday morning, 9AM Pacific (5pm > London)? If anyone's interested please reply with what dates work. > I'll be sure to post the details back here with the zoom link in case > anyone wants to join that didn't get a chance to reply, as well as a > link to the recorded call. > > Jon > > On Fri, Apr 12, 2019 at 10:41 AM Benedict Elliott Smith > wrote: >> >> +1 >> >> I’m also just as excited to see some standardised workloads and test >>> bed. At the moment we’re benefiting from some large contributors doing >>> their own proprietary performance testing, which is super valuable and >>> something we’ve lacked before. But I’m also keen to see some more >>> representative workloads that are reproducible by anybody in the community >>> take shape. >> >> >>> On 12 Apr 2019, at 18:09, Aleksey Yeshchenko >>> wrote: >>> >>> Hey Jon, >>> >>> This sounds exciting and pretty useful, thanks. >>> >>> Looking forward to using tlp-stress for validating 15066 performance. >>> >>> We should touch base some time next week to pick a comprehensive set >>> of workloads and versions, perhaps? >>> >>> On 12 Apr 2019, at 16:34, Jon Haddad wrote: I don't want to derail the discussion about Stabilizing Internode Messaging, so I'm starting this as a separate thread. There was a comment that Josh made [1] about doing performance testing with real clusters as well as a lot of microbenchmarks, and I'm 100% in support of this. We've been working on some tooling at TLP for the last several months to make this a lot easier. One of the goals has been to help improve the 4.0 testing process. The first tool we have is tlp-stress [2]. It's designed with a "get started in 5 minutes" mindset. My goal was to ship a stress tool >>> that ships with real workloads out of the box that can be easily tweaked, similar to how fio allows you to design a disk workload and tweak it with paramaters. Included are stress workloads that stress LWTs (two different types), materialized views, counters, time series, and key-value workloads. Each workload can be modified easily to change compaction strategies, concurrent operations, number of partitions. We can run workloads for a set number of iterations or a custom duration. We've used this *extensively* at TLP to help our customers and most of our blog posts that discuss performance use it as well. It exports data to both a CSV format and auto sets up prometheus for metrics collection / aggregation. As an example, we were able to determine that the compression length set on the paxos tables imposes a significant overhead when using the Locking LWT workload, which simulates locking and unlocking of rows. See CASSANDRA-15080 for details. We have documentation [3] on the TLP website. The second tool we've been working on is tlp-cluster [4]. This tool is designed to help provision AWS instances for the purposes of testing. To be clear, I don't expect, or want,
Jira Updates
Some exciting news from the Jira changes (maybe).
We’re done! We’ve even achieved the stretch goals.
Does anyone have any further suggestions from the new workflow, after having
tried it out for a bit? Speak up while we have the easy capability to make
changes!
- Somebody has proposed making Change Category a multi-select list, which would
require recreating this. Does anybody have an opinion on this?
- Would anybody prefer that one of the new fields be recreated as a
project-specific select-list, like Platform and Impacts (see below), so we can
manage the options?
Gavin McDonald kindly set us up with two plugins. With the ScriptRunner
plugin, I’ve setup scripted listeners to maintain the Authors field (based on
Assignee) and Priority field (based on updates to Severity), as well as
initialising both of these for all issues. Try it out - it works surprisingly
well, but please let me know ASAP if there are any bugs.
The Authors management may be slightly unintuitive, in that if assignee is
updated we always overwrite Authors to the value of the assignee. We could
maybe do something more sophisticated by checking what the prior value of
Assignee was, but this looked very fiddly. I think in the vast majority of
circumstances this will work as we expect; if there are multiple authors
actively working I expect the assignee not to change; if the assignee is
cleared we probably want to clear the Authors field; and if it is set to a new
value it’s probably the new author. If somebody has suggestions for a better
approach, please chime in.
The other plugin provided us with project-specific select lists - which we’ve
used for Platform and Impacts. These can be modified by any project admin, so
we can maintain them ourselves. Impacts and Platform have a default value of
None and All respectively, to avoid UX ugliness with this multi-select list;
don’t ask me why this makes them prettier, but it does. I had originally
proposed using these values so that we could force the user to specify a value,
but opted to not make the UX ugly, while supporting project ownership of the
options. Opinions on this decision welcome.
For posterity, here are the ScriptRunner scripts I used and setup:
= LISTENER TO SET PRIORITY WHEN SEVERITY UPDATED
import com.atlassian.jira.component.ComponentAccessor
import com.atlassian.jira.event.type.EventDispatchOption
import com.atlassian.jira.issue.MutableIssue
import com.atlassian.jira.util.ImportUtils
import com.atlassian.jira.user.ApplicationUser;
import com.atlassian.jira.issue.util.DefaultIssueChangeHolder
import com.atlassian.jira.issue.util.IssueChangeHolder
import com.atlassian.jira.issue.priority.Priority
import com.atlassian.jira.issue.context.IssueContext
import com.atlassian.jira.issue.customfields.option.Option
import com.atlassian.jira.issue.customfields.option.Options
import com.atlassian.jira.issue.customfields.manager.OptionsManager
def issue = event.issue as MutableIssue
def severityChange = null !=
event?.getChangeLog()?.getRelated("ChildChangeItem")?.find {it.field ==
"Severity"}
if (issue.issueType.name == "Bug" && severityChange) {
def issueManager = ComponentAccessor.getIssueManager()
def customFieldManager = ComponentAccessor.getCustomFieldManager()
def cf = customFieldManager.getCustomFieldObject("customfield_12313820" );
def constantsManager = ComponentAccessor.getConstantsManager()
Priority pUrgent = constantsManager.getPriorities().find {it.name ==
"Urgent"}
Priority pHigh = constantsManager.getPriorities().find {it.name == "High"}
Priority pNormal = constantsManager.getPriorities().find {it.name ==
"Normal"}
Priority pLow = constantsManager.getPriorities().find {it.name == "Low"}
def optManager = ComponentAccessor.getOptionsManager();
Options severities =
optManager.getOptions(cf.getRelevantConfig(IssueContext.GLOBAL))
Option sCritical = severities.getOptionForValue("Critical", null)
Option sNormal = severities.getOptionForValue("Normal", null)
Option sLow = severities.getOptionForValue("Low", null)
def sdUser = ComponentAccessor.getUserManager().getUserByKey('robot')
def priority = issue.getPriority()
def severity = issue.getCustomFieldValue(cf)
if (severity != null) {
def newPriority = priority;
if (severity == sCritical) {
newPriority = pUrgent
} else if (severity == sNormal) {
newPriority = pNormal
} else if (severity == sLow) {
newPriority = pLow
}
if (newPriority != priority) {
issue.setPriority(newPriority)
issueManager.updateIssue(sdUser, issue,
EventDispatchOption.DO_NOT_DISPATCH, false)
}
}
}
= LISTENER TO SET AUTHORS WHEN ASSIGNEE UPDATED
import com.atlassian.jira.issue.Issue
import com.atlassian.jira.issue.ModifiedValue
import com.atlassian.jira.issue.util.DefaultIssueChangeHolder
import
