Hi Stefan. Sorry for the late reply. Just wanted to close the loop on this.
Glad you found the demo useful. On related node, we found out what the issues was when we were going through the demo. It was to do with one of the AZs in our account being unavailable. Even though AWS says an AZ exists, it may still be unavailable for use. When provisioning a cluster, you might get an error like this in the log: aws_instance.cassandra.3: Error launching source instance: Unsupported: > Your requested instance type (r3.2xlarge) is not supported in your > requested Availability Zone (us-west-2d). Please retry your request by not > specifying an Availability Zone or choosing us-west-2c, us-west-2a, > us-west-2b. To address this, we've added an --azs flag which can be used as a workaround when an AZ unavailable. You can use it by supplying the letter of the AZ you want. For example --azs abc. Regards, Anthony On Wed, 17 Apr 2019 at 15:49, Stefan Miklosovic < stefan.mikloso...@instaclustr.com> wrote: > Thanks Anthony for going that proverbial extra mile to cover people in > different time zones too. > > I believe other people will find your talk as helpful as we did. > > Regards > > On Wed, 17 Apr 2019 at 10:08, Anthony Grasso <anthony.gra...@gmail.com> > wrote: > > > > Hi Stefan and devs, > > > > I have set up a zoom link for the TLP tool set intro that will be on in > an > > hours time (17 April 2019 @ 11:00AM AEST): https://zoom.us/j/272648772 > > > > This link is open so if anyone else wishes to join they are welcome to do > > so. I will be covering the same topics Jon is covering in his meeting > > tomorrow. > > > > Regards, > > Anthony > > > > > > On Wed, 17 Apr 2019 at 08:29, Anthony Grasso <anthony.gra...@gmail.com> > > wrote: > > > > > Hi Stefan, > > > > > > Thanks for sending the invite out! > > > > > > Just wondering what do you think of the idea of having a Zoom meeting > that > > > anyone can join? This way anyone else interested can join us as well. > I can > > > set that up if you like? > > > > > > Cheers, > > > Anthony > > > > > > On Tue, 16 Apr 2019 at 21:24, Stefan Miklosovic < > > > stefan.mikloso...@instaclustr.com> wrote: > > > > > >> Hi Anthony, > > >> > > >> sounds good. I ve sent you Hangouts meeting invitation privately. > > >> > > >> Regards > > >> > > >> On Tue, 16 Apr 2019 at 14:53, Anthony Grasso < > anthony.gra...@gmail.com> > > >> wrote: > > >> > > > >> > Hi Stefan, > > >> > > > >> > I have been working with Jon on developing the tool set. I can do a > Zoom > > >> > call tomorrow (Wednesday) at 11am AEST if that works for you? We > can go > > >> > through all the same information that Jon is going to go through in > his > > >> > call. Note that I am in the same timezone as you, so if tomorrow > > >> morning is > > >> > no good we can always do the afternoon. > > >> > > > >> > Cheers, > > >> > Anthony > > >> > > > >> > > > >> > On Sat, 13 Apr 2019 at 22:38, Stefan Miklosovic < > > >> > stefan.mikloso...@instaclustr.com> wrote: > > >> > > > >> > > Hi Jon, > > >> > > > > >> > > I would like be on that call too but I am off on Thursday. > > >> > > > > >> > > I am from Australia so 5pm London time is ours 2am next day so > your > > >> > > Wednesday morning is my Thursday night. Wednesday early morning so > > >> > > your Tuesday morning and London's afternoon would be the best. > > >> > > > > >> > > Recording the thing would be definitely helpful too. > > >> > > > > >> > > On Sat, 13 Apr 2019 at 07:45, Jon Haddad <j...@jonhaddad.com> > wrote: > > >> > > > > > >> > > > I'd be more than happy to hop on a call next week to give you > both > > >> > > > (and anyone else interested) a tour of our dev tools. Maybe > > >> something > > >> > > > early morning on my end, which should be your evening, could > work? > > >> > > > > > >> > > > I can set up a Zoom conference to get everyone acquainted. We > can > > >> > > > record and post it for any who can't make it. > > >> > > > > > >> > > > I'm thinking Tuesday, Wednesday, or Thursday morning, 9AM > Pacific > > >> (5pm > > >> > > > London)? If anyone's interested please reply with what dates > work. > > >> > > > I'll be sure to post the details back here with the zoom link in > > >> case > > >> > > > anyone wants to join that didn't get a chance to reply, as well > as a > > >> > > > link to the recorded call. > > >> > > > > > >> > > > Jon > > >> > > > > > >> > > > On Fri, Apr 12, 2019 at 10:41 AM Benedict Elliott Smith > > >> > > > <bened...@apache.org> wrote: > > >> > > > > > > >> > > > > +1 > > >> > > > > > > >> > > > > I’m also just as excited to see some standardised workloads > and > > >> test > > >> > > bed. At the moment we’re benefiting from some large contributors > > >> doing > > >> > > their own proprietary performance testing, which is super > valuable and > > >> > > something we’ve lacked before. But I’m also keen to see some more > > >> > > representative workloads that are reproducible by anybody in the > > >> community > > >> > > take shape. > > >> > > > > > > >> > > > > > > >> > > > > > On 12 Apr 2019, at 18:09, Aleksey Yeshchenko > > >> > > <alek...@apple.com.INVALID> wrote: > > >> > > > > > > > >> > > > > > Hey Jon, > > >> > > > > > > > >> > > > > > This sounds exciting and pretty useful, thanks. > > >> > > > > > > > >> > > > > > Looking forward to using tlp-stress for validating 15066 > > >> performance. > > >> > > > > > > > >> > > > > > We should touch base some time next week to pick a > > >> comprehensive set > > >> > > of workloads and versions, perhaps? > > >> > > > > > > > >> > > > > > > > >> > > > > >> On 12 Apr 2019, at 16:34, Jon Haddad <j...@jonhaddad.com> > > >> wrote: > > >> > > > > >> > > >> > > > > >> I don't want to derail the discussion about Stabilizing > > >> Internode > > >> > > > > >> Messaging, so I'm starting this as a separate thread. > There > > >> was a > > >> > > > > >> comment that Josh made [1] about doing performance testing > > >> with real > > >> > > > > >> clusters as well as a lot of microbenchmarks, and I'm 100% > in > > >> > > support > > >> > > > > >> of this. We've been working on some tooling at TLP for the > > >> last > > >> > > > > >> several months to make this a lot easier. One of the goals > > >> has been > > >> > > > > >> to help improve the 4.0 testing process. > > >> > > > > >> > > >> > > > > >> The first tool we have is tlp-stress [2]. It's designed > with > > >> a "get > > >> > > > > >> started in 5 minutes" mindset. My goal was to ship a > stress > > >> tool > > >> > > that > > >> > > > > >> ships with real workloads out of the box that can be easily > > >> tweaked, > > >> > > > > >> similar to how fio allows you to design a disk workload and > > >> tweak it > > >> > > > > >> with paramaters. Included are stress workloads that stress > > >> LWTs > > >> > > (two > > >> > > > > >> different types), materialized views, counters, time > series, > > >> and > > >> > > > > >> key-value workloads. Each workload can be modified easily > to > > >> change > > >> > > > > >> compaction strategies, concurrent operations, number of > > >> partitions. > > >> > > > > >> We can run workloads for a set number of iterations or a > custom > > >> > > > > >> duration. We've used this *extensively* at TLP to help our > > >> > > customers > > >> > > > > >> and most of our blog posts that discuss performance use it > as > > >> well. > > >> > > > > >> It exports data to both a CSV format and auto sets up > > >> prometheus for > > >> > > > > >> metrics collection / aggregation. As an example, we were > able > > >> to > > >> > > > > >> determine that the compression length set on the paxos > tables > > >> > > imposes > > >> > > > > >> a significant overhead when using the Locking LWT workload, > > >> which > > >> > > > > >> simulates locking and unlocking of rows. See > CASSANDRA-15080 > > >> for > > >> > > > > >> details. > > >> > > > > >> > > >> > > > > >> We have documentation [3] on the TLP website. > > >> > > > > >> > > >> > > > > >> The second tool we've been working on is tlp-cluster [4]. > > >> This tool > > >> > > > > >> is designed to help provision AWS instances for the > purposes of > > >> > > > > >> testing. To be clear, I don't expect, or want, this tool > to > > >> be used > > >> > > > > >> for production environments. It's designed to assist with > the > > >> > > > > >> Cassandra build process by generating deb packages or > re-using > > >> the > > >> > > > > >> ones that have already been uploaded. Here's a short list > of > > >> the > > >> > > > > >> things you'll care about: > > >> > > > > >> > > >> > > > > >> 1. Create instances in AWS for Cassandra using any instance > > >> size and > > >> > > > > >> number of nodes. Also create tlp-stress instances and a > box > > >> for > > >> > > > > >> monitoring > > >> > > > > >> 2. Use any available build of Cassandra, with a quick > option to > > >> > > change > > >> > > > > >> YAML config. For example: tlp-stress use 3.11.4 -c > > >> > > > > >> concurrent_writes:256 > > >> > > > > >> 3. Do custom builds just by pointing to a local Cassandra > git > > >> repo. > > >> > > > > >> They can be used the same way as #2. > > >> > > > > >> 4. tlp-stress is automatically installed on the stress box. > > >> > > > > >> 5. Everything's installed with pure bash. I considered > > >> something > > >> > > more > > >> > > > > >> complex, but since this is for development only, it turns > out > > >> the > > >> > > > > >> simplest tool possible works well and it means it's easily > > >> > > > > >> configurable. Just drop in your own bash script starting > with > > >> a > > >> > > > > >> number in a XX_script_name.sh format and it gets run. > > >> > > > > >> 6. The monitoring box is running Prometheus. It auto > scrapes > > >> > > > > >> Cassandra using the Instaclustr metrics library. > > >> > > > > >> 7. Grafana is also installed automatically. There's a > couple > > >> sample > > >> > > > > >> graphs there now. We plan on having better default graphs > > >> soon. > > >> > > > > >> > > >> > > > > >> For the moment it installs java 8 only but that should be > > >> easily > > >> > > > > >> fixable to use java 11 to test ZGC (it's on my radar). > > >> > > > > >> > > >> > > > > >> Documentation for tlp-cluster is here [5]. > > >> > > > > >> > > >> > > > > >> There's still some things to work out in the tool, and > we've > > >> been > > >> > > > > >> working hard to smooth out the rough edges. I still > haven't > > >> > > announced > > >> > > > > >> anything WRT tlp-cluster on the TLP blog, because I don't > > >> think it's > > >> > > > > >> quite ready for public consumption, but I think the folks > on > > >> this > > >> > > list > > >> > > > > >> are smart enough to see the value in it even if it has a > few > > >> warts > > >> > > > > >> still. > > >> > > > > >> > > >> > > > > >> I don't consider myself familiar enough with the networking > > >> patch to > > >> > > > > >> give it a full review, but I am qualified to build tools to > > >> help > > >> > > test > > >> > > > > >> it and go through the testing process myself. From what I > can > > >> tell > > >> > > > > >> the patch is moving the codebase in a positive direction > and > > >> I'd > > >> > > like > > >> > > > > >> to help build confidence in it so we can get it merged in. > > >> > > > > >> > > >> > > > > >> We'll continue to build out and improve the tooling with > the > > >> goal of > > >> > > > > >> making it easier for people to jump into the QA side of > things. > > >> > > > > >> > > >> > > > > >> Jon > > >> > > > > >> > > >> > > > > >> [1] > > >> > > > > >> > https://lists.apache.org/thread.html/742009c8a77999f4b62062509f087b670275f827d0c1895bf839eece@%3Cdev.cassandra.apache.org%3E > > >> > > > > >> [2] https://github.com/thelastpickle/tlp-stress > > >> > > > > >> [3] http://thelastpickle.com/tlp-stress/ > > >> > > > > >> [4] https://github.com/thelastpickle/tlp-cluster > > >> > > > > >> [5] http://thelastpickle.com/tlp-cluster/ > > >> > > > > >> > > >> > > > > >> > > >> > > > --------------------------------------------------------------------- > > >> > > > > >> To unsubscribe, e-mail: > dev-unsubscr...@cassandra.apache.org > > >> > > > > >> For additional commands, e-mail: > dev-h...@cassandra.apache.org > > >> > > > > >> > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> --------------------------------------------------------------------- > > >> > > > > > To unsubscribe, e-mail: > dev-unsubscr...@cassandra.apache.org > > >> > > > > > For additional commands, e-mail: > dev-h...@cassandra.apache.org > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > > >> --------------------------------------------------------------------- > > >> > > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > > >> > > > For additional commands, e-mail: dev-h...@cassandra.apache.org > > >> > > > > > >> > > > > >> > > > --------------------------------------------------------------------- > > >> > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > > >> > > For additional commands, e-mail: dev-h...@cassandra.apache.org > > >> > > > > >> > > > > >> > > >> --------------------------------------------------------------------- > > >> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > > >> For additional commands, e-mail: dev-h...@cassandra.apache.org > > >> > > >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > >
