I've added support for auto-scaling and I've tested that it works well. Please see: https://github.com/Agirish/drill-helm-charts#autoscaling-drill-clusters And I have a script to test this: https://github.com/Agirish/drill-helm-charts/blob/master/scripts/runCPULoadTest.sh
In case of cloud deployments like GKE, where I had a Loadbalancer (and an external IP), I could connect with external clients as well, using the JDBC Drillbit direct connection. On Tue, Mar 24, 2020 at 9:17 AM Arina Ielchiieva <[email protected]> wrote: > Please see https://issues.apache.org/jira/browse/DRILL-7563 < > https://issues.apache.org/jira/browse/DRILL-7563>, maybe it will be > helpful. > > Kind regards, > Arina > > > On Mar 24, 2020, at 6:04 PM, Dobes Vandermeer <[email protected]> wrote: > > > > I was able to get drill up and running inside a k8s cluster but I didn't > connect to it from outside the cluster, so the DNS names were always > resolvable by the client(s). > > > > I had to run it as a statefulset to ensure the DNS names are stable, > otherwise the drillbits couldn't talk to each other, either. > > > > On 3/24/2020 6:37:44 AM, Jaimes, Rafael - 0993 - MITLL < > [email protected]> wrote: > > I’m seeing a problem with scaling the number of pod instances in the > replication controller because they aren’t reporting their hostnames > properly. This was a common problem that got fixed in scalable > architectures like ZooKeeper and Kafka (see reference at bottom I think > this was related). > > > > In Drill’s case, ZooKeeper is able to see all of the drillbits, however, > the hostnames are only locally addressable within the cluster, so as soon > as you perform a query it fails since the client can’t find the drillbit > that it got assigned, its hostname isn’t externally addressable. > > > > Kafka fixes this by allowing an override for advertised names. Has > anyone gotten Drill to scale in a K8s cluster? > > > > https://issues.apache.org/jira/browse/KAFKA-1070 > >
