David, that plan looks good to me too. Is anyone interested in implementing it (maybe David or Don?). Seems to me like the steps needed would be,
1) Do a PR to implement the Dockerfile changes in the Druid repo (remove MySQL from the Dockerfile that will be used for DockerHub). 2) Work with Infra to set up automated tag builds. Examples - https://jira.apache.org/jira/browse/INFRA-13838, https://jira.apache.org/jira/browse/INFRA-12781. 3) Potentially add another Dockerfile FROM the autobuilt image that adds MySQL and that interested users could run on their own. On Tue, Mar 5, 2019 at 11:19 AM Charles Allen <[email protected]> wrote: > I would support that. > > On Tue, Mar 5, 2019 at 11:13 AM David Glasser <[email protected]> > wrote: > > > Is the following a reasonable solution from both a usability and > > legal perspective? > > - Write a Dockerfile that has everything except the GPL jar on it > > (including the Druid code that talks to the jar if configured to use > MySQL) > > - Automatically publish that Docker image to the ASF account on DockerHub > > - Also include a short Dockerfile in the repo that starts `FROM` our > > auto-built account and has a line or two of wget to download the jar > > (similar to the wget currently in the Dockerfile) > > - Tell users who want to use MySQL that they must publish that extra > > layered image themselves > > > > --dave > > > > > > On Tue, Mar 5, 2019 at 9:59 AM Charles Allen <[email protected] > > .invalid> > > wrote: > > > > > Honestly we're at a very strange impasse. On one hand I don't think the > > ASF > > > project can adopt an official docker image unless ASF legal says its > ok. > > > "Official" releases are source code anyways (as my understanding goes), > > and > > > binary artifacts are convenience things. Unfortunately I do not see a > > path > > > forward unless some entity is willing to take on a stance similar as > > > outlined in > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_LEGAL-2D437&d=DwIFaQ&c=ncDTmphkJTvjIDPh0hpF_w&r=HrLGT1qWNhseJBMYABL0GFSZESht5gBoLejor3SqMSo&m=x7yglOWgfT8oqHqcgn_BnihcsuEZ-gE5QzWksV1PCDg&s=XPfeziOkU4SCezu5EyBUj-pEGcxOY7XVHU1rdNobwrw&e= > > . This is > > > pretty new territory from a legal perspective (the fact that docker > > images > > > are layers makes it even more interesting). > > > > > > At this point I think the safest thing to do is something that is "no > > more > > > GPL dependent than other containers in the apache repo", which would > mean > > > not adding in GPL binaries. Which means switching to postgres. I don't > > > foresee an aggressive legal stance on this issue, meaning it might > take a > > > while as people watch where the industry is going. > > > > > > > > > > > > On Tue, Mar 5, 2019 at 8:20 AM Don Bowman <[email protected]> wrote: > > > > > > > where do we stand on this? > > > > the PR is in and accepted, but i feel we need to have this built as > > part > > > of > > > > the release artifacts and on dockerhub to foster adoption. > > > > if the only issue is the mysql connector i can remove it in favour of > > the > > > > postgres connector. > > > > > > > > > > > > On Mon, 18 Feb 2019 at 13:58, Don Bowman <[email protected]> wrote: > > > > > > > > > i can just remove the mysql, the postgres works, i was just > assuming > > > > folks > > > > > wanted it. > > > > > > > > > > > > > > > On Mon, 18 Feb 2019 at 16:58, Gian Merlino <[email protected]> > wrote: > > > > > > > > > >> A discussion is progressing on > > > > >> > > > > > > > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_LEGAL-2D437&d=DwIFaQ&c=ncDTmphkJTvjIDPh0hpF_w&r=HrLGT1qWNhseJBMYABL0GFSZESht5gBoLejor3SqMSo&m=SDcL2cv8y5vfiK64aTakmAF8xiMVateJ6QQ3JTsegRI&s=uXRQeP8EnjcHXQmG5oJoTQN2ztfu7N0YCNLpS_aj93g&e= > > > . > > > > It doesn't seem to have > > > > >> got anywhere firm yet. > > > > >> > > > > >> On Fri, Feb 8, 2019 at 12:23 PM Gian Merlino <[email protected]> > > wrote: > > > > >> > > > > >> > I don't think anything is strictly needed from you at this > point, > > > but > > > > >> > things happen when people drive them, and participation in that > > > effort > > > > >> > would help make sure it gets done. I think at this point the > tasks > > > on > > > > >> our > > > > >> > end are watching LEGAL-437 for advice (or making it moot by > > removing > > > > the > > > > >> > MySQL jar), asking Infra to set up automated builds once that is > > > > sorted > > > > >> > out, and building some kind of consensus around how we'll label > > and > > > > >> promote > > > > >> > the Docker images. > > > > >> > > > > > >> > On Fri, Feb 8, 2019 at 12:13 PM Don Bowman <[email protected]> > > > wrote: > > > > >> > > > > > >> >> i'd be fine w/ removing the mysql, i'm using postgresql for the > > > > >> metadata. > > > > >> >> if this is the case we should consider relfecting postgres as > the > > > > >> default > > > > >> >> metadata in the docs. > > > > >> >> however, i think this is mere aggregation under the gpl > license, > > > and > > > > >> the > > > > >> >> docker image tends to have other (e.g. bash) gpl code. druid's > > > start > > > > >> >> scripts are all bash-specific as an example. > > > > >> >> > > > > >> >> I'm not clear if anything further is needed of me, i'm hoping > to > > > get > > > > an > > > > >> >> automated build going into dockerhub, and tagged w/ each > > release. i > > > > >> think > > > > >> >> this will help adoption. > > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> On Fri, 8 Feb 2019 at 14:22, Gian Merlino <[email protected]> > > wrote: > > > > >> >> > > > > >> >> > First off thanks a lot for your work here Don!! > > > > >> >> > > > > > >> >> > I really do think, though, that we need to be careful about > the > > > > >> >> inclusion > > > > >> >> > of the MySQL connector jar. ASF legal has been clear in the > > past > > > > that > > > > >> >> ASF > > > > >> >> > projects should not distribute it as part of binary > convenience > > > > >> >> releases: > > > > >> >> > > > > > > > > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_LEGAL-2D200&d=DwIFaQ&c=ncDTmphkJTvjIDPh0hpF_w&r=HrLGT1qWNhseJBMYABL0GFSZESht5gBoLejor3SqMSo&m=SDcL2cv8y5vfiK64aTakmAF8xiMVateJ6QQ3JTsegRI&s=tzsmBm2IIaa5BS9lQrTc9e0GDt09RmMiI4gfn9CoHT4&e= > > > . > > > > I think having the > > > > >> >> > Dockerfile in the repo is probably fine: in that case we are > > not > > > > >> >> > distributing the jar itself, just, essentially, a pointer to > > how > > > to > > > > >> >> > download it. But if we start offering a prebuilt Docker > image, > > it > > > > is > > > > >> >> less > > > > >> >> > clear to me if that is fine or not. In the interests of > > resolving > > > > >> this > > > > >> >> > question one way or the other, I opened a question asking > about > > > > this > > > > >> >> > specific situation: > > > > > > > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_LEGAL-2D437&d=DwIFaQ&c=ncDTmphkJTvjIDPh0hpF_w&r=HrLGT1qWNhseJBMYABL0GFSZESht5gBoLejor3SqMSo&m=SDcL2cv8y5vfiK64aTakmAF8xiMVateJ6QQ3JTsegRI&s=uXRQeP8EnjcHXQmG5oJoTQN2ztfu7N0YCNLpS_aj93g&e= > > > > . > > > > >> >> > > > > > >> >> > About Dylan's questions: my feeling is that we should go > ahead > > > and > > > > >> >> enable > > > > >> >> > automated pushes to Docker Hub, and provide some appropriate > > > > language > > > > >> >> > around what people should expect out of it. I don't think > > > > >> >> 'experimental' is > > > > >> >> > the right word, but we should be clear around exactly what > > > contract > > > > >> we > > > > >> >> are > > > > >> >> > adhering to. Is it something people can expect to be > published > > > with > > > > >> each > > > > >> >> > release? Is it something that we are going to build and test > as > > > > part > > > > >> of > > > > >> >> the > > > > >> >> > release process, or are we going to publish it via automation > > > > without > > > > >> >> any > > > > >> >> > testing? Is it something we expect people to use in > production, > > > or > > > > >> >> > something we only expect people to use for evaluation? If it > is > > > > >> >> something > > > > >> >> > we expect people to use in production, do we expect them to > use > > > it > > > > in > > > > >> >> any > > > > >> >> > particular way? Will we be guaranteeing certain things (file > > > > layout, > > > > >> >> etc) > > > > >> >> > that provide a stable interface for people to build derived > > > images > > > > >> from? > > > > >> >> > > > > > >> >> > The path of least resistance to answering these questions is > to > > > say > > > > >> that > > > > >> >> > the Docker image is provided in the hopes that it is useful, > > but > > > > that > > > > >> >> it is > > > > >> >> > done via an automated build, without any pre-release testing, > > and > > > > >> >> without > > > > >> >> > any particular guarantees about the 'interface' it provides. > If > > > > this > > > > >> is > > > > >> >> the > > > > >> >> > case then I would suggest putting it up on Docker Hub with an > > > > >> >> appropriate > > > > >> >> > disclaimer and not promoting it too much. (We might very well > > end > > > > up > > > > >> >> > pushing images every once in a while that don't work right, > and > > > it > > > > >> would > > > > >> >> > reflect poorly on the project to have those be prominently > > > > >> linked-to.) > > > > >> >> It > > > > >> >> > becomes easier to strengthen these guarantees if we add an > > > > automated > > > > >> >> test > > > > >> >> > suite that we can run before releases which verifies > > > functionality > > > > >> and > > > > >> >> > interface adherence. > > > > >> >> > > > > > >> >> > On Fri, Feb 8, 2019 at 7:14 AM Rajiv Mordani > > > > >> >> <[email protected]> > > > > >> >> > wrote: > > > > >> >> > > > > > >> >> > > This is purely a packaging exercise. I don't see a reason > to > > > mark > > > > >> >> this as > > > > >> >> > > experimental. > > > > >> >> > > > > > > >> >> > > Rajiv. > > > > >> >> > > ________________________________ > > > > >> >> > > From: Dylan Wylie <[email protected]> > > > > >> >> > > Sent: Friday, February 8, 2019 6:08:47 AM > > > > >> >> > > To: [email protected] > > > > >> >> > > Subject: Re: docker build > > > > >> >> > > > > > > >> >> > > I believe all we have to do is submit a ticket to Apache's > > > > >> >> Infrastructure > > > > >> >> > > team and then we'll have some automatic process that'll > > > > >> automatically > > > > >> >> > > update docker-hub with images relating to each release. > > > > >> >> > > > > > > >> >> > > I guess there's two open questions I think we should reach > a > > > > >> >> consensus on > > > > >> >> > > (others feel free to add more!). > > > > >> >> > > > > > > >> >> > > - Are we as a community happy to "support" an additional > > > release > > > > >> >> > artefact? > > > > >> >> > > I'm happy to try to incorporate this into my employer's > > testing > > > > >> >> > > infrastructure to help catch any regressions on future > > releases > > > > but > > > > >> >> > that's > > > > >> >> > > just one data point on each release. > > > > >> >> > > > > > > >> >> > > - Along the same vein, do we follow the same process as we > do > > > > with > > > > >> new > > > > >> >> > > features and mark this as experimental for some time? > > > > >> >> > > > > > > >> >> > > On Fri, 8 Feb 2019 at 13:25, Don Bowman <[email protected]> > > > > wrote: > > > > >> >> > > > > > > >> >> > > > Now that > > > > >> >> > > > > > > >> >> > > > > > >> >> > > > > >> > > > > > > > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__na01.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Fgithub.com-252Fapache-252Fincubator-2Ddruid-252Fpull-252F6896-26amp-3Bdata-3D02-257C01-257Crmordani-2540vmware.com-257C942b2af1dfb740fcbed308d68dcef937-257Cb39138ca3cee4b4aa4d6cd83d9dd62f0-257C0-257C1-257C636852317419449405-26amp-3Bsdata-3DEXigZIBkKiatM0rEgyQRoxA9ER8u8amiAfPN0MghzjE-253D-26amp-3Breserved-3D0&d=DwIFaQ&c=ncDTmphkJTvjIDPh0hpF_w&r=HrLGT1qWNhseJBMYABL0GFSZESht5gBoLejor3SqMSo&m=SDcL2cv8y5vfiK64aTakmAF8xiMVateJ6QQ3JTsegRI&s=I0Jmt-SqSpozqPWg-3MxWry3mQC_oFP2v9FhGUTL5Ls&e= > > > > >> >> > > is merged > > > > >> >> > > > (thank you!) > > > > >> >> > > > > > > > >> >> > > > who can get this set to build into Dockerhub? Presumably > > > > >> >> automatically > > > > >> >> > > on a > > > > >> >> > > > 'tag' of the repo. > > > > >> >> > > > > > > > >> >> > > > Once that is done it is much more convenient for folks to > > use > > > > >> this > > > > >> >> > tool. > > > > >> >> > > > > > > > >> >> > > > --don > > > > >> >> > > > > > > > >> >> > > > > > > >> >> > > > > > >> >> > > > > >> > > > > > >> > > > > > > > > > > > > > > >
