Hi Maxime, I’d be glad to help with this. Haven’t read the whole thread and I might need a little guidance but seems doable. Is there already an issue for this on GitHub? I’ll check anyway and in case there isn’t I’ll open one.
Best, Gianluca Ciccarelli Data Engineer @ Bolt On 28 May 2019, 21:54 +0300, Maxime Beauchemin <maximebeauche...@gmail.com>, wrote: > @community, it'd be great if someone could volunteer to do this (making > the python library "requests" an optional dependency). It appears it's used > in its simplest form in a few places (requests.get) which should be easy to > replace with urrlib2. The Druid connector uses a more advanced feature > (basic auth), but that could be made just an optional dependency, as we're > deprecating the Druid connector, to be replaced by the SQLAlchemy Druid > connector/dialect. There it's just a matter of moving `pydruid` and > `requests` to the "extra_requires" section of our setup.py, and doing late > imports. > > If not it may take me a moment to get to do this on top of everything else > needed for the ASF official release. > > Max > > On Sat, May 25, 2019 at 12:33 AM Bolke de Bruin <bdbr...@gmail.com> wrote: > > > https://issues.apache.org/jira/browse/LEGAL-192 > > > > Details why. It comes down to that you can use LGPL, but only if it’s an > > optional dependency. This is policy of the ASF in order not to limit > > downstream usage of its own products. Ie. A Non optional dependency would > > require downstream usage of Superset to abide by the LGPL (“reverse > > engineering”) without a way out. > > > > As our use of the requests modules is fairly limited I suggest to > > replicate the functionality that is required. Another option is to fork > > requests and kick out the chardet dependency which is only used in one > > place and probably not relevant to Superset. > > > > Cheers > > Bolke > > > > Verstuurd vanaf mijn iPad > > > > > Op 23 mei 2019 om 01:37 heeft Maxime Beauchemin < > > maximebeauche...@gmail.com> het volgende geschreven: > > > > > > Related, about requests/chardet > > > https://github.com/kennethreitz/requests/issues/3389 > > > > > > For the source code release, do [unbundled] deps need to be cleared? From > > > my understanding we only needed to clear the code we ship. > > > > > > If that's the case we've got work to do on the JS side. > > > > > > Max > > > > > > > On Wed, May 22, 2019 at 4:11 PM Bolke de Bruin <bdbr...@gmail.com> > > wrote: > > > > > > > > Oef chardet is pulled in by requests. The usage of chardet (ie > > triggered) > > > > is unlikely as it is only used when the encoding is not set in headers. > > > > > > > > You could ask the maintainer of chardet to release under another > > license. > > > > This can be tough as their might be several people to contact that need > > to > > > > agree with relicensing. Or do a PR to make the usage of chardet > > optional in > > > > requests. Or use urllib and maybe create a wrapper that mimics requests. > > > > > > > > B. > > > > > > > > > > > > Verstuurd vanaf mijn iPad > > > > > > > > > Op 23 mei 2019 om 00:03 heeft Alan Gates <alanfga...@gmail.com> het > > > > volgende geschreven: > > > > > > > > > > +1 with caveats, see below. I looked at the LICENSE, NOTICE, and > > > > > DISCLAIMER files, checked for any binary files (executables, there's > > > > plenty > > > > > of image files in the distribution), and looked over the licenses of > > the > > > > > dependencies. > > > > > > > > > > More information on the dependencies: > > > > > I found https://pypi.org/project/pip-licenses/ which explains how to > > > > check > > > > > licenses, very useful. > > > > > > > > > > The licenses of modules that will be pulled in when a system is > > compiled > > > > or > > > > > run matter, as the system won't run without them. So it isn't ok to > > > > have a > > > > > GPL licensed library that's necessarily pulled in at compile/runtime, > > as > > > > to > > > > > run the product you'll still be pulling in the GPL which will > > > > > basically > > > > > turn the whole thing GPL. (Optional or contrib components are > > different, > > > > > as users can choose not to run with them if they aren't ok with the > > > > license > > > > > of the optional component.) > > > > > > > > > > Running the above on the modules in setup.py, I see that the vast > > > > majority > > > > > are BSD, MIT, Apache, or PSFL, all of which are fine. The ones that > > > > aren't > > > > > in that category are: > > > > > certifi MPL-2.0: This is ok, as it's binary > > > > > chardet LGPL Not Ok > > > > > click UNKNOWN > > > > > jsonschema UNKNOWN > > > > > python-dateutil Dual License > > > > > python-dotenv UNKNOWN > > > > > python-geohash UNKNOWN > > > > > python3-openid UNKNOWN > > > > > > > > > > The MPL one is fine since it's included in binary form. The unknown > > and > > > > > dual license need some digging to determine what they are. chardet, > > the > > > > > LGPL one, is not ok. > > > > > > > > > > Since this is an incubating release I am still voting +1, with the > > caveat > > > > > that the unknown licenses need to be figured out before the next > > release, > > > > > and the LGPL dependency will have to be removed. Right now I think > > > > getting > > > > > a release out is more important than fixing these issues. > > > > > > > > > > Alan. > > > > > > > > > > On Wed, May 22, 2019 at 2:01 PM Maxime Beauchemin < > > > > > maximebeauche...@gmail.com> wrote: > > > > > > > > > > > Oh actually the commands above just shows the dep tree. > > > > > > > > > > > > For deps in python there's > > > > > > https://github.com/dhatim/python-license-check > > > > > > > > > > > > On the JS side I did some work here to attempt building the LICENSE > > file > > > > > > dynamically as the dep tree evolves > > > > > > https://github.com/apache/incubator-superset/pull/5801 > > > > > > > > > > > > I thought validating the licenses of deps wasn't necessary for > > > > > > source > > > > > > releases though. We may want to start the conversation on > > > > > > convenience > > > > > > releases. To me having solid docker images (or just dockerfiles if > > > > images > > > > > > are troublesome) (that are lean and optimized to build fast) would > > > > > > be > > > > > > ideal, especially if they are used in CI. > > > > > > > > > > > > Max > > > > > > > > > > > > On Wed, May 22, 2019 at 1:52 PM Maxime Beauchemin < > > > > > > maximebeauche...@gmail.com> wrote: > > > > > > > > > > > > > Python: > > > > > > > pip install pipdeptree && pipdeptree > > > > > > > > > > > > > > NPM: > > > > > > > cd superset/assets && npm ls > > > > > > > > > > > > > > > On Wed, May 22, 2019 at 11:09 AM Alan Gates > > > > > > > > <alanfga...@gmail.com> > > > > > > > wrote: > > > > > > > > > > > > > > > Yes, I checked, it works now. I just haven't yet because I'm > > > > > > > > still > > > > > > > > looking > > > > > > > > at all the dependencies it pulls in. Maven makes this super > > > > > > > > easy to > > > > do, > > > > > > > > but I need to learn enough about python setuptools to figure > > > > > > > > out how > > > > to > > > > > > > > check the licenses on those modules. > > > > > > > > > > > > > > > > Alan. > > > > > > > > > > > > > > > > On Wed, May 22, 2019 at 10:56 AM Bolke de Bruin > > > > > > > > <bdbr...@gmail.com> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Is the signature now verifiable? Otherwise it won’t pass the > > > > > > > > > IPMC > > ... > > > > > > > > > > > > > > > > > > Verstuurd vanaf mijn iPad > > > > > > > > > > > > > > > > > > > > Op 22 mei 2019 om 19:26 heeft Maxime Beauchemin < > > > > > > > > > > maximebeauche...@gmail.com> het volgende geschreven: > > > > > > > > > > > > > > > > > > > > Oops, changing thread title this time around > > > > > > > > > > > > > > > > > > > > Vote passes! > > > > > > > > > > > > > > > > > > > > +3 binding votes (Max, Jeff & Abhishek) > > > > > > > > > > +1 non-binding vote (Ville) > > > > > > > > > > > > > > > > > > > > No neutral or negative votes. > > > > > > > > > > > > > > > > > > > > On Tue, May 21, 2019 at 12:31 AM Jeff Feng > > > > > > > > <jeff.f...@airbnb.com.invalid > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > +1 binding > > > > > > > > > > > > > > > > > > > > > > On Mon, May 20, 2019 at 3:54 PM Maxime Beauchemin < > > > > > > > > > > > maximebeauche...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > @Alan, looks like I messed up the signature somehow. I > > > > > > > > > > > > got > > tangled > > > > > > > > into > > > > > > > > > > > > adding a new entry (moving from my gmail to my > > > > > > > > > > > > apache.org > > > > > > address), > > > > > > > > > > > > deleting the old one and my svn kungfu is beyond > > > > > > > > > > > > rusty... > > > > > > > > > > > > > > > > > > > > > > > > Oh I think I just forgot to run "svn commit" (maybe i > > > > > > > > > > > > ran "svn > > > > > > > > update" > > > > > > > > > > > > instead?), so you should just have to import that new > > > > > > > > > > > > KEYS file > > > > > > and > > > > > > > > it > > > > > > > > > > > > should work. > > > > > > > > > > > > > > > > > > > > > > > > Sorry about the confusion. All of this is pretty > > > > > > > > > > > > error-prone, > > > > > > > > > especially > > > > > > > > > > > > the [few] first time[s] around. > > > > > > > > > > > > > > > > > > > > > > > > Max > > > > > > > > > > > > > > > > > > > > > > > > On Mon, May 20, 2019 at 11:29 AM Abhishek Sharma < > > > > > > > > > > > > abhioncbr.apa...@gmail.com> > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > +1 binding. > > > > > > > > > > > > > > > > > > > > > > > > > > Newly built docker image > > > > > > > > > > > > > < > > > > > > > > > > > > https://cloud.docker.com/u/abhioncbr/repository/docker/abhioncbr/docker-superset > > > > > > > > > > > > > working fine. > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > Abhishek > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, May 20, 2019 at 2:03 PM Alan Gates < > > alanfga...@gmail.com > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > Max, when I check the signature (gpg --verify ) it > > > > > > > > > > > > > > tells me: > > > > > > > > > > > > > > gpg: Signature made Sat May 18 15:36:55 2019 PDT > > > > > > > > > > > > > > gpg: using RSA key > > > > > > > > > > > > > 8CA186C4568E92301E5F2491A3B3BE2CCC1BB7E4 > > > > > > > > > > > > > > gpg: Can't check signature: No public key > > > > > > > > > > > > > > > > > > > > > > > > > > > > I imported the KEYS file referenced in your > > > > > > > > > > > > > > message, but it > > > > > > > > doesn't > > > > > > > > > > > > > appear > > > > > > > > > > > > > > to contain that key. I think you need to either > > > > > > > > > > > > > > generate a > > new > > > > > > > > > > > > signature > > > > > > > > > > > > > > with the key in the file and upload that .asc file > > > > > > > > > > > > > > to the dist > > > > > > > > site > > > > > > > > > > > (no > > > > > > > > > > > > > > need to rerole the release itself) or place the key > > > > > > > > > > > > > > you used > > > > > > into > > > > > > > > the > > > > > > > > > > > > > KEYS > > > > > > > > > > > > > > file. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Alan. > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sat, May 18, 2019 at 4:01 PM Maxime Beauchemin < > > > > > > > > > > > > > > maximebeauche...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Dear all, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The source release 0.33.0 RC1 for Apache Superset > > > > > > > > > > > > > > > is baked > > and > > > > > > > > > > > > > available > > > > > > > > > > > > > > > at: > > > > > > > > > > > > > > > https://dist.apache.org/repos/dist/dev/incubator/superset/, > > > > > > > > public > > > > > > > > > > > > > > > keys are available > > > > > > > > > > > > > > > at > > > > > > https://dist.apache.org/repos/dist/release/incubator/superset/KEYS > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > We're now attempting to use 0.33 as the base for > > > > > > > > > > > > > > > the first > > > > > > > > release > > > > > > > > > > > as > > > > > > > > > > > > > > > opposed to 0.32 in previous attempts. Many > > > > > > > > > > > > > > > license-related > > > > > > issues > > > > > > > > > > > had > > > > > > > > > > > > > > been > > > > > > > > > > > > > > > solved by the process shipping visualizations as > > > > > > > > > > > > > > > plugins, and > > > > > > > > that > > > > > > > > > > > > > > > migration wasn't completed on 0.32. This is the > > > > > > > > > > > > > > > third ASF > > > > > > release > > > > > > > > > > > > > > candidate > > > > > > > > > > > > > > > of Superset *We're still ironing out our release > > > > > > > > > > > > > > > process, so > > > > > > > > please > > > > > > > > > > > > > bear > > > > > > > > > > > > > > > with us and help if you can*. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > As I went along, I documented the process in > > [yet-to-be-merged] > > > > > > > > > > > > > > > RELEASING/README.md in the repo, latest edits here > > > > > > > > > > > > > > > https://github.com/apache/incubator-superset/pull/7539 > > > > > > > > > > > > > > > . As > > > > > > part > > > > > > > > > > > of > > > > > > > > > > > > > > > `RELEASING/`, we ship docker files to help > > > > > > > > > > > > > > > package and test > > > > > > > > > > > releases. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > For context the `0.33` release branch was cut at > > > > > > > > > > > > > > > SHA > > 51068f007, > > > > > > > > > > > that > > > > > > > > > > > > > was > > > > > > > > > > > > > > > merged on master on Apr 17th. From that common > > > > > > > > > > > > > > > ancestor, the > > > > > > > > > > > > following > > > > > > > > > > > > > > list > > > > > > > > > > > > > > > of commit was added as cherry-picks. The SHAs in > > > > > > > > > > > > > > > the list > > > > > > bellow > > > > > > > > > > > > > > reference > > > > > > > > > > > > > > > the cherries on the release branch, PR number are > > > > > > > > > > > > > > > available > > to > > > > > > > > get > > > > > > > > > > > > more > > > > > > > > > > > > > > > details. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 7591a709 (tag: 0.33.0rc1, apache/release--0.33) > > > > > > > > > > > > > > > 0.33.0rc1 > > > > > > > > > > > > > > > b7ffdb8b Improve instructions > > > > > > > > > > > > > > > 4bbd68c6 Change babytux to open image in birth > > > > > > > > > > > > > > > dashboard > > > > > > > > > > > > > > > eaa679e8 remove unused LICENSE entries > > > > > > > > > > > > > > > 42d50f9d Add Roboto font to LICENSE, remove > > > > > > > > > > > > > > > glyphicons files > > > > > > > > > > > > > > > 5ae2836b Address COPYRIGHT + LICENSE issues > > > > > > > > > > > > > > > ea807f20 [WiP] Improvements related to ASF > > > > > > > > > > > > > > > release process > > > > > > > > > > > > > > > c57ef5dc 0.31.0rc1.dev1 > > > > > > > > > > > > > > > 51068f00 Adding permission for > > > > > > > > > > > > > > > can_only_access_owned_queries > > > > > > > > > > > (#7234) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > > > *Jeff Feng* > > > > > > > > > > > Product Lead > > > > > > > > > > > m: (949)-610-5108 > > > > > > > > > > > twitter: @jtfeng > > > > > > > > > > > >
