Bug#829046: pagure dependencies (JavaScript libraries) packaged
On Sat, 24 Dec 2016 16:07:05 -0500 Sergio Durigan Juniorwrote: > OK, managed to reproduce it here. Bleach issue is fixed now, but I get 5 failures and 3 errors. Attaching log.== ERROR: test_group_user_delete (tests.test_pagure_flask_ui_groups.PagureFlaskGroupstests) Test the group_user_delete endpoint. -- Traceback (most recent call last): File "tests/test_pagure_flask_ui_groups.py", line 425, in test_group_user_delete output = self.app.get('/new/') File "/usr/lib/python2.7/dist-packages/werkzeug/test.py", line 778, in get return self.open(*args, **kw) File "/usr/lib/python2.7/dist-packages/flask/testing.py", line 127, in open follow_redirects=follow_redirects) File "/usr/lib/python2.7/dist-packages/werkzeug/test.py", line 751, in open response = self.run_wsgi_app(environ, buffered=buffered) File "/usr/lib/python2.7/dist-packages/werkzeug/test.py", line 668, in run_wsgi_app rv = run_wsgi_app(self.application, environ, buffered=buffered) File "/usr/lib/python2.7/dist-packages/werkzeug/test.py", line 871, in run_wsgi_app app_rv = app(environ, start_response) File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1994, in __call__ return self.wsgi_app(environ, start_response) File "/home/pravi/forge/debian/git/pagure-new/.pybuild/pythonX.Y_2.7/build/tests/../pagure/proxy.py", line 71, in __call__ return self.app(environ, start_response) File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1985, in wsgi_app response = self.handle_exception(e) File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1540, in handle_exception reraise(exc_type, exc_value, tb) File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1982, in wsgi_app response = self.full_dispatch_request() File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1614, in full_dispatch_request rv = self.handle_user_exception(e) File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1517, in handle_user_exception reraise(exc_type, exc_value, tb) File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1612, in full_dispatch_request rv = self.dispatch_request() File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1598, in dispatch_request return self.view_functions[rule.endpoint](**req.view_args) File "/home/pravi/forge/debian/git/pagure-new/.pybuild/pythonX.Y_2.7/build/tests/../pagure/__init__.py", line 328, in decorated_function return function(*args, **kwargs) File "/home/pravi/forge/debian/git/pagure-new/.pybuild/pythonX.Y_2.7/build/tests/../pagure/ui/app.py", line 371, in new_project user = pagure.lib.search_user(SESSION, username=flask.g.fas_user.username) File "/home/pravi/forge/debian/git/pagure-new/.pybuild/pythonX.Y_2.7/build/tests/../pagure/lib/__init__.py", line 180, in search_user output = query.first() File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2690, in first ret = list(self[0:1]) File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2482, in __getitem__ return list(res) File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2790, in __iter__ return self._execute_and_instances(context) File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2813, in _execute_and_instances result = conn.execute(querycontext.statement, self._params) File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 945, in execute return meth(self, multiparams, params) File "/usr/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection return connection._execute_clauseelement(self, multiparams, params) File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement compiled_sql, distilled_params File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context context) File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1393, in _handle_dbapi_exception exc_info File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context context) File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 470, in do_execute cursor.execute(statement, parameters) OperationalError: (pysqlite2.dbapi2.OperationalError) no such table: users [SQL: u'SELECT users.id AS users_id, users.user AS users_user, users.fullname AS users_fullname, users.public_ssh_key AS users_public_ssh_key, users.default_email AS users_default_email, users.password AS users_password, users.token AS
Bug#829046: pagure dependencies (JavaScript libraries) packaged
On Saturday, December 24 2016, Pirate Praveen wrote: > On ഞായര് 25 ഡിസംബര് 2016 12:01 രാവിലെ, Sergio Durigan Junior wrote: >> Yep, I did it before building pagure. > > It is failing here on sbuild as well. OK, managed to reproduce it here. -- Sergio GPG key ID: 237A 54B1 0287 28BF 00EF 31F4 D0EB 7628 65FC 5E36 Please send encrypted e-mail if possible http://sergiodj.net/ signature.asc Description: PGP signature
Bug#829046: pagure dependencies (JavaScript libraries) packaged
On ഞായര് 25 ഡിസംബര് 2016 12:01 രാവിലെ, Sergio Durigan Junior wrote: > Yep, I did it before building pagure. It is failing here on sbuild as well. aviau, can you try to build it? signature.asc Description: OpenPGP digital signature
Bug#829046: pagure dependencies (JavaScript libraries) packaged
On Saturday, December 24 2016, Pirate Praveen wrote: > On ശനി 24 ഡിസംബര് 2016 11:58 വൈകു, Sergio Durigan Junior wrote: >> Hm, strange. I'm using pbuilder/cowdancer to build the package, so it >> should catch this. I'll look into it. > > Did you update pbuilder recently? You have to run 'pbuilder update' often. Yep, I did it before building pagure. -- Sergio GPG key ID: 237A 54B1 0287 28BF 00EF 31F4 D0EB 7628 65FC 5E36 Please send encrypted e-mail if possible http://sergiodj.net/ signature.asc Description: PGP signature
Bug#829046: pagure dependencies (JavaScript libraries) packaged
On ശനി 24 ഡിസംബര് 2016 11:58 വൈകു, Sergio Durigan Junior wrote: > Hm, strange. I'm using pbuilder/cowdancer to build the package, so it > should catch this. I'll look into it. Did you update pbuilder recently? You have to run 'pbuilder update' often. signature.asc Description: OpenPGP digital signature
Bug#829046: pagure dependencies (JavaScript libraries) packaged
On Saturday, December 24 2016, Pirate Praveen wrote: > On ശനി 24 ഡിസംബര് 2016 07:26 വൈകു, Sergio Durigan Junior wrote: >> Yep, I forgot :-/. Anyway, pushed now. Sorry about that. > > I'm still getting the same error, > > from html5lib.sanitizer import HTMLSanitizer > ImportError: No module named sanitizer > > I don't see either python-bleach or python-html5lib is updated. > > This bug is still open > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=844943 May be you have > an older version of python-html5lib in your system. We have to fix this > bug to proceed further. Hm, strange. I'm using pbuilder/cowdancer to build the package, so it should catch this. I'll look into it. -- Sergio GPG key ID: 237A 54B1 0287 28BF 00EF 31F4 D0EB 7628 65FC 5E36 Please send encrypted e-mail if possible http://sergiodj.net/ signature.asc Description: PGP signature
Bug#829046: pagure dependencies (JavaScript libraries) packaged
On ശനി 24 ഡിസംബര് 2016 07:26 വൈകു, Sergio Durigan Junior wrote: > Yep, I forgot :-/. Anyway, pushed now. Sorry about that. I'm still getting the same error, from html5lib.sanitizer import HTMLSanitizer ImportError: No module named sanitizer I don't see either python-bleach or python-html5lib is updated. This bug is still open https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=844943 May be you have an older version of python-html5lib in your system. We have to fix this bug to proceed further. signature.asc Description: OpenPGP digital signature
Bug#829046: pagure dependencies (JavaScript libraries) packaged
On Saturday, December 24 2016, Pirate Praveen wrote: > On ശനി 24 ഡിസംബര് 2016 11:37 രാവിലെ, Sergio Durigan Junior wrote: >> Well, I've done some tests here and I think I have a package that >> builds. I've updated pagure to an intermediary version, 2.6, and the >> package builds fine on pbuilder/cowbuilder using the latest unstable. >> You can find it here: >> >> http://git.sergiodj.net/?p=debian/pagure-new.git;a=summary >> >> Can you give it a try? I think we can go with that (even for stretch), >> if you are OK with it. The package contains bundled JS libraries, and >> minifies them during the build, so it's not perfect in this aspect. But >> we can fix this later, when we have more time. > > I tried to build, but debian directory is missing there. May be you > forgot to push new commits? Yep, I forgot :-/. Anyway, pushed now. Sorry about that. -- Sergio GPG key ID: 237A 54B1 0287 28BF 00EF 31F4 D0EB 7628 65FC 5E36 Please send encrypted e-mail if possible http://sergiodj.net/ signature.asc Description: PGP signature
Bug#829046: pagure dependencies (JavaScript libraries) packaged
On ശനി 24 ഡിസംബര് 2016 11:37 രാവിലെ, Sergio Durigan Junior wrote: > Well, I've done some tests here and I think I have a package that > builds. I've updated pagure to an intermediary version, 2.6, and the > package builds fine on pbuilder/cowbuilder using the latest unstable. > You can find it here: > > http://git.sergiodj.net/?p=debian/pagure-new.git;a=summary > > Can you give it a try? I think we can go with that (even for stretch), > if you are OK with it. The package contains bundled JS libraries, and > minifies them during the build, so it's not perfect in this aspect. But > we can fix this later, when we have more time. I tried to build, but debian directory is missing there. May be you forgot to push new commits? signature.asc Description: OpenPGP digital signature
Bug#829046: pagure dependencies (JavaScript libraries) packaged
On ശനി 24 ഡിസംബര് 2016 11:37 രാവിലെ, Sergio Durigan Junior wrote: > Right, that's nice indeed, but it demands work anyway... But it's > really good that grunt and gulp are now available. Exploding dependencies in new releases is not pagure specific issue. Its there for every web app, be it gitlab, diaspora or pagure. We have to keep packaging new dependencies, I don't see another way around it. signature.asc Description: OpenPGP digital signature
Bug#829046: pagure dependencies (JavaScript libraries) packaged
On Saturday, December 24 2016, Pirate Praveen wrote: > On ശനി 24 ഡിസംബര് 2016 10:54 രാവിലെ, Sergio Durigan Junior wrote: >> IMO, if you have time, here's what I would do: >> >> - If you manage to get the building issue sorted out, and if there are >> no other critical issues, then I think the package is "good enough" >> (for a very relaxed definition of "good") to be put on stretch. As >> Praveen said, we will have 2 months to fix remaining bugs. >> >> - If it is not possible to fix the critical bugs, then I'd suggest going >> with the strectch-backports alternative. It's not a bad option IMHO, >> and gives us even more time to really fix the underlying problems. > > Its too close to Dec 25, lets take the backports path. Well, I've done some tests here and I think I have a package that builds. I've updated pagure to an intermediary version, 2.6, and the package builds fine on pbuilder/cowbuilder using the latest unstable. You can find it here: http://git.sergiodj.net/?p=debian/pagure-new.git;a=summary Can you give it a try? I think we can go with that (even for stretch), if you are OK with it. The package contains bundled JS libraries, and minifies them during the build, so it's not perfect in this aspect. But we can fix this later, when we have more time. >> Something really concerning me is the fact that pagure keeps adding JS >> dependencies that are either not packaged on Debian, or that are >> packaged as part of another, bigger package (usually some big ruby >> extension or so). And even in the second case, when the file is already >> in Debian, it is not really following our guidelines; for example, the >> .js and .min.js files are being shipped without being built from >> source. > > We now have grunt and gulp in main (rollup and babel are on their way), > so rebuilding them would be much easier than before (reimplementing the > build in rules). > > The ruby ones are usually just wrappers for the js part. It would be > easy to separate them if needed. ruby packages can just add a symlink > instead of embedded copies when we package the js part separately. Right, that's nice indeed, but it demands work anyway... But it's really good that grunt and gulp are now available. Thanks, -- Sergio GPG key ID: 237A 54B1 0287 28BF 00EF 31F4 D0EB 7628 65FC 5E36 Please send encrypted e-mail if possible http://sergiodj.net/ signature.asc Description: PGP signature
Bug#829046: pagure dependencies (JavaScript libraries) packaged
On ശനി 24 ഡിസംബര് 2016 10:54 രാവിലെ, Sergio Durigan Junior wrote: > IMO, if you have time, here's what I would do: > > - If you manage to get the building issue sorted out, and if there are > no other critical issues, then I think the package is "good enough" > (for a very relaxed definition of "good") to be put on stretch. As > Praveen said, we will have 2 months to fix remaining bugs. > > - If it is not possible to fix the critical bugs, then I'd suggest going > with the strectch-backports alternative. It's not a bad option IMHO, > and gives us even more time to really fix the underlying problems. Its too close to Dec 25, lets take the backports path. > Something really concerning me is the fact that pagure keeps adding JS > dependencies that are either not packaged on Debian, or that are > packaged as part of another, bigger package (usually some big ruby > extension or so). And even in the second case, when the file is already > in Debian, it is not really following our guidelines; for example, the > .js and .min.js files are being shipped without being built from > source. We now have grunt and gulp in main (rollup and babel are on their way), so rebuilding them would be much easier than before (reimplementing the build in rules). The ruby ones are usually just wrappers for the js part. It would be easy to separate them if needed. ruby packages can just add a symlink instead of embedded copies when we package the js part separately. signature.asc Description: OpenPGP digital signature
Bug#829046: pagure dependencies (JavaScript libraries) packaged
On Tuesday, December 20 2016, Pirate Praveen wrote: > On തിങ്കള് 19 ഡിസംബര് 2016 11:07 വൈകു, Sergio Durigan Junior wrote: >> Things are starting to calm down, and I intend to get back to packaging >> pagure. My intention is to get the latest version imported onto my >> repository and work on it. However, I don't think it will be possible >> to finish everything by Dec 25th. I know it would be good to get pagure >> on stretch, but IMHO it is better to make sure everything is working >> fine first. > > Dec 25 is the last day of uploading to sid so it can migrate to stretch > before freeze. We have time till stretch release (at least 2 months > more) to fix bugs. We should try our best to have it in stretch and if > not have it in stretch-backports. Hm, guys, I am not sure I'll be able to get this ready until the 25th. There are still issues with the package and well, tomorrow is christmas eve, so... I'll keep trying to get the package in a good shape. I think it is, but as Praveen mentioned there is still at least one issue when building it. Also, I'm not even considering updating the package to the latest version (2.10.1) now, because that will certainly introduce a lot of new issues. IMO, if you have time, here's what I would do: - If you manage to get the building issue sorted out, and if there are no other critical issues, then I think the package is "good enough" (for a very relaxed definition of "good") to be put on stretch. As Praveen said, we will have 2 months to fix remaining bugs. - If it is not possible to fix the critical bugs, then I'd suggest going with the strectch-backports alternative. It's not a bad option IMHO, and gives us even more time to really fix the underlying problems. Something really concerning me is the fact that pagure keeps adding JS dependencies that are either not packaged on Debian, or that are packaged as part of another, bigger package (usually some big ruby extension or so). And even in the second case, when the file is already in Debian, it is not really following our guidelines; for example, the .js and .min.js files are being shipped without being built from source. Anyway, some food for thought. I'll keep you posted if I manage to make things work on my side. Thanks, -- Sergio GPG key ID: 237A 54B1 0287 28BF 00EF 31F4 D0EB 7628 65FC 5E36 Please send encrypted e-mail if possible http://sergiodj.net/ signature.asc Description: PGP signature
Bug#829046: pagure dependencies (JavaScript libraries) packaged
On തിങ്കള് 19 ഡിസംബര് 2016 11:07 വൈകു, Sergio Durigan Junior wrote: > Things are starting to calm down, and I intend to get back to packaging > pagure. My intention is to get the latest version imported onto my > repository and work on it. However, I don't think it will be possible > to finish everything by Dec 25th. I know it would be good to get pagure > on stretch, but IMHO it is better to make sure everything is working > fine first. Dec 25 is the last day of uploading to sid so it can migrate to stretch before freeze. We have time till stretch release (at least 2 months more) to fix bugs. We should try our best to have it in stretch and if not have it in stretch-backports. signature.asc Description: OpenPGP digital signature
Bug#829046: pagure dependencies (JavaScript libraries) packaged
On Monday, December 19 2016, Pirate Praveen wrote: > On Mon, 12 Dec 2016 06:21:35 +1100 Ben Finneywrote: >> I don't understand Grunt or Gulp, but from what I could tell those >> were just being used for typical build tasks. I do understand Make, so >> I reproduced the build tasks in Debian's ârulesâ makefile :-) > > Btw I just uploaded gulp also to debian (its in NEW). Well, the upstream > provides a Gruntfile.js or Gulpfile.js, so usually you need to only > remove hinting and styling tasks (jshint, jscs etc). For grunt, you also > need to provide /usr/lib/nodejs as path for tasks (if someone helps make > that default for grunt, we can remove that step as well). > >> Please try the library packages (they are currently in âexperimentalâ) >> and let me know whether they are sufficient to allow Pagure packaging >> to continue. > > Since Sergio has been silent for sometime and freeze is approaching fast > (we have only time till Dec 25, less than a week to get pagure into > stretch), I will copy his repo to collab-maint and will try to upload it > to experimental. Hey guys, First of all, thank you very much for all the work you have done these last weeks. I am sorry I was absent and did to participate; I have been very busy with work and unable to concentrate on anything else. Things are starting to calm down, and I intend to get back to packaging pagure. My intention is to get the latest version imported onto my repository and work on it. However, I don't think it will be possible to finish everything by Dec 25th. I know it would be good to get pagure on stretch, but IMHO it is better to make sure everything is working fine first. I'll see if I can continue my work tonight and let you know of any progress. Cheers, -- Sergio GPG key ID: 237A 54B1 0287 28BF 00EF 31F4 D0EB 7628 65FC 5E36 Please send encrypted e-mail if possible http://sergiodj.net/ signature.asc Description: PGP signature
Bug#829046: pagure dependencies (JavaScript libraries) packaged
Control: block -1 by 844943 On തിങ്കള് 19 ഡിസംബര് 2016 05:55 വൈകു, Pirate Praveen wrote: > When I try to build the package, I get the following error > > File "/usr/lib/python2.7/dist-packages/bleach/__init__.py", line 8, in > > from html5lib.sanitizer import HTMLSanitizer > ImportError: No module named sanitizer > > apt-file-list python-html5lib shows > > python-html5lib: > /usr/lib/python2.7/dist-packages/html5lib/filters/sanitizer.py > > So I think we need to patch python-bleach. > This is already reported as #844943 signature.asc Description: OpenPGP digital signature
Bug#829046: pagure dependencies (JavaScript libraries) packaged
[ccing everyone who showed interest] On Mon, 19 Dec 2016 16:35:07 +0530 Pirate Praveenwrote: > Since Sergio has been silent for sometime and freeze is approaching fast > (we have only time till Dec 25, less than a week to get pagure into > stretch), I will copy his repo to collab-maint and will try to upload it > to experimental. > Its now pushed to collab-maint. Since all dependencies are now packaged, we should test it well and try to get it into stretch by Dec 25. When I try to build the package, I get the following error File "/usr/lib/python2.7/dist-packages/bleach/__init__.py", line 8, in from html5lib.sanitizer import HTMLSanitizer ImportError: No module named sanitizer apt-file-list python-html5lib shows python-html5lib: /usr/lib/python2.7/dist-packages/html5lib/filters/sanitizer.py So I think we need to patch python-bleach. signature.asc Description: OpenPGP digital signature
Bug#829046: pagure dependencies (JavaScript libraries) packaged
On Mon, 12 Dec 2016 06:21:35 +1100 Ben Finneywrote: > I don't understand Grunt or Gulp, but from what I could tell those > were just being used for typical build tasks. I do understand Make, so > I reproduced the build tasks in Debian's ârulesâ makefile :-) Btw I just uploaded gulp also to debian (its in NEW). Well, the upstream provides a Gruntfile.js or Gulpfile.js, so usually you need to only remove hinting and styling tasks (jshint, jscs etc). For grunt, you also need to provide /usr/lib/nodejs as path for tasks (if someone helps make that default for grunt, we can remove that step as well). > Please try the library packages (they are currently in âexperimentalâ) > and let me know whether they are sufficient to allow Pagure packaging > to continue. Since Sergio has been silent for sometime and freeze is approaching fast (we have only time till Dec 25, less than a week to get pagure into stretch), I will copy his repo to collab-maint and will try to upload it to experimental. signature.asc Description: OpenPGP digital signature
Bug#829046: pagure dependencies (JavaScript libraries) packaged
On 05-Sep-2016, Sergio Durigan Junior wrote: > […] from my recent experience packaging some JS libs, I fear that > the building issue will not be solved in the near future. I have packaged the two remaining JavaScript libraries that were blocking this RFP, ‘jquery-at.js’ and ‘jquery-caret.js’. I don't understand Grunt or Gulp, but from what I could tell those were just being used for typical build tasks. I do understand Make, so I reproduced the build tasks in Debian's ‘rules’ makefile :-) Please try the library packages (they are currently in ‘experimental’) and let me know whether they are sufficient to allow Pagure packaging to continue. -- \“I knew it was a shocking thing to say, but … no-one has the | `\right to spend their life without being offended.” —Philip | _o__) Pullman, 2010-03-28 | Ben Finneysignature.asc Description: PGP signature