Re: developer.gnome.org and meson

2017-08-13 Thread Nirbheek Chauhan 
On Fri, Aug 11, 2017 at 4:10 PM, Stefan Sauer  wrote:
> On 08/09/2017 10:11 PM, Nirbheek Chauhan wrote:
> > Somewhat relatedly, the only reason why it takes so long to build docs
> > is because we haven't been improving gtk-doc. There is little
> > technical reason why building our documentation has to be *so* slow.
> > For instance, there's Hotdoc which as a proof-of-concept does the same
> > work, but much faster. So perhaps our time is better spent figuring
> > out why gtk-doc eats CPU (single-threaded!) and fixing that.
> It is not a secret why gtk-doc is slow. It is slow because libxslt and
> the xsl-stylesheets are slow. In particular libxslt is single threaded.

Thanks for the clarification, Stefan :)

> In the gtk-doc-1.26 I just release, everything was ported to python.

Yes, I was following the port to Python. It was great! Another step
towards getting rid of Perl from our toolchain. I have a vested
interest in this because it makes bootstrapping builds on, for
example, Windows much easier.

> Next step is to remove file generating code from the xslt pipeline. Then
> we can replace xslt by e.g. a markdown toolchain. I appreciate what was
> done in hotdoc, but I'd like to provide a step by step migration path
> for existing modules. I never understood why what has been done could
> not have been made inside gtk-doc as well.
> I appreciate anyone helping (like jussi helping a lot with porting from
> perl to python).
>

I completely agree, my mention of Hotdoc wasn't an endorsement of it,
but rather to say that there is room for improvement in the time it
takes to build our docs. A gradual migration to a markdown-based
toolchain would do wonders. Do you have any bugs about the work that's
needed so people can dive in if they can?

Cheers,
Nirbheek
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: developer.gnome.org and meson

2017-08-11 Thread Stefan Sauer 
On 08/09/2017 10:11 PM, Nirbheek Chauhan wrote:
> On Thu, Aug 10, 2017 at 1:27 AM, Sébastien Wilmet  wrote:
>> On Wed, Aug 09, 2017 at 03:20:38PM +0100, Emmanuele Bassi wrote:
>>> After all, Linux
>>> distributions rebuild the documentation when building the binary
>>> packages anyway
>> I see that in the gspell-doc package on Fedora 26, some html pages have
>> links to /home/seb/jhbuild/... (a problem with the gtkdoc-fixxref
>> config). That's my home directory :-) so definitely Fedora didn't
>> re-generate the docs (and it would be painfully slow, it takes several
>> hours on my machine to generate the GTK+ docs).
>>
> Somewhat relatedly, the only reason why it takes so long to build docs
> is because we haven't been improving gtk-doc. There is little
> technical reason why building our documentation has to be *so* slow.
> For instance, there's Hotdoc which as a proof-of-concept does the same
> work, but much faster. So perhaps our time is better spent figuring
> out why gtk-doc eats CPU (single-threaded!) and fixing that.
It is not a secret why gtk-doc is slow. It is slow because libxslt and
the xsl-stylesheets are slow. In particular libxslt is single threaded.
In the gtk-doc-1.26 I just release, everything was ported to python.
Next step is to remove file generating code from the xslt pipeline. Then
we can replace xslt by e.g. a markdown toolchain. I appreciate what was
done in hotdoc, but I'd like to provide a step by step migration path
for existing modules. I never understood why what has been done could
not have been made inside gtk-doc as well.
I appreciate anyone helping (like jussi helping a lot with porting from
perl to python).

Stefan

>
> Cheers,
> Nirbheek
> ___
> desktop-devel-list mailing list
> desktop-devel-list@gnome.org
> https://mail.gnome.org/mailman/listinfo/desktop-devel-list


___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: developer.gnome.org and meson

2017-08-09 Thread Sébastien Wilmet 
On Wed, Aug 09, 2017 at 09:03:36PM +0100, Emmanuele Bassi wrote:
> > (and it would be painfully slow, it takes several
> > hours on my machine to generate the GTK+ docs).
> 
> Distributions typically use something slightly more beefy than a
> typical PC hardware for their builds.

They have also slightly more things to build.
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: developer.gnome.org and meson

2017-08-09 Thread Nirbheek Chauhan 
On Thu, Aug 10, 2017 at 1:27 AM, Sébastien Wilmet  wrote:
> On Wed, Aug 09, 2017 at 03:20:38PM +0100, Emmanuele Bassi wrote:
>> After all, Linux
>> distributions rebuild the documentation when building the binary
>> packages anyway
>
> I see that in the gspell-doc package on Fedora 26, some html pages have
> links to /home/seb/jhbuild/... (a problem with the gtkdoc-fixxref
> config). That's my home directory :-) so definitely Fedora didn't
> re-generate the docs (and it would be painfully slow, it takes several
> hours on my machine to generate the GTK+ docs).
>

Somewhat relatedly, the only reason why it takes so long to build docs
is because we haven't been improving gtk-doc. There is little
technical reason why building our documentation has to be *so* slow.
For instance, there's Hotdoc which as a proof-of-concept does the same
work, but much faster. So perhaps our time is better spent figuring
out why gtk-doc eats CPU (single-threaded!) and fixing that.

Cheers,
Nirbheek
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: developer.gnome.org and meson and devhelp

2017-08-09 Thread Sébastien Wilmet 
On Wed, Aug 09, 2017 at 08:33:09AM -0500, mcatanz...@gnome.org wrote:
> developer.gnome.org is going to have some problems because for meson modules
> 'ninja dist' does not include generated gtk-doc files in the tarball. At
> least one maintainer is working around this by manually generating tarballs
> with gtk-doc included instead of using 'ninja dist'. I don't recommend doing
> that since that's equivalent to skipping distcheck. It's better to use
> meson's dist target. developer.gnome.org is just going to have to learn to
> build docs itself.
> 
> Is anybody working on developer.gnome.org? Anyone interested in taking on
> this task? Otherwise it is going to be stuck with outdated docs.

When implementing this, it would be nice to take also into account the
following feature request for Devhelp, to download the latest versions
of the API references directly in Devhelp (something that I would like
to implement in the future, I don't know when):
https://bugzilla.gnome.org/show_bug.cgi?id=761284
"Have the latest stable/unstable GNOME API references"

I think the need for Devhelp is similar to the need for
developer.gnome.org: storing somewhere on gnome.org a list of tarballs
with the docs, alongside maybe an XML/JSON/whatever file listing the
tarballs with some additional info. Then library-web or Devhelp can
download the XML file, then download the new tarballs, andthen do its
thing.

--
Sébastien
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: developer.gnome.org and meson

2017-08-09 Thread Emmanuele Bassi 
On 9 August 2017 at 20:57, Sébastien Wilmet  wrote:
> On Wed, Aug 09, 2017 at 03:20:38PM +0100, Emmanuele Bassi wrote:
>> After all, Linux
>> distributions rebuild the documentation when building the binary
>> packages anyway
>
> I see that in the gspell-doc package on Fedora 26, some html pages have
> links to /home/seb/jhbuild/... (a problem with the gtkdoc-fixxref
> config). That's my home directory :-) so definitely Fedora didn't
> re-generate the docs

That's all kinds of broken — that's why distro re-generate the docs, typically.

> (and it would be painfully slow, it takes several
> hours on my machine to generate the GTK+ docs).

Distributions typically use something slightly more beefy than a
typical PC hardware for their builds.

Ciao,
 Emmanuele.

-- 
https://www.bassi.io
[@] ebassi [@gmail.com]
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: developer.gnome.org and meson

2017-08-09 Thread Sébastien Wilmet 
On Wed, Aug 09, 2017 at 03:20:38PM +0100, Emmanuele Bassi wrote:
> After all, Linux
> distributions rebuild the documentation when building the binary
> packages anyway

I see that in the gspell-doc package on Fedora 26, some html pages have
links to /home/seb/jhbuild/... (a problem with the gtkdoc-fixxref
config). That's my home directory :-) so definitely Fedora didn't
re-generate the docs (and it would be painfully slow, it takes several
hours on my machine to generate the GTK+ docs).

--
Sébastien
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: developer.gnome.org and meson

2017-08-09 Thread Michael Biebl 
I would prefer if building from git is the same as building from a
dist tarball, which means I wouldn't ship any pre-generated files in
the tarballs unless it's absolutely necessary.
I don't consider the additional dependencies for building the docs an
issue, at least for distro builds it isn't.
I guess it can be useful to not require those dependencies during a
bootstrap phase, but simply being able to disable the documentation is
sufficient for that.

2017-08-09 16:20 GMT+02:00 Emmanuele Bassi :
> In the medium-to-long term, I'd really appreciate if
> developer.gnome.org stopped trying to extract documentation from
> random locations inside tarballs, munge the cross-references, and
> published the HTML on a static website. This would avoid having to
> generate documentation at all, except when needed. After all, Linux
> distributions rebuild the documentation when building the binary
> packages anyway, so shipping documentation in release tarballs is
> pretty much for the benefit of developer.gnome.org to begin with.
>
> Ideally, with the switch to Gitlab, we'd be able to run CI targets for
> each module; that would allow us to build the API reference (and any
> other documentation we deem worthy of publishing), ensure that the
> cross-references pointed to a well-known URL prefix as part of the
> build itself, and publish them when pushing a release tag.
>
> Additionally, GitLab pages[0] would ensure that any module with
> documentation would have it published, without necessarily teaching
> developer.gnome.org how to do it.
>
> Ciao,
>  Emmanuele.
>
> [0]: https://about.gitlab.com/features/pages/
>
>
> On 9 August 2017 at 15:12, Bastien Nocera  wrote:
>> On Wed, 2017-08-09 at 08:33 -0500, mcatanz...@gnome.org wrote:
>>> Hi,
>>>
>>> developer.gnome.org is going to have some problems because for meson
>>> modules 'ninja dist' does not include generated gtk-doc files in the
>>> tarball. At least one maintainer is working around this by manually
>>> generating tarballs with gtk-doc included instead of using 'ninja
>>> dist'. I don't recommend doing that since that's equivalent to
>>> skipping
>>> distcheck. It's better to use meson's dist target.
>>> developer.gnome.org
>>> is just going to have to learn to build docs itself.
>>>
>>> Is anybody working on developer.gnome.org? Anyone interested in
>>> taking
>>> on this task? Otherwise it is going to be stuck with outdated docs.
>>
>> I filed this:
>> https://github.com/mesonbuild/meson/issues/2166
>>
>> I don't know whether that's something we'd want longer term, but it's a
>> win short-term.
>>
>> Cheers
>> ___
>> desktop-devel-list mailing list
>> desktop-devel-list@gnome.org
>> https://mail.gnome.org/mailman/listinfo/desktop-devel-list
>
>
>
> --
> https://www.bassi.io
> [@] ebassi [@gmail.com]
> ___
> desktop-devel-list mailing list
> desktop-devel-list@gnome.org
> https://mail.gnome.org/mailman/listinfo/desktop-devel-list



-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: developer.gnome.org and meson

2017-08-09 Thread Emmanuele Bassi 
In the medium-to-long term, I'd really appreciate if
developer.gnome.org stopped trying to extract documentation from
random locations inside tarballs, munge the cross-references, and
published the HTML on a static website. This would avoid having to
generate documentation at all, except when needed. After all, Linux
distributions rebuild the documentation when building the binary
packages anyway, so shipping documentation in release tarballs is
pretty much for the benefit of developer.gnome.org to begin with.

Ideally, with the switch to Gitlab, we'd be able to run CI targets for
each module; that would allow us to build the API reference (and any
other documentation we deem worthy of publishing), ensure that the
cross-references pointed to a well-known URL prefix as part of the
build itself, and publish them when pushing a release tag.

Additionally, GitLab pages[0] would ensure that any module with
documentation would have it published, without necessarily teaching
developer.gnome.org how to do it.

Ciao,
 Emmanuele.

[0]: https://about.gitlab.com/features/pages/


On 9 August 2017 at 15:12, Bastien Nocera  wrote:
> On Wed, 2017-08-09 at 08:33 -0500, mcatanz...@gnome.org wrote:
>> Hi,
>>
>> developer.gnome.org is going to have some problems because for meson
>> modules 'ninja dist' does not include generated gtk-doc files in the
>> tarball. At least one maintainer is working around this by manually
>> generating tarballs with gtk-doc included instead of using 'ninja
>> dist'. I don't recommend doing that since that's equivalent to
>> skipping
>> distcheck. It's better to use meson's dist target.
>> developer.gnome.org
>> is just going to have to learn to build docs itself.
>>
>> Is anybody working on developer.gnome.org? Anyone interested in
>> taking
>> on this task? Otherwise it is going to be stuck with outdated docs.
>
> I filed this:
> https://github.com/mesonbuild/meson/issues/2166
>
> I don't know whether that's something we'd want longer term, but it's a
> win short-term.
>
> Cheers
> ___
> desktop-devel-list mailing list
> desktop-devel-list@gnome.org
> https://mail.gnome.org/mailman/listinfo/desktop-devel-list



-- 
https://www.bassi.io
[@] ebassi [@gmail.com]
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: developer.gnome.org and meson

2017-08-09 Thread Bastien Nocera 
On Wed, 2017-08-09 at 08:33 -0500, mcatanz...@gnome.org wrote:
> Hi,
> 
> developer.gnome.org is going to have some problems because for meson 
> modules 'ninja dist' does not include generated gtk-doc files in the 
> tarball. At least one maintainer is working around this by manually 
> generating tarballs with gtk-doc included instead of using 'ninja 
> dist'. I don't recommend doing that since that's equivalent to
> skipping 
> distcheck. It's better to use meson's dist target.
> developer.gnome.org 
> is just going to have to learn to build docs itself.
> 
> Is anybody working on developer.gnome.org? Anyone interested in
> taking 
> on this task? Otherwise it is going to be stuck with outdated docs.

I filed this:
https://github.com/mesonbuild/meson/issues/2166

I don't know whether that's something we'd want longer term, but it's a
win short-term.

Cheers
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: developer.gnome.org and meson

2017-08-09 Thread Frederic Peters 
mcatanz...@gnome.org wrote:

> developer.gnome.org is going to have some problems because for meson modules
> 'ninja dist' does not include generated gtk-doc files in the tarball. At
> least one maintainer is working around this by manually generating tarballs
> with gtk-doc included instead of using 'ninja dist'. I don't recommend doing
> that since that's equivalent to skipping distcheck. It's better to use
> meson's dist target. developer.gnome.org is just going to have to learn to
> build docs itself.

But that often requires a full build environment, which is the reason
it doesn't do that.  Continuous or whatever CI system should include a
step publishing built documentation files, it will then be "easy" for
developer.gnome.org to use those.



Fred
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

developer.gnome.org and meson

2017-08-09 Thread mcatanzaro 

Hi,

developer.gnome.org is going to have some problems because for meson 
modules 'ninja dist' does not include generated gtk-doc files in the 
tarball. At least one maintainer is working around this by manually 
generating tarballs with gtk-doc included instead of using 'ninja 
dist'. I don't recommend doing that since that's equivalent to skipping 
distcheck. It's better to use meson's dist target. developer.gnome.org 
is just going to have to learn to build docs itself.


Is anybody working on developer.gnome.org? Anyone interested in taking 
on this task? Otherwise it is going to be stuck with outdated docs.


Michael

___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list