Re: [asterisk-dev] Final Preview: docs.asterisk.org

[asterisk]

On 7/28/2023 8:48 PM, George Joseph wrote:
On Fri, Jul 28, 2023 at 1:52 PM > wrote:



It's at the very bottom of the README:
> If you're always going to build just 1 branch's dynamic
documentation,
> you can skip the Makefile..inc file and just place everything in
the
> main Makefile.inc:

The first Makefile..inc has an extra period world's least
important
typo.


Ahha!   It's 'Makefile..inc' in the source README.md but the 
'' is getting stripped. :)


Ah, probably should've noticed that, actually, Makefile.inc twice in a 
row doesn't make any sense, if I was actually thinking about it...



Circling back to one other thing now that this seems good to go, what
exactly did you change for reducing the file sizes / is that
included in
the current iteration, without mike?


The addition of "navigation.prune" under features in mkdocs.yml did 
most of it, and yes, it's currently included.


I'm still seeing a lot of
extraneous whitespace in the pages. 244 KB per page isn't huge,
but just
at a cursory glance, 



Can you give me an example of an html page that's that big?  Most I 
see are in the 80-100k range


I think all of them - for example: 
https://docs.asterisk.org/Asterisk_20_Documentation/API_Documentation/Dialplan_Applications/ADSIProg/


This one is "only" 131 KB, but if you go and view source, you can scroll 
down a bit and see often hundreds of newlines, tabs, and spaces at a 
time in a row. I can't work how that's creeping in from the markdown, so 
I don't think it's from the markdown. That's why I thought we might need 
to do it manually, e.g. using tr or something like that. So regardless 
of page size, I think we could likely prune all the pages down just by 
eliminating whitespace.




I think we could probably cut the size 10-20% just
by getting rid of the whitespace. In some places, there are just
hundreds of newlines in a row for no reason.


Give me an example page.

If this is just what the
tool generates, I understand that, we don't have any control over
that,
just wanted to know. We could remove it all pretty easily with sed
probably, and think could be a final "post processing" step in the
Makefile, run recursively on all files. What do you think?


I tried to do exactly that but it didn't result in much savings and I 
got nervous about accidentally deleting multiple "blank" lines without 
knowing whether you might be in a "" block or not.   I was going 
to try html-minifier but just haven't got to it yet.


Yeah, I guess that could be tricky. But how much is the  tag 
actually used? On the page linked above, for example, I only see it 
once, and, ironically, there isn't any extraneous whitespace in it.  I 
took a look at a few different types of pages and this appears to be the 
case for all of them. So in our particular case, it seems like it might 
be okay to do a simple delete, since  shouldn't be affected by 
consecutive whitespace.


--
_
-- Bandwidth and Colocation Provided by http://www.api-digital.com --

asterisk-dev mailing list
To UNSUBSCRIBE or update options visit:
  http://lists.digium.com/mailman/listinfo/asterisk-dev

Re: [asterisk-dev] Final Preview: docs.asterisk.org

[George Joseph]

On Fri, Jul 28, 2023 at 6:48 PM George Joseph  wrote:

>
>
> On Fri, Jul 28, 2023 at 1:52 PM  wrote:
>
>>
>> It's at the very bottom of the README:
>> > If you're always going to build just 1 branch's dynamic documentation,
>> > you can skip the Makefile..inc file and just place everything in the
>> > main Makefile.inc:
>>
>> The first Makefile..inc has an extra period world's least important
>> typo.
>>
>
> Ahha!   It's 'Makefile..inc' in the source README.md but the
> '' is getting stripped. :)
>
>
>
>> Circling back to one other thing now that this seems good to go, what
>> exactly did you change for reducing the file sizes / is that included in
>> the current iteration, without mike?
>
>
> The addition of "navigation.prune" under features in mkdocs.yml did most
> of it, and yes, it's currently included.
>
>
>> I'm still seeing a lot of
>> extraneous whitespace in the pages. 244 KB per page isn't huge, but just
>> at a cursory glance,
>
>
> Can you give me an example of an html page that's that big?  Most I see
> are in the 80-100k range
>
>
>> I think we could probably cut the size 10-20% just
>> by getting rid of the whitespace. In some places, there are just
>> hundreds of newlines in a row for no reason.
>
>
> Give me an example page.
>
>
>> If this is just what the
>> tool generates, I understand that, we don't have any control over that,
>> just wanted to know. We could remove it all pretty easily with sed
>> probably, and think could be a final "post processing" step in the
>> Makefile, run recursively on all files. What do you think?
>>
>
> I tried to do exactly that but it didn't result in much savings and I got
> nervous about accidentally deleting multiple "blank" lines without knowing
> whether you might be in a "" block or not.   I was going to
> try html-minifier but just haven't got to it yet.
>

Oh, there's a mkdocs plugin called "minify".  I can give that a try.



>
>
>
-- 
_
-- Bandwidth and Colocation Provided by http://www.api-digital.com --

asterisk-dev mailing list
To UNSUBSCRIBE or update options visit:
   http://lists.digium.com/mailman/listinfo/asterisk-dev

Re: [asterisk-dev] Final Preview: docs.asterisk.org

[George Joseph]

On Fri, Jul 28, 2023 at 1:52 PM  wrote:

>
> It's at the very bottom of the README:
> > If you're always going to build just 1 branch's dynamic documentation,
> > you can skip the Makefile..inc file and just place everything in the
> > main Makefile.inc:
>
> The first Makefile..inc has an extra period world's least important
> typo.
>

Ahha!   It's 'Makefile..inc' in the source README.md but the
'' is getting stripped. :)



> Circling back to one other thing now that this seems good to go, what
> exactly did you change for reducing the file sizes / is that included in
> the current iteration, without mike?


The addition of "navigation.prune" under features in mkdocs.yml did most of
it, and yes, it's currently included.


> I'm still seeing a lot of
> extraneous whitespace in the pages. 244 KB per page isn't huge, but just
> at a cursory glance,


Can you give me an example of an html page that's that big?  Most I see are
in the 80-100k range


> I think we could probably cut the size 10-20% just
> by getting rid of the whitespace. In some places, there are just
> hundreds of newlines in a row for no reason.


Give me an example page.


> If this is just what the
> tool generates, I understand that, we don't have any control over that,
> just wanted to know. We could remove it all pretty easily with sed
> probably, and think could be a final "post processing" step in the
> Makefile, run recursively on all files. What do you think?
>

I tried to do exactly that but it didn't result in much savings and I got
nervous about accidentally deleting multiple "blank" lines without knowing
whether you might be in a "" block or not.   I was going to
try html-minifier but just haven't got to it yet.
-- 
_
-- Bandwidth and Colocation Provided by http://www.api-digital.com --

asterisk-dev mailing list
To UNSUBSCRIBE or update options visit:
   http://lists.digium.com/mailman/listinfo/asterisk-dev

Re: [asterisk-dev] Final Preview: docs.asterisk.org

[asterisk]

On 7/24/2023 8:27 AM, George Joseph wrote:



Only one at the time... I just found an extra period in
Makefile..inc in
the latest README, but haven't noticed anything else.

Not seeing it.  Probably fixed already.


It's at the very bottom of the README:
If you're always going to build just 1 branch's dynamic documentation, 
you can skip the Makefile..inc file and just place everything in the 
main Makefile.inc:


The first Makefile..inc has an extra period world's least important 
typo.



> Makefile.inc:
> ASTERISK_XML_FILE := /core-en_US.xml
> SKIP_ARI := yes
> BRANCHES := 20

Got it - this makes a lot more sense now! And yes, you read my
mind, I
don't care about ARI so that did the trick. I noticed the no-mike
branch
no longer exists, but looks like it was merged into main now, so I
gave
that a go and it got much further (sorry, I know it's been a while
and I
wasn't able to test this quickly).

Couldn't have asked for an easier process! It seems like I can just
clone the repo, copy my Makefile.inc in there, and run make. The
above
was on a relatively fast CPU, but it seems it shouldn't take
longer than
maybe 2 minutes.

The result is a 1.6 GB directory, 



Eh what?  When I build everything, temp/site is only 574M.  Maybe need 
to clean everything out or is your own stuff just that big?


No, I probably screwed it up somehow; "make clean" didn't remove any of 
the generated files, but it didn't give me a target error so I just 
assumed that would do what it needed to do. This time I explicitly did 
an rm -rf temp/site beforehand to ensure it would be clean.




 From what I tried initially, I should be able to solve this by
deleting
everything in the docs directory except index.md and the favicon,
which
ensures that there simply is no static content to build. That should
bring down both the size and the build time. I don't mind doing
that at
all, just wondering is there a good way to not build the static
content,
or would that be the best way?


Do a git pull :)
You should now be able to do...
"make BRANCH=master NO_STATIC=yes"

You can add NO_STATIC=yes to your makefile.inc instead.

Thanks George! This looks much more promising:

root@debian11:/usr/src/documentation# rm -rf temp/site/
root@debian11:/usr/src/documentation# make BRANCH=20
Creating ./temp/build-20
Setting Up Core Dynamic Documentation for branch '20'
  Generating markdown from Asterisk XML
Building to ./temp/site
INFO -  Cleaning site directory
INFO -  Building documentation to directory: 
/usr/src/documentation/temp/site

INFO -  Documentation built in 15.67 seconds

Now it's only 230 MB in total. The site builds quickly and it's exactly 
what I was looking for when I opened it up - perfect!


Circling back to one other thing now that this seems good to go, what 
exactly did you change for reducing the file sizes / is that included in 
the current iteration, without mike? I'm still seeing a lot of 
extraneous whitespace in the pages. 244 KB per page isn't huge, but just 
at a cursory glance, I think we could probably cut the size 10-20% just 
by getting rid of the whitespace. In some places, there are just 
hundreds of newlines in a row for no reason. If this is just what the 
tool generates, I understand that, we don't have any control over that, 
just wanted to know. We could remove it all pretty easily with sed 
probably, and think could be a final "post processing" step in the 
Makefile, run recursively on all files. What do you think?


--
_
-- Bandwidth and Colocation Provided by http://www.api-digital.com --

asterisk-dev mailing list
To UNSUBSCRIBE or update options visit:
  http://lists.digium.com/mailman/listinfo/asterisk-dev