Re: [LEDE-DEV] Call defines for minifying scripting languages
On 10/03/2016 01:00 PM, Jan-Tarek Butt wrote: > Hi, > > My Idea ist to create calldefs to minify all scriptes there are > interpreted by run time. > > As example see the mail "[PATCH] LuaSrcDiet call define for lua > code Minifying". > > I plan to write also calldefs for shellscripts, python and perl. > > [...] > When it comes to shell-scripts, I'd start at a prior step: why does an embedded system need to run and interpret so many shell-scripts at all? OWRT has been shell-script heavy from the very beginning, but luckily over the last few years started to move more and more of that scripted functionality into binaries (e.g. procd, netifd). This transition only started, and imho the goal to reach is to get rid of all shell-scripts on target (beside init-scripts, maybe). Therefore, instead of optimizing them, I'd opt to treat them as technical debt which needs to be resolved by replacing scripts with binaries - personally, I won't spend too much time on something to-become-legacy mid-term. Cheers, Zefir ___ Lede-dev mailing list Lede-dev@lists.infradead.org http://lists.infradead.org/mailman/listinfo/lede-dev
Re: [LEDE-DEV] Call defines for minifying scripting languages
* Karl Palsson[04.10.2016 08:38]: [...] > about the scripts _at all_ right up until they try and look in > one to debug something behaving oddly. At _that_ point they will > care _very much_ that it's all minified garbage, with some > presumption that they could "just get the source somewhere else" I have the same impression. But making it a non-default option for advanced users/self-builders is ok. Lets see what he can reach in the end... bye, bastian ___ Lede-dev mailing list Lede-dev@lists.infradead.org http://lists.infradead.org/mailman/listinfo/lede-dev
Re: [LEDE-DEV] Call defines for minifying scripting languages
On Mon, Oct 03, 2016 at 01:00:19PM +0200, Jan-Tarek Butt wrote: > My Idea ist to create calldefs to minify all scriptes there are > interpreted by run time. > > 1. Reducing memory size on firmware images. > this is absolutely negligible. as others already pointed out, scripts compress quite well. also, they are actually only a rather small fraction of the image's total size. > 2. Strip out all comments (which makes us able to do better code > commenting) > you mean, we can make the scripts *yet bigger* because it won't negatively affect the image's size. except that point 1, so this isn't actually a limiting factor. writing said comments and keeping them up-to-date is the problem. > 3. and so on. > i think mostly "whatever". ;) i'll also pre-emptively point out that minification has a rather insignificant impact on run time - the only thing that is slowed down is the tokenization/parsing, and that is again rather insiginificant compared to everything else executing a script involves. but hey, make some actual measurements - maybe you'll surprise me. ___ Lede-dev mailing list Lede-dev@lists.infradead.org http://lists.infradead.org/mailman/listinfo/lede-dev
Re: [LEDE-DEV] Call defines for minifying scripting languages
* Jan-Tarek Butt[03.10.2016 16:55]: > currently I am working on a Shellscript minifyer > there are not realy exsist one so I write a new one. hopefully in haskell 8-) you made my day 8-) ROFL! I think this is a bad idea. In practise you safe nothing but make debugging harder, the JFFS2 or squashfs already compresses text/shellscripts very good. bye, bastian ___ Lede-dev mailing list Lede-dev@lists.infradead.org http://lists.infradead.org/mailman/listinfo/lede-dev
Re: [LEDE-DEV] Call defines for minifying scripting languages
On 10/03/16 14:18, Piotr Dymacz wrote: > Hello, > > 2016-10-03 13:14 GMT+02:00 Karl Palsson: >> >> Jan-Tarek Butt wrote: > [snip] >>> 1. Reducing memory size on firmware images. >> >> But will it? They're in the squashfs image, it's already been >> demonstrated before that compressing things before can actually >> have negative impacts. > > That's true, my vote for 'optimization-only' approach and keep fs do > the rest (compression). > >>> 2. Strip out all comments (which makes us able to do better >>> code commenting) 3. and so on. > > IMHO, we could remove all comments from all scripts (not only Lua) in > target rootfs and work on better documentation outside the code. > > As an example of a huge, old comment, left in one of ar71xx base-files > scripts, please see [1]. > It's inside every ar71xx image... does it really make sense to have it > there, for devices without related hardware? Exactly that's my point. I mean that is really not necessary. And the comments will be stay inside the git. Only on the final images there will be gone. cheers Tarek signature.asc Description: OpenPGP digital signature ___ Lede-dev mailing list Lede-dev@lists.infradead.org http://lists.infradead.org/mailman/listinfo/lede-dev
Re: [LEDE-DEV] Call defines for minifying scripting languages
On 10/03/16 13:14, Karl Palsson wrote: > > Jan-Tarek Buttwrote: >> Hi, >> >> My Idea ist to create calldefs to minify all scriptes there are >> interpreted by run time. >> >> As example see the mail "[PATCH] LuaSrcDiet call define for lua >> code Minifying". >> >> I plan to write also calldefs for shellscripts, python and >> perl. >> >> They brings vareous of features. >> >> 1. Reducing memory size on firmware images. > > But will it? They're in the squashfs image, it's already been > demonstrated before that compressing things before can actually > have negative impacts. The minifing proces is on compile time so I cant see negative impacts for the squashfs. >> 2. Strip out all comments (which makes us able to do better >> code commenting) 3. and so on. >> >> One negative point will be there: >> >> the minified code an not realy human readable but if some one >> want to read this script or work on it on routers, they can >> easily copy the unminified script via scp. > > This is a _massive_ downside IMO, and absolutely not something > that should be enabled by default. There's already far too much > undocumented/underdocumented internal behaviours of scripts and > processes in LEDE/OpenWrt, and minifying scripts and stripping > comments actively works to preserve that situation. Sorry I think I have formulated it wrong. Inside the git we can documented the code and while the compile process the comments will drop from all shellscrips who run the calldef in its makefile. so only on the finish compiled images are the minified scripts. and of cause this is optionaly and looks like this [0]. So you can just call ist inside the package Makefile. [0] https://lists.ffnw.de//pipermail/dev/2016-July/001444.html cheers Tarek signature.asc Description: OpenPGP digital signature ___ Lede-dev mailing list Lede-dev@lists.infradead.org http://lists.infradead.org/mailman/listinfo/lede-dev
Re: [LEDE-DEV] Call defines for minifying scripting languages
Hello, 2016-10-03 13:14 GMT+02:00 Karl Palsson: > > Jan-Tarek Butt wrote: [snip] >> 1. Reducing memory size on firmware images. > > But will it? They're in the squashfs image, it's already been > demonstrated before that compressing things before can actually > have negative impacts. That's true, my vote for 'optimization-only' approach and keep fs do the rest (compression). >> 2. Strip out all comments (which makes us able to do better >> code commenting) 3. and so on. IMHO, we could remove all comments from all scripts (not only Lua) in target rootfs and work on better documentation outside the code. As an example of a huge, old comment, left in one of ar71xx base-files scripts, please see [1]. It's inside every ar71xx image... does it really make sense to have it there, for devices without related hardware? >> One negative point will be there: >> >> the minified code an not realy human readable but if some one >> want to read this script or work on it on routers, they can >> easily copy the unminified script via scp. > > This is a _massive_ downside IMO, and absolutely not something > that should be enabled by default. There's already far too much > undocumented/underdocumented internal behaviours of scripts and > processes in LEDE/OpenWrt, and minifying scripts and stripping > comments actively works to preserve that situation. [snip] +1 for keeping scripts human readable. +10 for getting scripts documented, someday, somewhere... Wiki? [1] https://goo.gl/yQFpDV Cheers, Piotr ___ Lede-dev mailing list Lede-dev@lists.infradead.org http://lists.infradead.org/mailman/listinfo/lede-dev
Re: [LEDE-DEV] Call defines for minifying scripting languages
Jan-Tarek Buttwrote: > Hi, > > My Idea ist to create calldefs to minify all scriptes there are > interpreted by run time. > > As example see the mail "[PATCH] LuaSrcDiet call define for lua > code Minifying". > > I plan to write also calldefs for shellscripts, python and > perl. > > They brings vareous of features. > > 1. Reducing memory size on firmware images. But will it? They're in the squashfs image, it's already been demonstrated before that compressing things before can actually have negative impacts. > 2. Strip out all comments (which makes us able to do better > code commenting) 3. and so on. > > One negative point will be there: > > the minified code an not realy human readable but if some one > want to read this script or work on it on routers, they can > easily copy the unminified script via scp. This is a _massive_ downside IMO, and absolutely not something that should be enabled by default. There's already far too much undocumented/underdocumented internal behaviours of scripts and processes in LEDE/OpenWrt, and minifying scripts and stripping comments actively works to preserve that situation. Sincerely, Karl Palsson signature.asc Description: OpenPGP Digital Signature ___ Lede-dev mailing list Lede-dev@lists.infradead.org http://lists.infradead.org/mailman/listinfo/lede-dev