Hello Waldek/Nadav/All, Totally agreed Waldek! And maybe we could also define a policy to phase out apps that are not maintained for a long time eg: let's suppose we have an apps that is not updated for more than 3 years we do a survey in the mailing list, then, if no one is using, we remove it from apps
Also, is there any all to modularise the apps and the external directories? I'm asking this, because I have no interesting in many apps and yet, I have to download the whole apps As for the external, I don't use the ARM 64 (aarch64 if I got it right), so, for my case, it's totally useless Kind Regards, Geraldo Netto Sapere Aude => Non dvcor, dvco http://exdev.sf.net/ On 18 February 2018 at 12:47, Waldek Kozaczuk <jwkozac...@gmail.com> wrote: > If I may also add my 2 cents. I think it might be also worth to think > about priorities meaning which apps we care and which not so much. And the > make sure that at least the most important ones have fixed build process, > are well documented (README) and tested. > > Here would be my high level breakdown of the apps with first category > being most important: > > 1. Runtimes - Java (openjdk* apps), NodeJS, Erlang, Python, Ruby, Free > Pascal, etc > 2. Stateless apps - redis, memchached, twemproxy, etc (other apps that > could be ZFS-less) - things that do not require writing to a file system > (ROFS) > 3. Everything else - even though one can run MySQL or ElasticSearch on > OSv - I do not think many people will do it for real > > Lastly it would be nice to provide a Docker-based build process for some > of these apps. I am not big fan of Docker as a runtime platform (because I > am fan of OSv) I think that Docker is an excellent platform to provide > consistent/repeatable process to build OSv apps. Look here for examples - > https://cloudrouter.org/cloudrouter/2015/04/27/build-osv-images.html and > https://github.com/mikelangelo-project/capstan-packages/ > > Finally (even though I used lastly above ;-)) please bear in mind that > many OSv users may not care about Java app makefiles as in JVM-land they > would use tools like Maven, Gradle or to build. > > Regards, > Waldek > > On Sun, Feb 18, 2018 at 7:08 AM, Nadav Har'El <n...@scylladb.com> wrote: > >> On Sun, Feb 18, 2018 at 5:27 AM, Geraldo Netto <geraldone...@gmail.com> >> wrote: >> >>> Hello Waldek/All, >>> >>> Indeed, I must say, not only this, but many other apps have broken urls >>> or broken builds >>> >> >> This was the main thing that bothered me about your "change wget to wget >> -c" series of 50 >> patches - in many of these changes, the fix didn't really fix anything - >> either the module did >> not compile at all before your patch (and also after it), or the Makefile >> was written in a way >> that it doesn't matter at all whether you did wget or wget -c because, >> for example, the files >> were deleted and re-downloaded from scratch every time. >> >> I think as a general comment, we need to at least try to *test* every >> patch. If a patch >> can't be tested because the code doesn't actually work both before and >> after the patch, >> it's a hint there's a more important patch to be written instead of the >> patch that doesn't >> fix anything :-) >> >> >>> For example, this is my 'quick list' after running random builds in >>> random apps: >>> - busybox that was updated recently (somewhere in the end of 2017) looks >>> broken for me now >>> - Apache Derby that is compiled with a newer jdk and because of that, >>> does not run >>> >> >> Although it would be great to fix all of these issues, it will take work, >> and in many cases >> the original module was written by someone who had a personal interest in >> this specific >> application, and today there's nobody who cares. So if we don't fix some >> of these modules, >> the least we could do is to open issues in the bug tracker and present >> what you found >> didn't run, and why you think it doesn't - perhaps someone else would >> care enough to >> fix them. >> >> I created https://github.com/cloudius-systems/osv/issues/947 now about >> what needs >> to be done to fix apache-derby (with fairly easy instructions how, in >> case you want to >> contribute a patch :-) ). >> >> >> >>> - Erlang, Free Pascal and some other apps require their own >>> compilers/tools/sdk to be installed before building their packages and we >>> don't have a check for them >>> >> >> While it would be nice to have a programatic check, an almost-good-enough >> alternative >> is to have a README in the module directory explaining what you need to >> get it running. >> In any case, I think it should be considered a must to have a README in >> each module >> directory - in some of those, I am not even sure what this module does or >> how I'm supposed >> to use it after building an image with it. >> >> >>> - we don't have a unified structure to handle many common cases >>> >> >> This I agree but see more on that below (it's a great idea, but will >> require work). >> >> >>> - netperf url seems broken >>> - rpm url is broken for me and on mint (ubuntu based) it requires >>> libmagic and some other libs to be installed >>> - leveldb i think i have submitted some patches to fix and upgrade the >>> build >>> - rhino url seems broken >>> - embulk url seems broken >>> - groonga build seems broken >>> - plus all the problems you're facing >>> >>> >>> Actually, I would like to suggest a different approach for the building, >>> maybe use a single unified Makefile or shell script >>> that uses a configuration file per app like capstran, in case some >>> custom shell script is required, we can also define some standard 'hooks' >>> >> >> We could do this, but it would be a lot of work to provide something >> useful enough for all application needs. >> The nice thing about the shell or Makefile (or both) is that it lets you >> do anything :-) >> >> I wonder if we can learn from existing open source distributions >> projects. For example, for RPM building too, they >> take upstream tars, mix them with local patches, run the Makefile, and >> collect part of the resulting files into an RPM. >> Could we use some of their tooling? >> I guess we could do better with our own special tools, and it will >> definitely be very useful, it just will be an effort to write them. >> >> >>> >>> By now, I see we need the follow tasks: >>> >>> check >>> - could be used to check for dependencies, like check if any host >>> library is missing >>> - could be used to check for the downloaded file >>> if we don't have the md5 of the file, we could pre-compute the hash in >>> our local host and put it to compare >>> wget -c just makes sure the file is there but it does not check for >>> the file size/hash as You/Nadav noticed >>> >>> build >>> - could be a Makefile or a shell script, I would suggest to define a >>> standard way of creating it - currently, we have many different approaches >>> to do the same thing >>> we could pass the cpu number automatically, I don't know why, i'm >>> still getting the make message "jobserver unavailable: using -j1. Add `+' >>> to parent make rule" in some builds >>> >>> unpack (to untar/unzip/un<anything>) >>> - just needs to unpack the source/binary >>> >>> rootfs >>> - put all the content on rootfs, etc >>> >>> manifest >>> - prepares the usr.manifest file >>> >>> clean >>> - a standard way to clean the apps without removing the source/binary >>> >>> Do you have any suggestion/comment? >>> >>> >>> Kind Regards, >>> >>> Geraldo Netto >>> Sapere Aude => Non dvcor, dvco >>> http://exdev.sf.net/ >>> >>> On 17 February 2018 at 21:20, Waldek Kozaczuk <jwkozac...@gmail.com> >>> wrote: >>> >>>> After I applied this patch and previous and this is an error I am >>>> getting: >>>> >>>> 2018-02-17 18:12:45 7786 [Note] InnoDB: Waiting for purge to start >>>> 2018-02-17 18:12:45 7786 [Note] InnoDB: 5.6.21 started; log sequence >>>> number 1625987 >>>> ERROR: 1193 Unknown system variable 'table_type' >>>> 2018-02-17 18:12:45 7786 [ERROR] Aborting >>>> >>>> 2018-02-17 18:12:45 7786 [Note] Binlog end >>>> 2018-02-17 18:12:45 7786 [Note] InnoDB: FTS optimize thread exiting. >>>> 2018-02-17 18:12:45 7786 [Note] InnoDB: Starting shutdown... >>>> 2018-02-17 18:12:47 7786 [Note] InnoDB: Shutdown completed; log >>>> sequence number 1625997 >>>> 2018-02-17 18:12:47 7786 [Note] ./bin/mysqld: Shutdown complete >>>> >>>> Importing /home/wkozaczuk/projects/osv/modules/java-base/module.py >>>> Importing /home/wkozaczuk/projects/osv/apps/fonts/module.py >>>> No module.py in /home/wkozaczuk/projects/osv/modules/ca-certificates >>>> No module.py in /home/wkozaczuk/projects/osv/modules/libz >>>> Preparing usr.manifest >>>> Appending /home/wkozaczuk/projects/osv/apps/mysql/usr.manifest to >>>> usr.manifest >>>> Preparing bootfs.manifest >>>> Saving command line to /home/wkozaczuk/projects/osv/b >>>> uild/release.x64/cmdline >>>> Building into build/release.x64 >>>> GEN gen/include/osv/version.h >>>> MKBOOTFS build/release.x64/bootfs.bin >>>> AS bootfs.s >>>> LINK loader.elf >>>> LIBOSV.SO >>>> STRIP loader.elf -> loader-stripped.elf >>>> LZ loader-stripped.elf >>>> OBJCOPY loader-stripped.elf.lz -> loader-stripped.elf.lz.o >>>> LINK lzloader.elf >>>> ALIGN lzloader.elf >>>> DD loader.img boot.bin >>>> DD loader.img lzloader.elf >>>> IMGEDIT build/release.x64/loader.img >>>> IMGEDIT build/release.x64/loader.img >>>> Traceback (most recent call last): >>>> File "/home/wkozaczuk/projects/osv/scripts/upload_manifest.py", line >>>> 161, in <module> >>>> main() >>>> File "/home/wkozaczuk/projects/osv/scripts/upload_manifest.py", line >>>> 144, in main >>>> manifest = read_manifest(options.manifest) >>>> File "/home/wkozaczuk/projects/osv/scripts/manifest_common.py", line >>>> 59, in read_manifest >>>> hostpath = components[1].strip() >>>> IndexError: list index out of range >>>> ./scripts/build failed: $SRC/scripts/upload_manifest.py -o usr.img -m >>>> usr.manifest -D jdkbase=$jdkbase -D gccbase=$gccbase -D >>>> glibcbase=$glibcbase -D miscbase=$miscbase >>>> >>>> Also as Nadav pointed out when one stops (Ctrl-C) after download (wget) >>>> next run happily does NOT do anything. >>>> >>>> Waldek >>>> >>>> On Tuesday, January 16, 2018 at 2:44:29 PM UTC-5, Geraldo Netto wrote: >>>>> >>>>> Signed-off-by: geraldo netto <gerald...@gmail.com> >>>>> --- >>>>> mysql/GET | 5 +---- >>>>> 1 file changed, 1 insertion(+), 4 deletions(-) >>>>> >>>>> diff --git a/mysql/GET b/mysql/GET >>>>> index 57ed3f8..8871124 100755 >>>>> --- a/mysql/GET >>>>> +++ b/mysql/GET >>>>> @@ -6,12 +6,9 @@ BUILDDIR=$BASEDIR/build >>>>> ROOTDIR=$BASEDIR/install >>>>> SRCDIR=$BASEDIR/mysql-$VERSION >>>>> >>>>> -if [ ! -f mysql-$VERSION.tar.gz ]; then >>>>> - wget -c http://dev.mysql.com/get/Downl >>>>> oads/MySQL-5.6/mysql-$VERSION.tar.gz >>>>> -fi >>>>> <http://dev.mysql.com/get/Downloads/MySQL-5.6/mysql-$VERSION.tar.gz-fi> >>>>> +wget -c http://dev.mysql.com/get/Downloads/MySQL-5.6/mysql-$VERSION. >>>>> tar.gz >>>>> tar zxf mysql-$VERSION.tar.gz >>>>> >>>>> - >>>>> mkdir -p $BUILDDIR >>>>> mkdir -p $ROOTDIR/etc >>>>> >>>>> -- >>>>> 2.7.4 >>>>> >>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "OSv Development" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to osv-dev+unsubscr...@googlegroups.com. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "OSv Development" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to osv-dev+unsubscr...@googlegroups.com. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> > -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to osv-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
