Re: [opensc-devel] Managing the queue line of a compilation farm
Dear Martin, Perfect. Just contact me privately on IM to open an account for Jerrit on the farm dedicated host. Kind regards, Jean-Michel > Excellent! > > Providing different Linux packages outside of their native > environments (official package trees) would be a nice thing, > especially if not so common platforms can be used. At the same time, > maintaining *installable* packages (repositories) and debugging them > if things go wrong is a time consuming task. From usage statistics in > .ee there are some distros on your list that have maybe a single user > :) > > Compiling against different versions and different environments is > good and the more varied the installation base is, the better > different gotchas can be caught. But for optimal results, the outcomes > should be tested as well (I guess the actual compilation-time > differences are within minor versions of gcc/clang/pcsc-lite/openssl) > > Connecting a dedicated smart card reader + smart card and running some > automated tests in addition to compilation on each and every platform, > that would be a real cherry. > > > Martin -- Jean-Michel Pouré - Gooze - http://www.gooze.eu smime.p7s Description: S/MIME cryptographic signature ___ opensc-devel mailing list opensc-devel@lists.opensc-project.org http://www.opensc-project.org/mailman/listinfo/opensc-devel
Re: [opensc-devel] Managing the queue line of a compilation farm
Hello, On Fri, Mar 16, 2012 at 11:14, Jean-Michel Pouré - GOOZE wrote: > GOOZE is working on a compilation farm to compile OpenSC and > pcsc-lite/libccid for the following platforms: > > GNU/Linux: > * Debian sid 368/amd64 > * Debian wheezy 368/amd64 > * Debian squeeze 368/amd64 > * Ubuntu precise 368/amd64 > * Ubuntu oneiric 368/amd64 > * Ubuntu natty 368/amd64 > * Ubuntu maverick 368/amd64 > * Fedora 15 > * Fedora 16 > * Fedora 17 > * Cent OS 6 > * OpenSuse 12.1 > * OpenSuse 11.4 Excellent! Providing different Linux packages outside of their native environments (official package trees) would be a nice thing, especially if not so common platforms can be used. At the same time, maintaining *installable* packages (repositories) and debugging them if things go wrong is a time consuming task. From usage statistics in .ee there are some distros on your list that have maybe a single user :) Compiling against different versions and different environments is good and the more varied the installation base is, the better different gotchas can be caught. But for optimal results, the outcomes should be tested as well (I guess the actual compilation-time differences are within minor versions of gcc/clang/pcsc-lite/openssl) Connecting a dedicated smart card reader + smart card and running some automated tests in addition to compilation on each and every platform, that would be a real cherry. Martin ___ opensc-devel mailing list opensc-devel@lists.opensc-project.org http://www.opensc-project.org/mailman/listinfo/opensc-devel
Re: [opensc-devel] Managing the queue line of a compilation farm
Le samedi 17 mars 2012 à 22:15 +0100, Viktor Tarasov a écrit : > Excellent, I will prepare the guidelines for the procedure. > Thanks. Perfect. During the week-end, I installed a fresh Super-micro server at the office. It is running a Debian x64 with encrypted file system on a 60G solid state drive. The hardware is running with an IPMI card providing cold/hot reset, serial over IP and monitoring to ensure the server is always up and running. It is running on electrical backup for 24x24 operation. So we stand ready to provide the first packages whenever needed. > You should not be disappointed - the naming policy is open for > suggestions. > The current one is just the rapid 'working' solution. This is a translation problem, sorry. I am not "disappointed". :) About naming, I would suggest to mimic VLC lightlies, at this URL: http://nightlies.videolan.org/build/win32/?C=M;O=D For Windows: opensc-$version-$branch-$mmdd-$hhmm-$platform-$debug.$suffix For Mac OSX: opensc-$version-$branch-$mmdd-$hhmm-$platform.$suffix Where: branch = "staging" or "sm" if we have two branches, or simply "git" if there is only one branch. version = 0.12.3 (major-minor). mmdd = 20120319 (19 March 2012) hhmm = 0112 (one hour 12 minutes) platform = x32 or x64 debug = "debug" or none. suffix = 7z, zip, exe or msi So we can have as many compilations as needed during the day. To comply with Debian rules: We may have to add a leading "O-" to the Debian versions. And set priority to zero. We have to think about a simple way to accommodate with several GIT versions. Maybe it could be a virtual apache host, i.e. gitversion.domain.org. So we can provide different APT repositories. Kind regards, -- Jean-Michel Pouré - Gooze - http://www.gooze.eu smime.p7s Description: S/MIME cryptographic signature ___ opensc-devel mailing list opensc-devel@lists.opensc-project.org http://www.opensc-project.org/mailman/listinfo/opensc-devel
Re: [opensc-devel] Managing the queue line of a compilation farm
Le 16/03/2012 19:29, Jean-Michel Pouré - GOOZE a écrit : >>> For example, in https://opensc.fr/jenkins/ the build on two windows >>> platform depend-on and triggered-by the build on debian. > To build under Debian packages, we probably need: > https://launchpad.net/jenkins.pbuilder.plugin > > pbuilder is the Debian chrooted package builder. > > If you allow us, we may trigger pbuilder on our farm using SSH. The > problem with pbuilder is that it runs under root account (or sudo which > is the same). So we will probably need to install a fresh server with > all Debian and Ubuntu chroots. Builds will auto-sign packages. Priority > should be set to zero, just like in Debian backports. Ok. > I am guite disapointed by OpenSC-SM naming under Windows. From a user > point of view, it should be more explicit and include a readable naming > space, like for example name-year-date-32x.msi. Even if this means a > single build everyday. You should not be disappointed - the naming policy is open for suggestions. The current one is just the rapid 'working' solution. > Can we proceed with Debian stuff? We can cope with all GNU/Linux builds > under all platforms including x32, x64 and ARM, providing signed > packages and APT/RPMs repositories. Excellent, I will prepare the guidelines for the procedure. Thanks. > Kind regards, Kind regards, Viktor. ___ opensc-devel mailing list opensc-devel@lists.opensc-project.org http://www.opensc-project.org/mailman/listinfo/opensc-devel
Re: [opensc-devel] Managing the queue line of a compilation farm
Dear Ludovic and all, > Do you know the OpenSUSE build service [1]? > [1] http://en.opensuse.org/Build_Service I am testing OpenSuSE build service and it seems powerful. OpenSuSE is a suitable service to build RPMs and is not Debian centric. It cannot really be used for Debian packaging. Of course we could us it to build Debian packages, but features are too limited. Maybe we can use OpenSUSE to rebuild packages and/or build from GIT for RPM platforms. I'll get back to you. Kind regards, -- Jean-Michel Pouré - Gooze - http://www.gooze.eu smime.p7s Description: S/MIME cryptographic signature ___ opensc-devel mailing list opensc-devel@lists.opensc-project.org http://www.opensc-project.org/mailman/listinfo/opensc-devel
Re: [opensc-devel] Managing the queue line of a compilation farm
> > For example, in https://opensc.fr/jenkins/ the build on two windows > > platform depend-on and triggered-by the build on debian. To build under Debian packages, we probably need: https://launchpad.net/jenkins.pbuilder.plugin pbuilder is the Debian chrooted package builder. If you allow us, we may trigger pbuilder on our farm using SSH. The problem with pbuilder is that it runs under root account (or sudo which is the same). So we will probably need to install a fresh server with all Debian and Ubuntu chroots. Builds will auto-sign packages. Priority should be set to zero, just like in Debian backports. I am guite disapointed by OpenSC-SM naming under Windows. From a user point of view, it should be more explicit and include a readable naming space, like for example name-year-date-32x.msi. Even if this means a single build everyday. Can we proceed with Debian stuff? We can cope with all GNU/Linux builds under all platforms including x32, x64 and ARM, providing signed packages and APT/RPMs repositories. Kind regards, -- Jean-Michel Pouré - Gooze - http://www.gooze.eu smime.p7s Description: S/MIME cryptographic signature ___ opensc-devel mailing list opensc-devel@lists.opensc-project.org http://www.opensc-project.org/mailman/listinfo/opensc-devel
Re: [opensc-devel] Managing the queue line of a compilation farm
Le vendredi 16 mars 2012 à 15:20 +0100, Viktor Tarasov a écrit : > Not sure that I understand the term 'queue management' > but in jenkins you can create a chain of dependent builds/tests/etc, > transfer the artifacts from one build to another, ... > For example, in https://opensc.fr/jenkins/ the build on two windows > platform depend-on and triggered-by the build on debian. > In your list you mention Windows: 32 and 64 msi. > Will it be cross-compilation? Jenkins is perfect! We need OpenSC-SM and OpenSC staging. I just opened an account on Jenkins, username=gooze. What do I need to connect to slave hosts? We can provide Mac OS X, Debian and Ubuntu slaves for different architectures. Kind regards, -- Jean-Michel Pouré - Gooze - http://www.gooze.eu smime.p7s Description: S/MIME cryptographic signature ___ opensc-devel mailing list opensc-devel@lists.opensc-project.org http://www.opensc-project.org/mailman/listinfo/opensc-devel
Re: [opensc-devel] Managing the queue line of a compilation farm
On Fri, Mar 16, 2012 at 2:47 PM, Jean-Michel Pouré - GOOZE wrote: > Le vendredi 16 mars 2012 à 12:56 +0100, Ludovic Rousseau a écrit : > > Do you know the OpenSUSE build service [1]? > > I checked OBS but it does not support all architectures. The reason is > simple: OpenSUSE wants to convince that SuSE is the best. Simple old > story. > > But you are right: SuSE job management software seems to be free, so I > will test it. > > From http://popcon.debian.org/ it seems that i386, AMD64 and ARM are the > GNU/Linux widely used platforms. We ordered a couple of ARM boards with > 1G onboard today. This will allow to provide Debian ARM and Android > based systems. > > If you needs more platforms, let me know. > > GOOZE really wants to contribute to OpenSC and help. At present, the > compilation farm is running, but we don't have a queue management > system. If you know one, any generic and opensource solution, let us > know. > > What solution for job queue management is using OpenSC? > Not sure that I understand the term 'queue management' but in jenkins you can create a chain of dependent builds/tests/etc, transfer the artifacts from one build to another, ... For example, in https://opensc.fr/jenkins/ the build on two windows platform depend-on and triggered-by the build on debian. In your list you mention Windows: 32 and 64 msi. Will it be cross-compilation? > > Kind regards, > -- > Jean-Michel Pouré - Gooze - http://www.gooze.eu > Kind wishes, Viktor. ___ opensc-devel mailing list opensc-devel@lists.opensc-project.org http://www.opensc-project.org/mailman/listinfo/opensc-devel
Re: [opensc-devel] Managing the queue line of a compilation farm
Le vendredi 16 mars 2012 à 12:56 +0100, Ludovic Rousseau a écrit : > Do you know the OpenSUSE build service [1]? I checked OBS but it does not support all architectures. The reason is simple: OpenSUSE wants to convince that SuSE is the best. Simple old story. But you are right: SuSE job management software seems to be free, so I will test it. From http://popcon.debian.org/ it seems that i386, AMD64 and ARM are the GNU/Linux widely used platforms. We ordered a couple of ARM boards with 1G onboard today. This will allow to provide Debian ARM and Android based systems. If you needs more platforms, let me know. GOOZE really wants to contribute to OpenSC and help. At present, the compilation farm is running, but we don't have a queue management system. If you know one, any generic and opensource solution, let us know. What solution for job queue management is using OpenSC? Kind regards, -- Jean-Michel Pouré - Gooze - http://www.gooze.eu smime.p7s Description: S/MIME cryptographic signature ___ opensc-devel mailing list opensc-devel@lists.opensc-project.org http://www.opensc-project.org/mailman/listinfo/opensc-devel
Re: [opensc-devel] Managing the queue line of a compilation farm
Le 16 mars 2012 10:14, Jean-Michel Pouré - GOOZE a écrit : > Dear all, Hello, > GOOZE is working on a compilation farm to compile OpenSC and > pcsc-lite/libccid for the following platforms: Good idea. > GNU/Linux: > * Debian sid 368/amd64 > * Debian wheezy 368/amd64 > * Debian squeeze 368/amd64 > * Ubuntu precise 368/amd64 > * Ubuntu oneiric 368/amd64 > * Ubuntu natty 368/amd64 > * Ubuntu maverick 368/amd64 > * Fedora 15 > * Fedora 16 > * Fedora 17 > * Cent OS 6 > * OpenSuse 12.1 > * OpenSuse 11.4 > > Mac OS X > * Mac OS X 10.7 > * Mac OS X 10.6 > > Windows: > * x32 msi > * x64 msi > > Platforms are able to compile in a chroot environment using pbuilder > (Debian) and mock (RPM based). We are still looking for a queue manager > to glue all machines together and distribute compilation works. > > Do you have any idea how to queue, manage and monitor compilation works? > Each distribution has its own queue management system. But as all this > relies on bash commands, I wonder if a general solution could be used. > > Any idea? Do you know the OpenSUSE build service [1]? It should already do what you want to (re)do at Gooze. I have not yet used the Open Build Service. It is on my todo list for another project. Bye [1] http://en.opensuse.org/Build_Service -- Dr. Ludovic Rousseau ___ opensc-devel mailing list opensc-devel@lists.opensc-project.org http://www.opensc-project.org/mailman/listinfo/opensc-devel
[opensc-devel] Managing the queue line of a compilation farm
Dear all, GOOZE is working on a compilation farm to compile OpenSC and pcsc-lite/libccid for the following platforms: GNU/Linux: * Debian sid 368/amd64 * Debian wheezy 368/amd64 * Debian squeeze 368/amd64 * Ubuntu precise 368/amd64 * Ubuntu oneiric 368/amd64 * Ubuntu natty 368/amd64 * Ubuntu maverick 368/amd64 * Fedora 15 * Fedora 16 * Fedora 17 * Cent OS 6 * OpenSuse 12.1 * OpenSuse 11.4 Mac OS X * Mac OS X 10.7 * Mac OS X 10.6 Windows: * x32 msi * x64 msi Platforms are able to compile in a chroot environment using pbuilder (Debian) and mock (RPM based). We are still looking for a queue manager to glue all machines together and distribute compilation works. Do you have any idea how to queue, manage and monitor compilation works? Each distribution has its own queue management system. But as all this relies on bash commands, I wonder if a general solution could be used. Any idea? Kind regards, -- Jean-Michel Pouré - Gooze - http://www.gooze.eu smime.p7s Description: S/MIME cryptographic signature ___ opensc-devel mailing list opensc-devel@lists.opensc-project.org http://www.opensc-project.org/mailman/listinfo/opensc-devel
