Re: rc.d scripts to control multiple instances of the same daemon?
On Tue, Jun 25, 2013 at 03:44:31PM -0400, Garrett Wollman wrote: I'm in the process of (re)writing an rc.d script for kadmind (security/krb5). Unlike the main Kerberos daemon, kadmind needs to have a separate instance for each realm on the server -- it can't support multiple realms in a single process. What I need to be able to do: 1) Have different flags and pidfiles for each instance. 2) Be able to start, stop, restart, and status each individual instance by giving its name on the command line. 3) Have all instances start/stop automatically when a specific instance isn't specified. I've looked around for examples of good practice to emulate, and haven't found much. The closest to what I want looks to be vboxheadless, but I'm uncomfortable with the amount of mechanism from rc.subr that it needs to reimplement. Are there any better examples? -GAWollman ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org This is a simple multi instance rc.d script: http://svnweb.freebsd.org/ports/head/www/fcgiwrap/files/fcgiwrap.in?revision=307542view=markup regards, Bapt pgptAOgDKVF01.pgp Description: PGP signature
Importing tradcpp (traditional (KR-style) C macro preprocessor) into base?
Hi, I have been working in importing tradcpp (developped by David A. Holland from NetBSD) into the ports tree, it is a traditional (KR-style) C macro preprocessor BSD licensed. I first worked on it so that imake can work properly without gcc. I discovered that some part of the base system still needs a traditional preprocessor, like (calendar), what I propose it to import tradcpp into the base system (not the version in port right now but what will become version 0.2). It mostly behave like gcpp, and I'm able to properly use calendar along with tradcpp with this small patch: http://people.freebsd.org/~bapt/tradcpp.diff Any objections against me importing it? regards, Bapt pgpYMMUnszxFD.pgp Description: PGP signature
Re: Importing tradcpp (traditional (KR-style) C macro preprocessor) into base?
On Wed, Jun 12, 2013 at 12:23:44AM +0200, Matthias Andree wrote: Am 12.06.2013 00:11, schrieb Baptiste Daroussin: Hi, I have been working in importing tradcpp (developped by David A. Holland from NetBSD) into the ports tree, it is a traditional (KR-style) C macro preprocessor BSD licensed. I first worked on it so that imake can work properly without gcc. I discovered that some part of the base system still needs a traditional preprocessor, like (calendar), what I propose it to import tradcpp into the base system (not the version in port right now but what will become version 0.2). It mostly behave like gcpp, and I'm able to properly use calendar along with tradcpp with this small patch: http://people.freebsd.org/~bapt/tradcpp.diff Any objections against me importing it? Shouldn't we fix calendar and imake so that they can use a modern cpp, instead of going back 25 years? Or am I missing the point here? To be more specific, some people have express some concern about the lack of support for traditional cpp in base, that's why I'm proposing this, now personnaly I don't care if tradcpp remains in ports (for imake, imake is not a matter of fixing imake, but rather all the users of imake). If we think it is not worth having a traditional cpp, I won't import it. cpp has not be design at first for this kind of usage, but someone of our vendors rely on a traditional cpp anyway. Just it exists, it is rather small, it is BSDL and actively maintainer, so :) concerning calendar(1) another approach is available here: bin/178463 regards, Bapt pgpmRBOgMOwVs.pgp Description: PGP signature
Re: Importing tradcpp (traditional (KR-style) C macro preprocessor) into base?
On Tue, Jun 11, 2013 at 03:35:54PM -0700, Xin Li wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 06/11/13 15:11, Baptiste Daroussin wrote: Hi, I have been working in importing tradcpp (developped by David A. Holland from NetBSD) into the ports tree, it is a traditional (KR-style) C macro preprocessor BSD licensed. I first worked on it so that imake can work properly without gcc. I discovered that some part of the base system still needs a traditional preprocessor, like (calendar), what I propose it to import tradcpp into the base system (not the version in port right now but what will become version 0.2). It mostly behave like gcpp, and I'm able to properly use calendar along with tradcpp with this small patch: http://people.freebsd.org/~bapt/tradcpp.diff Any objections against me importing it? Looking at the manual page, it looks like that the only reason is to support #include's? I think it would be better to just fix it than importing a new (old) preprocessor... Diane has proposed a patch to go that way: http://www.freebsd.org/cgi/query-pr.cgi?pr=bin%2F178463cat= If one wants to review Bapt pgp_jtHvxMfWp.pgp Description: PGP signature
Re: Fwd: GSOC: Qt front-ends
On Wed, Apr 24, 2013 at 02:03:56PM -0400, Justin Edward Muniz wrote: I think the interface to pkgng and freebsd-update are still interesting; at least more worthwhile than the kernel configuration one. I think the pkgng one has the edge, since packages are updated far more often than base, and it's easier to track base. Now you are at a stage where you should make your own decision; which one looks the most interesting to you? Once you decide on an area of interest, you can just start hacking :) Chris That's good to hear. I am sure that you are right, a pkgng GUI would probably see more use in general. I am definitely close to making my decision, but this thread has been so much help, I am glad for the insight. The coding is what I look forward to the most :D imho a pkgng frontend should be done via packagekit, just write a pkgng backend for packagekit and you will gain for FreeBSD a KDE frontend and a GTK frontend. That said any frontend at convenience to contributor will be anyway good :) regards, Bapt pgpNCIZpQMvf5.pgp Description: PGP signature
Re: Fix overlinking in base aka import pkgconf
On Sat, Dec 15, 2012 at 03:22:34AM +0200, Konstantin Belousov wrote: On Sat, Dec 15, 2012 at 12:54:19AM +0100, Baptiste Daroussin wrote: Hi, Some of our binary are overlinked, the way we handle the linking doesn't help for that. What do you mean there ? Do you mean that some libraries specified for the linking stage of the final binary are not needed for the execution ? I mean some library are registrer in the binary with DT_NEEDED tag where they don't need to. On proposition could be to use pkgconf https://github.com/pkgconf/pkgconf which is BSD license pkg-config implementation 100% compatible with pkg-config. What I propose is to create a new PCADD variable for the Makefiles. PCADD will invoke pkgconf to gather the libraries and the cflags for a given project. The second thing would be to create .pc files for all of our libraries. for example: usr.bin/fstat dynamic build is overlinked And how this is better than just removing the unneeded library from the Makefile ? In that case to statically build you need -lkvm -lutil -lprocstat but if you build dynamically you will only need -lproctast meaning you don't need to be directly linked to libutil and libkvm. This means a breakage of libutil will only have inpact on procstat and no more on fstat for example. For the port consumption, I believe that the better solution is to provide a pack of the .pc files describing base libraries, most likely as port. Yeah the port is another thing which yes can probably be done that way. Using .pc for the base system build is overkill, it does not add anything that cannot be accomplished by our existing build system. IMO. Probably. The thing is with pkgconf, fstat does not need to know that procstat needs libkvm and libutil for static link, it just has to know it needs procstat and pkgconf does all the magic. and pkgconf is really small (only 48k on an amd64 box) Other solution would be to reinvent the same thing using our framework? Maybe a LDSADD (LD STATIC ADD) to differenciate both? regards, Bapt pgp0iaFrpW0w9.pgp Description: PGP signature
Fix overlinking in base aka import pkgconf
Hi,
Some of our binary are overlinked, the way we handle the linking doesn't help
for that.
On proposition could be to use pkgconf https://github.com/pkgconf/pkgconf which
is BSD license pkg-config implementation 100% compatible with pkg-config.
What I propose is to create a new PCADD variable for the Makefiles.
PCADD will invoke pkgconf to gather the libraries and the cflags for a given
project.
The second thing would be to create .pc files for all of our libraries.
for example:
usr.bin/fstat dynamic build is overlinked
With the following simple patch we can solve the problem:
Index: Makefile
===
--- Makefile(revision 243899)
+++ Makefile(working copy)
@@ -5,7 +5,7 @@
SRCS= fstat.c fuser.c main.c
LINKS= ${BINDIR}/fstat ${BINDIR}/fuser
DPADD= ${LIBKVM} ${LIBUTIL} ${LIBPROCSTAT}
-LDADD= -lkvm -lutil -lprocstat
+PCADD= procstat
MAN1= fuser.1 fstat.1
This requires the following .pc files quick and dirty ones:
- util.pc:
prefix=/usr
libdir=${prefix}/lib
includedir=${prefix}/include
Name: util
Libs: -L${libdir} -lutil
Cflags: -I${includedir}
- kvm.pc:
prefix=/usr
libdir=${prefix}/lib
includedir=${prefix}/include
Name: kvm
Libs: -L${libdir} -lkvm
Cflags: -I${includedir}
- procstat.pc:
prefix=/usr
libdir=${prefix}/lib
includedir=${prefix}/include
Name: procstat
Requires.private: kvm util
Libs: -L${libdir} -lprocstat
Cflags: -I${includedir}
The quick and dirty patch for our framework is:
Index: bsd.prog.mk
===
--- bsd.prog.mk (revision 243899)
+++ bsd.prog.mk (working copy)
@@ -36,6 +36,18 @@
LDFLAGS+= -static
.endif
+.if defined(PCADD)
+PCFLAGS!= ${PKGCONF} --cflags ${PCADD}
+.if defined(NO_SHARED) (${NO_SHARED} != no ${NO_SHARED} != NO)
+PCLIBS!= ${PKGCONF} --libs --static ${PCADD}
+.else
+PCLIBS!= ${PKGCONF} --libs ${PCADD}
+.endif
+CFLAGS+= ${PCFLAGS}
+CXXFLAGS+= ${PCFLAGS}
+LDADD+=${PCLIBS}
+.endif
+
.if defined(PROG_CXX)
PROG= ${PROG_CXX}
.endif
Index: sys.mk
===
--- sys.mk (revision 243899)
+++ sys.mk (working copy)
@@ -137,6 +137,8 @@
PC ?= pc
PFLAGS ?=
+PKGCONF?= pkgconf
+
RC ?= f77
RFLAGS ?=
Of course a lot of work is needed to get something really well integrated but
I wanted feedback first before working on anything like this :)
regards,
Bapt
pgpHDhiOQ0RBS.pgp
Description: PGP signature
Re: FreeBSD needs Git to ensure repo integrity [was: 2012 incident]
On Mon, Nov 19, 2012 at 07:08:13PM -0800, Zach Leslie wrote: http://www.fossil-scm.org/ I'm not fossil user, but it's BSD licensed in written in C. Baptise Daroussin probably could tell us more about fossil pro and cons. This misses one of of the main points raised in the original post. The proliferation of git as a revision control system. Also, this particular tool bails out on the unix philosophy, with its web gui, ticket tracker etc. Do one thing. Do it well. Look at the internal of fossil and how things are done in fossil and you would understand that the last sentence is totally wrong. Fossil has really nice features that could nicely fits with FreeBSD workflows and greatly improves it. It has most of the new shiny feature everyone can expect from a dvcs, but it also has it drawbacks: The converted repositories (I did convert docs, src and ports) with full history kept: branches, tags, etc. is huge and the first clone would be painful to do. On the other side you have multiple working copies open on the same clone which is really nice. Some of the operations can be slow, Jörg Sonnenberger wrote an analysis about this one the fossil wiki, but don't remember the link sorry. From my testing, apart from the do we really need a new scm question? I am a big fan of fossil and find it easier and cleaner than all the other scm I know, I use git for pkgng and other projects, I use a lot mercurial on some other area, and fossil remains my favorite :). But I really don't think it could fit FreeBSD's requirements as it is now. but there are lots of room of improvements. The learning curve to fossil is probably really easy. On of the last thing is that fossil lacks keyword expansion. That said I'm happy with svn on FreeBSD, I still from time to time do conversion of out different tree to fossil for fun, but no more and I won't advocate for any vcs change. Bapt pgpCt99BR1uby.pgp Description: PGP signature
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Fri, Oct 26, 2012 at 11:11:52AM -0700, David O'Brien wrote: On Fri, Oct 26, 2012 at 12:12:44AM +0200, Baptiste Daroussin wrote: Do be able to get the ports tree working with bmake asap, I also asked him to MFC it to 9.1, from latest reply he got positive answer from re@ about this, but was waiting for something I don't remember. :tu/:tl is in releng/9.1, so it will also be in 9.1-RELEASE. -- -- David (obr...@freebsd.org) Perfect thanks, I wasn't sure Bapt pgpfjZZjJ4ItK.pgp Description: PGP signature
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Fri, Oct 26, 2012 at 10:02:00PM +0100, Chris Rees wrote:
On 26 Oct 2012 21:51, Simon J. Gerraty s...@juniper.net wrote:
On Fri, 26 Oct 2012 21:00:26 +0100, Chris Rees writes:
:L -- seems that bmake's use for this is kinda pointless; returning the
name of the variable; we could swap that usage over directly.
Acutally it is very useful.
The debugging facilities in dirdeps.mk rely on it.
The junos build uses it in many other places too.
:U -- with bmake has non-optional arguments, so for example:
${VAR:U} - pmake behaviour
${VAR:Uval} - make behaviour.
Would that be acceptable? I can get a patch in if that's popular.
No, please don't do that.
I'm trying to reduce the divergence b/w freebsd and netbsd.
In that case we have a switch time on the order of years, not weeks; 8.3 is
supported until May '14, and unless we get a :tl etc MFC into 8, even
longer. All this time the ports tree must work with pmake.
:tl/:tu has already been MFCed to 8 iirc.
I don't want to discourage you or belittle your excellent work here, but
Marcel made me very nervous with his comment on the process being a few
weeks.
Chris
pgpnZ5uCglcDf.pgp
Description: PGP signature
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Thu, Oct 25, 2012 at 11:01:27PM +0100, Chris Rees wrote: On 25 October 2012 22:32, Garrett Cooper yaneg...@gmail.com wrote: On Thu, Oct 25, 2012 at 2:23 PM, Marcel Moolenaar mar...@xcllnt.net wrote: ... I think there are 2 reasons why not to: 1. The people working on ATF have not raised this concern and have expressed that using the WITH_BMAKE knob is but a small price to pay. So let's work the bmake side and be able to get rid of the knob as soon as possible. It is annoying with the magnitude of build-related errors, but I have a workaround. 2. More knobs isn't better -- we must have none of the knobs in the end, so the more we create, the more work we have to get rid of them. That's just more work spent not focusing on the task at hand and thus more time wasted. Yes, but not being able to update one's machine makes me sad panda. In short: this isn't a 2-knob problem by any stretch of the imagination. The real issue is that I need to take the patch Simon developed, run with it, and in parallel he needs to -- and hopefully already is -- engage portmgr to get it through a number of exp- runs to make sure bmake does what it's supposed to do with his patch. Backwards compatibility will need to be maintained for ports because ports has to work on multiple versions of FreeBSD [where bmake isn't yet available/present], so maybe a fork in the road for bsd.port.mk should be devised in order to make everything work. Now you've terrified me, and probably most other ports people too. Is there a Wiki page where the actual benefits of moving to bmake are made clear? This is a major, *major* upheaval, and having two versions of bsd.port.mk for years is simply not an option. Not much test has been done on the ports tree about it, from what I have tested so far, except from the :tu :tl difference the ports seems to work ootb with both bmake and make, I asked obrien to MFC the support for :tl :tu in make(1) to all available platform which he did. Do be able to get the ports tree working with bmake asap, I also asked him to MFC it to 9.1, from latest reply he got positive answer from re@ about this, but was waiting for something I don't remember. regards, Bapt pgpp7TctkVsBS.pgp Description: PGP signature
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Thu, Oct 25, 2012 at 10:21:59PM +0100, Chris Rees wrote: On 25 October 2012 22:15, David O'Brien obr...@freebsd.org wrote: On Mon, Oct 08, 2012 at 09:11:29AM -0700, Marcel Moolenaar wrote: two independent efforts (ATF bmake) and there was no indication that one would be greatly benefitted from the other. At least not to the point of creating a dependency. It seems we do have the situation where folks feel there is a dependency between the two. Before we can switch permanently to bmake, we need to do the following first: 1. Request an EXP ports build with bmake as make(1). This should tell us the damage of switching to bmake for ports. 2. In parallel with 1: build www docs with bmake and assess the damage 3. Fix all the damage It could be a while (many weeks) before we get to 4, so the question Given the time this will take, I feel we need to add another knob to the Bmake build so that 'make world' gives one both the FreeBSD make as /usr/bin/make and Bmake as /usr/bin/bmake. We really aren't going to have any luck yet... [crees@pegasus]/usr/ports% sudo make MAKE=/usr/bin/bmake index | head Generating INDEX-9 - please wait..bmake: /usr/ports/Mk/bsd.port.mk line 5127: warning: duplicate script for target -depends ignored bmake: /usr/ports/Mk/bsd.port.mk line 5124: warning: using previous script for -depends defined here bmake: /usr/ports/Mk/bsd.port.mk line 5127: warning: duplicate script for target -depends ignored bmake: /usr/ports/Mk/bsd.port.mk line 5124: warning: using previous script for -depends defined here bmake: /usr/ports/Mk/bsd.port.mk line 5127: warning: duplicate script for target -depends ignored bmake: /usr/ports/Mk/bsd.port.mk line 5124: warning: using previous script for -depends defined here bmake: /usr/ports/Mk/bsd.port.mk line 5124: warning: duplicate script for target -depends ignored bmake: /usr/ports/Mk/bsd.port.mk line 5124: warning: using previous script for -depends defined here bmake: /usr/ports/Mk/bsd.port.mk line 5124: warning: duplicate script for target -depends ignored bmake: /usr/ports/Mk/bsd.port.mk line 5124: warning: using previous script for -depends defined here Looks like a few missing .if !target s, but the breakage is pretty big even for simple things :/ Have you converted the :U to :tu and :L to :tl? regards, Bapt pgpGGmrYitgPo.pgp Description: PGP signature
Playing with include-what-you-use shows interesting stuff
Hi, I've been playing with the include-what-you-use[1] llvm tool for some on my personnal projects, as it works very well, I have also played with it on our source tree starting with the bin directory. It shows some interesting results, while the default output is quite aggressive, I just chose to remove the useless headers in each sources. It show some interesting results which seems to come from maybe bad includes in some of our headers. Apparently some of the #include sys/param.h are false positive because others headers shouldn't include it for example (according to des) here is a diff showing what I find that can be removed: http://people.freebsd.org/~bapt/include-what-you-use.diff I think it shouldn't be applied as it but be more analyzed. regards, Bapt pgpVqC7bBkTJA.pgp Description: PGP signature
flex or reflex
Hi, I am willing to update our flex in base, my first motivation is to be able to have reentrant lexer in base, I first went to the http://flex.sourceforge.net derivative from flex 2.5.4, I've imported it in contrib, and I'm able to build the whole base using the 2.5.35 version (almost vanilla) and with just one or two small fixes from from .l files (mostly adding %option nounistd to fix warnings) One of the major problem of this version is that it uses m4 (it is compatible with our m4 version in base - the recently updated one). Another alternative is to use reflex (http://www.invisible-island.net/reflex/reflex.html) which seems a good one because, it is more respectful of the POSIX lex unfortunately it doesn't seem to be able to create reentrant lexer. Given this, I think it is better for us to choose flex. Of course it is still possible to add reentrant feature to our flex, but it would be more painful. After this I plan to import byacc http://www.invisible-island.net/byacc/byacc.html which can generate reentrant parser. regards, Bapt pgpjmD50DkxMC.pgp Description: PGP signature
Re: iotop (dtrace?)
On Tue, Oct 25, 2011 at 10:34:39PM +0200, Stefan Bethke wrote: I've got two systems with a constantly high rate of disk I/O that sometimes seems to be overwhelmed from it. Before trying to decide if a hardware upgrade will help, I'd like to figure out which processes generate the load. I've found a couple scripts named iotop which appear to produce what I would be interested in, but they appear to require Solaris or Linux. Has someone ported over one of them, or would have a suggestion how to go about writing a custom dtrace script to gather this kind of information? I can successfully run a couple of sample dtrace scripts on these 8-stable amd64 boxes. Can't 'top -mio' do the job? regards, Bapt pgpDBtL848D9o.pgp Description: PGP signature
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
2011/3/31 Andriy Gapon a...@freebsd.org: on 29/03/2011 23:29 Baptiste Daroussin said the following: ok let's try to say it simpler :) the main goal is to keep it simple for now, simple and rock solid, so that we can replace pkg_install and do some cleanup in the ports tree, add the must have features while doing that. And only when we will be ready for that and that portmgr have decided that it is mature enough to replace pkg_install, only after that we will start improving with new features and new changes. I thinks changing the package name scheme is not a must have feature, it for sure is and intresting feature, but what about pushing to after the first stable release? managing architecture as we plan to do it is enough imho. Oh, yes, I realize all this and totally agree with it. Given how huge and how visible our ports and packages systems are, it's better to be slow and cautious. All the ideas that I suggested were more for the next step than for now. And noted in my personnal TODO list :) Thank you for the work! -- Andriy Gapon ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
2011/3/29 Andriy Gapon a...@freebsd.org: on 28/03/2011 21:22 Julien Laffaye said the following: On Mon, Mar 28, 2011 at 6:59 PM, Garrett Cooper gcoo...@freebsd.org wrote: III. Package naming that includes architecture, major OS version (for API/ABI), maybe more. This could be provided in the manifest. Doing it in the filename sort of turns into a mess, as I've discovered working at Cisco :). Actually, it *is* in the +MANIFEST of pkgng packages archives :-) Well, by the package name I meant not only a package file name. Let's imagine that we do support installing i386 packages on amd64 in parallel to amd64 packages. And for some reason I want to have both 32-bit and 64-bit versions of, say, firefox; e.g. for benchmarking. If the packages would have the same name, then that would be impossible. I think that having some thing in package name in addition to package metadata could have certain benefits. -- Andriy Gapon I understand but I think pkgng is already quite radical changement. More change is taking the risk that it would be rejected in the end, we still do not have any reply from portmgr, there is no insurance pkgng will in the end replace pkg_install. Currently pkgng requires only very few changes from the ports infrastruture, I don't know the cost of changing the name scheme. If I'm not clear enough, supporting both 32bits and 64bits packages at the same time on amd64 or arches that could support this kind of installation, is a large change we don't want to take the responsability of :) and implementing this in pkgng would significate we already choose how it should work. regards, Bapt ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
2011/3/29 Baptiste Daroussin b...@freebsd.org: 2011/3/29 Andriy Gapon a...@freebsd.org: on 28/03/2011 21:22 Julien Laffaye said the following: On Mon, Mar 28, 2011 at 6:59 PM, Garrett Cooper gcoo...@freebsd.org wrote: III. Package naming that includes architecture, major OS version (for API/ABI), maybe more. This could be provided in the manifest. Doing it in the filename sort of turns into a mess, as I've discovered working at Cisco :). Actually, it *is* in the +MANIFEST of pkgng packages archives :-) Well, by the package name I meant not only a package file name. Let's imagine that we do support installing i386 packages on amd64 in parallel to amd64 packages. And for some reason I want to have both 32-bit and 64-bit versions of, say, firefox; e.g. for benchmarking. If the packages would have the same name, then that would be impossible. I think that having some thing in package name in addition to package metadata could have certain benefits. -- Andriy Gapon I understand but I think pkgng is already quite radical changement. More change is taking the risk that it would be rejected in the end, we still do not have any reply from portmgr, there is no insurance pkgng will in the end replace pkg_install. Currently pkgng requires only very few changes from the ports infrastruture, I don't know the cost of changing the name scheme. If I'm not clear enough, supporting both 32bits and 64bits packages at the same time on amd64 or arches that could support this kind of installation, is a large change we don't want to take the responsability of :) and implementing this in pkgng would significate we already choose how it should work. regards, Bapt seems it was not clear :) ok let's try to say it simpler :) the main goal is to keep it simple for now, simple and rock solid, so that we can replace pkg_install and do some cleanup in the ports tree, add the must have features while doing that. And only when we will be ready for that and that portmgr have decided that it is mature enough to replace pkg_install, only after that we will start improving with new features and new changes. I thinks changing the package name scheme is not a must have feature, it for sure is and intresting feature, but what about pushing to after the first stable release? managing architecture as we plan to do it is enough imho. But I can be wrong. regards, Bapt ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
2011/3/29 Tim Kientzle kient...@freebsd.org: II. Package signing. That would be really nice. Right know we only planned to sign the repo database, so we can trust the sah256 of the packages stored in the database. Then if the package has the same sha256 as the one in the repo database it is considered trusted. If we want a per-package signing, we would have a tarball in a tarball. I really expected this to have been mentioned already, but this approach (tarball in a tarball) is taken by Debian packages, and I don't remember hearing of any issues related to it. I don't think it's worth discounting from the start without giving some considerationg, but I will defer to the people actually doing the work. If you use libarchive-style streaming, it's even pretty straightforward to read and extract such things without having to create a bunch of temporary files. You just need to be careful about compression. Tim ok but what is the problem with signing only the repository then rely on digest? I am not sure we need more that this. second question howto sign? pgp? ssl? First would be the easiest way to go but we don't have in base anything to check signatures (maybe we should in that case investigating to import netpgp), ssl why not? but which algorithm? what security officer would prefer? We are ok to investigate that part, but we need more information about what is expected. regards, Bapt ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
2011/3/26 Mike Meyer m...@mired.org: On Fri, 25 Mar 2011 15:14:52 +0100 Baptiste Daroussin b...@freebsd.org wrote: 2011/3/25 Alexander Leidinger alexan...@leidinger.net: - the register command can analyse elf files when registering a new port to discover forgotten dependencies if necessary. (done in alpha using libelf) This will probably fail if LD_LIBRARY_PATH is used, or if we are installing linuxulator ports. this isn't activated by default, and if activated is only intended to work on freebsd elf files. This is done to workaround some bugguy ports not to be used in production, pkg register shows in warning in that case so that user/maintainers are warned they have something to fix. How about dealing with 32-bit x86 packages on an amd64 install? mike -- Mike Meyer m...@mired.org http://www.mired.org/consulting.html Independent Software developer/SCM consultant, email for more information. O ascii ribbon campaign - stop html mail - www.asciiribbon.org ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org 32bits x86 packages on amd64 simply are ignored by the elf introspection (currently) ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
[ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
Hi all, miwi@ launched the new thing called Experimental Call For Testing, it's our turn :) Julien Laffaye (jlaffaye@) and I, helped by Philippe Pepiot (huge contributor) have been working since the end of the last GSoC on a rewrite of pkg_install. pkgng is a binary package manager written from scratch for FreeBSD. After a long period of technology testing, (json, tinycdb, bdb, etc) and we now have achieved to implement the basic functionnality. We would greatly approciate to have some feedback, wider testing, patching, documenting etc, before implementing the higher level features. pkgng is built on top of a new libpkg, which allow to deal with the database of installed packages, to deal with remote repositories, manage packages: creation, installation gathering informations, registering new ports. features supported are or will be : - smooth integration with bsd.port.mk (including bsd.pkg.mk line 2486) which allow to have a bsd.port.mk which deal with both pkg_install and pkgng. (done in alpha) - the register command can analyse elf files when registering a new port to discover forgotten dependencies if necessary. (done in alpha using libelf) - the register command has two mode available : when dealing with old fashion ports it just registers the package, in new mode it does everything that would have been done by pkg add when installing the package : should display messages, execute post-install, execute @exec etc. (old fashion done in the alpha) - pkg add supports two mode : the old fashion one (no real upgrade support) and new one: upgrade scripts supported. (old fashion in the alpha) - new scripts supported +PREINSTALL +POSTINSTALL, +PREDEINSTALL, +POSTDEINSTALL, +PREUPGRADE, +POSTUPGRADE as well as the old fashion scripts : +INSTALL +DEINSTALL +UPGRADE (all supported *UPGRADES aren't supported in the alpha) - new +MANIFEST (plist-like format) with new metadatas : options, arch, os version, etc. (done in the alpha) - pkgng supports checking arch of the package which means that users won't be able to install sparc64 binary package into amd64 machines. (not done yet) - a special architecture all allows to specify when a package can be used on every architecture. (not done yet) - @dirrm and @dirrmtry are now deprecated, pkgng can discover itself which directory has to be removed. (done in the alpha but needs love :)) - new repository (apt-like feature) (only the repository generation is done) - real support for reverse dependency (no ugly +REQUIRED_BY) (done in the alpha) - test unit (libcheck) on libpkg. (done in the alpha needs some more love) - many more In term of technology we decided to use a sqlite3 database, and to prevent potential trolling, sqlite3 is used in it's amalgamation form which means it is incorporated in the code sources (as recommanded by sqlite developpers like a statically linked library) on build we only activate the features we need in sqlite. The alpha release come with an experimental tool pkg2ng to convert an existing package database to the new pkgng database format. So one can test pkgng without rebuild all its packages. One of the thing we are thinking about pkgng is to perhaps be able to provide it only as a ports (with simple script in base to boostrap/install it). That would allow pkgng to live with the ports to be able to easily integrate new features without having to support very old version of pkgng. design: pkgng is composed of : - a clean library libpkg that does all the work - a modern cli frontend (pkg) which accept subcommands, basically type pkg add, pkg info, pkg create etc. a dedicated subcommand exists for ports: pkg register which goal is to only supported adding to the database what is already installed. more informations can be found here: http://git.etoilebsd.net/pkgng/tree/docs/GOALS, http://git.etoilebsd.net/pkgng/tree/README http://git.etoilebsd.net/pkgng/tree/docs/TODO To download the alpha: http://git.etoilebsd.net/pkgng/snapshot/pkgng-0.1-alpha1.tar.gz Build it with debugging information: make DEBUG_FLAGS=-g -O0 -DDEBUG Developpement site: http://git.etoilebsd.net/pkgng/ IRC chan: #pkgng@freenode Beware that pkgng is in alpha states, it can kill kittens and eat puppies, and for sure it will do it so now you are warned. regards, bapt, pgpbgmBRFdsib.pgp Description: PGP signature
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
2011/3/25 Alexander Leidinger alexan...@leidinger.net: Quoting Baptiste Daroussin b...@freebsd.org (from Fri, 25 Mar 2011 11:11:11 +0100): pkgng is a binary package manager written from scratch for FreeBSD. I didn't had a look at it, just some comments about some parts you explained. features supported are or will be : - the register command can analyse elf files when registering a new port to discover forgotten dependencies if necessary. (done in alpha using libelf) This will probably fail if LD_LIBRARY_PATH is used, or if we are installing linuxulator ports. this isn't activated by default, and if activated is only intended to work on freebsd elf files. This is done to workaround some bugguy ports not to be used in production, pkg register shows in warning in that case so that user/maintainers are warned they have something to fix. - a special architecture all allows to specify when a package can be used on every architecture. (not done yet) What if a package is able to install on a subset, e.g. the linuxulator ports are for amd64 and i386? No clue for that at the moment but we are open to suggestions. What about DB corruption/loss? Do you keep the /var/db/pkg/package/xxx files even with pkgng and only use the DB as a way to speed up some work (so the DB corruption just requires to run pkg2ng), or are you lost of the DB is lost? Nothing is done about DB corruption/loss, I am not sure we need to do something. Maybe. Currently a filesystem corruption/loss on /var/db/pkg would do the same. but it is sqlite so we can perhaps provide a way to get compressed dump so user can periodically backup their database. regards, Bapt ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
2011/3/25 Pietro Cerutti g...@freebsd.org: On 2011-Mar-25, 15:03, Julien Laffaye wrote: What about DB corruption/loss? Do you keep the /var/db/pkg/package/xxx files even with pkgng and only use the DB as a way to speed up some work (so the DB corruption just requires to run pkg2ng), or are you lost of the DB is lost? Nothing is done about DB corruption/loss, I am not sure we need to do something. Maybe. I would say for sure. Info: In Solaris 10 sqlite is used for the service managenemt framework (SMF). It is possible that the DB is corrupt in some bad situations. In this case you have to rebuild the DB (script provided, been there, had to use it). If sqlite is properly used with transactions, it is very hard to corrupt the database. But if hardware lies to us and say that the data is on disk whereas it isnt... what can we do? Another potential problem is fsync(), but if it is broken on FreeBSD we want to fix it! BTW, the goal is to only have the database and not the flat files. If you are paranoid about power outage, use something like zfs snapshots... No need to look for strange scenarios, I'm surely going to sudo rm -f the file more sooner than later, so... maybe just save a copy? -- Pietro Cerutti The FreeBSD Project g...@freebsd.org PGP Public Key: http://gahr.ch/pgp I think we can provide a periodic script activable by users (I let other decide if it has to be activated by default or not) that does a pkg backup /path/to/file/backup and xz it. because copying can be a huge. 40Mo for the database here, corresponding to 70Mo in the old format and to 600 packages. the dump xzed is only 3Mo regards, Bapt ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install
2011/3/25 Yuri y...@rawbw.com: On 03/25/2011 03:11, Baptiste Daroussin wrote: Julien Laffaye (jlaffaye@) and I, helped by Philippe Pepiot (huge contributor) have been working since the end of the last GSoC on a rewrite of pkg_install. pkgng is a binary package manager written from scratch for FreeBSD. How does it relate to portmaster and portupgrade packages, which both have (or include) supposedly the same functionality? Yuri both have to be adapted, portupgrade throught maybe some ruby bindings to libpkg, portmaster by patching it to use pkg frontend instead of pkg_* tools (as I did for the ports (see ports/bsd.pkgng.mk in the git tree) regards, Bapt ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
[PATCH] sync libedit with netbsd
Hi all, Here is a new version of the patch, this time it only upgrades libedit without installing the readline compatibility headers (as I have been suggested for a first step) Their should be no regression with this patch. If it gets committed I'll send the FreeBSD's enhancement to upstream so that we can keep sources in sync. http://people.freebsd.org/~bapt/libedit-netbsd-sync.patch regards, Bapt ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [PATCH] update to the latest libedit version and remove libreadline deps
Yet another version of the patch, I hope the last one http://people.freebsd.org/~bapt/update-libedit.patch Everything should be working as it used to do before. Now gdbtui is almost working. why almost because everything works except Ctrl-D (EOF), I know where the bug is (lib/libedit/read.c : function: FUN(el,gets)(EditLine *el, int *nread), the read pb is in readchar I guess) but I can't find a way to fix it and it seems to me a really minor regression. I think we can live with this as this bug appear only when libedit doesn't directly gets its input but get is through a third party interface (ncurses in that case) and gdbtui is the only program in base working like this. Also thanks for the information about exp-run, if anyone is about to accept this patch and wanted to commit it, I'll follow those informations. regards, Bapt ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [PATCH] update to the latest libedit version and remove libreadline deps
Thanks all for your returns, I'll update my patch during the next week. Concerning the reverts I'll try to reintegrate them and then send them to upstream, Because I think it is better to keep in sync to easier futures updates. regards, Bapt ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
[PATCH] update to the latest libedit version and remove libreadline deps
Hi all, I've updated libedit to the latest version available in the netbsd cvs. UTF8 support is disabled for now has it seems to be experimental and segfault. I also patch and tested all the sources that used to be linked against libreadline so that it now uses libedit making libreadline unused (I guess, perhaps I have missed some) beware that there are collision between libreadline and libedit (/usr/include/readline/readline.h) is provided by both of them. You can find the patch against current here: http://people.freebsd.org/~bapt/update-libedit.patch regards, Bapt ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
