Re: Uses/compiler.mk does not trigger under RBPi

[Tim Kientzle]

What specific ports are you having trouble with?

Tim


> On Oct 2, 2015, at 2:56 PM, José Pérez  wrote:
> 
> Hi Michelle,
> thank you for your suggestion. Is this another workaround?
> 
> I mean: many port Makefiles are affected, in the sense that when built on 
> Intel/AMD the ports just work because Uses/compiler.mk is sucked in 
> automatically, but it is not on ARM.
> 
> So, shall I report a bug on all the ports that use COMPILER_TYPE, or is there 
> a way to have ARM trigger Uses/compiler.mk?
> 
> Thank you.
> 
> Regards,
> 
> 
> ---
> José Pérez
> 
> El 2015-10-02 02:29, Michelle Sullivan escribió:
>> José Pérez wrote:
>>> Hi,
>>> I've notice that Uses/compiler.mk is not triggered and as a
>>> consequence does to set COMPILER_TYPE.
>>> me@raspberry-pi:~ % cat Makefile
>>> all:
>>>@${ECHO_CMD} ${LOCALBASE}
>>>@${ECHO_CMD} ${COMPILER_TYPE}
>>> .include 
>>> me@raspberry-pi:~ % make
>>> /usr/local
>>> me@raspberry-pi:~ %
>>> As a result building ports is a nightmare.
>>> Note how in AMD64 it works:
>>> me@amd64:~ % make
>>> /usr/local
>>> clang
>>> me@amd64:~ %
>>> As a workaround I set COMPILER_TYPE=clang in /etc/make.conf but this
>>> is just an ugly hack.
>>> Can some expert trow a little light on this? Thank you.
>>> Regards,
>> Try adding:
>> USES+=   compiler
>> to the makefile first...
>> (and it still has some issues but that should solve the first)
> ___
> freebsd-...@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-arm
> To unsubscribe, send any mail to "freebsd-arm-unsubscr...@freebsd.org"

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: devel/gobject-introspection failure on ARM

[Tim Kientzle]

On Jan 27, 2013, at 7:57 AM, George Mitchell wrote:

 System: Raspberry Pi
 uname: r245840M (Alie Tan's image from 25 January)
 ports: svnversion 308518
 
 Build dies with message sizeof(ArrayTypeBlob) is expected to be 8 but
 is 12.  (Complete build log attached.)  I made a naive attempt to fix
 it by rearranging the order of the structure members, but obviously I
 don't understand structure packing on the ARM and it didn't help.

The easiest way to hack around this is usually to
sprinkle packed decorators on a lot of structure
definitions.

  It also didn't get rid of the huge number of cast increases required
 alignment of target type warnings.

How troublesome these are depends on the processor.
I think the ARMv6 on the RaspberryPi is late enough to
support misaligned accesses.  It's a big performance hit
but better than crashing.

 I note we're at version 0.10.8 of this package, but upstream is at
 1.34.2.  (It requires glib 2.34.1, though, and we're only at 2.28.8).
 
 What's the best way to proceed?  

Given the version numbers you quote, I expect that
a newer glib would be a good start.

Good luck,

Tim

___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: rtld or lang/gcc cannot find libgcc_s.so.1

[Tim Kientzle]

On Feb 23, 2012, at 9:16 AM, Alexander Kabaev wrote:

 On Tue, 21 Feb 2012 21:11:13 -0800
 Tim Kientzle t...@kientzle.com wrote:
 
 
 If I understand correctly, the libgcc in base is pretty stripped
 down compared to regular libgcc, because most of that
 stuff is in our libc instead. 
 
 
 You understand it a bit wrong, but your conclusions are correct. libgcc
 in base is not stripped in any way and is supposed to be identical to
 one coming from upstream.

So where is __umodsi3 supposed to be defined for ARM?

In FreeBSD, libgcc refers to it but does not define it.
It's defined in libc.

I stumbled across this trying to link some freestanding
ARM code using the native cross-compilers.  The link
failed if I used -nostdlib because of a handful of symbols
such as this.

Tim

___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: rtld or lang/gcc cannot find libgcc_s.so.1

[Tim Kientzle]

On Feb 21, 2012, at 3:39 PM, Daniel Eischen wrote:

 On Tue, 21 Feb 2012, Steve Kargl wrote:
 
 3) Add a new option to ldconfig to prepend new libraries to
  the hints files and fix the ports to use this option instead
  of -m.
 
 You don't want system binaries that want /lib/libgcc_s.so.1
 to use /usr/local/lib/gccXX/libgcc_s.so.1, though.  Wouldn't
 your option 3 do that?

Why not?  Would it cause problems?

Is libgcc from GCC 4.6 incompatible with /lib/libgcc?

If I understand correctly, the libgcc in base is pretty stripped
down compared to regular libgcc, because most of that
stuff is in our libc instead.  So if there were compatibility
problems, I'd expect those to show up when GCC 4.6 linked
programs against /usr/local/.../libgcc and /lib/libc.

Tim

___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: [ECFT] pkgng 0.1-alpha1: a replacement for pkg_install

[Tim Kientzle]

 II. Package signing.
 
 That would be really nice.
 
 Right know we only planned to sign the repo database, so we can trust
 the sah256 of the packages stored in the database. Then if the package
 has the same sha256 as the one in the repo database it is considered
 trusted.
 If we want a per-package signing, we would have a tarball in a tarball.
 
 I really expected this to have been mentioned already, but this approach 
 (tarball in a tarball) is taken by Debian packages, and I don't remember 
 hearing of any issues related to it.  I don't think it's worth discounting 
 from the start without giving some considerationg, but I will defer to the 
 people actually doing the work.

If you use libarchive-style streaming, it's even
pretty straightforward to read and extract such
things without having to create a bunch of
temporary files.

You just need to be careful about compression.

Tim

___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: what next for the pkg_install rewrite

[Tim Kientzle]

On Sep 2, 2010, at 12:34 AM, David Forsythe wrote:
 In the mean time, I'll keep building libpkg too work with the pkg
 system in its current state.

Any really viable plan to change the manifest structure
will have to include a means to continue to support the
old structure during the transition period.

Tim

___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: what next for the pkg_install rewrite

[Tim Kientzle]

On Sep 2, 2010, at 12:34 AM, David Forsythe wrote:

 Separating ports and packages is silly, because they need to coexist.
 Like gcooper pointed out, ports should be using the pkg tools to build
 and install packages.

I've been impressed with how MacPorts handles this.
If I understand correctly, you can specify a package you
want and the options you want with it and the system will
download a pre-built package with those options
(if one exists) or pull the source and  build the port,
all transparently.

Even better, the central repository sees all of these
requests, so can prioritize which option sets are
most popular.

Tim

___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: ports and PBIs

[Tim Kientzle]

Garrett Cooper wrote:

If I'm understanding you correctly you're saying it's an issue when I do:

pkg_add A B C

# 1 year passes

pkg_add D

# D depends on A, B, C, of different revisions. pkg_add barfs because
it can't find the applications, etc.

This is something that's been hashed over a number of times (a few of
which I've participated in in #bsdports). There needs to be a simple
update command which will handle the action of upgrading packages,
because there isn't a proper command that will do so today.


I'm not convinced that the simple update command you
mention is actually feasible, much less desirable.
(If I want to try out the new Firefox, why does that
imply that my year-old Gimp has to be upgraded?)

As for feasibility, here's the easy problem:
   A2.7 requires B3.6
 ... one year passes ...
   A4.8 now requires B7.2
But A4.8 is incompatible with B3.6 and A2.7 is
incompatible with B7.2.  So neither A nor B
can be updated separately without breaking the system.

Here's the hard problem:
   A2.7 requires B3.6
 ... one year passes ...
   I want to install C1.0 which requires B7.2
   but there hasn't been a new release of A that
   works with B7.2.
So I now simply cannot have both C1.0 and A2.7
installed at the same time because they require
different versions of B.

PBI avoids both of these problems.  It may
be unsuitable for embedded systems[1], but
I see no reason we should not extend the existing
ports/packages system with additional tools that
target certain use cases, and PBI seems a good
fit for the desktop case.

Tim

[1] Actually, PBI might work just fine even for
embedded if we address the disk bloat issue.  One
approach would be to make
   /Package/Bar/libfoo-2.8.7.so
a symlink or hardlink to
   /Package/Shared/libfoo-2.8.7.so-MD5-hash
This gives easy sharing of identical files.
It's even easy to handle at install time:
  * Installer writes libfoo-2.8.7.so to
 /Package/Shared/libfoo-2.8.7.so-temp-PID of installer
  * Installer computes hash of file as it's written
  * Installer renames file (delete if rename fails with EEXIST)
  * Installer writes symlink or hardlink into /Package/Bar

___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: ports and PBIs

[Tim Kientzle]

Julian Elischer wrote:

On 4/10/10 12:07 PM, Tim Kientzle wrote:

[1] Actually, PBI might work just fine even for
embedded if we address the disk bloat issue. One
approach would be to make
/Package/Bar/libfoo-2.8.7.so
a symlink or hardlink to
/Package/Shared/libfoo-2.8.7.so-MD5-hash
This gives easy sharing of identical files.


yeah that's more or less what we were thinking..
hardlinks allow you to garbage collect when the last pbi that needs 
something is replaced/removed.


The point of /Package/Shared in this design is
basically that it provides a list of all of
the files that can be shared, so you
avoid doing a full disk search to identify other
places that might have this file.  You could
accomplish the same goal by building and
storing a database of sharable files somewhere,
of course.

(Curiously, no one has mentioned filesystem-level
deduping yet as the big hammer solution...  ;-)

The LD_LIBRARY_PATH issue is the most interesting
problem here.  I don't immediately see a solution that
doesn't include teaching ld-elf.so.1 about some form
of per-application library path.

Tim
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: ports and PBIs

[Tim Kientzle]

Sam Fourman Jr. wrote:

I do have a question, assuming PBI's were merged officially into the
FreeBSD ports tree,
say I had PostgreSQL Server installed, via PBI. then I wanted to tweak
a setting so I:

cd /usr/ports/databases/postgresql84-server/  make deinstall clean

would the PBI at this point be removed? or no because it is self contained?


Basically, I believe the proposal here is to add:
  * make pbi
to the ports build system to create a PBI from a
port and possibly add
  * make installpbi
  * make deinstallpbi
to install/deinstall just the resulting PBI.

In particular, I don't think anyone is suggesting
removing or changing any existing ports/package
capability.  People who are happy with the existing
ports/package system could continue using it exactly
as-is.

This would imply that you might build Postgres
and install it both as a port/package and simultaneously
as a PBI.  I'm not sure what that would mean, though.

The big question, of course: what impact would the
addition of make pbi have on existing port/package
support efforts?  Is this creating extra work for
existing maintainers?  Should it be optional
(enabled per-port) or somehow default?

I suspect the next step is for someone to put forward
a proposed implementation of make pbi so those
interested can start trying it out and see what the
impacts are.  (If only the GSoC proposal deadline
hadn't already passed. ;-)

Tim

___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: ports and PBIs

[Tim Kientzle]

per...@pluto.rain.com wrote:

[dropped current@ since it doesn't take non-member posts]

Tim Kientzle kient...@freebsd.org wrote:


The LD_LIBRARY_PATH issue is the most interesting
problem here.  I don't immediately see a solution that
doesn't include teaching ld-elf.so.1 about some form
of per-application library path.


Maybe install PBI executables with a wrapper of some
sort which would set LD_LIBRARY_PATH and exec the
real executable...


PBIs already do something like this, but this
doesn't work for setuid/setgid executables.

Tim
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: Finding slowdowns in pkg_install (continuations of previous threads)

[Tim Kientzle]

4. CSV files available at: 
http://students.washington.edu/youshi10/posted/atk-results.tgz.


I've posted HTML results of the interpreted spreadsheet on 
http://students.washington.edu/posted/atk.htm. I'll provide commentary 
tomorrow after I get some sleep.


I think the second one should be:
http://students.washington.edu/youshi10/posted/atk.htm

Unfortunately, I get Permission Denied here for both
of those URLs.

Tim Kientzle
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]

Re: Finding slowdowns in pkg_install (continuations of previous threads)

[Tim Kientzle]

   The following blog post has all of my commentary on the results I 
have: 
http://blogs.freebsdish.org/gcooper/2007/07/14/modifications-to-pkg_install-the-positive-and-negative-implications/. 



I tried to unroll strcmp a bit by checking for the first character of the

 command, then run strcmp ...

There's a somewhat more straightforward optimization that
relies on this same idea:

switch(cmd[0]) {
case 'c':
/* Commands that start with 'c' */
if (strcmp(cmd, 'cwd') == 0)
return (CMD_CWD);
/* FALLTHROUGH */
case 'd':
/* Commands that start with 'd' */

 etc
/* FALLTHROUGH */
default:
/* Unrecognized command. */
}

This is a little cleaner and easier to read
and may even be faster than the code you
presented in your blog.  Note that the fall through
ensures that all unrecognized commands end up at
the same place.  If unrecognized commands are
very rare (they should be), then the fallthrough
is not a performance issue.


/** malloc buffer large enough to hold +CONTENTS **/

while(!feof(file_p)) {

/** add content via fgetc **/
}


Yuck.  Try this instead:

   struct stat st;
   int fd;
   char *buff;

   fd = open(file);
   fstat(fd, st);
   buff = malloc(st.st_size + 1);
   read(fd, buff, st.st_size);
   buff[st.st_size] = '\0';
   close(fd);

Plus some error checking, of course.  You can
use stdio if you prefer:

   FILE *f;

   f = fopen(file, r);
   fstat(fileno(f), st);
   buff = malloc(st.st_size + 1);
   fread(buff, 1, st.st_size, f);
   buff[st.st_size] = '\0';
   fclose(f);

Either way, this is a lot more efficient than
tens of thousands of calls to fgetc().

Cheers,

Tim Kientzle
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]

Re: Finding slowdowns in pkg_install (continuations of previous threads)

[Tim Kientzle]

   I'm currently running a gamut of tests (500 tests, per package -- 
128 total on my server), and outputting all data to CSV files to 
interpret later, using another Perl script to interpret calculated 
averages and standard deviations.


Excellent!  Much-needed work.

   Using basic printf(2)'s with clock_gettime(2) I have determined 
that the majority of the issues are disk-bound (as Tom Kientzle put 
it).


Next question:  What are those disk operations and are any
of them redundant?


The scope of my problem is not to analyze tar,...


I've spent the last three years+ doing exactly that.
Make sure you're using the newest bsdtar/libarchive,
which has some very noticable performance improvements.

but I've 
discovered that a lot of time is spent in reading and interpreting the 
+CONTENTS and related files (most notably in parsing commands to be 
honest).


Oh?  That's interesting.  Is data being re-parsed (in which case
some structural changes to parse it once and store the results
may help)?  Or is the parser just slow?

   Will post more conclusive results tomorrow once all of my results 
are available.


I don't follow ports@ so didn't see these conclusive results
of yours.  I'm very interested, though.

Tim Kientzle
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]

Re: Finding slowdowns in pkg_install (continuations of previous threads)

[Tim Kientzle]

   -I tried ... buffering ...  the +CONTENTS file parsing function, and the
majority of the time it yielded good results 


One approach I prototyped sometime back was to use
libarchive in pkg_add as follows:
  * Open the archive
  * Read +CONTENTS directly into memory (it's
guaranteed to always be first in the archive)
  * Parse all of +CONTENTS at once
  * Continue scanning the archive, disposing
of each file as it appears in the archive.

Based on my experience with this, I would
suggest you just read all of +CONTENTS
directly into memory at once and parse
the whole thing in a single shot.
fopen(), then fstat() to get the size,
then allocate a buffer and read the whole
thing, then fclose().  You can then
parse it all at once.

As a bonus, your parser then becomes a nice
little bit of reusable code that reads
a block of memory and returns a structure describing
the package metadata.

Tim Kientzle
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]

Re: bsdtar and packages vs. unionfs

[Tim Kientzle]

Ulrich Spoerlein wrote:

Tim Kientzle wrote:

The way I see it, bsdtar(1) extracts the symlink libcharset.so, and then
tries to stat(2) instead of lstat(2) it, before libcharset.so.1 is
extracted. The questions is: why?


Oh.  I see.  *That* old bug.  sigh  Try this patch:

Index: archive_read_extract.c
===
--- archive_read_extract.c  (revision 177)
+++ archive_read_extract.c  (working copy)
@@ -1243,7 +1243,7 @@
/* Already have stat() data available. */
} else if (fd = 0  fstat(fd, extract-st) == 0)
extract-pst = extract-st;
-   else if (stat(name, extract-st) == 0)
+   else if (lstat(name, extract-st) == 0)
extract-pst = extract-st;
else {
archive_set_error(a, errno,
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]

Re: Cannot package converters/libiconv inside clean chroot

[Tim Kientzle]

There are at least two issues here, one is pkg_add refusing a valid
(AFAICS) tbz file, the other is bsdtar(1) choosing a different tar
format based on unionfs(!).


Those two symlink entries have an opaque file flag.
This explains the format change (bsdtar uses the extended
pax format when it sees a file with flags set, since
ustar can't store those).

I would guess that pkg_add is invoking bsdtar with
-p (restore permissions), bsdtar is restoring the
'opaque' flag, and then pkg_add is tripping over
those symlinks for some reason when it tries to
move them.

To test this hypothesis, try stripping those flags
with:

   tar -cjf new-libiconv-1.9.2_2.tbz --format=ustar @libiconv-1.9.2_2.tbz

The new-libiconv tarfile should be identical except
it won't have the file flags stored.  If pkg_add
likes new-libiconv but not libiconv, then it must
be those opaque flags.

I don't know if this is a bug in unionfs, in pkg_add,
or in bsdtar.  I think we need someone who understands
the 'opaque' flag to chime in here.

Tim
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]

Re: Cannot package converters/libiconv inside clean chroot

[Tim Kientzle]

Ulrich Spoerlein wrote:

While I can pkg_delete/pkg_add the libtool15 package at will, the
libiconv package fails to pkg_add because bsdtar(1) is whining:

roadrunner# pkg_add -v /usr/ports/packages/All/libiconv-1.9.2_2.tbz 
lib/libcharset.so: Couldn't stat file: No such file or directory

lib/libiconv.so: Couldn't stat file: No such file or directory

It doesn't matter if I 'make package' or 'pkg_create -b' the package. I
can extract it just fine using the cmd line tar(1) though.

roadrunner# tar tvvf /usr/ports/packages/All/libiconv-1.9.2_2.tbz 


tar xvf works as well as tar tvf?  It sounds like bsdtar is
refusing to recreate the symlink on disk.


What do the '1' in the second column mean? The manpage just refers to
ls(1). If it is the inode link count, how can most of the entries be
zero?


Most tar entries don't store a link count, so it shows up
as zero.  Apparently, the two symlink entries are getting
stored with pax format for some reason.  (The 'pax' format
is an extension to 'ustar'; a single archive can mix entries
in different formats.  By default, bsdtar prefers 'ustar',
using 'pax' only for entries that require it.)

Sounds like 'tar' inside the chroot is different than the
one outside.  You've probably found a bug in the newest
RELENG_6 tar.

Can you email me the troublesome package file?

I'll also try to reproduce this here.

Thanks,

Tim Kientzle
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]

Re: Cannot package converters/libiconv inside clean chroot

[Tim Kientzle]

Ulrich Spoerlein wrote:


I'm facing a strange problem, where I cannot pkg_add a libiconv.tbz
which was build inside a clean chroot. I did the following (on
RELENG_6).


I just tried following the steps you listed and
I don't see the same error:

# pkg_add -v libiconv-1.9.2_2.tbz
Requested space: 5744640 bytes, free space: 7040139264 bytes in 
/var/tmp/instmp.mWa5XA

extract: Package name is libiconv-1.9.2_2
extract: CWD to /usr/local
extract: /usr/local/man/man1/iconv.1.gz
extract: /usr/local/man/man3/iconv.3.gz
.. additional lines deleted 
extract: execute '/sbin/ldconfig -m /usr/local/lib'
extract: CWD to .
Running mtree for libiconv-1.9.2_2..
mtree -U -f +MTREE_DIRS -d -e -p /usr/local /dev/null
Attempting to record package into /var/db/pkg/libiconv-1.9.2_2..
Package libiconv-1.9.2_2 registered in /var/db/pkg/libiconv-1.9.2_2

Please send me the .tbz created on your system,
so I can try to figure out why pkg_add is failing.

Thanks,

Tim Kientzle
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]