[tcpdump-workers] Re: Dropping support in tcpdump for older versions of libpcap?

2024-04-25 Thread Denis Ovsienko
On Fri, 19 Apr 2024 11:18:47 -0700
Guy Harris  wrote:

> On Apr 19, 2024, at 5:49 AM, Denis Ovsienko 
> wrote:
> 
> > On Fri, 12 Apr 2024 18:49:05 -0700
> > Guy Harris  wrote:  
> 
>   ...
> 
> > Since tcpdump is the reference implementation of a program that uses
> > libpcap, it may be a good occasion to improve the solution space
> > such that other software can copy something that works well in
> > tcpdump.  It is not entirely obvious the LIBPCAP_HAVE_PCAP_
> > macros would be worth the burden of maintenance, but the version
> > macros should be a straightforward improvement, something such as:
> > 
> > #define PCAP_VERSION_MAJOR 1
> > #define PCAP_VERSION_MINOR 11
> > #define PCAP_VERSION_PATCHLEVEL 0
> > #define PCAP_VERSION_AT_LEAST(a, b, c) ...
> > 
> > (The GCC and Clang version checks in compiler-tests.h would be
> > examples of a good macro structure; Sun C, XL C and HP C version
> > checks look unwieldy and error-prone).  
> 
> Presumably meaning that we should export version information in the
> way GCC and Clang do, rather than in the ways that Sun/Oracle C, XL C
> and HP C do, the latter being why we have to go through all that
> extra pain (they provide a single #define with the version number
> components packed in it - or two different defines in different
> versions as XL C does - rather than separate #defines for major and
> minor versions, as GCC and Clang do).

On a second thought, the best way to describe the desired result would
be that from the library users' point of view the version macros should
be easy to use correctly and difficult to use incorrectly.  This would
justify some inconvenience in the library code, if necessary.

An advantage of correctly sized BCD versions is that two packed integer
values compare in a straightforward way, so every end user does not
have to remember how to compare two version triplets correctly.  A
disadvantage of BCD versions is the need to unpack them if one needs
individual components.

Perhaps the best balance would be in defining the individual version
components as individual macros, and defining one-way "make this a BCD"
and "make the current libpcap version a BCD" macros.

For example, instead of

// require libpcap >= 1.10.2
PCAP_VERSION_MAJOR > 1 ||
(PCAP_VERSION_MAJOR == 1 && PCAP_VERSION_MINOR > 10) ||
(PCAP_VERSION_MAJOR == 1 && PCAP_VERSION_MINOR == 10 &&
PCAP_VERSION_PATCH >= 2)

the users could use the following:

// require libpcap >= 1.10.2
PCAP_VERSION_BCD_CURRENT >= PCAP_VERSION_BCD(1, 10, 2)

(where the exact number of bits allocated to each version component and
the definitions of PCAP_VERSION_BCD_CURRENT and PCAP_VERSION_BCD() are
an internal detail so long as the same values compare the same in
different versions of libpcap)

> > There could be a run-time check as well:
> > 
> > extern int pcap_version_at_least (unsigned char major, unsigned char
> > minor, unsigned char patchlevel);  
> 
> So how would that be used?
> 
> If a program is dynamically linked with libpcap, and includes calls
> to routines that were added in libpcap 1.12 or later, if you try to
> run it with libpcap 1.11, the run-time linker will fail to load it,
> as some symbols requested by the executable won't be present in the
> library. The only OS on which this can be made to work is macOS, with
> its weak linking mechanism:

[...]

> But *all* of those require either run-time checks for a particular OS
> version in macOS, in cases where you're using the libpcap that comes
> with macOS, or require loading the library at run time, finding
> particular routines at run time, and checking at run time whether the
> routine was found.

Thank you for the overview of weak linking means.  I suppose it would be
best to keep out of this space to keep the task relatively manageable.
The only use case for the C function I currently see is the
command-line version tester.

> > The latter could be available via a build helper binary, such as
> > (using the binary operators from test(1) and version-aware
> > comparison):
> > 
> > pcap-version -ge 1 # same as 1 0 0
> > pcap-version -ge 1 10 # same as 1 10 0
> > pcap-version -ne 1 10 4
> > pcap-version -eq 1 10 4
> > pcap-version -ge 1 9 1 && pcap-version -le 1 9 3  
> 
> So would this be used in a Makefile/configure
> script/CMakeFile.txt/etc. to check whether the libpcap on the system
> is sufficiently recent to include the routines your program needs,
> and fail if it isn't?

Yes (as the tcpdump dependency would be).  And not just the routines,
but their behaviour, as some software could require.

> >> Is there any reason not to require libpcap 1.0 or later?  

[tcpdump-workers] SITA ECN code is going to retire soon

2024-04-25 Thread Denis Ovsienko
Hello all.

The libpcap module in pcap-sita.c has been defunct for a while: there is
no support for "--with-pcap=sita", so the source cannot be compiled by
normal means, and "make pcap-sita.o" makes it clear it would fail to
compile anyway.

I have confirmed with Fulko Hew -- the original contributor of this
code -- that there are no known remaining users of this module.  Unless
anybody justifies the need to keep the SITA module, it will be removed
from libpcap 1.11.0.  The LINKTYPE allocation and the specification at
https://www.tcpdump.org/linktypes/LINKTYPE_SITA.html are going to
remain for historic purposes.

-- 
Denis Ovsienko
___
tcpdump-workers mailing list -- tcpdump-workers@lists.tcpdump.org
To unsubscribe send an email to tcpdump-workers-le...@lists.tcpdump.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


[tcpdump-workers] Re: Dropping support in tcpdump for older versions of libpcap?

2024-04-19 Thread Denis Ovsienko
On Fri, 12 Apr 2024 18:49:05 -0700
Guy Harris  wrote:

> A while ago, tcpdump and its configuration script were modified -
> mainly by Bill Fenner, as I remember - so that it didn't require a
> contemporary version of libpcap, and could be built with older
> versions of libpcap.
> 
> The intent, as I remember, was to allow somebody who was using a
> system that provided both libpcap and tcpdump to build a more recent
> version of tcpdump without having to download and build a newer
> version of libpcap.

Specifically, tcpdump tests for particular functions one-by-one and
then enables specific code paths depending on specific HAVE_PCAP_x
macros.  This covers some use cases, however...

First, the solution is not always convenient: every program that uses
libpcap has to make the build-time checks without much assistance from
libpcap headers: i.e. instead of hinging the conditionals on
hypothetical PCAP_HAVE_PCAP_ macros pre-defined (or not) in a
libpcap header the checks have to be duplicated: tcpdump uses one set
in Autoconf and another in CMake.  Other software has to use its own
means to detect particular functions.  This complicates usage
unnecessarily (as soon as a particular revision of libpcap has
compiled, the fact it implements a particular function is a constant).

Second, the solution space does not always match the problem space:
sometimes the software that uses libpcap needs to know not just that a
particular function is available, but also whether it implements a
particular detail (default behaviour, a feature or a bug/fix).  In
practice this boils down to checking libpcap version.  The latter is not
as trivial as it should be.  I remember one program parsing the [string]
result of pcap_lib_version() because there wasn't a  more appropriate
way to do it.

Since tcpdump is the reference implementation of a program that uses
libpcap, it may be a good occasion to improve the solution space such
that other software can copy something that works well in tcpdump.  It
is not entirely obvious the LIBPCAP_HAVE_PCAP_ macros would be worth
the burden of maintenance, but the version macros should be a
straightforward improvement, something such as:

#define PCAP_VERSION_MAJOR 1
#define PCAP_VERSION_MINOR 11
#define PCAP_VERSION_PATCHLEVEL 0
#define PCAP_VERSION_AT_LEAST(a, b, c) ...

(The GCC and Clang version checks in compiler-tests.h would be examples
of a good macro structure; Sun C, XL C and HP C version checks look
unwieldy and error-prone).

There could be a run-time check as well:

extern int pcap_version_at_least (unsigned char major, unsigned char
minor, unsigned char patchlevel);

The latter could be available via a build helper binary, such as (using
the binary operators from test(1) and version-aware comparison):

pcap-version -ge 1 # same as 1 0 0
pcap-version -ge 1 10 # same as 1 10 0
pcap-version -ne 1 10 4
pcap-version -eq 1 10 4
pcap-version -ge 1 9 1 && pcap-version -le 1 9 3

Obviously, any such improvements would not cover any earlier releases of
libpcap, so would need time to propagate.

Would this make sense?

> Currently, at least in theory, we support versions of libpcap at
> least as old as 0.4, which was the last version released by LBL.
> 
> tcpdump, for example, supports versions of libpcap that don't include
> pcap_findalldevs(); that routine first appeared in libpcap 0.7, which
> was released in 2001, almost 23 years ago.
> 
> It also supports versions of libpcap that don't include pcap_create()
> and pcap_activate(); those first appeared in libpcap 1.0, which was
> released in 2008, almost 16 years ago.
> 
> Is there any reason not to require libpcap 1.0 or later?  If there
> is, is there any reason not to require libpcap 0.7 or later?

Such use cases may exist, but I am not aware of any.

-- 
Denis Ovsienko
___
tcpdump-workers mailing list -- tcpdump-workers@lists.tcpdump.org
To unsubscribe send an email to tcpdump-workers-le...@lists.tcpdump.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


[tcpdump-workers] Re: openwrt Conclusions from CVE-2024-3094 (libxz disaster)

2024-04-03 Thread Denis Ovsienko


> This would be a bit of additional work, but security and confidence
> are usually at the opposite ends of the same scale.

s/confidence/convenience/

I knew what I meant, of course.

-- 
    Denis Ovsienko
___
tcpdump-workers mailing list -- tcpdump-workers@lists.tcpdump.org
To unsubscribe send an email to tcpdump-workers-le...@lists.tcpdump.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


[tcpdump-workers] Re: openwrt Conclusions from CVE-2024-3094 (libxz disaster)

2024-04-02 Thread Denis Ovsienko
On Tue, 2 Apr 2024 14:06:28 +0200
Francois-Xavier Le Bail via tcpdump-workers
 wrote:

> Even if we keep the tarball archive, we could have a host compromise
> (bad autoconf, etc.) and if the "configure" script is generated on
> it, we risk to open a door to an attack.
> 
> Thus, don't deliver "configure" in the tarball and ask to run
> "./autogen.sh" with autotools installed.

Let's consider the problem and the solution.

If the host is fully compromised (a remote attacker made themselves the
root user of my computer's OS), there is little point in trying to
game around it.  I delete configure, they put the virus into
autogen.sh.  I delete autogen.sh, they put the virus into mkdep.  I
delete mkdep, they put the virus into Makefile.in.  I delete
Makefile.in, they put the virus into CMakeLists.txt.  I delete
CMakeLists.txt, they put their own file of any name and contents into
the tarball because they are not limited by the contents of the git
repository.  They just need me to sign and publish a tarball with
their contents.

If the host is mostly clean, but the Autoconf package is compromised
through the upstream or otherwise, it means the compromised Autoconf
package is not limited to placing the virus into configure.  It may
happily produce a correct clean configure from a correct clean
configure.ac, and then add the virus into configure.ac or autogen.sh or
mkdep or Makefile.in or...

If the host is clean and Autoconf is clean and every other package is
clean, but the developer is compromised, then in that case why the
compromised developer would limit themselves to the configure script
if they have full control of the release archive contents and the git
repository contents?  What would prevent a compromised developer from
publishing a "secure, configure-free" tarball with a virus in some
other file (configure.ac, etc.)?

If the host is clean and all packages are clean and the developer is
clean and the tarball has no configure script, then the users that want
to compile the source will need to supply their own Autoconf, which can
be compromised as realistically as yours or mine, so effectively
excluding the configure script does not solve the problem, but shifts it
to somebody else's side of the fence and multiplies the number of times
it will need to be solved after the release.

It would not be impossible for every user that builds from source to
arrange a compatible version of Autoconf (you have to build it first in
pkgsrc, for example, and the dependencies) and to verify that their
Autoconf is not compromised.  Some users take exactly this approach and
run "autoreconf -f" as the very first step no matter if the tarball
included configure or not.  Other users choose not to duplicate the work
unnecessarily, but to trust the software developers to figure the
details out correctly once and to use a clean release environment
correctly once, and to publish a tarball that is not compromised, has
a ready to run configure script and a signature.  This is a reasonable
expectation.

A possible way to add confidence to the process could be additional
verification.

Individual computers and OSes can be compromised.  To that end, for
example, the release maker could run "make releasetar" on at least two
separate physical computers, each with a different OS and a different
architecture, and let a script compare the results.  Computers are small
and very affordable now.

Individual developers can be compromised.  When one developer produces
the signed tarballs at their end, another could run "make releasetar"
at their end and let a script compare the contents and verify the
signatures.

This would be a bit of additional work, but security and confidence are
usually at the opposite ends of the same scale.

-- 
Denis Ovsienko
___
tcpdump-workers mailing list -- tcpdump-workers@lists.tcpdump.org
To unsubscribe send an email to tcpdump-workers-le...@lists.tcpdump.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


[tcpdump-workers] Re: openwrt Conclusions from CVE-2024-3094 (libxz disaster)

2024-04-01 Thread Denis Ovsienko
r example, when one starts with a fresh install of NetBSD
  with just make, cc and a tarball of pkgsrc release.  Before you can
  use git, you have to actually compile git, which means downloading a
  tarball of git (Can you get a working copy of a clone of the git
  repository of git without using git?  How?).  And before that it means
  downloading tarballs of all dependencies of git and building those.
  That's why other packages usually do not have git as a build
  dependency, but use release tarballs.  In some scenarios
  bootstrapping is an important factor.  The world does not entirely
  consists of high-end build servers.

* Speaking of "reproducible builds", building from a self-contained
  source package such as .src.rpm with a signed tarball (and any
  patches, etc.) in it is way more reproducible than building from a
  text file that says "in theory, once upon a time there was a
  repository online over there with a tag named such and such".  How
  much of that will hold in 5 years time?  10 years time?  Repositories
  come and go. Tags get created and deleted, by accident or
  deliberately.  Entire git hosting providers and parts of the Internet
  go offline on a regular basis.

  Let's recognise that knowing where the source supposedly is and
  actually having the actual source on the actual computer where it
  needs to be built can be (and sometimes is) two different things.
  This difference is easy to ignore, until it is not.  Release tarballs
  among other things are a time-proven means to provide the source for
  reproducible builds, and the best course of action is not to destroy
  it under the heat of a moment.

* The root cause was a compromised developer.  The technique was
  obfuscation.  To that end, a lot of work has been done in tcpdump and
  libpcap to remove dead code and to clean up working code.  This is a
  long-term effort, plenty of hard work has been done on zero budget
  and some hard work still remains to be done, but the difference is
  already visible.  Let's continue this work.  Clean and maintainable
  code does not prevent compromising a developer in principle, but it
  makes obfuscation more likely to manifest if attempted.

  That said, compromising a developer would be a very serious blow to
  any project, and nobody would like to have such a problem in the
  first place.  Certain things potentially could be done in this project
  to minimise, if not to eliminate, such possibility, but this would be
  neither trivial nor quick, and it would cost, in terms of money and
  otherwise.  Is there a sufficiently funded demand for it from a
  trustworthy party?  Let me see it.

To sum it up, please do not make any major changes right now.  If
anybody wants to suggest such a change, please try to structure the
proposal along the following lines:

1. If the matter is truly urgent, please state why.
2. What specifically seems to be the problem?
3. What is the root cause of the problem?
4. What is the structure of the problem?
5. Which specific parts of the problem space you suggest to address
   first?
6. What is the proposed solution and how well does the solution space
   match the problem space?
7. How does the solution mitigate/reduce/eliminate the root cause of the
   problem?
8. What would be the best way to confirm the solution actually works as
   expected after it has been implemented?
9. What is the rollback plan, in case the solution did not work?
10. How is the proposed solution supposed to work long-term and
what is the impact on backward compatibility, if any?

Thank you.

-- 
Denis Ovsienko
___
tcpdump-workers mailing list -- tcpdump-workers@lists.tcpdump.org
To unsubscribe send an email to tcpdump-workers-le...@lists.tcpdump.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


[tcpdump-workers] Re: SIGINFO/SIGUSR1 and SIGUSR2

2024-03-28 Thread Denis Ovsienko
On Thu, 28 Mar 2024 15:28:00 -0700
Guy Harris  wrote:

> On Mar 28, 2024, at 2:19 PM, Denis Ovsienko 
> wrote:
> 
> > Yes, AIX and Haiku sometimes make portability issues manifest.  
> 
> And, in this case, Solaris doesn't have SIGINFO, either; SunOS
> 0.x-4.x didn't have it, as BSD hadn't picked it up, and they didn't
> pass it along to be put into SVR4, so it's not in the SVR4-based
> SunOS 5.x.
> 
> As noted, neither does Linux.
> 
> I.e., at this point, if it's not named "somethingBSD" or "Mac OS X/OS
> X/macOS", it doesn't have SIGINFO.

illumos declares it as number 41 and implements it normally.

SIGINFO causes:

tcpdump: 54 packets captured, 924 packets received by filter, 0 packets
dropped by kernel
(tcpdump continues to run)

SIGUSR1 causes:

User Signal 1
(tcpdump exits)

> > Changing the compiled-in defaults would be one thing, and given how
> > long ago the current behaviour was implemented, it would be best to
> > think twice before changing it.  There are users with learned
> > keystrokes and scripts that work, let's keep it this way when
> > possible.  
> 
> The only change I'm suggesting to the compiled-in defaults is to
> change the default for SIGUSR1 from the current default of
> "print_stats if the system doesn't have SIGINFO, kill the process if
> it doesn't" to "print_stats regardless of whether the system has
> SIGINFO"; neither the default for SIGINFO (print_status if the system
> has it) nor the default for SIGUSR2 (flush_savefile) would be changed.
> 
> I don't see a way in which any remotely reasonable learned keystroke
> or script would depend on SIGUSR1 killing the process on *BSD/macOS,
> so I don't see an issue with SIGINFO *and* SIGUSR1 both causing stats
> to be printed.

Alright, this time it makes sense.  Should be reasonably backward
compatible.

> > Allowing to override the defaults at run time  
> 
> Which is what I was talking about there.



-- 
Denis Ovsienko
___
tcpdump-workers mailing list -- tcpdump-workers@lists.tcpdump.org
To unsubscribe send an email to tcpdump-workers-le...@lists.tcpdump.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


[tcpdump-workers] Re: SIGINFO/SIGUSR1 and SIGUSR2

2024-03-28 Thread Denis Ovsienko
On Thu, 28 Mar 2024 12:44:29 -0700
Guy Harris  wrote:

> > [--siginfo=] (on platforms with SIGINFO only)  
[...]
> so I suspect few, if any, UN*Xes have, or had, SIGINFO without also
> having SIGUSR1 and SIGUSR2.

Thank you for providing the historic background.  The above was
supposed to mean "only on platforms that have SIGINFO (and any other
signals)".

> As for "not UN*X but tries hard to look like it", Haiku has SIGUSR1
> and SIGUSR2 but not SIGINFO.
> 
> SIGINFO is largely a BSDism, not adopted by Linux or System V Release
> 4 (which may have come out before *BSD* added it) or Haiku or AIX:
> 
>   
> https://www.ibm.com/docs/en/aix/7.3?topic=s-sigaction-sigvec-signal-subroutine
> 
> etc..

Yes, AIX and Haiku sometimes make portability issues manifest.

> So I wouldn't worry abut platforms that only have SIGINFO; given
> that, on the platforms that offer it (BSDs, including CupertinoBSD),
> it's defined to mean "give me a status report" - unlike SIGUSR1 and
> SIGUSR2, which are explicitly defined *not* to have a standard
> meaning, leaving it up to the application to choose how to use it - I
> wouldn't bother with a --siginfo option.
> 
> Instead, we could have SIGUSR1 default to "print statistics" even on
> systems that *have* SIGINFO, continue to have SIGUSR2 default to
> "flush the savefile", and allow --sigusr1= and --sigusr2= to reassign
> either of those to "flush_savefile" or "rotate_savefile".  That means
> you can't, on platforms without SIGINFO, have "print_stats",
> "flush_savefile", and "rotate_savefile" signals, but that's because
> you don't have three signals to reassign.  On platforms *with*
> SIGINFO, you can use the other two for "flush_savefile" and
> "rotate_savefile".

Changing the compiled-in defaults would be one thing, and given how long
ago the current behaviour was implemented, it would be best to think
twice before changing it.  There are users with learned keystrokes and
scripts that work, let's keep it this way when possible.

Allowing to override the defaults at run time would be another thing,
which would not depend on the compiled-in defaults.  Besides the obvious
"I need to use this action for this one tcpdump process somehow and it
does not have a default signal" use case, it would also make it
possible, for instance, to run one group of tcpdump processes with
"--sigusr1=ignore --sigusr2=rotate_savefile" and another with
"--sigusr1=rotate_savefile --sigusr2=ignore" and then to use killall
remembering only the correct signal to use for the group, but not which
group comprises which PIDs.

This feature would only need to detect the set of user-configurable
signals at build time, to show the set and its default actions on
request, and to allow changing individual actions, then the users could
make sense of that as each use case requires.

-- 
Denis Ovsienko
___
tcpdump-workers mailing list -- tcpdump-workers@lists.tcpdump.org
To unsubscribe send an email to tcpdump-workers-le...@lists.tcpdump.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


[tcpdump-workers] SIGINFO/SIGUSR1 and SIGUSR2

2024-03-28 Thread Denis Ovsienko
Hello all.

There is a rather old pull request at [1], which was supposed to make
use of the then-unused SIGUSR2, but whilst it was waiting, another pull
request used the signal for another code path.

There is a potential way to manage this kind of contention by
naming the available actions and disassociating them from the available
signals.  For example, let's call the existing SIGINFO/SIGUSR1 action
"print_stats", the existing SIGUSR2 action -- "flush_savefile" and the
action proposed in the pull request -- "rotate_savefile".  Perhaps an
easy action would be "ignore" to do nothing instead of the default
something.  Then these command-line options would allow to associate
the signals with the actions:

[--siginfo=] (on platforms with SIGINFO only)
[--sigusr1=]
[--sigusr2=]

In this case the following defaults would provide backward
compatibility:

(on platforms with SIGINFO)
--siginfo=print_stats
--sigusr2=flush_savefile

(on platforms without SIGINFO)
--sigusr1=print_stats
--sigusr2=flush_savefile

Then, if tcpdump prints the defaults and the available actions on
--help, it should be straightforward for the user to change the
response as necessary, including the action from the pull request:

--sigusr2=rotate_savefile

1: https://github.com/the-tcpdump-group/tcpdump/pull/570

-- 
Denis Ovsienko
___
tcpdump-workers mailing list -- tcpdump-workers@lists.tcpdump.org
To unsubscribe send an email to tcpdump-workers-le...@lists.tcpdump.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


[tcpdump-workers] HP-UX support and portability

2024-03-12 Thread Denis Ovsienko
Hello all.

HP-UX is one of the OSes nominally supported by libpcap and tcpdump,
but it is rather difficult (or expensive) to find a live HP-UX host with
shell for testing and development.  So before I forget again, this is
far from ideal, but is the best reference material I managed to find for
reasoning about HP-UX portability:

http://hpux.connect.org.uk/hppd/hpux/Networking/Admin/libpcap-1.10.4/
http://hpux.connect.org.uk/hppd/hpux/Networking/Admin/tcpdump-4.99.4/

The "source code" tarballs are not the pristine release archives, but
archives of working copies after applying the patches and running
./configure, so there is a config.h with (or without) the various
HAVE_ macros.

The binary packages also include description of changes applied before
building, perhaps some of that could be up-streamed.

-- 
Denis Ovsienko
___
tcpdump-workers mailing list -- tcpdump-workers@lists.tcpdump.org
To unsubscribe send an email to tcpdump-workers-le...@lists.tcpdump.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


[tcpdump-workers] Re: tcpslice licence

2024-03-04 Thread Denis Ovsienko
Hello.

It has been a while, and this problem still needs a solution.


On Mon, 3 Aug 2020 13:22:35 -0700
Guy Harris  wrote:

[...]
> The first step I'd take would be to get rid of the GPLed headers in
> favor of BSD-licensed headers, e.g. taking the ip.h, tcp.h, and udp.h
> headers from tcpdump and changing the code to work with them.

This is no longer a part of the problem space.

> What remains are:
[...]
> so I'll try contacting Mr. Ainslie to see whether we can replace the
> 3-clause-plus-one-unnumbered-clause LBL license with the 3-clause
> LB(N)L license in libpcap, tcpdump, and tcpslice.

I suppose this did not work.  Let's put the
"3-clause-plus-one-unnumbered-clause LBL license" into a LICENSE file
then?

-- 
Denis Ovsienko
___
tcpdump-workers mailing list -- tcpdump-workers@lists.tcpdump.org
To unsubscribe send an email to tcpdump-workers-le...@lists.tcpdump.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


[tcpdump-workers] Re: Removing untested libpcap support for older platforms

2024-02-28 Thread Denis Ovsienko
On Thu, 5 Oct 2023 20:27:41 -0700
Guy Harris  wrote:

[...]
> Should we also consider removing support for some older UN*X
> platforms, such as:
> 
>   SunOS prior to SunOS 4 - pcap-nit.c; the last such version,
> SunOS 3.5, was released in January 1988
> 
>   SunOS 4.x - pcap-snit.c; the last such version, SunOS 4.1.5,
> was released in November 1994
> 
>   HP-UX 9 - some code in pcap-dlpi.c;  the last such version,
> 9.10, was released in early-to-mid 1995
> 
>   HP-UX 10 prior to 10.20 - some code in pcap-dlpi.c; - the
> only such version, 10.0, was released in 1995
> 
>   SINIX - some code in pcap-dlpi.c; - the last release, Reliant
> UNIX 5.45, was released some time in the early 2000s (?)
> 
>   IRIX - pcap-snoop.c; - the last update was released some time
> in the mid-to-late 2000s (?)
> 
>   DEC OSF/1^W^WDigital UNIX^WTru64 UNIX - pcap-pf.c; the last
> release, Tru64 UNIX 5.1B-6, was released in October 2010
[...]

For posterity, the master branches no longer support the above OSes
even nominally.  Occasional dead-like code still surfaces, for example,
libpcap still uses a small buffer size on AIX to work around a problem
in AIX 3.5 (not in Wikipedia, supposedly dates to 1993-1994) and uses
conditionals for LynxOS, with which it has had no contact for good 10
years (if anyone knows that's not the case, please provide particulars).
I guess most of such artefacts eventually will be gone too.

In tcpdump README.md now includes a more or less complete list of
retired OSes, please improve what you find in need of improvement.

-- 
Denis Ovsienko
___
tcpdump-workers mailing list -- tcpdump-workers@lists.tcpdump.org
To unsubscribe send an email to tcpdump-workers-le...@lists.tcpdump.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


[tcpdump-workers] Re: Pcap debug at runtime

2023-03-18 Thread Denis Ovsienko
On Tue, 14 Mar 2023 08:33:40 +0100
Francois-Xavier Le Bail  wrote:

> After an update based on a Denis's idea, the configuration use now an
> environment variable instead of configurations files. 
> If the environment variable INSTRUMENT is
> - unset or set to an empty string, print nothing, like with no
> instrumentation
> - set to "all" or "a", print all the functions names
> - set to "global" or "g", print only the global functions names
> 
> Note that before the change, the default was to print all functions.
> Now it is to print nothing.

For posterity, this is where the idea originates:

$ LD_DEBUG=help date
Valid options for the LD_DEBUG environment variable are:

  libsdisplay library search paths
  reloc   display relocation processing
  files   display progress for input file
  symbols display symbol table processing
  bindingsdisplay information about symbol binding
  versionsdisplay version dependencies
  scopes  display scope information
  all all previous options combined
  statistics  display relocation statistics
  unused  determined unused DSOs
  helpdisplay this help message and exit

To direct the debugging output into a file instead of standard output
a filename can be specified using the LD_DEBUG_OUTPUT environment
variable.

-- 
Denis Ovsienko
___
tcpdump-workers mailing list -- tcpdump-workers@lists.tcpdump.org
To unsubscribe send an email to tcpdump-workers-le...@lists.tcpdump.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


[tcpdump-workers] Tru64 UNIX

2023-03-11 Thread Denis Ovsienko
Hello all.

I hope this finds you well.  After quite a few rounds of cleanups the
"lbl" directory exists only in libpcap, and the only file there is
os-osf5.h, which stands for Tru64 UNIX 5.x.  If anyone has such a
system running and would like libpcap to support it and can provide
resources to enable such support, please shout before long.  Otherwise
notions of Tru64/Digital/OSF will be removed where they get in the way
of present day development and maintenance.  The OS vendor finished
Tru64 support 10 years ago.

-- 
Denis Ovsienko
___
tcpdump-workers mailing list -- tcpdump-workers@lists.tcpdump.org
To unsubscribe send an email to tcpdump-workers-le...@lists.tcpdump.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


[tcpdump-workers] Re: Pcap debug at runtime

2023-03-01 Thread Denis Ovsienko
On Tue, 28 Feb 2023 17:01:51 +0100
Francois-Xavier Le Bail  wrote:

> In addition to printf()/fprintf(), here is a brand new way to help
> debugging a program using libpcap, currently only tested on Debian
> Linux (stable).
> 
> (Similar method to the one available with tcpdump and tcpslice.)
> 
> The goal is to generate instrumentation calls for entry and exit to
> functions. Just after function entry and just before function exit,
> the profiling functions are called and print the function names with
> indentation and call level. If entering in a function, print also the
> calling function name with file name and line number. There may be a
> small shift in the line number.
> 
> To use it:
> (There will be a doc entry based on this topic later.)

Thank you for putting this together.  Does the FAQ look the best place
for such documentation?

-- 
Denis Ovsienko
___
tcpdump-workers mailing list -- tcpdump-workers@lists.tcpdump.org
To unsubscribe send an email to tcpdump-workers-le...@lists.tcpdump.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


[tcpdump-workers] Re: TCP Header Flags

2023-02-27 Thread Denis Ovsienko
On Mon, 27 Feb 2023 03:31:27 +0100
Francois-Xavier Le Bail  wrote:

> There are already some doc/site with bad use of "tcp-psh" instead of
> "tcp-push" like: https://packetlife.net/media/library/12/tcpdump.pdf

The author has been notified about the error.

> https://github.com/the-tcpdump-group/tcpdump/issues/846

This indeed had happened.  Would it be correct to say only once in a
long time?

> https://github.com/tcpdump-examples/how-to-use-tcpdump

The fragment titled "Isolate TCP PSH flags." along with many others
is an old copy from https://danielmiessler.com/study/tcpdump/ and the
original document now says "tcp-push".

> https://blog.codefarm.me/2018/12/29/tcpdump-examples/

Another old copy from https://danielmiessler.com/study/tcpdump/.

> https://twitter.com/dc9221/status/1254154374143754241/photo/1

The above is a series of screenshots of
https://packetlife.net/media/library/12/tcpdump.pdf with the copyright
replaced with somebody else's.  See also
https://packetlife.net/blog/2016/mar/9/dont-be-discouraged-plagiarists/

> https://github.com/marciopocebon/TCPDUMP-1

Another old copy from https://danielmiessler.com/study/tcpdump/.

> ...
> Thus, the problem already exists in reverse.

It has been this way for 32 years, so let's either fix this properly
with backward compatibility notes, or not at all.  And in any case,
with a clear understanding which is the right thing.

-- 
Denis Ovsienko
___
tcpdump-workers mailing list -- tcpdump-workers@lists.tcpdump.org
To unsubscribe send an email to tcpdump-workers-le...@lists.tcpdump.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


[tcpdump-workers] Re: TCP Header Flags

2023-02-26 Thread Denis Ovsienko
On Sun, 26 Feb 2023 15:46:56 +0100
Francois-Xavier Le Bail  wrote:

[...]
> >> I wonder if there would be any other incurred future maintenance.  
> 
> The proposed patch is:
> 
> diff --git a/pcap-filter.manmisc.in b/pcap-filter.manmisc.in
> index 10aeb42d..864cd238 100644
> --- a/pcap-filter.manmisc.in
> +++ b/pcap-filter.manmisc.in
> @@ -1027,7 +1027,7 @@ The following ICMPv6 type field values are
> available: .BR \%icmp6-multicastrouterterm .
>  .IP
>  The following TCP flags field values are available: \fBtcp-fin\fP,
> -\fBtcp-syn\fP, \fBtcp-rst\fP, \fBtcp-push\fP,
> +\fBtcp-syn\fP, \fBtcp-rst\fP, \fBtcp-psh\fP (or \fBtcp-push\fP),
>  \fBtcp-ack\fP, \fBtcp-urg\fP, \fBtcp-ece\fP,
>  \fBtcp-cwr\fP.
>  .LP
> diff --git a/scanner.l b/scanner.l
> index 85fe395a..7cc39f77 100644
> --- a/scanner.l
> +++ b/scanner.l
> @@ -475,6 +475,7 @@ tcp-fin { yylval->h = 0x01;
> return NUM; } tcp-syn{ yylval->h = 0x02;
> return NUM; } tcp-rst{ yylval->h = 0x04;
> return NUM; } tcp-push   { yylval->h = 0x08; return NUM; }
> +tcp-psh{ yylval->h = 0x08; return NUM; }
>  tcp-ack{ yylval->h = 0x10; return NUM; }
>  tcp-urg{ yylval->h = 0x20; return NUM; }
>  tcp-ece{ yylval->h = 0x40; return NUM; }
> 
> (the tcpdump man page will need an update too.)

Obviously, the change would be easy to make.  But what comes to my mind
is next 5 or 10 years of answering the same question: "Why tcp-psh is in
the man page/my new book/stackoverflow/whatever and it works in my
development environment, but some production servers reject the syntax?
These are on a very expensive long term support contract, so everything
is supposed just to work, right?  Right?"

You could argue that there would be a note in the "backward
compatibility" section, but I could argue that only a fraction of users
ever reads any documentation that comes directly with the software.  It
would be better to have some other problems solved before looking at
this discrepancy again.

Specifically, I wonder if it would be practicable to process all
remaining longjmp() backlog in the next few months.  The script reports
35 not yet converted files ranging from 110 to 2755 CLOC each.

-- 
Denis Ovsienko
___
tcpdump-workers mailing list -- tcpdump-workers@lists.tcpdump.org
To unsubscribe send an email to tcpdump-workers-le...@lists.tcpdump.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


[tcpdump-workers] Re: [tcpdump] About struct in_addr / struct in6_addr

2023-02-20 Thread Denis Ovsienko
On Sun, 17 Jul 2022 15:29:39 -0700
Guy Harris via tcpdump-workers 
wrote:

> On Jul 17, 2022, at 11:09 AM, Francois-Xavier Le Bail
>  wrote:
> 
> > Remain some stuff about 'struct in6_addr'. Any need to keep them?
> > 
> > $ git grep -l 'struct in6_addr'
> > CMakeLists.txt
> > cmakeconfig.h.in
> > config.h.in
> > configure
> > configure.ac
> > netdissect-stdinc.h  
> 
> That's there for the benefit of OSes whose APIs don't have standard
> IPv6 support; if there are any left that we care about (or if there
> are old non-IPv6 versions we care about for any OSes we support),
> then it might be useful, but I'm not sure it would build (we use
> gethostbyaddr(), so *maybe* it'll compile, and maybe gethostbyaddr()
> will fail when passed AF_INET6 and the code will just show the IPv6
> address rather than a name).

After a closer look at this code it turns out the only remaining case
of struct in6_addr is Windows-specific win32_gethostbyaddr(), so the
conditional declaration of struct in6_addr can be at least moved into
the same "#ifdef _WIN32" block and at least Autoconf-specific checks
for struct in6_addr are moot.  If anyone could confirm that supported
versions of Windows always have struct in6_addr, that could be removed
altogether and the only occurrence of struct in6_addr would be in
win_gethostbyaddr().

AF_INET6 looks a bit more convoluted.  There is some code that uses
AF_INET6 to dissect wire encoding, which is usually a wrong idea.  For
example, pimv2_addr_print() switches on AF_INET and AF_INET6, and the
PIMv2 header encoding (RFC 4601 Section 4.9.1) clearly says the AF is
the IANA AF [1]:

1: IP
2: IP6

Which is different from most OS definitions of AF_INET and AF_INET6,
but this function has been implemented this way since 1999, and somehow
it seems to be able to decode a few PIMv2 packet captures I found on
the Internet.  So cases like this will require more attention and some
of the remaining AF_INET6 instances may become wire encoding constants
rather than the OS AF_INET6 constant.

The only sound reason for tcpdump to use AF_INET6 seems to be in
ip6addr_string() for address-to-name resolving.  That use case already
has nd_ipv6 and could have a simple "#ifdef AF_INET6" guard.  This
would eliminate respective logistics in Autoconf and CMake and would
make OS IPv6 support a nice-to-have, but not an essential dependency.

1:
https://www.iana.org/assignments/address-family-numbers/address-family-numbers.xhtml

-- 
Denis Ovsienko
___
tcpdump-workers mailing list -- tcpdump-workers@lists.tcpdump.org
To unsubscribe send an email to tcpdump-workers-le...@lists.tcpdump.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


[tcpdump-workers] Re: Z-Wave G.9959 TAP Specification version 0.94

2023-02-19 Thread Denis Ovsienko
On Thu, 9 Feb 2023 15:36:45 -0800
Chris Brandson  wrote:

> Hopefully this is sufficient to allocate a new DLT.

Thank you for waiting, 297 now stands for LINKTYPE_ZWAVE_TAP and
DLT_ZWAVE_TAP at https://www.tcpdump.org/linktypes.html and in libpcap
git repository.  Good luck with your project and please remember to keep
the published specification up to date.

-- 
    Denis Ovsienko
___
tcpdump-workers mailing list -- tcpdump-workers@lists.tcpdump.org
To unsubscribe send an email to tcpdump-workers-le...@lists.tcpdump.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


[tcpdump-workers] Re: [tcpdump] About struct in_addr / struct in6_addr

2023-02-18 Thread Denis Ovsienko
On Sun, 17 Jul 2022 16:11:01 -0700
Guy Harris via tcpdump-workers 
wrote:

> On Jul 17, 2022, at 3:39 PM, Bill Fenner  wrote:
> 
> > IMO it is safe to drop support for OSes lacking native IPv6
> > support.  
> 
> Yeah.  Back when IPv6 support was added to tcpdump, it was an
> experimental new technology and the configure script had to figure
> out which of several add-on IPv6 packages you had installed.  Now a
> significant amount of Wikipedia vandalism comes from IPv6 addresses
> rather than IPv4 addresses. :-)

OS IPv6 support would be a very reasonable requirement for tcpdump 5.

-- 
Denis Ovsienko
___
tcpdump-workers mailing list -- tcpdump-workers@lists.tcpdump.org
To unsubscribe send an email to tcpdump-workers-le...@lists.tcpdump.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


[tcpdump-workers] Re: [tcpdump] About HAVE_NO_PRINTF_Z

2023-02-16 Thread Denis Ovsienko
On Mon, 30 Jan 2023 21:30:50 +
Denis Ovsienko via tcpdump-workers 
wrote:

> The changes to detect printf() issues (which now correctly fails the
> build on Solaris 9 instead of masking the post-build test failures)
> are now tcpdump draft pull request #1031.  Please treat this as a
> request for comments and if something does not look right, make your
> comments either there or on the mailing list.
> 
> Given no objections, this will be merged in a few weeks.

Done as advised.

-- 
Denis Ovsienko
___
tcpdump-workers mailing list -- tcpdump-workers@lists.tcpdump.org
To unsubscribe send an email to tcpdump-workers-le...@lists.tcpdump.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


[tcpdump-workers] Re: mailman3 list imported

2023-02-16 Thread Denis Ovsienko
On Wed, 15 Feb 2023 14:14:15 -0500
Michael Richardson via tcpdump-workers
 wrote:

> The mailing list has been moved from a mailman2 host to a mailman3
> host. I had subscribed everyone with an option to confirm, but that
> was a bad idea. I have now found the import21 command, and imported
> the "pickle" file from the mailman2 installation.

Thank you Michael.

-- 
Denis Ovsienko
___
tcpdump-workers mailing list -- tcpdump-workers@lists.tcpdump.org
To unsubscribe send an email to tcpdump-workers-le...@lists.tcpdump.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


Re: [tcpdump-workers] AC_LBL_FIXINCLUDES does not make it into configure

2023-02-08 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Fri, 27 Jan 2023 12:53:28 +
Denis Ovsienko via tcpdump-workers 
wrote:

[...]
> I do not understand yet if AC_LBL_C_INLINE could just disappear safely
> altogether instead of being replaced with AC_C_INLINE.

In order to play safe, it has been replaced with an AC_C_INLINE in both
tcpdump and libpcap.  And the last copy of AC_LBL_CHECK_LIB has been
removed from tcpdump.

-- 
    Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] Z-Wave G.9959 TAP Specification version 0.94

2023-02-06 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Mon, 12 Dec 2022 17:06:07 -0800
Chris Brandson via tcpdump-workers 
wrote:

> Hello Tcp Dump Workers,
> 
> The latest 0.94 draft Z-Wave G.9959 TAP Specification is available at:
> 
> https://gitlab.com/exegin/zwave-g9959-tap
> <https://gitlab.com/exegin/zwave-g9959-tap>
> 
> This version incorporates feedback from Denis Ovsienko, James Ko, and
> Sam Freemantle, whose contributions are appreciated.
> 
> Since there were no signifigant issues identified we requested that a
> DLT value be assigned to this specification. Once assigned the
> document will be updated with the DLT value and published as version
> 1.0.
> 
> Comments and feedback are still appreciated.

Hello Chris.

Thank you for waiting.  I believe the last point I raised about the
proposed encoding was that the currently specified header design makes
it non-trivial to access any MPDU data beyond the Length header.
However, if you find the current design good enough to serve the
intended purpose, it would be wrong to argue, even more so since the
header has a version field and can be improved later as and if required.

Are there any other objections to make this DLT allocation?

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] Pcap delivers packets every 200ms

2023-02-02 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Thu, 2 Feb 2023 15:42:55 +
Paschal Chukwuebuk Amusuo via tcpdump-workers
 wrote:

> Hi,
> 
> Please, is there any way to force pcap to deliver packets once it
> receives the packet? Currently, pcap delivers packets to my
> application at intervals and it batches the packets before delivering
> them. There are substantial time differences between when the packet
> is received by pcap and when it is finally delivered by the
> application.

Please see
https://www.tcpdump.org/manpages/pcap_set_immediate_mode.3pcap.html and
try the function.

> In the screenshot I attached, 6 packets were received within 400ms
> but all delivered at the same time.

This list strips non-text attachments (for clarity HTML is considered
not text for this purpose), so the screenshot didn't make it through.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] [tcpdump] About HAVE_NO_PRINTF_Z

2023-01-30 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Sat, 14 Jan 2023 16:57:45 +
Denis Ovsienko via tcpdump-workers 
wrote:

[...]
> So it would be practicable to restore Solaris 9 support in the
> trailing edge department of tcpdump using compile-time-conditional
> length modifier.  If anybody volunteers to implement that, it would
> be wrong to get in the way, I guess.

Alright, there are no volunteers at this time.

> > > Whatever is the eventual solution, in any case from today's
> > > perspective it makes sense to stop masking these test
> > > failures. 
> > 
> > Agreed.
> > Do you mean: Maybe just let the tests fail on Solaris, and document
> > that this happens?  
> 
> As the first step in any case, yes.  And make the build fail early if
> the required length modifier is not supported.

The changes to detect printf() issues (which now correctly fails the
build on Solaris 9 instead of masking the post-build test failures) are
now tcpdump draft pull request #1031.  Please treat this as a request
for comments and if something does not look right, make your comments
either there or on the mailing list.

Given no objections, this will be merged in a few weeks.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] CPPFLAGS in C-only context

2023-01-28 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Mon, 23 Jan 2023 22:16:24 +
Denis Ovsienko via tcpdump-workers 
wrote:

> It looks like either in a C project context CPPFLAGS works in a
> non-obvious way, or is a no-op.

...or, rather, is the C preprocessor flags variable (just as
"./configure --help" says it), and C++ compiler flags variable has
always been CXXFLAGS.

-- 
    Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] AC_LBL_FIXINCLUDES does not make it into configure

2023-01-27 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Fri, 27 Jan 2023 01:40:48 -0800
Guy Harris  wrote:

> On Jan 22, 2023, at 9:59 AM, Denis Ovsienko via tcpdump-workers
>  wrote:
> 
> > I have also removed AC_LBL_C_INLINE and a conditional substitute for
> > Tru64 pfopen() from tcpslice.  Interestingly, tcpslice and tcpdump,
> > which don't call pfopen(), used to have this substitute, and
> > libpcap, which does call pfopen(), does not have it.  
> 
> That dates back to tcpdump 3.4.  I don't know why they decided to
> compile pfopen() into tcpdump and tcpdslice if it's not in a system
> library, rather than compiling it into libpcap if it's not in a
> system library.  Perhaps they wanted to be able to build versions of
> libpcap that would work both on Tru64 UNIX versions in which pfopen()
> is in a system library and versions in which it's not and all you
> have is pfopen.c source code under /usr/examples.
> 
> I don't know what older versions those might be, and I suspect we
> have little if any reason to continue to make it possible to build
> tcpdump or tcpslice on those older versions - it looks as if Tru64
> UNIX 4.x and 5.x have pfopen() in system libraries; according to
> 
>   https://en.wikipedia.org/wiki/Tru64_UNIX
> 
> 4.0A through 4.0F all date back to the previous millennium.

Thank you for confirming.

> > In tcpdump it is a bit more entrenched, so I did not touch it yet.  
> 
> It looks as if you removed the pfopen() stuff from tcpdump's
> configure script in 43670fb635503e69cdbf8055134a0befb94d2e15.

Yes, the Autoconf problem/solution space has been getting smaller.  On
one hand, there is build logic that has been obsolete for a while, and
on the other there is logic that is very much in the loop, but has not
been converted to the 2.69 dialect yet.  Quite a few more rounds of
improvements remain to be done in both departments.

Specifically, losing Ultrix props should be among the safest things to
do.

> The AC_LBL_C_INLINE stuff is still there, but doesn't look *that*
> entrenched; are there any compilers that we need to support and that
> *don't* support C99 inline?  If not, we could just remove the call
> from configure.ac and the definition from aclocal.m4.

In 2002 in commit b1263c6 you wrote it was some HP C compiler that
Autoconf 2.13 could not drive.  I have never seen HP-UX in the wild, but
assuming the amount of improvement made in Autoconf during the
subsequent 10 years (Autoconf 2.69 is from 2012) and the amount of
improvement made in HP-UX (which had the most recent release in 2022),
to me it would make the most sense to say the problem AC_LBL_C_INLINE
solved (HP C specifics) no longer exists unless proven otherwise, and
AC_LBL_C_INLINE should be removed with a good change log entry.

I do not understand yet if AC_LBL_C_INLINE could just disappear safely
altogether instead of being replaced with AC_C_INLINE.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] AC_LBL_FIXINCLUDES does not make it into configure

2023-01-25 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
Hello all.

I have removed some unused Autoconf macros here and there, including
AC_LBL_CHECK_LIB in tcpslice and libpcap.  This macro still remains in
tcpdump, where it was a part of the initial git commit in 1999, and
where it became unused in 2001 (commit d57c77a), with the current
comment saying:

dnl We keep it around for reference purposes in case it's ever
dnl useful in the future.

If it still a good reference?

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


[tcpdump-workers] CPPFLAGS in C-only context

2023-01-23 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
Hello all.

There is one other thing in Autoconf files of tcpslice and tcpdump that
jumped out at me:

savedcppflags="$CPPFLAGS"
CPPFLAGS="$CPPFLAGS $V_INCLS"
AC_CHECK_HEADERS(pcap/pcap-inttypes.h)
AC_CHECK_FUNCS(pcap_lib_version)
CPPFLAGS="$savedcppflags"

It looks like either in a C project context CPPFLAGS works in a
non-obvious way, or is a no-op.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] AC_LBL_FIXINCLUDES does not make it into configure

2023-01-22 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Thu, 19 Jan 2023 15:57:30 -0800
Guy Harris  wrote:

> On Jan 19, 2023, at 3:20 PM, Denis Ovsienko 
> wrote:
> 
> > * AC_LBL_SSLEAY -- is there anything useful to take from here?  
> 
> No, it's been replaced by the "Check for OpenSSL/libressl libcrypto"
> code in configure.ac.

Thank you, removed.

I have also removed AC_LBL_C_INLINE and a conditional substitute for
Tru64 pfopen() from tcpslice.  Interestingly, tcpslice and tcpdump,
which don't call pfopen(), used to have this substitute, and libpcap,
which does call pfopen(), does not have it.

In tcpdump it is a bit more entrenched, so I did not touch it yet.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] AC_LBL_FIXINCLUDES does not make it into configure

2023-01-19 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Wed, 18 Jan 2023 15:53:21 -0800
Guy Harris  wrote:

> On Jan 18, 2023, at 3:32 PM, Denis Ovsienko via tcpdump-workers
>  wrote:
> 
> > As it turns out, there is another unused macro
> > (AC_LBL_HAVE_RUN_PATH), tcpslice became the first to lose this
> > luggage.  
> 
> Unused in libpcap back to 0.4 and tcpdump back to 3.4, so it may be
> another one used in some LBL projects but not libpcap or tcpdump.

So, libpcap and tcpdump have lost a few unused macros as well.  I did
NOT touch the following unused AC_DEFUNs in tcpdump aclocal.m4:

* AC_LBL_CHECK_64BIT_FORMAT -- this used to define the PRI[doux]64
  conversion specifiers before C99 became required/presumed.  I wonder
  if this presumption is subject to the same problem as the sizeof
  length modifier: the compiler might be C99, but the libc may be not.
  If trying to compensate for this [again] would be a waste of effort
  and the best move would be to delete the macro, at least the build
  system could verify that all the required printf() features are in
  place -- in fact, a single piece of C code could exercise as many
  printf() features as required in one go and fail if at least one of
  those failed.  Or would it be over-engineering?

* AC_LBL_SSLEAY -- is there anything useful to take from here?

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] AC_LBL_FIXINCLUDES does not make it into configure

2023-01-18 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Wed, 18 Jan 2023 09:22:28 -0500
Michael Richardson  wrote:

> I haven't heard a clear argument as to what autoconf/configure gets
> us today over cmake, and maintaining both seems a huge waste of
> energy.

There's a comment buried in one of the build files: a release tarball
with a configure script in it does not require Autoconf to build the
source, whereas the same tarball with CMake files in it does require
CMake to build the source.

As it turns out, there is another unused macro (AC_LBL_HAVE_RUN_PATH),
tcpslice became the first to lose this luggage.

-- 
    Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] AC_LBL_FIXINCLUDES does not make it into configure

2023-01-18 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Tue, 17 Jan 2023 18:35:58 -0800
Guy Harris  wrote:

> If you're curious what AC_LBL_FIXINCLUDES is for, and why most
> platforms don't need it, continue reading.

Thank you for explaining the context Guy, it is very educational.  Is
AC_LBL_UNION_WAIT of a similar origin?  Neither tcpdump nor libpcap use
it.

-- 
    Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


[tcpdump-workers] AC_LBL_FIXINCLUDES does not make it into configure

2023-01-17 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
Hello all.

In tcpdump commit cee234c there are three messages changed in
aclocal.m4, but only two messages changed in the resulting configure
script.  After a brief look it is clear that it is the third message
(the one in AC_LBL_FIXINCLUDES) that does not make it to the script,
but I don't understand whether this means a bug or just some dead code.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] [tcpdump] About HAVE_NO_PRINTF_Z

2023-01-14 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Thu, 12 Jan 2023 10:40:24 -0500
Michael Richardson  wrote:

> Denis Ovsienko via tcpdump-workers
>  wrote:
> > Thank you for the detailed analysis, Guy.  I did not realize
> > the support is in the libc space rather than C99 space.  Do you
> > think it would be appropriate to use [again] %lu instead of %zu
> > for size_t and %ld instead of %z for ssize_t on systems that
> > don't have %zu and %z?  
> 
> We could... but...
> It's a significant hassle to maintain this and validate this for
> future patches as well... for a product that basically nobody out
> there uses anymore.

The tests below confirm it works almost as I thought: when %z is not
available, using %l still prints the correct value, even when long is
64-bit and sizeof is 32-bit (with a compile-time warning, of course).
In other words, this is the old behaviour from several years ago:

---
int main (void)
{
int i;

printf ("sizeof (size_t) == %u\n", sizeof (size_t));
printf ("sizeof (ssize_t) == %u\n", sizeof (ssize_t));
printf ("sizeof() as %%u: %u\n", sizeof (i));
printf ("sizeof() as %%d: %d\n", sizeof (i));
printf ("sizeof() as %%lu: %lu\n", sizeof (i));
printf ("sizeof() as %%ld: %ld\n", sizeof (i));
printf ("sizeof() as %%zu: %zu\n", sizeof (i));
printf ("sizeof() as %%zd: %zd\n", sizeof (i));
return 0;
}
---
# 32-bit ARM Linux
sizeof (size_t) == 4
sizeof (ssize_t) == 4
sizeof() as %u: 4
sizeof() as %d: 4
sizeof() as %lu: 4
sizeof() as %ld: 4
sizeof() as %zu: 4
sizeof() as %zd: 4
---
# 64-bit ARM Linux
sizeof (size_t) == 8
sizeof (ssize_t) == 8
sizeof() as %u: 4
sizeof() as %d: 4
sizeof() as %lu: 4
sizeof() as %ld: 4
sizeof() as %zu: 4
sizeof() as %zd: 4
---
# SPARC and x86 Solaris 9
sizeof (size_t) == 4
sizeof (ssize_t) == 4
sizeof() as %u: 4
sizeof() as %d: 4
sizeof() as %lu: 4
sizeof() as %ld: 4
sizeof() as %zu: zu
sizeof() as %zd: zd
---

So it would be practicable to restore Solaris 9 support in the trailing
edge department of tcpdump using compile-time-conditional length
modifier.  If anybody volunteers to implement that, it would be wrong
to get in the way, I guess.

> > Whatever is the eventual solution, in any case from today's
> > perspective it makes sense to stop masking these test failures.
> >  
> 
> Agreed.
> Do you mean: Maybe just let the tests fail on Solaris, and document
> that this happens?

As the first step in any case, yes.  And make the build fail early if
the required length modifier is not supported.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] [tcpdump] About HAVE_NO_PRINTF_Z

2023-01-12 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Thu, 12 Jan 2023 00:10:22 -0800
Guy Harris via tcpdump-workers 
wrote:

> So, if we care about %z support:
> 
>   on UN*Xes, we'd have to drop support for OSes that don't
> provide a C library that supports it;

Thank you for the detailed analysis, Guy.  I did not realize the support
is in the libc space rather than C99 space.  Do you think it would be
appropriate to use [again] %lu instead of %zu for size_t and %ld instead
of %z for ssize_t on systems that don't have %zu and %z?

For that the HAVE_NO_PRINTF macro would have to be ported to CMake, but
then the currently conditional tests would be able to pass everywhere
(i.e. on Solaris 9).

Whatever is the eventual solution, in any case from today's perspective
it makes sense to stop masking these test failures.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] Autoconf with Debian patches

2023-01-08 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Sat, 7 Jan 2023 18:47:37 -0800
Guy Harris  wrote:

> On Jan 7, 2023, at 8:51 AM, Denis Ovsienko 
> wrote:
> 
> > On Fri, 6 Jan 2023 17:13:20 -0800
> > Guy Harris  wrote:
> >   
> >> On Jan 6, 2023, at 3:31 PM, Denis Ovsienko 
> >> wrote:
> >>   
> >>> It is the latter, and a custom Autoconf seems an unreasonable
> >>> requirement for contributing.
> >> 
> >> Reasonable, or unreasonable?  
> > 
> > Unreasonable, if it is more complicated than installing an Autoconf
> > package using the package manager of the OS.  
> 
> Which it is likely to be.

Okay, let's not go out of the packaged Autoconf space then.

> >> (By the way, have other Linux distributions applied the same
> >> changes that Debian and its derivatives have?  If not, then users
> >> of those distributions would be in the same situation as macOS and
> >> FreeBSD users.)  
> > 
> > I do not remember to what extent these patches have propagated
> > beyond Debian and Ubuntu.  Maybe somebody else has other
> > distributions ready to check?  
> 
> Fedora 36 and later appear to ship autoconf 2.71; the Debian sid
> package for autoconf 2.71 applies no patches to it, as, I presume,
> all of the Debian packages are applied (the off_t patch is already
> incorporated in 2.71).  Debian's currently shipping 2.69, which
> requires their pile of patches.

Thank you for this information.  Let me add that Ubuntu 20.04 defaults
to 2.69, but Ubuntu 22.04, FreeBSD, NetBSD, OpenBSD and OmniOS all
currently default to Autoconf 2.71.  Would it make the most sense to
make 2.71 the nominal version (especially for releases), but to maintain
backward compatibility with 2.69 for quite a while longer?

> Fedora shipped autoconf 2.69, without a patch like the Debian off_t
> patch but with a patch like the Debian "add runstatedir" patch.  I
> don't know what RHEL has.
> 
> Looking at the Arch Linux repository, there doesn't appear to be a
> version of the off_t patch from when they shipped 2.69; they're
> currently shipping 2.71.  The same applies to Gentoo.
> 
> But at least some of them have 2.71 patches, so there's no guarantee
> that all the releases that have 2.71 will generate exactly the same
> script.

Identical behaviour everywhere would be unrealistic.  Minimizing the
average noise level in pull requests should be realistic, although not
critical, but nice to have.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] Has anyone got a clang-format for the tcpdump style?

2023-01-07 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Wed, 4 Jan 2023 08:40:21 -0500
Bill Fenner via tcpdump-workers 
wrote:

> Hi,
> 
> I know the tcpdump style follows a bunch of bsd patterns, since it
> came from Berkeley in the first place.  Does anyone have a
> clang-format config that reflects these coding conventions?  One of
> the problems I have in upstreaming Arista-developed tcpdump code is
> making sure that the code fits in well with its surroundings, and
> using clang-format for that purpose would sure be easier.

Bill, do you know if it would be practicable to apply code style per
file instead of doing a flag day on entire repository?

Also it might help before enforcing the style to get tcpdump 5.0 ready
(to remove backporting into 4.99 from the problem space) and to
merge/close as many pull requests as possible.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] Autoconf with Debian patches

2023-01-07 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Fri, 6 Jan 2023 17:13:20 -0800
Guy Harris  wrote:

> On Jan 6, 2023, at 3:31 PM, Denis Ovsienko 
> wrote:
> 
> > It is the latter, and a custom Autoconf seems an unreasonable
> > requirement for contributing.  
> 
> Reasonable, or unreasonable?

Unreasonable, if it is more complicated than installing an Autoconf
package using the package manager of the OS.

> Whatever version is chosen as the standard autoconf, if the goal is
> to have the version of the configure script in the repository always
> be generated by the standard autoconf, some users will have to build
> and install that version if they will be changing the configure
> script, and, for other contributions, they'll either have to build or
> install that version or they will have to take care not to check in
> the configure script if they haven't changed configure.ac or
> aclocal.m4.
> 
> (By the way, have other Linux distributions applied the same changes
> that Debian and its derivatives have?  If not, then users of those
> distributions would be in the same situation as macOS and FreeBSD
> users.)

I do not remember to what extent these patches have propagated beyond
Debian and Ubuntu.  Maybe somebody else has other distributions ready to
check?

> > Or the --runstatedir and LARGE_OFF_T bits between releases could
> > appear and disappear at random  
> 
> Meaning we let users check in the configure script in whatever form
> it exists on their machine?

Yes.  The obvious drawbacks of that would be lowering signal-to-noise
ratio in the diffs and potentially making subsequent git-cherry-pick
more likely to fail due to merge conflicts.

> > until it is a release time, then the standard
> > could be enforced as and if necessary.  
> 
> I.e., part of the process of making a release would be regenerating
> the configuration file using Debian autoconf and checking in the
> regenerated version.?

Yes.  This bit looks relatively simple.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] Autoconf with Debian patches

2023-01-06 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Fri, 6 Jan 2023 14:49:54 -0800
Guy Harris  wrote:

> On Jan 6, 2023, at 2:24 PM, Denis Ovsienko 
> wrote:
> 
> > On Fri, 6 Jan 2023 13:25:14 -0800
> > Guy Harris  wrote:
> >   
> >> If we switch to making Debian Autoconf the new standard and keeping
> >> the generated configure script in the repository, would that mean
> >> that developers working from the repository would either have to
> >> install Debian Autoconf or use "git add -p" instead of "git add"?  
> > 
> > Yes.  Right now it is the other way around (contributors that use
> > Debian or its derivatives have to filter their output).  So perhaps
> > this switch would not be convenient for macOS and FreeBSD users.  
> 
> If we go that way, we should document it when addressing developers.

Yes, and it would be nice to have this documented even if we don't
(currently the contribution guidelines do not discuss this).

> Is there a place where people can download a tarball for Debian
> autoconf and just do ./configure, make, and make install, or will
> they have to download the Debian package and apply the patches?  If
> the latter, we should, at minimum, give documentation on how to do
> that - or we could just do that ourselves and have a "Debian
> autoconf" source tarball to download.

It is the latter, and a custom Autoconf seems an unreasonable
requirement for contributing.

> An alternative would be *not* to keep the generated configure script
> in the repository (that's what Wireshark ended up doing before it
> ceased to use autoconf/automake), and generate it as part of the
> release-build process, which we would do on a machine on which Debian
> autoconf was installed.

Or the --runstatedir and LARGE_OFF_T bits between releases could appear
and disappear at random until it is a release time, then the standard
could be enforced as and if necessary.

> That requires that developers have autoconf installed if they're not
> going to be using CMake, but there are already tools they need
> installed (a C compiler, make, Flex, Bison/Berkeley YACC, ...) so I
> don't see that as a problem.
> 
> It also means that configure.ac and aclocal.m4 would have to work
> with various sufficiently-recent versions of autoconf.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] Autoconf with Debian patches

2023-01-06 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Fri, 6 Jan 2023 13:25:14 -0800
Guy Harris  wrote:

> If we switch to making Debian Autoconf the new standard and keeping
> the generated configure script in the repository, would that mean
> that developers working from the repository would either have to
> install Debian Autoconf or use "git add -p" instead of "git add"?

Yes.  Right now it is the other way around (contributors that use
Debian or its derivatives have to filter their output).  So perhaps
this switch would not be convenient for macOS and FreeBSD users.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


[tcpdump-workers] Autoconf with Debian patches

2023-01-04 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
Hello all.

As some have experienced before, attempts to regenerate the configure
script often result in two groups of unnecessary changes (runstatedir
and LARGE_OFF_T), both of which come from Debian-specific patches to
Autoconf because traditionally the configure scripts were generated
using non-Debian Autoconf.  In practice this means that a regenerated
revision of a configure script almost always requires "git add -p"
instead of "git add".

This has been discussed in some detail in [1], and my understanding is
that making Debian Autoconf the new standard should make this problem
smaller (it certainly would in my development environment).  Would
anybody like to make their point for or against such a switch in one of
the next releases?

1: https://github.com/the-tcpdump-group/tcpslice/pull/12

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


[tcpdump-workers] tcpdump 4.99.2 & libpcap 1.10.2

2022-12-31 Thread Denis Ovsienko via tcpdump-workers
g-control.h to handle compiling with clang-cl (issues
#1101 and #1115).
  Cleanup various leftover cruft in the configure script.
  Fix building without protochain support. (GH #852)
  Check for a usable YACC (or Bison) and {F}lex in CMake, as we do
in autotools.
  Only check for a C++ compiler on Haiku, as that's the only
platform with C++ code, and make sure they generate code for
the same instruction set bit-width (both 32-bit or both 64-bit)
(issue #1112).
  On Solaris, check the target bit-width and set PKG_CONFIG_PATH
appropriately, to handle the mess that is the D-Bus library
package (issue #1112).
  Fix generation of pcap-config and libpcap.pc files (issue #1062).
  pcap-config: don't assume the system library directory is /usr/lib.
  pcap-config: add a --static-pcap-only flag.
  Cirrus CI: Use the same configuration as for the main branch.
  Add four libpcap test files.
  Update Npcap SDK to 1.13.
  Makefile.in: Use TEST_DIST, like for tcpdump.
  Remove awk code from mkdep.
  Cirrus CI: Add the libssl-dev package in the Linux task.
  Cirrus CI: Add the openssl@3 brew package in the macOS task.
  Get "make shellcheck" to pass again.
  CMake: Build valgrindtest only if Autoconf would.
  CMake: use ${CMAKE_INSTALL_SBINDIR} rather than just sbin.
  CMake: use NUL: as the null device on Windows.
  autoconf: fix typo in test of macOS version.
  Makefile.in: Add two missing files in EXTRA_DIST.
  autotools, cmake: provide an rpath option if necessary.
  configure: get rid of the attempt to auto-run PKG_PROG_PKG_CONFIG.
  configure: use PKG_CHECK_MODULES to run pkg-config.
Documentation:
  Add README.solaris.md.
  Add SCTP to pcap-filter(7).
  Note that = and == are the same operator in filters (issue #1044).
  Update INSTALL.md, README.md, and README.solaris.md.
  Update and clean up CONTRIBUTING.md.
  Trim documentation of support for now-dead UN*Xe and older
versions of other UN*Xes.
  Move the "how to allocate a LINKTYPE_/DLT_ value" documentation to
the web site.
  Clean up man pages.
  Move README.capture-module to the web site.
  Improve some protocol details in pcap-filter(7).
  Refine "relop" notes in pcap-filter(7).
  In pcap-filter(7) "domain" is an id.
  Discuss backward compatibility in pcap-filter(7).
  Other improvements to pcap-filter(7).
  Document pcap_breakloop(3PCAP) interaction with threads better.
  Document PCAP_ERROR_NOT_ACTIVATED for more routines.
8<8<8<8<8<8<

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


[tcpdump-workers] CI news November-December 2022

2022-12-19 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
Hello all.

I hope this finds you well.  Below you can find a digest of the CI
infrastructure works since the previous update.

* macOS CI has been switched from AMD64 to AArch64 following some
  infrastructure changes in Cirrus CI.  Everything seems to work as
  before.
* On freebsd-aarch64 GCC has been upgraded from 12.1 to 12.2 and Clang
  has been upgraded from 14.0 to 15.0.  The latter upgrade started to
  produce a couple additional warnings, one of which still remains to be
  addressed.
* openbsd-amd64 and openbsd-mips64 have been upgraded from 7.1 to 7.2,
  without apparent changes.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] Public Release of Z-Wave G.9959 TAP Specification

2022-12-08 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Tue, 6 Dec 2022 07:55:06 -0800
Chris Brandson via tcpdump-workers 
wrote:

> Hello Denis,
> 
> I agree with all of your suggestions, have made the changes and
> pushed to the repo.

Thank you Chris.

I hope you don't mind one more question about the encoding.  Would it be
correct to say that the only reliable way to access the MPDU beyond the
Length field (for example, to start reading the MSDU) would be as
follows?

1. Iterate through the TLV space until an RF Info TLV is found or there
   is no more TLVs.
2. If the TLV was found, read Region, Data Rate and Frequency from the
   TLV and derive Channel Configuration ("1,2" or "3"), otherwise
   default to "1,2".
3. Somehow (by parsing MHR?) tell if the MPDU is singlecast or
   multicast.
4. From Channel Configuration and singlecast/multicast tell which one of
   the four possible MPDU formats to use.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] Public Release of Z-Wave G.9959 TAP Specification

2022-12-05 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Mon, 5 Dec 2022 11:53:11 -0800
Chris Brandson via tcpdump-workers 
wrote:

> Hello everyone,
> 
> We are pleased to publish the draft Z-Wave G.9959 TAP Specification
[...]

Hello Chris.

Thank you for preparing the document.  I am not familiar with the
standard to make more substantial comments, but a few things seem to
require some attention:

* Page 7: "Length is a minimum of 4 octets and must be a multiple of 4.
  The addition of new TLVs does not and must not require incrementing
  the version number." -- this definition automatically creates a
  problem space on the receiving/reading end of this encoding because
  slightly more than 75% of all possible length values are invalid by
  definition. The classic solution would be not to include the first 4
  octets into the length, and to count in multiples of 4, this way any
  length value would be valid, free of underflows and better consumable
  by simple parsers such as BPF bytecode.

* Page 8: "Z-Wave PHY Payload" -- is this the PPDU from Figure A.7 of
  G.9959 2015/01, starting from "preamble sequence"?

* Page 9: "length - number of octets for type in value field (not
  including padding)" -- this definition looks better, but in the next
  three TLV diagrams the length does include the T and the L.

* Page 10: If the list of region code points is a copy/subset of an
  external registry, it would help to give an exact reference.  Ditto
  bit rate code points.  Clearly there is some relation with tables A.1
  and A.2 from G.9959 2015/01.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


[tcpdump-workers] macOS CI is switching from AMD64 to AArch64

2022-11-26 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
Hello all.

Apparently, the current provider of macOS CI is switching their
infrastructure from AMD64 to AArch64 (M1) by the end of this year, so
tcpslice, tcpdump and libpcap are going to follow the suit, which will
also upgrade macOS from 11 to either 12 or 13 (to be decided).  If you
have any comments or suggestions in this regard, please post to the
list.

Cheers.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


[tcpdump-workers] CI news September-October 2022

2022-10-30 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
Hello list.

I hope this finds you well.  Below you can find a digest of the CI
infrastructure works since the previous update.

* OmniOS has seen some improvements to its Clang packages, so
  illumos-amd64 now uses Clang 15.0 (was 11.1).

* netbsd-amd64 has been upgraded from NetBSD 9.2 to 9.3, this among
  other things upgraded GCC from 10.3 to 12.1 and Clang from 13.0 to
  13.1; thanks to a recent bug fix by Guy Harris tcpdump now can build
  with the pkgsrc version of libpcap when using CMake, this is now a
  part of the build matrix on this worker.

* netbsd-aarch64 has been upgraded from NetBSD 9.2 to 9.3 and from
  pkgsrc 2022Q2 to 2022Q3, this among other things upgraded GCC from
  12.1 to 12.2 and Clang from 13.0 to 14.0; the system libpcap finally
  has a bug fix for the long-standing NetBSD bug 55901 and can be (and
  now is) used in tcpdump builds.

* linux-riscv64 has been switched from Fedora 36 to Ubuntu 22.04,
  neither of which is perfect, but the latter seems to be easier to
  maintain; this upgrades GCC from 10.3 to 11.2, but for now removes
  Clang, which turned out to be broken on this specific hardware
  (Ubuntu bug 1994071).

Cheers.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


[tcpdump-workers] tcpslice 1.6

2022-10-20 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
Hello list.

tcpslice 1.6 has been released with the following change log.  Thanks
to everyone who contributed!


- Call pcap_dump_close() on the output file.

- Implement new flags in ./configure: --enable-instrument-functions,
  --without-libnids, --without-libosipparser2 and --without-libooh323c.

- autoconf: Add the option to print functions and files names

- Update config.{guess,sub}, timestamps 2022-01-09,2022-01-03

- configure: use pcap-config --static-pcap-only if available

- Remove awk code from mkdep.

- Refine the man page.

- Refine the documentation files.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] upcoming tcpslice release

2022-10-16 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Sat, 15 Oct 2022 15:21:42 -0700
Guy Harris  wrote:

> On Oct 15, 2022, at 8:03 AM, Denis Ovsienko via tcpdump-workers
>  wrote:
> 
> > As it turns out, on Linux tcpslice currently fails to build with the
> > current master branch of libpcap.  This reproduces in all Linux CI
> > builds and also on my Ubuntu 20.04 PC.  The root cause seems to be
> > in libpcap via pcap-config:
> > 
> > /usr/bin/ld: cannot find -lsystemd
> > clang: error: linker command failed with exit code 1 (use -v to see
> > invocation)
> > 
> > LIBS='../libpcap/libpcap.a -lnl-genl-3 -lnl-3  -ldbus-1 -lpthread
> > -lsystemd  '  
> 
> Fixed in
> 
> commit 588f0bb996230a84a8cf10ddf30cc514e3ba5a68 (HEAD -> master,
[...]

Thank you Guy.

> This should only be an issue for programs that link statically with
> libpcap (libpcap.a) but don't link completely statically.  I don't
> know if anything does that other than tcpdump (from which this change
> was taken) and tcpslice, if they're building with a libpcap.a from a
> libpcap source and build tree in the same parent directory as the
> tcpdump/tcpslice source and build tree.

Is this semantics worth adding to pcap-config(1)?

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] upcoming tcpslice release

2022-10-15 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
As it turns out, on Linux tcpslice currently fails to build with the
current master branch of libpcap.  This reproduces in all Linux CI
builds and also on my Ubuntu 20.04 PC.  The root cause seems to be in
libpcap via pcap-config:

/usr/bin/ld: cannot find -lsystemd
clang: error: linker command failed with exit code 1 (use -v to see invocation)

LIBS='../libpcap/libpcap.a -lnl-genl-3 -lnl-3  -ldbus-1 -lpthread -lsystemd  '

With earlier snapshots of libpcap the problem does not happen.  I tried
to bisect libpcap, but without much success.  For now I tend to see it
as a libpcap issue that should not block tcpslice release, but other
opinions are welcome.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


[tcpdump-workers] upcoming tcpslice release

2022-10-13 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
Hello list.

Unless anyone objects, the next release of tcpslice (1.6) will be made
soonish to flush the few buffered minor bugfixes and to take another
item off my to-do list.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


[tcpdump-workers] ARMv7 unaligned access

2022-08-21 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
Hello list.

Based on some prior online research, I used to think that ARMv7 is a
hardware that allows (by default or after simple tweaking) to terminate
a userspace process when it tries to perform an unaligned memory
access.  Which is exactly the behaviour wanted for the CI purposes.

Not long ago I moved the linux-armv7l Buildbot worker to a true ARMv7
board (RPI2B v1.1).  Consistent with the documentation [1], the kernel
seems to have the right knob in place, which a boot-time script sets to
the desired position:

$ cat /proc/cpu/alignment
User:   0
System: 0 (0x0)
Skipped:0
Half:   0
Word:   0
DWord:  0
Multi:  0
User faults:4 (signal)

However, this does not seem to have any effect, in that this short C
program, which does trigger a SIGBUS/SIGILL on other OSes and
architectures that do not like unaligned access, just completes
successfully on linux-armv7l:

-
#include 
#include 
/*
#include 
*/

int main (void)
{
/*
sysmips (MIPS_FIXADE, 0);
*/
uint32_t *p;
union
{
uint32_t u32[2];
uint8_t u8[8];
} u;
u.u32[0] = 0;
u.u32[1] = 1;
p = (uint32_t *)(u.u8 + 0);
*p = 3;
p = (uint32_t *)(u.u8 + 1);
*p = 3;
p = (uint32_t *)(u.u8 + 2);
*p = 3;
p = (uint32_t *)(u.u8 + 3);
*p = 3;

uint8_t u8[20];
for (size_t i = 0; i < 10; i++)
{
p = (uint32_t *)(u8 + i);
*p = 4;
}

return 0;
}
-

Is there anything obvious I am missing in these interpretations?

1: https://docs.kernel.org/arm/mem_alignment.html

-- 
    Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] configure script problem while working on extention

2022-08-15 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Sun, 14 Aug 2022 11:49:57 -0700
Guy Harris via tcpdump-workers 
wrote:

> Or is this a ZIP archive provided by somebody other than tcpdump.org?

github.com -> code -> download ZIP. I vaguely remember there was a
"download tar.gz" there as well, but not anymore.  Anyway, git clone is
better order of magnitudes, in that it allows to tell which commit the
working copy is at.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


[tcpdump-workers] CI news June-August 2022

2022-08-14 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
Hello list.

Below you can find a digest of the CI infrastructure works since the
previous update.

* FreeBSD/AArch64 in release 13.1 notably improved its ability to run as
  a virtual machine, and freebsd-aarch64 has been migrated to OSU OSL
  infrastructure.  Thanks to sysmocom for hosting this Buildbot worker
  for more than a year on their own RPI and thanks to OSU OSL for
  sponsoring the VM.  Upgrading FreeBSD from 13.0 to 13.1 has also
  upgraded Clang from 11.0 to 13.0.
* netbsd-aarch64 pkgsrc has been upgraded from 2022Q1 to 2022Q2, which
  has upgraded GCC from 10.3 to 12.1.
* netbsd-mips64 has been upgraded from 9.99.93 to 9.99.98 and pkgsrc has
  been upgraded from 2021Q4 to 2022Q2.
* openbsd-mips64 has been upgraded from 7.0 to 7.1, which has upgraded
  Clang from 11.1 to 13.0.
* linux-armv7l hardware has been downgraded to RPI2B v1.1 (ARMv7-A).

Cheers.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] configure script problem while working on extention

2022-08-14 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Fri, 12 Aug 2022 16:27:29 +0200
Christian via tcpdump-workers  wrote:

> Then I opened the tcpdump.zip archive within the libpcap directory.
> step into the directory, call ./configure and it build. success!

The git clone of libpcap and the git clone of tcpdump are supposed to
be next to each other, not one inside the other.  Also the libpcap
directory must be named "libpcap" because tcpdump build process is
checking in ../libpcap for the "local" libpcap.  If the check is
unsuccessful, tcpdump tries to build with the "system" libpcap.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] About formalization of pcap expressions

2022-08-14 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Sat, 23 Jul 2022 17:24:12 +0200
Francois-Xavier Le Bail via tcpdump-workers
 wrote:

> Hi,
> 
> FYI, the author of the following document try a formalization of pcap
> expressions:
> https://www.seas.upenn.edu/~nsultana/files/pcap_semantics.pdf In this
> document, reference to a tool: https://gitlab.com/niksu/caper

The tool now produces one of the outputs for BPF Exam and can be tried
without the need to rig up a local OCaml environment.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] NetBSD CI breakage

2022-07-14 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Wed, 13 Jul 2022 23:16:18 -0700
Guy Harris  wrote:

> I changed it to a slightly different fix.

Thank you.  The signed device index hinted at some logic error, but I
didn't understand if it required a more sophisticated fix.

-- 
    Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


[tcpdump-workers] NetBSD CI breakage

2022-07-10 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
Hello all.

The last CI build of the libpcap-1.10 branch failed on netbsd-aarch64
because the latter now uses GCC 12.  Commit 4e7f6e8 makes a lazy fix
for that in the master branch; if a more sophisticated solution is not
required, a simple cherry-pick into libpcap-1.10 should be sufficient
to pass CI again.

Cheers.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] What's the correct new API to request pcap_linux to not open an eventfd

2022-07-05 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Fri, 1 Jul 2022 13:55:30 -0400
Bill Fenner via tcpdump-workers 
wrote:

> If we set
> pcap_nonblock after pcap_create and before pcap_activate, we get -3 -
> which I don't get at all, unless, -3 means "you didn't activate the
> pcap yet". My naive reading of the Linux pcap_getnonblock code says
> it'll return the integer value of a bool, and I don't know how that
> can be -3.

Hello Bill.

The pcap_setnonblock(3PCAP) man page describes two functions:

int pcap_setnonblock(pcap_t *p, int nonblock, char *errbuf);
int pcap_getnonblock(pcap_t *p, char *errbuf);

If your comments concern pcap_getnonblock(), what you describe is
consistent with the documented behaviour:

In pcap/pcap.h:

#define PCAP_ERROR_NOT_ACTIVATED-3  /* the capture needs to
be activated */

In the man page:

RETURN VALUE
   pcap_getnonblock()  returns  the  current ``non-blocking''
   state of the capture descriptor; it always  returns  0  on
   ``savefiles''.   If  called  on  a capture handle that has
   been created but not  activated,  PCAP_ERROR_NOT_ACTIVATED
   is returned.  If there is another error, PCAP_ERROR is re‐
   turned and errbuf is filled in with an  appropriate  error
   message.

Did you mean there is an issue with pcap_setnonblock()?

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] endianness of portable BPF bytecode (DRAFT revision 4)

2022-06-30 Thread Denis Ovsienko via tcpdump-workers
rd whether optimization  was  requested  for  the
   compilation or not, usually this is the optimize input argument
   to pcap_compile().  Note that some link-layer header types  and
   filter keywords disable the optimization automatically in libp‐
   cap.

   Type is 3, Length is 1, Value contains 1 or 0.

   Netmask TLV
   Allows to  record  the  value  of  netmask  input  argument  to
   pcap_compile().

   Type is 4, Length is 4, Value contains a 32-bit IPv4 netmask.

   Comment TLV
   Allows  to  record  a free-form text, for example, the name and
   version of the program that generated the file.

   Type is 5, Length is variable, Value contains a UTF-8 string.

   Timestamp TLV
   Allows to record when the compilation was performed.

   Type is 6, Length is 8, Value contains a 64-bit Unix timestamp.

SOFTWARE SUPPORT
   None at the time of this writing.

SEE ALSO
   pcap-savefile(5)

 30 June 2022 CBPF-SAVEFILE(5)

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] endianness of portable BPF bytecode (DRAFT revision 3)

2022-06-25 Thread Denis Ovsienko via tcpdump-workers
ent  the  major
   version and reset the minor version to 0.

   InstructionCount is the last field of the fixed header, it con-
   tains the number of bytecode instructions following the header.
   By  convention, valid BPF bytecode must consist of at least one
   instruction, so in a valid cBPF savefile this field value is at
   least 1.

   The  file  format  thus far minimizes the overhead for software
   that only writes or reads cBPF bytecode.  If there is any  data
   after the last instruction, it is the trailing TLV space, which
   mostly contains meta-data for human  interpretation.   It  con-
   tains TLVs in the format specified below.

INSTRUCTION FORMAT
0   1   2   3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | opcode|   jt  |   jf  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   k   |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   This  is  the traditional encoding of a cBPF instruction.  Note
   that usually its endianness depends on the machine, but in this
   format it is fixed.

TLV FORMAT
0   1   2   3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |  Type |Length=m   |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   |
   ~  Value (m bytes)  ~
   |   |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   All  TLVs  are optional.  Every TLV may appear in the same cBPF
   savefile at most once.  Length value does not include Type  and
   Length.   Code  points  for Type and the associated Length con-
   straints are defined below.

   EOF TLV
   Allows to mark the end of TLV space (hence of the file) explic-
   itly  to make it clear that the file is not truncated.  If this
   TLV is present in the TLV space, it may appear the last only.

   Type is 0, Length is 0, Value is empty.

   LinkTypeValue TLV
   Allows to record the link-layer header type value used for  the
   compilation, usually this is either the linktype input argument
   to  pcap_open_dead(3PCAP)  or  the  dlt   input   argument   to
   pcap_set_datalink(3PCAP).  By convention link-layer header type
   values are limited to 16 bits.

   Type is 1, Length is 2, Value contains an integer.

   LinkTypeName TLV
   Allows to record the input argument  to
   pcap_datalink_name_to_val(3PCAP)  if  the  latter  was  used to
   translate a DLT name into the DLT  value  (the  same  name  can
   sometimes produce different values in different contexts).

   Type is 2, Length is variable, Value contains an ASCII string.

   SnapLen TLV
   Allows  to record the snapshot length used for the compilation,
   usually this is the snaplen input argument to  pcap_open_dead()
   or pcap_set_snaplen(3PCAP).

   Type is 3, Length is 4, Value contains an integer.

   Filter TLV
   Allows  to  record the filter expression that was compiled into
   the bytecode,  usually  this  is  the  str  input  argument  to
   pcap_compile(3PCAP).

   Type is 4, Length is variable, Value contains an ASCII string.

   OptReq TLV
   Allows  to  record  whether  optimization was requested for the
   compilation or not, usually this is the optimize input argument
   to  pcap_compile().  Note that some link-layer header types and
   filter keywords disable the optimization automatically in libp-
   cap.

   Type is 5, Length is 1, Value contains 1 or 0.

   Netmask TLV
   Allows  to  record  the  value  of  netmask  input  argument to
   pcap_compile().

   Type is 6, Length is 4, Value contains a 32-bit IPv4 netmask.

   Comment TLV
   Allows to record a free-form text, for example,  the  name  and
   version of the program that generated the file.

   Type is 7, Length is variable, Value contains a UTF-8 string.

   Timestamp TLV
   Allows to record when the compilation was performed.

   Type is 8, Length is 8, Value contains a 64-bit Unix timestamp.

SOFTWARE SUPPORT
   None at the time of this writing.

SEE ALSO
   pcap-savefile(5)

 24 June 2022 CBPF-SAVEFILE(5)

-- 
Denis Ovsienko
--- End Message ---
_

Re: [tcpdump-workers] NetBSD breakage

2022-06-21 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Thu, 12 Aug 2021 01:18:45 +0100
Denis Ovsienko via tcpdump-workers 
wrote:

> On Wed, 11 Aug 2021 15:38:34 -0700
> Guy Harris  wrote:
> 
> > I've checked in a change to remove the include of grammar.h from
> > gencode.c; it builds without problems on macOS, and I suspect it
> > will build without problems everywhere  
> 
> Thank you, NetBSD has recovered now. I am going to remove
> HAVE_SYS_TYPES_H from pcap/bpf.h because it does not belong there.

As usual, better late than never, so the above has been done in commit
19e2a37 in the master branch.  I am going to cherry-pick that into the
libpcap-1.10 branch before the 1.10.2 release.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] pcap_compile_nopcap() not in man pages

2022-06-14 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Wed, 12 Aug 2020 12:59:35 -0700
Guy Harris  wrote:

[...]

> So, for now, my inclination is to 1) deprecate pcap_compile_nopcap()
> (complete with marking it as deprecated, so code that uses it gets a
> compile-time warning on compilers where the deprecation macro is
> supported) and 2) not document it.

Done and done.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] endianness of portable BPF bytecode

2022-06-11 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Sat, 11 Jun 2022 10:51:41 -0400
Michael Richardson  wrote:

> Denis Ovsienko via tcpdump-workers
>  wrote:
> > Below is a draft of such a file format.  It addresses the
> > following needs:
> > * There is a header with a signature string to avoid false
> > positive detection as some other file type that begins exactly
> > with particular bytecode (ran into this during disassembly
> > experiments).  
> 
> Looks good, but I suggest that rather than 'savefile', that it
> contain something about the program that wrote it, and then a CRC32
> on the first 12 bytes.
> That protects against a .txt file that starts with with cBPFsavefile
> :-)

Thank you for the feedback.  The essential thing is to declare what the
file is, in that sense I agree a binary marker would work better.  It
could be, for example, 0xa1, 0xb2, 0xc3, 0xcb (along the lines of
sf-pcap.c, but not yet taken).  With that in place, the program that
generates the file can leave a "XYZ BPF compiler was 'ere" string in an
optional TLV, if it wants.

The CRC is an interesting point, but it would need to cover more data
to make a meaningful difference in the sense of data integrity.  Thus it
would need to be as close to the end of file as possible in order not
to block/buffer subsequent output data (fseek() does not work on pipes).
This way, to be a part of the initial version, the CRC would need to be
very carefully defined.

> https://datatracker.ietf.org/doc/draft-ietf-cbor-file-magic/ is
> waiting on editing, and deals with these issues from a CBOR point of
> view. Instructions *could* just be a CBOR array or CBOR arrays

Maybe in a later version.  Let's keep the initial encoding as simple as
possible, for both reading and writing.

> > TLV format:  
> 
> I'm really not keen for Yet-Another-TLV format.
> I don't think that the disk space is so constrained to use 24-bits
> rather than 32-bits for the TL header.

Fine, let it be 16+16 bits then.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] endianness of portable BPF bytecode

2022-06-10 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Fri, 10 Jun 2022 14:26:34 -0700
Guy Harris  wrote:

> On Jun 10, 2022, at 1:59 PM, Denis Ovsienko via tcpdump-workers
>  wrote:
> 
> > Below is a draft of such a file format.  It addresses the following
> > needs:
> > * There is a header with a signature string to avoid false positive
> >  detection as some other file type that begins exactly with
> > particular bytecode (ran into this during disassembly experiments).
> > * There are version fields to address possible future changes to the
> >  encoding (either backward-compatible or not).  
> 
> Is the idea that a change that's backward-compatible (so that code
> that handles the new format needs no changes to handle the old
> format, but code that handles only the old format can't handle the
> new format) would involve a change to the minor version number, but a
> change that's not backward-compatible (so that to handle both
> versions would require two code paths for the two versions) would
> involve a change to the major version number?

Yes, more or less.  The draft format had a couple more fields not long
ago, with those the version semantics seemed more apparent (for a while
I thought Linux kernel cBPF is a superset of libpcap cBPF, but upon a
closer inspection of the header files they seem identical).

In any case, it is very convenient to be able to cycle a major version
and to redefine everything beyond the signature and the version fields,
that's the idea. Forward- and backward-compatibility between minor
versions can be considered now.

> > File format:
> > 
> > 0   1   2   3
> > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
> > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> > |  'c'  |  'B'  |  'P'  |  'F'  |
> > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+  
> 
> Is the 'c' part of the retronym "cBPF" for the "classic BPF"
> instruction set, as opposed to the eBPF instruction set?  (I didn't
> find any file format for saving eBPF programs, so this format could
> be used for that as well, with the magic number 'e' 'B' 'P' 'F'.)

Yes, it is.  In online documentation "eBPF" seems to clash with "BPF"
a lot, so it seems better to avoid the confusion early.

As it turned out after some research, the nominal binary format for eBPF
is ELF.  This is one of the most useful online documents I found:
https://www.man7.org/linux/man-pages/man8/tc-bpf.8.html
As you can see there and in the references into Linux kernel
documentation, ELF eBPF seems to cover different bit widths, relocation
types, debug information, lookup maps, multiple executable sections and
what not.

However, most of these features significantly overshoot the packet
capture problem space on one hand, and don't seem to address simple
practical needs of capturing parameters and context of a cBPF
compilation and reproducing it later.  So I figured it would be better
to leave eBPF solution space alone and to use a separate
purpose-designed file format for cBPF.

Most of the meta-data TLVs below are purposed to help a developer to
understand the context and reproduce the compilation.

> > Type=0x02 (LINKTYPE_ID)
> > Length=4
> > Value=  
> 
> This could be 2 bytes long - pcapng limits link-layer types to 16
> bits, and pcap now can use the upper 16 bits of the link-layer type
> field for other purposes.

Fine.

> > Type=0x03 (LINKTYPE_NAME)
> > Length is variable
> > Value=  
> 
> E.g. either its LINKTYPE_xxx name or its DLT_xxx name?

Yes.  The intent is to capture the input to
pcap_datalink_name_to_val() if the latter was involved.

> > Type=0x04 (COMMENT)
> > Length is variabe
> > Value= > description>  
> 
> "Generating software description" as in the code that generated the
> BPF program?

"libpcap x.y.z", "my script v1.0" or something like that.

> > Type=0x05 (TIMESTAMP)
> > Length=8
> > Value=  
> 

> Is this the time the code was generated?

Yes.

> Is it a 64-bit time_t, or a 32-bit time_t and a 32-bit
> microseconds/nanoseconds value?  I'd recommend the former, unless we
> expect classic BPF to be dead by 2038.

It is the 64-bit integer time.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] endianness of portable BPF bytecode

2022-06-10 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Thu, 2 Jun 2022 20:58:38 +0100
Denis Ovsienko via tcpdump-workers 
wrote:

> If there is no convention in place yet, I would like to propose
> declaring big-endian as the implicit/default byte order, then
> particular file format(s) with headers can override that as needed.

Below is a draft of such a file format.  It addresses the following
needs:
* There is a header with a signature string to avoid false positive
  detection as some other file type that begins exactly with particular
  bytecode (ran into this during disassembly experiments).
* There are version fields to address possible future changes to the
  encoding (either backward-compatible or not).
* The explicit instruction count enables detection of truncated or
  malformed files on one hand, on the other it divides the bytecode
  from the optional meta-data.
* The optional meta-data allows to captures some of the bytecode
  compilation context to aid in debugging.

All multiple-byte fields are big-endian.

File format:

 0   1   2   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|  'c'  |  'B'  |  'P'  |  'F'  |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|  's'  |  'a'  |  'v'  |  'e'  |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|  'f'  |  'i'  |  'l'  |  'e'  |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|   MajorVer=0  |MinorVer   |   InstructionCount=n  |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|   |
| instruction 1 |
|   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|   |
| instruction 2 |
|   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|   |
~   ~
|   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|   |
| instruction n |
|   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|   |
|   optional trailing TLV space |
|   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


Instruction:

 0   1   2   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| opcode|   jt  |   jf  |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|   k   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


TLV format:

 0   1   2   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|  Type | Length|   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+   ~
|   |
~  Value (0 or more bytes)  ~
|   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Length value does not include Type and Length.

Type=0x00 (FILTER)
Length is variable
Value=

Type=0x01 (OPTIMIZED)
Length=1
Value= (0 for off, 1 for on)

Type=0x02 (LINKTYPE_ID)
Length=4
Value=

Type=0x03 (LINKTYPE_NAME)
Length is variable
Value=

Type=0x04 (COMMENT)
Length is variabe
Value=

Type=0x05 (TIMESTAMP)
Length=8
Value=

-- 
    Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


[tcpdump-workers] BPF Exam

2022-06-05 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
Hello list.

A while ago I tried to comprehend a few BPF-related bug reports in
libpcap and found it difficult to follow the logic of filter
compilation and optimization.  On one hand, there is the universally
available, but basic "tcpdump -d" pseudocode listing.  On the other,
there is the off-by-default optimizer debug mode in libpcap and the
associated visopts.py script (and the C source code, of course).  But it
seemed as if something in between of these levels of detail would be
more convenient in some cases.

One of the bug reports (798) contained a block of BPF bytecode
disassembly with visualized conditional jump instructions, which seemed
useful and led me to discover Radare2.  After some experimentation and
integration it became possible to produce several different types of
debugging information easily on one page (called "BPF Exam"):

https://www.tcpdump.org/bpfexam/

As usual, there is some space for future improvements, but this revision
looks ready for general use.  Currently the page allows at most 1 form
submission per second to limit the impact on the server resources,
other than that everything should be self-explanatory.  Feedback is
welcome on the list.

Cheers.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


[tcpdump-workers] endianness of portable BPF bytecode

2022-06-02 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
Hello list.

In the usual workflow a correct BPF expression eventually compiles to a
series of BPF instructions, each of which is a 64-bit structure
encoding the (opcode, jt, jf, k) tuple.

So long as the compiled bytecode appears only between the OS kernel and
libpcap, its endianness has a limited scope.  However, as soon as the
bytecode needs to be saved to a portable file (or fed through a pipe or
a socket), it needs either to abide by an endianness convention, or to
use a header indicator such as the one in pcap-savefile(5).  There is a
couple open requests that would require this to be specified:

https://github.com/the-tcpdump-group/libpcap/issues/211
https://github.com/the-tcpdump-group/libpcap/pull/353

Also in Radare2 (which might be handy for BPF bytecode disassembly)
the code does not seem to be entirely certain about the input data
endianness.

As far as it was possible to infer from the 1993 Usenix paper, the
original development of BPF was done entirely on big-endian systems.
If there is no convention in place yet, I would like to propose
declaring big-endian as the implicit/default byte order, then
particular file format(s) with headers can override that as needed.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


[tcpdump-workers] CI news January-May

2022-05-05 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
Hello list.

Below you can find a digest of some works done on the CI infrastructure
during past few months.

First of all, the illumos-amd64 Buildbot worker has been replaced.  It
used to be an OpenIndiana VM from a commercial hosting provider.
Several months ago the VM started to have frequent random reboots and
there was no solution to the problem.  Now it is an OmniOS zone hosted
and managed by Andy Fiddaman of the OmniOS project (thank you!), and the
worker is stable again.  Other useful effects of this switch are
upgrades to GCC (from 10.3 to 11.2) and Clang (from 9.0 to 11.1, 13.0
and 14.0).

Likewise, linux-armv7l has been switched from Raspbian 10 to Ubuntu
22.04, which has upgraded GCC (from 8.3 to 11.2) and Clang (from 7.0 to
14.0).  It still runs on AArch64 hardware, which makes it useless for
the purpose of unaligned access testing (AArch64 handles unaligned
access in hardware).  If anybody is willing to host this worker on an
actual ARMv7l hardware, please let me know.

openbsd-amd64 has been upgraded from OpenBSD 7.0 to 7.1, which resulted
in a Clang upgrade as well (from 11.1 to 13.0).

Two NetBSD workers have been updated to more recent pkgsrc
releases/snapshots, although without any notable side effects.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


[tcpdump-workers] WinPcap CI

2022-02-02 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
Hello list.

www.winpcap.org has been down for a couple days, which is breaking
libpcap CI in Appveyor.  Now that Npcap is the recommended SDK, would it
be a good occasion to stop doing CI with WinPcap?

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


[tcpdump-workers] project news January 2022

2022-01-13 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
Hello list.

I almost typed "2021" into the subject of this e-mail, as 2022 still
feels futuristic, yet it is here.

Today is exactly 31 years from the first public release of tcpdump
(2.0 by LBL), which at the time was still the same program as libpcap.
As you can see on the "old releases" page [1], it was not until summer
1994 when libpcap 0.0 came into existence.  Perhaps some people could
explain the timeline much better, feel free to make your input if you
can.  In any case, it feels like a valid excuse to have a nice pizza.

Thanks to everyone who contributed, in one or another way, to this
project during this long time.  Unfortunately, often contributions get
stuck in the long queue of pull requests.  Please excuse the current
maintainers for not handling the incoming contribution stream fast
enough, as it takes time at the receiving end as well to proof-read and
merge changes safely.  Sometimes this time is available, and sometimes
it is not.

The web site received many assorted updates and cleanups in recent
couple months.  As far as the content goes, a number of things that
looked out of place and/or date have been put right.  From the visual
aspect, the content now uses screen space more effectively and it uses
more consistent style, especially the man pages.  Various documents now
link to each other better than before.  The FAQ now has a few new
entries, but, as usual, it is never complete, so if you can suggest new
entries (ideally complete with answers), that would be welcome.

Finally, assorted maintenance work was done about the CI:

* openbsd-mips64 has been upgraded from 6.9 to 7.0, which upgraded
  Clang from 10.0 to 11.0 and finally made ccache available.  However,
  the performance gain was offset with the addition of GCC 8.4.0, which
  was not available before, so the end effect is that the full CI round
  on this worker still takes about the same time, but now covers two
  compilers instead of one.
* illumos-amd64 has reached OpenIndiana 2021.10 as the latter became
  available.
* netbsd-aarch64 pkgsrc has been upgraded from 2021Q3 to 2021Q4, which
  among other things has upgraded Clang from 12.0 to 13.0.

1: https://www.tcpdump.org/old_releases.html

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] Request for new LINKTYPE/DLT ZBOSS_NCP

2022-01-09 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Tue, 2 Nov 2021 09:46:30 +0300
Eugene Exarevsky via tcpdump-workers
 wrote:

> Hi all,
> 
> Please, allocate a new LINKTYPE/DLT called ZBOSS_NCP.
> 
> The background is following: we (DSR) are developing Zigbee protocol 
> stack ZBOSS (
> The stack has serial commands interface - NCP.
> We implemented a dissector of that NCP protocol for Wireshark and
> want to publish it. The idea is that we can display NCP commands and
> Zigbee traffic between the stack and Zigbee transceiver in the same
> log. But, indeed, NCP serial protocol is not related to 802.15.4
> itself, so it will be wrong to base NCP parser on 803.15.4 protocol
> parser in Wireshark.
> We suppose separate PCAP DLT/WTAP_ENCAP  will be the best choice.

As a follow-up, this request did not go well.  It was merged before it
was fully reviewed, subsequent points were not addressed and the
requester seems to be out of contact.

It is difficult to see how the header type can be useful if it is not
sufficiently well defined, so I am going to revert the changes made in
libpcap pull request 1061 and tcpdump-htdocs pull request 25, unless
anyone can suggest a better way to resolve this.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] Selectively suppressing CI on some sites for a commit?

2022-01-07 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Thu, 6 Jan 2022 23:35:30 -0800
Guy Harris  wrote:

[...]
> It appears that a "GitHub skip hook" may have been first introduce in
> Buildbot 0.9.11:
> 
>   https://docs.buildbot.net/0.9.11/relnotes/index.html
> 
> with the hook being configurable by a regex match.  The 0.9.11
> documentation of the "skips" parameter of the GitHub hook:
> 
>   https://docs.buildbot.net/0.9.11/manual/cfg-wwwhooks.html#chsrc-GitHub
> 
> does not say anything about the skip item having to be on the first
> line of the commit message; it does say that the default parameter is
> 
>   [r'\[ *skip *ci *\]', r'\[ *ci *skip *\]']
> 
> so either [skip ci] or [ci skip] (with arbitrary numbers of blanks
> thrown in after [, between the words, or before ]) should work.

The matter is, it seems to disregard the non-default value.  This is
what has been in the configuration since 2021-04-14:

'skips': [
r'\[ *skip *ci *\]',  # default
r'\[ *ci *skip *\]',  # default
r'\[ *skip *bb *\]',  # does not seem to work
r'\[ *skip *buildbot *\]',  # idem
],

That had no effect on the day (Buildbot version likely 3.0.3 or 3.1.0):
https://github.com/the-tcpdump-group/tcpslice/commit/1222006

This is now on the list of potential improvements.

[...]
> So:
> 
>   to suppress *all* builds, put [skip ci] on the first line;
> 
>   to suppress only AppVeyor builds (which currently means "do
> only UN*X builds"), put [skip appveyor] on the first line;
> 
>   to suppress only Cirrus builds (which means "skip x86-64
> Linux, x86-64 macOS, and x86-64 FreeBSD", but that doesn't suppress
> ARM64 FreeBSD or non-x86-64 Linux, so I'm not sure how useful it is),
> put [skip cirrus] on the first line;
> 
>   to suppress only our buildbot builds, put [skip ci] somewhere
> *other* than the first line;
> 
>   to supporess any set of builders that's the union of the
> three lines above, do the items for the builders in question.

ci.html now includes this information, thank you for summarizing the
current state.

> There does not seem to be a way to do *only* Windows builds.  Putting
> [skip cirrus] on the first line and [skip ci] elsewhere in the commit
> message is the closest to that, but it won't suppress the OpenCSW
> builds, meaning "only Windows and Solaris".

This solution space does not match the problem space perfectly, but the
amount of useful feedback is worth the cost.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] Selectively suppressing CI on some sites for a commit?

2022-01-06 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Thu, 6 Jan 2022 14:11:54 -0800
Guy Harris via tcpdump-workers 
wrote:

> I've just updated the libpcap .appveyor.yml to get Npcap from
> npcap.com (the Npcap site has been moved there); I added [skip
> cirrus] to skip Cirrus CI for that change, and it appears to work.

That's nice to know.  Either this is a relatively recent skip pattern in
Cirrus CI, or I didn't notice it before (see my message to the list
from 21 August 2020).  Do you think https://www.tcpdump.org/ci.html
should document [skip cirrus] and [skip appveyor]?

> Are there other comments to add to suppress OpenCSW CI and to
> suppress the other CI sites that have been set up?  The only one I
> want *not* suppressed is AppVeyor.

Not immediately, or not at all.  However, there are only two Buildbot
places where all skip patterns are processed (or not).

ci.tcpdump.org recognizes [skip ci] because that's the default
behaviour in that version of Buildbot.  Following the documentation,
several months and Buildbot versions ago I tried adding [skip buildbot]
to the list of skip patterns, but for some reason it had no effect
(could be a user error or a bug). Would it help to try again?

I am not familiar with OpenCSW Buildbot setup, but from the build
history it is obvious it disregards [skip ci], so it looks likely it
would disregard [skip buildbot] too.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] pcap_datalink_ex() in linktypes.html

2021-12-18 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Wed, 8 Dec 2021 23:02:58 +
Denis Ovsienko via tcpdump-workers 
wrote:

> I am going to merge the short Markdown file into the long HTML
> page at some later point

Alright, libpcap/doc/DLT_ALLOCATE_HOWTO.md has been merged into
tcpdump-htdocs/linktypes.html, also there is a FAQ entry now for
VLAN/MPLS, proof-reading and feedback are welcome.

However, the pcap_datalink_ex() issue still remains as stated earlier.

-- 
    Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] pcap_datalink_ex() in linktypes.html

2021-12-08 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Sun, 28 Nov 2021 21:11:27 +
Denis Ovsienko via tcpdump-workers 
wrote:

> Hello all.
> 
> Whilst updating some markup in linktypes.html, I wanted to change the
[...]

On a related note, it does not look right that linktypes.html in the
middle of the explanation refers to a different file in a different
repository (DLT_ALLOCATE_HOWTO.md in libpcap), which is only a few
paragraphs long and on its own is not sufficient to get everything
right. I am going to merge the short Markdown file into the long HTML
page at some later point, if you see a good reason not to, please shout.

-- 
    Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] [libpcap] Keep Win32/Prj/* files ?

2021-12-06 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Mon, 6 Dec 2021 12:31:33 -0800
Guy Harris  wrote:

[...]
> The CMake files are likely to be better maintained than the "use
> Visual Studio directly" files, as you don't need Visual Studio, and
> don't need to know how Visual Studio solution or project files work
> internal, in order to modify the CMake files.

The only way it makes sense to me is "these files can and should be
removed".

On this note, is there anything in the TODO file that is still
relevant? If yes, would it be difficult to convert such content into
actionable ("do this and this to solve this problem") tasks/issues so
the file would be fine to remove?

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] [libpcap] Keep Win32/Prj/* files ?

2021-12-06 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Mon, 29 Nov 2021 19:20:32 +0100
Francois-Xavier Le Bail via tcpdump-workers
 wrote:

> Hello,
> 
> The information on building libpcap on Windows with Visual Studio is
> here:
> https://github.com/the-tcpdump-group/libpcap/blob/master/doc/README.Win32.md
> (The supported way to build libpcap (and tcpdump) on Windows is with
> CMake)
> 
> Does anyone use these files?
> Win32/Prj/wpcap.sln
> Win32/Prj/wpcap.vcxproj
> Win32/Prj/wpcap.vcxproj.filters

It looks like CMake has superseded these files, as far as it is
possible to tell without Windows.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


[tcpdump-workers] VLAN/MPLS FAQ

2021-11-30 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
Hello list.

Whilst looking through tcpdump and libpcap issues, I noted that the
specifics of "vlan" and "mpls" keywords in pcap filters is one of the
most common causes of confusion. So it should be in the FAQ.

That said, it is not clear if it would be better to have one long FAQ
entry or a few shorter ones, and whether it would be better to copy or
to rephrase what has already been explained in detail in at least 10 bug
reports, or to provide the links, or to write something from scratch.
What would make the most sense in your opinion?

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


[tcpdump-workers] pcap_datalink_ex() in linktypes.html

2021-11-28 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
Hello all.

Whilst updating some markup in linktypes.html, I wanted to change the
reference to pcap_datalink_ex() into a hyperlink to respective man page,
but it turned out there is no such function in libpcap in the first
place. There is pcap_datalink_ext(), which has been around since 2007,
but is not mentioned in any of libpcap man pages. Clearly that means at
least one loose end, but after a brief look I could not tell exactly
how to address that.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


[tcpdump-workers] project news November 2021

2021-11-05 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
Hello list.

I hope this finds you well. Below you can find an overview of assorted
recent developments.

The read-only git mirror service has moved from bpf.tcpdump.org to
git.tcpdump.org. The latter works both as a browsable web-site and as a
git clone URL (either https:// or git:// at your choice). The mirror now
uses webhooks to keep any lag behind the read-write repositories at
github.com to one minute or less.

A by-product of the new mirror is automatic propagation of changes from
the tcpdump-htdocs repository to the live www.tcpdump.org web site. This
way trivial web site updates are as simple as "commit and push".

To simplify the web site setup and maintenance, the only remaining
country-specific hostnames below no longer serve any content:

www.ca.tcpdump.org
www.de.tcpdump.org
www.us.tcpdump.org

Currently these hostnames resolve to a host that redirects all requests
to www.tcpdump.org with the same URL path, and in early 2022 these are
going to disappear from DNS. If you come across any makefiles, RPM spec
files, scripts etc. that use any of the three hostnames above to
download the .tar.gz source code archives, please help to fix them to
use www.tcpdump.org and HTTPS.

The web site now provides man pages in several sets: one for the
current git master branch (as it used to be) and a few more for recent
releases. To minimise confusion, each man page now has a banner at the
top to tell what it documents.

As a minor related point, much of the HTTP 404 (not found) response
codes for www.tcpdump.org accounted for /favicon.ico, so I used the
image by Ali Abdulkadir from tcpdump pull request 652 to plug that. It
looks alright, but if you see a better solution, please make your point.

In the CI department it has been mostly routine maintenance since the
previous update:

* netbsd-aarch64 pkgsrc packages have been upgraded from 2021Q2 to
  2021Q3, which among other updates includes Clang (from 10.0.1 to
  12.0.1).
* Ditto netbsd-mips64, also the OS snapshot has been upgraded from
  9.99.86 to 9.99.90 (from early July to early October).
* openbsd-amd64 has been upgraded from 6.9 to 7.0, which among other
  updates includes Clang (from 10.0.1 to 11.1.0), Autoconf (from 2.69
  to 2.71) and GCC (from 8.4 to 11.2).

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


[tcpdump-workers] CI news September 2021 (RISC-V, illumos)

2021-09-15 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
Hello list.

This is a semi-periodic summary of recent developments in the continuous
integration space of the tcpdump group.

It took a few months since Francois-Xavier Le Bail discovered this
opportunity, but finally the Buildbot setup includes a hardware RISC-V
worker, which runs on an Allwinner Nezha D1 development board provided
by RISC-V International. [1] The OS (Fedora Linux) currently comes from
Institute of Software, Chinese Academy of Sciences. [2]

Another new Buildbot worker is an AMD64 VM running the OpenIndiana
2021.04 distribution of illumos. This derives from my earlier
experiments with Solaris 11/AMD64, which may not be used in this CI
setup due to licence conditions. Overall research in this direction has
already identified a few issues in tcpdump and libpcap, some of which
still remain to be addressed.

The Buildbot master has been migrated from SQLite3 to MariaDB. The
database size has been slowly growing since the service start and
recently reached around 0.5GB, most of which accounts for the build
logs. Now the web-interface feels a bit more responsive and should be
easier to scale in future if necessary.

1: https://riscv.org/risc-v-developer-boards/
2: http://www.iscas.ac.cn/

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

Re: [tcpdump-workers] [tcpdump] Finding inconsistent outputs of tcpdump with different compilers

2021-08-15 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Mon, 9 Aug 2021 16:27:27 +0200
Francois-Xavier Le Bail via tcpdump-workers
 wrote:

> cmp $TMPDIR/out1 $TMPDIR/out2 >/dev/null
> [ $? = 1 ] && echo "$p" && diff $TMPDIR/out1 $TMPDIR/out2 &&
> echo

Differential analysis is indeed a good solution when the problem is to
derive certain facts from two or more uncertain inputs.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

[tcpdump-workers] assorted build issues

2021-08-15 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
Hello list.

I hope this finds you well.

To follow up on the earlier point about the build problem space mapping,
below are a few issues that don't fail building and testing, but might
mean potential problems:

* Builds that use Autoconf 2.71 generate quite a number of Autoconf
  warnings:

configure.ac:27: warning: The macro `AC_PROG_CC_C99' is obsolete.
configure.ac:27: You should run autoupdate.
./lib/autoconf/c.m4:1659: AC_PROG_CC_C99 is expanded from...
configure.ac:27: the top level
configure.ac:31: warning: The macro `AC_TRY_COMPILE' is obsolete.
configure.ac:31: You should run autoupdate.
(and so on)

* Builds that use CMake generate varying amounts of CMake warnings
  depending on which CMake version it is. The most common one is this:

CMake Deprecation Warning at CMakeLists.txt:19 (cmake_minimum_required):
  Compatibility with CMake < 2.8.12 will be removed from a future
version of CMake.
  Update the VERSION argument  value or use a ... suffix to
tell CMake that the project does not need compatibility with older
  versions.

* libpcap Autoconf builds do not use correct compiler flags for XL C
  (CMake is fine) on Linux:

/opt/ibm/xlC/16.1.1/bin/.orig/xlc: warning: 1501-269 fpic is not
supported on this Operating System platform.  Option fpic will be
ignored.

/opt/ibm/xlC/16.1.1/bin/.orig/xlc: warning: 1501-269 fpic is not
supported on this Operating System platform.  Option fpic will be
ignored.

(and so on)

-- 
    Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

Re: [tcpdump-workers] NetBSD breakage

2021-08-11 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Wed, 11 Aug 2021 15:38:34 -0700
Guy Harris  wrote:

> I've checked in a change to remove the include of grammar.h from
> gencode.c; it builds without problems on macOS, and I suspect it will
> build without problems everywhere

Thank you, NetBSD has recovered now. I am going to remove
HAVE_SYS_TYPES_H from pcap/bpf.h because it does not belong there.

-- 
    Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

[tcpdump-workers] NetBSD breakage

2021-08-11 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
Hello list.

I have spent some time looking into tcpdump NetBSD builds, which all
have failed as a consequence of my libpcap commit 0c21cb27, which had
broken the workaround earlier introduced in libpcap commit bb02779.

As Guy has noted in a comment just now, the matter isn't just about
bpf_u_int32, but also BPF_MEMWORDS. These two simple props (one of
which also depends on u_int) currently have a side effect of including
pcap/dlt.h, which involves enough system headers to touch .
Then the problem is that the NetBSD header and the libpcap header
compete for the same DLT_MATCHING_MAX (ifdef/undef/define), so there is
only one way to include them to have the right definition at libpcap
build time.

The other matter is that the gencode.h/grammar.h pair works best when
it is included early.

I have been following header dependencies for some time, and it is clear
that it would be nice to have one of the public headers make trivial
declarations that do not introduce side effects. So, for example,
pcap-inttypes.h could have the bpf_int32/bpf_u_int32 block instead of
pcap/bpf.h (plenty of files use these types). In this case it would
also make sense to merge pcap-types.h into pcap-inttypes.h. I do not
see what would be a good place for BPF_MEMWORDS, but clearly it is a
named constant and having it declared should not depend on any other
headers at all.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

Re: [tcpdump-workers] build failures on Solaris

2021-08-08 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Sun, 8 Aug 2021 12:02:53 -0700
Guy Harris  wrote:

[...]
> I've checked in a change to explicitly tell CMake "this is a C-only
> project, don't check for a C++ compiler", so it should now think it's
> building 64-bit when building with GCC.
> 
> See whether that fixes things.

Thank you for investigating this problem and making the changes. All
results below come from the unstable11s host (with OpenCSW packages).
I confirm this change has fixed GCC+CMake case when building tcpdump
with the system libpcap (4 builds), in that CMake starts to detect
libpcap features it did not before, for example:

-- Looking for pcap_findalldevs
-- Looking for pcap_findalldevs - found
("tcpdump -D" works, although does not print anything)

However, in the same context building tcpdump with local libpcap in
../libpcap (4 builds) no longer works, in that the resulting tcpdump
binary segfaults itself and makes ldd segfault:

ldd /tmp/tcpdump_build_matrix.XXYkrCSb/bin/tcpdump
ldd: /tmp/tcpdump_build_matrix.XXYkrCSb/bin/tcpdump: execution failed
due to signal 11 (core dumped)

file /tmp/tcpdump_build_matrix.XXYkrCSb/bin/tcpdump
/tmp/tcpdump_build_matrix.XXYkrCSb/bin/tcpdump: ELF 64-bit MSB
executable SPARCV9 Version 1, dynamically linked, not stripped, no
debugging information available

/tmp/tcpdump_build_matrix.XXYkrCSb/bin/tcpdump --version
Segmentation Fault (core dumped)

Autoconf build with GCC works with both the system and the local
libpcap (8 builds), as before.

Sun C works with both CMake and Autoconf, with both the system and the
local libpcap (16 builds), as before.

Let me know in case you need more testing/diagnostics.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

Re: [tcpdump-workers] build failures on Solaris

2021-08-08 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Sun, 8 Aug 2021 01:22:09 -0700
Guy Harris  wrote:

[...]
> I've checked in changes to:
> 
>   check the bit-width of the build in autotools;
> 
>   on Solaris, use the results of the bit-width checks for
> autotools and CMake to figure out which version of pcap-config to run.
> 
> See if that clears up the Solaris 11 with GCC build.

GCC+CMake fails early now (see attached). GCC+Autoconf works, as before.

-- 
Denis Ovsienko
> MATRIX_CC=gcc MATRIX_CMAKE=yes ./build_matrix.sh
SunOS unstable11s 5.11 11.3 sun4u sparc SUNW,SPARC-Enterprise
OS identification: SunOS-5.11
Sunday, August  8, 2021 11:02:23 AM CEST
PREFIX set to '/tmp/tcpdump_build_matrix.XXoP3bDd'
Use system libpcap
make: Fatal error: Don't know how to make target `distclean'
(Ignoring the make error.)
= SETUP 1: BUILD_LIBPCAP=no REMOTE=? CC=gcc CMAKE=yes CRYPTO=no SMB=no =
$ ./build.sh
gcc (GCC) 7.3.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Compiler identification: gcc-7.3.0
$ rm -rf build
$ mkdir build
$ cd build
$ cmake -DWITH_CRYPTO=no -DENABLE_SMB=no -DEXTRA_CFLAGS=-Werror 
-DCMAKE_INSTALL_PREFIX=/tmp/tcpdump_build_matrix.XXoP3bDd ..
-- The C compiler identification is GNU 7.3.0
-- The CXX compiler identification is SunPro 5.9.0
-- Check for working C compiler: /usr/bin/gcc
-- Check for working C compiler: /usr/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/CC
-- Check for working CXX compiler: /usr/bin/CC -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at CMakeLists.txt:62 (message):
  Architecture flags must be set in both CFLAGS and CXXFLAGS


-- Configuring incomplete, errors occurred!
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

[tcpdump-workers] CI news July-August 2021

2021-08-05 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
Hello list.

Besides the matters recently discussed on the list, a few notable
developments in The Tcpdump Group CI took place since the previous
update:

* Another MIPS worker has been doing builds in the pool for a month. It
  runs a snapshot of NetBSD-current. Although NetBSD runs
  single-processor only on this hardware, it looks stable enough. This
  is the so-called "n32" variety of 64-bit MIPS, which means mostly
  32-bit userspace on a 64-bit kernel. The "n64" variety (64-bit
  userspace) came into existence not long ago, but it is not usable
  enough yet.
* The Linux/POWER9 worker in addition to GCC and Clang recently started
  to use IBM XL C. Consequently, it currently builds the largest
  matrices: 48 permutations for tcpdump, 12 for libpcap and 6 for
  tcpslice.
* The build scripts in most cases started to request compilers to treat
  warnings as errors. To that end, most warnings have been addressed
  and only one exemption remains (libpcap on NetBSD). This way it
  should be easier to notice subtle regressions than before.

Cheers.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

Re: [tcpdump-workers] build failures on Solaris

2021-08-02 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Mon, 2 Aug 2021 14:31:18 -0700
Guy Harris  wrote:

>   solaris11$ /usr/bin/pcap-config --libs
>   -L/usr/lib  -lpcap
>   solaris11$ /usr/bin/amd64/pcap-config --libs
>   -L/usr/lib/amd64 -R/usr/lib/amd64 -lpcap
> 
> on my x86-64 Solaris 11 VM.

$ uname -a
SunOS gcc-solaris11 5.11 11.3 sun4u sparc SUNW,SPARC-Enterprise
$ echo $PATH
/usr/sbin:/usr/bin:/opt/csw/bin
$ type pcap-config
pcap-config is /usr/bin/pcap-config
$ pcap-config --libs
-L/usr/lib  -lpcap
$ /usr/bin/sparcv9/pcap-config --libs
-L/usr/lib/sparcv9 -R/usr/lib/sparcv9 -lpcap
$ /opt/csw/bin/pcap-config --libs
-L/opt/csw/lib -R/opt/csw/lib -lpcap
$ /opt/csw/bin/sparcv9/pcap-config --libs
-L/opt/csw/lib/64 -R/opt/csw/lib/64 -lpcap

Though it is not clear why the 64-bit directories are not in PATH.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

Re: [tcpdump-workers] build failures on Solaris

2021-08-01 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Sun, 1 Aug 2021 15:45:39 -0700
Guy Harris  wrote:

[...]
> So where do the Solaris 11 hosts show up on the buildbot site?

They don't. Solaris 9 and 11 are not in any CI at all. OpenCSW Buildbot
runs on Solaris 10, but uses a single build script. I used OpenCSW
shell access to see how realistic it is to do a manual full matrix CI
on each Solaris version, and it had chased some bugs out. The gcc211
Solaris host had added to the list a bit (not yet described). That was
on SPARC.

If you would like to host a Solaris 11 Buildbot worker, it would need
Python 3.6 or newer plus the usual tools (CMake, git...). Also it would
need to be firewalled from private networks and to have means to
restore to a known-good snapshot conveniently if the VM becomes
compromised through a pull request. Buildbot workers run fine behind
NAT.

[...]
> > As I have discovered just now, it does not reproduce on OpenCSW host
> > gcc211:  
> 
> Probably some annoying combination of one or more of "different
> compilers", "later version of CMake", "at least some versions of cc
> and gcc build 32-bit binaries by default even on Solaris 11 on a
> 64-bit machine(!)", and so on.
> 
> This is going to take a fair bit of cleanup, not the least of which
> includes forcing build with both autotools *and* CMake to default to
> 64-bit builds on 64-bit Solaris.

For clarity, there is no rush to fix every obscure issue in this
problem space, but it is useful to have the problem space mapped.

One purpose for that would be stating something like "Only these
particular kinds of Solaris are expected/known to work if they meet
this and this requirement." in the documentation. I do not have a use
case for Solaris, so I cannot define such a baseline, so I had tried
everything that was available, for worse or better.

Another purpose would be to cover as much as possible at least with
easy manual pre-release matrix builds, and ideally with CI.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

Re: [tcpdump-workers] build failures on Solaris

2021-07-31 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Sat, 31 Jul 2021 14:55:32 -0700
Guy Harris  wrote:

[...]
> What version of CMake is being used, and how was it installed?
> 
> My Solaris 11 x86-64 virtual machine has CMake 2.8.6 in
> /usr/ccs/bin/cmake, installed from Sun^WOracle's Image Packaging
> System repositories, and I'm not seeing that behavior - the test
> programs are linked with -lpcap, as is tcpdump.

This issue reproduces on OpenCSW host unstable11s:

# CMake 3.14.3 (OpenCSW package)
# GCC 7.3.0

MATRIX_CC=gcc \
MATRIX_CMAKE=yes \
MATRIX_BUILD_LIBPCAP=no \
./build_matrix.sh 
[...]
$ /tmp/tcpdump_build_matrix.XXVrYyid/bin/tcpdump -D
/tmp/tcpdump_build_matrix.XXVrYyid/bin/tcpdump: illegal option -- D
tcpdump version 5.0.0-PRE-GIT
libpcap version unknown

As I have discovered just now, it does not reproduce on OpenCSW host
gcc211:

# CMake 3.14.3 (OpenCSW package)
# GCC 5.5.0

mkdir ~/bin
ln -s /opt/csw/bin/gmake ~/bin/make
PATH=~/bin:$PATH \
MATRIX_CC=gcc \
MATRIX_CMAKE=yes \
MATRIX_BUILD_LIBPCAP=no \
./build_matrix.sh
[...]
Tested setup count: 4

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

[tcpdump-workers] build failures on Solaris

2021-07-31 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Sat, 24 Jul 2021 00:11:27 +0100
Denis Ovsienko via tcpdump-workers 
wrote:

> There are more related corner cases of varying severity,
> some of which I might have time to describe later.

So, as promised... Several warnings are now documented in the build
scripts (look for TCPSLICE_TAINTED=yes and TCPDUMP_TAINTED=yes).
Another problem that is briefly mentioned in a comment is tcpdump build
issues on Solaris specific to CMake and system libpcap. Hopefully my
notes have captured the details right.

# Solaris 10 with Sun C #
The pre-compile libpcap feature test programs link with the system
libpcap.so fine and run fine. The tcpdump binary links fine, but fails
to run:

$ /tmp/tcpdump_build_matrix.XXtwaqbh/bin/tcpdump --version
ld.so.1: tcpdump: fatal: libpcap.so.1: open failed: No such file or
directory

$ ldd /tmp/tcpdump_build_matrix.XXyKaiEo/bin/tcpdump

libpcap.so.1 =>  (file not found)
libsocket.so.1 =>/lib/libsocket.so.1
libnsl.so.1 =>   /lib/libnsl.so.1
libsmi.so.2 =>   (file not found)
libc.so.1 => /lib/libc.so.1
libmp.so.2 =>/lib/libmp.so.2
libmd.so.1 =>/lib/libmd.so.1
libscf.so.1 =>   /lib/libscf.so.1
libdoor.so.1 =>  /lib/libdoor.so.1
libuutil.so.1 => /lib/libuutil.so.1
libgen.so.1 =>   /lib/libgen.so.1
libm.so.2 => /lib/libm.so.2
/platform/SUNW,SPARC-Enterprise-T5220/lib/libc_psr.so.1
/platform/SUNW,SPARC-Enterprise-T5220/lib/libmd_psr.so.1

$ pcap-config --libs  
-L/opt/csw/lib -R/opt/csw/lib -lpcap

It looks like the test programs use the runpath, but tcpdump does not.

# Solaris 11 with GCC #
This is the opposite: the pre-compile libpcap feature test programs
fail to link so all libpcap feature tests fail. However, libpcap is
detected as available and the build process resorts to missing/ and
produces a binary of tcpdump that is mostly functional:

$ /tmp/tcpdump_build_matrix.XX06MD.a/bin/tcpdump -D
/tmp/tcpdump_build_matrix.XX06MD.a/bin/tcpdump: illegal option -- D

The problem seems to be that the feature test linking instead of using
the flags returned by pcap-config points exactly to the 32-bit version
of libpcap and fails:

$ pcap-config --libs
-L/usr/lib  -lpcap

/usr/bin/gcc
-DCHECK_FUNCTION_EXISTS=pcap_list_datalinks
CMakeFiles/cmTC_d38b1.dir/CheckFunctionExists.c.o  -o cmTC_d38b1
/usr/lib/libpcap.so

ld: fatal: file /usr/lib/libpcap.so: wrong ELF class: ELFCLASS32
collect2: error: ld returned 1 exit status

It likely has to do with these CMake variables:

TCPDUMP_LINK_LIBRARIES=/usr/lib/libpcap.so;socket;nsl
PCAP_LIBRARIES=/usr/lib/libpcap.so

Soon after that tcpdump linking does not force the 32-bit library and
works:

/usr/bin/gccCMakeFiles/tcpdump.dir/fptype.c.o
CMakeFiles/tcpdump.dir/tcpdump.c.o CMakeFiles/tcpdump.dir/bpf_dump.c.o
CMakeFiles/tcpdump.dir/missing/pcap_dump_ftell.c.o
CMakeFiles/tcpdump.dir/missing/datalinks.c.o
CMakeFiles/tcpdump.dir/missing/dlnames.c.o  -o tcpdump libnetdissect.a
-lpcap -lsocket -lnsl 

This automatically uses the 64-bit library (not in /usr/lib).

Autoconf build does not have the problem with feature tests because it
uses exactly what pcap-config has returned and lets the linker figure
the details out:

configure:6157: checking for pcap_findalldevs
configure:6157: gcc -o conftest -I/usr/inet6/include -g -O2
conftest.c -L/usr/lib  -lpcap -lsocket -lnsl  >&5
configure:6157: $? = 0

Sun C does not have the problem with feature tests: CMake invokes it
same way as GCC, but Sun C manages to produce feature test programs
that work as expected:

/opt/solarisstudio12.4/bin/suncc
-DCHECK_FUNCTION_EXISTS=pcap_list_datalinks
CMakeFiles/cmTC_97fb2.dir/CheckFunctionExists.c.o  -o cmTC_97fb2
/usr/lib/libpcap.so

This looks like some heuristics or a short-cut built into Sun C, but it
seems wrong to depend on it. Moreover, Sun C does not have to guess hard
for tcpdump linking because the flags are just correct:

/opt/solarisstudio12.4/bin/sunccCMakeFiles/tcpdump.dir/fptype.c.o
CMakeFiles/tcpdump.dir/tcpdump.c.o  -o tcpdump libnetdissect.a -lpcap
-lsocket -lnsl 

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

Re: [tcpdump-workers] compiler warnings on AIX and Solaris

2021-07-25 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Sat, 24 Jul 2021 10:55:19 -0700
Guy Harris  wrote:

[...]

> If there's a way to force XL C to treat it as a hard error, we need
> to update the AC_LBL_CHECK_UNKNOWN_WARNING_OPTION_ERROR autoconf
> macro to set the compiler up to use it when testing whether compiler
> options are supported.
> 
> If there *isn't* a way to do that, the configure-script test also
> needs to scan the standard error of the compilation and look for the
> warning, and treat that as an indication of lack of support as well.
> (I think the equivalent test provided as part of CMake may already do
> that.)

The current master branch produces a much cleaner output for "make -s"
with XL C, thank you!

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

Re: [tcpdump-workers] compiler warnings on AIX and Solaris

2021-07-24 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
[...]

> So the only way to fix this is to turn off the warnings; change
> 39f09d68ce7ebe9e229c9bf5209bfc30a8f51064 adds macros to disable and
> re-enable -Wcast-qual and wraps the offending code in getopt_long.c
> with those macros, so the problem should be fixed on Solaris 9.

The fix works, in that GCC on both OSes now compiles getopt_long.c
cleanly. Neither Sun C on Solaris nor XL C on AIX detected the problem
before, so there is no change there.

Thank you!

[...]

> If there's a way to force XL C to treat it as a hard error, we need
> to update the AC_LBL_CHECK_UNKNOWN_WARNING_OPTION_ERROR autoconf
> macro to set the compiler up to use it when testing whether compiler
> options are supported.

I have tried xlc -qhaltonmsg=1501-289 and -qhalt=w flags, but to no
apparent effect. It looks like warnings that come from the command-line
arguments parsing space are not handled the same way as warnings that
come from the C language compilation space.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

[tcpdump-workers] compiler warnings on AIX and Solaris

2021-07-23 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
Hello list.

As it turns out, on Solaris 9 it is impossible to compile current
tcpdump with CFLAGS=-Werror because missing/getopt_long.c yields a few
warnings (attached). As far as the current revisions of this file go in
FreeBSD, NetBSD and OpenBSD, FreeBSD seems to be the closest and just a
bit newer than the current tcpdump copy (OpenBSD revision 1.22 -> 1.26).
However, it seems unlikely that porting the newer revision would make
the warnings go away, because, for example, permute_args() has not
changed at all.

The same problem stands on AIX 7, and in addition there is an issue
specific to XL C compiler, in that ./configure detects that the
compiler does not support -W, but then proceeds to flex every -W
option anyway, which the compiler treats as a soft error, so ./configure
takes each option as supported, so at the compile time every -W
option ends up in CFLAGS, so for every .c file xlc yields 15 warnings
(see attached). It works, but looks messy.

So for now the build matrix scripts do not specify -Werror for these
two cases. There are more related corner cases of varying severity,
some of which I might have time to describe later.

Cheers.

-- 
    Denis Ovsienko
./missing/getopt_long.c: In function 'permute_args':
./missing/getopt_long.c:163:5: error: cast discards '__attribute__((const))' 
qualifier from pointer target type [-Werror=cast-qual]
./missing/getopt_long.c:165:5: error: cast discards '__attribute__((const))' 
qualifier from pointer target type [-Werror=cast-qual]
./missing/getopt_long.c: In function 'parse_long_options':
./missing/getopt_long.c:295:14: error: cast discards '__attribute__((const))' 
qualifier from pointer target type [-Werror=cast-qual]
./missing/getopt_long.c: In function 'getopt_internal':
./missing/getopt_long.c:547:13: error: cast discards '__attribute__((const))' 
qualifier from pointer target type [-Werror=cast-qual]
cc1: all warnings being treated as errors
*** Error code 1
The following command caused the error:
gcc -std=gnu99 -W -Wall -Wcast-qual -Wmissing-prototypes -Wold-style-definition 
-Wpointer-arith -Wpointer-sign -Wshadow -Wsign-compare -Wstrict-prototypes 
-Wwrite-strings -DHAVE_CONFIG_H -I. -I../libpcap -Werror -o getopt_long.o -c 
./missing/getopt_long.c
make: Fatal error: Command failed for target `getopt_long.o'
checking whether the compiler fails when given an unknown warning option... no
checking whether the compiler supports the -W option... no
checking whether the compiler supports the -Wall option... yes
checking whether the compiler supports the -Wcomma option... yes
checking whether the compiler supports the -Wdocumentation option... yes
checking whether the compiler supports the -Wformat-nonliteral option... yes
checking whether the compiler supports the -Wmissing-noreturn option... yes
checking whether the compiler supports the -Wmissing-prototypes option... yes
checking whether the compiler supports the -Wmissing-variable-declarations 
option... yes
checking whether the compiler supports the -Wpointer-arith option... yes
checking whether the compiler supports the -Wpointer-sign option... yes
checking whether the compiler supports the -Wshadow option... yes
checking whether the compiler supports the -Wsign-compare option... yes
checking whether the compiler supports the -Wstrict-prototypes option... yes
checking whether the compiler supports the -Wunused-parameter option... yes
checking whether the compiler supports the -Wused-but-marked-unused option... 
yes
checking whether the compiler supports the -Wunreachable-code option... yes
checking whether -Wunreachable-code generates warnings from ntohs()... no
checking whether the compiler supports the -Wshorten-64-to-32 option... yes


xlc: 1501-289 (W) Option -Wcomma was incorrectly specified. The option will be 
ignored.
xlc: 1501-289 (W) Option -Wdocumentation was incorrectly specified. The option 
will be ignored.
xlc: 1501-289 (W) Option -Wformat-nonliteral was incorrectly specified. The 
option will be ignored.
xlc: 1501-289 (W) Option -Wmissing-noreturn was incorrectly specified. The 
option will be ignored.
xlc: 1501-289 (W) Option -Wmissing-prototypes was incorrectly specified. The 
option will be ignored.
xlc: 1501-289 (W) Option -Wmissing-variable-declarations was incorrectly 
specified. The option will be ignored.
xlc: 1501-289 (W) Option -Wpointer-arith was incorrectly specified. The option 
will be ignored.
xlc: 1501-289 (W) Option -Wpointer-sign was incorrectly specified. The option 
will be ignored.
xlc: 1501-289 (W) Option -Wshadow was incorrectly specified. The option will be 
ignored.
xlc: 1501-289 (W) Option -Wsign-compare was incorrectly specified. The option 
will be ignored.
xlc: 1501-289 (W) Option -Wstrict-prototypes was incorrectly specified. The 
option will be ignored.
xlc: 1501-289 (W) Option -Wunused-parameter was incorrectly specified. The 
option will be ignored.
xlc: 1501-289 (W) Option -Wused-but-marked-unused was incorrec

[tcpdump-workers] CI news June 2021

2021-06-26 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
Hello list.

I hope you have a nice summer weekend. Below you can find a digest of
improvements made in the Buildbot infrastructure in the last couple
months.

* After some OS shuffling one more worker had moved from my home lab to
  OSU OSL. As a result, linux-aarch64 became a VM on an eMAG AArch64
  server, and it seems to work well.
* Two MIPS (specifically, Octeon) workers had joined the fleet, these
  run 32-bit Linux and 64-bit OpenBSD.
* NetBSD workers had been upgraded from 9.1 to 9.2 and started to use
  GCC 10 instead of the default GCC 7. Autoconf had been upgraded from
  2.69 to 2.71.

Cheers.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

Re: [tcpdump-workers] CI status update

2021-05-06 Thread Denis Ovsienko via tcpdump-workers
--- Begin Message ---
On Mon, 5 Apr 2021 10:42:39 +0100
Denis Ovsienko via tcpdump-workers 
wrote:

> Hello list.
> 
> I hope you have a great day. Please find below a brief overview of
> some current continuous integration matters.

Please find below a few recent developments since the last update a
month ago:

* The matrix now includes Linux on ppc64le again (courtesy of OSU OSL).
* NetBSD and OpenBSD AMD64 worker VMs are now hosted by OSU OSL.
* OpenBSD (AMD64 at OSU OSL and AArch64 at own infrastructure) has been
  upgraded from 6.8 to 6.9.
* FreeBSD (AArch64 at sysmocom) has been upgraded to 13.0-RELEASE.
* The Buildbot master now sends e-mails only when a build is broken or
  is changing status (it used to spam a bit too much before). Still it
  does not guess the recipient correctly every time.

Have a nice day.

-- 
Denis Ovsienko
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers

  1   2   3   >