Re: [pmacct-discussion] effort to relicense pmacct from GPL to a BSD-style license

[Vincent Bernat]

 ❦  8 janvier 2020 14:52 +01, Job Snijders :

> REQUEST TO THE PMACCT CONTRIBUTOR COMMUNITY
> ---
>
> If you have contributed to the pmacct project (your name may be listed
> below), please consider a reply-all to this email expressing your
> explicit consent (or disapproval) to change the license governing your
> contributions to the pmacct project, to the following license:
>
> """
> Permission to use, copy, modify, and distribute this software for
> any purpose with or without fee is hereby granted, provided that the
> above copyright notice and this permission notice appear in all
> copies.
>
> THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL
> WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED
> WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE
> AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL
> DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA
> OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER
> TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
> PERFORMANCE OF THIS SOFTWARE.
> """
>
> ---

I am OK with the change.
-- 
Keep it simple to make it faster.
- The Elements of Programming Style (Kernighan & Plauger)

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

Re: [pmacct-discussion] [RFC] Patches to allow capturing specific PF_PACKET protocols

[Vincent Bernat]

 ❦ 30 août 2017 13:58 GMT, Paolo Lucente  :

> Fantastic, please proceed. I guess the patch should be wrapped around
> some version checking of libpcap a-la if greater or equal than current
> release in master then compile the code. I did jus review configure.ac
> and libpcap version is not captured plus i seem to read libpcap folks
> are anti pkg-config, that would have made things extremely simple. We
> may have to resort to something a-la MySQL (since they also do have a
> mysql-config thing and only in 5.7 added pkg-config support):
>
> https://github.com/pmacct/pmacct/blob/master/m4/ax_lib_mysql.m4

Another quick solution is to check PCAP_VERSION_(MAJOR|MINOR).

If you use autoconf, you can directly check for the existence of the
function, like this is already done with AC_CHECK_LIB (or you could use
AC_CHECK_FUNCS([pcap_set_protocol])).

In fact, after checking Lennert's patch, it's already done this way.
-- 
Make sure special cases are truly special.
- The Elements of Programming Style (Kernighan & Plauger)

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

Re: [pmacct-discussion] building version 1.6.1

[Vincent Bernat]

 ❦  9 novembre 2016 11:56 -0500, Stephen Clark  :

 LIBS . : -L/usr/pgsql-9.4/lib -ldl -L/usr/local/lib -lpfring
 -lpcap -lrt -lnuma -lz -lpthread
>>> If libpfring is linked to one version of libpcap and your local libpcap
>>> is another one, there will a version conflict unless libpcap symbols are
>>> versioned (they are not). Could you check with ldd what is in libpfring,
>>> dependency-wise?
>> Yes, that is probably what is happening. The pf_ring from ntopng
>> replace libpcap with its own
>> version that uses pf_ring.
>>
>>
>>
>>
>
> Yes, that is probably what is happening. The pf_ring from ntopng
> replace libpcap with its own
> version that uses pf_ring.

So, if libpfring is exporting directly the bpf_validate symbols (could
you check with objdump -T?), you can try to not link with -lpcap at all.
-- 
What I tell you three times is true.
-- Lewis Carroll

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

Re: [pmacct-discussion] Problem compiling on Debian Jessie

[Vincent Bernat]

 ❦  6 juillet 2016 09:04 CEST, Johan Sjöberg  :

> Yes, don't know why it complained about MySQL client
> libraries. Configure also completed successfully without
> --enable-mysql. So it was somehow related to MySQL.

It's just that MySQL was the next test. Your compilation is likely to
fail as well after the succesful configure. -lnuma should be explicitely
tested but the code for that has been commented out. I suppose there is
a good reason for that.
-- 
Kindness is a language which the deaf can hear and the blind can read.
-- Mark Twain

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

Re: [pmacct-discussion] Problem compiling on Debian Jessie

[Vincent Bernat]

 ❦  5 juillet 2016 14:58 CEST, Johan Sjöberg  :

> We have been running pmacct 1.2.5 on Debian Squeeze for many years. We
> are now trying to set up a new server, running Debian Jessie. I have
> downloaded pmacct 1.6.0 (and also 1.5.3), and tried compiling it.
>
> On 1.6.0, I get an error that the MySQL client libraries cannot be
> found:

With 1.6, provides a pastebin of your config.log. It would help to
pinpoint the problem.
-- 
Many a writer seems to think he is never profound except when he can't
understand his own meaning.
-- George D. Prentice

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

Re: [pmacct-discussion] pcap/ulog performance on Linux

[Vincent Bernat]

 ❦ 29 février 2016 19:30 +0100, Vincent Bernat <ber...@luffy.cx> :

>> You are right with your assumption. Support of NFLOG has been requested,
>> ie. to support IPv6, but is still pending and i don't have it currently
>> on my radar (ie. 1.6.0 / 1.6.1). 
>
> Can I add it through the use of libnetfilter-log? Or do you want it
> without any external dependencies? I already did a couple of daemons
> with libnetfilter-log.

And a related question, should the added support be in addition to ULOG
or replace it? If replace it, would it be inside the uacctd daemon or a
separate nflacctd daemon?

My preferences are ordered like this:

 1. replace ULOG support by NFLOG support
 2. add NFLOG support inside uacctd daemon
 3. use a separate nflacctd daemon

-- 
Use debugging compilers.
- The Elements of Programming Style (Kernighan & Plauger)

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

Re: [pmacct-discussion] pcap/ulog performance on Linux

[Vincent Bernat]

 ❦ 29 février 2016 18:12 GMT, Paolo Lucente  :

> You are right with your assumption. Support of NFLOG has been requested,
> ie. to support IPv6, but is still pending and i don't have it currently
> on my radar (ie. 1.6.0 / 1.6.1). 

Can I add it through the use of libnetfilter-log? Or do you want it
without any external dependencies? I already did a couple of daemons
with libnetfilter-log.
-- 
Lay on, MacDuff, and curs'd be him who first cries, "Hold, enough!".
-- Shakespeare

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

Re: [pmacct-discussion] pcap/ulog performance on Linux

[Vincent Bernat]

 ❦ 26 février 2016 17:53 +0100, Vincent Bernat <ber...@luffy.cx> :

> Has someone already done some simple benchmarks to check the performance
> of libpcap-based capture with libpcap >= 1.0.0 (mmap support) and Linux
> kernel >= 2.6.27 (64bit support). There is another milestone which is
> libpcap >= 1.5 and Linux kernel >= 3.10 (support for TPACKET_V3)?
>
> How would this compare with the performance with ULOG?

I did try to use uacctd. It seems that it is unable to receive what I
send over NFLOG. Am I correct to assume that it supports only ULOG
(despite the NETLINK_NFLOG mention in the source code)? All other
NFLOG-enabled programs are using libnetlink-log library which configures
everything over netlink instead of with bind/setsockopt.
-- 
Grief can take care of itself; but to get the full value of a joy you must
have somebody to divide it with.
-- Mark Twain

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

[pmacct-discussion] pcap/ulog performance on Linux

[Vincent Bernat]

Hey!

Has someone already done some simple benchmarks to check the performance
of libpcap-based capture with libpcap >= 1.0.0 (mmap support) and Linux
kernel >= 2.6.27 (64bit support). There is another milestone which is
libpcap >= 1.5 and Linux kernel >= 3.10 (support for TPACKET_V3)?

How would this compare with the performance with ULOG?
-- 
Choose a data representation that makes the program simple.
- The Elements of Programming Style (Kernighan & Plauger)

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

Re: [pmacct-discussion] pmacct-contribs now on GitHub

[Vincent Bernat]

 ❦ 11 mai 2013 11:10 CEST, George-Cristian Bîrzan g...@birzan.org :

 Out of curiosity, is there any plan to move the main repo to
 git/mercurial?

If needed, I have already done the conversion using git-cvsimport:
 https://github.com/vincentbernat/pmacct

The commit messages do not respect git expectation (first line should be
a subject), so the list of commits is a bit odd.
-- 
Vincent Bernat | vincent.ber...@dailymotion.com
Dailymotion | 140 Boulevard Malesherbes | 75017 Paris | France

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

[pmacct-discussion] (no subject)

[Vincent Bernat]

Hi!

Here is a patch to use INSERT ... ON DUPLICATE KEY UPDATE instead of
just UPDATE. This increases the performance by an order of magnitude
with MySQL. This relies on the fact that the WHERE clause is
equivalent to the primary key.

There is no option for this because it supported by MySQL since ages.

There also a side fix because on 64 bit arch, %lu is for a long int
which is usually not uint32_t. printf should use PRIu64 and PRIu32.


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

[pmacct-discussion] [PATCH 1/2] mysql: use INSERT ... ON DUPLICATE KEY UPDATE ... instead of just UPDATE

[Vincent Bernat]

The regular INSERT is kept, mostly in case where the user has
requested do not use UPDATE. This still allows pmacct to do bulk
INSERT. However, the regular INSERT should not run in other cases,
hence getting better performance.
---
 src/mysql_plugin.c |   25 +
 1 file changed, 13 insertions(+), 12 deletions(-)

diff --git a/src/mysql_plugin.c b/src/mysql_plugin.c
index edb99df..03a1925 100644
--- a/src/mysql_plugin.c
+++ b/src/mysql_plugin.c
@@ -318,19 +318,7 @@ int MY_cache_dbop(struct DBdesc *db, struct db_cache 
*cache_elem, struct insert_
 
   for (num = 0; set[num].type; num++)
 (*set[num].handler)(cache_elem, idata, num, ptr_set, NULL);
-  
-  /* sending UPDATE query */
-  if (!config.sql_dont_try_update) {
-strncpy(sql_data, update_clause, SPACELEFT(sql_data));
-strncat(sql_data, set_clause, SPACELEFT(sql_data));
-strncat(sql_data, where_clause, SPACELEFT(sql_data));
-
-ret = mysql_query(db-desc, sql_data);
-if (ret) goto signal_error; 
-  }
 
-  if (config.sql_dont_try_update || (mysql_affected_rows(db-desc) == 0)) {
-/* UPDATE failed, trying with an INSERT query */ 
 #if defined HAVE_64BIT_COUNTERS
 if (have_flows) snprintf(ptr_values, SPACELEFT(values_clause), , %llu, 
%llu, %llu), cache_elem-packet_counter, cache_elem-bytes_counter, 
cache_elem-flows_counter);
 else snprintf(ptr_values, SPACELEFT(values_clause), , %llu, %llu), 
cache_elem-packet_counter, cache_elem-bytes_counter);
@@ -338,7 +326,20 @@ int MY_cache_dbop(struct DBdesc *db, struct db_cache 
*cache_elem, struct insert_
 if (have_flows) snprintf(ptr_values, SPACELEFT(values_clause), , %lu, 
%lu, %lu), cache_elem-packet_counter, cache_elem-bytes_counter, 
cache_elem-flows_counter);
 else snprintf(ptr_values, SPACELEFT(values_clause), , %lu, %lu), 
cache_elem-packet_counter, cache_elem-bytes_counter);
 #endif
+  
+  /* sending UPDATE query. Use an INSERT ... ON DUPLICATE KEY UPDATE ... */
+  if (!config.sql_dont_try_update) {
+strncpy(sql_data, insert_clause, sizeof(sql_data));
+strncat(sql_data, values_clause, SPACELEFT(sql_data));
+strncat(sql_data,  ON DUPLICATE KEY UPDATE , SPACELEFT(sql_data));
+strncat(sql_data, set_clause + 4, SPACELEFT(sql_data)); /* SET clause 
without SET  */
+
+ret = mysql_query(db-desc, sql_data);
+if (ret) goto signal_error; 
+  }
 
+  if (config.sql_dont_try_update || (mysql_affected_rows(db-desc) == 0)) {
+/* UPDATE failed, trying with an INSERT query */ 
 if (config.sql_multi_values) { 
   multi_values_handling:
   if (!idata-mv.buffer_elem_num) {
-- 
1.7.10.4


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

[pmacct-discussion] [PATCH 2/2] Use PRIu64 and PRIu32 for when formatting u64 and u32.

[Vincent Bernat]

There is not guarantee that %llu is appropriate to print
u_int64_t. See:
  
http://stackoverflow.com/questions/2844/how-do-you-printf-an-unsigned-long-long-int
---
 src/mysql_plugin.c   |   14 +++---
 src/pgsql_plugin.c   |   35 +++
 src/pmacct-defines.h |4 
 src/pmacct.c |   50 ++
 src/print_plugin.c   |   23 ---
 src/sql_common.c |   20 +++-
 src/sqlite3_plugin.c |   12 +---
 7 files changed, 60 insertions(+), 98 deletions(-)

diff --git a/src/mysql_plugin.c b/src/mysql_plugin.c
index 03a1925..5faed2e 100644
--- a/src/mysql_plugin.c
+++ b/src/mysql_plugin.c
@@ -319,13 +319,13 @@ int MY_cache_dbop(struct DBdesc *db, struct db_cache 
*cache_elem, struct insert_
   for (num = 0; set[num].type; num++)
 (*set[num].handler)(cache_elem, idata, num, ptr_set, NULL);
 
-#if defined HAVE_64BIT_COUNTERS
-if (have_flows) snprintf(ptr_values, SPACELEFT(values_clause), , %llu, 
%llu, %llu), cache_elem-packet_counter, cache_elem-bytes_counter, 
cache_elem-flows_counter);
-else snprintf(ptr_values, SPACELEFT(values_clause), , %llu, %llu), 
cache_elem-packet_counter, cache_elem-bytes_counter);
-#else
-if (have_flows) snprintf(ptr_values, SPACELEFT(values_clause), , %lu, 
%lu, %lu), cache_elem-packet_counter, cache_elem-bytes_counter, 
cache_elem-flows_counter);
-else snprintf(ptr_values, SPACELEFT(values_clause), , %lu, %lu), 
cache_elem-packet_counter, cache_elem-bytes_counter);
-#endif
+  if (have_flows) snprintf(ptr_values, SPACELEFT(values_clause),
+   , %PRIpmcounter, %PRIpmcounter, 
%PRIpmcounter),
+   cache_elem-packet_counter, 
cache_elem-bytes_counter,
+   cache_elem-flows_counter);
+  else snprintf(ptr_values,
+SPACELEFT(values_clause), , %PRIpmcounter, 
%PRIpmcounter),
+cache_elem-packet_counter, cache_elem-bytes_counter);
   
   /* sending UPDATE query. Use an INSERT ... ON DUPLICATE KEY UPDATE ... */
   if (!config.sql_dont_try_update) {
diff --git a/src/pgsql_plugin.c b/src/pgsql_plugin.c
index d1e9c55..8e07ff5 100644
--- a/src/pgsql_plugin.c
+++ b/src/pgsql_plugin.c
@@ -311,19 +311,15 @@ int PG_cache_dbop_copy(struct DBdesc *db, struct db_cache 
*cache_elem, struct in
 num++;
   }
 
-#if defined HAVE_64BIT_COUNTERS
-  if (have_flows) snprintf(ptr_values, SPACELEFT(values_clause), 
%s%llu%s%llu%s%llu\n, delim_buf, cache_elem-packet_counter,
-   
delim_buf, cache_elem-bytes_counter,
-   
delim_buf, cache_elem-flows_counter);
-  else snprintf(ptr_values, SPACELEFT(values_clause), %s%llu%s%llu\n, 
delim_buf, cache_elem-packet_counter,
-   
delim_buf, cache_elem-bytes_counter);
-#else
-  if (have_flows) snprintf(ptr_values, SPACELEFT(values_clause), 
%s%lu%s%lu%s%lu\n, delim_buf, cache_elem-packet_counter,
-   
delim_buf, cache_elem-bytes_counter,
-   
delim_buf, cache_elem-flows_counter);
-  else snprintf(ptr_values, SPACELEFT(values_clause), %s%lu%s%lu\n, 
delim_buf, cache_elem-packet_counter,
-   
delim_buf, cache_elem-bytes_counter);
-#endif
+  if (have_flows) snprintf(ptr_values, SPACELEFT(values_clause),
+   
%s%PRIpmcounter%s%PRIpmcounter%s%PRIpmcounter\n,
+   delim_buf, cache_elem-packet_counter,
+   delim_buf, cache_elem-bytes_counter,
+   delim_buf, cache_elem-flows_counter);
+  else snprintf(ptr_values, SPACELEFT(values_clause),
+%s%PRIpmcounter%s%PRIpmcounter\n,
+delim_buf, cache_elem-packet_counter,
+delim_buf, cache_elem-bytes_counter);
   strncpy(sql_data, values_clause, sizeof(sql_data));
   
   if (PQputCopyData(db-desc, sql_data, strlen(sql_data))  0) { // avoid 
strlen() 
@@ -386,13 +382,12 @@ int PG_cache_dbop(struct DBdesc *db, struct db_cache 
*cache_elem, struct insert_
   if (config.sql_dont_try_update || (!PG_affected_rows(ret))) {
 /* UPDATE failed, trying with an INSERT query */ 
 strncpy(sql_data, insert_clause, sizeof(sql_data));
-#if defined HAVE_64BIT_COUNTERS
-if (have_flows) snprintf(ptr_values, SPACELEFT(values_clause), , %llu, 
%llu, %llu), cache_elem-packet_counter, cache_elem-bytes_counter, 
cache_elem-flows_counter);
-else snprintf(ptr_values, SPACELEFT(values_clause), , %llu, %llu), 
cache_elem-packet_counter, cache_elem-bytes_counter);
-#else
-if (have_flows)