Re: [PLUG] monitoring MAGNITUDE of internet activity - how?

[Richard Owlett]

On 03/04/2017 11:02 AM, Galen Seitz wrote:
> On 03/04/17 00:41, Richard Owlett wrote:
>> On 03/02/2017 01:48 PM, Keith Lofstrom wrote:
>>> On Wed, Mar 01, 2017 at 04:33:57PM -0600, Richard Owlett wrote:
 Is there a tool which can be
  "turned on"  at time t0
  "turned off" at time t1
 which will report the number of
   "uploaded"   bytes
   "downloaded" bytes
 in that interval?
>>>
>>> [snip]
>>
>> For my needs I'll skip aiming at a spreadsheet and write some simple
>> bash scripts to gather data and do some basic formatting. For anything
>> more complex I'll use Tcl/Tk as I use it in another project. The more
>> practice the better.
>>[snip]
>
> If you want to use this as a learning exercise, go for it.  If not,
> then I feel you are making this more complicated than necessary.
>
> First capture your data as Robert suggested.  Change the 10 to
> whatever time interval is desired.
> cp /proc/net/dev /tmp/foo; sleep 10; cat /proc/net/dev >> /tmp/foo
>
> Now open the file using libreoffice.  Under Separator options, select
> "Space" and "Merge delimiters".  Click OK.  Your task, as you have
> described it, is now about 95% complete.

Learning is *A* motivation.
However using LibreOffice to substitute for a few lines of bash 
scripting seems like driving an 18-wheeler to the corner grocery for a 
quart of milk ;/
YMMV

Now all I have to do is learn bash and possibly awk and some grep nuances.

Retirement *IS* for learning.



___
PLUG mailing list
PLUG@lists.pdxlinux.org
http://lists.pdxlinux.org/mailman/listinfo/plug

Re: [PLUG] monitoring MAGNITUDE of internet activity - how?

[Galen Seitz]

On 03/04/17 00:41, Richard Owlett wrote:
> On 03/02/2017 01:48 PM, Keith Lofstrom wrote:
>> On Wed, Mar 01, 2017 at 04:33:57PM -0600, Richard Owlett wrote:
>>> Is there a tool which can be
>>>  "turned on"  at time t0
>>>  "turned off" at time t1
>>> which will report the number of
>>>   "uploaded"   bytes
>>>   "downloaded" bytes
>>> in that interval?
>>
>> For your purposes, perhaps you pipe a standard interface
>> reporting tool into a text file that you can edit into
>> .cvs format for a spreadsheet.
>>
>> However, sometimes the easiest way to do stuff is to learn
>> a scripting language and automate a task.  The good thing
>> about scripting languages is that you can always look at
>> the program to remind yourself how it works.
>> [snip]
> 
> For my needs I'll skip aiming at a spreadsheet and write some simple 
> bash scripts to gather data and do some basic formatting. For anything 
> more complex I'll use Tcl/Tk as I use it in another project. The more 
> practice the better.
> 
> I someone asks "How much can be done in bash?", I suggest browsing
> http://www.tldp.org/LDP/Bash-Beginners-Guide/html/Bash-Beginners-Guide.html
> and
> http://www.tldp.org/LDP/abs/html .

If you want to use this as a learning exercise, go for it.  If not, then
I feel you are making this more complicated than necessary.

First capture your data as Robert suggested.  Change the 10 to whatever
time interval is desired.
cp /proc/net/dev /tmp/foo; sleep 10; cat /proc/net/dev >> /tmp/foo

Now open the file using libreoffice.  Under Separator options, select
"Space" and "Merge delimiters".  Click OK.  Your task, as you have
described it, is now about 95% complete.

galen
-- 
Galen Seitz
gal...@seitzassoc.com
___
PLUG mailing list
PLUG@lists.pdxlinux.org
http://lists.pdxlinux.org/mailman/listinfo/plug

Re: [PLUG] monitoring MAGNITUDE of internet activity - how?

[Richard Owlett]

On 03/02/2017 01:48 PM, Keith Lofstrom wrote:
> On Wed, Mar 01, 2017 at 04:33:57PM -0600, Richard Owlett wrote:
>> Is there a tool which can be
>>  "turned on"  at time t0
>>  "turned off" at time t1
>> which will report the number of
>>   "uploaded"   bytes
>>   "downloaded" bytes
>> in that interval?
>
> For your purposes, perhaps you pipe a standard interface
> reporting tool into a text file that you can edit into
> .cvs format for a spreadsheet.
>
> However, sometimes the easiest way to do stuff is to learn
> a scripting language and automate a task.  The good thing
> about scripting languages is that you can always look at
> the program to remind yourself how it works.
> [snip]

For my needs I'll skip aiming at a spreadsheet and write some simple 
bash scripts to gather data and do some basic formatting. For anything 
more complex I'll use Tcl/Tk as I use it in another project. The more 
practice the better.

I someone asks "How much can be done in bash?", I suggest browsing
http://www.tldp.org/LDP/Bash-Beginners-Guide/html/Bash-Beginners-Guide.html
and
http://www.tldp.org/LDP/abs/html .




___
PLUG mailing list
PLUG@lists.pdxlinux.org
http://lists.pdxlinux.org/mailman/listinfo/plug

Re: [PLUG] monitoring MAGNITUDE of internet activity - how?

[Keith Lofstrom]

On Wed, Mar 01, 2017 at 04:33:57PM -0600, Richard Owlett wrote:
> Is there a tool which can be
>  "turned on"  at time t0
>  "turned off" at time t1
> which will report the number of
>   "uploaded"   bytes
>   "downloaded" bytes
> in that interval?

For your purposes, perhaps you pipe a standard interface
reporting tool into a text file that you can edit into
.cvs format for a spreadsheet. 

However, sometimes the easiest way to do stuff is to learn
a scripting language and automate a task.  The good thing
about scripting languages is that you can always look at
the program to remind yourself how it works.

A decade ago, I wrote a cheesy perl hack to wrap around
"ifconfig" to look at total bandwidth used, then modified
it a year ago to look at specific interface usage.  It is
very dependent on the behavior of /sbin/ifconfig for an
old distro, but it is easy to modify.  It would be even
easier if "standard" tool maintainers were more
intelligent and responsible and less "creative".

Keith


--
#!/usr/bin/perl
#  ifbr60 - average traffic rate for 60 seconds
#  depends on specific behavior of /sbin/ifconfig
#  V0.1.1  Keith Lofstrom   KLIC   2016 Jan 30

my $delay = 60 ;

use bigint ;  # the byte counts can be very large
my $ifc;  # interface name
my $encap  ;  # type of encapsulation
my $mac;  # mac (hardware) address
my $updn   ;  # interface up or down?
my $inet   ;  # ipv4 internet address
my $txrx   ;  # sum of transmit and receive bytes
my $err;  # sum of transmit and receive errors

my $ifcNum = scalar(@ARGV);
my @r, @t;

for my $pass (0..1) {
  my $ifcnt = -1 ;
  foreach my $port (@ARGV) {
$ifcnt  += 1 ;  # start at 0
open (IFOUT, "/sbin/ifconfig |" );
  while () {
if( /(\w+)\s+Link encap:(\w+)/ ) {
   $ifc   = $1 ;
} elsif( /inet addr:([\d\.]+)/ ) {
   $inet = $1 ;
} elsif( /RX bytes:(\d+).+TX bytes:(\d+)/ ) {
   if( $port eq $ifc ) {
  printf "%4s%14s%16s.r%16s.t\n", $ifc,$inet,$1,$2 ;
  if( $pass == 0 ) {
 $r[$ifcnt] = $1 ;
 $t[$ifcnt] = $2 ;
  } else {
 my $rd = ( $1 - $r[$ifcnt] ) / $delay ;
 my $td = ( $2 - $t[$ifcnt] ) / $delay ;
 printf "%4s%14s%15.3f/s %15.3f/s\n\n", $ifc,$inet,$rd,$td ;
  }
   }
}
  } 
   }
   if( $pass == 0 ) {
  printf("\n");
  sleep $delay;
   }
} 
close IFOUT;


-- 
Keith Lofstrom  kei...@keithl.com
___
PLUG mailing list
PLUG@lists.pdxlinux.org
http://lists.pdxlinux.org/mailman/listinfo/plug

Re: [PLUG] monitoring MAGNITUDE of internet activity - how?

[Richard Owlett]

On 03/01/2017 07:23 PM, King Beowulf wrote:
> On 03/01/2017 02:33 PM, Richard Owlett wrote:
>> *NOTE BENE*
>> Capitalization in subject line semantically important :>
>>
>> I have significant bandwidth constraints.
>> I have been asked to participate in a project requiring significant data
>> transfers.
>> I wish to run a test case to estimate the connectivity "cost".
>> If it is significant, I use Debian Jessie.
>> For the test case the source/destination is of secondary importance.
>>
>> Is there a tool which can be
>>   "turned on"  at time t0
>>   "turned off" at time t1
>> which will report the number of
>>"uploaded"   bytes
>>"downloaded" bytes
>> in that interval?
>>
>> If the tool reports more, no problem -- it will likely be ignored.
>> TIA
>>
>
> I forgot about this one.  Probably easier to use as a shell script:
> http://humdi.net/vnstat/
>

Not fine grained enough for the project which triggered my question.
However, it can likely answer questions that haven't arisen yet. As it's 
in the Debian repository I'll install and give it a test run.




___
PLUG mailing list
PLUG@lists.pdxlinux.org
http://lists.pdxlinux.org/mailman/listinfo/plug

Re: [PLUG] monitoring MAGNITUDE of internet activity - how?

[Richard Owlett]

On 03/01/2017 05:32 PM, Richard Owlett wrote:
> On 03/01/2017 04:44 PM, Robert Citek wrote:
>> Would /proc/net/dev give you the info you are looking for?
>
> I don't know.
> Will investigate when awake.
> Thank you.

A good night's sleep and a cup of coffee (even if decaf) yields a 
*POSITIVE* verdict. It and some simple scripts will allow me record with 
appropriate resolution/granularity different processes which could last 
from minutes to hours. It seems "made for" the problem that prompted my 
question.


>
>
>>
>> $ sudo cat /proc/net/dev
>> Inter-|   Receive|  Transmit
>>  face |bytespackets errs drop fifo frame compressed
>> multicast|bytespackets errs drop fifo colls carrier compressed
>>   eth0: 16839226   26454000 0  0 0
>> 16903410   27268000 0   0  0
>>   eth1:   0   0000 0  0 0
>> 578   7000 0   0  0
>> lo:   0   0000 0  0 0
>>   0   0000 0   0  0
>>
>> Capture at t0 and t1, parse, diff.
>>
>> Regards,
>> - Robert
>>
>> On Wed, Mar 1, 2017 at 2:33 PM, Richard Owlett  wrote:
>>> *NOTE BENE*
>>> Capitalization in subject line semantically important :>
>>>
>>> I have significant bandwidth constraints.
>>> I have been asked to participate in a project requiring significant data
>>> transfers.
>>> I wish to run a test case to estimate the connectivity "cost".
>>> If it is significant, I use Debian Jessie.
>>> For the test case the source/destination is of secondary importance.
>>>
>>> Is there a tool which can be
>>>   "turned on"  at time t0
>>>   "turned off" at time t1
>>> which will report the number of
>>>"uploaded"   bytes
>>>"downloaded" bytes
>>> in that interval?
>>>
>>> If the tool reports more, no problem -- it will likely be ignored.
>>> TIA
>>>
>>> ___
>>> PLUG mailing list
>>> PLUG@lists.pdxlinux.org
>>> http://lists.pdxlinux.org/mailman/listinfo/plug
>> ___
>> PLUG mailing list
>> PLUG@lists.pdxlinux.org
>> http://lists.pdxlinux.org/mailman/listinfo/plug
>>
>
>
> ___
> PLUG mailing list
> PLUG@lists.pdxlinux.org
> http://lists.pdxlinux.org/mailman/listinfo/plug
>


___
PLUG mailing list
PLUG@lists.pdxlinux.org
http://lists.pdxlinux.org/mailman/listinfo/plug

Re: [PLUG] monitoring MAGNITUDE of internet activity - how?

[King Beowulf]

On 03/01/2017 02:33 PM, Richard Owlett wrote:
> *NOTE BENE*
> Capitalization in subject line semantically important :>
> 
> I have significant bandwidth constraints.
> I have been asked to participate in a project requiring significant data 
> transfers.
> I wish to run a test case to estimate the connectivity "cost".
> If it is significant, I use Debian Jessie.
> For the test case the source/destination is of secondary importance.
> 
> Is there a tool which can be
>   "turned on"  at time t0
>   "turned off" at time t1
> which will report the number of
>"uploaded"   bytes
>"downloaded" bytes
> in that interval?
> 
> If the tool reports more, no problem -- it will likely be ignored.
> TIA
> 

I forgot about this one.  Probably easier to use as a shell script:
http://humdi.net/vnstat/

-Ed

___
PLUG mailing list
PLUG@lists.pdxlinux.org
http://lists.pdxlinux.org/mailman/listinfo/plug

Re: [PLUG] monitoring MAGNITUDE of internet activity - how?

[King Beowulf]

On 03/01/2017 02:33 PM, Richard Owlett wrote:
> *NOTE BENE*
> Capitalization in subject line semantically important :>
> 
> I have significant bandwidth constraints.
> I have been asked to participate in a project requiring significant data 
> transfers.
> I wish to run a test case to estimate the connectivity "cost".
> If it is significant, I use Debian Jessie.
> For the test case the source/destination is of secondary importance.
> 
> Is there a tool which can be
>   "turned on"  at time t0
>   "turned off" at time t1
> which will report the number of
>"uploaded"   bytes
>"downloaded" bytes
> in that interval?
> 
> If the tool reports more, no problem -- it will likely be ignored.
> TIA
> 

'iftop' is a nifty bandwidth tool.  it has a bunch of options

also, if you run a OpenWRT compatible Wifi router, you can give firmware
from

 https://www.gargoyle-router.com/

Its a customization of OpenWrt that can provide heaps of bandwidth data.

___
PLUG mailing list
PLUG@lists.pdxlinux.org
http://lists.pdxlinux.org/mailman/listinfo/plug

Re: [PLUG] monitoring MAGNITUDE of internet activity - how?

[Richard Owlett]

On 03/01/2017 04:44 PM, Robert Citek wrote:
> Would /proc/net/dev give you the info you are looking for?

I don't know.
Will investigate when awake.
Thank you.


>
> $ sudo cat /proc/net/dev
> Inter-|   Receive|  Transmit
>  face |bytespackets errs drop fifo frame compressed
> multicast|bytespackets errs drop fifo colls carrier compressed
>   eth0: 16839226   26454000 0  0 0
> 16903410   27268000 0   0  0
>   eth1:   0   0000 0  0 0
> 578   7000 0   0  0
> lo:   0   0000 0  0 0
>   0   0000 0   0  0
>
> Capture at t0 and t1, parse, diff.
>
> Regards,
> - Robert
>
> On Wed, Mar 1, 2017 at 2:33 PM, Richard Owlett  wrote:
>> *NOTE BENE*
>> Capitalization in subject line semantically important :>
>>
>> I have significant bandwidth constraints.
>> I have been asked to participate in a project requiring significant data
>> transfers.
>> I wish to run a test case to estimate the connectivity "cost".
>> If it is significant, I use Debian Jessie.
>> For the test case the source/destination is of secondary importance.
>>
>> Is there a tool which can be
>>   "turned on"  at time t0
>>   "turned off" at time t1
>> which will report the number of
>>"uploaded"   bytes
>>"downloaded" bytes
>> in that interval?
>>
>> If the tool reports more, no problem -- it will likely be ignored.
>> TIA
>>
>> ___
>> PLUG mailing list
>> PLUG@lists.pdxlinux.org
>> http://lists.pdxlinux.org/mailman/listinfo/plug
> ___
> PLUG mailing list
> PLUG@lists.pdxlinux.org
> http://lists.pdxlinux.org/mailman/listinfo/plug
>


___
PLUG mailing list
PLUG@lists.pdxlinux.org
http://lists.pdxlinux.org/mailman/listinfo/plug

Re: [PLUG] monitoring MAGNITUDE of internet activity - how?

[Robert Citek]

Would /proc/net/dev give you the info you are looking for?

$ sudo cat /proc/net/dev
Inter-|   Receive|  Transmit
 face |bytespackets errs drop fifo frame compressed
multicast|bytespackets errs drop fifo colls carrier compressed
  eth0: 16839226   26454000 0  0 0
16903410   27268000 0   0  0
  eth1:   0   0000 0  0 0
578   7000 0   0  0
lo:   0   0000 0  0 0
  0   0000 0   0  0

Capture at t0 and t1, parse, diff.

Regards,
- Robert

On Wed, Mar 1, 2017 at 2:33 PM, Richard Owlett  wrote:
> *NOTE BENE*
> Capitalization in subject line semantically important :>
>
> I have significant bandwidth constraints.
> I have been asked to participate in a project requiring significant data
> transfers.
> I wish to run a test case to estimate the connectivity "cost".
> If it is significant, I use Debian Jessie.
> For the test case the source/destination is of secondary importance.
>
> Is there a tool which can be
>   "turned on"  at time t0
>   "turned off" at time t1
> which will report the number of
>"uploaded"   bytes
>"downloaded" bytes
> in that interval?
>
> If the tool reports more, no problem -- it will likely be ignored.
> TIA
>
> ___
> PLUG mailing list
> PLUG@lists.pdxlinux.org
> http://lists.pdxlinux.org/mailman/listinfo/plug
___
PLUG mailing list
PLUG@lists.pdxlinux.org
http://lists.pdxlinux.org/mailman/listinfo/plug

[PLUG] monitoring MAGNITUDE of internet activity - how?

[Richard Owlett]

*NOTE BENE*
Capitalization in subject line semantically important :>

I have significant bandwidth constraints.
I have been asked to participate in a project requiring significant data 
transfers.
I wish to run a test case to estimate the connectivity "cost".
If it is significant, I use Debian Jessie.
For the test case the source/destination is of secondary importance.

Is there a tool which can be
  "turned on"  at time t0
  "turned off" at time t1
which will report the number of
   "uploaded"   bytes
   "downloaded" bytes
in that interval?

If the tool reports more, no problem -- it will likely be ignored.
TIA

___
PLUG mailing list
PLUG@lists.pdxlinux.org
http://lists.pdxlinux.org/mailman/listinfo/plug