Re: [SailfishDevel] network statistics for sailfish
I'm investigating to see if it is possible to create a network monitor for SailfishOs. Something like https://play.google.com/store/apps/details?id=com.mobidia.android.mdmfeature=search_result I can read simple received/sent statistics from /proc/net/dev , but it would be more interesting to see the amount of data used per application. This is possible with libpcap and maybe with iptables, but both require root privileges (or at least the CAP_NET_RAW capability). So hopefully someone can answer the following questions: - Is there any possibility for a Harbour app to acquire extra priviliges/capabilities ? - If not can you think of any other way to report per-application network statistics ? libpcap can't detect application level data transfer, and neither can iptables plus you don't want to set your interface promiscuous, nor make a complicated iptables thing... afaik, there is no way in linux to filter out per application, because, you'd need to track which application bound what source port and track data that way... it's just not doable without realistically root access, a lot of tracing of all applications (which would slow down) and possibly fucking up the networking/internet access of the phone... most likely you'd have to recompile the kernel even... which might be an option is cgroups (though i doubt it), though i'm unsure if at this time network traffic can be accounted/limited accross cgroups though it could be a nice feature. iow, give it up... :-) plus, when you get to the point of dozens of openvpn networks and bridging and vlans and voip QoS in a different vlan and such, things are gonna get complicated enough... ___ SailfishOS.org Devel mailing list
Re: [SailfishDevel] network statistics for sailfish
On 16 Dec 2013, at 21:09, AL13N al...@rmail.be wrote: I'm investigating to see if it is possible to create a network monitor for SailfishOs. Something like https://play.google.com/store/apps/details?id=com.mobidia.android.mdmfeature=search_result I can read simple received/sent statistics from /proc/net/dev , but it would be more interesting to see the amount of data used per application. This is possible with libpcap and maybe with iptables, but both require root privileges (or at least the CAP_NET_RAW capability). So hopefully someone can answer the following questions: - Is there any possibility for a Harbour app to acquire extra priviliges/capabilities ? - If not can you think of any other way to report per-application network statistics ? libpcap can't detect application level data transfer, and neither can iptables Have a look at the source of for example Nethogs. Libpcap is used to detect packet transfer and then some clever things are done to deduct the application that initiated the transfer. So it's definitely possible. plus you don't want to set your interface promiscuous, nor make a complicated iptables thing... afaik, there is no way in linux to filter out per application, because, you'd need to track which application bound what source port and track data that way... iow, give it up... :-) plus, when you get to the point of dozens of openvpn networks and bridging and vlans and voip QoS in a different vlan and such, things are gonna get complicated enough... Probably that will be the end conclusion, but Android is also Linux, so how do the Android network monitor apps get their information? Does the android stack/Dalvik give extra hooks for network inspection ? thanks anyway for your information, Winfried ___ SailfishOS.org Devel mailing list
Re: [SailfishDevel] network statistics for sailfish
2013/12/16, W. Dobbe winfried.do...@xmsnet.nl: Probably that will be the end conclusion, but Android is also Linux, so how do the Android network monitor apps get their information? Does the android stack/Dalvik give extra hooks for network inspection ? Android is not Linux. Android is as much Linux as iOS is BSD. Android runs on top of a Linux kernel, but that's all Android and GNU/Linux have in common. Network inspection is probably exposed by the Android Java API. Martin ___ SailfishOS.org Devel mailing list
Re: [SailfishDevel] network statistics for sailfish
Hey, 2013/12/16 Martin Grimme martin.gri...@gmail.com: 2013/12/16, W. Dobbe winfried.do...@xmsnet.nl: Probably that will be the end conclusion, but Android is also Linux, so how do the Android network monitor apps get their information? Does the android stack/Dalvik give extra hooks for network inspection ? Android is not Linux. Android is as much Linux as iOS is BSD. Android runs on top of a Linux kernel, but that's all Android and GNU/Linux have in common. Network inspection is probably exposed by the Android Java API. In Android, every application has its own UID, so the bandwidth per app problem can be reduced to bandwidth per user in that case. If you look around the Android sources, there's a way to access these stats: https://github.com/CyanogenMod/android_frameworks_base/blob/cm-10.2/core/java/android/net/NetworkStats.java And this is how the stats are collected: https://github.com/CyanogenMod/android_frameworks_base/blob/cm-10.2/services/java/com/android/server/net/NetworkStatsCollection.java https://github.com/CyanogenMod/android_frameworks_base/blob/cm-10.2/services/java/com/android/server/net/NetworkStatsRecorder.java https://github.com/CyanogenMod/android_frameworks_base/blob/cm-10.2/services/java/com/android/server/net/NetworkStatsService.java With that said, it's definitely possible with some motivation to do per-process bandwidth tracking, we did something like this for a Uni course some years ago: http://thp.io/2010/bwmon/ This uses the ip_conntrack kernel module or - with more recent kernels - conntrack-tools: http://conntrack-tools.netfilter.org/ There's even some documentation on how to use the bwmon utility: http://thp.io/2010/bwmon/manual.html If you plan on using that for the mobile device use case, think about rewriting the thing in C/C++ instead of Python and make it less resource-hungry by reducing wake-ups. As Harbour apps are now prefixed with harbour- and have strict requirements for .desktop and icon file naming, it should be easy to make a system that calculates per-app bandwidth usage and shows the icon of the app and the name of the app without too much work. HTH :) Thomas ___ SailfishOS.org Devel mailing list
Re: [SailfishDevel] network statistics for sailfish
On 16 Dec 2013, at 23:40, Thomas Perl th.p...@gmail.com wrote: Hey, 2013/12/16 Martin Grimme martin.gri...@gmail.com: 2013/12/16, W. Dobbe winfried.do...@xmsnet.nl: Probably that will be the end conclusion, but Android is also Linux, so how do the Android network monitor apps get their information? Does the android stack/Dalvik give extra hooks for network inspection ? Android is not Linux. Android is as much Linux as iOS is BSD. Android runs on top of a Linux kernel, but that's all Android and GNU/Linux have in common. Network inspection is probably exposed by the Android Java API. In Android, every application has its own UID, so the bandwidth per app problem can be reduced to bandwidth per user in that case. If you look around the Android sources, there's a way to access these stats: https://github.com/CyanogenMod/android_frameworks_base/blob/cm-10.2/core/java/android/net/NetworkStats.java And this is how the stats are collected: https://github.com/CyanogenMod/android_frameworks_base/blob/cm-10.2/services/java/com/android/server/net/NetworkStatsCollection.java https://github.com/CyanogenMod/android_frameworks_base/blob/cm-10.2/services/java/com/android/server/net/NetworkStatsRecorder.java https://github.com/CyanogenMod/android_frameworks_base/blob/cm-10.2/services/java/com/android/server/net/NetworkStatsService.java Hi Thomas, thanks a lot for the insight in Androids monitoring. I have 0 experience with Android myself. With that said, it's definitely possible with some motivation to do per-process bandwidth tracking, we did something like this for a Uni course some years ago: http://thp.io/2010/bwmon/ This uses the ip_conntrack kernel module or - with more recent kernels - conntrack-tools: http://conntrack-tools.netfilter.org/ There's even some documentation on how to use the bwmon utility: http://thp.io/2010/bwmon/manual.html If you plan on using that for the mobile device use case, think about rewriting the thing in C/C++ instead of Python and make it less resource-hungry by reducing wake-ups. As Harbour apps are now prefixed with harbour- and have strict requirements for .desktop and icon file naming, it should be easy to make a system that calculates per-app bandwidth usage and shows the icon of the app and the name of the app without too much work. Hmm, I didn't know conntrack. I'll have a look at that. At first sight it still looks like you need root privileges for conntrack though :-( The problem is not to get the stats, if Nethog can do it, so can a Sailfish app. Problem is to do it in such a way that it will be accepted in Harbour. Still, some more leads to investigate this week. Thanks. HTH :) Thomas ___ SailfishOS.org Devel mailing list
Re: [SailfishDevel] network statistics for sailfish
2013/12/16 W. Dobbe winfried.do...@xmsnet.nl: Have a look at the source of for example Nethogs. Libpcap is used to detect packet transfer and then some clever things are done to deduct the application that initiated the transfer. Speaking of Nethogs, I recently made a package for it: https://build.merproject.org/package/show?package=nethogsproject=home%3Azuh It turned out to be useful for benchmarking one's harbour app network usage. Does need root though. -- Kalle Vahlman, Movial Creative Technologies Inc. Porkkalankatu 20, FI-00180 Helsinki Tel +358 9 8567 6400 Fax +358 9 8567 6401 www.movial.com ___ SailfishOS.org Devel mailing list
