On Tue, 20 Sep 2016 01:08:48 -0300 Gustavo Sverzut Barbieri <barbi...@gmail.com> said:
> On Mon, Sep 19, 2016 at 8:44 PM, Carsten Haitzler <ras...@rasterman.com> > wrote: > > On Mon, 19 Sep 2016 11:31:58 -0300 Gustavo Sverzut Barbieri > > <barbi...@gmail.com> said: > > > >> Hi all, > >> > >> Today I did commit support for libproxy in ecore_con's new API, > >> Efl.Net.Dialer.Tcp, Efl.Net.Dialer.Http and Efl.Net.Dialer.Websocket > >> (implicit from HTTP). > > > > ummm i just am looking now (i have a lot of catching up to do). can you undo > > the changes to configure.ac and make this a dlopen/dlsym approach? well use > > eina_module. look a the old curl eina_module approach. look at what i did in > > ecore_audio too. in fact i need to look over all the efl net code, but you > > want to use emile for ssl stuff and eina_module for curl too there too (i > > haven't looked yet). there are very good reasons to do all of this. > > I've checked what you did for curl, since I use that and need more symbols. > > my question and concern are just: > - not being able to compile without libproxy. If we always detect, > should we remove --enable-libproxy from configure.ac? yes. it moves to runtime not compile-time. several reasons: 1. makes compiling simpler. people dont have to find dependencies and those -dev/devel pkgs too. if they add libproxy later after building efl magically the feature works without a rebuild. 2. speed up startup time with less linking going on for apps especially for features they may never use 3. saves memory - symbol tables and dirtying private pages to do the symbol fixups is expensive. i nuked like 320k or so before 1.18 release of actual real memory usage that was dirty pages from rarely used libraries.features. it's expensive and so only load in if/when needed to save this cost. this cost is private pages PER PROCESS using efl so it multiples and is not a one-off cost. 4. the idea of isolating such costs into a proxy daemon/process probably is a good thing and that saves adding the above cost for any process then needing to do proxy pac file parsing etc. and isolates the cost into a single daemon. > - I ask the above because libproxy is a monster, it will pull in a JS > loader, glib, C++... all of that, per process. If using pacrunner > (from connman guys) you can just link with libdbus, less bad. in GENERAL go for a dlopen(eina_module) style approach, BUT if a daemon makes sense, then do that. it makes more sense for us to do our own daemon for efl like efreet. dbus is not a good solution for this. supporting pacrunner later of course is doable, but i would not do this out of the box because pacrunner is not everywhere, dbus session buses do not exist when you do things like use the console (terminology using fbcon is a major problem when it comes to ethumb and was for efreet because no session bus is active thus everything bound to a session bus is broken). i will eventually rewrite ethumb features likely in elm and drop ethumb because of such issues. also ethumb design is broken for SMACK systems so we really do need a "fork off a slave process that is custom for your app only" method of working. for proxies i see the value of a single efl.net.proxy daemon service to serve as a proxy handler - send relevant info to this daemon, let it decide and then answer back with what connection to make. so basically take the libproxy stuff you just committed, move it to a daemon (like efreet does now), and connect to it (execute then connect if connect fails). if you don't, then i'll eventually need to do this. keep this in mind as a design point. for data that is "global" (efreet style stuff, proxy info etc.) then a single daemon doing the work will be the best solution. maybe in future we might/can merge these daemons into a single "efl daemon". > > emile too needs to switch to using this approach internally as well. > > > >> Libproxy (https://libproxy.github.io) tries to dynamically find a > >> proxy for a given URL, unlike plain $http_proxy, $no_proxy... which > >> needs a mess to get right in complex environments such as big > >> corporations. It will also use PAC - ProxyAutoConfiguration, those > >> JavaScript like files named wpad.dat, as well as the wpad host on the > >> local network, where it tries to find the JS file. > >> > >> More than that, libproxy uses the configuration from Desktop > >> Environments. There are none for E (some volunteer?), but we can use > >> the one from Gnome3/Gnome or KDE. It will also handle envvars and some > >> /etc/sysconf configuration. > > > > what kind of config? you mean like a dbus interface exposed from e to > > provide proxy info? i haven't looked... > > whatever you want, like > https://github.com/libproxy/libproxy/tree/master/libproxy/modules > > they have config_envvars that check http_proxy, no_proxy... but they > also have one for gnome that uses gsettings. err so you mean write code into libproxy to get proxy settings from e? a quick look at the above implies that... is there a way of hooking into this e.g. from the above "efl proxy daemon"? actually i'm sure there is... either way a single daemon would solve a lot of the above issues of libproxy pulling in half the universe into every app needing to do network stuff. > >> Just be aware that libproxy is nasty and will load many crap in your > >> application binary, including mozjs to interpret JS mentioned above. > > > > ugh. i know. pac files. js in there. ugh. it's times like this that all the > > "web tech crowd" makes me want to cry. it's like they never cared about > > performance... ever. just stuff some js anywhere because we have a browser > > using it already and hey whats an extra few mb to something already needing > > 1gb to run... :( oh wait1 we already have js spinning so it's no cost to > > us! argh! > > indeed. At least pacrunner isolates this in their daemon, not every > process. And the file is shared... not to say they picked a light JS > engine (duktape) as the wpad.dat is super-simple. yeah. for now i think the best soln for us is a single efreet-like daemon that is spawned if you cant connect to it and THIS daemon deals with proxies. efl/net would auto spawn it like efreet does to efreetd and connect and ask it to process a proxy connection (ie here is a target ip/name/url - tell me what to do". > >> A better solution is to use libproxy replacement from pacrunner > >> (http://git.kernel.org/cgit/network/connman/pacrunner.git/), that one > >> will only pull in libdbus and the JS is done at the daemon, which also > >> keeps shared cache not replicating it in all processes. > > > > much better, OR we even spin off a process - let it execute as a child, read > > back results in parent (and cache them until proxy data changes). > > less bad, but still one per process doing networking. yeah... or just 1 process for all efl apps. ie like efreetd. > >> Be traditional libproxy or pacrunner's alternative, the current code > >> should be more dynamic, however needs more testing. If you work inside > >> a proxy, give it a try and report problems, you can use the > >> src/examples/ecore/efl_io_copier or efl_net_dialer_http_example to > >> test. > > > > we do indeed live inside one at work. and there is the PAC file from hell... > > > >> It's important to get more testing as in near future I'm converting > >> parts of E/EFL that used ecore_con to use this and I don't want to > >> break things :-) > > > > please please... don't race! you bring up very important things above. also > > dlopen the sucker. don't make it a configure option + link option. runtime > > link as needed. duplicate whatever elements of the libproxy headers needed > > to talk to it. > > I can do this very easily, 3 functions, no public struct, just returns > a char **. that's even better! the simpler the headers/structs/api's needed to interface to it the easier it is to make it runtime only. even if it is ONLY accessed by a single daemon it simplified compilation. :) > > we need to do much more of this for "rarely used dependencies". unless it's > > needed by like 80%+ of apps, then it should be dlopen'd instead of linked. > > linking is EXPENSIVE. > > > > also the issue running the PAC files... it makes sense to ecore_exe() a > > slave and have it figure it out once and store the result (ok ok PAC files > > technically have to be re-executed per connect because they can switch per > > target... urgh). we COULd do it like efreet with a daemon approach for now. > > we can support pacrunner as a daemon later too and avoid running our own. > > in fact this is likely the best approach - do it the efreetd way. no no. do > > not use dbus. yes i know it does activation for us. no. this is far more > > trouble, but i won't get into that here. just connect to a local user ipc > > service of our own, if not there, spawn the daemon and try connecting again > > a bit later. repeat until connect succeeded. do it in background async. > > I'd say the sanest route is: > > - dlopen trick on libproxy (3 functions) > > - hint packagers on optional dependency on libproxy > > - don't bother with libproxy.so (traditional) impact, hint users to > go pacrunner. Note my code already uses libproxy 'get proxies for url' > in a thread, as they recommend in their project. > > - use pacrunner where performance matters. > > - push distros to package pacrunner (glib + dbus + curl, very simple) we pushed connman years ago. not again. distros just don't want to. we need to DIY and provide our own "pacrunner". that's easy enough for us as we do it with efreetd already. we have lots of infra to do this easily. if we just have an eflnet "proxy handler" and efl.net spawns and talks to it we have a solution that doesn't require pacrunner to be packaged and so on (going by history of connman i just dont want to go here again), and it's TRIVIAL for us to support. you have there all the relevant code already. just needs to be put into its own binary, with an ipc/whatever conn for the user and some code to connect and/or launch it. > AFAIU (still trying to get more details on the working), pacrunner > will interact with connman's proxy configuration... I need to stop and > read/ask which one is the controller. But that's the ideal situation, > we already do connman, set property there and it spreads to the whole > system. the problem is 99% of users dont want or use connman. after years i've seen that and i don't see that binding ourselves even deeper is a good idea. connman works fine. i use it personally. but it's just not palatable for distros or users. providing our own little proxy daemon process like efreetd would be a self-sustained solution that is GUARANTEED to be installed and work in every environment. -- ------------- Codito, ergo sum - "I code, therefore I am" -------------- The Rasterman (Carsten Haitzler) ras...@rasterman.com ------------------------------------------------------------------------------ _______________________________________________ enlightenment-devel mailing list enlightenment-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
