[Ilugc] different network I/O handling methods
Dear LUG, Been a long while since I initiated a mail to LUG. I thought I will discuss different network I/O schemes or rather the way CPU and memory is used to perform what most of us do today. Be it google search or facebook or mail or whatever this is what is key. Right from publishing SSLC results to Deepavali mails to some event like earthquake jamming communications, the ability to scale with load and the load handling capability of servers is put to test. I am an expert in server side gear. I have been doing this the last 10 years and more and starting from OpenBSD kernel, IPsec, routing to Apache to web development I have seen quite a bit of programming techniques. It is a no brainer that if you want performance you write in C. You may know Linus Torvald's comments on C++. Okay I am not really concerned about programming language right now. Only the high level design and coding will just follow. Broadly put there are a few ways by which multi processing is done. 1) fork() based process creation(Apache pre forked model) 2) multi threading 3) event based I/O loop handling You should read the paper by Ousterhout, the author of Tcl about why threading is a bad idea and event loops are better. This has been repeatedly demonstrated not only in my real life experience but also by the success of Twisted Python or nginx server. Apache used only two approaches, 1.x pre forked model and 2.x threading. nginx did better through event based I/O. A lot of UNIX Programmers do not know about select(2). It is the easiest way to handle multiple sockets, be it STDIN, TCP or UDP. You could also use poll(2). Nowadays I just use poll.It is better than select. You also have many methods to monitor file I/O. FreeBSD uses kqueue(2) and Linux has inotify and friends. All UNICes have event(3) library which is a generic library for network and file I/O using the event loop model. If you have stayed with me so far then you can also perhaps guess why a straight forward light weight process(thread) cannot scale or improve. It most of the time causes bugs and performance loss. Reason is simple. By having multiple network connections you get performance. This is how download accelerators work. But using pipelining also improves performance which is not using multiple connections. How to resolve this paradox? Multiple network connections get you the bandwidth that are shared on the LAN, so you can use the WAN bandwidth to your advantage. However in protocols like SMTP and HTTP, creating a separate connection for every mail or web request is a drag on performance. Enough for now. -Girish -- G3 Tech Networking appliance company web: http://g3tech.in mail: gir...@g3tech.in ___ ILUGC Mailing List: http://www.ae.iitm.ac.in/mailman/listinfo/ilugc
Re: [Ilugc] different network I/O handling methods
Been a long while since I initiated a mail to LUG. I thought I will discuss different network I/O schemes or rather the way CPU and memory is used to perform what most of us do today. It will be better if you put this in a web page and provide the link. It is difficult to read, mostly because, formatting plain text just don't go well together unless you put a lot of effort in. It is a no brainer that if you want performance you write in C. You may know Linus Torvald's comments on C++. Who gives a damn about what he thinks about C or C++ for that matter. I hate C++ myself and he didn't have say it. I hate java too but I guess the discussion on programming languages I hate doesn't belong here. People who like C++/Java, please don't feel offended, this is just a personal opinion/choice. -- 0 ___ ILUGC Mailing List: http://www.ae.iitm.ac.in/mailman/listinfo/ilugc
Re: [Ilugc] why embedded Linux not having swap partition.
On Thu, Mar 29, 2012 at 7:38 AM, kenneth gonsalves law...@thenilgiris.comwrote: On Wed, 2012-03-28 at 15:21 +0200, Krishna wrote: This thread is full of theoretical stuff, lot of useful info but nowadays I only deal with practical reality. This is No Theory, I am not understanding what you mean by practical reality. Can you explain the difference? How far is it from practical? if we say it, it is theory. If he says it, it is practical. I agree with you ;) ___ ILUGC Mailing List: http://www.ae.iitm.ac.in/mailman/listinfo/ilugc
Re: [Ilugc] different network I/O handling methods
Hi, On Sat, Mar 31, 2012 at 8:24 PM, Girish Venkatachalam girishvenkatacha...@gmail.com wrote: 3) event based I/O loop handling You should read the paper by Ousterhout, the author of Tcl about why threading is a bad idea and event loops are better. This is something that is easy to get wrong. I still see a lot of people argue that multi-threaded is the way to go. But event based approaches are a lot better. Also, I guess beyond a limit, multi-threaded approach doesn't scale. You could also use poll(2). Nowadays I just use poll.It is better than select. You also have many methods to monitor file I/O. FreeBSD uses kqueue(2) and Linux has inotify and friends. All UNICes have event(3) library which is a generic library for network and file I/O using the event loop model. In GNU/Linux, we now have epoll(7). Epoll is an event based I/O notification based library. scales better normally, but can be a little tricky if we are not used to event loops/event based programming. Thanks -- Thank you Balachandran Sivakumar Arise Awake and stop not till the goal is reached. - Swami Vivekananda Mail: benignb...@gmail.com Blog: http://benignbala.wordpress.com/ ___ ILUGC Mailing List: http://www.ae.iitm.ac.in/mailman/listinfo/ilugc
Re: [Ilugc] different network I/O handling methods
On 31-Mar-2012, at 2:19 PM, Balachandran Sivakumar wrote: In GNU/Linux, we now have epoll(7). Epoll is an event based I/O notification based library. scales better normally, but can be a little tricky if we are not used to event loops/event based programming. Thanks And then there are libev[1] and libevent[2] libraries which offers a nice abstraction over poll, kqueue, epoll, select etc. for events based programming. GLIb[3] has a very easy interface to write an event loop too. Feels OOP but in C if thats your cup of tea ;). Needless to say, programs written using these libraries are portable across multiple platforms. [1]: http://software.schmorp.de/pkg/libev.html [2]: http://libevent.org/ [3]: http://developer.gnome.org/glib/2.30/glib-The-Main-Event-Loop.html - Sudharshan ___ ILUGC Mailing List: http://www.ae.iitm.ac.in/mailman/listinfo/ilugc
[Ilugc] Any report making tools in php for MYSQL database?
Hi Guys, I am new to php mysql report report making. Any tools available?. I basically need to have a layout setup. It can be making pdf.give any clues. My database table having four columns when user select date. particular date, data will generated to pdf file. -Ganesh. Did I learn something today? If not, I wasted it. ___ ILUGC Mailing List: http://www.ae.iitm.ac.in/mailman/listinfo/ilugc
Re: [Ilugc] Any report making tools in php for MYSQL database?
Hi, On Sun, Apr 1, 2012 at 9:03 AM, Ganesh Kumar bugcy...@gmail.com wrote: Hi Guys, I am new to php mysql report report making. Any tools available?. I basically need to have a layout setup. It can be making pdf.give any clues. Should it be specifically in PHP? If not, you can try Python's report lab. Another option is to use phpmyadmin. IIRC, it has options to generate some reports. For multiple users, you can provide logins with restricted privileges. But it needs to be tried out because I am not sure of the exact set of features that it supports. Thanks -- Thank you Balachandran Sivakumar Arise Awake and stop not till the goal is reached. - Swami Vivekananda Mail: benignb...@gmail.com Blog: http://benignbala.wordpress.com/ ___ ILUGC Mailing List: http://www.ae.iitm.ac.in/mailman/listinfo/ilugc
Re: [Ilugc] Any report making tools in php for MYSQL database?
On Sun, Apr 1, 2012 at 9:03 AM, Ganesh Kumar bugcy...@gmail.com wrote: Hi Guys, I am new to php mysql report report making. Any tools available?. I basically need to have a layout setup. It can be making pdf.give any clues. My database table having four columns when user select date. particular date, data will generated to pdf file. Dunno what you mean by layout. You have several different PDF generation Perl modules on CPAN. They take input in various forms, text is obviously the default choice. So you can use an SQL query tool that can get you text output of your columns and you can generate a table from it as a PDF file. I use PDF::Table, PDF::API2. -Girish ___ ILUGC Mailing List: http://www.ae.iitm.ac.in/mailman/listinfo/ilugc
