I buffer input from a socket so that socket reads are performed in large chunks and extract characters from that buffer as needed by parsers etc.
To write data to a socket I buffer the write into a memory mapped file and then use sendfile (TransmitFile on Windows) to move it out efficiently with a minimum of buffer shadowing. Serving files uses the same mechanism. To send a short message to a socket I buffer the message and use a chunk of code I lifted from Berkeley Unix to handle a send to a non-blocking socket and accommodate the partial writes and uncertain events which may happen. An HTTP message created by the application server (an RPC or a file created by the embedded server page processor) hits the network with a single sendfile and a served file has the header in a message followed by a sendfile. To get good performance using HTTP protocol I built the socket interface and the HTTP layer so that I could remove inefficiencies and so that I could be certain of eliminating any buffer overflow vulnerabilities inherent in using a library. Using a carefully hand coded HTTP parser lifted performance. When handling AJAX traffic the major time spent in a 20MS turn around transaction is spent in the Javascript and and communication interface in the client browser. The next largest overhead is in the Sqlite DB access, as one would expect. Sqlite activity will max out processor usage but the server overhead is relatively light. On a simple test server, an 800MHZ Pentium with SCSI disks salvaged from a dumpster and running Ubuntu this server can scale up to handling a large number of concurrent clients. On a multi-processor IBM P-Series server running SUSE Enterprise Server its performance is enterprise level. On a 166MHZ IBM RS/6000 43P recycled from a junk pile and running AIX it has strong departmental performance. I also turn Nagle's algorithm on and off to try to take advantage of a possible capability of the network interface combining small packets, although experience has shown me that this is not always present. RPCs require that Nagles be turned off. The magic potion is the ability to embed Sqlite in the application server and avoid IPCs and multiple processes. One application of this server uses remote Windows programs which perform real time access to scales and use RPCs encapsulated in XML. Mostly the RPCs use JSON to browsers. JS Alex Katebi wrote: > Hi John, > > I was writing a message into the socket partially. I had done this to > avoid coping. This caused many transmissions for a single message. At the > other end I was doing MSG_PEEK optional flag which was wasteful. I had done > this to dump the message for debugging. > > By building my message into memory. I was able to send the entire message > in one socket write. At the other end of the socket I did the > MSG_WAITALLoptional for the > recv function. This caused the receiver to be active only when a > usable message peace was received. > > One might say that a misbehaving client could still slow down the server. A > server should respond with a notification before dropping this client for > sending runt messages. > > What is your design like? > > Thanks, > -Alex > > > On Tue, Jun 17, 2008 at 11:57 AM, John Stanton <[EMAIL PROTECTED]> wrote: > >> What did you change? What was causing the lag? >> >> Alex Katebi wrote: >>> slowness is fixed. Can't tell the difference between client/server speed >>> from library. >>> >>> On Sat, Jun 14, 2008 at 8:32 PM, Alex Katebi <[EMAIL PROTECTED]> >> wrote: >>> >>>> Hi All, >>>> >>>> Looks like there is some interest. I will announce when I release it. >>>> Currently I am developing an interactive user shell client. This shell is >>>> used for my client/server development. It can also be embedded for any >>>> clients user interface. >>>> The request/response is a little slow for some reason. I need to fix this >>>> issue. >>>> >>>> Thanks, >>>> -Alex >>>> >>>> >>>> >>>> On Mon, Jun 2, 2008 at 11:40 AM, Alex Katebi <[EMAIL PROTECTED]> >>>> wrote: >>>> >>>> >>>>> Hi All, >>>>> >>>>> I am using remote procedure calls (RPC) for SQLite in my application. >> I >>>>> have implemented a few SQLite RPC functions that I needed successfully. >>>>> I am wondering if there are other people like me who need this. >>>>> If there are enough people who could benefit from this I can make it >>>>> available as an open source public domain software. >>>>> Then people can add more functions as needed. >>>>> >>>>> Thanks, >>>>> -Alex >>>>> >>>> >> > _______________________________________________ >>> sqlite-users mailing list >>> sqlite-users@sqlite.org >>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users >> _______________________________________________ >> sqlite-users mailing list >> sqlite-users@sqlite.org >> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users >> > _______________________________________________ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
