Re: Libev on windows, binding to the D language
On Tue, Jun 03, 2008 at 11:21:29PM +0100, James Mansion <[EMAIL PROTECTED]>
wrote:
> Marc Lehmann wrote:
> >- get rid of silly, idiotic, and threatening disclaimers.
> >
> Good grief. Get a life, Marc.
You might have missed reality completely:
> Anyone coding in a professional capacity in a medium or large business
> is liable to have absolutely no say in whether such disclaimers get
> added.
This is total bullshit:
- I have full say in that.
- the original poster with the disclaimer also had that capacity.
- every other person with such a stupid disclaimer in the past was
capable of switching it off.
> In some cases its effectively a regulatory requirement.
It's trivial to work around this so-called requirement in all cases I have
seen so far.
Note also that in some countries, you can get into jail for doing
"interesting" things, such as voting for the opposition party, or
demonstrating against the government. It's the law...
The fact that such stupid requirements exist does not mean I need to
support them.
Your argument of "it's a requirement in some cases" has no value
whatsoever. It is stupid, it is anti-social.
> Its hardly condusive to helping libev users if they can't post here to get
True - but what you desribe ("cannot post here") has not happened, and there
is no indication that it will ever keep people from posting here.
This is juts a red herring.
> support without forwarding stuff home and then posting from home (I've had
> to do that, but I'd rather not hang out anywhere someone takes that sort of
> attitude).
You are free to go if you disagree, that's your right. At least, I think it
should be your right, maybe it's a regulatory requirement where you live...
> >- take responsibility - if you send a virus, you clearly are responsible
> > for that.
> >
> If you send it, maybe. If it arrives with a virus in, that's something
> else again.
> There's no guarantee that plain mail arrives as sent.
Another red herring. The fatc that mail could be manipulated does not mean
it will be manipulated. This kind of argument is just stupid.
> The reality is that most people who know enough to even consider using
> something like libev probably know that, but that doesn't mean that its not
> worth pointing out to everone else.
It is pointless to point it out here, as it has nothing to do with the issue
at hand.
And if you disagree with the policies of this list, you are free to leave.
Threatening with legal action, or forcing the receive to make extra
checks, is just plain stupid, and very unfriendly towards other people on
this list.
--
The choice of a Deliantra, the free code+content MORPG
-==- _GNU_ http://www.deliantra.net
==-- _ generation
---==---(_)__ __ __ Marc Lehmann
--==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED]
-=/_/_//_/\_,_/ /_/\_\
___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Re: libeio, Windows
On Tue, Jun 03, 2008 at 04:00:40PM -0600, Tony Arcieri <[EMAIL PROTECTED]> wrote: > All right, all that said, it looks like libeio can't be used as the basis of > a Proactor (particularly one that needs to deal with sockets in addition to > files, etc). My mistake. Just as libev can't be used for the same thing: it doesn't work on files. I don't think that logic makes much sense - for files, libeio can be used, for file descriptors supporting non-blicking operation, libev can be used. They are complementary, not exclusive to each other. Using one does not preclude using the other, in fatc, most programs would probbaly take advantage of both if both are available. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Re: Compilation on windows
On Wed, Jun 04, 2008 at 02:46:59PM +1000, Chris Hulbert <[EMAIL PROTECTED]> wrote: > indeed the first parameter of the select() function is ignored so it Close to truth: winsocket's select function indeed ignores it, most other implementations of it do not (on unix, you rarely have more than one, but on windows you have a dozen or so select()'s). > doesn't matter, we could leave it as zero even. Thanks for analysing this and your patch! I will add a dummy definition of NFDBITS when libev is configured for winsockets select - could you give CVS a try? (You don't have to: your patch should work fine for your environment, but I cnanot use it directly) You cna chekc out a copy as described here under Anonymous CVS checkout: http://software.schmorp.de/pkg/libev.html (the CVS version also contains another fix for non-blocking connects on windows which might be worth using). -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Re: libeio and plattform specific aio
On Tue, Jun 03, 2008 at 11:51:47PM +0200, Aleksandar Lazic <[EMAIL PROTECTED]> wrote: > what do you think, will libeio be able to use the 'best' aio for the > plattform, or are there all only part of the solution? It would be, but you need to be careful - linux doesn't have working AIO for normal files - the libc on linux emulates it using threads, and libeio's threads are much lower on resource usage and are also faster. solaris has kernel-aio, but it also uses threads, so it's not clearly better. in fact, you lose scheduling control when using them. something similar is true for kqueue - you might actually get better performance and more control via threads. windows would be kind of closest :) so while libeio certainly could transparently use native aio, this currently isn't on my priority list, as on most platforms it will just use threads anyways or will not work very well. if somebody sent a patch for some platform and maintained it, that would of course be very welcome (especially if some benchmark showed how this affects resource usage and efficiency), but it is likely too early to use native aio on most operating systems. kqueue is probably closest to something like that, followed by windows. note, however, that rarely-used features in kqueue have been broken in 100% of the cases in the past, so don't get your hopes up. (of course, threading is usually broken on bsd's as well, so it might not be much worse). -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Re: High-performance Windows Sockets Applications ;-)
On Wed, Jun 04, 2008 at 08:52:13AM +0200, Aleksandar Lazic <[EMAIL PROTECTED]> wrote: > there is a High-performance Windows Sockets Applications description, > anybody used it? > > http://msdn.microsoft.com/en-us/library/ms738551%28VS.85%29.aspx Thanks, that looks interesting, but after reading through it, here are the vastly suprising tips we get from microsoft to achieve high performance :) - don't connect more often than neecssary - disable nagle (great suggestion in general...) - transfer less data if possible - compress your data if possible and as for future improvements: - use only one conenction and keep it open - combine multiple replies into one send - use threads to overlap (blocking) network I/O with computations Now, except for the very last suggestion, that's hardly any news to anybody. And if the last suggestion is meant seriously, then, well, forget about libev, on windows, you do not use non-blocking I/O at all, you don't use events, you use THREADS. and that might well be true... to get perfromance out of windows, use threads. preferably thread pools. and thats the best windows can do (windows threads are quite slow and high-overhead, but completion ports effetcively force their use). now, for windows specifically, the problems are elsewhere - for sockets, you can actually get "async. notifications" (they are synchronous of course), but you need a window handle. from what i have gathered so far, requiring a window handle in a pure server application is a bad idea, and forcing one into the libev API seems to be a bad idea as well - I personally prefer an API-compatible library that "just" works on windows as well over ahigh-performance library that uses a different API - I can then use the platform-native approach. that kind of leaves threads - microsoft also recomemnds to spawn a thread for every 63 (sometimes 64) handles you have and then wait for them (64 is the maximum number of objects a thread cna wait for at any one time). this might be much more efficient than the current select backend, and afaik doesn't need a window handle. and it might even work for non-select handles (for example, you cna wait for the console in that way, but there is little else you can wait on). another option would be overlapping I/O - apparently some windows versions support waiting for readyness notifications that way, but then only on some handles. my suspicion is that when you force any windows mechanism into the posix model, you will get lots of headaches, and most of the performance gain will be eaten up by the workarounds required. but then, if somebody wrote code and it worked and was performant I would be the last one to complain. The issue is simply that the libev/posix model doesn't map on windows, and will never ahcieve bets possible performance. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Re: High-performance Windows Sockets Applications ;-)
On Mit 04.06.2008 10:07, Marc Lehmann wrote: On Wed, Jun 04, 2008 at 08:52:13AM +0200, Aleksandar Lazic <[EMAIL PROTECTED]> wrote: there is a High-performance Windows Sockets Applications description, anybody used it? http://msdn.microsoft.com/en-us/library/ms738551%28VS.85%29.aspx Thanks, that looks interesting, but after reading through it, here are the vastly suprising tips we get from microsoft to achieve high performance :) [snipp a lot of input] Thanks for your opinion, I'am not a windows developer but it should be noticed that M$ have a statement about High-performance Sockets. BR Aleks PS: sorry for the thread catchup :-( ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Re: libeio and plattform specific aio
On Mit 04.06.2008 09:52, Marc Lehmann wrote: On Tue, Jun 03, 2008 at 11:51:47PM +0200, Aleksandar Lazic <[EMAIL PROTECTED]> wrote: what do you think, will libeio be able to use the 'best' aio for the plattform, or are there all only part of the solution? It would be, but you need to be careful - linux doesn't have working AIO for normal files - the libc on linux emulates it using threads, and libeio's threads are much lower on resource usage and are also faster. I haven't know this, thanks. [snipp] so while libeio certainly could transparently use native aio, this currently isn't on my priority list, as on most platforms it will just use threads anyways or will not work very well. if somebody sent a patch for some platform and maintained it, that would of course be very welcome (especially if some benchmark showed how this affects resource usage and efficiency), but it is likely too early to use native aio on most operating systems. So, you mean threads are the plattform independed solution. There are some thread implementations out there do you mean the 'build in' or bundled with libeio, maybe pthreads or something similar? Aleks ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Windows Compilation problem
Hi, I'm having some trouble getting libev to compile on windows. I am embedding it as recommended. Here are the errors I receive initially: cl /nologo /TP /Ilib /c src\ev_cpp.cpp /Fobuild\ev_cpp.obj ev_cpp.cpp c:\mud\lib\ev\ev_win32.c(85) : error C3861: '_open_osfhandle': identifier not fo und c:\mud\lib\ev\ev_win32.c(86) : error C3861: '_open_osfhandle': identifier not fo und lib\ev/ev.c(708) : error C3861: '_get_osfhandle': identifier not found lib\ev/ev.c(759) : error C3861: '_get_osfhandle': identifier not found lib\ev/ev.c(773) : warning C4806: '==' : unsafe operation: no value of type 'boo l' promoted to type 'int' can equal the given constant lib\ev/ev.c(981) : error C3861: '_get_osfhandle': identifier not found lib\ev/ev.c(1033) : error C3861: 'write': identifier not found lib\ev/ev.c(1052) : error C3861: 'read': identifier not found c:\mud\lib\ev\ev_select.c(137) : error C2065: 'NFDBITS' : undeclared identifier lib\ev/ev.c(1365) : error C3861: 'close': identifier not found lib\ev/ev.c(1366) : error C3861: 'close': identifier not found lib\ev/ev.c(1376) : error C3861: 'close': identifier not found lib\ev/ev.c(1461) : error C3861: 'close': identifier not found lib\ev/ev.c(1462) : error C3861: 'close': identifier not found I poked around and discovered that the "io.h" header has the unreferenced functions. I added that in my ev_cpp.c file above the ev.c include. This reduced the errors to the following: cl /nologo /TP /Ilib /c src\ev_cpp.cpp /Fobuild\ev_cpp.obj ev_cpp.cpp lib\ev/ev.c(773) : warning C4806: '==' : unsafe operation: no value of type 'boo l' promoted to type 'int' can equal the given constant lib\ev/ev.c(981) : error C2664: 'ioctlsocket' : cannot convert parameter 3 from 'int *' to 'u_long *' Types pointed to are unrelated; conversion requires reinterpret_cast, C- style cast or function-style cast c:\mud\lib\ev\ev_select.c(137) : error C2065: 'NFDBITS' : undeclared identifier According to the docs, the NFDBITS problem should be fixed by defining EV_SELECT_USE_FD_SET, which seems to be defined automatically for windows. Any pointers on this? I am using libev-3.41. Thanks, Matt ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Re: High-performance Windows Sockets Applications ;-)
On Jun 4, 2008, at 4:07 AM, Marc Lehmann wrote: - use threads to overlap (blocking) network I/O with computations Now, except for the very last suggestion, that's hardly any news to anybody. And if the last suggestion is meant seriously, then, well, forget about libev, on windows, you do not use non-blocking I/O at all, you don't use events, you use THREADS. and that might well be true... to get perfromance out of windows, use threads. preferably thread pools. I think they mean that you want to use separate threads for computation and data communication With the data communication threads, IIRR, being the same # as CPUs (or something like that)... I think I recall finding that IOCP either uses a thread-pool or interrupts currently executing code for its callbacks... so you'd have IOCP perform data management and on completion do the minimum amt of work (queue up the data to be processed). This doesn't quite fit into libev's workflow directly (booo!), though one 'could' create some interesting layers... Ex: something like libeio's interface could take advantage of some items transparently, or a layer for libev that handles data transport and alerts you to readiness for data available in filled buffers or buffers have room to put data (kinda like the bufferevent abstraction...). ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Re: libeio and plattform specific aio
On Wed, Jun 04, 2008 at 12:10:42PM +0200, Aleksandar Lazic <[EMAIL PROTECTED]> wrote: > >would of course be very welcome (especially if some benchmark showed > >how this affects resource usage and efficiency), but it is likely too > >early to use native aio on most operating systems. > > So, you mean threads are the plattform independed solution. Yes, pthreads to be exact (there is a pthreads implementation for win32 which I use with my BDB module, which in turn uses the same xthreads.h and threadpool implementation as libeio, and works fine). And apart from win32, pthreads are ubiquitious - and if you don'tr have pthreads, you don't have aio anyways. > There are some thread implementations out there do you mean the 'build > in' or bundled with libeio, maybe pthreads or something similar? libeio requires pthreads (the xthread.h implementation once used native threads on windows, but it turned out to be a real pain to get stable condition variables on windows, which is why it now relies on the pthreads-for-win32 library). -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Re: Windows Compilation problem
On Wed, Jun 04, 2008 at 04:19:36AM -0700, Matt Tolton <[EMAIL PROTECTED]> wrote: > I'm having some trouble getting libev to compile on windows. I am > embedding it as recommended. Here are the errors I receive initially: for the NFDBITS issue, you could try out current CVS, which should contain an (untested) workaround for that. the io.h issue is surprising - with my microsoft sdk, I get those definitions when I include windows.h with WIN32_LEAN_AND_MEAN - why it would require an additional io.h include for you is puzzling. > I poked around and discovered that the "io.h" header has the > unreferenced functions. I added that in my ev_cpp.c file above the > ev.c include. This reduced the errors to the following: I think that's the way to go for the moment. > lib\ev/ev.c(773) : warning C4806: '==' : unsafe operation: no value of type > 'boo > l' promoted to type 'int' can equal the given constant Hmm, that might actually be a real bug... (it is of no consquence as long as libev is used correctly, though). > lib\ev/ev.c(981) : error C2664: 'ioctlsocket' : cannot convert parameter 3 > from > 'int *' to 'u_long *' This is also a bug - could you check out CVS, which will have a possible fix for that as well in a few minutes? (It also doesn't affect correctness, when it compiles, as unsigned long is compatible to int on windows for the values used). > According to the docs, the NFDBITS problem should be fixed by defining > EV_SELECT_USE_FD_SET, which seems to be defined automatically for > windows. It should, but the recent workaround for non-blocking connects apparentlsy reintroduced the dependency on NFDBITS. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Re: High-performance Windows Sockets Applications ;-)
On Wed, Jun 04, 2008 at 11:37:50AM -0400, Thomas Harning <[EMAIL PROTECTED]> wrote: > >and that might well be true... to get perfromance out of windows, use > >threads. preferably thread pools. > > > I think they mean that you want to use separate threads for > computation and data communication Exactly: use threads (and blocking operations) and not non-blocking operations. > I think I recall finding that IOCP either uses a thread-pool or > interrupts currently executing code for its callbacks... so you'd have > IOCP perform data management and on completion do the minimum amt of > work (queue up the data to be processed). Note that IOCP are the wrong approach to sockets - they work fine for files, but not for sockets. Window smight force you to use them for sockets to get performance, but thats another issue. > This doesn't quite fit into libev's workflow directly (booo!), though > one 'could' create some interesting layers... It's not libev's workflow, it's the workflow of about every other OS out there. It's standardised by POSIX as well. One should be careful before punishing the majority of well-designed platforms just because of the atrocity of one platform with a business agenda of being incompatible. > Ex: something like libeio's interface could take advantage of some > items transparently, or a layer for libev that handles data transport > and alerts you to readiness for data available in filled buffers or > buffers have room to put data (kinda like the bufferevent > abstraction...). Not quite - libevent's buffer thingy is the right thing here - it could do the right thing on unix and abuse iocps on windows because its the only thing that works well there. I cannot imagine that this interface would be even remotely similar to libeio's api, though, the problem is just too different. I also think it is pointless to implement such an abstraction just because of a single abysmally broken platform (glib does that for example, and the result is quite portable, and quite horrible). If one restricts oneself to sockets, however, the other approaches possible with winsockets should be explored as well - sockets *do* support the posix model, just the mapping is non-trivial (yeah, you need threads, and a window handle) (but the mapping from the broken windows select to posix semantics is quite horrible as well, so it doesn't mean it isn't doable). For libev, using async notification for sockets is definitely more useful and has more future then forcing everybody to use some api only required for windows performance. If somebody made himself the work to implement a specific winsockets backend that used async notification and needs extra configuration in the form of a window handle, that would be nice. And it would mean that applications can take advantage of it without sacrificing speed and good design just because of windows. windows *is* a second class citizen, and performance on it is quite unimportant, as long as you can write programs that work efficiently elsewhere and still work correctly on windows. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Re: Windows Compilation problem
Hi,
Thanks for the assistance.
I grabbed the current version from CVS and that seemed to fix the
compilation issues, so long as I included io.h manually. However, now
when I run the program, it is crashing on line 16 of ev.c, in the
fd_reify function when it calls _get_osfhandle().
Any pointers here? The fd that I'm using is a socket obtained from
the winsock socket() call. Here is the code used to initialize it:
void initServerSocket() {
int res;
int serverfd = socket(AF_INET, SOCK_STREAM, 0);
if (serverfd == -1)
THROW("socket() failed");
if (!SetSocketNonBlocking(serverfd))
THROW("error setting socket nonblocking");
sockaddr server_addr;
init_sockaddr("0.0.0.0", 5111, &server_addr);
res = bind(serverfd, &server_addr, sizeof(server_addr));
if (res == -1)
THROW("bind() failed");
res = listen(serverfd, 10);
if (res == -1)
THROW("listen() failed");
ev::io * server_watcher = new ev::io;
server_watcher->set(0);
server_watcher->set(serverfd, EV_READ);
server_watcher->start();
LOGI << "Server socket initialized.";
}
Thanks,
Matt
___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Re: Windows Compilation problem
> when I run the program, it is crashing on line 16 of ev.c, in the Sorry, typo. I meant line 716. ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Re: libeio, Windows
On Wed, Jun 4, 2008 at 1:36 AM, Marc Lehmann <[EMAIL PROTECTED]> wrote: > On Tue, Jun 03, 2008 at 04:00:40PM -0600, Tony Arcieri <[EMAIL PROTECTED]> > wrote: > I don't think that logic makes much sense - for files, libeio can be used, > for file descriptors supporting non-blicking operation, libev can be used. > > They are complementary, not exclusive to each other. Using one does not > preclude using the other, in fatc, most programs would probbaly take > advantage of both if both are available. > My main use case would be a generic Proactor for file descriptors, hence my concern over a lack of support from sockets. If I'm understanding you correctly, libeio doesn't fit Proactor semantics. That said, I entirely agree that both patterns (I/O completions vs. readiness notifications) need do be accessible alongside each other, as there's use cases for both. -- Tony Arcieri medioh.com ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Re: Windows Compilation problem
Hi, I've debugged this a little bit further, and have more information to give. In poking around on the net and in the libev code, I couldn't determine why it was necessary to use _get_osfhandle(). I tried the following experiment, and changed my ev_cpp.cpp to look like this: #include #define EV_FD_TO_WIN32_HANDLE(fh) fh #include "ev_cpp.h" #include This allowed things to work until a socket was closed and fd_ebadf (and hence fd_valid) was called, causing another call to _get_osfhandle(). FYI, the place that that function is crashing is at the following assertion: _VALIDATE_CLEAR_OSSERR_RETURN((_osfile(fh) & FOPEN), EBADF, -1); Might I ask...what is the _get_osfhandle() call for? It seems like it is ok to just use the handle returned by socket() in my case. Thanks, Matt ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
