Re: [cp-patches] RFC: epoll-based selector
Anthony Green wrote: > On Mon, 2006-09-18 at 18:38 -0700, Casey Marshall wrote: >> Here is a patch to implement a Selector using the epoll_wait call on >> Linux, the preferred event notification facility in the 2.6 series kernels. > > This is great Casey. Have you ever tried running Azureus? It stresses > both the nio selector code as well as our crypto infrastructure -- two > areas where you are expert. > > I realize that GNU Classpath != libgcj, but it's worth mentioning that > azureus still doesn't run nearly as smoothly on gcj as it does on Sun > java. I blame nio and crypto, but I don't have any proof :-) I'm > curious to hear how azureus fares on vanilla GNU Classpath (vs Sun, for > instance). > I haven't run Azureus on classpath or GCJ yet. I'd actually be surprised if the epoll selector even makes a difference in that app (even if Classpath's default selector does some just plain inefficient things), unless you really are interacting with thousands of peers at once.
Re: [cp-patches] RFC: epoll-based selector
On Mon, 2006-09-18 at 18:38 -0700, Casey Marshall wrote: > Here is a patch to implement a Selector using the epoll_wait call on > Linux, the preferred event notification facility in the 2.6 series kernels. This is great Casey. Have you ever tried running Azureus? It stresses both the nio selector code as well as our crypto infrastructure -- two areas where you are expert. I realize that GNU Classpath != libgcj, but it's worth mentioning that azureus still doesn't run nearly as smoothly on gcj as it does on Sun java. I blame nio and crypto, but I don't have any proof :-) I'm curious to hear how azureus fares on vanilla GNU Classpath (vs Sun, for instance). AG
Re: [cp-patches] RFC: epoll-based selector
Casey Marshall wrote:
> Note, too, that you can disable epoll (or kqueue) at run time by setting
> the system property "gnu.java.nio.selectorImpl" to any string other than
> "epoll" (or "kqueue"), and use the old select() based version. Maybe we
> can detect ENOSYS at run time, and disable epoll selectors.
>
OK. I've attached a revised patch that tries epoll selectors if support
was compiled in, and if that fails with ENOSYS, we set a flag that
disables trying it again. In any case, if creating an epoll selector
fails, we fall back on the select- (actually, whatever backs VMSelector)
based version.
Warning, I've neither tried, nor tried to compile, this yet.
### Eclipse Workspace Patch 1.0
#P classpath
Index: include/Makefile.am
===
RCS file: /cvsroot/classpath/classpath/include/Makefile.am,v
retrieving revision 1.70
diff -u -r1.70 Makefile.am
--- include/Makefile.am 17 Sep 2006 07:31:41 - 1.70
+++ include/Makefile.am 19 Sep 2006 21:46:56 -
@@ -128,6 +128,7 @@
$(top_srcdir)/include/gnu_java_net_VMPlainDatagramSocketImpl.h \
$(top_srcdir)/include/gnu_java_net_VMPlainSocketImpl.h \
$(top_srcdir)/include/gnu_java_net_local_LocalSocketImpl.h \
+$(top_srcdir)/include/gnu_java_nio_EpollSelectorImpl.h \
$(top_srcdir)/include/gnu_java_nio_FileChannelImpl.h \
$(top_srcdir)/include/gnu_java_nio_KqueueSelectorImpl.h \
$(top_srcdir)/include/gnu_java_nio_VMChannel.h \
@@ -227,6 +228,9 @@
$(top_srcdir)/include/gnu_java_nio_KqueueSelectorImpl.h:
$(top_srcdir)/gnu/java/nio/KqueueSelectorImpl.java
$(JAVAH) -o $@ gnu.java.nio.KqueueSelectorImpl
+$(top_srcdir)/include/gnu_java_nio_EpollSelectorImpl.h:
$(top_srcdir)/gnu/java/nio/EpollSelectorImpl.java
+ $(JAVAH) -o $@ gnu.java.nio.EpollSelectorImpl
+
$(top_srcdir)/include/gnu_java_nio_charset_iconv_IconvDecoder.h:
$(top_srcdir)/gnu/java/nio/charset/iconv/IconvDecoder.java
$(JAVAH) -o $@ gnu.java.nio.charset.iconv.IconvDecoder
$(top_srcdir)/include/gnu_java_nio_charset_iconv_IconvEncoder.h:
$(top_srcdir)/gnu/java/nio/charset/iconv/IconvEncoder.java
Index: gnu/java/nio/SelectorProviderImpl.java
===
RCS file: /cvsroot/classpath/classpath/gnu/java/nio/SelectorProviderImpl.java,v
retrieving revision 1.9
diff -u -r1.9 SelectorProviderImpl.java
--- gnu/java/nio/SelectorProviderImpl.java 17 Sep 2006 07:31:41 -
1.9
+++ gnu/java/nio/SelectorProviderImpl.java 19 Sep 2006 21:46:56 -
@@ -53,6 +53,7 @@
private static final String SELECTOR_IMPL_KQUEUE = "kqueue";
private static final String SELECTOR_IMPL_EPOLL = "epoll";
private static final String SELECTOR_IMPL = "gnu.java.nio.selectorImpl";
+ private static boolean epoll_failed = false;
public SelectorProviderImpl ()
{
@@ -76,12 +77,31 @@
String selectorImpl = "default";
if (KqueueSelectorImpl.kqueue_supported())
selectorImpl = SELECTOR_IMPL_KQUEUE;
+if (EpollSelectorImpl.epoll_supported() && !epoll_failed)
+ selectorImpl = SELECTOR_IMPL_EPOLL;
selectorImpl = SystemProperties.getProperty(SELECTOR_IMPL, selectorImpl);
if (selectorImpl.equals(SELECTOR_IMPL_KQUEUE))
return new KqueueSelectorImpl(this);
+
if (selectorImpl.equals(SELECTOR_IMPL_EPOLL))
- throw new UnsupportedOperationException("epoll selector not yet
implemented");
+ {
+// We jump through these hoops because even though epoll may look
+// like it's available (sys/epoll.h exists, and you can link against
+// all the epoll functions) it may not be available in the kernel
+// (especially 2.4 kernels), meaning you will get ENOSYS at run time.
+//
+// Madness!
+try
+ {
+return new EpollSelectorImpl(this);
+ }
+catch (InternalError e)
+ {
+// epoll_create throws this on ENOSYS.
+epoll_failed = true;
+ }
+ }
return new SelectorImpl (this);
}
Index: configure.ac
===
RCS file: /cvsroot/classpath/classpath/configure.ac,v
retrieving revision 1.183
diff -u -r1.183 configure.ac
--- configure.ac17 Sep 2006 07:31:41 - 1.183
+++ configure.ac19 Sep 2006 21:46:56 -
@@ -357,7 +357,7 @@
fcntl.h \
sys/mman.h \
magic.h \
-sys/event.h])
+sys/event.h sys/epoll.h])
AC_EGREP_HEADER(uint32_t, stdint.h, AC_DEFINE(HAVE_INT32_DEFINED, 1, [Define
to 1 if you have uint32_t]))
AC_EGREP_HEADER(uint32_t, inttypes.h, AC_DEFINE(HAVE_INT32_DEFINED, 1,
[Define to 1 if you have uint32_t]))
@@ -375,7 +375,7 @@
mmap munmap mincore msync madvise getpagesize sysconf \
lstat readlink \
inet_aton inet_addr inet_pton \
- getif
Re: [cp-patches] RFC: epoll-based selector
> "Casey" == Casey Marshall <[EMAIL PROTECTED]> writes: Casey> Does libgcj use Classpath's NIO? I thought for a lot of these things Casey> libgcj had its own implementation in CNI. libgcj doesn't use it or even build it. There could be an issue when we port this to CNI, but I can't really look into it at the moment. Casey> Maybe we can detect ENOSYS at run time, and disable epoll Casey> selectors. Yeah. For my uses I'm not concerned -- my users won't be running 2.4 kernels -- but this would be friendly for folks stuck there. Tom
Re: [cp-patches] RFC: epoll-based selector
David Daney wrote: > Casey Marshall wrote: >> Here is a patch to implement a Selector using the epoll_wait call on >> Linux, the preferred event notification facility in the 2.6 series >> kernels. >> >> This seems to test OK, so I'll probably just commit this soon. It >> compiles OK on systems that do and don't have epoll, and on Linux the >> Mauve tests for Selector pass. >> > I have not tested the patch, but am afraid that it may fail on linux > systems without epoll support (2.4.x kernels I think lack epoll). I am > running glibc-2.3.3 which has sys/epoll.h but I think that the epoll > syscalls may fail at runtime with ENOSYS. > > Also I am running gcj/libgcj cross compiler and do not know how this > patch will interact with libgcj. > > Well those are my concerns. I guess if things get broken by this we can > add a configure switch to manually disable it. > Thanks for mentioning this. I suppose I should have known that trying to write software for Linux would be a minefield of crap like this :-\ Does libgcj use Classpath's NIO? I thought for a lot of these things libgcj had its own implementation in CNI. Note, too, that you can disable epoll (or kqueue) at run time by setting the system property "gnu.java.nio.selectorImpl" to any string other than "epoll" (or "kqueue"), and use the old select() based version. Maybe we can detect ENOSYS at run time, and disable epoll selectors. Thanks.
Re: [cp-patches] RFC: epoll-based selector
Casey Marshall wrote: Here is a patch to implement a Selector using the epoll_wait call on Linux, the preferred event notification facility in the 2.6 series kernels. This seems to test OK, so I'll probably just commit this soon. It compiles OK on systems that do and don't have epoll, and on Linux the Mauve tests for Selector pass. I have not tested the patch, but am afraid that it may fail on linux systems without epoll support (2.4.x kernels I think lack epoll). I am running glibc-2.3.3 which has sys/epoll.h but I think that the epoll syscalls may fail at runtime with ENOSYS. Also I am running gcj/libgcj cross compiler and do not know how this patch will interact with libgcj. Well those are my concerns. I guess if things get broken by this we can add a configure switch to manually disable it. David Daney.
[cp-patches] RFC: epoll-based selector
Here is a patch to implement a Selector using the epoll_wait call on
Linux, the preferred event notification facility in the 2.6 series kernels.
This seems to test OK, so I'll probably just commit this soon. It
compiles OK on systems that do and don't have epoll, and on Linux the
Mauve tests for Selector pass.
### Eclipse Workspace Patch 1.0
#P classpath
Index: include/Makefile.am
===
RCS file: /cvsroot/classpath/classpath/include/Makefile.am,v
retrieving revision 1.70
diff -u -r1.70 Makefile.am
--- include/Makefile.am 17 Sep 2006 07:31:41 - 1.70
+++ include/Makefile.am 19 Sep 2006 01:34:43 -
@@ -128,6 +128,7 @@
$(top_srcdir)/include/gnu_java_net_VMPlainDatagramSocketImpl.h \
$(top_srcdir)/include/gnu_java_net_VMPlainSocketImpl.h \
$(top_srcdir)/include/gnu_java_net_local_LocalSocketImpl.h \
+$(top_srcdir)/include/gnu_java_nio_EpollSelectorImpl.h \
$(top_srcdir)/include/gnu_java_nio_FileChannelImpl.h \
$(top_srcdir)/include/gnu_java_nio_KqueueSelectorImpl.h \
$(top_srcdir)/include/gnu_java_nio_VMChannel.h \
@@ -227,6 +228,9 @@
$(top_srcdir)/include/gnu_java_nio_KqueueSelectorImpl.h:
$(top_srcdir)/gnu/java/nio/KqueueSelectorImpl.java
$(JAVAH) -o $@ gnu.java.nio.KqueueSelectorImpl
+$(top_srcdir)/include/gnu_java_nio_EpollSelectorImpl.h:
$(top_srcdir)/gnu/java/nio/EpollSelectorImpl.java
+ $(JAVAH) -o $@ gnu.java.nio.EpollSelectorImpl
+
$(top_srcdir)/include/gnu_java_nio_charset_iconv_IconvDecoder.h:
$(top_srcdir)/gnu/java/nio/charset/iconv/IconvDecoder.java
$(JAVAH) -o $@ gnu.java.nio.charset.iconv.IconvDecoder
$(top_srcdir)/include/gnu_java_nio_charset_iconv_IconvEncoder.h:
$(top_srcdir)/gnu/java/nio/charset/iconv/IconvEncoder.java
Index: gnu/java/nio/SelectorProviderImpl.java
===
RCS file: /cvsroot/classpath/classpath/gnu/java/nio/SelectorProviderImpl.java,v
retrieving revision 1.9
diff -u -r1.9 SelectorProviderImpl.java
--- gnu/java/nio/SelectorProviderImpl.java 17 Sep 2006 07:31:41 -
1.9
+++ gnu/java/nio/SelectorProviderImpl.java 19 Sep 2006 01:34:43 -
@@ -76,11 +76,15 @@
String selectorImpl = "default";
if (KqueueSelectorImpl.kqueue_supported())
selectorImpl = SELECTOR_IMPL_KQUEUE;
+if (EpollSelectorImpl.epoll_supported())
+ selectorImpl = SELECTOR_IMPL_EPOLL;
selectorImpl = SystemProperties.getProperty(SELECTOR_IMPL, selectorImpl);
if (selectorImpl.equals(SELECTOR_IMPL_KQUEUE))
return new KqueueSelectorImpl(this);
if (selectorImpl.equals(SELECTOR_IMPL_EPOLL))
+ return new EpollSelectorImpl(this);
+if (selectorImpl.equals(SELECTOR_IMPL_EPOLL))
throw new UnsupportedOperationException("epoll selector not yet
implemented");
return new SelectorImpl (this);
Index: configure.ac
===
RCS file: /cvsroot/classpath/classpath/configure.ac,v
retrieving revision 1.183
diff -u -r1.183 configure.ac
--- configure.ac17 Sep 2006 07:31:41 - 1.183
+++ configure.ac19 Sep 2006 01:34:43 -
@@ -357,7 +357,7 @@
fcntl.h \
sys/mman.h \
magic.h \
-sys/event.h])
+sys/event.h sys/epoll.h])
AC_EGREP_HEADER(uint32_t, stdint.h, AC_DEFINE(HAVE_INT32_DEFINED, 1, [Define
to 1 if you have uint32_t]))
AC_EGREP_HEADER(uint32_t, inttypes.h, AC_DEFINE(HAVE_INT32_DEFINED, 1,
[Define to 1 if you have uint32_t]))
@@ -375,7 +375,7 @@
mmap munmap mincore msync madvise getpagesize sysconf \
lstat readlink \
inet_aton inet_addr inet_pton \
- getifaddrs kqueue kevent])
+ getifaddrs kqueue kevent epoll_create])
LIBMAGIC=
AC_CHECK_LIB(magic, magic_open, LIBMAGIC=-lmagic)
Index: native/jni/java-nio/Makefile.am
===
RCS file: /cvsroot/classpath/classpath/native/jni/java-nio/Makefile.am,v
retrieving revision 1.25
diff -u -r1.25 Makefile.am
--- native/jni/java-nio/Makefile.am 17 Sep 2006 07:31:43 - 1.25
+++ native/jni/java-nio/Makefile.am 19 Sep 2006 01:34:43 -
@@ -7,6 +7,7 @@
gnu_java_nio_charset_iconv_IconvEncoder.c \
java_nio_MappedByteBufferImpl.c \
java_nio_VMDirectByteBuffer.c \
+ gnu_java_nio_EpollSelectorImpl.c \
gnu_java_nio_KqueueSelectorImpl.c
libjavanio_la_LIBADD = $(top_builddir)/native/jni/classpath/jcl.lo \
Index: include/gnu_java_nio_EpollSelectorImpl.h
===
RCS file: include/gnu_java_nio_EpollSelectorImpl.h
diff -N include/gnu_java_nio_EpollSelectorImpl.h
--- /dev/null 1 Jan 1970 00:00:00 -
+++ include/gnu_java
