Re: [libvirt] [RFC PATCH] set tap mtu
On Mon, Nov 17, 2008 at 06:39:45PM -0800, Chris Wright wrote: Below is a simple PoC for setting a large MTU size on a tap device. With this we are able to improve net i/o throughput substantially (~40% improvement on TX and ~130% improvement on RX). This is just RFC because it's hardcoded to an MTU of 9000 for any tap device. Thoughts on the best way to add this kind of support? Well if we want it to be configurable per guest then we'd add it to the XML, in the target device field, alongside the VIF name, eg interface type='network' source network='default'/ target dev='vnet7' mtu='9000'/ mac address=11:22:33:44:55:66/ /interface libvirt can handle this stuff directly for QEMU and LXC drivers but for Xen, VIF setup is done by an /etc/xen/scripts/vif-bridge script. AFAIK, XenD has no config param for MTU, but there's no reason we can't write a patch for XenD to accept an MTU param and pass that through to vif-bridge. If large MTUs are useful for Xen's netback of course... If we don't need this configurable, is there any downside to setting a MTU of 9000 for all TAP devices we create. I assume that PTMUD will ensure that the guest only sends packets = 1500 if the physical NIC connected to the bridge doesn't have such a large MTU, or if the guest doesn't do PMTUD, then the bridge code will do fragementation as needed ? Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :| -- Libvir-list mailing list Libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] ANNOUNCE: Ruby bindings for libvirt, version 0.1.0
I am pleased to announce the release of version 0.1.0 of the ruby-libvirt bindings. NEWS: * Add binding for virConnectFindStoragePoolSources (clalance) * Fix dom_migrate (clalance) * Add the MIGRATE_LIVE (enum virDomainMigrateFlags) flag * Slight improvements of the unit tests The main site for the bindings is http://libvirt.org/ruby API docs can be found at http://libvirt.org/ruby/api/index.html Tarballs/src RPM's are at http://libvirt.org/ruby/download Packages for Fedora and EPEL5 are in the works and should show up on a friendly mirror near you soon. David -- Libvir-list mailing list Libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [RFC PATCH] set tap mtu
On Tue, 2008-11-18 at 14:57 +, Daniel P. Berrange wrote: On Mon, Nov 17, 2008 at 06:39:45PM -0800, Chris Wright wrote: Below is a simple PoC for setting a large MTU size on a tap device. With this we are able to improve net i/o throughput substantially (~40% improvement on TX and ~130% improvement on RX). This is just RFC because it's hardcoded to an MTU of 9000 for any tap device. Thoughts on the best way to add this kind of support? Well if we want it to be configurable per guest then we'd add it to the XML, in the target device field, alongside the VIF name, eg interface type='network' source network='default'/ target dev='vnet7' mtu='9000'/ mac address=11:22:33:44:55:66/ /interface Yeah, I was just about to suggest that. libvirt can handle this stuff directly for QEMU and LXC drivers but for Xen, VIF setup is done by an /etc/xen/scripts/vif-bridge script. AFAIK, XenD has no config param for MTU, but there's no reason we can't write a patch for XenD to accept an MTU param and pass that through to vif-bridge. If large MTUs are useful for Xen's netback of course... If we don't need this configurable, is there any downside to setting a MTU of 9000 for all TAP devices we create. I assume that PTMUD will ensure that the guest only sends packets = 1500 if the physical NIC connected to the bridge doesn't have such a large MTU, or if the guest doesn't do PMTUD, then the bridge code will do fragementation as needed ? My take on it is that it needs to be an opt in thing - if path MTU discovery was sufficient to deal with all cases then we'd probably have an mtu 1500 everywhere ... Herbert? Cheers, Mark. -- Libvir-list mailing list Libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PATCH 2/2] Java bindings for domain events
This patch allows the remote driver to work with an asynchronous
EventImpl (it's the only one using an externally-supplied one), assuming
libvirt is compiled with pthread support. (Without pthreads, this code
is harmless in a single-threaded environment.)
Basically it uses a mutex to protect reads from the RPC socket in such a
way that message reads (in their entirety) are done atomically
(otherwise the remoteDomainEventFired() can race the call() code that
reads replies events).
In addition, I update the EventImpl handle to prevent
remoteDomainEventFired() from being called everytime a reply is sent.
(This helps us dispatch events in a timely manner, though it's not
strictly necessary. Without it, any events coming in during a call()
won't be dispatched until the call drops the socket lock (because
remoteDomainEventFired() will be stuck awaiting the lock).
Dave
diff --git a/src/remote_internal.c b/src/remote_internal.c
index 2ca7930..59128f6 100644
--- a/src/remote_internal.c
+++ b/src/remote_internal.c
@@ -116,6 +116,7 @@ struct private_data {
virDomainEventQueuePtr domainEvents;
/* Timer for flushing domainEvents queue */
int eventFlushTimer;
+PTHREAD_MUTEX_T(lock); /* Serializes socket reads w/async EventImpl */
};
#define GET_PRIVATE(conn,retcode) \
@@ -700,6 +701,9 @@ doRemoteOpen (virConnectPtr conn,
} /* switch (transport) */
+/* This must precede the first call() */
+priv-eventFlushTimer = -1;
+
/* Try and authenticate with server */
if (remoteAuthenticate(conn, priv, 1, auth, authtype) == -1)
goto failed;
@@ -744,6 +748,8 @@ doRemoteOpen (virConnectPtr conn,
}
}
+pthread_mutex_init(priv-lock, NULL);
+
if(VIR_ALLOC(priv-callbackList)0) {
error(conn, VIR_ERR_INVALID_ARG, _(Error allocating callbacks list));
goto failed;
@@ -1250,6 +1256,8 @@ doRemoteClose (virConnectPtr conn, struct private_data *priv)
/* Free queued events */
virDomainEventQueueFree(priv-domainEvents);
+pthread_mutex_destroy(priv-lock);
+
return 0;
}
@@ -4536,11 +4544,11 @@ static int really_read (virConnectPtr conn, struct private_data *priv,
* else Bad Things will happen in the XDR code.
*/
static int
-call (virConnectPtr conn, struct private_data *priv,
- int flags /* if we are in virConnectOpen */,
- int proc_nr,
- xdrproc_t args_filter, char *args,
- xdrproc_t ret_filter, char *ret)
+really_call (virConnectPtr conn, struct private_data *priv,
+ int flags /* if we are in virConnectOpen */,
+ int proc_nr,
+ xdrproc_t args_filter, char *args,
+ xdrproc_t ret_filter, char *ret)
{
char buffer[REMOTE_MESSAGE_MAX];
char buffer2[4];
@@ -4596,16 +4604,18 @@ call (virConnectPtr conn, struct private_data *priv,
really_write (conn, priv, flags REMOTE_CALL_IN_OPEN, buffer, len-4) == -1)
return -1;
+pthread_mutex_lock(priv-lock);
+
retry_read:
/* Read and deserialise length word. */
if (really_read (conn, priv, flags REMOTE_CALL_IN_OPEN, buffer2, sizeof buffer2) == -1)
-return -1;
+goto unlock_return_err;
xdrmem_create (xdr, buffer2, sizeof buffer2, XDR_DECODE);
if (!xdr_int (xdr, len)) {
error (flags REMOTE_CALL_IN_OPEN ? NULL : conn,
VIR_ERR_RPC, _(xdr_int (length word, reply)));
-return -1;
+goto unlock_return_err;
}
xdr_destroy (xdr);
@@ -4615,12 +4625,14 @@ retry_read:
if (len 0 || len REMOTE_MESSAGE_MAX) {
error (flags REMOTE_CALL_IN_OPEN ? NULL : conn,
VIR_ERR_RPC, _(packet received from server too large));
-return -1;
+goto unlock_return_err;
}
/* Read reply header and what follows (either a ret or an error). */
if (really_read (conn, priv, flags REMOTE_CALL_IN_OPEN, buffer, len) == -1)
-return -1;
+goto unlock_return_err;
+
+pthread_mutex_unlock(priv-lock);
/* Deserialise reply header. */
xdrmem_create (xdr, buffer, len, XDR_DECODE);
@@ -4729,8 +4741,33 @@ retry_read:
xdr_destroy (xdr);
return -1;
}
+
+ unlock_return_err:
+pthread_mutex_unlock(priv-lock);
+return -1;
+}
+
+static int call (virConnectPtr conn, struct private_data *priv,
+ int flags /* if we are in virConnectOpen */,
+ int proc_nr,
+ xdrproc_t args_filter, char *args,
+ xdrproc_t ret_filter, char *ret)
+{
+int rv;
+if (priv-eventFlushTimer = 0)
+virEventUpdateHandle(priv-sock, 0);
+rv = really_call(conn, priv, flags, proc_nr,
+ args_filter, args,
+ ret_filter, ret);
+if (priv-eventFlushTimer = 0)
+virEventUpdateHandle(priv-sock,
+ VIR_EVENT_HANDLE_READABLE |
+
Re: [libvirt] [PATCH 1/2] Java bindings for domain events
On Tue, Nov 18, 2008 at 11:06:10AM -0500, David Lively wrote: The attached patch (against libvirt-java) implements Java bindings for libvirt domain events. This version provides a libvirt EventImpl running in its own Java Thread, and provides per-Connect synchronization that makes using the bindings thread-safe. (Note the Domain, Network, StoragePool, and StorageVol methods also synchronize on their Connect object, as required by libvirt. I have similar changes for NodeDevice.java that need to be made when that code is checked in.) This version of the patch also implements and uses an enum class (DomainEvent.Type), as suggested by Tóth István. IMPORTANT: THIS PATCH WILL BREAK THINGS UNLESS THE NEXT [PATCH 2/2] IS APPLIED TO libvirt FIRST. Also, libvirt must be compiled WITH_PTHREADS for Java events to work. my main concern is to keep Java bindings working on Windows. Does our Win32 build includes pthreads ? I would love to get some positive feedback on that point (or even better that the java bindings do indeed work on Win32 with that patch). Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ [EMAIL PROTECTED] | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/ -- Libvir-list mailing list Libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 1/2] Java bindings for domain events
On Tue, Nov 18, 2008 at 11:06:10AM -0500, David Lively wrote: The attached patch (against libvirt-java) implements Java bindings for libvirt domain events. This version provides a libvirt EventImpl running in its own Java Thread, and provides per-Connect synchronization that makes using the bindings thread-safe. (Note the Domain, Network, StoragePool, and StorageVol methods also synchronize on their Connect object, as required by libvirt. I have similar changes for NodeDevice.java that need to be made when that code is checked in.) I don't particularly like the event loop code because it is adding a huge pile of non-portable JNI code that won't work on Windows, which lacks pipe() and poll(). Java already provides a portable pure Java API for building a poll() like event loop in form of NIO. http://www.xhaus.com/alan/python/jynio/select.html The addition of the 'synchronized' annotatioons is something we need regardless of the rest of the patch, since our API contract dictates that only a single thread is allowed to use a single virConnectPtr at once. This version of the patch also implements and uses an enum class (DomainEvent.Type), as suggested by Tóth István. IMPORTANT: THIS PATCH WILL BREAK THINGS UNLESS THE NEXT [PATCH 2/2] IS APPLIED TO libvirt FIRST. Also, libvirt must be compiled WITH_PTHREADS for Java events to work. Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :| -- Libvir-list mailing list Libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 2/2] Java bindings for domain events
On Tue, Nov 18, 2008 at 11:18:38AM -0500, David Lively wrote: This patch allows the remote driver to work with an asynchronous EventImpl (it's the only one using an externally-supplied one), assuming libvirt is compiled with pthread support. (Without pthreads, this code is harmless in a single-threaded environment.) I don't like this patch, since it is only making one tiny part of the API thread-safe, for one tiny use case. eg, if someone uses events from the Xen driver in Java with threads, they'll crash burn, since only the remote driver is being protected. IMHO, if we want to allow multi-threaded access to a single virConnectPtr object then we need to do the full job and make the whole thing safe, including adding thread-local error objects, to replace the inherantly non-thread safe global per-virConnect error objects. The other alternative that we've discussed in the past is to add the ability to 'clone' an existing virConnectPtr object. The idea being that a 2nd thread could be given a clone, and safely use that. Internally we'd share synchronize on relevant state information, so we didn't actually need to setup a separate network connection re-auth for each. As it stands, this patch adding semi-thread safe access will make it harder for us to pursue this 2nd option of cloning virConnect, without breaking the Java bindings in future. Basically it uses a mutex to protect reads from the RPC socket in such a way that message reads (in their entirety) are done atomically (otherwise the remoteDomainEventFired() can race the call() code that reads replies events). Why doesn't the Java code simply synchronize on the virConnect object before invoking the FD event callbacks. That will ensure another thread has finished whatever API call it was doing In addition, I update the EventImpl handle to prevent remoteDomainEventFired() from being called everytime a reply is sent. (This helps us dispatch events in a timely manner, though it's not strictly necessary. Without it, any events coming in during a call() won't be dispatched until the call drops the socket lock (because remoteDomainEventFired() will be stuck awaiting the lock). I don't really like the idea of applying any of this patch, since I think it'll cause us unexpected complications when doing proper thread support in the next release of libvirt, and risk us breaking the java bindings. Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :| -- Libvir-list mailing list Libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 2/2] Java bindings for domain events
On Tue, 2008-11-18 at 17:05 +, Daniel P. Berrange wrote: On Tue, Nov 18, 2008 at 11:18:38AM -0500, David Lively wrote: This patch allows the remote driver to work with an asynchronous EventImpl (it's the only one using an externally-supplied one), assuming libvirt is compiled with pthread support. (Without pthreads, this code is harmless in a single-threaded environment.) I don't like this patch, since it is only making one tiny part of the API thread-safe, for one tiny use case. eg, if someone uses events from the Xen driver in Java with threads, they'll crash burn, since only the remote driver is being protected. Currently ONLY the remote driver (and yes, soon - the xen driver, which would also need thread-safety changes) use an EventImpl supplied externally. All others use the libvirtd-provided synchronous impl. Why doesn't the Java code simply synchronize on the virConnect object before invoking the FD event callbacks. That will ensure another thread has finished whatever API call it was doing Note that the Java code uses (Java's builtin) Connect-level lock to avoid concurrent calls using the same virConnect. This even applies to domain event delivery - note that Connect.fireDomainEventCallbacks is (in the Java patch) a synchronized method. A FD event callback isn't associated with a particular virConnect object, and EventImpls aren't virConnect-specific, so I can't lock the virConnect (see below). When we exposed virEventRegisterImpl, we said that externally-supplied event impls must not make callbacks when a virConnect is being used. If the Java EventImpl were following this rule, we wouldn't need this patch. BUT because the EventImpl can't know which virConnect (if any) is involved in a particular callback, the only way to satisfy this rule is to lock ALL Connect objects before making an EventImpl callback. This is (IMO) both way too restrictive, and (I'm a little less sure of this) not restrictive enough. (I suspect we'd find that making two callbacks concurrently can break things, even if no virConnects are in use at the time.) I think we have to allow externally supplied EventImpls to make callbacks without regard to which virConnect, etc. objects are in use, since EventImpls don't know anything about virConnect, etc. This doesn't require all of libvirt to be made thread-safe. So I view this fix (and the one necessary for the xen driver once it starts using an externally-supplied EventImpl) as providing just enough concurrency control to allow an asynchronous EventImpl, while still making the libvirt user (the Java bindings, in my case) responsible for avoiding concurrent virConnect usage. -- Libvir-list mailing list Libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 1/2] Java bindings for domain events
On Tue, 2008-11-18 at 16:51 +, Daniel P. Berrange wrote: On Tue, Nov 18, 2008 at 11:06:10AM -0500, David Lively wrote: The attached patch (against libvirt-java) implements Java bindings for libvirt domain events. This version provides a libvirt EventImpl running in its own Java Thread, and provides per-Connect synchronization that makes using the bindings thread-safe. (Note the Domain, Network, StoragePool, and StorageVol methods also synchronize on their Connect object, as required by libvirt. I have similar changes for NodeDevice.java that need to be made when that code is checked in.) I don't particularly like the event loop code because it is adding a huge pile of non-portable JNI code that won't work on Windows, which lacks pipe() and poll(). Java already provides a portable pure Java API for building a poll() like event loop in form of NIO. http://www.xhaus.com/alan/python/jynio/select.html Yeah, Daniel V and I had briefly considered this, and rejected it on the basis of it's complicated and (more importantly) some negative feedback I hear from our Java folks on the java.nio Select mechanism. But I agree the java.nio Select mechanism should greatly decrease the amount of JNI code in the Java EventImpl. I need to look over the docs again, but I think it's just a matter of implementing a SelectableChannel on top of a fd. (That JNI code will presumably be very different in Win32 and Unix, but it should be a relatively small amount of JNI code in comparison to my current impl.) So I'll look over the java.nio Select documentation and start thinking about a more portable approach ... (and also talk more with our Java folks about their Select gripes). Dave The addition of the 'synchronized' annotatioons is something we need regardless of the rest of the patch, since our API contract dictates that only a single thread is allowed to use a single virConnectPtr at once. This version of the patch also implements and uses an enum class (DomainEvent.Type), as suggested by Tóth István. IMPORTANT: THIS PATCH WILL BREAK THINGS UNLESS THE NEXT [PATCH 2/2] IS APPLIED TO libvirt FIRST. Also, libvirt must be compiled WITH_PTHREADS for Java events to work. Daniel -- Libvir-list mailing list Libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 1/2] Java bindings for domain events
On Tue, Nov 18, 2008 at 01:12:42PM -0500, David Lively wrote: On Tue, 2008-11-18 at 16:51 +, Daniel P. Berrange wrote: On Tue, Nov 18, 2008 at 11:06:10AM -0500, David Lively wrote: The attached patch (against libvirt-java) implements Java bindings for libvirt domain events. This version provides a libvirt EventImpl running in its own Java Thread, and provides per-Connect synchronization that makes using the bindings thread-safe. (Note the Domain, Network, StoragePool, and StorageVol methods also synchronize on their Connect object, as required by libvirt. I have similar changes for NodeDevice.java that need to be made when that code is checked in.) I don't particularly like the event loop code because it is adding a huge pile of non-portable JNI code that won't work on Windows, which lacks pipe() and poll(). Java already provides a portable pure Java API for building a poll() like event loop in form of NIO. http://www.xhaus.com/alan/python/jynio/select.html Yeah, Daniel V and I had briefly considered this, and rejected it on the basis of it's complicated and (more importantly) some negative feedback I hear from our Java folks on the java.nio Select mechanism. But I agree the java.nio Select mechanism should greatly decrease the amount of JNI code in the Java EventImpl. I need to look over the docs again, but I think it's just a matter of implementing a SelectableChannel on top of a fd. (That JNI code will presumably be very different in Win32 and Unix, but it should be a relatively small amount of JNI code in comparison to my current impl.) So I'll look over the java.nio Select documentation and start thinking about a more portable approach ... (and also talk more with our Java folks about their Select gripes). I guess it's better to invest a bit more time and come up with a solution relying as much as possible on Java threading, I/O and synchronization. After all we should capitalize as much as possible on the portability work done in the Java engine, and limit the C part of the bindings to the strict minimum JNI (as much as possible). On one hand we want the bindings to be the easiest possible to use and avoid threading limitation imposed to the client code, on the other hand limit the C part on those issue, of course that means growing the java side of the bindings, but it really should be easier to maintain and port than equivalent C code, even if NIO is not the nicest Java API :-\ Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ [EMAIL PROTECTED] | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/ -- Libvir-list mailing list Libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
