NO-JIRA: [c] advanced documentation - threads, buffering Added an "advanced" page and 2 new topics - multi-threading and some explanation about proton buffering.
Project: http://git-wip-us.apache.org/repos/asf/qpid-proton/repo Commit: http://git-wip-us.apache.org/repos/asf/qpid-proton/commit/caefcc71 Tree: http://git-wip-us.apache.org/repos/asf/qpid-proton/tree/caefcc71 Diff: http://git-wip-us.apache.org/repos/asf/qpid-proton/diff/caefcc71 Branch: refs/heads/master Commit: caefcc7158517357c110e045605bdf0f976f0eed Parents: d28fecf Author: Alan Conway <acon...@redhat.com> Authored: Mon Apr 30 13:13:14 2018 -0400 Committer: Alan Conway <acon...@redhat.com> Committed: Tue May 29 09:15:45 2018 -0400 ---------------------------------------------------------------------- c/docs/CMakeLists.txt | 9 ++++++++- c/docs/advanced.md | 5 +++++ c/docs/buffering.md | 48 +++++++++++++++++++++++++++++++++++++++++++++ c/docs/threads.md | 42 +++++++++++++++++++++++++++++++++++++++ c/docs/user.doxygen.in | 3 ++- 5 files changed, 105 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/qpid-proton/blob/caefcc71/c/docs/CMakeLists.txt ---------------------------------------------------------------------- diff --git a/c/docs/CMakeLists.txt b/c/docs/CMakeLists.txt index 7756e48..f1f7fb5 100644 --- a/c/docs/CMakeLists.txt +++ b/c/docs/CMakeLists.txt @@ -18,10 +18,17 @@ # find_package(Doxygen) + if (DOXYGEN_FOUND) configure_file (${CMAKE_CURRENT_SOURCE_DIR}/user.doxygen.in ${CMAKE_CURRENT_BINARY_DIR}/user.doxygen) - add_custom_target (docs-c COMMAND ${DOXYGEN_EXECUTABLE} user.doxygen) + + file(GLOB docsrc ../include/proton/*.h *.md) + add_custom_target (docs-c COMMAND + COMMAND ${CMAKE_COMMAND} -E remove_directory html # get rid of old files + COMMAND ${DOXYGEN_EXECUTABLE} user.doxygen + DEPENDS ${docsrc}) + add_dependencies (docs docs-c) # HTML files are generated to ./html - put those in the install. http://git-wip-us.apache.org/repos/asf/qpid-proton/blob/caefcc71/c/docs/advanced.md ---------------------------------------------------------------------- diff --git a/c/docs/advanced.md b/c/docs/advanced.md new file mode 100644 index 0000000..6293958 --- /dev/null +++ b/c/docs/advanced.md @@ -0,0 +1,5 @@ +## Advanced topics + +* @subpage threads +* @subpage io_page +* @subpage buffering http://git-wip-us.apache.org/repos/asf/qpid-proton/blob/caefcc71/c/docs/buffering.md ---------------------------------------------------------------------- diff --git a/c/docs/buffering.md b/c/docs/buffering.md new file mode 100644 index 0000000..a938205 --- /dev/null +++ b/c/docs/buffering.md @@ -0,0 +1,48 @@ +## Buffering {#buffering} + +AMQP and proton have mechanisms to allow an application to control it's use of memory. + +### Outgoing Data + +The unit of memory control in AMQP is the *session*. +`pn_session_outgoing_bytes()` tells you the total bytes buffered for all +outgoing deliveries on all sending links belonging to that session. + +Each call to `pn_link_send()` adds to the session's outgoing byte total. Each +time proton writes data to the network it reduces the total. To control the +memory used by a session, check `pn_session_outgoing_bytes()` before +calling `pn_link_send()`. If it is too high, stop sending until the link +gets a @ref PN_LINK_FLOW event. + +The AMQP protocol allows peers to exchange session limits so they can predict +their buffering requirements for incoming data ( +`pn_session_set_incoming_capacity()` and +`pn_session_set_outgoing_window()`). Proton will not exceed those limits when +sending to or receiving from the peer. However proton does *not* limit the +amount of data buffered in local memory at the request of the application. It +is up to the application to ensure it does not request more memory than it wants +proton to use. + +#### Priority + +Data written on different links can be interleaved with data from any other link +on the same connection when sending to the peer. Proton does not make any formal +guarantee of fairness, and does not enforce any kind of priority when deciding +how to order frames for sending. Using separate links and/or sessions for +high-priority messages means their frames *can* be sent before already-buffered +low-priority frames, but there is no guarantee that they *will*. + +If you need to ensure swift delivery of higher-priority messages on the same +connection as lower-priority ones, then you should control the amount of data +buffered by proton, and buffer the backlog of low-priority backlog in your own +application. + +There is no point in letting proton buffer more than the outgoing session limits +since that's all it can transmit without peer confirmation. You may want to +buffer less, depending on how you value the trade-off between reducing max +latency for high-priority messages (smaller buffer) and increasing max +throughput under load (bigger buffer). + +### Incoming Data + +To Be Done... <!-- FIXME aconway 2018-05-03: --> http://git-wip-us.apache.org/repos/asf/qpid-proton/blob/caefcc71/c/docs/threads.md ---------------------------------------------------------------------- diff --git a/c/docs/threads.md b/c/docs/threads.md new file mode 100644 index 0000000..8c2113b --- /dev/null +++ b/c/docs/threads.md @@ -0,0 +1,42 @@ +## Multithreading {#threads} + +The @ref pn_proactor allows you to create multi-threaded client and server +applications. You can call `pn_proactor_run` in multiple threads to create a +thread pool for proton to handle events and IO. + +The @ref proactor functions are safe to call from any thread. Most other +functions are **not** safe to call concurrently. The following rules applies: +objects and events associated with *different* connections can safely be used +concurrently. Objects and events associated with *the same* connection cannot. + +The proactor helps you to handle connections safely: `pn_proactor_get()` and +`pn_proactor_wait()` will never return events associated with the same +connection to different threads concurrently. As long as you process only the +objects available from the event returned by the proactor, you are in no danger. + +When you need to process objects belonging to a *different* connection (for +example if you receive a message on one connection and want to send it on +another) you must use locks and thread safe data structures and the +`pn_connection_wake()` function. + +### The wake function + +`pn_connection_wake()` can be called from any thread as long as the +`pn_connection_t` has not been freed. It allows any thread to "wake up" a +connection by generating a @ref PN\_CONNECTION\_WAKE event for that connection. + +For example, supposed a message arrives on connection A and must be sent on +connection B. The thread that receives events for A from the proactor can safely +call `pn_link_recv()` on A's link to decode the @ref message. However it cannot +call `pn_link_send()` on any link belonging to B. + +The decoded @ref message is independent of any connection, so the thread +handling A can store the message in a thread-safe data structure, and call +`pn_connection_wake()` to notify B. Shortly after, some thread will get a @ref +PN\_CONNECTION\_WAKE event for B, and can retrieve the message and send it +on B's link. + +@note Be careful to ensure (using a mutex for example) that connection B +cannot be deleted before connection A calls `pn_connection_wake`. Connections +are automatically deleted by the proactor after the @ref PN\_TRANSPORT\_CLOSED event +git http://git-wip-us.apache.org/repos/asf/qpid-proton/blob/caefcc71/c/docs/user.doxygen.in ---------------------------------------------------------------------- diff --git a/c/docs/user.doxygen.in b/c/docs/user.doxygen.in index 03324ee..ae6894a 100644 --- a/c/docs/user.doxygen.in +++ b/c/docs/user.doxygen.in @@ -53,7 +53,8 @@ WARNINGS = YES # Configuration options related to the input files INPUT = @CMAKE_SOURCE_DIR@/c/include \ - @CMAKE_SOURCE_DIR@/c/examples + @CMAKE_SOURCE_DIR@/c/examples \ + @CMAKE_CURRENT_SOURCE_DIR@ FILE_PATTERNS = *.h *.md *.dox EXCLUDE_PATTERNS = @CMAKE_SOURCE_DIR@/c/examples/*.c \ @CMAKE_SOURCE_DIR@/c/examples/*.h \ --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@qpid.apache.org For additional commands, e-mail: commits-h...@qpid.apache.org
