Re: Request for review/discussion of proposed SSL additions for C++ broker/client on Windows
Steve Huston wrote: For some background info on Windows Schannel (the native SSL implementation on Windows) I wrote a blog overview of it recently - you may want to read that before the review meeting for this patch (http://stevehuston.wordpress.com/2009/12/29/how-to-use-schannel-for-ssl -sockets-on-windows/) I just commented to your blog: from the sound of it, it should be possible to use a single architecture for SChannel and OpenSSL. Its not something I've done, but its on my radar to look at it. James - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
Re: Accepting large contributions (was Re: 10000 msgs limit per session)
Rajith Attapattu wrote: IMO it's not a very accurate picture. Well, we can debate whether the assortment of protocol support levels actually has that affect, but its not really important. Having said that, there would still be quite a bit of work maintaining them and we would need an owner/maintainer to champion a client to ensure that end user needs are properly met. From my pov the whole point would be that 'maintaining them' becomes 'somebody else's problem'. You need to maintain the core swig centrally and allow feedback from sip etc devs to shape the raw API, but beyond maintaining enough code to ckeck that its working (and that definitely does not need to use platform idioms - and is a side effect of testing the core anyway so little extra effory) thats job done. The total effort is still great - sure - but the steering and release management of the core is much more focussed. I definitely agree that a passive state-machine implementation of the protocol core is the ideal. In the case that its 'too hard' then some sort of abstraction similar to the OpenSSL BIO might be adequate, at least up to a point. - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
Re: Accepting large contributions (was Re: 10000 msgs limit per session)
Aidan Skinner wrote: On Thu, Dec 3, 2009 at 10:11 PM, James Mansion ja...@mansionfamily.plus.com wrote: Eh, I don't really want to get rid of the actively maintained clients we already have. In particular, Java and C# derive enormous benefits from purely managed code in terms of portability, JITability etc. Well, *some* benefits. There's not much point having jitability and portability if the result is that resources are spread thinly and none of the non-C[++] implementations works properly. I don't think the functionality and portability story is very good at the moment. Also, I think the 'native VM only' angle is oversold be zealots. Certainly in the investment banking industry its very common to find that quant libraries etc are written in C++ precisely because its portable between Java, .Net and so on, so apps with significant business logic (ie ones that matter) are infrequently pure VM systems anyway. I really don't think there's a problem exposing APIs that look a bit ugly or non-standard in the client languages so long as the full API is present - as an enabler for developers who care about idiomatic presentation. . James - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
Re: Accepting large contributions (was Re: 10000 msgs limit per session)
Aidan Skinner wrote: It's particularly important where we're importing something which duplicates (fully or partially) existing functionality, if only so that the situation is sufficiently clear to people trying to make an informed decision about what best suits their needs. Perhaps the QPID project itself could focus on just C++ for all the protocol handling and use swig (or similar) to create wrappers, so the code volume to support multiple client languages will be much smaller. Then native clients can be completely independant projects. AMQP is supposed to be interoperable on the wire: there shouldn't be a requirement for QPID to provide a lot of native client support, though some scripting (for some definition of that term) is clearly handy for testing etc. James - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
Re: New messaging API - base on AMQP 1.0, or on Java JMS?
Jonathan Robie wrote: In the languages besides Java, there is no standard messaging API to fall behind. I'll say something more on Java in a separate message. If its market share of deployed MOM components that lie strangely unused, its MSMQ. If market share of paid-for MOM matters, isn't MQSeries effectively a de-facto standard? But as I've said before, last time I checked the MQSeries API really only worked in threaded clients. James - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
Re: [jira] Created: (QPID-2017) Files for Async Store interface layer (NOTE, read carefully - license issues)
Kim van der Riet wrote: IANAL, you may be correct. However, this decision was made about 2 years ago out of an abundance of caution regarding non-apache-compatible licenses. The reason LGPL was selected was that it is suited for use with libraries (which this is - msgstore.so) which can be run against the apache-licensed qpidd. OK, so its self-inflicted. Shame its not dual-licensed as Apache/LGPL with a tear-off option. Or BSD. Its just that it seemed to me similar to the APRutil/BDB situation - my recollection was that the BDB interface code was retained in-tree but that it was not compiled by default in case a user infected their own code. It probably is better that it evolves into an interface implementation anyway - hopefully Windows will pick up a Jet implementation. Presumably you could ship a SQLite one as a refeernce. (Some things, COM did get right - we've been rather hasty as an industry to abandon it, just because of the registry - which is actually not a direct requirement anyway) Thanks James - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
Re: [jira] Created: (QPID-2017) Files for Async Store interface layer (NOTE, read carefully - license issues)
Kim van der Riet (JIRA) wrote: They are encumbered with BDB, which is used to save the broker state. For this reason, these files are LGPL Why are they encumbered with BDB? You aren't redistributing BDB are you? And even if you deem that referencing symbols in BDB is somehow 'redistribution' of (part of) BDB, why does that make it LGPL? Surely it just requires: 'Redistributions in any form must be accompanied by information on how to obtain complete source code for the DB software and any accompanying software that uses the DB software.' (And that 's more like GPL, than LGPL) I'm not sure how referencing BDB symbols can be redistributing BDB, but I guess Sleepycat and Oracle have no interest in clarity on that. The symbols are discussed in books and documentation and used from code that could be considered as the source of snippets used. APR has code that can use BDB doesn't it? James - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
Re: [c++]: Sketch of a high-level, protocol independent API
Rafael Schloming wrote: (good stuff) What he said, +1. Unfortunately, as it were, if you want to go there, don't start from here. Alan, Gordon: How are you expecting flow of control to work wrt number of driver threads and flow of control in any callbacks? This is arguably a lot more important than whether the API has a C or C++ flavour. James - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
Re: [c++]: race condition in client when handling session close due to execution.exception
Gordon Sim wrote: However, following the execution.exception is the session.detach as required by the AMQP 0-10 spec. If the session goes out of scope before this is handled, then the whole connection ends up being destroyed due to the NotAttachedException thrown. To me that suggests that some representation of a Channel object *must* live on until the client and server have both agreed that it has been closed, otherwise something might cause the client to try to reuse the channel while there are still messages in transit for it. I think the time to do this is when the use-count is going to zero - depending on how the smart pointer works this will either be just in time and the channel can be marked as a zombie, or just too late and you'll need to insert a stand-in zombie. It does mean that the client would not correctly handle an erroneous attempt by the broker to send frames on a session that has genuinely been detached. That isn't as serious in my view and seems like a worthwhile trade for getting the issue describe in the jira resolved. Surely the problem is also that the broker has (or will shortly) send frames that the protocol requires - that's hardly erroneous? James - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
[jira] Commented: (QPID-1780) Remove boost dependencies from C++ client API
[ https://issues.apache.org/jira/browse/QPID-1780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12693993#action_12693993 ] James Mansion commented on QPID-1780: - I'm not sure the position is any better on RHEL or other *nix systems - the system-supplied Boost package evolves slowly and I'm likely to want a more modern one. All platforms will often have the client built from source in a customer C++ application, I suspect, and packagers will choose a common Boost library for distributions. If there is going to be a lightweight C API with a supported swig/sip/whatever interface, why worry? Shouldn't that be the supported binary client? Remove boost dependencies from C++ client API - Key: QPID-1780 URL: https://issues.apache.org/jira/browse/QPID-1780 Project: Qpid Issue Type: Bug Components: C++ Client Affects Versions: M4 Reporter: Steve Huston Currently there is a dependency on Boost headers being present when building a Qpid client application. I believe that Qpid client programmers should not have to be exposed to Boost just for building Qpid applications. It's a Bad Idea to impose a requirement such as this on all users - Boost is a supporting artifact of the implementation and shouldn't be a requirement for users. More pointedly, it could cause version skew on Windows installed kits. Unlike on RHEL, Windows doesn't offer Boost packages as part of the vendor-supplied OS kit. So, Windows users needing Boost must get their kit somewhere else. This makes it impossible to predict what version of Boost will (or won't) be on a user's system. To take care of this, the Windows installable kit is currently including the needed Boost DLLs that the kit is built against within the install. This removes the need to have Boost installed by the user prior to installing/using Qpid and isolates Qpid's binaries from whatever Boost version is present if there is one. This is a Good Thing except that when trying to build a client app (such as the Qpid examples) Boost ends up getting pulled in from the client API at build time. This is bad because the user now needs to go install Boost. If it's not the same version that the installed kit was built against, there's a chance that object sizes, methods, etc. will differ between what Qpid was built with and what the user installed. The case that brought this up immediately is that qpid/SessionId.h derives from boost::totally_ordered1SessionId. Since qpid/client/Session_Base_0_10.h defines a method which returns a SessionId by value, it needs to pull in SessionId.h which yields a compile error if Boost is not available. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
Re: async server core in qpid code
Gordon Sim wrote: That is the intention. There are as you say a few AMQP specifics still remaining (mainly around the protocol initiation header) that need to be refactored out of this code and hidden behind the codec abstraction. I had a quick try on the train the other day. Looks like there are AMQP protocol specifics pulled into the exception header. Haven't had another try yet. James - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
Re: AMQP 1.0
Robert Greig wrote: JMS may not be perfect but I can just about guarantee that most of the apps written in Java for Qpid using the JMS API will work just fine when we release a Qpid broker than supports the 1-0 protocol. Who's feeling confident about Python?? I'm surprised about that. Do you think jdbc provides similar portability? The API might be pretty common, but there is no way on earth that makes non-trivial code portable between Sybase and PostgreSQL, for example - and nor even does sticking hibernate on top to generate SQL for you help either, for that matter. In that particular case the Big Deal will be the concurrency model more than anything else, followed by subtleties relating to careful use of update-in-place and index coverage (from the perspective of starting with Sybase anyway). I would have thought that differences in the semantics of message ack batching, fan-out, authentication, subscription etc will be subtle but ultimately irreconcilable, and the ability to use a familiar JMS API will be a portability benefit in theory more than in practice. Time will tell. Clearly there is a body that want JMS with an ability to get at AMQP facilities, and a body that want to write an AMQP application that happens to use Java. - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
async server core in qpid code
This code (Poller etc) is currently in common (at least on Windows) - which also has a pile of AMQP-specific code. Would anyone (else) like to have an organisation which would see it usable as a simple general-purpose server core? Not least, this would make it easier to test and measure it, or develop alternative implementations (eg one using Boost asio, or a direct threaded implementation, or one using libevent or libev). James - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
Re: AMQP 1.0
Gordon Sim wrote: For me the first part of the process is a protocol independent API and some thinking around how to support multiple protocols in the broker (allowing messages published over one protocol to be delivered over another etc). You mean an internal support API into which the 1.0 and 0.10 bits etc can be plugged? Another important aspect is ensuring that adding support for 1.0 doesn't break previously supported versions. How important is that, really? There is no version overlap in the Java and C++ brokers currently. You aren't at v1.0 yet, of AMQP or QPid. Is anyone actually in production? Thank you very much for investing your time in the project, it is great to have your interest! You could read to much into it. I'm trying to warn that my interest isn't realistically going to result in coding. I can think about things, comment, and play. But I can't contribute in a 'show me the code' way. Some projects can't accomodate that. I can see why there is a preference for action rather than bike-shed design, but it does limit the sort of contribution that people can make - after all, even a prioritisation of features from potential users is a contribution at some level. James - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
Re: AMQP 1.0
Aidan Skinner wrote: The Java client uses JMS for this to an extent, but we still need a way of exposing AMQP specific things in ways that are as version independent as possible (such as the mandatory flag). I can see that there is value in reaching out to developers who have JMS code or who wish to retain JMS capability, *please* don't hobble Java users who want the full-fat qpid experience by making everything fit with JMS. Case in point: the Java client for postgres is annoyingly limited as soon as you want to receive async notifications and so on. I would much prefer the most efficient and direct mapping to AMQP facilities possible in each language, and an adapter to legacy APIs. Sun's failure to consider interoperability with other language environments continues to haunt it. :-( There is much to learn from ActiveMQ interoperability. There are lots of JMS implementations. My own interest in AMQP is largely driven by it NOT being JMS - so there is a chance that my C#, Python and C++ code can be first class citizens. Clearly other people will have different priorities. I'm interested to know why a (happy) JMS user would be interested in AMQP though: JMS has things which are handy in Java but will be awkward in a heterogeneous system. Perhaps something to consider is trying to make the C/C++ client as lightweight as possible and having a reference SWIG wrapper. In this case I would ideally look for the client to be 'passive' and allow the host to do raw connection establishment and raw IO so it acts as a protocol engine and avoid issues of event loop integration and so on. James - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
Re: [jira] Created: (QPID-1745) C++ Windows; IntegerTypes.h need not define size_t
Steve Huston (JIRA) wrote: The qpid/cpp/src/qpid/sys/windows/IntegerTypes.h file typedefs size_t. This is not necessary for VC9, and the definition conflicts with the proper def for 64-bit builds. Removing the typedef for size_t works for both 32- and 64-bit builds. More to the point, why not just pull the types defined in boost into the qpid namespace on all platforms, and use those? While defining these types in the default namespace is making up for a standardisation defiiciency, its likely to conflict with user code. James - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
AMQP 1.0
I notice that just as I start to try to understand the implementation, the AMQP 1.0 spec is 'all change'. Well, significantly so it seems. The current codebase doesn't seem entirely seperated between the 'how' and the 'what' of the implementation - its not clear how much will carry over into a 1.0 world: even the SSL support is quite different it seems and the whole design seems to 'follow on' from flow control. Which is fair - I don't think you can retrofit it - and I've tried, elsewhere. What's the plan for the 1.0 implementation - is there a design document or implementation plan? Who is doing it? How much of the existing infrastructure will be revisited? James - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
Re: Problem running tests/client_test
Gordon Sim wrote: James Mansion wrote: The WindowsSasl implementation doesn't do anything Windows-ish. Why isn't this basic implementation the default if CyrusSasl is not available? There is code in the ConnectionHandler that is intended to cover the case where a specific SaslFactory is not available that does broadly the same thing. Isn't it cleaner to have it as a basic Sasl handler? This will need revisiting anyway won't it, since in 1.0 there will potentially be a TLS handler inserted in the codec chain ahead of the SaslHandler? I'm trying to get my head around the processing chain at the moment - is there a design doc anywhere? James - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
Re: Problem running tests/client_test
Steve Huston wrote: Hi James, This is a known issue: https://issues.apache.org/jira/browse/QPID-1733 I'm working on a fix for it. Workaround is to start with broker with --auth no. OK, thanks. Perhaps teh JIRA organisation could be improved? - the Java tests have a JIRA section and I looked for one for C++ - and I looked in the C++ client since it seems that the empty response from the client is in error rather than the list from the broker. Is teher a release note I've failed to read with this sort of thing in it? James - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
Build failure for C++ tests with VC2008 and Boost 1.38
Where to Jira?
I see:
ClientSessionTest.cpp
.\ClientSessionTest.cpp(149) : error C2447: '{' : missing function
header (old-style formal list?)
.\ClientSessionTest.cpp(170) : error C2447: '{' : missing function
header (old-style formal list?)
I'm using VS2008 with Boost 1.38, QPID_AUTO_TEST_CASE_EXPECTED_FAILURES
expands to BOOST_AUTO_TEST_CASE_EXPECTED_FAILURES, but that doesn't seem
to expand to something that ends with the start of a function definition.
Rather it expands to:
struct testSuspend0Timeout_id; static struct
testSuspend0Timeout_exp_fail_num_spec : boost::unit_test::ut_detail::
auto_tc_exp_failtestSuspend0Timeout_id {
testSuspend0Timeout_exp_fail_num_spec() : boost::unit_test::ut_detail::
auto_tc_exp_failtestSuspend0Timeout_id ( 1 ) {} }
testSuspend0Timeout_exp_fail_num_spec_inst;
{
ClientSessionFixture fix;
fix.session.suspend();
I don't have older boost versions installed - is this a boost change?
James
-
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscr...@qpid.apache.org
Re: Build error for qpid m4 with Win7 SDK
Steve Huston wrote: Hi James, This situation arose a few other times while I was porting Qpid to Windows. The approach taken was to rename the item in Qpid to avoid Microsoft's macro. I haven't tried anything related to Win7, so haven't hit that one, but if you attach a patch to a JIRA entry, I'll help get it into Qpid. I don't seem to have a working patch program so I can't tell if its adequate - but I can attach a svn diff - is that OK? James - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
[jira] Created: (QPID-1737) qpid C++ will not build with WinSDK 7
qpid C++ will not build with WinSDK 7 - Key: QPID-1737 URL: https://issues.apache.org/jira/browse/QPID-1737 Project: Qpid Issue Type: Bug Components: C++ Broker Affects Versions: M4 Environment: XP 32 bit with latest SDK from Microsoft (SDK 7 beta) Reporter: James Mansion I'm trying to build M4 of AMQP on XP with the Win7 SDK Beta, and I'm seeing that Manageable::STATUS_INVALID_PARAMETER is being expanded to: Manageable::((DWORD )0xC00DL); I think this is because src\qpid/sys/windows/uuid.h has included rpc.h and included a lot of the SDK as a result. STATUS_INVALID_PARAMETER is defined in ntstatus.h and winnt.h. The include structure is rather Byzantine - I see there's a JIRA to tidy things up, though that is focussed on client applications. I suspect that one way or another this will bite again even if sufficient 'undefs' are added in the code now. I know this is Microsoft's bad for namespace pollution, but the practical answer is to change the name of this constant. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
Re: Build error for qpid m4 with Win7 SDK
Steve Huston wrote: Hi James, Svn diff is perfect - thanks. QPID-1737. - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
[jira] Updated: (QPID-1737) qpid C++ will not build with WinSDK 7
[ https://issues.apache.org/jira/browse/QPID-1737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Mansion updated QPID-1737: Attachment: SIP.patch Patch attached - replaces STATUS_INVALID_PARAMETER with STATUS_PARAMETER_INVALID. qpid C++ will not build with WinSDK 7 - Key: QPID-1737 URL: https://issues.apache.org/jira/browse/QPID-1737 Project: Qpid Issue Type: Bug Components: C++ Broker Affects Versions: M4 Environment: XP 32 bit with latest SDK from Microsoft (SDK 7 beta) Reporter: James Mansion Attachments: SIP.patch I'm trying to build M4 of AMQP on XP with the Win7 SDK Beta, and I'm seeing that Manageable::STATUS_INVALID_PARAMETER is being expanded to: Manageable::((DWORD )0xC00DL); I think this is because src\qpid/sys/windows/uuid.h has included rpc.h and included a lot of the SDK as a result. STATUS_INVALID_PARAMETER is defined in ntstatus.h and winnt.h. The include structure is rather Byzantine - I see there's a JIRA to tidy things up, though that is focussed on client applications. I suspect that one way or another this will bite again even if sufficient 'undefs' are added in the code now. I know this is Microsoft's bad for namespace pollution, but the practical answer is to change the name of this constant. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
Internals documentation
Is there any documentation on the CPU flow inside the broker and the major elements of the broker? James - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
Build error for qpid m4 with Win7 SDK
I'm trying to build M4 of AMQP on XP with the Win7 SDK Beta, and I'm seeing that Manageable::STATUS_INVALID_PARAMETER is being expanded to: Manageable::((DWORD )0xC00DL); I think this is because src\qpid/sys/windows/uuid.h has included rpc.h and included a lot of the SDK as a result. STATUS_INVALID_PARAMETER is defined in ntstatus.h and winnt.h. The include structure is rather Byzantine - I see there's a JIRA to tidy things up, though that is focussed on client applications. I suspect that one way or another this will bite again even if sufficient 'undefs' are added in the code now. I know this is Microsoft's bad for namespace pollution, but the practical answer is to change the name of this constant. Any plans? James - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
