Re: Review Request: paged queue implementation as flow to disk replacement
On 09/24/2012 09:05 PM, Carl Trieloff wrote: I assume this queue will not support queue-browsing for example, is that correct? No, browsing is supported. However large numbers of browsers going through a large queue at different speeds could lead to a lot of thrashing. - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
Re: Review Request: paged queue implementation as flow to disk replacement
> On Sept. 24, 2012, 7:14 p.m., Alan Conway wrote: > > I also like the approach. You're correct that HA could replace it's use of > > eachMessage with a cursor, and it probably should do so anyway. > > > > I think you will need to consider the case where a single message takes up > > more than one page. Not just for very large messages, but for message > > streams with occasional large messages where most traffic is smaller - it > > may be sub-optimal to force the page size across the board just to cope > > with rare large messages. I agree ultimately it would be nice to allow messages of any size. I was considering for an early access implementation to enforce a maximum message size (which would be the largest message that could fit in a single page), and then reject anything larger. However I don't think spanning pages is too difficult to achieve, though it does add some complexity. - Gordon --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7209/#review11853 --- On Sept. 21, 2012, 3:14 p.m., Gordon Sim wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/7209/ > --- > > (Updated Sept. 21, 2012, 3:14 p.m.) > > > Review request for qpid, Kenneth Giusti and Ted Ross. > > > Description > --- > > == The Problem == > > We want to be able to handle a large, growing queue without exhausting > the limited supply of memory. To do this we want to make use of the > filesystem. > > == Overview of Approach == > > My design proposal allows a queue to be configured as enabling > 'paging'. This option cannot be used in combination with LVQ or > priority queue options at present (due to the specific ordering > requirements of those options). > > A queue for which paging is enabled will be backed by a file. This > file will be logically split into fixed size 'pages'. Each page will > hold a contiguous sequence of messages within it. The corresponding > segment of the file for each page may be mapped into memory allowing > representations of the contained messages to be stored to-, and > recovered from-, disk. The recording of message representation to disk > for paging is entirely orthogonal to any persistence of the data for > recovery purposes. This allows the encoded form to be much simpler > since we don't need to consider recovery after broker failure. > > The queue is thus comprised of a sequence of pages. Only a fixed > number of pages need be loaded at any given time. This frees the > broker from having to store all the messages in memory. When a message > from an unloaded page is required, that page can be reloaded. This may > necessitate unloading some other page to stay within the allowed > number of loaded pages. > > New pages can be created as needed, extending the file without > explicit limit (obviously the filesystem has some finite limit). The > sequence of pages that make up the queue need not match the sequence > of segments within the backing file. This means that pages can be > reused when they are emptied of all messages. > > == The Design == > > A specific Messages implementation is used to implement a queue > supporting paging in the manner desctibed above. > > On a posix system it relies on mmap/munmap/msync for the mapping of > the file into memory. This will (eventually) be abstracted behind an > abstraction allowing platforms that don't support those calls to > supply alternative implementations. > > The central structure in the paged queue is a map of Page instances, > keyed by a sequence number. The key represents the sequence of the > first message contained by the page. > > All pages are the same size. Each corresponds to a particular offset > in the file. A Page instance can be in the loaded or unloaded > state. When loaded, the messages it contains are held in a standard > deque from which they can be returned as needed. When loaded, the > segment in the file it is backed by will be mapped into a particular > region in memory. > > To add messages to a page it must be loaded. When a messages is added, > it is pushed onto the dequeue and also encoded into the region of > memory to which the file segment it represents is mapped. > > A page also contains two sequence sets. One tracks all the messages > that are enqueued within the page, the other all the messages which > have been acquired (the latter is a strict subset of the > former). These sequence sets are always in memory. This means each > enqueued message will be tracked in memory and thus the memory will > grow as the queue grows. However the maximum memory required per > message in the unloaded state is two sequence ranges (assuming both > the enqueued set and acquired set a
Re: Java broker proposal: move firewall rules into ACL file (QPID-4334)
Where the Java broker is concerned it wasnt moved in the timeframe of that
mail thread simply because the ACL system in use at the time was limited
and in need of removal more than anything else (which happened some time
ago), so it didnt make sense to combine them at the time and it just hasn't
happened since it did become worthwhile. Now there are new reasons it would
be useful to combine them, to the extent it makes enough sense for us to
combine the functionality now rather than later.
I cant really say I recall any other discussions on this subject except the
other postings on JIRA and/or the user list requesting the functionality it
would offer.
Robbie
On 25 September 2012 01:19, Rajith Attapattu wrote:
> I'm not really against the change proposed by Philip.
> But wanted to highlight the history around this area, in case anybody
> remembers the reasons behind abandoning the previous effort.
> It's always best to make a decision after considering all aspects.
> I hope somebody still remembers why this didn't take place.
>
> Rajith
>
> On Mon, Sep 24, 2012 at 8:05 PM, Robbie Gemmell
> wrote:
> > The previous thread in [1] is of little relevance now as none of the ACL
> > code under discussion there exists anymore. It is easy enough to argue
> that
> > such restrictions would are best served by a real firewall, but there are
> > reasons it still proves useful functionality for the broker to have and
> we
> > have users we want to keep supporting the functionality for, even if we
> are
> > changing the config (which doesnt particularly concern me, we are slowly
> > moving towards an end state with config on the Java broker that will
> change
> > config for almost everything..making this change now will just reduce the
> > amount of future change slightly). Ultimately the functionality already
> > exists, simplistically this is just going to be tweaking where the
> > implementation lives and its config, though in the end it may actually
> add
> > functionality too as a side effect (e.g. user specific network
> restriction,
> > which I dont think is currently supported on the Java side).
> >
> > The docs for the existing xml config for the Java broker effort lives at
> > https://cwiki.apache.org/qpid/firewall-configuration.html and details
> its
> > current hostname wildcarding and CIDR network matching.
> >
> > Robbie
> >
> > On 24 September 2012 21:23, Rajith Attapattu wrote:
> >
> >> The last time this came up for discussion there was some push back on
> the
> >> list.
> >> This was proposed here [1] due to some requests from the users and
> >> there was even a patch for the c++ broker attached here [2]
> >> However this didn't go through due to some discussion that happened on
> the
> >> list.
> >> Unfortunately I can't seem to find a reference to this on the mailing
> >> list archives.
> >>
> >> Does anybody recall the reasons ?
> >>
> >> I vaguely remember that one of the reasons was that this could be
> >> handled more effectively with a firewall than ACL.
> >>
> >> [1]
> >>
> http://apache-qpid-developers.2158895.n2.nabble.com/IP-white-lists-for-brokers-td4127195.html
> >> [2] https://issues.apache.org/jira/browse/QPID-2305
> >>
> >> On Mon, Sep 24, 2012 at 3:33 PM, Chuck Rolke wrote:
> >> > I think the proposal makes sense and I'd like to see it common to all
> >> brokers.
> >> >
> >> > To date the C++ broker ACL code has used only literal text strings for
> >> host names as defined by the connection agent. Resolving network names
> >> and/or subnets adds new code.
> >> >
> >> > Your proposed syntax is basically OK. The C++ broker supports IPv4,
> >> IPv6, and RDMA. Could you specify the "--from-network " property
> more
> >> fully?
> >> >
> >> > Do you think this can make it in the next release?
> >> >
> >> > -Chuck
> >> >
> >> > - Original Message -
> >> >> From: "Phil Harvey"
> >> >> To: dev@qpid.apache.org
> >> >> Sent: Monday, September 24, 2012 11:14:48 AM
> >> >> Subject: Java broker proposal: move firewall rules into ACL file
> >> (QPID-4334)
> >> >>
> >> >> I'm working on https://issues.apache.org/jira/browse/QPID-4334
> >> >> ("[Java
> >> >> broker] move the Firewall functionality into the ACL plugin") and
> >> >> want to
> >> >> gather opinions on the desired behaviour.
> >> >>
> >> >> My main questions are:
> >> >> - Are we happy to make this change to the Java Broker?
> >> >> - If so, what is the nicest ACL syntax for firewall rules?
> >> >>
> >> >> The motivation for this work is to:
> >> >>
> >> >> (1) rationalise our set of plugins, thus making the implementation of
> >> >> QPID-4335 ("[java broker] replace current plugin system with a
> >> >> simplified
> >> >> system") easier;
> >> >> (2) make life simpler for our users.
> >> >>
> >> >> I expect the second point will be more contentious, hence this email.
> >> >>
> >> >> Putting myself in the user's shoes, I believe it makes sense for
> >> >> access
> >> >> control and firewall configuration to be done in one
Re: Java broker proposal: move firewall rules into ACL file (QPID-4334)
I'm not really against the change proposed by Philip.
But wanted to highlight the history around this area, in case anybody
remembers the reasons behind abandoning the previous effort.
It's always best to make a decision after considering all aspects.
I hope somebody still remembers why this didn't take place.
Rajith
On Mon, Sep 24, 2012 at 8:05 PM, Robbie Gemmell
wrote:
> The previous thread in [1] is of little relevance now as none of the ACL
> code under discussion there exists anymore. It is easy enough to argue that
> such restrictions would are best served by a real firewall, but there are
> reasons it still proves useful functionality for the broker to have and we
> have users we want to keep supporting the functionality for, even if we are
> changing the config (which doesnt particularly concern me, we are slowly
> moving towards an end state with config on the Java broker that will change
> config for almost everything..making this change now will just reduce the
> amount of future change slightly). Ultimately the functionality already
> exists, simplistically this is just going to be tweaking where the
> implementation lives and its config, though in the end it may actually add
> functionality too as a side effect (e.g. user specific network restriction,
> which I dont think is currently supported on the Java side).
>
> The docs for the existing xml config for the Java broker effort lives at
> https://cwiki.apache.org/qpid/firewall-configuration.html and details its
> current hostname wildcarding and CIDR network matching.
>
> Robbie
>
> On 24 September 2012 21:23, Rajith Attapattu wrote:
>
>> The last time this came up for discussion there was some push back on the
>> list.
>> This was proposed here [1] due to some requests from the users and
>> there was even a patch for the c++ broker attached here [2]
>> However this didn't go through due to some discussion that happened on the
>> list.
>> Unfortunately I can't seem to find a reference to this on the mailing
>> list archives.
>>
>> Does anybody recall the reasons ?
>>
>> I vaguely remember that one of the reasons was that this could be
>> handled more effectively with a firewall than ACL.
>>
>> [1]
>> http://apache-qpid-developers.2158895.n2.nabble.com/IP-white-lists-for-brokers-td4127195.html
>> [2] https://issues.apache.org/jira/browse/QPID-2305
>>
>> On Mon, Sep 24, 2012 at 3:33 PM, Chuck Rolke wrote:
>> > I think the proposal makes sense and I'd like to see it common to all
>> brokers.
>> >
>> > To date the C++ broker ACL code has used only literal text strings for
>> host names as defined by the connection agent. Resolving network names
>> and/or subnets adds new code.
>> >
>> > Your proposed syntax is basically OK. The C++ broker supports IPv4,
>> IPv6, and RDMA. Could you specify the "--from-network " property more
>> fully?
>> >
>> > Do you think this can make it in the next release?
>> >
>> > -Chuck
>> >
>> > - Original Message -
>> >> From: "Phil Harvey"
>> >> To: dev@qpid.apache.org
>> >> Sent: Monday, September 24, 2012 11:14:48 AM
>> >> Subject: Java broker proposal: move firewall rules into ACL file
>> (QPID-4334)
>> >>
>> >> I'm working on https://issues.apache.org/jira/browse/QPID-4334
>> >> ("[Java
>> >> broker] move the Firewall functionality into the ACL plugin") and
>> >> want to
>> >> gather opinions on the desired behaviour.
>> >>
>> >> My main questions are:
>> >> - Are we happy to make this change to the Java Broker?
>> >> - If so, what is the nicest ACL syntax for firewall rules?
>> >>
>> >> The motivation for this work is to:
>> >>
>> >> (1) rationalise our set of plugins, thus making the implementation of
>> >> QPID-4335 ("[java broker] replace current plugin system with a
>> >> simplified
>> >> system") easier;
>> >> (2) make life simpler for our users.
>> >>
>> >> I expect the second point will be more contentious, hence this email.
>> >>
>> >> Putting myself in the user's shoes, I believe it makes sense for
>> >> access
>> >> control and firewall configuration to be done in one place, using
>> >> rules
>> >> such as:
>> >>
>> >> ACL ALLOW all ACCESS VIRTUALHOST FROM-NETWORK="123.456.789/24"
>> >> ACL DENY-LOG all ACCESS VIRTUALHOST
>> >> FROM-HOSTNAME=".*\.uat.mycompany\.com"
>> >>
>> >> I therefore propose to enhance the "ACCESS VIRTUALHOST" ACL rule to
>> >> support
>> >> the same network and hostname predicates that are currently supported
>> >> by
>> >> the firewall Java broker plugin. This will make the firewall plugin
>> >> redundant, so it will be deleted.
>> >>
>> >> The objections I'm anticipating are:
>> >>
>> >> - This will break require users to modify their config when they
>> >> upgrade.
>> >> I think this minor inconvenience is outweighed by the motivations
>> >> stated
>> >> above.
>> >>
>> >> - This will cause the Java and C++ ACL syntax to diverge further. I
>> >> don't
>> >> know if this is a showstopper. I understand that this enhancement
>> >> was
>> >> previously discuss
Re: Java broker proposal: move firewall rules into ACL file (QPID-4334)
The previous thread in [1] is of little relevance now as none of the ACL
code under discussion there exists anymore. It is easy enough to argue that
such restrictions would are best served by a real firewall, but there are
reasons it still proves useful functionality for the broker to have and we
have users we want to keep supporting the functionality for, even if we are
changing the config (which doesnt particularly concern me, we are slowly
moving towards an end state with config on the Java broker that will change
config for almost everything..making this change now will just reduce the
amount of future change slightly). Ultimately the functionality already
exists, simplistically this is just going to be tweaking where the
implementation lives and its config, though in the end it may actually add
functionality too as a side effect (e.g. user specific network restriction,
which I dont think is currently supported on the Java side).
The docs for the existing xml config for the Java broker effort lives at
https://cwiki.apache.org/qpid/firewall-configuration.html and details its
current hostname wildcarding and CIDR network matching.
Robbie
On 24 September 2012 21:23, Rajith Attapattu wrote:
> The last time this came up for discussion there was some push back on the
> list.
> This was proposed here [1] due to some requests from the users and
> there was even a patch for the c++ broker attached here [2]
> However this didn't go through due to some discussion that happened on the
> list.
> Unfortunately I can't seem to find a reference to this on the mailing
> list archives.
>
> Does anybody recall the reasons ?
>
> I vaguely remember that one of the reasons was that this could be
> handled more effectively with a firewall than ACL.
>
> [1]
> http://apache-qpid-developers.2158895.n2.nabble.com/IP-white-lists-for-brokers-td4127195.html
> [2] https://issues.apache.org/jira/browse/QPID-2305
>
> On Mon, Sep 24, 2012 at 3:33 PM, Chuck Rolke wrote:
> > I think the proposal makes sense and I'd like to see it common to all
> brokers.
> >
> > To date the C++ broker ACL code has used only literal text strings for
> host names as defined by the connection agent. Resolving network names
> and/or subnets adds new code.
> >
> > Your proposed syntax is basically OK. The C++ broker supports IPv4,
> IPv6, and RDMA. Could you specify the "--from-network " property more
> fully?
> >
> > Do you think this can make it in the next release?
> >
> > -Chuck
> >
> > - Original Message -
> >> From: "Phil Harvey"
> >> To: dev@qpid.apache.org
> >> Sent: Monday, September 24, 2012 11:14:48 AM
> >> Subject: Java broker proposal: move firewall rules into ACL file
> (QPID-4334)
> >>
> >> I'm working on https://issues.apache.org/jira/browse/QPID-4334
> >> ("[Java
> >> broker] move the Firewall functionality into the ACL plugin") and
> >> want to
> >> gather opinions on the desired behaviour.
> >>
> >> My main questions are:
> >> - Are we happy to make this change to the Java Broker?
> >> - If so, what is the nicest ACL syntax for firewall rules?
> >>
> >> The motivation for this work is to:
> >>
> >> (1) rationalise our set of plugins, thus making the implementation of
> >> QPID-4335 ("[java broker] replace current plugin system with a
> >> simplified
> >> system") easier;
> >> (2) make life simpler for our users.
> >>
> >> I expect the second point will be more contentious, hence this email.
> >>
> >> Putting myself in the user's shoes, I believe it makes sense for
> >> access
> >> control and firewall configuration to be done in one place, using
> >> rules
> >> such as:
> >>
> >> ACL ALLOW all ACCESS VIRTUALHOST FROM-NETWORK="123.456.789/24"
> >> ACL DENY-LOG all ACCESS VIRTUALHOST
> >> FROM-HOSTNAME=".*\.uat.mycompany\.com"
> >>
> >> I therefore propose to enhance the "ACCESS VIRTUALHOST" ACL rule to
> >> support
> >> the same network and hostname predicates that are currently supported
> >> by
> >> the firewall Java broker plugin. This will make the firewall plugin
> >> redundant, so it will be deleted.
> >>
> >> The objections I'm anticipating are:
> >>
> >> - This will break require users to modify their config when they
> >> upgrade.
> >> I think this minor inconvenience is outweighed by the motivations
> >> stated
> >> above.
> >>
> >> - This will cause the Java and C++ ACL syntax to diverge further. I
> >> don't
> >> know if this is a showstopper. I understand that this enhancement
> >> was
> >> previously discussed for the C++ broker, and I'd be particularly
> >> interested
> >> to hear current views on this from the C++ folks.
> >>
> >> Let me know what you think.
> >>
> >> Thanks
> >> Phil
> >>
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
> > For additional commands, e-mail: dev-h...@qpid.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@
Re: Java broker proposal: move firewall rules into ACL file (QPID-4334)
The last time this came up for discussion there was some push back on the list.
This was proposed here [1] due to some requests from the users and
there was even a patch for the c++ broker attached here [2]
However this didn't go through due to some discussion that happened on the list.
Unfortunately I can't seem to find a reference to this on the mailing
list archives.
Does anybody recall the reasons ?
I vaguely remember that one of the reasons was that this could be
handled more effectively with a firewall than ACL.
[1]
http://apache-qpid-developers.2158895.n2.nabble.com/IP-white-lists-for-brokers-td4127195.html
[2] https://issues.apache.org/jira/browse/QPID-2305
On Mon, Sep 24, 2012 at 3:33 PM, Chuck Rolke wrote:
> I think the proposal makes sense and I'd like to see it common to all brokers.
>
> To date the C++ broker ACL code has used only literal text strings for host
> names as defined by the connection agent. Resolving network names and/or
> subnets adds new code.
>
> Your proposed syntax is basically OK. The C++ broker supports IPv4, IPv6, and
> RDMA. Could you specify the "--from-network " property more fully?
>
> Do you think this can make it in the next release?
>
> -Chuck
>
> - Original Message -
>> From: "Phil Harvey"
>> To: dev@qpid.apache.org
>> Sent: Monday, September 24, 2012 11:14:48 AM
>> Subject: Java broker proposal: move firewall rules into ACL file (QPID-4334)
>>
>> I'm working on https://issues.apache.org/jira/browse/QPID-4334
>> ("[Java
>> broker] move the Firewall functionality into the ACL plugin") and
>> want to
>> gather opinions on the desired behaviour.
>>
>> My main questions are:
>> - Are we happy to make this change to the Java Broker?
>> - If so, what is the nicest ACL syntax for firewall rules?
>>
>> The motivation for this work is to:
>>
>> (1) rationalise our set of plugins, thus making the implementation of
>> QPID-4335 ("[java broker] replace current plugin system with a
>> simplified
>> system") easier;
>> (2) make life simpler for our users.
>>
>> I expect the second point will be more contentious, hence this email.
>>
>> Putting myself in the user's shoes, I believe it makes sense for
>> access
>> control and firewall configuration to be done in one place, using
>> rules
>> such as:
>>
>> ACL ALLOW all ACCESS VIRTUALHOST FROM-NETWORK="123.456.789/24"
>> ACL DENY-LOG all ACCESS VIRTUALHOST
>> FROM-HOSTNAME=".*\.uat.mycompany\.com"
>>
>> I therefore propose to enhance the "ACCESS VIRTUALHOST" ACL rule to
>> support
>> the same network and hostname predicates that are currently supported
>> by
>> the firewall Java broker plugin. This will make the firewall plugin
>> redundant, so it will be deleted.
>>
>> The objections I'm anticipating are:
>>
>> - This will break require users to modify their config when they
>> upgrade.
>> I think this minor inconvenience is outweighed by the motivations
>> stated
>> above.
>>
>> - This will cause the Java and C++ ACL syntax to diverge further. I
>> don't
>> know if this is a showstopper. I understand that this enhancement
>> was
>> previously discussed for the C++ broker, and I'd be particularly
>> interested
>> to hear current views on this from the C++ folks.
>>
>> Let me know what you think.
>>
>> Thanks
>> Phil
>>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
> For additional commands, e-mail: dev-h...@qpid.apache.org
>
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
Re: Review Request: paged queue implementation as flow to disk replacement
On 09/21/2012 11:14 AM, Gordon Sim wrote: > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/7209/ > --- > > Review request for qpid, Kenneth Giusti and Ted Ross. > > > Description > --- > > == The Problem == > > We want to be able to handle a large, growing queue without exhausting > the limited supply of memory. To do this we want to make use of the > filesystem. > > == Overview of Approach == > > My design proposal allows a queue to be configured as enabling > 'paging'. This option cannot be used in combination with LVQ or > priority queue options at present (due to the specific ordering > requirements of those options). > > A queue for which paging is enabled will be backed by a file. This > file will be logically split into fixed size 'pages'. Each page will > hold a contiguous sequence of messages within it. The corresponding > segment of the file for each page may be mapped into memory allowing > representations of the contained messages to be stored to-, and > recovered from-, disk. The recording of message representation to disk > for paging is entirely orthogonal to any persistence of the data for > recovery purposes. This allows the encoded form to be much simpler > since we don't need to consider recovery after broker failure. > > The queue is thus comprised of a sequence of pages. Only a fixed > number of pages need be loaded at any given time. This frees the > broker from having to store all the messages in memory. When a message > from an unloaded page is required, that page can be reloaded. This may > necessitate unloading some other page to stay within the allowed > number of loaded pages. > > New pages can be created as needed, extending the file without > explicit limit (obviously the filesystem has some finite limit). The > sequence of pages that make up the queue need not match the sequence > of segments within the backing file. This means that pages can be > reused when they are emptied of all messages. > > == The Design == > > A specific Messages implementation is used to implement a queue > supporting paging in the manner desctibed above. > > On a posix system it relies on mmap/munmap/msync for the mapping of > the file into memory. This will (eventually) be abstracted behind an > abstraction allowing platforms that don't support those calls to > supply alternative implementations. > > The central structure in the paged queue is a map of Page instances, > keyed by a sequence number. The key represents the sequence of the > first message contained by the page. > > All pages are the same size. Each corresponds to a particular offset > in the file. A Page instance can be in the loaded or unloaded > state. When loaded, the messages it contains are held in a standard > deque from which they can be returned as needed. When loaded, the > segment in the file it is backed by will be mapped into a particular > region in memory. > > To add messages to a page it must be loaded. When a messages is added, > it is pushed onto the dequeue and also encoded into the region of > memory to which the file segment it represents is mapped. > > A page also contains two sequence sets. One tracks all the messages > that are enqueued within the page, the other all the messages which > have been acquired (the latter is a strict subset of the > former). These sequence sets are always in memory. This means each > enqueued message will be tracked in memory and thus the memory will > grow as the queue grows. However the maximum memory required per > message in the unloaded state is two sequence ranges (assuming both > the enqueued set and acquired set are sparse and the message is > recorded in both). In general it is anticipated the memory used will > be even less than this. Of course additionally there is the memory > overhead of the map of pages which will grow as the queue grows even > though not all these pages are in the loaded state. Of course the > expectation is that the saving in memory by having most of the pages > in a large queue in the unloaded state, in which they do not hold the > actual messages, but merely the two sequence sets mentioned above, is > significant. > > Having the acquired state held in sequence sets avoids having to > update the file every time a messages state changes. The state of the > message instances can be set based on the sequence sets when the page > is loaded. The sequence sets are also currently updated based on the > message states when the page is unloaded (this is because at present > it is the MessageDistributor that sets the state to acquired, and that > is not done via the Messages instance - that maybe worth changing). > > When a subscriber moves through a queue (Messages::next()) the > QueueCursor tracks its poisition. In a paged queue, we can find page > the next message is in by consulting the map of
[jira] [Resolved] (QPID-4222) qpid-cluster -C ignores credentials provided
[
https://issues.apache.org/jira/browse/QPID-4222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
michael j. goulish resolved QPID-4222.
--
Resolution: Fixed
> qpid-cluster -C ignores credentials provided
>
>
> Key: QPID-4222
> URL: https://issues.apache.org/jira/browse/QPID-4222
> Project: Qpid
> Issue Type: Bug
> Components: Python Tools
>Affects Versions: 0.14
>Reporter: Pavel Moravec
>Priority: Minor
> Labels: patch
> Attachments: 0001-qpid-cluster-C-ignores-credentials.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> Description of problem:
> qpid-cluster -C shall connect to all brokers of the cluster to gather all
> connections to the brokers. However it ignores credentials from broker URL.
> So any connection attempt to a broker with authentication fails.
> Version-Release number of selected component (if applicable):
> any (incl. 0.12)
> How reproducible:
> 100%
> Steps to Reproduce:
> 1. Start clustered broker with authentication enabled (sasl config without
> anonymous mechanism)
> 2. SASL database to have default credentials stored (guest/guest for realm
> QPID)
> 3. Run:
> qpid-cluster -C guest/guest@${HOSTNAME}:5672
>
> Actual results:
> Traceback (most recent call last):
> File "/usr/bin/qpid-cluster", line 316, in ?
> sys.exit(main())
> File "/usr/bin/qpid-cluster", line 307, in main
> raise Exception("Failed: %s - %s" % (e.__class__.__name__, e))
> Exception: Failed: ConnectionFailed - (None, 'No acceptable SASL
> authentication mechanism available')
> Expected results:
> list of connections of the cluster to be printed out
> Additional info:
> Patch to be provided
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Resolved] (QPID-4221) qpid-cluster -c does not work
[ https://issues.apache.org/jira/browse/QPID-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] michael j. goulish resolved QPID-4221. -- Resolution: Fixed > qpid-cluster -c does not work > - > > Key: QPID-4221 > URL: https://issues.apache.org/jira/browse/QPID-4221 > Project: Qpid > Issue Type: Bug > Components: Python Tools >Affects Versions: 0.14 >Reporter: Pavel Moravec >Priority: Minor > Labels: patch > Attachments: 0001-qpid-cluster-c-doesnt-work.patch, > qpid-cluster-ipv6.patch > > Original Estimate: 1h > Remaining Estimate: 1h > > Description of problem: > qpid-cluster -c shall view client connections to specified member, but > it does not work at all. > Version-Release number of selected component (if applicable): > any > How reproducible: > 100% > Steps to Reproduce: > 1. Setup a qpid cluster > 2. Run qpid-cluster to discover ID of a broker. Output > is something like: > $ qpid-cluster localhost:5672 > Cluster Name: test > Cluster Status: ACTIVE > Cluster Size: 2 >Members: ID=1.2.3.4:28840 URL=amqp:tcp:1.2.3.4:5672 > : ID=1.2.3.5:21260 URL=amqp:tcp:1.2.3.5:5672 > $ > 3. Query for connections on one node using broker ID from previous output, > i.e.: > $ qpid-cluster -c 1.2.3.4:28840 localhost:5672 > > Actual results: > Output identical to the one in point 2. is written: > Cluster Name: test > Cluster Status: ACTIVE > Cluster Size: 2 >Members: ID=1.2.3.4:28840 URL=amqp:tcp:1.2.3.4:5672 > : ID=1.2.3.5:21260 URL=amqp:tcp:1.2.3.5:5672 > Expected results: > List of connections, i.e. something like: > Clients on Member: ID=1.2.3.5:21260: > dhcp-1-223:5672-1.2.3.4 > dhcp-1-223:5672-1.2.3.5 > localhost.localdomain:5672-127.0.0.1 > Additional info: > Patch attached, trivial omitting of setting up variables at options parsing -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Closed] (QPID-4003) qpid-cluster to allow --sasl-mechanism option
[ https://issues.apache.org/jira/browse/QPID-4003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] michael j. goulish closed QPID-4003. Resolution: Fixed Fix Version/s: 0.18 > qpid-cluster to allow --sasl-mechanism option > - > > Key: QPID-4003 > URL: https://issues.apache.org/jira/browse/QPID-4003 > Project: Qpid > Issue Type: Improvement > Components: Python Tools >Affects Versions: 0.14 >Reporter: Pavel Moravec >Priority: Minor > Labels: patch > Fix For: 0.18 > > Attachments: qpid-cluster-mechList.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > Description of problem: > qpid-cluster shall have option --sasl-mechanism to select which SASL > authentication mechanism it wishes to use. Other qpid tools have it. > That makes sense e.g. when more mechanisms are allowed and one wishes to > authenticate qpid-cluster via Kerberos. Currently qpid-cluster picks up the > most secure method (DIGEST-MD5) that can't be changed. > Trivial patch is already proposed. > Version-Release number of selected component (if applicable): > qpid 0.14 > How reproducible: > 100% > Steps to Reproduce: > 1. don't limit auth.mechanisms in /etc/sasl2/qpidd.conf (comment out > mech_list there, if present) > 2. have auth=yes in a clustered broker > 3. Setup Kerberos credentials. > 4. Try to run qpid-cluster authenticating via Kerberos. > > Actual results: > There is no option to pick up a mechanism (in our case, GSSAPI). > Expected results: > Have --sasl-mechanism option available. > Additional info: > Easy patch proposed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
Re: Java broker proposal: move firewall rules into ACL file (QPID-4334)
I think the proposal makes sense and I'd like to see it common to all brokers.
To date the C++ broker ACL code has used only literal text strings for host
names as defined by the connection agent. Resolving network names and/or
subnets adds new code.
Your proposed syntax is basically OK. The C++ broker supports IPv4, IPv6, and
RDMA. Could you specify the "--from-network " property more fully?
Do you think this can make it in the next release?
-Chuck
- Original Message -
> From: "Phil Harvey"
> To: dev@qpid.apache.org
> Sent: Monday, September 24, 2012 11:14:48 AM
> Subject: Java broker proposal: move firewall rules into ACL file (QPID-4334)
>
> I'm working on https://issues.apache.org/jira/browse/QPID-4334
> ("[Java
> broker] move the Firewall functionality into the ACL plugin") and
> want to
> gather opinions on the desired behaviour.
>
> My main questions are:
> - Are we happy to make this change to the Java Broker?
> - If so, what is the nicest ACL syntax for firewall rules?
>
> The motivation for this work is to:
>
> (1) rationalise our set of plugins, thus making the implementation of
> QPID-4335 ("[java broker] replace current plugin system with a
> simplified
> system") easier;
> (2) make life simpler for our users.
>
> I expect the second point will be more contentious, hence this email.
>
> Putting myself in the user's shoes, I believe it makes sense for
> access
> control and firewall configuration to be done in one place, using
> rules
> such as:
>
> ACL ALLOW all ACCESS VIRTUALHOST FROM-NETWORK="123.456.789/24"
> ACL DENY-LOG all ACCESS VIRTUALHOST
> FROM-HOSTNAME=".*\.uat.mycompany\.com"
>
> I therefore propose to enhance the "ACCESS VIRTUALHOST" ACL rule to
> support
> the same network and hostname predicates that are currently supported
> by
> the firewall Java broker plugin. This will make the firewall plugin
> redundant, so it will be deleted.
>
> The objections I'm anticipating are:
>
> - This will break require users to modify their config when they
> upgrade.
> I think this minor inconvenience is outweighed by the motivations
> stated
> above.
>
> - This will cause the Java and C++ ACL syntax to diverge further. I
> don't
> know if this is a showstopper. I understand that this enhancement
> was
> previously discussed for the C++ broker, and I'd be particularly
> interested
> to hear current views on this from the C++ folks.
>
> Let me know what you think.
>
> Thanks
> Phil
>
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
Re: Review Request: paged queue implementation as flow to disk replacement
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7209/#review11853 --- I also like the approach. You're correct that HA could replace it's use of eachMessage with a cursor, and it probably should do so anyway. I think you will need to consider the case where a single message takes up more than one page. Not just for very large messages, but for message streams with occasional large messages where most traffic is smaller - it may be sub-optimal to force the page size across the board just to cope with rare large messages. - Alan Conway On Sept. 21, 2012, 3:14 p.m., Gordon Sim wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/7209/ > --- > > (Updated Sept. 21, 2012, 3:14 p.m.) > > > Review request for qpid, Kenneth Giusti and Ted Ross. > > > Description > --- > > == The Problem == > > We want to be able to handle a large, growing queue without exhausting > the limited supply of memory. To do this we want to make use of the > filesystem. > > == Overview of Approach == > > My design proposal allows a queue to be configured as enabling > 'paging'. This option cannot be used in combination with LVQ or > priority queue options at present (due to the specific ordering > requirements of those options). > > A queue for which paging is enabled will be backed by a file. This > file will be logically split into fixed size 'pages'. Each page will > hold a contiguous sequence of messages within it. The corresponding > segment of the file for each page may be mapped into memory allowing > representations of the contained messages to be stored to-, and > recovered from-, disk. The recording of message representation to disk > for paging is entirely orthogonal to any persistence of the data for > recovery purposes. This allows the encoded form to be much simpler > since we don't need to consider recovery after broker failure. > > The queue is thus comprised of a sequence of pages. Only a fixed > number of pages need be loaded at any given time. This frees the > broker from having to store all the messages in memory. When a message > from an unloaded page is required, that page can be reloaded. This may > necessitate unloading some other page to stay within the allowed > number of loaded pages. > > New pages can be created as needed, extending the file without > explicit limit (obviously the filesystem has some finite limit). The > sequence of pages that make up the queue need not match the sequence > of segments within the backing file. This means that pages can be > reused when they are emptied of all messages. > > == The Design == > > A specific Messages implementation is used to implement a queue > supporting paging in the manner desctibed above. > > On a posix system it relies on mmap/munmap/msync for the mapping of > the file into memory. This will (eventually) be abstracted behind an > abstraction allowing platforms that don't support those calls to > supply alternative implementations. > > The central structure in the paged queue is a map of Page instances, > keyed by a sequence number. The key represents the sequence of the > first message contained by the page. > > All pages are the same size. Each corresponds to a particular offset > in the file. A Page instance can be in the loaded or unloaded > state. When loaded, the messages it contains are held in a standard > deque from which they can be returned as needed. When loaded, the > segment in the file it is backed by will be mapped into a particular > region in memory. > > To add messages to a page it must be loaded. When a messages is added, > it is pushed onto the dequeue and also encoded into the region of > memory to which the file segment it represents is mapped. > > A page also contains two sequence sets. One tracks all the messages > that are enqueued within the page, the other all the messages which > have been acquired (the latter is a strict subset of the > former). These sequence sets are always in memory. This means each > enqueued message will be tracked in memory and thus the memory will > grow as the queue grows. However the maximum memory required per > message in the unloaded state is two sequence ranges (assuming both > the enqueued set and acquired set are sparse and the message is > recorded in both). In general it is anticipated the memory used will > be even less than this. Of course additionally there is the memory > overhead of the map of pages which will grow as the queue grows even > though not all these pages are in the loaded state. Of course the > expectation is that the saving in memory by having most of the pages > in a large queue in the unloaded state, in which they do not hol
Re: Review Request: qpid-4330 windows client hang/fault
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7179/#review11842 --- Ship it! This addresses a root cause of the client crash at exit. Well done. - Chug Rolke On Sept. 19, 2012, 8:32 p.m., Cliff Jansen wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/7179/ > --- > > (Updated Sept. 19, 2012, 8:32 p.m.) > > > Review request for qpid, Andrew Stitcher, Chug Rolke, and Steve Huston. > > > Description > --- > > This patch introduces a primitive > qpid::sys::SystemInfo::threadSafeShutdown(). It always returns true on Linux > and Solaris. On Windows, it returns true for FreeLibrary and false if exit() > has been called, main() returns, or the Qpid libraries have been statically > linked. > > It is used to fix the noted static destructor problems in the main Jira. > > Most of the work is plumbing related to obtain knowledge of how the module is > terminated in time to be useful to the consumer of the threadSafeShutdown > call. > > > This addresses bug qpid-4330. > https://issues.apache.org/jira/browse/qpid-4330 > > > Diffs > - > > > http://svn.apache.org/repos/asf/qpid/trunk/qpid/cpp/include/qpid/sys/SystemInfo.h > 1387463 > http://svn.apache.org/repos/asf/qpid/trunk/qpid/cpp/src/CMakeLists.txt > 1387463 > http://svn.apache.org/repos/asf/qpid/trunk/qpid/cpp/src/Makefile.am 1387463 > > http://svn.apache.org/repos/asf/qpid/trunk/qpid/cpp/src/qpid/client/ConnectionImpl.cpp > 1387463 > > http://svn.apache.org/repos/asf/qpid/trunk/qpid/cpp/src/qpid/client/windows/ClientDllMain.cpp > PRE-CREATION > > http://svn.apache.org/repos/asf/qpid/trunk/qpid/cpp/src/qpid/sys/posix/SystemInfo.cpp > 1387463 > > http://svn.apache.org/repos/asf/qpid/trunk/qpid/cpp/src/qpid/sys/solaris/SystemInfo.cpp > 1387463 > > http://svn.apache.org/repos/asf/qpid/trunk/qpid/cpp/src/qpid/sys/windows/QpidDllMain.h > PRE-CREATION > > http://svn.apache.org/repos/asf/qpid/trunk/qpid/cpp/src/qpid/sys/windows/Socket.cpp > 1387463 > > http://svn.apache.org/repos/asf/qpid/trunk/qpid/cpp/src/qpid/sys/windows/SystemInfo.cpp > 1387463 > > http://svn.apache.org/repos/asf/qpid/trunk/qpid/cpp/src/qpid/sys/windows/Thread.cpp > 1387463 > > Diff: https://reviews.apache.org/r/7179/diff/ > > > Testing > --- > > > Thanks, > > Cliff Jansen > >
[jira] [Comment Edited] (QPID-4291) Renamed the Ruby gemfile to "qpid_messaging"
[ https://issues.apache.org/jira/browse/QPID-4291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13461935#comment-13461935 ] Darryl L. Pierce edited comment on QPID-4291 at 9/25/12 4:42 AM: - Rebased the patches, cleaning up how it's used as a Ruby gem. Also checked in the ChangeLog file and added it to the generated gem. was (Author: mcpierce): Rebased the patches, cleaning up how it's used as a Ruby gem. > Renamed the Ruby gemfile to "qpid_messaging" > > > Key: QPID-4291 > URL: https://issues.apache.org/jira/browse/QPID-4291 > Project: Qpid > Issue Type: Improvement > Components: Ruby Client >Affects Versions: 0.18 >Reporter: Darryl L. Pierce >Assignee: Darryl L. Pierce > Fix For: 0.19 > > Attachments: > 0001-Some-cleanup-on-the-rdoc-generation-and-markups.patch, > 0002-Renamed-the-gemfile-to-qpid_messaging.patch > > > Since the name "qpid" is already claimed on Rubygems.org, we've had to change > the name of our gem to "qpid_messaging". > This patch set includes changes needed in the internals for the Qpid Ruby > bindings to use this new naming. Users will need to change: > require "qpid" > to: > require "qpid_messaging" > to avoid any potential name collisions. Otherwise, no other changes are > required to their code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Updated] (QPID-4291) Renamed the Ruby gemfile to "qpid_messaging"
[ https://issues.apache.org/jira/browse/QPID-4291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce updated QPID-4291: --- Attachment: (was: 0003-Bumped-the-release-to-0.18.patch) > Renamed the Ruby gemfile to "qpid_messaging" > > > Key: QPID-4291 > URL: https://issues.apache.org/jira/browse/QPID-4291 > Project: Qpid > Issue Type: Improvement > Components: Ruby Client >Affects Versions: 0.18 >Reporter: Darryl L. Pierce >Assignee: Darryl L. Pierce > Fix For: 0.19 > > Attachments: > 0001-Some-cleanup-on-the-rdoc-generation-and-markups.patch, > 0002-Renamed-the-gemfile-to-qpid_messaging.patch > > > Since the name "qpid" is already claimed on Rubygems.org, we've had to change > the name of our gem to "qpid_messaging". > This patch set includes changes needed in the internals for the Qpid Ruby > bindings to use this new naming. Users will need to change: > require "qpid" > to: > require "qpid_messaging" > to avoid any potential name collisions. Otherwise, no other changes are > required to their code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (QPID-4291) Renamed the Ruby gemfile to "qpid_messaging"
[ https://issues.apache.org/jira/browse/QPID-4291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13461935#comment-13461935 ] Darryl L. Pierce commented on QPID-4291: Rebased the patches, cleaning up how it's used as a Ruby gem. > Renamed the Ruby gemfile to "qpid_messaging" > > > Key: QPID-4291 > URL: https://issues.apache.org/jira/browse/QPID-4291 > Project: Qpid > Issue Type: Improvement > Components: Ruby Client >Affects Versions: 0.18 >Reporter: Darryl L. Pierce >Assignee: Darryl L. Pierce > Fix For: 0.19 > > Attachments: > 0001-Some-cleanup-on-the-rdoc-generation-and-markups.patch, > 0002-Renamed-the-gemfile-to-qpid_messaging.patch > > > Since the name "qpid" is already claimed on Rubygems.org, we've had to change > the name of our gem to "qpid_messaging". > This patch set includes changes needed in the internals for the Qpid Ruby > bindings to use this new naming. Users will need to change: > require "qpid" > to: > require "qpid_messaging" > to avoid any potential name collisions. Otherwise, no other changes are > required to their code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Updated] (QPID-4291) Renamed the Ruby gemfile to "qpid_messaging"
[ https://issues.apache.org/jira/browse/QPID-4291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce updated QPID-4291: --- Attachment: 0002-Renamed-the-gemfile-to-qpid_messaging.patch 0001-Some-cleanup-on-the-rdoc-generation-and-markups.patch > Renamed the Ruby gemfile to "qpid_messaging" > > > Key: QPID-4291 > URL: https://issues.apache.org/jira/browse/QPID-4291 > Project: Qpid > Issue Type: Improvement > Components: Ruby Client >Affects Versions: 0.18 >Reporter: Darryl L. Pierce >Assignee: Darryl L. Pierce > Fix For: 0.19 > > Attachments: > 0001-Some-cleanup-on-the-rdoc-generation-and-markups.patch, > 0002-Renamed-the-gemfile-to-qpid_messaging.patch > > > Since the name "qpid" is already claimed on Rubygems.org, we've had to change > the name of our gem to "qpid_messaging". > This patch set includes changes needed in the internals for the Qpid Ruby > bindings to use this new naming. Users will need to change: > require "qpid" > to: > require "qpid_messaging" > to avoid any potential name collisions. Otherwise, no other changes are > required to their code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Updated] (QPID-4291) Renamed the Ruby gemfile to "qpid_messaging"
[ https://issues.apache.org/jira/browse/QPID-4291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce updated QPID-4291: --- Attachment: 0003-Bumped-the-release-to-0.18.patch > Renamed the Ruby gemfile to "qpid_messaging" > > > Key: QPID-4291 > URL: https://issues.apache.org/jira/browse/QPID-4291 > Project: Qpid > Issue Type: Improvement > Components: Ruby Client >Affects Versions: 0.18 >Reporter: Darryl L. Pierce >Assignee: Darryl L. Pierce > Fix For: 0.19 > > Attachments: > 0001-Some-cleanup-on-the-rdoc-generation-and-markups.patch, > 0002-Renamed-the-gemfile-to-qpid_messaging.patch > > > Since the name "qpid" is already claimed on Rubygems.org, we've had to change > the name of our gem to "qpid_messaging". > This patch set includes changes needed in the internals for the Qpid Ruby > bindings to use this new naming. Users will need to change: > require "qpid" > to: > require "qpid_messaging" > to avoid any potential name collisions. Otherwise, no other changes are > required to their code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Updated] (QPID-4291) Renamed the Ruby gemfile to "qpid_messaging"
[ https://issues.apache.org/jira/browse/QPID-4291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce updated QPID-4291: --- Attachment: (was: 0003-Bumped-the-release-to-0.18.patch) > Renamed the Ruby gemfile to "qpid_messaging" > > > Key: QPID-4291 > URL: https://issues.apache.org/jira/browse/QPID-4291 > Project: Qpid > Issue Type: Improvement > Components: Ruby Client >Affects Versions: 0.18 >Reporter: Darryl L. Pierce >Assignee: Darryl L. Pierce > Fix For: 0.19 > > > Since the name "qpid" is already claimed on Rubygems.org, we've had to change > the name of our gem to "qpid_messaging". > This patch set includes changes needed in the internals for the Qpid Ruby > bindings to use this new naming. Users will need to change: > require "qpid" > to: > require "qpid_messaging" > to avoid any potential name collisions. Otherwise, no other changes are > required to their code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Updated] (QPID-4291) Renamed the Ruby gemfile to "qpid_messaging"
[ https://issues.apache.org/jira/browse/QPID-4291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce updated QPID-4291: --- Attachment: (was: 0002-Renamed-the-gemfile-to-qpid_messaging.patch) > Renamed the Ruby gemfile to "qpid_messaging" > > > Key: QPID-4291 > URL: https://issues.apache.org/jira/browse/QPID-4291 > Project: Qpid > Issue Type: Improvement > Components: Ruby Client >Affects Versions: 0.18 >Reporter: Darryl L. Pierce >Assignee: Darryl L. Pierce > Fix For: 0.19 > > > Since the name "qpid" is already claimed on Rubygems.org, we've had to change > the name of our gem to "qpid_messaging". > This patch set includes changes needed in the internals for the Qpid Ruby > bindings to use this new naming. Users will need to change: > require "qpid" > to: > require "qpid_messaging" > to avoid any potential name collisions. Otherwise, no other changes are > required to their code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Updated] (QPID-4291) Renamed the Ruby gemfile to "qpid_messaging"
[ https://issues.apache.org/jira/browse/QPID-4291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce updated QPID-4291: --- Attachment: (was: 0001-Some-cleanup-on-the-rdoc-generation-and-markups.patch) > Renamed the Ruby gemfile to "qpid_messaging" > > > Key: QPID-4291 > URL: https://issues.apache.org/jira/browse/QPID-4291 > Project: Qpid > Issue Type: Improvement > Components: Ruby Client >Affects Versions: 0.18 >Reporter: Darryl L. Pierce >Assignee: Darryl L. Pierce > Fix For: 0.19 > > > Since the name "qpid" is already claimed on Rubygems.org, we've had to change > the name of our gem to "qpid_messaging". > This patch set includes changes needed in the internals for the Qpid Ruby > bindings to use this new naming. Users will need to change: > require "qpid" > to: > require "qpid_messaging" > to avoid any potential name collisions. Otherwise, no other changes are > required to their code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Updated] (QPID-4291) Renamed the Ruby gemfile to "qpid_messaging"
[ https://issues.apache.org/jira/browse/QPID-4291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce updated QPID-4291: --- Attachment: 0003-Bumped-the-release-to-0.18.patch 0002-Renamed-the-gemfile-to-qpid_messaging.patch 0001-Some-cleanup-on-the-rdoc-generation-and-markups.patch > Renamed the Ruby gemfile to "qpid_messaging" > > > Key: QPID-4291 > URL: https://issues.apache.org/jira/browse/QPID-4291 > Project: Qpid > Issue Type: Improvement > Components: Ruby Client >Affects Versions: 0.18 >Reporter: Darryl L. Pierce >Assignee: Darryl L. Pierce > Fix For: 0.19 > > Attachments: > 0001-Some-cleanup-on-the-rdoc-generation-and-markups.patch, > 0002-Renamed-the-gemfile-to-qpid_messaging.patch, > 0003-Bumped-the-release-to-0.18.patch > > > Since the name "qpid" is already claimed on Rubygems.org, we've had to change > the name of our gem to "qpid_messaging". > This patch set includes changes needed in the internals for the Qpid Ruby > bindings to use this new naming. Users will need to change: > require "qpid" > to: > require "qpid_messaging" > to avoid any potential name collisions. Otherwise, no other changes are > required to their code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
Re: Review Request: qpid-4330 windows client hang/fault
> On Sept. 21, 2012, 7:22 p.m., Chug Rolke wrote:
> > I ran this patch with extra OutputDebugString calls when the new logic
> > checks threadSafeShutdown.
> >
> > In a typical client program like drain threadSafeShutdown is false every
> > time as expected for IOThreads and for WinSockSetup.
> >
> > On the windows broker only ~WinSockSetup checks the threadSafeShutdown.
> > Should the broker threads check for safe shutdown too?
No. The broker code creates and destroys these resources in the context of
main(), before main() returns or exit().
threadSafeShutdown() only returns meaningful info if called after the C++
runtime has started its housecleaning for the DLL. In practice that means if
called from an atexit() routine (not used in Qpid), or from a static destructor.
Unlike the broker, the client library uses a static destructor ("static
IOThread io...") to "know" when to clean up.
- Cliff
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7179/#review11796
---
On Sept. 19, 2012, 8:32 p.m., Cliff Jansen wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7179/
> ---
>
> (Updated Sept. 19, 2012, 8:32 p.m.)
>
>
> Review request for qpid, Andrew Stitcher, Chug Rolke, and Steve Huston.
>
>
> Description
> ---
>
> This patch introduces a primitive
> qpid::sys::SystemInfo::threadSafeShutdown(). It always returns true on Linux
> and Solaris. On Windows, it returns true for FreeLibrary and false if exit()
> has been called, main() returns, or the Qpid libraries have been statically
> linked.
>
> It is used to fix the noted static destructor problems in the main Jira.
>
> Most of the work is plumbing related to obtain knowledge of how the module is
> terminated in time to be useful to the consumer of the threadSafeShutdown
> call.
>
>
> This addresses bug qpid-4330.
> https://issues.apache.org/jira/browse/qpid-4330
>
>
> Diffs
> -
>
>
> http://svn.apache.org/repos/asf/qpid/trunk/qpid/cpp/include/qpid/sys/SystemInfo.h
> 1387463
> http://svn.apache.org/repos/asf/qpid/trunk/qpid/cpp/src/CMakeLists.txt
> 1387463
> http://svn.apache.org/repos/asf/qpid/trunk/qpid/cpp/src/Makefile.am 1387463
>
> http://svn.apache.org/repos/asf/qpid/trunk/qpid/cpp/src/qpid/client/ConnectionImpl.cpp
> 1387463
>
> http://svn.apache.org/repos/asf/qpid/trunk/qpid/cpp/src/qpid/client/windows/ClientDllMain.cpp
> PRE-CREATION
>
> http://svn.apache.org/repos/asf/qpid/trunk/qpid/cpp/src/qpid/sys/posix/SystemInfo.cpp
> 1387463
>
> http://svn.apache.org/repos/asf/qpid/trunk/qpid/cpp/src/qpid/sys/solaris/SystemInfo.cpp
> 1387463
>
> http://svn.apache.org/repos/asf/qpid/trunk/qpid/cpp/src/qpid/sys/windows/QpidDllMain.h
> PRE-CREATION
>
> http://svn.apache.org/repos/asf/qpid/trunk/qpid/cpp/src/qpid/sys/windows/Socket.cpp
> 1387463
>
> http://svn.apache.org/repos/asf/qpid/trunk/qpid/cpp/src/qpid/sys/windows/SystemInfo.cpp
> 1387463
>
> http://svn.apache.org/repos/asf/qpid/trunk/qpid/cpp/src/qpid/sys/windows/Thread.cpp
> 1387463
>
> Diff: https://reviews.apache.org/r/7179/diff/
>
>
> Testing
> ---
>
>
> Thanks,
>
> Cliff Jansen
>
>
[jira] [Updated] (QPID-4343) cluster initial update stall when a queue has >10k messages with message groups set
[
https://issues.apache.org/jira/browse/QPID-4343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Moravec updated QPID-4343:
Attachment: bz854666.patch
Patch proposal.
Instead of sending one too-huge-to-encode AMQP message from UpdateClient to
update state of MessageGroupManager, more state updates are sent - one per each
message group. As a message group consists of few messages only, this approach
should not hit the original problem any more.
a/src/qpid/cluster/UpdateClient.cpp has to be changed to send potentially more
updates by one StatefulQueueObserver.
a/src/qpid/broker/QueueFlowLimit.h changed is a direct consequence of that
MessageGroupManager::getState and MessageGroupManager::setState in fact does
the same as before but without the "for (GroupMap::const_iterator .." loop done
from UpdateClient.
> cluster initial update stall when a queue has >10k messages with message
> groups set
> ---
>
> Key: QPID-4343
> URL: https://issues.apache.org/jira/browse/QPID-4343
> Project: Qpid
> Issue Type: Bug
> Components: C++ Broker
>Affects Versions: 0.18
>Reporter: Pavel Moravec
> Labels: patch
> Attachments: bz854666.patch
>
>
> Description of problem:
> Having qpid broker in a cluster and using message groups, an attempt to join
> a clustered peer causes cluster stall during initial update process, when
> some queue has >10k messages with message groups set.
> The reason is that updater node sends information about message groups in
> ClusterConnectionQueueObserverStateBody message (exactly one message per one
> queue). If some queue has "too much" messages with msg.groups, such
> ClusterConnectionQueueObserverStateBody message does not fit into one AMQP
> frame and it is silently(!) dropped by the updater.
> Updatee node then waits for the message while updater node (and consequently
> whole cluster) waits for updatee to mark itself as ready.
> Version-Release number of selected component (if applicable):
> 0.14-21, almost surely in 0.18
> How reproducible:
> 100%
> Steps to Reproduce:
> 1. Have 2node cluster with 1 node running
> 2. Produce at least 10k messages with message groups to it:
> qpid-send --group-key "GROUP_KEY" -m 1 -a "groupQ; {create:always,
> node:{type:queue, x-declare:{ arguments:{'qpid.group_header_key':'GROUP_KEY',
> 'qpid.shared_msg_group':1 "
> 3. (re)start 2nd node twice - due to some unknown reason, the first start
> succeeds while the second does not.
> Actual results:
> New joiner stalls the cluster.
> Expected results:
> No broker joining a cluster can stall the cluster.
> Additional info:
> patch proposed
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Created] (QPID-4343) cluster initial update stall when a queue has >10k messages with message groups set
Pavel Moravec created QPID-4343:
---
Summary: cluster initial update stall when a queue has >10k
messages with message groups set
Key: QPID-4343
URL: https://issues.apache.org/jira/browse/QPID-4343
Project: Qpid
Issue Type: Bug
Components: C++ Broker
Affects Versions: 0.18
Reporter: Pavel Moravec
Attachments: bz854666.patch
Description of problem:
Having qpid broker in a cluster and using message groups, an attempt to join a
clustered peer causes cluster stall during initial update process, when some
queue has >10k messages with message groups set.
The reason is that updater node sends information about message groups in
ClusterConnectionQueueObserverStateBody message (exactly one message per one
queue). If some queue has "too much" messages with msg.groups, such
ClusterConnectionQueueObserverStateBody message does not fit into one AMQP
frame and it is silently(!) dropped by the updater.
Updatee node then waits for the message while updater node (and consequently
whole cluster) waits for updatee to mark itself as ready.
Version-Release number of selected component (if applicable):
0.14-21, almost surely in 0.18
How reproducible:
100%
Steps to Reproduce:
1. Have 2node cluster with 1 node running
2. Produce at least 10k messages with message groups to it:
qpid-send --group-key "GROUP_KEY" -m 1 -a "groupQ; {create:always,
node:{type:queue, x-declare:{ arguments:{'qpid.group_header_key':'GROUP_KEY',
'qpid.shared_msg_group':1 "
3. (re)start 2nd node twice - due to some unknown reason, the first start
succeeds while the second does not.
Actual results:
New joiner stalls the cluster.
Expected results:
No broker joining a cluster can stall the cluster.
Additional info:
patch proposed
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
Java broker proposal: move firewall rules into ACL file (QPID-4334)
I'm working on https://issues.apache.org/jira/browse/QPID-4334 ("[Java
broker] move the Firewall functionality into the ACL plugin") and want to
gather opinions on the desired behaviour.
My main questions are:
- Are we happy to make this change to the Java Broker?
- If so, what is the nicest ACL syntax for firewall rules?
The motivation for this work is to:
(1) rationalise our set of plugins, thus making the implementation of
QPID-4335 ("[java broker] replace current plugin system with a simplified
system") easier;
(2) make life simpler for our users.
I expect the second point will be more contentious, hence this email.
Putting myself in the user's shoes, I believe it makes sense for access
control and firewall configuration to be done in one place, using rules
such as:
ACL ALLOW all ACCESS VIRTUALHOST FROM-NETWORK="123.456.789/24"
ACL DENY-LOG all ACCESS VIRTUALHOST FROM-HOSTNAME=".*\.uat.mycompany\.com"
I therefore propose to enhance the "ACCESS VIRTUALHOST" ACL rule to support
the same network and hostname predicates that are currently supported by
the firewall Java broker plugin. This will make the firewall plugin
redundant, so it will be deleted.
The objections I'm anticipating are:
- This will break require users to modify their config when they upgrade.
I think this minor inconvenience is outweighed by the motivations stated
above.
- This will cause the Java and C++ ACL syntax to diverge further. I don't
know if this is a showstopper. I understand that this enhancement was
previously discussed for the C++ broker, and I'd be particularly interested
to hear current views on this from the C++ folks.
Let me know what you think.
Thanks
Phil
[jira] [Updated] (QPID-4291) Renamed the Ruby gemfile to "qpid_messaging"
[ https://issues.apache.org/jira/browse/QPID-4291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce updated QPID-4291: --- Attachment: (was: 0001-Some-cleanup-on-the-rdoc-generation-and-markups.patch) > Renamed the Ruby gemfile to "qpid_messaging" > > > Key: QPID-4291 > URL: https://issues.apache.org/jira/browse/QPID-4291 > Project: Qpid > Issue Type: Improvement > Components: Ruby Client >Affects Versions: 0.18 >Reporter: Darryl L. Pierce >Assignee: Darryl L. Pierce > Fix For: 0.19 > > > Since the name "qpid" is already claimed on Rubygems.org, we've had to change > the name of our gem to "qpid_messaging". > This patch set includes changes needed in the internals for the Qpid Ruby > bindings to use this new naming. Users will need to change: > require "qpid" > to: > require "qpid_messaging" > to avoid any potential name collisions. Otherwise, no other changes are > required to their code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Updated] (QPID-4291) Renamed the Ruby gemfile to "qpid_messaging"
[ https://issues.apache.org/jira/browse/QPID-4291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darryl L. Pierce updated QPID-4291: --- Attachment: (was: 0002-Renamed-the-gemfile-to-qpid_messaging.patch) > Renamed the Ruby gemfile to "qpid_messaging" > > > Key: QPID-4291 > URL: https://issues.apache.org/jira/browse/QPID-4291 > Project: Qpid > Issue Type: Improvement > Components: Ruby Client >Affects Versions: 0.18 >Reporter: Darryl L. Pierce >Assignee: Darryl L. Pierce > Fix For: 0.19 > > > Since the name "qpid" is already claimed on Rubygems.org, we've had to change > the name of our gem to "qpid_messaging". > This patch set includes changes needed in the internals for the Qpid Ruby > bindings to use this new naming. Users will need to change: > require "qpid" > to: > require "qpid_messaging" > to avoid any potential name collisions. Otherwise, no other changes are > required to their code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
Re: Review Request: Implement a ListMessage interface to accept amqp/list messages as java.util.List
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6816/
---
(Updated Sept. 24, 2012, 2:24 p.m.)
Review request for qpid, Gordon Sim, Robbie Gemmell, Weston Price, and Keith
Wall.
Changes
---
Added the missing files. Thx Gordon for pointing it out.
Description
---
We have a need to support amqp encoded list messages that are understood across
all language clients. In JMS this is a bit difficult as there is no direct list
message interface like the map message.
I wanted to introduce a way of getting list message support while using only
JMS interfaces.
The list message support is implemented as a special case of Map message, where
the only entry is a list (it can be nested lists as well).
The application notifies the client of their intention via a message header.
MapMessage m = _session.createMapMessage();
m.setBooleanProperty("encode-as-list", true);
m.setObject("amqp/list", myList);
_producer.send(m);
The resulting message body will be encoded as an AMQP 0-10 list. A C++ or
python client can then directly read it as a list message.
On the receiving side, it will be presented as a MapMessage with a single entry
retrieved using "amqp/list".
List list = (List)m.getObject("amqp/list");
Please feel free to provide feedback, including alternative ways of
implementing this.
All feedback are most welcomed!
This addresses bug QPID-3906.
https://issues.apache.org/jira/browse/QPID-3906
Diffs (updated)
-
http://svn.apache.org/repos/asf/qpid/trunk/qpid/java/client/src/main/java/org/apache/qpid/client/AMQConnection.java
1389396
http://svn.apache.org/repos/asf/qpid/trunk/qpid/java/client/src/main/java/org/apache/qpid/client/AMQSession.java
1389396
http://svn.apache.org/repos/asf/qpid/trunk/qpid/java/client/src/main/java/org/apache/qpid/client/message/AMQPEncodedListMessage.java
PRE-CREATION
http://svn.apache.org/repos/asf/qpid/trunk/qpid/java/client/src/main/java/org/apache/qpid/client/message/AMQPEncodedListMessageFactory.java
PRE-CREATION
http://svn.apache.org/repos/asf/qpid/trunk/qpid/java/client/src/main/java/org/apache/qpid/client/message/JMSStreamMessage.java
1389396
http://svn.apache.org/repos/asf/qpid/trunk/qpid/java/client/src/main/java/org/apache/qpid/client/message/MessageFactoryRegistry.java
1389396
http://svn.apache.org/repos/asf/qpid/trunk/qpid/java/client/src/main/java/org/apache/qpid/jms/ConnectionURL.java
1389396
http://svn.apache.org/repos/asf/qpid/trunk/qpid/java/client/src/main/java/org/apache/qpid/jms/ListMessage.java
PRE-CREATION
http://svn.apache.org/repos/asf/qpid/trunk/qpid/java/client/src/main/java/org/apache/qpid/jms/Session.java
1389396
http://svn.apache.org/repos/asf/qpid/trunk/qpid/java/common/src/main/java/org/apache/qpid/configuration/ClientProperties.java
1389396
Diff: https://reviews.apache.org/r/6816/diff/
Testing
---
AMQPEncodedListMessageTest is added to cover several cases.
Thanks,
rajith attapattu
Re: [jira] [Commented] (QPID-4291) Renamed the Ruby gemfile to "qpid_messaging"
On Wed, Sep 19, 2012 at 09:13:08AM +1100, Robbie Gemmell (JIRA) wrote: > > [ > https://issues.apache.org/jira/browse/QPID-4291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13458237#comment-13458237 > ] > > Robbie Gemmell commented on QPID-4291: > -- > > Bumped fix-for to 0.19 since 0.18 already went out before the JIRA was opened. > > > Renamed the Ruby gemfile to "qpid_messaging" > > > > > > Key: QPID-4291 > > URL: https://issues.apache.org/jira/browse/QPID-4291 > > Project: Qpid > > Issue Type: Improvement > > Components: Ruby Client > >Affects Versions: 0.18 > >Reporter: Darryl L. Pierce > >Assignee: Darryl L. Pierce > > Fix For: 0.19 > > > > Attachments: > > 0001-Some-cleanup-on-the-rdoc-generation-and-markups.patch, > > 0002-Renamed-the-gemfile-to-qpid_messaging.patch > > > > > > Since the name "qpid" is already claimed on Rubygems.org, we've had to > > change the name of our gem to "qpid_messaging". > > This patch set includes changes needed in the internals for the Qpid Ruby > > bindings to use this new naming. Users will need to change: > > require "qpid" > > to: > > require "qpid_messaging" > > to avoid any potential name collisions. Otherwise, no other changes are > > required to their code. Can I get a review and an OK on this change? It only affects the name of the Ruby gem artifact, plus some clean up on Rdoc annotations. No change to code. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpjKZAQ91z6Q.pgp Description: PGP signature
[jira] [Assigned] (QPID-4342) Derby profile keeps timing out on Jenkins CI (slow store initialisation?)
[
https://issues.apache.org/jira/browse/QPID-4342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Keith Wall reassigned QPID-4342:
Assignee: Keith Wall
> Derby profile keeps timing out on Jenkins CI (slow store initialisation?)
> -
>
> Key: QPID-4342
> URL: https://issues.apache.org/jira/browse/QPID-4342
> Project: Qpid
> Issue Type: Bug
> Components: Java Broker
>Reporter: Keith Wall
>Assignee: Keith Wall
> Fix For: 0.19
>
>
> We've seen a number of timeouts on Jenkins when running the Derby profile
> over the last few days. From the look of the logs, it suggests that store
> initialisation is taking an excessive length of time . 8 or 9 seconds in some
> cases.
> https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-Java-Java-DBY-TestMatrix/200/
> https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-Java-Java-DBY-TestMatrix/198/
> https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-Java-Java-DBY-TestMatrix/197/
> The pattern I see in the logs is as follows:
> {noformat}
> main 2012-09-24 04:01:29,885 INFO
> [security.auth.manager.PrincipalDatabaseAuthenticationManager] Initialised
> CRAM-MD5 SASL provider successfully
> main 2012-09-24 04:01:29,885 INFO
> [security.auth.manager.PrincipalDatabaseAuthenticationManager] Additional
> SASL providers successfully registered.
> main 2012-09-24 04:01:29,885 INFO [qpid.message.virtualhost.created] [Broker]
> VHT-1001 : Created : test
> main 2012-09-24 04:01:29,885 DEBUG [qpid.server.store.EventManager] Received
> event BEFORE_INIT
> main 2012-09-24 04:01:29,885 INFO [qpid.message.configstore.created] [Broker]
> [vh(/test)/ms(DerbyMessageStore)] CFG-1001 : Created
> ### What is going on here?
> main 2012-09-24 04:01:40,551 DEBUG [qpid.server.store.EventManager] Received
> event AFTER_INIT
> main 2012-09-24 04:01:40,551 INFO [qpid.message.messagestore.created]
> [Broker] [vh(/test)/ms(DerbyMessageStore)] MST-1001 : Created
> main 2012-09-24 04:01:40,551 INFO [qpid.message.transactionlog.created]
> [Broker] [vh(/test)/ms(DerbyMessageStore)] TXN-1001 : Created
> m
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Created] (QPID-4342) Derby profile keeps timing out on Jenkins CI (slow store initialisation?)
Keith Wall created QPID-4342:
Summary: Derby profile keeps timing out on Jenkins CI (slow store
initialisation?)
Key: QPID-4342
URL: https://issues.apache.org/jira/browse/QPID-4342
Project: Qpid
Issue Type: Bug
Components: Java Broker
Reporter: Keith Wall
Fix For: 0.19
We've seen a number of timeouts on Jenkins when running the Derby profile over
the last few days. From the look of the logs, it suggests that store
initialisation is taking an excessive length of time . 8 or 9 seconds in some
cases.
https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-Java-Java-DBY-TestMatrix/200/
https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-Java-Java-DBY-TestMatrix/198/
https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-Java-Java-DBY-TestMatrix/197/
The pattern I see in the logs is as follows:
{noformat}
main 2012-09-24 04:01:29,885 INFO
[security.auth.manager.PrincipalDatabaseAuthenticationManager] Initialised
CRAM-MD5 SASL provider successfully
main 2012-09-24 04:01:29,885 INFO
[security.auth.manager.PrincipalDatabaseAuthenticationManager] Additional SASL
providers successfully registered.
main 2012-09-24 04:01:29,885 INFO [qpid.message.virtualhost.created] [Broker]
VHT-1001 : Created : test
main 2012-09-24 04:01:29,885 DEBUG [qpid.server.store.EventManager] Received
event BEFORE_INIT
main 2012-09-24 04:01:29,885 INFO [qpid.message.configstore.created] [Broker]
[vh(/test)/ms(DerbyMessageStore)] CFG-1001 : Created
### What is going on here?
main 2012-09-24 04:01:40,551 DEBUG [qpid.server.store.EventManager] Received
event AFTER_INIT
main 2012-09-24 04:01:40,551 INFO [qpid.message.messagestore.created] [Broker]
[vh(/test)/ms(DerbyMessageStore)] MST-1001 : Created
main 2012-09-24 04:01:40,551 INFO [qpid.message.transactionlog.created]
[Broker] [vh(/test)/ms(DerbyMessageStore)] TXN-1001 : Created
m
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
Re: Review Request: Implement a ListMessage interface to accept amqp/list messages as java.util.List
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6816/#review11830
---
Is this missing the added files?
- Gordon Sim
On Sept. 21, 2012, 3:52 p.m., rajith attapattu wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/6816/
> ---
>
> (Updated Sept. 21, 2012, 3:52 p.m.)
>
>
> Review request for qpid, Gordon Sim, Robbie Gemmell, Weston Price, and Keith
> Wall.
>
>
> Description
> ---
>
> We have a need to support amqp encoded list messages that are understood
> across all language clients. In JMS this is a bit difficult as there is no
> direct list message interface like the map message.
> I wanted to introduce a way of getting list message support while using only
> JMS interfaces.
>
> The list message support is implemented as a special case of Map message,
> where the only entry is a list (it can be nested lists as well).
> The application notifies the client of their intention via a message header.
> MapMessage m = _session.createMapMessage();
> m.setBooleanProperty("encode-as-list", true);
> m.setObject("amqp/list", myList);
> _producer.send(m);
>
> The resulting message body will be encoded as an AMQP 0-10 list. A C++ or
> python client can then directly read it as a list message.
>
> On the receiving side, it will be presented as a MapMessage with a single
> entry retrieved using "amqp/list".
> List list = (List)m.getObject("amqp/list");
>
> Please feel free to provide feedback, including alternative ways of
> implementing this.
> All feedback are most welcomed!
>
>
> This addresses bug QPID-3906.
> https://issues.apache.org/jira/browse/QPID-3906
>
>
> Diffs
> -
>
>
> http://svn.apache.org/repos/asf/qpid/trunk/qpid/java/client/src/main/java/org/apache/qpid/client/AMQConnection.java
> 1388554
>
> http://svn.apache.org/repos/asf/qpid/trunk/qpid/java/client/src/main/java/org/apache/qpid/client/AMQSession.java
> 1388554
>
> http://svn.apache.org/repos/asf/qpid/trunk/qpid/java/client/src/main/java/org/apache/qpid/client/message/JMSStreamMessage.java
> 1388554
>
> http://svn.apache.org/repos/asf/qpid/trunk/qpid/java/client/src/main/java/org/apache/qpid/client/message/MessageFactoryRegistry.java
> 1388554
>
> http://svn.apache.org/repos/asf/qpid/trunk/qpid/java/client/src/main/java/org/apache/qpid/jms/ConnectionURL.java
> 1388554
>
> http://svn.apache.org/repos/asf/qpid/trunk/qpid/java/client/src/main/java/org/apache/qpid/jms/Session.java
> 1388554
>
> http://svn.apache.org/repos/asf/qpid/trunk/qpid/java/common/src/main/java/org/apache/qpid/configuration/ClientProperties.java
> 1388554
>
> Diff: https://reviews.apache.org/r/6816/diff/
>
>
> Testing
> ---
>
> AMQPEncodedListMessageTest is added to cover several cases.
>
>
> Thanks,
>
> rajith attapattu
>
>
[jira] [Assigned] (QPID-4334) [Java broker] move the Firewall functionality into the ACL plugin
[ https://issues.apache.org/jira/browse/QPID-4334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Harvey reassigned QPID-4334: --- Assignee: Philip Harvey > [Java broker] move the Firewall functionality into the ACL plugin > - > > Key: QPID-4334 > URL: https://issues.apache.org/jira/browse/QPID-4334 > Project: Qpid > Issue Type: Improvement > Components: Java Broker >Reporter: Robbie Gemmell >Assignee: Philip Harvey > Fix For: 0.19 > > > Firewall rules are currently expressed separately from ACls, but this plugin > is effectively an access control plugin and it is felt that the functionality > would thus be better expressed using ACL rules. Such request has been made in > the past, including a patch for the C++ broker as in use by a user. > The firewall functionality will be moved into the ACL plugin, and the > firewall plugin removed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Closed] (QPID-4333) [Java broker] remove the Slow Consumer Detection plugin
[ https://issues.apache.org/jira/browse/QPID-4333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keith Wall closed QPID-4333. > [Java broker] remove the Slow Consumer Detection plugin > --- > > Key: QPID-4333 > URL: https://issues.apache.org/jira/browse/QPID-4333 > Project: Qpid > Issue Type: Task > Components: Java Broker >Reporter: Robbie Gemmell >Assignee: Keith Wall > Fix For: 0.19 > > > The Slow Consumer Detection plugin has gone largely unused since its addition > and is fairly heavily tied to Commons Configuration XML and the cumbersome > plugin system the broker currently uses. In order to ease the transition away > from Commons Configuration and XML config the SCD plugin will be removed, > reducing the legacy cruft that needs to be fixed during the process. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Resolved] (QPID-4333) [Java broker] remove the Slow Consumer Detection plugin
[ https://issues.apache.org/jira/browse/QPID-4333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keith Wall resolved QPID-4333. -- Resolution: Fixed Reviewed, no comments from me. > [Java broker] remove the Slow Consumer Detection plugin > --- > > Key: QPID-4333 > URL: https://issues.apache.org/jira/browse/QPID-4333 > Project: Qpid > Issue Type: Task > Components: Java Broker >Reporter: Robbie Gemmell >Assignee: Keith Wall > Fix For: 0.19 > > > The Slow Consumer Detection plugin has gone largely unused since its addition > and is fairly heavily tied to Commons Configuration XML and the cumbersome > plugin system the broker currently uses. In order to ease the transition away > from Commons Configuration and XML config the SCD plugin will be removed, > reducing the legacy cruft that needs to be fixed during the process. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
Re: qpid-cluster broker recovery failed
Gordon Sim wrote > certainly look like they are recovered links. > > The TCP support is compiled in to the broker, but the factory is only > registered when the plugins are initialised. > > However, the store is recovered *before* the plugin initialisation. That > seems to me to be a bug. > > Do the links eventually get created? I.e. do the errors stop eventually? > The link management logic should keep retrying (with a backoff strategy) > until they are established, and the plugins should all eventually be > initialised. hi Gordon， thanks for your replay. In my test, it looks like the link would not get created, it would keep retrying and throwing error messages in 64 seconds interval, following is my output log. [root@localhost .libs]# ./lt-qpidd --auth 0 -p 5682 --cluster-name cluster_two --load-module ./cluster.so --data-dir=/tmp/5682 2012-09-24 10:22:25 [Unspecified] notice Journal "TplStore": Created 2012-09-24 10:22:25 [Unspecified] notice Store module initialized; store-dir=/tmp/5682 2012-09-24 10:22:25 [HA] notice Initializing CPG 2012-09-24 10:22:25 [HA] notice Cluster store state: clean cluster-id=0888a5cb-9f2d-4f37-8c51-050e42aee723 shutdown-id=9f7b9e38-1488-4c7a-951b-4a1c13f7a706 2012-09-24 10:22:25 [HA] notice cluster(172.26.184.45:24761 PRE_INIT) configuration change: 172.26.184.45:24761 2012-09-24 10:22:25 [HA] notice cluster(172.26.184.45:24761 PRE_INIT) Members joined: 172.26.184.45:24761 2012-09-24 10:22:25 [Unspecified] notice Journal "queue-test": Created 2012-09-24 10:22:25 [Unspecified] warning Journal "queue-test": Recovery found 4 files (different from --num-jfiles value of 8). 2012-09-24 10:22:25 [Unspecified] warning Journal "queue-test": Recovery found file size = 2 (different from --jfile-size-pgs value of 24). 2012-09-24 10:22:25 [Broker] error Link connection to 172.26.184.35:5682 failed: Unsupported transport type: tcp (qpid/broker/Broker.cpp:1006) 2012-09-24 10:22:27 [Broker] error Link connection to 172.26.184.35:5682 failed: Unsupported transport type: tcp (qpid/broker/Broker.cpp:1006) 2012-09-24 10:22:31 [Broker] error Link connection to 172.26.184.35:5682 failed: Unsupported transport type: tcp (qpid/broker/Broker.cpp:1006) 2012-09-24 10:22:39 [Broker] error Link connection to 172.26.184.35:5682 failed: Unsupported transport type: tcp (qpid/broker/Broker.cpp:1006) 2012-09-24 10:22:55 [Broker] error Link connection to 172.26.184.35:5682 failed: Unsupported transport type: tcp (qpid/broker/Broker.cpp:1006) 2012-09-24 10:23:27 [Broker] error Link connection to 172.26.184.35:5682 failed: Unsupported transport type: tcp (qpid/broker/Broker.cpp:1006) 2012-09-24 10:24:31 [Broker] error Link connection to 172.26.184.35:5682 failed: Unsupported transport type: tcp (qpid/broker/Broker.cpp:1006) 2012-09-24 10:25:35 [Broker] error Link connection to 172.26.184.35:5682 failed: Unsupported transport type: tcp (qpid/broker/Broker.cpp:1006) .. Best Regards Rong -- View this message in context: http://qpid.2158936.n2.nabble.com/qpid-cluster-broker-recovery-failed-tp7582404p7582513.html Sent from the Apache Qpid developers mailing list archive at Nabble.com. - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
Re: AMQP 1.0 support for JMS client via Proton
On 09/21/2012 10:51 PM, Rajith Attapattu wrote: Gordon I'm not comparing the API's here. I'm comparing the *implementations of those API's*. My assumption is that the Qpid API implementation will likely use a logging façade like slf4j and perhaps some other dependencies, while the Messenger implementation will be dependency free and more compact. I am challenging that assumption (and the inference you drew from it regarding the attractiveness of particular - as yet unimplemented - APIs in particular environments). Why would the needs of users of these APIs with regard to logging be different? If they aren't why would we pursue two different approaches to meeting those needs and why would we align a particular approach with a particular API? [...] I agree with you that there is a lot of overlap btw the JMS and Qpid API in terms of functionality and behaviour. Infact the Qpid API seems like a superset of the JMS API I also agree from a design point of view it seems cleaner. But as I said I'm not discounting either approach. It's always good to investigate all approaches and look into all kinds of factor not just technical. Are there non-technical advantages that you see with option #2? Until recently I have considered only option #1 (JMS --> Qpid API --> Proton, but I have heard people looking at option #2 as well. It's always good to consider all options Certainly. I am not suggesting otherwise, merely trying to establish the case for the second option you presented. I can understand a reluctance to have another public API.; but from what you are saying that isn't really a factor in you're mind with option #2? (While you are laying out *all* the options, it might be useful to include the current approach for JMS over AMQP 1.0, as included in the 0.18 release. What are the particular issues that we would want to address by picking another approach?). - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
