Re: how to programatically create thousands of topics (for marketdata)?

2009-03-25 Thread falconair
I just tried your solution, it works :) I didn't realize I could create topics on the fly. falconair, Since I don't do this kind of thing myself, I may not understand all of the issues in your application domain. Let me see if I understand what the issues are. 1. There are many topics. You w

Re: how to programatically create thousands of topics (for marketdata)?

2009-03-25 Thread Jonathan Robie
falconair, Since I don't do this kind of thing myself, I may not understand all of the issues in your application domain. Let me see if I understand what the issues are. 1. There are many topics. You want to use hierarchies to manage them. Wildcards do that well in the topic exchange. 2. Y

Re: how to programatically create thousands of topics (for marketdata)?

2009-03-25 Thread Gordon Sim
falconair wrote: You are right, that is indeed how I currently use qpid. However, for marketdata, there might be thousands of symbols. Currently, every topic I use has to be defined in my properties file as such: topic.a=x.a topic.b=x.b ... I can't list all the symbols (once for level1, again

Re: how to programatically create thousands of topics (for marketdata)?

2009-03-25 Thread falconair
You are right, that is indeed how I currently use qpid. However, for marketdata, there might be thousands of symbols. Currently, every topic I use has to be defined in my properties file as such: topic.a=x.a topic.b=x.b ... I can't list all the symbols (once for level1, again for level2 then

Re: how to programatically create thousands of topics (for marketdata)?

2009-03-25 Thread Gordon Sim
falconair wrote: I would like to publish market data through qpid. Since I have massive amount of data flowing through the pipes, I can't publish all that data to a single topic, since the client may only be interested in one or two symbols. The usual solution to use a hierarchical topic naming

Re: how to programatically create thousands of topics (for marketdata)?

2009-03-24 Thread falconair
Hi think Headers exchange might be the one I need. Unfortunately I can't find any developer-friendly docs on it. MRG manual doesn't mention it and qpid.apache.org doesn't seem to describe how to use it from a Java client (google doesn't help much). Any pointers on how I can access Headers ex

Re: how to programatically create thousands of topics (for marketdata)?

2009-03-24 Thread Robert Greig
2009/3/24 falconair : > After sending the original message, I realized that I might be able to take > advantage of selectors or 'where clause' like functionality.  In other words, > I could subscribe to level1 but limit the values to "symbol=MSFT" (which > means only MSFT should travel down the

Re: how to programatically create thousands of topics (for marketdata)?

2009-03-24 Thread Carl Trieloff
falconair wrote: The main reason I can't have pre-defined topics (in properties file or database) is because symbols are changed every day (some may be added, some may be deleted, some may just change slightly). Maintaining an accurate list of these symbols is not trivial (some large banks ha

Re: how to programatically create thousands of topics (for marketdata)?

2009-03-24 Thread falconair
The main reason I can't have pre-defined topics (in properties file or database) is because symbols are changed every day (some may be added, some may be deleted, some may just change slightly). Maintaining an accurate list of these symbols is not trivial (some large banks have groups dedicate

Re: how to programatically create thousands of topics (for marketdata)?

2009-03-24 Thread Carl Trieloff
falconair wrote: I would like to publish market data through qpid. Since I have massive amount of data flowing through the pipes, I can't publish all that data to a single topic, since the client may only be interested in one or two symbols. The usual solution to use a hierarchical topic nami

Re: how to programatically create thousands of topics (for marketdata)?

2009-03-24 Thread Jonathan Robie
falconair wrote: I would like to publish market data through qpid. Since I have massive amount of data flowing through the pipes, I can't publish all that data to a single topic, since the client may only be interested in one or two symbols. The usual solution to use a hierarchical topic nami

how to programatically create thousands of topics (for marketdata)?

2009-03-24 Thread falconair
I would like to publish market data through qpid. Since I have massive amount of data flowing through the pipes, I can't publish all that data to a single topic, since the client may only be interested in one or two symbols. The usual solution to use a hierarchical topic naming scheme. In ot