Re: [google-appengine] Re: Announcing the Matcher API for Trusted Testers
No, it's not on the roadmap. You can check this roadmap here: http://code.google.com/appengine/docs/roadmap.html Note that our roadmap is not an exhaustive list of everything we intend to do, just a list of the highest priority items for us. -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger: http://googleappengine.blogspot.com Reddit: http://www.reddit.com/r/appengine Twitter: http://twitter.com/app_engine On Tue, Oct 19, 2010 at 7:27 PM, MasterGaurav wrote: > Hi Ikai, > > That's a fantastic news! > > btw wondering if "processing pipeline" architecture is somewhere down > the line in the roadmap? > I have been working with FAST and OpenPipe, to name a couple, for > document processing... and it will be great to have such a feature > incorporated. Is something in roadmap yet? Or any visbility on the > same? > > > > -Gaurav > www.mastergaurav.com > > > On Oct 20, 5:36 am, "Ikai Lan (Google)" > > > > wrote: > > That's my fault, here's a working link: > > > > https://spreadsheets.google.com/a/google.com/viewform?hl=en&formkey=d... > > > > And yes, Java support is coming. > > > > -- > > Ikai Lan > > Developer Programs Engineer, Google App Engine > > Blogger:http://googleappengine.blogspot.com > > Reddit:http://www.reddit.com/r/appengine > > Twitter:http://twitter.com/app_engine > > > > On Tue, Oct 19, 2010 at 5:29 PM, Nickolas Daskalou >wrote: > > > > > Hi Ikai, > > > > > I've tried accessing the trusted tester list but I get this permission > > > error from Google Docs: > > > > > We're sorry, does not have permission to access this > > > spreadsheet. > > > > > You are signed in as , but that email address doesn't > > > have permission to access this spreadsheet. (Sign in as a different > user or > > > request access to this document) > > > > > where is this email address I'm sending from now > (it's > > > my Google Apps + Google Account email address). > > > > > Nick > > > > > On 20 October 2010 11:10, Ikai Lan (Google) > > > > > > > > > wrote: > > > > >> Hey everyone, > > > > >> I wanted to announce that we are accepting signups for trusted testers > for > > >> the Python Matcher API, which is available for local testing in the > 1.3.8 > > >> SDK. The Matcher API allows developers to take advantage of Google's > high > > >> performance matching infrastructure. Developers will be able to > register a > > >> large number of queries for incoming documents to be matched against. > The > > >> API will then match these queries against numerical and text > properties of > > >> incoming data at a very high rate. > > > > >> To better illustrate what the Matcher API can do, let’s pretend you > are > > >> building a site that notifies users on stock ticker price changes. > That is - > > >> a user of the site might sign up and register to receive alerts > anytime > > >> BRK.A is greater than $500 but lower than $525 (by the way, if > Berkshire > > >> Hathaway is ever in this price range, sell everything you have and > buy. > > >> Disclosure: I am not a registered financial advisor). Here’s how this > might > > >> be implemented on App Engine before: > > > > >> 1. When a user wants to create a new alert, a new AlertCondition > entity is > > >> created. This entity records the ticker_symbol, min_price, max_price, > and > > >> email to notify. > > > > >> 2. On an incoming notification of a stock price change of BRK.A > between > > >> $500 and $525, we filter AlertCondition entities. Entities that match > are > > >> returned, and from these entities, we create offline tasks to email > each of > > >> the users about the price change. > > > > >> This works decently, given that we don’t have many stock price changes > or > > >> many alerts in the system. As the number of AlertConditions go up, we > will > > >> need to change our application to break the queries into multiple > pages, or > > >> even move them into task queues. Unfortunately for us, stock prices > change > > >> very frequently, and (we hope) we will have many users. Fetching tens > of > > >> thousands of Alert Conditions from the datastore can take on the order > of > > >> seconds, causing the implementation detailed above to be difficult to > scale > > >> for our expected usage. > > > > >> How does the Matcher API help us solve this problem? > > >> --- > > > > >> The Matcher API allows us to register a set of queries, then filter > > >> incoming documents against these queries in a scalable, > high-performance > > >> fashion. The type of problem being solved in the stock price > notification > > >> example is a good example of how the Matcher API can be used. Here’s > what > > >> we’d have to do in our application using the Matcher API: > > > > >> 1. When a user wants to create a new alert, we acquire an instance of > a > > >> Python matcher object and register queries. > > > > >> 2. On an incoming stock price change, we run the alert against the > matcher > > >> and try to find all the queries that matched. As qu
Re: [google-appengine] Re: Announcing the Matcher API for Trusted Testers
I need to post a correction to my early example. Here's the code you should
use:
# First, define a schema for our StockTopic
schema = {str: ["symbol"], float: ["price"]}
matcher.subscribe(dict,
"symbol:GOOG AND price > 500 AND price < 525",
"ikai:GOOG",
schema=schema,
topic="StockTopic")
# Note: A subscription is active if the state returned by list
is: matcher.SubscriptionState.OK
# After waiting for the subscription to become active, you can do the
following:
change = { "symbol" : "GOOG", "price" : 515.0 }
matcher.match(change, topic="StockTopic")
The reason you need to use this code is because you can't just pass an
arbitrary dictionary. You need to register a schema to be used if you don't
use a db.Model as the first argument to matcher.subscribe(), and you need to
pass it to the subscribe() method.
Note that the subscription needs to become active. This isn't instantaneous,
but it's pretty darned fast.
Anyway, here's the trusted tester signup link for anyone that didn't catch
it:
https://spreadsheets.google.com/a/google.com/viewform?hl=en&formkey=dG5XNnlVWXJYWG1yS0ExV2RmTW5EZEE6MQ#gid=0
--
Ikai Lan
Developer Programs Engineer, Google App Engine
Blogger: http://googleappengine.blogspot.com
Reddit: http://www.reddit.com/r/appengine
Twitter: http://twitter.com/app_engine
On Wed, Oct 20, 2010 at 9:37 AM, Ikai Lan (Google)
> wrote:
> No, it's not on the roadmap. You can check this roadmap here:
>
> http://code.google.com/appengine/docs/roadmap.html
>
> Note that our roadmap is not an exhaustive list of everything we intend to
> do, just a list of the highest priority items for us.
>
>
> --
> Ikai Lan
> Developer Programs Engineer, Google App Engine
> Blogger: http://googleappengine.blogspot.com
> Reddit: http://www.reddit.com/r/appengine
> Twitter: http://twitter.com/app_engine
>
>
>
> On Tue, Oct 19, 2010 at 7:27 PM, MasterGaurav wrote:
>
>> Hi Ikai,
>>
>> That's a fantastic news!
>>
>> btw wondering if "processing pipeline" architecture is somewhere down
>> the line in the roadmap?
>> I have been working with FAST and OpenPipe, to name a couple, for
>> document processing... and it will be great to have such a feature
>> incorporated. Is something in roadmap yet? Or any visbility on the
>> same?
>>
>>
>>
>> -Gaurav
>> www.mastergaurav.com
>>
>>
>> On Oct 20, 5:36 am, "Ikai Lan (Google)"
>>
>> >
>> wrote:
>> > That's my fault, here's a working link:
>> >
>> > https://spreadsheets.google.com/a/google.com/viewform?hl=en&formkey=d.
>> ..
>> >
>> > And yes, Java support is coming.
>> >
>> > --
>> > Ikai Lan
>> > Developer Programs Engineer, Google App Engine
>> > Blogger:http://googleappengine.blogspot.com
>> > Reddit:http://www.reddit.com/r/appengine
>> > Twitter:http://twitter.com/app_engine
>> >
>> > On Tue, Oct 19, 2010 at 5:29 PM, Nickolas Daskalou > >wrote:
>> >
>> > > Hi Ikai,
>> >
>> > > I've tried accessing the trusted tester list but I get this permission
>> > > error from Google Docs:
>> >
>> > > We're sorry, does not have permission to access
>> this
>> > > spreadsheet.
>> >
>> > > You are signed in as , but that email address
>> doesn't
>> > > have permission to access this spreadsheet. (Sign in as a different
>> user or
>> > > request access to this document)
>> >
>> > > where is this email address I'm sending from now
>> (it's
>> > > my Google Apps + Google Account email address).
>> >
>> > > Nick
>> >
>> > > On 20 October 2010 11:10, Ikai Lan (Google) <
>> ikai.l+gro...@google.com <
>> ikai.l%2bgro...@google.com >
>> > > > wrote:
>> >
>> > >> Hey everyone,
>> >
>> > >> I wanted to announce that we are accepting signups for trusted
>> testers for
>> > >> the Python Matcher API, which is available for local testing in the
>> 1.3.8
>> > >> SDK. The Matcher API allows developers to take advantage of Google's
>> high
>> > >> performance matching infrastructure. Developers will be able to
>> register a
>> > >> large number of queries for incoming documents to be matched against.
>> The
>> > >> API will then match these queries against numerical and text
>> properties of
>> > >> incoming data at a very high rate.
>> >
>> > >> To better illustrate what the Matcher API can do, let’s pretend you
>> are
>> > >> building a site that notifies users on stock ticker price changes.
>> That is -
>> > >> a user of the site might sign up and register to receive alerts
>> anytime
>> > >> BRK.A is greater than $500 but lower than $525 (by the way, if
>> Berkshire
>> > >> Hathaway is ever in this price range, sell everything you have and
>> buy.
>> > >> Disclosure: I am not a registered financial advisor). Here’s how this
>> might
>> > >> be implemented on App Engine before:
>> >
>> > >> 1. When a user wants to create a new alert, a new AlertCondition
>> entity is
>> > >> created. This entity records the ticker_symbol, min_price, max_price,
>> and
>> > >> email to notify.
>> >
>> >
