Re: [google-appengine] Announcing the Matcher API for Trusted Testers
Hi Ikai,
I've tried accessing the trusted tester list but I get this permission error
from Google Docs:
We're sorry, my email address does not have permission to access this
spreadsheet.
You are signed in as my email address, but that email address doesn't have
permission to access this spreadsheet. (Sign in as a different user or
request access to this document)
where my email address is this email address I'm sending from now (it's my
Google Apps + Google Account email address).
Nick
On 20 October 2010 11:10, Ikai Lan (Google)
ikai.l+gro...@google.comikai.l%2bgro...@google.com
wrote:
Hey everyone,
I wanted to announce that we are accepting signups for trusted testers for
the Python Matcher API, which is available for local testing in the 1.3.8
SDK. The Matcher API allows developers to take advantage of Google's high
performance matching infrastructure. Developers will be able to register a
large number of queries for incoming documents to be matched against. The
API will then match these queries against numerical and text properties of
incoming data at a very high rate.
To better illustrate what the Matcher API can do, let’s pretend you are
building a site that notifies users on stock ticker price changes. That is -
a user of the site might sign up and register to receive alerts anytime
BRK.A is greater than $500 but lower than $525 (by the way, if Berkshire
Hathaway is ever in this price range, sell everything you have and buy.
Disclosure: I am not a registered financial advisor). Here’s how this might
be implemented on App Engine before:
1. When a user wants to create a new alert, a new AlertCondition entity is
created. This entity records the ticker_symbol, min_price, max_price, and
email to notify.
2. On an incoming notification of a stock price change of BRK.A between
$500 and $525, we filter AlertCondition entities. Entities that match are
returned, and from these entities, we create offline tasks to email each of
the users about the price change.
This works decently, given that we don’t have many stock price changes or
many alerts in the system. As the number of AlertConditions go up, we will
need to change our application to break the queries into multiple pages, or
even move them into task queues. Unfortunately for us, stock prices change
very frequently, and (we hope) we will have many users. Fetching tens of
thousands of Alert Conditions from the datastore can take on the order of
seconds, causing the implementation detailed above to be difficult to scale
for our expected usage.
How does the Matcher API help us solve this problem?
---
The Matcher API allows us to register a set of queries, then filter
incoming documents against these queries in a scalable, high-performance
fashion. The type of problem being solved in the stock price notification
example is a good example of how the Matcher API can be used. Here’s what
we’d have to do in our application using the Matcher API:
1. When a user wants to create a new alert, we acquire an instance of a
Python matcher object and register queries.
2. On an incoming stock price change, we run the alert against the matcher
and try to find all the queries that matched. As queries are found, the
matcher API enqueues tasks to process the results offline. Unlike the
implementation using the datastore in the earlier example, the Matcher API
performs checks in parallel and runs using a completely different service
optimized for this use case. A single price change notification could match
against hundreds of thousands of queries in the time span of a few seconds.
Let’s show this example in code (also posted here:
http://pastie.org/1234174):
# We’re going to call subscribe. Here’s what we’re passing:
# dict - this means we are going to match against a Python dictionary. We
can also
# pass a db.Model type to match against. For instance, StockPrice
# “symbol: GOOG AND price 500 AND price 525” - this is our query
# “ikai:GOOG” - this is the name of our subscription. We’ll use this to map
back to our
# User. This must be unique, so we are using the User key and
ticket combination
matcher.subscribe(dict, “symbol: GOOG AND price 500 AND price 525”,
“ikai:GOOG”)
# When a new stock price update comes in, we create a Python dictionary
representing
# all the parts we care about
change = { symbol : GOOG, price : 515 }
matcher.match(change)
# The code above doesn’t execute right away. It makes an API call to
Google’s
# matcher service, which, upon completion, begins dispatching matches to a
# task queue at the URI path /_ah/matcher. You’ll need to define the task
queue handler:
application = webapp.WSGIApplication(
[('/_ah/matcher', ChangeNotificationHandler)])
# You'd define the handler, a web handler for the results:
class ChangeNotificationHandler(webapp.RequestHandler):
def post(self):
user_ids =
Re: [google-appengine] Announcing the Matcher API for Trusted Testers
That's my fault, here's a working link:
https://spreadsheets.google.com/a/google.com/viewform?hl=enformkey=dG5XNnlVWXJYWG1yS0ExV2RmTW5EZEE6MQ#gid=0
And yes, Java support is coming.
--
Ikai Lan
Developer Programs Engineer, Google App Engine
Blogger: http://googleappengine.blogspot.com
Reddit: http://www.reddit.com/r/appengine
Twitter: http://twitter.com/app_engine
On Tue, Oct 19, 2010 at 5:29 PM, Nickolas Daskalou n...@daskalou.comwrote:
Hi Ikai,
I've tried accessing the trusted tester list but I get this permission
error from Google Docs:
We're sorry, my email address does not have permission to access this
spreadsheet.
You are signed in as my email address, but that email address doesn't
have permission to access this spreadsheet. (Sign in as a different user or
request access to this document)
where my email address is this email address I'm sending from now (it's
my Google Apps + Google Account email address).
Nick
On 20 October 2010 11:10, Ikai Lan (Google)
ikai.l+gro...@google.comikai.l%2bgro...@google.com
wrote:
Hey everyone,
I wanted to announce that we are accepting signups for trusted testers for
the Python Matcher API, which is available for local testing in the 1.3.8
SDK. The Matcher API allows developers to take advantage of Google's high
performance matching infrastructure. Developers will be able to register a
large number of queries for incoming documents to be matched against. The
API will then match these queries against numerical and text properties of
incoming data at a very high rate.
To better illustrate what the Matcher API can do, let’s pretend you are
building a site that notifies users on stock ticker price changes. That is -
a user of the site might sign up and register to receive alerts anytime
BRK.A is greater than $500 but lower than $525 (by the way, if Berkshire
Hathaway is ever in this price range, sell everything you have and buy.
Disclosure: I am not a registered financial advisor). Here’s how this might
be implemented on App Engine before:
1. When a user wants to create a new alert, a new AlertCondition entity is
created. This entity records the ticker_symbol, min_price, max_price, and
email to notify.
2. On an incoming notification of a stock price change of BRK.A between
$500 and $525, we filter AlertCondition entities. Entities that match are
returned, and from these entities, we create offline tasks to email each of
the users about the price change.
This works decently, given that we don’t have many stock price changes or
many alerts in the system. As the number of AlertConditions go up, we will
need to change our application to break the queries into multiple pages, or
even move them into task queues. Unfortunately for us, stock prices change
very frequently, and (we hope) we will have many users. Fetching tens of
thousands of Alert Conditions from the datastore can take on the order of
seconds, causing the implementation detailed above to be difficult to scale
for our expected usage.
How does the Matcher API help us solve this problem?
---
The Matcher API allows us to register a set of queries, then filter
incoming documents against these queries in a scalable, high-performance
fashion. The type of problem being solved in the stock price notification
example is a good example of how the Matcher API can be used. Here’s what
we’d have to do in our application using the Matcher API:
1. When a user wants to create a new alert, we acquire an instance of a
Python matcher object and register queries.
2. On an incoming stock price change, we run the alert against the matcher
and try to find all the queries that matched. As queries are found, the
matcher API enqueues tasks to process the results offline. Unlike the
implementation using the datastore in the earlier example, the Matcher API
performs checks in parallel and runs using a completely different service
optimized for this use case. A single price change notification could match
against hundreds of thousands of queries in the time span of a few seconds.
Let’s show this example in code (also posted here:
http://pastie.org/1234174):
# We’re going to call subscribe. Here’s what we’re passing:
# dict - this means we are going to match against a Python dictionary. We
can also
# pass a db.Model type to match against. For instance, StockPrice
# “symbol: GOOG AND price 500 AND price 525” - this is our query
# “ikai:GOOG” - this is the name of our subscription. We’ll use this to
map back to our
# User. This must be unique, so we are using the User key and
ticket combination
matcher.subscribe(dict, “symbol: GOOG AND price 500 AND price 525”,
“ikai:GOOG”)
# When a new stock price update comes in, we create a Python dictionary
representing
# all the parts we care about
change = { symbol : GOOG, price : 515 }
matcher.match(change)
# The code above doesn’t execute right away. It
