Re: Tips on speeding up OR queries?
Maybe you can do it in Javascipt on the Browser? Richard On Dec 2, 3:24 am, Andrew Badera [EMAIL PROTECTED] wrote: Which reinforces my it's a GAE problem stance ... GAE needs to accept the reality of the demands that will be placed on that system. Long-running operations against foreign servers is one among many such issues. On Mon, Dec 1, 2008 at 9:23 PM, Amir Michail [EMAIL PROTECTED] wrote: On Dec 1, 9:08 pm, Amir Michail [EMAIL PROTECTED] wrote: On Dec 1, 8:15 pm, Andrew Badera [EMAIL PROTECTED] wrote: The timeout I see as more of a GAE issue, not a Twitter one. You get what you pay for, on both sides of that equation. It's not a CPU usage issue in the GAE. It's just that the request to the twitter search API takes too long and urlfetch times out in the GAE. Hopefully a timeout parameter will be added to urlfetch soon... Amir Actually... Increasing the timeout in production isn't possible at present, and you'll always be bound by our overall request limits. http://groups.google.com/group/google-appengine/msg/f59528628a3bb86e Amir No SLA, no billing from Twitter, beta and no billing on GAE's part ... and I think we can agree, GAE hasn't been particular forthcoming nor cooperative about what constitutes a true mcycle or a long-running request. That said, perhaps there's some optimization to be done on OR ... Thanks- - Andy Badera - [EMAIL PROTECTED] - (518) 641-1280 -http://higherefficiency.net/ -http://changeroundup.com/ -http://flipbitsnotburgers.blogspot.com/ -http://andrew.badera.us/ - Google me:http://www.google.com/search?q=andrew+badera On Mon, Dec 1, 2008 at 8:10 PM, Amir Michail [EMAIL PROTECTED] wrote: Hi, OR search queries can take a long time and are causing lots of timeouts with google app engine. Amir
Re: Tips on speeding up OR queries?
I do OR queries through the search API all day long and they are always blazingly fast from my perspective. Do you have some numbers you could share regarding the time it takes to do the query? Also, having the client do it in their browser through javascript is also a very easy and viable alternative (of course, depending on the application). I have used that method many many times as well. -Chad On Tue, Dec 2, 2008 at 6:54 AM, Richie [EMAIL PROTECTED] wrote: Maybe you can do it in Javascipt on the Browser? Richard On Dec 2, 3:24 am, Andrew Badera [EMAIL PROTECTED] wrote: Which reinforces my it's a GAE problem stance ... GAE needs to accept the reality of the demands that will be placed on that system. Long-running operations against foreign servers is one among many such issues. On Mon, Dec 1, 2008 at 9:23 PM, Amir Michail [EMAIL PROTECTED] wrote: On Dec 1, 9:08 pm, Amir Michail [EMAIL PROTECTED] wrote: On Dec 1, 8:15 pm, Andrew Badera [EMAIL PROTECTED] wrote: The timeout I see as more of a GAE issue, not a Twitter one. You get what you pay for, on both sides of that equation. It's not a CPU usage issue in the GAE. It's just that the request to the twitter search API takes too long and urlfetch times out in the GAE. Hopefully a timeout parameter will be added to urlfetch soon... Amir Actually... Increasing the timeout in production isn't possible at present, and you'll always be bound by our overall request limits. http://groups.google.com/group/google-appengine/msg/f59528628a3bb86e Amir No SLA, no billing from Twitter, beta and no billing on GAE's part ... and I think we can agree, GAE hasn't been particular forthcoming nor cooperative about what constitutes a true mcycle or a long-running request. That said, perhaps there's some optimization to be done on OR ... Thanks- - Andy Badera - [EMAIL PROTECTED] - (518) 641-1280 -http://higherefficiency.net/ -http://changeroundup.com/ -http://flipbitsnotburgers.blogspot.com/ -http://andrew.badera.us/ - Google me:http://www.google.com/search?q=andrew+badera On Mon, Dec 1, 2008 at 8:10 PM, Amir Michail [EMAIL PROTECTED] wrote: Hi, OR search queries can take a long time and are causing lots of timeouts with google app engine. Amir
Re: Tips on speeding up OR queries?
On Dec 2, 11:24 am, Chad Etzel [EMAIL PROTECTED] wrote: I do OR queries through the search API all day long and they are always blazingly fast from my perspective. Do you have some numbers you could share regarding the time it takes to do the query? Queries of this form can be very slow: web 2.0 from:techcrunch OR from:kevinrose OR from:leolaporte OR from:scobleizer Amir Also, having the client do it in their browser through javascript is also a very easy and viable alternative (of course, depending on the application). I have used that method many many times as well. -Chad On Tue, Dec 2, 2008 at 6:54 AM, Richie [EMAIL PROTECTED] wrote: Maybe you can do it in Javascipt on the Browser? Richard On Dec 2, 3:24 am, Andrew Badera [EMAIL PROTECTED] wrote: Which reinforces my it's a GAE problem stance ... GAE needs to accept the reality of the demands that will be placed on that system. Long-running operations against foreign servers is one among many such issues. On Mon, Dec 1, 2008 at 9:23 PM, Amir Michail [EMAIL PROTECTED] wrote: On Dec 1, 9:08 pm, Amir Michail [EMAIL PROTECTED] wrote: On Dec 1, 8:15 pm, Andrew Badera [EMAIL PROTECTED] wrote: The timeout I see as more of a GAE issue, not a Twitter one. You get what you pay for, on both sides of that equation. It's not a CPU usage issue in the GAE. It's just that the request to the twitter search API takes too long and urlfetch times out in the GAE. Hopefully a timeout parameter will be added to urlfetch soon... Amir Actually... Increasing the timeout in production isn't possible at present, and you'll always be bound by our overall request limits. http://groups.google.com/group/google-appengine/msg/f59528628a3bb86e Amir No SLA, no billing from Twitter, beta and no billing on GAE's part ... and I think we can agree, GAE hasn't been particular forthcoming nor cooperative about what constitutes a true mcycle or a long-running request. That said, perhaps there's some optimization to be done on OR ... Thanks- - Andy Badera - [EMAIL PROTECTED] - (518) 641-1280 -http://higherefficiency.net/ -http://changeroundup.com/ -http://flipbitsnotburgers.blogspot.com/ -http://andrew.badera.us/ - Google me:http://www.google.com/search?q=andrew+badera On Mon, Dec 1, 2008 at 8:10 PM, Amir Michail [EMAIL PROTECTED] wrote: Hi, OR search queries can take a long time and are causing lots of timeouts with google app engine. Amir
Search API feature request: follows:username
Hi, I would like to be able to issue queries like this: web 2.0 follows:techcrunch This would identify tweets containing web 2.0 posted by people following techcrunch. Amir
Re: Search API feature request: follows:username
On Dec 2, 1:33 pm, Matt Sanford [EMAIL PROTECTED] wrote: Hi Amir, This is something we've talked about internally for some time. Please open a ticket athttp://code.google.com/p/twitter-api/issuesso we can track your request. Done: http://code.google.com/p/twitter-api/issues/detail?id=172 Amir Thanks; — Matt Sanford (@mzsanford) On Dec 2, 2008, at 10:31 AM, Amir Michail wrote: Hi, I would like to be able to issue queries like this: web 2.0 follows:techcrunch This would identify tweets containing web 2.0 posted by people following techcrunch. Amir --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Twitter Development Talk group. To post to this group, send email to twitter-development-talk@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/twitter-development-talk?hl=en -~--~~~~--~~--~--~---
Re: A general status update from Twitter's API Team
We'll keep the current version running for a stretch (probably six months tops) as developers transition over to the new version of the API. On Tue, Dec 2, 2008 at 12:33, Chad Etzel [EMAIL PROTECTED] wrote: Thanks for the update! For those of us doing current development with the API, will the current version be kept around for a while (as a legacy version I guess) so that we may continue development as the new API is being rolled out? Or will it be a cut-over situation when the new API is released? I understand that eventually the current API version will be retired... but looking for guidance in the short-term. Thanks, -Chad On Tue, Dec 2, 2008 at 3:27 PM, Alex Payne [EMAIL PROTECTED] wrote: Hi all, Just wanted to give you an update on what's going on Twitter API land. Firstly, my colleague on the API Team, Matt Sanford (@mzsanford), is in town from Seattle and working from the Twitter offices. We're trying to make the most of this in-person time to clear out administrivia and plan the next several weeks of work. We've just finished cleaning up the list of API issues and enhancement requests (http://code.google.com/p/twitter-api/issues/list). We've closed, updated, re-prioritized, and generally attended to all tickets in the system. We have a number of fixes that are waiting on other parts of the Twitter engineering team to ship, and we've tried to clearly note which tickets aren't going to be dealt with until the next major release of the API. Just yesterday, Matt finished working with our Operations team to move Twitter Search to Twitter's data center. The Search API should now return results more quickly, and we believe that we've increased our queries per second (QPS) capacity as well. Additionally, Matt has been working with our User Experience (UX) team on a beta of OAuth support. The UX component of this work is almost complete, and we should be ready for our first deploy in the next week or ten days. The only potential blocker to this launch is the database schema changes it entails, which may be delayed by our Operations team as part of a broader set of database work. Having completed performance tests to our satisfaction, a colleague of ours has been testing our HTTP-based firehose solution for correctness and stability. So far he's uncovered no issues, and we should be starting a beta period with this service in a matter of days. Apologies for not having the beta going by Thanksgiving, but hopefully this additional testing will mean fewer issues and a reduced time-to-production. Our next major priority remains the rewrite of the Twitter API, which encompasses a variety of backend and frontend changes. We were hoping to have much of this work completed by the end of the year, and while I believe it'll be underway, I don't expect that it will be complete until early next year. If you have any questions about our priorities and projects, please let us know. Thanks! -- Alex Payne - API Lead, Twitter, Inc. http://twitter.com/al3x -- Alex Payne - API Lead, Twitter, Inc. http://twitter.com/al3x
Re: A general status update from Twitter's API Team
On Tue, Dec 2, 2008 at 2:27 PM, Alex Payne [EMAIL PROTECTED] wrote: Additionally, Matt has been working with our User Experience (UX) team on a beta of OAuth support. The UX component of this work is almost complete, and we should be ready for our first deploy in the next week or ten days. Nifty. Anything y'all can share about the thinking behind your OAuth UX decisions would be very helpful (not just how it ends up looking, but the sorts of things that were of concern, differerent options you considered, etc). That stuff's pure gold for others facing similar sorts of decisions. Not totally on topic, I'm just saying... -cks -- Christopher St. John http://artofsystems.blogspot.com
Re: Difference between mobile and sms source?
On 2 Dec 2008, at 21:12, fastest963 wrote: What is the difference between the mobile and sms source? Mobile means from m.twitter.com and SMS means, erm, SMS. -Stut -- http://stut.net/
Re: A general status update from Twitter's API Team
Sure, I'll talk to the UX folks about writing some of that up. OAuth is still in its early stages, and it seems most every organization that implements it ends up taking some slightly different paths. On Tue, Dec 2, 2008 at 13:03, Christopher St John [EMAIL PROTECTED] wrote: On Tue, Dec 2, 2008 at 2:27 PM, Alex Payne [EMAIL PROTECTED] wrote: Additionally, Matt has been working with our User Experience (UX) team on a beta of OAuth support. The UX component of this work is almost complete, and we should be ready for our first deploy in the next week or ten days. Nifty. Anything y'all can share about the thinking behind your OAuth UX decisions would be very helpful (not just how it ends up looking, but the sorts of things that were of concern, differerent options you considered, etc). That stuff's pure gold for others facing similar sorts of decisions. Not totally on topic, I'm just saying... -cks -- Christopher St. John http://artofsystems.blogspot.com -- Alex Payne - API Lead, Twitter, Inc. http://twitter.com/al3x
Re: New API methods for updating profile design and images
Hi, Does anyone have a sample of what the HTTP Request POST body should look like? Thanks, Lien On Oct 23, 7:54 am, Alex Payne [EMAIL PROTECTED] wrote: Not any time soon, but we'll keep it in mind. On Wed, Oct 22, 2008 at 10:34 PM, Richie [EMAIL PROTECTED] wrote: Cool. Are there any plans on allowing to update images by adding an url- parameter instead of raw multipart images? This would make the API much simpler to use, but I guess there might be security issues. Richie. On Oct 23, 2:07 am, Alex Payne [EMAIL PROTECTED] wrote: We missed that bit, but we'll be adding a parameter to the update_profile_background_image method that lets you set whether or not it tiles. On Wed, Oct 22, 2008 at 2:52 PM, Richie [EMAIL PROTECTED] wrote: Hi Alex, this looks great. I got some ideas for this allready. One question: How to setprofilebackgroundimagetiles? Thanks, Richie On Oct 21, 8:10 pm, Alex Payne [EMAIL PROTECTED] wrote: Hi all! Some new API methods for you to play with: - /account/update_profile_colors updates the colors on a user's profile(also returned via the /users/show API method) - /account/update_profile_image sets a newprofileimagefor a user - /account/update_profile_background_image sets, you guessed it, a new backgroundimagefor a user'sprofile You can find them all documented underhttp://apiwiki.twitter.com/REST%20API%20Documentation#AccountMethods. The clear use for these methods is a third-partyprofiledesign customizer (basically, a Twitter theme site). Geo apps might want to grab a photo of where the user is at and set it as theirprofile backgroundimage. That sort of thing. Enjoy, and let us know if you find any bugs. -- Alex Payne - API Lead, Twitter, Inc.http://twitter.com/al3x -- Alex Payne - API Lead, Twitter, Inc.http://twitter.com/al3x -- Alex Payne - API Lead, Twitter, Inc.http://twitter.com/al3x
Re: Return user information in verify_credentials instead of just string representing authorized?
Thanks, I filed Issue 173: http://code.google.com/p/twitter-api/issues/detail?id=173 On Oct 30, 1:46 pm, Alex Payne [EMAIL PROTECTED] wrote: Sure, that's a thing we could do. Please request it:http://code.google.com/p/twitter-api/issues/entry On Thu, Oct 30, 2008 at 11:56 AM,LienTran [EMAIL PROTECTED] wrote: Return user information in verify_credentials instead of just string representing authorized? Useful if we could get the user id at this time instead of having to make a separate call to get the data. -- Alex Payne - API Lead, Twitter, Inc.http://twitter.com/al3x
INCOMPATIBILITY ALERT: response body of /account/verify_credentials changing Dec 10th
As per http://code.google.com/p/twitter-api/issues/detail?id=173 we'll be changing the /account/verify_credentials method to return the representation of the authenticated user. Because some applications depend on the contents of this response, we're delaying this change until December 10th, 2008. Please update your applications to verify by response code, not by the response body for this method. If you get a 200 back, you're verified. If you get a 401 back, you're not. If you can't ship an update in 8 days, please let us know and we'll push the date out further. -- Alex Payne - API Lead, Twitter, Inc. http://twitter.com/al3x
Re: Rate limit exceeded for whitelisted app
The updated estimate I've just received from our ops guys is more than 15 minutes and less than 12 hours. They have to restore from a nightly database backup. Said backups are quite large, and take some time to get through. Thanks for your patience. On Tue, Dec 2, 2008 at 18:39, Yu-Shan Fung [EMAIL PROTECTED] wrote: Thanks for being so responsive. You guys rock! On Tue, Dec 2, 2008 at 6:37 PM, Alex Payne [EMAIL PROTECTED] wrote: Just talked to our Operations team. It looks like some database maintenance inadvertently truncated our table of whitelisted users. We're restoring that as I type and everything will be back to normal shortly. On Tue, Dec 2, 2008 at 18:25, Alex Payne [EMAIL PROTECTED] wrote: No, there's no change in policy, but perhaps we have a bug. Yours is the second report of a rate limit issue. On Tue, Dec 2, 2008 at 18:23, Yu-Shan Fung [EMAIL PROTECTED] wrote: Hi, Our app (mrtweet.net), which has been whitelisted (@mrtweet) since a couple of weeks back, has suddenly began seeing the rate limit exceeded error since around 3:45pm (pacific) today. Was there a change in policy, or do I have to reapply for whitelisting? Thanks! Yu-Shan. -- Alex Payne - API Lead, Twitter, Inc. http://twitter.com/al3x -- Alex Payne - API Lead, Twitter, Inc. http://twitter.com/al3x -- Reality is that which, when you stop believing in it, doesn't go away. - Philip K. Dick, American Writer -- Alex Payne - API Lead, Twitter, Inc. http://twitter.com/al3x
Re: uaing HTTP Basic Authentication with VB
Hi there, My VB knowledge is very old and very poor, and my .NET is only slightly better. Take a look at: request.PreAuthenticate = True request.Credentials = new NetworkCredential(UserName, Password) I could easily be wrong. Take a look at http://msdn.microsoft.com/en-us/library/system.net.webrequest_members(VS.71).aspx Thanks; — Matt Sanford (@mzsanford) On Dec 2, 6:58 pm, jpdenoyer [EMAIL PROTECTED] wrote: Does anyone know how to use a VB (VB express 2005) program to access a webpage that requires HTTP Basic Authentication for access? Below is the code I have so far. It obviously does not work because there is no authentification info supplied (and I do not know how to supply the info). Public Class Form1 Const URL As String = http://twitter.com/statuses/ friends_timeline.xml?count=1 Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load Dim request As WebRequest = WebRequest.Create(URL) Dim response As WebResponse = request.GetResponse() Dim rssStream As Stream = response.GetResponseStream() Dim rssDoc As XmlDocument = New XmlDocument() rssDoc.Load(rssStream) DisplayNode(rssDoc, 0) End Sub Private Sub DisplayNode(ByVal node As XmlNode, ByVal depth As Integer) ' Define the indent level. Dim Indent As New String( c, depth * 4) ' Display the node type. TextBox1.Text = (Indent node.NodeType.ToString() _ : node.Name ) ' Display the node content, if applicable. If node.Value String.Empty Then Console.WriteLine(Indent Value: node.Value) End If ' Display all nested nodes. Dim Child As XmlNode For Each Child In node.ChildNodes DisplayNode(Child, depth + 1) Next End Sub End Class
uaing HTTP Basic Authentication with VB
Does anyone know how to use a VB (VB express 2005) program to access a webpage that requires HTTP Basic Authentication for access? Below is the code I have so far. It obviously does not work because there is no authentification info supplied (and I do not know how to supply the info). Public Class Form1 Const URL As String = http://twitter.com/statuses/ friends_timeline.xml?count=1 Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load Dim request As WebRequest = WebRequest.Create(URL) Dim response As WebResponse = request.GetResponse() Dim rssStream As Stream = response.GetResponseStream() Dim rssDoc As XmlDocument = New XmlDocument() rssDoc.Load(rssStream) DisplayNode(rssDoc, 0) End Sub Private Sub DisplayNode(ByVal node As XmlNode, ByVal depth As Integer) ' Define the indent level. Dim Indent As New String( c, depth * 4) ' Display the node type. TextBox1.Text = (Indent node.NodeType.ToString() _ : node.Name ) ' Display the node content, if applicable. If node.Value String.Empty Then Console.WriteLine(Indent Value: node.Value) End If ' Display all nested nodes. Dim Child As XmlNode For Each Child In node.ChildNodes DisplayNode(Child, depth + 1) Next End Sub End Class
