Re: [google-appengine] Re: can we assume time is sync across all servers?
I was referring to the absence of an AppEngine clock skew API, not the absence of clock skew. On Tue, Mar 8, 2011 at 1:07 PM, Ikai Lan (Google) wrote: > Ideally, we wouldn't have to worry about clock skew period. Clock skew is a > fact of life when working with distributed systems. > > Ikai Lan > Developer Programs Engineer, Google App Engine > Blog: http://googleappengine.blogspot.com > Twitter: http://twitter.com/app_engine > Reddit: http://www.reddit.com/r/appengine > > > > On Tue, Mar 8, 2011 at 8:48 AM, saidimu apale wrote: > >> It would be interesting if there was an AppEngine app whose sole purpose >> was to provide a simple API that used your earlier gist to provide a moving >> average of clock-skew within AppEngine. >> >> It could also plot the collected time series as a quick visual guide to >> time in the AppEngine domain. >> >> Anyone interested in co-developing such an app? >> >> saidimu >> >> PS Ideally we shouldn't have to worry about this, or we should have a >> ready-made AppEngine API, but alas... >> >> On Fri, Mar 4, 2011 at 5:13 PM, Calvin wrote: >> >>> I just discovered a previous invention that's similar to this idea: >>> >>> http://www.adeptus-mechanicus.com/codex/htpdate/htpdate.html >>> >>> They attempt to reduce inaccuracies by averaging multiple sources. I >>> suppose that if you checked three source you could exclude one if it >>> differed wildly from the other two. >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Google App Engine" group. >>> To post to this group, send email to google-appengine@googlegroups.com. >>> To unsubscribe from this group, send email to >>> google-appengine+unsubscr...@googlegroups.com. >>> For more options, visit this group at >>> http://groups.google.com/group/google-appengine?hl=en. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: can we assume time is sync across all servers?
Ideally, we wouldn't have to worry about clock skew period. Clock skew is a fact of life when working with distributed systems. Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine On Tue, Mar 8, 2011 at 8:48 AM, saidimu apale wrote: > It would be interesting if there was an AppEngine app whose sole purpose > was to provide a simple API that used your earlier gist to provide a moving > average of clock-skew within AppEngine. > > It could also plot the collected time series as a quick visual guide to > time in the AppEngine domain. > > Anyone interested in co-developing such an app? > > saidimu > > PS Ideally we shouldn't have to worry about this, or we should have a > ready-made AppEngine API, but alas... > > On Fri, Mar 4, 2011 at 5:13 PM, Calvin wrote: > >> I just discovered a previous invention that's similar to this idea: >> >> http://www.adeptus-mechanicus.com/codex/htpdate/htpdate.html >> >> They attempt to reduce inaccuracies by averaging multiple sources. I >> suppose that if you checked three source you could exclude one if it >> differed wildly from the other two. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: can we assume time is sync across all servers?
It would be interesting if there was an AppEngine app whose sole purpose was to provide a simple API that used your earlier gist to provide a moving average of clock-skew within AppEngine. It could also plot the collected time series as a quick visual guide to time in the AppEngine domain. Anyone interested in co-developing such an app? saidimu PS Ideally we shouldn't have to worry about this, or we should have a ready-made AppEngine API, but alas... On Fri, Mar 4, 2011 at 5:13 PM, Calvin wrote: > I just discovered a previous invention that's similar to this idea: > > http://www.adeptus-mechanicus.com/codex/htpdate/htpdate.html > > They attempt to reduce inaccuracies by averaging multiple sources. I > suppose that if you checked three source you could exclude one if it > differed wildly from the other two. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: can we assume time is sync across all servers?
I just discovered a previous invention that's similar to this idea: http://www.adeptus-mechanicus.com/codex/htpdate/htpdate.html They attempt to reduce inaccuracies by averaging multiple sources. I suppose that if you checked three source you could exclude one if it differed wildly from the other two. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: can we assume time is sync across all servers?
Yeah, I can't find the thread where this was originally discussed, but the hack does make the assumption that Google is more careful about keeping the servers that serve 'www' requests in sync than they are about keeping the App Engine cluster in sync. The best use of this would be to point it at a server that is under your control, and is always served by a single machine. It should only query for time delta once when each App Engine instance is started up, so it shouldn't put that server under heavy load. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: can we assume time is sync across all servers?
>From the gist: - the difference between the time on AppEngine's proxy servers (as detected in the response headers) and "local time" on AppEngine's serving machines. local_now = datetime.utcnow() > google_time = urlfetch.fetch('http://www.google.com/robots.txt', > method=urlfetch.HEAD) > google_now = datetime.strptime(google_time.headers['date'], '%a, %d %b %Y > %H:%M:%S GMT') > swatch_delta = local_now - google_now This works if the proxy servers don't also have greatly varying clock-skew, else the delta can swing in wildly unexpected ways. saidimu On Fri, Mar 4, 2011 at 2:13 PM, Ikai Lan (Google) wrote: > I'm trying to think of a solution that doesn't involve fetching data from a > third party time service, but I'm drawing a blank. How did you detect the > time skew in the first place? > > Ikai Lan > Developer Programs Engineer, Google App Engine > Blog: http://googleappengine.blogspot.com > Twitter: http://twitter.com/app_engine > Reddit: http://www.reddit.com/r/appengine > > > > On Thu, Mar 3, 2011 at 4:57 PM, Eric Ka Ka Ng wrote: > >> Hi Gopal, Calvin, and Ikai, >> >> thx all for your sharing. Calvin, your handy script implements Gopal idea >> could be very useful to many of us ;) >> >> Ikai, yes, later we could do more testing and sampling, and with Calvin's >> script, to log the time (and delta with google time) however, this is not a >> problem frequently happen and we are currently busying with some other >> application dev tasks, guess it may take some time to log and analyze this >> problem, and would keep you posted. just thinking what kinds of 'examples' >> we could provide? (you want log? data in DS? script to produce the log / >> data? ) >> >> >> btw, how could we explain Brett's data pipelines' talk at 35:50 or i >> interpret wrongly? and i believe Gopal, Calvin faced time sync issues and >> thus have the idea and implementation to solve this (just me recently aware >> about this problem). how much time is off did you experience? >> >> thx a lot! >> >> - eric >> >> On 4 March 2011 03:12, Ikai Lan (Google) wrote: >> >>> Can you provide examples where the time is off by 30 minutes? This seems >>> incorrect. >>> >>> There's definitely clock skew, but we're talking milliseconds, seconds or >>> at worst, worst, worst maybe a minute or two. >>> >>> Ikai Lan >>> Developer Programs Engineer, Google App Engine >>> Blog: http://googleappengine.blogspot.com >>> Twitter: http://twitter.com/app_engine >>> Reddit: http://www.reddit.com/r/appengine >>> >>> >>> >>> On Thu, Mar 3, 2011 at 5:54 AM, Ng Ka Ka Eric wrote: >>> Hi Raymond, Just watched the video. Thx for pointing this reference. I really didnt expect that the machines are not time synced (can be off for 40mins?!) are there any technical difficulties to sync them? And in this case, for whatever reason if we want to record the time when the request is made into DS, how can we do that? - eric Sent from my iPhone On 2011年3月3日, at 下午6:20, "Raymond C." wrote: According to Brett's data pipelines' talk ( http://www.youtube.com/watch?v=zSDC_TU7rtc, around 35:50), its not -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Google App Engine" group. >>> To post to this group, send email to google-appengine@googlegroups.com. >>> To unsubscribe from this group, send email to >>> google-appengine+unsubscr...@googlegroups.com. >>> For more options, visit this group at >>> http://groups.google.com/group/google-appengine?hl=en. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@go
Re: [google-appengine] Re: can we assume time is sync across all servers?
I'm trying to think of a solution that doesn't involve fetching data from a third party time service, but I'm drawing a blank. How did you detect the time skew in the first place? Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine On Thu, Mar 3, 2011 at 4:57 PM, Eric Ka Ka Ng wrote: > Hi Gopal, Calvin, and Ikai, > > thx all for your sharing. Calvin, your handy script implements Gopal idea > could be very useful to many of us ;) > > Ikai, yes, later we could do more testing and sampling, and with Calvin's > script, to log the time (and delta with google time) however, this is not a > problem frequently happen and we are currently busying with some other > application dev tasks, guess it may take some time to log and analyze this > problem, and would keep you posted. just thinking what kinds of 'examples' > we could provide? (you want log? data in DS? script to produce the log / > data? ) > > > btw, how could we explain Brett's data pipelines' talk at 35:50 or i > interpret wrongly? and i believe Gopal, Calvin faced time sync issues and > thus have the idea and implementation to solve this (just me recently aware > about this problem). how much time is off did you experience? > > thx a lot! > > - eric > > On 4 March 2011 03:12, Ikai Lan (Google) wrote: > >> Can you provide examples where the time is off by 30 minutes? This seems >> incorrect. >> >> There's definitely clock skew, but we're talking milliseconds, seconds or >> at worst, worst, worst maybe a minute or two. >> >> Ikai Lan >> Developer Programs Engineer, Google App Engine >> Blog: http://googleappengine.blogspot.com >> Twitter: http://twitter.com/app_engine >> Reddit: http://www.reddit.com/r/appengine >> >> >> >> On Thu, Mar 3, 2011 at 5:54 AM, Ng Ka Ka Eric wrote: >> >>> Hi Raymond, >>> >>> Just watched the video. Thx for pointing this reference. >>> >>> >>> I really didnt expect that the machines are not time synced (can be off >>> for 40mins?!) are there any technical difficulties to sync them? And in this >>> case, for whatever reason if we want to record the time when the request is >>> made into DS, how can we do that? >>> >>> - eric >>> >>> Sent from my iPhone >>> >>> On 2011年3月3日, at 下午6:20, "Raymond C." wrote: >>> >>> According to Brett's data pipelines' talk ( >>> http://www.youtube.com/watch?v=zSDC_TU7rtc, around 35:50), its not >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Google App Engine" group. >>> To post to this group, send email to google-appengine@googlegroups.com. >>> To unsubscribe from this group, send email to >>> google-appengine+unsubscr...@googlegroups.com. >>> For more options, visit this group at >>> http://groups.google.com/group/google-appengine?hl=en. >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Google App Engine" group. >>> To post to this group, send email to google-appengine@googlegroups.com. >>> To unsubscribe from this group, send email to >>> google-appengine+unsubscr...@googlegroups.com. >>> For more options, visit this group at >>> http://groups.google.com/group/google-appengine?hl=en. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: can we assume time is sync across all servers?
I haven't experienced time-sync issues. I just know that it would be a problem if I did, so I developed a contingency hack. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: can we assume time is sync across all servers?
Sorry if my respond misled you. To clarify I think Brett just saying we should not assume the time is always in sync and use it for criterial logic, but it should be fairly close most of the time. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: can we assume time is sync across all servers?
Hi Gopal, Calvin, and Ikai, thx all for your sharing. Calvin, your handy script implements Gopal idea could be very useful to many of us ;) Ikai, yes, later we could do more testing and sampling, and with Calvin's script, to log the time (and delta with google time) however, this is not a problem frequently happen and we are currently busying with some other application dev tasks, guess it may take some time to log and analyze this problem, and would keep you posted. just thinking what kinds of 'examples' we could provide? (you want log? data in DS? script to produce the log / data? ) btw, how could we explain Brett's data pipelines' talk at 35:50 or i interpret wrongly? and i believe Gopal, Calvin faced time sync issues and thus have the idea and implementation to solve this (just me recently aware about this problem). how much time is off did you experience? thx a lot! - eric On 4 March 2011 03:12, Ikai Lan (Google) wrote: > Can you provide examples where the time is off by 30 minutes? This seems > incorrect. > > There's definitely clock skew, but we're talking milliseconds, seconds or > at worst, worst, worst maybe a minute or two. > > Ikai Lan > Developer Programs Engineer, Google App Engine > Blog: http://googleappengine.blogspot.com > Twitter: http://twitter.com/app_engine > Reddit: http://www.reddit.com/r/appengine > > > > On Thu, Mar 3, 2011 at 5:54 AM, Ng Ka Ka Eric wrote: > >> Hi Raymond, >> >> Just watched the video. Thx for pointing this reference. >> >> >> I really didnt expect that the machines are not time synced (can be off >> for 40mins?!) are there any technical difficulties to sync them? And in this >> case, for whatever reason if we want to record the time when the request is >> made into DS, how can we do that? >> >> - eric >> >> Sent from my iPhone >> >> On 2011年3月3日, at 下午6:20, "Raymond C." wrote: >> >> According to Brett's data pipelines' talk ( >> http://www.youtube.com/watch?v=zSDC_TU7rtc, around 35:50), its not >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: can we assume time is sync across all servers?
Can you provide examples where the time is off by 30 minutes? This seems incorrect. There's definitely clock skew, but we're talking milliseconds, seconds or at worst, worst, worst maybe a minute or two. Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine On Thu, Mar 3, 2011 at 5:54 AM, Ng Ka Ka Eric wrote: > Hi Raymond, > > Just watched the video. Thx for pointing this reference. > > > I really didnt expect that the machines are not time synced (can be off for > 40mins?!) are there any technical difficulties to sync them? And in this > case, for whatever reason if we want to record the time when the request is > made into DS, how can we do that? > > - eric > > Sent from my iPhone > > On 2011年3月3日, at 下午6:20, "Raymond C." wrote: > > According to Brett's data pipelines' talk ( > http://www.youtube.com/watch?v=zSDC_TU7rtc, around 35:50), its not > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: can we assume time is sync across all servers?
I wrote this little gist to try and solve the issue you're experiencing: https://gist.github.com/836228 It needs a lot of testing, and assumes that the time is synced on the servers google uses to serve robots.txt. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: can we assume time is sync across all servers?
urlfetch google.com and check date header... ;) On Thu, Mar 3, 2011 at 7:24 PM, Ng Ka Ka Eric wrote: > Hi Raymond, > Just watched the video. Thx for pointing this reference. > > I really didnt expect that the machines are not time synced (can be off for > 40mins?!) are there any technical difficulties to sync them? And in this > case, for whatever reason if we want to record the time when the request is > made into DS, how can we do that? > > - eric > Sent from my iPhone > On 2011年3月3日, at 下午6:20, "Raymond C." wrote: > > According to Brett's data pipelines' talk > (http://www.youtube.com/watch?v=zSDC_TU7rtc, around 35:50), its not > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: can we assume time is sync across all servers?
Hi Raymond, Just watched the video. Thx for pointing this reference. I really didnt expect that the machines are not time synced (can be off for 40mins?!) are there any technical difficulties to sync them? And in this case, for whatever reason if we want to record the time when the request is made into DS, how can we do that? - eric Sent from my iPhone On 2011年3月3日, at 下午6:20, "Raymond C." wrote: > According to Brett's data pipelines' talk > (http://www.youtube.com/watch?v=zSDC_TU7rtc, around 35:50), its not > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.