[appengine-java] Will user service be available for DeferredTask?
Hi, In case, the deferred task need to identify the user who ran the task, will the state of userservice be available for the task being executed? thanks, -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine-java/-/-cAeWEhnW9gJ. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] LocalUnitTesting, setting up queue name??
Hi, How to setup the name of the queue, if the code that uses queue other than default queue?? protected final LocalServiceTestHelper helper = new LocalServiceTestHelper( new LocalUserServiceTestConfig(), new LocalDatastoreServiceTestConfig(), * new LocalTaskQueueTestConfig()* * .setDisableAutoTaskExecution(false)* * .setCallbackClass(* * LocalTaskQueueTestConfig.DeferredTaskCallback.class)* * .setTaskExecutionLatch(latch)*); In the above code snippet, I cannot set the queue name to initialize. Thanks, -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine-java/-/dv2Q4jAOxFgJ. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [google-appengine] Re: CNAME mapping stopped working 25 minutes ago
I have the same problem here, can you help me with this? Domain Name: http://app.dziennik.edu.pl maps to http://dziennikel.appspot.com http://www.dziennik.edu.pl maps to http://dl-cms.appspot.com For some people domain name mapped over in the GA to App Engine application (and the ghs.google.com by CNAME) does not work, while the *.appspot.com domain is reachable. Steps to Reproduce (if applicable): http://app.dziennik.edu.pl doesn't work. http://dziennikel.appspot.com works, so it's not an app engine issue. Can you help me resolve this? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/Bt3eeOjhk7wJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] DNS resolution problem for www domain?
I have a similar problem, my www.dziennik.edu.pl is not reachable by some people, can you help me diagnose this? ; DiG 9.6.-ESV-R5 app.dziennik.edu.pl @ns10.az.pl ;; global options: +cmd ;; Got answer: ;; -HEADER- opcode: QUERY, status: NXDOMAIN, id: 30298 ;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 0 ;; WARNING: recursion requested but not available ;; QUESTION SECTION: ;app.dziennik.edu.pl. IN A ;; ANSWER SECTION: app.dziennik.edu.pl.3600IN CNAME ghs.google.com. ;; AUTHORITY SECTION: . 3600IN SOA ns10.az.pl. admin.az.pl. 2011072100 10800 3600 604800 3600 ;; Query time: 143 msec ;; SERVER: 62.146.113.3#53(62.146.113.3) ;; WHEN: Sat Sep 24 05:08:12 2011 ;; MSG SIZE rcvd: 114 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/8LKdYXnlFKcJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
RE: [google-appengine] DNS resolution problem for www domain?
Who is it not reachable for? Where are they Physically who is their provider? You are Up for me from 23 locations I have access to on 3 continents. From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Janusz Skonieczny Sent: Saturday, September 24, 2011 3:11 AM To: google-appengine@googlegroups.com Subject: Re: [google-appengine] DNS resolution problem for www domain? I have a similar problem, my www.dziennik.edu.pl is not reachable by some people, can you help me diagnose this? ; DiG 9.6.-ESV-R5 app.dziennik.edu.pl @ns10.az.pl ;; global options: +cmd ;; Got answer: ;; -HEADER- opcode: QUERY, status: NXDOMAIN, id: 30298 ;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 0 ;; WARNING: recursion requested but not available ;; QUESTION SECTION: ;app.dziennik.edu.pl. IN A ;; ANSWER SECTION: app.dziennik.edu.pl. 3600IN CNAME ghs.google.com. ;; AUTHORITY SECTION: . 3600IN SOA ns10.az.pl. admin.az.pl. 2011072100 10800 3600 604800 3600 ;; Query time: 143 msec ;; SERVER: 62.146.113.3#53(62.146.113.3) ;; WHEN: Sat Sep 24 05:08:12 2011 ;; MSG SIZE rcvd: 114 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/8LKdYXnlFKcJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: How to throttle or batch transactional updates to an entity group
Yeah it's a pretty in depth topic, and I know I've watched the video several times over. I hope it works for you :) Steve On 11-09-23 07:06 PM, Nikolaj wrote: Steve, I thought I had gotten everything out of that talk but clearly I was sleeping in class! The memcache fork join queue looks very interesting - I'm definitely giving that a try. Nikolaj On Sep 23, 1:00 am, Steve Sherriest...@wasteofpaper.com wrote: Definitely checkout Brett Slatkin's Building high-throughput data pipelines with Google App Engine http://www.google.com/events/io/2010/sessions/high-throughput-data-pi... I/O Session which should give you a fundamental understanding of the caveats involved with batch writing. You have to do a little work to make sure writes are not added to a batch that's already applied (or is otherwise invalid), but Brett explains how to do that with memcache locking. Steve On 11-09-22 07:11 PM, Brandon Wirtz wrote: Start a another datastore that doesn't use the same entity, but uses an incremental system. On Write: In a table with format Writes to do, Entity to Overwrite, Data for Entity 1,Brandon Wirtz, +$120 2,Brandon Wirtz, -$120 3,Brandon Wirtz, +$0.66 Increment Write to do count, and store transaction. If a Read request for the entity comes in: Get Entity Get Latest committed Write To Do Value Check Write Cache Recalculate Value based on Entity Value and All Changes On Batch: For batch size calculate all values, and update entities Store Late todo value as Last committed write. You can now batch as often as every 2 seconds and not hit the entity write limit. -Original Message- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Nikolaj Sent: Thursday, September 22, 2011 3:59 PM To: Google App Engine Subject: [google-appengine] How to throttle or batch transactional updates to an entity group Hi there, I'm developing an application that requires me to create lots of child entities within the same entity group transactionally. The tasks being generated to do this come in unpredictable bursts - sometimes 100/s for an entity group. This is clearly exceeding the 1/s limit for transaction contention and causing some real problems for my poor application. I need a way to either batch the required updates for each entity group so I can run through them serially or otherwise schedule each update to run on a 1/s basis. The ideas I've come up with so far: 1. Use some kind of memcache'd timestamp or counter to schedule or throttle the updates. My question is how to do this in a way that will work with 100's of tasks hungrily looking for their time slot? Should I do a CAS for when the last task was scheduled (would this even work with the contention we're talking about?) or can I use an atomic operation like incr() somehow? 2. Store the work items somewhere and then at some later time query for the work items for an entity group and create the children in serial. I'd like to avoid this, if possible, as it creates many other headaches. My application is already using the taskqueue to great effect. This really seems like a queue problem - if only there was some kind of dynamic subqueue where I could add tasks with a 'namespace' of the entity group. Then entity groups could run in parallel, but the entity group items would still be in serial. Does something like that exist? Any suggestions or experiences on how I can work around this would be appreciated. Many thanks, Nikolaj -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Apps For Domains is a MAJOR failing of AppEngine
In case any of you were following along, the google docs team found a workaround for the viewer not liking .docx files: Begin forwarded message: From: googleappeng...@googlecode.com Subject: Re: Issue 5956 in googleappengine: Google Apps Viewer will not show .docx files served from blobstore Date: September 23, 2011 7:04:12 PM EDT To: joshuaesm...@charter.net Reply-To: codesite-nore...@google.com Comment #3 on issue 5956 by slang...@google.com: Google Apps Viewer will not show .docx files served from blobstore http://code.google.com/p/googleappengine/issues/detail?id=5956 Viewer team has found the root cause and will work on a fix. In the meantime they have provided advice on a workaround. Append the filetype extension as an escaped query param in their download url that they pass to us. E.g., %26.docx for docx -- You received this message because you starred the issue. You may adjust your notification preferences at: https://code.google.com/hosting/settings On Sep 22, 2011, at 5:41 PM, Stuart Langley wrote: I'll point it at the right people. Thanks. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/_3RPsFgBeNoJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] DNS resolution problem for www domain?
All of them in Poland, here is some info i have collected. I'm sorry in
advance for the quality of it I'm a network newbe, and I do not
have physical to those machines, I asked some people to run a couple
commands on their boxes.
#
# ping dziennik.edu.pl
#
Pinging dziennik.edu.pl [62.146.68.215] with 32 bytes of data:
Reply from 62.146.68.215: bytes=32 time=66ms TTL=48
Reply from 62.146.68.215: bytes=32 time=66ms TTL=48
Reply from 62.146.68.215: bytes=32 time=65ms TTL=48
Reply from 62.146.68.215: bytes=32 time=65ms TTL=48
Ping statistics for 62.146.68.215:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 65ms, Maximum = 66ms, Average = 65ms
#
# ping app.dziennik.edu.pl
#
Ping request could not find host app.dziennik.edu.pl. Please check the name and
try again.
#
# ping ghs.google.com
#
Pinging ghs.l.google.com [74.125.79.121] with 32 bytes of data:
Reply from 74.125.79.121: bytes=32 time=40ms TTL=51
Reply from 74.125.79.121: bytes=32 time=41ms TTL=52
Reply from 74.125.79.121: bytes=32 time=42ms TTL=52
Reply from 74.125.79.121: bytes=32 time=41ms TTL=52
Ping statistics for 74.125.79.121:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 40ms, Maximum = 42ms, Average = 41ms
#
# ipconfig /all
#
Windows IP Configuration
Host Name . . . . . . . . . . . . : Home1
Primary Dns Suffix . . . . . . . :
Node Type . . . . . . . . . . . . : Hybrid
IP Routing Enabled. . . . . . . . : No
WINS Proxy Enabled. . . . . . . . : No
DNS Suffix Search List. . . . . . : chello.pl
Ethernet adapter Local Area Connection 2:
Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : TAP-Win32 Adapter V9
Physical Address. . . . . . . . . : 00-FF-E6-93-7F-5F
DHCP Enabled. . . . . . . . . . . : Yes
Autoconfiguration Enabled . . . . : Yes
Ethernet adapter Local Area Connection:
Connection-specific DNS Suffix . : chello.pl
Description . . . . . . . . . . . : Realtek PCIe GBE Family Controller
Physical Address. . . . . . . . . : E0-CB-4E-81-94-81
DHCP Enabled. . . . . . . . . . . : Yes
Autoconfiguration Enabled . . . . : Yes
Link-local IPv6 Address . . . . . : fe80::29ea:ea5b:c308:5fb3%11(Preferred)
IPv4 Address. . . . . . . . . . . : 87.207.162.187(Preferred)
Subnet Mask . . . . . . . . . . . : 255.255.254.0
Lease Obtained. . . . . . . . . . : 24 wrze˜nia 2011 09:33:25
Lease Expires . . . . . . . . . . : 30 wrze˜nia 2011 06:09:11
Default Gateway . . . . . . . . . : 87.207.162.1
DHCP Server . . . . . . . . . . . : 10.131.40.1
DHCPv6 IAID . . . . . . . . . . . : 249613134
DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-13-77-89-56-E0-CB-4E-81-94-81
DNS Servers . . . . . . . . . . . : 62.179.1.63
62.179.1.62
NetBIOS over Tcpip. . . . . . . . : Enabled
Tunnel adapter isatap.chello.pl:
Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . : chello.pl
Description . . . . . . . . . . . : Microsoft ISATAP Adapter
Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes
Tunnel adapter Teredo Tunneling Pseudo-Interface:
Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Teredo Tunneling Pseudo-Interface
Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes
IPv6 Address. . . . . . . . . . . :
2001:0:5ef5:79fd:14aa:2509:a830:5d44(Preferred)
Link-local IPv6 Address . . . . . : fe80::14aa:2509:a830:5d44%12(Preferred)
Default Gateway . . . . . . . . . :
NetBIOS over Tcpip. . . . . . . . : Disabled
Tunnel adapter Reusable Microsoft 6To4 Adapter:
Connection-specific DNS Suffix . : chello.pl
Description . . . . . . . . . . . : Microsoft 6to4 Adapter #2
Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes
IPv6 Address. . . . . . . . . . . : 2002:57cf:a2bb::57cf:a2bb(Preferred)
Default Gateway . . . . . . . . . : 2002:c058:6301::c058:6301
DNS Servers . . . . . . . . . . . : 62.179.1.63
62.179.1.62
NetBIOS over Tcpip. . . . . . . . : Disabled
Tunnel adapter isatap.{E6937F5F-F54B-452B-81C8-ED551FB63D45}:
Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Microsoft ISATAP Adapter #2
Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes
#
# ipconfig
[google-appengine] Re: API call datastore_v3.Put() required more quota ?
Any official engineer present on this group ? On Sep 14, 7:21 am, @zghanv/- azgha...@gmail.com wrote: I'm trying to delete testing data from my app from datastore admin ... but it's giving following error. My High Replication Data is 100%, now i want to reset all data. How can i remove that. Thanks. --- Delete Job Status There was a problem kicking off the jobs. The error was: The API call datastore_v3.Put() required more quota than is available. --- Dashboard status ... CPU Time 30% 1.95 of 6.50 CPU hours Outgoing Bandwidth 2% 0.02 of 1.00 GBytes Incoming Bandwidth 0% 0.00 of 1.00 GBytes Total Stored Data 0% 0.00 of 1.00 GBytes Recipients Emailed 0% 0 of 2,000 High Replication Data 100% 0.50 of 0.50 GBytes This resource is currently experiencing a short-term quota limit. Backend Usage 0% $0.00 of $0.72 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
RE: [google-appengine] Re: API call datastore_v3.Put() required more quota ?
Sleep in non-GAE Environments says Do other things and check back on me in X time With instance Hour billing Sleep would only save you money if you have concurrency to handle multiple tasks, and this freed CPU Cycles, AND you weren't sleeping longer than the thing you were waiting to have happen. From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Gerald Tan Sent: Friday, September 16, 2011 9:51 AM To: google-appengine@googlegroups.com Subject: [google-appengine] Re: API call datastore_v3.Put() required more quota ? From my experience (at least with Java) I believe sleep() continues to eat cpu time despite the cpu not really doing anything. My cpu times skyrocketed when I used sleep() with long-held http connections to implement http push back before channel api was available. I recommend breaking the work into taskqueue tasks instead. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/vJC56brjHYwJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] public, dynamic backends are magical (in a good way!!)
hey guys--
If you're using Tasks and haven't tried public, dynamic Backends yet,
they're wonderful:
- for tiny apps, you can offload another $0.72/day for free (they're
dynamic!)
- for medium apps, it's a super easy way to get more RAM
- easy migration from Tasks running on frontends to running them on
public, dynamic backends (incl development, debugging, etc.)
- nearly identical debugging/production methods as frontend Tasks
- since they're public, you can launch Tasks directly on the backend,
removing the frontend from the equation, e.g. you can test out how a
Task will perform on a backend, before committing to the architecture
change.
- cool trick: keep the fast Tasks running on frontends, and use the
(expensive) backends for RAM-intensive jobs-- this is a tiny code
change, and also makes it easier to debug large jobs because the
console isn't flooded with little jobs. For my app, I use Tasks for
various indexing jobs in an online marketplace-- now, I send the
large sellers with 1000s of SKUs, to a B4 backend.
GAE team:
- I've noticed that my backends can 500-error but not produce a stack
trace in the logs? are other people seeing this?
my choice of migration: (python)
1. create backends.xml, define *one* backend, e.g.
backends:
- name: backend2647324# since it's public, give it a hard-to-
guess name
class: B4# big+fast without blowing the free budget
options: dynamic, public
2. add backend-enabled queued, by copying queues from frontends:
queue:
- name: image-processor
...other options here...
- name: image-processor-backend # note how I append -backend
to the name
target: backend2647324
...other options here...
3. in your code, start firing Tasks at the backend
# see bottom for taskname() trick
taskqueue.add(url=..., name=taskname(some-identifier-i-will-
recognize),
queue_name=image-thumbnailer+(-backend if
is_gigantic_image(...) else ))
4. modify your deploy script to update both the front backends.
Mine (legacy) is in perl:
# deploy bar == deploys to foo-bar and to foo-bar-backend
# deploy bar-backend == deploys to foo-bar-backend only
# deploy bar-frontend == deploys to foo-bar only
$DEST = pop(@ARGV);
my $push_frontend = 1;
if ($DEST =~ s/-backend//) { $push_frontend = 0; }
my $push_backend = 1;
if ($DEST =~ s/-frontend//) { $push_backend = 0; }
if ($push_frontend) {
open(FH, echo .$ENV{GAEPASSWD}.|python2.5 .$ENV{GAEDIR}./
appcfg.py --email=.$ENV{GAEUSER}. --passin --application=foo-$DEST
update .|);
while (FH) { print; }
close FH;
}
if ($push_backend) {
open(FH, echo .$ENV{GAEPASSWD}.|python2.5 .$ENV{GAEDIR}./
appcfg.py --email=.$ENV{GAEUSER}. --passin --application=foo-$DEST
backends . update|);
while (FH) { print; }
close FH;
}
that's it! now, some jobs will go to the frontends and others to the
backends.
hope this helps,
adam
def taskname(string):
safely create Google App Engine Task names.
# add timestamp and random for ultimate uniqueness, strip disallowed
chars
rnd_salt = str(random.randint(10, 99))
return re.sub(r'[^a-zA-Z0-9-]', '-', string +
str(datetime.datetime.now())) + - + rnd_salt
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: calling out the app engine team on ssl for custom domains
Jeff - thanks for the link to Cloudflare. It certainly seems like an interesting option. The risk with implementing them is that it's one more layer that can fail. Does anyone else have feedback about how well it works?Also, does anyone have thoughts on whether the other benefits of Cloudflare (edge-caching and security layer) is relevant to Appengine users, is it essentially redundant? On Sep 23, 2:50 am, Jeff Schnitzer j...@infohazard.org wrote: Try this out: http://blog.cloudflare.com/ssl-on-custom-domains-for-appengine-and-other When our business is ready to launch, this is our intended plan for always-SSL for our app. Cloudflare does edge caching for all content too - possibly better than Google's (wholly undocumented and not guaranteed) edge cache. I hate to plug a solution that I haven't tried yet, but it looks promising. Try it out and let me know, or you can wait for my report towards the end of the year. Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: calling out the app engine team on ssl for custom domains
I too wonder about the availability of cloud fare. I'd like to find out how it goes for you if you end up using it. Steve On 11-09-24 02:48 PM, johnP wrote: Jeff - thanks for the link to Cloudflare. It certainly seems like an interesting option. The risk with implementing them is that it's one more layer that can fail. Does anyone else have feedback about how well it works?Also, does anyone have thoughts on whether the other benefits of Cloudflare (edge-caching and security layer) is relevant to Appengine users, is it essentially redundant? On Sep 23, 2:50 am, Jeff Schnitzerj...@infohazard.org wrote: Try this out: http://blog.cloudflare.com/ssl-on-custom-domains-for-appengine-and-other When our business is ready to launch, this is our intended plan for always-SSL for our app. Cloudflare does edge caching for all content too - possibly better than Google's (wholly undocumented and not guaranteed) edge cache. I hate to plug a solution that I haven't tried yet, but it looks promising. Try it out and let me know, or you can wait for my report towards the end of the year. Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
RE: [google-appengine] Re: calling out the app engine team on ssl for custom domains
You will be most unhappy. Many of my SEO clients are FORMER cloudflare users who found out the hard way that when Cloudflare mistakes Google Bot for a DDoS Attack they get delisted, or when one of Google's Human Quality validation techs visit the site and get a captcha challenge they get Delisted. Cloudflare also has issues that because adult sites often use it, that they get blocked by large organizations firewall. -Original Message- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Steve Sherrie Sent: Saturday, September 24, 2011 11:51 AM To: google-appengine@googlegroups.com Subject: Re: [google-appengine] Re: calling out the app engine team on ssl for custom domains I too wonder about the availability of cloud fare. I'd like to find out how it goes for you if you end up using it. Steve On 11-09-24 02:48 PM, johnP wrote: Jeff - thanks for the link to Cloudflare. It certainly seems like an interesting option. The risk with implementing them is that it's one more layer that can fail. Does anyone else have feedback about how well it works?Also, does anyone have thoughts on whether the other benefits of Cloudflare (edge-caching and security layer) is relevant to Appengine users, is it essentially redundant? On Sep 23, 2:50 am, Jeff Schnitzerj...@infohazard.org wrote: Try this out: http://blog.cloudflare.com/ssl-on-custom-domains-for-appengine-and-ot her When our business is ready to launch, this is our intended plan for always-SSL for our app. Cloudflare does edge caching for all content too - possibly better than Google's (wholly undocumented and not guaranteed) edge cache. I hate to plug a solution that I haven't tried yet, but it looks promising. Try it out and let me know, or you can wait for my report towards the end of the year. Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: calling out the app engine team on ssl for custom domains
There ya go then. Steve On 11-09-24 03:31 PM, Brandon Wirtz wrote: You will be most unhappy. Many of my SEO clients are FORMER cloudflare users who found out the hard way that when Cloudflare mistakes Google Bot for a DDoS Attack they get delisted, or when one of Google's Human Quality validation techs visit the site and get a captcha challenge they get Delisted. Cloudflare also has issues that because adult sites often use it, that they get blocked by large organizations firewall. -Original Message- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Steve Sherrie Sent: Saturday, September 24, 2011 11:51 AM To: google-appengine@googlegroups.com Subject: Re: [google-appengine] Re: calling out the app engine team on ssl for custom domains I too wonder about the availability of cloud fare. I'd like to find out how it goes for you if you end up using it. Steve On 11-09-24 02:48 PM, johnP wrote: Jeff - thanks for the link to Cloudflare. It certainly seems like an interesting option. The risk with implementing them is that it's one more layer that can fail. Does anyone else have feedback about how well it works?Also, does anyone have thoughts on whether the other benefits of Cloudflare (edge-caching and security layer) is relevant to Appengine users, is it essentially redundant? On Sep 23, 2:50 am, Jeff Schnitzerj...@infohazard.org wrote: Try this out: http://blog.cloudflare.com/ssl-on-custom-domains-for-appengine-and-ot her When our business is ready to launch, this is our intended plan for always-SSL for our app. Cloudflare does edge caching for all content too - possibly better than Google's (wholly undocumented and not guaranteed) edge cache. I hate to plug a solution that I haven't tried yet, but it looks promising. Try it out and let me know, or you can wait for my report towards the end of the year. Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Write Ops incurred during Model.put()
Yes, the write ops for changing an entity depends on how many indexed properties you change. It is: 1 op for the entity 2 ops for each changed index value (1 to remove the old value and 1 to add the new) Since it is not a fixed value and depends on what you actually modify, we do not (can not) show it in the dev_appserver's Datastore Viewer. On Tue, Sep 20, 2011 at 11:09 AM, Alex Epshteyn alexander.epsht...@gmail.com wrote: So the Write Ops value in the new dev_appserver's Datastore Viewer applies only to creating new entities of a kind, updating the entity i is presumably much cheaper if only a few properties are changed? On Tue, Sep 20, 2011 at 12:27 PM, Simon Knott knott.si...@gmail.com wrote: Hi, According to Alfred, who's a Googler who appears to know lots about the datastore, only the updated values cause index write operations - see https://groups.google.com/d/msg/google-appengine/mjnSqQWOfqU/cgPVeHbrR8oJ for more info. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/OcSq1WpF5noJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Write Ops incurred during Model.put()
Thanks for the definitive clarification, Alfred. It would be awesome if you guys could show the actual Read/Write Ops incurred by each URI under the Current Load section of the live dashboard. On Sat, Sep 24, 2011 at 4:00 PM, Alfred Fuller arfuller+appeng...@google.com wrote: Yes, the write ops for changing an entity depends on how many indexed properties you change. It is: 1 op for the entity 2 ops for each changed index value (1 to remove the old value and 1 to add the new) Since it is not a fixed value and depends on what you actually modify, we do not (can not) show it in the dev_appserver's Datastore Viewer. On Tue, Sep 20, 2011 at 11:09 AM, Alex Epshteyn alexander.epsht...@gmail.com wrote: So the Write Ops value in the new dev_appserver's Datastore Viewer applies only to creating new entities of a kind, updating the entity i is presumably much cheaper if only a few properties are changed? On Tue, Sep 20, 2011 at 12:27 PM, Simon Knott knott.si...@gmail.com wrote: Hi, According to Alfred, who's a Googler who appears to know lots about the datastore, only the updated values cause index write operations - see https://groups.google.com/d/msg/google-appengine/mjnSqQWOfqU/cgPVeHbrR8oJ for more info. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/OcSq1WpF5noJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] public, dynamic backends are magical (in a good way!!)
I wish i could +1 on postings like this ! Gonna try it out tomorrow ;]
Thnx.
Bart Thate
programming schizofrenic - till freedom come!
[!] http://protocol.by/botfather
[!] http://botfather.blogspot.com
[!] http://jsonbot.org
On Sat, Sep 24, 2011 at 8:23 PM, Adam Sah adam@gmail.com wrote:
hey guys--
If you're using Tasks and haven't tried public, dynamic Backends yet,
they're wonderful:
- for tiny apps, you can offload another $0.72/day for free (they're
dynamic!)
- for medium apps, it's a super easy way to get more RAM
- easy migration from Tasks running on frontends to running them on
public, dynamic backends (incl development, debugging, etc.)
- nearly identical debugging/production methods as frontend Tasks
- since they're public, you can launch Tasks directly on the backend,
removing the frontend from the equation, e.g. you can test out how a
Task will perform on a backend, before committing to the architecture
change.
- cool trick: keep the fast Tasks running on frontends, and use the
(expensive) backends for RAM-intensive jobs-- this is a tiny code
change, and also makes it easier to debug large jobs because the
console isn't flooded with little jobs. For my app, I use Tasks for
various indexing jobs in an online marketplace-- now, I send the
large sellers with 1000s of SKUs, to a B4 backend.
GAE team:
- I've noticed that my backends can 500-error but not produce a stack
trace in the logs? are other people seeing this?
my choice of migration: (python)
1. create backends.xml, define *one* backend, e.g.
backends:
- name: backend2647324# since it's public, give it a hard-to-
guess name
class: B4# big+fast without blowing the free budget
options: dynamic, public
2. add backend-enabled queued, by copying queues from frontends:
queue:
- name: image-processor
...other options here...
- name: image-processor-backend # note how I append -backend
to the name
target: backend2647324
...other options here...
3. in your code, start firing Tasks at the backend
# see bottom for taskname() trick
taskqueue.add(url=..., name=taskname(some-identifier-i-will-
recognize),
queue_name=image-thumbnailer+(-backend if
is_gigantic_image(...) else ))
4. modify your deploy script to update both the front backends.
Mine (legacy) is in perl:
# deploy bar == deploys to foo-bar and to foo-bar-backend
# deploy bar-backend == deploys to foo-bar-backend only
# deploy bar-frontend == deploys to foo-bar only
$DEST = pop(@ARGV);
my $push_frontend = 1;
if ($DEST =~ s/-backend//) { $push_frontend = 0; }
my $push_backend = 1;
if ($DEST =~ s/-frontend//) { $push_backend = 0; }
if ($push_frontend) {
open(FH, echo .$ENV{GAEPASSWD}.|python2.5 .$ENV{GAEDIR}./
appcfg.py --email=.$ENV{GAEUSER}. --passin --application=foo-$DEST
update .|);
while (FH) { print; }
close FH;
}
if ($push_backend) {
open(FH, echo .$ENV{GAEPASSWD}.|python2.5 .$ENV{GAEDIR}./
appcfg.py --email=.$ENV{GAEUSER}. --passin --application=foo-$DEST
backends . update|);
while (FH) { print; }
close FH;
}
that's it! now, some jobs will go to the frontends and others to the
backends.
hope this helps,
adam
def taskname(string):
safely create Google App Engine Task names.
# add timestamp and random for ultimate uniqueness, strip disallowed
chars
rnd_salt = str(random.randint(10, 99))
return re.sub(r'[^a-zA-Z0-9-]', '-', string +
str(datetime.datetime.now())) + - + rnd_salt
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Typo in Official GAE Pricing Page
Hey Gregory, While you are fixing that page please divide all prices by ten, so I can come back to GAE :) On Sep 24, 12:39 am, Gregory D'alesandre gr...@google.com wrote: Thank you for highlighting, it is indeed inaccurate on the external page and we are in the process of fixing it! Greg On Sep 23, 2011 7:40 PM, Albert albertpa...@gmail.com wrote: There's a possible typo in the Official GAE Pricing Page. In my Billing History, it says... Datastore Writes: $1.00/Million Ops Datastore Reads: $0.70/Million Ops Small Datastore Operations: $0.10/Million Ops That looks correct, but in the official pricing page here http://www.google.com/enterprise/cloud/appengine/pricing.html, it says... Datastore API $0.01/10k write ops $0.07/10k read ops $0.01/100k small ops $0.07/10k read ops. That would translate to $7.00 / Million Read Ops. That can't be right. Please check. Thanks! -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] CloudFlare https traffic blocked?
Hi, We're trying CloudFlare to tide us over until Google enables SSL for custom domains. I'm open to suggestions, but so far this seems to be the simplest solution to get ready for Facebook's October 1st https requirement for canvas applications. For some reason, it seems traffic is being blocked when coming through CloudFlare over port 443. Going directly to our appspot domain works fine. The folks at CloudFlare don't seem to be aware of any issues - is anyone here able to confirm whether this still works, or perhaps even give a better alternative (for various reasons, Facebook must be pointing to our custom domain)? Thanks, Owen Wiggins Co-Founder, Inspirado Games -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/3GJGNZEq3PsJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] CloudFlare https traffic blocked?
Thank you for your quick reply, Brandon! I believe CloudFlare uses the industry standard X-Forwarded-For header: http://code.google.com/appengine/forum/?place=msg%2Fgoogle-appengine%2F4D1IGqCh4LA%2FrguR4Zqtj08J We're not seeing excessive traffic this week either - not sure what would be considered a possible DDoS by Google, though. Regardless, I'm really in a tough spot here and would love an alternative if anyone can suggest something. I'm sure many other developers are in a similar position... -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/cRtKFojKcBMJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
RE: [google-appengine] CloudFlare https traffic blocked?
No, they say that, but many of their headers have private IP ranges, reciprocal IP's, and other issues. They have a hacked together product running on borrowed infrastructure, I make a LOT of money cleaning up after them. And I feel bad because most the people who end up getting burned by them can't afford it. From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Owen Wiggins Sent: Saturday, September 24, 2011 8:44 PM To: google-appengine@googlegroups.com Subject: Re: [google-appengine] CloudFlare https traffic blocked? Thank you for your quick reply, Brandon! I believe CloudFlare uses the industry standard X-Forwarded-For header: http://code.google.com/appengine/forum/?place=msg%2Fgoogle-appengine%2F4D1IG qCh4LA%2FrguR4Zqtj08J We're not seeing excessive traffic this week either - not sure what would be considered a possible DDoS by Google, though. Regardless, I'm really in a tough spot here and would love an alternative if anyone can suggest something. I'm sure many other developers are in a similar position... -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/cRtKFojKcBMJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] CloudFlare https traffic blocked?
Thanks, Brandon. What I'm really after is a definitive answer on whether there's an issue on the AppEngine side. I'm currently working with CloudFlare support to resolve the issue. However, the $22 I've spent doesn't have me committed either way - I'm open to any solutions or alternatives. The only reason I'm trying CloudFlare is to get SSL working until Google has a permanent solution ready. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/KK75kBAyEloJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
RE: [google-appengine] CloudFlare https traffic blocked?
CF always blames GAE, but none of my sites that are proxied have this issue because My headers are right. BTW Using CF as your SSL Solution opens a Whole host of issues, because of who you share resources with and CFs Less then optimal policies on security. From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Owen Wiggins Sent: Saturday, September 24, 2011 9:19 PM To: google-appengine@googlegroups.com Subject: Re: [google-appengine] CloudFlare https traffic blocked? Thanks, Brandon. What I'm really after is a definitive answer on whether there's an issue on the AppEngine side. I'm currently working with CloudFlare support to resolve the issue. However, the $22 I've spent doesn't have me committed either way - I'm open to any solutions or alternatives. The only reason I'm trying CloudFlare is to get SSL working until Google has a permanent solution ready. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/KK75kBAyEloJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
