[google-appengine] Where to submit BigQuery performance issues?
Hi, Today BigQuery is taking far more time to complete query jobs than usual. Two days ago a query of my system used to be executed within 6 seconds, today it is taking from 1 to 6 minutes (without any increase of the dataset). The BigQuery Google Groups https://groups.google.com/forum/#!forum/bigquery-discuss is disabled for new posts, StackOverflow is just not suitable for this kind of questions, the BigQuery announce https://groups.google.com/forum/#!forum/bigquery-announce and BigQuery Downtime Notify https://groups.google.com/forum/#!forum/bigquery-downtime-notify groups are read-only. I'm posting here to the App Engine groups because I didn't find the right place to go. Where should I post about it and get answers? Thanks. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/d/optout.
Re: [google-appengine] Re: Private API with Cloud Endpoints
Hi Less, Thanks again for your answer. It took me a few hours but I read the whole tutorial on the blog post and tried to work with it, including its link about Obtaining an access token (http://developer.android.com/google/auth/http-auth.html#obtain). If I leave the code as it is, it does require a google account to work. However since you told me it was possible to do this without this requirement, I tried to determine where is located the google account requirement in the code. My belief is that it is in the pickUserAccount() and more specifically on the use of newChooseAccountIntent. It seems like it is possible to provide it an Account object that could be my own login system, and thus in the end it would be possible to avoid google accounts. However could you please confirm that's what you were talking about in your last post? Le mercredi 17 décembre 2014 19:34:25 UTC+1, Les Vogel a écrit : Kevin, The blog post http://android-developers.blogspot.com/2013/01/verifying-back-end-calls-from-android.html allows things to work w/o you users logging in, it's just a bit complex. I would strongly suggest you look at that. On Wed, Dec 17, 2014 at 4:26 AM, Gannicus kevin.k...@gmail.com javascript: wrote: Hi Les, First of all thank you very much for your answers and your help. Like you said, security depends on how much time/money... I want to spend. For now as a beginner I just want to use a minimal protection. I will start things off by studying your links about @clientsIds and @Audiences. However the first link ( http://android-developers.blogspot.fr/2013/01/verifying-back-end-calls-from-android.html) mentions Google accounts: It turns out that Google Play services http://developer.android.com/google/play-services/index.html, now available on every compatible device running Android release 2.2 or higher, offers a good solution to this problem, based on the use of Google Accounts, Hopefully it's not necessary to ask my users for their google accounts. I will try it anyway, or else I will just create my own token system. Le mardi 16 décembre 2014 00:11:23 UTC+1, Les Vogel a écrit : Hi Kevin, I was reminded of the following today: http://android-developers.blogspot.com/2013/01/verifying- back-end-calls-from-android.html In cloud endpoints, you can use @clientIds and @audiences to whitelist your client ids. https://cloud.google.com/appengine/docs/java/endpoints/auth It should lock things down for you. Les On Mon, Dec 15, 2014 at 10:58 AM, Les Vogel le...@google.com wrote: Security is a lot of things to a lot of people. For this, we can think about it as raising the cost for your adversary, while trying to stay within your cost of development (skill, experience, time to market, etc.) A determined, well funded adversary can usually bypass / break most security systems. There are lots of papers on this available on the net -- search for security cost benefit analysis You know your app, and what you are trying to secure, so, it's really up to you to decide what is the appropriate method to secure your api. I'm trying to suggest a range of things, they each have different cost / benefits. The constant token method is fairly simple, and as you pointed out, vulnerable to a decompilation attack. You can improve it by replacing the using of a single constant that can be easily found by decompiling the code, uses several values that must be sent through a formula to get the correct token value. Google Endpoints used with Google accounts (including Google for Business / Education / Government accounts) provides good security easily -- it didn't sound like this met your needs. We have the Google Identity Toolkit https://developers.google.com/identity-toolkit/, which will allow you to securely login your users, it also supports Google, Yahoo, AOL, Microsoft, Facebook, and Paypal logins as well as your own. In the Abelana-gcp https://github.com/GoogleCloudPlatform/abelana-gcp and Abelana-android https://github.com/GoogleCloudPlatform/abelana-android apps, I used the Identity Toolkit for user identity, then rolled by own access tokens that I used for all but one api (the login one). (My understanding is that direct support for access tokens is coming) The code is in Go for AppEngine and Java for the Android app. You might also take a method used for XCRF prevention that involves encrypting a random # and a expiration time on the first server reply, then requiring that value to be passed on all subsequent requests. It's a bit more work and requires storing the value on the server side. The last suggestion I gave had a link for verifying a certificate -- looking at it today it appears that AppEngine is providing id to your client, not the other way around. I'll look around to see if there is a way to ask TLS to bring the client certificate info to the API. But
Re: [google-appengine] Re: Private API with Cloud Endpoints
Hi Les, Thanks again for your answer. It took me a few hours but I read the whole tutorial on the blog post and tried to work with it, including its link about Obtaining an access token ( http://developer.android.com/google/auth/http-auth.html#obtain). If I leave the code as it is, it does require a google account to work. However since you told me it was possible to do this without this requirement, I tried to determine where is located the google account requirement in the code. My belief is that it is in the pickUserAccount() and more specifically on the use of newChooseAccountIntent. It seems like it is possible to provide it an Account object that could be my own login system, and thus in the end it would be possible to avoid google accounts. However could you please confirm that's what you were talking about in your last post? Le mercredi 17 décembre 2014 19:34:25 UTC+1, Les Vogel a écrit : Kevin, The blog post http://android-developers.blogspot.com/2013/01/verifying-back-end-calls-from-android.html allows things to work w/o you users logging in, it's just a bit complex. I would strongly suggest you look at that. On Wed, Dec 17, 2014 at 4:26 AM, Gannicus kevin.k...@gmail.com javascript: wrote: Hi Les, First of all thank you very much for your answers and your help. Like you said, security depends on how much time/money... I want to spend. For now as a beginner I just want to use a minimal protection. I will start things off by studying your links about @clientsIds and @Audiences. However the first link ( http://android-developers.blogspot.fr/2013/01/verifying-back-end-calls-from-android.html) mentions Google accounts: It turns out that Google Play services http://developer.android.com/google/play-services/index.html, now available on every compatible device running Android release 2.2 or higher, offers a good solution to this problem, based on the use of Google Accounts, Hopefully it's not necessary to ask my users for their google accounts. I will try it anyway, or else I will just create my own token system. Le mardi 16 décembre 2014 00:11:23 UTC+1, Les Vogel a écrit : Hi Kevin, I was reminded of the following today: http://android-developers.blogspot.com/2013/01/verifying- back-end-calls-from-android.html In cloud endpoints, you can use @clientIds and @audiences to whitelist your client ids. https://cloud.google.com/appengine/docs/java/endpoints/auth It should lock things down for you. Les On Mon, Dec 15, 2014 at 10:58 AM, Les Vogel le...@google.com wrote: Security is a lot of things to a lot of people. For this, we can think about it as raising the cost for your adversary, while trying to stay within your cost of development (skill, experience, time to market, etc.) A determined, well funded adversary can usually bypass / break most security systems. There are lots of papers on this available on the net -- search for security cost benefit analysis You know your app, and what you are trying to secure, so, it's really up to you to decide what is the appropriate method to secure your api. I'm trying to suggest a range of things, they each have different cost / benefits. The constant token method is fairly simple, and as you pointed out, vulnerable to a decompilation attack. You can improve it by replacing the using of a single constant that can be easily found by decompiling the code, uses several values that must be sent through a formula to get the correct token value. Google Endpoints used with Google accounts (including Google for Business / Education / Government accounts) provides good security easily -- it didn't sound like this met your needs. We have the Google Identity Toolkit https://developers.google.com/identity-toolkit/, which will allow you to securely login your users, it also supports Google, Yahoo, AOL, Microsoft, Facebook, and Paypal logins as well as your own. In the Abelana-gcp https://github.com/GoogleCloudPlatform/abelana-gcp and Abelana-android https://github.com/GoogleCloudPlatform/abelana-android apps, I used the Identity Toolkit for user identity, then rolled by own access tokens that I used for all but one api (the login one). (My understanding is that direct support for access tokens is coming) The code is in Go for AppEngine and Java for the Android app. You might also take a method used for XCRF prevention that involves encrypting a random # and a expiration time on the first server reply, then requiring that value to be passed on all subsequent requests. It's a bit more work and requires storing the value on the server side. The last suggestion I gave had a link for verifying a certificate -- looking at it today it appears that AppEngine is providing id to your client, not the other way around. I'll look around to see if there is a way to ask TLS to bring the client certificate info to the API. But
Re: [google-appengine] Re: Private API with Cloud Endpoints
Hi Kevin, I'll write a longer answer shortly, however, if you could send me lesv@g... your code, I'll take a peek. On Thu, Dec 18, 2014 at 9:52 AM, Gannicus kevin.kaido...@gmail.com wrote: Hi Les, Thanks again for your answer. It took me a few hours but I read the whole tutorial on the blog post and tried to work with it, including its link about Obtaining an access token (http://developer.android.com/ google/auth/http-auth.html#obtain). If I leave the code as it is, it does require a google account to work. However since you told me it was possible to do this without this requirement, I tried to determine where is located the google account requirement in the code. My belief is that it is in the pickUserAccount() and more specifically on the use of newChooseAccountIntent. It seems like it is possible to provide it an Account object that could be my own login system, and thus in the end it would be possible to avoid google accounts. However could you please confirm that's what you were talking about in your last post? Le mercredi 17 décembre 2014 19:34:25 UTC+1, Les Vogel a écrit : Kevin, The blog post http://android-developers.blogspot.com/2013/01/verifying-back-end-calls-from-android.html allows things to work w/o you users logging in, it's just a bit complex. I would strongly suggest you look at that. On Wed, Dec 17, 2014 at 4:26 AM, Gannicus kevin.k...@gmail.com wrote: Hi Les, First of all thank you very much for your answers and your help. Like you said, security depends on how much time/money... I want to spend. For now as a beginner I just want to use a minimal protection. I will start things off by studying your links about @clientsIds and @Audiences. However the first link (http://android-developers. blogspot.fr/2013/01/verifying-back-end-calls-from-android.html) mentions Google accounts: It turns out that Google Play services http://developer.android.com/google/play-services/index.html, now available on every compatible device running Android release 2.2 or higher, offers a good solution to this problem, based on the use of Google Accounts, Hopefully it's not necessary to ask my users for their google accounts. I will try it anyway, or else I will just create my own token system. Le mardi 16 décembre 2014 00:11:23 UTC+1, Les Vogel a écrit : Hi Kevin, I was reminded of the following today: http://android-developers.blogspot.com/2013/01/verifying-bac k-end-calls-from-android.html In cloud endpoints, you can use @clientIds and @audiences to whitelist your client ids. https://cloud.google.com/appengine/docs/java/endpoints/auth It should lock things down for you. Les On Mon, Dec 15, 2014 at 10:58 AM, Les Vogel le...@google.com wrote: Security is a lot of things to a lot of people. For this, we can think about it as raising the cost for your adversary, while trying to stay within your cost of development (skill, experience, time to market, etc.) A determined, well funded adversary can usually bypass / break most security systems. There are lots of papers on this available on the net -- search for security cost benefit analysis You know your app, and what you are trying to secure, so, it's really up to you to decide what is the appropriate method to secure your api. I'm trying to suggest a range of things, they each have different cost / benefits. The constant token method is fairly simple, and as you pointed out, vulnerable to a decompilation attack. You can improve it by replacing the using of a single constant that can be easily found by decompiling the code, uses several values that must be sent through a formula to get the correct token value. Google Endpoints used with Google accounts (including Google for Business / Education / Government accounts) provides good security easily -- it didn't sound like this met your needs. We have the Google Identity Toolkit https://developers.google.com/identity-toolkit/, which will allow you to securely login your users, it also supports Google, Yahoo, AOL, Microsoft, Facebook, and Paypal logins as well as your own. In the Abelana-gcp https://github.com/GoogleCloudPlatform/abelana-gcp and Abelana-android https://github.com/GoogleCloudPlatform/abelana-android apps, I used the Identity Toolkit for user identity, then rolled by own access tokens that I used for all but one api (the login one). (My understanding is that direct support for access tokens is coming) The code is in Go for AppEngine and Java for the Android app. You might also take a method used for XCRF prevention that involves encrypting a random # and a expiration time on the first server reply, then requiring that value to be passed on all subsequent requests. It's a bit more work and requires storing the value on the server side. The last suggestion I gave had a link for verifying a certificate -- looking at it today it appears that AppEngine is providing id to your
Re: [google-appengine] Re: Private API with Cloud Endpoints
Hi Kevin, Not sure why you are calling *pickUserAccount*() or the *newChooseAccountIntent* as your use case shouldn't either of those. Please take a look at this paragraph. You’ll need to call the Google Play services GoogleAuthUtil http://developer.android.com/reference/com/google/android/gms/auth/GoogleAuthUtil.html#getToken(android.content.Context, java.lang.String, java.lang.String) class to get an ID token; the procedure is as described inObtaining an Access Token http://developer.android.com/google/play-services/auth.html#obtain. There’s one extra bit of magic: the value of the scope argument to the getToken(email, scope) method. It has to be the string *audience:server:client_id:X*, where X is the Client ID of for the Web app, as described above. If our Client ID were the example value given above, the value of the scope argument would be*audience:server:client_id:9414861317621.apps.googleusercontent.com http://9414861317621.apps.googleusercontent.com*. I've highlighted what you need to do. You get a client id from cloud.google.com/console click on your project, then API's and Auth, then create a client ID for an Android Application, if you don't already have one, using the instructions in the blog post. You may find things work better if you also create a Consent screen, that shouldn't be necessary, but I've seen occasions where I get a better result by doing so. On Thu, Dec 18, 2014 at 1:04 PM, Les Vogel l...@google.com wrote: Hi Kevin, I'll write a longer answer shortly, however, if you could send me lesv@g... your code, I'll take a peek. On Thu, Dec 18, 2014 at 9:52 AM, Gannicus kevin.kaido...@gmail.com wrote: Hi Les, Thanks again for your answer. It took me a few hours but I read the whole tutorial on the blog post and tried to work with it, including its link about Obtaining an access token (http://developer.android.com/ google/auth/http-auth.html#obtain). If I leave the code as it is, it does require a google account to work. However since you told me it was possible to do this without this requirement, I tried to determine where is located the google account requirement in the code. My belief is that it is in the pickUserAccount() and more specifically on the use of newChooseAccountIntent. It seems like it is possible to provide it an Account object that could be my own login system, and thus in the end it would be possible to avoid google accounts. However could you please confirm that's what you were talking about in your last post? Le mercredi 17 décembre 2014 19:34:25 UTC+1, Les Vogel a écrit : Kevin, The blog post http://android-developers.blogspot.com/2013/01/verifying-back-end-calls-from-android.html allows things to work w/o you users logging in, it's just a bit complex. I would strongly suggest you look at that. On Wed, Dec 17, 2014 at 4:26 AM, Gannicus kevin.k...@gmail.com wrote: Hi Les, First of all thank you very much for your answers and your help. Like you said, security depends on how much time/money... I want to spend. For now as a beginner I just want to use a minimal protection. I will start things off by studying your links about @clientsIds and @Audiences. However the first link (http://android-developers. blogspot.fr/2013/01/verifying-back-end-calls-from-android.html) mentions Google accounts: It turns out that Google Play services http://developer.android.com/google/play-services/index.html, now available on every compatible device running Android release 2.2 or higher, offers a good solution to this problem, based on the use of Google Accounts, Hopefully it's not necessary to ask my users for their google accounts. I will try it anyway, or else I will just create my own token system. Le mardi 16 décembre 2014 00:11:23 UTC+1, Les Vogel a écrit : Hi Kevin, I was reminded of the following today: http://android-developers.blogspot.com/2013/01/verifying-bac k-end-calls-from-android.html In cloud endpoints, you can use @clientIds and @audiences to whitelist your client ids. https://cloud.google.com/appengine/docs/java/endpoints/auth It should lock things down for you. Les On Mon, Dec 15, 2014 at 10:58 AM, Les Vogel le...@google.com wrote: Security is a lot of things to a lot of people. For this, we can think about it as raising the cost for your adversary, while trying to stay within your cost of development (skill, experience, time to market, etc.) A determined, well funded adversary can usually bypass / break most security systems. There are lots of papers on this available on the net -- search for security cost benefit analysis You know your app, and what you are trying to secure, so, it's really up to you to decide what is the appropriate method to secure your api. I'm trying to suggest a range of things, they each have different cost / benefits. The constant token method is fairly simple, and as you pointed out, vulnerable to a decompilation attack. You can
[google-appengine] What are my options for transferring data from one project to another?
We're considering starting from scratch our app, but I'd need to keep the data (migrated). Our first option would be just to use a new module/version and being careful with the new model names. But I was wondering if it's possible to just start a new project completely? Since keys are linked to the project app, I guess it's not possible to just import a backup? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/d/optout.
Re: [google-appengine] What are my options for transferring data from one project to another?
On Fri, Dec 19, 2014 at 12:43 AM, John Louis Del Rosario joh...@gmail.com wrote: We're considering starting from scratch our app, but I'd need to keep the data (migrated). Our first option would be just to use a new module/version and being careful with the new model names. But I was wondering if it's possible to just start a new project completely? Since keys are linked to the project app, I guess it's not possible to just import a backup? I've never tried it myself, but apparently the datastore admin can migrate datastore backups between apps: https://cloud.google.com/appengine/docs/adminconsole/datastoreadmin#restoring_data_to_another_app As for the keys the export/import process should handle it, but just in case I'm missing something: how do you generate your datastore keys? Are you letting App Engine issue numerical IDs, setting a custom name, using a helper library, or anything similar? - -Vinny P Technology Media Consultant Chicago, IL App Engine Code Samples: http://www.learntogoogleit.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/d/optout.
Re: [google-appengine] Random blobstore entries produce error 500
On Mon, Dec 15, 2014 at 3:38 PM, Marcin marcin.kury...@innodia.pl wrote: All files are listed in blobviewer only download action causes the error 500. Is it a maintenance or a temporary issue or should I be really worried about my data? If you try to partially download your blobs (see the Blob Byte Ranges section here: https://cloud.google.com/appengine/docs/go/blobstore/#Go_Serving_a_blob) do you get the same 500 error, or do you get the correct data? - -Vinny P Technology Media Consultant Chicago, IL App Engine Code Samples: http://www.learntogoogleit.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/d/optout.
Re: [google-appengine] Re: How to add email alias to app engine?
On Tue, Dec 16, 2014 at 12:36 PM, Abilash Amar abilash.am...@gmail.com wrote: Did you get this to work without creating another google app domain? On Monday, 12 July 2010 18:10:52 UTC+5:30, Hugo Visser wrote: I have an app that is set to apps domain only mode for authentication. Now I want to send some email from that app. It is working when I use one of the administrator accounts, but I like to set up an email alias and I'm failing to get that working. No, you won't get it to work. Google Apps will only create user accounts on the primary domain - you can map an email address on the alias domain to the primary, but the account itself will always be considered to be on the primary domain. In other words, Google Apps won't let you create *user @ alias . com* (if *alias . com* is an alias domain) as a full user account. This means that an alias domain can't configure a valid email account on the domain, and so App Engine will refuse to send email with a from address of a non-valid account. The only workarounds are to: use Sendgrid/Mandrill/etc to send email https://cloud.google.com/appengine/docs/python/mail/sendgrid#send_an_emailor (as you stated) creating a new Google Apps account and registering the alias domain as a primary domain. - -Vinny P Technology Media Consultant Chicago, IL App Engine Code Samples: http://www.learntogoogleit.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/d/optout.
