[google-appengine] Re: SSL for app engine custom domain
Nick, Now, that is good news, therefore, I think I shall hold off on decisions until I see the Custom Domains SSL process. Looking forward to it. Oh, there is something else to, is it possible to disable the .appspot domain and just serve the custom domain, or do I have to implement my own redirect in the code? Regards On Friday, August 14, 2015 at 6:41:48 PM UTC+3, Richard Cheesmar wrote: Hi, Is it going to be possible anytime soon to setup SSL for a custom domain without a Google App account directly from the Google cloud developers console? Signing up for yet another Google account is bordering on making me insane. I have setup a custom domain via the Google cloud developers console for my app project, and this is serving as a naked url...no problem. Then I tried to setup a Google App account with an existing account linked to my Google App Engine project as an owner, but it didn't like it. So I setup a new one (deleted now) and re-verified the custom domain, which then stopped the serving of the custom domain. I then deleted the Google App account and it started serving the custom domain again. I have no idea why, what or how... I have purchased an SSL certificate from Comodo, not yet activated, and all I want to do is set it up without giving myself an emotional hernia, I get enough of those from coding! And, yes, read the docs, but you know, they suck! A step by step idiots example would be great! In fact back to the question at the beginning, is it going to be possible to setup SSL for a custom domain from the Google cloud developers console? Thanks one and all -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/f651f90e-d041-4d48-aca4-4b501796740c%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [google-appengine] Re: Java app and scheduler settings
Nick, I completely disagree. The CSV import job is not so heavy because stream a file from GCS and enqueue a task for every row. We designed that job with the GAE rules in mind. Moreover that jobs are very few compared to the app traffic and are handled by a backend module, but my instances restarts on the default module. When the app is used the instances restart *at the same time* every hour or less. I still have no errors and I still have no evidence that our is wrong. We are sure to have followed all the rules and all the docs. We tried to configure the instance class to F4 but was the same. If the app has very few traffic the instances last days but as the traffic rise the instances begin restarting. I have not responded to the issue because I found it shallow and offensive, seems that someone looked at the logs and seeing an url like importcsv found the gold mine. The issue mention a warning in the log about a concurrency issue as an evidence of our fault, well, the warning is to log when our 'entity group limiter' start to limit the write rate on a specific entity group to consume less resources. Now I'm on vacation, the instances restart, I'm very upset, in September I will decide the role of GCP on our business. Il giorno mar 18 ago 2015 alle 01:28 Nick (Cloud Platform Support) pay...@google.com ha scritto: For the record of any future users: an investigation of the issue reported by Stefano appeared to turn up that the cause of the instances shutting down was low instance specs combined with a memory-heavy CSV import job. Mauricio, I've located the support ticket you have open, and it appears that you're currently receiving help there for a similar but unrelated issue. I encourage you to follow up there. On Friday, August 7, 2015 at 7:11:49 AM UTC-4, Stefano Ciccarelli wrote: Hello to all! Our application has always suffered from high latencies due to instances respawning continuously. We always used custom settings in automatic-scaling to try to save money. Following this post ( http://googlecloudplatform.blogspot.it/2015/08/How-to-Troubleshoot-Latency-in-Your-App-Engine-Application.html) I've decided to reset the automatic-scheduling settings of our app to default values. So I've configured only min-idle-instances to 1 (otherwise the /_ah/warmup handler is never called). The instance class is F2. After two days this is our state. So: - why I have 4 instances 2 days old not used? (The logs says that one instance served ~20 request today, but the other instances stopped to serve yesterday) - why the scheduler stop and spawn instances every 40/50 minutes if 4 instances are there to do nothing? - why the resident instance is restarted every 40/50 minutes if it is supposed to be... resident? - is there something wrong with our code? Thanks [image: nimbus-image-1438944970211.png] -- *Stefano Ciccarelli* GAE Application Division / Director stefano.ciccare...@mmbsoftware.it *M.M.B. s.r.l.* via Granarolo, 177/7 - 48018 Faenza (RA) - Italy tel. +39.0546.637711 - fax +39.0546.46077 www.mmbsoftware.it - i...@mmbsoftware.it Le informazioni contenute in questa comunicazione sono riservate e destinate esclusivamente alla/e persona/e o all'ente sopra indicati. E' vietato ai soggetti diversi dai destinatari qualsiasi uso, copia, diffusione di quanto in esso contenuto sia ai sensi dell'art. 616 c.p., sia ai sensi del DL n. 196/03. Se questa comunicazione Vi e' pervenuta per errore, Vi preghiamo di rispondere a questa e-mail e successivamente cancellarla dal Vostro sistema. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/1ce70118-f593-4ea3-b6d3-47be766d36f2%40googlegroups.com https://groups.google.com/d/msgid/google-appengine/1ce70118-f593-4ea3-b6d3-47be766d36f2%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- *Stefano Ciccarelli* GAE Application Division / Director stefano.ciccare...@mmbsoftware.it *M.M.B. s.r.l.* via Granarolo, 177/7 - 48018 Faenza (RA) - Italy tel. +39.0546.637711 - fax +39.0546.46077 www.mmbsoftware.it - i...@mmbsoftware.it Le informazioni contenute in questa comunicazione sono riservate e destinate esclusivamente alla/e persona/e o all'ente sopra indicati. E' vietato ai soggetti diversi dai destinatari qualsiasi uso, copia, diffusione di quanto in esso contenuto sia ai sensi dell'art. 616 c.p., sia ai sensi del DL n. 196/03. Se questa comunicazione Vi e' pervenuta per errore, Vi preghiamo di rispondere a questa e-mail e
[google-appengine] Re: SSL certificate for custom domains in GoogleApps
Hi Nick, Thanks for your answer. The problem could be what you say. But I just checked the Active in the Admin console - Domains. The domain is active, I think no more actions are required. Is there any other option to make a reverify? What do you mean by checking the SubjectAltNames (SAN)? I bought the domain in Dinahosting, could it be this the problem? Thanks for your help. Xevi El martes, 18 de agosto de 2015, 1:00:28 (UTC+2), Nick (Cloud Platform Support) escribió: Hi Xevi, I think the error message might be saying your certificate contains references to domains which aren't on your verified domain. You should double-check your certificate and check not only the main domain but also any SubjectAltNames. On Thursday, August 13, 2015 at 5:05:35 PM UTC-4, Xevi Farrarons Clusella wrote: Hi Nick, Thanks for your answer :) It's great to know that I should be able to add the SSL certificate for the alias domain. Let me explain better: I follow this procedure: https://support.google.com/a/answer/2644386?hl=en The domain alias is verified. But I stuck at point 4. When I press the upload button, it appears a message saying: the ssl certificate references subdomain outside of managed domain. At the link, where you could read: appenginetest.com, in our case our main domain, but not the alias domain. I think that this is the issue, we cannot select for which domain.. What do you think, how could you add the SSL for the alias domain? Thanks in advance, Xevi El jueves, 13 de agosto de 2015, 21:54:40 (UTC+2), Nick escribió: You can use an SSL certificate for a domain alias in Google apps. First add and verify the domain as an alias domain of your apps account, then follow the normal procedure of adding the app and cert. Be aware that you cannot send email from the alias domain without adding specific users as Appengine users. You could also look at mailgun or sendgrid etc. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/105650d0-e5ef-4568-b833-23298eacead6%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [google-appengine] At what point is querying down a hierarchical tree faster than querying through all the end nodes?
Hey Karl! I definitely would leverage Google's API for this problem, but I actually was more interested in the approach for this hierarchical type of problem. To flatten or not to flatten, that is the question! But yes, APis would allow me to check for user error when it comes to cities and such. I think my current format doesn't allow me to figure this out efficiently. I would need to get the city, find neighbouring cities and then find restaurants in those cities. Actually, I could submit their original query and ask some other api for nearby cities. Once I got cities I could also run queries for the same restaurant name in those neighbouring cities. Even better I think, would be when I create my restaurant information, I find the neighbouring cities using some other API, and create restauants there pointing back to the correct city... For instance City 1 and City2 are close to each other. When I put a restaurant via, Country, State, City1.Restaurant, I could also run a query through some other api for neighbouring cities. Once I get that information I could put the restaurant information into, Country, State, City2, Restaurant. Then if someone screws up and searches Country,State.City2.Restaurant instead of the correct City1 alternative, I would still be able to give them information that points back to the correct city. This means extra time for creating data, but hopefully less for searching through it. Thanks Karl! On Monday, August 17, 2015 at 5:41:25 PM UTC-7, Karl MacMillan wrote: On Sunday, August 16, 2015, Sanket More sanke...@gmail.com javascript: wrote: Hey Nick! After I submitted my question, I went through lots of information sources and gathered some information. I feel like my question was based on many misconceptions about google app engine's datastore. I think I have a strategy for the restaurant problem. The user will provide me with country, state, city and the restaurant name, And I will provide the user with location(s). So all I really need is one specific entity and the response will simply be the information in that entity. And, I don't really need to index country, state, city, and restaurant name with each other independently. I just need one string that holds the country state city and restaurant. In fact, I think that string is unique. So, I am thinking about having a restaurant kind with key string country_state_city_restaurant Once the user has finished submitting the form, I can get by country_state_city_restaurant and return some information to the user. I wanted a smarter solution than this, because I feel that this solution won't scale well. Furthermore, if I am flattening my tree structure like this, wouldn't gql be faster? On Sunday, August 16, 2015 at 4:14:33 PM UTC-7, Nick wrote: The general rule is optimize for your primary use case, denormalise to support others. In addition, enforce and support constraints In your code. If your primary case is restaurant search, embed and index all necessary data. This will be the fastest solution and easiest to work with On Appengine extra data doesn't cost you unless it's indexed. There is a careful balance between optimising for speed, cost and transactional integrity. My advice is if this is your first go, expect to make mistakes and to need to reengineer your data a couple of times. Make that process as easy as possible. Just a suggestion - have you thought about letting the user query through the Google maps or similar API to find the restaurants and then based on those results returning information from your app? These location based queries are hard - both technically and from matching the users expectations. Especially as I'm betting that the city that a user thinks a restaurant is located in often differs from official addresses, especially in large cities. This strategy could let you both control the user experience and leverage the hard work of others. Karl -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/04705cc7-3673-4d82-984a-61c64b099fe8%40googlegroups.com https://groups.google.com/d/msgid/google-appengine/04705cc7-3673-4d82-984a-61c64b099fe8%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to
Re: [google-appengine] Re: Java app and scheduler settings
It sounds like the root problem is OOM errors fail to produce good log messages. What CSV parser are you using? If you're on Java, most of them are crap. Jackson has a pretty good CSV plugin, but you probably still need to be very careful about streaming - it's really easy to blow up the heap with large CSV imports. I've had this problem before. Jeff On Tue, Aug 18, 2015 at 12:27 AM, Stefano Ciccarelli stefano.ciccare...@mmbsoftware.it wrote: Nick, I completely disagree. The CSV import job is not so heavy because stream a file from GCS and enqueue a task for every row. We designed that job with the GAE rules in mind. Moreover that jobs are very few compared to the app traffic and are handled by a backend module, but my instances restarts on the default module. When the app is used the instances restart *at the same time* every hour or less. I still have no errors and I still have no evidence that our is wrong. We are sure to have followed all the rules and all the docs. We tried to configure the instance class to F4 but was the same. If the app has very few traffic the instances last days but as the traffic rise the instances begin restarting. I have not responded to the issue because I found it shallow and offensive, seems that someone looked at the logs and seeing an url like importcsv found the gold mine. The issue mention a warning in the log about a concurrency issue as an evidence of our fault, well, the warning is to log when our 'entity group limiter' start to limit the write rate on a specific entity group to consume less resources. Now I'm on vacation, the instances restart, I'm very upset, in September I will decide the role of GCP on our business. Il giorno mar 18 ago 2015 alle 01:28 Nick (Cloud Platform Support) pay...@google.com ha scritto: For the record of any future users: an investigation of the issue reported by Stefano appeared to turn up that the cause of the instances shutting down was low instance specs combined with a memory-heavy CSV import job. Mauricio, I've located the support ticket you have open, and it appears that you're currently receiving help there for a similar but unrelated issue. I encourage you to follow up there. On Friday, August 7, 2015 at 7:11:49 AM UTC-4, Stefano Ciccarelli wrote: Hello to all! Our application has always suffered from high latencies due to instances respawning continuously. We always used custom settings in automatic-scaling to try to save money. Following this post ( http://googlecloudplatform.blogspot.it/2015/08/How-to-Troubleshoot-Latency-in-Your-App-Engine-Application.html) I've decided to reset the automatic-scheduling settings of our app to default values. So I've configured only min-idle-instances to 1 (otherwise the /_ah/warmup handler is never called). The instance class is F2. After two days this is our state. So: - why I have 4 instances 2 days old not used? (The logs says that one instance served ~20 request today, but the other instances stopped to serve yesterday) - why the scheduler stop and spawn instances every 40/50 minutes if 4 instances are there to do nothing? - why the resident instance is restarted every 40/50 minutes if it is supposed to be... resident? - is there something wrong with our code? Thanks [image: nimbus-image-1438944970211.png] -- *Stefano Ciccarelli* GAE Application Division / Director stefano.ciccare...@mmbsoftware.it *M.M.B. s.r.l.* via Granarolo, 177/7 - 48018 Faenza (RA) - Italy tel. +39.0546.637711 - fax +39.0546.46077 www.mmbsoftware.it - i...@mmbsoftware.it Le informazioni contenute in questa comunicazione sono riservate e destinate esclusivamente alla/e persona/e o all'ente sopra indicati. E' vietato ai soggetti diversi dai destinatari qualsiasi uso, copia, diffusione di quanto in esso contenuto sia ai sensi dell'art. 616 c.p., sia ai sensi del DL n. 196/03. Se questa comunicazione Vi e' pervenuta per errore, Vi preghiamo di rispondere a questa e-mail e successivamente cancellarla dal Vostro sistema. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/1ce70118-f593-4ea3-b6d3-47be766d36f2%40googlegroups.com https://groups.google.com/d/msgid/google-appengine/1ce70118-f593-4ea3-b6d3-47be766d36f2%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- *Stefano Ciccarelli* GAE Application Division / Director stefano.ciccare...@mmbsoftware.it *M.M.B. s.r.l.* via Granarolo, 177/7 - 48018 Faenza (RA) - Italy tel.
Re: [google-appengine] Re: Problem authenticating to GAE app using GoogleCredential OAuth2
Hi Julian, So, after some extensive testing and reading around online, it appears that this is no longer possible, and I encourage you to make a public issue tracker feature request http://code.google.com/p/google-appengine/issues/list explaining in simple terms, minus the code, your desired use-case, of making HTTP calls to your app on routes protected by login: admin. From what I can see from looking around online, the old method which used a certain ClientLogin endpoint to get the token passed to the server as the ACSID cookie is no longer active. The closest thing to signing-in with a service account that I could find was service account Apps domain user impersonation in the Server to Server OAuth2 https://developers.google.com/identity/protocols/OAuth2ServiceAccount docs. After extensive testing, implementing code which built the credential, built an HttpTransport for it, it was not possible to get the app to recognize the calling java code as a login: admin user, even when impersonating a user which has admin status (standalone java code compiled with classpath by hand). So, feel free to post the public issue tracker feature request link in this thread once you've made it, and I'll be following it and helping to get it processed. Best wishes, Nick On Friday, August 14, 2015 at 5:44:08 PM UTC-4, Julian Bunn wrote: Hi Nick, Yes ... here are the relevant details (extracted from my updated question on stack overflow). First the secure URL specs: !-- Secure sensitive URLs -- security-constraint web-resource-collection url-pattern/gcm/home/url-pattern url-pattern/gcm/send/url-pattern /web-resource-collection auth-constraint role-nameadmin/role-name /auth-constraint /security-constraint Previously, in the client application, I was using ClientLogin to authenticate with Google before calling the endpoint. This is the code I was using, that extracts the Auth token which it then uses as a Cookie on HTTP GET to the above endpoints. public static String loginToGoogle(String userid, String password, String appUrl) throws Exception { HttpClient client = new DefaultHttpClient(); HttpPost post = new HttpPost( https://www.google.com/accounts/ClientLogin;); MultipartEntity reqEntity = new MultipartEntity(); reqEntity.addPart(accountType, new StringBody(HOSTED_OR_GOOGLE, text/plain, Charset.forName(UTF-8))); reqEntity.addPart(Email, new StringBody(userid)); reqEntity.addPart(Passwd, new StringBody(password)); reqEntity.addPart(service, new StringBody(ah)); reqEntity.addPart(source, new StringBody(myappname)); post.setEntity(reqEntity); HttpResponse response = client.execute(post); if (response.getStatusLine().getStatusCode() == 200) { InputStream input = response.getEntity().getContent(); String result = IOUtils.toString(input); String authToken = getAuthToken(result); post = new HttpPost(appUrl + /_ah/login?auth= + authToken); response = client.execute(post); Header[] cookies = response.getHeaders(SET-COOKIE); for (Header cookie : cookies) { if (cookie.getValue().startsWith(ACSID=)) { return cookie.getValue(); } } throw new Exception(ACSID cookie cannot be found); } else throw new Exception(Error obtaining ACSID); } private static String getAuthToken(String responseText) throws Exception { LineNumberReader reader = new LineNumberReader(new StringReader( responseText)); String line = reader.readLine(); while (line != null) { line = line.trim(); if (line.startsWith(Auth=)) { return line.substring(5); } line = reader.readLine(); } throw new Exception(Could not find Auth token); } Calling the gcm endpoint: HttpGet get = new HttpGet(httpURL); get.setHeader(Cookie, authCookie); HttpResponse response = client.execute(get); response.getEntity().writeTo(System.out); where authCookie is the token obtained from loginToGoogle above. Thanks so much for helping with this! Julian On Fri, Aug 14, 2015 at 2:03 PM, Nick (Cloud Platform Support) pay...@google.com wrote: A quick question, is it possible you could provide the skeleton code for your client project? It appears to be a standalone java program, rather than a web app, yes? On Wednesday, August 12, 2015 at 5:00:53 PM UTC-4, Julian Bunn wrote: Hi Nick, Thanks for much for persisting with your help!
Re: [google-appengine] Re: Problem authenticating to GAE app using GoogleCredential OAuth2
Hi Nick, Thanks very much for investigating this so thoroughly and following up. I will complete a feature request, as you suggest, probably sometime tomorrow. It's reassuring that you too were unable to make it work - I'd tried so many different things myself, I was beginning to doubt my competence :-) Julian On Tue, Aug 18, 2015 at 4:30 PM, Nick (Cloud Platform Support) pay...@google.com wrote: Hi Julian, So, after some extensive testing and reading around online, it appears that this is no longer possible, and I encourage you to make a public issue tracker feature request http://code.google.com/p/google-appengine/issues/list explaining in simple terms, minus the code, your desired use-case, of making HTTP calls to your app on routes protected by login: admin. From what I can see from looking around online, the old method which used a certain ClientLogin endpoint to get the token passed to the server as the ACSID cookie is no longer active. The closest thing to signing-in with a service account that I could find was service account Apps domain user impersonation in the Server to Server OAuth2 https://developers.google.com/identity/protocols/OAuth2ServiceAccount docs. After extensive testing, implementing code which built the credential, built an HttpTransport for it, it was not possible to get the app to recognize the calling java code as a login: admin user, even when impersonating a user which has admin status (standalone java code compiled with classpath by hand). So, feel free to post the public issue tracker feature request link in this thread once you've made it, and I'll be following it and helping to get it processed. Best wishes, Nick On Friday, August 14, 2015 at 5:44:08 PM UTC-4, Julian Bunn wrote: Hi Nick, Yes ... here are the relevant details (extracted from my updated question on stack overflow). First the secure URL specs: !-- Secure sensitive URLs -- security-constraint web-resource-collection url-pattern/gcm/home/url-pattern url-pattern/gcm/send/url-pattern /web-resource-collection auth-constraint role-nameadmin/role-name /auth-constraint /security-constraint Previously, in the client application, I was using ClientLogin to authenticate with Google before calling the endpoint. This is the code I was using, that extracts the Auth token which it then uses as a Cookie on HTTP GET to the above endpoints. public static String loginToGoogle(String userid, String password, String appUrl) throws Exception { HttpClient client = new DefaultHttpClient(); HttpPost post = new HttpPost( https://www.google.com/accounts/ClientLogin;); MultipartEntity reqEntity = new MultipartEntity(); reqEntity.addPart(accountType, new StringBody(HOSTED_OR_GOOGLE, text/plain, Charset.forName(UTF-8))); reqEntity.addPart(Email, new StringBody(userid)); reqEntity.addPart(Passwd, new StringBody(password)); reqEntity.addPart(service, new StringBody(ah)); reqEntity.addPart(source, new StringBody(myappname)); post.setEntity(reqEntity); HttpResponse response = client.execute(post); if (response.getStatusLine().getStatusCode() == 200) { InputStream input = response.getEntity().getContent(); String result = IOUtils.toString(input); String authToken = getAuthToken(result); post = new HttpPost(appUrl + /_ah/login?auth= + authToken); response = client.execute(post); Header[] cookies = response.getHeaders(SET-COOKIE); for (Header cookie : cookies) { if (cookie.getValue().startsWith(ACSID=)) { return cookie.getValue(); } } throw new Exception(ACSID cookie cannot be found); } else throw new Exception(Error obtaining ACSID); } private static String getAuthToken(String responseText) throws Exception { LineNumberReader reader = new LineNumberReader(new StringReader( responseText)); String line = reader.readLine(); while (line != null) { line = line.trim(); if (line.startsWith(Auth=)) { return line.substring(5); } line = reader.readLine(); } throw new Exception(Could not find Auth token); } Calling the gcm endpoint: HttpGet get = new HttpGet(httpURL); get.setHeader(Cookie, authCookie); HttpResponse response = client.execute(get); response.getEntity().writeTo(System.out); where authCookie is the token obtained from loginToGoogle above. Thanks so much for helping with this! Julian On Fri, Aug 14, 2015 at 2:03 PM, Nick (Cloud Platform Support) pay...@google.com wrote: A quick question, is it possible you could provide the skeleton code for your client project? It appears to be a standalone java program, rather than a web app, yes? On Wednesday, August 12, 2015 at 5:00:53 PM UTC-4, Julian Bunn
[google-appengine] Re: Sort and subsort in datastore?
Hi Keith, For the datastore to be scalable, indexing has to be done this way, and datastore cannot think, only scan an index. This is the only way to keep datastore scalable when there's a really big amount of data in it. You can get more information to the limits of querying and indexes in this article https://cloud.google.com/appengine/docs/java/datastore/queries#Java_Restrictions_on_queries. Also, when I was learning datastore, I found that this video https://www.youtube.com/watch?v=tx5gdoNpcZM was of great help to understand the reasoning behind the limitations, so it might be worth watching it :). Now, with that in mind, there's no way to do what you want straightforward. With the limitations on inequality filters, you can't filter for both date and relevance at once. You could get around this by having each entity have boolean properties for this month, this week, this day, etc. With that, you could do a query for all this week = true, sort by relevance. A cron job would be needed to update the entities each day though, and depending on your use case and what's in your datastore, could end up being costly. Your method of grabbing everything, then sorting yourself, is another idea that would work. Even if it will require a bit longer time to process your data Personally, I'd use our automated system https://cloud.google.com/bigquery/loading-data-cloud-datastore?hl=en to backup your data to BigQuery daily, and then you could query the way you want easily. Cheers! On Monday, August 17, 2015 at 11:15:18 PM UTC-4, Keith Chima wrote: My datastore has stored entities with a timestamp and relevance. I want to get the most relevant results from today, this week, month, or all time. The first sort would be timestamp, followed by relevance, pulling 10 at a time. However, this: Filter timeMaxFilter = new FilterPredicate(timestamp, FilterOperator.GREATER_THAN_OR_EQUAL, getEarlierTimeStamp(Calendar.HOUR, -24)); if(timeFrame.equalsIgnoreCase(day)){ timeMaxFilter = new FilterPredicate(timestamp, FilterOperator.GREATER_THAN_OR_EQUAL, getEarlierTimeStamp(Calendar.HOUR, -24)); q.setFilter(timeMaxFilter); } q.addSort(relevance); PreparedQuery pq = datastore.prepare(q); for (Entity result : pq.asList(withLimit(10).offset(10*pageNumber))) { } *Generates this error message:* [INFO] java.lang.IllegalArgumentException: The first sort property must be the s ame as the property to which the inequality filter is applied. In your query th e first sort property is relevance but the inequality filter is on relevance. [INFO] at com.google.appengine.api.datastore.DatastoreApiHelper.translateError( DatastoreApiHelper.java:53) I understand why- they reindex in order of timestamp, because they want to be able to return the query in flat time, so they require me to sort by timestamp first. However, I feel like they can't be putting such a strict requirement on their customers. I could probably pull back ALL results in order of relevance, and then pull off the top 10. However, that seems wasteful. Is there a better way? If not, how would I set a FilterOrder with no limit, meaning ALL of the entries so I could sort them on my end? Thanks, -Keith -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/1bd36a20-3de1-44b7-b7f5-08c5e0367f17%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[google-appengine] Re: app engine usage report beyond 90 days
Hello Nick, thank you for the info. On Tuesday, August 18, 2015 at 1:33:31 AM UTC+3, Nick (Cloud Platform Support) wrote: Hi Ivan, Unfortunately the data is not persisted beyond 90 days. In order to securely store logs beyond that, you should look into Logs Export https://cloud.google.com/logging/docs/export/configure_export. I hope this helps you set up a solution which will prevent such difficulties in the future. I sincerely hope that your logs data was not extremely critical. Best of luck! Nick On Monday, August 17, 2015 at 11:27:32 AM UTC-4, Ivan Golubev wrote: Hello guys, does anyone know how to fetch Google App Engine usage report beyond the 90 days, say for 9 months ? I mean this feature : Download Usage Reports (CSV) https://appengine.google.com/billing/history.csv This is located on page https://appengine.google.com/billing/history -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/568166c2-1ba9-41ad-af94-1026817a8c92%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[google-appengine] Re: How to whitelist a App Engine URL with Google Groups.
Hi Paula, I'm not quite sure what you mean by whitelist an App Engine URL. What exactly are you trying to do? Is there a document which mentioned whitelisting? Sincerely, Nick On Monday, August 17, 2015 at 7:28:36 PM UTC-4, Paula Byrne wrote: Hi there I am trying to learn how to whitelist an App Engine URL with Google Groups. Does anyone have any advice on this? Thanks Paula -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/0e3e9708-a715-4aad-9224-45dd3cd9c251%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.