Hello Remko, Just to clear any confusion, the product I was referring to was “Firestore in Datastore mode”. Although the names are similar, “Firestore” and “Firebase” are different in that “Firebase” is a development platform and “Firestore” is the new-generation of NoSQL Cloud Database.
In the past, you were using the “Datastore” product and the DB API library to access the database in that product. On January 31st 2019, “Cloud Firestore” became Generally Available [1]. Afterward, an automatic upgrade to Firestore was executed (see [2] for more details), and we made available the new product available in Datastore mode. Although this allows users of the legacy Datastore still feel at home, the architecture of Firestore is different and this is why we recommend migrating from DB to the NDB client library. Here [3] is the correct documentation I meant to send you in my previous message about this migration. You may use it as reference to compare with your own implementation to target potential issues leading to the contention errors. So to address your inquiry, switching to a different library is not directly what causes the data to be held in a different architectural product, but the fact that this new library is optimized to work with the new product being Firestore. Therefore, I recommend looking at the limits specific to Firestore as outlined here [4], which may be different than the ones for Datastore, and test how your application queries it resulting in more than 1/sec mutations on a single entity group. Please note that cross entity group contention can cause unavailability, so reading an entity group in a transaction counts as a “write” and thus is limited to 1 per second. For a transaction across entities, all participant entities must be uncontended in order for the transaction to succeed. Let me know if you need further help. [1] https://cloud.google.com/datastore/release-notes#January_31_2019 [2] https://cloud.google.com/datastore/docs/upgrade-to-firestore#automatic_upgrade_to_cloud_firestore_in_datastore_mode [3] https://cloud.google.com/appengine/docs/standard/python/ndb/db_to_ndb [4] https://cloud.google.com/datastore/docs/concepts/limits On Monday, March 2, 2020 at 2:07:38 PM UTC-5, Remko Tronçon wrote: > > Hi Pierre-Yves, > > Thanks for your comments. I'm aware of what causes contention errors, and > how to resolve them. However, I'm just surprised that moving from the DB > (*not* NDB) library to the Cloud NDB library suddenly causes contention > errors. (and it's significant; e.g. a model that never used to have > problems with 20 shards now still has problems with 60 shards). > > I am also aware about the caching behavior. I have not configured any > cache. However, since I'm coming from DB (which isn't cached at all > either), and since this is about problems with writing (which would write > through the cache anyway AFAIK), I don't see how cache changes can cause > this. > > Unfortunately, since I'm not coming from NDB, I can't say whether it's the > move from DB->NDB or NDB->Cloud NDB that causes this. I did the entire jump > in one time. > > Could you explain why you bring up 'Firebase in Datastore mode'? Am I > suddenly using a different architecture by switching to a different library > to access the same data? > > thanks, > Remko > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/cc41b04a-8256-4bda-bb45-2905e8aa5163%40googlegroups.com.
