Hi Dmitriy, Quick start for a new user is a perfect concern. I thought about it as well. I and would like to outline that I am completely ok with an approach in your patch.
пн, 18 мар. 2019 г. в 16:16, Dmitriy Pavlov <dpav...@apache.org>: > > Hi Ivan, > > reasonable points, but we had a long and hot discussion about this logic. > > Reason for selecting auto-generation was users that just try to use Apache > Ignite with native persistence for the first time. They must be able to > test features without any extensive setup, but before go-live we expect > them to study the product a little bit more. There are not so many reasons > to use the same folder (or even same device) for 2+ nodes. > > Sincerely, > Dmitriy Pavlov > > вс, 17 мар. 2019 г. в 13:34, Павлухин Иван <vololo...@gmail.com>: > > > Hi Dmitriy, > > > > It is a pitty that we have to guess here. Some comments: > > 1. The usage scenario is not very complex and erything might work well > > with aforementioned patch. Unfortunately some tradeoff and extra > > transformations are involved and consequently it is hard to predict > > how many users will be troubled. But on a bright side the change is > > seemless and perhaps nobody will be really troubled. > > 2. From "honest" code perspective introducing nodeStorageName seems > > better to me because no (surprising) transformations are involved. > > 3. Also, it seems to me that original logic for choosing storage was > > not designed well. From the first glance it looks strange that need > > folders for a particular node in different places. Is not it much > > clear to have all data of the node in a single directory isolated from > > other nodes? AFAIK Hazelcast follows this approach. They generate an > > error when someone is trying to launch second instance in the same > > working directory. Perhaps, we at least should think about changing > > our layout and consider including such change in some future release. > > > > Thank you! > > > > пт, 15 мар. 2019 г. в 17:55, Dmitriy Pavlov <dpav...@apache.org>: > > > > > > Hi Ivan, yes, after the restart we need to re-use both folder name and > > UUID > > > as consistent ID. > > > > > > About returning exact directory name: I'm just guessing here, but the > > > persistent store is not the only folder, there are several more folders, > > > e.g. binary-meta, etc. > > > > > > About several nodes reusing same root folder, yes, the proposed solution > > > will not works because 00 is hardcoded here. But I expect it will be too > > > rare case. > > > > > > Sincerely, > > > Dmitriy Pavlov > > > > > > пт, 15 мар. 2019 г. в 11:29, Павлухин Иван <vololo...@gmail.com>: > > > > > > > Hi Dmitriy, > > > > > > > > It looks like that case with only one node in working directory is > > > > covered. But what is problem is solved? As I understand before fix in > > > > such case node after restart uses proper folder and proper > > > > consistentId. After fix we can specify automatically generated UUID in > > > > configuration and everything will go smooth. Is it the main problem > > > > case? If yes then such simplicity is very attractive. > > > > > > > > But honestly I do not fully understand why do we need to return a node > > > > to node00-UUID directory after that directory was deleted? > > > > > > > > чт, 14 мар. 2019 г. в 17:21, Dmitriy Pavlov <dpav...@apache.org>: > > > > > > > > > > Hi, I've prepared demo PR for approach 1, just checking UUID and > > trying > > > > to > > > > > use node00-UUID. https://github.com/apache/ignite/pull/6266/ > > Actually > > > > for > > > > > this approach it is really simple, no API change is required. > > > > > > > > > > I guess the case is not very often because automatic folders > > assignment > > > > > works well in most cases. > > > > > > > > > > WDYT? > > > > > > > > > > ср, 13 мар. 2019 г. в 09:42, Павлухин Иван <vololo...@gmail.com>: > > > > > > > > > > > Alex, > > > > > > > > > > > > Why do we need > > > > > > > - check if consistent ID is set to a string 'nodeXX-UUID'. In > > this > > > > case > > > > > > > the consistent ID is set to UUID, and the storage folder is > > chosen > > > > > > > according to the proper rules. This change has a minimal chance > > to > > > > affect > > > > > > > current users because it's unlikely that somebody is using > > > > auto-generated > > > > > > > folder naming scheme as consistent ID. > > > > > > ? > > > > > > It looks hacky as well. The thing I do not like here is that a > > > > > > consistentId specified in configuration is not a consistentId used > > by > > > > > > a node sometimes. > > > > > > > > > > > > Can we go just with > > > > > > > - either check if consistent ID is an instance of UUID and then > > take > > > > the > > > > > > > appropriate folder. This approach is straightforward, but may > > affect > > > > > > > current users > > > > > > ? > > > > > > > > > > > > And as a last chance a user will have a possibility to rename a > > > > directory. > > > > > > > > > > > > > > > > > > вт, 12 мар. 2019 г. в 19:33, Alexey Goncharuk < > > > > alexey.goncha...@gmail.com > > > > > > >: > > > > > > > > > > > > > > Igniters, > > > > > > > > > > > > > > I came across the same issue during development and found no sane > > > > > > > workaround for this issue. I believe the solution should be as > > > > simple as > > > > > > > possible because we are already adding a warning to let users > > know > > > > that > > > > > > it > > > > > > > is good to specify a consistent ID in production deployments. > > > > > > > > > > > > > > As for the current solution, I do not like adding a new > > configuration > > > > > > > property like 'storageFolder' because it's another way to shoot > > > > yourself > > > > > > in > > > > > > > the leg (e.g. different nodes have different consistent IDs but > > > > > > configured > > > > > > > to have the same storage folder). > > > > > > > Why can't we: > > > > > > > - either check if consistent ID is an instance of UUID and then > > > > take the > > > > > > > appropriate folder. This approach is straightforward, but may > > affect > > > > > > > current users > > > > > > > - check if consistent ID is set to a string 'nodeXX-UUID'. In > > this > > > > case > > > > > > > the consistent ID is set to UUID, and the storage folder is > > chosen > > > > > > > according to the proper rules. This change has a minimal chance > > to > > > > affect > > > > > > > current users because it's unlikely that somebody is using > > > > auto-generated > > > > > > > folder naming scheme as consistent ID. > > > > > > > > > > > > > > Thoughts? > > > > > > > > > > > > > > вт, 12 мар. 2019 г. в 16:51, Dmitriy Pavlov <dpav...@apache.org > > >: > > > > > > > > > > > > > > > Hi Igniters, > > > > > > > > > > > > > > > > A full description can be found at wiki page > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood#IgnitePersistentStore-underthehood-SubfoldersGeneration > > > > > > > > > > > > > > > > I like the idea of introducing a property like nodeStorageName > > to > > > > > > specify > > > > > > > > exactly name of a directory to use. For example, it could have > > > > higher > > > > > > > > priority then IgniteConfiguration.getConsistentId(). It will > > > > impact the > > > > > > > > mentioned algorithm, but it seems to be easier to understand by > > > > users. > > > > > > > > > > > > > > > > If nobody else minds, I will try to implement this idea. > > > > > > > > > > > > > > > > Sincerely, > > > > > > > > Dmitriy Pavlov > > > > > > > > > > > > > > > > > > > > > > > > сб, 2 мар. 2019 г. в 08:33, Павлухин Иван <vololo...@gmail.com > > >: > > > > > > > > > > > > > > > > > Dmitriy, > > > > > > > > > > > > > > > > > > You wrote > > > > > > > > > > we need an order to scan and lock random-UUID based > > folders. > > > > > > > > > > > > > > > > > > It would be great if you provide a discussion about that > > order to > > > > > > > > > complete the picture. Currently I cannot understand why the > > > > order is > > > > > > > > > important. > > > > > > > > > > > > > > > > > > Also, couple of raw thoughts: > > > > > > > > > 1. Can we extend a directory lookup procedure when > > consistent id > > > > is > > > > > > > > > specified to check nodeXX-consistentId directories as well? > > > > > > > > > 2. Can we introduce a property like nodeStorageName to > > specify > > > > exact > > > > > > > > > name of a directory to use? It looks like a straightforward > > and > > > > > > > > > universal workaround. Or is node index better in some sense? > > Why? > > > > > > > > > > > > > > > > > > Please, share your thoughts. > > > > > > > > > > > > > > > > > > чт, 28 февр. 2019 г. в 17:43, Dmitriy Pavlov < > > dpav...@apache.org > > > > >: > > > > > > > > > > > > > > > > > > > > Hi Ivan, > > > > > > > > > > > > > > > > > > > > Yes, you catch me, I'm a little bit cheating with lazy > > > > consensus on > > > > > > > > code > > > > > > > > > > modification without providing a PR because I was expecting > > > > that > > > > > > nobody > > > > > > > > > > comes to discussion. I will prepare PR shortly. And since > > we > > > > anyway > > > > > > > > have > > > > > > > > > a > > > > > > > > > > discussion, I will not apply anything by lazy approval. > > > > > > > > > > > > > > > > > > > > - storageNodeIndex without consistent ID will not work. > > > > > > > > > > cfg.getDataStorageConfiguration().setNodeIdx() will be > > > > required > > > > > > only > > > > > > > > for > > > > > > > > > > case we have consistent ID. > > > > > > > > > > > > > > > > > > > > Hi Stanislav, > > > > > > > > > > > > > > > > > > > > We can't use only consistent ID because > > > > > > > > > > > > > > > > > > > > 1) we need an order to scan and lock random-UUID based > > folders. > > > > > > Node > > > > > > > > > index > > > > > > > > > > provides the order of scan. I can find the corresponding > > > > > > discussion, > > > > > > > > but > > > > > > > > > I > > > > > > > > > > guess it is not needed. > > > > > > > > > > 2) we need to separate backward compatible folders from new > > > > > > random-UUID > > > > > > > > > > based folders. Using UUID as folder will not allow us to > > scan > > > > only > > > > > > new > > > > > > > > > name > > > > > > > > > > format folders. > > > > > > > > > > > > > > > > > > > > I guess specifying node index is a quite rare case and good > > > > JavaDoc > > > > > > > > will > > > > > > > > > > always help. > > > > > > > > > > > > > > > > > > > > DataStorageConfiguration().setNodeIdx() JavaDoc may > > include > > > > > > following > > > > > > > > > > notes: > > > > > > > > > > Node index used for persistent store folders in case > > several > > > > nodes > > > > > > > > reuse > > > > > > > > > > one persistent store root folder. > > > > > > > > > > > > > > > > > > > > Sincerely, > > > > > > > > > > Dmitriy Pavlov > > > > > > > > > > > > > > > > > > > > чт, 28 февр. 2019 г. в 08:03, Павлухин Иван < > > > > vololo...@gmail.com>: > > > > > > > > > > > > > > > > > > > > > Dmitiy, > > > > > > > > > > > > > > > > > > > > > > Could please clarify one thing: > > > > > > > > > > > 1. Will it be enough to use only storageNodeIndex in > > order to > > > > > > reuse > > > > > > > > > > > the same persistence folders when consistentId is > > > > auto-generated? > > > > > > > > E.g. > > > > > > > > > > > I have a configuration with storageNodeIndex=1 and > > without > > > > > > explicitly > > > > > > > > > > > specified consistentId, will the node after restart use > > the > > > > same > > > > > > > > > > > persistence folder as before restart? > > > > > > > > > > > > > > > > > > > > > > Also a side note: > > > > > > > > > > > > Please share your vision. I'm going to apply this > > change by > > > > > > lazy > > > > > > > > > > > consensus > > > > > > > > > > > in 3 days. > > > > > > > > > > > What do you mean by "apply"? I have not seen any PR yet. > > > > > > > > > > > > > > > > > > > > > > ср, 27 февр. 2019 г. в 21:12, Dmitriy Pavlov < > > > > dpav...@apache.org > > > > > > >: > > > > > > > > > > > > > > > > > > > > > > > > Hi Igniters, > > > > > > > > > > > > > > > > > > > > > > > > I would like to fix the issue > > > > > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-11432 > > about > > > > > > > > specifying > > > > > > > > > some > > > > > > > > > > > > previous randomly generated UUID as a new consistent > > ID. > > > > Folder > > > > > > > > > > > generation > > > > > > > > > > > > algorithm here ( > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood > > > > > > > > > > > ) > > > > > > > > > > > > allows two options > > > > > > > > > > > > -node00+random UUID > > > > > > > > > > > > - consistendId > > > > > > > > > > > > > > > > > > > > > > > > I would like to add to Ignite configuration new > > property > > > > > > nodeIndex > > > > > > > > in > > > > > > > > > > > > addition to consistent Id. New Property will be named > > as > > > > > > > > > > > storageNodeIndex, > > > > > > > > > > > > int, zero-based. > > > > > > > > > > > > This will add the third option of subfolders > > processing: > > > > > > > > > > > > node{storageNodeIndex}+consistentID > > > > > > > > > > > > > > > > > > > > > > > > Please share your vision. I'm going to apply this > > change by > > > > > > lazy > > > > > > > > > > > consensus > > > > > > > > > > > > in 3 days. > > > > > > > > > > > > > > > > > > > > > > > > Sincerely, > > > > > > > > > > > > Dmitriy Pavlov > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > Best regards, > > > > > > > > > > > Ivan Pavlukhin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > Best regards, > > > > > > > > > Ivan Pavlukhin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Best regards, > > > > > > Ivan Pavlukhin > > > > > > > > > > > > > > > > > > > > > > -- > > > > Best regards, > > > > Ivan Pavlukhin > > > > > > > > > > > > -- > > Best regards, > > Ivan Pavlukhin > > -- Best regards, Ivan Pavlukhin
