I've noticed that the dsl_dataset_t that points to a given dataset changes during the life time of a 'zfs create' command. We start out with one dsl_dataset_t* during dmu_objset_create_sync() but by the time we are later mounting the dataset we have a different in memory dsl_dataset_t* referring to the same dataset.
This causes me a big issue with per dataset provided encryption keys, but not with a per pool provided encryption key. When keyscope=dataset we pass the key value used as the wrapping key down over the ioctl as part of the dataset creation. This makes its way via dmu_objset_create_sync() to dsl_crypto_key_gen() which generates the actual encryption key and wraps it using the key that came over the ioctl, it then stores the wrapped key on disk using zap_update. All this part works just fine. The last thing that dsl_crypto_key_gen() does is place the encryption key into ds>ds_key so it can be used later. When keyscope=pool the only difference with the above is that the wrapping key is already present in spa->spa_key. The problem occurs when the userland zfs_create() attempts to mount the newly created dataset. The dsl_dataset_t* that was valid during the creation isn't available anymore and since there is no key we can't mount the dataset. Have I done something fundamentally wrong by hanging the crypto key off the dsl_dataset_t ? Is there a better choice that is always going to be in memory for a given dataset through the whole 'zfs create' operation ? -- Darren J Moffat
