Re: [ndctl PATCH] monitor: removing the requirement of default config
On Mon, 2019-04-01 at 06:20 +, qi.f...@fujitsu.com wrote: > > Hi Vishal, > > Thanks for your comment. > I think it might be misunderstood. > > Current scenario is: > 1. Check if "-c " option is provided > a. if it is, make the as config_file > b. if it is not, make default configuration file(/etc/ndctl/monitor.conf) > as config_file > 2. Try to read options from config_file > a. if config_file cannot be opened, then check if "-c" option is provided >(1) if it is, stop starting monitor (user may make a wrong in "-c" > option) >(2) if it is not, finish read_config_file (default configuration > file(/etc/ndctl/monitor.conf) is missing) > b. if config_file can be opened successfully, merge the options from > config_file with cli options > > It is consistent with the ideal scenario as you mentioned. Hi Qi, Looking at it in more detail, I see you're right. I've applied the patch - thanks for the explanation. > > Thanks, > QI Fuli ___ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm
Re: [PATCH] mm: Fix modifying of page protection by insert_pfn_pmd()
Hi, [This is an automated email] This commit has been processed because it contains a -stable tag. The stable tag indicates that it's relevant for the following trees: all The bot has tested the following trees: v5.0.5, v4.19.32, v4.14.109, v4.9.166, v4.4.177, v3.18.137. v5.0.5: Build OK! v4.19.32: Build OK! v4.14.109: Build OK! v4.9.166: Failed to apply! Possible dependencies: 82b0f8c39a38 ("mm: join struct fault_env and vm_fault") 953c66c2b22a ("mm: THP page cache support for ppc64") a00cc7d9dd93 ("mm, x86: add support for PUD-sized transparent hugepages") b5bc66b71310 ("mm: update mmu_gather range correctly") fd60775aea80 ("mm, thp: avoid unlikely branches for split_huge_pmd") v4.4.177: Failed to apply! Possible dependencies: 01871e59af5c ("mm, dax: fix livelock, allow dax pmd mappings to become writeable") 01c8f1c44b83 ("mm, dax, gpu: convert vm_insert_mixed to pfn_t") 0e749e54244e ("dax: increase granularity of dax_clear_blocks() operations") 34c0fd540e79 ("mm, dax, pmem: introduce pfn_t") 52db400fcd50 ("pmem, dax: clean up clear_pmem()") 606b5908 ("bpf: split HAVE_BPF_JIT into cBPF and eBPF variant") a00cc7d9dd93 ("mm, x86: add support for PUD-sized transparent hugepages") b2e0d1625e19 ("dax: fix lifetime of in-kernel dax mappings with dax_map_atomic()") b329f95d70f3 ("ARM: 8479/2: add implementation for arm-smccc") e37e43a497d5 ("x86/mm/64: Enable vmapped stacks (CONFIG_HAVE_ARCH_VMAP_STACK=y)") f25748e3c34e ("mm, dax: convert vmf_insert_pfn_pmd() to pfn_t") v3.18.137: Failed to apply! Possible dependencies: 047fc8a1f9a6 ("libnvdimm, nfit, nd_blk: driver for BLK-mode access persistent memory") 2a3746984c98 ("x86: Use new cache mode type in track_pfn_remap() and track_pfn_insert()") 34c0fd540e79 ("mm, dax, pmem: introduce pfn_t") 4c1eaa2344fb ("drivers/block/pmem: Fix 32-bit build warning in pmem_alloc()") 5cad465d7fa6 ("mm: add vmf_insert_pfn_pmd()") 61031952f4c8 ("arch, x86: pmem api for ensuring durability of persistent memory updates") 62232e45f4a2 ("libnvdimm: control (ioctl) messages for nvdimm_bus and nvdimm devices") 83e0abae ("staging: android: binder: move to the "real" part of the kernel") 957e3facd147 ("gcov: enable GCOV_PROFILE_ALL from ARCH Kconfigs") 9e853f2313e5 ("drivers/block/pmem: Add a driver for persistent memory") 9f53f9fa4ad1 ("libnvdimm, pmem: add libnvdimm support to the pmem driver") b94d5230d06e ("libnvdimm, nfit: initial libnvdimm infrastructure and NFIT support") cb389b9c0e00 ("dax: drop size parameter to ->direct_access()") dd22f551ac0a ("block: Change direct_access calling convention") e2e05394e4a3 ("pmem, dax: have direct_access use __pmem annotation") ec776ef6bbe1 ("x86/mm: Add support for the non-standard protected e820 type") f0dc089ce217 ("libnvdimm: enable iostat") f25748e3c34e ("mm, dax: convert vmf_insert_pfn_pmd() to pfn_t") How should we proceed with this patch? -- Thanks, Sasha ___ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm
Re: [PATCH v5 00/10] mm: Sub-section memory hotplug support
On 27.03.19 01:20, Dan Williams wrote: > On Tue, Mar 26, 2019 at 1:04 AM Michal Hocko wrote: >> >> On Mon 25-03-19 13:03:47, Dan Williams wrote: >>> On Mon, Mar 25, 2019 at 3:20 AM Michal Hocko wrote: >> [...] > User-defined memory namespaces have this problem, but 2MB is the > default alignment and is sufficient for most uses. What does prevent users to go and use a larger alignment? >>> >>> Given that we are living with 64MB granularity on mainstream platforms >>> for the foreseeable future, the reason users can't rely on a larger >>> alignment to address the issue is that the physical alignment may >>> change from one boot to the next. >> >> I would love to learn more about this inter boot volatility. Could you >> expand on that some more? I though that the HW configuration presented >> to the OS would be more or less stable unless the underlying HW changes. > > Even if the configuration is static there can be hardware failures > that prevent a DIMM, or a PCI device to be included in the memory map. > When that happens the BIOS needs to re-layout the map and the result > is not guaranteed to maintain the previous alignment. > >>> No, you can't just wish hardware / platform firmware won't do this, >>> because there are not enough platform resources to give every hardware >>> device a guaranteed alignment. >> >> Guarantee is one part and I can see how nobody wants to give you >> something as strong but how often does that happen in the real life? > > I expect a "rare" event to happen everyday in a data-center fleet. > Failure rates tend towards 100% daily occurrence at scale and in this > case the kernel has everything it needs to mitigate such an event. > > Setting aside the success rate of a software-alignment mitigation, the > reason I am charging this hill again after a 2 year hiatus is the > realization that this problem is wider spread than the original > failing scenario. Back in 2017 the problem seemed limited to custom > memmap= configurations, and collisions between PMEM and System RAM. > Now it is clear that the collisions can happen between PMEM regions > and namespaces as well, and the problem spans platforms from multiple > vendors. Here is the most recent collision problem: > https://github.com/pmem/ndctl/issues/76, from a third-party platform. > > The fix for that issue uncovered a bug in the padding implementation, > and a fix for that bug would result in even more hacks in the nvdimm > code for what is a core kernel deficiency. Code review of those > changes resulted in changing direction to go after the core > deficiency. > >>> The effect is that even if the driver deploys a software alignment >>> mitigation when it first sees the persistent memory range, that >>> alignment can be violated on a subsequent boot leading to data being >>> unavailable. There is no facility to communicate to the administrator >>> what went wrong in this scenario as several events can trigger a >>> physical map layout change. Add / remove of hardware and hardware >>> failure are the most likely causes. >> >> This is indeed bad and unexpected! That is exactly something to have in >> the chagelog! > > Apologies that was indeed included in the 2017 changelog (see: "a user > could inadvertently lose access to nvdimm namespaces" note here: > https://lwn.net/Articles/717383/), and I failed to carry it forward. > >> >>> An additional pain point for users is that EFI pre-boot environment >>> has little chance to create a namespace that Linux might be able to >>> use. The section size is an arbitrary Linux constraint and we should >>> not encode something Linux specific that might change in the future >>> into OS agnostic software. >> >> This looks like a fair point but please keep in mind that there hotplug >> restrictions are on other platforms as well (4MB on Windows IIRC) so >> there will be some knowledge required all the time. Besides that there >> are likely to be some restrictions depending on the implementation. > > Windows does not have an equivalent constraint, so it's only Linux > that imposes an arbitrary alignment restriction on pmem to agents like > EFI. > >> [...] > Right, as stated in the cover letter, this does not remove all those > assumptions, it only removes the ones that impact > devm_memremap_pages(). Specifying that sub-section is only supported > in the 'want_memblock=false' case to arch_add_memory(). And this is exactly the problem. Having different assumptions depending on whether there is a memblock interface or not is utterly wrong and a maintainability mess. >>> >>> In this case I disagree with you. The hotplug code already has the >>> want_memblock=false semantic in the implementation. >> >> want_memblock was a hack to allow memory hotplug to not have user >> visible sysfs interface. It was added to reduce the code duplication >> IIRC. Besides that this hasn't changed the underlying assumptions about >> hotplugable units
Returned mail: Data format error
This Message was undeliverable due to the following reason: Your message was not delivered because the destination computer was not reachable within the allowed queue period. The amount of time a message is queued before it is returned depends on local configura- tion parameters. Most likely there is a network problem that prevented delivery, but it is also possible that the computer is turned off, or does not have a mail system running right now. Your message was not delivered within 5 days: Host 154.69.21.136 is not responding. The following recipients did not receive this message: Please reply to postmas...@lists.01.org if you feel this message to be in error. ___ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm
RE: [ndctl PATCH] monitor: removing the requirement of default config
> -Original Message- > From: Verma, Vishal L [mailto:vishal.l.ve...@intel.com] > Sent: Saturday, March 30, 2019 2:37 AM > To: linux-nvdimm@lists.01.org; Qi, Fuli/斉 福利 > Subject: Re: [ndctl PATCH] monitor: removing the requirement of default > config > > > On Fri, 2019-03-29 at 20:01 +0900, QI Fuli wrote: > > Removing the requirement of default configuration file. > > If "-c, --config-file" option is not specified, default configuration > > file should be dispensable. > > > > Link: https://github.com/pmem/ndctl/issues/83 > > Signed-off-by: QI Fuli > > > > --- > > ndctl/monitor.c | 7 +-- > > 1 file changed, 5 insertions(+), 2 deletions(-) > > > > diff --git a/ndctl/monitor.c b/ndctl/monitor.c > > index 346a6df..6829a6b 100644 > > --- a/ndctl/monitor.c > > +++ b/ndctl/monitor.c > > @@ -484,8 +484,11 @@ static int read_config_file(struct ndctl_ctx *ctx, > struct monitor *_monitor, > > > > f = fopen(config_file, "r"); > > if (!f) { > > - err(, "config-file: %s cannot be opened\n", > config_file); > > - rc = -errno; > > + if (_monitor->config_file) { > > + err(, "config-file: %s cannot be > opened\n", > > + config_file); > > + rc = -errno; > > + } > > goto out; > > } > > > Hi Qi, > > Thanks for the quick patch. However, this makes it so that the only way > in which a config file gets used is by explicitly specifying it with a > -c option, and that kind of makes a 'default' config file pointless.. > > The ideal scenario would be: > 1. Check if default config exists >a. if it does, use options from there >b. if any cli options provided, use those to override the ones in > config file (if any) > > 2. If default config file is missing, only use cli options > > 3. If -c provided, use that, but perform the same treatment as > 1b > above, i.e. any cli options override anything in the config file from > -c > as well. Hi Vishal, Thanks for your comment. I think it might be misunderstood. Current scenario is: 1. Check if "-c " option is provided a. if it is, make the as config_file b. if it is not, make default configuration file(/etc/ndctl/monitor.conf) as config_file 2. Try to read options from config_file a. if config_file cannot be opened, then check if "-c" option is provided (1) if it is, stop starting monitor (user may make a wrong in "-c" option) (2) if it is not, finish read_config_file (default configuration file(/etc/ndctl/monitor.conf) is missing) b. if config_file can be opened successfully, merge the options from config_file with cli options It is consistent with the ideal scenario as you mentioned. Thanks, QI Fuli ___ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm