Re: [RFC PATCH] kvm: Use huge pages for DAX-backed files
On 2018-10-31 at 09:49 Paolo Bonzini wrote:
> > On 2018-10-29 at 20:10 Dan Williams wrote:
> >> The property of DAX pages that requires special coordination is the
> >> fact that the device hosting the pages can be disabled at will. The
> >> get_dev_pagemap() api is the interface to pin a device-pfn so that you
> >> can safely perform a pfn_to_page() operation.
> >>
> >> Have the pages that kvm uses in this path already been pinned by vfio?
>
> No, VFIO is not involved here.
>
> The pages that KVM uses are never pinned. Soon after we grab them and
> we build KVM's page table, we do put_page in mmu_set_spte (via
> kvm_release_pfn_clean). From that point on the MMU notifier will take
> care of invalidating SPT before the page disappears from the mm's page
> table.
I looked into this a little, and I think it's safe in terms of DAX's
get_dev_pagemap() refcounts to have kvm_is_reserved_pfn() treat
DAX pages as not reserved. kvm_is_reserved_pfn() is used before a
pfn_to_page() call, so the concern was that the pages didn't have a DAX
refcount.
Many of the times that KVM looks at the PFN, it's holding the KVM
mmu_lock, which keeps the pages in the host-side VMA. (IIUC).
Functions like kvm_set_pfn_dirty() fall into this category.
The other times, such as the vmexit path I mentioned before, go through
a gfn_to_pfn call. Under the hood, those call one of the
get_user_pages() functions during hva_to_pfn, and those do the
right thing w.r.t. get_dev_pagemap().
One of the other things I noticed was some places in KVM make a
distinction between kvm_is_reserved_pfn and PageReserved:
void kvm_set_pfn_dirty(kvm_pfn_t pfn)
{
if (!kvm_is_reserved_pfn(pfn)) {
struct page *page = pfn_to_page(pfn);
if (!PageReserved(page))
SetPageDirty(page);
}
}
I think we want to SetPageDirty for DAX, so making PageReserved be true
for DAX seems like the way to go, or we'll need more KVM-specific
changes. Apologies is this was discussed in the previous thread on this
topic and is redundant.
Thanks,
Barret
___
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
Re: [RFC v2 01/14] kunit: test: add KUnit test runner core
On 10/23/2018 05:57 PM, Brendan Higgins wrote:
> Add core facilities for defining unit tests; this provides a common way
> to define test cases, functions that execute code which is under test
> and determine whether the code under test behaves as expected; this also
> provides a way to group together related test cases in test suites (here
> we call them test_modules).
>
> Just define test cases and how to execute them for now; setting
> expectations on code will be defined later.
>
> Signed-off-by: Brendan Higgins
> ---
> include/kunit/test.h | 165 ++
> kunit/Kconfig| 17 +
> kunit/Makefile | 1 +
> kunit/test.c | 168 +++
> 4 files changed, 351 insertions(+)
> create mode 100644 include/kunit/test.h
> create mode 100644 kunit/Kconfig
> create mode 100644 kunit/Makefile
> create mode 100644 kunit/test.c
>
> diff --git a/include/kunit/test.h b/include/kunit/test.h
> new file mode 100644
> index 0..e0b14b227ac44
> --- /dev/null
> +++ b/include/kunit/test.h
> @@ -0,0 +1,165 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Base unit test (KUnit) API.
> + *
> + * Copyright (C) 2018, Google LLC.
> + * Author: Brendan Higgins
> + */
> +
> +#ifndef _KUNIT_TEST_H
> +#define _KUNIT_TEST_H
> +
> +#include
> +#include
> +
> +struct test;
> +
> +/**
> + * struct test_case - represents an individual test case.
> + * @run_case: the function representing the actual test case.
> + * @name: the name of the test case.
> + *
> + * A test case is a function with the signature, ``void (*)(struct test *)``
> + * that makes expectations (see TEST_EXPECT_TRUE()) about code under test.
> Each
> + * test case is associated with a &struct test_module and will be run after
> the
> + * module's init function and followed by the module's exit function.
> + *
> + * A test case should be static and should only be created with the
> TEST_CASE()
> + * macro; additionally, every array of test cases should be terminated with
> an
> + * empty test case.
> + *
> + * Example:
> + *
> + * .. code-block:: c
> + *
> + * void add_test_basic(struct test *test)
> + * {
> + * TEST_EXPECT_EQ(test, 1, add(1, 0));
> + * TEST_EXPECT_EQ(test, 2, add(1, 1));
> + * TEST_EXPECT_EQ(test, 0, add(-1, 1));
> + * TEST_EXPECT_EQ(test, INT_MAX, add(0, INT_MAX));
> + * TEST_EXPECT_EQ(test, -1, add(INT_MAX, INT_MIN));
> + * }
> + *
> + * static struct test_case example_test_cases[] = {
> + * TEST_CASE(add_test_basic),
> + * {},
> + * };
> + *
> + */
> +struct test_case {
> + void (*run_case)(struct test *test);
> + const char name[256];
> +
> + /* private: internal use only. */
> + bool success;
> +};
> +
Introducing a prefix kunit_* might be a good idea for the API.
This comment applies to the rest of patches as well.
thanks,
-- Shuah
___
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
Re: [RFC v2 00/14] kunit: introduce KUnit, the Linux kernel unit testing framework
Hi Brendan,
On 10/23/2018 05:57 PM, Brendan Higgins wrote:
> This patch set proposes KUnit, a lightweight unit testing and mocking
> framework for the Linux kernel.
>
> Unlike Autotest and kselftest, KUnit is a true unit testing framework;
> it does not require installing the kernel on a test machine or in a VM
> and does not require tests to be written in userspace running on a host
> kernel. Additionally, KUnit is fast: From invocation to completion KUnit
> can run several dozen tests in under a second. Currently, the entire
> KUnit test suite for KUnit runs in under a second from the initial
> invocation (build time excluded).
>
> KUnit is heavily inspired by JUnit, Python's unittest.mock, and
> Googletest/Googlemock for C++. KUnit provides facilities for defining
> unit test cases, grouping related test cases into test suites, providing
> common infrastructure for running tests, mocking, spying, and much more.
>
> ## What's so special about unit testing?
>
> A unit test is supposed to test a single unit of code in isolation,
> hence the name. There should be no dependencies outside the control of
> the test; this means no external dependencies, which makes tests orders
> of magnitudes faster. Likewise, since there are no external dependencies,
> there are no hoops to jump through to run the tests. Additionally, this
> makes unit tests deterministic: a failing unit test always indicates a
> problem. Finally, because unit tests necessarily have finer granularity,
> they are able to test all code paths easily solving the classic problem
> of difficulty in exercising error handling code.
>
> ## Is KUnit trying to replace other testing frameworks for the kernel?
>
> No. Most existing tests for the Linux kernel are end-to-end tests, which
> have their place. A well tested system has lots of unit tests, a
> reasonable number of integration tests, and some end-to-end tests. KUnit
> is just trying to address the unit test space which is currently not
> being addressed.
>
> ## More information on KUnit
>
> There is a bunch of documentation near the end of this patch set that
> describes how to use KUnit and best practices for writing unit tests.
> For convenience I am hosting the compiled docs here:
> https://google.github.io/kunit-docs/third_party/kernel/docs/
>
> ## Changes Since Last Version
>
> - Updated patchset to apply cleanly on 4.19.
> - Stripped down patchset to focus on just the core features (I dropped
>mocking, spying, and the MMIO stuff for now; you can find these
>patches here: https://kunit-review.googlesource.com/c/linux/+/1132),
>as suggested by Rob.
> - Cleaned up some of the commit messages and tweaked commit order a
>bit based on suggestions.
>
Framework looks good. I think it would be helpful to include a real test
in the patch series to get a feel for how effective it is.
On one hand, KUnit stands on its own as its own and maybe it should be placed in
under tools/testing/KUnit, however I am wondering would it be beneficial for the
framework to under selftests.
I am a bit concerned about the number of test framework we have at the moment
and
are we running the risk of fragmenting the landscape. I am concerned if this
would
lead to developer confusion as to where to add tests.
That being said, I don't have a strong opinion one way or the other.
btw I started playing with kunit following the instructions and ran into
problems:
./tools/testing/kunit/kunit.py
usage: kunit.py [-h] {run,new} ...
Helps writing and running KUnit tests.
positional arguments:
{run,new}
run Runs KUnit tests.
new Prints out boilerplate for writing new tests.
optional arguments:
-h, --help show this help message and exit
./tools/testing/kunit/kunit.py run
Regenerating .config ...
ERROR:root:Provided Kconfig is not contained in validated .config!
thanks,
-- Shuah
___
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
Change your password immediately. Your account has been hacked.
I greet you! I have bad news for you. 11/08/2018 - on this day I hacked your operating system and got full access to your account linux-nvdimm@lists.01.org It is useless to change the password, my malware intercepts it every time. How it was: In the software of the router to which you were connected that day, there was a vulnerability. I first hacked this router and placed my malicious code on it. When you entered in the Internet, my trojan was installed on the operating system of your device. After that, I made a full dump of your disk (I have all your address book, history of viewing sites, all files, phone numbers and addresses of all your contacts). A month ago, I wanted to lock your device and ask for a small amount of money to unlock. But I looked at the sites that you regularly visit, and came to the big delight of your favorite resources. I'm talking about sites for adults. I want to say - you are a big pervert. You have unbridled fantasy! After that, an idea came to my mind. I made a screenshot of the intimate website where you have fun (you know what it is about, right?). After that, I took off your joys (using the camera of your device). It turned out beautifully, do not hesitate. I am strongly belive that you would not like to show these pictures to your relatives, friends or colleagues. I think $867 is a very small amount for my silence. Besides, I spent a lot of time on you! I accept money only in Bitcoins. My BTC wallet: 17vzpL7n29egdeJF1hvUE4tKV81MqsW4wF You do not know how to replenish a Bitcoin wallet? In any search engine write "how to send money to btc wallet". It's easier than send money to a credit card! For payment you have a little more than two days (exactly 50 hours). Do not worry, the timer will start at the moment when you open this letter. Yes, yes .. it has already started! After payment, my virus and dirty photos with you self-destruct automatically. Narrative, if I do not receive the specified amount from you, then your device will be blocked, and all your contacts will receive a photos with your "joys". I want you to be prudent. - Do not try to find and destroy my virus! (All your data is already uploaded to a remote server) - Do not try to contact me (this is not feasible, I sent you an email from your account) - Various security services will not help you; formatting a disk or destroying a device will not help either, since your data is already on a remote server. P.S. I guarantee you that I will not disturb you again after payment, as you are not my single victim. This is a hacker code of honor. >From now on, I advise you to use good antiviruses and update them regularly >(several times a day)! Don't be mad at me, everyone has their own work. Farewell. ___ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm
