[Xen-devel] [qemu-mainline test] 133317: regressions - trouble: broken/fail/pass
flight 133317 qemu-mainline real [real] http://logs.test-lab.xenproject.org/osstest/logs/133317/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-i386-xl-qemuu-debianhvm-amd64-xsmbroken test-amd64-amd64-pairbroken test-amd64-i386-libvirt-xsm broken test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm 4 host-install(4) broken REGR. vs. 133284 test-amd64-amd64-pair 4 host-install/src_host(4) broken REGR. vs. 133284 test-amd64-i386-libvirt-xsm 4 host-install(4)broken REGR. vs. 133284 test-armhf-armhf-xl-arndale 6 xen-install fail REGR. vs. 133284 test-amd64-i386-qemuu-rhel6hvm-intel 12 guest-start/redhat.repeat fail REGR. vs. 133284 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow 10 debian-hvm-install fail REGR. vs. 133284 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm 10 debian-hvm-install fail REGR. vs. 133284 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 debian-hvm-install fail REGR. vs. 133284 test-amd64-amd64-xl-qemuu-ovmf-amd64 10 debian-hvm-install fail REGR. vs. 133284 test-amd64-amd64-libvirt-vhd 10 debian-di-installfail REGR. vs. 133284 test-amd64-i386-xl-qemuu-debianhvm-amd64 10 debian-hvm-install fail REGR. vs. 133284 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install fail REGR. vs. 133284 test-armhf-armhf-xl-vhd 10 debian-di-installfail REGR. vs. 133284 Tests which did not succeed, but are not blocking: test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 133284 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 133284 test-armhf-armhf-libvirt 14 saverestore-support-checkfail like 133284 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 133284 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail like 133284 test-amd64-i386-xl-pvshim12 guest-start fail never pass test-arm64-arm64-xl-credit1 13 migrate-support-checkfail never pass test-arm64-arm64-xl-credit1 14 saverestore-support-checkfail never pass test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail never pass test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail never pass test-arm64-arm64-xl-xsm 13 migrate-support-checkfail never pass test-arm64-arm64-xl-xsm 14 saverestore-support-checkfail never pass test-arm64-arm64-xl-credit2 13 migrate-support-checkfail never pass test-arm64-arm64-xl 13 migrate-support-checkfail never pass test-amd64-i386-libvirt 13 migrate-support-checkfail never pass test-arm64-arm64-xl-credit2 14 saverestore-support-checkfail never pass test-arm64-arm64-xl 14 saverestore-support-checkfail never pass test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check fail never pass test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2 fail never pass test-armhf-armhf-xl 13 migrate-support-checkfail never pass test-armhf-armhf-xl 14 saverestore-support-checkfail never pass test-amd64-amd64-libvirt 13 migrate-support-checkfail never pass test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail never pass test-armhf-armhf-xl-credit2 13 migrate-support-checkfail never pass test-armhf-armhf-xl-credit2 14 saverestore-support-checkfail never pass test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail never pass test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail never pass test-armhf-armhf-libvirt 13 migrate-support-checkfail never pass test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail never pass test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop fail never pass test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass test-armhf-armhf-xl-credit1 13 migrate-support-checkfail never pass test-armhf-armhf-xl-credit1 14 saverestore-support-checkfail never pass test-armhf-armhf-xl-rtds 13 migrate-support-checkfail never pass test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail never pass version targeted for testing: qemuu2e68b8620637a4ee8c79b5724144b726af1e261b baseline version: qemuu1e36232994c8ad01774501d2e299deba3a2469af Last test of basis 133284 2019-02-17 02:55:00 Z4 days Failing since133302 2019-02-18 14:42:26 Z2 days2 attempts Testing same since 133317 2019-02-20 00:05:34 Z1 days1
Re: [Xen-devel] [PATCH v4 3/4] x86/mm: handle foreign mappings in p2m_entry_modify
> From: Roger Pau Monné [mailto:roger@citrix.com]
> Sent: Tuesday, February 19, 2019 10:01 PM
>
> On Tue, Feb 19, 2019 at 06:14:00AM +, Tian, Kevin wrote:
> > > From: Roger Pau Monne [mailto:roger@citrix.com]
> > > Sent: Tuesday, February 19, 2019 1:27 AM
> > > diff --git a/xen/arch/x86/mm/shadow/common.c
> > > b/xen/arch/x86/mm/shadow/common.c
> > > index fe48c4a02b..ad670de515 100644
> > > --- a/xen/arch/x86/mm/shadow/common.c
> > > +++ b/xen/arch/x86/mm/shadow/common.c
> > > @@ -3189,7 +3189,8 @@ shadow_write_p2m_entry(struct domain *d,
> > > unsigned long gfn,
> > > sh_unshadow_for_p2m_change(d, gfn, p, new, level);
> > >
> > > p2m_entry_modify(p2m_get_hostp2m(d),
> > > p2m_flags_to_type(l1e_get_flags(new)),
> > > - p2m_flags_to_type(l1e_get_flags(*p)), level);
> > > + p2m_flags_to_type(l1e_get_flags(*p)),
> > > l1e_get_mfn(new),
> > > + l1e_get_mfn(*p), level);
> > >
> > > /* Update the entry with new content */
> > > safe_write_pte(p, new);
> > > diff --git a/xen/include/asm-x86/p2m.h b/xen/include/asm-x86/p2m.h
> > > index f4ec2becbd..2801a8ccca 100644
> > > --- a/xen/include/asm-x86/p2m.h
> > > +++ b/xen/include/asm-x86/p2m.h
> > > @@ -932,11 +932,14 @@ int p2m_set_ioreq_server(struct domain *d,
> > > unsigned int flags,
> > > struct hvm_ioreq_server *p2m_get_ioreq_server(struct domain *d,
> > >unsigned int *flags);
> > >
> > > -static inline void p2m_entry_modify(struct p2m_domain *p2m,
> > > p2m_type_t nt,
> > > -p2m_type_t ot, unsigned int level)
> > > +static inline int p2m_entry_modify(struct p2m_domain *p2m,
> p2m_type_t
> > > nt,
> > > + p2m_type_t ot, mfn_t nfn, mfn_t ofn,
> > > + unsigned int level)
> > > {
> > > -if ( level != 1 || nt == ot )
> > > -return;
> > > +BUG_ON(level > 1 && (nt == p2m_ioreq_server || nt ==
> > > p2m_map_foreign));
> > > +
> > > +if ( level != 1 || (nt == ot && mfn_eq(nfn, ofn)) )
> > > +return 0;
> >
> > could also return here in case of nt==ot==p2m_ioreq_server,
> > otherwise p2m->ioreq.entry_count might be unnecessarily
> > inc/dec if mfn changes while type stays as p2m_ioreq_server...
>
> The original code that open-coded the handling of p2m_ioreq_server
> didn't have this optimization, and as said by George I'm not sure the
> extra check is going to be worth it. I expect changing the mfn of an
> entry with type p2m_ioreq_server is not something very common.
>
OK. Reviewed-by: Kevin Tian
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [xen-unstable-smoke test] 133343: tolerable all pass - PUSHED
flight 133343 xen-unstable-smoke real [real] http://logs.test-lab.xenproject.org/osstest/logs/133343/ Failures :-/ but no regressions. Tests which did not succeed, but are not blocking: test-armhf-armhf-xl 13 migrate-support-checkfail never pass test-armhf-armhf-xl 14 saverestore-support-checkfail never pass test-amd64-amd64-libvirt 13 migrate-support-checkfail never pass test-arm64-arm64-xl-xsm 13 migrate-support-checkfail never pass test-arm64-arm64-xl-xsm 14 saverestore-support-checkfail never pass version targeted for testing: xen db2af23d15077605f286d8ef86c8f5d9c1b8302a baseline version: xen 1bcd0b43a16b7a48ec9afce3887c6c841b687abb Last test of basis 133312 2019-02-19 11:00:37 Z1 days Failing since133329 2019-02-20 12:00:39 Z0 days4 attempts Testing same since 17 2019-02-20 18:00:46 Z0 days3 attempts People who touched revisions under test: Andrew Cooper George Dunlap Jan Beulich Roger Pau Monne Roger Pau Monné Varad Gautam Wei Liu jobs: build-arm64-xsm pass build-amd64 pass build-armhf pass build-amd64-libvirt pass test-armhf-armhf-xl pass test-arm64-arm64-xl-xsm pass test-amd64-amd64-xl-qemuu-debianhvm-i386 pass test-amd64-amd64-libvirt pass sg-report-flight on osstest.test-lab.xenproject.org logs: /home/logs/logs images: /home/logs/images Logs, config files, etc. are available at http://logs.test-lab.xenproject.org/osstest/logs Explanation of these reports, and of osstest in general, is at http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master Test harness code can be found at http://xenbits.xen.org/gitweb?p=osstest.git;a=summary Pushing revision : To xenbits.xen.org:/home/xen/git/xen.git 1bcd0b43a1..db2af23d15 db2af23d15077605f286d8ef86c8f5d9c1b8302a -> smoke ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [RESEND PATCH 3/7] mm/gup: Change GUP fast to use flags rather than a write 'bool'
Hi Ira
Martin and I looked at your patch and agree that it doesn't change
functionality for Orangefs.
Reviewed-by: Mike Marshall
On Wed, Feb 20, 2019 at 12:32 AM wrote:
>
> From: Ira Weiny
>
> To facilitate additional options to get_user_pages_fast() change the
> singular write parameter to be gup_flags.
>
> This patch does not change any functionality. New functionality will
> follow in subsequent patches.
>
> Some of the get_user_pages_fast() call sites were unchanged because they
> already passed FOLL_WRITE or 0 for the write parameter.
>
> Signed-off-by: Ira Weiny
> ---
> arch/mips/mm/gup.c | 11 ++-
> arch/powerpc/kvm/book3s_64_mmu_hv.c| 4 ++--
> arch/powerpc/kvm/e500_mmu.c| 2 +-
> arch/powerpc/mm/mmu_context_iommu.c| 4 ++--
> arch/s390/kvm/interrupt.c | 2 +-
> arch/s390/mm/gup.c | 12 ++--
> arch/sh/mm/gup.c | 11 ++-
> arch/sparc/mm/gup.c| 9 +
> arch/x86/kvm/paging_tmpl.h | 2 +-
> arch/x86/kvm/svm.c | 2 +-
> drivers/fpga/dfl-afu-dma-region.c | 2 +-
> drivers/gpu/drm/via/via_dmablit.c | 3 ++-
> drivers/infiniband/hw/hfi1/user_pages.c| 3 ++-
> drivers/misc/genwqe/card_utils.c | 2 +-
> drivers/misc/vmw_vmci/vmci_host.c | 2 +-
> drivers/misc/vmw_vmci/vmci_queue_pair.c| 6 --
> drivers/platform/goldfish/goldfish_pipe.c | 3 ++-
> drivers/rapidio/devices/rio_mport_cdev.c | 4 +++-
> drivers/sbus/char/oradax.c | 2 +-
> drivers/scsi/st.c | 3 ++-
> drivers/staging/gasket/gasket_page_table.c | 4 ++--
> drivers/tee/tee_shm.c | 2 +-
> drivers/vfio/vfio_iommu_spapr_tce.c| 3 ++-
> drivers/vhost/vhost.c | 2 +-
> drivers/video/fbdev/pvr2fb.c | 2 +-
> drivers/virt/fsl_hypervisor.c | 2 +-
> drivers/xen/gntdev.c | 2 +-
> fs/orangefs/orangefs-bufmap.c | 2 +-
> include/linux/mm.h | 4 ++--
> kernel/futex.c | 2 +-
> lib/iov_iter.c | 7 +--
> mm/gup.c | 10 +-
> mm/util.c | 8
> net/ceph/pagevec.c | 2 +-
> net/rds/info.c | 2 +-
> net/rds/rdma.c | 3 ++-
> 36 files changed, 81 insertions(+), 65 deletions(-)
>
> diff --git a/arch/mips/mm/gup.c b/arch/mips/mm/gup.c
> index 0d14e0d8eacf..4c2b4483683c 100644
> --- a/arch/mips/mm/gup.c
> +++ b/arch/mips/mm/gup.c
> @@ -235,7 +235,7 @@ int __get_user_pages_fast(unsigned long start, int
> nr_pages, int write,
> * get_user_pages_fast() - pin user pages in memory
> * @start: starting user address
> * @nr_pages: number of pages from start to pin
> - * @write: whether pages will be written to
> + * @gup_flags: flags modifying pin behaviour
> * @pages: array that receives pointers to the pages pinned.
> * Should be at least nr_pages long.
> *
> @@ -247,8 +247,8 @@ int __get_user_pages_fast(unsigned long start, int
> nr_pages, int write,
> * requested. If nr_pages is 0 or negative, returns 0. If no pages
> * were pinned, returns -errno.
> */
> -int get_user_pages_fast(unsigned long start, int nr_pages, int write,
> - struct page **pages)
> +int get_user_pages_fast(unsigned long start, int nr_pages,
> + unsigned int gup_flags, struct page **pages)
> {
> struct mm_struct *mm = current->mm;
> unsigned long addr, len, end;
> @@ -273,7 +273,8 @@ int get_user_pages_fast(unsigned long start, int
> nr_pages, int write,
> next = pgd_addr_end(addr, end);
> if (pgd_none(pgd))
> goto slow;
> - if (!gup_pud_range(pgd, addr, next, write, pages, ))
> + if (!gup_pud_range(pgd, addr, next, gup_flags & FOLL_WRITE,
> + pages, ))
> goto slow;
> } while (pgdp++, addr = next, addr != end);
> local_irq_enable();
> @@ -289,7 +290,7 @@ int get_user_pages_fast(unsigned long start, int
> nr_pages, int write,
> pages += nr;
>
> ret = get_user_pages_unlocked(start, (end - start) >> PAGE_SHIFT,
> - pages, write ? FOLL_WRITE : 0);
> + pages, gup_flags);
>
> /* Have to be a bit careful with return values */
> if (nr > 0) {
> diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c
> b/arch/powerpc/kvm/book3s_64_mmu_hv.c
> index bd2dcfbf00cd..8fcb0a921e46 100644
> --- a/arch/powerpc/kvm/book3s_64_mmu_hv.c
> +++
[Xen-devel] [xen-unstable test] 133316: regressions - trouble: blocked/broken/fail/pass
flight 133316 xen-unstable real [real] http://logs.test-lab.xenproject.org/osstest/logs/133316/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-armhf-armhf-libvirt-raw broken build-i386-pvops broken test-amd64-amd64-xl-pvhv2-intel broken test-amd64-amd64-xl-qemut-win7-amd64 broken build-i386-pvops 4 host-install(4)broken REGR. vs. 133300 test-amd64-amd64-xl-qemut-win7-amd64 4 host-install(4) broken REGR. vs. 133300 test-amd64-amd64-xl-pvhv2-intel 4 host-install(4) broken REGR. vs. 133300 test-armhf-armhf-libvirt-raw 4 host-install(4)broken REGR. vs. 133300 test-xtf-amd64-amd64-37 xen-boot fail REGR. vs. 133300 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install fail REGR. vs. 133300 test-armhf-armhf-xl-vhd 10 debian-di-installfail REGR. vs. 133300 Tests which did not succeed, but are not blocking: test-amd64-i386-xl-qemut-debianhvm-amd64-xsm 1 build-check(1) blocked n/a test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 1 build-check(1) blocked n/a test-amd64-i386-xl1 build-check(1) blocked n/a test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a test-amd64-i386-xl-qemuu-win10-i386 1 build-check(1) blocked n/a test-amd64-i386-livepatch 1 build-check(1) blocked n/a test-amd64-i386-libvirt 1 build-check(1) blocked n/a test-amd64-i386-xl-qemut-win10-i386 1 build-check(1) blocked n/a test-amd64-i386-xl-qemuu-debianhvm-amd64 1 build-check(1) blocked n/a test-amd64-i386-xl-shadow 1 build-check(1) blocked n/a test-amd64-i386-xl-xsm1 build-check(1) blocked n/a test-amd64-i386-qemuu-rhel6hvm-amd 1 build-check(1) blocked n/a test-amd64-i386-xl-pvshim 1 build-check(1) blocked n/a test-amd64-i386-libvirt-pair 1 build-check(1) blocked n/a test-amd64-i386-freebsd10-i386 1 build-check(1) blocked n/a test-amd64-i386-xl-qemut-debianhvm-amd64 1 build-check(1) blocked n/a test-amd64-i386-examine 1 build-check(1) blocked n/a test-amd64-i386-libvirt-xsm 1 build-check(1) blocked n/a test-amd64-i386-xl-qemuu-ws16-amd64 1 build-check(1) blocked n/a test-amd64-i386-qemuu-rhel6hvm-intel 1 build-check(1) blocked n/a test-amd64-i386-xl-qemuu-ovmf-amd64 1 build-check(1) blocked n/a test-amd64-i386-xl-qemut-win7-amd64 1 build-check(1) blocked n/a test-amd64-i386-xl-qemut-ws16-amd64 1 build-check(1) blocked n/a test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 1 build-check(1) blocked n/a test-amd64-i386-qemut-rhel6hvm-intel 1 build-check(1) blocked n/a test-amd64-i386-pair 1 build-check(1) blocked n/a test-amd64-i386-freebsd10-amd64 1 build-check(1) blocked n/a test-amd64-i386-xl-raw1 build-check(1) blocked n/a test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked n/a test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a test-amd64-i386-rumprun-i386 1 build-check(1) blocked n/a test-amd64-i386-xl-qemuu-win7-amd64 1 build-check(1) blocked n/a test-amd64-i386-migrupgrade 1 build-check(1) blocked n/a test-amd64-i386-qemut-rhel6hvm-amd 1 build-check(1) blocked n/a test-armhf-armhf-libvirt 14 saverestore-support-checkfail like 133300 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 133300 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stopfail like 133300 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 133300 test-arm64-arm64-xl-credit1 13 migrate-support-checkfail never pass test-arm64-arm64-xl-xsm 13 migrate-support-checkfail never pass test-arm64-arm64-xl-credit1 14 saverestore-support-checkfail never pass test-arm64-arm64-xl-xsm 14 saverestore-support-checkfail never pass test-arm64-arm64-xl 13 migrate-support-checkfail never pass test-arm64-arm64-xl 14 saverestore-support-checkfail never pass test-arm64-arm64-xl-credit2 13 migrate-support-checkfail never pass test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail never pass test-arm64-arm64-xl-credit2 14 saverestore-support-checkfail never pass test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail never pass test-amd64-amd64-libvirt 13 migrate-support-checkfail never pass test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail
[Xen-devel] [ovmf test] 133318: trouble: broken/pass
flight 133318 ovmf real [real] http://logs.test-lab.xenproject.org/osstest/logs/133318/ Failures and problems with tests :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-amd64-xl-qemuu-ovmf-amd64 broken test-amd64-amd64-xl-qemuu-ovmf-amd64 4 host-install(4) broken REGR. vs. 133291 version targeted for testing: ovmf 68c67d3a2a33261e41ff0123129b4e9759617f71 baseline version: ovmf c417c1b33d06ef6ae96adb373201a5a3c3b38772 Last test of basis 133291 2019-02-18 01:41:15 Z3 days Failing since133305 2019-02-19 00:41:21 Z2 days2 attempts Testing same since 133318 2019-02-20 00:37:14 Z1 days1 attempts People who touched revisions under test: Bob Feng Dandan Bi Fan, ZhijuX Feng, Bob C Jiaxin Wu Jordan Justen Julien Grall Max Knutsen Ray Ni Ruiyu Ni Sami Mujawar Shenglei Zhang Star Zeng Wu Jiaxin Zhiju.Fan jobs: build-amd64-xsm pass build-i386-xsm pass build-amd64 pass build-i386 pass build-amd64-libvirt pass build-i386-libvirt pass build-amd64-pvopspass build-i386-pvops pass test-amd64-amd64-xl-qemuu-ovmf-amd64 broken test-amd64-i386-xl-qemuu-ovmf-amd64 pass sg-report-flight on osstest.test-lab.xenproject.org logs: /home/logs/logs images: /home/logs/images Logs, config files, etc. are available at http://logs.test-lab.xenproject.org/osstest/logs Explanation of these reports, and of osstest in general, is at http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master Test harness code can be found at http://xenbits.xen.org/gitweb?p=osstest.git;a=summary broken-job test-amd64-amd64-xl-qemuu-ovmf-amd64 broken broken-step test-amd64-amd64-xl-qemuu-ovmf-amd64 host-install(4) Not pushing. (No revision log; it would be 705 lines long.) ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [RESEND PATCH 3/7] mm/gup: Change GUP fast to use flags rather than a write 'bool'
Hi Ira,
On Wed, Feb 20, 2019 at 11:01 AM wrote:
>
> From: Ira Weiny
>
> To facilitate additional options to get_user_pages_fast() change the
> singular write parameter to be gup_flags.
>
> This patch does not change any functionality. New functionality will
> follow in subsequent patches.
>
> Some of the get_user_pages_fast() call sites were unchanged because they
> already passed FOLL_WRITE or 0 for the write parameter.
>
> Signed-off-by: Ira Weiny
> ---
> arch/mips/mm/gup.c | 11 ++-
> arch/powerpc/kvm/book3s_64_mmu_hv.c| 4 ++--
> arch/powerpc/kvm/e500_mmu.c| 2 +-
> arch/powerpc/mm/mmu_context_iommu.c| 4 ++--
> arch/s390/kvm/interrupt.c | 2 +-
> arch/s390/mm/gup.c | 12 ++--
> arch/sh/mm/gup.c | 11 ++-
> arch/sparc/mm/gup.c| 9 +
> arch/x86/kvm/paging_tmpl.h | 2 +-
> arch/x86/kvm/svm.c | 2 +-
> drivers/fpga/dfl-afu-dma-region.c | 2 +-
> drivers/gpu/drm/via/via_dmablit.c | 3 ++-
> drivers/infiniband/hw/hfi1/user_pages.c| 3 ++-
> drivers/misc/genwqe/card_utils.c | 2 +-
> drivers/misc/vmw_vmci/vmci_host.c | 2 +-
> drivers/misc/vmw_vmci/vmci_queue_pair.c| 6 --
> drivers/platform/goldfish/goldfish_pipe.c | 3 ++-
> drivers/rapidio/devices/rio_mport_cdev.c | 4 +++-
> drivers/sbus/char/oradax.c | 2 +-
> drivers/scsi/st.c | 3 ++-
> drivers/staging/gasket/gasket_page_table.c | 4 ++--
> drivers/tee/tee_shm.c | 2 +-
> drivers/vfio/vfio_iommu_spapr_tce.c| 3 ++-
> drivers/vhost/vhost.c | 2 +-
> drivers/video/fbdev/pvr2fb.c | 2 +-
> drivers/virt/fsl_hypervisor.c | 2 +-
> drivers/xen/gntdev.c | 2 +-
> fs/orangefs/orangefs-bufmap.c | 2 +-
> include/linux/mm.h | 4 ++--
> kernel/futex.c | 2 +-
> lib/iov_iter.c | 7 +--
> mm/gup.c | 10 +-
> mm/util.c | 8
> net/ceph/pagevec.c | 2 +-
> net/rds/info.c | 2 +-
> net/rds/rdma.c | 3 ++-
> 36 files changed, 81 insertions(+), 65 deletions(-)
>
> diff --git a/arch/mips/mm/gup.c b/arch/mips/mm/gup.c
> index 0d14e0d8eacf..4c2b4483683c 100644
> --- a/arch/mips/mm/gup.c
> +++ b/arch/mips/mm/gup.c
> @@ -235,7 +235,7 @@ int __get_user_pages_fast(unsigned long start, int
> nr_pages, int write,
> * get_user_pages_fast() - pin user pages in memory
> * @start: starting user address
> * @nr_pages: number of pages from start to pin
> - * @write: whether pages will be written to
> + * @gup_flags: flags modifying pin behaviour
> * @pages: array that receives pointers to the pages pinned.
> * Should be at least nr_pages long.
> *
> @@ -247,8 +247,8 @@ int __get_user_pages_fast(unsigned long start, int
> nr_pages, int write,
> * requested. If nr_pages is 0 or negative, returns 0. If no pages
> * were pinned, returns -errno.
> */
> -int get_user_pages_fast(unsigned long start, int nr_pages, int write,
> - struct page **pages)
> +int get_user_pages_fast(unsigned long start, int nr_pages,
> + unsigned int gup_flags, struct page **pages)
> {
> struct mm_struct *mm = current->mm;
> unsigned long addr, len, end;
> @@ -273,7 +273,8 @@ int get_user_pages_fast(unsigned long start, int
> nr_pages, int write,
> next = pgd_addr_end(addr, end);
> if (pgd_none(pgd))
> goto slow;
> - if (!gup_pud_range(pgd, addr, next, write, pages, ))
> + if (!gup_pud_range(pgd, addr, next, gup_flags & FOLL_WRITE,
> + pages, ))
> goto slow;
> } while (pgdp++, addr = next, addr != end);
> local_irq_enable();
> @@ -289,7 +290,7 @@ int get_user_pages_fast(unsigned long start, int
> nr_pages, int write,
> pages += nr;
>
> ret = get_user_pages_unlocked(start, (end - start) >> PAGE_SHIFT,
> - pages, write ? FOLL_WRITE : 0);
> + pages, gup_flags);
>
> /* Have to be a bit careful with return values */
> if (nr > 0) {
> diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c
> b/arch/powerpc/kvm/book3s_64_mmu_hv.c
> index bd2dcfbf00cd..8fcb0a921e46 100644
> --- a/arch/powerpc/kvm/book3s_64_mmu_hv.c
> +++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c
> @@ -582,7 +582,7 @@ int kvmppc_book3s_hv_page_fault(struct kvm_run *run,
> struct kvm_vcpu *vcpu,
> /*
[Xen-devel] [xen-unstable-smoke test] 133340: trouble: blocked/broken/pass
flight 133340 xen-unstable-smoke real [real] http://logs.test-lab.xenproject.org/osstest/logs/133340/ Failures and problems with tests :-( Tests which did not succeed and are blocking, including tests which could not be run: build-armhf broken build-armhf 4 host-install(4)broken REGR. vs. 133312 Tests which did not succeed, but are not blocking: test-armhf-armhf-xl 1 build-check(1) blocked n/a test-amd64-amd64-libvirt 13 migrate-support-checkfail never pass test-arm64-arm64-xl-xsm 13 migrate-support-checkfail never pass test-arm64-arm64-xl-xsm 14 saverestore-support-checkfail never pass version targeted for testing: xen db2af23d15077605f286d8ef86c8f5d9c1b8302a baseline version: xen 1bcd0b43a16b7a48ec9afce3887c6c841b687abb Last test of basis 133312 2019-02-19 11:00:37 Z1 days Failing since133329 2019-02-20 12:00:39 Z0 days3 attempts Testing same since 17 2019-02-20 18:00:46 Z0 days2 attempts People who touched revisions under test: Andrew Cooper George Dunlap Jan Beulich Roger Pau Monne Roger Pau Monné Varad Gautam Wei Liu jobs: build-arm64-xsm pass build-amd64 pass build-armhf broken build-amd64-libvirt pass test-armhf-armhf-xl blocked test-arm64-arm64-xl-xsm pass test-amd64-amd64-xl-qemuu-debianhvm-i386 pass test-amd64-amd64-libvirt pass sg-report-flight on osstest.test-lab.xenproject.org logs: /home/logs/logs images: /home/logs/images Logs, config files, etc. are available at http://logs.test-lab.xenproject.org/osstest/logs Explanation of these reports, and of osstest in general, is at http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master Test harness code can be found at http://xenbits.xen.org/gitweb?p=osstest.git;a=summary broken-job build-armhf broken broken-step build-armhf host-install(4) Not pushing. commit db2af23d15077605f286d8ef86c8f5d9c1b8302a Author: Jan Beulich Date: Wed Feb 20 17:07:17 2019 +0100 x86/shadow: don't pass wrong L4 MFN to guest_walk_tables() 64-bit PV guest user mode runs on a different L4 table. Make sure - the accessed bit gets set in the correct table (and in log-dirty mode the correct page gets marked dirty) during guest walks, - the correct table gets audited by sh_audit_gw(), - correct info gets logged by print_gw(). Signed-off-by: Jan Beulich Reviewed-by: Andrew Cooper Acked-by: George Dunlap Release-acked-by: Juergen Gross commit b22c900c44a2db8db1c53e269e152206e55c273f Author: Varad Gautam Date: Wed Feb 20 17:06:25 2019 +0100 x86/pmtimer: fix hvm_acpi_sleep_button behavior Commit 19fb14622e941 "x86/pmtimer: move ACPI registers from PMTState to hvm_domain" misconfigures pm1a_sts for hvm_acpi_sleep_button with PWRBTN_STS instead of SLPBTN_STS, which leads to XEN_DOMCTL_SENDTRIGGER_SLEEP causing guest powerdowns. Fix this. Signed-off-by: Varad Gautam Reviewed-by: Andrew Cooper commit 3c5552954c5c63860ccc01c6bc4f9c077bc26072 Author: Andrew Cooper Date: Fri Feb 1 16:56:38 2019 + x86/vpmu: Improve documentation and parsing for vpmu= The behaviour of vpmu= being exclusive of vpmu=bts|ipc|arch is odd and contrary to Xen's normal command line parsing behaviour. Rewrite the parsing to use the normal form, but retain the previous behaviour where the use of bts/ipc/arch implies vpmu=true. Parts of the documenation are stale, most notibly the HVM-only statement. Update it for consistency and correctness. Signed-off-by: Andrew Cooper Release-acked-by: Juergen Gross Reviewed-by: Jan Beulich commit 1e12872d29cc36c61894e347dd3409d7d206699d Author: Roger Pau Monne Date: Tue Feb 19 16:26:08 2019 +0100 libs/gnttab: add missing FreeBSD functions The FreeBSD implementation is missing the following functions: osdep_gnttab_dmabuf_exp_from_refs osdep_gnttab_dmabuf_exp_wait_released osdep_gnttab_dmabuf_imp_to_refs osdep_gnttab_dmabuf_imp_release Which all deal with dmabufs, that only exists on Linux. Implement them using abort, since such functions should never be called on FreeBSD. FTR, I
Re: [Xen-devel] [PATCH v4 0/9] mm: Use vm_map_pages() and vm_map_pages_zero() API
On Fri, Feb 15, 2019 at 8:06 AM Souptick Joarder wrote: > > Previouly drivers have their own way of mapping range of > kernel pages/memory into user vma and this was done by > invoking vm_insert_page() within a loop. > > As this pattern is common across different drivers, it can > be generalized by creating new functions and use it across > the drivers. > > vm_map_pages() is the API which could be used to map > kernel memory/pages in drivers which has considered vm_pgoff. > > vm_map_pages_zero() is the API which could be used to map > range of kernel memory/pages in drivers which has not considered > vm_pgoff. vm_pgoff is passed default as 0 for those drivers. > > We _could_ then at a later "fix" these drivers which are using > vm_map_pages_zero() to behave according to the normal vm_pgoff > offsetting simply by removing the _zero suffix on the function > name and if that causes regressions, it gives us an easy way to revert. > > Tested on Rockchip hardware and display is working fine, including talking > to Lima via prime. > > v1 -> v2: > Few Reviewed-by. > > Updated the change log in [8/9] > > In [7/9], vm_pgoff is treated in V4L2 API as a 'cookie' > to select a buffer, not as a in-buffer offset by design > and it always want to mmap a whole buffer from its beginning. > Added additional changes after discussing with Marek and > vm_map_pages() could be used instead of vm_map_pages_zero(). > > v2 -> v3: > Corrected the documentation as per review comment. > > As suggested in v2, renaming the interfaces to - > *vm_insert_range() -> vm_map_pages()* and > *vm_insert_range_buggy() -> vm_map_pages_zero()*. > As the interface is renamed, modified the code accordingly, > updated the change logs and modified the subject lines to use the > new interfaces. There is no other change apart from renaming and > using the new interface. > > Patch[1/9] & [4/9], Tested on Rockchip hardware. > > v3 -> v4: > Fixed build warnings on patch [8/9] reported by kbuild test robot. > > Souptick Joarder (9): > mm: Introduce new vm_map_pages() and vm_map_pages_zero() API > arm: mm: dma-mapping: Convert to use vm_map_pages() > drivers/firewire/core-iso.c: Convert to use vm_map_pages_zero() > drm/rockchip/rockchip_drm_gem.c: Convert to use vm_map_pages() > drm/xen/xen_drm_front_gem.c: Convert to use vm_map_pages() > iommu/dma-iommu.c: Convert to use vm_map_pages() > videobuf2/videobuf2-dma-sg.c: Convert to use vm_map_pages() > xen/gntdev.c: Convert to use vm_map_pages() > xen/privcmd-buf.c: Convert to use vm_map_pages_zero() If no further comment, is it fine to take these patches to -mm tree for regression ? > > arch/arm/mm/dma-mapping.c | 22 ++ > drivers/firewire/core-iso.c| 15 +--- > drivers/gpu/drm/rockchip/rockchip_drm_gem.c| 17 + > drivers/gpu/drm/xen/xen_drm_front_gem.c| 18 ++--- > drivers/iommu/dma-iommu.c | 12 +--- > drivers/media/common/videobuf2/videobuf2-core.c| 7 ++ > .../media/common/videobuf2/videobuf2-dma-contig.c | 6 -- > drivers/media/common/videobuf2/videobuf2-dma-sg.c | 22 ++ > drivers/xen/gntdev.c | 11 ++- > drivers/xen/privcmd-buf.c | 8 +-- > include/linux/mm.h | 4 ++ > mm/memory.c| 81 > ++ > mm/nommu.c | 14 > 13 files changed, 134 insertions(+), 103 deletions(-) > > -- > 1.9.1 > ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH RFC 00/39] x86/KVM: Xen HVM guest support
On 2/20/19 3:39 PM, Marek Marczykowski-Górecki wrote: On Wed, Feb 20, 2019 at 08:15:30PM +, Joao Martins wrote: 2. PV Driver support (patches 17 - 39) We start by redirecting hypercalls from the backend to routines which emulate the behaviour that PV backends expect i.e. grant table and interdomain events. Next, we add support for late initialization of xenbus, followed by implementing frontend/backend communication mechanisms (i.e. grant tables and interdomain event channels). Finally, introduce xen-shim.ko, which will setup a limited Xen environment. This uses the added functionality of Xen specific shared memory (grant tables) and notifications (event channels). Does it mean backends could be run in another guest, similarly as on real Xen? AFAIK virtio doesn't allow that as virtio backends need I'm afraid not. For now grant operations (map/unmap) can only be done by backends to the local KVM instance. Ankur arbitrary write access to guest memory. But grant tables provide enough abstraction to do that safely. ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH RFC 00/39] x86/KVM: Xen HVM guest support
On 2/20/19 1:09 PM, Paolo Bonzini wrote: On 20/02/19 21:15, Joao Martins wrote: 2. PV Driver support (patches 17 - 39) We start by redirecting hypercalls from the backend to routines which emulate the behaviour that PV backends expect i.e. grant table and interdomain events. Next, we add support for late initialization of xenbus, followed by implementing frontend/backend communication mechanisms (i.e. grant tables and interdomain event channels). Finally, introduce xen-shim.ko, which will setup a limited Xen environment. This uses the added functionality of Xen specific shared memory (grant tables) and notifications (event channels). I am a bit worried by the last patches, they seem really brittle and prone to breakage. I don't know Xen well enough to understand if the lack of support for GNTMAP_host_map is fixable, but if not, you have to define a completely different hypercall. I assume you are aware of most of this, but just in case, here's the flow when a backend driver wants to map a grant-reference in the host: it allocates an empty struct page (via ballooning) and does a map_grant_ref(GNTMAP_host_map) hypercall. In response, Xen validates the grant-reference and maps it onto the address associated with the struct page. After this, from the POV of the underlying network/block drivers, these struct pages can be used as just regular pages. To support this in a KVM environment, where AFAICS no remapping of pages is possible, the idea was to make minimal changes to the backend drivers such that map_grant_ref() could just return the PFN from which the backend could derive the struct page. To ensure that backends -- when running in this environment -- have been modified to deal with these new semantics, our map_grant_ref() implementation explicitly disallows the GNTMAP_host_map flag. Now if I'm reading you right, you would prefer something more straightforward -- perhaps similar semantics but a new flag that makes this behaviour explicit? Of course, tests are missing. You should use the tools/testing/selftests/kvm/ framework, and ideally each patch should come with coverage for the newly-added code. Agreed. Thanks Ankur Thanks, Paolo ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [xen-4.9-testing test] 133314: regressions - trouble: blocked/broken/fail/pass
flight 133314 xen-4.9-testing real [real] http://logs.test-lab.xenproject.org/osstest/logs/133314/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-amd64-i386-pvgrub broken test-amd64-amd64-xl-qemuu-win10-i386 broken test-amd64-amd64-xl broken build-armhf-pvopsbroken test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm broken test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow broken test-amd64-amd64-xl-qemut-debianhvm-amd64 broken test-amd64-amd64-xl-xsm broken test-amd64-amd64-pygrub broken test-amd64-i386-xl-qemuu-ovmf-amd64 broken test-xtf-amd64-amd64-2 broken test-amd64-i386-xl-qemut-win10-i386 broken build-armhf-pvops 4 host-install(4)broken REGR. vs. 132889 build-arm64 broken in 133252 build-arm64-xsm broken in 133252 build-arm64-pvopsbroken in 133252 build-arm644 host-install(4) broken in 133252 REGR. vs. 132889 build-arm64-pvops 4 host-install(4) broken in 133252 REGR. vs. 132889 build-arm64-xsm4 host-install(4) broken in 133252 REGR. vs. 132889 test-amd64-i386-libvirt-pair 22 guest-migrate/src_host/dst_host fail REGR. vs. 132889 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop fail in 133252 REGR. vs. 132889 build-i386-pvops 6 kernel-build fail in 133295 REGR. vs. 132889 Tests which are failing intermittently (not blocking): test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 4 host-install(4) broken pass in 133281 test-amd64-i386-xl-qemut-win10-i386 4 host-install(4) broken pass in 133281 test-amd64-i386-xl-qemuu-ovmf-amd64 4 host-install(4) broken pass in 133281 test-amd64-amd64-pygrub 4 host-install(4) broken pass in 133295 test-xtf-amd64-amd64-24 host-install(4) broken pass in 133295 test-amd64-amd64-xl-qemut-debianhvm-amd64 4 host-install(4) broken pass in 133295 test-amd64-amd64-i386-pvgrub 4 host-install(4) broken pass in 133295 test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm 4 host-install(4) broken pass in 133295 test-amd64-amd64-xl 4 host-install(4) broken pass in 133295 test-amd64-amd64-xl-xsm 4 host-install(4) broken pass in 133295 test-amd64-amd64-xl-qemuu-win10-i386 4 host-install(4) broken pass in 133295 test-amd64-i386-xl-qemut-ws16-amd64 14 guest-localmigrate fail in 133252 pass in 133314 test-amd64-amd64-xl-qemut-debianhvm-amd64 13 guest-saverestore fail in 133281 pass in 133295 test-armhf-armhf-libvirt 6 xen-install fail in 133281 pass in 133295 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow 15 guest-saverestore.2 fail in 133281 pass in 133314 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-localmigrate/x10 fail in 133281 pass in 133314 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 15 guest-saverestore.2 fail in 133281 pass in 133314 test-amd64-amd64-xl-qemuu-ws16-amd64 16 guest-localmigrate/x10 fail in 133281 pass in 133314 test-amd64-i386-xl-qemut-win7-amd64 15 guest-saverestore.2 fail in 133281 pass in 133314 test-amd64-amd64-xl-qemut-ws16-amd64 16 guest-localmigrate/x10 fail in 133295 pass in 133252 test-amd64-i386-xl-qemuu-debianhvm-amd64 10 debian-hvm-install fail pass in 133281 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 debian-hvm-install fail pass in 133281 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm 10 debian-hvm-install fail pass in 133281 test-amd64-i386-xl-qemut-ws16-amd64 16 guest-localmigrate/x10 fail pass in 133281 test-amd64-amd64-libvirt-pair 22 guest-migrate/src_host/dst_host fail pass in 133295 test-amd64-amd64-xl-qemut-ws16-amd64 14 guest-localmigrate fail pass in 133295 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm 10 debian-hvm-install fail pass in 133295 test-amd64-amd64-qemuu-nested-intel 17 debian-hvm-install/l1/l2 fail pass in 133295 test-amd64-amd64-libvirt-vhd 10 debian-di-install fail pass in 133295 Tests which did not succeed, but are not blocking: test-arm64-arm64-libvirt-xsm 1 build-check(1) blocked in 133252 n/a build-arm64-libvirt 1 build-check(1) blocked in 133252 n/a test-arm64-arm64-xl-credit2 1 build-check(1) blocked in 133252 n/a test-arm64-arm64-xl-credit1 1 build-check(1) blocked in 133252 n/a test-arm64-arm64-xl-xsm 1 build-check(1) blocked in 133252 n/a test-arm64-arm64-xl 1 build-check(1) blocked in 133252 n/a test-amd64-i386-xl-xsm1 build-check(1) blocked in 133295 n/a test-amd64-i386-xl-qemuu-win7-amd64 1 build-check(1)blocked in 133295 n/a
Re: [Xen-devel] [PATCH RFC 00/39] x86/KVM: Xen HVM guest support
On Wed, Feb 20, 2019 at 08:15:30PM +, Joao Martins wrote: > 2. PV Driver support (patches 17 - 39) > > We start by redirecting hypercalls from the backend to routines > which emulate the behaviour that PV backends expect i.e. grant > table and interdomain events. Next, we add support for late > initialization of xenbus, followed by implementing > frontend/backend communication mechanisms (i.e. grant tables and > interdomain event channels). Finally, introduce xen-shim.ko, > which will setup a limited Xen environment. This uses the added > functionality of Xen specific shared memory (grant tables) and > notifications (event channels). Does it mean backends could be run in another guest, similarly as on real Xen? AFAIK virtio doesn't allow that as virtio backends need arbitrary write access to guest memory. But grant tables provide enough abstraction to do that safely. -- Best Regards, Marek Marczykowski-Górecki Invisible Things Lab A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? signature.asc Description: PGP signature ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [linux-linus test] 133313: regressions - trouble: blocked/broken/fail/pass
flight 133313 linux-linus real [real] http://logs.test-lab.xenproject.org/osstest/logs/133313/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: build-armhf broken test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow broken test-amd64-i386-xl-qemut-win7-amd64 broken test-amd64-amd64-xl-xsm broken test-amd64-amd64-pairbroken test-amd64-i386-xl-xsm broken test-amd64-amd64-pair 4 host-install/src_host(4) broken REGR. vs. 132911 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 4 host-install(4) broken REGR. vs. 132911 test-amd64-i386-xl-qemut-win7-amd64 4 host-install(4) broken REGR. vs. 132911 test-amd64-i386-xl-xsm4 host-install(4)broken REGR. vs. 132911 test-amd64-amd64-xl-xsm 4 host-install(4)broken REGR. vs. 132911 build-armhf 4 host-install(4)broken REGR. vs. 132911 test-arm64-arm64-libvirt-xsm 16 guest-start/debian.repeat fail REGR. vs. 132911 test-amd64-i386-xl-qemut-debianhvm-amd64-xsm 10 debian-hvm-install fail REGR. vs. 132911 test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm 10 debian-hvm-install fail REGR. vs. 132911 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install fail REGR. vs. 132911 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm 10 debian-hvm-install fail REGR. vs. 132911 test-amd64-i386-xl-qemuu-debianhvm-amd64 10 debian-hvm-install fail REGR. vs. 132911 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm 10 debian-hvm-install fail REGR. vs. 132911 test-amd64-i386-xl-qemuu-ovmf-amd64 10 debian-hvm-install fail REGR. vs. 132911 test-amd64-amd64-qemuu-nested-intel 17 debian-hvm-install/l1/l2 fail REGR. vs. 132911 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow 10 debian-hvm-install fail REGR. vs. 132911 test-amd64-amd64-i386-pvgrub 10 debian-di-installfail REGR. vs. 132911 Tests which did not succeed, but are not blocking: test-armhf-armhf-xl-rtds 1 build-check(1) blocked n/a test-armhf-armhf-xl-vhd 1 build-check(1) blocked n/a build-armhf-libvirt 1 build-check(1) blocked n/a test-armhf-armhf-xl-multivcpu 1 build-check(1) blocked n/a test-armhf-armhf-xl 1 build-check(1) blocked n/a test-armhf-armhf-libvirt 1 build-check(1) blocked n/a test-armhf-armhf-xl-credit2 1 build-check(1) blocked n/a test-armhf-armhf-xl-credit1 1 build-check(1) blocked n/a test-armhf-armhf-libvirt-raw 1 build-check(1) blocked n/a test-armhf-armhf-xl-arndale 1 build-check(1) blocked n/a test-armhf-armhf-examine 1 build-check(1) blocked n/a test-armhf-armhf-xl-cubietruck 1 build-check(1) blocked n/a test-amd64-amd64-examine 4 memdisk-try-append fail like 132804 test-amd64-amd64-rumprun-amd64 17 rumprun-demo-xenstorels/xenstorels.repeat fail like 132911 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 132911 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 132911 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 132911 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stopfail like 132911 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 132911 test-amd64-amd64-libvirt 13 migrate-support-checkfail never pass test-amd64-i386-xl-pvshim12 guest-start fail never pass test-arm64-arm64-xl-credit2 13 migrate-support-checkfail never pass test-arm64-arm64-xl-credit2 14 saverestore-support-checkfail never pass test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail never pass test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail never pass test-arm64-arm64-xl 13 migrate-support-checkfail never pass test-arm64-arm64-xl-credit1 13 migrate-support-checkfail never pass test-arm64-arm64-xl 14 saverestore-support-checkfail never pass test-arm64-arm64-xl-credit1 14 saverestore-support-checkfail never pass test-amd64-i386-libvirt 13 migrate-support-checkfail never pass test-arm64-arm64-xl-xsm 13 migrate-support-checkfail never pass test-arm64-arm64-xl-xsm 14 saverestore-support-checkfail never pass test-amd64-i386-libvirt-xsm 13 migrate-support-checkfail never pass test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail never pass test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check fail never pass test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check fail never pass test-amd64-amd64-libvirt-vhd 12
Re: [Xen-devel] xen/evtchn and forced threaded irq
Hi Boris,
On 2/20/19 9:46 PM, Boris Ostrovsky wrote:
On 2/20/19 3:46 PM, Julien Grall wrote:
(+ Andrew and Jan for feedback on the event channel interrupt)
Hi Boris,
Thank you for the your feedback.
On 2/20/19 8:04 PM, Boris Ostrovsky wrote:
On 2/20/19 1:05 PM, Julien Grall wrote:
Hi,
On 20/02/2019 17:07, Boris Ostrovsky wrote:
On 2/20/19 9:15 AM, Julien Grall wrote:
Hi Boris,
Thank you for your answer.
On 20/02/2019 00:02, Boris Ostrovsky wrote:
On Tue, Feb 19, 2019 at 05:31:10PM +, Julien Grall wrote:
Hi all,
I have been looking at using Linux RT in Dom0. Once the guest is
started,
the console is ending to have a lot of warning (see trace below).
After some investigation, this is because the irq handler will now
be threaded.
I can reproduce the same error with the vanilla Linux when passing
the option
'threadirqs' on the command line (the trace below is from 5.0.0-rc7
that has
not RT support).
FWIW, the interrupt for port 6 is used to for the guest to
communicate with
xenstore.
From my understanding, this is happening because the interrupt
handler is now
run in a thread. So we can have the following happening.
Interrupt context | Interrupt thread
|
receive interrupt port 6 |
clear the evtchn port |
set IRQF_RUNTHREAD |
kick interrupt thread |
| clear IRQF_RUNTHREAD
| call evtchn_interrupt
receive interrupt port 6 |
clear the evtchn port |
set IRQF_RUNTHREAD |
kick interrupt thread |
| disable interrupt port 6
| evtchn->enabled = false
| []
|
| *** Handling the second
interrupt ***
| clear IRQF_RUNTHREAD
| call evtchn_interrupt
| WARN(...)
I am not entirely sure how to fix this. I have two solutions in
mind:
1) Prevent the interrupt handler to be threaded. We would also
need to
switch from spin_lock to raw_spin_lock as the former may sleep on
RT-Linux.
2) Remove the warning
I think access to evtchn->enabled is racy so (with or without the
warning) we can't use it reliably.
Thinking about it, it would not be the only issue. The ring is sized
to contain only one instance of the same event. So if you receive
twice the event, you may overflow the ring.
Hm... That's another argument in favor of "unthreading" the handler.
I first thought it would be possible to unthread it. However,
wake_up_interruptible is using a spin_lock. On RT spin_lock can sleep,
so this cannot be used in an interrupt context.
So I think "unthreading" the handler is not an option here.
That sounds like a different problem. I.e. there are two issues:
* threaded interrupts don't work properly (races, ring overflow)
* evtchn_interrupt() (threaded or not) has spin_lock(), which is not
going to work for RT
I am afraid that's not correct, you can use spin_lock() in threaded
interrupt handler.
In non-RT handler -- yes, but not in an RT one (in fact, isn't this what
you yourself said above?)
In RT-linux, interrupt handlers are threaded by default. So the handler
will not run in the interrupt context. Hence, it will be safe to call
spin_lock.
However, if you force the handler to not be threaded (IRQF_NO_THREAD),
it will run in interrupt context.
Another alternative could be to queue the irq if !evtchn->enabled
and
handle it in evtchn_write() (which is where irq is supposed to be
re-enabled).
What do you mean by queue? Is it queueing in the ring?
No, I was thinking about having a new structure for deferred
interrupts.
Hmmm, I am not entirely sure what would be the structure here. Could
you expand your thinking?
Some sort of a FIFO that stores {irq, data} tuple. It could obviously be
implemented as a ring but not necessarily as Xen shared ring (if that's
what you were referring to).
The underlying question is what happen if you miss an interrupt. Is it
going to be ok?
This I am not sure about. I thought yes since we are signaling the
process only once.
I have CCed Andrew and Jan to see if they can help here.
Cheers,
--
Julien Grall
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] xen/evtchn and forced threaded irq
On 2/20/19 3:46 PM, Julien Grall wrote:
> (+ Andrew and Jan for feedback on the event channel interrupt)
>
> Hi Boris,
>
> Thank you for the your feedback.
>
> On 2/20/19 8:04 PM, Boris Ostrovsky wrote:
>> On 2/20/19 1:05 PM, Julien Grall wrote:
>>> Hi,
>>>
>>> On 20/02/2019 17:07, Boris Ostrovsky wrote:
On 2/20/19 9:15 AM, Julien Grall wrote:
> Hi Boris,
>
> Thank you for your answer.
>
> On 20/02/2019 00:02, Boris Ostrovsky wrote:
>> On Tue, Feb 19, 2019 at 05:31:10PM +, Julien Grall wrote:
>>> Hi all,
>>>
>>> I have been looking at using Linux RT in Dom0. Once the guest is
>>> started,
>>> the console is ending to have a lot of warning (see trace below).
>>>
>>> After some investigation, this is because the irq handler will now
>>> be threaded.
>>> I can reproduce the same error with the vanilla Linux when passing
>>> the option
>>> 'threadirqs' on the command line (the trace below is from 5.0.0-rc7
>>> that has
>>> not RT support).
>>>
>>> FWIW, the interrupt for port 6 is used to for the guest to
>>> communicate with
>>> xenstore.
>>>
>>> From my understanding, this is happening because the interrupt
>>> handler is now
>>> run in a thread. So we can have the following happening.
>>>
>>> Interrupt context | Interrupt thread
>>> |
>>> receive interrupt port 6 |
>>> clear the evtchn port |
>>> set IRQF_RUNTHREAD |
>>> kick interrupt thread |
>>> | clear IRQF_RUNTHREAD
>>> | call evtchn_interrupt
>>> receive interrupt port 6 |
>>> clear the evtchn port |
>>> set IRQF_RUNTHREAD |
>>> kick interrupt thread |
>>> | disable interrupt port 6
>>> | evtchn->enabled = false
>>> | []
>>> |
>>> | *** Handling the second
>>> interrupt ***
>>> | clear IRQF_RUNTHREAD
>>> | call evtchn_interrupt
>>> | WARN(...)
>>>
>>> I am not entirely sure how to fix this. I have two solutions in
>>> mind:
>>>
>>> 1) Prevent the interrupt handler to be threaded. We would also
>>> need to
>>> switch from spin_lock to raw_spin_lock as the former may sleep on
>>> RT-Linux.
>>>
>>> 2) Remove the warning
>>
>> I think access to evtchn->enabled is racy so (with or without the
>> warning) we can't use it reliably.
>
> Thinking about it, it would not be the only issue. The ring is sized
> to contain only one instance of the same event. So if you receive
> twice the event, you may overflow the ring.
Hm... That's another argument in favor of "unthreading" the handler.
>>>
>>> I first thought it would be possible to unthread it. However,
>>> wake_up_interruptible is using a spin_lock. On RT spin_lock can sleep,
>>> so this cannot be used in an interrupt context.
>>>
>>> So I think "unthreading" the handler is not an option here.
>>
>> That sounds like a different problem. I.e. there are two issues:
>> * threaded interrupts don't work properly (races, ring overflow)
>> * evtchn_interrupt() (threaded or not) has spin_lock(), which is not
>> going to work for RT
>
> I am afraid that's not correct, you can use spin_lock() in threaded
> interrupt handler.
In non-RT handler -- yes, but not in an RT one (in fact, isn't this what
you yourself said above?)
>
>> The first can be fixed by using non-threaded handlers.
>
> The two are somewhat related, if you use a non-threaded handler then
> you are not going to help the RT case.
>
> In general, the unthreaded solution should be used in the last resort.
>
>>>
>
>>
>> Another alternative could be to queue the irq if !evtchn->enabled
>> and
>> handle it in evtchn_write() (which is where irq is supposed to be
>> re-enabled).
> What do you mean by queue? Is it queueing in the ring?
No, I was thinking about having a new structure for deferred
interrupts.
>>>
>>> Hmmm, I am not entirely sure what would be the structure here. Could
>>> you expand your thinking?
>>
>> Some sort of a FIFO that stores {irq, data} tuple. It could obviously be
>> implemented as a ring but not necessarily as Xen shared ring (if that's
>> what you were referring to).
>
> The underlying question is what happen if you miss an interrupt. Is it
> going to be ok?
This I am not sure about. I thought yes since we are signaling the
process only once.
-boris
> If no, then we
Re: [Xen-devel] XEN on R-CAR H3
ср, 20 февр. 2019 г., 22:14 Julien Grall :
> Hi Amit,
Hi, Julien, Amit.
Sorry for formatting, writing from my mobile.
If I am not mistaken, the diff between BSP's and mainline device trees is
in reserved memory area. BSP device tree (1) contains reserved memory
regions, but the mainline one (2) doesn't.
>From the log you provided, I see that Xen is trying to copy device-tree to
the address which is located in reserved area (0x5800). FYI, we always
remove these reserved area nodes from the device-tree. Maybe that's why we
didn't face an issue. Julien, what do you think, can this be a reason?
(1)
https://git.kernel.org/pub/scm/linux/kernel/git/horms/renesas-bsp.git/tree/arch/arm64/boot/dts/renesas/r8a7795-h3ulcb.dts?h=v4.14.75-ltsi/rcar-3.9.3.rc1
(2)
https://elixir.bootlin.com/linux/v5.0-rc7/source/arch/arm64/boot/dts/renesas/r8a7795-h3ulcb.dts
> Thank you for the report.
>
> On 2/19/19 4:46 PM, Amit Tomer wrote:
> > (XEN) CPU7 MIDR (0x410fd034) does not match boot CPU MIDR (0x411fd073),
> > (XEN) disable cpu (see big.LITTLE.txt under docs/).
> > (XEN) CPU7 never came online
> > (XEN) Failed to bring up CPU 7 (error -5)
> > (XEN) Brought up 4 CPUs
> > (XEN) P2M: 44-bit IPA with 44-bit PA and 8-bit VMID
> > (XEN) P2M: 4 levels with order-0 root, VTCR 0x80043594
> > (XEN) I/O virtualisation disabled
> > (XEN) build-id: 74f80103afa98953c029eea87d69696bcd5ef69d
> > (XEN) alternatives: Patching with alt table 002abba8 ->
> 002ac1f0
> > (XEN) CPU0 will call ARM_SMCCC_ARCH_WORKAROUND_1 on exception entry
> > (XEN) CPU2 will call ARM_SMCCC_ARCH_WORKAROUND_1 on exception entry
> > (XEN) CPU3 will call ARM_SMCCC_ARCH_WORKAROUND_1 on exception entry
> > (XEN) CPU1 will call ARM_SMCCC_ARCH_WORKAROUND_1 on exception entry
> > (XEN) *** LOADING DOMAIN 0 ***
> > (XEN) Loading Domd0 kernel from boot module @ 7a00
> > (XEN) Allocating 1:1 mappings totalling 512MB for dom0:
> > (XEN) BANK[0] 0x005000-0x007000 (512MB)
> > (XEN) Grant table range: 0x004800-0x004804
> > (XEN) Allocating PPI 16 for event channel interrupt
> > (XEN) Loading zImage from 7a00 to
> 5008-5188
> > (XEN) Loading dom0 DTB to 0x5800-0x58010a48
> > (XEN)
> > (XEN)
> > (XEN) Panic on CPU 0:
> > (XEN) Unable to copy the DTB to dom0 memory (left = 68168 bytes)
> > (XEN)
>
> This is a bit odd. The function copy_to_guest_phys_flush_dcache can
> only fail when the P2M entry is invalid or it is not a RAM page.
>
> From the log, it can't even copy the first page. However, this seems
> to belong to the RAM (see BANK[0] message). Would you mind to apply the
> following patch and send the log?
>
>
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index d9836779d1..08b9cd2c44 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -1805,6 +1805,8 @@ static void __init dtb_load(struct kernel_info
> *kinfo)
> printk("Loading dom0 DTB to 0x%"PRIpaddr"-0x%"PRIpaddr"\n",
> kinfo->dtb_paddr, kinfo->dtb_paddr +
> fdt_totalsize(kinfo->fdt));
>
> +dump_p2m_lookup(kinfo->d, kinfo->dtb_paddr);
> +
> left = copy_to_guest_phys_flush_dcache(kinfo->d, kinfo->dtb_paddr,
> kinfo->fdt,
> fdt_totalsize(kinfo->fdt));
>
> Cheers,
>
> --
> Julien Grall
>
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [xen-unstable-smoke test] 133337: trouble: broken/pass
flight 17 xen-unstable-smoke real [real] http://logs.test-lab.xenproject.org/osstest/logs/17/ Failures and problems with tests :-( Tests which did not succeed and are blocking, including tests which could not be run: test-armhf-armhf-xl broken test-armhf-armhf-xl 4 host-install(4)broken REGR. vs. 133312 Tests which did not succeed, but are not blocking: test-amd64-amd64-libvirt 13 migrate-support-checkfail never pass test-arm64-arm64-xl-xsm 13 migrate-support-checkfail never pass test-arm64-arm64-xl-xsm 14 saverestore-support-checkfail never pass version targeted for testing: xen db2af23d15077605f286d8ef86c8f5d9c1b8302a baseline version: xen 1bcd0b43a16b7a48ec9afce3887c6c841b687abb Last test of basis 133312 2019-02-19 11:00:37 Z1 days Failing since133329 2019-02-20 12:00:39 Z0 days2 attempts Testing same since 17 2019-02-20 18:00:46 Z0 days1 attempts People who touched revisions under test: Andrew Cooper George Dunlap Jan Beulich Roger Pau Monne Roger Pau Monné Varad Gautam Wei Liu jobs: build-arm64-xsm pass build-amd64 pass build-armhf pass build-amd64-libvirt pass test-armhf-armhf-xl broken test-arm64-arm64-xl-xsm pass test-amd64-amd64-xl-qemuu-debianhvm-i386 pass test-amd64-amd64-libvirt pass sg-report-flight on osstest.test-lab.xenproject.org logs: /home/logs/logs images: /home/logs/images Logs, config files, etc. are available at http://logs.test-lab.xenproject.org/osstest/logs Explanation of these reports, and of osstest in general, is at http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master Test harness code can be found at http://xenbits.xen.org/gitweb?p=osstest.git;a=summary broken-job test-armhf-armhf-xl broken broken-step test-armhf-armhf-xl host-install(4) Not pushing. commit db2af23d15077605f286d8ef86c8f5d9c1b8302a Author: Jan Beulich Date: Wed Feb 20 17:07:17 2019 +0100 x86/shadow: don't pass wrong L4 MFN to guest_walk_tables() 64-bit PV guest user mode runs on a different L4 table. Make sure - the accessed bit gets set in the correct table (and in log-dirty mode the correct page gets marked dirty) during guest walks, - the correct table gets audited by sh_audit_gw(), - correct info gets logged by print_gw(). Signed-off-by: Jan Beulich Reviewed-by: Andrew Cooper Acked-by: George Dunlap Release-acked-by: Juergen Gross commit b22c900c44a2db8db1c53e269e152206e55c273f Author: Varad Gautam Date: Wed Feb 20 17:06:25 2019 +0100 x86/pmtimer: fix hvm_acpi_sleep_button behavior Commit 19fb14622e941 "x86/pmtimer: move ACPI registers from PMTState to hvm_domain" misconfigures pm1a_sts for hvm_acpi_sleep_button with PWRBTN_STS instead of SLPBTN_STS, which leads to XEN_DOMCTL_SENDTRIGGER_SLEEP causing guest powerdowns. Fix this. Signed-off-by: Varad Gautam Reviewed-by: Andrew Cooper commit 3c5552954c5c63860ccc01c6bc4f9c077bc26072 Author: Andrew Cooper Date: Fri Feb 1 16:56:38 2019 + x86/vpmu: Improve documentation and parsing for vpmu= The behaviour of vpmu= being exclusive of vpmu=bts|ipc|arch is odd and contrary to Xen's normal command line parsing behaviour. Rewrite the parsing to use the normal form, but retain the previous behaviour where the use of bts/ipc/arch implies vpmu=true. Parts of the documenation are stale, most notibly the HVM-only statement. Update it for consistency and correctness. Signed-off-by: Andrew Cooper Release-acked-by: Juergen Gross Reviewed-by: Jan Beulich commit 1e12872d29cc36c61894e347dd3409d7d206699d Author: Roger Pau Monne Date: Tue Feb 19 16:26:08 2019 +0100 libs/gnttab: add missing FreeBSD functions The FreeBSD implementation is missing the following functions: osdep_gnttab_dmabuf_exp_from_refs osdep_gnttab_dmabuf_exp_wait_released osdep_gnttab_dmabuf_imp_to_refs osdep_gnttab_dmabuf_imp_release Which all deal with dmabufs, that only exists on Linux. Implement them using abort, since such functions should never be called on FreeBSD. FTR, I realized those functions where missing when attempting to
Re: [Xen-devel] [PATCH RFC 00/39] x86/KVM: Xen HVM guest support
On 20/02/19 21:15, Joao Martins wrote: > 2. PV Driver support (patches 17 - 39) > > We start by redirecting hypercalls from the backend to routines > which emulate the behaviour that PV backends expect i.e. grant > table and interdomain events. Next, we add support for late > initialization of xenbus, followed by implementing > frontend/backend communication mechanisms (i.e. grant tables and > interdomain event channels). Finally, introduce xen-shim.ko, > which will setup a limited Xen environment. This uses the added > functionality of Xen specific shared memory (grant tables) and > notifications (event channels). I am a bit worried by the last patches, they seem really brittle and prone to breakage. I don't know Xen well enough to understand if the lack of support for GNTMAP_host_map is fixable, but if not, you have to define a completely different hypercall. Of course, tests are missing. You should use the tools/testing/selftests/kvm/ framework, and ideally each patch should come with coverage for the newly-added code. Thanks, Paolo ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] xen/evtchn and forced threaded irq
(+ Andrew and Jan for feedback on the event channel interrupt)
Hi Boris,
Thank you for the your feedback.
On 2/20/19 8:04 PM, Boris Ostrovsky wrote:
On 2/20/19 1:05 PM, Julien Grall wrote:
Hi,
On 20/02/2019 17:07, Boris Ostrovsky wrote:
On 2/20/19 9:15 AM, Julien Grall wrote:
Hi Boris,
Thank you for your answer.
On 20/02/2019 00:02, Boris Ostrovsky wrote:
On Tue, Feb 19, 2019 at 05:31:10PM +, Julien Grall wrote:
Hi all,
I have been looking at using Linux RT in Dom0. Once the guest is
started,
the console is ending to have a lot of warning (see trace below).
After some investigation, this is because the irq handler will now
be threaded.
I can reproduce the same error with the vanilla Linux when passing
the option
'threadirqs' on the command line (the trace below is from 5.0.0-rc7
that has
not RT support).
FWIW, the interrupt for port 6 is used to for the guest to
communicate with
xenstore.
From my understanding, this is happening because the interrupt
handler is now
run in a thread. So we can have the following happening.
Interrupt context | Interrupt thread
|
receive interrupt port 6 |
clear the evtchn port |
set IRQF_RUNTHREAD |
kick interrupt thread |
| clear IRQF_RUNTHREAD
| call evtchn_interrupt
receive interrupt port 6 |
clear the evtchn port |
set IRQF_RUNTHREAD |
kick interrupt thread |
| disable interrupt port 6
| evtchn->enabled = false
| []
|
| *** Handling the second
interrupt ***
| clear IRQF_RUNTHREAD
| call evtchn_interrupt
| WARN(...)
I am not entirely sure how to fix this. I have two solutions in mind:
1) Prevent the interrupt handler to be threaded. We would also
need to
switch from spin_lock to raw_spin_lock as the former may sleep on
RT-Linux.
2) Remove the warning
I think access to evtchn->enabled is racy so (with or without the
warning) we can't use it reliably.
Thinking about it, it would not be the only issue. The ring is sized
to contain only one instance of the same event. So if you receive
twice the event, you may overflow the ring.
Hm... That's another argument in favor of "unthreading" the handler.
I first thought it would be possible to unthread it. However,
wake_up_interruptible is using a spin_lock. On RT spin_lock can sleep,
so this cannot be used in an interrupt context.
So I think "unthreading" the handler is not an option here.
That sounds like a different problem. I.e. there are two issues:
* threaded interrupts don't work properly (races, ring overflow)
* evtchn_interrupt() (threaded or not) has spin_lock(), which is not
going to work for RT
I am afraid that's not correct, you can use spin_lock() in threaded
interrupt handler.
The first can be fixed by using non-threaded handlers.
The two are somewhat related, if you use a non-threaded handler then you
are not going to help the RT case.
In general, the unthreaded solution should be used in the last resort.
Another alternative could be to queue the irq if !evtchn->enabled and
handle it in evtchn_write() (which is where irq is supposed to be
re-enabled).
What do you mean by queue? Is it queueing in the ring?
No, I was thinking about having a new structure for deferred interrupts.
Hmmm, I am not entirely sure what would be the structure here. Could
you expand your thinking?
Some sort of a FIFO that stores {irq, data} tuple. It could obviously be
implemented as a ring but not necessarily as Xen shared ring (if that's
what you were referring to).
The underlying question is what happen if you miss an interrupt. Is it
going to be ok? If no, then we have to record everything and can't
kill/send an error to the user app because it is not its fault.
This means a FIFO would not be a viable. How do you size it? Static (i.e
pre-defined) size is not going to be possible because you don't know how
many interrupt you are going to receive before the thread handler runs.
You can't make it grow dynamically as it make become quite big for the
same reason.
Discussing with my team, a solution that came up would be to introduce
one atomic field per event to record the number of event received. I
will explore that solution tomorrow.
Cheers,
--
Julien Grall
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [PATCH RFC 36/39] drivers/xen: xen_shim_domain() support
Enable /dev/xen/{gntdev,evtchn} and /proc/xen/ for xen_shim_domain().
These interfaces will be used by xenstored to initialize its
event channel port and the kva used to communicate with the
xenbus driver.
Signed-off-by: Joao Martins
---
drivers/xen/evtchn.c | 4 +++-
drivers/xen/privcmd.c | 5 -
drivers/xen/xenbus/xenbus_xs.c | 2 +-
drivers/xen/xenfs/super.c | 7 ---
drivers/xen/xenfs/xensyms.c| 4
5 files changed, 16 insertions(+), 6 deletions(-)
diff --git a/drivers/xen/evtchn.c b/drivers/xen/evtchn.c
index 6d1a5e58968f..367390eb5883 100644
--- a/drivers/xen/evtchn.c
+++ b/drivers/xen/evtchn.c
@@ -708,13 +708,14 @@ static int __init evtchn_init(void)
{
int err;
- if (!xen_domain())
+ if (!xen_domain() && !xen_shim_domain_get())
return -ENODEV;
/* Create '/dev/xen/evtchn'. */
err = misc_register(_miscdev);
if (err != 0) {
pr_err("Could not register /dev/xen/evtchn\n");
+ xen_shim_domain_put();
return err;
}
@@ -726,6 +727,7 @@ static int __init evtchn_init(void)
static void __exit evtchn_cleanup(void)
{
misc_deregister(_miscdev);
+ xen_shim_domain_put();
}
module_init(evtchn_init);
diff --git a/drivers/xen/privcmd.c b/drivers/xen/privcmd.c
index b24ddac1604b..a063ad5c8260 100644
--- a/drivers/xen/privcmd.c
+++ b/drivers/xen/privcmd.c
@@ -999,12 +999,13 @@ static int __init privcmd_init(void)
{
int err;
- if (!xen_domain())
+ if (!xen_domain() && !xen_shim_domain_get())
return -ENODEV;
err = misc_register(_dev);
if (err != 0) {
pr_err("Could not register Xen privcmd device\n");
+ xen_shim_domain_put();
return err;
}
@@ -1012,6 +1013,7 @@ static int __init privcmd_init(void)
if (err != 0) {
pr_err("Could not register Xen hypercall-buf device\n");
misc_deregister(_dev);
+ xen_shim_domain_put();
return err;
}
@@ -1022,6 +1024,7 @@ static void __exit privcmd_exit(void)
{
misc_deregister(_dev);
misc_deregister(_privcmdbuf_dev);
+ xen_shim_domain_put();
}
module_init(privcmd_init);
diff --git a/drivers/xen/xenbus/xenbus_xs.c b/drivers/xen/xenbus/xenbus_xs.c
index bd6db3703972..062c0a5fa415 100644
--- a/drivers/xen/xenbus/xenbus_xs.c
+++ b/drivers/xen/xenbus/xenbus_xs.c
@@ -735,7 +735,7 @@ static void xs_reset_watches(void)
{
int err;
- if (!xen_hvm_domain() || xen_initial_domain())
+ if (!xen_hvm_domain() || xen_initial_domain() || xen_shim_domain())
return;
if (xen_strict_xenbus_quirk())
diff --git a/drivers/xen/xenfs/super.c b/drivers/xen/xenfs/super.c
index 71ddfb4cf61c..2ef326645880 100644
--- a/drivers/xen/xenfs/super.c
+++ b/drivers/xen/xenfs/super.c
@@ -64,7 +64,8 @@ static int xenfs_fill_super(struct super_block *sb, void
*data, int silent)
};
return simple_fill_super(sb, XENFS_SUPER_MAGIC,
- xen_initial_domain() ? xenfs_init_files : xenfs_files);
+ xen_initial_domain() || xen_shim_domain() ?
+ xenfs_init_files : xenfs_files);
}
static struct dentry *xenfs_mount(struct file_system_type *fs_type,
@@ -84,7 +85,7 @@ MODULE_ALIAS_FS("xenfs");
static int __init xenfs_init(void)
{
- if (xen_domain())
+ if (xen_domain() || xen_shim_domain())
return register_filesystem(_type);
return 0;
@@ -92,7 +93,7 @@ static int __init xenfs_init(void)
static void __exit xenfs_exit(void)
{
- if (xen_domain())
+ if (xen_domain() || xen_shim_domain())
unregister_filesystem(_type);
}
diff --git a/drivers/xen/xenfs/xensyms.c b/drivers/xen/xenfs/xensyms.c
index c6c73a33c44d..79bccb53d344 100644
--- a/drivers/xen/xenfs/xensyms.c
+++ b/drivers/xen/xenfs/xensyms.c
@@ -8,6 +8,7 @@
#include
#include
#include
+#include
#include "xenfs.h"
@@ -114,6 +115,9 @@ static int xensyms_open(struct inode *inode, struct file
*file)
struct xensyms *xs;
int ret;
+ if (xen_shim_domain())
+ return -EINVAL;
+
ret = seq_open_private(file, _seq_ops,
sizeof(struct xensyms));
if (ret)
--
2.11.0
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [PATCH RFC 32/39] xen/balloon: xen_shim_domain() support
Xen ballooning uses hollow struct pages (with the underlying PFNs being
populated/unpopulated via hypercalls) which are used by the grant logic
to map grants from other domains.
For purposes of a KVM based xen-shim, this model is not useful --
mapping is unnecessary since all guest memory is already mapped in the
KVM host. The simplest option is to just translate grant references to
GPAs (essentially a get_page() on the appropriate GPA.)
This patch provides an alternate balloon allocation mechanism where in
the allocation path we just provide a constant struct page
(corresponding to page 0.) This allows the calling code -- which does a
page_to_pfn() on the returned struct page -- to remain unchanged before
doing the grant operation (which in this case would fill in the real
struct page.)
Co-developed-by: Ankur Arora
Signed-off-by: Joao Martins
Signed-off-by: Ankur Arora
---
arch/x86/kvm/xen-shim.c | 31 +++
drivers/xen/balloon.c | 15 ++-
include/xen/balloon.h | 7 +++
3 files changed, 52 insertions(+), 1 deletion(-)
diff --git a/arch/x86/kvm/xen-shim.c b/arch/x86/kvm/xen-shim.c
index 61fdceb63ec2..4086d92a4bfb 100644
--- a/arch/x86/kvm/xen-shim.c
+++ b/arch/x86/kvm/xen-shim.c
@@ -13,11 +13,40 @@
#include
#include
#include
+#include
#define BITS_PER_EVTCHN_WORD (sizeof(xen_ulong_t)*8)
static struct kvm_xen shim = { .domid = XEN_SHIM_DOMID };
+static int shim_alloc_pages(int nr_pages, struct page **pages)
+{
+ int i;
+
+ /*
+* We provide page 0 instead of NULL because we'll effectively
+* do the inverse operation while deriving the pfn to pass to
+* xen for mapping.
+*/
+ for (i = 0; i < nr_pages; i++)
+ pages[i] = pfn_to_page(0);
+
+ return 0;
+}
+
+static void shim_free_pages(int nr_pages, struct page **pages)
+{
+ int i;
+
+ for (i = 0; i < nr_pages; i++)
+ pages[i] = NULL;
+}
+
+static struct xen_balloon_ops shim_balloon_ops = {
+ .alloc_pages = shim_alloc_pages,
+ .free_pages = shim_free_pages,
+};
+
static void shim_evtchn_setup(struct shared_info *s)
{
int cpu;
@@ -65,6 +94,7 @@ static int __init shim_register(void)
mutex_init(_lock);
kvm_xen_register_lcall();
+ xen_balloon_ops = _balloon_ops;
/* We can handle hypercalls after this point */
xen_shim_domain = 1;
@@ -94,6 +124,7 @@ static void __exit shim_exit(void)
xen_shim_domain = 0;
kvm_xen_unregister_lcall();
+ xen_balloon_ops = NULL;
HYPERVISOR_shared_info = NULL;
free_page((unsigned long) shim.shinfo);
shim.shinfo = NULL;
diff --git a/drivers/xen/balloon.c b/drivers/xen/balloon.c
index ceb5048de9a7..00375fa6c122 100644
--- a/drivers/xen/balloon.c
+++ b/drivers/xen/balloon.c
@@ -138,7 +138,7 @@ enum bp_state {
static DEFINE_MUTEX(balloon_mutex);
-struct balloon_stats balloon_stats;
+struct balloon_stats balloon_stats __read_mostly;
EXPORT_SYMBOL_GPL(balloon_stats);
/* We increase/decrease in batches which fit in a page */
@@ -158,6 +158,9 @@ static DECLARE_DELAYED_WORK(balloon_worker,
balloon_process);
#define GFP_BALLOON \
(GFP_HIGHUSER | __GFP_NOWARN | __GFP_NORETRY | __GFP_NOMEMALLOC)
+struct xen_balloon_ops *xen_balloon_ops;
+EXPORT_SYMBOL(xen_balloon_ops);
+
/* balloon_append: add the given page to the balloon. */
static void __balloon_append(struct page *page)
{
@@ -589,6 +592,11 @@ int alloc_xenballooned_pages(int nr_pages, struct page
**pages)
struct page *page;
int ret;
+ if (xen_shim_domain() && xen_balloon_ops)
+ return xen_balloon_ops->alloc_pages(nr_pages, pages);
+
+ WARN_ON_ONCE(xen_shim_domain());
+
mutex_lock(_mutex);
balloon_stats.target_unpopulated += nr_pages;
@@ -634,6 +642,11 @@ void free_xenballooned_pages(int nr_pages, struct page
**pages)
{
int i;
+ if (xen_shim_domain() && xen_balloon_ops)
+ return xen_balloon_ops->free_pages(nr_pages, pages);
+
+ WARN_ON_ONCE(xen_shim_domain());
+
mutex_lock(_mutex);
for (i = 0; i < nr_pages; i++) {
diff --git a/include/xen/balloon.h b/include/xen/balloon.h
index 4914b93a23f2..9ba6a7e91d5e 100644
--- a/include/xen/balloon.h
+++ b/include/xen/balloon.h
@@ -22,6 +22,13 @@ struct balloon_stats {
extern struct balloon_stats balloon_stats;
+struct xen_balloon_ops {
+ int (*alloc_pages)(int nr_pages, struct page **pages);
+ void (*free_pages)(int nr_pages, struct page **pages);
+};
+
+extern struct xen_balloon_ops *xen_balloon_ops;
+
void balloon_set_new_target(unsigned long target);
int alloc_xenballooned_pages(int nr_pages, struct page **pages);
--
2.11.0
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [PATCH RFC 17/39] x86/xen: export vcpu_info and shared_info
From: Ankur Arora
Also remove __init annotations from xen_evtchn_2l/fifo_init().
This allows us to support 2-level event channel ABI on
xen_shim_domain().
Signed-off-by: Ankur Arora
---
arch/x86/xen/enlighten.c | 3 +++
drivers/xen/events/events_2l.c | 2 +-
drivers/xen/events/events_fifo.c | 2 +-
include/xen/xen-ops.h| 2 +-
4 files changed, 6 insertions(+), 3 deletions(-)
diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
index 750f46ad018a..73b9736e89d2 100644
--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -50,6 +50,8 @@ DEFINE_PER_CPU(struct vcpu_info, xen_vcpu_info);
/* Linux <-> Xen vCPU id mapping */
DEFINE_PER_CPU(uint32_t, xen_vcpu_id);
EXPORT_PER_CPU_SYMBOL(xen_vcpu_id);
+EXPORT_PER_CPU_SYMBOL(xen_vcpu);
+EXPORT_PER_CPU_SYMBOL(xen_vcpu_info);
enum xen_domain_type xen_domain_type = XEN_NATIVE;
EXPORT_SYMBOL_GPL(xen_domain_type);
@@ -79,6 +81,7 @@ EXPORT_SYMBOL(xen_start_flags);
* page as soon as fixmap is up and running.
*/
struct shared_info *HYPERVISOR_shared_info = _dummy_shared_info;
+EXPORT_SYMBOL_GPL(HYPERVISOR_shared_info);
/*
* Flag to determine whether vcpu info placement is available on all
diff --git a/drivers/xen/events/events_2l.c b/drivers/xen/events/events_2l.c
index 8edef51c92e5..b5acf4b09971 100644
--- a/drivers/xen/events/events_2l.c
+++ b/drivers/xen/events/events_2l.c
@@ -369,7 +369,7 @@ static const struct evtchn_ops evtchn_ops_2l = {
.resume= evtchn_2l_resume,
};
-void __init xen_evtchn_2l_init(void)
+void xen_evtchn_2l_init(void)
{
pr_info("Using 2-level ABI\n");
evtchn_ops = _ops_2l;
diff --git a/drivers/xen/events/events_fifo.c b/drivers/xen/events/events_fifo.c
index 76b318e88382..453b4b05f238 100644
--- a/drivers/xen/events/events_fifo.c
+++ b/drivers/xen/events/events_fifo.c
@@ -430,7 +430,7 @@ static int xen_evtchn_cpu_dead(unsigned int cpu)
return 0;
}
-int __init xen_evtchn_fifo_init(void)
+int xen_evtchn_fifo_init(void)
{
int cpu = smp_processor_id();
int ret;
diff --git a/include/xen/xen-ops.h b/include/xen/xen-ops.h
index 4969817124a8..92fc45075500 100644
--- a/include/xen/xen-ops.h
+++ b/include/xen/xen-ops.h
@@ -10,7 +10,7 @@
#include
DECLARE_PER_CPU(struct vcpu_info *, xen_vcpu);
-
+DECLARE_PER_CPU(struct vcpu_info, xen_vcpu_info);
DECLARE_PER_CPU(uint32_t, xen_vcpu_id);
static inline uint32_t xen_vcpu_nr(int cpu)
{
--
2.11.0
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [PATCH RFC 33/39] xen/grant-table: xen_shim_domain() support
With xen-shim, allocate_xenballooned_pages() only allocates a
place-holder page (pfn 0) expecting a subsequent map_grant_ref to fix
it up.
However, this means that, until the grant operation
(GNTTABOP_map_grant_ref) provides a valid page, we cannot set
PagePrivate or save any state.
This patch elides the setting of that state if xen_shim_domain(). In
addition, gnttab_map_refs() now fills in the appropriate page returned
from the grant operation.
Signed-off-by: Joao Martins
---
drivers/xen/grant-table.c | 15 +++
1 file changed, 11 insertions(+), 4 deletions(-)
diff --git a/drivers/xen/grant-table.c b/drivers/xen/grant-table.c
index 7ea6fb6a2e5d..ab05b70d98bb 100644
--- a/drivers/xen/grant-table.c
+++ b/drivers/xen/grant-table.c
@@ -804,7 +804,7 @@ int gnttab_alloc_pages(int nr_pages, struct page **pages)
int ret;
ret = alloc_xenballooned_pages(nr_pages, pages);
- if (ret < 0)
+ if (ret < 0 || xen_shim_domain())
return ret;
ret = gnttab_pages_set_private(nr_pages, pages);
@@ -1045,6 +1045,11 @@ int gnttab_map_refs(struct gnttab_map_grant_ref *map_ops,
{
struct xen_page_foreign *foreign;
+ if (xen_shim_domain()) {
+ pages[i] = virt_to_page(map_ops[i].host_addr);
+ continue;
+ }
+
SetPageForeign(pages[i]);
foreign = xen_page_foreign(pages[i]);
foreign->domid = map_ops[i].dom;
@@ -1085,8 +1090,10 @@ int gnttab_unmap_refs(struct gnttab_unmap_grant_ref
*unmap_ops,
if (ret)
return ret;
- for (i = 0; i < count; i++)
- ClearPageForeign(pages[i]);
+ for (i = 0; i < count; i++) {
+ if (!xen_shim_domain())
+ ClearPageForeign(pages[i]);
+ }
return clear_foreign_p2m_mapping(unmap_ops, kunmap_ops, pages, count);
}
@@ -1113,7 +1120,7 @@ static void __gnttab_unmap_refs_async(struct
gntab_unmap_queue_data* item)
int pc;
for (pc = 0; pc < item->count; pc++) {
- if (page_count(item->pages[pc]) > 1) {
+ if (page_count(item->pages[pc]) > 1 && !xen_shim_domain()) {
unsigned long delay = GNTTAB_UNMAP_REFS_DELAY *
(item->age + 1);
schedule_delayed_work(>gnttab_work,
msecs_to_jiffies(delay));
--
2.11.0
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [PATCH RFC 38/39] xen-blkback: xen_shim_domain() support
With xen_shim_domain() enabled, the struct pages used for grant mappings
are only valid in the interval between GNTTABOP_map_grant_ref and
GNTTABOP_unmap_grant_ref.
Ensure that we do not cache the struct page outside that duration.
Also, update the struct page for the segment after
GNTTABOP_map_grant_ref, for the subsequent tracking or when doing IO.
In addition, we disable persistent grants for this configuration, since
the map/unmap overhead is fairly small (in the fast path, page lookup,
get_page() and put_page().) This reduces the memory usage in
blkback/blkfront.
Co-developed-by: Ankur Arora
Signed-off-by: Joao Martins
Signed-off-by: Ankur Arora
---
drivers/block/xen-blkback/blkback.c | 19 ---
drivers/block/xen-blkback/xenbus.c | 5 +++--
2 files changed, 19 insertions(+), 5 deletions(-)
diff --git a/drivers/block/xen-blkback/blkback.c
b/drivers/block/xen-blkback/blkback.c
index d51d88be88e1..20c15e6787b1 100644
--- a/drivers/block/xen-blkback/blkback.c
+++ b/drivers/block/xen-blkback/blkback.c
@@ -167,6 +167,15 @@ static inline void put_free_pages(struct xen_blkif_ring
*ring, struct page **pag
unsigned long flags;
int i;
+ /*
+* We don't own the struct page after unmap has been called.
+* Reallocation is cheap anyway for the shim domain case, so free it.
+*/
+ if (xen_shim_domain()) {
+ gnttab_free_pages(num, page);
+ return;
+ }
+
spin_lock_irqsave(>free_pages_lock, flags);
for (i = 0; i < num; i++)
list_add([i]->lru, >free_pages);
@@ -825,7 +834,7 @@ static int xen_blkbk_map(struct xen_blkif_ring *ring,
*/
again:
for (i = map_until; i < num; i++) {
- uint32_t flags;
+ uint32_t flags = 0;
if (use_persistent_gnts) {
persistent_gnt = get_persistent_gnt(
@@ -846,7 +855,8 @@ static int xen_blkbk_map(struct xen_blkif_ring *ring,
addr = vaddr(pages[i]->page);
pages_to_gnt[segs_to_map] = pages[i]->page;
pages[i]->persistent_gnt = NULL;
- flags = GNTMAP_host_map;
+ if (!xen_shim_domain())
+ flags = GNTMAP_host_map;
if (!use_persistent_gnts && ro)
flags |= GNTMAP_readonly;
gnttab_set_map_op([segs_to_map++], addr,
@@ -880,6 +890,8 @@ static int xen_blkbk_map(struct xen_blkif_ring *ring,
goto next;
}
pages[seg_idx]->handle = map[new_map_idx].handle;
+ if (xen_shim_domain())
+ pages[seg_idx]->page =
pages_to_gnt[new_map_idx];
} else {
continue;
}
@@ -1478,7 +1490,7 @@ static int __init xen_blkif_init(void)
{
int rc = 0;
- if (!xen_domain())
+ if (!xen_domain() && !xen_shim_domain_get())
return -ENODEV;
if (xen_blkif_max_ring_order > XENBUS_MAX_RING_GRANT_ORDER) {
@@ -1508,6 +1520,7 @@ static void __exit xen_blkif_exit(void)
{
xen_blkif_interface_exit();
xen_blkif_xenbus_exit();
+ xen_shim_domain_put();
}
module_exit(xen_blkif_exit);
diff --git a/drivers/block/xen-blkback/xenbus.c
b/drivers/block/xen-blkback/xenbus.c
index 424e2efebe85..11be0c052b77 100644
--- a/drivers/block/xen-blkback/xenbus.c
+++ b/drivers/block/xen-blkback/xenbus.c
@@ -871,7 +871,8 @@ static void connect(struct backend_info *be)
xen_blkbk_barrier(xbt, be, be->blkif->vbd.flush_support);
- err = xenbus_printf(xbt, dev->nodename, "feature-persistent", "%u", 1);
+ err = xenbus_printf(xbt, dev->nodename, "feature-persistent", "%u",
+ !xen_shim_domain());
if (err) {
xenbus_dev_fatal(dev, err, "writing %s/feature-persistent",
dev->nodename);
@@ -1059,7 +1060,7 @@ static int connect_ring(struct backend_info *be)
}
pers_grants = xenbus_read_unsigned(dev->otherend, "feature-persistent",
0);
- be->blkif->vbd.feature_gnt_persistent = pers_grants;
+ be->blkif->vbd.feature_gnt_persistent = pers_grants &&
!xen_shim_domain();
be->blkif->vbd.overflow_max_grants = 0;
/*
--
2.11.0
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [PATCH RFC 34/39] xen/gntdev: xen_shim_domain() support
From: Ankur Arora
GNTTABOP_map_grant_ref treats host_addr as an OUT parameter for
xen_shim_domaim().
Accordingly it's updated in struct gnttab_unmap_grant_ref before it gets
used via GNTTABOP_unmap_grant_ref.
Co-developed-by: Joao Martins
Signed-off-by: Ankur Arora
Signed-off-by: Joao Martins
---
drivers/xen/gntdev.c | 10 --
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c
index 5efc5eee9544..8540a51f7597 100644
--- a/drivers/xen/gntdev.c
+++ b/drivers/xen/gntdev.c
@@ -351,6 +351,8 @@ int gntdev_map_grant_pages(struct gntdev_grant_map *map)
}
map->unmap_ops[i].handle = map->map_ops[i].handle;
+ if (xen_shim_domain())
+ map->unmap_ops[i].host_addr = map->map_ops[i].host_addr;
if (use_ptemod)
map->kunmap_ops[i].handle = map->kmap_ops[i].handle;
#ifdef CONFIG_XEN_GRANT_DMA_ALLOC
@@ -1122,7 +1124,9 @@ static int gntdev_mmap(struct file *flip, struct
vm_area_struct *vma)
(map->flags & GNTMAP_readonly))
goto out_unlock_put;
} else {
- map->flags = GNTMAP_host_map;
+ map->flags = 0;
+ if (!xen_shim_domain())
+ map->flags = GNTMAP_host_map;
if (!(vma->vm_flags & VM_WRITE))
map->flags |= GNTMAP_readonly;
}
@@ -1207,7 +1211,7 @@ static int __init gntdev_init(void)
{
int err;
- if (!xen_domain())
+ if (!xen_domain() && !xen_shim_domain_get())
return -ENODEV;
use_ptemod = !xen_feature(XENFEAT_auto_translated_physmap);
@@ -1215,6 +1219,7 @@ static int __init gntdev_init(void)
err = misc_register(_miscdev);
if (err != 0) {
pr_err("Could not register gntdev device\n");
+ xen_shim_domain_put();
return err;
}
return 0;
@@ -1223,6 +1228,7 @@ static int __init gntdev_init(void)
static void __exit gntdev_exit(void)
{
misc_deregister(_miscdev);
+ xen_shim_domain_put();
}
module_init(gntdev_init);
--
2.11.0
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [PATCH RFC 20/39] xen-blkback: module_exit support
Implement module_exit to allow users to do module unload of blkback.
We prevent users from module unload whenever there are still interfaces
allocated, in other words, do module_get on xen_blkif_alloc() and
module_put on xen_blkif_free().
Signed-off-by: Joao Martins
---
drivers/block/xen-blkback/blkback.c | 8
drivers/block/xen-blkback/common.h | 2 ++
drivers/block/xen-blkback/xenbus.c | 14 ++
3 files changed, 24 insertions(+)
diff --git a/drivers/block/xen-blkback/blkback.c
b/drivers/block/xen-blkback/blkback.c
index fd1e19f1a49f..d51d88be88e1 100644
--- a/drivers/block/xen-blkback/blkback.c
+++ b/drivers/block/xen-blkback/blkback.c
@@ -1504,5 +1504,13 @@ static int __init xen_blkif_init(void)
module_init(xen_blkif_init);
+static void __exit xen_blkif_exit(void)
+{
+ xen_blkif_interface_exit();
+ xen_blkif_xenbus_exit();
+}
+
+module_exit(xen_blkif_exit);
+
MODULE_LICENSE("Dual BSD/GPL");
MODULE_ALIAS("xen-backend:vbd");
diff --git a/drivers/block/xen-blkback/common.h
b/drivers/block/xen-blkback/common.h
index 1d3002d773f7..3415c558e115 100644
--- a/drivers/block/xen-blkback/common.h
+++ b/drivers/block/xen-blkback/common.h
@@ -376,8 +376,10 @@ struct phys_req {
blkif_sector_t sector_number;
};
int xen_blkif_interface_init(void);
+void xen_blkif_interface_exit(void);
int xen_blkif_xenbus_init(void);
+void xen_blkif_xenbus_exit(void);
irqreturn_t xen_blkif_be_int(int irq, void *dev_id);
int xen_blkif_schedule(void *arg);
diff --git a/drivers/block/xen-blkback/xenbus.c
b/drivers/block/xen-blkback/xenbus.c
index a4bc74e72c39..424e2efebe85 100644
--- a/drivers/block/xen-blkback/xenbus.c
+++ b/drivers/block/xen-blkback/xenbus.c
@@ -181,6 +181,8 @@ static struct xen_blkif *xen_blkif_alloc(domid_t domid)
init_completion(>drain_complete);
INIT_WORK(>free_work, xen_blkif_deferred_free);
+ __module_get(THIS_MODULE);
+
return blkif;
}
@@ -328,6 +330,8 @@ static void xen_blkif_free(struct xen_blkif *blkif)
/* Make sure everything is drained before shutting down */
kmem_cache_free(xen_blkif_cachep, blkif);
+
+ module_put(THIS_MODULE);
}
int __init xen_blkif_interface_init(void)
@@ -341,6 +345,11 @@ int __init xen_blkif_interface_init(void)
return 0;
}
+void xen_blkif_interface_exit(void)
+{
+ kmem_cache_destroy(xen_blkif_cachep);
+}
+
/*
* sysfs interface for VBD I/O requests
*/
@@ -1115,3 +1124,8 @@ int xen_blkif_xenbus_init(void)
{
return xenbus_register_backend(_blkbk_driver);
}
+
+void xen_blkif_xenbus_exit(void)
+{
+ xenbus_unregister_driver(_blkbk_driver);
+}
--
2.11.0
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [PATCH RFC 35/39] xen/xenbus: xen_shim_domain() support
From: Ankur Arora
Fixup the gnttab unmap_ops (and other data structures) to handle
host_addr as an OUT parameter from the call to GNTTABOP_map_grant_ref.
Also, allow xenstored to be hosted in XS_LOCAL mode for
xen_shim_domain() -- this means that it does not need to acquire
xenstore evtchn and pfn externally.
Co-developed-by: Joao Martins
Signed-off-by: Ankur Arora
Signed-off-by: Joao Martins
---
drivers/xen/xenbus/xenbus_client.c | 23 +--
drivers/xen/xenbus/xenbus_dev_backend.c | 4 ++--
drivers/xen/xenbus/xenbus_dev_frontend.c | 4 ++--
drivers/xen/xenbus/xenbus_probe.c| 8
4 files changed, 25 insertions(+), 14 deletions(-)
diff --git a/drivers/xen/xenbus/xenbus_client.c
b/drivers/xen/xenbus/xenbus_client.c
index ada1c9aa6525..b22149a789d4 100644
--- a/drivers/xen/xenbus/xenbus_client.c
+++ b/drivers/xen/xenbus/xenbus_client.c
@@ -487,8 +487,11 @@ static int __xenbus_map_ring(struct xenbus_device *dev,
"mapping in shared page %d from domain
%d",
gnt_refs[i], dev->otherend_id);
goto fail;
- } else
+ } else {
handles[i] = map[i].handle;
+ if (xen_shim_domain())
+ addrs[i] = map[i].host_addr;
+ }
}
return GNTST_okay;
@@ -498,7 +501,8 @@ static int __xenbus_map_ring(struct xenbus_device *dev,
if (handles[i] != INVALID_GRANT_HANDLE) {
memset([j], 0, sizeof(unmap[j]));
gnttab_set_unmap_op([j], (phys_addr_t)addrs[i],
- GNTMAP_host_map, handles[i]);
+
!xen_shim_domain()?GNTMAP_host_map:0,
+ handles[i]);
j++;
}
}
@@ -546,7 +550,7 @@ static int xenbus_map_ring_valloc_hvm(struct xenbus_device
*dev,
void **vaddr)
{
struct xenbus_map_node *node;
- int err;
+ int i, err;
void *addr;
bool leaked = false;
struct map_ring_valloc_hvm info = {
@@ -572,9 +576,16 @@ static int xenbus_map_ring_valloc_hvm(struct xenbus_device
*dev,
);
err = __xenbus_map_ring(dev, gnt_ref, nr_grefs, node->handles,
- info.phys_addrs, GNTMAP_host_map, );
+ info.phys_addrs,
+ !xen_shim_domain() ? GNTMAP_host_map : 0,
+ );
node->nr_handles = nr_grefs;
+ if (xen_shim_domain()) {
+ for (i = 0; i < nr_grefs; i++)
+ node->hvm.pages[i] = virt_to_page(info.phys_addrs[i]);
+ }
+
if (err)
goto out_free_ballooned_pages;
@@ -882,7 +893,7 @@ int xenbus_unmap_ring(struct xenbus_device *dev,
for (i = 0; i < nr_handles; i++)
gnttab_set_unmap_op([i], vaddrs[i],
- GNTMAP_host_map, handles[i]);
+ !xen_shim_domain()?GNTMAP_host_map:0,
handles[i]);
if (HYPERVISOR_grant_table_op(GNTTABOP_unmap_grant_ref, unmap, i))
BUG();
@@ -926,7 +937,7 @@ static const struct xenbus_ring_ops ring_ops_hvm = {
.unmap = xenbus_unmap_ring_vfree_hvm,
};
-void __init xenbus_ring_ops_init(void)
+void xenbus_ring_ops_init(void)
{
#ifdef CONFIG_XEN_PV
if (!xen_feature(XENFEAT_auto_translated_physmap))
diff --git a/drivers/xen/xenbus/xenbus_dev_backend.c
b/drivers/xen/xenbus/xenbus_dev_backend.c
index edba5fecde4d..b605c87bff76 100644
--- a/drivers/xen/xenbus/xenbus_dev_backend.c
+++ b/drivers/xen/xenbus/xenbus_dev_backend.c
@@ -119,11 +119,11 @@ static struct miscdevice xenbus_backend_dev = {
.fops = _backend_fops,
};
-static int __init xenbus_backend_init(void)
+int xenbus_backend_init(void)
{
int err;
- if (!xen_initial_domain())
+ if (!xen_initial_domain() && !xen_shim_domain())
return -ENODEV;
err = misc_register(_backend_dev);
diff --git a/drivers/xen/xenbus/xenbus_dev_frontend.c
b/drivers/xen/xenbus/xenbus_dev_frontend.c
index a4080d04a01c..c6fca6cca6c8 100644
--- a/drivers/xen/xenbus/xenbus_dev_frontend.c
+++ b/drivers/xen/xenbus/xenbus_dev_frontend.c
@@ -680,11 +680,11 @@ static struct miscdevice xenbus_dev = {
.fops = _xenbus_fops,
};
-static int __init xenbus_frontend_init(void)
+static int xenbus_frontend_init(void)
{
int err;
- if (!xen_domain())
+ if (!xen_domain() && !xen_shim_domain())
return -ENODEV;
err = misc_register(_dev);
diff --git a/drivers/xen/xenbus/xenbus_probe.c
b/drivers/xen/xenbus/xenbus_probe.c
index 2e0ed46b05e7..bbc405cd01ef 100644
---
[Xen-devel] [PATCH RFC 31/39] xen-shim: introduce shim domain driver
From: Ankur Arora
xen-shim.ko sets up and tears down state needed to support Xen
backends. The underlying primitives that are exposed are interdomain
event-channels and grant-table map/unmap/copy.
We setup the following:
* Initialize shared_info and vcpu_info pages, essentially setting
up event-channel state.
* Set up features (this allows xen_feature() to work)
* Initialize event-channel subsystem (select event ops and related
setup.)
* Initialize xenbus and tear it down on module exit.
This functionality would be used by the backend drivers (e.g. netback,
scsiback, blkback etc) in order to drive guest I/O.
Co-developed-by: Joao Martins
Signed-off-by: Ankur Arora
Signed-off-by: Joao Martins
---
arch/x86/include/asm/kvm_host.h | 2 +
arch/x86/kvm/Kconfig | 10 +++
arch/x86/kvm/Makefile| 1 +
arch/x86/kvm/xen-shim.c | 107 +++
arch/x86/xen/enlighten.c | 45 +
drivers/xen/events/events_2l.c | 2 +-
drivers/xen/events/events_base.c | 6 +-
drivers/xen/features.c | 1 +
drivers/xen/xenbus/xenbus_dev_frontend.c | 4 +-
include/xen/xen.h| 5 ++
include/xen/xenbus.h | 3 +
11 files changed, 182 insertions(+), 4 deletions(-)
create mode 100644 arch/x86/kvm/xen-shim.c
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 55609e919e14..6bdae8649d56 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -896,6 +896,8 @@ struct kvm_grant_table {
/* Xen emulation context */
struct kvm_xen {
u64 xen_hypercall;
+
+#define XEN_SHIM_DOMID 0
domid_t domid;
gfn_t shinfo_addr;
diff --git a/arch/x86/kvm/Kconfig b/arch/x86/kvm/Kconfig
index 72fa955f4a15..47347df282dc 100644
--- a/arch/x86/kvm/Kconfig
+++ b/arch/x86/kvm/Kconfig
@@ -96,6 +96,16 @@ config KVM_MMU_AUDIT
This option adds a R/W kVM module parameter 'mmu_audit', which allows
auditing of KVM MMU events at runtime.
+config XEN_SHIM
+ tristate "Xen hypercall emulation shim"
+ depends on KVM
+ depends on XEN
+ default m
+ help
+ Shim to support Xen hypercalls on non-Xen hosts. It intercepts grant
+ table and event channels hypercalls same way as Xen hypervisor. This is
+ useful for having Xen backend drivers work on KVM.
+
# OK, it's a little counter-intuitive to do this, but it puts it neatly under
# the virtualization menu.
source "drivers/vhost/Kconfig"
diff --git a/arch/x86/kvm/Makefile b/arch/x86/kvm/Makefile
index c1eaabbd0a54..a96a96a002a7 100644
--- a/arch/x86/kvm/Makefile
+++ b/arch/x86/kvm/Makefile
@@ -18,3 +18,4 @@ kvm-amd-y += svm.o pmu_amd.o
obj-$(CONFIG_KVM) += kvm.o
obj-$(CONFIG_KVM_INTEL)+= kvm-intel.o
obj-$(CONFIG_KVM_AMD) += kvm-amd.o
+obj-$(CONFIG_XEN_SHIM) += xen-shim.o
diff --git a/arch/x86/kvm/xen-shim.c b/arch/x86/kvm/xen-shim.c
new file mode 100644
index ..61fdceb63ec2
--- /dev/null
+++ b/arch/x86/kvm/xen-shim.c
@@ -0,0 +1,107 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2019 Oracle and/or its affiliates. All rights reserved.
+ *
+ * Xen hypercall emulation shim
+ */
+
+#define pr_fmt(fmt) "KVM:" KBUILD_MODNAME ": " fmt
+
+#include
+
+#include
+#include
+#include
+#include
+
+#define BITS_PER_EVTCHN_WORD (sizeof(xen_ulong_t)*8)
+
+static struct kvm_xen shim = { .domid = XEN_SHIM_DOMID };
+
+static void shim_evtchn_setup(struct shared_info *s)
+{
+ int cpu;
+
+ /* Point Xen's shared_info to the domain's sinfo page */
+ HYPERVISOR_shared_info = s;
+
+ /* Evtchns will be marked pending on allocation */
+ memset(s->evtchn_pending, 0, sizeof(s->evtchn_pending));
+ /* ... but we do mask all of -- dom0 expect it. */
+ memset(s->evtchn_mask, 1, sizeof(s->evtchn_mask));
+
+ for_each_possible_cpu(cpu) {
+ struct vcpu_info *vcpu_info;
+ int i;
+
+ /* Direct CPU mapping as far as dom0 is concerned */
+ per_cpu(xen_vcpu_id, cpu) = cpu;
+
+ vcpu_info = _cpu(xen_vcpu_info, cpu);
+ memset(vcpu_info, 0, sizeof(*vcpu_info));
+
+ vcpu_info->evtchn_upcall_mask = 0;
+
+ vcpu_info->evtchn_upcall_pending = 0;
+ for (i = 0; i < BITS_PER_EVTCHN_WORD; i++)
+ clear_bit(i, _info->evtchn_pending_sel);
+
+ per_cpu(xen_vcpu, cpu) = vcpu_info;
+ }
+}
+
+static int __init shim_register(void)
+{
+ struct shared_info *shinfo;
+
+ shinfo = (struct shared_info *)get_zeroed_page(GFP_KERNEL);
+ if (!shinfo) {
+ pr_err("Failed to allocate shared_info page\n");
+ return -ENOMEM;
+ }
+ shim.shinfo = shinfo;
+
+ idr_init(_to_evt);
+
[Xen-devel] [PATCH RFC 18/39] x86/xen: make hypercall_page generic
From: Ankur Arora
Export hypercall_page as a generic interface which can be implemented
by other hypervisors. With this change, hypercall_page now points to
the newly introduced xen_hypercall_page which is seeded by Xen, or to
one that is filled in by a different hypervisor.
Signed-off-by: Ankur Arora
---
arch/x86/include/asm/xen/hypercall.h | 12 +++-
arch/x86/xen/enlighten.c | 1 +
arch/x86/xen/enlighten_hvm.c | 3 ++-
arch/x86/xen/enlighten_pv.c | 1 +
arch/x86/xen/enlighten_pvh.c | 3 ++-
arch/x86/xen/xen-asm_32.S| 2 +-
arch/x86/xen/xen-asm_64.S| 2 +-
arch/x86/xen/xen-head.S | 8
8 files changed, 19 insertions(+), 13 deletions(-)
diff --git a/arch/x86/include/asm/xen/hypercall.h
b/arch/x86/include/asm/xen/hypercall.h
index ef05bea7010d..1a3cd6680e6f 100644
--- a/arch/x86/include/asm/xen/hypercall.h
+++ b/arch/x86/include/asm/xen/hypercall.h
@@ -86,11 +86,13 @@ struct xen_dm_op_buf;
* there aren't more than 5 arguments...)
*/
-extern struct { char _entry[32]; } hypercall_page[];
+struct hypercall_entry { char _entry[32]; };
+extern struct hypercall_entry xen_hypercall_page[128];
+extern struct hypercall_entry *hypercall_page;
-#define __HYPERCALL"call hypercall_page+%c[offset]"
+#define __HYPERCALLCALL_NOSPEC
#define __HYPERCALL_ENTRY(x) \
- [offset] "i" (__HYPERVISOR_##x * sizeof(hypercall_page[0]))
+ [thunk_target] "0" (hypercall_page + __HYPERVISOR_##x)
#ifdef CONFIG_X86_32
#define __HYPERCALL_RETREG "eax"
@@ -116,7 +118,7 @@ extern struct { char _entry[32]; } hypercall_page[];
register unsigned long __arg4 asm(__HYPERCALL_ARG4REG) = __arg4; \
register unsigned long __arg5 asm(__HYPERCALL_ARG5REG) = __arg5;
-#define __HYPERCALL_0PARAM "=r" (__res), ASM_CALL_CONSTRAINT
+#define __HYPERCALL_0PARAM "=" (__res), ASM_CALL_CONSTRAINT
#define __HYPERCALL_1PARAM __HYPERCALL_0PARAM, "+r" (__arg1)
#define __HYPERCALL_2PARAM __HYPERCALL_1PARAM, "+r" (__arg2)
#define __HYPERCALL_3PARAM __HYPERCALL_2PARAM, "+r" (__arg3)
@@ -208,7 +210,7 @@ xen_single_call(unsigned int call,
asm volatile(CALL_NOSPEC
: __HYPERCALL_5PARAM
-: [thunk_target] "a" (_page[call])
+: [thunk_target] "0" (hypercall_page + call)
: __HYPERCALL_CLOBBER5);
return (long)__res;
diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
index 73b9736e89d2..b36a10e6b5d7 100644
--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -20,6 +20,7 @@
#include "smp.h"
#include "pmu.h"
+struct hypercall_entry *hypercall_page;
EXPORT_SYMBOL_GPL(hypercall_page);
/*
diff --git a/arch/x86/xen/enlighten_hvm.c b/arch/x86/xen/enlighten_hvm.c
index 0e75642d42a3..40845e3e9a96 100644
--- a/arch/x86/xen/enlighten_hvm.c
+++ b/arch/x86/xen/enlighten_hvm.c
@@ -105,8 +105,9 @@ static void __init init_hvm_pv_info(void)
pv_info.name = "Xen HVM";
msr = cpuid_ebx(base + 2);
- pfn = __pa(hypercall_page);
+ pfn = __pa(xen_hypercall_page);
wrmsr_safe(msr, (u32)pfn, (u32)(pfn >> 32));
+ hypercall_page = xen_hypercall_page;
}
xen_setup_features();
diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c
index c54a493e139a..e1537713b57d 100644
--- a/arch/x86/xen/enlighten_pv.c
+++ b/arch/x86/xen/enlighten_pv.c
@@ -1197,6 +1197,7 @@ asmlinkage __visible void __init xen_start_kernel(void)
if (!xen_start_info)
return;
+ hypercall_page = xen_hypercall_page;
xen_domain_type = XEN_PV_DOMAIN;
xen_start_flags = xen_start_info->flags;
diff --git a/arch/x86/xen/enlighten_pvh.c b/arch/x86/xen/enlighten_pvh.c
index 35b7599d2d0b..d57a8ad1769e 100644
--- a/arch/x86/xen/enlighten_pvh.c
+++ b/arch/x86/xen/enlighten_pvh.c
@@ -30,8 +30,9 @@ void __init xen_pvh_init(void)
xen_start_flags = pvh_start_info.flags;
msr = cpuid_ebx(xen_cpuid_base() + 2);
- pfn = __pa(hypercall_page);
+ pfn = __pa(xen_hypercall_page);
wrmsr_safe(msr, (u32)pfn, (u32)(pfn >> 32));
+ hypercall_page = xen_hypercall_page;
}
void __init mem_map_via_hcall(struct boot_params *boot_params_p)
diff --git a/arch/x86/xen/xen-asm_32.S b/arch/x86/xen/xen-asm_32.S
index c15db060a242..ee4998055ea9 100644
--- a/arch/x86/xen/xen-asm_32.S
+++ b/arch/x86/xen/xen-asm_32.S
@@ -121,7 +121,7 @@ xen_iret_end_crit:
hyper_iret:
/* put this out of line since its very rarely used */
- jmp hypercall_page + __HYPERVISOR_iret * 32
+ jmp xen_hypercall_page + __HYPERVISOR_iret * 32
.globl xen_iret_start_crit, xen_iret_end_crit
diff --git a/arch/x86/xen/xen-asm_64.S b/arch/x86/xen/xen-asm_64.S
index 1e9ef0ba30a5..2172d6aec9a3 100644
---
[Xen-devel] [PATCH RFC 19/39] xen/xenbus: xenbus uninit support
This allows reinitialization of xenbus which is useful for
xen_shim_domain() support. Cleaning xenbus state means cancelling
pending watch events, and deleting all watches, closing xenstore event
channel and finally stopping xenbus/xenwatch kthreads alongside
unregistering /proc/xen.
Signed-off-by: Joao Martins
---
drivers/xen/xenbus/xenbus.h| 2 ++
drivers/xen/xenbus/xenbus_client.c | 5
drivers/xen/xenbus/xenbus_probe.c | 51 +++---
drivers/xen/xenbus/xenbus_xs.c | 38
4 files changed, 93 insertions(+), 3 deletions(-)
diff --git a/drivers/xen/xenbus/xenbus.h b/drivers/xen/xenbus/xenbus.h
index 092981171df1..e0e586d81d48 100644
--- a/drivers/xen/xenbus/xenbus.h
+++ b/drivers/xen/xenbus/xenbus.h
@@ -96,6 +96,7 @@ extern wait_queue_head_t xb_waitq;
extern struct mutex xb_write_mutex;
int xs_init(void);
+void xs_deinit(void);
int xb_init_comms(void);
void xb_deinit_comms(void);
int xs_watch_msg(struct xs_watch_event *event);
@@ -129,6 +130,7 @@ int xenbus_read_otherend_details(struct xenbus_device
*xendev,
char *id_node, char *path_node);
void xenbus_ring_ops_init(void);
+void xenbus_ring_ops_deinit(void);
int xenbus_dev_request_and_reply(struct xsd_sockmsg *msg, void *par);
void xenbus_dev_queue_reply(struct xb_req_data *req);
diff --git a/drivers/xen/xenbus/xenbus_client.c
b/drivers/xen/xenbus/xenbus_client.c
index e17ca8156171..ada1c9aa6525 100644
--- a/drivers/xen/xenbus/xenbus_client.c
+++ b/drivers/xen/xenbus/xenbus_client.c
@@ -935,3 +935,8 @@ void __init xenbus_ring_ops_init(void)
#endif
ring_ops = _ops_hvm;
}
+
+void xenbus_ring_ops_deinit(void)
+{
+ ring_ops = NULL;
+}
diff --git a/drivers/xen/xenbus/xenbus_probe.c
b/drivers/xen/xenbus/xenbus_probe.c
index 5b471889d723..2e0ed46b05e7 100644
--- a/drivers/xen/xenbus/xenbus_probe.c
+++ b/drivers/xen/xenbus/xenbus_probe.c
@@ -741,6 +741,21 @@ static int __init xenstored_local_init(void)
return err;
}
+static void xenstored_local_deinit(void)
+{
+ struct evtchn_close close;
+ void *page = NULL;
+
+ page = gfn_to_virt(xen_store_gfn);
+ free_page((unsigned long)page);
+
+ close.port = xen_store_evtchn;
+
+ HYPERVISOR_event_channel_op(EVTCHNOP_close, );
+
+ xen_store_evtchn = 0;
+}
+
static int xenbus_resume_cb(struct notifier_block *nb,
unsigned long action, void *data)
{
@@ -765,7 +780,11 @@ static struct notifier_block xenbus_resume_nb = {
.notifier_call = xenbus_resume_cb,
};
-static int __init xenbus_init(void)
+#ifdef CONFIG_XEN_COMPAT_XENFS
+struct proc_dir_entry *xen_procfs;
+#endif
+
+int xenbus_init(void)
{
int err = 0;
uint64_t v = 0;
@@ -833,13 +852,39 @@ static int __init xenbus_init(void)
* Create xenfs mountpoint in /proc for compatibility with
* utilities that expect to find "xenbus" under "/proc/xen".
*/
- proc_create_mount_point("xen");
+ xen_procfs = proc_create_mount_point("xen");
#endif
out_error:
return err;
}
-
+EXPORT_SYMBOL_GPL(xenbus_init);
postcore_initcall(xenbus_init);
+void xenbus_deinit(void)
+{
+ if (!xen_domain())
+ return;
+
+#ifdef CONFIG_XEN_COMPAT_XENFS
+ proc_remove(xen_procfs);
+ xen_procfs = NULL;
+#endif
+
+ xs_deinit();
+ xenstored_ready = 0;
+
+ switch (xen_store_domain_type) {
+ case XS_LOCAL:
+ xenstored_local_deinit();
+ xen_store_interface = NULL;
+ break;
+ default:
+ pr_warn("Xenstore state unknown\n");
+ break;
+ }
+ xenbus_ring_ops_deinit();
+}
+EXPORT_SYMBOL_GPL(xenbus_deinit);
+
MODULE_LICENSE("GPL");
diff --git a/drivers/xen/xenbus/xenbus_xs.c b/drivers/xen/xenbus/xenbus_xs.c
index 49a3874ae6bb..bd6db3703972 100644
--- a/drivers/xen/xenbus/xenbus_xs.c
+++ b/drivers/xen/xenbus/xenbus_xs.c
@@ -866,6 +866,7 @@ static int xenwatch_thread(void *unused)
for (;;) {
wait_event_interruptible(watch_events_waitq,
+kthread_should_stop() ||
!list_empty(_events));
if (kthread_should_stop())
@@ -917,6 +918,8 @@ static struct notifier_block xs_reboot_nb = {
.notifier_call = xs_reboot_notify,
};
+static struct task_struct *xenwatch_task;
+
int xs_init(void)
{
int err;
@@ -932,9 +935,44 @@ int xs_init(void)
task = kthread_run(xenwatch_thread, NULL, "xenwatch");
if (IS_ERR(task))
return PTR_ERR(task);
+ xenwatch_task = task;
/* shutdown watches for kexec boot */
xs_reset_watches();
return 0;
}
+
+void cancel_watches(void)
+{
+ struct xs_watch_event *event, *tmp;
+
+ /* Cancel pending watch events. */
+ spin_lock(_events_lock);
+
[Xen-devel] [PATCH RFC 00/39] x86/KVM: Xen HVM guest support
Hey, Presented herewith a series that allows KVM to boot Xen x86 HVM guests (with their respective frontends and backends). On the hypervisor side, the approach is to keep the implementation similar to how HyperV was done on x86 KVM. On the backend driver side, the intent is to reuse Xen support. Note that this is an RFC so there are bugs and room for improvement. The intent is to get overall feedback before proceeding further. Running Xen guests on KVM, enables the following use-cases: * Run unmodified Xen HVM images; * Facilitate development/testing of Xen guests and Xen PV drivers; There has been a similar proposal in the past with Xenner, albeit this work has the following advantages over it: * Allows use of existing Xen PV drivers such as block, net etc, both as frontends and backends; * Xen tooling will see the same UABI as on Xen. This means things like xenstored, xenstore-list, xenstore-read run unmodified. Optionally, userspace VMM can emulate xenstore operations; This work is divided in two parts: 1. Xen HVM ABI support (patches 1 - 16) Support the necessary mechanisms to allow HVM guests to boot *without* PV frontends exposed to the guest. We start by intercepting hypercalls made by the guest, followed by event channel IPIs/VIRQs (for PV IPI, timers, spinlocks), pvclock and steal clock. Ankur Arora (1): KVM: x86/xen: support upcall vector Boris Ostrovsky (1): KVM: x86/xen: handle PV spinlocks slowpath Joao Martins (14): KVM: x86: fix Xen hypercall page msr handling KVM: x86/xen: intercept xen hypercalls if enabled KVM: x86/xen: register shared_info page KVM: x86/xen: setup pvclock updates KVM: x86/xen: update wallclock region KVM: x86/xen: register vcpu info KVM: x86/xen: register vcpu time info region KVM: x86/xen: register steal clock KVM: x86: declare Xen HVM guest capability KVM: x86/xen: evtchn signaling via eventfd KVM: x86/xen: store virq when assigning evtchn KVM: x86/xen: handle PV timers oneshot mode KVM: x86/xen: handle PV IPI vcpu yield KVM: x86: declare Xen HVM evtchn offload capability Documentation/virtual/kvm/api.txt | 23 ++ arch/x86/include/asm/kvm_host.h | 46 arch/x86/kvm/Makefile |2 +- arch/x86/kvm/irq.c| 25 ++- arch/x86/kvm/irq_comm.c | 11 + arch/x86/kvm/trace.h | 33 +++ arch/x86/kvm/x86.c| 60 +- arch/x86/kvm/x86.h|1 + arch/x86/kvm/xen.c| 1025 + arch/x86/kvm/xen.h| 48 + include/linux/kvm_host.h | 24 +++ include/uapi/linux/kvm.h | 73 ++- 12 files changed, 1361 insertions(+), 10 deletions(-) 2. PV Driver support (patches 17 - 39) We start by redirecting hypercalls from the backend to routines which emulate the behaviour that PV backends expect i.e. grant table and interdomain events. Next, we add support for late initialization of xenbus, followed by implementing frontend/backend communication mechanisms (i.e. grant tables and interdomain event channels). Finally, introduce xen-shim.ko, which will setup a limited Xen environment. This uses the added functionality of Xen specific shared memory (grant tables) and notifications (event channels). Note that patch 19 is useful to Xen on its own. Ankur Arora (11): x86/xen: export vcpu_info and shared_info x86/xen: make hypercall_page generic KVM: x86/xen: backend hypercall support KVM: x86/xen: grant map support KVM: x86/xen: grant unmap support KVM: x86/xen: interdomain evtchn support KVM: x86/xen: evtchn unmask support KVM: x86/xen: add additional evtchn ops xen-shim: introduce shim domain driver xen/gntdev: xen_shim_domain() support xen/xenbus: xen_shim_domain() support Joao Martins (12): xen/xenbus: xenbus uninit support xen-blkback: module_exit support KVM: x86/xen: domid allocation KVM: x86/xen: grant table init KVM: x86/xen: grant table grow support KVM: x86/xen: grant copy support xen/balloon: xen_shim_domain() support xen/grant-table: xen_shim_domain() support drivers/xen: xen_shim_domain() support xen-netback: xen_shim_domain() support xen-blkback: xen_shim_domain() support KVM: x86: declare Xen HVM Dom0 capability [See the entire series diffstat at the end]
Re: [Xen-devel] XEN on R-CAR H3
Hi Amit,
Thank you for the report.
On 2/19/19 4:46 PM, Amit Tomer wrote:
> (XEN) CPU7 MIDR (0x410fd034) does not match boot CPU MIDR (0x411fd073),
> (XEN) disable cpu (see big.LITTLE.txt under docs/).
> (XEN) CPU7 never came online
> (XEN) Failed to bring up CPU 7 (error -5)
> (XEN) Brought up 4 CPUs
> (XEN) P2M: 44-bit IPA with 44-bit PA and 8-bit VMID
> (XEN) P2M: 4 levels with order-0 root, VTCR 0x80043594
> (XEN) I/O virtualisation disabled
> (XEN) build-id: 74f80103afa98953c029eea87d69696bcd5ef69d
> (XEN) alternatives: Patching with alt table 002abba8 ->
> 002ac1f0
> (XEN) CPU0 will call ARM_SMCCC_ARCH_WORKAROUND_1 on exception entry
> (XEN) CPU2 will call ARM_SMCCC_ARCH_WORKAROUND_1 on exception entry
> (XEN) CPU3 will call ARM_SMCCC_ARCH_WORKAROUND_1 on exception entry
> (XEN) CPU1 will call ARM_SMCCC_ARCH_WORKAROUND_1 on exception entry
> (XEN) *** LOADING DOMAIN 0 ***
> (XEN) Loading Domd0 kernel from boot module @ 7a00
> (XEN) Allocating 1:1 mappings totalling 512MB for dom0:
> (XEN) BANK[0] 0x005000-0x007000 (512MB)
> (XEN) Grant table range: 0x004800-0x004804
> (XEN) Allocating PPI 16 for event channel interrupt
> (XEN) Loading zImage from 7a00 to
> 5008-5188
> (XEN) Loading dom0 DTB to 0x5800-0x58010a48
> (XEN)
> (XEN)
> (XEN) Panic on CPU 0:
> (XEN) Unable to copy the DTB to dom0 memory (left = 68168 bytes)
> (XEN)
This is a bit odd. The function copy_to_guest_phys_flush_dcache can
only fail when the P2M entry is invalid or it is not a RAM page.
From the log, it can't even copy the first page. However, this seems
to belong to the RAM (see BANK[0] message). Would you mind to apply the
following patch and send the log?
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index d9836779d1..08b9cd2c44 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -1805,6 +1805,8 @@ static void __init dtb_load(struct kernel_info *kinfo)
printk("Loading dom0 DTB to 0x%"PRIpaddr"-0x%"PRIpaddr"\n",
kinfo->dtb_paddr, kinfo->dtb_paddr + fdt_totalsize(kinfo->fdt));
+dump_p2m_lookup(kinfo->d, kinfo->dtb_paddr);
+
left = copy_to_guest_phys_flush_dcache(kinfo->d, kinfo->dtb_paddr,
kinfo->fdt,
fdt_totalsize(kinfo->fdt));
Cheers,
--
Julien Grall
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] xen/evtchn and forced threaded irq
On 2/20/19 1:05 PM, Julien Grall wrote:
> Hi,
>
> On 20/02/2019 17:07, Boris Ostrovsky wrote:
>> On 2/20/19 9:15 AM, Julien Grall wrote:
>>> Hi Boris,
>>>
>>> Thank you for your answer.
>>>
>>> On 20/02/2019 00:02, Boris Ostrovsky wrote:
On Tue, Feb 19, 2019 at 05:31:10PM +, Julien Grall wrote:
> Hi all,
>
> I have been looking at using Linux RT in Dom0. Once the guest is
> started,
> the console is ending to have a lot of warning (see trace below).
>
> After some investigation, this is because the irq handler will now
> be threaded.
> I can reproduce the same error with the vanilla Linux when passing
> the option
> 'threadirqs' on the command line (the trace below is from 5.0.0-rc7
> that has
> not RT support).
>
> FWIW, the interrupt for port 6 is used to for the guest to
> communicate with
> xenstore.
>
> From my understanding, this is happening because the interrupt
> handler is now
> run in a thread. So we can have the following happening.
>
> Interrupt context | Interrupt thread
> |
> receive interrupt port 6 |
> clear the evtchn port |
> set IRQF_RUNTHREAD |
> kick interrupt thread |
> | clear IRQF_RUNTHREAD
> | call evtchn_interrupt
> receive interrupt port 6 |
> clear the evtchn port |
> set IRQF_RUNTHREAD |
> kick interrupt thread |
> | disable interrupt port 6
> | evtchn->enabled = false
> | []
> |
> | *** Handling the second
> interrupt ***
> | clear IRQF_RUNTHREAD
> | call evtchn_interrupt
> | WARN(...)
>
> I am not entirely sure how to fix this. I have two solutions in mind:
>
> 1) Prevent the interrupt handler to be threaded. We would also
> need to
> switch from spin_lock to raw_spin_lock as the former may sleep on
> RT-Linux.
>
> 2) Remove the warning
I think access to evtchn->enabled is racy so (with or without the
warning) we can't use it reliably.
>>>
>>> Thinking about it, it would not be the only issue. The ring is sized
>>> to contain only one instance of the same event. So if you receive
>>> twice the event, you may overflow the ring.
>>
>> Hm... That's another argument in favor of "unthreading" the handler.
>
> I first thought it would be possible to unthread it. However,
> wake_up_interruptible is using a spin_lock. On RT spin_lock can sleep,
> so this cannot be used in an interrupt context.
>
> So I think "unthreading" the handler is not an option here.
That sounds like a different problem. I.e. there are two issues:
* threaded interrupts don't work properly (races, ring overflow)
* evtchn_interrupt() (threaded or not) has spin_lock(), which is not
going to work for RT
The first can be fixed by using non-threaded handlers.
>
>>
>>>
Another alternative could be to queue the irq if !evtchn->enabled and
handle it in evtchn_write() (which is where irq is supposed to be
re-enabled).
>>> What do you mean by queue? Is it queueing in the ring?
>>
>>
>> No, I was thinking about having a new structure for deferred interrupts.
>
> Hmmm, I am not entirely sure what would be the structure here. Could
> you expand your thinking?
Some sort of a FIFO that stores {irq, data} tuple. It could obviously be
implemented as a ring but not necessarily as Xen shared ring (if that's
what you were referring to).
-boris
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [linux-3.18 test] 133307: regressions - trouble: blocked/broken/fail/pass
flight 133307 linux-3.18 real [real] http://logs.test-lab.xenproject.org/osstest/logs/133307/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-i386-xl-raw broken test-amd64-i386-libvirt broken test-amd64-i386-xl-qemuu-ws16-amd64 broken test-amd64-i386-xl-qemuu-win10-i386 broken test-amd64-i386-xl-qemut-win7-amd64 broken test-amd64-i386-xl-qemuu-win7-amd64 broken test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm broken test-amd64-amd64-xl-qcow2broken test-armhf-armhf-xl-vhd broken test-amd64-i386-qemuu-rhel6hvm-amd broken test-armhf-armhf-xl broken test-amd64-i386-rumprun-i386 broken test-amd64-amd64-xl-pvhv2-intel broken test-amd64-i386-pair broken build-i386-xsm broken test-amd64-amd64-xl-qemuu-debianhvm-amd64 broken build-i386-xsm4 host-install(4)broken REGR. vs. 128858 build-arm64 broken in 133132 build-arm64-xsm broken in 133132 build-arm64-pvopsbroken in 133132 build-arm64-pvops 3 capture-logs broken in 133132 REGR. vs. 128858 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-boot fail REGR. vs. 128858 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 7 xen-boot fail REGR. vs. 128858 test-amd64-i386-freebsd10-amd64 7 xen-boot fail REGR. vs. 128858 test-amd64-amd64-xl-shadow7 xen-boot fail REGR. vs. 128858 test-amd64-amd64-amd64-pvgrub 7 xen-bootfail REGR. vs. 128858 test-amd64-amd64-pair10 xen-boot/src_hostfail REGR. vs. 128858 test-amd64-amd64-rumprun-amd64 7 xen-boot fail REGR. vs. 128858 test-amd64-amd64-xl-credit2 7 xen-boot fail REGR. vs. 128858 test-amd64-amd64-xl 7 xen-boot fail REGR. vs. 128858 test-amd64-amd64-pair11 xen-boot/dst_hostfail REGR. vs. 128858 test-amd64-i386-freebsd10-i386 7 xen-boot fail REGR. vs. 128858 test-amd64-amd64-libvirt 7 xen-boot fail REGR. vs. 128858 test-amd64-amd64-xl-pvshim7 xen-boot fail REGR. vs. 128858 test-amd64-amd64-xl-qemuu-ovmf-amd64 7 xen-boot fail REGR. vs. 128858 test-amd64-amd64-xl-xsm 7 xen-boot fail REGR. vs. 128858 test-amd64-amd64-libvirt-xsm 7 xen-boot fail REGR. vs. 128858 test-amd64-amd64-i386-pvgrub 7 xen-boot fail REGR. vs. 128858 test-amd64-amd64-qemuu-nested-intel 7 xen-boot fail REGR. vs. 128858 test-amd64-amd64-xl-multivcpu 7 xen-bootfail REGR. vs. 128858 test-amd64-i386-examine 8 reboot fail REGR. vs. 128858 test-amd64-amd64-libvirt-pair 10 xen-boot/src_host fail REGR. vs. 128858 test-amd64-amd64-libvirt-pair 11 xen-boot/dst_host fail REGR. vs. 128858 test-amd64-i386-xl7 xen-boot fail REGR. vs. 128858 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-boot fail REGR. vs. 128858 test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm 7 xen-boot fail REGR. vs. 128858 test-amd64-i386-rumprun-i386 7 xen-boot fail in 133288 REGR. vs. 128858 test-amd64-amd64-xl-qemuu-debianhvm-amd64 7 xen-boot fail in 133288 REGR. vs. 128858 test-amd64-i386-pair 10 xen-boot/src_host fail in 133288 REGR. vs. 128858 test-amd64-i386-pair 11 xen-boot/dst_host fail in 133288 REGR. vs. 128858 test-amd64-i386-xl-raw7 xen-boot fail in 133288 REGR. vs. 128858 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm 7 xen-boot fail in 133288 REGR. vs. 128858 Tests which are failing intermittently (not blocking): test-amd64-i386-qemuu-rhel6hvm-amd 4 host-install(4)broken pass in 133288 test-armhf-armhf-xl 4 host-install(4) broken pass in 133288 test-amd64-i386-xl-qemuu-ws16-amd64 4 host-install(4) broken pass in 133288 test-amd64-i386-libvirt 4 host-install(4) broken pass in 133288 test-amd64-amd64-xl-pvhv2-intel 4 host-install(4) broken pass in 133288 test-amd64-amd64-xl-qemuu-debianhvm-amd64 4 host-install(4) broken pass in 133288 test-amd64-i386-rumprun-i386 4 host-install(4) broken pass in 133288 test-amd64-i386-xl-qemut-win7-amd64 4 host-install(4) broken pass in 133288 test-amd64-i386-xl-qemuu-win10-i386 4 host-install(4) broken pass in 133288 test-amd64-i386-xl-qemuu-win7-amd64 4 host-install(4) broken pass in 133288 test-amd64-amd64-xl-qcow2 4 host-install(4) broken pass in 133288
Re: [Xen-devel] [PATCH] iommu: leave IOMMU enabled by default during kexec crash transition
On 20/02/2019 08:55, Jan Beulich wrote: > > Everything, absolutely everything is possible as a cause for a crash. > I don't see why device isolation would matter here at all. Page table > corruption (be it IOMMU or CPU one) can be caused by > malfunctioning kernel code, entirely unrelated to what devices do. > Yes, I blindly assumed that stray DMA is the most common source of kernel data corruption. Unfortunately, we don't really have a choice now - almost all platforms these days raise some form of an error in the event of unhanded translation fault (at least most of those we have now in our labs). So it's either slight chance of entering the crash kernel with corrupted IOMMU pages or not entering it at all in most of the cases. Igor ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH] iommu: leave IOMMU enabled by default during kexec crash transition
On 20/02/2019 09:01, Jan Beulich wrote: > But isn't it a valid question whether keeping interrupt remapping > enabled is helpful or potentially making things worse? The > description of the patch discusses the DMA translation aspects > only. Unless the crash kernel would always operate in polling > mode only, it needs to have interrupts routed to the right > handler(s). Whether that's guaranteed with remapping left > enabled is not something that goes without saying, imo. > IR copy logic works as follows - IRT is copied as is and still in functioning state while remapable interrupts might be in-flight. New IRT entries are allocated in addition to existing ones which makes the transition transparent for a device. I'll mention this aspect in the commit message since it doesn't seem obvious - you're right. Igor ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [xen-4.10-testing test] 133311: trouble: blocked/broken/fail/pass
flight 133311 xen-4.10-testing real [real] http://logs.test-lab.xenproject.org/osstest/logs/133311/ Failures and problems with tests :-( Tests which did not succeed and are blocking, including tests which could not be run: build-armhf broken test-amd64-amd64-xl-multivcpu broken test-amd64-amd64-qemuu-nested-intel broken test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm broken test-amd64-i386-xl-qemut-win7-amd64 broken test-amd64-amd64-xl-credit1 broken build-armhf 4 host-install(4)broken REGR. vs. 132966 test-amd64-i386-xl-qemut-debianhvm-amd64-xsm broken in 133292 test-amd64-i386-livepatchbroken in 133292 test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm broken in 133292 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm broken in 133292 test-amd64-i386-xl-qemut-debianhvm-amd64 broken in 133292 Tests which are failing intermittently (not blocking): test-amd64-i386-xl-qemut-debianhvm-amd64-xsm 4 host-install(4) broken in 133292 pass in 133311 test-amd64-i386-xl-qemut-debianhvm-amd64 4 host-install(4) broken in 133292 pass in 133311 test-amd64-i386-livepatch4 host-install(4) broken in 133292 pass in 133311 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 4 host-install(4) broken in 133292 pass in 133311 test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm 4 host-install(4) broken in 133292 pass in 133311 test-amd64-amd64-qemuu-nested-intel 4 host-install(4) broken pass in 133292 test-amd64-amd64-xl-multivcpu 4 host-install(4) broken pass in 133292 test-amd64-amd64-xl-credit1 4 host-install(4) broken pass in 133292 test-amd64-i386-xl-qemut-win7-amd64 4 host-install(4) broken pass in 133292 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 4 host-install(4) broken pass in 133292 test-amd64-amd64-xl-qemut-debianhvm-amd64 10 debian-hvm-install fail in 133278 pass in 133311 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 10 debian-hvm-install fail pass in 133278 test-amd64-amd64-xl-qemuu-debianhvm-amd64 10 debian-hvm-install fail pass in 133292 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm 10 debian-hvm-install fail pass in 133292 test-amd64-i386-xl-qemuu-debianhvm-amd64 10 debian-hvm-install fail pass in 133292 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow 10 debian-hvm-install fail pass in 133292 test-amd64-amd64-qemuu-nested-amd 10 debian-hvm-installfail pass in 133292 test-amd64-amd64-libvirt-vhd 10 debian-di-install fail pass in 133292 Tests which did not succeed, but are not blocking: test-armhf-armhf-xl-credit1 1 build-check(1) blocked n/a test-armhf-armhf-xl-vhd 1 build-check(1) blocked n/a test-armhf-armhf-xl-multivcpu 1 build-check(1) blocked n/a test-armhf-armhf-xl-credit2 1 build-check(1) blocked n/a test-armhf-armhf-xl-cubietruck 1 build-check(1) blocked n/a test-armhf-armhf-xl-arndale 1 build-check(1) blocked n/a test-armhf-armhf-xl-rtds 1 build-check(1) blocked n/a test-armhf-armhf-libvirt 1 build-check(1) blocked n/a test-armhf-armhf-libvirt-raw 1 build-check(1) blocked n/a build-armhf-libvirt 1 build-check(1) blocked n/a test-armhf-armhf-xl 1 build-check(1) blocked n/a test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check fail in 133292 never pass test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2 fail in 133292 never pass test-amd64-amd64-libvirt-vhd 12 migrate-support-check fail in 133292 never pass test-armhf-armhf-xl-credit2 13 migrate-support-check fail in 133292 never pass test-armhf-armhf-xl-credit2 14 saverestore-support-check fail in 133292 never pass test-armhf-armhf-xl-credit1 13 migrate-support-check fail in 133292 never pass test-armhf-armhf-xl-credit1 14 saverestore-support-check fail in 133292 never pass test-amd64-i386-xl-qemut-win7-amd64 17 guest-stopfail in 133292 never pass test-armhf-armhf-libvirt13 migrate-support-check fail in 133292 never pass test-armhf-armhf-libvirt 14 saverestore-support-check fail in 133292 never pass test-armhf-armhf-xl-cubietruck 13 migrate-support-check fail in 133292 never pass test-armhf-armhf-xl-cubietruck 14 saverestore-support-check fail in 133292 never pass test-armhf-armhf-xl-rtds13 migrate-support-check fail in 133292 never pass test-armhf-armhf-xl-rtds 14 saverestore-support-check fail in 133292 never pass test-armhf-armhf-xl 13 migrate-support-check fail in 133292 never pass test-armhf-armhf-xl 14 saverestore-support-check fail in 133292 never pass test-armhf-armhf-xl-arndale 13 migrate-support-check fail in 133292 never pass test-armhf-armhf-xl-arndale 14
Re: [Xen-devel] XEN on R-CAR H3
Hi, > We use earlier BSP version and we didn't face the similar issue. > > We have a plan to switch to recent BSP version (3.15). So, I will come > up with updates when we migrate. Thanks for this information, we have now built it for 3.9 but yet to test the images. Also, it would be great if you can let us know the XEN version you use with BSP images? Thanks, -Amit. ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH] x86/shadow: don't use map_domain_page_global() on paths that may not fail
On 20/02/2019 15:15, Jan Beulich wrote: > The assumption (according to one comment) and hope (according to > another) that map_domain_page_global() can't fail are both wrong on > large enough systems. Do away with the guest_vtable field altogether, > and establish / tear down the desired mapping as necessary. > > The alternatives, discarded as being undesirable, would have been to > either crash the guest in sh_update_cr3() when the mapping fails, or to > bubble up an error indicator, which upper layers would have a hard time > to deal with (other than again by crashing the guest). > > Signed-off-by: Jan Beulich Reviewed-by: Andrew Cooper ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH] iommu: leave IOMMU enabled by default during kexec crash transition
On 20/02/2019 08:48, Jan Beulich wrote: > > Some entity needs to decide whether to add the respective command > line option to the crash kernel's command line. It should be this same > entity to tell Xen whether to keep the IOMMU enabled while invoking > the crash kernel. I am merely guessing that this entity is the kexec > tool. > I was just double checking and it seems (assuming the device reset correctly in the crash kernel) everything seem to work even without enabling intel_iommu in the command line - newer kernels handle this case by explicitly disabling translation in any case: they expect it to be enabled by the previous kernel. I intended Xen command line option be more of a fallback mechanism for those still using ancient kdump kernels. For all the others the transition should be transparent. With the above said, having an orchestration tool on top of that seems a bit of overkill. >> I'm not following your last question. Could you elaborate? > > I'm simply asking what the bare metal equivalent behavior is here, > i.e. how command line option addition and IOMMU state are being > kept in sync in that case. Just for reference, in the hope that what > they do is sane, and hence we could follow that model. > As mentioned above, Linux (as the main kernel) currently only supports secondary kernels that expect IOMMU to be enabled and have basic DMAR understanding which means crash kernel is supposed to be up to date with the main one. On the other hand, Linux (as a crash kernel) expects the main kernel (which might be Xen) to do either of things: keep IOMMU enabled or disabled - both cases are handled. Igor ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] XEN on R-CAR H3
On 19.02.19 18:46, Amit Tomer wrote:
Hi
Hi
[1] https://elinux.org/R-Car/Boards/Yocto-Gen3
We tried BSP release v3.15.0 from above link but see following message:
[ 45.518865] => setenv xen_addr_r 0x4800;setenv fdt_addr_r
0x4a00;setenv kernel_addr_r 0x7a00
[ 52.430467] => setenv fdt_high 0x;fdt addr $fdt_addr_r;fdt resize
[ 55.348938] => setenv xen_bootargs console=dtuart dom0_mem=512M
[ 57.661868] => setenv dom0_bootargs console=hvc0,115200n8
earlycon=xenboot debug clk_ignore_unused root=/dev/mmcblk1 rw rootwait
[ 59.813185] => fdt set /chosen xen,xen-bootargs \"$xen_bootargs\";
[ 62.037351] => fdt set /chosen xen,dom0-bootargs
\"$dom0_bootargs\";fdt mknode /chosen modules
[ 65.765571] => fdt set /chosen/modules '#address-cells' <1>;fdt set
/chosen/modules '#size-cells' <1>;fdt mknode /chosen/modules module@0
[ 68.086099] => fdt set /chosen/modules/module@0 compatible
"xen,linux-zimage" "xen,multiboot-module"
[ 70.205113] => fdt set /chosen/modules/module@0 reg <
$kernel_addr_r 0x180 >
[ 72.165412] => booti ${xen_addr_r} - ${fdt_addr_r}
[ 74.021004] ## Flattened Device Tree blob at 4a00
[ 74.026244]Booting using the fdt blob at 0x4a00
[ 74.031691]reserving fdt memory region: addr=4800 size=4
[ 74.038310]reserving fdt memory region: addr=4a00 size=12000
[ 74.044936]Loading Device Tree to 7d71, end
7d724fff ... OK
[ 74.054036]
[ 74.055489] Starting kernel ...
[ 74.058755]
- UART enabled -
- CPU booting -
- Current EL 0008 -
- Xen starting at EL2 -
- Zero BSS -
- Setting up control registers -
- Turning on paging -
- Ready -
(XEN) Checking for initrd in /chosen
(XEN) RAM: 4800 - 7fff
(XEN) RAM: 0005 - 00053fff
(XEN) RAM: 0006 - 00063fff
(XEN) RAM: 0007 - 00073fff
(XEN)
(XEN) MODULE[0]: 7d71 - 7d722000 Device Tree
(XEN) MODULE[1]: 7a00 - 7b80 Kernel
(XEN) RESVD[0]: 4800 - 4804
(XEN) RESVD[1]: 4a00 - 4a012000
(XEN) RESVD[2]: 7d71 - 7d722000
(XEN)
(XEN)
(XEN) Command line: console=dtuart dom0_mem=512M
(XEN) PFN compression on bits 19...19
(XEN) Domain heap initialised
(XEN) Booting using Device Tree
(XEN) Platform: Generic System
(XEN) Taking dtuart configuration from /chosen/stdout-path
(XEN) Looking for dtuart at "serial0", options "115200n8"
(XEN) WARNING: UART configuration is not supported
Xen 4.12.0-rc
(XEN) Xen version 4.12.0-rc (amit@) (aarch64-linux-gnu-gcc (Linaro GCC
7.3-2018.05) 7.3.1 20180425 [linaro-7.3-2018.05 revision
d29120a424ecfbc167ef90065c0eeb7f91977701]) debug=y Wed Feb 6
16:23:25 I9
(XEN) Latest ChangeSet: Thu Jan 24 14:03:48 2019 + git:755eb64
(XEN) Processor: 411fd073: "ARM Limited", variant: 0x1, part 0xd07, rev 0x3
(XEN) 64-bit Execution:
(XEN) Processor Features:
(XEN) Exception Levels: EL3:64+32 EL2:64+32 EL1:64+32 EL0:64+32
(XEN) Extensions: FloatingPoint AdvancedSIMD
(XEN) Debug Features: 10305106
(XEN) Auxiliary Features:
(XEN) Memory Model Features: 1124
(XEN) ISA Features: 00011120
(XEN) 32-bit Execution:
(XEN) Processor Features: 0131:00011011
(XEN) Instruction Sets: AArch32 A32 Thumb Thumb-2 Jazelle
(XEN) Extensions: GenericTimer Security
(XEN) Debug Features: 03010066
(XEN) Auxiliary Features:
(XEN) Memory Model Features: 10201105 4000 0126 02102211
(XEN) ISA Features: 02101110 13112111 21232042 01112131 00011142 00011121
(XEN) Using SMC Calling Convention v1.1
(XEN) Using PSCI v1.0
(XEN) SMP: Allowing 8 CPUs
(XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27 Freq: 8333 KHz
(XEN) GICv2 initialization:
(XEN) gic_dist_addr=f101
(XEN) gic_cpu_addr=f102
(XEN) gic_hyp_addr=f104
(XEN) gic_vcpu_addr=f106
(XEN) gic_maintenance_irq=25
(XEN) GICv2: Adjusting CPU interface base to 0xf102f000
(XEN) GICv2: 512 lines, 8 cpus, secure (IID 0200043b).
(XEN) Using scheduler: SMP Credit Scheduler rev2 (credit2)
(XEN) Initializing Credit2 scheduler
(XEN) load_precision_shift: 18
(XEN) load_window_shift: 30
(XEN) underload_balance_tolerance: 0
(XEN) overload_balance_tolerance: -3
(XEN) runqueues arrangement: socket
(XEN) cap enforcement granularity: 10ms
(XEN) load tracking window length 1073741824 ns
(XEN) Adding cpu 0 to runqueue 0
(XEN) First cpu on runqueue, activating
(XEN) Allocated console ring of 64 KiB.
(XEN) Bringing up CPU1
- CPU 0001 booting -
- Current EL 0008 -
- Xen starting at EL2 -
- Setting up control registers -
- Turning on paging -
- Ready -
(XEN) Adding cpu 1 to runqueue 0
(XEN) CPU 1 booted.
(XEN) Bringing up CPU2
- CPU 0002
Re: [Xen-devel] [RESEND PATCH 0/7] Add FOLL_LONGTERM to GUP fast and use it
On Wed, Feb 20, 2019 at 07:19:30AM -0800, Christoph Hellwig wrote: > On Tue, Feb 19, 2019 at 09:30:33PM -0800, ira.we...@intel.com wrote: > > From: Ira Weiny > > > > Resending these as I had only 1 minor comment which I believe we have > > covered > > in this series. I was anticipating these going through the mm tree as they > > depend on a cleanup patch there and the IB changes are very minor. But they > > could just as well go through the IB tree. > > > > NOTE: This series depends on my clean up patch to remove the write parameter > > from gup_fast_permitted()[1] > > > > HFI1, qib, and mthca, use get_user_pages_fast() due to it performance > > advantages. These pages can be held for a significant time. But > > get_user_pages_fast() does not protect against mapping of FS DAX pages. > > This I don't get - if you do lock down long term mappings performance > of the actual get_user_pages call shouldn't matter to start with. > > What do I miss? A couple of points. First "longterm" is a relative thing and at this point is probably a misnomer. This is really flagging a pin which is going to be given to hardware and can't move. I've thought of a couple of alternative names but I think we have to settle on if we are going to use FL_LAYOUT or something else to solve the "longterm" problem. Then I think we can change the flag to a better name. Second, It depends on how often you are registering memory. I have spoken with some RDMA users who consider MR in the performance path... For the overall application performance. I don't have the numbers as the tests for HFI1 were done a long time ago. But there was a significant advantage. Some of which is probably due to the fact that you don't have to hold mmap_sem. Finally, architecturally I think it would be good for everyone to use *_fast. There are patches submitted to the RDMA list which would allow the use of *_fast (they reworking the use of mmap_sem) and as soon as they are accepted I'll submit a patch to convert the RDMA core as well. Also to this point others are looking to use *_fast.[2] As an asside, Jasons pointed out in my previous submission that *_fast and *_unlocked look very much the same. I agree and I think further cleanup will be coming. But I'm focused on getting the final solution for DAX at the moment. Ira ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] xen/evtchn and forced threaded irq
Hi, On 20/02/2019 17:07, Boris Ostrovsky wrote: On 2/20/19 9:15 AM, Julien Grall wrote: Hi Boris, Thank you for your answer. On 20/02/2019 00:02, Boris Ostrovsky wrote: On Tue, Feb 19, 2019 at 05:31:10PM +, Julien Grall wrote: Hi all, I have been looking at using Linux RT in Dom0. Once the guest is started, the console is ending to have a lot of warning (see trace below). After some investigation, this is because the irq handler will now be threaded. I can reproduce the same error with the vanilla Linux when passing the option 'threadirqs' on the command line (the trace below is from 5.0.0-rc7 that has not RT support). FWIW, the interrupt for port 6 is used to for the guest to communicate with xenstore. From my understanding, this is happening because the interrupt handler is now run in a thread. So we can have the following happening. Interrupt context | Interrupt thread | receive interrupt port 6 | clear the evtchn port | set IRQF_RUNTHREAD | kick interrupt thread | | clear IRQF_RUNTHREAD | call evtchn_interrupt receive interrupt port 6 | clear the evtchn port | set IRQF_RUNTHREAD | kick interrupt thread | | disable interrupt port 6 | evtchn->enabled = false | [] | | *** Handling the second interrupt *** | clear IRQF_RUNTHREAD | call evtchn_interrupt | WARN(...) I am not entirely sure how to fix this. I have two solutions in mind: 1) Prevent the interrupt handler to be threaded. We would also need to switch from spin_lock to raw_spin_lock as the former may sleep on RT-Linux. 2) Remove the warning I think access to evtchn->enabled is racy so (with or without the warning) we can't use it reliably. Thinking about it, it would not be the only issue. The ring is sized to contain only one instance of the same event. So if you receive twice the event, you may overflow the ring. Hm... That's another argument in favor of "unthreading" the handler. I first thought it would be possible to unthread it. However, wake_up_interruptible is using a spin_lock. On RT spin_lock can sleep, so this cannot be used in an interrupt context. So I think "unthreading" the handler is not an option here. Another alternative could be to queue the irq if !evtchn->enabled and handle it in evtchn_write() (which is where irq is supposed to be re-enabled). What do you mean by queue? Is it queueing in the ring? No, I was thinking about having a new structure for deferred interrupts. Hmmm, I am not entirely sure what would be the structure here. Could you expand your thinking? Cheers, -- Julien Grall ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH V2 3/3] xen/arm: Add SCIFA UART support for early printk
On 19.02.19 12:09, Julien Grall wrote: Hi Oleksandr, Hi Julien Actually, the main difference for the "early printk" support is in two reg offsets: +#define SCIFA_SCASSR 0x14 /* Serial status register */ +#define SCIFA_SCAFTDR 0x20 /* Transmit FIFO data register */ +#define SCIF_SCFSR 0x10 /* Serial status register */ +#define SCIF_SCFTDR 0x0c /* Transmit FIFO data register */ I am not mistaken, we will have to introduce two options to cover this case, as the offsets are not correlated with each other, no? You don't need two options. For instance, you can only introduce an option SCIF_VERSION that would be 0 for SCIF and 61 (ascii 'a') for SCIFA. Then in the code, you can use SCIF_VERSION to decides which sets of macros you are using. I think I understand the idea. Will try. -- Regards, Oleksandr Tyshchenko ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [xen-unstable-smoke test] 133329: regressions - FAIL
flight 133329 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/133329/
Regressions :-(
Tests which did not succeed and are blocking,
including tests which could not be run:
test-amd64-amd64-xl-qemuu-debianhvm-i386 10 debian-hvm-install fail REGR. vs.
133312
Tests which did not succeed, but are not blocking:
test-arm64-arm64-xl-xsm 13 migrate-support-checkfail never pass
test-arm64-arm64-xl-xsm 14 saverestore-support-checkfail never pass
test-amd64-amd64-libvirt 13 migrate-support-checkfail never pass
test-armhf-armhf-xl 13 migrate-support-checkfail never pass
test-armhf-armhf-xl 14 saverestore-support-checkfail never pass
version targeted for testing:
xen 1e12872d29cc36c61894e347dd3409d7d206699d
baseline version:
xen 1bcd0b43a16b7a48ec9afce3887c6c841b687abb
Last test of basis 133312 2019-02-19 11:00:37 Z1 days
Testing same since 133329 2019-02-20 12:00:39 Z0 days1 attempts
People who touched revisions under test:
Roger Pau Monne
Roger Pau Monné
Wei Liu
jobs:
build-arm64-xsm pass
build-amd64 pass
build-armhf pass
build-amd64-libvirt pass
test-armhf-armhf-xl pass
test-arm64-arm64-xl-xsm pass
test-amd64-amd64-xl-qemuu-debianhvm-i386 fail
test-amd64-amd64-libvirt pass
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images
Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs
Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master
Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary
Not pushing.
commit 1e12872d29cc36c61894e347dd3409d7d206699d
Author: Roger Pau Monne
Date: Tue Feb 19 16:26:08 2019 +0100
libs/gnttab: add missing FreeBSD functions
The FreeBSD implementation is missing the following functions:
osdep_gnttab_dmabuf_exp_from_refs
osdep_gnttab_dmabuf_exp_wait_released
osdep_gnttab_dmabuf_imp_to_refs
osdep_gnttab_dmabuf_imp_release
Which all deal with dmabufs, that only exists on Linux. Implement them
using abort, since such functions should never be called on FreeBSD.
FTR, I realized those functions where missing when attempting to use
pygrub:
Traceback (most recent call last):
File "/usr/local/lib/xen/bin/pygrub", line 19, in
import xen.lowlevel.xc
ImportError: /usr/local/lib/libxengnttab.so.1: Undefined symbol
"osdep_gnttab_dmabuf_exp_from_refs"
Fixes: ee8105 ("libgnttab: Add support for Linux dma-buf")
Signed-off-by: Roger Pau Monné
Acked-by: Wei Liu
Release-acked-by: Juergen Gross
Reviewed-by: Oleksandr Andrushchenko
(qemu changes not included)
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH V2 1/3] xen/arm: drivers: scif: Add support for SCIFA compatible UARTs
On 19.02.19 12:00, Julien Grall wrote: Hi Oleksandr, On 2/18/19 8:14 PM, Oleksandr wrote: On 18.02.19 16:00, Julien Grall wrote: Hi, Hi Hi Julien On 01/02/2019 12:37, Oleksandr Tyshchenko wrote: From: Oleksandr Tyshchenko Extend existing driver to be able to handle SCIFA interface as well. In general a patch should do only one thing. In this case, this should have been split in 2 patches: one to extend the driver, the second to add support for SCIFA. Please split the patch accordingly. Not entirely clear to me how the current patch should be split... - The first patch will be just a copy of the current patch, but without new compatible string (SCIFA). Without anything related to SCIFA. This patch would only contain the rework of the code. - The second patch will add new compatible string. + anything related to SCIFA (Macros and the SCIFA element in the array port_params. It is clear now. Will do. Cheers, -- Regards, Oleksandr Tyshchenko ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] xen/evtchn and forced threaded irq
On 2/20/19 9:15 AM, Julien Grall wrote: > Hi Boris, > > Thank you for your answer. > > On 20/02/2019 00:02, Boris Ostrovsky wrote: >> On Tue, Feb 19, 2019 at 05:31:10PM +, Julien Grall wrote: >>> Hi all, >>> >>> I have been looking at using Linux RT in Dom0. Once the guest is >>> started, >>> the console is ending to have a lot of warning (see trace below). >>> >>> After some investigation, this is because the irq handler will now >>> be threaded. >>> I can reproduce the same error with the vanilla Linux when passing >>> the option >>> 'threadirqs' on the command line (the trace below is from 5.0.0-rc7 >>> that has >>> not RT support). >>> >>> FWIW, the interrupt for port 6 is used to for the guest to >>> communicate with >>> xenstore. >>> >>> From my understanding, this is happening because the interrupt >>> handler is now >>> run in a thread. So we can have the following happening. >>> >>> Interrupt context | Interrupt thread >>> | >>> receive interrupt port 6 | >>> clear the evtchn port | >>> set IRQF_RUNTHREAD | >>> kick interrupt thread | >>> | clear IRQF_RUNTHREAD >>> | call evtchn_interrupt >>> receive interrupt port 6 | >>> clear the evtchn port | >>> set IRQF_RUNTHREAD | >>> kick interrupt thread | >>> | disable interrupt port 6 >>> | evtchn->enabled = false >>> | [] >>> | >>> | *** Handling the second >>> interrupt *** >>> | clear IRQF_RUNTHREAD >>> | call evtchn_interrupt >>> | WARN(...) >>> >>> I am not entirely sure how to fix this. I have two solutions in mind: >>> >>> 1) Prevent the interrupt handler to be threaded. We would also need to >>> switch from spin_lock to raw_spin_lock as the former may sleep on >>> RT-Linux. >>> >>> 2) Remove the warning >> >> I think access to evtchn->enabled is racy so (with or without the >> warning) we can't use it reliably. > > Thinking about it, it would not be the only issue. The ring is sized > to contain only one instance of the same event. So if you receive > twice the event, you may overflow the ring. Hm... That's another argument in favor of "unthreading" the handler. > >> >> Another alternative could be to queue the irq if !evtchn->enabled and >> handle it in evtchn_write() (which is where irq is supposed to be >> re-enabled). > What do you mean by queue? Is it queueing in the ring? No, I was thinking about having a new structure for deferred interrupts. -boris > If so, wouldn't it be racy as described above? ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] Reducing or removing direct map from xen (was Re: Ongoing/future speculative mitigation work)
On Wed, Feb 20, 2019 at 01:09:56PM +, Wei Liu wrote: [...] > I think under-allocate-then-map looks plausible. xmalloc will need > to allocate pages, put them into an array and call __vmap on that array > directly. The biggest issue with this approach is that we now need an array of 1UL
[Xen-devel] [PATCH 2/6] osstest: introduce a helper to create a weblink to a directory
Signed-off-by: Roger Pau Monné
---
Osstest/TestSupport.pm | 13 +
1 file changed, 13 insertions(+)
diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index c6da5ee9..12427d11 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -125,6 +125,7 @@ BEGIN {
toolstack guest_create guest_create_paused
await_webspace_fetch_byleaf create_webfile
+ create_weblink
file_link_contents get_timeout
setup_netboot_di setup_netboot_local host_netboot_file
subst_netboot_template setup_netboot_memdisk
@@ -2739,6 +2740,18 @@ sub create_webfile ($$$) {
return $wf_url;
}
+sub create_weblink ($$$) {
+my ($ho, $tail, $target) = @_;
+my $wf_rhs= hostnamepath($ho)."_".$tail;
+my $wf_common= $c{WebspaceCommon}.$wf_rhs;
+my $wf_url= $c{WebspaceUrl}.$wf_common;
+my $wf_file= $c{WebspaceFile}.$wf_common;
+
+unlink $wf_file;
+symlink $target, $wf_file or die "$wf_file $!";
+return $wf_url;
+}
+
#-- netboot handling --
sub file_link_contents ($$$) {
--
2.17.2 (Apple Git-113)
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [PATCH 5/6] osstest: introduce a script to build a FreeBSD package repository
The building of the binary packages itself is done with the poudriere
tool, that given a set of port names generates a binary package
repository with the requested packages and all it's dependencies.
Add a packages build job in the FreeBSD flight. Note that the binary
packages generated are tied to the base system used to build them when
not using a FreeBSD stable branch, stable branches guarantee ABI
stability.
Fetching the ports tree that contain the Makefiles to build the ports
is slightly more complicated than what would expected, since the
FreeBSD base system doesn't contain a git client the fetching is done
form the svn repository using the svnlite tool from the base system.
This is required in order to assure that bootstrapping the binary
repository doesn't depend on any external tools not found in the base
system.
Signed-off-by: Roger Pau Monné
---
ap-common | 6 ++
ap-fetch-version | 19 +++--
cr-daily-branch | 57 +--
make-freebsd-flight | 8 ++-
sg-run-job| 9 ++-
ts-freebsd-build-packages | 145 ++
6 files changed, 215 insertions(+), 29 deletions(-)
create mode 100755 ts-freebsd-build-packages
diff --git a/ap-common b/ap-common
index 87df7948..8083b6c2 100644
--- a/ap-common
+++ b/ap-common
@@ -41,6 +41,11 @@
: ${PUSH_TREE_FREEBSD:=$XENBITS:/home/xen/git/freebsd.git}
: ${BASE_TREE_FREEBSD:=git://xenbits.xen.org/freebsd.git}
+: ${TREE_FREEBSD_PORTS:=git://github.com/freebsd/freebsd-ports.git}
+: ${TREE_FREEBSD_PORTS_SVN:=https://svn.FreeBSD.org/ports/head}
+: ${PUSH_TREE_FREEBSD_PORTS:=$XENBITS:/home/xen/git/freebsd-ports.git}
+: ${BASE_TREE_FREEBSD_PORTS:=git://xenbits.xen.org/freebsd-ports.git}
+
: ${TREE_LIBVIRT:=git://libvirt.org/libvirt.git}
: ${PUSH_TREE_LIBVIRT:=$XENBITS:/home/xen/git/libvirt.git}
: ${BASE_TREE_LIBVIRT:=git://xenbits.xen.org/libvirt.git}
@@ -88,6 +93,7 @@ fi
: ${LOCALREV_OVMF:=daily-cron.$branch}
: ${LOCALREV_XTF:=daily-cron.$branch}
: ${LOCALREV_FREEBSD:=daily-cron.$branch}
+: ${LOCALREV_FREEBSD_PORTS:=daily-cron.$branch}
: ${TREEBASE_LINUX_XCP:=http://hg.uk.xensource.com/carbon/trunk/linux-2.6.27}
diff --git a/ap-fetch-version b/ap-fetch-version
index 87725bf0..a0963cc5 100755
--- a/ap-fetch-version
+++ b/ap-fetch-version
@@ -108,11 +108,22 @@ ovmf)
;;
freebsd-*)
branchcore=${branch#freebsd-}
- if [ "x$branchcore" != "xmaster" ]; then
- branchcore="stable/$branchcore"
+ subbranch=${branchcore#*--}
+ branchcore=${branchcore%--"$subbranch"}
+
+ if [ "$subbranch" = "$branchcore" ]; then
+ if [ "$branchcore" != master ]; then
+ branchcore="stable/$branchcore"
+ fi
+ repo_tree_rev_fetch_git freebsd \
+ $TREE_FREEBSD $branchcore $LOCALREV_FREEBSD
+ else
+ if [ "$subbranch" != ports ]; then
+ exit 1
+ fi
+ repo_tree_rev_fetch_git freebsd-ports \
+ $TREE_FREEBSD_PORTS master $LOCALREV_FREEBSD_PORTS
fi
- repo_tree_rev_fetch_git freebsd \
- $TREE_FREEBSD $branchcore $LOCALREV_FREEBSD
;;
osstest)
if [ "x$OSSTEST_USE_HEAD" = "xy" ] ; then
diff --git a/cr-daily-branch b/cr-daily-branch
index 49b8ad8e..971f4c01 100755
--- a/cr-daily-branch
+++ b/cr-daily-branch
@@ -47,19 +47,28 @@ determine_version () {
local tversionvar=$1
local tbranch=$2
local treevarwhich=$3
- if [ "x$tbranch" = "x$branch" ] && [ "x$force_baseline" = x ]; then
-if [ "x$FORCE_REVISION" != x ]; then
-tversion="$FORCE_REVISION"
-else
- tversion=`$AP_FETCH_PFX ./ap-fetch-version "$tbranch"`
-fi
- determine_tree "$treevarwhich" "" _${treevarwhich}
- determine_tree "$treevarwhich" "" _${treevarwhich}_THIS
- else
+
+ case "$tbranch" in
+ "$branch"|"$branch"--*)
+ if [ "x$force_baseline" = x ]; then
+ if [ "x$FORCE_REVISION" != x ]; then
+ tversion="$FORCE_REVISION"
+ else
+ tversion=`$AP_FETCH_PFX ./ap-fetch-version
"$tbranch"`
+ fi
+ determine_tree "$treevarwhich" "" _${treevarwhich}
+ determine_tree "$treevarwhich" "" _${treevarwhich}_THIS
+ eval "$tversionvar=$tversion"
+ return
+ fi
+ ;& # fallthrough
+ *)
tversion=`$AP_FETCH_PFX ./ap-fetch-version-baseline "$tbranch"`
determine_tree "$treevarwhich" BASE_ _${treevarwhich}
determine_tree "$treevarwhich" BASE_ _${treevarwhich}_THIS
- fi
+ ;;
+ esac
+
eval
[Xen-devel] [PATCH 3/6] osstest: allow to perform multiple anoints in the same transaction
In the same way allow to perform several fetches in the same retrieve
transaction.
Further patches will anoint a FreeBSD image and a binary ports tree
in the same transaction, and it's required to do it in the same
transaction in order to avoid inconsistencies when fetching them.
Note that most of the changes in this patch is code movement in order
to place the database accessors inside of a loop that iterates over
the input parameters.
Signed-off-by: Roger Pau Monné
---
mg-anoint | 167 --
1 file changed, 88 insertions(+), 79 deletions(-)
diff --git a/mg-anoint b/mg-anoint
index d09124b3..08447b8e 100755
--- a/mg-anoint
+++ b/mg-anoint
@@ -10,12 +10,12 @@
#
# ./mg-anoint destroy REFKEY
#
-# ./mg-anoint anoint [ANOINT-OPTION...] REFKEY FLIGHT JOB
+# ./mg-anoint anoint [ANOINT-OPTION...] REFKEY FLIGHT JOB [REFKEY FLIGHT
JOB...]
# ANOINT-OPTIONs are:
#--allow-blessed=BLESSING,... default is from `prepare'
#--allow-job-status=STATUS,... default is only `pass'
#
-# ./mg-anoint retrieve [--tolerate-unprepared] REFKEY
+# ./mg-anoint retrieve [--tolerate-unprepared] REFKEY [REFKEY...]
# => FLIGHT JOB
# if nothing anointed yet, prints nothing and exits 0
# if anointment not prepared, fails
@@ -189,8 +189,7 @@ sub cmd_anoint {
die "unknown option $_ ?";
}
}
-die unless @ARGV==3;
-my ($refkey, $flight, $job) = @ARGV;
+die unless @ARGV%3==0;
fail_unless_can_anoint();
prep_queries();
@@ -228,69 +227,74 @@ END
db_retry($dbh_tests, [], sub {
@o = ();
-$task_q->execute($refkey);
-
- # find the task row (ie, the anointment kind)
- my ($task, $refinfo) = $task_q->fetchrow_array();
- die "no such anointment kind \`$refkey' (no prepare?)\n"
- unless defined $task;
- my %params;
- foreach (split /\s+/, $refinfo) {
- die unless m/=/;
- $params{$`} = $';
- }
- my %blessings;
- $blessings{$_}++ foreach
- grep /./,
- (split /,/, $params{blessings}),
- (split /,/, $allow_blessed);
-
- my %jobstatus;
- $jobstatus{pass}++;
- $jobstatus{$_}++ foreach grep /./, split /,/, $allow_jobstatus;
-
- # check the to-be-anointed flight's blessing
- $newflight_q->execute($flight);
- my $frow = $newflight_q->fetchrow_hashref();
- die "flight $flight missing" unless $frow;
- die "flight $flight not started" unless defined $frow->{started};
-
- # check the job status
- $newjob_q->execute($flight, $job);
- my ($jstatus) = $newjob_q->fetchrow_array();
- die "job $flight.$job missing" unless defined $jstatus;
- die "job $flight.$job status $jstatus" unless $jobstatus{$jstatus};
-
- push @o, "flight $flight blessed $frow->{blessing}".
-" started ".show_abs_time($frow->{started});
-
- die "flight $flight blessing $frow->{blessing}".
- " (not $params{blessings} / $allow_blessed)"
- unless $blessings{ $frow->{blessing} };
-
- # check to-be-annointed flight is most recent
- $mostrecent_q->execute($task);
- my $mostrecent = $mostrecent_q->fetchrow_hashref();
- die "flight $flight not newer than $mostrecent->{flight}"
- unless $frow->{started} > ($mostrecent->{started} // 0);
-
- # expire old anointments
- $count_q->execute($task);
- my ($current) = $count_q->fetchrow_array();
- my $want_delete = ($current+1) - $params{keep};
- push @o, "anointment $refkey: currently $current anointed";
- if ($want_delete > 0) {
- $todelete_q->execute($task, $want_delete);
- while (my $d = $todelete_q->fetchrow_hashref()) {
- push @o, " expiring $d->{flight}.$d->{job} [/$d->{shareix}]".
- " started ".show_abs_time($d->{started});
- $delete_res_q->execute($task, $d->{flight}, $d->{shareix});
- }
- }
- # at last!
- $insert_q->execute($flight,$task,$task,$job);
- push @o, "anointed $flight.$job";
+ for (my $i=0; $i < @ARGV; $i+=3) {
+ my ($refkey, $flight, $job) = @ARGV[$i..$i+2];
+
+ $task_q->execute($refkey);
+
+ # find the task row (ie, the anointment kind)
+ my ($task, $refinfo) = $task_q->fetchrow_array();
+ die "no such anointment kind \`$refkey' (no prepare?)\n"
+ unless defined $task;
+ my %params;
+ foreach (split /\s+/, $refinfo) {
+ die unless m/=/;
+ $params{$`} = $';
+ }
+ my %blessings;
+ $blessings{$_}++ foreach
+ grep /./,
+ (split /,/, $params{blessings}),
+ (split /,/, $allow_blessed);
+
+ my
[Xen-devel] [PATCH 4/6] osstest: introduce a helper to get the svn revision of a git commit
This only works when the svn revision is stored as a git note
with the format 'revision='.
Such conversion is required in order to bootstrap a FreeBSD system
without relying on external package repositories. FreeBSD base system
only contains a subversion client (no git client), and thus in order
to fetch the ports repository (that contain the external packages
build makefiles) svn must be used.
Signed-off-by: Roger Pau Monné
---
cri-common | 35 +++
1 file changed, 27 insertions(+), 8 deletions(-)
diff --git a/cri-common b/cri-common
index 8d2d26cf..2655a9ac 100644
--- a/cri-common
+++ b/cri-common
@@ -38,12 +38,10 @@ besteffort_repo () {
cached_repo "$1" '[fetch=try]'
}
-repo_tree_rev_fetch_git () {
- local treename=$1
- local remoteurl=$2
- local remotetag=$3
- local localtag=$4
+repo_get_realurl () {
+ local remoteurl=$1
local proxy=`getconfig GitCacheProxy`
+
case $remoteurl in
$proxy*)
local realurl="$remoteurl" ;;
@@ -52,14 +50,35 @@ repo_tree_rev_fetch_git () {
*)
local realurl="$remoteurl" ;;
esac
- if ! test -d $repos/$treename; then
- git clone --bare $realurl $repos/$treename >&2
- fi
+
+ echo $realurl
+}
+
+repo_tree_rev_fetch_git () {
+ local treename=$1
+ local remoteurl=$2
+ local remotetag=$3
+ local localtag=$4
+ local realurl=`repo_get_realurl $remoteurl`
+
cd $repos/$treename
git fetch -f $realurl $remotetag:$localtag >&2
git rev-parse $localtag^0
}
+repo_tree_git2svn_rev () {
+ local treename=$1
+ local remoteurl=$2
+ local gitrev=$3
+ local realurl=`repo_get_realurl $remoteurl`
+
+ cd $repos/$treename
+ git fetch -f $realurl refs/notes/commits:refs/notes/commits >&2
+ git notes show $gitrev | \
+ sed -n 's/^.*revision=\([0-9][0-9]*\).*$/\1/p'
+}
+
+
select_prevxenbranch () {
prevxenbranch=`./cri-getprevxenbranch $xenbranch`
}
--
2.17.2 (Apple Git-113)
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [PATCH 6/6] osstest: use a locally built pkg repository for FreeBSD
This removes the dependency on the official pkg repository, which is
dangerous when not testing stable branches, since the ABI of the
development branch is not stable, and thus it's easy for packages to
get out of sync, or for osstest to test an old FreeBSD version that
has an ABI different than the one used to build the official
packages.
The output of the package build test is tested together with the newly
built image, and if there are no regressions both are anointed in
lockstep in order to prevent temporary discrepancies between the
installer and the package repository.
Note that in order to bootstrap the first run it's possible to
manually set the package repository to use with an environment
variable:
- FREEBSD_PACKAGES__BUILDJOB: points to the flight.job that
contains the binary package repository.
- FREEBSD_PACKAGES: points to a local directory that contains the
binary repository.
It's also possible to set the directory that contains the package
repository in the configuration file using FreeBSDPackages.
Signed-off-by: Roger Pau Monné
---
cr-daily-branch | 35 +++
make-freebsd-flight | 10 +++--
mfi-common | 94 +++--
ts-build-prep-freebsd | 43 +++
ts-freebsd-host-install | 13 --
5 files changed, 147 insertions(+), 48 deletions(-)
diff --git a/cr-daily-branch b/cr-daily-branch
index 971f4c01..28cbd61a 100755
--- a/cr-daily-branch
+++ b/cr-daily-branch
@@ -321,15 +321,26 @@ freebsd)
esac
IFS=$'\n'
+count=0
for anointed in \
-`./mg-anoint list-prepared "freebsd build $freebsd_branch *"`; do
+`./mg-anoint list-prepared "freebsd* build $freebsd_branch *"`; do
# Retrieve previous successful FreeBSD build for each arch.
freebsd_arch=${anointed##* }
-freebsd_envvar="FREEBSD_${freebsd_arch^^}_BUILDJOB"
+freebsd_name=${anointed%% *}
+freebsd_name=${freebsd_name/-/_}
+freebsd_envvar="${freebsd_name^^}_${freebsd_arch^^}_BUILDJOB"
if [ "x${!freebsd_envvar}" = "x" ]; then
-flight_job=`./mg-anoint retrieve "$anointed"`
-export ${freebsd_envvar}=${flight_job/ /.}
+ envvars[$count]="$freebsd_envvar"
+ refkeys[$count]="$anointed"
+ count=$((count+1))
+fi
+done
+count=0
+for flight_job in `./mg-anoint retrieve ${refkeys[@]}`; do
+if [ "$flight_job" != "ERROR" ]; then
+ export ${envvars[$count]}=${flight_job/ /.}
fi
+count=$((count+1))
done
unset IFS
@@ -542,17 +553,23 @@ freebsd-*)
[ "x$OSSTEST_BLESSING" == "xreal" ]; then
IFS=$'\n'
for anointed in `./mg-anoint list-prepared \
- "freebsd build $freebsd_branch *"`; do
+ "freebsd* build $freebsd_branch *"`; do
# Update anointed versions
# NB: failure to update an anointed build for a specific arch
# should not be fatal, and it's not an issue if one of the
# arches gets slightly out of sync with the other ones.
freebsd_arch=${anointed##* }
-if ./mg-anoint anoint "$anointed" \
- $flight build-$freebsd_arch-freebsd; then
-echo "Anointed artifacts from build-$freebsd_arch-freebsd"
-fi
+freebsd_name=${anointed%% *}
+ # Rely on the fact that the job suffix is the same as the
+ # anointment refkey. Ie:
+ # refkey: freebsd job: build--freebsd
+ # refkey: freebsd-packages job: build--freebsd-packages
+anoint="$anoint \"$anointed\" $flight \
+build-$freebsd_arch-$freebsd_name"
done
+ if ./mg-anoint anoint $anoint; then
+ echo "Anointed build artifacts from flight"
+ fi
unset IFS
fi
;;
diff --git a/make-freebsd-flight b/make-freebsd-flight
index fc3d2d83..0458a33b 100755
--- a/make-freebsd-flight
+++ b/make-freebsd-flight
@@ -45,12 +45,14 @@ for arch in "$arches"; do
recipe_testinstall=true"
create_freebsd_pkg_build_job build-$arch-freebsd-packages
-# Create an identical job that's going to use the build output from
-# the previous one.
+# Create a build job that going to use the output of both the jobs
+# above in order to test the newly built FreeBSD and packages
+freebsd_runvars="$freebsd_runvars \
+ freebsdpackagesbuildjob=build-$arch-freebsd-packages"
create_freebsd_build_job build-$arch-freebsd-again
-# Create a Xen build job that's going to use the output from the first
-# FreeBSD build job.
+# Create a Xen build job that's going to use the output from the
+# FreeBSD build jobs.
create_xen_build_job build-$arch-xen-freebsd build-xen-freebsd \
host_hostflags=arch-$arch,purpose-build all_host_os=freebsd \
$freebsd_runvars
diff --git a/mfi-common b/mfi-common
index
[Xen-devel] [PATCH 0/6] osstest: create a local binary FreeBSD package repository
Hello, In order to reliably run FreeBSD test with the development branch (aka HEAD) osstest needs it's custom binary package repository that contains the packages built against the specific version of FreeBSD under test. FreeBSD HEAD doesn't have any ABI guarantees, and as such binary packages are tied to the base system used to build them. This series introduces a new job to the FreeBSD specific flight, that builds a local binary package repository against the FreeBSD version under test. Note that this repository only contains the dependencies required to build Xen. The output of the package building job anointed together with the installer as a pair, so they are used in conjunction. Note that the package building job is (like the installer building job) only used by the FReeBSD specific flight, the output however will be consumed by other flights, just like the FreeBSD installer. The runvar changes for a FreeBSD flight are: +build-amd64-freebsd-packages all_host_os freebsd +build-amd64-freebsd-packages all_hostflagsPropEq:Firmware:bios:bios +build-amd64-freebsd-packages arch amd64 +build-amd64-freebsd-packages freebsdbuildjob build-amd64-freebsd +build-amd64-freebsd freebsdpackagesbuildjob 133315.build-amd64-freebsd-packages +build-amd64-freebsd-againfreebsdpackagesbuildjob build-amd64-freebsd-packages +build-amd64-freebsd-packages freebsdpackagesbuildjob 133315.build-amd64-freebsd-packages +build-amd64-xen-freebsd freebsdpackagesbuildjob build-amd64-freebsd-packages +build-amd64-freebsd-packages host_hostflags arch-amd64,purpose-build +build-amd64-freebsd-packages recipe_skipbuildprep true +build-amd64-freebsd-packages recipe_testinstall true +build-amd64-freebsd-packages revision_freebsdports +build-amd64-freebsd-packages svnrevision_freebsdports 483590 +build-amd64-freebsd-packages svntree_freebsdports https://svn.FreeBSD.org/ports/head +build-amd64-freebsd-packages tree_freebsdports git://github.com/freebsd/freebsd-ports.git And for a xen-unstable flight: +test-amd64-amd64-examine freebsdpackagesbuildjob 133315.build-amd64-freebsd-packages The following links show the result of a successful FreeBSD flight including the binary package repository build: http://logs.test-lab.xenproject.org/osstest/logs/133321/ Note that before applying the series the output of the above flight should be anointed, so follow up flights can used this output as the initial seed: $ ./mg-anoint anoint "freebsd build master amd64" 133321 build-amd64-freebsd \ "freebsd-packages build master amd64" 133321 build-amd64-freebsd-packages I've pushed the patch series to my git repo: git://xenbits.xen.org/people/royger/osstest.git freebsd-pkg Thanks, Roger. Roger Pau Monne (6): osstest: introduce a helper to stash a whole directory osstest: introduce a helper to create a weblink to a directory osstest: allow to perform multiple anoints in the same transaction osstest: introduce a helper to get the svn revision of a git commit osstest: introduce a script to build a FreeBSD package repository osstest: use a locally built pkg repository for FreeBSD Osstest/TestSupport.pm| 32 +++- ap-common | 6 ++ ap-fetch-version | 19 - cr-daily-branch | 92 ++--- cri-common| 35 ++-- make-freebsd-flight | 14 ++-- mfi-common| 94 - mg-anoint | 167 -- sg-run-job| 9 +- ts-build-prep-freebsd | 43 ++ ts-freebsd-build-packages | 145 + ts-freebsd-host-install | 13 --- 12 files changed, 506 insertions(+), 163 deletions(-) create mode 100755 ts-freebsd-build-packages -- 2.17.2 (Apple Git-113) ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [PATCH] tools/xentop: Display '-' when stats are not available.
From: Pritha Srivastava
Displaying 0 is misleading.
Signed-off-by: Pritha Srivastava
Signed-off-by: Ronan Abhamon
---
tools/xenstat/libxenstat/src/xenstat.c | 6 +
tools/xenstat/libxenstat/src/xenstat.h | 3 +
tools/xenstat/libxenstat/src/xenstat_linux.c | 47 +++---
tools/xenstat/libxenstat/src/xenstat_priv.h | 1 +
tools/xenstat/xentop/xentop.c| 159 +++
5 files changed, 158 insertions(+), 58 deletions(-)
diff --git a/tools/xenstat/libxenstat/src/xenstat.c
b/tools/xenstat/libxenstat/src/xenstat.c
index fbe44f3c56..8fa12d7bc0 100644
--- a/tools/xenstat/libxenstat/src/xenstat.c
+++ b/tools/xenstat/libxenstat/src/xenstat.c
@@ -729,6 +729,12 @@ unsigned long long xenstat_vbd_wr_sects(xenstat_vbd * vbd)
return vbd->wr_sects;
}
+/* Returns error while getting stats (1 if error happened, 0 otherwise) */
+unsigned int xenstat_vbd_error(xenstat_vbd * vbd)
+{
+ return vbd->error;
+}
+
/*
* Tmem functions
*/
diff --git a/tools/xenstat/libxenstat/src/xenstat.h
b/tools/xenstat/libxenstat/src/xenstat.h
index 47ec60e14d..fe13b65727 100644
--- a/tools/xenstat/libxenstat/src/xenstat.h
+++ b/tools/xenstat/libxenstat/src/xenstat.h
@@ -193,6 +193,9 @@ unsigned long long xenstat_vbd_wr_reqs(xenstat_vbd * vbd);
unsigned long long xenstat_vbd_rd_sects(xenstat_vbd * vbd);
unsigned long long xenstat_vbd_wr_sects(xenstat_vbd * vbd);
+/* Returns error while getting stats (1 if error happened, 0 otherwise) */
+unsigned int xenstat_vbd_error(xenstat_vbd * vbd);
+
/*
* Tmem functions - extract tmem information
*/
diff --git a/tools/xenstat/libxenstat/src/xenstat_linux.c
b/tools/xenstat/libxenstat/src/xenstat_linux.c
index 7cdd3bf91f..9421ca43c8 100644
--- a/tools/xenstat/libxenstat/src/xenstat_linux.c
+++ b/tools/xenstat/libxenstat/src/xenstat_linux.c
@@ -436,13 +436,15 @@ int xenstat_collect_vbds(xenstat_node * node)
ret = sscanf(dp->d_name, "%3s-%u-%u", buf, , );
if (ret != 3)
continue;
+ if (!(strstr(buf, "vbd")) && !(strstr(buf, "tap")))
+ continue;
if (strcmp(buf,"vbd") == 0)
vbd.back_type = 1;
else if (strcmp(buf,"tap") == 0)
vbd.back_type = 2;
else
- continue;
+ vbd.back_type = 0;
domain = xenstat_node_domain(node, domid);
if (domain == NULL) {
@@ -453,36 +455,29 @@ int xenstat_collect_vbds(xenstat_node * node)
continue;
}
- if((read_attributes_vbd(dp->d_name, "statistics/oo_req", buf,
256)<=0)
- || ((ret = sscanf(buf, "%llu", _reqs)) != 1))
- {
- continue;
- }
-
- if((read_attributes_vbd(dp->d_name, "statistics/rd_req", buf,
256)<=0)
- || ((ret = sscanf(buf, "%llu", _reqs)) != 1))
+ if (vbd.back_type == 1 || vbd.back_type == 2)
{
- continue;
- }
- if((read_attributes_vbd(dp->d_name, "statistics/wr_req", buf,
256)<=0)
- || ((ret = sscanf(buf, "%llu", _reqs)) != 1))
- {
- continue;
- }
-
- if((read_attributes_vbd(dp->d_name, "statistics/rd_sect", buf,
256)<=0)
- || ((ret = sscanf(buf, "%llu", _sects)) != 1))
- {
- continue;
+ vbd.error = 0;
+
+ if ((read_attributes_vbd(dp->d_name,
"statistics/oo_req", buf, 256)<=0) ||
+ ((ret = sscanf(buf, "%llu", _reqs)) !=
1) ||
+ (read_attributes_vbd(dp->d_name,
"statistics/rd_req", buf, 256)<=0) ||
+ ((ret = sscanf(buf, "%llu", _reqs)) !=
1) ||
+ (read_attributes_vbd(dp->d_name,
"statistics/wr_req", buf, 256)<=0) ||
+ ((ret = sscanf(buf, "%llu", _reqs)) !=
1) ||
+ (read_attributes_vbd(dp->d_name,
"statistics/rd_sect", buf, 256)<=0) ||
+ ((ret = sscanf(buf, "%llu", _sects)) !=
1) ||
+ (read_attributes_vbd(dp->d_name,
"statistics/wr_sect", buf, 256)<=0) ||
+ ((ret = sscanf(buf, "%llu", _sects)) !=
1))
+ {
+ vbd.error = 1;
+ }
}
-
- if((read_attributes_vbd(dp->d_name, "statistics/wr_sect", buf,
256)<=0)
- || ((ret = sscanf(buf, "%llu", _sects)) != 1))
+ else
{
- continue;
+ vbd.error = 1;
}
-
if ((xenstat_save_vbd(domain, )) ==
Re: [Xen-devel] [RESEND PATCH 0/7] Add FOLL_LONGTERM to GUP fast and use it
On Tue, Feb 19, 2019 at 09:30:33PM -0800, ira.we...@intel.com wrote: > From: Ira Weiny > > Resending these as I had only 1 minor comment which I believe we have covered > in this series. I was anticipating these going through the mm tree as they > depend on a cleanup patch there and the IB changes are very minor. But they > could just as well go through the IB tree. > > NOTE: This series depends on my clean up patch to remove the write parameter > from gup_fast_permitted()[1] > > HFI1, qib, and mthca, use get_user_pages_fast() due to it performance > advantages. These pages can be held for a significant time. But > get_user_pages_fast() does not protect against mapping of FS DAX pages. This I don't get - if you do lock down long term mappings performance of the actual get_user_pages call shouldn't matter to start with. What do I miss? ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [PATCH] x86/shadow: don't use map_domain_page_global() on paths that may not fail
The assumption (according to one comment) and hope (according to
another) that map_domain_page_global() can't fail are both wrong on
large enough systems. Do away with the guest_vtable field altogether,
and establish / tear down the desired mapping as necessary.
The alternatives, discarded as being undesirable, would have been to
either crash the guest in sh_update_cr3() when the mapping fails, or to
bubble up an error indicator, which upper layers would have a hard time
to deal with (other than again by crashing the guest).
Signed-off-by: Jan Beulich
--- a/xen/arch/x86/mm/shadow/multi.c
+++ b/xen/arch/x86/mm/shadow/multi.c
@@ -175,18 +175,22 @@ static inline bool
sh_walk_guest_tables(struct vcpu *v, unsigned long va, walk_t *gw,
uint32_t pfec)
{
-return guest_walk_tables(v, p2m_get_hostp2m(v->domain), va, gw, pfec,
#if GUEST_PAGING_LEVELS == 3 /* PAE */
- INVALID_MFN,
- v->arch.paging.shadow.gl3e
+return guest_walk_tables(v, p2m_get_hostp2m(v->domain), va, gw, pfec,
+ INVALID_MFN, v->arch.paging.shadow.gl3e);
#else /* 32 or 64 */
- (((v->arch.flags & TF_kernel_mode) ||
- is_pv_32bit_vcpu(v))
- ? pagetable_get_mfn(v->arch.guest_table)
- : pagetable_get_mfn(v->arch.guest_table_user)),
- v->arch.paging.shadow.guest_vtable
+const struct domain *d = v->domain;
+mfn_t root_mfn = ((v->arch.flags & TF_kernel_mode) || is_pv_32bit_domain(d)
+ ? pagetable_get_mfn(v->arch.guest_table)
+ : pagetable_get_mfn(v->arch.guest_table_user));
+void *root_map = map_domain_page(root_mfn);
+bool ok = guest_walk_tables(v, p2m_get_hostp2m(d), va, gw, pfec,
+root_mfn, root_map);
+
+unmap_domain_page(root_map);
+
+return ok;
#endif
- );
}
/* This validation is called with lock held, and after write permission
@@ -226,8 +230,9 @@ shadow_check_gwalk(struct vcpu *v, unsig
perfc_incr(shadow_check_gwalk);
#if GUEST_PAGING_LEVELS >= 3 /* PAE or 64... */
#if GUEST_PAGING_LEVELS >= 4 /* 64-bit only... */
-l4p = (guest_l4e_t *)v->arch.paging.shadow.guest_vtable;
+l4p = map_domain_page(gw->l4mfn);
mismatch |= (gw->l4e.l4 != l4p[guest_l4_table_offset(va)].l4);
+unmap_domain_page(l4p);
l3p = map_domain_page(gw->l3mfn);
mismatch |= (gw->l3e.l3 != l3p[guest_l3_table_offset(va)].l3);
unmap_domain_page(l3p);
@@ -235,13 +240,11 @@ shadow_check_gwalk(struct vcpu *v, unsig
mismatch |= (gw->l3e.l3 !=
v->arch.paging.shadow.gl3e[guest_l3_table_offset(va)].l3);
#endif
+#endif
l2p = map_domain_page(gw->l2mfn);
mismatch |= (gw->l2e.l2 != l2p[guest_l2_table_offset(va)].l2);
unmap_domain_page(l2p);
-#else
-l2p = (guest_l2e_t *)v->arch.paging.shadow.guest_vtable;
-mismatch |= (gw->l2e.l2 != l2p[guest_l2_table_offset(va)].l2);
-#endif
+
if ( !(guest_can_use_l2_superpages(v) &&
(guest_l2e_get_flags(gw->l2e) & _PAGE_PSE)) )
{
@@ -3862,7 +3865,8 @@ sh_update_linear_entries(struct vcpu *v)
}
-/* Removes vcpu->arch.paging.shadow.guest_vtable and vcpu->arch.shadow_table[].
+/*
+ * Removes vcpu->arch.shadow_table[].
* Does all appropriate management/bookkeeping/refcounting/etc...
*/
static void
@@ -3873,23 +3877,6 @@ sh_detach_old_tables(struct vcpu *v)
int i = 0;
- vcpu->arch.paging.shadow.guest_vtable
-
-
-#if GUEST_PAGING_LEVELS == 3
-/* PAE guests don't have a mapping of the guest top-level table */
-ASSERT(v->arch.paging.shadow.guest_vtable == NULL);
-#else
-if ( v->arch.paging.shadow.guest_vtable )
-{
-if ( shadow_mode_external(d) || shadow_mode_translate(d) )
-unmap_domain_page_global(v->arch.paging.shadow.guest_vtable);
-v->arch.paging.shadow.guest_vtable = NULL;
-}
-#endif // !NDEBUG
-
-
-
vcpu->arch.shadow_table[]
@@ -4044,29 +4031,12 @@ sh_update_cr3(struct vcpu *v, int do_loc
#endif
gmfn = pagetable_get_mfn(v->arch.guest_table);
-
-
- vcpu->arch.paging.shadow.guest_vtable
-
-#if GUEST_PAGING_LEVELS == 4
-if ( shadow_mode_external(d) || shadow_mode_translate(d) )
-{
-if ( v->arch.paging.shadow.guest_vtable )
-unmap_domain_page_global(v->arch.paging.shadow.guest_vtable);
-v->arch.paging.shadow.guest_vtable = map_domain_page_global(gmfn);
-/* PAGING_LEVELS==4 implies 64-bit, which means that
- * map_domain_page_global can't fail */
-BUG_ON(v->arch.paging.shadow.guest_vtable == NULL);
-}
-else
-v->arch.paging.shadow.guest_vtable = __linear_l4_table;
-#elif GUEST_PAGING_LEVELS == 3
+#if GUEST_PAGING_LEVELS == 3
/*
* On PAE
[Xen-devel] [linux-4.9 test] 133308: trouble: blocked/broken/fail/pass
flight 133308 linux-4.9 real [real] http://logs.test-lab.xenproject.org/osstest/logs/133308/ Failures and problems with tests :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-amd64-xl-qemuu-win10-i386 broken test-amd64-amd64-xl-qemuu-win7-amd64 broken test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadowbroken test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsmbroken build-amd64-libvirt broken test-armhf-armhf-xl-arndale broken test-amd64-i386-xl-qemuu-debianhvm-amd64-xsmbroken test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrictbroken test-amd64-amd64-xl-qcow2broken build-amd64-libvirt 4 host-install(4)broken REGR. vs. 132748 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm broken in 133290 test-amd64-i386-xl-xsm broken in 133290 Tests which are failing intermittently (not blocking): test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 4 host-install(4) broken in 133290 pass in 133308 test-amd64-i386-xl-xsm 4 host-install(4) broken in 133290 pass in 133308 test-armhf-armhf-xl-arndale 4 host-install(4) broken pass in 133290 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 4 host-install(4) broken pass in 133290 test-amd64-amd64-xl-qcow2 4 host-install(4) broken pass in 133290 test-amd64-amd64-xl-qemuu-win10-i386 4 host-install(4) broken pass in 133290 test-amd64-amd64-xl-qemuu-win7-amd64 4 host-install(4) broken pass in 133290 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm 4 host-install(4) broken pass in 133290 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 4 host-install(4) broken pass in 133290 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow 4 host-install(4) broken pass in 133290 test-amd64-amd64-xl-qemuu-ovmf-amd64 16 guest-localmigrate/x10 fail in 133276 pass in 133308 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm 16 guest-localmigrate/x10 fail in 133290 pass in 133308 test-amd64-amd64-xl-qemut-debianhvm-amd64 16 guest-localmigrate/x10 fail pass in 133276 test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm 10 debian-hvm-install fail pass in 133290 test-amd64-amd64-qemuu-nested-intel 17 debian-hvm-install/l1/l2 fail pass in 133290 test-amd64-amd64-i386-pvgrub 10 debian-di-install fail pass in 133290 test-armhf-armhf-libvirt-raw 10 debian-di-install fail pass in 133290 Tests which did not succeed, but are not blocking: test-amd64-amd64-libvirt 1 build-check(1) blocked n/a test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a test-amd64-amd64-libvirt-xsm 1 build-check(1) blocked n/a test-amd64-amd64-libvirt-pair 1 build-check(1) blocked n/a test-amd64-amd64-libvirt-vhd 1 build-check(1) blocked n/a test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop fail in 133290 like 132748 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install fail in 133290 never pass test-amd64-amd64-libvirt13 migrate-support-check fail in 133290 never pass test-amd64-amd64-libvirt-xsm 13 migrate-support-check fail in 133290 never pass test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check fail in 133290 never pass test-armhf-armhf-xl-arndale 13 migrate-support-check fail in 133290 never pass test-armhf-armhf-xl-arndale 14 saverestore-support-check fail in 133290 never pass test-amd64-amd64-libvirt-vhd 12 migrate-support-check fail in 133290 never pass test-armhf-armhf-libvirt-raw 12 migrate-support-check fail in 133290 never pass test-armhf-armhf-libvirt-raw 13 saverestore-support-check fail in 133290 never pass test-amd64-amd64-xl-qemuu-win10-i386 10 windows-install fail in 133290 never pass test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 132748 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 132748 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 132748 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 132748 test-amd64-amd64-xl-pvhv2-intel 12 guest-start fail never pass test-amd64-i386-xl-pvshim12 guest-start fail never pass test-arm64-arm64-xl-credit1 13 migrate-support-checkfail never pass test-arm64-arm64-xl-credit1 14 saverestore-support-checkfail never pass test-arm64-arm64-xl-credit2 13 migrate-support-checkfail never pass test-arm64-arm64-xl-credit2 14 saverestore-support-checkfail never pass test-arm64-arm64-xl 13 migrate-support-checkfail never pass test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail never pass test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail never pass test-arm64-arm64-xl
Re: [Xen-devel] [PATCH 4/4] x86/vmx: Properly flush the TLB when an altp2m is modified
>>> On 19.02.19 at 23:18, wrote:
> @@ -4319,17 +4320,42 @@ bool vmx_vmenter_helper(const struct cpu_user_regs
> *regs)
>
> if ( paging_mode_hap(curr->domain) )
> {
> -struct ept_data *ept = _get_hostp2m(curr->domain)->ept;
> +struct ept_data *ept = _get_hostp2m(currd)->ept;
> unsigned int cpu = smp_processor_id();
> +unsigned int inv = 0; /* None => Single => All */
> +struct ept_data *single = NULL; /* Single eptp, iff inv == 1 */
>
> if ( cpumask_test_cpu(cpu, ept->invalidate) )
> {
> cpumask_clear_cpu(cpu, ept->invalidate);
> -if ( nestedhvm_enabled(curr->domain) )
> -__invept(INVEPT_ALL_CONTEXT, 0);
> -else
> -__invept(INVEPT_SINGLE_CONTEXT, ept->eptp);
> +
> +/* Automatically invalidate all contexts if nested. */
> +inv += 1 + nestedhvm_enabled(currd);
> +single = ept;
> +}
> +
> +if ( altp2m_active(curr->domain) )
Please use currd here as well.
Reviewed-by: Jan Beulich
Jan
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH 3/4] x86/vmx: Fix security issue when a guest balloons out the #VE info page
>>> On 19.02.19 at 23:18, wrote:
> @@ -58,25 +57,67 @@ altp2m_vcpu_destroy(struct vcpu *v)
>
> int altp2m_vcpu_enable_ve(struct vcpu *v, gfn_t gfn)
> {
> +struct domain *d = v->domain;
> +struct altp2mvcpu *a = _altp2m(v);
> p2m_type_t p2mt;
> +mfn_t mfn;
> +struct page_info *pg;
> +int rc;
> +
> +/* Early exit path if #VE is already configured. */
> +if ( a->veinfo_pg )
> +return -EEXIST;
> +
> +mfn = get_gfn(d, gfn_x(gfn), );
> +
> +/*
> + * Looking for a plain piece of guest writeable RAM. Take an extra page
> + * reference to reflect our intent to point the VMCS at it.
> + */
> +if ( mfn_eq(mfn, INVALID_MFN) || !p2m_is_ram(p2mt) ||
> + p2m_is_readonly(p2mt) || !get_page(pg = mfn_to_page(mfn), d) )
p2m_is_ram() is pretty broad a class, and p2m_is_readonly() removes
only usefully p2m_ram_ro and p2m_ram_shared from the set. In
particular p2m_ioreq_server is thus permitted, as are all p2m_paging_*
which don't produce INVALID_MFN. I don't think that's what you want.
I also don't think you really want to exclude p2m_ram_logdirty - if
anything you might need to convert that type to p2m_ram_rw in case
you find the page in that state.
Can't you simply call check_get_page_from_gfn() here anyway?
Furthermore I'm uncertain if get_page() is quite enough: Don't you
also want a writable type reference? It may not strictly be needed at
this point, but I think we're trying to make the distinction in new code,
like was e.g. done in recent Viridian changes.
> +{
> +rc = -EINVAL;
> +goto out;
> +}
>
> -if ( !gfn_eq(vcpu_altp2m(v).veinfo_gfn, INVALID_GFN) ||
> - mfn_eq(get_gfn_query_unlocked(v->domain, gfn_x(gfn), ),
> -INVALID_MFN) )
> -return -EINVAL;
> +/*
> + * Update veinfo_pg, making sure to be safe with concurrent hypercalls.
> + * The first caller to make veinfo_pg become non-NULL will program its
> MFN
> + * into the VMCS, so must not be clobbered. Callers which lose the race
> + * back off with -EEXIST.
> + */
> +if ( cmpxchg(>veinfo_pg, NULL, pg) != NULL )
> +{
> +put_page(pg);
> +rc = -EEXIST;
> +goto out;
> +}
>
> -vcpu_altp2m(v).veinfo_gfn = gfn;
> +rc = 0;
> altp2m_vcpu_update_vmfunc_ve(v);
>
> -return 0;
> + out:
> +put_gfn(d, gfn_x(gfn));
Is there any reason why you need to hold the GFN ref until here? It would
seem to me that it can be dropped the moment you've obtained the general
page ref.
Jan
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 23/25] hw/ipmi: Assert outlen > outpos
On Wed, Feb 20, 2019 at 02:02:30AM +0100, Philippe Mathieu-Daudé wrote:
> A througfull audit show that all time data is added to outbuf[],
> 'outlen' is incremented. Then at creation and each time
> continue_send() returns it pass thru check_reset which resets
> 'outpos', thus we always have 'outlen >= outpos'.
Perhaps: "A thorough audit shows that outlen is always incremented
when data is always added to outbuf[]. Then at creation and each
time continus_send() returns it assures if outpos reaches outlen,
both values are reset to zero, except in the case of sending
a reset where a new command is added."
This is certainly the design intent, thank you for the thorough
audit.
> Also due to the check on entry, we know outlen != 0.
> We can then add an assertion on 'outlen > outpos', which will
> helps the next patch to safely convert 'outlen - outpos' as an
I was a little confused by "next patch", there is no following
patch in the series for this. Maybe "next part of the patch"?
> unsigned type (size_t).
>
> Make this assertion explicit by casting 'outlen - outpos' size_t.
>
> Signed-off-by: Philippe Mathieu-Daudé
Outside of the minor grammer issues, this looks good. I have
noticed the inconsistent signed/unsigned usage in qemu and IMHO
it's likely to lead to very bad bugs at some point. There have
been studies that show that unsigned values tend to be more
buggy in usage due to underflows, but for a length value that
will eventually be converted to an unsigned value, what is
here is better, I think.
Both outpos and outlen are unsigned, so the size_t() cast is
not really necessary, but I guess it makes it clear.
Reviewed-by: Corey Minyard
> ---
> hw/ipmi/ipmi_bmc_extern.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/hw/ipmi/ipmi_bmc_extern.c b/hw/ipmi/ipmi_bmc_extern.c
> index bf0b7ee0f5..ca61b04942 100644
> --- a/hw/ipmi/ipmi_bmc_extern.c
> +++ b/hw/ipmi/ipmi_bmc_extern.c
> @@ -107,8 +107,9 @@ static void continue_send(IPMIBmcExtern *ibe)
> goto check_reset;
> }
> send:
> +assert(ibe->outlen > ibe->outpos);
> ret = qemu_chr_fe_write(>chr, ibe->outbuf + ibe->outpos,
> -ibe->outlen - ibe->outpos);
> +(size_t)(ibe->outlen - ibe->outpos));
> if (ret > 0) {
> ibe->outpos += ret;
> }
> --
> 2.20.1
>
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [Qemu-devel] [PATCH v3 00/25] chardev: Convert qemu_chr_write() to take a size_t argument
On 2/20/19 5:30 AM, Daniel P. Berrangé wrote: >> Since Paolo you suggested the change, could you give some convincing >> arguments that it's worth taking the plunge? > > The chardev write/read methods will end up calling libc read/write > methods, whose parameters are "size_t count". In my mind, that's the convincing reason. We should model our read/write after the libc read/write, which means size_t input and ssize_t returns. > > Thus if there is QEMU code that could currently (mistakenly) pass a > negative value for length to qemu_chr_write, unless something stops > it, this is going to be cast to a size_t when we finally call read/ > write on the FD, leading to a large positive value & array out of > bounds read/write. > > IOW we already have inconsistent use of signed vs unsigned in our code > which has potential to cause bugs. Converting chardev to use size_t > we get rid fo the mismatch with the underlying libc APIs we call, > which ultimately eliminates an area of risk longer term. There is a > chance it could uncover some pre-existing dormant bugs, but provided > we do due diligence to check callers I think its a win to be consistent > with libc APIs in size_t usage for read/write. And hopefully this exercise of making the conversion serves as a good audit to help us gain confidence in our code and/or fix bugs it uncovers. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] xen/evtchn and forced threaded irq
Hi Boris, Thank you for your answer. On 20/02/2019 00:02, Boris Ostrovsky wrote: On Tue, Feb 19, 2019 at 05:31:10PM +, Julien Grall wrote: Hi all, I have been looking at using Linux RT in Dom0. Once the guest is started, the console is ending to have a lot of warning (see trace below). After some investigation, this is because the irq handler will now be threaded. I can reproduce the same error with the vanilla Linux when passing the option 'threadirqs' on the command line (the trace below is from 5.0.0-rc7 that has not RT support). FWIW, the interrupt for port 6 is used to for the guest to communicate with xenstore. From my understanding, this is happening because the interrupt handler is now run in a thread. So we can have the following happening. Interrupt context| Interrupt thread | receive interrupt port 6 | clear the evtchn port| set IRQF_RUNTHREAD | kick interrupt thread| |clear IRQF_RUNTHREAD |call evtchn_interrupt receive interrupt port 6 | clear the evtchn port| set IRQF_RUNTHREAD | kick interrupt thread| |disable interrupt port 6 |evtchn->enabled = false |[] | |*** Handling the second interrupt *** |clear IRQF_RUNTHREAD |call evtchn_interrupt |WARN(...) I am not entirely sure how to fix this. I have two solutions in mind: 1) Prevent the interrupt handler to be threaded. We would also need to switch from spin_lock to raw_spin_lock as the former may sleep on RT-Linux. 2) Remove the warning I think access to evtchn->enabled is racy so (with or without the warning) we can't use it reliably. Thinking about it, it would not be the only issue. The ring is sized to contain only one instance of the same event. So if you receive twice the event, you may overflow the ring. Another alternative could be to queue the irq if !evtchn->enabled and handle it in evtchn_write() (which is where irq is supposed to be re-enabled). What do you mean by queue? Is it queueing in the ring? If so, wouldn't it be racy as described above? Cheers, -- Julien Grall ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH 1/4] xen/common: Break domain_unmap_resources() out of domain_kill()
>>> On 19.02.19 at 23:46, wrote:
> On 19/02/2019 22:39, Razvan Cojocaru wrote:
>> On 2/20/19 12:18 AM, Andrew Cooper wrote:
>>> A subsequent change is going to need an x86-specific unmapping step, so take
>>> the opportunity to split the current vcpu unmapping out into a dedicated
> path.
>>>
>>> No practical change.
>>>
>>> Signed-off-by: Andrew Cooper
>>> ---
>>> CC: Jan Beulich
>>> CC: Wei Liu
>>> CC: Roger Pau Monné
>>> CC: Razvan Cojocaru
>>> CC: Tamas K Lengyel
>>> CC: Juergen Gross
>>> ---
>>> xen/common/domain.c | 16 +---
>>> xen/include/xen/domain.h | 4
>>> 2 files changed, 17 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/xen/common/domain.c b/xen/common/domain.c
>>> index 32bca8d..e66f7ea 100644
>>> --- a/xen/common/domain.c
>>> +++ b/xen/common/domain.c
>>> @@ -700,10 +700,21 @@ int rcu_lock_live_remote_domain_by_id(domid_t dom,
> struct domain **d)
>>> return 0;
>>> }
>>>
>>> +static void domain_unmap_resources(struct domain *d)
>>> +{
>>> +struct vcpu *v;
>>> +
>>> +for_each_vcpu ( d, v )
>>> +{
>>> +unmap_vcpu_info(v);
>>> +
>>> +arch_vcpu_unmap_resources(v);
>>> +}
>>> +}
>>> +
>>> int domain_kill(struct domain *d)
>>> {
>>> int rc = 0;
>>> -struct vcpu *v;
>>>
>>> if ( d == current->domain )
>>> return -EINVAL;
>>> @@ -732,13 +743,12 @@ int domain_kill(struct domain *d)
>>> d->tmem_client = NULL;
>>> /* fallthrough */
>>> case DOMDYING_dying:
>>> +domain_unmap_resources(d);
>>> rc = domain_relinquish_resources(d);
>>> if ( rc != 0 )
>>> break;
>>> if ( cpupool_move_domain(d, cpupool0) )
>>> return -ERESTART;a
>>> -for_each_vcpu ( d, v )
>>> -unmap_vcpu_info(v);
>> So before this change there was a leak of some sort here? I see that
>> before this patch unmap_vcpu_info() was called only if rc == 0, but it
>> is now called unconditionally _before_ domain_relinquish_resources().
>>
>> This does appear to change the behaviour of the code in the error case.
>>
>> If that's intended:
>> Reviewed-by: Razvan Cojocaru
>
> domain_kill() is a very long running hypercall, and takes about 15
> minutes of wallclock time for domains with 1T of RAM (before the idle
> scrubbing changes went in).
>
> It will frequently break out with -ERESTART for continuation purposes,
> but doesn't fail outright.
>
> However, with hindsight it would probably be better just to put the
> altp2m case in domain_relinquish_resources() and do away with this patch
> entirely.
While this patch could have my ack, I'd clearly prefer you going the
described alternative route.
Jan
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH 2/4] x86/altp2m: Rework #VE enable/disable paths
>>> On 20.02.19 at 00:10, wrote:
> On 2/20/19 12:18 AM, Andrew Cooper wrote:
>> Split altp2m_vcpu_{enable,disable}_ve() out of the
>> HVMOP_altp2m_vcpu_{enable,disable}_notify marshalling logic. A future
> change
>> is going to need to call altp2m_vcpu_disable_ve() from the domain_kill()
> path.
>>
>> While at it, clean up the logic in altp2m_vcpu_{initialise,destroy}().
>> altp2m_vcpu_reset() has no external callers, so fold it into its two
>> callsites. This in turn allows for altp2m_vcpu_destroy() to reuse
>> altp2m_vcpu_disable_ve() rather than opencoding it.
>>
>> No practical change.
>>
>> Signed-off-by: Andrew Cooper
>
> Reviewed-by: Razvan Cojocaru
Acked-by: Jan Beulich
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 09/11] optee: add support for RPC commands
Hi,
On 19/02/2019 16:14, Volodymyr Babchuk wrote:
Hi Julien,
Julien Grall writes:
Hi Volodymyr,
On 18/12/2018 21:11, Volodymyr Babchuk wrote:
From: Volodymyr Babchuk
OP-TEE can issue multiple RPC requests. We are interested mostly in
request that asks NW to allocate/free shared memory for OP-TEE
needs, because mediator need to do address translation in the same
NIT: the mediator needs
way as it was done for shared buffers registered by NW.
As mediator now accesses shared command buffer, we need to shadow
it in the same way, as we shadow request buffers for STD calls.
This is a bit confusing, does it means patch #8 is not doing the right thing?
No, it was patch #6 :)
And I can't say that it did something wrong. Remember that prior to the
last patch in series DomU can't use the mediator. And for Dom0 it is okay to
map RPC command buffer directly. Description of patch #4 mentions that
we need all patches in the series for a complete mediator.
Not all the memory in Dom0 is 1:1 mapped. So you may end up to use the wrong
address here.
But, it is not very intuitive to have to read the commit message of patch #4 to
understand that patch #8 is fixing a flaw in patch #6.
Technically, earlier patch should not have allowed to use shared command buffer
until now. While I appreciate it is hard to split big series, we at least need
to write clear commit message. In that case "now accesses" clear lead to think
something wrong has been done before. So a reminder in the commit message would
help the reviewer here.
Signed-off-by: Volodymyr Babchuk
---
Changes from v2:
- Use access_guest_memory_by_ipa() instead of direct mapping
xen/arch/arm/tee/optee.c | 136 +--
1 file changed, 130 insertions(+), 6 deletions(-)
diff --git a/xen/arch/arm/tee/optee.c b/xen/arch/arm/tee/optee.c
index cfc3b34df7..bf3535946d 100644
--- a/xen/arch/arm/tee/optee.c
+++ b/xen/arch/arm/tee/optee.c
@@ -67,6 +67,8 @@ struct optee_std_call {
struct shm_rpc {
struct list_head list;
struct page_info *guest_page;
+struct optee_msg_arg *xen_arg;
+paddr_t guest_ipa;
uint64_t cookie;
};
@@ -290,6 +292,11 @@ static struct shm_rpc
*allocate_and_pin_shm_rpc(struct optee_domain *ctx,
P2M_ALLOC);
if ( !shm_rpc->guest_page )
goto err;
+shm_rpc->guest_ipa = gaddr;
+
+shm_rpc->xen_arg = alloc_xenheap_page();
Based on the discussion in patch #6, I think you want to use to
allocate the memory from domheap.
Yes, will do.
I already did it for #6 and now there is a lots of places
where I am mapping/unmapping that page. So, it is somewhat
inconvenient...
Well, security is always inconvenient until we found a flaw ;).
Cheers,
--
Julien Grall
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 24/25] chardev: Let qemu_chr_fe_write[_all] use size_t type argument
On Wed, Feb 20, 2019 at 2:08 AM Philippe Mathieu-Daudé
wrote:
>
> All caller have been audited and call these functions with
> unsigned arguments.
>
> Most of them use a size_t argument, or directly pass sizeof().
>
> One case is unclear: the mux_chr_write() call in chardev/char-mux.c.
> There we add an assert (which will be removed in few patches) and
> cast the parameter as size_t to make explicit this value is unsigned.
mux_chr_write() is called indirectly from qemu_chr_fe_write(), so the
same argument applies here. Fine with or without the assert().
>
> Suggested-by: Paolo Bonzini
> Signed-off-by: Philippe Mathieu-Daudé
Reviewed-by: Marc-André Lureau
> ---
> chardev/char-fe.c | 4 ++--
> chardev/char-mux.c| 3 ++-
> include/chardev/char-fe.h | 4 ++--
> 3 files changed, 6 insertions(+), 5 deletions(-)
>
> diff --git a/chardev/char-fe.c b/chardev/char-fe.c
> index f3530a90e6..ab2a01709d 100644
> --- a/chardev/char-fe.c
> +++ b/chardev/char-fe.c
> @@ -31,7 +31,7 @@
> #include "chardev/char-io.h"
> #include "chardev/char-mux.h"
>
> -int qemu_chr_fe_write(CharBackend *be, const uint8_t *buf, int len)
> +int qemu_chr_fe_write(CharBackend *be, const uint8_t *buf, size_t len)
> {
> Chardev *s = be->chr;
>
> @@ -42,7 +42,7 @@ int qemu_chr_fe_write(CharBackend *be, const uint8_t *buf,
> int len)
> return qemu_chr_write(s, buf, len, false);
> }
>
> -int qemu_chr_fe_write_all(CharBackend *be, const uint8_t *buf, int len)
> +int qemu_chr_fe_write_all(CharBackend *be, const uint8_t *buf, size_t len)
> {
> Chardev *s = be->chr;
>
> diff --git a/chardev/char-mux.c b/chardev/char-mux.c
> index 23aa82125d..7a3ff21db4 100644
> --- a/chardev/char-mux.c
> +++ b/chardev/char-mux.c
> @@ -38,7 +38,8 @@ static int mux_chr_write(Chardev *chr, const uint8_t *buf,
> int len)
> MuxChardev *d = MUX_CHARDEV(chr);
> int ret;
> if (!d->timestamps) {
> -ret = qemu_chr_fe_write(>chr, buf, len);
> +assert(len >= 0);
> +ret = qemu_chr_fe_write(>chr, buf, (size_t)len);
> } else {
> int i;
>
> diff --git a/include/chardev/char-fe.h b/include/chardev/char-fe.h
> index aa1b864ccd..5fb2c2e7ec 100644
> --- a/include/chardev/char-fe.h
> +++ b/include/chardev/char-fe.h
> @@ -203,7 +203,7 @@ guint qemu_chr_fe_add_watch(CharBackend *be, GIOCondition
> cond,
> *
> * Returns: the number of bytes consumed (0 if no associated Chardev)
> */
> -int qemu_chr_fe_write(CharBackend *be, const uint8_t *buf, int len);
> +int qemu_chr_fe_write(CharBackend *be, const uint8_t *buf, size_t len);
>
> /**
> * qemu_chr_fe_write_all:
> @@ -217,7 +217,7 @@ int qemu_chr_fe_write(CharBackend *be, const uint8_t
> *buf, int len);
> *
> * Returns: the number of bytes consumed (0 if no associated Chardev)
> */
> -int qemu_chr_fe_write_all(CharBackend *be, const uint8_t *buf, int len);
> +int qemu_chr_fe_write_all(CharBackend *be, const uint8_t *buf, size_t len);
>
> /**
> * qemu_chr_fe_read_all:
> --
> 2.20.1
>
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v2 for-4.12] x86/vpmu: Improve documentation and parsing for vpmu=
>>> On 20.02.19 at 14:29, wrote: > The behaviour of vpmu= being exclusive of vpmu=bts|ipc|arch is odd and > contrary to Xen's normal command line parsing behaviour. Rewrite the parsing > to use the normal form, but retain the previous behaviour where the use of > bts/ipc/arch implies vpmu=true. > > Parts of the documenation are stale, most notibly the HVM-only statement. > Update it for consistency and correctness. > > Signed-off-by: Andrew Cooper > Release-acked-by: Juergen Gross Reviewed-by: Jan Beulich ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 23/25] hw/ipmi: Assert outlen > outpos
Hi
On Wed, Feb 20, 2019 at 2:08 AM Philippe Mathieu-Daudé
wrote:
>
> A througfull audit show that all time data is added to outbuf[],
througfull? :) thorough?
> 'outlen' is incremented. Then at creation and each time
> continue_send() returns it pass thru check_reset which resets
> 'outpos', thus we always have 'outlen >= outpos'.
> Also due to the check on entry, we know outlen != 0.
> We can then add an assertion on 'outlen > outpos', which will
hmm, I think you could assert 'outlen >= outpos' only. I don't buy
your argument about 'outlen > outpos' (because outlen != 0?)
> helps the next patch to safely convert 'outlen - outpos' as an
> unsigned type (size_t).
>
> Make this assertion explicit by casting 'outlen - outpos' size_t.
>
> Signed-off-by: Philippe Mathieu-Daudé
> ---
> hw/ipmi/ipmi_bmc_extern.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/hw/ipmi/ipmi_bmc_extern.c b/hw/ipmi/ipmi_bmc_extern.c
> index bf0b7ee0f5..ca61b04942 100644
> --- a/hw/ipmi/ipmi_bmc_extern.c
> +++ b/hw/ipmi/ipmi_bmc_extern.c
> @@ -107,8 +107,9 @@ static void continue_send(IPMIBmcExtern *ibe)
> goto check_reset;
> }
> send:
> +assert(ibe->outlen > ibe->outpos);
> ret = qemu_chr_fe_write(>chr, ibe->outbuf + ibe->outpos,
> -ibe->outlen - ibe->outpos);
> +(size_t)(ibe->outlen - ibe->outpos));
> if (ret > 0) {
> ibe->outpos += ret;
> }
> --
> 2.20.1
>
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [PATCH v2 for-4.12] x86/vpmu: Improve documentation and parsing for vpmu=
The behaviour of vpmu= being exclusive of vpmu=bts|ipc|arch is odd and
contrary to Xen's normal command line parsing behaviour. Rewrite the parsing
to use the normal form, but retain the previous behaviour where the use of
bts/ipc/arch implies vpmu=true.
Parts of the documenation are stale, most notibly the HVM-only statement.
Update it for consistency and correctness.
Signed-off-by: Andrew Cooper
Release-acked-by: Juergen Gross
---
CC: Jan Beulich
CC: Wei Liu
CC: Roger Pau Monné
v2:
* Have vpmu=0 superceed previous bts/ipc/arch settings, for better
consistency similar Xen options
* Rephrase the documentation to better indicate the relationship between the
sub-options.
---
docs/misc/xen-command-line.pandoc | 44 +
xen/arch/x86/cpu/vpmu.c | 68 ++-
2 files changed, 54 insertions(+), 58 deletions(-)
diff --git a/docs/misc/xen-command-line.pandoc
b/docs/misc/xen-command-line.pandoc
index c8d1ced..a03c0b4 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -2108,36 +2108,38 @@ Use Virtual Processor ID support if available. This
prevents the need for TLB
flushes on VM entry and exit, increasing performance.
### vpmu (x86)
-> `= ( | { bts | ipc | arch [, ...] } )`
+= List of [ , bts, ipc, arch ]
-> Default: `off`
+Applicability: x86. Default: false
-Switch on the virtualized performance monitoring unit for HVM guests.
+Controls for Performance Monitoring Unit virtualisation.
-If the current cpu isn't supported a message like
-'VPMU: Initialization failed. ...'
-is printed on the hypervisor serial log.
+Performance monitoring facilities tend to be very hardware specific, and
+provide access to a wealth of low level processor information.
-For some Intel Nehalem processors a quirk handling exist for an unknown
-wrong behaviour (see `handle_pmc_quirk()`).
+* An overall boolean can be used to enable or disable vPMU support. vPMU is
+disabled by default.
-If 'vpmu=bts' is specified the virtualisation of the Branch Trace Store (BTS)
-feature is switched on on Intel processors supporting this feature.
+When enabled, guests have full access to all performance counter settings,
+including model specific functionality. This is a superset of the
+functionality offered by `ipc` and/or `arch`, but a subset of the
+functionality offered by `bts`.
-vpmu=ipc enables performance monitoring, but restricts the counters to the
-most minimum set possible: instructions, cycles, and reference cycles. These
-can be used to calculate instructions per cycle (IPC).
+Xen's watchdog functionality is implemented using performance counters.
+As a result, use of the **watchdog** option will override and disable
+vPMU.
-vpmu=arch enables performance monitoring, but restricts the counters to the
-pre-defined architectural events only. These are exposed by cpuid, and listed
-in the Pre-Defined Architectural Performance Events table from the Intel 64
-and IA-32 Architectures Software Developer's Manual, Volume 3B, System
-Programming Guide, Part 2.
+* The `bts` option enables performance monitoring, and permits additional
+access to the Branch Trace Store controls. BTS is an Intel feature where
+the processor can write data into a buffer whenever a branch occurs.
+However, as this feature isn't virtualised, a misconfiguration by the
+guest can lock the entire system up.
-If a boolean is not used, combinations of flags are allowed, comma separated.
-For example, vpmu=arch,bts.
+* The `ipc` option allows access to the most minimal set of counters
+possible: instructions, cycles, and reference cycles. These can be used
+to calculate instructions per cycle (IPC).
-Note that if **watchdog** option is also specified vpmu will be turned off.
+* The `arch` option allows access to the pre-defined architectural events.
*Warning:*
As the virtualisation is not 100% safe, don't use the vpmu flag on
diff --git a/xen/arch/x86/cpu/vpmu.c b/xen/arch/x86/cpu/vpmu.c
index 13da7d0..8324d62 100644
--- a/xen/arch/x86/cpu/vpmu.c
+++ b/xen/arch/x86/cpu/vpmu.c
@@ -42,19 +42,9 @@ CHECK_pmu_cntr_pair;
CHECK_pmu_data;
CHECK_pmu_params;
-/*
- * "vpmu" : vpmu generally enabled (all counters)
- * "vpmu=off" : vpmu generally disabled
- * "vpmu=bts" : vpmu enabled and Intel BTS feature switched on.
- * "vpmu=ipc" : vpmu enabled for IPC counters only (most restrictive)
- * "vpmu=arch" : vpmu enabled for predef arch counters only (restrictive)
- * flag combinations are allowed, eg, "vpmu=ipc,bts".
- */
static unsigned int __read_mostly opt_vpmu_enabled;
unsigned int __read_mostly vpmu_mode = XENPMU_MODE_OFF;
unsigned int __read_mostly vpmu_features = 0;
-static int parse_vpmu_params(const char *s);
-custom_param("vpmu", parse_vpmu_params);
static DEFINE_SPINLOCK(vpmu_lock);
static unsigned vpmu_count;
@@ -64,37 +54,41 @@ static
Re: [Xen-devel] [PATCH v3 04/25] chardev: Let qemu_chr_be_can_write() return a size_t types
Hi
On Wed, Feb 20, 2019 at 12:26 PM Philippe Mathieu-Daudé
wrote:
>
> On 2/20/19 11:40 AM, Marc-André Lureau wrote:
> > Hi
> >
> > On Wed, Feb 20, 2019 at 2:04 AM Philippe Mathieu-Daudé
> > wrote:
> >>
> >> In the previous commit we added an assert to be sure than
> >> qemu_chr_be_can_write() will never return a negative value.
> >> We can now change its prototype to return a size_t.
> >> Adapt the backends accordingly.
> >
> > Each variable you change to an unsigned type, we should check it isn't
> > used with negative values.
>
> Fortunately the preprocessor can help here!
>
> Oh I forgot to write down the steps I ran:
>
> # enable warnings
> $ configure \
> --extra-cflags='-Wtype-limits -Wsign-conversion -Wsign-compare' \
> --disable-werror
>
> # since there are many sign abuse, build blindly
> $ make 2>/dev/null
>
> # now refresh the source we modified
> $ git diff --name-only origin/master \
> | egrep \.c\$ \
> | xargs touch
>
> # build again and carefully watch the warnings
> # (there are many unuseful #include warnings, ignore them)
> $ make
>
> >>
> >> Suggested-by: Paolo Bonzini
> >> Signed-off-by: Philippe Mathieu-Daudé
> >> ---
> >> chardev/baum.c| 6 +++---
> >> chardev/char-fd.c | 2 +-
> >> chardev/char-pty.c| 4 ++--
> >> chardev/char-socket.c | 7 ---
> >> chardev/char-udp.c| 4 ++--
> >> chardev/char-win.c| 2 +-
> >> chardev/char.c| 2 +-
> >> chardev/msmouse.c | 4 ++--
> >> chardev/spice.c | 2 +-
> >> chardev/wctablet.c| 4 ++--
> >> hw/bt/hci-csr.c | 2 +-
> >> include/chardev/char-fd.h | 2 +-
> >> include/chardev/char.h| 2 +-
> >> ui/console.c | 6 +++---
> >> 14 files changed, 25 insertions(+), 24 deletions(-)
> >>
> >> diff --git a/chardev/baum.c b/chardev/baum.c
> >> index 78b0c87625..1d69d62158 100644
> >> --- a/chardev/baum.c
> >> +++ b/chardev/baum.c
> >> @@ -265,7 +265,7 @@ static int baum_deferred_init(BaumChardev *baum)
> >> static void baum_chr_accept_input(struct Chardev *chr)
> >> {
> >> BaumChardev *baum = BAUM_CHARDEV(chr);
> >> -int room, first;
> >> +size_t room, first;
> >>
> >> if (!baum->out_buf_used)
> >> return;
> >> @@ -292,7 +292,7 @@ static void baum_write_packet(BaumChardev *baum, const
> >> uint8_t *buf, int len)
> >> {
> >> Chardev *chr = CHARDEV(baum);
> >> uint8_t io_buf[1 + 2 * len], *cur = io_buf;
> >> -int room;
> >> +size_t room;
> >> *cur++ = ESC;
> >> while (len--)
> >> if ((*cur++ = *buf++) == ESC)
> >> @@ -303,7 +303,7 @@ static void baum_write_packet(BaumChardev *baum, const
> >> uint8_t *buf, int len)
> >> /* Fits */
> >> qemu_chr_be_write(chr, io_buf, len);
> >> } else {
> >> -int first;
> >> +size_t first;
> >> uint8_t out;
> >> /* Can't fit all, send what can be, and store the rest. */
> >> qemu_chr_be_write(chr, io_buf, room);
> >
> > baum room & first are only used for non-negative capacity values. ack
> >
> >> diff --git a/chardev/char-fd.c b/chardev/char-fd.c
> >> index 2421d8e216..0fe2822869 100644
> >> --- a/chardev/char-fd.c
> >> +++ b/chardev/char-fd.c
> >> @@ -43,7 +43,7 @@ static gboolean fd_chr_read(QIOChannel *chan,
> >> GIOCondition cond, void *opaque)
> >> {
> >> Chardev *chr = CHARDEV(opaque);
> >> FDChardev *s = FD_CHARDEV(opaque);
> >> -int len;
> >> +size_t len;
> >> uint8_t buf[CHR_READ_BUF_LEN];
> >> ssize_t ret;
> >>
> >
> > fd len is only used for non-negative buffer size. ack
> >
> >> diff --git a/chardev/char-pty.c b/chardev/char-pty.c
> >> index f6ddef..eae25f043b 100644
> >> --- a/chardev/char-pty.c
> >> +++ b/chardev/char-pty.c
> >> @@ -34,7 +34,7 @@
> >> typedef struct {
> >> Chardev parent;
> >> QIOChannel *ioc;
> >> -int read_bytes;
> >> +size_t read_bytes;
> >>
> >
> > Only set with values returned from qemu_chr_be_can_write(), ack
> >
> >> int connected;
> >> GSource *timer_src;
> >> @@ -132,7 +132,7 @@ static gboolean pty_chr_read(QIOChannel *chan,
> >> GIOCondition cond, void *opaque)
> >> {
> >> Chardev *chr = CHARDEV(opaque);
> >> PtyChardev *s = PTY_CHARDEV(opaque);
> >> -gsize len;
> >> +size_t len;
> >> uint8_t buf[CHR_READ_BUF_LEN];
> >> ssize_t ret;
> >
> > pty len is only used for non-negative buffer size. ack
> >
> >> diff --git a/chardev/char-socket.c b/chardev/char-socket.c
> >> index 262a59b64f..4010c343e0 100644
> >> --- a/chardev/char-socket.c
> >> +++ b/chardev/char-socket.c
> >> @@ -60,7 +60,7 @@ typedef struct {
> >> GSource *hup_source;
> >> QCryptoTLSCreds *tls_creds;
> >> TCPChardevState state;
> >> -int max_size;
> >> +size_t max_size;
> >
> > Only set with values returned from qemu_chr_be_can_write(), ack
> >
> >> int do_telnetopt;
> >> int do_nodelay;
> >> int
Re: [Xen-devel] [PATCH v3 08/11] optee: add support for arbitrary shared memory
On 19/02/2019 15:59, Volodymyr Babchuk wrote: Hello Julien, Hi Volodymyr, Julien Grall writes: Hi Volodymyr, On 18/12/2018 21:11, Volodymyr Babchuk wrote: From: Volodymyr Babchuk Shared memory is widely used by NW to communicate with TAs in OP-TEE. NW can share part of own memory with TA or OP-TEE core, by registering it OP-TEE, or by providing a temporal reference. Anyways, information about such memory buffers are sent to OP-TEE as a list of pages. This mechanism is described in optee_msg.h. Mediator should step in when NW tries to share memory with OP-TEE for two reasons: 1. Do address translation from IPA to PA. 2. Pin domain pages till they are mapped into OP-TEE or TA Looking at the code, I think the page are mapped while OP-TEE is using them. If so, it should be s/till/while/. address space, so domain can't transfer this pages to other domain or balloon out them. > Address translation is done by translate_noncontig(...) function. It allocates new buffer from xenheap and then walks on guest provided list of pages, translates addresses and stores PAs into newly allocated buffer. This buffer will be provided to OP-TEE instead of original buffer from the guest. This buffer will be free at the end of standard call. In the same time this function pins pages and stores them in struct optee_shm_buf object. This object will live all the time, when given SHM buffer is known to OP-TEE. It will be freed after guest unregisters shared buffer. At this time pages will be unpinned. We don't need to do any special reference counting because OP-TEE tracks buffer on its side. So, mediator will unpin pages only when OP-TEE returns successfully from OPTEE_MSG_CMD_UNREGISTER_SHM call. Signed-off-by: Volodymyr Babchuk --- Changes from v2: - Made sure that guest does not tries to register shared buffer with the same cookie twice - Fixed coding style - Use access_guest_memory_by_ipa() instead of direct memory mapping xen/arch/arm/tee/optee.c | 274 +++ 1 file changed, 274 insertions(+) diff --git a/xen/arch/arm/tee/optee.c b/xen/arch/arm/tee/optee.c index 771148e940..cfc3b34df7 100644 --- a/xen/arch/arm/tee/optee.c +++ b/xen/arch/arm/tee/optee.c @@ -37,6 +37,10 @@ */ #define MAX_RPC_SHMSMAX_STD_CALLS +/* Maximum total number of pages that guest can share with OP-TEE */ +#define MAX_TOTAL_SMH_BUF_PG16384 Please explain in the commit message and code how you came up to this value. Basically it is taken from head. I just needed some number. You see, number of registered shared memory buffer solely depends on free heap in OP-TEE. But the same heap is used for other purposes, so it is hard to tell how much pages can be shared with OP-TEE. I assumed that 64M ought to be enough for anybody. Such explanation would be fine for me in the commit message and the code. The goal is to log how we came up with the value (even if it is random). This would help us if we need to refine the value in the future. Probably it is worth to make this configurable via domctl interface. It is not strictly necessary for now. We can refine it in the future if needed. [...] +page_offset = param->u.tmem.buf_ptr & (OPTEE_MSG_NONCONTIG_PAGE_SIZE - 1); + +/* Size of the user buffer in bytes */ +size = ROUNDUP(param->u.tmem.size + page_offset, + OPTEE_MSG_NONCONTIG_PAGE_SIZE); + +num_pages = DIV_ROUND_UP(size, OPTEE_MSG_NONCONTIG_PAGE_SIZE); + +order = get_order_from_bytes(get_pages_list_size(num_pages)); + +pages_data_xen_start = alloc_xenheap_pages(order, 0); By using alloc_xenheap_pages, you may end-up allocation more memory than necessary when the order is getting bigger. What is the bigger order you expect here? It depend on MAX_TOTAL_SMH_BUF_PG. One page can contain up to 511 entries, plus reference to the next one. So, basically at max I will need about 32 pages, which gives order 5-6. I think, I can try xzalloc or domheap there. But looks like there is no xmem_pool for the domheap. So, I still will be forced to use alloc_domheap_pages(). xmem_pool is not used when you allocate buffer larger than PAGE_SIZE. Instead, xmalloc will allocate an order bigger than free unused page (see xmalloc_whole_pages). It is not overly critical to allocate more for now, but it would be nice to add it as a TODO in the code. Cheers, -- Julien Grall ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] Reducing or removing direct map from xen (was Re: Ongoing/future speculative mitigation work)
On Wed, Feb 20, 2019 at 02:00:52PM +0100, Roger Pau Monné wrote: > On Wed, Feb 20, 2019 at 12:29:01PM +, Wei Liu wrote: > > On Thu, Jan 24, 2019 at 11:44:55AM +, Wei Liu wrote: > > > 3. Implement xenheap using vmap infrastructure > > > > > > This helps preserve xenheap's "always mapped" property. At the moment, > > > vmap relies on xenheap, we want to turn this relationship around. > > > > > > There is a loop what needs breaking in the new world: > > > > > > alloc_xenheap_pages -> vmap -> __vmap -> map_pages_to_xen -> > > > virt_to_xen_l1e -> alloc_xen_pagetable -> alloc_xenheap_page -> vmap > > > ... > > > > > > Two options were proposed to break this loop: > > > > > > 3.1 Pre-populate all page tables for vmap region > > > > Now that we have this ... > > > > > 3.2 Switch page table allocation to use domheap page > > > > > > > > > The other work item is to track page<->virt relationship so that > > > conversion functions (_to_virt etc) continue to work. For PoC purpose, > > > putting a void * into page_info is good enough. But in the future we > > > want to have a separate array for tracking so that page_info stays power > > > of two in size. > > > > > > > I started working on some prototyping code for the rest of this major > > work item. Conversion functions are a bit messy to deal with (I have no > > idea whether my modifications are totally correct at this point), but > > the most major issue I see is an optimisation done by xmalloc which > > isn't compatible with vmap. > > > > So xmalloc has this optimisation: it will allocate a high-order page > > from xenheap when necessary and then attempt to break that up and return > > the unused portion. Vmap uses bitmap to track address space usage, and > > it mandates a guard page before every address space allocation. What > > xmalloc does is to free a portion of the address space, which isn't > > really supported by vmap. > > > > I came up with two options yesterday: > > > > 1. Remove the optimisation in xmalloc > > 2. Make vmap able to break up allocation > > > > Neither looks great to me. The first is simple but potentially wasteful > > (how much is wasted?). The second requires non-trivial modification to > > vmap, essentially removing the mandatory guard page. In comparison the > > first is easier and safer. > > > > I would like to hear people's thought on this. Comments are welcome. > > The PV dom0 builder does something similar to this, it tries to > allocate a page that has an order equal or higher than the order of > the request size, and then frees up the unused part. > > I've used another approach for the PVH dom0 builder, which is to never > allocate more than what's required, and instead always under-allocate. > This has the benefit of not splitting high order pages, but requires > multiple calls to the allocation function. See > pvh_populate_memory_range in hvm/dom0_build.c and it's usage of > get_order_from_pages. I think a similar approach could be implemented > in xmalloc? > The usage in PV dom0 build is not an issue because those pages are domheap pages. On a related topic, I have to fix that instance since it treats domheap pages like xenheap pages, which will be very wrong in the future. Your example of PVH dom0 build uses domheap pages too, so that's not an issue. I think under-allocate-then-map looks plausible. xmalloc will need to allocate pages, put them into an array and call __vmap on that array directly. Wei. > Roger. ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] Reducing or removing direct map from xen (was Re: Ongoing/future speculative mitigation work)
On Wed, Feb 20, 2019 at 12:29:01PM +, Wei Liu wrote: > On Thu, Jan 24, 2019 at 11:44:55AM +, Wei Liu wrote: > > 3. Implement xenheap using vmap infrastructure > > > > This helps preserve xenheap's "always mapped" property. At the moment, > > vmap relies on xenheap, we want to turn this relationship around. > > > > There is a loop what needs breaking in the new world: > > > > alloc_xenheap_pages -> vmap -> __vmap -> map_pages_to_xen -> > > virt_to_xen_l1e -> alloc_xen_pagetable -> alloc_xenheap_page -> vmap ... > > > > Two options were proposed to break this loop: > > > > 3.1 Pre-populate all page tables for vmap region > > Now that we have this ... > > > 3.2 Switch page table allocation to use domheap page > > > > > > The other work item is to track page<->virt relationship so that > > conversion functions (_to_virt etc) continue to work. For PoC purpose, > > putting a void * into page_info is good enough. But in the future we > > want to have a separate array for tracking so that page_info stays power > > of two in size. > > > > I started working on some prototyping code for the rest of this major > work item. Conversion functions are a bit messy to deal with (I have no > idea whether my modifications are totally correct at this point), but > the most major issue I see is an optimisation done by xmalloc which > isn't compatible with vmap. > > So xmalloc has this optimisation: it will allocate a high-order page > from xenheap when necessary and then attempt to break that up and return > the unused portion. Vmap uses bitmap to track address space usage, and > it mandates a guard page before every address space allocation. What > xmalloc does is to free a portion of the address space, which isn't > really supported by vmap. > > I came up with two options yesterday: > > 1. Remove the optimisation in xmalloc > 2. Make vmap able to break up allocation > > Neither looks great to me. The first is simple but potentially wasteful > (how much is wasted?). The second requires non-trivial modification to > vmap, essentially removing the mandatory guard page. In comparison the > first is easier and safer. > > I would like to hear people's thought on this. Comments are welcome. The PV dom0 builder does something similar to this, it tries to allocate a page that has an order equal or higher than the order of the request size, and then frees up the unused part. I've used another approach for the PVH dom0 builder, which is to never allocate more than what's required, and instead always under-allocate. This has the benefit of not splitting high order pages, but requires multiple calls to the allocation function. See pvh_populate_memory_range in hvm/dom0_build.c and it's usage of get_order_from_pages. I think a similar approach could be implemented in xmalloc? Roger. ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] Reducing or removing direct map from xen (was Re: Ongoing/future speculative mitigation work)
On Thu, Jan 24, 2019 at 11:44:55AM +, Wei Liu wrote: > 3. Implement xenheap using vmap infrastructure > > This helps preserve xenheap's "always mapped" property. At the moment, > vmap relies on xenheap, we want to turn this relationship around. > > There is a loop what needs breaking in the new world: > > alloc_xenheap_pages -> vmap -> __vmap -> map_pages_to_xen -> > virt_to_xen_l1e -> alloc_xen_pagetable -> alloc_xenheap_page -> vmap ... > > Two options were proposed to break this loop: > > 3.1 Pre-populate all page tables for vmap region Now that we have this ... > 3.2 Switch page table allocation to use domheap page > > > The other work item is to track page<->virt relationship so that > conversion functions (_to_virt etc) continue to work. For PoC purpose, > putting a void * into page_info is good enough. But in the future we > want to have a separate array for tracking so that page_info stays power > of two in size. > I started working on some prototyping code for the rest of this major work item. Conversion functions are a bit messy to deal with (I have no idea whether my modifications are totally correct at this point), but the most major issue I see is an optimisation done by xmalloc which isn't compatible with vmap. So xmalloc has this optimisation: it will allocate a high-order page from xenheap when necessary and then attempt to break that up and return the unused portion. Vmap uses bitmap to track address space usage, and it mandates a guard page before every address space allocation. What xmalloc does is to free a portion of the address space, which isn't really supported by vmap. I came up with two options yesterday: 1. Remove the optimisation in xmalloc 2. Make vmap able to break up allocation Neither looks great to me. The first is simple but potentially wasteful (how much is wasted?). The second requires non-trivial modification to vmap, essentially removing the mandatory guard page. In comparison the first is easier and safer. I would like to hear people's thought on this. Comments are welcome. Wei. ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 19/25] s390/ebcdic: Use size_t to iterate over arrays
On 2/20/19 10:40 AM, Cornelia Huck wrote:
> On Wed, 20 Feb 2019 02:02:26 +0100
> Philippe Mathieu-Daudé wrote:
>
>> Signed-off-by: Philippe Mathieu-Daudé
>> ---
>> include/hw/s390x/ebcdic.h | 8
>> 1 file changed, 4 insertions(+), 4 deletions(-)
>>
>> diff --git a/include/hw/s390x/ebcdic.h b/include/hw/s390x/ebcdic.h
>> index 69a04cab62..d89174e113 100644
>> --- a/include/hw/s390x/ebcdic.h
>> +++ b/include/hw/s390x/ebcdic.h
>> @@ -83,18 +83,18 @@ static const uint8_t ascii2ebcdic[] = {
>> 0x90, 0x3F, 0x3F, 0x3F, 0x3F, 0xEA, 0x3F, 0xFF
>> };
>>
>> -static inline void ebcdic_put(uint8_t *p, const char *ascii, int len)
>> +static inline void ebcdic_put(uint8_t *p, const char *ascii, size_t len)
>> {
>> -int i;
>> +size_t i;
>>
>> for (i = 0; i < len; i++) {
>> p[i] = ascii2ebcdic[(uint8_t)ascii[i]];
>> }
>> }
>>
>> -static inline void ascii_put(uint8_t *p, const char *ebcdic, int len)
>> +static inline void ascii_put(uint8_t *p, const char *ebcdic, size_t len)
>> {
>> -int i;
>> +size_t i;
>>
>> for (i = 0; i < len; i++) {
>> p[i] = ebcdic2ascii[(uint8_t)ebcdic[i]];
>
> Making the passed len parameter a size_t makes sense; but using a
> size_t as an array iterator looks a bit unidiomatic... it's not wrong,
> though.
This is to silent the "-Wsign-conversion -Wsign-compare" warnings...
I am still not sure what is the best to do.
> Acked-by: Cornelia Huck
Thanks!
Phil.
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [Qemu-devel] [PATCH v3 25/25] chardev: Let qemu_chr_write[_all] use size_t
On 2/20/19 11:42 AM, Marc-André Lureau wrote: > Hi > > On Wed, Feb 20, 2019 at 11:39 AM Daniel P. Berrangé > wrote: >> >> On Wed, Feb 20, 2019 at 02:02:32AM +0100, Philippe Mathieu-Daudé wrote: >> >>> diff --git a/include/chardev/char.h b/include/chardev/char.h >>> index 0341dd1ba2..2e3b5a15ca 100644 >>> --- a/include/chardev/char.h >>> +++ b/include/chardev/char.h >>> @@ -221,7 +221,7 @@ void qemu_chr_set_feature(Chardev *chr, >>>ChardevFeature feature); >>> QemuOpts *qemu_chr_parse_compat(const char *label, const char *filename, >>> bool permit_mux_mon); >>> -int qemu_chr_write(Chardev *s, const uint8_t *buf, int len, bool >>> write_all); >>> +int qemu_chr_write(Chardev *s, const uint8_t *buf, size_t len, bool >>> write_all); >> >> Seeing this cleanup reminds me that I think we ought to change >> the chardev read & write functions to take "void *buf" instead. >> as is done for regular libc read/write functions. This would >> avoid casts in the callers between char */uint8_t * >> >> Something to think about for a future cleanup jobsame applies >> for the QIOChannel APIs which take a 'char *buf', annoyingly >> different from the chardev APIs :-( Both ought to have void *buf > > I fully agree, and I started that conversion some time ago, and lost > it somewhere. That should be easier than changing the sign of return > values though! I thought about it too and am happy both of you agree. The "part #3" update the prototypes to return ssize_t instead of int. I can do this change there. >> >> >> Regards, >> Daniel >> -- >> |: https://berrange.com -o-https://www.flickr.com/photos/dberrange >> :| >> |: https://libvirt.org -o-https://fstop138.berrange.com >> :| >> |: https://entangle-photo.org-o-https://www.instagram.com/dberrange >> :| ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [Qemu-devel] [PATCH v3 00/25] chardev: Convert qemu_chr_write() to take a size_t argument
On Wed, Feb 20, 2019 at 11:53:42AM +0100, Marc-André Lureau wrote: > Hi > > On Wed, Feb 20, 2019 at 2:02 AM Philippe Mathieu-Daudé > wrote: > > > > Hi, > > > > This series convert the chardev::qemu_chr_write() to take unsigned > > length argument. To do so I went through all caller and checked if > > there are no negative value possible. > > > Changing signedness is problematic and can easily introduce bugs that > are easy to miss during review. > > I agree with Cornelia about idiomatic use of int. Changing "int" for > "size_t" isn't systematically a clear win. > > Even Google C++ style recommends to avoid unsigned types "(except for > representing bitfields or modular arithmetic). Do not use an unsigned > type merely to assert that a variable is non-negative." > https://google.github.io/styleguide/cppguide.html#Integer_Types - see > rationale > > Since Paolo you suggested the change, could you give some convincing > arguments that it's worth taking the plunge? The chardev write/read methods will end up calling libc read/write methods, whose parameters are "size_t count". Thus if there is QEMU code that could currently (mistakenly) pass a negative value for length to qemu_chr_write, unless something stops it, this is going to be cast to a size_t when we finally call read/ write on the FD, leading to a large positive value & array out of bounds read/write. IOW we already have inconsistent use of signed vs unsigned in our code which has potential to cause bugs. Converting chardev to use size_t we get rid fo the mismatch with the underlying libc APIs we call, which ultimately eliminates an area of risk longer term. There is a chance it could uncover some pre-existing dormant bugs, but provided we do due diligence to check callers I think its a win to be consistent with libc APIs in size_t usage for read/write. Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :| ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 04/25] chardev: Let qemu_chr_be_can_write() return a size_t types
On 2/20/19 11:40 AM, Marc-André Lureau wrote:
> Hi
>
> On Wed, Feb 20, 2019 at 2:04 AM Philippe Mathieu-Daudé
> wrote:
>>
>> In the previous commit we added an assert to be sure than
>> qemu_chr_be_can_write() will never return a negative value.
>> We can now change its prototype to return a size_t.
>> Adapt the backends accordingly.
>
> Each variable you change to an unsigned type, we should check it isn't
> used with negative values.
Fortunately the preprocessor can help here!
Oh I forgot to write down the steps I ran:
# enable warnings
$ configure \
--extra-cflags='-Wtype-limits -Wsign-conversion -Wsign-compare' \
--disable-werror
# since there are many sign abuse, build blindly
$ make 2>/dev/null
# now refresh the source we modified
$ git diff --name-only origin/master \
| egrep \.c\$ \
| xargs touch
# build again and carefully watch the warnings
# (there are many unuseful #include warnings, ignore them)
$ make
>>
>> Suggested-by: Paolo Bonzini
>> Signed-off-by: Philippe Mathieu-Daudé
>> ---
>> chardev/baum.c| 6 +++---
>> chardev/char-fd.c | 2 +-
>> chardev/char-pty.c| 4 ++--
>> chardev/char-socket.c | 7 ---
>> chardev/char-udp.c| 4 ++--
>> chardev/char-win.c| 2 +-
>> chardev/char.c| 2 +-
>> chardev/msmouse.c | 4 ++--
>> chardev/spice.c | 2 +-
>> chardev/wctablet.c| 4 ++--
>> hw/bt/hci-csr.c | 2 +-
>> include/chardev/char-fd.h | 2 +-
>> include/chardev/char.h| 2 +-
>> ui/console.c | 6 +++---
>> 14 files changed, 25 insertions(+), 24 deletions(-)
>>
>> diff --git a/chardev/baum.c b/chardev/baum.c
>> index 78b0c87625..1d69d62158 100644
>> --- a/chardev/baum.c
>> +++ b/chardev/baum.c
>> @@ -265,7 +265,7 @@ static int baum_deferred_init(BaumChardev *baum)
>> static void baum_chr_accept_input(struct Chardev *chr)
>> {
>> BaumChardev *baum = BAUM_CHARDEV(chr);
>> -int room, first;
>> +size_t room, first;
>>
>> if (!baum->out_buf_used)
>> return;
>> @@ -292,7 +292,7 @@ static void baum_write_packet(BaumChardev *baum, const
>> uint8_t *buf, int len)
>> {
>> Chardev *chr = CHARDEV(baum);
>> uint8_t io_buf[1 + 2 * len], *cur = io_buf;
>> -int room;
>> +size_t room;
>> *cur++ = ESC;
>> while (len--)
>> if ((*cur++ = *buf++) == ESC)
>> @@ -303,7 +303,7 @@ static void baum_write_packet(BaumChardev *baum, const
>> uint8_t *buf, int len)
>> /* Fits */
>> qemu_chr_be_write(chr, io_buf, len);
>> } else {
>> -int first;
>> +size_t first;
>> uint8_t out;
>> /* Can't fit all, send what can be, and store the rest. */
>> qemu_chr_be_write(chr, io_buf, room);
>
> baum room & first are only used for non-negative capacity values. ack
>
>> diff --git a/chardev/char-fd.c b/chardev/char-fd.c
>> index 2421d8e216..0fe2822869 100644
>> --- a/chardev/char-fd.c
>> +++ b/chardev/char-fd.c
>> @@ -43,7 +43,7 @@ static gboolean fd_chr_read(QIOChannel *chan, GIOCondition
>> cond, void *opaque)
>> {
>> Chardev *chr = CHARDEV(opaque);
>> FDChardev *s = FD_CHARDEV(opaque);
>> -int len;
>> +size_t len;
>> uint8_t buf[CHR_READ_BUF_LEN];
>> ssize_t ret;
>>
>
> fd len is only used for non-negative buffer size. ack
>
>> diff --git a/chardev/char-pty.c b/chardev/char-pty.c
>> index f6ddef..eae25f043b 100644
>> --- a/chardev/char-pty.c
>> +++ b/chardev/char-pty.c
>> @@ -34,7 +34,7 @@
>> typedef struct {
>> Chardev parent;
>> QIOChannel *ioc;
>> -int read_bytes;
>> +size_t read_bytes;
>>
>
> Only set with values returned from qemu_chr_be_can_write(), ack
>
>> int connected;
>> GSource *timer_src;
>> @@ -132,7 +132,7 @@ static gboolean pty_chr_read(QIOChannel *chan,
>> GIOCondition cond, void *opaque)
>> {
>> Chardev *chr = CHARDEV(opaque);
>> PtyChardev *s = PTY_CHARDEV(opaque);
>> -gsize len;
>> +size_t len;
>> uint8_t buf[CHR_READ_BUF_LEN];
>> ssize_t ret;
>
> pty len is only used for non-negative buffer size. ack
>
>> diff --git a/chardev/char-socket.c b/chardev/char-socket.c
>> index 262a59b64f..4010c343e0 100644
>> --- a/chardev/char-socket.c
>> +++ b/chardev/char-socket.c
>> @@ -60,7 +60,7 @@ typedef struct {
>> GSource *hup_source;
>> QCryptoTLSCreds *tls_creds;
>> TCPChardevState state;
>> -int max_size;
>> +size_t max_size;
>
> Only set with values returned from qemu_chr_be_can_write(), ack
>
>> int do_telnetopt;
>> int do_nodelay;
>> int *read_msgfds;
>> @@ -493,10 +493,11 @@ static gboolean tcp_chr_read(QIOChannel *chan,
>> GIOCondition cond, void *opaque)
>> Chardev *chr = CHARDEV(opaque);
>> SocketChardev *s = SOCKET_CHARDEV(opaque);
>> uint8_t buf[CHR_READ_BUF_LEN];
>> -int len, size;
>> +size_t len;
>
> len is only used for non-negative buffer size. ack
>
>> +
Re: [Xen-devel] [PATCH v3 17/25] net/filter-mirror: Use size_t
Hi
On Wed, Feb 20, 2019 at 2:07 AM Philippe Mathieu-Daudé
wrote:
>
> Since iov_size() returns a size_t, no need to use a signed type.
And it is the variable only purpose.
>
> Signed-off-by: Philippe Mathieu-Daudé
Reviewed-by: Marc-André Lureau
> ---
> net/filter-mirror.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/net/filter-mirror.c b/net/filter-mirror.c
> index 3a61cf21e8..97b52d0544 100644
> --- a/net/filter-mirror.c
> +++ b/net/filter-mirror.c
> @@ -48,7 +48,7 @@ static int filter_send(MirrorState *s,
> {
> NetFilterState *nf = NETFILTER(s);
> int ret = 0;
> -ssize_t size = 0;
> +size_t size = 0;
> uint32_t len = 0;
> char *buf;
>
> --
> 2.20.1
>
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 16/25] tpm: Use size_t to hold sizes
On Wed, Feb 20, 2019 at 2:06 AM Philippe Mathieu-Daudé
wrote:
>
> Avoid to use a signed type to hold an unsigned value.
>
> Signed-off-by: Philippe Mathieu-Daudé
Reviewed-by: Marc-André Lureau
> ---
> hw/tpm/tpm_emulator.c | 7 ---
> 1 file changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/hw/tpm/tpm_emulator.c b/hw/tpm/tpm_emulator.c
> index 70f4b10284..931e56f6ed 100644
> --- a/hw/tpm/tpm_emulator.c
> +++ b/hw/tpm/tpm_emulator.c
> @@ -87,17 +87,18 @@ static int tpm_emulator_ctrlcmd(TPMEmulator *tpm,
> unsigned long cmd, void *msg,
> {
> CharBackend *dev = >ctrl_chr;
> uint32_t cmd_no = cpu_to_be32(cmd);
> -ssize_t n = sizeof(uint32_t) + msg_len_in;
> +size_t sz = sizeof(uint32_t) + msg_len_in;
> +ssize_t n;
> uint8_t *buf = NULL;
> int ret = -1;
>
> qemu_mutex_lock(>mutex);
>
> -buf = g_alloca(n);
> +buf = g_alloca(sz);
> memcpy(buf, _no, sizeof(cmd_no));
> memcpy(buf + sizeof(cmd_no), msg, msg_len_in);
>
> -n = qemu_chr_fe_write_all(dev, buf, n);
> +n = qemu_chr_fe_write_all(dev, buf, sz);
> if (n <= 0) {
> goto end;
> }
> --
> 2.20.1
>
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 14/25] virtio-serial: Let VirtIOSerialPortClass::have_data() use size_t
On Wed, Feb 20, 2019 at 2:06 AM Philippe Mathieu-Daudé
wrote:
>
> Both callers in hw/char/virtio-serial-bus.c provide unsigned values,
> even the trace event display an unsigned value.
> Convert the have_data() handler to take an unsigned value.
>
> Signed-off-by: Philippe Mathieu-Daudé
Reviewed-by: Marc-André Lureau
> ---
> It is funny/scary that there are big comments about how to treat
> errors to set the return value, then the return value is simply
> ignored by the caller.
have_data() return value? yes, the code may have change and the
comment doesn't seem to reflect accurately what's going on.
> ---
> hw/char/virtio-console.c | 2 +-
> include/hw/virtio/virtio-serial.h | 2 +-
> 2 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/hw/char/virtio-console.c b/hw/char/virtio-console.c
> index 2cbe1d4ed5..19639dca3b 100644
> --- a/hw/char/virtio-console.c
> +++ b/hw/char/virtio-console.c
> @@ -45,7 +45,7 @@ static gboolean chr_write_unblocked(GIOChannel *chan,
> GIOCondition cond,
>
> /* Callback function that's called when the guest sends us data */
> static ssize_t flush_buf(VirtIOSerialPort *port,
> - const uint8_t *buf, ssize_t len)
> + const uint8_t *buf, size_t len)
> {
> VirtConsole *vcon = VIRTIO_CONSOLE(port);
> ssize_t ret;
> diff --git a/include/hw/virtio/virtio-serial.h
> b/include/hw/virtio/virtio-serial.h
> index 12657a9f39..f1a5ccf4f7 100644
> --- a/include/hw/virtio/virtio-serial.h
> +++ b/include/hw/virtio/virtio-serial.h
> @@ -81,7 +81,7 @@ typedef struct VirtIOSerialPortClass {
> * 'len'. In this case, throttling will be enabled for this port.
> */
> ssize_t (*have_data)(VirtIOSerialPort *port, const uint8_t *buf,
> - ssize_t len);
> + size_t len);
> } VirtIOSerialPortClass;
>
> /*
> --
> 2.20.1
>
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 12/25] xen: Let buffer_append() return the size consumed
Hi
On Wed, Feb 20, 2019 at 2:06 AM Philippe Mathieu-Daudé
wrote:
>
> The buffer.size and buffer.consumed fields are only updated within the
> buffer_append() body. We can simply let buffer_append() return the
> difference (the buffer consumed).
>
> Signed-off-by: Philippe Mathieu-Daudé
This is weird, why not introduce a buffer_size() function instead?
> ---
> hw/char/xen_console.c | 13 -
> 1 file changed, 8 insertions(+), 5 deletions(-)
>
> diff --git a/hw/char/xen_console.c b/hw/char/xen_console.c
> index 083b2c8e2a..1a30014a11 100644
> --- a/hw/char/xen_console.c
> +++ b/hw/char/xen_console.c
> @@ -48,7 +48,7 @@ struct XenConsole {
> int backlog;
> };
>
> -static void buffer_append(struct XenConsole *con)
> +static ssize_t buffer_append(struct XenConsole *con)
> {
> struct buffer *buffer = >buffer;
> XENCONS_RING_IDX cons, prod, size;
> @@ -59,8 +59,9 @@ static void buffer_append(struct XenConsole *con)
> xen_mb();
>
> size = prod - cons;
> -if ((size == 0) || (size > sizeof(intf->out)))
> -return;
> +if ((size == 0) || (size > sizeof(intf->out))) {
> +goto out;
> +}
>
> if ((buffer->capacity - buffer->size) < size) {
> buffer->capacity += (size + 1024);
> @@ -89,6 +90,9 @@ static void buffer_append(struct XenConsole *con)
> if (buffer->consumed > buffer->max_capacity - over)
> buffer->consumed = buffer->max_capacity - over;
> }
> +
> + out:
> +return buffer->size - buffer->consumed;
> }
>
> static void buffer_advance(struct buffer *buffer, size_t len)
> @@ -281,8 +285,7 @@ static void con_event(struct XenLegacyDevice *xendev)
> struct XenConsole *con = container_of(xendev, struct XenConsole, xendev);
> ssize_t size;
>
> -buffer_append(con);
> -size = con->buffer.size - con->buffer.consumed;
> +size = buffer_append(con);
> if (size) {
> xencons_send(con, size);
> }
> --
> 2.20.1
>
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 02/25] chardev: Assert IOCanReadHandler can not be negative
On 2/20/19 11:03 AM, Marc-André Lureau wrote:
> Hi
>
> On Wed, Feb 20, 2019 at 2:03 AM Philippe Mathieu-Daudé
> wrote:
>>
>> The backend should not return a negative length to read.
>> We will later change the prototype of IOCanReadHandler to return an
>> unsigned length. Meanwhile make sure the return length is positive.
>>
>> Suggested-by: Paolo Bonzini
>> Signed-off-by: Philippe Mathieu-Daudé
>
> In such patch, you should do extensive review of existing callbacks,
> or find a convincing argument that this can't break.
Argh I missed that.
> The problem is there are a lot of can_read callbacks, and it's not
> trivial. The *first* of git-grep is rng_egd_chr_can_read()
>
> 57 QSIMPLEQ_FOREACH(req, >parent.requests, next) {
> 58 size += req->size - req->offset;
> 59 }
> 60
> 61 return size;
>
> Clearly not obvious if it returns >= 0.
>
> Another approach is to look at the caller and the return value
> handling. If none handle negative values (or would have wrong
> behaviour with negative values), the assert() is perhaps justified, as
> it could prevent from doing more harm.
I'll go and audit all of them.
Thanks for the review!
Phil.
>> ---
>> chardev/char.c | 5 -
>> 1 file changed, 4 insertions(+), 1 deletion(-)
>>
>> diff --git a/chardev/char.c b/chardev/char.c
>> index f6d61fa5f8..71ecd32b25 100644
>> --- a/chardev/char.c
>> +++ b/chardev/char.c
>> @@ -159,12 +159,15 @@ int qemu_chr_write(Chardev *s, const uint8_t *buf, int
>> len, bool write_all)
>> int qemu_chr_be_can_write(Chardev *s)
>> {
>> CharBackend *be = s->be;
>> +int receivable_bytes;
>>
>> if (!be || !be->chr_can_read) {
>> return 0;
>> }
>>
>> -return be->chr_can_read(be->opaque);
>> +receivable_bytes = be->chr_can_read(be->opaque);
>> +assert(receivable_bytes >= 0);
>> +return receivable_bytes;
>> }
>>
>> void qemu_chr_be_write_impl(Chardev *s, uint8_t *buf, int len)
>> --
>> 2.20.1
>>
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 11/25] xen: Let xencons_send() take a 'size' argument
On Wed, Feb 20, 2019 at 2:05 AM Philippe Mathieu-Daudé
wrote:
>
> The single caller of xencons_send(), con_event() already use the
> difference 'con->buffer.size - con->buffer.consumed'.
> Deduplicate by passing the difference as an argument.
>
> Signed-off-by: Philippe Mathieu-Daudé
Reviewed-by: Marc-André Lureau
> ---
> hw/char/xen_console.c | 12 +++-
> 1 file changed, 7 insertions(+), 5 deletions(-)
>
> diff --git a/hw/char/xen_console.c b/hw/char/xen_console.c
> index 91f34ef06c..083b2c8e2a 100644
> --- a/hw/char/xen_console.c
> +++ b/hw/char/xen_console.c
> @@ -144,11 +144,10 @@ static void xencons_receive(void *opaque, const uint8_t
> *buf, int len)
> xen_pv_send_notify(>xendev);
> }
>
> -static void xencons_send(struct XenConsole *con)
> +static void xencons_send(struct XenConsole *con, ssize_t size)
> {
> -ssize_t len, size;
> +ssize_t len;
>
> -size = con->buffer.size - con->buffer.consumed;
> if (qemu_chr_fe_backend_connected(>chr)) {
> len = qemu_chr_fe_write(>chr,
> con->buffer.data + con->buffer.consumed,
> @@ -280,10 +279,13 @@ static void con_disconnect(struct XenLegacyDevice
> *xendev)
> static void con_event(struct XenLegacyDevice *xendev)
> {
> struct XenConsole *con = container_of(xendev, struct XenConsole, xendev);
> +ssize_t size;
>
> buffer_append(con);
> -if (con->buffer.size - con->buffer.consumed)
> -xencons_send(con);
> +size = con->buffer.size - con->buffer.consumed;
> +if (size) {
> +xencons_send(con, size);
> +}
> }
>
> /* */
> --
> 2.20.1
>
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 09/25] vhost-user: Express sizeof with size_t
On Wed, Feb 20, 2019 at 2:05 AM Philippe Mathieu-Daudé
wrote:
>
> VHOST_USER_HDR_SIZE uses offsetof(), thus is an expression of type
> size_t. Update the format string accordingly.
>
> Signed-off-by: Philippe Mathieu-Daudé
Reviewed-by: Marc-André Lureau
> ---
> hw/virtio/vhost-user.c | 14 --
> 1 file changed, 8 insertions(+), 6 deletions(-)
>
> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> index 564a31d12c..2eb7143d3d 100644
> --- a/hw/virtio/vhost-user.c
> +++ b/hw/virtio/vhost-user.c
> @@ -215,11 +215,12 @@ static int vhost_user_read(struct vhost_dev *dev,
> VhostUserMsg *msg)
> struct vhost_user *u = dev->opaque;
> CharBackend *chr = u->user->chr;
> uint8_t *p = (uint8_t *) msg;
> -int r, size = VHOST_USER_HDR_SIZE;
> +int r;
> +size_t size = VHOST_USER_HDR_SIZE;
>
> r = qemu_chr_fe_read_all(chr, p, size);
> if (r != size) {
> -error_report("Failed to read msg header. Read %d instead of %d."
> +error_report("Failed to read msg header. Read %d instead of %zu."
> " Original request %d.", r, size, msg->hdr.request);
> goto fail;
> }
> @@ -235,7 +236,7 @@ static int vhost_user_read(struct vhost_dev *dev,
> VhostUserMsg *msg)
> /* validate message size is sane */
> if (msg->hdr.size > VHOST_USER_PAYLOAD_SIZE) {
> error_report("Failed to read msg header."
> -" Size %d exceeds the maximum %zu.", msg->hdr.size,
> +" Size %u exceeds the maximum %zu.", msg->hdr.size,
> VHOST_USER_PAYLOAD_SIZE);
> goto fail;
> }
> @@ -246,7 +247,7 @@ static int vhost_user_read(struct vhost_dev *dev,
> VhostUserMsg *msg)
> r = qemu_chr_fe_read_all(chr, p, size);
> if (r != size) {
> error_report("Failed to read msg payload."
> - " Read %d instead of %d.", r, msg->hdr.size);
> + " Read %d instead of %u.", r, msg->hdr.size);
> goto fail;
> }
> }
> @@ -300,7 +301,8 @@ static int vhost_user_write(struct vhost_dev *dev,
> VhostUserMsg *msg,
> {
> struct vhost_user *u = dev->opaque;
> CharBackend *chr = u->user->chr;
> -int ret, size = VHOST_USER_HDR_SIZE + msg->hdr.size;
> +int ret;
> +size_t size = VHOST_USER_HDR_SIZE + msg->hdr.size;
>
> /*
> * For non-vring specific requests, like VHOST_USER_SET_MEM_TABLE,
> @@ -320,7 +322,7 @@ static int vhost_user_write(struct vhost_dev *dev,
> VhostUserMsg *msg,
> ret = qemu_chr_fe_write_all(chr, (const uint8_t *) msg, size);
> if (ret != size) {
> error_report("Failed to write msg."
> - " Wrote %d instead of %d.", ret, size);
> + " Wrote %d instead of %zu.", ret, size);
> return -1;
> }
>
> --
> 2.20.1
>
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 07/25] gdbstub: Let put_buffer() use size_t
On Wed, Feb 20, 2019 at 2:04 AM Philippe Mathieu-Daudé
wrote:
>
> All callers provide a size_t argument, we can safely use size_t
> for this function.
>
> Signed-off-by: Philippe Mathieu-Daudé
Reviewed-by: Marc-André Lureau
> ---
> gdbstub.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/gdbstub.c b/gdbstub.c
> index 69340d7cd1..860e9bb7c7 100644
> --- a/gdbstub.c
> +++ b/gdbstub.c
> @@ -475,10 +475,10 @@ static int gdb_continue_partial(GDBState *s, char
> *newstates)
> return res;
> }
>
> -static void put_buffer(GDBState *s, const uint8_t *buf, int len)
> +static void put_buffer(GDBState *s, const uint8_t *buf, size_t len)
> {
> #ifdef CONFIG_USER_ONLY
> -int ret;
> +ssize_t ret;
>
> while (len > 0) {
> ret = send(s->fd, buf, len, 0);
> --
> 2.20.1
>
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 06/25] gdbstub: Use size_t to hold GDBState::last_packet_len
On Wed, Feb 20, 2019 at 2:04 AM Philippe Mathieu-Daudé
wrote:
>
> In put_packet_binary() we have:
>
> uint8_t *p;
> for(;;) {
> p = s->last_packet;
> *(p++) = ...
> s->last_packet_len = p - s->last_packet;
> put_buffer(s, (uint8_t *)s->last_packet, s->last_packet_len);
>
> The 'p' pointer start at s->last_packet, then is only incremented.
> Since we have "p >= s->last_packet", we are sure than
> "p - s->last_packet >= 0", thus "p - s->last_packet" is positive.
>
> The few other places where s->last_packet_len is set is with constant
> positive values.
>
> It makes sense to use size_t to hold last_packet_len values.
>
> Signed-off-by: Philippe Mathieu-Daudé
Reviewed-by: Marc-André Lureau
> ---
> gdbstub.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/gdbstub.c b/gdbstub.c
> index 76eca3bb7e..69340d7cd1 100644
> --- a/gdbstub.c
> +++ b/gdbstub.c
> @@ -323,7 +323,7 @@ typedef struct GDBState {
> int line_sum; /* running checksum */
> int line_csum; /* checksum at the end of the packet */
> uint8_t last_packet[MAX_PACKET_LENGTH + 4];
> -int last_packet_len;
> +size_t last_packet_len;
> int signal;
> #ifdef CONFIG_USER_ONLY
> int fd;
> --
> 2.20.1
>
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 05/25] gdbstub: Use size_t for strlen() return value
On Wed, Feb 20, 2019 at 2:04 AM Philippe Mathieu-Daudé
wrote:
>
> Since strlen() returns an unsigned value, it is pointless to
> convert it to a signed one. Use size_t to hold its return value.
>
> Signed-off-by: Philippe Mathieu-Daudé
Reviewed-by: Marc-André Lureau
(it looks like the variable is hiding len from outer scope btw)
> ---
> gdbstub.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/gdbstub.c b/gdbstub.c
> index bc774ae992..76eca3bb7e 100644
> --- a/gdbstub.c
> +++ b/gdbstub.c
> @@ -1693,7 +1693,7 @@ static int gdb_handle_packet(GDBState *s, const char
> *line_buf)
> }
> #else /* !CONFIG_USER_ONLY */
> else if (strncmp(p, "Rcmd,", 5) == 0) {
> -int len = strlen(p + 5);
> +size_t len = strlen(p + 5);
>
> if ((len % 2) != 0) {
> put_packet(s, "E01");
> --
> 2.20.1
>
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 00/25] chardev: Convert qemu_chr_write() to take a size_t argument
On Wed, 20 Feb 2019 11:53:42 +0100 Marc-André Lureau wrote: > Hi > > On Wed, Feb 20, 2019 at 2:02 AM Philippe Mathieu-Daudé > wrote: > > > > Hi, > > > > This series convert the chardev::qemu_chr_write() to take unsigned > > length argument. To do so I went through all caller and checked if > > there are no negative value possible. > > > Changing signedness is problematic and can easily introduce bugs that > are easy to miss during review. > > I agree with Cornelia about idiomatic use of int. Changing "int" for > "size_t" isn't systematically a clear win. > > Even Google C++ style recommends to avoid unsigned types "(except for > representing bitfields or modular arithmetic). Do not use an unsigned > type merely to assert that a variable is non-negative." > https://google.github.io/styleguide/cppguide.html#Integer_Types - see > rationale > > Since Paolo you suggested the change, could you give some convincing > arguments that it's worth taking the plunge? FWIW, using an explicitly unsigned type for a length sounds fine; but not all conversions are really convincing (albeit not wrong). ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 00/25] chardev: Convert qemu_chr_write() to take a size_t argument
Hi On Wed, Feb 20, 2019 at 2:02 AM Philippe Mathieu-Daudé wrote: > > Hi, > > This series convert the chardev::qemu_chr_write() to take unsigned > length argument. To do so I went through all caller and checked if > there are no negative value possible. Changing signedness is problematic and can easily introduce bugs that are easy to miss during review. I agree with Cornelia about idiomatic use of int. Changing "int" for "size_t" isn't systematically a clear win. Even Google C++ style recommends to avoid unsigned types "(except for representing bitfields or modular arithmetic). Do not use an unsigned type merely to assert that a variable is non-negative." https://google.github.io/styleguide/cppguide.html#Integer_Types - see rationale Since Paolo you suggested the change, could you give some convincing arguments that it's worth taking the plunge? thanks > > I'm having headaches with the Xen backend, talking with Marc-André > he suggested I ask help to the Xen maintainers. > > Since the series is becoming quite big, I splitted it: > - part 1: convert qemu_chr_write() > - part 2: convert IOReadHandler > > part 1 can be reviewed and merged without part 2. > > The git diffstat is not huge, but there are various chardev subsystems > so many maintainers to ask for Ack. > > v2: https://lists.gnu.org/archive/html/qemu-devel/2018-10/msg02396.html > v1: https://lists.gnu.org/archive/html/qemu-devel/2018-10/msg02200.html > (from Prasad J Pandit) > Changes requested by Paolo: > https://lists.gnu.org/archive/html/qemu-devel/2018-10/msg02294.html > > Please review, > > Phil. > > Philippe Mathieu-Daudé (25): > chardev: Simplify IOWatchPoll::fd_can_read as a GSourceFunc > chardev: Assert IOCanReadHandler can not be negative > chardev/wctablet: Use unsigned type to hold unsigned value > chardev: Let qemu_chr_be_can_write() return a size_t types > gdbstub: Use size_t for strlen() return value > gdbstub: Use size_t to hold GDBState::last_packet_len > gdbstub: Let put_buffer() use size_t > ui/gtk: Remove pointless cast > vhost-user: Express sizeof with size_t > usb-redir: Verify usbredirparser_write get called with positive count > xen: Let xencons_send() take a 'size' argument > xen: Let buffer_append() return the size consumed > RFC xen: Let buffer_append() return a size_t > virtio-serial: Let VirtIOSerialPortClass::have_data() use size_t > spapr-vty: Let vty_putchars() use size_t > tpm: Use size_t to hold sizes > net/filter-mirror: Use size_t > s390x/3270: Let insert_IAC_escape_char() use size_t > s390/ebcdic: Use size_t to iterate over arrays > s390x/sclp: Use a const variable to improve readability > s390x/sclp: Use size_t in process_mdb() > s390x/sclp: Let write_console_data() take a size_t > hw/ipmi: Assert outlen > outpos > chardev: Let qemu_chr_fe_write[_all] use size_t type argument > chardev: Let qemu_chr_write[_all] use size_t > > chardev/baum.c| 6 +++--- > chardev/char-fd.c | 6 +++--- > chardev/char-fe.c | 4 ++-- > chardev/char-io.c | 6 +++--- > chardev/char-mux.c| 3 ++- > chardev/char-pty.c| 8 > chardev/char-socket.c | 13 +++-- > chardev/char-udp.c| 8 > chardev/char-win.c| 2 +- > chardev/char.c| 15 +-- > chardev/msmouse.c | 4 ++-- > chardev/spice.c | 2 +- > chardev/trace-events | 2 +- > chardev/wctablet.c| 9 + > gdbstub.c | 8 > hw/bt/hci-csr.c | 2 +- > hw/char/sclpconsole-lm.c | 12 +++- > hw/char/spapr_vty.c | 2 +- > hw/char/terminal3270.c| 7 --- > hw/char/virtio-console.c | 2 +- > hw/char/xen_console.c | 24 +++- > hw/ipmi/ipmi_bmc_extern.c | 3 ++- > hw/tpm/tpm_emulator.c | 7 --- > hw/usb/redirect.c | 6 +- > hw/virtio/vhost-user.c| 14 -- > include/chardev/char-fd.h | 2 +- > include/chardev/char-fe.h | 4 ++-- > include/chardev/char-io.h | 2 +- > include/chardev/char.h| 4 ++-- > include/hw/ppc/spapr_vio.h| 2 +- > include/hw/s390x/ebcdic.h | 8 > include/hw/virtio/virtio-serial.h | 2 +- > include/sysemu/replay.h | 2 +- > net/filter-mirror.c | 2 +- > replay/replay-char.c | 2 +- > stubs/replay.c| 2 +- > ui/console.c | 6 +++--- > ui/gtk.c | 2 +- > 38 files changed, 119 insertions(+), 96 deletions(-) > > -- > 2.20.1 > ___ Xen-devel mailing list Xen-devel@lists.xenproject.org
Re: [Xen-devel] [PATCH v3 21/25] s390x/sclp: Use size_t in process_mdb()
On Wed, 20 Feb 2019 02:02:28 +0100 Philippe Mathieu-Daudé wrote: > Since it is unlikely we have sizeof(mdbo->mto.message) < 0, > we can convert this variable to an unsigned type. > > Signed-off-by: Philippe Mathieu-Daudé > --- > hw/char/sclpconsole-lm.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) Reviewed-by: Cornelia Huck ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 22/25] s390x/sclp: Let write_console_data() take a size_t
On Wed, 20 Feb 2019 02:02:29 +0100 Philippe Mathieu-Daudé wrote: > Since all callers provide an unsigned value, we can safely > use a size_t argument. > > Signed-off-by: Philippe Mathieu-Daudé > --- > hw/char/sclpconsole-lm.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) Reviewed-by: Cornelia Huck ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 20/25] s390x/sclp: Use a const variable to improve readability
On Wed, 20 Feb 2019 02:02:27 +0100 Philippe Mathieu-Daudé wrote: > We will reuse this variable in the next patch. > > Signed-off-by: Philippe Mathieu-Daudé > --- > hw/char/sclpconsole-lm.c | 7 --- > 1 file changed, 4 insertions(+), 3 deletions(-) > > diff --git a/hw/char/sclpconsole-lm.c b/hw/char/sclpconsole-lm.c > index dbc91a1e5b..49543e2c83 100644 > --- a/hw/char/sclpconsole-lm.c > +++ b/hw/char/sclpconsole-lm.c > @@ -210,13 +210,14 @@ static int process_mdb(SCLPEvent *event, MDBO *mdbo) > int rc; > int len; > uint8_t buffer[SIZE_BUFFER]; > - > -len = be16_to_cpu(mdbo->length); > -len -= sizeof(mdbo->length) + sizeof(mdbo->type) > +const size_t hlen = sizeof(mdbo->length) > ++ sizeof(mdbo->type) > + sizeof(mdbo->mto.line_type_flags) > + sizeof(mdbo->mto.alarm_control) > + sizeof(mdbo->mto._reserved); > > +len = be16_to_cpu(mdbo->length); > +len -= hlen; > assert(len <= SIZE_BUFFER); > > /* convert EBCDIC SCLP contents to ASCII console message */ I'd probably merge this with the next patch, though. Reviewed-by: Cornelia Huck ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [Qemu-devel] [PATCH v3 25/25] chardev: Let qemu_chr_write[_all] use size_t
Hi On Wed, Feb 20, 2019 at 11:39 AM Daniel P. Berrangé wrote: > > On Wed, Feb 20, 2019 at 02:02:32AM +0100, Philippe Mathieu-Daudé wrote: > > > diff --git a/include/chardev/char.h b/include/chardev/char.h > > index 0341dd1ba2..2e3b5a15ca 100644 > > --- a/include/chardev/char.h > > +++ b/include/chardev/char.h > > @@ -221,7 +221,7 @@ void qemu_chr_set_feature(Chardev *chr, > >ChardevFeature feature); > > QemuOpts *qemu_chr_parse_compat(const char *label, const char *filename, > > bool permit_mux_mon); > > -int qemu_chr_write(Chardev *s, const uint8_t *buf, int len, bool > > write_all); > > +int qemu_chr_write(Chardev *s, const uint8_t *buf, size_t len, bool > > write_all); > > Seeing this cleanup reminds me that I think we ought to change > the chardev read & write functions to take "void *buf" instead. > as is done for regular libc read/write functions. This would > avoid casts in the callers between char */uint8_t * > > Something to think about for a future cleanup jobsame applies > for the QIOChannel APIs which take a 'char *buf', annoyingly > different from the chardev APIs :-( Both ought to have void *buf I fully agree, and I started that conversion some time ago, and lost it somewhere. That should be easier than changing the sign of return values though! > > > Regards, > Daniel > -- > |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o-https://fstop138.berrange.com :| > |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :| ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 04/25] chardev: Let qemu_chr_be_can_write() return a size_t types
Hi
On Wed, Feb 20, 2019 at 2:04 AM Philippe Mathieu-Daudé
wrote:
>
> In the previous commit we added an assert to be sure than
> qemu_chr_be_can_write() will never return a negative value.
> We can now change its prototype to return a size_t.
> Adapt the backends accordingly.
Each variable you change to an unsigned type, we should check it isn't
used with negative values.
>
> Suggested-by: Paolo Bonzini
> Signed-off-by: Philippe Mathieu-Daudé
> ---
> chardev/baum.c| 6 +++---
> chardev/char-fd.c | 2 +-
> chardev/char-pty.c| 4 ++--
> chardev/char-socket.c | 7 ---
> chardev/char-udp.c| 4 ++--
> chardev/char-win.c| 2 +-
> chardev/char.c| 2 +-
> chardev/msmouse.c | 4 ++--
> chardev/spice.c | 2 +-
> chardev/wctablet.c| 4 ++--
> hw/bt/hci-csr.c | 2 +-
> include/chardev/char-fd.h | 2 +-
> include/chardev/char.h| 2 +-
> ui/console.c | 6 +++---
> 14 files changed, 25 insertions(+), 24 deletions(-)
>
> diff --git a/chardev/baum.c b/chardev/baum.c
> index 78b0c87625..1d69d62158 100644
> --- a/chardev/baum.c
> +++ b/chardev/baum.c
> @@ -265,7 +265,7 @@ static int baum_deferred_init(BaumChardev *baum)
> static void baum_chr_accept_input(struct Chardev *chr)
> {
> BaumChardev *baum = BAUM_CHARDEV(chr);
> -int room, first;
> +size_t room, first;
>
> if (!baum->out_buf_used)
> return;
> @@ -292,7 +292,7 @@ static void baum_write_packet(BaumChardev *baum, const
> uint8_t *buf, int len)
> {
> Chardev *chr = CHARDEV(baum);
> uint8_t io_buf[1 + 2 * len], *cur = io_buf;
> -int room;
> +size_t room;
> *cur++ = ESC;
> while (len--)
> if ((*cur++ = *buf++) == ESC)
> @@ -303,7 +303,7 @@ static void baum_write_packet(BaumChardev *baum, const
> uint8_t *buf, int len)
> /* Fits */
> qemu_chr_be_write(chr, io_buf, len);
> } else {
> -int first;
> +size_t first;
> uint8_t out;
> /* Can't fit all, send what can be, and store the rest. */
> qemu_chr_be_write(chr, io_buf, room);
baum room & first are only used for non-negative capacity values. ack
> diff --git a/chardev/char-fd.c b/chardev/char-fd.c
> index 2421d8e216..0fe2822869 100644
> --- a/chardev/char-fd.c
> +++ b/chardev/char-fd.c
> @@ -43,7 +43,7 @@ static gboolean fd_chr_read(QIOChannel *chan, GIOCondition
> cond, void *opaque)
> {
> Chardev *chr = CHARDEV(opaque);
> FDChardev *s = FD_CHARDEV(opaque);
> -int len;
> +size_t len;
> uint8_t buf[CHR_READ_BUF_LEN];
> ssize_t ret;
>
fd len is only used for non-negative buffer size. ack
> diff --git a/chardev/char-pty.c b/chardev/char-pty.c
> index f6ddef..eae25f043b 100644
> --- a/chardev/char-pty.c
> +++ b/chardev/char-pty.c
> @@ -34,7 +34,7 @@
> typedef struct {
> Chardev parent;
> QIOChannel *ioc;
> -int read_bytes;
> +size_t read_bytes;
>
Only set with values returned from qemu_chr_be_can_write(), ack
> int connected;
> GSource *timer_src;
> @@ -132,7 +132,7 @@ static gboolean pty_chr_read(QIOChannel *chan,
> GIOCondition cond, void *opaque)
> {
> Chardev *chr = CHARDEV(opaque);
> PtyChardev *s = PTY_CHARDEV(opaque);
> -gsize len;
> +size_t len;
> uint8_t buf[CHR_READ_BUF_LEN];
> ssize_t ret;
pty len is only used for non-negative buffer size. ack
> diff --git a/chardev/char-socket.c b/chardev/char-socket.c
> index 262a59b64f..4010c343e0 100644
> --- a/chardev/char-socket.c
> +++ b/chardev/char-socket.c
> @@ -60,7 +60,7 @@ typedef struct {
> GSource *hup_source;
> QCryptoTLSCreds *tls_creds;
> TCPChardevState state;
> -int max_size;
> +size_t max_size;
Only set with values returned from qemu_chr_be_can_write(), ack
> int do_telnetopt;
> int do_nodelay;
> int *read_msgfds;
> @@ -493,10 +493,11 @@ static gboolean tcp_chr_read(QIOChannel *chan,
> GIOCondition cond, void *opaque)
> Chardev *chr = CHARDEV(opaque);
> SocketChardev *s = SOCKET_CHARDEV(opaque);
> uint8_t buf[CHR_READ_BUF_LEN];
> -int len, size;
> +size_t len;
len is only used for non-negative buffer size. ack
> +int size;
>
> if ((s->state != TCP_CHARDEV_STATE_CONNECTED) ||
> -s->max_size <= 0) {
> +s->max_size == 0) {
> return TRUE;
> }
> len = sizeof(buf);
> diff --git a/chardev/char-udp.c b/chardev/char-udp.c
> index b6e399e983..d4f40626e4 100644
> --- a/chardev/char-udp.c
> +++ b/chardev/char-udp.c
> @@ -39,7 +39,7 @@ typedef struct {
> uint8_t buf[CHR_READ_BUF_LEN];
> int bufcnt;
> int bufptr;
> -int max_size;
> +size_t max_size;
Only set with values returned from qemu_chr_be_can_write(), ack
> } UdpChardev;
>
> #define UDP_CHARDEV(obj) OBJECT_CHECK(UdpChardev, (obj), TYPE_CHARDEV_UDP)
> @@ -58,7 +58,7 @@ static void udp_chr_flush_buffer(UdpChardev *s)
> Chardev
Re: [Xen-devel] [Qemu-devel] [PATCH v3 25/25] chardev: Let qemu_chr_write[_all] use size_t
On Wed, Feb 20, 2019 at 02:02:32AM +0100, Philippe Mathieu-Daudé wrote: > diff --git a/include/chardev/char.h b/include/chardev/char.h > index 0341dd1ba2..2e3b5a15ca 100644 > --- a/include/chardev/char.h > +++ b/include/chardev/char.h > @@ -221,7 +221,7 @@ void qemu_chr_set_feature(Chardev *chr, >ChardevFeature feature); > QemuOpts *qemu_chr_parse_compat(const char *label, const char *filename, > bool permit_mux_mon); > -int qemu_chr_write(Chardev *s, const uint8_t *buf, int len, bool write_all); > +int qemu_chr_write(Chardev *s, const uint8_t *buf, size_t len, bool > write_all); Seeing this cleanup reminds me that I think we ought to change the chardev read & write functions to take "void *buf" instead. as is done for regular libc read/write functions. This would avoid casts in the callers between char */uint8_t * Something to think about for a future cleanup jobsame applies for the QIOChannel APIs which take a 'char *buf', annoyingly different from the chardev APIs :-( Both ought to have void *buf Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :| ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 03/25] chardev/wctablet: Use unsigned type to hold unsigned value
Hi
On Wed, Feb 20, 2019 at 2:03 AM Philippe Mathieu-Daudé
wrote:
>
> TabletChardev::query is an array of uint8_t.
> Use the same type to hold it (this also silent a -Wsign-conversion
> warning in the trace function).
>
> Signed-off-by: Philippe Mathieu-Daudé
Reviewed-by: Marc-André Lureau
> ---
> chardev/trace-events | 2 +-
> chardev/wctablet.c | 5 +++--
> 2 files changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/chardev/trace-events b/chardev/trace-events
> index d0e5f3bbc1..562bfe70e9 100644
> --- a/chardev/trace-events
> +++ b/chardev/trace-events
> @@ -5,7 +5,7 @@ wct_init(void) ""
> wct_cmd_re(void) ""
> wct_cmd_st(void) ""
> wct_cmd_sp(void) ""
> -wct_cmd_ts(int input) "0x%02x"
> +wct_cmd_ts(uint8_t input) "0x%02x"
> wct_cmd_other(const char *cmd) "%s"
> wct_speed(int speed) "%d"
>
> diff --git a/chardev/wctablet.c b/chardev/wctablet.c
> index 35dbd29a33..cf7a08a363 100644
> --- a/chardev/wctablet.c
> +++ b/chardev/wctablet.c
> @@ -207,7 +207,8 @@ static int wctablet_chr_write(struct Chardev *chr,
>const uint8_t *buf, int len)
> {
> TabletChardev *tablet = WCTABLET_CHARDEV(chr);
> -unsigned int i, clen;
> +size_t i;
> +unsigned int clen;
> char *pos;
>
> if (tablet->line_speed != 9600) {
> @@ -269,7 +270,7 @@ static int wctablet_chr_write(struct Chardev *chr,
>
> } else if (strncmp((char *)tablet->query, "TS", 2) == 0 &&
> clen == 3) {
> -unsigned int input = tablet->query[2];
> +uint8_t input = tablet->query[2];
> uint8_t codes[7] = {
> 0xa3,
> ((input & 0x80) == 0) ? 0x7e : 0x7f,
> --
> 2.20.1
>
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [xen-unstable-coverity test] 133324: all pass - PUSHED
flight 133324 xen-unstable-coverity real [real] http://logs.test-lab.xenproject.org/osstest/logs/133324/ Perfect :-) All tests in this flight passed as required version targeted for testing: xen 1bcd0b43a16b7a48ec9afce3887c6c841b687abb baseline version: xen 365aabb6e5023cee476adf81106729efd49c644f Last test of basis 133286 2019-02-17 09:18:47 Z3 days Testing same since 133324 2019-02-20 09:18:56 Z0 days1 attempts People who touched revisions under test: George Dunlap Jan Beulich Razvan Cojocaru Roger Pau Monne Roger Pau Monné jobs: coverity-amd64 pass sg-report-flight on osstest.test-lab.xenproject.org logs: /home/logs/logs images: /home/logs/images Logs, config files, etc. are available at http://logs.test-lab.xenproject.org/osstest/logs Explanation of these reports, and of osstest in general, is at http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master Test harness code can be found at http://xenbits.xen.org/gitweb?p=osstest.git;a=summary Pushing revision : To xenbits.xen.org:/home/xen/git/xen.git 365aabb6e5..1bcd0b43a1 1bcd0b43a16b7a48ec9afce3887c6c841b687abb -> coverity-tested/smoke ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
