Re: [PATCH 3/4 V3] Remap the device table of IOMMU in encrypted manner for kdump

2018-06-20 Thread lijiang
在 2018年06月21日 00:42, Tom Lendacky 写道:
> On 6/16/2018 3:27 AM, Lianbo Jiang wrote:
>> In kdump mode, it will copy the device table of IOMMU from the old
>> device table, which is encrypted when SME is enabled in the first
>> kernel. So we must remap it in encrypted manner in order to be
>> automatically decrypted when we read.
>>
>> Signed-off-by: Lianbo Jiang 
>> ---
>> Some changes:
>> 1. add some comments
>> 2. clean compile warning.
>>
>>  drivers/iommu/amd_iommu_init.c | 15 ++-
>>  1 file changed, 14 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/iommu/amd_iommu_init.c b/drivers/iommu/amd_iommu_init.c
>> index 904c575..a20af4c 100644
>> --- a/drivers/iommu/amd_iommu_init.c
>> +++ b/drivers/iommu/amd_iommu_init.c
>> @@ -889,11 +889,24 @@ static bool copy_device_table(void)
>>  }
>>  
>>  old_devtb_phys = entry & PAGE_MASK;
>> +
>> +/*
>> + *  When sme enable in the first kernel, old_devtb_phys includes the
>> + *  memory encryption mask(sme_me_mask), we must remove the memory
>> + *  encryption mask to obtain the true physical address in kdump mode.
>> + */
>> +if (mem_encrypt_active() && is_kdump_kernel())
>> +old_devtb_phys = __sme_clr(old_devtb_phys);
>> +
> 
> You can probably just use "if (is_kdump_kernel())" here, since memory
> encryption is either on in both the first and second kernel or off in
> both the first and second kernel.  At which point __sme_clr() will do
> the proper thing.
> 
> Actually, this needs to be done no matter what.  When doing either the
> ioremap_encrypted() or the memremap(), the physical address should not
> include the encryption bit/mask.
> 
> Thanks,
> Tom
> 
Thanks for your comments. If we don't remove the memory encryption mask, it will
return false because the 'old_devtb_phys >= 0x1ULL' may become true.

Lianbo
>>  if (old_devtb_phys >= 0x1ULL) {
>>  pr_err("The address of old device table is above 4G, not 
>> trustworthy!\n");
>>  return false;
>>  }
>> -old_devtb = memremap(old_devtb_phys, dev_table_size, MEMREMAP_WB);
>> +old_devtb = (mem_encrypt_active() && is_kdump_kernel())
>> +? (__force void *)ioremap_encrypted(old_devtb_phys,
>> +dev_table_size)
>> +: memremap(old_devtb_phys, dev_table_size, MEMREMAP_WB);> +
>>  if (!old_devtb)
>>  return false;
>>  
>>
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH 2/4 V3] Allocate pages for kdump without encryption when SME is enabled

2018-06-20 Thread lijiang
在 2018年06月21日 09:53, Baoquan He 写道:
> On 06/16/18 at 04:27pm, Lianbo Jiang wrote:
>> When SME is enabled in the first kernel, we will allocate pages
>> for kdump without encryption in order to be able to boot the
>> second kernel in the same manner as kexec, which helps to keep
>> the same code style.
>>
>> Signed-off-by: Lianbo Jiang 
>> ---
>>  kernel/kexec_core.c | 12 
>>  1 file changed, 12 insertions(+)
>>
>> diff --git a/kernel/kexec_core.c b/kernel/kexec_core.c
>> index 20fef1a..3c22a9b 100644
>> --- a/kernel/kexec_core.c
>> +++ b/kernel/kexec_core.c
>> @@ -471,6 +471,16 @@ static struct page 
>> *kimage_alloc_crash_control_pages(struct kimage *image,
>>  }
>>  }
>>  
>> +if (pages) {
>> +unsigned int count, i;
>> +
>> +pages->mapping = NULL;
>> +set_page_private(pages, order);
>> +count = 1 << order;
>> +for (i = 0; i < count; i++)
>> +SetPageReserved(pages + i);
> 
> I guess you might imitate the kexec case, however kexec get pages from
> buddy. Crash pages are reserved in memblock, these codes might make no sense.
> 
Thanks for your comments.
We have changed the attribute of pages, so the original attribute of pages will 
be
restored when they free.

Thanks.
Lianbo
>> +arch_kexec_post_alloc_pages(page_address(pages), 1 << order, 0);
>> +}
>>  return pages;
>>  }
>>  
>> @@ -865,6 +875,7 @@ static int kimage_load_crash_segment(struct kimage 
>> *image,
>>  result  = -ENOMEM;
>>  goto out;
>>  }
>> +arch_kexec_post_alloc_pages(page_address(page), 1, 0);
>>  ptr = kmap(page);
>>  ptr += maddr & ~PAGE_MASK;
>>  mchunk = min_t(size_t, mbytes,
>> @@ -882,6 +893,7 @@ static int kimage_load_crash_segment(struct kimage 
>> *image,
>>  result = copy_from_user(ptr, buf, uchunk);
>>  kexec_flush_icache_page(page);
>>  kunmap(page);
>> +arch_kexec_pre_free_pages(page_address(page), 1);
>>  if (result) {
>>  result = -EFAULT;
>>  goto out;
>> -- 
>> 2.9.5
>>
>>
>> ___
>> kexec mailing list
>> ke...@lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/kexec
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

RE: [PATCH 0/7 v5] Support for fsl-mc bus and its devices in SMMU

2018-06-20 Thread Nipun Gupta
Hi Robin/Greg k-h,

Will this patch-set be taken for the next kernel release (and via which tree)?

Thanks,
Nipun

> -Original Message-
> From: Nipun Gupta
> Sent: Sunday, May 20, 2018 7:20 PM
> To: robin.mur...@arm.com; will.dea...@arm.com; robh...@kernel.org;
> r...@kernel.org; mark.rutl...@arm.com; catalin.mari...@arm.com;
> gre...@linuxfoundation.org; Laurentiu Tudor ;
> bhelg...@google.com
> Cc: h...@lst.de; j...@8bytes.org; m.szyprow...@samsung.com;
> shawn...@kernel.org; frowand.l...@gmail.com; iommu@lists.linux-
> foundation.org; linux-ker...@vger.kernel.org; devicet...@vger.kernel.org;
> linux-arm-ker...@lists.infradead.org; linuxppc-...@lists.ozlabs.org; linux-
> p...@vger.kernel.org; Bharat Bhushan ;
> stuyo...@gmail.com; Leo Li ; Nipun Gupta
> 
> Subject: [PATCH 0/7 v5] Support for fsl-mc bus and its devices in SMMU
> 
> This patchset defines IOMMU DT binding for fsl-mc bus and adds
> support in SMMU for fsl-mc bus.
> 
> The patch series is based on top of dma-mapping tree (for-next branch):
> http://git.infradead.org/users/hch/dma-mapping.git
> 
> These patches
>   - Define property 'iommu-map' for fsl-mc bus (patch 1)
>   - Integrates the fsl-mc bus with the SMMU using this
> IOMMU binding (patch 2,3,4)
>   - Adds the dma configuration support for fsl-mc bus (patch 5, 6)
>   - Updates the fsl-mc device node with iommu/dma related changes (patch 7)
> 
> Changes in v2:
>   - use iommu-map property for fsl-mc bus
>   - rebase over patchset https://patchwork.kernel.org/patch/10317337/
> and make corresponding changes for dma configuration of devices on
> fsl-mc bus
> 
> Changes in v3:
>   - move of_map_rid in drivers/of/address.c
> 
> Changes in v4:
>   - move of_map_rid in drivers/of/base.c
> 
> Changes in v5:
>   - break patch 5 in two separate patches (now patch 5/7 and patch 6/7)
>   - add changelog text in patch 3/7 and patch 5/7
>   - typo fix
> 
> Nipun Gupta (7):
>   Docs: dt: add fsl-mc iommu-map device-tree binding
>   iommu: of: make of_pci_map_rid() available for other devices too
>   iommu: support iommu configuration for fsl-mc devices
>   iommu: arm-smmu: Add support for the fsl-mc bus
>   bus: fsl-mc: support dma configure for devices on fsl-mc bus
>   bus: fsl-mc: set coherent dma mask for devices on fsl-mc bus
>   arm64: dts: ls208xa: comply with the iommu map binding for fsl_mc
> 
>  .../devicetree/bindings/misc/fsl,qoriq-mc.txt  |  39 
>  arch/arm64/boot/dts/freescale/fsl-ls208xa.dtsi |   6 +-
>  drivers/bus/fsl-mc/fsl-mc-bus.c|  16 +++-
>  drivers/iommu/arm-smmu.c   |   7 ++
>  drivers/iommu/iommu.c  |  21 +
>  drivers/iommu/of_iommu.c   |  25 -
>  drivers/of/base.c  | 102 
> +
>  drivers/of/irq.c   |   5 +-
>  drivers/pci/of.c   | 101 
>  include/linux/fsl/mc.h |   8 ++
>  include/linux/iommu.h  |   2 +
>  include/linux/of.h |  11 +++
>  include/linux/of_pci.h |  10 --
>  13 files changed, 231 insertions(+), 122 deletions(-)
> 
> --
> 1.9.1

___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu


Re: [PATCH 0/4 V3] Support kdump for AMD secure memory encryption(SME)

2018-06-20 Thread lijiang
在 2018年06月21日 09:21, Baoquan He 写道:
> On 06/16/18 at 04:27pm, Lianbo Jiang wrote:
>> It is convenient to remap the old memory encrypted to the second kernel by
>> calling ioremap_encrypted().
>>
>> When sme enabled on AMD server, we also need to support kdump. Because
>> the memory is encrypted in the first kernel, we will remap the old memory
>> encrypted to the second kernel(crash kernel), and sme is also enabled in
>> the second kernel, otherwise the old memory encrypted can not be decrypted.
>> Because simply changing the value of a C-bit on a page will not
>> automatically encrypt the existing contents of a page, and any data in the
>> page prior to the C-bit modification will become unintelligible. A page of
>> memory that is marked encrypted will be automatically decrypted when read
>> from DRAM and will be automatically encrypted when written to DRAM.
>>
>> For the kdump, it is necessary to distinguish whether the memory is
>> encrypted. Furthermore, we should also know which part of the memory is
>> encrypted or decrypted. We will appropriately remap the memory according
>> to the specific situation in order to tell cpu how to deal with the
>> data(encrypted or decrypted). For example, when sme enabled, if the old
>> memory is encrypted, we will remap the old memory in encrypted way, which
>> will automatically decrypt the old memory encrypted when we read those data
>> from the remapping address.
>>
>>  --
>> | first-kernel | second-kernel | kdump support |
>> |  (mem_encrypt=on|off)|   (yes|no)|
>> |--+---+---|
>> | on   | on| yes   |
>> | off  | off   | yes   |
>> | on   | off   | no|
> 
> 
>> | off  | on| no|
> 
> It's not clear to me here. If 1st kernel sme is off, in 2nd kernel, when
> you remap the old memory with non-sme mode, why did it fail?
> 
Thank you, Baoquan.
For kdump, there are two cases that doesn't need to support:

1. SME on(first kernel), but SME off(second kernel).
Because the old memory is encrypted, we can't decrypt the old memory if SME is 
off
in the second kernel(in kdump mode).

2. SME off(first kernel), but SME on(second kernel)
Maybe this situation doesn't have significance in actual deployment, 
furthermore, it
will also increase the complexity of the code. It's just for testing, maybe it 
is
unnecessary to support it, because the old memory is unencrypted.

Thanks.
Lianbo
> And please run scripts/get_maintainer.pl and add maintainers of
> component which is affected in patch to CC list.
Great! I forgot CC maintainers, thanks for your reminder.

Lianbo
> 
>> |__|___|___|
>>
>> This patch is only for SME kdump, it is not support SEV kdump.
>>
>> Test tools:
>> makedumpfile[v1.6.3]: https://github.com/LianboJ/makedumpfile
>> commit e1de103eca8f (A draft for kdump vmcore about AMD SME)
>> Author: Lianbo Jiang 
>> Date:   Mon May 14 17:02:40 2018 +0800
>> Note: This patch can only dump vmcore in the case of SME enabled.
>>
>> crash-7.2.1: https://github.com/crash-utility/crash.git
>> commit 1e1bd9c4c1be (Fix for the "bpf" command display on Linux 4.17-rc1)
>> Author: Dave Anderson 
>> Date:   Fri May 11 15:54:32 2018 -0400
>>
>> Test environment:
>> HP ProLiant DL385Gen10 AMD EPYC 7251
>> 8-Core Processor
>> 32768 MB memory
>> 600 GB disk space
>>
>> Linux 4.17-rc7:
>> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
>> commit b04e217704b7 ("Linux 4.17-rc7")
>> Author: Linus Torvalds 
>> Date:   Sun May 27 13:01:47 2018 -0700
>>
>> Reference:
>> AMD64 Architecture Programmer's Manual
>> https://support.amd.com/TechDocs/24593.pdf
>>
>> Some changes:
>> 1. remove the sme_active() check in __ioremap_caller().
>> 2. remove the '#ifdef' stuff throughout this patch.
>> 3. put some logic into the early_memremap_pgprot_adjust() and clean the
>> previous unnecessary changes, for example: arch/x86/include/asm/dmi.h,
>> arch/x86/kernel/acpi/boot.c, drivers/acpi/tables.c.
>> 4. add a new file and modify Makefile.
>> 5. clean compile warning in copy_device_table() and some compile error.
>> 6. split the original patch into four patches, it will be better for
>> review.
>>
>> Some known issues:
>> 1. about SME
>> Upstream kernel doesn't work when we use kexec in the follow command. The
>> system will hang.
>> (This issue doesn't matter with the kdump patch.)
>>
>> Reproduce steps:
>>  # kexec -l /boot/vmlinuz-4.17.0-rc7+ 
>> --initrd=/boot/initramfs-4.17.0-rc7+.img 
>> --command-line="root=/dev/mapper/rhel_hp--dl385g10--03-root ro 
>> mem_encrypt=on rd.lvm.lv=rhel_hp-dl385g10-03/root 
>> rd.lvm.lv=rhel_hp-dl385g10-03/swap console=ttyS0,115200n81 LANG=en_US.UTF-8 
>> earlyprintk=serial debug nokaslr"
>>  # kexec -e (or reboot)
>>
>> The system will hang:
>> [ 1248.932239] kexec_core: Starting new kernel
>> early console in 

Re: [PATCH 4/4 V3] Help to dump the old memory encrypted into vmcore file

2018-06-20 Thread Baoquan He
On 06/16/18 at 04:27pm, Lianbo Jiang wrote:
> In kdump mode, we need to dump the old memory into vmcore file,
> if SME is enabled in the first kernel, we must remap the old
> memory in encrypted manner, which will be automatically decrypted
> when we read from DRAM. It helps to parse the vmcore for some tools.

It implies 2nd kernel is sme enabled, right?

Then what if 1st kernel has no sme enabled?

> 
> Signed-off-by: Lianbo Jiang 
> ---
> Some changes:
> 1. add a new file and modify Makefile.
> 2. remove some code in sev_active().
> 
>  arch/x86/kernel/Makefile |  1 +
>  arch/x86/kernel/crash_dump_encrypt.c | 53 
> 
>  fs/proc/vmcore.c | 20 ++
>  include/linux/crash_dump.h   | 11 
>  4 files changed, 79 insertions(+), 6 deletions(-)
>  create mode 100644 arch/x86/kernel/crash_dump_encrypt.c
> 
> diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
> index 02d6f5c..afb5bad 100644
> --- a/arch/x86/kernel/Makefile
> +++ b/arch/x86/kernel/Makefile
> @@ -96,6 +96,7 @@ obj-$(CONFIG_KEXEC_CORE)+= machine_kexec_$(BITS).o
>  obj-$(CONFIG_KEXEC_CORE) += relocate_kernel_$(BITS).o crash.o
>  obj-$(CONFIG_KEXEC_FILE) += kexec-bzimage64.o
>  obj-$(CONFIG_CRASH_DUMP) += crash_dump_$(BITS).o
> +obj-$(CONFIG_AMD_MEM_ENCRYPT)+= crash_dump_encrypt.o
>  obj-y+= kprobes/
>  obj-$(CONFIG_MODULES)+= module.o
>  obj-$(CONFIG_DOUBLEFAULT)+= doublefault.o
> diff --git a/arch/x86/kernel/crash_dump_encrypt.c 
> b/arch/x86/kernel/crash_dump_encrypt.c
> new file mode 100644
> index 000..e44ef33
> --- /dev/null
> +++ b/arch/x86/kernel/crash_dump_encrypt.c
> @@ -0,0 +1,53 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + *   Memory preserving reboot related code.
> + *
> + *   Created by: Lianbo Jiang (liji...@redhat.com)
> + *   Copyright (C) RedHat Corporation, 2018. All rights reserved
> + */
> +
> +#include 
> +#include 
> +#include 
> +#include 
> +
> +/**
> + * copy_oldmem_page_encrypted - copy one page from "oldmem encrypted"
> + * @pfn: page frame number to be copied
> + * @buf: target memory address for the copy; this can be in kernel address
> + *   space or user address space (see @userbuf)
> + * @csize: number of bytes to copy
> + * @offset: offset in bytes into the page (based on pfn) to begin the copy
> + * @userbuf: if set, @buf is in user address space, use copy_to_user(),
> + *   otherwise @buf is in kernel address space, use memcpy().
> + *
> + * Copy a page from "oldmem encrypted". For this page, there is no pte
> + * mapped in the current kernel. We stitch up a pte, similar to
> + * kmap_atomic.
> + */
> +
> +ssize_t copy_oldmem_page_encrypted(unsigned long pfn, char *buf,
> + size_t csize, unsigned long offset, int userbuf)
> +{
> + void  *vaddr;
> +
> + if (!csize)
> + return 0;
> +
> + vaddr = (__force void *)ioremap_encrypted(pfn << PAGE_SHIFT,
> +   PAGE_SIZE);
> + if (!vaddr)
> + return -ENOMEM;
> +
> + if (userbuf) {
> + if (copy_to_user((void __user *)buf, vaddr + offset, csize)) {
> + iounmap((void __iomem *)vaddr);
> + return -EFAULT;
> + }
> + } else
> + memcpy(buf, vaddr + offset, csize);
> +
> + set_iounmap_nonlazy();
> + iounmap((void __iomem *)vaddr);
> + return csize;
> +}
> diff --git a/fs/proc/vmcore.c b/fs/proc/vmcore.c
> index a45f0af..5200266 100644
> --- a/fs/proc/vmcore.c
> +++ b/fs/proc/vmcore.c
> @@ -25,6 +25,8 @@
>  #include 
>  #include 
>  #include "internal.h"
> +#include 
> +#include 
>  
>  /* List representing chunks of contiguous memory areas and their offsets in
>   * vmcore file.
> @@ -86,7 +88,8 @@ static int pfn_is_ram(unsigned long pfn)
>  
>  /* Reads a page from the oldmem device from given offset. */
>  static ssize_t read_from_oldmem(char *buf, size_t count,
> - u64 *ppos, int userbuf)
> + u64 *ppos, int userbuf,
> + bool encrypted)
>  {
>   unsigned long pfn, offset;
>   size_t nr_bytes;
> @@ -108,8 +111,11 @@ static ssize_t read_from_oldmem(char *buf, size_t count,
>   if (pfn_is_ram(pfn) == 0)
>   memset(buf, 0, nr_bytes);
>   else {
> - tmp = copy_oldmem_page(pfn, buf, nr_bytes,
> - offset, userbuf);
> + tmp = encrypted ? copy_oldmem_page_encrypted(pfn,
> + buf, nr_bytes, offset, userbuf)
> + : copy_oldmem_page(pfn, buf, nr_bytes,
> +offset, userbuf);
> +
>   if (tmp < 0)
>   return tmp;
>  

Re: [PATCH 2/4 V3] Allocate pages for kdump without encryption when SME is enabled

2018-06-20 Thread Baoquan He
On 06/16/18 at 04:27pm, Lianbo Jiang wrote:
> When SME is enabled in the first kernel, we will allocate pages
> for kdump without encryption in order to be able to boot the
> second kernel in the same manner as kexec, which helps to keep
> the same code style.
> 
> Signed-off-by: Lianbo Jiang 
> ---
>  kernel/kexec_core.c | 12 
>  1 file changed, 12 insertions(+)
> 
> diff --git a/kernel/kexec_core.c b/kernel/kexec_core.c
> index 20fef1a..3c22a9b 100644
> --- a/kernel/kexec_core.c
> +++ b/kernel/kexec_core.c
> @@ -471,6 +471,16 @@ static struct page 
> *kimage_alloc_crash_control_pages(struct kimage *image,
>   }
>   }
>  
> + if (pages) {
> + unsigned int count, i;
> +
> + pages->mapping = NULL;
> + set_page_private(pages, order);
> + count = 1 << order;
> + for (i = 0; i < count; i++)
> + SetPageReserved(pages + i);

I guess you might imitate the kexec case, however kexec get pages from
buddy. Crash pages are reserved in memblock, these codes might make no sense.

> + arch_kexec_post_alloc_pages(page_address(pages), 1 << order, 0);
> + }
>   return pages;
>  }
>  
> @@ -865,6 +875,7 @@ static int kimage_load_crash_segment(struct kimage *image,
>   result  = -ENOMEM;
>   goto out;
>   }
> + arch_kexec_post_alloc_pages(page_address(page), 1, 0);
>   ptr = kmap(page);
>   ptr += maddr & ~PAGE_MASK;
>   mchunk = min_t(size_t, mbytes,
> @@ -882,6 +893,7 @@ static int kimage_load_crash_segment(struct kimage *image,
>   result = copy_from_user(ptr, buf, uchunk);
>   kexec_flush_icache_page(page);
>   kunmap(page);
> + arch_kexec_pre_free_pages(page_address(page), 1);
>   if (result) {
>   result = -EFAULT;
>   goto out;
> -- 
> 2.9.5
> 
> 
> ___
> kexec mailing list
> ke...@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu


Re: [PATCH 0/4 V3] Support kdump for AMD secure memory encryption(SME)

2018-06-20 Thread Baoquan He
On 06/16/18 at 04:27pm, Lianbo Jiang wrote:
> It is convenient to remap the old memory encrypted to the second kernel by
> calling ioremap_encrypted().
> 
> When sme enabled on AMD server, we also need to support kdump. Because
> the memory is encrypted in the first kernel, we will remap the old memory
> encrypted to the second kernel(crash kernel), and sme is also enabled in
> the second kernel, otherwise the old memory encrypted can not be decrypted.
> Because simply changing the value of a C-bit on a page will not
> automatically encrypt the existing contents of a page, and any data in the
> page prior to the C-bit modification will become unintelligible. A page of
> memory that is marked encrypted will be automatically decrypted when read
> from DRAM and will be automatically encrypted when written to DRAM.
> 
> For the kdump, it is necessary to distinguish whether the memory is
> encrypted. Furthermore, we should also know which part of the memory is
> encrypted or decrypted. We will appropriately remap the memory according
> to the specific situation in order to tell cpu how to deal with the
> data(encrypted or decrypted). For example, when sme enabled, if the old
> memory is encrypted, we will remap the old memory in encrypted way, which
> will automatically decrypt the old memory encrypted when we read those data
> from the remapping address.
> 
>  --
> | first-kernel | second-kernel | kdump support |
> |  (mem_encrypt=on|off)|   (yes|no)|
> |--+---+---|
> | on   | on| yes   |
> | off  | off   | yes   |
> | on   | off   | no|


> | off  | on| no|

It's not clear to me here. If 1st kernel sme is off, in 2nd kernel, when
you remap the old memory with non-sme mode, why did it fail?

And please run scripts/get_maintainer.pl and add maintainers of
component which is affected in patch to CC list.

> |__|___|___|
> 
> This patch is only for SME kdump, it is not support SEV kdump.
> 
> Test tools:
> makedumpfile[v1.6.3]: https://github.com/LianboJ/makedumpfile
> commit e1de103eca8f (A draft for kdump vmcore about AMD SME)
> Author: Lianbo Jiang 
> Date:   Mon May 14 17:02:40 2018 +0800
> Note: This patch can only dump vmcore in the case of SME enabled.
> 
> crash-7.2.1: https://github.com/crash-utility/crash.git
> commit 1e1bd9c4c1be (Fix for the "bpf" command display on Linux 4.17-rc1)
> Author: Dave Anderson 
> Date:   Fri May 11 15:54:32 2018 -0400
> 
> Test environment:
> HP ProLiant DL385Gen10 AMD EPYC 7251
> 8-Core Processor
> 32768 MB memory
> 600 GB disk space
> 
> Linux 4.17-rc7:
> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
> commit b04e217704b7 ("Linux 4.17-rc7")
> Author: Linus Torvalds 
> Date:   Sun May 27 13:01:47 2018 -0700
> 
> Reference:
> AMD64 Architecture Programmer's Manual
> https://support.amd.com/TechDocs/24593.pdf
> 
> Some changes:
> 1. remove the sme_active() check in __ioremap_caller().
> 2. remove the '#ifdef' stuff throughout this patch.
> 3. put some logic into the early_memremap_pgprot_adjust() and clean the
> previous unnecessary changes, for example: arch/x86/include/asm/dmi.h,
> arch/x86/kernel/acpi/boot.c, drivers/acpi/tables.c.
> 4. add a new file and modify Makefile.
> 5. clean compile warning in copy_device_table() and some compile error.
> 6. split the original patch into four patches, it will be better for
> review.
> 
> Some known issues:
> 1. about SME
> Upstream kernel doesn't work when we use kexec in the follow command. The
> system will hang.
> (This issue doesn't matter with the kdump patch.)
> 
> Reproduce steps:
>  # kexec -l /boot/vmlinuz-4.17.0-rc7+ 
> --initrd=/boot/initramfs-4.17.0-rc7+.img 
> --command-line="root=/dev/mapper/rhel_hp--dl385g10--03-root ro mem_encrypt=on 
> rd.lvm.lv=rhel_hp-dl385g10-03/root rd.lvm.lv=rhel_hp-dl385g10-03/swap 
> console=ttyS0,115200n81 LANG=en_US.UTF-8 earlyprintk=serial debug nokaslr"
>  # kexec -e (or reboot)
> 
> The system will hang:
> [ 1248.932239] kexec_core: Starting new kernel
> early console in extract_kernel
> input_data: 0x00087e91c3b4
> input_len: 0x0067fcbd
> output: 0x00087d40
> output_len: 0x01b6fa90
> kernel_total_size: 0x01a9d000
> trampoline_32bit: 0x00099000
> 
> Decompressing Linux...
> Parsing ELF...[-here the system will hang]
> 
> 2. about SEV
> Upstream kernel(Host OS) doesn't work in host side, some drivers about
> SEV always go wrong in host side. We can't boot SEV Guest OS to test
> kdump patch. Maybe it is more reasonable to improve SEV in another
> patch. When some drivers can work in host side and it can also boot
> Virtual Machine(SEV Guest OS), it will be suitable to fix SEV for kdump.
> 
> [  369.426131] INFO: task systemd-udevd:865 blocked for more than 120 seconds.
> [  

Re: usb HC busted?

2018-06-20 Thread Sudip Mukherjee
Hi Mathias, Andy,

On Thu, Jun 07, 2018 at 10:40:03AM +0300, Mathias Nyman wrote:
> On 06.06.2018 19:45, Sudip Mukherjee wrote:
> > Hi Andy,
> > 
> > And we meet again. :)
> > 
> > On Wed, Jun 06, 2018 at 06:36:35PM +0300, Andy Shevchenko wrote:
> > > On Wed, 2018-06-06 at 17:12 +0300, Mathias Nyman wrote:
> > > > On 04.06.2018 18:28, Sudip Mukherjee wrote:
> > > > > On Thu, May 24, 2018 at 04:35:34PM +0300, Mathias Nyman wrote:
> > > > > > 
> > > 
> > > > Odd and unlikely, but to me this looks like some issue in allocating
> > > > dma memory
> > > > from pool using dma_pool_zalloc()
> > > > 
> > > > 
> > > > Here's the story:
> > > > Sudip sees usb issues on a Intel Atom based board with 4.14.2 kernel.
> > > > All tracing points to dma_pool_zalloc() returning the same dma address
> > > > block on
> > > > consecutive calls.
> > > > 
> > > > In the failing case dma_pool_zalloc() is called 3 - 6us apart.
> > > > 
> > > > <...>-26362 [002]   1186.756739: xhci_ring_mem_detail: MATTU
> > > > xhci_segment_alloc dma @ 0x2d92b000
> > > > <...>-26362 [002]   1186.756745: xhci_ring_mem_detail: MATTU
> > > > xhci_segment_alloc dma @ 0x2d92b000
> > > > <...>-26362 [002]   1186.756748: xhci_ring_mem_detail: MATTU
> > > > xhci_segment_alloc dma @ 0x2d92b000
> > > > 
> > > > dma_pool_zalloc() is called from xhci_segment_alloc() in
> > > > drivers/usb/host/xhci-mem.c
> > > > see:
> > > > https://elixir.bootlin.com/linux/v4.14.2/source/drivers/usb/host/xhci-
> > > > mem.c#L52
> > > > 
> > > > prints above are custom traces added right after dma_pool_zalloc()
> > > 
> > > For better understanding it would be good to have dma_pool_free() calls
> > > debugged as well.
> > 
> 
> Sudip has a full (394M unpacked) trace at:
> https://drive.google.com/open?id=1h-3r-1lfjg8oblBGkzdRIq8z3ZNgGZx-
> 



> But then it gets stuck, for the whole ring2 dma_pool_zalloc() just returns 
> the same dma address as the last segment for
> ring1:0x2d92b000. Last part of trace snippet is just another ring being freed.

A gentle ping on this. Any idea on what the problem might be and any
possible fix?

--
regards
Sudip
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu


Re: [RFC PATCH 17/23] watchdog/hardlockup/hpet: Convert the timer's interrupt to NMI

2018-06-20 Thread Ricardo Neri
On Tue, Jun 19, 2018 at 05:25:09PM -0700, Randy Dunlap wrote:
> On 06/19/2018 05:15 PM, Ricardo Neri wrote:
> > On Sat, Jun 16, 2018 at 03:24:49PM +0200, Thomas Gleixner wrote:
> >> On Fri, 15 Jun 2018, Ricardo Neri wrote:
> >>> On Fri, Jun 15, 2018 at 11:19:09AM +0200, Thomas Gleixner wrote:
>  On Thu, 14 Jun 2018, Ricardo Neri wrote:
> > Alternatively, there could be a counter that skips reading the HPET 
> > status
> > register (and the detection of hardlockups) for every X NMIs. This would
> > reduce the overall frequency of HPET register reads.
> 
>  Great plan. So if the watchdog is the only NMI (because perf is off) then
>  you delay the watchdog detection by that count.
> >>>
> >>> OK. This was a bad idea. Then, is it acceptable to have an read to an HPET
> >>> register per NMI just to check in the status register if the HPET timer
> >>> caused the NMI?
> >>
> >> The status register is useless in case of MSI. MSI is edge triggered 
> >>
> >> The only register which gives you proper information is the counter
> >> register itself. That adds an massive overhead to each NMI, because the
> >> counter register access is synchronized to the HPET clock with hardware
> >> magic. Plus on larger systems, the HPET access is cross node and even
> >> slower.
> > 
> > It starts to sound that the HPET is too slow to drive the hardlockup 
> > detector.
> > 
> > Would it be possible to envision a variant of this implementation? In this
> > variant, the HPET only targets a single CPU. The actual hardlockup detector
> > is implemented by this single CPU sending interprocessor interrupts to the
> > rest of the CPUs.
> > 
> > In this manner only one CPU has to deal with the slowness of the HPET; the
> > rest of the CPUs don't have to read or write any HPET registers. A sysfs
> > entry could be added to configure which CPU will have to deal with the HPET
> > timer. However, profiling could not be done accurately on such CPU.
> 
> Please forgive my simple question:
> 
> What happens when this one CPU is the one that locks up?

I think that in this particular case this one CPU would check for hardlockups
on itself when it receives the NMI from the HPET timer. It would also issue
NMIs to the other monitored processors.

Thanks and BR,
Ricardo
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu


Re: [PATCH v2 1/2] iommu/vt-d: add definitions for PFSID

2018-06-20 Thread Jacob Pan
Hi Joerg/David,

Any comments on this? I will rebase to 18-rc1.
Thanks,

Jacob

On Thu,  7 Jun 2018 09:56:59 -0700
Jacob Pan  wrote:

> When SRIOV VF device IOTLB is invalidated, we need to provide
> the PF source ID such that IOMMU hardware can gauge the depth
> of invalidation queue which is shared among VFs. This is needed
> when device invalidation throttle (DIT) capability is supported.
> 
> This patch adds bit definitions for checking and tracking PFSID.
> 
> Signed-off-by: Jacob Pan 
> Cc: sta...@vger.kernel.org
> Cc: "Ashok Raj" 
> Cc: "Lu Baolu" 
> ---
>  drivers/iommu/intel-iommu.c | 1 +
>  include/linux/intel-iommu.h | 3 +++
>  2 files changed, 4 insertions(+)
> 
> diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
> index 749d8f2..3d77d61 100644
> --- a/drivers/iommu/intel-iommu.c
> +++ b/drivers/iommu/intel-iommu.c
> @@ -422,6 +422,7 @@ struct device_domain_info {
>   struct list_head global; /* link to global list */
>   u8 bus; /* PCI bus number */
>   u8 devfn;   /* PCI devfn number */
> + u16 pfsid;  /* SRIOV physical function source
> ID */ u8 pasid_supported:3;
>   u8 pasid_enabled:1;
>   u8 pri_supported:1;
> diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
> index ef169d6..af1c05f 100644
> --- a/include/linux/intel-iommu.h
> +++ b/include/linux/intel-iommu.h
> @@ -114,6 +114,7 @@
>   * Extended Capability Register
>   */
>  
> +#define ecap_dit(e)  ((e >> 41) & 0x1)
>  #define ecap_pasid(e)((e >> 40) & 0x1)
>  #define ecap_pss(e)  ((e >> 35) & 0x1f)
>  #define ecap_eafs(e) ((e >> 34) & 0x1)
> @@ -284,6 +285,7 @@ enum {
>  #define QI_DEV_IOTLB_SID(sid)((u64)((sid) & 0x) << 32)
>  #define QI_DEV_IOTLB_QDEP(qdep)  (((qdep) & 0x1f) << 16)
>  #define QI_DEV_IOTLB_ADDR(addr)  ((u64)(addr) & VTD_PAGE_MASK)
> +#define QI_DEV_IOTLB_PFSID(pfsid) (((u64)(pfsid & 0xf) << 12) |
> ((u64)(pfsid & 0xfff) << 52)) #define QI_DEV_IOTLB_SIZE   1
>  #define QI_DEV_IOTLB_MAX_INVS32
>  
> @@ -308,6 +310,7 @@ enum {
>  #define QI_DEV_EIOTLB_PASID(p)   (((u64)p) << 32)
>  #define QI_DEV_EIOTLB_SID(sid)   ((u64)((sid) & 0x) << 16)
>  #define QI_DEV_EIOTLB_QDEP(qd)   ((u64)((qd) & 0x1f) << 4)
> +#define QI_DEV_EIOTLB_PFSID(pfsid) (((u64)(pfsid & 0xf) << 12) |
> ((u64)(pfsid & 0xfff) << 52)) #define QI_DEV_EIOTLB_MAX_INVS  32
>  
>  #define QI_PGRP_IDX(idx) (((u64)(idx)) << 55)

[Jacob Pan]
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu


Re: [PATCH 3/4 V3] Remap the device table of IOMMU in encrypted manner for kdump

2018-06-20 Thread Tom Lendacky
On 6/16/2018 3:27 AM, Lianbo Jiang wrote:
> In kdump mode, it will copy the device table of IOMMU from the old
> device table, which is encrypted when SME is enabled in the first
> kernel. So we must remap it in encrypted manner in order to be
> automatically decrypted when we read.
> 
> Signed-off-by: Lianbo Jiang 
> ---
> Some changes:
> 1. add some comments
> 2. clean compile warning.
> 
>  drivers/iommu/amd_iommu_init.c | 15 ++-
>  1 file changed, 14 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/iommu/amd_iommu_init.c b/drivers/iommu/amd_iommu_init.c
> index 904c575..a20af4c 100644
> --- a/drivers/iommu/amd_iommu_init.c
> +++ b/drivers/iommu/amd_iommu_init.c
> @@ -889,11 +889,24 @@ static bool copy_device_table(void)
>   }
>  
>   old_devtb_phys = entry & PAGE_MASK;
> +
> + /*
> +  *  When sme enable in the first kernel, old_devtb_phys includes the
> +  *  memory encryption mask(sme_me_mask), we must remove the memory
> +  *  encryption mask to obtain the true physical address in kdump mode.
> +  */
> + if (mem_encrypt_active() && is_kdump_kernel())
> + old_devtb_phys = __sme_clr(old_devtb_phys);
> +

You can probably just use "if (is_kdump_kernel())" here, since memory
encryption is either on in both the first and second kernel or off in
both the first and second kernel.  At which point __sme_clr() will do
the proper thing.

Actually, this needs to be done no matter what.  When doing either the
ioremap_encrypted() or the memremap(), the physical address should not
include the encryption bit/mask.

Thanks,
Tom

>   if (old_devtb_phys >= 0x1ULL) {
>   pr_err("The address of old device table is above 4G, not 
> trustworthy!\n");
>   return false;
>   }
> - old_devtb = memremap(old_devtb_phys, dev_table_size, MEMREMAP_WB);
> + old_devtb = (mem_encrypt_active() && is_kdump_kernel())
> + ? (__force void *)ioremap_encrypted(old_devtb_phys,
> + dev_table_size)
> + : memremap(old_devtb_phys, dev_table_size, MEMREMAP_WB);> +
>   if (!old_devtb)
>   return false;
>  
> 
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu


Re: [PATCH 1/4 V3] Add a function(ioremap_encrypted) for kdump when AMD sme enabled

2018-06-20 Thread Tom Lendacky
On 6/16/2018 3:27 AM, Lianbo Jiang wrote:
> It is convenient to remap the old memory encrypted to the second
> kernel by calling ioremap_encrypted().
> 
> Signed-off-by: Lianbo Jiang 
> ---
> Some changes:
> 1. remove the sme_active() check in __ioremap_caller().
> 2. put some logic into the early_memremap_pgprot_adjust() for
> early memremap.
> 
>  arch/x86/include/asm/io.h |  3 +++
>  arch/x86/mm/ioremap.c | 28 
>  2 files changed, 23 insertions(+), 8 deletions(-)
> 
> diff --git a/arch/x86/include/asm/io.h b/arch/x86/include/asm/io.h
> index f6e5b93..989d60b 100644
> --- a/arch/x86/include/asm/io.h
> +++ b/arch/x86/include/asm/io.h
> @@ -192,6 +192,9 @@ extern void __iomem *ioremap_cache(resource_size_t 
> offset, unsigned long size);
>  #define ioremap_cache ioremap_cache
>  extern void __iomem *ioremap_prot(resource_size_t offset, unsigned long 
> size, unsigned long prot_val);
>  #define ioremap_prot ioremap_prot
> +extern void __iomem *ioremap_encrypted(resource_size_t phys_addr,
> + unsigned long size);
> +#define ioremap_encrypted ioremap_encrypted
>  
>  /**
>   * ioremap -   map bus memory into CPU space
> diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
> index c63a545..e365fc4 100644
> --- a/arch/x86/mm/ioremap.c
> +++ b/arch/x86/mm/ioremap.c
> @@ -24,6 +24,7 @@
>  #include 
>  #include 
>  #include 
> +#include 
>  
>  #include "physaddr.h"
>  
> @@ -131,7 +132,8 @@ static void __ioremap_check_mem(resource_size_t addr, 
> unsigned long size,
>   * caller shouldn't need to know that small detail.
>   */
>  static void __iomem *__ioremap_caller(resource_size_t phys_addr,
> - unsigned long size, enum page_cache_mode pcm, void *caller)
> + unsigned long size, enum page_cache_mode pcm,
> + void *caller, bool encrypted)
>  {
>   unsigned long offset, vaddr;
>   resource_size_t last_addr;
> @@ -199,7 +201,7 @@ static void __iomem *__ioremap_caller(resource_size_t 
> phys_addr,
>* resulting mapping.
>*/
>   prot = PAGE_KERNEL_IO;
> - if (sev_active() && mem_flags.desc_other)
> + if ((sev_active() && mem_flags.desc_other) || encrypted)
>   prot = pgprot_encrypted(prot);
>  
>   switch (pcm) {
> @@ -291,7 +293,7 @@ void __iomem *ioremap_nocache(resource_size_t phys_addr, 
> unsigned long size)
>   enum page_cache_mode pcm = _PAGE_CACHE_MODE_UC_MINUS;
>  
>   return __ioremap_caller(phys_addr, size, pcm,
> - __builtin_return_address(0));
> + __builtin_return_address(0), false);
>  }
>  EXPORT_SYMBOL(ioremap_nocache);
>  
> @@ -324,7 +326,7 @@ void __iomem *ioremap_uc(resource_size_t phys_addr, 
> unsigned long size)
>   enum page_cache_mode pcm = _PAGE_CACHE_MODE_UC;
>  
>   return __ioremap_caller(phys_addr, size, pcm,
> - __builtin_return_address(0));
> + __builtin_return_address(0), false);
>  }
>  EXPORT_SYMBOL_GPL(ioremap_uc);
>  
> @@ -341,7 +343,7 @@ EXPORT_SYMBOL_GPL(ioremap_uc);
>  void __iomem *ioremap_wc(resource_size_t phys_addr, unsigned long size)
>  {
>   return __ioremap_caller(phys_addr, size, _PAGE_CACHE_MODE_WC,
> - __builtin_return_address(0));
> + __builtin_return_address(0), false);
>  }
>  EXPORT_SYMBOL(ioremap_wc);
>  
> @@ -358,14 +360,21 @@ EXPORT_SYMBOL(ioremap_wc);
>  void __iomem *ioremap_wt(resource_size_t phys_addr, unsigned long size)
>  {
>   return __ioremap_caller(phys_addr, size, _PAGE_CACHE_MODE_WT,
> - __builtin_return_address(0));
> + __builtin_return_address(0), false);
>  }
>  EXPORT_SYMBOL(ioremap_wt);
>  
> +void __iomem *ioremap_encrypted(resource_size_t phys_addr, unsigned long 
> size)
> +{
> + return __ioremap_caller(phys_addr, size, _PAGE_CACHE_MODE_WB,
> + __builtin_return_address(0), true);
> +}
> +EXPORT_SYMBOL(ioremap_encrypted);
> +
>  void __iomem *ioremap_cache(resource_size_t phys_addr, unsigned long size)
>  {
>   return __ioremap_caller(phys_addr, size, _PAGE_CACHE_MODE_WB,
> - __builtin_return_address(0));
> + __builtin_return_address(0), false);
>  }
>  EXPORT_SYMBOL(ioremap_cache);
>  
> @@ -374,7 +383,7 @@ void __iomem *ioremap_prot(resource_size_t phys_addr, 
> unsigned long size,
>  {
>   return __ioremap_caller(phys_addr, size,
>   pgprot2cachemode(__pgprot(prot_val)),
> - __builtin_return_address(0));
> + __builtin_return_address(0), false);
>  }
>  EXPORT_SYMBOL(ioremap_prot);
>  
> @@ -688,6 +697,9 @@ pgprot_t __init 
> early_memremap_pgprot_adjust(resource_size_t phys_addr,
>   if (encrypted_prot && 

Re: [RESEND 0/6] iommu: ipmmu-vmsa: Add more Gen3 SoCs support

2018-06-20 Thread Simon Horman
[CC Alex Williamson]

On Thu, Jun 14, 2018 at 12:48:20PM +0200, Jacopo Mondi wrote:
> Hello,
>this series collects patches sent to mailing lists in late 2017 to add
> support for a number of R-Car Gen3 SoCs to the ipmmu-vmsa driver.
> 
> Part of the series these patches were part of went in through Alex Williamson
> tree and are now included in v4.17.
> 
> These patches have not yet been collected, and need to be rebased on v4.17,
> and this what this series serves for.
> 
> The series:
> - Increses the micro-TLB size to 48bits to accomodate newer SoCs
> - Adds support for M3-N, M3-W, V3-M and E3 to ipmmu-vmsa driver
> - Documents M3-N, V3H and E3 device tree bindings
> 
> All patches have been acked/reviewed by, if nothing big happens they should
> be ready to be collected.

All patches:

Reviewed-by: Simon Horman 

Joerg, Alex,

I'd be most grateful if we could find a way for these patches to progress
into mainline.

> 
> Thanks
>j
> 
> Jacopo Mondi (2):
>   dt-bindings: iommu/ipmmu-vmsa: Add R-Car M3-N (R8A77965)
>   iommu/ipmmu-vmsa: Hook up R8A77965 DT matching code
> 
> Magnus Damm (3):
>   iommu/ipmmu-vmsa: Increase maximum micro-TLBS to 48
>   iommu/ipmmu-vmsa: Hook up r8a7796 DT matching code
>   iommu/ipmmu-vmsa: Document R-Car V3H and E3 IPMMU DT bindings
> 
> Simon Horman (1):
>   iommu/ipmmu-vmsa: Hook up r8a779(70|95) DT matching code
> 
>  .../bindings/iommu/renesas,ipmmu-vmsa.txt  |  3 ++
>  drivers/iommu/ipmmu-vmsa.c | 38 
> ++
>  2 files changed, 34 insertions(+), 7 deletions(-)
> 
> --
> 2.7.4
> 
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu


Re: [RFC PATCH 17/23] watchdog/hardlockup/hpet: Convert the timer's interrupt to NMI

2018-06-20 Thread Thomas Gleixner
On Tue, 19 Jun 2018, Ricardo Neri wrote:
> On Sat, Jun 16, 2018 at 03:24:49PM +0200, Thomas Gleixner wrote:
> > The status register is useless in case of MSI. MSI is edge triggered 
> > 
> > The only register which gives you proper information is the counter
> > register itself. That adds an massive overhead to each NMI, because the
> > counter register access is synchronized to the HPET clock with hardware
> > magic. Plus on larger systems, the HPET access is cross node and even
> > slower.
> 
> It starts to sound that the HPET is too slow to drive the hardlockup detector.
> 
> Would it be possible to envision a variant of this implementation? In this
> variant, the HPET only targets a single CPU. The actual hardlockup detector
> is implemented by this single CPU sending interprocessor interrupts to the
> rest of the CPUs.

And these IPIs must be NMIs which need to have a software based indicator
that the watchdog needs to be checked, which is going to create yet another
can of race conditions and in the worst case 'unknown NMI' splats. Not
pretty either.

Thanks,

tglx
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu


Re: [PATCH 18/25] MIPS: loongson64: use generic dma noncoherent ops

2018-06-20 Thread Christoph Hellwig
On Tue, Jun 19, 2018 at 04:19:25PM -0700, Paul Burton wrote:
> Hi Christoph,
> 
> On Fri, Jun 15, 2018 at 01:08:47PM +0200, Christoph Hellwig wrote:
> > -static inline unsigned long plat_dma_addr_to_phys(struct device *dev,
> > -   dma_addr_t dma_addr)
> > -{
> > -#if defined(CONFIG_CPU_LOONGSON2F) && defined(CONFIG_64BIT)
> > -   return (dma_addr > 0x8fff) ? dma_addr : (dma_addr & 0x0fff);
> > -#else
> > -   return dma_addr & 0x7fff;
> > -#endif
> > -}
> 
> ...
> 
> > +phys_addr_t __dma_to_phys(struct device *dev, dma_addr_t dma_addr)
> > +{
> > +#if defined(CONFIG_CPU_LOONGSON2F) && defined(CONFIG_64BIT)
> > +   if (dma_addr > 0x8fff)
> > +   return dma_addr;
> > +#endif
> > +   return dma_addr & 0x0fff;
> > +}
> 
> Thanks for putting in the work here - I've applied patches 1-17 to
> mips-next so far, but I'm struggling to convince myself that the above
> is correct.
> 
> In the original code we have 3 distinct cases:
> 
>   64b Loongson2F, dma_addr > 0x8fff  -> dma_addr
>   64b Loongson2F, dma_addr <= 0x8fff -> dma_addr & 0x0fff
>   Everything else-> dma_addr & 0x7fff
> 
> In the new __dma_to_phys() though only the first case remains the same.
> 
> Is this intentional or a mixup?

No, this is a mixup.  I hadn't noticed one case was 0x0fff and
the other 0x7fff.

Below is the minimal fixup that keeps things as much as it was before.

---
>From 174213f535c07774cdbbccd42c76164893729cf5 Mon Sep 17 00:00:00 2001
From: Christoph Hellwig 
Date: Wed, 20 Jun 2018 09:11:15 +0200
Subject: MIPS: loongson64: use generic dma noncoherent ops

Provide phys_to_dma/dma_to_phys helpers, everything else is generic.

Signed-off-by: Christoph Hellwig 
---
 arch/mips/Kconfig |  1 +
 .../asm/mach-loongson64/dma-coherence.h   | 69 ---
 arch/mips/loongson64/Kconfig  |  2 -
 arch/mips/loongson64/common/Makefile  |  1 +
 arch/mips/loongson64/common/dma.c | 18 +
 5 files changed, 20 insertions(+), 71 deletions(-)
 delete mode 100644 arch/mips/include/asm/mach-loongson64/dma-coherence.h
 create mode 100644 arch/mips/loongson64/common/dma.c

diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig
index 326bd73bc5bf..e192e484b1b8 100644
--- a/arch/mips/Kconfig
+++ b/arch/mips/Kconfig
@@ -1827,6 +1827,7 @@ config CPU_LOONGSON2
select CPU_SUPPORTS_64BIT_KERNEL
select CPU_SUPPORTS_HIGHMEM
select CPU_SUPPORTS_HUGEPAGES
+   select ARCH_HAS_PHYS_TO_DMA
 
 config CPU_LOONGSON1
bool
diff --git a/arch/mips/include/asm/mach-loongson64/dma-coherence.h 
b/arch/mips/include/asm/mach-loongson64/dma-coherence.h
deleted file mode 100644
index 651dd2eb3ee5..
--- a/arch/mips/include/asm/mach-loongson64/dma-coherence.h
+++ /dev/null
@@ -1,69 +0,0 @@
-/*
- * This file is subject to the terms and conditions of the GNU General Public
- * License.  See the file "COPYING" in the main directory of this archive
- * for more details.
- *
- * Copyright (C) 2006, 07  Ralf Baechle 
- * Copyright (C) 2007 Lemote, Inc. & Institute of Computing Technology
- * Author: Fuxin Zhang, zhan...@lemote.com
- *
- */
-#ifndef __ASM_MACH_LOONGSON64_DMA_COHERENCE_H
-#define __ASM_MACH_LOONGSON64_DMA_COHERENCE_H
-
-#ifdef CONFIG_SWIOTLB
-#include 
-#endif
-
-struct device;
-
-static inline dma_addr_t plat_map_dma_mem(struct device *dev, void *addr,
- size_t size)
-{
-   return virt_to_phys(addr) | 0x8000;
-}
-
-static inline dma_addr_t plat_map_dma_mem_page(struct device *dev,
-  struct page *page)
-{
-   return page_to_phys(page) | 0x8000;
-}
-
-static inline unsigned long plat_dma_addr_to_phys(struct device *dev,
-   dma_addr_t dma_addr)
-{
-#if defined(CONFIG_CPU_LOONGSON2F) && defined(CONFIG_64BIT)
-   return (dma_addr > 0x8fff) ? dma_addr : (dma_addr & 0x0fff);
-#else
-   return dma_addr & 0x7fff;
-#endif
-}
-
-static inline void plat_unmap_dma_mem(struct device *dev, dma_addr_t dma_addr,
-   size_t size, enum dma_data_direction direction)
-{
-}
-
-static inline int plat_dma_supported(struct device *dev, u64 mask)
-{
-   /*
-* we fall back to GFP_DMA when the mask isn't all 1s,
-* so we can't guarantee allocations that must be
-* within a tighter range than GFP_DMA..
-*/
-   if (mask < DMA_BIT_MASK(24))
-   return 0;
-
-   return 1;
-}
-
-static inline int plat_device_is_coherent(struct device *dev)
-{
-   return 0;
-}
-
-static inline void plat_post_dma_flush(struct device *dev)
-{
-}
-
-#endif /* __ASM_MACH_LOONGSON64_DMA_COHERENCE_H */
diff --git a/arch/mips/loongson64/Kconfig b/arch/mips/loongson64/Kconfig
index a785bf8da3f3..c865b4b9b775 100644
--- a/arch/mips/loongson64/Kconfig
+++ b/arch/mips/loongson64/Kconfig
@@ -13,7 +13,6 @@ config LEMOTE_FULOONG2E
select CSRC_R4K