Re: [PATCH V8 1/5] crypto: Multi-buffer encryption infrastructure support
On Fri, May 11, 2018 at 01:24:42AM +, Dey, Megha wrote: > > Are you suggesting that the SIMD wrapper, will do what is currently being > done by the ' mcryptd_queue_worker ' function (assuming FPU is not disabled) > i.e dispatching the job to the inner algorithm? > > I have got rid of the mcryptd layer( have an inner layer, outer SIMD layer, > handled the pointers and completions accordingly), but still facing some > issues after removing the per cpu mcryptd_cpu_queue. Why don't you post what you've got and we can work it out together? Thanks, -- Email: Herbert Xu Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
RE: [PATCH V8 1/5] crypto: Multi-buffer encryption infrastructure support
>-Original Message- >From: Herbert Xu [mailto:herb...@gondor.apana.org.au] >Sent: Monday, May 7, 2018 2:35 AM >To: Dey, Megha >Cc: linux-ker...@vger.kernel.org; linux-crypto@vger.kernel.org; >da...@davemloft.net >Subject: Re: [PATCH V8 1/5] crypto: Multi-buffer encryption infrastructure >support > >On Tue, May 01, 2018 at 10:39:15PM +, Dey, Megha wrote: >> >> crypto/simd.c provides a simd_skcipher_create_compat. I have used the >> same template to introduce simd_ahash_create_compat which would wrap >around the inner hash algorithm. >> >> Hence we would still register 2 algs, outer and inner. > >Right. > >> Currently we have outer_alg -> mcryptd alg -> inner_alg >> >> Mcryptd is mainly providing the following: >> 1. Ensuring the lanes(8 in case of AVX2) are full before dispatching to the >lower inner algorithm. This is obviously why we would expect better >performance for multi-buffer as opposed to the present single-buffer >algorithms. >> 2. If there no new incoming jobs, issue a flush. >> 3. A glue layer which sends the correct pointers and completions. >> >> If we get rid of mcryptd, these functions needs to be done by someone. Since >all multi-buffer algorithms would require this tasks, where do you suggest >these >helpers live, if not the current mcryptd.c? > >That's the issue. I don't think mcryptd is doing any of these claimed >functions >except for hosting the flush helper which could really live anywhere. > >All these functions are actually being carried out in the inner algorithm >already. > >> I am not sure if you are suggesting that we need to get rid of the mcryptd >work queue itself. In that case, we would need to execute in the context of the >job requesting the crypto transformation. > >Which is fine as long as you can disable the FPU. If not the simd wrapper will >defer the job to kthread context as required. Hi Herbert, Are you suggesting that the SIMD wrapper, will do what is currently being done by the ' mcryptd_queue_worker ' function (assuming FPU is not disabled) i.e dispatching the job to the inner algorithm? I have got rid of the mcryptd layer( have an inner layer, outer SIMD layer, handled the pointers and completions accordingly), but still facing some issues after removing the per cpu mcryptd_cpu_queue. > >Cheers, >-- >Email: Herbert Xu Home Page: >http://gondor.apana.org.au/~herbert/ >PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
Re: [PATCH v2] async_pq: Remove VLA usage
On Sat, May 5, 2018 at 12:58 AM, Kyle Spiers wrote:
> In the quest to remove VLAs from the kernel[1], this moves the
> allocation of coefs and blocks from the stack to being kmalloc()ed.
>
> [1] https://lkml.org/lkml/2018/3/7/621
>
> Signed-off-by: Kyle Spiers
Reviewed-by: Kees Cook
Is this something that should go via Vinod, Dan, or direct through
Herbert's crypto tree?
Thanks!
-Kees
> ---
> Forgot to add slab.h
> ---
> crypto/async_tx/async_pq.c | 18 ++
> crypto/async_tx/raid6test.c | 9 -
> 2 files changed, 22 insertions(+), 5 deletions(-)
>
> diff --git a/crypto/async_tx/async_pq.c b/crypto/async_tx/async_pq.c
> index 56bd612927ab..af1912313a23 100644
> --- a/crypto/async_tx/async_pq.c
> +++ b/crypto/async_tx/async_pq.c
> @@ -194,9 +194,9 @@ async_gen_syndrome(struct page **blocks, unsigned int
> offset, int disks,
> (src_cnt <= dma_maxpq(device, 0) ||
> dma_maxpq(device, DMA_PREP_CONTINUE) > 0) &&
> is_dma_pq_aligned(device, offset, 0, len)) {
> - struct dma_async_tx_descriptor *tx;
> + struct dma_async_tx_descriptor *tx = NULL;
> enum dma_ctrl_flags dma_flags = 0;
> - unsigned char coefs[src_cnt];
> + unsigned char *coefs;
> int i, j;
>
> /* run the p+q asynchronously */
> @@ -207,6 +207,9 @@ async_gen_syndrome(struct page **blocks, unsigned int
> offset, int disks,
> * sources and update the coefficients accordingly
> */
> unmap->len = len;
> + coefs = kmalloc_array(src_cnt, sizeof(*coefs), GFP_KERNEL);
> + if (!coefs)
> + goto out;
> for (i = 0, j = 0; i < src_cnt; i++) {
> if (blocks[i] == NULL)
> continue;
> @@ -240,7 +243,9 @@ async_gen_syndrome(struct page **blocks, unsigned int
> offset, int disks,
> }
>
> tx = do_async_gen_syndrome(chan, coefs, j, unmap, dma_flags,
> submit);
> +out:
> dmaengine_unmap_put(unmap);
> + kfree(coefs);
> return tx;
> }
>
> @@ -298,8 +303,8 @@ async_syndrome_val(struct page **blocks, unsigned int
> offset, int disks,
> {
> struct dma_chan *chan = pq_val_chan(submit, blocks, disks, len);
> struct dma_device *device = chan ? chan->device : NULL;
> - struct dma_async_tx_descriptor *tx;
> - unsigned char coefs[disks-2];
> + struct dma_async_tx_descriptor *tx = NULL;
> + unsigned char *coefs = NULL;
> enum dma_ctrl_flags dma_flags = submit->cb_fn ? DMA_PREP_INTERRUPT :
> 0;
> struct dmaengine_unmap_data *unmap = NULL;
>
> @@ -318,6 +323,9 @@ async_syndrome_val(struct page **blocks, unsigned int
> offset, int disks,
> __func__, disks, len);
>
> unmap->len = len;
> + coefs = kmalloc_array(disks - 2, sizeof(*coefs), GFP_KERNEL);
> + if (!coefs)
> + goto out;
> for (i = 0; i < disks-2; i++)
> if (likely(blocks[i])) {
> unmap->addr[j] = dma_map_page(dev, blocks[i],
> @@ -423,6 +431,8 @@ async_syndrome_val(struct page **blocks, unsigned int
> offset, int disks,
> async_tx_sync_epilog(submit);
> tx = NULL;
> }
> +out:
> + kfree(coefs);
> dmaengine_unmap_put(unmap);
>
> return tx;
> diff --git a/crypto/async_tx/raid6test.c b/crypto/async_tx/raid6test.c
> index dad95f45b88f..4237a5ae8f42 100644
> --- a/crypto/async_tx/raid6test.c
> +++ b/crypto/async_tx/raid6test.c
> @@ -24,6 +24,7 @@
> #include
> #include
> #include
> +#include
>
> #undef pr
> #define pr(fmt, args...) pr_info("raid6test: " fmt, ##args)
> @@ -81,11 +82,16 @@ static void raid6_dual_recov(int disks, size_t bytes, int
> faila, int failb, stru
> init_async_submit(&submit, 0, NULL, NULL, NULL,
> addr_conv);
> tx = async_gen_syndrome(ptrs, 0, disks, bytes,
> &submit);
> } else {
> - struct page *blocks[disks];
> + struct page **blocks;
> struct page *dest;
> int count = 0;
> int i;
>
> + blocks = kmalloc_array(disks, sizeof(*blocks),
> + GFP_KERNEL);
> + if (!blocks)
> + return;
> +
> /* data+Q failure. Reconstruct data from P,
> * then rebuild syndrome
> */
> @@ -101,6 +107,7 @@ static void raid6_dual_recov(int disks, size_t bytes, int
> faila, int failb, stru
>
> init_async_submit(&submit, 0,
Re: [PATCH RESEND 1/2] Add DOWNLOAD_FIRMWARE SEV command
Use a prefix for the subject pls:
Subject: [PATCH RESEND 1/2] crypto: ccp: Add DOWNLOAD_FIRMWARE SEV command
or
Subject: [PATCH RESEND 1/2] crypto/ccp: Add DOWNLOAD_FIRMWARE SEV command
or so.
On Wed, May 09, 2018 at 11:18:27AM -0500, Janakarajan Natarajan wrote:
> The DOWNLOAD_FIRMWARE command, added as of SEV API v0.15, allows the OS
> to install SEV firmware newer than the currently active SEV firmware.
>
> For the new SEV firmware to be applied it must:
> * Pass the validation test performed by the existing firmware.
> * Be of the same build or a newer build compared to the existing firmware.
>
> For more information please refer to "Section 5.11 DOWNLOAD_FIRMWARE" of
> https://support.amd.com/TechDocs/55766_SEV-KM%20API_Specification.pdf
>
> Signed-off-by: Janakarajan Natarajan
> ---
> drivers/crypto/ccp/psp-dev.c | 96
> +++-
> drivers/crypto/ccp/psp-dev.h | 4 ++
> include/linux/psp-sev.h | 12 ++
> 3 files changed, 102 insertions(+), 10 deletions(-)
>
> diff --git a/drivers/crypto/ccp/psp-dev.c b/drivers/crypto/ccp/psp-dev.c
> index d95ec52..1c78a2e 100644
> --- a/drivers/crypto/ccp/psp-dev.c
> +++ b/drivers/crypto/ccp/psp-dev.c
> @@ -22,11 +22,17 @@
> #include
> #include
> #include
> +#include
>
> #include "sp-dev.h"
> #include "psp-dev.h"
>
> +#define SEV_VERSION_CHECK(_maj, _min)\
> + ((psp_master->api_major) >= _maj && \
> + (psp_master->api_minor) >= _min)
> +
> #define DEVICE_NAME "sev"
> +#define SEV_FW_FILE "amd-sev.fw"
Can we put this firmware in an amd folder like
amd/sev.fw
or so, like we to with the microcode?
/lib/firmware/amd-ucode/
├── microcode_amd.bin
├── microcode_amd.bin.asc
├── microcode_amd_fam15h.bin
├── microcode_amd_fam15h.bin.asc
├── microcode_amd_fam16h.bin
└── microcode_amd_fam16h.bin.asc
It is tidier this way in the fw directory.
> static DEFINE_MUTEX(sev_cmd_mutex);
> static struct sev_misc_dev *misc_dev;
> @@ -112,6 +118,7 @@ static int sev_cmd_buffer_len(int cmd)
> case SEV_CMD_RECEIVE_UPDATE_DATA: return sizeof(struct
> sev_data_receive_update_data);
> case SEV_CMD_RECEIVE_UPDATE_VMSA: return sizeof(struct
> sev_data_receive_update_vmsa);
> case SEV_CMD_LAUNCH_UPDATE_SECRET: return sizeof(struct
> sev_data_launch_secret);
> + case SEV_CMD_DOWNLOAD_FIRMWARE: return sizeof(struct
> sev_data_download_firmware);
> default:return 0;
> }
>
> @@ -378,6 +385,77 @@ void *psp_copy_user_blob(u64 __user uaddr, u32 len)
> }
> EXPORT_SYMBOL_GPL(psp_copy_user_blob);
>
> +static int get_sev_api_version(void)
sev_get_api_version()
like the rest.
> +{
> + struct sev_user_data_status *status;
> + int error, ret;
> +
> + status = &psp_master->status_cmd_buf;
> + ret = sev_platform_status(status, &error);
> + if (ret) {
> + dev_err(psp_master->dev,
> + "SEV: failed to get status. Error: %#x\n", error);
> + return 1;
> + }
> +
> + psp_master->api_major = status->api_major;
> + psp_master->api_minor = status->api_minor;
> + psp_master->build = status->build;
> +
> + return 0;
> +}
> +
> +/* Don't fail if SEV FW couldn't be updated. Continue with existing SEV FW */
> +static void sev_update_firmware(struct device *dev)
> +{
> + struct sev_data_download_firmware *data;
> + const struct firmware *firmware;
> + int ret, error, order;
> + struct page *p;
> + u64 data_size;
> +
> + ret = request_firmware(&firmware, SEV_FW_FILE, dev);
> + if (ret < 0)
> + return;
> +
> + /*
> + * SEV FW expects the physical address given to it to be 32
> + * byte aligned. Memory allocated has structure placed at the
> + * beginning followed by the firmware being passed to the SEV
> + * FW. Allocate enough memory for data structure + alignment
> + * padding + SEV FW.
> + */
> + data_size = ALIGN(sizeof(struct sev_data_download_firmware), 32);
> +
> + order = get_order(firmware->size + data_size);
> + p = alloc_pages(GFP_KERNEL, order);
> + if (!p)
> + goto fw_err;
> +
> + /*
> + * Copy firmware data to a kernel allocated contiguous
> + * memory region.
> + */
> + data = page_address(p);
> + memcpy(page_address(p) + data_size, firmware->data, firmware->size);
> +
> + data->address = __psp_pa(page_address(p) + data_size);
> + data->len = firmware->size;
> +
> + ret = sev_do_cmd(SEV_CMD_DOWNLOAD_FIRMWARE, data, &error);
> + if (ret) {
> + dev_dbg(dev, "Failed to update SEV firmware: %#x\n", error);
> + } else {
> + dev_info(dev, "SEV firmware update successful\n");
> + get_sev_api_version();
Call that function in the caller of sev_update_firmware() and in its
success path. For that, make sev_update_firmwa
[PATCH 08/18] random: Remove pr_fmt duplicate logging prefixes
Converting pr_fmt from a simple define to use KBUILD_MODNAME added
some duplicate logging prefixes to existing uses.
Remove them.
Signed-off-by: Joe Perches
---
drivers/char/hw_random/via-rng.c | 10 +-
drivers/char/random.c| 16 +++-
2 files changed, 12 insertions(+), 14 deletions(-)
diff --git a/drivers/char/hw_random/via-rng.c b/drivers/char/hw_random/via-rng.c
index ffe9b0c6c647..b9367e055468 100644
--- a/drivers/char/hw_random/via-rng.c
+++ b/drivers/char/hw_random/via-rng.c
@@ -24,6 +24,8 @@
* warranty of any kind, whether express or implied.
*/
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
#include
#include
#include
@@ -137,8 +139,7 @@ static int via_rng_init(struct hwrng *rng)
* register */
if (((c->x86 == 6) && (c->x86_model >= 0x0f)) || (c->x86 > 6)){
if (!boot_cpu_has(X86_FEATURE_XSTORE_EN)) {
- pr_err(PFX "can't enable hardware RNG "
- "if XSTORE is not enabled\n");
+ pr_err("can't enable hardware RNG if XSTORE is not
enabled\n");
return -ENODEV;
}
return 0;
@@ -176,7 +177,7 @@ static int via_rng_init(struct hwrng *rng)
unneeded */
rdmsr(MSR_VIA_RNG, lo, hi);
if ((lo & VIA_RNG_ENABLE) == 0) {
- pr_err(PFX "cannot enable VIA C3 RNG, aborting\n");
+ pr_err("cannot enable VIA C3 RNG, aborting\n");
return -ENODEV;
}
@@ -202,8 +203,7 @@ static int __init mod_init(void)
pr_info("VIA RNG detected\n");
err = hwrng_register(&via_rng);
if (err) {
- pr_err(PFX "RNG registering failed (%d)\n",
- err);
+ pr_err("RNG registering failed (%d)\n", err);
goto out;
}
out:
diff --git a/drivers/char/random.c b/drivers/char/random.c
index cd888d4ee605..d1e35cfcce8e 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -693,7 +693,7 @@ static void credit_entropy_bits(struct entropy_store *r,
int nbits)
}
if (unlikely(entropy_count < 0)) {
- pr_warn("random: negative entropy/overflow: pool %s count %d\n",
+ pr_warn("negative entropy/overflow: pool %s count %d\n",
r->name, entropy_count);
WARN_ON(1);
entropy_count = 0;
@@ -857,7 +857,7 @@ static int crng_fast_load(const char *cp, size_t len)
invalidate_batched_entropy();
crng_init = 1;
wake_up_interruptible(&crng_init_wait);
- pr_notice("random: fast init done\n");
+ pr_notice("fast init done\n");
}
return 1;
}
@@ -942,16 +942,14 @@ static void crng_reseed(struct crng_state *crng, struct
entropy_store *r)
crng_init = 2;
process_random_ready_list();
wake_up_interruptible(&crng_init_wait);
- pr_notice("random: crng init done\n");
+ pr_notice("crng init done\n");
if (unseeded_warning.missed) {
- pr_notice("random: %d get_random_xx warning(s) missed "
- "due to ratelimiting\n",
+ pr_notice("%d get_random_xx warning(s) missed due to
ratelimiting\n",
unseeded_warning.missed);
unseeded_warning.missed = 0;
}
if (urandom_warning.missed) {
- pr_notice("random: %d urandom warning(s) missed "
- "due to ratelimiting\n",
+ pr_notice("%d urandom warning(s) missed due to
ratelimiting\n",
urandom_warning.missed);
urandom_warning.missed = 0;
}
@@ -1380,7 +1378,7 @@ static size_t account(struct entropy_store *r, size_t
nbytes, int min,
ibytes = 0;
if (unlikely(entropy_count < 0)) {
- pr_warn("random: negative entropy count: pool %s count %d\n",
+ pr_warn("negative entropy count: pool %s count %d\n",
r->name, entropy_count);
WARN_ON(1);
entropy_count = 0;
@@ -1596,7 +1594,7 @@ static void _warn_unseeded_randomness(const char
*func_name, void *caller,
print_once = true;
#endif
if (__ratelimit(&unseeded_warning))
- pr_notice("random: %s called from %pS with crng_init=%d\n",
+ pr_notice("%s called from %pS with crng_init=%d\n",
func_name, caller, crng_init);
}
--
2.15.0
[PATCH 00/18] Convert default pr_fmt from empty to KBUILD_MODNAME
pr_ logging uses allow a prefix to be specified with a
specific #define pr_fmt
The default of pr_fmt in printk.h is #define pr_fmt(fmt) fmt
so no prefixing of logging output is generically done.
There are several output logging uses like dump_stack() that are
unprefixed and should remain unprefixed.
This patch series attempts to convert the default #define of pr_fmt to
KBUILD_MODNAME ": " fmt and as well update the various bits of the kernel
that should _not_ be prefixed by adding #define pr_fmt(fmt) fmt to those
compilation units that do not want output message prefixing.
There are about 1200 uses of #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
that could be removed if the default is changed.
A script that does this removal (and removes any blank lines that follow)
for the linux-kernel tree is included below:
$ git grep -P --name-only "define\s+pr_fmt\b" | \
grep -v include/linux/printk.h | \
xargs perl -i -e 'local $/; while (<>) {s/(\n)*[ \t]*#[ \t]*define[
\t]+pr_fmt[ \t]*\([ \t]*(\w+)[ \t]*\)[ \t]*KBUILD_MODNAME[ \t]*\": \"[ \t]*\2[
\t]*\n\s*/\1\n/; s/^\n//; print;}'
This script should be run after this patch series is applied.
The above script output diff is currently:
1198 files changed, 70 insertions(+), 2241 deletions(-)
Joe Perches (18):
kernel: Use pr_fmt
lib: Use pr_fmt
printk: Convert pr_fmt from blank define to KBUILD_MODNAME
x86: Remove pr_fmt duplicate logging prefixes
x86/mtrr: Rename main.c to mtrr.c and remove duplicate prefixes
net: Remove pr_fmt duplicate logging prefixes
blk-mq: Remove pr_fmt duplicate logging prefixes
random: Remove pr_fmt duplicate logging prefixes
ptp: Remove pr_fmt duplicate logging prefixes
efifb: Remove pr_fmt duplicate logging prefixes
proc: Remove pr_fmt duplicate logging prefixes
uprobes: Remove pr_fmt duplicate logging prefixes
printk: Remove pr_fmt duplicate logging prefixes
lib/mpi: Remove pr_fmt duplicate logging prefixes
security: Remove pr_fmt duplicate logging prefixes
aoe: Remove pr_fmt duplicate logging prefixes
security: encrypted-keys: Remove pr_fmt duplicate logging prefixes
rcu: Use pr_fmt to prefix "rcu: " to logging output
arch/x86/events/amd/ibs.c | 2 +-
arch/x86/kernel/cpu/mtrr/Makefile | 2 +-
arch/x86/kernel/cpu/mtrr/{main.c => mtrr.c}| 33 ++---
arch/x86/kernel/e820.c | 32 ++--
arch/x86/kernel/hpet.c | 5 +-
arch/x86/kernel/uprobes.c | 4 +-
arch/x86/mm/numa.c | 22 -
block/blk-mq.c | 9 ++--
drivers/block/aoe/aoeblk.c | 29 ++-
drivers/block/aoe/aoechr.c | 11 ++---
drivers/block/aoe/aoecmd.c | 34 ++---
drivers/block/aoe/aoedev.c | 19 +++-
drivers/block/aoe/aoemain.c| 6 +--
drivers/block/aoe/aoenet.c | 19 +++-
drivers/char/hw_random/via-rng.c | 10 ++--
drivers/char/random.c | 16 +++---
drivers/ptp/ptp_clock.c| 4 +-
drivers/video/fbdev/efifb.c| 48 +-
fs/proc/root.c | 6 +--
include/linux/printk.h | 2 +-
kernel/acct.c | 2 +
kernel/async.c | 14 +++---
kernel/audit_tree.c| 2 +-
kernel/backtracetest.c | 8 +--
kernel/crash_core.c| 29 ++-
kernel/events/uprobes.c| 3 +-
kernel/exit.c | 2 +
kernel/hung_task.c | 13 +++--
kernel/kprobes.c | 20 +---
kernel/module.c| 59 +++
kernel/panic.c | 3 ++
kernel/params.c| 13 +++--
kernel/pid.c | 2 +
kernel/printk/printk.c | 2 +-
kernel/profile.c | 2 +
kernel/range.c | 2 +-
kernel/rcu/rcu_segcblist.c | 2 +
kernel/rcu/rcuperf.c | 10 ++--
kernel/rcu/rcutorture.c| 46 +-
kernel/rcu/srcutiny.c | 2 +
kernel/rcu/srcutree.c | 5 +-
kernel/rcu/tiny.c | 3 ++
kernel/rcu/tree.c | 8 +--
kernel/rcu/tree_plugin.h | 67 +++---
kernel/rcu/update.c| 19 +---
kernel/relay.c | 5 +-
kernel/seccomp.c | 4 +-
kernel/signal
[PATCH -resend 08/27] x86: assembly, annotate aliases
_key_expansion_128 is an alias to _key_expansion_256a, __memcpy to memcpy, xen_syscall32_target to xen_sysenter_target, and so on. Annotate them all using the new SYM_FUNC_START_ALIAS, SYM_FUNC_START_LOCAL_ALIAS, and SYM_FUNC_END_ALIAS. This will make the tools generating the debuginfo happy. Signed-off-by: Jiri Slaby Cc: Herbert Xu Cc: "David S. Miller" Cc: Thomas Gleixner Cc: Ingo Molnar Cc: "H. Peter Anvin" Cc: Cc: Boris Ostrovsky Cc: Juergen Gross Reviewed-by: Juergen Gross [xen parts] Cc: Cc: --- arch/x86/crypto/aesni-intel_asm.S | 5 ++--- arch/x86/lib/memcpy_64.S | 4 ++-- arch/x86/lib/memmove_64.S | 4 ++-- arch/x86/lib/memset_64.S | 4 ++-- arch/x86/xen/xen-asm_64.S | 4 ++-- 5 files changed, 10 insertions(+), 11 deletions(-) diff --git a/arch/x86/crypto/aesni-intel_asm.S b/arch/x86/crypto/aesni-intel_asm.S index b482ac1a1fb3..c85ecb163c78 100644 --- a/arch/x86/crypto/aesni-intel_asm.S +++ b/arch/x86/crypto/aesni-intel_asm.S @@ -1761,8 +1761,7 @@ ENDPROC(aesni_gcm_finalize) #endif -.align 4 -_key_expansion_128: +SYM_FUNC_START_LOCAL_ALIAS(_key_expansion_128) SYM_FUNC_START_LOCAL(_key_expansion_256a) pshufd $0b, %xmm1, %xmm1 shufps $0b0001, %xmm0, %xmm4 @@ -1773,8 +1772,8 @@ SYM_FUNC_START_LOCAL(_key_expansion_256a) movaps %xmm0, (TKEYP) add $0x10, TKEYP ret -ENDPROC(_key_expansion_128) SYM_FUNC_END(_key_expansion_256a) +SYM_FUNC_END_ALIAS(_key_expansion_128) SYM_FUNC_START_LOCAL(_key_expansion_192a) pshufd $0b01010101, %xmm1, %xmm1 diff --git a/arch/x86/lib/memcpy_64.S b/arch/x86/lib/memcpy_64.S index 9a53a06e5a3e..4911b1c61aa8 100644 --- a/arch/x86/lib/memcpy_64.S +++ b/arch/x86/lib/memcpy_64.S @@ -26,7 +26,7 @@ * Output: * rax original destination */ -ENTRY(__memcpy) +SYM_FUNC_START_ALIAS(__memcpy) ENTRY(memcpy) ALTERNATIVE_2 "jmp memcpy_orig", "", X86_FEATURE_REP_GOOD, \ "jmp memcpy_erms", X86_FEATURE_ERMS @@ -40,7 +40,7 @@ ENTRY(memcpy) rep movsb ret ENDPROC(memcpy) -ENDPROC(__memcpy) +SYM_FUNC_END_ALIAS(__memcpy) EXPORT_SYMBOL(memcpy) EXPORT_SYMBOL(__memcpy) diff --git a/arch/x86/lib/memmove_64.S b/arch/x86/lib/memmove_64.S index bbec69d8223b..50c1648311b3 100644 --- a/arch/x86/lib/memmove_64.S +++ b/arch/x86/lib/memmove_64.S @@ -26,7 +26,7 @@ */ .weak memmove -ENTRY(memmove) +SYM_FUNC_START_ALIAS(memmove) ENTRY(__memmove) /* Handle more 32 bytes in loop */ @@ -208,6 +208,6 @@ ENTRY(__memmove) 13: retq ENDPROC(__memmove) -ENDPROC(memmove) +SYM_FUNC_END_ALIAS(memmove) EXPORT_SYMBOL(__memmove) EXPORT_SYMBOL(memmove) diff --git a/arch/x86/lib/memset_64.S b/arch/x86/lib/memset_64.S index 9bc861c71e75..927ac44d34aa 100644 --- a/arch/x86/lib/memset_64.S +++ b/arch/x86/lib/memset_64.S @@ -19,7 +19,7 @@ * * rax original destination */ -ENTRY(memset) +SYM_FUNC_START_ALIAS(memset) ENTRY(__memset) /* * Some CPUs support enhanced REP MOVSB/STOSB feature. It is recommended @@ -43,8 +43,8 @@ ENTRY(__memset) rep stosb movq %r9,%rax ret -ENDPROC(memset) ENDPROC(__memset) +SYM_FUNC_END_ALIAS(memset) EXPORT_SYMBOL(memset) EXPORT_SYMBOL(__memset) diff --git a/arch/x86/xen/xen-asm_64.S b/arch/x86/xen/xen-asm_64.S index 417b339e5c8e..e8f6f482bb20 100644 --- a/arch/x86/xen/xen-asm_64.S +++ b/arch/x86/xen/xen-asm_64.S @@ -164,13 +164,13 @@ ENDPROC(xen_sysenter_target) #else /* !CONFIG_IA32_EMULATION */ -ENTRY(xen_syscall32_target) +SYM_FUNC_START_ALIAS(xen_syscall32_target) ENTRY(xen_sysenter_target) lea 16(%rsp), %rsp /* strip %rcx, %r11 */ mov $-ENOSYS, %rax pushq $0 jmp hypercall_iret -ENDPROC(xen_syscall32_target) ENDPROC(xen_sysenter_target) +SYM_FUNC_END_ALIAS(xen_syscall32_target) #endif /* CONFIG_IA32_EMULATION */ -- 2.16.3
[PATCH -resend 06/27] x86: crypto, annotate local functions
Use the newly added SYM_FUNC_START_LOCAL to annotate starts of all functions which do not have ".globl" annotation, but their ends are annotated by ENDPROC. This is needed to balance ENDPROC for tools that are about to generate debuginfo. To be symmetric, we also convert their ENDPROCs to the new SYM_FUNC_END. Signed-off-by: Jiri Slaby Cc: Herbert Xu Cc: "David S. Miller" Cc: Thomas Gleixner Cc: Ingo Molnar Cc: "H. Peter Anvin" Cc: Cc: --- arch/x86/crypto/aesni-intel_asm.S| 49 arch/x86/crypto/camellia-aesni-avx-asm_64.S | 20 ++-- arch/x86/crypto/camellia-aesni-avx2-asm_64.S | 20 ++-- arch/x86/crypto/cast5-avx-x86_64-asm_64.S| 8 ++--- arch/x86/crypto/cast6-avx-x86_64-asm_64.S| 8 ++--- arch/x86/crypto/ghash-clmulni-intel_asm.S| 4 +-- arch/x86/crypto/serpent-avx-x86_64-asm_64.S | 8 ++--- arch/x86/crypto/serpent-avx2-asm_64.S| 8 ++--- arch/x86/crypto/twofish-avx-x86_64-asm_64.S | 8 ++--- 9 files changed, 62 insertions(+), 71 deletions(-) diff --git a/arch/x86/crypto/aesni-intel_asm.S b/arch/x86/crypto/aesni-intel_asm.S index e762ef417562..b482ac1a1fb3 100644 --- a/arch/x86/crypto/aesni-intel_asm.S +++ b/arch/x86/crypto/aesni-intel_asm.S @@ -1763,7 +1763,7 @@ ENDPROC(aesni_gcm_finalize) .align 4 _key_expansion_128: -_key_expansion_256a: +SYM_FUNC_START_LOCAL(_key_expansion_256a) pshufd $0b, %xmm1, %xmm1 shufps $0b0001, %xmm0, %xmm4 pxor %xmm4, %xmm0 @@ -1774,10 +1774,9 @@ _key_expansion_256a: add $0x10, TKEYP ret ENDPROC(_key_expansion_128) -ENDPROC(_key_expansion_256a) +SYM_FUNC_END(_key_expansion_256a) -.align 4 -_key_expansion_192a: +SYM_FUNC_START_LOCAL(_key_expansion_192a) pshufd $0b01010101, %xmm1, %xmm1 shufps $0b0001, %xmm0, %xmm4 pxor %xmm4, %xmm0 @@ -1799,10 +1798,9 @@ _key_expansion_192a: movaps %xmm1, 0x10(TKEYP) add $0x20, TKEYP ret -ENDPROC(_key_expansion_192a) +SYM_FUNC_END(_key_expansion_192a) -.align 4 -_key_expansion_192b: +SYM_FUNC_START_LOCAL(_key_expansion_192b) pshufd $0b01010101, %xmm1, %xmm1 shufps $0b0001, %xmm0, %xmm4 pxor %xmm4, %xmm0 @@ -1819,10 +1817,9 @@ _key_expansion_192b: movaps %xmm0, (TKEYP) add $0x10, TKEYP ret -ENDPROC(_key_expansion_192b) +SYM_FUNC_END(_key_expansion_192b) -.align 4 -_key_expansion_256b: +SYM_FUNC_START_LOCAL(_key_expansion_256b) pshufd $0b10101010, %xmm1, %xmm1 shufps $0b0001, %xmm2, %xmm4 pxor %xmm4, %xmm2 @@ -1832,7 +1829,7 @@ _key_expansion_256b: movaps %xmm2, (TKEYP) add $0x10, TKEYP ret -ENDPROC(_key_expansion_256b) +SYM_FUNC_END(_key_expansion_256b) /* * int aesni_set_key(struct crypto_aes_ctx *ctx, const u8 *in_key, @@ -1985,8 +1982,7 @@ ENDPROC(aesni_enc) * KEY * TKEYP (T1) */ -.align 4 -_aesni_enc1: +SYM_FUNC_START_LOCAL(_aesni_enc1) movaps (KEYP), KEY # key mov KEYP, TKEYP pxor KEY, STATE # round 0 @@ -2029,7 +2025,7 @@ _aesni_enc1: movaps 0x70(TKEYP), KEY AESENCLAST KEY STATE ret -ENDPROC(_aesni_enc1) +SYM_FUNC_END(_aesni_enc1) /* * _aesni_enc4:internal ABI @@ -2049,8 +2045,7 @@ ENDPROC(_aesni_enc1) * KEY * TKEYP (T1) */ -.align 4 -_aesni_enc4: +SYM_FUNC_START_LOCAL(_aesni_enc4) movaps (KEYP), KEY # key mov KEYP, TKEYP pxor KEY, STATE1# round 0 @@ -2138,7 +2133,7 @@ _aesni_enc4: AESENCLAST KEY STATE3 AESENCLAST KEY STATE4 ret -ENDPROC(_aesni_enc4) +SYM_FUNC_END(_aesni_enc4) /* * void aesni_dec (struct crypto_aes_ctx *ctx, u8 *dst, const u8 *src) @@ -2177,8 +2172,7 @@ ENDPROC(aesni_dec) * KEY * TKEYP (T1) */ -.align 4 -_aesni_dec1: +SYM_FUNC_START_LOCAL(_aesni_dec1) movaps (KEYP), KEY # key mov KEYP, TKEYP pxor KEY, STATE # round 0 @@ -2221,7 +2215,7 @@ _aesni_dec1: movaps 0x70(TKEYP), KEY AESDECLAST KEY STATE ret -ENDPROC(_aesni_dec1) +SYM_FUNC_END(_aesni_dec1) /* * _aesni_dec4:internal ABI @@ -2241,8 +2235,7 @@ ENDPROC(_aesni_dec1) * KEY * TKEYP (T1) */ -.align 4 -_aesni_dec4: +SYM_FUNC_START_LOCAL(_aesni_dec4) movaps (KEYP), KEY # key mov KEYP, TKEYP pxor KEY, STATE1# round 0 @@ -2330,7 +2323,7 @@ _aesni_dec4: AESDECLAST KEY STATE3 AESDECLAST KEY STATE4 ret -ENDPROC(_aesni_dec4) +SYM_FUNC_END(_aesni_dec4) /* * void aesni_ecb_enc(struct crypto_aes_ctx *ctx, const u8 *dst, u8 *src, @@ -2608,8 +2601,7 @@ ENDPROC(aesni_cbc_dec) * INC:== 1, in little endian * BSWAP_MASK == endian swapping mask */ -.align 4 -_aesni_inc_init: +SYM_FUNC_START_LOCAL(_aesni_inc_init) movaps .Lbswap_mask, BSWAP_MASK movaps IV, CTR
[PATCH -resend 26/27] x86_32: assembly, change all ENTRY+ENDPROC to SYM_FUNC_*
These are all functions which are invoked from elsewhere, so we annotate them as global using the new SYM_FUNC_START (and their ENDPROC's by SYM_FUNC_END.) Now, we can finally force ENTRY/ENDPROC to be undefined on X86. Signed-off-by: Jiri Slaby Cc: "H. Peter Anvin" Cc: Thomas Gleixner Cc: Ingo Molnar Cc: x...@kernel.org Cc: Herbert Xu Cc: "David S. Miller" Cc: Bill Metzenthen Cc: Matt Fleming Cc: Ard Biesheuvel Cc: linux-crypto@vger.kernel.org Cc: linux-...@vger.kernel.org --- arch/x86/boot/compressed/efi_stub_32.S | 4 ++-- arch/x86/boot/compressed/head_32.S | 12 +-- arch/x86/crypto/salsa20-i586-asm_32.S | 4 ++-- arch/x86/crypto/serpent-sse2-i586-asm_32.S | 8 arch/x86/crypto/twofish-i586-asm_32.S | 8 arch/x86/entry/entry_32.S | 24 +++--- arch/x86/kernel/head_32.S | 16 +++ arch/x86/lib/atomic64_386_32.S | 4 ++-- arch/x86/lib/atomic64_cx8_32.S | 32 +++--- arch/x86/lib/checksum_32.S | 8 arch/x86/math-emu/div_Xsig.S | 4 ++-- arch/x86/math-emu/div_small.S | 4 ++-- arch/x86/math-emu/mul_Xsig.S | 12 +-- arch/x86/math-emu/polynom_Xsig.S | 4 ++-- arch/x86/math-emu/reg_norm.S | 8 arch/x86/math-emu/reg_round.S | 4 ++-- arch/x86/math-emu/reg_u_add.S | 4 ++-- arch/x86/math-emu/reg_u_div.S | 4 ++-- arch/x86/math-emu/reg_u_mul.S | 4 ++-- arch/x86/math-emu/reg_u_sub.S | 4 ++-- arch/x86/math-emu/round_Xsig.S | 8 arch/x86/math-emu/shr_Xsig.S | 4 ++-- arch/x86/math-emu/wm_shrx.S| 8 arch/x86/math-emu/wm_sqrt.S| 4 ++-- arch/x86/platform/efi/efi_stub_32.S| 4 ++-- include/linux/linkage.h| 8 +++- 26 files changed, 103 insertions(+), 105 deletions(-) diff --git a/arch/x86/boot/compressed/efi_stub_32.S b/arch/x86/boot/compressed/efi_stub_32.S index 257e341fd2c8..ed6c351d34ed 100644 --- a/arch/x86/boot/compressed/efi_stub_32.S +++ b/arch/x86/boot/compressed/efi_stub_32.S @@ -24,7 +24,7 @@ */ .text -ENTRY(efi_call_phys) +SYM_FUNC_START(efi_call_phys) /* * 0. The function can only be called in Linux kernel. So CS has been * set to 0x0010, DS and SS have been set to 0x0018. In EFI, I found @@ -77,7 +77,7 @@ ENTRY(efi_call_phys) movlsaved_return_addr(%edx), %ecx pushl %ecx ret -ENDPROC(efi_call_phys) +SYM_FUNC_END(efi_call_phys) .previous .data diff --git a/arch/x86/boot/compressed/head_32.S b/arch/x86/boot/compressed/head_32.S index 7e8ab0bb6968..3fa36496af12 100644 --- a/arch/x86/boot/compressed/head_32.S +++ b/arch/x86/boot/compressed/head_32.S @@ -61,7 +61,7 @@ .hidden _egot __HEAD -ENTRY(startup_32) +SYM_FUNC_START(startup_32) cld /* * Test KEEP_SEGMENTS flag to see if the bootloader is asking @@ -142,14 +142,14 @@ ENTRY(startup_32) */ lealrelocated(%ebx), %eax jmp *%eax -ENDPROC(startup_32) +SYM_FUNC_END(startup_32) #ifdef CONFIG_EFI_STUB /* * We don't need the return address, so set up the stack so efi_main() can find * its arguments. */ -ENTRY(efi_pe_entry) +SYM_FUNC_START(efi_pe_entry) add $0x4, %esp call1f @@ -174,9 +174,9 @@ ENTRY(efi_pe_entry) pushl %eax pushl %ecx jmp 2f /* Skip efi_config initialization */ -ENDPROC(efi_pe_entry) +SYM_FUNC_END(efi_pe_entry) -ENTRY(efi32_stub_entry) +SYM_FUNC_START(efi32_stub_entry) add $0x4, %esp popl%ecx popl%edx @@ -205,7 +205,7 @@ fail: movlBP_code32_start(%esi), %eax lealstartup_32(%eax), %eax jmp *%eax -ENDPROC(efi32_stub_entry) +SYM_FUNC_END(efi32_stub_entry) #endif .text diff --git a/arch/x86/crypto/salsa20-i586-asm_32.S b/arch/x86/crypto/salsa20-i586-asm_32.S index 6014b7b9e52a..edeb4c3e7389 100644 --- a/arch/x86/crypto/salsa20-i586-asm_32.S +++ b/arch/x86/crypto/salsa20-i586-asm_32.S @@ -8,7 +8,7 @@ .text # enter salsa20_encrypt_bytes -ENTRY(salsa20_encrypt_bytes) +SYM_FUNC_START(salsa20_encrypt_bytes) mov %esp,%eax and $31,%eax add $256,%eax @@ -935,4 +935,4 @@ ENTRY(salsa20_encrypt_bytes) add $64,%esi # goto bytesatleast1 jmp ._bytesatleast1 -ENDPROC(salsa20_encrypt_bytes) +SYM_FUNC_END(salsa20_encrypt_bytes) diff --git a/arch/x86/crypto/serpent-sse2-i586-asm_32.S b/arch/x86/crypto/serpent-sse2-i586-asm_32.S index d348f1553a79..f3cebd3c6739 100644 --- a/arch/x86/crypto/serpent-sse2-i586-asm_32.S +++ b/arch/x86/crypto/serpent-sse2-i586-asm_32.S @@ -512,7 +512,7 @@ pxor t0,x3; \ movdqu x3,
[PATCH -resend 23/27] x86_64: assembly, change all ENTRY+ENDPROC to SYM_FUNC_*
These are all functions which are invoked from elsewhere, so we annotate them as global using the new SYM_FUNC_START. And their ENDPROC's by SYM_FUNC_END. And make sure ENTRY/ENDPROC is not defined on X86_64, given these were the last users. Signed-off-by: Jiri Slaby Reviewed-by: Rafael J. Wysocki [hibernate] Reviewed-by: Boris Ostrovsky [xen bits] Cc: "H. Peter Anvin" Cc: Thomas Gleixner Cc: Ingo Molnar Cc: x...@kernel.org Cc: Herbert Xu Cc: "David S. Miller" Cc: "Rafael J. Wysocki" Cc: Len Brown Cc: Pavel Machek Cc: Matt Fleming Cc: Ard Biesheuvel Cc: Boris Ostrovsky Cc: Juergen Gross Cc: linux-crypto@vger.kernel.org Cc: linux...@vger.kernel.org Cc: linux-...@vger.kernel.org Cc: xen-de...@lists.xenproject.org --- arch/x86/boot/compressed/efi_thunk_64.S| 4 +- arch/x86/boot/compressed/head_64.S | 16 +++--- arch/x86/boot/compressed/mem_encrypt.S | 8 +-- arch/x86/crypto/aes-i586-asm_32.S | 8 +-- arch/x86/crypto/aes-x86_64-asm_64.S| 4 +- arch/x86/crypto/aes_ctrby8_avx-x86_64.S| 12 ++--- arch/x86/crypto/aesni-intel_asm.S | 60 +++--- arch/x86/crypto/aesni-intel_avx-x86_64.S | 24 - arch/x86/crypto/blowfish-x86_64-asm_64.S | 16 +++--- arch/x86/crypto/camellia-aesni-avx-asm_64.S| 24 - arch/x86/crypto/camellia-aesni-avx2-asm_64.S | 24 - arch/x86/crypto/camellia-x86_64-asm_64.S | 16 +++--- arch/x86/crypto/cast5-avx-x86_64-asm_64.S | 16 +++--- arch/x86/crypto/cast6-avx-x86_64-asm_64.S | 24 - arch/x86/crypto/chacha20-avx2-x86_64.S | 4 +- arch/x86/crypto/chacha20-ssse3-x86_64.S| 8 +-- arch/x86/crypto/crc32-pclmul_asm.S | 4 +- arch/x86/crypto/crc32c-pcl-intel-asm_64.S | 4 +- arch/x86/crypto/crct10dif-pcl-asm_64.S | 4 +- arch/x86/crypto/des3_ede-asm_64.S | 8 +-- arch/x86/crypto/ghash-clmulni-intel_asm.S | 8 +-- arch/x86/crypto/poly1305-avx2-x86_64.S | 4 +- arch/x86/crypto/poly1305-sse2-x86_64.S | 8 +-- arch/x86/crypto/salsa20-x86_64-asm_64.S| 4 +- arch/x86/crypto/serpent-avx-x86_64-asm_64.S| 24 - arch/x86/crypto/serpent-avx2-asm_64.S | 24 - arch/x86/crypto/serpent-sse2-x86_64-asm_64.S | 8 +-- arch/x86/crypto/sha1-mb/sha1_mb_mgr_flush_avx2.S | 8 +-- arch/x86/crypto/sha1-mb/sha1_mb_mgr_submit_avx2.S | 4 +- arch/x86/crypto/sha1-mb/sha1_x8_avx2.S | 4 +- arch/x86/crypto/sha1_avx2_x86_64_asm.S | 4 +- arch/x86/crypto/sha1_ni_asm.S | 4 +- arch/x86/crypto/sha1_ssse3_asm.S | 4 +- arch/x86/crypto/sha256-avx-asm.S | 4 +- arch/x86/crypto/sha256-avx2-asm.S | 4 +- .../crypto/sha256-mb/sha256_mb_mgr_flush_avx2.S| 8 +-- .../crypto/sha256-mb/sha256_mb_mgr_submit_avx2.S | 4 +- arch/x86/crypto/sha256-mb/sha256_x8_avx2.S | 4 +- arch/x86/crypto/sha256-ssse3-asm.S | 4 +- arch/x86/crypto/sha256_ni_asm.S| 4 +- arch/x86/crypto/sha512-avx-asm.S | 4 +- arch/x86/crypto/sha512-avx2-asm.S | 4 +- .../crypto/sha512-mb/sha512_mb_mgr_flush_avx2.S| 8 +-- .../crypto/sha512-mb/sha512_mb_mgr_submit_avx2.S | 4 +- arch/x86/crypto/sha512-mb/sha512_x4_avx2.S | 4 +- arch/x86/crypto/sha512-ssse3-asm.S | 4 +- arch/x86/crypto/twofish-avx-x86_64-asm_64.S| 24 - arch/x86/crypto/twofish-x86_64-asm_64-3way.S | 8 +-- arch/x86/crypto/twofish-x86_64-asm_64.S| 8 +-- arch/x86/entry/entry_64.S | 10 ++-- arch/x86/entry/entry_64_compat.S | 4 +- arch/x86/kernel/acpi/wakeup_64.S | 8 +-- arch/x86/kernel/ftrace_64.S| 20 arch/x86/kernel/head_64.S | 12 ++--- arch/x86/lib/checksum_32.S | 8 +-- arch/x86/lib/clear_page_64.S | 12 ++--- arch/x86/lib/cmpxchg16b_emu.S | 4 +- arch/x86/lib/cmpxchg8b_emu.S | 4 +- arch/x86/lib/copy_page_64.S| 4 +- arch/x86/lib/copy_user_64.S| 16 +++--- arch/x86/lib/csum-copy_64.S| 4 +- arch/x86/lib/getuser.S | 16 +++--- arch/x86/lib/hweight.S | 8 +-- arch/x86/lib/iomap_copy_64.S | 4 +- arch/x86/lib/memcpy_64.S | 4 +- arch/x86/lib/memmove_64.S | 4 +- arch/x86/lib/memset_64.S | 4 +- arch/x86/lib/msr-reg.S | 8 +-- arch/x86/lib/putuser.S
