Re: [PATCHv2 1/2] arm64:vdso: Rewrite gettimeofday into C.

2017-08-01 Thread Will Deacon 
On Tue, Aug 01, 2017 at 11:30:11AM +0200, Robert Richter wrote:
> Will,
> 
> On 31.05.17 13:44:30, Will Deacon wrote:
> > Thanks for posting this, but please try to cc the maintainers in future -- I
> > almost missed it!
> > 
> > On Tue, May 30, 2017 at 05:34:19PM -0700, Andrew Pinski wrote:
> > > This allows the compiler to optimize the divide by 1000.
> > > And remove the other divide.
> > > 
> > > On ThunderX, gettimeofday improves by 32%.  On ThunderX 2,
> > > gettimeofday improves by 18%.
> > > 
> > > Note I noticed a bug in the old implementation of __kernel_clock_getres;
> > > it was checking only the lower 32bits of the pointer; this would work
> > > for most cases but could fail in a few.
> > > 
> > > Changes from v1:
> > > * Fixed bug in __kernel_clock_getres for checking the pointer argument.
> > > * Fix comments to refer to functions in arm64.
> > 
> > I tested this patch on a few platforms I have access to and didn't see the
> > perf regressions I saw when I looked at this in the past with an older
> > toolchain (it was mostly about the same, with a couple of improvements).
> > 
> > So, in principle, I'm not opposed to moving this into C. However, we're
> > currently close to a "vDSO-explosion" on arm64 with people wanting a compat
> > variant and also an ILP32 variant. When Kevin posted his compat variant
> > (also in C):
> > 
> >   http://lkml.kernel.org/r/20161206160353.14581-1-kevin.brod...@arm.com
> 
> from a technical pov there are no issues in a convertion to C. Since
> this fixes bad syscall performance and an alternative solution as
> pointed out here is not in sight very soon, would you be willing to
> get this series upstream. Should we update to latest kernel and resend
> the patches for v4.14?

No, I'd much rather get this right straight off the bat whilst there's an
incentive to do it properly. Otherwise we just end up maintaining something
which nobody will realistically rework, despite their best intentions.
"bad syscall performance" seems like a bit of an over-reaction if you look
at the cost of the vDSO relative to an actual trap.

Will

Re: [PATCHv2 1/2] arm64:vdso: Rewrite gettimeofday into C.

2017-08-01 Thread Will Deacon 
On Tue, Aug 01, 2017 at 11:30:11AM +0200, Robert Richter wrote:
> Will,
> 
> On 31.05.17 13:44:30, Will Deacon wrote:
> > Thanks for posting this, but please try to cc the maintainers in future -- I
> > almost missed it!
> > 
> > On Tue, May 30, 2017 at 05:34:19PM -0700, Andrew Pinski wrote:
> > > This allows the compiler to optimize the divide by 1000.
> > > And remove the other divide.
> > > 
> > > On ThunderX, gettimeofday improves by 32%.  On ThunderX 2,
> > > gettimeofday improves by 18%.
> > > 
> > > Note I noticed a bug in the old implementation of __kernel_clock_getres;
> > > it was checking only the lower 32bits of the pointer; this would work
> > > for most cases but could fail in a few.
> > > 
> > > Changes from v1:
> > > * Fixed bug in __kernel_clock_getres for checking the pointer argument.
> > > * Fix comments to refer to functions in arm64.
> > 
> > I tested this patch on a few platforms I have access to and didn't see the
> > perf regressions I saw when I looked at this in the past with an older
> > toolchain (it was mostly about the same, with a couple of improvements).
> > 
> > So, in principle, I'm not opposed to moving this into C. However, we're
> > currently close to a "vDSO-explosion" on arm64 with people wanting a compat
> > variant and also an ILP32 variant. When Kevin posted his compat variant
> > (also in C):
> > 
> >   http://lkml.kernel.org/r/20161206160353.14581-1-kevin.brod...@arm.com
> 
> from a technical pov there are no issues in a convertion to C. Since
> this fixes bad syscall performance and an alternative solution as
> pointed out here is not in sight very soon, would you be willing to
> get this series upstream. Should we update to latest kernel and resend
> the patches for v4.14?

No, I'd much rather get this right straight off the bat whilst there's an
incentive to do it properly. Otherwise we just end up maintaining something
which nobody will realistically rework, despite their best intentions.
"bad syscall performance" seems like a bit of an over-reaction if you look
at the cost of the vDSO relative to an actual trap.

Will

Re: [PATCHv2 1/2] arm64:vdso: Rewrite gettimeofday into C.

2017-08-01 Thread Robert Richter 
Will,

On 31.05.17 13:44:30, Will Deacon wrote:
> Thanks for posting this, but please try to cc the maintainers in future -- I
> almost missed it!
> 
> On Tue, May 30, 2017 at 05:34:19PM -0700, Andrew Pinski wrote:
> > This allows the compiler to optimize the divide by 1000.
> > And remove the other divide.
> > 
> > On ThunderX, gettimeofday improves by 32%.  On ThunderX 2,
> > gettimeofday improves by 18%.
> > 
> > Note I noticed a bug in the old implementation of __kernel_clock_getres;
> > it was checking only the lower 32bits of the pointer; this would work
> > for most cases but could fail in a few.
> > 
> > Changes from v1:
> > * Fixed bug in __kernel_clock_getres for checking the pointer argument.
> > * Fix comments to refer to functions in arm64.
> 
> I tested this patch on a few platforms I have access to and didn't see the
> perf regressions I saw when I looked at this in the past with an older
> toolchain (it was mostly about the same, with a couple of improvements).
> 
> So, in principle, I'm not opposed to moving this into C. However, we're
> currently close to a "vDSO-explosion" on arm64 with people wanting a compat
> variant and also an ILP32 variant. When Kevin posted his compat variant
> (also in C):
> 
>   http://lkml.kernel.org/r/20161206160353.14581-1-kevin.brod...@arm.com

from a technical pov there are no issues in a convertion to C. Since
this fixes bad syscall performance and an alternative solution as
pointed out here is not in sight very soon, would you be willing to
get this series upstream. Should we update to latest kernel and resend
the patches for v4.14?

Thanks,

-Robert

> 
> Nathan (who apparently needs to set his mail host address ;) was concerned
> about duplication between arm and arm64:
> 
>   
> http://lkml.kernel.org/r/87r35jmv3e.fsf@wedge.i-did-not-set--mail-host-address--so-tickle-me
> 
> I'm firmly of the opinion that we should try to write an arch-agnostic vDSO
> implementation in core code (lib/vdso or something) where the arch header
> provides things like:
> 
>   * The mechanism to read the counter
>   * The mechanism to issue a syscall
>   * A function to determine whether or not the current clocksource is
> suitable
> 
> I think the datapage format could be defined in core code and it would be
> worth looking to see how much the virtual mapping code can be consolidated
> too.
> 
> If we can get something that works for arm native, arm64 native, arm64
> compat and arm64 ilp32 then it's probably going to be useful for other
> architectures too, even if we need to add more customisation points in
> future.
> 
> I've spoken to Kevin about this, but I'm not sure whether he's had a chance
> to look at knocking up a prototype. A first stab could just unconditionally
> fallback to the system call.
> 
> Will

Re: [PATCHv2 1/2] arm64:vdso: Rewrite gettimeofday into C.

2017-08-01 Thread Robert Richter 
Will,

On 31.05.17 13:44:30, Will Deacon wrote:
> Thanks for posting this, but please try to cc the maintainers in future -- I
> almost missed it!
> 
> On Tue, May 30, 2017 at 05:34:19PM -0700, Andrew Pinski wrote:
> > This allows the compiler to optimize the divide by 1000.
> > And remove the other divide.
> > 
> > On ThunderX, gettimeofday improves by 32%.  On ThunderX 2,
> > gettimeofday improves by 18%.
> > 
> > Note I noticed a bug in the old implementation of __kernel_clock_getres;
> > it was checking only the lower 32bits of the pointer; this would work
> > for most cases but could fail in a few.
> > 
> > Changes from v1:
> > * Fixed bug in __kernel_clock_getres for checking the pointer argument.
> > * Fix comments to refer to functions in arm64.
> 
> I tested this patch on a few platforms I have access to and didn't see the
> perf regressions I saw when I looked at this in the past with an older
> toolchain (it was mostly about the same, with a couple of improvements).
> 
> So, in principle, I'm not opposed to moving this into C. However, we're
> currently close to a "vDSO-explosion" on arm64 with people wanting a compat
> variant and also an ILP32 variant. When Kevin posted his compat variant
> (also in C):
> 
>   http://lkml.kernel.org/r/20161206160353.14581-1-kevin.brod...@arm.com

from a technical pov there are no issues in a convertion to C. Since
this fixes bad syscall performance and an alternative solution as
pointed out here is not in sight very soon, would you be willing to
get this series upstream. Should we update to latest kernel and resend
the patches for v4.14?

Thanks,

-Robert

> 
> Nathan (who apparently needs to set his mail host address ;) was concerned
> about duplication between arm and arm64:
> 
>   
> http://lkml.kernel.org/r/87r35jmv3e.fsf@wedge.i-did-not-set--mail-host-address--so-tickle-me
> 
> I'm firmly of the opinion that we should try to write an arch-agnostic vDSO
> implementation in core code (lib/vdso or something) where the arch header
> provides things like:
> 
>   * The mechanism to read the counter
>   * The mechanism to issue a syscall
>   * A function to determine whether or not the current clocksource is
> suitable
> 
> I think the datapage format could be defined in core code and it would be
> worth looking to see how much the virtual mapping code can be consolidated
> too.
> 
> If we can get something that works for arm native, arm64 native, arm64
> compat and arm64 ilp32 then it's probably going to be useful for other
> architectures too, even if we need to add more customisation points in
> future.
> 
> I've spoken to Kevin about this, but I'm not sure whether he's had a chance
> to look at knocking up a prototype. A first stab could just unconditionally
> fallback to the system call.
> 
> Will

Re: [PATCHv2 1/2] arm64:vdso: Rewrite gettimeofday into C.

2017-06-01 Thread Nathan Lynch 
Will Deacon  writes:
> On Tue, May 30, 2017 at 05:34:19PM -0700, Andrew Pinski wrote:
>> Note I noticed a bug in the old implementation of __kernel_clock_getres;
>> it was checking only the lower 32bits of the pointer; this would work
>> for most cases but could fail in a few.
>> 
>> Changes from v1:
>> * Fixed bug in __kernel_clock_getres for checking the pointer argument.
>> * Fix comments to refer to functions in arm64.

FWIW: despite asking around, I've never been able to determine the
original rationale for putting clock_getres in a vdso.  It seems like it
originated in powerpc and crept into other implementations.  I think
clock_getres should be dropped from new vdso implementations unless its
inclusion can be justified.


> I tested this patch on a few platforms I have access to and didn't see the
> perf regressions I saw when I looked at this in the past with an older
> toolchain (it was mostly about the same, with a couple of improvements).
>
> So, in principle, I'm not opposed to moving this into C. However, we're
> currently close to a "vDSO-explosion" on arm64 with people wanting a compat
> variant and also an ILP32 variant. When Kevin posted his compat variant
> (also in C):
>
>   http://lkml.kernel.org/r/20161206160353.14581-1-kevin.brod...@arm.com
>
> Nathan (who apparently needs to set his mail host address ;) was concerned
> about duplication between arm and arm64:
>
>   
> http://lkml.kernel.org/r/87r35jmv3e.fsf@wedge.i-did-not-set--mail-host-address--so-tickle-me
>
> I'm firmly of the opinion that we should try to write an arch-agnostic vDSO
> implementation in core code (lib/vdso or something) where the arch header
> provides things like:
>
>   * The mechanism to read the counter
>   * The mechanism to issue a syscall
>   * A function to determine whether or not the current clocksource is
> suitable
>
> I think the datapage format could be defined in core code and it would be
> worth looking to see how much the virtual mapping code can be consolidated
> too.
>
> If we can get something that works for arm native, arm64 native, arm64
> compat and arm64 ilp32 then it's probably going to be useful for other
> architectures too, even if we need to add more customisation points in
> future.
>
> I've spoken to Kevin about this, but I'm not sure whether he's had a chance
> to look at knocking up a prototype. A first stab could just unconditionally
> fallback to the system call.

I was thinking something like CONFIG_GENERIC_VDSO that arches can opt
into over time makes sense.  But the generic code needs to be amenable
to being "sourced" by the arch code for multiple ABIs (compat, ILP32) in
a single build.

There are some additional (likely somewhat dated) considerations here,
from one of the arm vdso discusssions:

https://marc.info/?l=linux-arm-kernel=140972320130624=2

Re: [PATCHv2 1/2] arm64:vdso: Rewrite gettimeofday into C.

2017-06-01 Thread Nathan Lynch 
Will Deacon  writes:
> On Tue, May 30, 2017 at 05:34:19PM -0700, Andrew Pinski wrote:
>> Note I noticed a bug in the old implementation of __kernel_clock_getres;
>> it was checking only the lower 32bits of the pointer; this would work
>> for most cases but could fail in a few.
>> 
>> Changes from v1:
>> * Fixed bug in __kernel_clock_getres for checking the pointer argument.
>> * Fix comments to refer to functions in arm64.

FWIW: despite asking around, I've never been able to determine the
original rationale for putting clock_getres in a vdso.  It seems like it
originated in powerpc and crept into other implementations.  I think
clock_getres should be dropped from new vdso implementations unless its
inclusion can be justified.


> I tested this patch on a few platforms I have access to and didn't see the
> perf regressions I saw when I looked at this in the past with an older
> toolchain (it was mostly about the same, with a couple of improvements).
>
> So, in principle, I'm not opposed to moving this into C. However, we're
> currently close to a "vDSO-explosion" on arm64 with people wanting a compat
> variant and also an ILP32 variant. When Kevin posted his compat variant
> (also in C):
>
>   http://lkml.kernel.org/r/20161206160353.14581-1-kevin.brod...@arm.com
>
> Nathan (who apparently needs to set his mail host address ;) was concerned
> about duplication between arm and arm64:
>
>   
> http://lkml.kernel.org/r/87r35jmv3e.fsf@wedge.i-did-not-set--mail-host-address--so-tickle-me
>
> I'm firmly of the opinion that we should try to write an arch-agnostic vDSO
> implementation in core code (lib/vdso or something) where the arch header
> provides things like:
>
>   * The mechanism to read the counter
>   * The mechanism to issue a syscall
>   * A function to determine whether or not the current clocksource is
> suitable
>
> I think the datapage format could be defined in core code and it would be
> worth looking to see how much the virtual mapping code can be consolidated
> too.
>
> If we can get something that works for arm native, arm64 native, arm64
> compat and arm64 ilp32 then it's probably going to be useful for other
> architectures too, even if we need to add more customisation points in
> future.
>
> I've spoken to Kevin about this, but I'm not sure whether he's had a chance
> to look at knocking up a prototype. A first stab could just unconditionally
> fallback to the system call.

I was thinking something like CONFIG_GENERIC_VDSO that arches can opt
into over time makes sense.  But the generic code needs to be amenable
to being "sourced" by the arch code for multiple ABIs (compat, ILP32) in
a single build.

There are some additional (likely somewhat dated) considerations here,
from one of the arm vdso discusssions:

https://marc.info/?l=linux-arm-kernel=140972320130624=2

RE: [PATCHv2 1/2] arm64:vdso: Rewrite gettimeofday into C.

2017-06-01 Thread Pinski, Andrew 
> -Original Message-
> From: Will Deacon [mailto:will.dea...@arm.com]
> Sent: Wednesday, May 31, 2017 5:45 AM
> To: Pinski, Andrew <andrew.pin...@cavium.com>
> Cc: linux-arm-ker...@lists.infradead.org; linux-kernel@vger.kernel.org;
> Norov, Yuri <yuri.no...@cavium.com>; catalin.mari...@arm.com;
> nathan_ly...@mentor.com; kevin.brod...@arm.com; dave.mar...@arm.com;
> john.stu...@linaro.org; a...@arndb.de
> Subject: Re: [PATCHv2 1/2] arm64:vdso: Rewrite gettimeofday into C.
> 
> Hi Andrew,
> 
> Thanks for posting this, but please try to cc the maintainers in future --
> I almost missed it!

Oh sorry; I didn't know when I am supposed to CC the maintainers or not.  
Different project, different rules.

> 
> On Tue, May 30, 2017 at 05:34:19PM -0700, Andrew Pinski wrote:
> > This allows the compiler to optimize the divide by 1000.
> > And remove the other divide.
> >
> > On ThunderX, gettimeofday improves by 32%.  On ThunderX 2,
> > gettimeofday improves by 18%.
> >
> > Note I noticed a bug in the old implementation of
> > __kernel_clock_getres; it was checking only the lower 32bits of the
> > pointer; this would work for most cases but could fail in a few.
> >
> > Changes from v1:
> > * Fixed bug in __kernel_clock_getres for checking the pointer argument.
> > * Fix comments to refer to functions in arm64.
> 
> I tested this patch on a few platforms I have access to and didn't see the
> perf regressions I saw when I looked at this in the past with an older
> toolchain (it was mostly about the same, with a couple of improvements).
> 
> So, in principle, I'm not opposed to moving this into C. However, we're
> currently close to a "vDSO-explosion" on arm64 with people wanting a compat
> variant and also an ILP32 variant. When Kevin posted his compat variant
> (also in C):
> 
>   http://lkml.kernel.org/r/20161206160353.14581-1-kevin.brod...@arm.com
> 
> Nathan (who apparently needs to set his mail host address ;) was concerned
> about duplication between arm and arm64:
> 
>   http://lkml.kernel.org/r/87r35jmv3e.fsf@wedge.i-did-not-set--mail-host-
> address--so-tickle-me
> 
> I'm firmly of the opinion that we should try to write an arch-agnostic vDSO
> implementation in core code (lib/vdso or something) where the arch header
> provides things like:
> 
>   * The mechanism to read the counter
>   * The mechanism to issue a syscall
>   * A function to determine whether or not the current clocksource is
> suitable

> 
> I think the datapage format could be defined in core code and it would be
> worth looking to see how much the virtual mapping code can be consolidated
> too.
> 
> If we can get something that works for arm native, arm64 native, arm64
> compat and arm64 ilp32 then it's probably going to be useful for other
> architectures too, even if we need to add more customisation points in
> future.

To share code between the three vdso is a good goal and shouldn't be a hard to 
expand my patch to handle the arm compat vdso.  To expand it to the arm native 
code shouldn't be too hard.  The main thing is add a few #ifdef/#define in a 
header.
I will try to do that but I don't know when I will be able to finish it.

Thanks,
Andrew


> 
> I've spoken to Kevin about this, but I'm not sure whether he's had a chance
> to look at knocking up a prototype. A first stab could just unconditionally
> fallback to the system call.
> 
> Will

RE: [PATCHv2 1/2] arm64:vdso: Rewrite gettimeofday into C.

2017-06-01 Thread Pinski, Andrew 
> -Original Message-
> From: Will Deacon [mailto:will.dea...@arm.com]
> Sent: Wednesday, May 31, 2017 5:45 AM
> To: Pinski, Andrew 
> Cc: linux-arm-ker...@lists.infradead.org; linux-kernel@vger.kernel.org;
> Norov, Yuri ; catalin.mari...@arm.com;
> nathan_ly...@mentor.com; kevin.brod...@arm.com; dave.mar...@arm.com;
> john.stu...@linaro.org; a...@arndb.de
> Subject: Re: [PATCHv2 1/2] arm64:vdso: Rewrite gettimeofday into C.
> 
> Hi Andrew,
> 
> Thanks for posting this, but please try to cc the maintainers in future --
> I almost missed it!

Oh sorry; I didn't know when I am supposed to CC the maintainers or not.  
Different project, different rules.

> 
> On Tue, May 30, 2017 at 05:34:19PM -0700, Andrew Pinski wrote:
> > This allows the compiler to optimize the divide by 1000.
> > And remove the other divide.
> >
> > On ThunderX, gettimeofday improves by 32%.  On ThunderX 2,
> > gettimeofday improves by 18%.
> >
> > Note I noticed a bug in the old implementation of
> > __kernel_clock_getres; it was checking only the lower 32bits of the
> > pointer; this would work for most cases but could fail in a few.
> >
> > Changes from v1:
> > * Fixed bug in __kernel_clock_getres for checking the pointer argument.
> > * Fix comments to refer to functions in arm64.
> 
> I tested this patch on a few platforms I have access to and didn't see the
> perf regressions I saw when I looked at this in the past with an older
> toolchain (it was mostly about the same, with a couple of improvements).
> 
> So, in principle, I'm not opposed to moving this into C. However, we're
> currently close to a "vDSO-explosion" on arm64 with people wanting a compat
> variant and also an ILP32 variant. When Kevin posted his compat variant
> (also in C):
> 
>   http://lkml.kernel.org/r/20161206160353.14581-1-kevin.brod...@arm.com
> 
> Nathan (who apparently needs to set his mail host address ;) was concerned
> about duplication between arm and arm64:
> 
>   http://lkml.kernel.org/r/87r35jmv3e.fsf@wedge.i-did-not-set--mail-host-
> address--so-tickle-me
> 
> I'm firmly of the opinion that we should try to write an arch-agnostic vDSO
> implementation in core code (lib/vdso or something) where the arch header
> provides things like:
> 
>   * The mechanism to read the counter
>   * The mechanism to issue a syscall
>   * A function to determine whether or not the current clocksource is
> suitable

> 
> I think the datapage format could be defined in core code and it would be
> worth looking to see how much the virtual mapping code can be consolidated
> too.
> 
> If we can get something that works for arm native, arm64 native, arm64
> compat and arm64 ilp32 then it's probably going to be useful for other
> architectures too, even if we need to add more customisation points in
> future.

To share code between the three vdso is a good goal and shouldn't be a hard to 
expand my patch to handle the arm compat vdso.  To expand it to the arm native 
code shouldn't be too hard.  The main thing is add a few #ifdef/#define in a 
header.
I will try to do that but I don't know when I will be able to finish it.

Thanks,
Andrew


> 
> I've spoken to Kevin about this, but I'm not sure whether he's had a chance
> to look at knocking up a prototype. A first stab could just unconditionally
> fallback to the system call.
> 
> Will

Re: [PATCHv2 1/2] arm64:vdso: Rewrite gettimeofday into C.

2017-05-31 Thread Yury Norov 
On Wed, May 31, 2017 at 01:44:30PM +0100, Will Deacon wrote:
> Hi Andrew,
> 
> Thanks for posting this, but please try to cc the maintainers in future -- I
> almost missed it!
> 
> On Tue, May 30, 2017 at 05:34:19PM -0700, Andrew Pinski wrote:
> > This allows the compiler to optimize the divide by 1000.
> > And remove the other divide.
> > 
> > On ThunderX, gettimeofday improves by 32%.  On ThunderX 2,
> > gettimeofday improves by 18%.
> > 
> > Note I noticed a bug in the old implementation of __kernel_clock_getres;
> > it was checking only the lower 32bits of the pointer; this would work
> > for most cases but could fail in a few.
> > 
> > Changes from v1:
> > * Fixed bug in __kernel_clock_getres for checking the pointer argument.
> > * Fix comments to refer to functions in arm64.
> 
> I tested this patch on a few platforms I have access to and didn't see the
> perf regressions I saw when I looked at this in the past with an older
> toolchain (it was mostly about the same, with a couple of improvements).
> 
> So, in principle, I'm not opposed to moving this into C. However, we're
> currently close to a "vDSO-explosion" on arm64 with people wanting a compat
> variant and also an ILP32 variant.

Hi Will,

This is the patch for ilp32. It's based on v1 but should be OK for v2
too. If vdso rework will be upstreamed prior to ilp32 series, I'll
incorporate it in ilp32.

Yury

>From e6f22d8ea41e6f7919de30509ac95989cd8076a4 Mon Sep 17 00:00:00 2001
From: Yury Norov 
Date: Wed, 24 May 2017 15:02:43 +0300
Subject: [PATCH 28/28] arm64a/ilp32:vdso: Rewrite gettimeofday into C.

Signed-off-by: Yury Norov 
---
 arch/arm64/kernel/vdso-ilp32/Makefile | 20 +---
 arch/arm64/kernel/vdso/gettimeofday.c |  9 -
 2 files changed, 25 insertions(+), 4 deletions(-)

diff --git a/arch/arm64/kernel/vdso-ilp32/Makefile 
b/arch/arm64/kernel/vdso-ilp32/Makefile
index 0671e88ce860..ecf62e7d1c8b 100644
--- a/arch/arm64/kernel/vdso-ilp32/Makefile
+++ b/arch/arm64/kernel/vdso-ilp32/Makefile
@@ -11,10 +11,22 @@ obj-ilp32-vdso := gettimeofday-ilp32.o note-ilp32.o 
sigreturn-ilp32.o
 targets := $(obj-ilp32-vdso) vdso-ilp32.so vdso-ilp32.so.dbg
 obj-ilp32-vdso := $(addprefix $(obj)/, $(obj-ilp32-vdso))
 
-ccflags-y := -shared -fno-common -fno-builtin
+ccflags-y := -shared -fno-common -fno-builtin -fno-stack-protector
+ccflags-y += -DDISABLE_BRANCH_PROFILING
 ccflags-y += -nostdlib -Wl,-soname=linux-ilp32-vdso.so.1 \
$(call cc-ldoption, -Wl$(comma)--hash-style=sysv)
 
+# Force -O2 to avoid libgcc dependencies
+CFLAGS_REMOVE_gettimeofday-ilp32.o = -pg -Os
+CFLAGS_gettimeofday-ilp32.o = -O2 -mcmodel=tiny -mabi=ilp32
+
+# Disable gcov profiling for VDSO code
+GCOV_PROFILE := n
+
+# Workaround for bare-metal (ELF) toolchains that neglect to pass -shared
+# down to collect2, resulting in silent corruption of the vDSO image.
+ccflags-y += -Wl,-shared
+
 obj-y += vdso-ilp32.o
 extra-y += vdso-ilp32.lds vdso-ilp32-offsets.h
 CPPFLAGS_vdso-ilp32.lds += -P -C -U$(ARCH) -mabi=ilp32
@@ -46,8 +58,8 @@ $(obj)/vdso-ilp32-offsets.h: $(obj)/vdso-ilp32.so.dbg FORCE
 #$(obj-ilp32-vdso): %.o: $(src)/../vdso/$(subst -ilp32,,%.S)
 #  $(call if_changed_dep,vdso-ilp32as)
 
-$(obj)/gettimeofday-ilp32.o: $(src)/../vdso/gettimeofday.S
-   $(call if_changed_dep,vdso-ilp32as)
+$(obj)/gettimeofday-ilp32.o: $(src)/../vdso/gettimeofday.c
+   $(call if_changed_dep,vdso-ilp32cc)
 
 $(obj)/note-ilp32.o: $(src)/../vdso/note.S
$(call if_changed_dep,vdso-ilp32as)
@@ -60,6 +72,8 @@ $(obj)/sigreturn-ilp32.o: $(src)/../vdso/sigreturn.S
 # Actual build commands
 quiet_cmd_vdso-ilp32ld = VDSOILP32L $@
   cmd_vdso-ilp32ld = $(CC) $(c_flags) -mabi=ilp32  -Wl,-n -Wl,-T $^ -o $@
+quiet_cmd_vdso-ilp32as = VDSOILP32C $@
+  cmd_vdso-ilp32cc= $(CC) $(c_flags) -mabi=ilp32 -c -o $@ $<
 quiet_cmd_vdso-ilp32as = VDSOILP32A $@
   cmd_vdso-ilp32as = $(CC) $(a_flags) -mabi=ilp32 -c -o $@ $<
 
diff --git a/arch/arm64/kernel/vdso/gettimeofday.c 
b/arch/arm64/kernel/vdso/gettimeofday.c
index a0ab8b1bd53e..0c4a00b58963 100644
--- a/arch/arm64/kernel/vdso/gettimeofday.c
+++ b/arch/arm64/kernel/vdso/gettimeofday.c
@@ -26,8 +26,15 @@
 #include 
 #include 
 #include 
+
+#ifdef __ILP32__
+#undef BITS_PER_LONG
+#define BITS_PER_LONG 32
+#endif
+
 #include 
 
+
 extern struct vdso_data _vdso_data;
 
 static notrace int gettimeofday_fallback(struct timeval *_tv,
@@ -122,7 +129,7 @@ static notrace u64 get_clock_shifted_nsec(u64 cycle_last, 
u64 mult)
 
res = res - cycle_last;
/* We can only guarantee 56 bits of precision. */
-   res &= ~(0xff00ul<<48);
+   res &= ~(0xff00ull<<48);
return res * mult;
 }
 
-- 
2.11.0

Re: [PATCHv2 1/2] arm64:vdso: Rewrite gettimeofday into C.

2017-05-31 Thread Yury Norov 
On Wed, May 31, 2017 at 01:44:30PM +0100, Will Deacon wrote:
> Hi Andrew,
> 
> Thanks for posting this, but please try to cc the maintainers in future -- I
> almost missed it!
> 
> On Tue, May 30, 2017 at 05:34:19PM -0700, Andrew Pinski wrote:
> > This allows the compiler to optimize the divide by 1000.
> > And remove the other divide.
> > 
> > On ThunderX, gettimeofday improves by 32%.  On ThunderX 2,
> > gettimeofday improves by 18%.
> > 
> > Note I noticed a bug in the old implementation of __kernel_clock_getres;
> > it was checking only the lower 32bits of the pointer; this would work
> > for most cases but could fail in a few.
> > 
> > Changes from v1:
> > * Fixed bug in __kernel_clock_getres for checking the pointer argument.
> > * Fix comments to refer to functions in arm64.
> 
> I tested this patch on a few platforms I have access to and didn't see the
> perf regressions I saw when I looked at this in the past with an older
> toolchain (it was mostly about the same, with a couple of improvements).
> 
> So, in principle, I'm not opposed to moving this into C. However, we're
> currently close to a "vDSO-explosion" on arm64 with people wanting a compat
> variant and also an ILP32 variant.

Hi Will,

This is the patch for ilp32. It's based on v1 but should be OK for v2
too. If vdso rework will be upstreamed prior to ilp32 series, I'll
incorporate it in ilp32.

Yury

>From e6f22d8ea41e6f7919de30509ac95989cd8076a4 Mon Sep 17 00:00:00 2001
From: Yury Norov 
Date: Wed, 24 May 2017 15:02:43 +0300
Subject: [PATCH 28/28] arm64a/ilp32:vdso: Rewrite gettimeofday into C.

Signed-off-by: Yury Norov 
---
 arch/arm64/kernel/vdso-ilp32/Makefile | 20 +---
 arch/arm64/kernel/vdso/gettimeofday.c |  9 -
 2 files changed, 25 insertions(+), 4 deletions(-)

diff --git a/arch/arm64/kernel/vdso-ilp32/Makefile 
b/arch/arm64/kernel/vdso-ilp32/Makefile
index 0671e88ce860..ecf62e7d1c8b 100644
--- a/arch/arm64/kernel/vdso-ilp32/Makefile
+++ b/arch/arm64/kernel/vdso-ilp32/Makefile
@@ -11,10 +11,22 @@ obj-ilp32-vdso := gettimeofday-ilp32.o note-ilp32.o 
sigreturn-ilp32.o
 targets := $(obj-ilp32-vdso) vdso-ilp32.so vdso-ilp32.so.dbg
 obj-ilp32-vdso := $(addprefix $(obj)/, $(obj-ilp32-vdso))
 
-ccflags-y := -shared -fno-common -fno-builtin
+ccflags-y := -shared -fno-common -fno-builtin -fno-stack-protector
+ccflags-y += -DDISABLE_BRANCH_PROFILING
 ccflags-y += -nostdlib -Wl,-soname=linux-ilp32-vdso.so.1 \
$(call cc-ldoption, -Wl$(comma)--hash-style=sysv)
 
+# Force -O2 to avoid libgcc dependencies
+CFLAGS_REMOVE_gettimeofday-ilp32.o = -pg -Os
+CFLAGS_gettimeofday-ilp32.o = -O2 -mcmodel=tiny -mabi=ilp32
+
+# Disable gcov profiling for VDSO code
+GCOV_PROFILE := n
+
+# Workaround for bare-metal (ELF) toolchains that neglect to pass -shared
+# down to collect2, resulting in silent corruption of the vDSO image.
+ccflags-y += -Wl,-shared
+
 obj-y += vdso-ilp32.o
 extra-y += vdso-ilp32.lds vdso-ilp32-offsets.h
 CPPFLAGS_vdso-ilp32.lds += -P -C -U$(ARCH) -mabi=ilp32
@@ -46,8 +58,8 @@ $(obj)/vdso-ilp32-offsets.h: $(obj)/vdso-ilp32.so.dbg FORCE
 #$(obj-ilp32-vdso): %.o: $(src)/../vdso/$(subst -ilp32,,%.S)
 #  $(call if_changed_dep,vdso-ilp32as)
 
-$(obj)/gettimeofday-ilp32.o: $(src)/../vdso/gettimeofday.S
-   $(call if_changed_dep,vdso-ilp32as)
+$(obj)/gettimeofday-ilp32.o: $(src)/../vdso/gettimeofday.c
+   $(call if_changed_dep,vdso-ilp32cc)
 
 $(obj)/note-ilp32.o: $(src)/../vdso/note.S
$(call if_changed_dep,vdso-ilp32as)
@@ -60,6 +72,8 @@ $(obj)/sigreturn-ilp32.o: $(src)/../vdso/sigreturn.S
 # Actual build commands
 quiet_cmd_vdso-ilp32ld = VDSOILP32L $@
   cmd_vdso-ilp32ld = $(CC) $(c_flags) -mabi=ilp32  -Wl,-n -Wl,-T $^ -o $@
+quiet_cmd_vdso-ilp32as = VDSOILP32C $@
+  cmd_vdso-ilp32cc= $(CC) $(c_flags) -mabi=ilp32 -c -o $@ $<
 quiet_cmd_vdso-ilp32as = VDSOILP32A $@
   cmd_vdso-ilp32as = $(CC) $(a_flags) -mabi=ilp32 -c -o $@ $<
 
diff --git a/arch/arm64/kernel/vdso/gettimeofday.c 
b/arch/arm64/kernel/vdso/gettimeofday.c
index a0ab8b1bd53e..0c4a00b58963 100644
--- a/arch/arm64/kernel/vdso/gettimeofday.c
+++ b/arch/arm64/kernel/vdso/gettimeofday.c
@@ -26,8 +26,15 @@
 #include 
 #include 
 #include 
+
+#ifdef __ILP32__
+#undef BITS_PER_LONG
+#define BITS_PER_LONG 32
+#endif
+
 #include 
 
+
 extern struct vdso_data _vdso_data;
 
 static notrace int gettimeofday_fallback(struct timeval *_tv,
@@ -122,7 +129,7 @@ static notrace u64 get_clock_shifted_nsec(u64 cycle_last, 
u64 mult)
 
res = res - cycle_last;
/* We can only guarantee 56 bits of precision. */
-   res &= ~(0xff00ul<<48);
+   res &= ~(0xff00ull<<48);
return res * mult;
 }
 
-- 
2.11.0

Re: [PATCHv2 1/2] arm64:vdso: Rewrite gettimeofday into C.

2017-05-31 Thread Will Deacon 
Hi Andrew,

Thanks for posting this, but please try to cc the maintainers in future -- I
almost missed it!

On Tue, May 30, 2017 at 05:34:19PM -0700, Andrew Pinski wrote:
> This allows the compiler to optimize the divide by 1000.
> And remove the other divide.
> 
> On ThunderX, gettimeofday improves by 32%.  On ThunderX 2,
> gettimeofday improves by 18%.
> 
> Note I noticed a bug in the old implementation of __kernel_clock_getres;
> it was checking only the lower 32bits of the pointer; this would work
> for most cases but could fail in a few.
> 
> Changes from v1:
> * Fixed bug in __kernel_clock_getres for checking the pointer argument.
> * Fix comments to refer to functions in arm64.

I tested this patch on a few platforms I have access to and didn't see the
perf regressions I saw when I looked at this in the past with an older
toolchain (it was mostly about the same, with a couple of improvements).

So, in principle, I'm not opposed to moving this into C. However, we're
currently close to a "vDSO-explosion" on arm64 with people wanting a compat
variant and also an ILP32 variant. When Kevin posted his compat variant
(also in C):

  http://lkml.kernel.org/r/20161206160353.14581-1-kevin.brod...@arm.com

Nathan (who apparently needs to set his mail host address ;) was concerned
about duplication between arm and arm64:

  
http://lkml.kernel.org/r/87r35jmv3e.fsf@wedge.i-did-not-set--mail-host-address--so-tickle-me

I'm firmly of the opinion that we should try to write an arch-agnostic vDSO
implementation in core code (lib/vdso or something) where the arch header
provides things like:

  * The mechanism to read the counter
  * The mechanism to issue a syscall
  * A function to determine whether or not the current clocksource is
suitable

I think the datapage format could be defined in core code and it would be
worth looking to see how much the virtual mapping code can be consolidated
too.

If we can get something that works for arm native, arm64 native, arm64
compat and arm64 ilp32 then it's probably going to be useful for other
architectures too, even if we need to add more customisation points in
future.

I've spoken to Kevin about this, but I'm not sure whether he's had a chance
to look at knocking up a prototype. A first stab could just unconditionally
fallback to the system call.

Will

Re: [PATCHv2 1/2] arm64:vdso: Rewrite gettimeofday into C.

2017-05-31 Thread Will Deacon 
Hi Andrew,

Thanks for posting this, but please try to cc the maintainers in future -- I
almost missed it!

On Tue, May 30, 2017 at 05:34:19PM -0700, Andrew Pinski wrote:
> This allows the compiler to optimize the divide by 1000.
> And remove the other divide.
> 
> On ThunderX, gettimeofday improves by 32%.  On ThunderX 2,
> gettimeofday improves by 18%.
> 
> Note I noticed a bug in the old implementation of __kernel_clock_getres;
> it was checking only the lower 32bits of the pointer; this would work
> for most cases but could fail in a few.
> 
> Changes from v1:
> * Fixed bug in __kernel_clock_getres for checking the pointer argument.
> * Fix comments to refer to functions in arm64.

I tested this patch on a few platforms I have access to and didn't see the
perf regressions I saw when I looked at this in the past with an older
toolchain (it was mostly about the same, with a couple of improvements).

So, in principle, I'm not opposed to moving this into C. However, we're
currently close to a "vDSO-explosion" on arm64 with people wanting a compat
variant and also an ILP32 variant. When Kevin posted his compat variant
(also in C):

  http://lkml.kernel.org/r/20161206160353.14581-1-kevin.brod...@arm.com

Nathan (who apparently needs to set his mail host address ;) was concerned
about duplication between arm and arm64:

  
http://lkml.kernel.org/r/87r35jmv3e.fsf@wedge.i-did-not-set--mail-host-address--so-tickle-me

I'm firmly of the opinion that we should try to write an arch-agnostic vDSO
implementation in core code (lib/vdso or something) where the arch header
provides things like:

  * The mechanism to read the counter
  * The mechanism to issue a syscall
  * A function to determine whether or not the current clocksource is
suitable

I think the datapage format could be defined in core code and it would be
worth looking to see how much the virtual mapping code can be consolidated
too.

If we can get something that works for arm native, arm64 native, arm64
compat and arm64 ilp32 then it's probably going to be useful for other
architectures too, even if we need to add more customisation points in
future.

I've spoken to Kevin about this, but I'm not sure whether he's had a chance
to look at knocking up a prototype. A first stab could just unconditionally
fallback to the system call.

Will

[PATCHv2 1/2] arm64:vdso: Rewrite gettimeofday into C.

2017-05-30 Thread Andrew Pinski 
This allows the compiler to optimize the divide by 1000.
And remove the other divide.

On ThunderX, gettimeofday improves by 32%.  On ThunderX 2,
gettimeofday improves by 18%.

Note I noticed a bug in the old implementation of __kernel_clock_getres;
it was checking only the lower 32bits of the pointer; this would work
for most cases but could fail in a few.

Changes from v1:
* Fixed bug in __kernel_clock_getres for checking the pointer argument.
* Fix comments to refer to functions in arm64.


Signed-off-by: Andrew Pinski 
---
 arch/arm64/kernel/vdso/Makefile   |  13 +-
 arch/arm64/kernel/vdso/gettimeofday.S | 329 
 arch/arm64/kernel/vdso/gettimeofday.c | 345 ++
 3 files changed, 351 insertions(+), 336 deletions(-)
 delete mode 100644 arch/arm64/kernel/vdso/gettimeofday.S
 create mode 100644 arch/arm64/kernel/vdso/gettimeofday.c

diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile
index 62c84f7..55f352f 100644
--- a/arch/arm64/kernel/vdso/Makefile
+++ b/arch/arm64/kernel/vdso/Makefile
@@ -11,10 +11,15 @@ obj-vdso := gettimeofday.o note.o sigreturn.o
 targets := $(obj-vdso) vdso.so vdso.so.dbg
 obj-vdso := $(addprefix $(obj)/, $(obj-vdso))
 
-ccflags-y := -shared -fno-common -fno-builtin
+ccflags-y := -shared -fno-common -fno-builtin -fno-stack-protector
+ccflags-y += -DDISABLE_BRANCH_PROFILING
 ccflags-y += -nostdlib -Wl,-soname=linux-vdso.so.1 \
$(call cc-ldoption, -Wl$(comma)--hash-style=sysv)
 
+# Force -O2 to avoid libgcc dependencies
+CFLAGS_REMOVE_gettimeofday.o = -pg -Os
+CFLAGS_gettimeofday.o = -O2 -mcmodel=tiny
+
 # Disable gcov profiling for VDSO code
 GCOV_PROFILE := n
 
@@ -48,15 +53,9 @@ endef
 include/generated/vdso-offsets.h: $(obj)/vdso.so.dbg FORCE
$(call if_changed,vdsosym)
 
-# Assembly rules for the .S files
-$(obj-vdso): %.o: %.S FORCE
-   $(call if_changed_dep,vdsoas)
-
 # Actual build commands
 quiet_cmd_vdsold = VDSOL   $@
   cmd_vdsold = $(CC) $(c_flags) -Wl,-n -Wl,-T $^ -o $@
-quiet_cmd_vdsoas = VDSOA   $@
-  cmd_vdsoas = $(CC) $(a_flags) -c -o $@ $<
 
 # Install commands for the unstripped file
 quiet_cmd_vdso_install = INSTALL $@
diff --git a/arch/arm64/kernel/vdso/gettimeofday.S 
b/arch/arm64/kernel/vdso/gettimeofday.S
deleted file mode 100644
index e00b467..000
--- a/arch/arm64/kernel/vdso/gettimeofday.S
+++ /dev/null
@@ -1,329 +0,0 @@
-/*
- * Userspace implementations of gettimeofday() and friends.
- *
- * Copyright (C) 2012 ARM Limited
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program.  If not, see .
- *
- * Author: Will Deacon 
- */
-
-#include 
-#include 
-#include 
-
-#define NSEC_PER_SEC_LO16  0xca00
-#define NSEC_PER_SEC_HI16  0x3b9a
-
-vdso_data  .reqx6
-seqcnt .reqw7
-w_tmp  .reqw8
-x_tmp  .reqx8
-
-/*
- * Conventions for macro arguments:
- * - An argument is write-only if its name starts with "res".
- * - All other arguments are read-only, unless otherwise specified.
- */
-
-   .macro  seqcnt_acquire
-:  ldr seqcnt, [vdso_data, #VDSO_TB_SEQ_COUNT]
-   tbnzseqcnt, #0, b
-   dmb ishld
-   .endm
-
-   .macro  seqcnt_check fail
-   dmb ishld
-   ldr w_tmp, [vdso_data, #VDSO_TB_SEQ_COUNT]
-   cmp w_tmp, seqcnt
-   b.ne\fail
-   .endm
-
-   .macro  syscall_check fail
-   ldr w_tmp, [vdso_data, #VDSO_USE_SYSCALL]
-   cbnzw_tmp, \fail
-   .endm
-
-   .macro get_nsec_per_sec res
-   mov \res, #NSEC_PER_SEC_LO16
-   movk\res, #NSEC_PER_SEC_HI16, lsl #16
-   .endm
-
-   /*
-* Returns the clock delta, in nanoseconds left-shifted by the clock
-* shift.
-*/
-   .macro  get_clock_shifted_nsec res, cycle_last, mult
-   /* Read the virtual counter. */
-   isb
-   mrs x_tmp, cntvct_el0
-   /* Calculate cycle delta and convert to ns. */
-   sub \res, x_tmp, \cycle_last
-   /* We can only guarantee 56 bits of precision. */
-   movnx_tmp, #0xff00, lsl #48
-   and \res, x_tmp, \res
-   mul \res, \res, \mult
-   .endm
-
-   /*
-* Returns in res_{sec,nsec} the REALTIME timespec, based on the
-* "wall time" (xtime) and the clock_mono delta.
-*/
-   .macro  get_ts_realtime res_sec, res_nsec, \
-

[PATCHv2 1/2] arm64:vdso: Rewrite gettimeofday into C.

2017-05-30 Thread Andrew Pinski 
This allows the compiler to optimize the divide by 1000.
And remove the other divide.

On ThunderX, gettimeofday improves by 32%.  On ThunderX 2,
gettimeofday improves by 18%.

Note I noticed a bug in the old implementation of __kernel_clock_getres;
it was checking only the lower 32bits of the pointer; this would work
for most cases but could fail in a few.

Changes from v1:
* Fixed bug in __kernel_clock_getres for checking the pointer argument.
* Fix comments to refer to functions in arm64.


Signed-off-by: Andrew Pinski 
---
 arch/arm64/kernel/vdso/Makefile   |  13 +-
 arch/arm64/kernel/vdso/gettimeofday.S | 329 
 arch/arm64/kernel/vdso/gettimeofday.c | 345 ++
 3 files changed, 351 insertions(+), 336 deletions(-)
 delete mode 100644 arch/arm64/kernel/vdso/gettimeofday.S
 create mode 100644 arch/arm64/kernel/vdso/gettimeofday.c

diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile
index 62c84f7..55f352f 100644
--- a/arch/arm64/kernel/vdso/Makefile
+++ b/arch/arm64/kernel/vdso/Makefile
@@ -11,10 +11,15 @@ obj-vdso := gettimeofday.o note.o sigreturn.o
 targets := $(obj-vdso) vdso.so vdso.so.dbg
 obj-vdso := $(addprefix $(obj)/, $(obj-vdso))
 
-ccflags-y := -shared -fno-common -fno-builtin
+ccflags-y := -shared -fno-common -fno-builtin -fno-stack-protector
+ccflags-y += -DDISABLE_BRANCH_PROFILING
 ccflags-y += -nostdlib -Wl,-soname=linux-vdso.so.1 \
$(call cc-ldoption, -Wl$(comma)--hash-style=sysv)
 
+# Force -O2 to avoid libgcc dependencies
+CFLAGS_REMOVE_gettimeofday.o = -pg -Os
+CFLAGS_gettimeofday.o = -O2 -mcmodel=tiny
+
 # Disable gcov profiling for VDSO code
 GCOV_PROFILE := n
 
@@ -48,15 +53,9 @@ endef
 include/generated/vdso-offsets.h: $(obj)/vdso.so.dbg FORCE
$(call if_changed,vdsosym)
 
-# Assembly rules for the .S files
-$(obj-vdso): %.o: %.S FORCE
-   $(call if_changed_dep,vdsoas)
-
 # Actual build commands
 quiet_cmd_vdsold = VDSOL   $@
   cmd_vdsold = $(CC) $(c_flags) -Wl,-n -Wl,-T $^ -o $@
-quiet_cmd_vdsoas = VDSOA   $@
-  cmd_vdsoas = $(CC) $(a_flags) -c -o $@ $<
 
 # Install commands for the unstripped file
 quiet_cmd_vdso_install = INSTALL $@
diff --git a/arch/arm64/kernel/vdso/gettimeofday.S 
b/arch/arm64/kernel/vdso/gettimeofday.S
deleted file mode 100644
index e00b467..000
--- a/arch/arm64/kernel/vdso/gettimeofday.S
+++ /dev/null
@@ -1,329 +0,0 @@
-/*
- * Userspace implementations of gettimeofday() and friends.
- *
- * Copyright (C) 2012 ARM Limited
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program.  If not, see .
- *
- * Author: Will Deacon 
- */
-
-#include 
-#include 
-#include 
-
-#define NSEC_PER_SEC_LO16  0xca00
-#define NSEC_PER_SEC_HI16  0x3b9a
-
-vdso_data  .reqx6
-seqcnt .reqw7
-w_tmp  .reqw8
-x_tmp  .reqx8
-
-/*
- * Conventions for macro arguments:
- * - An argument is write-only if its name starts with "res".
- * - All other arguments are read-only, unless otherwise specified.
- */
-
-   .macro  seqcnt_acquire
-:  ldr seqcnt, [vdso_data, #VDSO_TB_SEQ_COUNT]
-   tbnzseqcnt, #0, b
-   dmb ishld
-   .endm
-
-   .macro  seqcnt_check fail
-   dmb ishld
-   ldr w_tmp, [vdso_data, #VDSO_TB_SEQ_COUNT]
-   cmp w_tmp, seqcnt
-   b.ne\fail
-   .endm
-
-   .macro  syscall_check fail
-   ldr w_tmp, [vdso_data, #VDSO_USE_SYSCALL]
-   cbnzw_tmp, \fail
-   .endm
-
-   .macro get_nsec_per_sec res
-   mov \res, #NSEC_PER_SEC_LO16
-   movk\res, #NSEC_PER_SEC_HI16, lsl #16
-   .endm
-
-   /*
-* Returns the clock delta, in nanoseconds left-shifted by the clock
-* shift.
-*/
-   .macro  get_clock_shifted_nsec res, cycle_last, mult
-   /* Read the virtual counter. */
-   isb
-   mrs x_tmp, cntvct_el0
-   /* Calculate cycle delta and convert to ns. */
-   sub \res, x_tmp, \cycle_last
-   /* We can only guarantee 56 bits of precision. */
-   movnx_tmp, #0xff00, lsl #48
-   and \res, x_tmp, \res
-   mul \res, \res, \mult
-   .endm
-
-   /*
-* Returns in res_{sec,nsec} the REALTIME timespec, based on the
-* "wall time" (xtime) and the clock_mono delta.
-*/
-   .macro  get_ts_realtime res_sec, res_nsec, \
-   clock_nsec, xtime_sec,