RE: Oops: 17 SMP ARM (v3.16-rc2)

2014-12-16 Thread Mattis Lorentzon
Hi Russell,

> Now because things have changed during the last merge window, I've got
> an even bigger problem sorting through that patch set and getting it
> back into a submittable state.  I've just sent out v2 for it onto the
> net...@vger.kernel.org mailing list.
>
> The initial version (marked RFC) attracted very little interest from
> testers, or acks.  I'd very much like to have some testing of it, so
> if you want to try it out, I can provide you with a git URL, patches or a
> combined patch.
 
We have run v3.16 for about three months now, and many millions of ssh
connections on eight separate systems, both without and with your network
patches. Our conclusion is that the patches clearly reduce the number of
network timeouts, and this is a great improvement. However, after a month
or so of uptime, the number of timeouts began to increase again, forcing us
to reboot the cards.

Best regards,
Mattis Lorentzon
***
Consider the environment before printing this message.

To read Autoliv's Information and Confidentiality Notice, follow this link:
http://www.autoliv.com/disclaimer.html
***

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


RE: Oops: 17 SMP ARM (v3.16-rc2)

2014-08-29 Thread Mattis Lorentzon
Iain,

> Interesting. We obviously have some differences in how we boot, my
> changes to your config to get it to boot basically amount to reverting the
> patch you attached and then enabling sata and mmc. So far I've been unable
> to get your config to fail.

Our version of U-boot doesn't support specifying a device tree separate from
the kernel, so we append it to the end of the kernel binary. We also enable
automatic configuration of IP addresses (CONFIG_IP_PNP). Our bootargs are:
console=ttymxc1,115200
ip=192.168.2.157:192.168.2.1:192.168.2.1:255.255.255.0:armcard:eth0:on
earlyprintk enable_wait_mode=off

> It would be good to know what makes my config work for you, I don't think
> I've done anything special with it.

With a couple of modifications (attached) we have been able to get your
config running on our Zynq boards as well, solving our ethernet issues.

The serial port and ethernet are essentially the only things we use. No disks,
no graphics, no USB, etc. which is why we tried to reduce the kernel
configuration to a bare minimum. We have no idea which disabled and/or
enabled options that are causing the stalls.

Best regards,
Mattis Lorentzon

***
Consider the environment before printing this message.

To read Autoliv's Information and Confidentiality Notice, follow this link:
http://www.autoliv.com/disclaimer.html
***


config.patch
Description: config.patch


RE: Oops: 17 SMP ARM (v3.16-rc2)

2014-08-26 Thread Mattis Lorentzon
Hi Iain, Russell and Fabio,

> The config is attached. Note that there's a lot of additional stuff enabled as
> I'm aiming for a single general purpose kernel that covers i.MX6, AM3359,
> Allwinner A10/A20 along with several versions of boards using those
> particular SoCs.
>
> Same kernel binary on all the boards I've tried this on, only real differences
> will be the devicetree and u-boot

Amazingly we have been able to run a complete nightly test on eight i.MX6
boards without hickups using Iain's config! We had to modify it slightly to get
it to boot, please find attached patch and Iain's patched config.

On Russell's suggestion we also began to disable flow control on the machines.
However it did not seem to make a difference because all our Zynq cards
stalled during the same test run (using our own Zynq config).

Iain's config seems promising and we will continue to run tests during the
next couple of days. We will also try to adapt Iain's config to our Zynq board.

Many thanks for all suggestions, patches and configs so far!

Best regards,
Mattis Lorentzon

***
Consider the environment before printing this message.

To read Autoliv's Information and Confidentiality Notice, follow this link:
http://www.autoliv.com/disclaimer.html
***


config.patch
Description: config.patch


config.gz
Description: config.gz


RE: Oops: 17 SMP ARM (v3.16-rc2)

2014-08-21 Thread Mattis Lorentzon
Fabio,

> What is the silicon version of the mx6 in your sabrelite? What GCC version do
> you use?

The silicon version is PCIMX6Q6AVT10AA and the GCC version we use is
arm-none-eabi-gcc (Fedora 2013.11.24-2.fc19) 4.8.1.

Iain,

> Up to now, my testing has been done with my own config, I'll now
> repeat the whole thing using the config Mattis posted to see if I can
> reproduce it that way.

Thanks for testing this. Could you also send me the config that you used for
your Sabrelite?

Do you know of any options that enable additional debug information about
the network driver state (full buffers etc.)?

Best regards,
Mattis Lorentzon

***
Consider the environment before printing this message.

To read Autoliv's Information and Confidentiality Notice, follow this link:
http://www.autoliv.com/disclaimer.html
***

RE: Oops: 17 SMP ARM (v3.16-rc2)

2014-08-14 Thread Mattis Lorentzon
Fabio,

> Do the stalls also happen on a pure 3.16 kernel?

Yes, we just tried this out overnight and we get the same stalls here.
We have seen similar problems on a Zynq-based board. It might be
worth noting that a common chip between all three boards is, for
example, the KSZ9021RN, while the FEC driver, for example, only
runs on the two iMX6-boards.

> How can we reproduce the error?

We mostly run SSH with benchmarks using NFS, it can probably be
triggered by using only SSH with the following loop:

# while : ; do ssh arm-card date; done

Our (pure) 3.16 kernel uses the following config.
http://lkml.iu.edu/hypermail/linux/kernel/1408.1/03045/config.gz

(We have quite generously disabled a lot of sub-systems in our config.)

Best regards,
Mattis Lorentzon

***
Consider the environment before printing this message.

To read Autoliv's Information and Confidentiality Notice, follow this link:
http://www.autoliv.com/disclaimer.html
***

RE: Oops: 17 SMP ARM (v3.16-rc2)

2014-08-14 Thread Mattis Lorentzon
Fabio and Russell,

> A working theory is that the switch (3Com Switch 4400) triggers the
> degeneration of the network stack from which Linux does not seem to
> recover, even if we later bypass the switch and directly connect the board to
> the server machine.

After a few more tests we have finally been able to trigger the exact same 
stalls
on the Sabrelite board with a direct network connection (i.e. without the 
switch).

Best regards,
Mattis Lorentzon

***
Consider the environment before printing this message.

To read Autoliv's Information and Confidentiality Notice, follow this link:
http://www.autoliv.com/disclaimer.html
***N�r��yb�X��ǧv�^�)޺{.n�+{zX����ܨ}���Ơz�&j:+v���zZ+��+zf���h���~i���z��w���?�&�)ߢf��^jǫy�m��@A�a���
0��h���i

RE: Oops: 17 SMP ARM (v3.16-rc2)

2014-08-13 Thread Mattis Lorentzon
Fabio and Russell,

> In order to try to narrow down whether this is a board issue, could you try to
> run the same kernel on a mx6q development board, such as mx6qsabresd,
> cubox-i, wandboard, etc?

Indeed, we have a Sabrelite development board and have run the same kernel
configuration (please find attached). Russells 30 FEC related patches are 
applied.
We have also tried with and without the extended interrupts entry in the DT.

All our tests seem to behave the same way on the Sabrelite as on our own board.
A working theory is that the switch (3Com Switch 4400) triggers the degeneration
of the network stack from which Linux does not seem to recover, even if we later
bypass the switch and directly connect the board to the server machine.

Since the problem is stochastic in nature we are not completely sure if we can
trigger the problem without the switch. It's the switch that allows us to run 
many
cards simultaneously and thus trigger the problem more easily. :-)

What are your thoughts?

Best regards,
Mattis Lorentzon

***
Consider the environment before printing this message.

To read Autoliv's Information and Confidentiality Notice, follow this link:
http://www.autoliv.com/disclaimer.html
***

config.gz
Description: config.gz


RE: Oops: 17 SMP ARM (v3.16-rc2)

2014-08-11 Thread Mattis Lorentzon
Russell and Fabio,

> I'd be interested to hear whether removing the
> 
>   interrupts-extended = ...
> 
> property from your board's DT file, thereby causing you to revert back to the
> default I list above, also fixes the instability you are seeing.

We have tried to remove the board specific interrupts-extended field and the
MX6QDL_PAD_GPIO_6__ENET_IRQ entry. Sadly this did not seem to improve
the stalls. Our interrupts look like this now:

150:  15519  0  0  0   GIC 150  2188000.ethernet
151:  0  0  0  0   GIC 151  2188000.ethernet

Our device tree might still be slightly incorrect. We have noticed that our
RGMII_INT is connected to GPIO 19 (P5) which might be nonstandard (we are
a bit surprised that this works at all). We are not quite sure how to configure
this properly.

Best regards,
Mattis Lorentzon
***
Consider the environment before printing this message.

To read Autoliv's Information and Confidentiality Notice, follow this link:
http://www.autoliv.com/disclaimer.html
***

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


RE: Oops: 17 SMP ARM (v3.16-rc2)

2014-08-07 Thread Mattis Lorentzon
Russell,

> Can you ascertain whether these stalls are a result of some failure of the
> receive side or the transmit side - you should be able to tell that if you 
> watch
> the packet counts via ifconfig on the stalled card.  Also, it would be useful 
> to
> know whether the FEC interrupt was firing.

grep eth /proc/interrupts
151:  0  0  0  0   GIC 151  2188000.ethernet
166:1205661  0  0  0  gpio-mxc   6  2188000.ethernet

The interrupt counter 166 increases regularly during the stalls.
Ifconfig indicates that the RX and TX  counters do not increase.

> I hope you have some kind of serial console on these cards?

Yes, indeed. Local stimuli seems to be able to unstall the network in a
somewhat random fashion. Running e.g. ifconfig or ping locally may
immediately or after up to about half a minute make the network responsive.
However, it usually degenerates again to a complete stall within seconds.
Without local stimuli the network does not appear to recover at all. The card
does not even respond to pings (again, most often without any apparent
error messages).

Running both of the following commands in parallel from the FC server seems
to trigger the problem within minutes (please note that the arm card stops
responding to both ping and ssh):

# while :; do ssh arm-card echo Ok; done
# ping arm-card

We have noticed the same problem on both the i.MX6 and the Zynq cards
(using KSZ9021 and Cadence GEM drivers). However, the number of
iterations required to trigger the problem vary. Sometimes it might stall after
less than 100, but in other cases the stalls begin after nearly 1 
iterations.
Once stalled (and unstalled after stimuli), the network on that particular card
degenerates a lot more often. Apart from the kernel, IP numbers and MAC
addresses, the software configurations are identical between the Zynq and
the i.MX6. Perhaps the fault is unrelated to the Freescale driver?

> Hmm.  Okay, I think the first thing we need to do is to work out why the
> silent stalls are happening.

Would you have any ideas on what to check next?

Best regards,
Mattis Lorentzon
***
Consider the environment before printing this message.

To read Autoliv's Information and Confidentiality Notice, follow this link:
http://www.autoliv.com/disclaimer.html
***

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


RE: Oops: 17 SMP ARM (v3.16-rc2)

2014-08-06 Thread Mattis Lorentzon
Russell,

> What is on the other end of the link?

16 ARM cards connected to a 3Com Switch 4400 connected to a Linux FC 20
machine (Intel Corporation 82541PI Gigabit Ethernet Controller rev 05).

There may be multiple problems. The backtrace has only been seen a few
times, on two different cards. Most of the time, the network for a random
card just stalls without any visible backtrace or error messages. The other
cards seem to be unaffected when this happens.

> What I would like to do is to stamp each packet in some way with an
> identifier marking its ring position, and then monitor the network to find out
> whether the packet at slot 85 was actually transmitted - that's made slightly
> harder because packets may be dropped at the receiver when operating in
> promisc mode.  This would then allow us to work out some likely causes.

We would be glad to run this test on our setup, do you have more detailed
information on how to set it up?

> Note that after the transmit watchdog, the interface should recover and start
> operating normally again - and that should not take "several minutes."

After a network stall, we usually have to powercycle the ARM hardware to
get it back to a usable state. These stalls last at least several minutes,
perhaps indefinitely. It does not seem to recover properly, and is no longer
reachable via the network.

Best regards,
Mattis Lorentzon
***
Consider the environment before printing this message.

To read Autoliv's Information and Confidentiality Notice, follow this link:
http://www.autoliv.com/disclaimer.html
***

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


RE: Oops: 17 SMP ARM (v3.16-rc2)

2014-08-05 Thread Mattis Lorentzon
Hi Fabio,

> Could this problem be the same one as reported at:
> http://www.spinics.net/lists/arm-kernel/msg347914.html ?

The problem you link to describes a permanent issue, our problem seems
to be sporadic as most of our tests work fine (at least for a while).

> Which Ethernet PHY do you use? Do you have pull-up in the MDIO line?

Our hardware has the KSZ9021RN PHY, so the MDIO line should be pull-up.

Do you know if there are debug options that could help us determine the
cause of the timeout?

Best regards,
Mattis Lorentzon

***
Consider the environment before printing this message.

To read Autoliv's Information and Confidentiality Notice, follow this link:
http://www.autoliv.com/disclaimer.html
***

RE: Oops: 17 SMP ARM (v3.16-rc2)

2014-08-05 Thread Mattis Lorentzon
Hi Russell!

> Now because things have changed during the last merge window, I've got an
> even bigger problem sorting through that patch set and getting it back into a
> submittable state.  I've just sent out v2 for it onto the
> net...@vger.kernel.org mailing list.
>
> The initial version (marked RFC) attracted very little interest from testers, 
> or
> acks.  I'd very much like to have some testing of it, so if you want to try 
> it out,
> I can provide you with a git URL, patches or a combined patch.

We have applied your V2 patch set of 30 patches on top of v3.16-rc2 and are
currently running some stability tests.

During our first test round we triggered a timeout which caused the fec driver
to become unresponsive for several minutes. The attached backtrace was
shown when the hardware was rebooted.

Best regards,
Mattis Lorentzon

***
Consider the environment before printing this message.

To read Autoliv's Information and Confidentiality Notice, follow this link:
http://www.autoliv.com/disclaimer.html
***
[ cut here ]
WARNING: CPU: 0 PID: 0 at net/sched/sch_generic.c:264 dev_watchdog+0x270/0x27c()
NETDEV WATCHDOG: eth0 (fec): transmit queue 0 timed out
Modules linked in:
CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.16.0-rc2+ #7
Backtrace: 
[<8001234c>] (dump_backtrace) from [<80012628>] (show_stack+0x18/0x1c)
 r6:0108 r5: r4:806ac3dc r3:
[<80012610>] (show_stack) from [<804d2c60>] (dump_stack+0x8c/0x9c)
[<804d2bd4>] (dump_stack) from [<80025c18>] (warn_slowpath_common+0x74/0x90)
 r5:0009 r4:8068fd70
[<80025ba4>] (warn_slowpath_common) from [<80025c6c>] 
(warn_slowpath_fmt+0x38/0x40)
 r8:806900c0 r7:9e160254 r6:9f4ec800 r5:9e16 r4:
[<80025c38>] (warn_slowpath_fmt) from [<803f4578>] (dev_watchdog+0x270/0x27c)
 r3:9e16 r2:8061ad58
[<803f4308>] (dev_watchdog) from [<8002fee8>] (call_timer_fn+0x74/0xec)
 r10:8068e008 r9:9e16 r8:803f4308 r7:0100 r6:8068e000 r5:0001
 r4:8068fdd8
[<8002fe74>] (call_timer_fn) from [<80030b2c>] (run_timer_softirq+0x1d4/0x254)
 r10:803f4308 r9:806900c0 r8:9e16 r7: r6:8068fe28 r5:806cc140
 r4:9e160284
[<80030958>] (run_timer_softirq) from [<8002a124>] (__do_softirq+0x168/0x2f0)
 r10:0001 r9:80690080 r8:4001 r7:8068e000 r6:0100 r5:80690084
 r4:0020
[<80029fbc>] (__do_softirq) from [<8002a5d0>] (irq_exit+0xc8/0x10c)
 r10:8068e000 r9:806cafd9 r8:0001 r7:f4000100 r6: r5:001d
 r4:8068e008
[<8002a508>] (irq_exit) from [<8000f304>] (handle_IRQ+0x4c/0x98)
 r5:001d r4:8068ae14
[<8000f2b8>] (handle_IRQ) from [<8000860c>] (gic_handle_irq+0x34/0x64)
 r6:8068ff20 r5:80696a40 r4:f400010c r3:00a0
[<800085d8>] (gic_handle_irq) from [<80013184>] (__irq_svc+0x44/0x58)
Exception stack(0x8068ff20 to 0x8068ff68)
ff20: 0001 0001  806996f0 8069652c 806964d8 806cafd9 804db068
ff40: 0001 806cafd9 8068e000 8068ff74  8068ff68 80061a1c 8000f664
ff60: 200f0013 
 r7:8068ff54 r6: r5:200f0013 r4:8000f664
[<8000f638>] (arch_cpu_idle) from [<8005d154>] (cpu_startup_entry+0x10c/0x164)
[<8005d048>] (cpu_startup_entry) from [<804cdefc>] (rest_init+0xc8/0xd8)
 r7:80683450 r3:
[<804cde34>] (rest_init) from [<80651c68>] (start_kernel+0x3a0/0x3ac)
 r5:0001 r4:806965d0
[<806518c8>] (start_kernel) from [<10008074>] (0x10008074)
---[ end trace b51f6196c5e036f0 ]---
fec 2188000.ethernet eth0: TX ring dump
Nr SC addr   len  SKB
  00x1c00 0x   66   (null)
  10x1c00 0x   66   (null)
  20x1c00 0x   66   (null)
  30x1c00 0x   66   (null)
  40x1c00 0x   66   (null)
  50x1c00 0x   66   (null)
  60x1c00 0x   66   (null)
  70x1c00 0x   66   (null)
  80x1c00 0x   66   (null)
  90x1c00 0x   66   (null)
 100x1c00 0x   66   (null)
 110x1c00 0x   66   (null)
 120x1c00 0x   66   (null)
 130x1c00 0x   66   (null)
 140x1c00 0x   66   (null)
 150x1c00 0x   66   (null)
 160x1c00 0x   66   (null)
 170x1c00 0x   66   (null)
 180x1c00 0x   66   (null)
 190x1c00 0x   66   (null)
 200x1c00 0x   66   (null)
 210x1c00 0x   66   (null)
 220x1c00 0x   66   (null)
 230x1c00 0x   66   (null)
 240x1c00 0x   66   (null)
 250x1c00 0x   66   (null)
 260x1c00 0x   66   (null)
 270x1c00 0x   66   (null)
 280x1c00 0x   66   (null)
 290x1c00 0x   66   (n

RE: Oops: 17 SMP ARM (v3.16-rc2)

2014-06-26 Thread Mattis Lorentzon
Thank you for your reply,

> On Wed, Jun 25, 2014 at 01:55:05PM +0000, Mattis Lorentzon wrote:
> > I have a similar issue with v3.16-rc2 as previously reported by Waldemar
> Brodkorb for v3.15-rc4.
> > https://lkml.org/lkml/2014/5/9/330
> 
> This URL returns no useful information.  I find that lkml.org is broken more
> times than not in recent years.  Please use a different archive site when
> referring to posts, thanks.

http://lkml.iu.edu/hypermail/linux/kernel/1405.1/01114.html

> I have had two iMX6 platforms running root-NFS for about the last six to nine
> months with various workloads, and have never seen this oops.
> Unfortunately, the description above gives very little information for what
> the mechanism to trigger this bug may be.  For example, if I wanted to
> reproduce it, what would I need to do?

We have managed to trigger the Oops by just transferring a large file over nfs
cat /mnt/foo > /dev/null
where foo is a file that is approximately 2 GB. There may be some packet losses
on this network, perhaps this differs from your workload?

> > The error is sporadic and it seems to occur more frequently when using
> perf.
> 
> So it occurs when not using perf?

Yes, certainly, see above.

We have done some more investigations, please find it in this mail:

http://lkml.iu.edu/hypermail/linux/kernel/1406.3/02190.html

The Oops seems to have been introduced somewhere between v3.12 and v3.13:

- The Oops is reproducible within seconds when running Linux 3.16-rc2.
- We have observed the Oops on 8 different hardware units and two different 
chipsets (Freescale i.MX6 and Xilinx Zynq).
- The Oops has not been seen on Linux 3.12 so it appears to be good.
- The Oops has been seen on Linux 3.13, 3.14, 3.15, 3.16-rc2 so these appear to 
be bad.

Configs and a couple of Oops reports are attached to the linked mail.

Best regards,
Mattis Lorentzon
***
Consider the environment before printing this message.

To read Autoliv's Information and Confidentiality Notice, follow this link:
http://www.autoliv.com/disclaimer.html
***

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Oops: 17 SMP ARM (v3.16-rc2)

2014-06-25 Thread Mattis Lorentzon
Hello kernel people,

I have a similar issue with v3.16-rc2 as previously reported by Waldemar 
Brodkorb for v3.15-rc4.
https://lkml.org/lkml/2014/5/9/330

We are running a benchmark application, sometimes using perf, with heavy 
traffic over NFS.
The error is sporadic and it seems to occur more frequently when using perf.

Linux imx6-test0 3.16.0-rc2+ #1 SMP Wed Jun 25 15:04:16 CEST 2014 armv7l armv7l 
armv7l GNU/Linux

Any help is greatly appreciated.

Best regards,
Mattis Lorentzon

Unable to handle kernel paging request at virtual address 
pgd = 9e338000
[] *pgd=2fffd821, *pte=, *ppte=
Internal error: Oops: 17 [#1] SMP ARM
Modules linked in:
CPU: 0 PID: 146 Comm: stereo Not tainted 3.16.0-rc2+ #1
task: 9e07a700 ti: 81c42000 task.ti: 81c42000
PC is at find_get_entry+0x60/0xfc
LR is at radix_tree_lookup_slot+0x1c/0x2c
pc : [<800a34d8>]lr : [<80290448>]psr: a013
sp : 81c43d98  ip :   fp : 81c43dcc
r10: 0001  r9 : 9e30e3c0  r8 : 02a7
r7 : 9f3758a0  r6 :   r5 : 0001  r4 : 
r3 : 81c43d84  r2 :   r1 : 02a7  r0 : 
Flags: NzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
Control: 10c5387d  Table: 2e33804a  DAC: 0015
Process stereo (pid: 146, stack limit = 0x81c42240)
Stack: (0x81c43d98 to 0x81c44000)
3d80:    
3da0: 800a3478 000a6000 81c43ecc  9f37589c  806cb02a 02a7
3dc0: 81c43e04 81c43dd0 800a406c 800a3484 80061ca0 9fc2dfe0 0013 0059
3de0: 9f37589c 9f375770 0300 02a7 9e30e3c0 02a7 81c43e94 81c43e08
3e00: 800a50c4 800a4040   801d1818  1000 00080001
3e20: 02a6 9f3757f4 0300 000a7000  801d1818 9e30e490 9f37567c
3e40: 81c43ee8 81c43ed4   804d87e0 80067098 0004 9f375770
3e60: 81c43e94 81c43e70 801d491c 81c43ee8 9f375770 81c43ed4 9e30e3c0 9e07a700
3e80: 76907000  81c43ebc 81c43e98 801d1818 800a4dfc 80061ca0 80061b0c
3ea0: 9f375770 0020  81c43f78 81c43f44 81c43ec0 800e1348 801d17b8
3ec0: 0010 81c43ed0 800e1764 76907000 0010  000a7000 00059000
3ee0: 81c43ecc 0001 9e30e3c0    9e07a700 
3f00:   0020  0010   
3f20: 9e30e3c0 9e30e3c0 76907000 81c43f78 9e30e3c0 0010 81c43f74 81c43f48
3f40: 800e1adc 800e12b8  0027cce0 0020  9e30e3c0 9e30e3c0
3f60: 0010 76907000 81c43fa4 81c43f78 800e2200 800e1a58 0020 
3f80: 0027cce0  0007cce0 0003 8000ebc4 81c42000  81c43fa8
3fa0: 8000ea00 800e21c8 0027cce0  0003 76907000 0010 
3fc0: 0027cce0  0007cce0 0003 0142b5a0   
3fe0:  7ec59d94 76dc26ac 76e1762c 6010 0003  
Backtrace:
[<800a3478>] (find_get_entry) from [<800a406c>] (pagecache_get_page+0x38/0x1d8)
 r8:02a7 r7:806cb02a r6: r5:9f37589c r4:
[<800a4034>] (pagecache_get_page) from [<800a50c4>] 
(generic_file_read_iter+0x2d4/0x750)
 r10:02a7 r9:9e30e3c0 r8:02a7 r7:0300 r6:9f375770 r5:9f37589c
 r4:0059
[<800a4df0>] (generic_file_read_iter) from [<801d1818>] 
(nfs_file_read+0x6c/0xa8)
 r10: r9:76907000 r8:9e07a700 r7:9e30e3c0 r6:81c43ed4 r5:9f375770
 r4:81c43ee8
[<801d17ac>] (nfs_file_read) from [<800e1348>] (new_sync_read+0x9c/0xc4)
 r6:81c43f78 r5: r4:0020
[<800e12ac>] (new_sync_read) from [<800e1adc>] (vfs_read+0x90/0x150)
 r8:0010 r7:9e30e3c0 r6:81c43f78 r5:76907000 r4:9e30e3c0
[<800e1a4c>] (vfs_read) from [<800e2200>] (SyS_read+0x44/0x98)
 r9:76907000 r8:0010 r7:9e30e3c0 r6:9e30e3c0 r5: r4:0020
[<800e21bc>] (SyS_read) from [<8000ea00>] (ret_fast_syscall+0x0/0x48)
 r9:81c42000 r8:8000ebc4 r7:0003 r6:0007cce0 r5: r4:0027cce0
Code: e1a01008 eb07b3d6 e350 0a1c (e5904000)
---[ end trace bebb56a5d6f464ed ]---

***
Consider the environment before printing this message.

To read Autoliv's Information and Confidentiality Notice, follow this link:
http://www.autoliv.com/disclaimer.html
***
Booting Linux on physical CPU 0x0
Linux version 3.16.0-rc2+ (mattisl@localhost.localdomain) (gcc version 4.8.2 
(GCC) ) #1 SMP Wed Jun 25 15:04:16 CEST 2014
CPU: ARMv7 Processor [412fc09a] revision 10 (ARMv7), cr=10c5387d
CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
Machine model: Freescale i.MX6 Quad SABRE Lite Board
bootconsole [earlycon0] enabled
Memory policy: Data cache writealloc
On node 0 totalpages: 131072
free_area_init_node: node 0, pgdat 806ca500, node_mem_map 9fbf7000
  Normal zone: 1024 pages used for memmap
  Normal zone: 0 pages reserved
  Normal zo