2017-04-07 18:38 GMT+03:00 Dan Williams <d...@redhat.com>: > On Thu, 2017-03-30 at 19:32 +0300, matti kaasinen wrote: > > I tried to analyze log files from one card having problematic > > connection. > > First my set-up briefly: > > 1) Embedded Linux card behind Huawei E3131 modem connection + NM & > > MM. > > 2) Drivers: Option and huawei_cdc_ncm. > > 3) Connection monitor with following functionality: > > - monitoring frequency 10 minutes > > - monitoring task, 1..18 ping operations with 10s timeout (max 3 > > minutes > > for one task) > > - if all the 18 ping operations fail = task fails => USB power lines > > are > > sequenced. > > How long is power to the card turned off? >
2 seconds > > > - any of 18 ping operations passes = task passes => wait to next task > > start. > > - if 12 successive tasks fail (2 hours) => boot card. > > 4) Occasionally bandwidth or field limited conditions that get modem > > firmware crash and boot frequently/at any time. It is not quite sure > > what > > crashes, but usb device gets re-numerated over and over. > > If it keeps enumerating, that's the firmware crashing. > > I forget if we've covered this or not, but are you sure there's > adequate power to the card? If the modem gets into a situation where > it thinks it needs to transmit at a higher power and then draws too > much power, that could be the issue. But I've seen enough firmware > changelogs to know that all kinds of firmware crash bugs get fixed all > the time too. > I tested this modem on my desk enclosed in metal enclosure to damp field. Then I fed modem directly through laboratory power supply, which did not stop operation described in 4). So, most likely firmware is crashing. > > > From logs it seems that: > > - Usually connection recovers automatically. > > - If it does not, power sequencing usually helps. > > - However, some times it does not help. If the first sequencing fails > > helping, next 10 power breaks fail too and 12'th task boots the card. > > It > > shows from the logs that USB enumeration does not work anymore when > > power > > sequencing stops working, which suggests that either udev or most > > likely > > some kernel driver has crashed. However, no kernel crash logs are > > printed. > > Yes, it could be a kernel driver issue here, but you'd need to enable > kernel driver debugging for the USB Host Controller to figure that out, > most likely. Which could require a kernel recompile. I found a way how to reload usb kernel drivers. It was a bit tricky finding module who built sysfs structures for usb power access (/sys/kernel/debug/musb-hdrc.YY.auto/softconnect) - it was not musb_hdrc but its child: musb_dsps. Correct sequence was switching usb power off, removing all the drivers in correct order and then reloading musb_dsps, which switched power on starting device enumeration. I used this sequence to monitoring tasks as follows: Instead of executing pure usb power sequencing I made following variations: Task #1 power sequencing Task #2 udev restart Task #3 usb driver reload Task #4 usb driver reload and power sequencing Task #5...9 usb driver reload and power sequencing with one more second off time Task #10...11 usb driver reload and power sequencing with 8 seconds off time Task #12 reboot I hope this helps figuring out correct sequence. If not, then I suppose, it's time kernel recompilation. It's OK, but It's kind of maintenance problem having several versions of kernel hanging around. Only problem with this test set-up seems now the fact that gsm conditions have been too good for 15 days on that card that usually ends up booting several times a week. I'll keep you informed on the results when I get any. Thanks, Matti
_______________________________________________ networkmanager-list mailing list networkmanager-list@gnome.org https://mail.gnome.org/mailman/listinfo/networkmanager-list
