Re: musb - high CPU load in DMA mode and dropouts during audio playback
Hi George, On 12.3.2014 12:15, George Cherian wrote: So it looks like, without this patch, hrtimer trick for compensation of early interrupt doesn't work and samples gets garbled during transfers (albeit i don't have HW USB analyzer to prove it). Alternative with fifo checking and workqueue, unfortunately hogs CPU. The basic issue is we get the DMA interrupt very early that it takes some time for the packet to flow out from the TX Fifo. In some cases we end up re-scheduling the workqueue multiple times since the TX FIFO is NYET empty. It seems, that rescheduling definitely helped to avoid data corruption. Alternatively do you have any reason not to try out the PIO mode? I already tried that at the beginning of my test as first workaround to remedy audio interface issues. It is best, what i achieved with Sitara SoC, but it is not stable when system is loaded with other things. Practically that means, audio is playing, but when there is for instance song database update from player app, which involves additional network transfers and reading of song tags, it starts to stutter badly. This get worse, when playing high resolution audio (eg. 192k/24bit). So to sum it up, it is not really usable from my tests. Best regards, Michal -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: musb - high CPU load in DMA mode and dropouts during audio playback
Hi George, I did few more tests and have further details for you. This time i modified my build procedure and according to Felipe's hint, I used ti-linux-3.12.y branch from TI repository and also clean 3.14-rc6 kernel, where i subsequently tried additional unmerged patches. With ti-linux-3.12.y I didn't notice any of CPU usage or stability, 80-90%, dropouts. Latest kernel behaved differently, CPU usage is low and audio after DAC is total rubbish (constant level noise). After playing with additional patches, i've found, that modifications in your third patch for different isochronous packet handling https://lkml.org/lkml/2014/1/24/187 causes high load. So it looks like, without this patch, hrtimer trick for compensation of early interrupt doesn't work and samples gets garbled during transfers (albeit i don't have HW USB analyzer to prove it). Alternative with fifo checking and workqueue, unfortunately hogs CPU. Thanks, Michal On 27.2.2014 12:18, Michal Šmucr wrote: Hi George, On 27.2.2014 5:15, George Cherian wrote: I too see the backtraces will send a patch soon to fix the same. Thanks, i applied it and most of backtraces was suppressed. Several times i was able to invocate it again, but this time, it was before complete stuck of playback application (kill -9 was only way to end it). From that moment, no other audio playback can be initiated until next reboot. If i tried to remedy situation by replugging of USB interface, it leaded to nice Oops (snippet from serial console is attached). During my testing am not seeing the CPU usage as you mention. That is interesting, i was able to reproduce it whenever i tried that, maybe it could be also some config issue. Just for more complete info, i'm using build scripts by Robert C. Nelson and there are couple other patches applied to kernel before build (now including your previous ISOCH. handling ones) - https://github.com/RobertCNelson/linux-dev/tree/am33x-v3.14/patches , but it doesn't seems related to issue. Could you please share you .config. You'll find it attached. I used aplay for testing playback and arecord for recording using USB headsets. I tried mpd as player, but i'm able to reproduce kworker load also with aplay or alsa built-in speaker-test (eg. speaker-test -c 2 -D plughw:0,0). Regarding hardware i tried few XMOS based UAC2 interfaces and recently also interface with TI PCM2904 USB codec, which could be very close to headset, you tried. It doesn't matter according to my tests. Thanks, Michal -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: musb - babble interrupt recovery
Hello, On 21.2.2014 15:09, Michal Šmucr wrote: I tried both patches separately on 3.14-rc3 and second patch alone helped here. I wasn't able to reproduce babble interrupt anymore, no matter, what i did with my hub and USB peripherals. Great! Unfortunately i found another appearance of Babble and this time without any workaround. Recently I tested HiFace USB audio interface (https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/sound/usb/hiface), mainly because it using bulk transfers instead of isochronous ones and i was curious, if it will be also affected by high CPU utilization issue, which i reported in my other posts. After connection to Beaglebone it seems to be correctly enumerated (albeit with one warning about its reported standard violating maxpacket length, but it shouldn't be lethal). Then after initiating of audio playback, Babble interrupt occurs and this is the end. Force cancellation of playback emits couple of warnings in dmesg, but this is probably sign of unfinished urbs. Just for completeness, audio interface works well on Intel NUC board with same kernel version. Best regards, Michal Bus 002 Device 002: ID 04b4:930b Cypress Semiconductor Corp. Device Descriptor: bLength18 bDescriptorType 1 bcdUSB 2.00 bDeviceClass 255 Vendor Specific Class bDeviceSubClass 255 Vendor Specific Subclass bDeviceProtocol 255 Vendor Specific Protocol bMaxPacketSize064 idVendor 0x04b4 Cypress Semiconductor Corp. idProduct 0x930b bcdDevice0.01 iManufacturer 1 M2TECH iProduct2 USB-SPDIF iSerial 0 bNumConfigurations 1 Configuration Descriptor: bLength 9 bDescriptorType 2 wTotalLength 25 bNumInterfaces 1 bConfigurationValue 0 iConfiguration 0 bmAttributes 0x60 (Missing must-be-set bit!) Self Powered Remote Wakeup MaxPower 100mA Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber0 bAlternateSetting 0 bNumEndpoints 1 bInterfaceClass 255 Vendor Specific Class bInterfaceSubClass255 Vendor Specific Subclass bInterfaceProtocol255 Vendor Specific Protocol iInterface 0 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x02 EP 2 OUT bmAttributes2 Transfer TypeBulk Synch Type None Usage Type Data wMaxPacketSize 0x0400 1x 1024 bytes bInterval 0 Device Qualifier (for other device speed): bLength10 bDescriptorType 6 bcdUSB 2.00 bDeviceClass 255 Vendor Specific Class bDeviceSubClass 255 Vendor Specific Subclass bDeviceProtocol 255 Vendor Specific Protocol bMaxPacketSize064 bNumConfigurations 1 Device Status: 0x0002 (Bus Powered) Remote Wakeup Enabled [0.00] Booting Linux on physical CPU 0x0 [0.00] Initializing cgroup subsys cpuset [0.00] Initializing cgroup subsys cpu [0.00] Initializing cgroup subsys cpuacct [0.00] Linux version 3.14.0-rc4-bone0 (msmucr@vmdebian-ct) (gcc version 4.8.3 20140106 (prerelease) (crosstool-NG linaro-1.13.1-4.8-2014.01 - Linaro GCC 2013.11) ) #1 SMP Sun Mar 2 15:10:51 CET 2014 [0.00] CPU: ARMv7 Processor [413fc082] revision 2 (ARMv7), cr=10c5387d [0.00] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache [0.00] Machine model: TI AM335x BeagleBone [0.00] cma: CMA: reserved 16 MiB at 9e80 [0.00] Memory policy: Data cache writeback [0.00] On node 0 totalpages: 130816 [0.00] free_area_init_node: node 0, pgdat c0b09000, node_mem_map dfaf2000 [0.00] Normal zone: 1024 pages used for memmap [0.00] Normal zone: 0 pages reserved [0.00] Normal zone: 130816 pages, LIFO batch:31 [0.00] CPU: All CPU(s) started in SVC mode. [0.00] AM335X ES2.0 (sgx neon ) [0.00] PERCPU: Embedded 9 pages/cpu @dfad7000 s14400 r8192 d14272 u36864 [0.00] pcpu-alloc: s14400 r8192 d14272 u36864 alloc=9*4096 [0.00] pcpu-alloc: [0] 0 [0.00] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 129792 [0.00] Kernel command line: console=ttyO0,115200n8 capemgr.disable_partno=BB-BONELT-HDMI,BB-BONELT-HDMIN,BB-BONE-EMMC-2G root=/dev/mmcblk0p2 ro rootfstype=ext4 rootwait fixrtc ip= [0.00] PID hash table entries: 2048 (order: 1, 8192 bytes) [0.00] Dentry cache hash table entries: 65536 (order: 6, 262144 bytes) [0.00
Re: musb - high CPU load in DMA mode and dropouts during audio playback
Hi George, On 27.2.2014 5:15, George Cherian wrote: I too see the backtraces will send a patch soon to fix the same. Thanks, i applied it and most of backtraces was suppressed. Several times i was able to invocate it again, but this time, it was before complete stuck of playback application (kill -9 was only way to end it). From that moment, no other audio playback can be initiated until next reboot. If i tried to remedy situation by replugging of USB interface, it leaded to nice Oops (snippet from serial console is attached). During my testing am not seeing the CPU usage as you mention. That is interesting, i was able to reproduce it whenever i tried that, maybe it could be also some config issue. Just for more complete info, i'm using build scripts by Robert C. Nelson and there are couple other patches applied to kernel before build (now including your previous ISOCH. handling ones) - https://github.com/RobertCNelson/linux-dev/tree/am33x-v3.14/patches , but it doesn't seems related to issue. Could you please share you .config. You'll find it attached. I used aplay for testing playback and arecord for recording using USB headsets. I tried mpd as player, but i'm able to reproduce kworker load also with aplay or alsa built-in speaker-test (eg. speaker-test -c 2 -D plughw:0,0). Regarding hardware i tried few XMOS based UAC2 interfaces and recently also interface with TI PCM2904 USB codec, which could be very close to headset, you tried. It doesn't matter according to my tests. Thanks, Michal config-3.14.0-rc4-bone0.gz Description: application/gzip [ 176.322424] usb 2-1: USB disconnect, device number 2 [ 179.654157] usb 2-1: new high-speed USB device number 3 using musb-hdrc [ 179.795133] usb 2-1: device v249c p930b is not supported [ 179.800758] usb 2-1: New USB device found, idVendor=249c, idProduct=930b [ 179.807928] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3 [ 179.815489] usb 2-1: Product: M2Tech USB Audio 2.0 [ 179.820540] usb 2-1: Manufacturer: M2Tech [ 179.824901] usb 2-1: SerialNumber: [ 329.707680] systemd-logind[2285]: New session 2 of user root. [ 671.601671] usb 2-1: USB disconnect, device number 3 [ 671.613917] Unable to handle kernel paging request at virtual address ffe6 [ 671.621545] pgd = c0004000 [ 671.624400] [ffe6] *pgd=9fef6821, *pte=, *ppte= [ 671.631045] Internal error: Oops: 17 [#1] SMP ARM [ 671.635995] Modules linked in: usb_f_acm u_serial usb_f_ecm g_multi usb_f_mass_storage usb_f_rndis u_ether libcomposite rpcsec_gss_krb5 nfsd snd_usb_audio snd_usbmidi_lib snd_hwdep snd_seq_midi snd_seq_midi_event snd_rawmidi snd_pcm snd_seq snd_seq_device snd_timer omap_aes snd omap_sham soundcore ti_am335x_adc kfifo_buf industrialio rtc_omap uio_pdrv_genirq uio [ 671.670042] CPU: 0 PID: 19 Comm: khubd Not tainted 3.14.0-rc4-bone0 #3 [ 671.676914] task: de0bf200 ti: de176000 task.ti: de176000 [ 671.682622] PC is at musb_g_tx+0x9c/0x18c [ 671.686856] LR is at cppi41_trans_done+0x48/0x134 [ 671.691814] pc : [c0515354]lr : [c05165c4]psr: a00f0193 [ 671.691814] sp : de177c38 ip : 0019 fp : c0b0ca00 [ 671.703880] r10: de01eef8 r9 : 200f0193 r8 : de01e010 [ 671.709378] r7 : de01eeb0 r6 : 3400 r5 : e0878de0 r4 : ffcc [ 671.716242] r3 : r2 : 0001 r1 : e0878de2 r0 : de01e010 [ 671.723109] Flags: NzCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment kernel [ 671.730887] Control: 10c5387d Table: 9e7a0019 DAC: 0015 [ 671.736931] Process khubd (pid: 19, stack limit = 0xde176248) [ 671.742976] Stack: (0xde177c38 to 0xde178000) [ 671.747567] 7c20: de027944 de01ee84 [ 671.756179] 7c40: dd281000 de01e010 0080 200f0193 008a c05165c4 0003 [ 671.764790] 7c60: 0001 c0166368 800f0093 dd399790 dd3a2b38 de027944 de01ee84 dd281000 [ 671.773401] 7c80: de01e010 0080 200f0193 c0516c44 de58ea80 c0074280 dd554580 00029903 [ 671.782009] 7ca0: 00029903 0004 dd281000 9e841720 0080 0001 [ 671.790620] 7cc0: 008a c040ad88 dd2a2e80 de005900 0021 [ 671.799230] 7ce0: a00f0013 c008037c de005900 dd2a2e80 1f065000 de005900 c0a741e4 [ 671.807842] 7d00: c0b11be4 0001 a00f0013 dd39 c0080550 0002 de005900 [ 671.816452] 7d20: c0a741e4 c0082fcc 0021 c007fbe4 0021 c000e244 fa20 de177d60 [ 671.825064] 7d40: 0021 c0008578 c0511c6c c06df880 a00f0013 de177d94 c06dffc0 [ 671.833675] 7d60: de01e010 a00f0013 34a6 747e de590080 de5a8800 de01e010 8300 [ 671.842286] 7d80: 0001 a00f0013 dd39 e0878de2 de177da8 c0511c6c c06df880 [ 671.850897] 7da0: a00f0013 de5a8800 ff98 a00f0013 de6ed3ec de6ed3f0 [ 671.859508] 7dc0: de62a800 c04f9d10 de26c200 de6ec000 de6ec000 13e8 bf0cc4bc [ 671.868118] 7de0:
Re: musb - high CPU load in DMA mode and dropouts during audio playback
On 18.2.2014 2:18, Michal Šmucr wrote: Hello, i'm fighting with audio playback problem on Beaglebone Black AM-3358. Class compilant USB soundcards with isochronous transfers. With DMA enabled in kernels 3.13.2 and 3.14-rc2 it has following symptoms. - it will stutter immediately after starts of playback - kworker thread shows about 80-90% CPU usage during playback - it presents with cppi41_dma_control warnings and traces, plus couple of transfer errors (i've enabled also CONFIG_DMADEVICES_DEBUG, so it dmesg is quite lengthy) http://vpub.smucr.cz/pub/bbb/debug/dmesg.log With force PIO at any tested kernel from 3.8.x to 3.14.x and DMA mode at 3.12.9 it is better, CPU load seems to be much lower, but it is not completely solved. Still during high interrupt load from other devices like NIC or second USB port, i'm having audio dropouts. Without any messages this time. I tested to apply following George Cherian's related patches http://marc.info/?l=linux-usbm=139081552015728w=2 usb: musb: musb_host: Enable ISOCH IN handling for AM335x host usb: musb: musb_cppi41: Make CPPI aware of high bandwidth transfers usb: musb: musb_cppi41: Handle ISOCH differently and not use the hrtimer and Adam Wozniak's http://marc.info/?l=linux-usbm=139101388408467w=2 high cpu load on omap3 using musb I also tried to find something related in TI git tree and compile linux-next, but without luck. Any tips, what to do next? Does anybody experienced same issue? Thanks, Michal -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: musb - babble interrupt recovery
Thank you for reply, On 21.2.2014 2:03, Felipe Balbi wrote: heh, you can drop the Mr. ;-) :-) We have another version in TI's tree (git.ti.com) which will be sent soonish for mainline. Just hashing out a few extra details. That sounds great and i didn't know about that repository. I'll check patches there and possibly try it on BBB, maybe there is also something, that could also address other issue, i'm having with musb (DMA errors and high CPU load) and then possibly post some field report. George and Roger have been involved in all the work and deserve all the credit for getting all of that working fine (well, plus Ravi, Sebastian and few others). As an user I also would like to join with thanks for work there. Best regards, Michal -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: musb - babble interrupt recovery
Hi Roger, thanks for those patches, actually i briefly approached them during my googling, but at first i didn't relate subject with Babble situation. Closely reading description about SUSPENDM bit behavior during resume makes sense. On 21.2.2014 13:03, Roger Quadros wrote: Could you please try the 2 patches posted here [1] for the babble problem, esp. patch 2. This is not about babble recovery but to prevent the babble from happening in the first place. I tried both patches separately on 3.14-rc3 and second patch alone helped here. I wasn't able to reproduce babble interrupt anymore, no matter, what i did with my hub and USB peripherals. Great! Best regards, Michal -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: v3.14-rc2+ WARNING: CPU: 0 PID: 24108 at fs/sysfs/group.c:216 sysfs_remove_group+0xa3/0xb0()
On 18. 2. 2014 21:51, Alan Stern wrote: On Tue, 18 Feb 2014, [windows-1252] Michal �mucr wrote: On 17.2.2014 20:48, Alan Stern wrote: This isn't a USB problem. Hello, I'm experiencing very similar messages after every disconnection of USB soundcard (snd-usb-audio module). During searching for other similar occurrences of message, i've found few patches. One kind basically prevents sysfs from popping those messages at sysfs group attribute removal - https://lkml.org/lkml/2013/11/23/102 and isn't included in 3.14-rc2, other kind of patches, if i got that correctly, fixes removal ordering or its visibility at underlying subsystem (eg. SCSI, PCI for Thunderbolt hotplugging) to make sysfs happy. Do you know, that there was some general decision, to intentionally don't modify sysfs and fix that for every subsystem, which triggers that messages? Yes, there was indeed such a decision. Just asking, because then it would be probably nice to modify something also at USB audio.. my dmesg with WARNING: http://vpub.smucr.cz/pub/bbb/debug/dmesg-reconnections.log and i probably not alone: http://forums.funtoo.org/viewtopic.php?pid=10327 If you want to pursue this, you should contact the alsa-devel mailing list and the maintainers of the Sound subsystem. Alan Stern Hello Alan, thank you for information. Best regards, Michal Smucr -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: v3.14-rc2+ WARNING: CPU: 0 PID: 24108 at fs/sysfs/group.c:216 sysfs_remove_group+0xa3/0xb0()
On 17.2.2014 9:41, Ronald wrote: I caught this by coincidence since I had netconsole attached to debug a nouveau MSI problem (deadlock). It happenned when I disconnected my external hardrive in thunar. I have attached my full dmesg. Sorry for third time, now without HTML, I've played with latest kernels yesterday and found the same messages after disconnection of one my USB soundcard. After some seeking at lists and testing, i believe this is inconvenient (i originally thought it is related to other problem, which i debugged), but harmless message. According to other threads, where was report, that this popped after Thuderbolt device removal. Error is printed during removal of device sysfs entries, which don't exist at that moment, because its parent sysfs folder is already deleted, if i got that correctly. relevant threads: https://bugzilla.kernel.org/show_bug.cgi?id=65281 http://www.spinics.net/lists/linux-scsi/msg70154.html http://forums.funtoo.org/viewtopic.php?pid=10327 https://lkml.org/lkml/2013/11/23/102 There was patch in latest mentioned thread by Mika Westerberg, but it is not possible to apply to 3.14 as there was some additional rework done at sysfs. Best regards, Michal -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: v3.14-rc2+ WARNING: CPU: 0 PID: 24108 at fs/sysfs/group.c:216 sysfs_remove_group+0xa3/0xb0()
On 17.2.2014 20:48, Alan Stern wrote: This isn't a USB problem. Hello, I'm experiencing very similar messages after every disconnection of USB soundcard (snd-usb-audio module). During searching for other similar occurrences of message, i've found few patches. One kind basically prevents sysfs from popping those messages at sysfs group attribute removal - https://lkml.org/lkml/2013/11/23/102 and isn't included in 3.14-rc2, other kind of patches, if i got that correctly, fixes removal ordering or its visibility at underlying subsystem (eg. SCSI, PCI for Thunderbolt hotplugging) to make sysfs happy. Do you know, that there was some general decision, to intentionally don't modify sysfs and fix that for every subsystem, which triggers that messages? Just asking, because then it would be probably nice to modify something also at USB audio.. my dmesg with WARNING: http://vpub.smucr.cz/pub/bbb/debug/dmesg-reconnections.log and i probably not alone: http://forums.funtoo.org/viewtopic.php?pid=10327 Best regards, Michal Smucr -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
musb - high CPU load in DMA mode and dropouts during audio playback
Hello, i'm fighting with audio playback problem on Beaglebone Black AM-3358. Class compilant USB soundcards with isochronous transfers. With DMA enabled in kernels 3.13.2 and 3.14-rc2 it has following symptoms. - it will stutter immediately after starts of playback - kworker thread shows about 80-90% CPU usage during playback - it presents with cppi41_dma_control warnings and traces, plus couple of transfer errors (i've enabled also CONFIG_DMADEVICES_DEBUG, so it dmesg is quite lengthy) http://vpub.smucr.cz/pub/bbb/debug/dmesg.log With force PIO at any tested kernel from 3.8.x to 3.14.x and DMA mode at 3.12.9 it is better, CPU load seems to be much lower, but it is not completely solved. Still during high interrupt load from other devices like NIC or second USB port, i'm having audio dropouts. Without any messages this time. I tested to apply following George Cherian's related patches http://marc.info/?l=linux-usbm=139081552015728w=2 usb: musb: musb_host: Enable ISOCH IN handling for AM335x host usb: musb: musb_cppi41: Make CPPI aware of high bandwidth transfers usb: musb: musb_cppi41: Handle ISOCH differently and not use the hrtimer and Adam Wozniak's http://marc.info/?l=linux-usbm=139101388408467w=2 high cpu load on omap3 using musb But it haven't changed anything for that issue. Thank you for help, Michal -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
musb - babble interrupt recovery
Hello, during searching for solution to problem with Beaglebone Black USB unreliable reconnections, which i had, when i tried to reconnect devices to external powered hub, I came to that older patch from Ravi Babu, which restarts musb after babble interrupt, https://lkml.org/lkml/2013/5/29/247 To be honest, I don't exactly know, why that situation happens on SoC and if is it hardware related, but it was working well for me and solved that omnipresent issue. Patch isn't applied to current kernels and i believe it will be very helpful to either use it or provide similar mechanism for recovery. Best regards, Michal Smucr -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
