Re: musb - high CPU load in DMA mode and dropouts during audio playback
On 3/12/2014 5:53 AM, Michal Šmucr wrote: Hi George, I did few more tests and have further details for you. This time i modified my build procedure and according to Felipe's hint, I used ti-linux-3.12.y branch from TI repository and also clean 3.14-rc6 kernel, where i subsequently tried additional unmerged patches. With ti-linux-3.12.y I didn't notice any of CPU usage or stability, 80-90%, dropouts. Latest kernel behaved differently, CPU usage is low and audio after DAC is total rubbish (constant level noise). After playing with additional patches, i've found, that modifications in your third patch for different isochronous packet handling https://lkml.org/lkml/2014/1/24/187 causes high load. So it looks like, without this patch, hrtimer trick for compensation of early interrupt doesn't work and samples gets garbled during transfers (albeit i don't have HW USB analyzer to prove it). Alternative with fifo checking and workqueue, unfortunately hogs CPU. The basic issue is we get the DMA interrupt very early that it takes some time for the packet to flow out from the TX Fifo. In some cases we end up re-scheduling the workqueue multiple times since the TX FIFO is NYET empty. Alternatively do you have any reason not to try out the PIO mode? Thanks, Michal On 27.2.2014 12:18, Michal Šmucr wrote: Hi George, On 27.2.2014 5:15, George Cherian wrote: I too see the backtraces will send a patch soon to fix the same. Thanks, i applied it and most of backtraces was suppressed. Several times i was able to invocate it again, but this time, it was before complete stuck of playback application (kill -9 was only way to end it). From that moment, no other audio playback can be initiated until next reboot. If i tried to remedy situation by replugging of USB interface, it leaded to nice Oops (snippet from serial console is attached). During my testing am not seeing the CPU usage as you mention. That is interesting, i was able to reproduce it whenever i tried that, maybe it could be also some config issue. Just for more complete info, i'm using build scripts by Robert C. Nelson and there are couple other patches applied to kernel before build (now including your previous ISOCH. handling ones) - https://github.com/RobertCNelson/linux-dev/tree/am33x-v3.14/patches , but it doesn't seems related to issue. Could you please share you .config. You'll find it attached. I used aplay for testing playback and arecord for recording using USB headsets. I tried mpd as player, but i'm able to reproduce kworker load also with aplay or alsa built-in speaker-test (eg. speaker-test -c 2 -D plughw:0,0). Regarding hardware i tried few XMOS based UAC2 interfaces and recently also interface with TI PCM2904 USB codec, which could be very close to headset, you tried. It doesn't matter according to my tests. Thanks, Michal -- -George -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: musb - high CPU load in DMA mode and dropouts during audio playback
Hi George, On 12.3.2014 12:15, George Cherian wrote: So it looks like, without this patch, hrtimer trick for compensation of early interrupt doesn't work and samples gets garbled during transfers (albeit i don't have HW USB analyzer to prove it). Alternative with fifo checking and workqueue, unfortunately hogs CPU. The basic issue is we get the DMA interrupt very early that it takes some time for the packet to flow out from the TX Fifo. In some cases we end up re-scheduling the workqueue multiple times since the TX FIFO is NYET empty. It seems, that rescheduling definitely helped to avoid data corruption. Alternatively do you have any reason not to try out the PIO mode? I already tried that at the beginning of my test as first workaround to remedy audio interface issues. It is best, what i achieved with Sitara SoC, but it is not stable when system is loaded with other things. Practically that means, audio is playing, but when there is for instance song database update from player app, which involves additional network transfers and reading of song tags, it starts to stutter badly. This get worse, when playing high resolution audio (eg. 192k/24bit). So to sum it up, it is not really usable from my tests. Best regards, Michal -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: musb - high CPU load in DMA mode and dropouts during audio playback
Hi George, I did few more tests and have further details for you. This time i modified my build procedure and according to Felipe's hint, I used ti-linux-3.12.y branch from TI repository and also clean 3.14-rc6 kernel, where i subsequently tried additional unmerged patches. With ti-linux-3.12.y I didn't notice any of CPU usage or stability, 80-90%, dropouts. Latest kernel behaved differently, CPU usage is low and audio after DAC is total rubbish (constant level noise). After playing with additional patches, i've found, that modifications in your third patch for different isochronous packet handling https://lkml.org/lkml/2014/1/24/187 causes high load. So it looks like, without this patch, hrtimer trick for compensation of early interrupt doesn't work and samples gets garbled during transfers (albeit i don't have HW USB analyzer to prove it). Alternative with fifo checking and workqueue, unfortunately hogs CPU. Thanks, Michal On 27.2.2014 12:18, Michal Šmucr wrote: Hi George, On 27.2.2014 5:15, George Cherian wrote: I too see the backtraces will send a patch soon to fix the same. Thanks, i applied it and most of backtraces was suppressed. Several times i was able to invocate it again, but this time, it was before complete stuck of playback application (kill -9 was only way to end it). From that moment, no other audio playback can be initiated until next reboot. If i tried to remedy situation by replugging of USB interface, it leaded to nice Oops (snippet from serial console is attached). During my testing am not seeing the CPU usage as you mention. That is interesting, i was able to reproduce it whenever i tried that, maybe it could be also some config issue. Just for more complete info, i'm using build scripts by Robert C. Nelson and there are couple other patches applied to kernel before build (now including your previous ISOCH. handling ones) - https://github.com/RobertCNelson/linux-dev/tree/am33x-v3.14/patches , but it doesn't seems related to issue. Could you please share you .config. You'll find it attached. I used aplay for testing playback and arecord for recording using USB headsets. I tried mpd as player, but i'm able to reproduce kworker load also with aplay or alsa built-in speaker-test (eg. speaker-test -c 2 -D plughw:0,0). Regarding hardware i tried few XMOS based UAC2 interfaces and recently also interface with TI PCM2904 USB codec, which could be very close to headset, you tried. It doesn't matter according to my tests. Thanks, Michal -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: musb - high CPU load in DMA mode and dropouts during audio playback
Hi George, On 27.2.2014 5:15, George Cherian wrote: I too see the backtraces will send a patch soon to fix the same. Thanks, i applied it and most of backtraces was suppressed. Several times i was able to invocate it again, but this time, it was before complete stuck of playback application (kill -9 was only way to end it). From that moment, no other audio playback can be initiated until next reboot. If i tried to remedy situation by replugging of USB interface, it leaded to nice Oops (snippet from serial console is attached). During my testing am not seeing the CPU usage as you mention. That is interesting, i was able to reproduce it whenever i tried that, maybe it could be also some config issue. Just for more complete info, i'm using build scripts by Robert C. Nelson and there are couple other patches applied to kernel before build (now including your previous ISOCH. handling ones) - https://github.com/RobertCNelson/linux-dev/tree/am33x-v3.14/patches , but it doesn't seems related to issue. Could you please share you .config. You'll find it attached. I used aplay for testing playback and arecord for recording using USB headsets. I tried mpd as player, but i'm able to reproduce kworker load also with aplay or alsa built-in speaker-test (eg. speaker-test -c 2 -D plughw:0,0). Regarding hardware i tried few XMOS based UAC2 interfaces and recently also interface with TI PCM2904 USB codec, which could be very close to headset, you tried. It doesn't matter according to my tests. Thanks, Michal config-3.14.0-rc4-bone0.gz Description: application/gzip [ 176.322424] usb 2-1: USB disconnect, device number 2 [ 179.654157] usb 2-1: new high-speed USB device number 3 using musb-hdrc [ 179.795133] usb 2-1: device v249c p930b is not supported [ 179.800758] usb 2-1: New USB device found, idVendor=249c, idProduct=930b [ 179.807928] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3 [ 179.815489] usb 2-1: Product: M2Tech USB Audio 2.0 [ 179.820540] usb 2-1: Manufacturer: M2Tech [ 179.824901] usb 2-1: SerialNumber: [ 329.707680] systemd-logind[2285]: New session 2 of user root. [ 671.601671] usb 2-1: USB disconnect, device number 3 [ 671.613917] Unable to handle kernel paging request at virtual address ffe6 [ 671.621545] pgd = c0004000 [ 671.624400] [ffe6] *pgd=9fef6821, *pte=, *ppte= [ 671.631045] Internal error: Oops: 17 [#1] SMP ARM [ 671.635995] Modules linked in: usb_f_acm u_serial usb_f_ecm g_multi usb_f_mass_storage usb_f_rndis u_ether libcomposite rpcsec_gss_krb5 nfsd snd_usb_audio snd_usbmidi_lib snd_hwdep snd_seq_midi snd_seq_midi_event snd_rawmidi snd_pcm snd_seq snd_seq_device snd_timer omap_aes snd omap_sham soundcore ti_am335x_adc kfifo_buf industrialio rtc_omap uio_pdrv_genirq uio [ 671.670042] CPU: 0 PID: 19 Comm: khubd Not tainted 3.14.0-rc4-bone0 #3 [ 671.676914] task: de0bf200 ti: de176000 task.ti: de176000 [ 671.682622] PC is at musb_g_tx+0x9c/0x18c [ 671.686856] LR is at cppi41_trans_done+0x48/0x134 [ 671.691814] pc : [c0515354]lr : [c05165c4]psr: a00f0193 [ 671.691814] sp : de177c38 ip : 0019 fp : c0b0ca00 [ 671.703880] r10: de01eef8 r9 : 200f0193 r8 : de01e010 [ 671.709378] r7 : de01eeb0 r6 : 3400 r5 : e0878de0 r4 : ffcc [ 671.716242] r3 : r2 : 0001 r1 : e0878de2 r0 : de01e010 [ 671.723109] Flags: NzCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment kernel [ 671.730887] Control: 10c5387d Table: 9e7a0019 DAC: 0015 [ 671.736931] Process khubd (pid: 19, stack limit = 0xde176248) [ 671.742976] Stack: (0xde177c38 to 0xde178000) [ 671.747567] 7c20: de027944 de01ee84 [ 671.756179] 7c40: dd281000 de01e010 0080 200f0193 008a c05165c4 0003 [ 671.764790] 7c60: 0001 c0166368 800f0093 dd399790 dd3a2b38 de027944 de01ee84 dd281000 [ 671.773401] 7c80: de01e010 0080 200f0193 c0516c44 de58ea80 c0074280 dd554580 00029903 [ 671.782009] 7ca0: 00029903 0004 dd281000 9e841720 0080 0001 [ 671.790620] 7cc0: 008a c040ad88 dd2a2e80 de005900 0021 [ 671.799230] 7ce0: a00f0013 c008037c de005900 dd2a2e80 1f065000 de005900 c0a741e4 [ 671.807842] 7d00: c0b11be4 0001 a00f0013 dd39 c0080550 0002 de005900 [ 671.816452] 7d20: c0a741e4 c0082fcc 0021 c007fbe4 0021 c000e244 fa20 de177d60 [ 671.825064] 7d40: 0021 c0008578 c0511c6c c06df880 a00f0013 de177d94 c06dffc0 [ 671.833675] 7d60: de01e010 a00f0013 34a6 747e de590080 de5a8800 de01e010 8300 [ 671.842286] 7d80: 0001 a00f0013 dd39 e0878de2 de177da8 c0511c6c c06df880 [ 671.850897] 7da0: a00f0013 de5a8800 ff98 a00f0013 de6ed3ec de6ed3f0 [ 671.859508] 7dc0: de62a800 c04f9d10 de26c200 de6ec000 de6ec000 13e8 bf0cc4bc [ 671.868118] 7de0:
Re: musb - high CPU load in DMA mode and dropouts during audio playback
On 18.2.2014 2:18, Michal Šmucr wrote: Hello, i'm fighting with audio playback problem on Beaglebone Black AM-3358. Class compilant USB soundcards with isochronous transfers. With DMA enabled in kernels 3.13.2 and 3.14-rc2 it has following symptoms. - it will stutter immediately after starts of playback - kworker thread shows about 80-90% CPU usage during playback - it presents with cppi41_dma_control warnings and traces, plus couple of transfer errors (i've enabled also CONFIG_DMADEVICES_DEBUG, so it dmesg is quite lengthy) http://vpub.smucr.cz/pub/bbb/debug/dmesg.log With force PIO at any tested kernel from 3.8.x to 3.14.x and DMA mode at 3.12.9 it is better, CPU load seems to be much lower, but it is not completely solved. Still during high interrupt load from other devices like NIC or second USB port, i'm having audio dropouts. Without any messages this time. I tested to apply following George Cherian's related patches http://marc.info/?l=linux-usbm=139081552015728w=2 usb: musb: musb_host: Enable ISOCH IN handling for AM335x host usb: musb: musb_cppi41: Make CPPI aware of high bandwidth transfers usb: musb: musb_cppi41: Handle ISOCH differently and not use the hrtimer and Adam Wozniak's http://marc.info/?l=linux-usbm=139101388408467w=2 high cpu load on omap3 using musb I also tried to find something related in TI git tree and compile linux-next, but without luck. Any tips, what to do next? Does anybody experienced same issue? Thanks, Michal -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: musb - high CPU load in DMA mode and dropouts during audio playback
Hi Michal, On 2/26/2014 7:07 PM, Michal Šmucr wrote: On 18.2.2014 2:18, Michal Šmucr wrote: Hello, i'm fighting with audio playback problem on Beaglebone Black AM-3358. Class compilant USB soundcards with isochronous transfers. what is the application you are using for play back. With DMA enabled in kernels 3.13.2 and 3.14-rc2 it has following symptoms. - it will stutter immediately after starts of playback - kworker thread shows about 80-90% CPU usage during playback - it presents with cppi41_dma_control warnings and traces, plus couple of transfer errors (i've enabled also CONFIG_DMADEVICES_DEBUG, so it dmesg is quite lengthy) http://vpub.smucr.cz/pub/bbb/debug/dmesg.log I too see the backtraces will send a patch soon to fix the same. During my testing am not seeing the CPU usage as you mention. Could you please share you .config. I used aplay for testing playback and arecord for recording using USB headsets. With force PIO at any tested kernel from 3.8.x to 3.14.x and DMA mode at 3.12.9 it is better, CPU load seems to be much lower, but it is not completely solved. Still during high interrupt load from other devices like NIC or second USB port, i'm having audio dropouts. Without any messages this time. I tested to apply following George Cherian's related patches http://marc.info/?l=linux-usbm=139081552015728w=2 usb: musb: musb_host: Enable ISOCH IN handling for AM335x host usb: musb: musb_cppi41: Make CPPI aware of high bandwidth transfers usb: musb: musb_cppi41: Handle ISOCH differently and not use the hrtimer and Adam Wozniak's http://marc.info/?l=linux-usbm=139101388408467w=2 high cpu load on omap3 using musb I also tried to find something related in TI git tree and compile linux-next, but without luck. Any tips, what to do next? Does anybody experienced same issue? Thanks, Michal -- -George -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
