After some time of googling I found a very similar problem with AVX. It's discussed here:
http://software.intel.com/en-us/forums/topic/280592 and the problem is probably in mixing SSE and AVX instructions. According to Intel, there should be a VZEROALL or a VZEROUPPER instruction between every SSE <-> AVX transition. For more information look here: http://software.intel.com/en-us/articles/intel-software-development-emulator/#TRANSITION or here (page 103): http://www.agner.org/optimize/microarchitecture.pdf I don't expect Libav mixes AVX and SSE but since I link it from java, there is a chance java uses SSEs. So is it possible to surround Libav AVX code by the VZEROALL or the VZEROUPPER instruction? _______________________________________________ libav-api mailing list [email protected] https://lists.libav.org/mailman/listinfo/libav-api
