I wouldn't consider the shaders too complex, though I'm not familiar
with how complex shaders generally are. Below is an example of a
shader that demonstrates the problem.
My goal is well under 60 frames. Since the camera only delivers
around ~20-25 frames, and I'm generally happy if I can pull more than
15 FPS out of my various. The below shader gets around 9 frames per
second, but I don't mind the framerate drop when I'm doing complex
operations.
The real problem is the out of order frame, visually it is a big a
problem. It seems to flicker 3 or 4 times at a frequency of around
once per second or so before it finally clears up. It is easiest to
see by moving the camera quickly from something dark to light. Though
it is noticeable even for small movements as the video will have an
odd jerkiness to it.
Example cell shader:
#extension GL_OES_EGL_image_external : require
precision mediump float;
varying vec2 outTexCoords;
uniform samplerExternalOES texture;
void main(void) {
vec4 color = texture2D(texture, outTexCoords);
color.r = (sign(color.r*8.0)*floor(abs(color.r*8.0)+0.5))/4.0;
color.g = (sign(color.g*8.0)*floor(abs(color.g*8.0)+0.5))/4.0;
color.b = (sign(color.b*8.0)*floor(abs(color.b*8.0)+0.5))/4.0;
const float threshold = 0.05;
vec3 rgb2lum = vec3(0.30, 0.59, 0.11);
float lum0 = dot(texture2D(texture, outTexCoords +
0.2*vec2(-0.0078125, 0.0078125)).rgb, rgb2lum);
float lum1 = dot(texture2D(texture, outTexCoords +
0.2*vec2( 0.00 , 0.0078125)).rgb, rgb2lum);
float lum2 = dot(texture2D(texture, outTexCoords +
0.2*vec2( 0.0078125, 0.0078125)).rgb, rgb2lum);
float lum3 = dot(texture2D(texture, outTexCoords +
0.2*vec2(-0.0078125, 0.00)).rgb, rgb2lum);
float lum4 = dot(texture2D(texture, outTexCoords +
0.2*vec2( 0.0, 0.0)).rgb, rgb2lum);
float lum5 = dot(texture2D(texture, outTexCoords +
0.2*vec2( 0.0078125, 0.007 )).rgb, rgb2lum);
float lum6 = dot(texture2D(texture, outTexCoords +
0.2*vec2(-0.0078125,-0.0078125)).rgb, rgb2lum);
float lum7 = dot(texture2D(texture, outTexCoords +
0.2*vec2( 0.00 , -0.0078125)).rgb, rgb2lum);
float lum8 = dot(texture2D(texture, outTexCoords +
0.2*vec2( 0.0078125,-0.0078125)).rgb, rgb2lum);
float x = lum2+ lum8+2.0*lum5-lum0-2.0*lum3-lum6;
float y = lum6+2.0*lum7+ lum8-lum0-2.0*lum1-lum2;
float edge = sign(max(threshold - (x*x + y*y),0.0));
color.rgb *= edge;
gl_FragColor = vec4(color.rgb, 1.0);
}
On Jan 10, 2:43 pm, Romain Guy <[email protected]> wrote:
> It sounds like the issue might be your fragment shader. How complex is
> it? On today's mobile GPUs you cannot do that many operations per
> fragment at 60fps.
>
>
>
>
>
>
>
>
>
> On Tue, Jan 10, 2012 at 1:08 PM, Brad Grimm <[email protected]> wrote:
> > I have noticed an interesting bug when using TextureView with the
> > Camera.
>
> > If I setup a TextureView, pass it a valid OpenGL texture and pass that
> > into the camera, everything works great. I can then swap out the
> > pixel shader with a simple shader and all still works great. But if I
> > swap out the pixel shader with a more complex shader the framerate
> > drops and stuttering appears.
>
> > What seems to be happening is that a frame is getting processed out of
> > order. After some hypothesizing I wondered if the hardware was being
> > sent too much to do. After trying a number of things I found that if
> > I just slept the gl processing thread for around 75 milliseconds or so
> > the framerate jumped back up and the stuttering disappeared.
>
> > This is an okay hack for now, but I worry that value is just an
> > arbitrary guess based on my shader, and that it may change with other
> > shaders or other phones. My question is: Is it possible to know when
> > it is safe to send another frame down to OpenGL without over-taxing
> > the system? (Or is something else going on here?)
>
> > I have tried using glFinish, and glFlush and neither of these helped.
> > I wondered if that was because the camera uses samplerExternalOES and
> > somehow that effects it?
>
> > Note: If I run the same OpenGL code using a SurfaceView instead I
> > don't get any stuttering. Of course the performance isn't as good so
> > that may be the reason it doesn't.
>
> > Any help is appreciated.
>
> > Thanks,
> > Brad
>
> > --
> > You received this message because you are subscribed to the Google
> > Groups "Android Developers" group.
> > To post to this group, send email to [email protected]
> > To unsubscribe from this group, send email to
> > [email protected]
> > For more options, visit this group at
> >http://groups.google.com/group/android-developers?hl=en
>
> --
> Romain Guy
> Android framework engineer
> [email protected]
--
You received this message because you are subscribed to the Google
Groups "Android Developers" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/android-developers?hl=en
