I want to start a discussion on how the Camera API could be substantially
improved regarding the performance of preview callbacks. As far as I am able
to tell (by browsing the git repository) no coding has been done to address
it.
The core of the problem is that the glue between the native Camera code and
the Java framework (in the form of android_hardware_Camera.cpp) allocates a
new Java array on each preview frame delivered to the PreviewCallback
supplied by the application. In short, this generates crazy amounts of
garbage and kills performance. A defect has been reported at:
http://code.google.com/p/android/issues/detail?id=2794
Having read some of the source code I remain ignorant as to why the API
operates in the way it does, and have assumed that the API is strongly
moulded by hardware/driver constraints. So I've tried to remain within
conservative (but guessed) boundaries for how the API can be adjusted.
My suggestion (which I acknowledge up-front might be unworkable since I
don't understand the constraints) is to expose a ByteBuffer via the Camera
class, together with a read lock, something like:
public ByteBuffer getPreviewData();
public Lock getPreviewLock();
The ByteBuffer would be backed directly by system memory that stores the
preview. Java applications wanting to use this data would have to:
* always obtain the lock before reading data from the buffer
* always explicitly release the lock after acquiring it
* accommodate whatever data encoding is used within the buffer
The application developer would essentially have two (and a half) choices:
1a) On acquiring the lock, copy the ByteBuffer into a Java array, release
the lock, process the preview. In this instance we are better off because
the developer is in control and knows how/when to reuse the Java array.
Excessive garbage collection can be avoided.
1b) As above except releasing the lock after processing has finished. This
would prevent the framework from supplying frame previews that the
application isn't ready to accept (see below).
2) Or, after acquiring the lock within the Java code, the ByteBuffer can
be passed to native code for processing, after which the Java application
would release the lock (though lock control could be performed by the native
code). This avoids mem-copying to the greatest extent possible and should
provide good performance.
The framework would attempt to take the lock before writing preview data to
the ByteBuffer and would release it immediately thereafter. If the lock were
already held (ie. by the application) then the preview frame would simply be
discarded. Discarding data in this way prevents applications from blocking
the system thread that is posting the preview frame. It also allows them to
signal back to the framework that they are not ready for more data, avoiding
unnecessary writes to the buffer.
I realize that this approach will probably force the framework to maintain
an extra preview buffer (since I assume that the camera driver writes
directly and unimpeded to the currently used buffer). For this reason, the
ByteBuffer would not be used by the framework, or made available to an
application, until a callback had been registered to indicate the
application's readiness to receive preview data in this way, something like:
public interface PreviewDataCallback {
void onPreviewFrameData(Camera camera);
}
with onPreviewFrameData() called after each write to the ByteBuffer.
In the event that an implementation of this (or something like it) can be
realized without an extra buffer, the calls to
PreviewCallback.onPreviewFrame() could then be driven from the ByteBuffer.
In either case, I can't see any reason why this improved API couldn't
coexist with the current implementation. But comments on its viability are
most welcome - I don't know enough to say if this suggestion is workable and
I'd love to see improvements in this area.
Tom.
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"android-framework" group.
To post to this group, send email to android-framework@googlegroups.com
To unsubscribe from this group, send email to
android-framework+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/android-framework?hl=en
-~----------~----~----~----~------~----~------~--~---
