Android Dev Intro - Some Concepts on Android Graphics Architecture

1. EGL and GLES


OpenGL ES defines an API for rendering graphics. It does not define a windowing 
system. To allow GLES to work on a variety of platforms, it is designed to be 
combined with a library that knows how to create and access windows through the 
operating system. The library used for Android is called EGL.


If you want to draw textured polygons, you use GLES calls; if you want to put 
your rendering on the screen, you use EGL calls.


Before you can do anything with GLES, you need to create a GL context. In EGL, 
this means creating an EGLContext and an EGLSurface. GLES operations apply to 
the current context, which is accessed through thread-local storage rather than 
passed around as an argument. This means you have to be careful about which 
thread your rendering code executes on, and which context is current on that thread.


The EGLSurface can be an off-screen buffer allocated by EGL (called a "pbuffer") 
or a window allocated by the operating system. EGL window surfaces are created 
with the eglCreateWindowSurface() call. It takes a "window object" as an argument, 
which on Android can be a SurfaceView, a SurfaceTexture, a SurfaceHolder, or a 
Surface -- all of which have a BufferQueue underneath. When you make this call, 
EGL creates a new EGLSurface object, and connects it to the producer interface of 
the window object's BufferQueue. From that point onward, rendering to that 
EGLSurface results in a buffer being dequeued, rendered into, and queued for use 
by the consumer. (The term "window" is indicative of the expected use, but bear 
in mind the output might not be destined to appear on the display.)


Only one EGLSurface can be associated with a Surface at a time -- you can have 
only one producer connected to a BufferQueue -- but if you destroy the EGLSurface 
it will disconnect from the BufferQueue and allow something else to connect.


A given thread can switch between multiple EGLSurfaces by changing what's 
"current." An EGLSurface must be current on only one thread at a time.


The public Surface class is implemented in the Java programming language. The 
equivalent in C/C++ is the ANativeWindow class, semi-exposed by the Android NDK. 
You can get the ANativeWindow from a Surface with the ANativeWindow_fromSurface() 
call. Just like its Java-language cousin, you can lock it, render in software, 
and unlock-and-post.


To create an EGL window surface from native code, you pass an instance of 
EGLNativeWindowType to eglCreateWindowSurface(). EGLNativeWindowType is just a 
synonym for ANativeWindow, so you can freely cast one to the other.


2. Surface and SurfaceHolder


The Surface class has been part of the public API since 1.0. Its description 
simply says, "Handle onto a raw buffer that is being managed by the screen 
compositor." The statement was accurate when initially written but falls well 
short of the mark on a modern system.


The Surface represents the producer side of a buffer queue that is often 
(but not always!) consumed by SurfaceFlinger. When you render onto a Surface, 
the result ends up in a buffer that gets shipped to the consumer. A Surface 
is not simply a raw chunk of memory you can scribble on.


Some things that work with Surfaces want a SurfaceHolder, notably SurfaceView. 
The original idea was that Surface represented the raw compositor-managed buffer, 
while SurfaceHolder was managed by the app and kept track of higher-level information 
like the dimensions and format.


3. UI Views and SurfaceView
All visible View objects are rendered to a SurfaceFlinger-created Surface that 
was set up by the WindowManager when the app was brought to the foreground. The 
layout and rendering is performed on the app's UI thread. Regardless of how many 
Layouts and Views you have, everything gets rendered into a single buffer. This 
is true whether or not the Views are hardware-accelerated.


When the SurfaceView's View component is about to become visible, the framework 
asks the WindowManager to ask SurfaceFlinger to create a new Surface. (This doesn't 
happen synchronously, which is why you should provide a callback that notifies you 
when the Surface creation finishes.) By default, the new Surface is placed behind 
the app UI Surface, but the default "Z-ordering" can be overridden to put the 
Surface on top.


Whatever you render onto this Surface will be composited by SurfaceFlinger, not by 
the app. This is the real power of SurfaceView: the Surface you get can be rendered 
by a separate thread or a separate process, isolated from any rendering performed by 
the app UI, and the buffers go directly to SurfaceFlinger. You can't totally ignore 
the UI thread -- you still have to coordinate with the Activity lifecycle, and you 
may need to adjust something if the size or position of the View changes -- but you 
have a whole Surface all to yourself, and blending with the app UI and other layers 
is handled by the Hardware Composer.


It's worth taking a moment to note that this new Surface is the producer side of a 
BufferQueue whose consumer is a SurfaceFlinger layer. You can update the Surface 
with any mechanism that can feed a BufferQueue. You can: use the Surface-supplied 
Canvas functions, attach an EGLSurface and draw on it with GLES, and configure a 
MediaCodec video decoder to write to it.


4. SurfaceView and GLSurfaceView


The GLSurfaceView class provides some helper classes that help manage EGL contexts, 
inter-thread communication, and interaction with the Activity lifecycle. That's it. 
You do not need to use a GLSurfaceView to use GLES.


For example, GLSurfaceView creates a thread for rendering and configures an EGL 
context there. The state is cleaned up automatically when the activity pauses. 
Most apps won't need to know anything about EGL to use GLES with GLSurfaceView.


5. Surface and SurfaceTexture


The SurfaceTexture class is a relative newcomer, added in Android 3.0 ("Honeycomb"). 
Just as SurfaceView is the combination of a Surface and a View, SurfaceTexture is 
the combination of a Surface and a GLES texture. Sort of.


If you look closely at the API you'll see the only way for an application to create 
a plain Surface is through a constructor that takes a SurfaceTexture as the sole 
argument. (Prior to API 11, there was no public constructor for Surface at all.) 
This might seem a bit backward if you view SurfaceTexture as a combination of a 
Surface and a texture.


Under the hood, SurfaceTexture is called GLConsumer, which more accurately reflects 
its role as the owner and consumer of a BufferQueue. When you create a Surface from 
a SurfaceTexture, what you're doing is creating an object that represents the producer 
side of the SurfaceTexture's BufferQueue.


6. TextureView and SurfaceTexture


The TextureView class was introduced in Android 4.0 ("Ice Cream Sandwich"). It's the 
most complex of the View objects discussed here, combining a View with a SurfaceTexture.