Hello strk,
Tuesday, September 19, 2006, 11:35:19 AM, you wrote:
s> A convenient way to write notes while stepping trough the code is
s> by using Doxygen comments
Ok, that could be the next step. However at the moment I try to make
an overview of the code...
s> Eh. For example, I don't like definitions to expose a 'display' method.
s> It should be the renderer having a display() method instead.
Ok, since we already started the discussion I attached the current
version of my notes to this mail. I hope it can be of some help - at
least being a basis for this discussion.
I'm not sure if I understand what you are actually trying to do but
giving the display() method to the renderer would mean reversing the
whole mechanism.
The display() method comes from the character_def class. Both a
MovieClip and shapes derive from it and they know how to draw
themselves. A MovieClip will draw it's own characters and the
child MovieClips. Characters know how to tell the renderer
what to draw. So the display() method of the root MC is used to call
all display() methods recursively and after all Flash is hierarchical.
It's the same thing how windowing systems work. You have a window,
that contains a frame which contains a label. They all have some
display() method.
The display() does actually more than just low level rendering.
s> I think all the intermediate representation only needed for rendering
s> should happen and be cached inside rendering classes, not character
s> definition.
Yes, that's what I thought too at first. However that would result
into a renderer that must organize a huge list of cache objects. To be
efficient it would have to associate cache objects to the shape
objects.
For example, say you have a simple movie with two sprites. Sprite A
does never change and sprite B continuously switches between 100% and
200% of it's own size.
That currently results in 1 cache object for sprite A and 2 cache
objects for sprite B (one mesh for each size factor). That's easy to
do when the cache resides in the sprite itself. How should the
renderer understand which cache object matches? He can't certainly
compare all edges...
Keep in mind that there are can be thousands of shapes which need to
be drawn in a frame.
>> server/parser/shape_character_def.cpp would be the place to look since
s> Ok, so you suggest that ActionScript code actually creates "definitions"
s> rather then "instances" ?
Uhm, are there character instances? I just know about MovieClip
definitions/instances..
s> Ok, so it does some sort of "optimization" for the SWF-defined shape,
s> correct ?
The tesselator is something very complex. I know what it does but
don't know how it does it as it is not documented. However, the author
states at the beginning of the tesselate.cpp file that the tesselator
is not perfect and that it does not need take care of some special
cases since the Flash IDE already simplifies shapes.
s> Maybe we won't have problems with this as the Drawing API
s> is very similar to the tags that define a shape (lineTo, moveTo, etc..)
I don't think so. Try this:
Draw a solid filled rectangle in the Flash IDE:
+--------------------------------+
|################################|
|################################|
|################################|
|################################|
|################################|
+--------------------------------+
Take the top side and drag it below the bottom side making a curve.
+ +
|. .|
|#.. ..#|
|###.. ..###|
|#####.. ..#####|
|#######.. ..#######|
+---------..----------..---------+
... ...
....
The left and right parts of the figure will be filled, the bottom part
won't (it has a "hole"). You will notice that the curve you just
created has been cut into three parts. You can stretch the middle part
without changing the outer parts. That means that Flash translated the
path to something that looks the same but in fact is drawn
differently. The problem is solved at design time (which can save
precious time when playing the movie).
If you draw this shape using ActionScript then the tesselator has to
deal with it. And I think this is where the various algorithms differ.
For example it could happen that the tesselator fills that hole.
A practical example would be the pentagram.
s> mm.. Note also that the Drawing API is a set of methods of the MovieClip
s> object (basically, sprites). So, we should be able to *add* lines and fills
s> to existing set of shapes (grouped into sprites). There's no concept of
s> *multiple* shape in ActionScript, it's like each MovieClip has an associated
s> *single* shape initially blank on which you can draw.
Exactly, and so it should not be difficult to implement it. lineTo()
and curveTo() would simply add items to the path and clear() would
delete it's contents. Of course each call needs to clear the cache to
make changes visible.
s> What about moving the decomposision and caching inside the renderer ?
See above...
s> Again, what about moving mesh computation inside the renderer ?
s> We might port the existing code into a specialized class which could
s> be used as it is or derived from by rendering classes.
s> The class would provide decomposition, meshing and cache.
Agree, but what about the cache? Misuse the shape objects as
containers for the renderer?
UdoGnash source code analysis.
An attempt to understand how Gnash works ;-)
Udo Giacomozzi, v0.1
Note I'm myself a beginner regarding Gnash and so some things mentioned here
may be wrong. Also, I'm writing this from the perspective of a future backend
developer, so I'll focus on that parts.
I use "may", "appears", "I think" ... terminology here because currently simly
everything is guessed in here. Once I get some parts confirmed I'll change this
so that well known parts can be distinguished.
NOTE: A quite nice intro to the SWF file format can be found at
http://www.the-labs.com/MacromediaFlash/SWF-Spec/SWFfileformat.html
A somewhat outdated but useful class reference can be found at
http://www.gnu.org/software/gnash/manual/doxygen/
Player
------
There seem to be two gnash.cpp, one in gui/ and one in backend/, the latter
being more than twice as big. However, it appears that the one in gui/ is the
correct one because there has been some redesign of the internal Gnash
architecture (support for multiple front-/backends).
gnash.cpp is used for both the standalone and the plugin mode.
First it uses the RcInitFile class from rc.cpp which checks the system for
"gnashrc" files. This file can tell gnash to set some verbosity level, log file,
security options, special settings and such. This may be useful when using
gnash as a plugin.
Then it goes on parsing the command line options which may override the
RcInitFile settings and finally find the SWF file name.
gnash.cpp registers some dummy callback function for the FSCommand function
(which is used to interact with the player or web browser scripting). The dummy
function does nothing except logging the call.
The sound handlers are then being intialized, but I won't go any deeper there
because it's currently far from my interest...
get_movie_info() (from server/impl.cpp) is then used to read the header of the
SWF file. The size of the stage, frame rate etc. are retrieved this way. The
function raises an exception when an error occurrs. Compressed SWF files seem
to be supported (using ZLIB).
There are "movie_width" and "movie_height" variables that contain the original
size of the movie, however they're already converted to pixels (SWF uses TWIPS,
where 1 pixel=20 twips).
Two additional variables, "width" and "height" contain the effective size of
the player window with the "scale" parameter already applied. This is a command
line option which sets the display ratio.
Now the "Gui" class (protected via auto_ptr) is initialized. The macro
GUI_CLASS is used to choose the correct implementing class (GtkGui, SDLGui,
KdeGui or whatever).
There is a special code fragment for the SDL GUI which calls a disableCoreTrap()
function - probably a quick hack not really belonging there?
It is possible to disable the renderer altogether via command line. In that case
a special "NullGui" (see gui/NullGui.cpp) instance is used.
The command line arguments are passed to the GUI, so that it may extract
additional settings from it (?).
The already-scaled width/height values and the SWF file name are passed to the
GUI so that it may initialize/prepare it's window. Just like the
Adobe/Macromedia SWF player the player tries to resize it's window to fit the
size of the movie.
create_library_movie() (from gui/gui.cpp) is then used to load the main SWF file
into a "movie_definition" class. There is a nice intro in
server/parser/movie_definition.h:
/// SWF Movies definitions are created by reading an SWF stream.
/// Gnash doesn't play SWF Movie definitions, but instances.
/// So you can play the same SWF file (Movie definiton) using
/// multiple instances.
///
/// A Movie definition is defined by the gnash::movie_definition class.
/// A Movie instance is defined by the gnash::movie_interface class.
///
/// A Movie instance exposes the ActionScript
/// Object base interface (gnash::as_object),
/// thus it can manage gnash::as_value members.
///
/// The implementation of SWF parsing for a Movie definition
/// is found in gnash::movie_def_impl::read.
/// Note that movie_definition is also used as a base class
/// to sprite_definition, which is a sub-movie defined in an SWF
/// file. This seems to be the only reason to have a
/// movie_def_impl class, being the top-level definition of
/// a movie (the one with a CharacterDictionary in it).
That instance==interface is created using create_library_movie_inst() right
after.
Then the command line is scanned for FlashVars that should be pre-initialized.
Using set_current_root() a global variable "s_current_root" is updated which
probably reflects the current root MC ("_root" in Flash). This variable may be
updated when an external movie is loaded and processed (via LoadMovieClip Flash
command I guess).
Viewport and background transparency are then reported to the GUI.
The "delay" is calculated which is the amount of theoretical milliseconds
between each frame (remember, the frame rate is fixed through the movie).
This delay is reported to the GUI using setCallback(), which may be renamed to
setDelay().
Using exit_timeout (-t parameter) it is possible to automatically exit the
player after some seconds (I guess for debugging purposes?).
Finally, "gui.run(app)" is called to start the whole thing.
The GUI
-------
I'll continue my analysis using the GTK GUI and the Cairo backend.
The class descendant is defined in gtksup.h and implemented in gtk.cpp. It
does all the necessary stuff to initialize a GTK window.
A class member named "glue" is the interface to the backend, in this case being
the GtkCairoGlue class. It is initialized in GtkGui::Init (again passing command
line parameters).
The "Gui" base class already defines a member called "_renderer" which is the
interface to the renderer class. The appropriate instance is created in
GtkGui::createWindow(), which in turn is called from within the main program.
The file "gtk_glue_cairo.cpp" apparently is the connection between the GUI and
the backend. It contains the actual code to *create* the renderer instance by
calling itself gdk_cairo_create(). It doesn't do much more useful and I did not
yet find the reason for the "glue" class (just to avoid a few IFDEFs?).
For clarity, here is the complete call stack for creating the renderer:
gui/gnash.cpp -> gui.createWindow(infile, width, height);
|
gui/gtk.cpp -> _renderer = glue.createRenderHandler();
|
gui/gtk_glue_cairo.cpp -> return create_render_handler_cairo(...);
|
backend/render_handler_cairo.cpp -> return new render_handler_cairo();
The frame delay (setCallback(), see above) is implemented using a GTK timer with
low priority that calls the method "advance_movie" from the base Gui class (see
gui/gui.cpp).
The GUI (in the "run" method) passes control to GTK via gtk_main(), which will
take care of displaying it's window and call the "advance_movie" method
periodically. It will not exit before the application should be terminated
(gtk_quit() probably).
advance_movie() itself advances the root movie (MC) by one frame. The parameter
of server/movie_instance.cpp:advance() is a float which I don't understand
why. There are a bunch of stacked advanceXXXX() calls but generally a frame
advace would always be 1.0. Maybe floats are required for tweening where it
would make sense, but I haven't checked this.
Anyway, movie_interface::advance() probably has the following (unchecked) tasks:
- update the display list (that is, what is visible on the screen) according
to the new frame
- apply tweening
- process ActionScript code
- define the current viewport?
It then calls display() of the root movie (movie_root class). That method calls
begin_display() of the renderer class and passes the current viewport, the
background color and the frame (=stage) size. The renderer normally clears the
framebuffer by filling the viewport with the background color.
Then the more generic display() method (of movie_interface, or more exactly
sprite_instance) is called, which draws the display list and finally
end_display() is told to the renderer class to commit the current frame.
More on the display() method in the next section!
Then gui::renderBuffer() is called, which again is located in gtk.cpp and passes
the call to glue::render(), which in the case of the Cairo backend does nothing.
This may be a function that can be used to swap a double buffer or tell
something to acceleration hardware but it seems that it's just a duplicate for
end_display(), except that the call goes through the GUI and so may be used to
blit the frame buffer on screen or something...
NOTE: This function probably could be very useful for the AGG backend. The
backend would render the frame the same way for any platform / WM combination.
The GUI then takes the finished frame buffer and blits it to the hardware
framebuffer (/dev/fb0), to some X window, or Windows GDI handle, or whatever.
This last step would fit well into renderBuffer() directly.
Rendering / Sprite instances
----------------------------
Back to the sprite_instance::display() method.
The sprite_instance class holds information for any existing (visible and
invisible) sprite on the stage. In the Flash world this would be a MovieClip
(MC) which itself can contain other MCs and stateful information such as
ActionScript variables. The root MC is basically a sprite too, just like any
MC has it's own timeline.
sprite_instance class inheritance:
ref_counted (server/ref_counted.h)
|
as_object (server/as_object.h)
|
movie_interface (server/movie_interface.h)
|
movie (server/movie.h)
|
character (server/character.h)
|
sprite_instance (server/sprite_instance.h)
The sprite contains a member "m_display_list" which is of type "DisplayList"
(defined in server/dlist.h). This is the display list of the sprite, that is,
what "characters" the sprite itself does contain. Each character has it's own
"depth" which defines the order of which the characters have to be displayed.
No two characters can be at the same depth.
sprite_instance::display() passes the call to display_list::display() which
goes through the list of characters and calls their own display() method.
Since MCs can contain mask layers the sprite may inform the renderer about it.
Basically, before a character of a mask is rendered (the mask "begins"), the
method renderer::begin_submit_mask() is called. Then normal rendering
operations follow until the mask is confirmed with renderer::end_submit_mask().
This enables the mask which remains active until render::disable_mask() is
called.
Say, we have a rectangle A partially masked by circle B and a triangle C that
does not have a mask (above it). This should work as follows:
1. renderer::begin_submit_mask()
2. circle B is drawn
3. renderer::end_submit_mask() --> the mask is now a circle
4. rectangle A is drawn, using the mask
5. render::disable_mask() --> mask not required anymore
6. triangle C is drawn normally
The way how the specific character (which may be a MC of it's own) is being
drawen depends on the character type (see following sections).
Outlines and shapes
-------------------
The file server/shape.h contains a few classes that are drawing primitives
(curves, shapes, ...). The'yre created while parsing the SWF file.
When a line or curve is read from the SWF file it is being handled by
server/parser/shape_character_def.cpp
It's read() method first loads the fill and line style. For a simple outline,
there would be no fill. Then it reads the actual shape records.
NOTE: Any shape (being it filled or not) begins in the SWF file with the
definiton of the fill styles, followed by the line styles and finally the
edges that define the shape itself. Along the edges path the previously
defined styles may be selected at any position.
Any shape consists of straight lines and/or curves.
So read():
- calls read_fill_styles() to read all the necessary fill styles of the shape
- calls read_line_styles() to do the same for the line styles
- reads the number of bits used for indexing fill and line styles later on
- loads the shape definition and stores it in the "current_path" variable
The current_path variable (of type "path") contains a list of objects of type
"edge". Straight lines are also stored as curves, whose control point equals
the anchor point. So, the path only contains a list of curves!
At the end the current_path is added to the class' "m_paths" vector.
The display() method of the class takes care that the path of the shape is
converted to a valid "mesh", that is a set of straight lines. This mesh is also
stored in some sort of cache so that it does not have to be re-calculated when
it's not necessary (important speed optimization).
Of course these straight lines are an approximation of the curves and so the
display() method keeps track of the error. If it raises a certain tolerance,
then the mesh needs to be recalculated. This may be necessary when the total
transformation matrix changes noticeably, for example when resizing a sprite.
The class "mesh_set" takes care of this (more on this later).
Regardless of whether the mesh has been loaded from cache or just recalculated,
it's own display() method is called to render it.
When new mashes have been calculated the complete cache is resorted so that
the best mashes (smallest error) are found at the beginning and unused mashes
are removed.
Mash sets / Tesselator
----------------------
Now, this is somewhat complicated...
Simple outlines (including curves) are converted to a list of straight lines
called "line strips". These line strips still have their style (width, color,
pattern) so it's up to the renderer to draw them correctly.
NOTE: Since straight lines are also "curves" here (see previous section),
they are still treated as such. [UNCHECKED] That means a simple line is split
into a number of smaller lines, each at the same angle. In fact, you won't
notice that it isn't a single line. This explains why the start-/endpoints of
these line sections have logarithmically increasing distances. This may be
something that can be optimized simply by comparing the control and anchor
points (when they're equal, then draw a single straight line and nothing
more).
The file parser/shape.cpp may be patched in method "path::tesselate()" using
add_line_segment() instead of add_curve_segment() when the is_straight()
function of the edge returns true.
Filled shapes, however, need to be converted into triangles so that they can
be rendered. The process of converting any polygon to a set of triangles is
called "tesselation". There are different tesselation algorithms and apparently
the tesselator used in GameSWF/Gnash is a class of it's own. However, for an
introduction see the Seidel's Algorithm at
http://www.cs.unc.edu/~dm/CODE/GEM/chapter.html
(Note I'm not a graphics specialist, I just did some research.)
Irrespective of the variant all tesselators work basically in the following way:
First the polygon is divided into trapezoids. Pratically they are
quadrilaterals with exactly horizontal upper and lower sides. Somewhat like
this:
-------------------------------
/ \
/ \
/ \
------------------------------------------
It may happen that the trapezoid is actually a triangle (when two corners of
the trapezoid are identical).
Each one of these trapezoids is split into two triangles by cutting it from the
upper right to the lower left or vice-versa.
-------------------------------
/ ...........\
/ ............ \
/........... \
------------------------------------------
It should not be complicated to fill a simple rectangle (at least with a solid
color).
NOTE: The Anti Grain Gravity (AGG) engine brings it's own tesselator, which
works very well (including anti-aliasing) so we should bypass the GameSWF
tesselator completely and let AGG do the work.
The "mesh_set" class (defined in server/shape.cpp) does the transformation
right in the constructor.
Inside the constructor, "collect_traps", a descendant of "trapezoid_acceptor",
is defined. The trapezoid acceptor is a class that accepts a number of
"trapezoid" instances.
Line strips (used for outlines) are simply copied to the mesh.
NOTE: The drawing of outlines is completely left up to the renderer. That's
strange because an outline actually is also a shape (think of a line that's
200 points thick). However, that's not a problem with AGG since it can draw
lines of all styles very well and after all that already works in the test
renderer.
Filled shapes, however, are converted to trapezoids, which are passed to a
"tri_stripper" class which simply generates the triangles out of them.
NOTE: The arguments to add_trapezoid() are simple the four points of the
trapezoid: upper left, upper right, lower left, lower right. Probably the
coordinates of the trapezoid can be upside-down.
The tri_stripper class can hold a list of triangles and is itself contained in
a list called "m_strips" that contains tri_strippers for each style used.
A mesh set is generated out of each tri_stripper (based on the triangles) which
is later fed to the renderer.
Drawing mesh sets
-----------------
The mesh set is given as-is to the renderer who is responsibe of actually
drawing them.
Refer to
- mesh_set::display()
- mesh::display()
-end of current version-
_______________________________________________
Gnash-dev mailing list
[email protected]
http://lists.gnu.org/mailman/listinfo/gnash-dev
