Re: [fonc] iconic representations of powerful ideas
On 6/3/2012 8:31 PM, Shawn Morel wrote: I'm a very visual learner / thinker. I usually find it mentally painful (yes brow furrowing, headache inducing) to think of hard (distant) ideas until I can find an image in my mind's eye. Understood that not everyone thinks like this :) I guess I often think visually as well, though with both a lot of pictures and text (but, how does one really know for certain?...). I also tend to be a bit of a "concrete" thinker (or, a "sensing type" in psychology terms). I was re-reading the original NSF grant proposal, in particular after reading this passage: "Key to the tractability of this approach is the separation of the kernel into two complementary facets: representation of executable specifications (structures of message-passing objects) forming symbolic expressions and the meaning of those specifications (interpretation of their structure) that yields concrete behavior." I was gliding along the surface of a dynamically shifting Klein bottle. Curious what other people might think. personally I don't much understand the core goals of the project all that well either. I lurk some, and respond if something interesting shows up, and sometimes make a fool of myself in the process, but oh well... as well, it sometimes seems to me like maybe I am some sort of "generalized antagonist" for many people or something, at least given how many often pointless arguments seem to pop up (in general). but, thinking of visual things: I had recently looked over the SWF spec, and noticed that to some degree, at this level Flash looks a good deal like "some sort of animated photoshop-like thing" (both seem to be composed of stacks of layers and similar). or, at least, I found it kind of interesting. then was recently left dealing with the idea of systems being driven from the top-down, rather than how I am more familiar with them in games: basically as interacting finite-state-machines (although top-down wouldn't likely replace FSMs, but they could be used in combination). and, recently devised a hack for creating "component layered JPEG images", or, basically, a hack to allow creating JPEGs which also contained alpha-blending, normal maps, specular maps, and luma maps (as an essentially 16-component JPEG image composed of multiple component layers, with individual JPEG images placed end-to-end with marker tags between them to mark each layer). the main purpose was mostly though that I could have more advanced video-mapped surfaces (and, for the most part, I use MJPEG AVIs for these). there wasn't any other "clearly better" way. among other things... dunno if anyone would find any of this all that interesting though. ___ fonc mailing list fonc@vpri.org http://vpri.org/mailman/listinfo/fonc
Re: [fonc] iconic representations of powerful ideas
BGB wrote: and, recently devised a hack for creating "component layered JPEG images", or, basically, a hack to allow creating JPEGs which also contained alpha-blending, normal maps, specular maps, and luma maps (as an essentially 16-component JPEG image composed of multiple component layers, with individual JPEG images placed end-to-end with marker tags between them to mark each layer). dunno if anyone would find any of this all that interesting though. well, I'd certainly be interested in seeing that hack! Mile Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. Yogi Berra ___ fonc mailing list fonc@vpri.org http://vpri.org/mailman/listinfo/fonc
Re: [fonc] iconic representations of powerful ideas
On 6/4/2012 6:48 AM, Miles Fidelman wrote:
BGB wrote:
and, recently devised a hack for creating "component layered JPEG
images", or, basically, a hack to allow creating JPEGs which also
contained alpha-blending, normal maps, specular maps, and luma maps
(as an essentially 16-component JPEG image composed of multiple
component layers, with individual JPEG images placed end-to-end with
marker tags between them to mark each layer).
dunno if anyone would find any of this all that interesting though.
well, I'd certainly be interested in seeing that hack!
Mile Fidelman
from a comment in my JPEG code:
<--
BGB Extensions:
APP11: BGBTech Tag
ASCIZ TagName
Tag-specific data until next marker.
"AlphaColor":
AlphaColor
RGBA as string ("red green blue alpha").
APP11 markers may indicate component layer:
FF,APP11,"CompLayer\0",
"RGB": Base RGB
"XYZ": Normal XYZ (XZY ordering)
"SpRGB": Specular RGB
"DASe": Depth, Alpha, Specular-Exponent
"LuRGB": Luma RGB
"Alpha": Mono alpha layer
Component Layouts:
3 component: (no marker, RGB)
4 component: RGB+Alpha
7 component: RGB+Alpha+LuRGB
8 component: RGB+XYZ+DASe
12 component: RGB+XYZ+SpRGB+DASe
16 component: RGB+XYZ+SpRGB+DASe+LuRGB
-->
"AlphaColor" was an prior extension, basically for in-image chroma-keys.
the RGB color specifies the color to be matched, and A specifies how
"strongly" the color is matched (IIRC, it is the "distance" to Alpha=128
or so).
it was imagined that this could be calculated dynamically per-image, but
doing so is costly, so typically a fixed color is specified during
encoding (such as cyan or magenta).
"CompLayer" is the component layers.
currently, this tag precedes the SOI tages.
example:
FF,APP11, "CompLayer\0", "RGB\0"
FF,SOI
...
FF,EOI
FF,APP11, "CompLayer\0", "XYZ\0"
FF,SOI
...
FF,EOI
...
basically:
most component-layers are generic 4:2:0 RGB/YUV layers (except the mono
alpha layer, which is monochrome).
the layers may share the same Huffman and Quantization tables (by having
only the first layer encode them).
the RGB layer always comes first, so a decoder that doesn't know of the
extension, will just see the basic RGB components. also all layers are
the same resolution.
this is hardly an "ideal" design, but was intended more to allow a
simple encoder/decoder tweak to handle it (currently, it is
encoded/decoded by a function which may accept 4 RGBA buffers, and may
shuffle things around slightly to encode them into the layers).
the in-program layers are:
RGBA;
XYZD ('D' may be used for parallax mapping, and represents the relative
depth of the pixel);
Specular (RGBe), this basically gives the reflection color and shininess
of surface pixels;
Luma (RGBA).
so, yes, it is all a bit of a hack...
there was also a little fudging to the my AVI code to allow these videos
to be used for surface video-mapping (basically, the video is streamed
into all 4 layers at the same time).
example use-cases of something like this would likely be things like
making animated textures which resemble moving parts (such as metal
gears and blinking lights), or alternatively as a partial alternative to
using 3D modeled character faces (the face moving is really the texture
and animation frames, rather than 3D geometry), however presently this
is likely a better fit for animated textures than for video-maps.
___
fonc mailing list
fonc@vpri.org
http://vpri.org/mailman/listinfo/fonc
Re: [fonc] iconic representations of powerful ideas
BGB wrote: On 6/4/2012 6:48 AM, Miles Fidelman wrote: BGB wrote: and, recently devised a hack for creating "component layered JPEG images", or, basically, a hack to allow creating JPEGs which also contained alpha-blending, normal maps, specular maps, and luma maps (as an essentially 16-component JPEG image composed of multiple component layers, with individual JPEG images placed end-to-end with marker tags between them to mark each layer). dunno if anyone would find any of this all that interesting though. well, I'd certainly be interested in seeing that hack! Mile Fidelman from a comment in my JPEG code: <-- BGB Extensions: APP11: BGBTech Tag ASCIZ TagName Tag-specific data until next marker. Pardon my cluelessness here, but what exactly are you showing as "JPEG code?" Is this part of a JPEG file header, part of code that's generating a JPEG file, what? And... I don't suppose you have any examples of such files - either behind URLs, or for download? Thanks, Miles -- In theory, there is no difference between theory and practice. In practice, there is. Yogi Berra ___ fonc mailing list fonc@vpri.org http://vpri.org/mailman/listinfo/fonc
Re: [fonc] iconic representations of powerful ideas
On 6/4/2012 12:59 PM, Miles Fidelman wrote: BGB wrote: On 6/4/2012 6:48 AM, Miles Fidelman wrote: BGB wrote: and, recently devised a hack for creating "component layered JPEG images", or, basically, a hack to allow creating JPEGs which also contained alpha-blending, normal maps, specular maps, and luma maps (as an essentially 16-component JPEG image composed of multiple component layers, with individual JPEG images placed end-to-end with marker tags between them to mark each layer). dunno if anyone would find any of this all that interesting though. well, I'd certainly be interested in seeing that hack! Mile Fidelman from a comment in my JPEG code: <-- BGB Extensions: APP11: BGBTech Tag ASCIZ TagName Tag-specific data until next marker. Pardon my cluelessness here, but what exactly are you showing as "JPEG code?" Is this part of a JPEG file header, part of code that's generating a JPEG file, what? the "JPEG code" reference was, basically, to my JPEG encoder/decoder (which implements ITU T.81 and JFIF), and is written in C. the design shouldn't be too hard to retrofit onto a "standard" encoder though (such as "libjpeg"), given it doesn't significantly alter the file-format, but as-is I use my own JPEG implementation. in this case, basically it is implemented as wrapper functions, which essentially just daisy-chain to the normal JPEG encoder/decoder functions. but, anyways, to try to explain the JPEG format: basically, it consists mostly of "marker tags", defined as byte values. there is no particular "file header" in JPEG, only marker tags, and data which follows tags, and the relative positioning and ordering of these marker tags. the byte value 0xFF is "magic" in JPEG, and serves to escape various tags (given in the following byte). it may not be used directly as a data value in encoded data (it needs to be escaped). for example, "0xFF, 0x00" escapes an 0xFF byte. so, a few major tags. SOI (Start Of Image, 0xD8) and EOI (Start Of Image, 0xD9) are also major, as these mark the start and end of an encoded image (all other image-encoding markers exist between these tags). DHT (Define Huffman Table, 0xC4), defines the Huffman table. DQT (Define Quantization Table, 0xDB), defines the Huffman table. SOF0 (Start Of Frame 0, 0xC0), defines the resolution and, components, and sub-sampling of the image (for color images, always 3 components and 4:2:0 in JFIF, and the color-space is always YUV / YCbCr). SOS (Start Of Scan, 0xDA), marks the start of the compressed image data. and, in this case: APP0-APP15 (0xE0-0xEF), which define application-specific extension tags. these are commonly used to encode various extensions features. if a codec sees one it doesn't recognize, it will typically skip over it (skip until the next marker tag). APP0 is used by JFIF, and APP1 and APP2 are used by EXIF. APP3-APP15 are used by various other people for various extensions. I ended up using APP11 because, AFAICT, no one else was using it. the usual convention for using these tags is to have them followed by an ASCIIZ string, which is what I was doing. I also ended up also using an ASCIIZ string to encode parameters as well. but, all this means that the layer tag will come before the SOI tag. so, for example (leaving out 0x prefixes): FF, E0, xx, xx, "JFIF", 00, ...,//JFIF "header" (often optional) FF, EB, "CompLayer", 00, "RGB", 00,//layer marker FF, D8, ..., FF, D9,//encoded image FF, EB, "CompLayer", 00, "XYZ", 00, FF, D8, ..., FF, D9,//encoded image ... And... I don't suppose you have any examples of such files - either behind URLs, or for download? sadly, not as of yet. I only recently implemented it, and it is mostly intended for AVI videos (I don't currently use it for textures, which typically give all this information as multiple image files). given the way AVI works, there would be no good way to use multiple images per frame in an AVI (thus, the major reason for creating this). (note: OpenDML specifies something similar already for encoding interlaced MJPEG videos). even if files were available though, most likely only the basic RGB image would be visible to existing applications (the other layers would be ignored). Thanks, Miles ___ fonc mailing list fonc@vpri.org http://vpri.org/mailman/listinfo/fonc
Re: [fonc] iconic representations of powerful ideas
Thanks! BGB wrote: On 6/4/2012 12:59 PM, Miles Fidelman wrote: BGB wrote: On 6/4/2012 6:48 AM, Miles Fidelman wrote: BGB wrote: and, recently devised a hack for creating "component layered JPEG images", or, basically, a hack to allow creating JPEGs which also contained alpha-blending, normal maps, specular maps, and luma maps (as an essentially 16-component JPEG image composed of multiple component layers, with individual JPEG images placed end-to-end with marker tags between them to mark each layer). dunno if anyone would find any of this all that interesting though. well, I'd certainly be interested in seeing that hack! Mile Fidelman from a comment in my JPEG code: <-- BGB Extensions: APP11: BGBTech Tag ASCIZ TagName Tag-specific data until next marker. Pardon my cluelessness here, but what exactly are you showing as "JPEG code?" Is this part of a JPEG file header, part of code that's generating a JPEG file, what? the "JPEG code" reference was, basically, to my JPEG encoder/decoder (which implements ITU T.81 and JFIF), and is written in C. the design shouldn't be too hard to retrofit onto a "standard" encoder though (such as "libjpeg"), given it doesn't significantly alter the file-format, but as-is I use my own JPEG implementation. in this case, basically it is implemented as wrapper functions, which essentially just daisy-chain to the normal JPEG encoder/decoder functions. but, anyways, to try to explain the JPEG format: basically, it consists mostly of "marker tags", defined as byte values. there is no particular "file header" in JPEG, only marker tags, and data which follows tags, and the relative positioning and ordering of these marker tags. the byte value 0xFF is "magic" in JPEG, and serves to escape various tags (given in the following byte). it may not be used directly as a data value in encoded data (it needs to be escaped). for example, "0xFF, 0x00" escapes an 0xFF byte. so, a few major tags. SOI (Start Of Image, 0xD8) and EOI (Start Of Image, 0xD9) are also major, as these mark the start and end of an encoded image (all other image-encoding markers exist between these tags). DHT (Define Huffman Table, 0xC4), defines the Huffman table. DQT (Define Quantization Table, 0xDB), defines the Huffman table. SOF0 (Start Of Frame 0, 0xC0), defines the resolution and, components, and sub-sampling of the image (for color images, always 3 components and 4:2:0 in JFIF, and the color-space is always YUV / YCbCr). SOS (Start Of Scan, 0xDA), marks the start of the compressed image data. and, in this case: APP0-APP15 (0xE0-0xEF), which define application-specific extension tags. these are commonly used to encode various extensions features. if a codec sees one it doesn't recognize, it will typically skip over it (skip until the next marker tag). APP0 is used by JFIF, and APP1 and APP2 are used by EXIF. APP3-APP15 are used by various other people for various extensions. I ended up using APP11 because, AFAICT, no one else was using it. the usual convention for using these tags is to have them followed by an ASCIIZ string, which is what I was doing. I also ended up also using an ASCIIZ string to encode parameters as well. but, all this means that the layer tag will come before the SOI tag. so, for example (leaving out 0x prefixes): FF, E0, xx, xx, "JFIF", 00, ...,//JFIF "header" (often optional) FF, EB, "CompLayer", 00, "RGB", 00,//layer marker FF, D8, ..., FF, D9,//encoded image FF, EB, "CompLayer", 00, "XYZ", 00, FF, D8, ..., FF, D9,//encoded image ... And... I don't suppose you have any examples of such files - either behind URLs, or for download? sadly, not as of yet. I only recently implemented it, and it is mostly intended for AVI videos (I don't currently use it for textures, which typically give all this information as multiple image files). given the way AVI works, there would be no good way to use multiple images per frame in an AVI (thus, the major reason for creating this). (note: OpenDML specifies something similar already for encoding interlaced MJPEG videos). even if files were available though, most likely only the basic RGB image would be visible to existing applications (the other layers would be ignored). Thanks, Miles ___ fonc mailing list fonc@vpri.org http://vpri.org/mailman/listinfo/fonc -- In theory, there is no difference between theory and practice. In practice, there is. Yogi Berra ___ fonc mailing list fonc@vpri.org http://vpri.org/mailman/listinfo/fonc
