Hey Andy, However if Avro is both fast and has good support for nested structures > with optional fields, and we could come up with some scheme where some > marker indicates a field should get the last previous value seen (as > opposed to just being null), then it might not be so crazy. >
Avro does have good support for nested structures with optional fields. The "last previous value" scheme is outside the scope of Avro at this point. If you could (informally) write up what you would need from your serialization system for the HBase WAL and send it over to [email protected], we'd really appreciate the guidance and will do our best to include your needs on the roadmap. Also, FWIW, we've filed a ticket to use Avro serialization for the HDFS edits log at https://issues.apache.org/jira/browse/HDFS-842. Regards, Jeff
