martin-g commented on issue #322: URL: https://github.com/apache/avro-rs/issues/322#issuecomment-3455239489
Hi, I don't have a good solution for you. The current stable version (0.20) is synchronous... At https://github.com/apache/avro-rs/pull/238 we are discussing how to add both sync and async APIs. https://github.com/apache/avro-rs/pull/246/ is a possible solution that provides async APIs and uses proc macros to remove the async part to produce sync APIs. You may give it a try, e.g. by using `tokio::spawn(async_writer.append(...))` but I have the feeling this will lead to corrupted avro file due to the interleaved writes to the writer. Another option is to use GenericSingleObjectWriter to produce the bytes for a single record in parallel and then append to a file behind a mutex or similar. > * Schema validation per-value appended is expensive - it would be really nice to have compile flags around it so it can be stripped out for production, or have a sampling rate attached to it to retain some runtime safety? You are the second person mentioning this - https://github.com/apache/avro-rs/issues/310#issuecomment-3387202633 Adding a feature flag to disable the validation is easy but risky. @Kriskras99 What do you think about this idea ? CC @mina-asham -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
