villebro commented on issue #41854: URL: https://github.com/apache/superset/issues/41854#issuecomment-4956185563
@michael-s-molina thanks for expanding with the codec functionality. I didn't look deeply into the implementation yet, but I wonder if we're mixing up REST API transport layer and backend persistence encoding/decoding? Here's my understanding: - The REST API needs to distinguish between binary and non-binary transport. For this it does binary transport with bas64 to keep the content type simple. The backend then decodes the base64 encoded data if it was marked as binary. - When persisting binary data _in the backend_, base64 isn't actually needed at all - Pickle is only needed for persisting Python objects, but other than that, binary data can just be stored directly as-is. For this reason I'd recommend refining the design as follows: - Confine base64 in the REST API layer where it's actually used (maybe have a flag `isBase64` which tells the backend if it needs to decode before encoding with the codec). - Replace the base64 codec with one called `binary` or `bytes` or similar. - Later on, we can consider adding a `codecParams` to the REST API so one can pass along extra params that are passed to the codec. For instance, if we were to expose a Marshmallow codec to the frontend, one could pass the name of the Marshmallow schema via the params. This would make it possible to validate the schema of an object being persisted. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
