I agree 100%! In the face of intra-container concurrency we should go in fully and find a solution that works even there.
Another slight wrinkle: Usually, the log-forwarder (whomever that may be) needs to know which container belongs to which user to namespace logs accordingly. We cannot really set that on the container itself, because there are pre-warm containers. The mapping of container -> userr is immutable though, since we don't reuse containers for different users (today). It would then be plausible to informer the log forwarder about the ContainerID -> user mapping to make it do the right thing. Note that this specific piece of information cannot really be part of the log's own context since the user must not be able to change it. @Tyson seems like you already put quite a bit of thought into this. Could you turn this into a proposal in itself to discuss seperately? Cheers, Markus -----Tyson Norris <[email protected]> wrote: ----- >To: "[email protected]" <[email protected]> >From: Tyson Norris <[email protected]> >Date: 07/20/2018 06:24PM >Subject: Re: Proposal on a future architecture of OpenWhisk > >On Logging, I think if you are considering enabling concurrent >activation processing, you will encounter that the only approach to >parsing logs to be associated with a specific activationId, is to >force the log output to be structured, and always include the >activationId with every log message. This requires a change at the >action container layer, but the simpler thing to do is to encourage >action containers to provide a structured logging context that action >developers can (and must) use to generate logs. > >An example is nodejs container - for the time being, we are hijacking >the stdout/stderr and injecting the activationId when any developer >code writes to stdout/stderr (as console.log/console.error). This may >not work as simply in all action containers, and isn’t great even in >nodejs. > >I would rather encourage action containers to provide a logging >context, where action devs use: log.info, log.debug, etc, and this >logging context does the needful to assert some structure to the log >format. In general, many (most?) languages have conventions (slf4xyz, >et al) for this already, and while you lose “random writes to >stdout”, I haven’t seen this be an actual problem. > >If you don’t deal with interleaved logs (typically because >activations don’t run concurrently), this this is less of an issue, >but regardless, writing log parsers is a solved problem that would >still be good to offload to external (not in OW controller/invoker) >systems (logstash, fluentd, splunk, etc). This obviously comes with a >caveat that logs parsing will be delayed, but that is OK from my >point of view, partly because most logs will never be viewed, and >partly because the log ingest systems are mostly fast enough already >to limit this delay to seconds or milliseconds. > >Thanks >Tyson
