Re: Logstore usage during `activation get`
Nick Mitchell wrote on 10/04/2018 12:06:14 PM: > > from a usability perspective, there is something quite attractive about > having ready access to logs (and activation records generally)... while > developing. i think most of the woes we encounter with logs come from high > volume actions (either exercised in prod, or during a high volume test)? Nick, I agree with this, but I think it is separable from the question of whether or not the logs are included in the activation API or if the user/tooling needs to make a separate call to a logs API. If we cleanly separated the two APIs, it might actually be easier to provide higher-quality implementations of both of them by using the appropriate backend technology for each. --dave > > i realize that this gets us into a broader topic, but it would be nice if > we could at least support ready access to say the last 1MB of activation > information per user. > > otherwise, we are sacrificing developer experience at the altar of service > provider bottom line. don't get me wrong, i am sympathetic with the > conundrum, but there are technical solutions that can satisfy both, if we > put our minds to it. > > nick > > On Thu, Oct 4, 2018 at 11:49 AM James W Dubee wrote: > > > Hey guys, > > > > For scalability concerns, I think it is preferable to separate logs from > > activation records entirely. Doing so would require users to use the logs > > API, or whatever tools are provided by the underly logging service. We > > should focus on serverless technology and delegate log handling to services > > that specialize in such areas. > > > > From experimenting with storing logs in a different data store other than > > where activations are stored, I've seen that there may be intermittent test > > failures for tests that rely on data being present in an activation > > record's log field. This may result when the activation record is retrieved > > before all the logs are stored in the separate log store. > > > > On a similar note, I don't think we should be storing activation responses > > at all. > > > > Regards, > > James Dubee > > > > > > > > [image: Inactive hide details for Rodric Rabbah ---10/02/2018 07:30:22 > > PM---> By "break this" do you mean at some point we should remov]Rodric > > Rabbah ---10/02/2018 07:30:22 PM---> By "break this" do you mean at some > > point we should remove the logs from the GET? > > > > From: Rodric Rabbah > > To: dev@openwhisk.apache.org > > Date: 10/02/2018 07:30 PM > > Subject: Re: Logstore usage during `activation get` > > -- > > > > > > > > > By "break this" do you mean at some point we should remove the logs from > > the GET? > > > > Yes. > > > > @dubee thoughts? Since you've worked on the elastic plugin. > > > > -r > > > > > > > >
Re: Logstore usage during `activation get`
"James W Dubee" wrote on 10/04/2018 11:45:34 AM: > From: "James W Dubee" > To: dev@openwhisk.apache.org > Date: 10/04/2018 11:49 AM > Subject: Re: Logstore usage during `activation get` > > On a similar note, I don't think we should be storing activation > responses at all. > +1 Responses can contain sensitive data; we should not be persisting them. --dave
Re: Logstore usage during `activation get`
from a usability perspective, there is something quite attractive about having ready access to logs (and activation records generally)... while developing. i think most of the woes we encounter with logs come from high volume actions (either exercised in prod, or during a high volume test)? i realize that this gets us into a broader topic, but it would be nice if we could at least support ready access to say the last 1MB of activation information per user. otherwise, we are sacrificing developer experience at the altar of service provider bottom line. don't get me wrong, i am sympathetic with the conundrum, but there are technical solutions that can satisfy both, if we put our minds to it. nick On Thu, Oct 4, 2018 at 11:49 AM James W Dubee wrote: > Hey guys, > > For scalability concerns, I think it is preferable to separate logs from > activation records entirely. Doing so would require users to use the logs > API, or whatever tools are provided by the underly logging service. We > should focus on serverless technology and delegate log handling to services > that specialize in such areas. > > From experimenting with storing logs in a different data store other than > where activations are stored, I've seen that there may be intermittent test > failures for tests that rely on data being present in an activation > record's log field. This may result when the activation record is retrieved > before all the logs are stored in the separate log store. > > On a similar note, I don't think we should be storing activation responses > at all. > > Regards, > James Dubee > > > > [image: Inactive hide details for Rodric Rabbah ---10/02/2018 07:30:22 > PM---> By "break this" do you mean at some point we should remov]Rodric > Rabbah ---10/02/2018 07:30:22 PM---> By "break this" do you mean at some > point we should remove the logs from the GET? > > From: Rodric Rabbah > To: dev@openwhisk.apache.org > Date: 10/02/2018 07:30 PM > Subject: Re: Logstore usage during `activation get` > -- > > > > > By "break this" do you mean at some point we should remove the logs from > the GET? > > Yes. > > @dubee thoughts? Since you've worked on the elastic plugin. > > -r > > > >
Re: Logstore usage during `activation get`
Hey guys, For scalability concerns, I think it is preferable to separate logs from activation records entirely. Doing so would require users to use the logs API, or whatever tools are provided by the underly logging service. We should focus on serverless technology and delegate log handling to services that specialize in such areas. >From experimenting with storing logs in a different data store other than where activations are stored, I've seen that there may be intermittent test failures for tests that rely on data being present in an activation record's log field. This may result when the activation record is retrieved before all the logs are stored in the separate log store. On a similar note, I don't think we should be storing activation responses at all. Regards, James Dubee From: Rodric Rabbah To: dev@openwhisk.apache.org Date: 10/02/2018 07:30 PM Subject:Re: Logstore usage during `activation get` > By "break this" do you mean at some point we should remove the logs from the GET? Yes. @dubee thoughts? Since you've worked on the elastic plugin. -r
