[jira] [Commented] (SLING-9389) Distribution Event Packages should contain queue item creation time
[ https://issues.apache.org/jira/browse/SLING-9389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17125354#comment-17125354 ] Timothee Maret commented on SLING-9389: --- bq. The idea is to capture a metric (using Sling Metrics) exposing the end to end distribution behavior (AEM to BP in this context). [~harshchiki] I think we should do exactly what your requirement describes: add Sling metrics in the implementations to monitor end to end latency. I think that we could achieve that without extending the event API. It's possible to dissociate metrics by agent, use case, etc. See for instance the [Bookeeper|https://github.com/apache/sling-org-apache-sling-distribution-journal/blob/4190c2befc6da8277e12ba019b8c41087cb217e1/src/main/java/org/apache/sling/distribution/journal/impl/subscriber/BookKeeper.java#L114-L115] which emits metrics by subscriber agent. bq. the information would not come down to the consumers of SCD apparently. It would come via the Sling metrics rather than the SCD API. Would that be satisfactory for your use case ? > Distribution Event Packages should contain queue item creation time > --- > > Key: SLING-9389 > URL: https://issues.apache.org/jira/browse/SLING-9389 > Project: Sling > Issue Type: Improvement > Components: Content Distribution >Reporter: Harsh Chiki >Assignee: Timothee Maret >Priority: Major > Attachments: image-2020-04-30-10-28-58-011.png, scdapi.patch, > scdcore.patch > > Time Spent: 1h 40m > Remaining Estimate: 0h > > Currently the Distribution Event package contains the following details: > * Distribution Component Name > * Distribution Component Kind > * Distribution Type > * Distribution Paths > > Improvement aims at adding the queue item creation time, essentially when the > the item was creation for the first time, and enqueue into the queue. The > value does not change over retries (on failure). > > The purpose to get this detail is to be able to capture metrics at the > consumer level. The consumers could have an event handler, which can capture > the duration which turns out to be (NOW MINUS queue item creation time thrown > in the distribution event package); NOW being the current time in the event > handler (consumer). > > \cc: [~shgu...@adobe.com], [~ashishc] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (SLING-9389) Distribution Event Packages should contain queue item creation time
[ https://issues.apache.org/jira/browse/SLING-9389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17124714#comment-17124714 ] Harsh Chiki commented on SLING-9389: Hi [~diru], While I'd revisit your suggestion (having quickly looked into it), the fundamental idea was to capture the end to end latency. In SCD Core, this (AFAIU) can best be captured from the time the distribution queue item was "created" (stored in repo) until it finally succeeds. May I request you to indicate any gaps in my understanding in achieving the aim (per the description), and if in general what you suggested can be helpful for some other use case (something like if one wants to monitoring and capture metrics - stage by stage the flow in SCD) > Distribution Event Packages should contain queue item creation time > --- > > Key: SLING-9389 > URL: https://issues.apache.org/jira/browse/SLING-9389 > Project: Sling > Issue Type: Improvement > Components: Content Distribution >Reporter: Harsh Chiki >Assignee: Timothee Maret >Priority: Major > Attachments: image-2020-04-30-10-28-58-011.png, scdapi.patch, > scdcore.patch > > Time Spent: 1h 40m > Remaining Estimate: 0h > > Currently the Distribution Event package contains the following details: > * Distribution Component Name > * Distribution Component Kind > * Distribution Type > * Distribution Paths > > Improvement aims at adding the queue item creation time, essentially when the > the item was creation for the first time, and enqueue into the queue. The > value does not change over retries (on failure). > > The purpose to get this detail is to be able to capture metrics at the > consumer level. The consumers could have an event handler, which can capture > the duration which turns out to be (NOW MINUS queue item creation time thrown > in the distribution event package); NOW being the current time in the event > handler (consumer). > > \cc: [~shgu...@adobe.com], [~ashishc] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (SLING-9389) Distribution Event Packages should contain queue item creation time
[ https://issues.apache.org/jira/browse/SLING-9389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17123667#comment-17123667 ] Dirk Rudolph commented on SLING-9389: - Hey [~harshchiki], quick question: couldn't you just take the time added to the DistributionPackageInfo here https://github.com/apache/sling-org-apache-sling-distribution-core/blob/70214c09f52d9d8fae0e727b572afc82f1d937a9/src/main/java/org/apache/sling/distribution/agent/impl/QueueingDistributionPackageProcessor.java#L119 and expose it DefaultDistributionEventFactory if set? Though I agree that having the approx. time the distribution request was created available in the distribution events would make sense. For example when forwarding the package imported event to a 3rd party system from multiple sling instances this timestamp could be useful for optimistic locking. > Distribution Event Packages should contain queue item creation time > --- > > Key: SLING-9389 > URL: https://issues.apache.org/jira/browse/SLING-9389 > Project: Sling > Issue Type: Improvement > Components: Content Distribution >Reporter: Harsh Chiki >Assignee: Timothee Maret >Priority: Major > Attachments: image-2020-04-30-10-28-58-011.png, scdapi.patch, > scdcore.patch > > Time Spent: 1h 40m > Remaining Estimate: 0h > > Currently the Distribution Event package contains the following details: > * Distribution Component Name > * Distribution Component Kind > * Distribution Type > * Distribution Paths > > Improvement aims at adding the queue item creation time, essentially when the > the item was creation for the first time, and enqueue into the queue. The > value does not change over retries (on failure). > > The purpose to get this detail is to be able to capture metrics at the > consumer level. The consumers could have an event handler, which can capture > the duration which turns out to be (NOW MINUS queue item creation time thrown > in the distribution event package); NOW being the current time in the event > handler (consumer). > > \cc: [~shgu...@adobe.com], [~ashishc] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (SLING-9389) Distribution Event Packages should contain queue item creation time
[ https://issues.apache.org/jira/browse/SLING-9389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17117406#comment-17117406 ] Harsh Chiki commented on SLING-9389: [~marett], request you to kindly review the PRs shared, or please let us know if you have questions/concerns with the requirement. cc: [~cschneider], [~ashishc], [~shgu...@adobe.com] > Distribution Event Packages should contain queue item creation time > --- > > Key: SLING-9389 > URL: https://issues.apache.org/jira/browse/SLING-9389 > Project: Sling > Issue Type: Improvement > Components: Content Distribution >Reporter: Harsh Chiki >Assignee: Timothee Maret >Priority: Major > Attachments: image-2020-04-30-10-28-58-011.png, scdapi.patch, > scdcore.patch > > Time Spent: 1.5h > Remaining Estimate: 0h > > Currently the Distribution Event package contains the following details: > * Distribution Component Name > * Distribution Component Kind > * Distribution Type > * Distribution Paths > > Improvement aims at adding the queue item creation time, essentially when the > the item was creation for the first time, and enqueue into the queue. The > value does not change over retries (on failure). > > The purpose to get this detail is to be able to capture metrics at the > consumer level. The consumers could have an event handler, which can capture > the duration which turns out to be (NOW MINUS queue item creation time thrown > in the distribution event package); NOW being the current time in the event > handler (consumer). > > \cc: [~shgu...@adobe.com], [~ashishc] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (SLING-9389) Distribution Event Packages should contain queue item creation time
[ https://issues.apache.org/jira/browse/SLING-9389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17107005#comment-17107005 ] Harsh Chiki commented on SLING-9389: [~marett], thanks for looking into it. > Is it correct that your requirement is to measure the end to end distribution >latency for each package on the source service (author) ? Yes. > If so, could you also clarify if it's a requirement for you to have the >information in the event, or if you could not have the information exposed as >a metric. * The idea is to capture a metric (using Sling Metrics) exposing the end to end distribution behavior (AEM to BP in this context). * IIUC if the information is captured within SCD as a metric (if that is the other option you are putting on the table) - the information would not come down to the consumers of SCD apparently. ** Since our scenario is AEM to BP publish - we wanted to essentially monitor the distribution from AEM to BP - end to end. ** Nothing wrong (in fact likely good to capture metrics in SCD), I am afraid if there are multiple Distribution Agents (and flows) the statistics would get skewed. It will be difficult (I guess impossible) to figure out the metrics per flow (AEM to BP as in this case). Do you think this problem stands true, and if we have a solution? Please let us know. \cc: [~ashishc], [~shgu...@adobe.com] > Distribution Event Packages should contain queue item creation time > --- > > Key: SLING-9389 > URL: https://issues.apache.org/jira/browse/SLING-9389 > Project: Sling > Issue Type: Improvement > Components: Content Distribution >Reporter: Harsh Chiki >Assignee: Timothee Maret >Priority: Major > Attachments: image-2020-04-30-10-28-58-011.png, scdapi.patch, > scdcore.patch > > Time Spent: 50m > Remaining Estimate: 0h > > Currently the Distribution Event package contains the following details: > * Distribution Component Name > * Distribution Component Kind > * Distribution Type > * Distribution Paths > > Improvement aims at adding the queue item creation time, essentially when the > the item was creation for the first time, and enqueue into the queue. The > value does not change over retries (on failure). > > The purpose to get this detail is to be able to capture metrics at the > consumer level. The consumers could have an event handler, which can capture > the duration which turns out to be (NOW MINUS queue item creation time thrown > in the distribution event package); NOW being the current time in the event > handler (consumer). > > \cc: [~shgu...@adobe.com], [~ashishc] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (SLING-9389) Distribution Event Packages should contain queue item creation time
[ https://issues.apache.org/jira/browse/SLING-9389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17106286#comment-17106286 ] Timothee Maret commented on SLING-9389: --- [~harshchiki] thanks I looked at it. Is it correct that your requirement is to measure the end to end distribution latency for each package on the source service (author) ? > Distribution Event Packages should contain queue item creation time > --- > > Key: SLING-9389 > URL: https://issues.apache.org/jira/browse/SLING-9389 > Project: Sling > Issue Type: Improvement > Components: Content Distribution >Reporter: Harsh Chiki >Assignee: Timothee Maret >Priority: Major > Attachments: image-2020-04-30-10-28-58-011.png, scdapi.patch, > scdcore.patch > > Time Spent: 50m > Remaining Estimate: 0h > > Currently the Distribution Event package contains the following details: > * Distribution Component Name > * Distribution Component Kind > * Distribution Type > * Distribution Paths > > Improvement aims at adding the queue item creation time, essentially when the > the item was creation for the first time, and enqueue into the queue. The > value does not change over retries (on failure). > > The purpose to get this detail is to be able to capture metrics at the > consumer level. The consumers could have an event handler, which can capture > the duration which turns out to be (NOW MINUS queue item creation time thrown > in the distribution event package); NOW being the current time in the event > handler (consumer). > > \cc: [~shgu...@adobe.com], [~ashishc] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (SLING-9389) Distribution Event Packages should contain queue item creation time
[ https://issues.apache.org/jira/browse/SLING-9389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105844#comment-17105844 ] Harsh Chiki commented on SLING-9389: [~marett], Gentle reminder. Kindly review the PRs and share your feedback. Thanks! \cc: [~shgu...@adobe.com], [~ashishc] > Distribution Event Packages should contain queue item creation time > --- > > Key: SLING-9389 > URL: https://issues.apache.org/jira/browse/SLING-9389 > Project: Sling > Issue Type: Improvement > Components: Content Distribution >Reporter: Harsh Chiki >Assignee: Timothee Maret >Priority: Major > Attachments: image-2020-04-30-10-28-58-011.png, scdapi.patch, > scdcore.patch > > Time Spent: 50m > Remaining Estimate: 0h > > Currently the Distribution Event package contains the following details: > * Distribution Component Name > * Distribution Component Kind > * Distribution Type > * Distribution Paths > > Improvement aims at adding the queue item creation time, essentially when the > the item was creation for the first time, and enqueue into the queue. The > value does not change over retries (on failure). > > The purpose to get this detail is to be able to capture metrics at the > consumer level. The consumers could have an event handler, which can capture > the duration which turns out to be (NOW MINUS queue item creation time thrown > in the distribution event package); NOW being the current time in the event > handler (consumer). > > \cc: [~shgu...@adobe.com], [~ashishc] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (SLING-9389) Distribution Event Packages should contain queue item creation time
[ https://issues.apache.org/jira/browse/SLING-9389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17096378#comment-17096378 ] Harsh Chiki commented on SLING-9389: Hi [~marett], [~ashishc], I have raised the PRs: * [https://github.com/apache/sling-org-apache-sling-distribution-api/pull/2] * [https://github.com/apache/sling-org-apache-sling-distribution-core/pull/41] Kindly review and share your feedback. Thanks \cc: [~shgu...@adobe.com] > Distribution Event Packages should contain queue item creation time > --- > > Key: SLING-9389 > URL: https://issues.apache.org/jira/browse/SLING-9389 > Project: Sling > Issue Type: Improvement > Components: Content Distribution >Reporter: Harsh Chiki >Assignee: Timothee Maret >Priority: Major > Attachments: image-2020-04-30-10-28-58-011.png, scdapi.patch, > scdcore.patch > > Time Spent: 40m > Remaining Estimate: 0h > > Currently the Dsitribution Event package contains the following details: > * Distribution Component Name > * Distribution Component Kind > * Distribution Type > * Distribution Paths > > Improvement aims at adding the queue item creation time, essentially when the > the item was creation for the first time, and enqueue into the queue. The > value does not change over retries (on failure). > > The purpose to get this detail is to be able to capture metrics at the > consumer level. The consumers could have an event handler, which can capture > the duration which turns out to be (NOW MINUS queue item creation time thrown > in the distribution event package); NOW being the current time in the event > handler (consumer). > > \cc: [~shgu...@adobe.com], [~ashishc] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (SLING-9389) Distribution Event Packages should contain queue item creation time
[
https://issues.apache.org/jira/browse/SLING-9389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17093799#comment-17093799
]
Harsh Chiki commented on SLING-9389:
Hi [~marett],
Please consider this a gentle reminder.
I worked out some basic changes to capture the queue item creation time, to be
put in the {{AGENT_PACKAGE_DISTRIBUTED}} and {{AGENT_PACKAGE_DROPPED}}. From
other paths, have sent out null. PFA SCD API and Core patches which of course
are very rough at the moment.
May I request your views, in general from this requirement pov, essentially:
* Whether we should add a new parameter in this internal API and internal
implementation, or a new method to capture time stamp as well. (Prior one is
done in the patch shared)
* Is it fine to send null in the other event topic cases?
* For scheduled import export cases.
* In our local experiments, we identified the queue item creation time stamp
is what one would need from an enqueue (first time creation, not on retries)
time stamp pov, so we intend to use
[https://github.com/apache/sling-org-apache-sling-distribution-core/blob/429f6bc225970c34e948d0ad41c38fe4c0ef9ef9/src/main/java/org/apache/sling/distribution/queue/impl/resource/ResourceQueueUtils.java#L219]
** Figured that
[https://github.com/apache/sling-org-apache-sling-distribution-core/blob/master/src/main/java/org/apache/sling/distribution/agent/impl/QueueingDistributionPackageProcessor.java#L119]
which is later used for getting {{globalStartTime}} here -
[https://github.com/apache/sling-org-apache-sling-distribution-core/blob/master/src/main/java/org/apache/sling/distribution/agent/impl/SimpleDistributionAgentQueueProcessor.java#L115.|https://github.com/apache/sling-org-apache-sling-distribution-core/blob/master/src/main/java/org/apache/sling/distribution/agent/impl/SimpleDistributionAgentQueueProcessor.java#L115]
** But, relied on the queue item creation time - generated in
`ResourceQueueUtils` linked above (in this comment)
Kindly have a look, and advise.
Thanks!
\cc: [~ashishc], [~shgu...@adobe.com]
> Distribution Event Packages should contain queue item creation time
> ---
>
> Key: SLING-9389
> URL: https://issues.apache.org/jira/browse/SLING-9389
> Project: Sling
> Issue Type: Improvement
> Components: Content Distribution
>Reporter: Harsh Chiki
>Assignee: Timothee Maret
>Priority: Major
> Attachments: scdapi.patch, scdcore.patch
>
>
> Currently the Dsitribution Event package contains the following details:
> * Distribution Component Name
> * Distribution Component Kind
> * Distribution Type
> * Distribution Paths
>
> Improvement aims at adding the queue item creation time, essentially when the
> the item was creation for the first time, and enqueue into the queue. The
> value does not change over retries (on failure).
>
> The purpose to get this detail is to be able to capture metrics at the
> consumer level. The consumers could have an event handler, which can capture
> the duration which turns out to be (NOW MINUS queue item creation time thrown
> in the distribution event package); NOW being the current time in the event
> handler (consumer).
>
> \cc: [~shgu...@adobe.com], [~ashishc]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (SLING-9389) Distribution Event Packages should contain queue item creation time
[ https://issues.apache.org/jira/browse/SLING-9389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17091219#comment-17091219 ] Harsh Chiki commented on SLING-9389: Hi [~marett], atm we need it on the source service (author). This is in conjunction to the AEM on BP publish cloud service, where the purpose is to effectively capture the round trip time of the distribution from AEM (author - source service) to BP (destination service). > Distribution Event Packages should contain queue item creation time > --- > > Key: SLING-9389 > URL: https://issues.apache.org/jira/browse/SLING-9389 > Project: Sling > Issue Type: Improvement > Components: Content Distribution >Reporter: Harsh Chiki >Assignee: Timothee Maret >Priority: Major > > Currently the Dsitribution Event package contains the following details: > * Distribution Component Name > * Distribution Component Kind > * Distribution Type > * Distribution Paths > > Improvement aims at adding the queue item creation time, essentially when the > the item was creation for the first time, and enqueue into the queue. The > value does not change over retries (on failure). > > The purpose to get this detail is to be able to capture metrics at the > consumer level. The consumers could have an event handler, which can capture > the duration which turns out to be (NOW MINUS queue item creation time thrown > in the distribution event package); NOW being the current time in the event > handler (consumer). > > \cc: [~shgu...@adobe.com], [~ashishc] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (SLING-9389) Distribution Event Packages should contain queue item creation time
[ https://issues.apache.org/jira/browse/SLING-9389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17090421#comment-17090421 ] Timothee Maret commented on SLING-9389: --- Hi [~harshchiki], do you need this information on the source service (author) or destination service (publish) ? > Distribution Event Packages should contain queue item creation time > --- > > Key: SLING-9389 > URL: https://issues.apache.org/jira/browse/SLING-9389 > Project: Sling > Issue Type: Improvement > Components: Content Distribution >Reporter: Harsh Chiki >Assignee: Timothee Maret >Priority: Major > > Currently the Dsitribution Event package contains the following details: > * Distribution Component Name > * Distribution Component Kind > * Distribution Type > * Distribution Paths > > Improvement aims at adding the queue item creation time, essentially when the > the item was creation for the first time, and enqueue into the queue. The > value does not change over retries (on failure). > > The purpose to get this detail is to be able to capture metrics at the > consumer level. The consumers could have an event handler, which can capture > the duration which turns out to be (NOW MINUS queue item creation time thrown > in the distribution event package); NOW being the current time in the event > handler (consumer). > > \cc: [~shgu...@adobe.com], [~ashishc] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (SLING-9389) Distribution Event Packages should contain queue item creation time
[ https://issues.apache.org/jira/browse/SLING-9389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17089433#comment-17089433 ] Harsh Chiki commented on SLING-9389: [~marett], we request you to have a look at the proposed improvement. We request your inputs on the requirement as posted and what technical points we need to be careful about. Following are the initial pointers we have (we are working on refining them further) * Queue item creation: [https://github.com/apache/sling-org-apache-sling-distribution-core/blob/429f6bc225970c34e948d0ad41c38fe4c0ef9ef9/src/main/java/org/apache/sling/distribution/queue/impl/resource/ResourceQueueUtils.java#L219] * Queue Processing where DISTRIBUTION/DROPPED events are fired - [https://github.com/apache/sling-org-apache-sling-distribution-core/blob/master/src/main/java/org/apache/sling/distribution/agent/impl/SimpleDistributionAgentQueueProcessor.java#L115-L169] * Add details to the distribution event here - [https://github.com/apache/sling-org-apache-sling-distribution-core/blob/a1850193d63ddedced4ad982df18a62345cdf41b/src/main/java/org/apache/sling/distribution/event/impl/DefaultDistributionEventFactory.java#L52-L70] ** DistributionEventFactory - and internal interface in SCD Core - [https://github.com/apache/sling-org-apache-sling-distribution-core/blob/429f6bc225970c34e948d0ad41c38fe4c0ef9ef9/src/main/java/org/apache/sling/distribution/event/impl/DistributionEventFactory.java#L28] ** DistributionEventFactoryImpl - internal to SCD Core - [https://github.com/apache/sling-org-apache-sling-distribution-core/blob/a1850193d63ddedced4ad982df18a62345cdf41b/src/main/java/org/apache/sling/distribution/event/impl/DefaultDistributionEventFactory.java#L52-L70] *** DistributionEventFactoryImpl though uses DistributionEventProperties "exported" by SCD api, where we will need to add a new property for queue creation time - [https://github.com/apache/sling-org-apache-sling-distribution-api/blob/master/src/main/java/org/apache/sling/distribution/event/DistributionEventProperties.java#L28] \cc: [~ashishc], [~shgu...@adobe.com] > Distribution Event Packages should contain queue item creation time > --- > > Key: SLING-9389 > URL: https://issues.apache.org/jira/browse/SLING-9389 > Project: Sling > Issue Type: Improvement >Reporter: Harsh Chiki >Priority: Major > > Currently the Dsitribution Event package contains the following details: > * Distribution Component Name > * Distribution Component Kind > * Distribution Type > * Distribution Paths > > Improvement aims at adding the queue item creation time, essentially when the > the item was creation for the first time, and enqueue into the queue. The > value does not change over retries (on failure). > > The purpose to get this detail is to be able to capture metrics at the > consumer level. The consumers could have an event handler, which can capture > the duration which turns out to be (NOW - queue item creation time thrown in > the distribution event package); NOW being the current time in the event > handler (consumer). > > \cc: [~shgu...@adobe.com], [~ashishc] -- This message was sent by Atlassian Jira (v8.3.4#803005)
