[
https://issues.apache.org/jira/browse/GOBBLIN-1930?focusedWorklogId=885445&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-885445
]
ASF GitHub Bot logged work on GOBBLIN-1930:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 16/Oct/23 17:55
Start Date: 16/Oct/23 17:55
Worklog Time Spent: 10m
Work Description: umustafi commented on code in PR #3800:
URL: https://github.com/apache/gobblin/pull/3800#discussion_r1361057598
##########
gobblin-service/src/main/java/org/apache/gobblin/service/monitoring/DagActionStoreChangeMonitor.java:
##########
@@ -130,30 +132,34 @@ protected void processMessage(DecodeableKafkaRecord
message) {
String changeIdentifier = tid + key;
if (!ChangeMonitorUtils.shouldProcessMessage(changeIdentifier,
dagActionsSeenCache, operation,
produceTimestamp.toString())) {
+ this.messageFilteredOutMeter.mark();
return;
}
+ // Used to easily log information to identify the dag action
+ DagActionStore.DagAction dagAction = new
DagActionStore.DagAction(flowGroup, flowName, flowExecutionId,
+ dagActionType);
+
// We only expect INSERT and DELETE operations done to this table. INSERTs
correspond to any type of
// {@link DagActionStore.FlowActionType} flow requests that have to be
processed. DELETEs require no action.
try {
if (operation.equals("INSERT")) {
if (dagActionType.equals(DagActionStore.FlowActionType.RESUME)) {
- log.info("Received insert dag action and about to send resume flow
request");
+ log.info("Received insert dag action and about to send resume flow
request for: {}", dagAction);
Review Comment:
I was thinking about unifying all the logs like this as well, let me make
the change. I will add a bit more context in message but use this format.
Issue Time Tracking
-------------------
Worklog Id: (was: 885445)
Time Spent: 1h 10m (was: 1h)
> Improve Logs & Metrics around Multi-active Launch Handling
> ----------------------------------------------------------
>
> Key: GOBBLIN-1930
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1930
> Project: Apache Gobblin
> Issue Type: Improvement
> Components: gobblin-service
> Reporter: Urmi Mustafi
> Assignee: Abhishek Tiwari
> Priority: Major
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Improve logging and metrics around multi-active launch flow event handling to
> identify any missing events between the {{MysqlMultiActiveLeaseArbiter}}
> committing the launch event to the {{dagActionStore}} and the
> {{DagActionMonitor}} receiving events for processing. We want to be able to
> distinguish between the following cases ofÂ
> * events that are never received by the {{DagActionMonitor}}
> * events incorrectly filtered out by the {{DagActionMonitor}}
> * any failed submissions of dags to the {{DagManager}} either upon leader
> change or from the {{DagActionChangeMonitor}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
