[
https://issues.apache.org/jira/browse/KAFKA-9385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kaikai.hou resolved KAFKA-9385.
-------------------------------
Resolution: Duplicate
> Connect cluster: connector task repeat like a splitbrain cluster problem
> -------------------------------------------------------------------------
>
> Key: KAFKA-9385
> URL: https://issues.apache.org/jira/browse/KAFKA-9385
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Reporter: kaikai.hou
> Priority: Major
> Attachments: 12_31_d8c7j_1.jpg
>
>
> I am using Debezium. And find a task repeat
> problem.[Jump|[https://issues.redhat.com/browse/DBZ-1573?jql=key%20in%20watchedIssues()]]
>
> 1. I push the Debezium image to our private image repository.
> 2. Deploy the connect cluster with the following *Deployment Config*：
> {code:java}
> //代码占位符
> apiVersion: apps.openshift.io/v1
> kind: DeploymentConfig
> metadata:
> annotations:
> openshift.io/generated-by: OpenShiftWebConsole
> creationTimestamp: '2019-10-14T07:45:41Z'
> generation: 29
> labels:
> app: debezium-test-cloud
> name: debezium-test-cloud
> namespace: test
> resourceVersion: '168496156'
> selfLink: >-
>
> /apis/apps.openshift.io/v1/namespaces/test/deploymentconfigs/debezium-test-cloud
> uid: 9f4f8f4d-ee56-11e9-a5a1-00163e0e008f
> spec:
> replicas: 2
> selector:
> app: debezium-test-cloud
> deploymentconfig: debezium-test-cloud
> strategy:
> activeDeadlineSeconds: 21600
> resources: {}
> rollingParams:
> intervalSeconds: 1
> maxSurge: 25%
> maxUnavailable: 25%
> timeoutSeconds: 600
> updatePeriodSeconds: 1
> type: Rolling
> template:
> metadata:
> annotations:
> openshift.io/generated-by: OpenShiftWebConsole
> creationTimestamp: null
> labels:
> app: debezium-test-cloud
> deploymentconfig: debezium-test-cloud
> spec:
> containers:
> - env:
> - name: BOOTSTRAP_SERVERS
> value: '192.168.100.228:9092'
> - name: GROUP_ID
> value: test-cloud
> - name: CONFIG_STORAGE_TOPIC
> value: base.test-cloud.config
> - name: OFFSET_STORAGE_TOPIC
> value: base.test-cloud.offset
> - name: STATUS_STORAGE_TOPIC
> value: base.test-cloud.status
> - name: CONNECT_KEY_CONVERTER_SCHEMAS_ENABLE
> value: 'true'
> - name: CONNECT_VALUE_CONVERTER_SCHEMAS_ENABLE
> value: 'true'
> - name: CONNECT_PRODUCER_MAX_REQUEST_SIZE
> value: '20971520'
> - name: CONNECT_DATABASE_HISTORY_KAFKA_RECOVERY_POLL_INTERVAL_MS
> value: '1000'
> - name: HEAP_OPTS
> value: '-XX:+UseContainerSupport -XX:MaxRAMPercentage=75.0'
> image:
> 'registry.cn-hangzhou.aliyuncs.com/eshine/debeziumconnect:1.0.0.Beta2'
> imagePullPolicy: IfNotPresent
> name: debezium-test-cloud
> ports:
> - containerPort: 8083
> protocol: TCP
> - containerPort: 8778
> protocol: TCP
> - containerPort: 9092
> protocol: TCP
> - containerPort: 9779
> protocol: TCP
> resources:
> limits:
> cpu: 400m
> memory: 1Gi
> requests:
> cpu: 200m
> memory: 1Gi
> terminationMessagePath: /dev/termination-log
> terminationMessagePolicy: File
> volumeMounts:
> - mountPath: /kafka/config
> name: debezium-test-cloud-1
> - mountPath: /kafka/data
> name: debezium-test-cloud-2
> - mountPath: /kafka/logs
> name: debezium-test-cloud-3
> dnsPolicy: ClusterFirst
> restartPolicy: Always
> schedulerName: default-scheduler
> securityContext: {}
> terminationGracePeriodSeconds: 30
> volumes:
> - emptyDir: {}
> name: debezium-test-cloud-1
> - emptyDir: {}
> name: debezium-test-cloud-2
> - emptyDir: {}
> name: debezium-test-cloud-3
> test: false
> triggers:
> - type: ConfigChange
> status:
> availableReplicas: 2
> conditions:
> - lastTransitionTime: '2019-11-25T06:44:30Z'
> lastUpdateTime: '2019-11-25T06:44:44Z'
> message: replication controller "debezium-test-cloud-15" successfully
> rolled out
> reason: NewReplicationControllerAvailable
> status: 'True'
> type: Progressing
> - lastTransitionTime: '2019-12-31T10:06:23Z'
> lastUpdateTime: '2019-12-31T10:06:23Z'
> message: Deployment config has minimum availability.
> status: 'True'
> type: Available
> details:
> causes:
> - type: Manual
> message: manual change
> latestVersion: 15
> observedGeneration: 29
> readyReplicas: 2
> replicas: 2
> unavailableReplicas: 0
> updatedReplicas: 2
> {code}
> 3. Connect cluster in openshift: one service with two pods
> 4.
> a). task_connector_1_0 and task_connector_3_0 were running in podA;
> task_connector_2_0 was running in PodB
> b) Then, PodA console follows error log: In attachment
> "12_31_d8c7j_1.jpg"
> !12_31_d8c7j_1.jpg!
> c) Then, Rebalance started;
> d) However, In PodB, all task (task_connector_1_0, task_connector_2_0,
> task_connector_3_0) are running. In PodA, still task_connector_1_0 and
> task_connector_3_0.
> e) So the repeat task appeared.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
