date:20200606

[GitHub] [kafka-site] brary commented on pull request #267: Updating upgrade docs for KIP-535 and KIP-562

2020-06-06 Thread GitBox



brary commented on pull request #267:
URL: https://github.com/apache/kafka-site/pull/267#issuecomment-640160892


   @mjsax Requesting review.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [kafka-site] brary opened a new pull request #267: Updating upgrade docs for KIP-535 and KIP-562

2020-06-06 Thread GitBox



brary opened a new pull request #267:
URL: https://github.com/apache/kafka-site/pull/267


   Documenting KIP-535 in the release notes and making some updates related to 
KIP-562.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Build failed in Jenkins: kafka-2.6-jdk8 #25

2020-06-06 Thread Apache Jenkins Server

See 


Changes:

[matthias] KAFKA-10097: Internalize checkpoint data (#8820)


--
[...truncated 6.26 MB...]

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldForwardRecordsFromSubtopologyToSubtopology[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldForwardRecordsFromSubtopologyToSubtopology[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldNotUpdateStoreForSmallerValue[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldNotUpdateStoreForSmallerValue[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldCreateStateDirectoryForStatefulTopology[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldCreateStateDirectoryForStatefulTopology[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldPunctuateIfWallClockTimeAdvances[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldPunctuateIfWallClockTimeAdvances[Eos enabled = false] PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordsFromKeyValuePairs STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordsFromKeyValuePairs PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithNullKeyAndDefaultTimestamp 
STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithNullKeyAndDefaultTimestamp 
PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithDefaultTimestamp 
STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithDefaultTimestamp 
PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithNullKey STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithNullKey PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateNullKeyConsumerRecordWithOtherTopicNameAndTimestampWithTimetamp 
STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateNullKeyConsumerRecordWithOtherTopicNameAndTimestampWithTimetamp 
PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordWithTimestamp STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordWithTimestamp PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullHeaders STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullHeaders PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithDefaultTimestamp STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithDefaultTimestamp PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicName STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicName PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithNullKey STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithNullKey PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecord STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecord PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateNullKeyConsumerRecord STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateNullKeyConsumerRecord PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordWithOtherTopicName STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordWithOtherTopicName PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > shouldAdvanceTime 
STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > shouldAdvanceTime 
PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithKeyValuePairs STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithKeyValuePairs PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithKeyValuePairsAndCustomTimestamps
 STARTED

Jenkins build is back to normal : kafka-trunk-jdk14 #195

2020-06-06 Thread Apache Jenkins Server

See

[jira] [Resolved] (KAFKA-10097) Avoid passing task checkpoint file externally

2020-06-06 Thread Boyang Chen (Jira)



 [ 
https://issues.apache.org/jira/browse/KAFKA-10097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Boyang Chen resolved KAFKA-10097.
-
Resolution: Fixed

> Avoid passing task checkpoint file externally
> -
>
> Key: KAFKA-10097
> URL: https://issues.apache.org/jira/browse/KAFKA-10097
> Project: Kafka
>  Issue Type: Improvement
>  Components: streams
>Reporter: Boyang Chen
>Assignee: Boyang Chen
>Priority: Major
>
> In StreamTask, we have the logic to generate a checkpoint offset map to be 
> materialized through StateManager#checkpoint. This map could be either empty 
> map or null, which the former case indicates to only pull down existing state 
> store checkpoint data, while the latter indicates no need to do a checkpoint 
> in the case such as we are suspending a task.
> Having two similar special logics for checkpointing could lead to unexpected 
> bugs, also we should think about separating the empty checkpoint case vs 
> passed-in checkpoint case.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: subscribe confirm

2020-06-06 Thread Matthias J. Sax

Roc,

if you want to subscribe to the mailing list, please follow the
instruction from the web page: https://kafka.apache.org/contact

-Matthias

On 6/6/20 9:32 AM, Roc Marshal wrote:
> flin...@126.com
> 



signature.asc
Description: OpenPGP digital signature

Jenkins build is back to normal : kafka-trunk-jdk11 #1546

2020-06-06 Thread Apache Jenkins Server

See

Build failed in Jenkins: kafka-trunk-jdk14 #194

2020-06-06 Thread Apache Jenkins Server

See 


Changes:

[manikumar] KAFKA-10033: Throw UnknownTopicOrPartitionException when modifying a


--
[...truncated 6.31 MB...]

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldAllowPrePopulatingStatesStoresWithCachingEnabled[Eos enabled = false] 
STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldAllowPrePopulatingStatesStoresWithCachingEnabled[Eos enabled = false] 
PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldReturnCorrectPersistentStoreTypeOnly[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldReturnCorrectPersistentStoreTypeOnly[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > shouldRespectTaskIdling[Eos 
enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > shouldRespectTaskIdling[Eos 
enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldUseSourceSpecificDeserializers[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldUseSourceSpecificDeserializers[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > shouldReturnAllStores[Eos 
enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > shouldReturnAllStores[Eos 
enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldNotCreateStateDirectoryForStatelessTopology[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldNotCreateStateDirectoryForStatelessTopology[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldApplyGlobalUpdatesCorrectlyInRecursiveTopologies[Eos enabled = false] 
STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldApplyGlobalUpdatesCorrectlyInRecursiveTopologies[Eos enabled = false] 
PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldReturnAllStoresNames[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldReturnAllStoresNames[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldPassRecordHeadersIntoSerializersAndDeserializers[Eos enabled = false] 
STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldPassRecordHeadersIntoSerializersAndDeserializers[Eos enabled = false] 
PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldProcessConsumerRecordList[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldProcessConsumerRecordList[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldUseSinkSpecificSerializers[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldUseSinkSpecificSerializers[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldFlushStoreForFirstInput[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldFlushStoreForFirstInput[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldProcessFromSourceThatMatchPattern[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldProcessFromSourceThatMatchPattern[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldCaptureSinkTopicNamesIfWrittenInto[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldCaptureSinkTopicNamesIfWrittenInto[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldUpdateStoreForNewKey[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldUpdateStoreForNewKey[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldSendRecordViaCorrectSourceTopicDeprecated[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldSendRecordViaCorrectSourceTopicDeprecated[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldPunctuateOnWallClockTime[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldPunctuateOnWallClockTime[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > shouldSetRecordMetadata[Eos 
enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > shouldSetRecordMetadata[Eos 
enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldNotUpdateStoreForLargerValue[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldNotUpdateStoreForLargerValue[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldReturnCorrectInMemoryStoreTypeOnly[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldReturnCorrectInMemoryStoreTypeOnly[Eos enabled = false] PASSED

[jira] [Created] (KAFKA-10113) LogTruncationException sets fetch offsets incorrectly

2020-06-06 Thread Jason Gustafson (Jira)

Jason Gustafson created KAFKA-10113:
---

 Summary: LogTruncationException sets fetch offsets incorrectly
 Key: KAFKA-10113
 URL: https://issues.apache.org/jira/browse/KAFKA-10113
 Project: Kafka
  Issue Type: Bug
Reporter: Jason Gustafson
Assignee: Jason Gustafson


LogTruncationException, which extends OffsetOutOfRangeException, takes the 
divergent offsets in the constructor. These are the first offsets known to 
diverge from what the consumer read. These are then passed to the 
OffsetOutOfRangeException incorrectly as the out of range fetch offsets. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

subscribe confirm

2020-06-06 Thread Roc Marshal

flin...@126.com

[jira] [Resolved] (KAFKA-9851) Revoking Connect tasks due to connectivity issues should also clear running assignment

2020-06-06 Thread Konstantine Karantasis (Jira)



 [ 
https://issues.apache.org/jira/browse/KAFKA-9851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantine Karantasis resolved KAFKA-9851.
---
Resolution: Fixed

> Revoking Connect tasks due to connectivity issues should also clear running 
> assignment
> --
>
> Key: KAFKA-9851
> URL: https://issues.apache.org/jira/browse/KAFKA-9851
> Project: Kafka
>  Issue Type: Bug
>Reporter: Konstantine Karantasis
>Assignee: Konstantine Karantasis
>Priority: Major
> Fix For: 2.3.2, 2.6.0, 2.4.2, 2.5.1
>
>
> https://issues.apache.org/jira/browse/KAFKA-9184 fixed an issue with workers 
> continuing to run tasks even after they'd lose connectivity with the broker 
> coordinator and they'd detect that they are out of the group. 
>  
> However, because the revocation of tasks in this case is voluntary and does 
> not come with an explicit assignment (containing revoked tasks) from the 
> leader worker, the worker that quits running its tasks due to connectivity 
> issues needs to also clear its running task assignment snapshot. 
> This will allow for proper restart of the stopped tasks after the worker 
> rejoins the group when connectivity returns and get assigned the same 
> connectors or tasks. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (KAFKA-10033) AdminClient should throw UnknownTopicOrPartitionException instead of UnknownServerException if altering configs of non-existing topic

2020-06-06 Thread Manikumar (Jira)



 [ 
https://issues.apache.org/jira/browse/KAFKA-10033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Manikumar resolved KAFKA-10033.
---
Fix Version/s: 2.7.0
   Resolution: Fixed

> AdminClient should throw UnknownTopicOrPartitionException instead of 
> UnknownServerException if altering configs of non-existing topic
> -
>
> Key: KAFKA-10033
> URL: https://issues.apache.org/jira/browse/KAFKA-10033
> Project: Kafka
>  Issue Type: Improvement
>  Components: admin
>Affects Versions: 2.4.0
>Reporter: Gregory Koshelev
>Assignee: Brian Byrne
>Priority: Major
>  Labels: pull-request-available
> Fix For: 2.7.0
>
>
> Currently, altering configs of non-existing topic leads to 
> {{UnknownServerException}}:
> {code}
> java.util.concurrent.ExecutionException: 
> org.apache.kafka.common.errors.UnknownServerException: Topic "kgn_test" does 
> not exist.
>   at 
> org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
>   at 
> org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
>   at 
> org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
>   at 
> org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260)
>   at 
> ru.kontur.vostok.hercules.stream.manager.kafka.KafkaManager.changeTtl(KafkaManager.java:130)
>   ... 10 common frames omitted
> Caused by: org.apache.kafka.common.errors.UnknownServerException: Topic 
> "kgn_test" does not exist.
> {code}
> The output above is produced due to {{AdminZkClient.validateTopicConfig}} 
> method:
> {code}
>   def validateTopicConfig(topic: String, configs: Properties): Unit = {
> Topic.validate(topic)
> if (!zkClient.topicExists(topic))
>   throw new AdminOperationException("Topic \"%s\" does not 
> exist.".format(topic))
> // remove the topic overrides
> LogConfig.validate(configs)
>   }
> {code}
> {{UnknownServerException}} is common exception but in this case cause is 
> pretty clear. So this can be fixed easily by using 
> {{UnknownTopicOrPartitionException}}.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (KAFKA-10094) In MirrorSourceConnector replace two-step assignment with single call

2020-06-06 Thread Jakob Homan (Jira)



 [ 
https://issues.apache.org/jira/browse/KAFKA-10094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jakob Homan resolved KAFKA-10094.
-
Resolution: Fixed

> In MirrorSourceConnector replace two-step assignment with single call
> -
>
> Key: KAFKA-10094
> URL: https://issues.apache.org/jira/browse/KAFKA-10094
> Project: Kafka
>  Issue Type: Improvement
>  Components: mirrormaker
>Reporter: Jakob Homan
>Assignee: Mandar Tillu
>Priority: Trivial
>  Labels: newbie
>
> n.b. This is a newbie ticket designed to be an introduction to contributing 
> for the assignee.
>  
> In MirrorSourceConnector::refreshTopicPartitions we have places where we 
> create a new HashSet and then addAll to the set.  We can replace both with a 
> direct call to the copy constructor.
>  
> {code:java}
> void refreshTopicPartitions()
> throws InterruptedException, ExecutionException {
> knownSourceTopicPartitions = findSourceTopicPartitions();
> knownTargetTopicPartitions = findTargetTopicPartitions();
> List upstreamTargetTopicPartitions = 
> knownTargetTopicPartitions.stream()
> .map(x -> new 
> TopicPartition(replicationPolicy.upstreamTopic(x.topic()), x.partition()))
> .collect(Collectors.toList());
> Set newTopicPartitions = new HashSet<>();
> newTopicPartitions.addAll(knownSourceTopicPartitions);
> newTopicPartitions.removeAll(upstreamTargetTopicPartitions);
> Set deadTopicPartitions = new HashSet<>();
> deadTopicPartitions.addAll(upstreamTargetTopicPartitions);{code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (KAFKA-9840) Consumer should not use OffsetForLeaderEpoch without current epoch validation

2020-06-06 Thread Jason Gustafson (Jira)



 [ 
https://issues.apache.org/jira/browse/KAFKA-9840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Gustafson resolved KAFKA-9840.

Fix Version/s: 2.6.0
   Resolution: Fixed

> Consumer should not use OffsetForLeaderEpoch without current epoch validation
> -
>
> Key: KAFKA-9840
> URL: https://issues.apache.org/jira/browse/KAFKA-9840
> Project: Kafka
>  Issue Type: Bug
>  Components: consumer
>Affects Versions: 2.4.1
>Reporter: Jason Gustafson
>Assignee: Boyang Chen
>Priority: Major
> Fix For: 2.6.0
>
>
> We have observed a case where the consumer attempted to detect truncation 
> with the OffsetsForLeaderEpoch API against a broker which had become a 
> zombie. In this case, the last epoch known to the consumer was higher than 
> the last epoch known to the zombie broker, so the broker returned -1 as both 
> the end offset and epoch in the response. The consumer did not check for this 
> in the response, which resulted in the following message:
> {code}
> Truncation detected for partition topic-1 at offset 
> FetchPosition{offset=11859, offsetEpoch=Optional[46], 
> currentLeader=LeaderAndEpoch{leader=broker-host (id: 3 rack: null), 
> epoch=-1}}, resetting offset to the first offset known to diverge 
> FetchPosition{offset=-1, offsetEpoch=Optional[-1], 
> currentLeader=LeaderAndEpoch{broker-host (id: 3 rack: null), epoch=-1}} 
> (org.apache.kafka.clients.consumer.internals.SubscriptionState:414)
> {code}
> There are a couple ways we the consumer can handle this situation better. 
> First, the reason we did not detect the zombie broker is that we did not 
> include the current leader epoch in the OffsetForLeaderEpoch request. This 
> was likely because of KAFKA-9212. Following this patch, we would not 
> initialize the current leader epoch from metadata responses because there are 
> cases that we cannot rely on it. But if the client cannot rely on being able 
> to detect zombies, then the epoch validation is less useful anyway. So the 
> simple solution is to not bother with the validation unless we have a 
> reliable current leader epoch.
> Second, the consumer needs to check for the case when the returned offset and 
> epoch are not defined. In this case, we have to treat this as a normal 
> OffsetOutOfRange case and invoke the reset policy. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

2020-06-06 Thread Alexandre Dupriez

Hi Satish,

A couple of questions specific to the section "Follower
Requests/Replication", pages 16:17 in the design document [1].

900. It is mentioned that followers fetch auxiliary states from the
remote storage.

900.a Does the consistency model of the external storage impacts reads
of leader epochs and other auxiliary data?

900.b What are the benefits of using a mechanism to store and access
the leader epochs which is different from other metadata associated to
tiered segments? What are the benefits of retrieving this information
on-demand from the follower rather than relying on propagation via the
topic __remote_log_metadata? What are the advantages over using a
dedicated control structure (e.g. a new record type) propagated via
this topic? Since in the document, different control paths are
operating in the system, how are the metadata stored in
__remote_log_metadata [which also include the epoch of the leader
which offloaded a segment] and the remote auxiliary states, kept in
sync?

900.c A follower can encounter an OFFSET_MOVED_TO_TIERED_STORAGE. Is
this in response to a Fetch or OffsetForLeaderEpoch request?

900.d What happens if, after a follower encountered an
OFFSET_MOVED_TO_TIERED_STORAGE response, its attempts to retrieve
leader epochs fail (for instance, because the remote storage is
temporarily unavailable)? Does the follower fallbacks to a mode where
it ignores tiered segments, and applies truncation using only locally
available information? What happens when access to the remote storage
is restored? How is the replica lineage inferred by the remote leader
epochs reconciled with the follower's replica lineage, which has
evolved? Does the follower remember fetching auxiliary states failed
in the past and attempt reconciliation? Is there a plan to offer
different strategies in this scenario, configurable via configuration?

900.e Is the leader epoch cache offloaded with every segment? Or when
a new checkpoint is detected? If that information is not always
offloaded to avoid duplicating data, how does the remote storage
satisfy the request to retrieve it?

900.f Since the leader epoch cache covers the entire replica lineage,
what happens if, after a leader epoch cache file is offloaded with a
given segment, the local epoch cache is truncated [not necessarily for
a range of offset included in tiered segments]? How are remote and
local leader epoch caches kept consistent?

900.g Consumer can also use leader epochs (e.g. to enable fencing to
protect against stale leaders). What differences would there be
between consumer and follower fetches? Especially, would consumers
also fetch leader epoch information from the remote storage?

900.h Assume a newly elected leader of a topic-partition detects more
recent segments are available in the external storage, with epochs >
its local epoch. Does it ignore these segments and their associated
epoch-to-offset vectors? Or try to reconstruct its local replica
lineage based on the data remotely available?

Thanks,
Alexandre

[1] 
https://docs.google.com/document/d/18tnobSas3mKFZFr8oRguZoj_tkD_sGzivuLRlMloEMs/edit?usp=sharing

Le jeu. 4 juin 2020 à 19:55, Satish Duggana  a écrit :
>
> Hi Jun,
> Please let us know if you have any comments on "transactional support"
> and "follower requests/replication" mentioned in the wiki.
>
> Thanks,
> Satish.
>
> On Tue, Jun 2, 2020 at 9:25 PM Satish Duggana  
> wrote:
> >
> > Thanks Jun for your comments.
> >
> > >100. It would be useful to provide more details on how those apis are 
> > >used. Otherwise, it's kind of hard to really assess whether the new apis 
> > >are sufficient/redundant. A few examples below.
> >
> > We will update the wiki and let you know.
> >
> > >100.1 deleteRecords seems to only advance the logStartOffset in Log. How 
> > >does that trigger the deletion of remote log segments?
> >
> > RLMTask for leader partition periodically checks whether there are
> > remote log segments earlier to logStartOffset and the respective
> > remote log segment metadata and data are deleted by using RLMM and
> > RSM.
> >
> > >100.2 stopReplica with deletion is used in 2 cases (a) replica 
> > >reassignment; (b) topic deletion. We only want to delete the tiered 
> > >metadata in the second case. Also, in the second case, who initiates the 
> > >deletion of the remote segment since the leader may not exist?
> >
> > Right, it is deleted only incase of topic deletion only. We will cover
> > the details in the KIP.
> >
> > >100.3 "LogStartOffset of a topic can be either in local or in remote 
> > >storage." If LogStartOffset exists in both places, which one is the source 
> > >of truth?
> >
> > I meant the logStartOffset can point to either of local segment or
> > remote segment but it is initialised and maintained in the Log class
> > like now.
> >
> > >100.4 List listRemoteLogSegments(TopicPartition 
> > >topicPartition, long minOffset): How is minOffset supposed to be used?
> >
> > Returns list of remote segments,

Build failed in Jenkins: kafka-2.6-jdk8 #24

2020-06-06 Thread Apache Jenkins Server

See 


Changes:

[matthias] MINOR: improve code encapsulation between StreamThread and 
TaskManager


--
[...truncated 6.06 MB...]

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldForwardRecordsFromSubtopologyToSubtopology[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldNotUpdateStoreForSmallerValue[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldNotUpdateStoreForSmallerValue[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldCreateStateDirectoryForStatefulTopology[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldCreateStateDirectoryForStatefulTopology[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldPunctuateIfWallClockTimeAdvances[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldPunctuateIfWallClockTimeAdvances[Eos enabled = false] PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordsFromKeyValuePairs STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordsFromKeyValuePairs PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithNullKeyAndDefaultTimestamp 
STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithNullKeyAndDefaultTimestamp 
PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithDefaultTimestamp 
STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithDefaultTimestamp 
PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithNullKey STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithNullKey PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateNullKeyConsumerRecordWithOtherTopicNameAndTimestampWithTimetamp 
STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateNullKeyConsumerRecordWithOtherTopicNameAndTimestampWithTimetamp 
PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordWithTimestamp STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordWithTimestamp PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullHeaders STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullHeaders PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithDefaultTimestamp STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithDefaultTimestamp PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicName STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicName PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithNullKey STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithNullKey PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecord STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecord PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateNullKeyConsumerRecord STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateNullKeyConsumerRecord PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordWithOtherTopicName STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordWithOtherTopicName PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > shouldAdvanceTime 
STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > shouldAdvanceTime 
PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithKeyValuePairs STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithKeyValuePairs PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithKeyValuePairsAndCustomTimestamps
 STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest >

Jenkins build is back to normal : kafka-trunk-jdk14 #193

2020-06-06 Thread Apache Jenkins Server

See

Build failed in Jenkins: kafka-2.6-jdk8 #23

2020-06-06 Thread Apache Jenkins Server

See 


Changes:

[rhauch] KAFKA-9570: Define SSL configs in all worker config classes, not just

[jason] KAFKA-9840; Skip End Offset validation when the leader epoch is not

[konstantine] KAFKA-9851: Revoking Connect tasks due to connectivity issues 
should


--
[...truncated 4.87 MB...]
kafka.admin.TopicCommandWithZKClientTest > testInvalidTopicLevelConfig PASSED

kafka.admin.TopicCommandWithZKClientTest > testAlterConfigs STARTED

kafka.admin.TopicCommandWithZKClientTest > testAlterConfigs PASSED

kafka.admin.TopicCommandWithZKClientTest > 
testConfigPreservationAcrossPartitionAlteration STARTED

kafka.admin.TopicCommandWithZKClientTest > 
testConfigPreservationAcrossPartitionAlteration PASSED

kafka.admin.TopicCommandWithZKClientTest > 
testTopicOperationsWithRegexSymbolInTopicName STARTED

kafka.admin.TopicCommandWithZKClientTest > 
testTopicOperationsWithRegexSymbolInTopicName PASSED

kafka.admin.TopicCommandWithZKClientTest > testCreateWithConfigs STARTED

kafka.admin.TopicCommandWithZKClientTest > testCreateWithConfigs PASSED

kafka.admin.TopicCommandWithZKClientTest > testAlterIfExists STARTED

kafka.admin.TopicCommandWithZKClientTest > testAlterIfExists PASSED

kafka.admin.TopicCommandWithZKClientTest > 
testDescribeAndListTopicsWithoutInternalTopics STARTED

kafka.admin.TopicCommandWithZKClientTest > 
testDescribeAndListTopicsWithoutInternalTopics PASSED

kafka.admin.TopicCommandWithZKClientTest > 
testCreateWithUnspecifiedReplicationFactorAndPartitionsWithZkClient STARTED

kafka.admin.TopicCommandWithZKClientTest > 
testCreateWithUnspecifiedReplicationFactorAndPartitionsWithZkClient PASSED

kafka.admin.TopicCommandWithZKClientTest > testCreateWithReplicaAssignment 
STARTED

kafka.admin.TopicCommandWithZKClientTest > testCreateWithReplicaAssignment 
PASSED

kafka.admin.TopicCommandWithZKClientTest > testAlterWithInvalidPartitionCount 
STARTED

kafka.admin.TopicCommandWithZKClientTest > testAlterWithInvalidPartitionCount 
PASSED

kafka.admin.TopicCommandWithZKClientTest > testCreate STARTED

kafka.admin.TopicCommandWithZKClientTest > testCreate PASSED

kafka.admin.TopicCommandWithZKClientTest > 
testDescribeAndListTopicsMarkedForDeletion STARTED

kafka.admin.TopicCommandWithZKClientTest > 
testDescribeAndListTopicsMarkedForDeletion PASSED

kafka.admin.TopicCommandWithZKClientTest > testDeleteIfExists STARTED

kafka.admin.TopicCommandWithZKClientTest > testDeleteIfExists PASSED

kafka.admin.TopicCommandWithZKClientTest > testDescribe STARTED

kafka.admin.TopicCommandWithZKClientTest > testDescribe PASSED

kafka.admin.TopicCommandWithZKClientTest > testAlterAssignment STARTED

kafka.admin.TopicCommandWithZKClientTest > testAlterAssignment PASSED

kafka.admin.TimeConversionTests > testDateTimeFormats STARTED

kafka.admin.TimeConversionTests > testDateTimeFormats PASSED

kafka.common.ZkNodeChangeNotificationListenerTest > testProcessNotification 
STARTED

kafka.common.ZkNodeChangeNotificationListenerTest > testProcessNotification 
PASSED

kafka.common.ZkNodeChangeNotificationListenerTest > 
testSwallowsProcessorException STARTED

kafka.common.ZkNodeChangeNotificationListenerTest > 
testSwallowsProcessorException PASSED

kafka.utils.CoreUtilsTest > testGenerateUuidAsBase64 STARTED

kafka.utils.CoreUtilsTest > testGenerateUuidAsBase64 PASSED

kafka.utils.CoreUtilsTest > testAbs STARTED

kafka.utils.CoreUtilsTest > testAbs PASSED

kafka.utils.CoreUtilsTest > testReplaceSuffix STARTED

kafka.utils.CoreUtilsTest > testReplaceSuffix PASSED

kafka.utils.CoreUtilsTest > testCircularIterator STARTED

kafka.utils.CoreUtilsTest > testCircularIterator PASSED

kafka.utils.CoreUtilsTest > testReadBytes STARTED

kafka.utils.CoreUtilsTest > testReadBytes PASSED

kafka.utils.CoreUtilsTest > testCsvList STARTED

kafka.utils.CoreUtilsTest > testCsvList PASSED

kafka.utils.CoreUtilsTest > testReadInt STARTED

kafka.utils.CoreUtilsTest > testReadInt PASSED

kafka.utils.CoreUtilsTest > testAtomicGetOrUpdate STARTED

kafka.utils.CoreUtilsTest > testAtomicGetOrUpdate PASSED

kafka.utils.CoreUtilsTest > testUrlSafeBase64EncodeUUID STARTED

kafka.utils.CoreUtilsTest > testUrlSafeBase64EncodeUUID PASSED

kafka.utils.CoreUtilsTest > testCsvMap STARTED

kafka.utils.CoreUtilsTest > testCsvMap PASSED

kafka.utils.CoreUtilsTest > testInLock STARTED

kafka.utils.CoreUtilsTest > testInLock PASSED

kafka.utils.CoreUtilsTest > testTryAll STARTED

kafka.utils.CoreUtilsTest > testTryAll PASSED

kafka.utils.CoreUtilsTest > testSwallow STARTED

kafka.utils.CoreUtilsTest > testSwallow PASSED

kafka.utils.CommandLineUtilsTest > testParseEmptyArg STARTED

kafka.utils.CommandLineUtilsTest > testParseEmptyArg PASSED

kafka.utils.CommandLineUtilsTest > testMaybeMergeOptionsOverwriteExisting 
STARTED

kafka.utils.CommandLineUtilsTest > testMaybeMergeOptionsOverwriteExisting PASSED

kafka.utils.CommandLineUtilsTest >

Build failed in Jenkins: kafka-2.4-jdk8 #221

2020-06-06 Thread Apache Jenkins Server

See 


Changes:

[rhauch] KAFKA-9570: Define SSL configs in all worker config classes, not just

[konstantine] KAFKA-9851: Revoking Connect tasks due to connectivity issues 
should


--
[...truncated 2.76 MB...]

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfKeyAndValueAndTimestampIsEqualWithNullForCompareKeyValueTimestampWithProducerRecord
 STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfKeyAndValueAndTimestampIsEqualWithNullForCompareKeyValueTimestampWithProducerRecord
 PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullForCompareKeyValueWithProducerRecord STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullForCompareKeyValueWithProducerRecord PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentForCompareKeyValueTimestampWithProducerRecord 
STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentForCompareKeyValueTimestampWithProducerRecord PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullProducerRecordWithExpectedRecordForCompareValueTimestamp 
STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullProducerRecordWithExpectedRecordForCompareValueTimestamp 
PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullExpectedRecordForCompareValue STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullExpectedRecordForCompareValue PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullReversForCompareKeyValueTimestampWithProducerRecord
 STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullReversForCompareKeyValueTimestampWithProducerRecord
 PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullProducerRecordForCompareKeyValue STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullProducerRecordForCompareKeyValue PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullReversForCompareKeyValueWithProducerRecord 
STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullReversForCompareKeyValueWithProducerRecord 
PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfKeyAndValueAndTimestampIsEqualForCompareKeyValueTimestamp STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfKeyAndValueAndTimestampIsEqualForCompareKeyValueTimestamp PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullForCompareKeyValueTimestampWithProducerRecord
 STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullForCompareKeyValueTimestampWithProducerRecord
 PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfKeyIsDifferentWithNullReversForCompareKeyValueWithProducerRecord 
STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfKeyIsDifferentWithNullReversForCompareKeyValueWithProducerRecord 
PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfKeyAndValueIsEqualWithNullForCompareKeyValueWithProducerRecord 
STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfKeyAndValueIsEqualWithNullForCompareKeyValueWithProducerRecord 
PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfKeyIsDifferentWithNullForCompareKeyValueTimestampWithProducerRecord 
STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfKeyIsDifferentWithNullForCompareKeyValueTimestampWithProducerRecord 
PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentForCompareKeyValueTimestamp STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentForCompareKeyValueTimestamp PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfValueAndTimestampIsEqualForCompareValueTimestampWithProducerRecord 
STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfValueAndTimestampIsEqualForCompareValueTimestampWithProducerRecord 
PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullReversForCompareKeyValueTimestamp STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullReversForCompareKeyValueTimestamp PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullReverseForCompareValueTimestampWithProducerRecord
 STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullReverseForCompareValueTimestampWithProducerRecord
 PASSED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest >

Build failed in Jenkins: kafka-trunk-jdk11 #1545

2020-06-06 Thread Apache Jenkins Server

See 


Changes:

[github] MINOR: improve code encapsulation between StreamThread and TaskManager


--
[...truncated 6.32 MB...]
org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithNullKey STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithNullKey PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecord STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecord PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateNullKeyConsumerRecord STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateNullKeyConsumerRecord PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordWithOtherTopicName STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordWithOtherTopicName PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > shouldAdvanceTime 
STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > shouldAdvanceTime 
PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithKeyValuePairs STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithKeyValuePairs PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithKeyValuePairsAndCustomTimestamps
 STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithKeyValuePairsAndCustomTimestamps
 PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithKeyValuePairs 
STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithKeyValuePairs PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordWithOtherTopicNameAndTimestamp STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordWithOtherTopicNameAndTimestamp PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithKeyValuePairsAndCustomTimestamps
 STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithKeyValuePairsAndCustomTimestamps
 PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicName STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicName PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordsFromKeyValuePairsWithCustomTimestampAndIncrementsAndNotAdvanceTime
 STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordsFromKeyValuePairsWithCustomTimestampAndIncrementsAndNotAdvanceTime
 PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateNullKeyConsumerRecordWithTimestampWithTimestamp STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateNullKeyConsumerRecordWithTimestampWithTimestamp PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithNullKeyAndDefaultTimestamp
 STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithNullKeyAndDefaultTimestamp
 PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordsFromKeyValuePairsWithTimestampAndIncrements STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordsFromKeyValuePairsWithTimestampAndIncrements PASSED

org.apache.kafka.streams.MockTimeTest > shouldGetNanosAsMillis STARTED

org.apache.kafka.streams.MockTimeTest > shouldGetNanosAsMillis PASSED

org.apache.kafka.streams.MockTimeTest > shouldSetStartTime STARTED

org.apache.kafka.streams.MockTimeTest > shouldSetStartTime PASSED

org.apache.kafka.streams.MockTimeTest > shouldNotAllowNegativeSleep STARTED

org.apache.kafka.streams.MockTimeTest > shouldNotAllowNegativeSleep PASSED

org.apache.kafka.streams.MockTimeTest > shouldAdvanceTimeOnSleep STARTED

org.apache.kafka.streams.MockTimeTest > shouldAdvanceTimeOnSleep PASSED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > shouldReturnIsOpen 
STARTED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > shouldReturnIsOpen 
PASSED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > 
shouldDeleteAndReturnPlainValue STARTED

Build failed in Jenkins: kafka-trunk-jdk14 #192

2020-06-06 Thread Apache Jenkins Server

See 

Changes:


--
Started by an SCM change
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on H23 (ubuntu) in workspace 

[WS-CLEANUP] Deleting project workspace...
[WS-CLEANUP] Deferred wipeout is used...
[WS-CLEANUP] Done
No credentials specified
Cloning the remote Git repository
Cloning repository https://github.com/apache/kafka.git
 > git init  # timeout=10
Fetching upstream changes from https://github.com/apache/kafka.git
 > git --version # timeout=10
 > git fetch --tags --progress -- https://github.com/apache/kafka.git 
 > +refs/heads/*:refs/remotes/origin/*
ERROR: Error cloning remote repo 'origin'
hudson.plugins.git.GitException: Command "git fetch --tags --progress -- 
https://github.com/apache/kafka.git +refs/heads/*:refs/remotes/origin/*" 
returned status code 128:
stdout: 
stderr: error: cannot fork() for rev-list: Resource temporarily unavailable
error: Could not run 'git rev-list'
error: cannot fork() for fetch-pack: Resource temporarily unavailable

at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2172)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1864)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:78)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:545)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:758)
at 
org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
at 
org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to 
H23
at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1743)
at 
hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
at hudson.remoting.Channel.call(Channel.java:957)
at 
org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
at sun.reflect.GeneratedMethodAccessor1196.invoke(Unknown 
Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
at com.sun.proxy.$Proxy137.execute(Unknown Source)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1152)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1192)
at hudson.scm.SCM.checkout(SCM.java:504)
at 
hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
at 
jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1815)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at 
hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
ERROR: Error cloning remote repo 'origin'
Retrying after 10 seconds
No credentials specified
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/kafka.git # timeout=10
Fetching upstream changes from https://github.com/apache/kafka.git
 > git --version # timeout=10
 > git fetch --tags --progress -- https://github.com/apache/kafka.git 
 > +refs/heads/*:refs/remotes/origin/*
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from 
https://github.com/apache/kafka.git
at

Jenkins build is back to normal : kafka-2.4-jdk8 #220

2020-06-06 Thread Apache Jenkins Server

See

Build failed in Jenkins: kafka-trunk-jdk14 #191

2020-06-06 Thread Apache Jenkins Server

See 

Changes:


--
Started by an SCM change
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on H23 (ubuntu) in workspace 

[WS-CLEANUP] Deleting project workspace...
[WS-CLEANUP] Deferred wipeout is used...
[WS-CLEANUP] Done
No credentials specified
Cloning the remote Git repository
Cloning repository https://github.com/apache/kafka.git
 > git init  # timeout=10
Fetching upstream changes from https://github.com/apache/kafka.git
 > git --version # timeout=10
 > git fetch --tags --progress -- https://github.com/apache/kafka.git 
 > +refs/heads/*:refs/remotes/origin/*
ERROR: Error cloning remote repo 'origin'
hudson.plugins.git.GitException: Command "git fetch --tags --progress -- 
https://github.com/apache/kafka.git +refs/heads/*:refs/remotes/origin/*" 
returned status code 128:
stdout: 
stderr: error: cannot fork() for rev-list: Resource temporarily unavailable
error: Could not run 'git rev-list'
error: cannot fork() for fetch-pack: Resource temporarily unavailable

at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2172)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1864)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:78)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:545)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:758)
at 
org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
at 
org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to 
H23
at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1743)
at 
hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
at hudson.remoting.Channel.call(Channel.java:957)
at 
org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
at sun.reflect.GeneratedMethodAccessor1196.invoke(Unknown 
Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
at com.sun.proxy.$Proxy137.execute(Unknown Source)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1152)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1192)
at hudson.scm.SCM.checkout(SCM.java:504)
at 
hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
at 
jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1815)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at 
hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
ERROR: Error cloning remote repo 'origin'
Retrying after 10 seconds
No credentials specified
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/kafka.git # timeout=10
Fetching upstream changes from https://github.com/apache/kafka.git
 > git --version # timeout=10
 > git fetch --tags --progress -- https://github.com/apache/kafka.git 
 > +refs/heads/*:refs/remotes/origin/*
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from 
https://github.com/apache/kafka.git
at

[GitHub] [kafka-site] brary commented on pull request #267: Updating upgrade docs for KIP-535 and KIP-562

[GitHub] [kafka-site] brary opened a new pull request #267: Updating upgrade docs for KIP-535 and KIP-562

Build failed in Jenkins: kafka-2.6-jdk8 #25

Jenkins build is back to normal : kafka-trunk-jdk14 #195

[jira] [Resolved] (KAFKA-10097) Avoid passing task checkpoint file externally

Re: subscribe confirm

Jenkins build is back to normal : kafka-trunk-jdk11 #1546

Build failed in Jenkins: kafka-trunk-jdk14 #194

[jira] [Created] (KAFKA-10113) LogTruncationException sets fetch offsets incorrectly

subscribe confirm

[jira] [Resolved] (KAFKA-9851) Revoking Connect tasks due to connectivity issues should also clear running assignment

[jira] [Resolved] (KAFKA-10033) AdminClient should throw UnknownTopicOrPartitionException instead of UnknownServerException if altering configs of non-existing topic

[jira] [Resolved] (KAFKA-10094) In MirrorSourceConnector replace two-step assignment with single call

[jira] [Resolved] (KAFKA-9840) Consumer should not use OffsetForLeaderEpoch without current epoch validation

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

Build failed in Jenkins: kafka-2.6-jdk8 #24

Jenkins build is back to normal : kafka-trunk-jdk14 #193

Build failed in Jenkins: kafka-2.6-jdk8 #23

Build failed in Jenkins: kafka-2.4-jdk8 #221

Build failed in Jenkins: kafka-trunk-jdk11 #1545

Build failed in Jenkins: kafka-trunk-jdk14 #192

Jenkins build is back to normal : kafka-2.4-jdk8 #220

Build failed in Jenkins: kafka-trunk-jdk14 #191

23 matches

Site Navigation

Mail list logo

Footer information