> So when the network is down between master and slave (eg slaves network card fails)... the master will keep ACK-ing messages it receieves in this case?
In general, yes. Like I said before, the master can be configured to initiate a quorum vote and will shut itself down if it's isolated. > If so -- is there some timeout setting that tells the master how long it should wait before considering the slave as failed? As soon as the network between the master and slave fails then the master considers the slave as failed. This is not configurable. > Aren't [messages, acks, etc] stored as transactions within the broker journal Only records which *need* to be stored as part of a transaction are stored as such. Lots of records stand on their own, not as a part of a transaction. All that said, I'm not sure it's really relevant. > So if there is an incomplete/corrupt transaction in the journal wouldn't the broker just roll it back and ignore it? Essentially, yes. At least that's what I understand from reading the code [1]. Justin [1] https://github.com/apache/activemq-artemis/blob/master/artemis-journal/src/main/java/org/apache/activemq/artemis/core/journal/impl/JournalImpl.java#L488 On Sun, Jun 23, 2019 at 7:24 PM warm-sun <warm-...@tutanota.com> wrote: > >>> Technically speaking, replication is asynchronous. However, the broker > will not send a response to the client until it has received a reply from > the slave that the data has been received. > ... > >>> If the network between the master and slave goes down then by default > >>> the master continues like nothing happened. > > So when the network is down between master and slave (eg slaves network > card > fails)... the master will keep ACK-ing messages it receieves in this case? > If so -- is there some timeout setting that tells the master how long it > should wait before considering the slave as failed? > > ----- > >>> For what it's worth, I've found that data integrity and high > performance > >>> and generally at odds with each other. > Agreed. A balance has to be found. We will be replicating with the built-in > replication on the LAN within the same data centers... and replicating via > asynchronous DRBD across data centers (best effort -- but not guaranteed) > > ----- > >>> I'm not sure the kind of data loss you're describing has ever been > >>> tested. As far as I know, Artemis expects the data it writes to the > >>> journal to still be in the journal when it is re-loaded. In general, I > >>> would expect that any solution designed to ensure data integrity would > >>> consider message loss or imperfect replication an unacceptable failure. > > A small amount of message loss may be acceptable across data center > replication -- but corrupt journals that a broker cannot use may not be. A > Couple of questions on this very important point: > 1) > Aren't [messages, acks, etc] stored as transactions within the broker > journal ie: > begin > message > end > > So if there is an incomplete/corrupt transaction in the journal wouldn't > the > broker just roll it back and ignore it? > This would lead to message loss but not journal corruption. Is this how the > journal / broker work? > > 2) > What if only journal blocks that were full were replicated to the other > data > center... would this ensure an uncorrupted journal? ie does the broker / > journal only store complete messages at the end of journal blocks (as the > journal block is getting full)? Another way to say this: can the journal > store 1/2 a message at the end of 1 journal block and the other 1/2 at the > next journal block? > > Case 1) would seem to be the preferable solution (imho)... is a feature > request for this likely to be implemented? > > > > -- > Sent from: > http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html >
