Hi, I have a hyperledger fabric cluster that uses a cluster of 3 kafka nodes + 3 zookeepers. Fabric doesn't support pruning yet so any change to recorded offsets is detected by it and it fails to reconnect to kafka cluster. Today morning all kafka nodes have gone offline and then slowly restarted. After that fabric doesn't work anymore. After checking the logs I've learnt that fabric has recorded last offset of 13356
2019-06-27 10:34:51.325 UTC [orderer/consensus/kafka] newChain -> INFO 144 [channel: audit] Starting chain with last persisted offset 13356 and last recorded block 2881 ,however when I compared it with kafka logs of the first node that started (kafka-0), I've found out that, although initially highwater mark was around this value, for some reason, during reboot, the node truncated the topic and I've lost all messages. The topic audit has 1 partition and replication factor of 3. I've added logs from kafka-0 that contain any reference to audit topic. Any suggestions, why the topic's offset has been truncated to 1? Thanks in advance, Jan
