[GitHub] activemq-artemis pull request #1607: ARTEMIS-1478 - ActiveMQ Artemis to Kafk...

[michaelandrepearce]

Github user michaelandrepearce commented on a diff in the pull request:

    https://github.com/apache/activemq-artemis/pull/1607#discussion_r146526503
  
    --- Diff: docs/user-manual/en/kafka-bridges.md ---
    @@ -0,0 +1,179 @@
    +# Apache ActiveMQ Kafka Bridge
    +
    +![ActiveMQ Artemis Kafka Bridge Logo](images/activemq-kafka-bridge.png)
    +
    +The function of a bridge is to consume messages from a source queue in 
Apache ActiveMQ Artemis, 
    +and forward them to a target topic, on a remote Apache Kafka server.
    +
    +By pairing Apache ActiveMQ Artemis and Apache Kafka with the bridge you 
could have a hybrid broker setup, 
    +having a data flow with CORE, AMQP, MQTT clients, as well as now Kafka 
clients also. 
    +Taking and giving the best features of both broker technologies when and 
where needed for a flow of data.
    +
    +![ActiveMQ Artemis Kafka Bridge 
Clients](images/activemq-kafka-bridge-clients.png)
    +
    +The intent is this will be a two way bridge, but currently the flow is a 
single direction 
    +from Apache ActiveMQ Artemis to Apache Kafka
    +
    +
    +The source and target servers are remote making bridging suitable
    +for reliably sending messages from one artemis cluster to kafka, 
    +for instance across a WAN, to the cloud, or internet and where the 
connection may be unreliable.
    +
    +The bridge has built in resilience to failure so if the target server
    +connection is lost, e.g. due to network failure, the bridge will retry
    +connecting to the target until it comes back online. When it comes back
    +online it will resume operation as normal.
    +
    +In summary, Apache ActiveMQ Kafka Bridge is a way to reliably connect 
separate 
    +Apache ActiveMQ Artemis server and Apache Kafka server together.
    +
    +## Configuring Kakfa Bridges
    +
    +Bridges are configured in `broker.xml`.  
    +Let's kick off
    +with an example (this is actually from the kafka bridge test example):
    +
    +
    +    <connector-services>
    +         <connector-service name="my-kafka-bridge">
    +            
<factory-class>org.apache.activemq.artemis.integration.kafka.bridge.KafkaProducerBridgeFactory</factory-class>
    +            <param key="bootstrap.servers" 
value="kafka-1.domain.local:9092,kafka-2.domain.local:9092,kafka-3.domain.local:9092"
 />
    +            <param key="queue-name" value="my.artemis.queue" />
    +            <param key="kafka-topic" value="my_kafka_topic" />
    +         </connector-service>
    +    </connector-services>
    +
    +In the above example we have shown the required parameters to
    +configure for a kakfa bridge. See below for a complete list of available 
configuration options. 
    +
    +### Serialization
    +By default the CoreMessageSerializer is used.
    +
    +#### CoreMessageSerializer
    +Default but can be explicitly set using
    +           
    +    <param key="value.serializer" 
value="org.apache.activemq.artemis.integration.kafka.protocol.core.CoreMessageSerializer"
 />
    +
    +This maps the Message properties to Record headers.
    +And then maps the payload binary as the Record value, encoding TextMessage
    +
    +This makes it easy to consume from Kafka using default deserializers
    +TextMessage using StringDeserializer and
    +ByteMessage using BytesDeserializer.
    +
    +Also to note, if you serialized an Apache Avro object into a byte array 
for the ByteMessage you could deserialize using an e
    +
    --- End diff --
    
    Good spot , Looks like something didn’t commit from my local properly 
I’ll correct.


---