[GitHub] [pulsar] jm7647 commented on issue #5454: mysql JDBC Sink - consumer error

[GitBox]

jm7647 commented on issue #5454: mysql JDBC Sink - consumer error
URL: https://github.com/apache/pulsar/issues/5454#issuecomment-547456603
 
 
   **Tried to do localrun. Below is result from main broker log.**
   _(Notice "Closing consumer: 0" - next "Closing consumer: 1" - it will 
continue in a loop)_
   
   
  14:25:12.806 [pulsar-io-50-3] INFO  
org.apache.pulsar.broker.service.ServerCnx - New connection from 
/135.25.67.53:33764
   14:25:12.810 [pulsar-io-50-3] INFO  
org.apache.pulsar.broker.service.ServerCnx - 
[/135.25.67.53:33764][persistent://public/default/pulsar-mysql-jdbc-sink-topic] 
Creating producer. producerId=0
   14:25:12.817 [ForkJoinPool.commonPool-worker-3] INFO  
org.apache.pulsar.broker.service.ServerCnx - [/135.25.67.53:33764] Created new 
producer: 
Producer{topic=PersistentTopic{topic=persistent://public/default/pulsar-mysql-jdbc-sink-topic},
 client=/135.25.67.53:33764, producerName=standalone-8-51, producerId=0}
   14:25:12.818 [bookkeeper-ml-workers-OrderedExecutor-0-0] WARN  
org.apache.bookkeeper.client.BookieWatcherImpl - New ensemble: [127.0.0.1:3181] 
is not adhering to Placement Policy. quarantinedBookies: []
   14:25:12.831 [pulsar-ordered-OrderedExecutor-7-0-EventThread] INFO  
org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for 
ledger: 25036
   14:25:12.831 [pulsar-ordered-OrderedExecutor-7-0-EventThread] INFO  
org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - 
[public/default/persistent/pulsar-mysql-jdbc-sink-topic] Created new ledger 
25036
   14:25:12.836 [pulsar-io-50-3] INFO  
org.apache.pulsar.broker.service.ServerCnx - 
[PersistentTopic{topic=persistent://public/default/pulsar-mysql-jdbc-sink-topic}][standalone-8-51]
 Closing producer on cnx /135.25.67.53:33764
   14:25:12.836 [pulsar-io-50-3] INFO  
org.apache.pulsar.broker.service.ServerCnx - 
[PersistentTopic{topic=persistent://public/default/pulsar-mysql-jdbc-sink-topic}][standalone-8-51]
 Closed producer on cnx /135.25.67.53:33764
   14:25:12.844 [pulsar-io-50-3] INFO  
org.apache.pulsar.broker.service.ServerCnx - Closed connection from 
/135.25.67.53:33764
   14:25:12.922 [pulsar-io-50-2] INFO  
org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:42854] Closing 
consumer: 0
   14:25:12.922 [pulsar-io-50-2] INFO  
org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumers
 - Removed consumer 
Consumer{subscription=PersistentSubscription{topic=persistent://public/default/pulsar-mysql-jdbc-sink-topic,
 name=public/default/pulsar-mysql-jdbc-sink}, consumerId=0, consumerName=d4dfc, 
address=/127.0.0.1:42854} with pending 1 acks
   14:25:12.922 [pulsar-io-50-2] INFO  
org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:42854] Closed consumer 
Consumer{subscription=PersistentSubscription{topic=persistent://public/default/pulsar-mysql-jdbc-sink-topic,
 name=public/default/pulsar-mysql-jdbc-sink}, consumerId=0, consumerName=d4dfc, 
address=/127.0.0.1:42854}
   14:25:14.810 [pulsar-io-50-2] INFO  
org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:42854] Subscribing on 
topic persistent://public/default/pulsar-mysql-jdbc-sink-topic / 
public/default/pulsar-mysql-jdbc-sink
   14:25:14.813 [ForkJoinPool.commonPool-worker-0] INFO  
org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - 
[public/default/persistent/pulsar-mysql-jdbc-sink-topic-public%2Fdefault%2Fpulsar-mysql-jdbc-sink]
 Rewind from 25036:1 to 25036:0
   14:25:14.813 [ForkJoinPool.commonPool-worker-0] INFO  
org.apache.pulsar.broker.service.persistent.PersistentTopic - 
[persistent://public/default/pulsar-mysql-jdbc-sink-topic] There are no 
replicated subscriptions on the topic
   14:25:14.813 [ForkJoinPool.commonPool-worker-0] INFO  
org.apache.pulsar.broker.service.persistent.PersistentTopic - 
[persistent://public/default/pulsar-mysql-jdbc-sink-topic][public/default/pulsar-mysql-jdbc-sink]
 Created new subscription for 1
   14:25:14.813 [ForkJoinPool.commonPool-worker-0] INFO  
org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:42854] Created 
subscription on topic persistent://public/default/pulsar-mysql-jdbc-sink-topic 
/ public/default/pulsar-mysql-jdbc-sink
   14:25:14.818 [pulsar-io-50-2] INFO  
org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:42854] Closing 
consumer: 1
   14:25:14.818 [pulsar-io-50-2] INFO  
org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumers
 - Removed consumer 
Consumer{subscription=PersistentSubscription{topic=persistent://public/default/pulsar-mysql-jdbc-sink-topic,
 name=public/default/pulsar-mysql-jdbc-sink}, consumerId=1, consumerName=03170, 
address=/127.0.0.1:42854} with pending 1 acks
   
   
   **And here is log from SINK - it is showing same NULL  error:**
   
   
   14:22:15.535 [pulsar-client-io-1-1] INFO  
org.apache.pulsar.client.impl.ConnectionPool - [[id: 0x1643772d, 
L:/127.0.0.1:42854 - R:localhost/127.0.0.1:6650]] Connected to server
   14:22:15.535 [pulsar-client-io-1-1] INFO  

[GitHub] [pulsar] jm7647 commented on issue #5454: mysql JDBC Sink - consumer error

[GitBox]

jm7647 commented on issue #5454: mysql JDBC Sink - consumer error
URL: https://github.com/apache/pulsar/issues/5454#issuecomment-546101921
 
 
   This  is producer code I used based on your sample - still same error:
   
   producer = client.create_producer(
   topic='pulsar-mysql-jdbc-sink-topic',
   properties={"EVENT": "INSERT"},
   schema=AvroSchema(Test))
   
   r = Test(id=1, name="Hello World")
   producer.send(r)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] jm7647 commented on issue #5454: mysql JDBC Sink - consumer error

[GitBox]

jm7647 commented on issue #5454: mysql JDBC Sink - consumer error
URL: https://github.com/apache/pulsar/issues/5454#issuecomment-545940153
 
 
   I was hoping you have some python example.
   There is nothing I could find on your web side regarding producer for that 
sink.
   All links related to builtin connectors are broken:
   
   https://pulsar.apache.org/docs/en/io-connectors/
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services