2020-03-02 09:50:54 UTC - Sijie Guo: `extraServerComponents` only enables the table service in bookkeeper to be used by function state.
in conf/functions_worker.yml, you need to configure `stateStorageServiceUrl` to enable the state. ---- 2020-03-02 10:30:30 UTC - Manuel Mueller: hello again, I am trying to follow the debezium guide (<https://pulsar.apache.org/docs/en/io-debezium-source/#configuration-2>) and seem to have a problem understanding how this is working. The connector seems to scan the changes in the database and publish this into -> `debezium-postgres-topic` … In step 4 it says I am supposed to create a consumer for “public/default/dbserver1.inventory.products” but this topic would not be “produced into” … ? I would understand if the debezium connector would create a topic for each table, but it does not? In addition - there seems to be an “offtopic” topic to be created -> what is this for? ---- 2020-03-02 10:49:52 UTC - Manuel Mueller: running it seems not to produce anything into - { “entriesAddedCounter” : 0, “numberOfEntries” : 0, “totalSize” : 0, “currentLedgerEntries” : 0, “currentLedgerSize” : 0, “lastLedgerCreatedTimestamp” : “2020-03-02T10:41:30.516Z”, “waitingCursorsCount” : 0, “pendingAddEntriesCount” : 0, “lastConfirmedEntry” : “117:-1”, “state” : “LedgerOpened”, “ledgers” : [ { “ledgerId” : 117, “entries” : 0, “size” : 0, “offloaded” : false } ], “cursors” : { } } ---- 2020-03-02 10:50:17 UTC - Vincent LE MAITRE: So I need to configure these 2 options, am i right ? It seems helm deployment template does not support these options yet. I will try to update this. Thanks ---- 2020-03-02 12:55:16 UTC - yannick: regarding schema in a topic the docs are very "light" when it comes to protobuf. what i dont 100% understand is how i save the schema. can i save the `.proto` within a pulsar topic ? or what is the format ---- 2020-03-02 13:17:49 UTC - Hans Martin Kristensen: @Hans Martin Kristensen has joined the channel ---- 2020-03-02 13:35:02 UTC - Hans Martin Kristensen: @Hans Martin Kristensen has joined the channel ---- 2020-03-02 13:45:20 UTC - Tamer: It will create a topic per table, first make sure that the `Source` is running successfully. Then list all topics to see the newly created topics for your table ---- 2020-03-02 13:46:38 UTC - Manuel Mueller: ah perfect - then there is already a problem - it doesn’t do it. All i can see right now is a “failed first connection” attempt as potential error. Thanks a lot for this information, cause this allows me to look into the right direction ---- 2020-03-02 13:47:10 UTC - Tamer: I have CDC running for more than 6 weeks now, pulling data from Postgres RDS with no issues. Once it is configured properly it will work ---- 2020-03-02 13:47:28 UTC - Kannan: Running pulsar in k8s with istio (enabled with mTLS), broker pods restarting with following errors. ```13:45:02.118 [main] INFO org.apache.bookkeeper.meta.zk.ZKMetadataDriverBase - Initialize zookeeper metadata driver with external zookeeper client : ledgersRootPath = /ledgers. 13:45:02.119 [main] WARN org.apache.bookkeeper.util.EventLoopUtil - Could not use Netty Epoll event loop: Could not initialize class io.netty.channel.epoll.EpollEventLoop 13:45:02.120 [main] WARN org.apache.bookkeeper.client.TopologyAwareEnsemblePlacementPolicy - Failed to resolve network location for 10.240.0.32, using default rack for it : /default-rack. 13:45:02.120 [main] INFO org.apache.bookkeeper.client.RackawareEnsemblePlacementPolicyImpl - Initialize rackaware ensemble placement policy @ <Bookie:10.240.0.32:0> @ /default-rack : org.apache.pulsar.zookeeper.ZkBookieRackAffinityMapping. 13:45:02.121 [main] INFO org.apache.bookkeeper.client.RackawareEnsemblePlacementPolicyImpl - Not weighted 13:45:02.121 [main] INFO org.apache.bookkeeper.client.BookKeeper - Weighted ledger placement is not enabled 13:45:02.124 [BookKeeperClientScheduler-OrderedScheduler-0-0] WARN org.apache.bookkeeper.client.TopologyAwareEnsemblePlacementPolicy - Failed to resolve network location for 10.240.0.65, using default rack for it : /default-rack. 13:45:02.124 [BookKeeperClientScheduler-OrderedScheduler-0-0] INFO <http://org.apache.bookkeeper.net|org.apache.bookkeeper.net>.NetworkTopologyImpl - Adding a new node: /default-rack/10.240.0.65:3181 13:45:02.124 [BookKeeperClientScheduler-OrderedScheduler-0-0] WARN org.apache.bookkeeper.client.TopologyAwareEnsemblePlacementPolicy - Failed to resolve network location for 10.240.0.73, using default rack for it : /default-rack. 13:45:02.124 [BookKeeperClientScheduler-OrderedScheduler-0-0] INFO <http://org.apache.bookkeeper.net|org.apache.bookkeeper.net>.NetworkTopologyImpl - Adding a new node: /default-rack/10.240.0.73:3181 13:45:02.156 [main] INFO org.eclipse.jetty.server.Server - jetty-9.4.20.v20190813; built: 2019-08-13T21:28:18.144Z; git: 84700530e645e812b336747464d6fbbf370c9a20; jvm 1.8.0_232-b09 13:45:02.190 [main] INFO org.eclipse.jetty.server.session - DefaultSessionIdManager workerName=node0 13:45:02.190 [main] INFO org.eclipse.jetty.server.session - No SessionScavenger set, using defaults 13:45:02.191 [main] INFO org.eclipse.jetty.server.session - node0 Scavenging every 660000ms 13:45:03.331 [main] INFO org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.s.ServletContextHandler@3d43fe{/,null,AVAILABLE} 13:45:03.439 [main] INFO org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.s.ServletContextHandler@74ad8d05{/,null,AVAILABLE} 13:45:04.000 [main] INFO org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.s.ServletContextHandler@40368a46{/admin,null,AVAILABLE} 13:45:04.339 [main] INFO org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.s.ServletContextHandler@53e3a87a{/admin/v2,null,AVAILABLE} 13:45:04.520 [main] INFO org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.s.ServletContextHandler@4dafba3e{/admin/v3,null,AVAILABLE} 13:45:04.601 [main] INFO org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.s.ServletContextHandler@7e9da981{/lookup,null,AVAILABLE} 13:45:04.602 [main] INFO org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.s.ServletContextHandler@4ed38226{/metrics,null,AVAILABLE} 13:45:04.604 [main] INFO org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.s.h.ContextHandler@676ff3b0{/static,null,AVAILABLE} 13:45:04.617 [main] INFO org.eclipse.jetty.server.AbstractConnector - Started PulsarServerConnector@1280851e{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} 13:45:04.617 [main] INFO org.eclipse.jetty.server.Server - Started @5846ms 13:45:04.618 [main] INFO org.apache.pulsar.broker.web.WebService - HTTP Service started at <http://0.0.0.0:8080> 13:45:04.618 [main] INFO org.apache.pulsar.broker.web.WebService - HTTPS Service disabled 13:45:04.619 [main] INFO org.apache.pulsar.broker.PulsarService - Starting name space service, bootstrap namespaces=[] 13:45:04.668 [main] INFO org.apache.pulsar.broker.loadbalance.LeaderElectionService - LeaderElectionService started 13:45:04.672 [main] INFO org.apache.pulsar.broker.loadbalance.LeaderElectionService - Broker [<http://pulsaristio-broker:8080>] is the follower now. Waiting for the watch to trigger... 13:45:04.690 [ForkJoinPool.commonPool-worker-0] INFO org.apache.pulsar.broker.cache.LocalZooKeeperCacheService - Successfully created local policies for /admin/local-policies/pulsar/pulsaristio/pulsaristio-broker:8080 -- Optional.empty 13:45:04.696 [main] INFO org.apache.pulsar.broker.namespace.OwnershipCache - Trying to acquire ownership of pulsar/pulsaristio/pulsaristio-broker:8080/0x00000000_0xffffffff 13:45:04.719 [pulsar-ordered-OrderedExecutor-7-0-EventThread] INFO org.apache.pulsar.broker.namespace.OwnershipCache - Successfully acquired ownership of /namespace/pulsar/pulsaristio/pulsaristio-broker:8080/0x00000000_0xffffffff 13:45:04.720 [main] INFO org.apache.pulsar.broker.namespace.NamespaceService - added heartbeat namespace name in local cache: ns=pulsar/pulsaristio/pulsaristio-broker:8080 13:45:04.720 [pulsar-3-1] INFO org.apache.pulsar.broker.PulsarService - Loading all topics on bundle: pulsar/pulsaristio/pulsaristio-broker:8080/0x00000000_0xffffffff 13:45:04.732 [main] INFO org.apache.pulsar.broker.PulsarService - Starting load management service ... 13:45:04.768 [main] ERROR org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl - Broker znode - [/loadbalance/brokers/pulsaristio-broker:8080] is own by different zookeeper-ssession 144117162184736776 13:45:04.769 [main] ERROR org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl - Unable to create znode - [/loadbalance/brokers/pulsaristio-broker:8080] for load balance on zookeeper org.apache.pulsar.broker.PulsarServerException: Broker-znode owned by different zk-session 144117162184736776 at org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl.start(ModularLoadManagerImpl.java:808) [org.apache.pulsar-pulsar-broker-2.5.0.jar:2.5.0] at org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerWrapper.start(ModularLoadManagerWrapper.java:110) [org.apache.pulsar-pulsar-broker-2.5.0.jar:2.5.0] at org.apache.pulsar.broker.PulsarService.startLoadManagementService(PulsarService.java:640) [org.apache.pulsar-pulsar-broker-2.5.0.jar:2.5.0] at org.apache.pulsar.broker.PulsarService.start(PulsarService.java:490) [org.apache.pulsar-pulsar-broker-2.5.0.jar:2.5.0] at org.apache.pulsar.PulsarBrokerStarter$BrokerStarter.start(PulsarBrokerStarter.java:264) [org.apache.pulsar-pulsar-broker-2.5.0.jar:2.5.0] at org.apache.pulsar.PulsarBrokerStarter.main(PulsarBrokerStarter.java:329) [org.apache.pulsar-pulsar-broker-2.5.0.jar:2.5.0] 13:45:04.772 [main] ERROR org.apache.pulsar.broker.PulsarService - org.apache.pulsar.broker.PulsarServerException: Broker-znode owned by different zk-session 144117162184736776 org.apache.pulsar.broker.PulsarServerException: org.apache.pulsar.broker.PulsarServerException: Broker-znode owned by different zk-session 144117162184736776 at org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl.start(ModularLoadManagerImpl.java:824) ~[org.apache.pulsar-pulsar-broker-2.5.0.jar:2.5.0] at org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerWrapper.start(ModularLoadManagerWrapper.java:110) ~[org.apache.pulsar-pulsar-broker-2.5.0.jar:2.5.0] at org.apache.pulsar.broker.PulsarService.startLoadManagementService(PulsarService.java:640) ~[org.apache.pulsar-pulsar-broker-2.5.0.jar:2.5.0] at org.apache.pulsar.broker.PulsarService.start(PulsarService.java:490) [org.apache.pulsar-pulsar-broker-2.5.0.jar:2.5.0] at org.apache.pulsar.PulsarBrokerStarter$BrokerStarter.start(PulsarBrokerStarter.java:264) [org.apache.pulsar-pulsar-broker-2.5.0.jar:2.5.0] at org.apache.pulsar.PulsarBrokerStarter.main(PulsarBrokerStarter.java:329) [org.apache.pulsar-pulsar-broker-2.5.0.jar:2.5.0] Caused by: org.apache.pulsar.broker.PulsarServerException: Broker-znode owned by different zk-session 144117162184736776 at org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl.start(ModularLoadManagerImpl.java:808) ~[org.apache.pulsar-pulsar-broker-2.5.0.jar:2.5.0] ... 5 more 13:45:04.773 [main] ERROR org.apache.pulsar.PulsarBrokerStarter - Failed to start pulsar service. ``` ---- 2020-03-02 13:47:54 UTC - Manuel Mueller: yea, I just followed the guideline literally - maybe it is some docker thing which did not work. ---- 2020-03-02 13:50:54 UTC - Tamer: Which Postgres version do you use? ---- 2020-03-02 13:52:19 UTC - Manuel Mueller: Currently their demo docker has psql (9.6.11) ---- 2020-03-02 13:54:57 UTC - Manuel Mueller: I think I will try to use my “own” setup and check if there are any errors rather than following theirs ---- 2020-03-02 13:55:06 UTC - Manuel Mueller: Will update this thread with the result ---- 2020-03-02 14:10:00 UTC - Ryan Slominski: Yeah, I'm considering topic compaction too. By itself it seems like it just cleans up old messages, but paired with a producer that periodically also publishes snapshots then you can combining snapshot topic and change topic into one. I'm not sure if that is better or worse than separate topic for snapshots and changes. One of the downsides is that getting the current state of entities always requires being prepared to rewind to latest snapshot and replay. With dedicated snapshot topic you could just grab latest message off snapshot topic. Also need to think through schema supporting different message payloads, plus retention/expiration considerations. Lots to consider. ---- 2020-03-02 14:20:04 UTC - Vincent LE MAITRE: When trying to run my function with "--state-storage-service-url <bk://service-name:3181>", the function execution does not start. Here is the function localrun output : `bin/pulsar-admin functions create --jar /tmp/pulsar-functions-0.0.1-SNAPSHOT-jar-with-dependencies.jar --classname com.hardisgroup.n1.poc.ExampleAggregateFunction --inputs <persistent://tenant_HardisGroup/example/aggregate-in> --tenant tenant_HardisGroup --namespace example` . ---- 2020-03-02 14:28:36 UTC - Vincent LE MAITRE: And here are the function logs file : `14:20:54,740 INFO [tenant_HardisGroup/example/ExampleAggregateFunction-0] [instance: 0] JavaInstanceRunnable - Starting Java Instance ExampleAggregateFunction :` Details = tenant: "tenant_HardisGroup" namespace: "example" name: "ExampleAggregateFunction" className: "com.hardisgroup.n1.poc.ExampleAggregateFunction" autoAck: true parallelism: 1 source { typeClassName: "[B" inputSpecs { key: "`<persistent://tenant_HardisGroup/example/aggregate-in>"` value { } } cleanupSubscription: true } sink { typeClassName: "[B" } resources { cpu: 1.0 ram: 1073741824 disk: 10737418240 } componentType: FUNCTION 14:20:54,750 INFO [tenant_HardisGroup/example/ExampleAggregateFunction-0] [instance: 0] JavaInstanceRunnable - Load JAR: /tmp/pulsar-functions-0.0.1-SNAPSHOT-jar-with-dependencies.jar 14:20:54,780 WARN [tenant_HardisGroup/example/ExampleAggregateFunction-0] [instance: 0] NarClassLoader - /tmp/pulsar-nar/pulsar-functions-0.0.1-SNAPSHOT-jar-with-dependencies.jar-unpacked does not contain META-INF/bundled-dependencies! 14:20:54,781 INFO [tenant_HardisGroup/example/ExampleAggregateFunction-0] [instance: 0] JavaInstanceRunnable - Initialize function class loader for function ExampleAggregateFunction at function cache manager ---- 2020-03-02 14:29:05 UTC - Vincent LE MAITRE: I do not understand where is my mistake... ---- 2020-03-02 15:53:06 UTC - Alexander Ursu: For sure! I'm still making updates to this as I go along, but am willing to share it as it evolves. How might I be able to get started? ---- 2020-03-02 17:06:21 UTC - Sijie Guo: <http://pulsar.apache.org/en/contributing/> ---- 2020-03-02 17:06:26 UTC - Sijie Guo: Here is the contributing guide. ---- 2020-03-02 19:04:39 UTC - Alexander Ursu: Question about the pulsar proxy when using service discovery through zookeeper. Is it not required to specify the port for `zookeeperServers` , and how come the port for the `configurationStoreServers` is 2184 but when initialising the metadata and setting `configurationStoreServers` for the brokers its port 2181? Just running a single cluster using docker containers btw. ---- 2020-03-02 19:15:55 UTC - Sijie Guo: you can provide the protobuf generated class. Pulsar java client will extract the schema info from the protobuf class. ---- 2020-03-02 19:17:16 UTC - Sijie Guo: the pod crashed and k8s attempted to restart the pod. but the znode is not expired yet. ---- 2020-03-02 19:17:49 UTC - Sijie Guo: I think you need to check why the broker crashed the first time. ---- 2020-03-02 19:19:41 UTC - Sijie Guo: the proxy should point to the local zookeeper cluster not the configuration store. hence the port is the default port 2181 ---- 2020-03-02 19:22:02 UTC - Alexander Ursu: There are variables for both in `proxy.conf` . I've been looking at this page here (<https://pulsar.apache.org/docs/en/administration-proxy/#option-1-use-service-discovery>) ---- 2020-03-02 19:57:29 UTC - Ryan Slominski: Looks like "./bin/pulsar-admin functions list" doesn't show local runner instances? +1 : Konstantinos Papalias ---- 2020-03-02 19:58:20 UTC - Ryan Slominski: Maybe because I'm using version 1.4.0 local function runner with a 1.5.0 server? 1.5.0 runner is broken... ---- 2020-03-02 20:01:32 UTC - Sijie Guo: localrunners don’t register functions into the system. those are managed externally out side of Pulsar. So `functions list` will not return those local runners. ---- 2020-03-02 20:06:03 UTC - Ryan Slominski: How do I debug then? For example I have a working function that simply echo input topic String into output topic String, but if I try to do something fancy like get previous input from state store (and push input into state store) then output the concatenation of current messages and previous then the local runner launches fine, but the function doesn't seem be invoked. The consumer client is never notified of any messages even after a producer client feeds messages into the input <http://topic.ca|topic.> ---- 2020-03-02 20:14:13 UTC - Ryan Slominski: Maybe I need to cleanup the state on the server after compiling a new implementation for a function? ---- 2020-03-02 20:16:26 UTC - Ryan Slominski: Log file doesn't show errors, but does show subscriptions being made and it looks like the function is referred to as topic itself: `Created subscription on topic <persistent://public/default/input> / public/default/myfunction` ---- 2020-03-02 20:21:49 UTC - Ryan Slominski: Weird... it's like using the context state breaks local runner functions silently. If I comment out state lines, recompile and re-run the function is all-of-the sudden honored again. Is this a known issue? ---- 2020-03-02 20:45:49 UTC - Ryan Slominski: Probably this issue: <https://github.com/apache/pulsar/issues/6215> ---- 2020-03-02 21:21:25 UTC - Kiran Chitturi: Hi, Is there any advantage to running proxy pods in a kubernetes deployment? We currently deploy pulsar without proxy and resolve broker addresses via service DNS. Is anyone running proxy pods along with broker pods and what’s the advantage of using proxy pods vs broker service DNS? Thanks! ---- 2020-03-02 21:21:32 UTC - Ryan Slominski: ```functionConfig.setRuntimeFlags("--state_storage_serviceurl <bk://localhost:4181>");``` ---- 2020-03-02 21:21:55 UTC - Ryan Slominski: Guessing something like that is needed. Still haven't gotten it to work yet. ---- 2020-03-02 21:23:44 UTC - Addison Higham: @Kiran Chitturi they are really useful if you are trying to expose pulsar outside the cluster, but if you only need it in cluster, I would say no proxy is fine ---- 2020-03-02 21:24:52 UTC - Kiran Chitturi: Thanks! yeah, our usecase is only internal right now ---- 2020-03-02 21:30:21 UTC - Mathieu Druart: @Sijie Guo Hi, we are still investigating this issue even if we have some trouble to find the most light example to reproduce it. Just to let you know, it seems correlated to the BatchMessageIdImpl implementation of MessageId. When the MessageIds are only regular MessageIdImpl the startingOffsets seems to work just fine, but not when it's BatchMessageIdImpl. And we are experiencing maybe another issue : we try to use the Java PulsarAdmin to get the last MessageId of a topic (admin.topics().getLastMessageId), but this method only gives us MessageIdImpl and not BatchMessageIdImpl, so we miss the batchIndex inside the Id ... but maybe we are doing something wrong ... ---- 2020-03-02 21:33:19 UTC - Mathieu Druart: we will continue our investigations tomorrow before loging any issues ---- 2020-03-02 21:38:02 UTC - Ryan Slominski: Are Pulsar Functions a good choice for conditionally publishing to a topic based on multiple input topics? For example if input topic A has value "1" and B has value "Hello" then publish "match" to output topic otherwise do nothing. My concerns are that there appears to be no way to determine which topic the message came from and if the input topics have different schema this may also be problematic (one could be a more complex object for example). Finally, it isn't clear how one would get the value of the other input topic - I guess either use the state saving feature of Functions to save state and once you have both values you can start checking for matches. Alternatively you could possibly use Client API to query the other topic (the one not triggering the function)? Thoughts? Maybe I should just create a custom client for this and forget Pulsar Functions? ---- 2020-03-02 23:23:16 UTC - Antti Kaikkonen: > My concerns are that there appears to be no way to determine which topic the message came from `context.getCurrentRecord().getTopicName()` I'm not sure if the input topics can have different schemas. ---- 2020-03-03 01:30:47 UTC - Ken Huang: Hi The [presto REST API](<https://pulsar.apache.org/docs/en/sql-rest-api/>) document says : If you use authentication, you must use the same username. The "username" here is the same as role in the pulsar? I try to use it, but it seems only read the config in "conf/presto/catalog/pulsar.properties" ```pulsar.auth-plugin = org.apache.pulsar.client.impl.auth.AuthenticationToken pulsar.auth-params = file:///pulsar/manning/security/authentication/jwt/admin-token.txt``` I want to query the message by token of role. What can I do, thanks~ ---- 2020-03-03 02:36:51 UTC - Eric Simon: I saw a reference to this issue being when TLS is enabled. I believe its for any instance of FunctionWorker that is not on the broker. Is that accurate? ---- 2020-03-03 02:44:30 UTC - Sijie Guo: for debugging localrunner, you can checkout the log files for the function. ---- 2020-03-03 02:45:34 UTC - Sijie Guo: for localrun accessing state, you need to point to the bookkeeper cluster. If you are running pulsar in a docker container environment, the bookkeeper service is not exposed publicly. ---- 2020-03-03 02:48:50 UTC - Sijie Guo: no presto has it is own authentication system. you need to configure the presto following presto’s documentation: <https://prestodb.io/docs/current/develop/password-authenticator.html> ---- 2020-03-03 02:49:42 UTC - Sijie Guo: correct ---- 2020-03-03 04:28:42 UTC - Prasad Reddy: @Prasad Reddy has joined the channel ---- 2020-03-03 05:04:24 UTC - futeng: @futeng has joined the channel ---- 2020-03-03 05:56:31 UTC - futeng: Hi I successfully used TLS authentication based on this <http://pulsar.apache.org/docs/en/security-tls-authentication/|Authentication using TLS>. The admin user (or role) is used in the documentation and the Broker configuration role name `superUserRoles=admin` is also required. But can I use other users as well? After all, in reality, the client user is different. I tried to generate a second client certificate, such as the common name is `admin-futeng` , and I have `admin-futeng.key-pk8.pem` and `admin-futeng.cert.pem` , which has been configured into `conf/client.conf`, but the `bin/pulsar-client` can not connect to the broker with this certificate. ---- 2020-03-03 07:12:05 UTC - futeng: I tested the other users, and it worked:grin:. I wonder what happened to the last user. ---- 2020-03-03 07:43:30 UTC - yannick: ugh, thats a cumbersome approach. plus we dont use java. some of the design decisions are really weird. ---- 2020-03-03 08:49:21 UTC - bleacha0721: @bleacha0721 has joined the channel ---- 2020-03-03 09:04:10 UTC - Svetislav Ćirić: @Svetislav Ćirić has joined the channel ----
