2020-01-31 12:27:33 UTC - Joshua Dunham: Yes, in a general sense. A BK cluster on a remote machine accessible over network. How do I configure pulsar to use that? ---- 2020-01-31 14:19:37 UTC - Ravi Shah: Cool, Thanks! ---- 2020-01-31 14:28:01 UTC - Alexandre DUVAL: Hi, we need something that autocompact topic continually ---- 2020-01-31 14:28:27 UTC - Alexandre DUVAL: So, shall I write an issue for feature request or it's already the case? ---- 2020-01-31 15:23:14 UTC - Martin Skogevall: @Martin Skogevall has joined the channel ---- 2020-01-31 16:41:59 UTC - Ryan Samo: Hey guys,
I'm dealing with an odd scenario and need a bit of advice. I have 6 broker nodes, 3 in rack-1 and 3 in rack-2. There is a DNS record which resolves to all 6 brokers. I have noticed a lop-sided amount of load on rack-1 vs. rack-2, specifically on 1 namespace which 1 partitioned topic with 18 partitions. If I run a $PULSAR_HOME/bin/pulsar-admin brokers namespaces command, I can see that the rack-1 brokers are all serving this namespace, but the rack-2 brokers are not. I then investigated the namespace bundles and our default is 64. If you do some quick math, of 64 bundles/18 topics = 3.56 Brokers so that makes sense that only the 3 rack-1 brokers are serving the namespace, my bad. If I run the following command ($PULSAR_HOME/bin/pulsar-admin namespaces policies), it shows that my num_bundles = 129, not 64 as I assume Pulsar is trying to auto-balance. My question is what setting would keep Pulsar from utilizing the other 3 broker nodes when attempting this balancing effort and is there something I can do to change that? I would like all 6 brokers to be utilized if possible. Thanks! ---- 2020-01-31 16:45:56 UTC - Ruslan Sheremet: @Ryan Samo I think you should enable internal load balancer between brokers, also for external cluster you should use proxy nodes. ---- 2020-01-31 16:47:33 UTC - Ryan Samo: Thanks @Ruslan Sheremet, is there a command for that or is it in the broker.conf? ---- 2020-01-31 16:52:23 UTC - Ruslan Sheremet: Yes, `Load balancer` section in `broker.conf` . But you should keep internal ZK cluster for each rack, communication between racks via proxy using `proxy.conf` and external ZK cluster ---- 2020-01-31 16:56:58 UTC - Ryan Samo: Right now the racks use a shared ZK cluster which also spans both racks, so all 6 brokers are using a quorum of ZKs. The proxy.conf contains the zkServers and Configuration store servers and both are set to the same zk nodes. Is this an issue? ---- 2020-01-31 16:58:22 UTC - Ryan Samo: Load balancer is enabled on all brokers as well ---- 2020-01-31 16:59:20 UTC - Ruslan Sheremet: @Ryan Samo Yes. Global ZK should looking only to proxies. Proxies is entry-point to the internal cluster, with internal ZK, BookKeper, brokers, LB for brokers. ---- 2020-01-31 17:00:56 UTC - Ryan Samo: We don’t use global-zookeeper.conf currently, only zookeeper.conf. Would that break things? ---- 2020-01-31 17:07:44 UTC - Ruslan Sheremet: Each rack should be a `cluster` , balancing between clusters should be via proxies and Global ZK for proxies. Internal clusters should have brokers, internal ZK cluster, BookKeeper cluster. ---- 2020-01-31 17:08:34 UTC - Sijie Guo: Ah I think there was an issue tracking it. Let me search it. ---- 2020-01-31 17:08:47 UTC - Ruslan Sheremet: Each distributed Topic can be handled by several clusters. So this can be scale out to any size horizontally ---- 2020-01-31 17:10:37 UTC - Ruslan Sheremet: Please look into Helm chart <https://github.com/apache/pulsar/tree/master/deployment/kubernetes/helm/pulsar> and `values.yaml` for clusters like your ---- 2020-01-31 17:10:48 UTC - Ryan Samo: Makes sense, but what if you have already committed to splitting brokers between racks and under the same dns and cluster name, can it still not use all 6 broker nodes? ---- 2020-01-31 17:12:15 UTC - Ruslan Sheremet: DNS handling should be only on proxy nodes and global ZK cluster. ---- 2020-01-31 17:13:08 UTC - Ruslan Sheremet: All internal clusters should be behind NAT and use private IP addresses ---- 2020-01-31 17:14:59 UTC - Ryan Samo: Ok thanks ---- 2020-01-31 17:16:22 UTC - Ruslan Sheremet: @Ryan Samo You can watch video: <https://www.youtube.com/watch?v=De6avNyQUMw> about scalable architecture of Pulsar, how to scale out it to any size. ---- 2020-01-31 17:17:50 UTC - Ryan Samo: Nice! Thank you ---- 2020-01-31 17:18:28 UTC - Ruslan Sheremet: @Ryan Samo Your case on <https://youtu.be/De6avNyQUMw?t=2109> ---- 2020-01-31 17:30:41 UTC - David Kjerrumgaard: @Ryan Samo Is your concern about data availability in the event of a single rack failure or are you wanting to evenly distribute the load across racks? For the former, that is handled by the stateless nature of the brokers themselves. In the event that all 3 brokers fail at once due to a rack outage, the brokers on rack-2 would take over serving by reading the data from the bookies. ---- 2020-01-31 17:31:22 UTC - David Kjerrumgaard: The key is to ensure that the data is distributed across racks at the bookie layer. ---- 2020-01-31 17:33:04 UTC - David Kjerrumgaard: @Ryan Samo You may also want to look at failure domains and anti-affinity groups to address data availability concerns in Pulsar. <https://github.com/apache/pulsar/wiki/PIP-7:-Pulsar-Failure-domain-and-Anti-affinity-namespaces> ---- 2020-01-31 17:49:32 UTC - Ryan Samo: @David Kjerrumgaard , my concern was that I have a very heavily used topic that for some reason is only being served by half my broker hardware. I would like to spread the load. It seems that the load is containing itself within the brokers in each rack even though I haven’t set anything that I’m aware of to force that behavior. Would the anti-affinity assist with this? Shared ZK not global per-say. 6 brokers split in half in same cluster ---- 2020-01-31 17:50:51 UTC - David Kjerrumgaard: @Ryan Samo Topics are dynamically assigned to brokers based on the load conditions of all brokers in the cluster. So if one or more of the brokers serving this topic starts getting over-used, they will automatically shed some of the load to other brokers. ---- 2020-01-31 17:51:37 UTC - David Kjerrumgaard: So you really don't have to worry about these scenarios, as Pulsar handles all of that for you automatically. ---- 2020-01-31 17:52:45 UTC - Ryan Samo: Ok, thanks. I just saw when I did a lookup command on the namespace it shows up only on the 3 brokers when I assumed it would show on all. ---- 2020-01-31 17:53:53 UTC - David Kjerrumgaard: Pulsar is just trying to find a broker that can handle the load for a topic rather than a "perfect" placement.....good enough will suffice :smiley: ---- 2020-01-31 17:54:25 UTC - David Kjerrumgaard: Also, if you are really unhappy with the placement you can try using the `pulsar-admin topics unload <persistent://tenant/namespace/topic>` command to force broker reassignment ---- 2020-01-31 17:56:06 UTC - Ryan Samo: Cool, thanks! I think I was more concerned with the bundle count and maybe there is something I should have set better. I appreciate it! ---- 2020-01-31 18:04:17 UTC - David Kjerrumgaard: Bundles are a namespace level construct, and try to distribute the topics within the namespace evenly across the brokers. It does NOT affect the distribution load of an individual partitioned topic. Topics are assigned to a bundle by taking the hash of the topic name, so with 64 bundles and only 6 brokers the distribution might get skewed..... ---- 2020-01-31 18:04:57 UTC - David Kjerrumgaard: 64 bundles is better when you have 1000+ topics. ---- 2020-01-31 18:14:00 UTC - Ryan Samo: Ok, I’m this case I have a single namespace with the single 18 partitioned topic. Default is 64 and the namespace now shows 129. Should I lower that number since I’m only dealing with 1 partitioned topic under 1 namespace? Can you shrink the number of bundles @David Kjerrumgaard ? ---- 2020-01-31 18:48:48 UTC - David Kjerrumgaard: @Ryan Samo The number of bundles is set when you create the namespace and can only ever increase AFAIK ---- 2020-01-31 18:49:17 UTC - Ryan Samo: Ok thanks! Good to know, learning a lot here :+1: ---- 2020-01-31 18:52:36 UTC - Ryan Samo: Manual unload is working by the way! +1 : David Kjerrumgaard ---- 2020-01-31 18:52:38 UTC - David Kjerrumgaard: Glad to help. In general you want the number of bundles to be equal to the number of brokers. Since it just a hashing function, you want the probability of an given topic's hash to be equal for all brokers. ---- 2020-01-31 18:53:13 UTC - David Kjerrumgaard: once you exceed the number of brokers, the probability distribution starts getting skewed. ---- 2020-01-31 18:54:02 UTC - David Kjerrumgaard: 64 values have to be mapped back to 6 brokers, and since 64 isn't evenly divisible, you will have some brokers who get more topics assigned to them than the others. ---- 2020-01-31 18:55:24 UTC - David Kjerrumgaard: The other unknown is how "busy" each topic is in terms of traffic. We cannot assume they are all equal, and in some scenarios one broker could end up with ALL the busy ones and the rest without much load at all. This is where the load shedding comes into play. :smiley: ---- 2020-01-31 18:56:33 UTC - Ryan Samo: I see, ok perfect! Thanks for bearing with me. I’ll be more cautious on namespace creations from now on ---- 2020-01-31 18:57:05 UTC - David Kjerrumgaard: no worries. Glad to help.......trial and error is a great way to learn ---- 2020-01-31 18:57:58 UTC - Ryan Samo: Sure is, trial by fire more like haha fire_engine : David Kjerrumgaard fire : Ryan Samo ---- 2020-01-31 19:02:35 UTC - Mikhail Markov: Hello guys ! I am writing a function and have a question. How to access messages property inside process function? I have only two variable 1. Raw message - String 2. Context any ideas ? ---- 2020-01-31 19:06:37 UTC - Ali Ahmed: @Ravi Shah Here you go <https://github.com/apache/pulsar/pull/6180/files> ---- 2020-01-31 19:08:05 UTC - Jerry Peng: @Mikhail Markov you can do the following using the Java API: ```Record record = context.getRecord(); properties = record.getProperties()``` ---- 2020-01-31 19:09:39 UTC - Mikhail Markov: @Jerry Peng You are awesome ! Thanks a lot :) ---- 2020-01-31 19:10:07 UTC - Jerry Peng: np! ---- 2020-01-31 19:34:00 UTC - Joshua Dunham: I think I've hit bug #2020 - Caused by: org.apache.bookkeeper.statelib.api.exceptions.StateStoreException: Failed to restore rocksdb ---- 2020-01-31 19:34:34 UTC - Joshua Dunham: Is there a way to recover from that? I've tried exec into the container (standalone) and ./bin/bookkeeper autorecovery but it didn't have any effect. ---- 2020-01-31 20:32:21 UTC - tcourdy: @tcourdy has joined the channel ---- 2020-02-01 00:43:09 UTC - ravi satya durga prasad Yenugula: @David Kjerrumgaard Its working but I am still getting this html error what is this ```<!doctype html><html lang="en"><head><title>HTTP Status 500 – Internal Server Error</title><style type="text/css">h1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} h2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} h3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} body {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} b {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} p {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;} a {color:black;} a.name {color:black;} .line {height:1px;background-color:#525D76;border:none;}</style></head><body><h1>HTTP Status 500 – Internal Server Error</h1></body></html>``` ---- 2020-02-01 00:47:13 UTC - ravi satya durga prasad Yenugula: Due this I am unable to save the Enviroment ---- 2020-02-01 00:47:38 UTC - ravi satya durga prasad Yenugula: Looks like Pulsar is not a user friendly :disappointed: ---- 2020-02-01 00:51:06 UTC - ravi satya durga prasad Yenugula: Hi everyone, I am trying to just setup pulsar in my enviroment to learn and try but I was unable to . Can any one help with a proper docker image and documentation it would be great . Thanks ---- 2020-02-01 00:51:26 UTC - ravi satya durga prasad Yenugula: ---- 2020-02-01 00:55:05 UTC - ravi satya durga prasad Yenugula: I was able to reach to this step after the community help by providing their docker-compose file / references by still it failed with some error. So proper documentation for the beginner would be apricated and It will help the community to grow / learn ---- 2020-02-01 01:20:54 UTC - Sijie Guo: you need to add the broker service url. `http://<broker-address>:8080` The container running Pulsar-Manager should be able to reach to the service url that you provided. The error you saw seems to indicate that you are not able to reach to the service url. ---- 2020-02-01 01:21:32 UTC - Sijie Guo: It would be good to read the documentation to understand some of the concepts if you haven’t. <http://pulsar.apache.org/docs/en/concepts-overview/> ---- 2020-02-01 01:23:00 UTC - ravi satya durga prasad Yenugula: Sure I will try it again. I am running manager and a standalone image and running the it not sure why it still did not found the service ---- 2020-02-01 01:24:19 UTC - Sijie Guo: is the manager and standalone in the same network? ---- 2020-02-01 01:27:53 UTC - ravi satya durga prasad Yenugula: here are the commands I ran ```docker pull apachepulsar/pulsar-manager:v0.1.0 docker run -it -p 9527:9527 -e REDIRECT_HOST=<http://192.168.0.104> -e REDIRECT_PORT=9527 -e DRIVER_CLASS_NAME=org.postgresql.Driver -e URL='jdbc:<postgresql://127.0.0.1:5432/pulsar_manager>' -e USERNAME=pulsar -e PASSWORD=pulsar -e LOG_LEVEL=DEBUG apachepulsar/pulsar-manager:v0.1.0 /bin/sh``` and I got a http inter error ```<!doctype html><html lang="en"><head><title>HTTP Status 500 – Internal Server Error</title><style type="text/css">h1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} h2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} h3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} body {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} b {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} p {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;} a {color:black;} a.name {color:black;} .line {height:1px;background-color:#525D76;border:none;}</style></head><body><h1>HTTP Status 500 – Internal Server Error</h1></body></html>``` Later I used a community provided docker file and ran the `docker-compose up` ---- 2020-02-01 01:29:02 UTC - ravi satya durga prasad Yenugula: I think they should in the same network do you see any issue in the config? ---- 2020-02-01 01:53:28 UTC - tuteng: Can you check file pulsar_manager.log? ---- 2020-02-01 01:54:09 UTC - ravi satya durga prasad Yenugula: It is in which location? ---- 2020-02-01 01:54:41 UTC - David Kjerrumgaard: Try <http://locaclhost:8080|http://locaclhost:8080> ---- 2020-02-01 01:55:06 UTC - ravi satya durga prasad Yenugula: ```This site can't be reached``` ---- 2020-02-01 01:55:30 UTC - tuteng: docker exec -it container-id /bin/bash ---- 2020-02-01 01:57:09 UTC - tuteng: I think your pulsar-manager and standalone are not in a network, you can try to create a network, or link through --link ---- 2020-02-01 01:58:12 UTC - ravi satya durga prasad Yenugula: ```OCI runtime exec failed: exec failed: container_linux.go:346: starting container process caused "exec: \"/bin/bash\": stat /bin/bash: no such file or directory": unknown``` ---- 2020-02-01 01:58:43 UTC - tuteng: Oh, sorry, is `docker exec -it container-id /bin/sh` ---- 2020-02-01 01:59:17 UTC - tuteng: You can try 'curl <http://pulsar:8080/metrics/>' in container pulsar-manager ---- 2020-02-01 01:59:36 UTC - ravi satya durga prasad Yenugula: "I think your pulsar-manager and standalone are not in a network, you can try to create a network, or link through --link" @David Kjerrumgaard any idea? ---- 2020-02-01 02:00:59 UTC - David Kjerrumgaard: The docker compose should do the port forwarding from the containers to your local host ---- 2020-02-01 02:01:39 UTC - ravi satya durga prasad Yenugula: ```curl : The remote name could not be resolved: 'pulsar' At line:1 char:1 + curl <http://pulsar:8080/metrics/> + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand``` ---- 2020-02-01 02:02:41 UTC - tuteng: Use name of container name standalone to replace your domain name `pulsar` ---- 2020-02-01 02:02:50 UTC - ravi satya durga prasad Yenugula: do I need to mention the port in the compose file? ---- 2020-02-01 02:03:53 UTC - ravi satya durga prasad Yenugula: do you mean ```<http://standalone:8080/metrics/>``` if so its did not work ---- 2020-02-01 02:05:29 UTC - tuteng: Please take a look at the screenshot for me. `docker ps` ---- 2020-02-01 02:05:38 UTC - ravi satya durga prasad Yenugula: ok ---- 2020-02-01 02:06:20 UTC - ravi satya durga prasad Yenugula: ---- 2020-02-01 02:07:37 UTC - tuteng: curl <http://personaldocs_standalone_1:8080/metrics/> ---- 2020-02-01 02:10:40 UTC - ravi satya durga prasad Yenugula: not working ---- 2020-02-01 02:10:47 UTC - tuteng: If not, please restart pulsar-manager with the following command, `docker rm pulsar-manager-container-id docker run -it --link personaldocs_standalone_1 -p 9527:9527 -e REDIRECT_HOST=<http://192.168.0.104> -e REDIRECT_PORT=9527 -e DRIVER_CLASS_NAME=org.postgresql.Driver -e URL='jdbc:<postgresql://127.0.0.1:5432/pulsar_manager>' -e USERNAME=pulsar -e PASSWORD=pulsar -e LOG_LEVEL=DEBUG apachepulsar/pulsar-manager:v0.1.0 /bin/sh` ---- 2020-02-01 02:11:32 UTC - ravi satya durga prasad Yenugula: ok ---- 2020-02-01 02:12:22 UTC - tuteng: The network between your two containers is not working, you can use the above method, or refer to this <https://docs.docker.com/engine/reference/commandline/network_create/> ---- 2020-02-01 02:13:48 UTC - ravi satya durga prasad Yenugula: `Error response from daemon: Cannot link to /personaldocs_standalone_1, as it does not belong to the default network.` ---- 2020-02-01 02:18:43 UTC - tuteng: <https://stackoverflow.com/questions/36489696/cannot-link-to-a-running-container-started-by-docker-compose> ---- 2020-02-01 02:19:58 UTC - ravi satya durga prasad Yenugula: ```NETWORK ID NAME DRIVER SCOPE a0b4b214efdc bridge bridge local 9a91889b999a celery_default bridge local c0d7ac59355c host host local 8e1eb8d59db3 none null local 56a0c7757de9 personaldocs_default bridge local``` ---- 2020-02-01 02:20:37 UTC - tuteng: If the network option is specified, add --network to the command line as well ---- 2020-02-01 02:21:14 UTC - tuteng: --network personaldocs_default ---- 2020-02-01 02:24:44 UTC - ravi satya durga prasad Yenugula: do you mean `docker --network` its not working ---- 2020-02-01 02:32:14 UTC - tuteng: Now, your problem is the network between your two containers, you can try `docker run -it --network personaldocs_default -p 9527:9527 -e REDIRECT_HOST=<http://192.168.0.104> -e REDIRECT_PORT=9527 -e DRIVER_CLASS_NAME=org.postgresql.Driver -e URL='jdbc:<postgresql://127.0.0.1:5432/pulsar_manager>' -e USERNAME=pulsar -e PASSWORD=pulsar -e LOG_LEVEL=DEBUG apachepulsar/pulsar-manager:v0.1.0 /bin/sh` for start pulsar-manager ---- 2020-02-01 02:35:04 UTC - ravi satya durga prasad Yenugula: still its same ---- 2020-02-01 02:35:46 UTC - ravi satya durga prasad Yenugula: ---- 2020-02-01 02:36:19 UTC - David Kjerrumgaard: The ports are specified in the compose.yml file ---- 2020-02-01 02:37:10 UTC - David Kjerrumgaard: `docker network ls` should return `standalone-manager_default` which is created automatically by docker-compose and allows the container to communicate ---- 2020-02-01 02:37:22 UTC - tuteng: Can you check pulsar-manager.log? In container of pulsar-manager /pulsar/ ---- 2020-02-01 02:37:27 UTC - ravi satya durga prasad Yenugula: :disappointed: I added it to the docker network and started the container but no use ---- 2020-02-01 02:38:46 UTC - David Kjerrumgaard: This is odd. What version docker are you running? ---- 2020-02-01 02:40:02 UTC - David Kjerrumgaard: This is the version I am using.... ---- 2020-02-01 02:40:57 UTC - David Kjerrumgaard: Change to the directory containing this file ---- 2020-02-01 02:41:32 UTC - David Kjerrumgaard: run `docker-compose up` ---- 2020-02-01 02:41:48 UTC - David Kjerrumgaard: See out from both containers ---- 2020-02-01 02:43:05 UTC - David Kjerrumgaard: navigate to <http://localhost:9527> and create the ennvironment ---- 2020-02-01 02:43:15 UTC - David Kjerrumgaard: ---- 2020-02-01 02:43:38 UTC - David Kjerrumgaard: and it works.... ---- 2020-02-01 02:43:54 UTC - David Kjerrumgaard: no other steps required...... ---- 2020-02-01 02:46:04 UTC - tuteng: ServiceUlr should is <http://personaldocs_standalone_1:8080> , <http://personaldocs_standalone_1:8080/metrics> for test ---- 2020-02-01 02:47:33 UTC - ravi satya durga prasad Yenugula: not working ---- 2020-02-01 02:48:07 UTC - ravi satya durga prasad Yenugula: only <http://localhost:9527/> is working ---- 2020-02-01 02:49:11 UTC - tuteng: Can you show me the log pulsar-manager.log? ---- 2020-02-01 02:50:42 UTC - tuteng: Is the network between your two containers open? Can you test it? Use tool telnet or ping ---- 2020-02-01 02:51:25 UTC - ravi satya durga prasad Yenugula: ```C:\Users\ravi.yenugula> docker network ls NETWORK ID NAME DRIVER SCOPE a0b4b214efdc bridge bridge local 9a91889b999a celery_default bridge local c0d7ac59355c host host local 8e1eb8d59db3 none null local 56a0c7757de9 personaldocs_default bridge local``` ---- 2020-02-01 02:51:31 UTC - ravi satya durga prasad Yenugula: ```56a0c7757de9 personaldocs_default bridge local``` ---- 2020-02-01 02:53:47 UTC - ravi satya durga prasad Yenugula: I was able to log in as its a light weight image so I cant telnet ---- 2020-02-01 02:54:42 UTC - David Kjerrumgaard: @ravi satya durga prasad Yenugula What value do you have for the `volumes:` section of the `pulsar-manager` service? I used `- /Users/david/pulsar-manager/data:/data` so if you have that, it will give you an error unless you also have a `/Users/david` directory on your local machine. ---- 2020-02-01 02:54:42 UTC - tuteng: docker exec -it pulsar-manager-container-id /bin/sh ? ---- 2020-02-01 02:55:32 UTC - David Kjerrumgaard: That volume is where the manager's embedded Postgres DB stores its data. So if that is mis-configured, then you will get errors ---- 2020-02-01 02:56:24 UTC - tuteng: pulsar-manager.log file is under path /pulsar-manager. ---- 2020-02-01 02:57:32 UTC - ravi satya durga prasad Yenugula: +1 : tuteng ---- 2020-02-01 02:59:15 UTC - ravi satya durga prasad Yenugula: where did you mentioned it ? ---- 2020-02-01 03:02:03 UTC - ravi satya durga prasad Yenugula: This is the command I use ```docker run -it --network personaldocs_default -p 9527:9527 -e REDIRECT_HOST=<http://192.168.0.104> -e REDIRECT_PORT=9527 -e DRIVER_CLASS_NAME=org.postgresql.Driver -e URL='jdbc:<postgresql://127.0.0.1:5432/pulsar_manager>' -e USERNAME=pulsar -e PASSWORD=pulsar -e LOG_LEVEL=DEBUG apachepulsar/pulsar-manager:v0.1.0 /bin/sh``` I dont see volume in it ---- 2020-02-01 03:06:22 UTC - tuteng: Please add parameter -v $PWD:/data ---- 2020-02-01 03:06:56 UTC - ravi satya durga prasad Yenugula: ```docker run -it --network personaldocs_default -p 9527:9527 -e REDIRECT_HOST=<http://192.168.0.104> -e REDIRECT_PORT=9527 -e DRIVER_CLASS_NAME=org.postgresql.Driver -e URL='jdbc:<postgresql://127.0.0.1:5432/pulsar_manager>' -e USERNAME=pulsar -e PASSWORD=pulsar -e LOG_LEVEL=DEBUG apachepulsar/pulsar-manager:v0.1.0 /bin/sh v $PWD:/data``` ---- 2020-02-01 03:07:05 UTC - ravi satya durga prasad Yenugula: is this the command I need to run? ---- 2020-02-01 03:07:28 UTC - tuteng: Refer to: <http://pulsar.apache.org/docs/en/administration-pulsar-manager/>, your database no mount data directory, result start failed ---- 2020-02-01 03:25:57 UTC - ravi satya durga prasad Yenugula: yesterday we already tried it and it did not work ---- 2020-02-01 03:26:14 UTC - ravi satya durga prasad Yenugula: ```At line:1 char:249 + ... SERNAME=pulsar -e PASSWORD=pulsar -e LOG_LEVEL=DEBUG -v $PWD:/data ap ... + ~~~~~ Variable reference is not valid. ':' was not followed by a valid variable name character. Consider using ${} to delimit the name. + CategoryInfo : ParserError: (:) [], ParentContainsErrorRecordException + FullyQualifiedErrorId : InvalidVariableReferenceWithDrive``` ---- 2020-02-01 03:31:09 UTC - tuteng: What is your operating system? ---- 2020-02-01 03:31:37 UTC - ravi satya durga prasad Yenugula: Windows and PWD worked in the power shell ---- 2020-02-01 03:34:31 UTC - tuteng: $v="/data" $PWD:$v ---- 2020-02-01 03:34:38 UTC - tuteng: Have you tried this? ---- 2020-02-01 03:35:00 UTC - tuteng: There is no windows environment on my side ---- 2020-02-01 03:35:31 UTC - ravi satya durga prasad Yenugula: I did not understand what you are tying to do ---- 2020-02-01 03:36:01 UTC - ravi satya durga prasad Yenugula: can you please provide me the full command ---- 2020-02-01 03:37:12 UTC - tuteng: ```$v="/data" docker run -it --network personaldocs_default -p 9527:9527 -e REDIRECT_HOST=<http://192.168.0.104> -e REDIRECT_PORT=9527 -e DRIVER_CLASS_NAME=org.postgresql.Driver -e URL='jdbc:<postgresql://127.0.0.1:5432/pulsar_manager>' -e USERNAME=pulsar -e PASSWORD=pulsar -e LOG_LEVEL=DEBUG -v $PWD:$v apachepulsar/pulsar-manager:v0.1.0 /bin/sh``` ---- 2020-02-01 03:37:56 UTC - ravi satya durga prasad Yenugula: nope it did not work ```At line:1 char:280 + ... SERNAME=pulsar -e PASSWORD=pulsar -e LOG_LEVEL=DEBUG -v $PWD:$v apach ... + ~~~~~ Variable reference is not valid. ':' was not followed by a valid variable name character. Consider using ${} to delimit the name. + CategoryInfo : ParserError: (:) [], ParentContainsErrorRecordException + FullyQualifiedErrorId : InvalidVariableReferenceWithDrive``` ---- 2020-02-01 03:40:14 UTC - tuteng: ```$v="/data" docker run -it --network personaldocs_default -p 9527:9527 -e REDIRECT_HOST=<http://192.168.0.104> -e REDIRECT_PORT=9527 -e DRIVER_CLASS_NAME=org.postgresql.Driver -e URL='jdbc:<postgresql://127.0.0.1:5432/pulsar_manager>' -e USERNAME=pulsar -e PASSWORD=pulsar -e LOG_LEVEL=DEBUG -v ${PWD}:${v} apachepulsar/pulsar-manager:v0.1.0 /bin/sh``` ---- 2020-02-01 03:41:14 UTC - ravi satya durga prasad Yenugula: Its Running..... Let's hope it works ---- 2020-02-01 03:45:54 UTC - ravi satya durga prasad Yenugula: server started but no use still I am getting the same old error ```<!doctype html><html lang="en"><head><title>HTTP Status 500 – Internal Server Error</title><style type="text/css">h1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} h2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} h3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} body {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} b {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} p {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;} a {color:black;} a.name {color:black;} .line {height:1px;background-color:#525D76;border:none;}</style></head><body><h1>HTTP Status 500 – Internal Server Error</h1></body></html>``` ---- 2020-02-01 03:46:13 UTC - ravi satya durga prasad Yenugula: :face_with_head_bandage: ---- 2020-02-01 03:47:36 UTC - tuteng: Can you check log pulsar-manager.log? ---- 2020-02-01 03:49:32 UTC - ravi satya durga prasad Yenugula: ---- 2020-02-01 03:50:02 UTC - ravi satya durga prasad Yenugula: ---- 2020-02-01 03:53:50 UTC - tuteng: ```It seems that it is still the mistake just now. The database did not start successfully. Error querying database. Cause: org.springframework.jdbc.CannotGetJdbcConnectionException: Failed to obtain JDBC Connection; nested exception is org.postgresql.util.PSQLException: Connection to 127.0.0.1:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.``` ---- 2020-02-01 03:54:34 UTC - ravi satya durga prasad Yenugula: I am using the image should handel it right ---- 2020-02-01 03:56:28 UTC - tuteng: Are you familiar with how to mount the directory correctly for docker under windows ---- 2020-02-01 03:57:09 UTC - ravi satya durga prasad Yenugula: nope is it using ```--voume=local-file-path``` ---- 2020-02-01 03:59:17 UTC - tuteng: --voume? ---- 2020-02-01 03:59:27 UTC - tuteng: <https://stackoverflow.com/questions/41485217/mount-current-directory-as-a-volume-in-docker-on-windows-10> ---- 2020-02-01 04:00:03 UTC - tuteng: docker run -it --network personaldocs_default -p 9527:9527 -e REDIRECT_HOST=<http://192.168.0.104> -e REDIRECT_PORT=9527 -e DRIVER_CLASS_NAME=org.postgresql.Driver -e URL='jdbc:<postgresql://127.0.0.1:5432/pulsar_manager>' -e USERNAME=pulsar -e PASSWORD=pulsar -e LOG_LEVEL=DEBUG -v ${PWD}:/data apachepulsar/pulsar-manager:v0.1.0 /bin/sh ---- 2020-02-01 04:01:00 UTC - tuteng: hope this will work properly. ---- 2020-02-01 05:52:13 UTC - Joe Francis: You can try rebuilding the bookie index bin/bookkeeper shell rebuild-db-ledger-locations-index ----
