Pushpendra, Are all of the nodes whose IPs are named in the etcd cluster setting running, or is just the Ellis node running?
Does the ellis node have IP connectivity to the other nodes? If at least half of them aren't running then the cluster won't have quorum and won't be able to start. Richard From: Kumar, Pushpendra [mailto:pushpendra.ku...@intel.com] Sent: 02 March 2018 07:50 To: Richard Whitehouse (projectclearwater.org) <richard.whiteho...@projectclearwater.org> Subject: RE: Problems in manual installation of clearwater Hi Richard, I have successfully installed the ellis, bono etc, but when I am trying to download the shared_config file on ellis-1, it gives like- ubuntu@ellis-1:~$ cw-config download shared_config Error: client: etcd cluster is unavailable or misconfigured; error #0: client: endpoint http://10.224.61.19:4000 exceeded header timeout error #0: client: endpoint http://10.224.61.19:4000 exceeded header timeout No changes to configuration can be made while the configuration database does not have quorum. Restore connectivity to the uncontactable nodes in the deployment and try again. ? Local_config file is- ubuntu@ellis-1:/etc/clearwater$ cat local_config local_ip=10.224.61.19 public_ip=10.224.61.19 public_hostname=ellis-1 etcd_cluster="10.224.61.19,10.224.61.20,10.224.61.27,10.224.61.34,10.224.61.39,10.224.61.48" Thanks, Pushpendra From: Kumar, Pushpendra Sent: Thursday, March 1, 2018 6:34 PM To: Richard Whitehouse (projectclearwater.org) <richard.whiteho...@projectclearwater.org<mailto:richard.whiteho...@projectclearwater.org>> Subject: RE: Problems in manual installation of clearwater Thanks Richard, Now that dpkg problem is solved in ellis. One thing, When I update using sudo apt-get update after after setting the debian package in clearwater.list, it give likes : ubuntu@ellis-1:~$ sudo apt-get update >> update1.txt ubuntu@ellis-1:~$ cat update1.txt Hit http://security.ubuntu.com trusty-security InRelease Ign http://repo.cw-ngv.com binary/ InRelease Ign http://in.archive.ubuntu.com trusty InRelease Hit http://security.ubuntu.com trusty-security/main Sources Get:1 http://repo.cw-ngv.com binary/ Release.gpg [819 B] Get:2 http://repo.cw-ngv.com binary/ Release [1,219 B] Get:3 http://in.archive.ubuntu.com trusty-updates InRelease [65.9 kB] Hit http://security.ubuntu.com trusty-security/restricted Sources Get:4 http://repo.cw-ngv.com binary/ Packages [23.0 kB] Hit http://security.ubuntu.com trusty-security/universe Sources Hit http://security.ubuntu.com trusty-security/multiverse Sources Get:5 http://in.archive.ubuntu.com trusty-backports InRelease [65.9 kB] Hit http://security.ubuntu.com trusty-security/main amd64 Packages Hit http://security.ubuntu.com trusty-security/restricted amd64 Packages Get:6 http://in.archive.ubuntu.com trusty Release.gpg [933 B] Hit http://security.ubuntu.com trusty-security/universe amd64 Packages Get:7 http://in.archive.ubuntu.com trusty-updates/main Sources [412 kB] Hit http://security.ubuntu.com trusty-security/multiverse amd64 Packages Ign http://repo.cw-ngv.com binary/ Translation-en_IN Ign http://repo.cw-ngv.com binary/ Translation-en Hit http://security.ubuntu.com trusty-security/main i386 Packages ->Will it be create problem later or its fine? Thanks, Pushpendra From: Richard Whitehouse (projectclearwater.org) [mailto:richard.whiteho...@projectclearwater.org] Sent: Thursday, March 1, 2018 4:31 PM To: Kumar, Pushpendra <pushpendra.ku...@intel.com<mailto:pushpendra.ku...@intel.com>>; clearwater@lists.projectclearwater.org<mailto:clearwater@lists.projectclearwater.org> Subject: RE: Problems in manual installation of clearwater What user are you running this as? Can you post the full log of the install process? Richard From: Kumar, Pushpendra [mailto:pushpendra.ku...@intel.com] Sent: 01 March 2018 10:46 To: Richard Whitehouse (projectclearwater.org) <richard.whiteho...@projectclearwater.org<mailto:richard.whiteho...@projectclearwater.org>> Subject: RE: Problems in manual installation of clearwater Hi, I have created the new clean ubuntu vm(without openSSH and DNSserver), still I am getting this error while installing ellis: usermod: user ellis is currently used by process 1343 dpkg: error processing package ellis (--configure): subprocess installed post-installation script returned error exit status 8 Processing triggers for libc-bin (2.19-0ubuntu6.9) ... Processing triggers for ureadahead (0.100.0-16) ... Errors were encountered while processing: ellis E: Sub-process /usr/bin/dpkg returned an error code (1) Note: On the same machine there is other vm which has installed ellis. Should there only be one ellis on one machine or it doesn't matter? Thanks From: Richard Whitehouse (projectclearwater.org) [mailto:richard.whiteho...@projectclearwater.org] Sent: Thursday, March 1, 2018 3:28 PM To: Kumar, Pushpendra <pushpendra.ku...@intel.com<mailto:pushpendra.ku...@intel.com>>; clearwater@lists.projectclearwater.org<mailto:clearwater@lists.projectclearwater.org> Subject: RE: Problems in manual installation of clearwater Pushpendra, I think there are two reasons you are hitting problems installing Clearwater: 1) The ellis node that you've created has a user called ellis, which as I described below is required to be a system user under which the ellis processes run. I'd suggest you create the node with a different username (e.g. ubuntu or clearwater). 2) The ellis server has bind9 installed on it, prior to installing Clearwater, which is conflicting with dnsmasq which Clearwater uses for caching DNS queries. You'll need to uninstall this before installing the Clearwater software. At a guess, you selected the 'DNS server' task selection when installing the Ubuntu VM. We'd recommend that the only task selection you make as part of installing the Ubuntu VM is 'OpenSSH server', as that's useful to log into the node remotely, which allows you to copy and paste commands, and upload files easily. We'll make a change to Clearwater so it detects this misconfiguration and requires it to be corrected prior to installing Clearwater. Hope this helps! Richard From: Kumar, Pushpendra [mailto:pushpendra.ku...@intel.com] Sent: 01 March 2018 03:41 To: Richard Whitehouse <richard.whiteho...@metaswitch.com<mailto:richard.whiteho...@metaswitch.com>> Subject: RE: Problems in manual installation of clearwater Thanks for replying. I am using virtualbox for installing nodes, I have installed ellis first time on that node (for reconfirm, I installed all the nodes again on virtualbox, but same errors (dpkg). thanks From: Kumar, Pushpendra Sent: Thursday, March 1, 2018 9:07 AM To: 'Richard Whitehouse' <richard.whiteho...@metaswitch.com<mailto:richard.whiteho...@metaswitch.com>> Subject: RE: Problems in manual installation of clearwater Hi Richard, Thanks for replying. I am using virtualbox for installing nodes, I have installed ellis first time on that node (for reconfirm, I installed all the nodes again on virtualbox). This is the output of netstat -pltun: [ellis]ellis@Ellis:~$ sudo netstat -pltun [sudo] password for ellis: Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 10.224.61.25:53 0.0.0.0:* LISTEN 25075/named tcp 0 0 127.0.0.1:53 0.0.0.0:* LISTEN 25075/named tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 28512/sshd tcp 0 0 127.0.0.1:953 0.0.0.0:* LISTEN 25075/named tcp 0 0 127.0.0.1:2812 0.0.0.0:* LISTEN 9667/monit tcp 0 0 127.0.0.1:8000 0.0.0.0:* LISTEN 2019/nginx tcp 0 0 127.0.0.1:3306 0.0.0.0:* LISTEN 8998/mysqld tcp 0 0 10.224.61.25:2380 0.0.0.0:* LISTEN 12039/etcd tcp6 0 0 :::53 :::* LISTEN 25075/named tcp6 0 0 :::22 :::* LISTEN 28512/sshd tcp6 0 0 ::1:953 :::* LISTEN 25075/named tcp6 0 0 :::4000 :::* LISTEN 12039/etcd tcp6 0 0 :::80 :::* LISTEN 2019/nginx udp 0 0 10.224.61.25:53 0.0.0.0:* 25075/named udp 0 0 127.0.0.1:53 0.0.0.0:* 25075/named udp 0 0 0.0.0.0:68 0.0.0.0:* 703/dhclient udp 0 0 10.224.61.25:123 0.0.0.0:* 9178/ntpd udp 0 0 127.0.0.1:123 0.0.0.0:* 9178/ntpd udp 0 0 0.0.0.0:123 0.0.0.0:* 9178/ntpd udp 0 0 0.0.0.0:49694 0.0.0.0:* 703/dhclient udp6 0 0 :::29724 :::* 703/dhclient udp6 0 0 :::53 :::* 25075/named udp6 0 0 fe80::a00:27ff:fe11:123 :::* 9178/ntpd udp6 0 0 ::1:123 :::* 9178/ntpd udp6 0 0 :::123 :::* 9178/ntpd I am also getting 502 Bad Gateway when I trying to connect ellis using http://ellis.iind.intel.com Thanks Pushpendra From: Richard Whitehouse [mailto:richard.whiteho...@metaswitch.com] Sent: Thursday, March 1, 2018 4:21 AM To: Kumar, Pushpendra <pushpendra.ku...@intel.com<mailto:pushpendra.ku...@intel.com>>; clearwater@lists.projectclearwater.org<mailto:clearwater@lists.projectclearwater.org> Subject: RE: Problems in manual installation of clearwater Pushpendra, usermod: user ellis is currently used by process 1343 dpkg: error processing package ellis (--configure): subprocess installed post-installation script returned error exit status 8 Processing triggers for libc-bin (2.19-0ubuntu6.9) ... Processing triggers for ureadahead (0.100.0-16) ... Errors were encountered while processing: ellis E: Sub-process /usr/bin/dpkg returned an error code (1) It sounds like you attempted to install ellis on a node was installed on a node in which there already was an ellis user - is that correct? Ellis requires a user to run the components as for security so that we aren't running components as root which don't require root privileges, and this is required to be ellis. dnsmasq: failed to create listening socket for 127.0.0.1: Address already in use dnsmasq: failed to create listening socket for port 53: Address already in use $sudo DEBIAN_FRONTEND=noninteractive apt-get install clearwater-management --yes Errors were encountered while processing: ellis E: Sub-process /usr/bin/dpkg returned an error code (1) >From these logs, and the similar logs regarding bono, sprout and >clearwater-management it looks like there's already a service running on the >nodes which is bound to port 53 before you install Clearwater. We've only regularly tested performing the manual install on a clean Ubuntu box, and if I create a new Ubuntu VM (e.g. the basic Ubuntu 14.0.4 VM ) I don't see anything already running on Port 53 before I install Clearwater. ubuntu@ip-10-0-162-214:~$ sudo netstat -pltun Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1133/sshd tcp6 0 0 :::22 :::* LISTEN 1133/sshd udp 0 0 0.0.0.0:5254 0.0.0.0:* 582/dhclient udp 0 0 0.0.0.0:68 0.0.0.0:* 582/dhclient udp6 0 0 :::21727 :::* 582/dhclient Can you clarify what image you are attempting to install ellis on where you are seeing these errors, and what's already installed on the box? Can you run the above netstat command? For the error: reload: Job is not running: clearwater-monit /usr/share/clearwater/infrastructure/scripts/memcached: line 43: /etc/memcached.conf: No such file or directory reload: Job is not running: clearwater-monit Can you provide the complete output of the vellum build log? It'd be useful to know in what context this error was output - had something in the build already failed? It sounds like it failed to install memcached on the vellum node. Regarding smtp_smarthost - this needs to be an SMTP server which can send mail so that Ellis can send password recovery emails. If it's not configured then password recovery emails won't work. See the entry in http://clearwater.readthedocs.io/en/stable/Clearwater_Configuration_Options_Reference.html#core-options for details about the SMTP options. Regarding home_domain, it's described in Clearwater Options reference - see http://clearwater.readthedocs.io/en/stable/Clearwater_Configuration_Options_Reference.html#core-options - it needs to be a domain which will resolve to the P-CSCFs (e.g. the bono nodes in the deployment). It's usually also the root domain for all of the domains used. iind.intel.com might be a good choice if you can configure DNS entries under that domain - you'll need to configure the DNS entries listed in http://clearwater.readthedocs.io/en/stable/Clearwater_DNS_Usage.html in this domain You will need to complete the DNS configuration before Ellis will work - it needs a DNS entry to exist in order to communicate with the other nodes in the deployment. Richard From: Kumar, Pushpendra [mailto:pushpendra.ku...@intel.com] Sent: 28 February 2018 19:00 To: Richard Whitehouse <richard.whiteho...@metaswitch.com<mailto:richard.whiteho...@metaswitch.com>> Subject: Problems in manual installation of clearwater Hi Richards, I need your help in Clearwater project manual installation, Its on high priority so please consider that. I am installing the clearwater usingg manual installation. I have created the 6 VMs on virtualbox (using bridge adapter in network setting, used the same IP as public_ip and local_ip in local.conf) as I follow http://clearwater.readthedocs.io/en/stable/Manual_Install.html. while installing I have faced some errors (mentioned below), it will be your great help if u guide some solutions for them: One more thing as I am using bridge adapter in network, I have not did any port forwarding as mention in document (I am able to ping vm from one to another i.e. they are are communicating) 1.in installtion of ellis: $sudo DEBIAN_FRONTEND=noninteractive apt-get install ellis --yes usermod: user ellis is currently used by process 1343 dpkg: error processing package ellis (--configure): subprocess installed post-installation script returned error exit status 8 Processing triggers for libc-bin (2.19-0ubuntu6.9) ... Processing triggers for ureadahead (0.100.0-16) ... Errors were encountered while processing: ellis E: Sub-process /usr/bin/dpkg returned an error code (1) Note: First time I install ellis I got this error, then I installed again in new node from scratch then also got the same error. dnsmasq: failed to create listening socket for 127.0.0.1: Address already in use dnsmasq: failed to create listening socket for port 53: Address already in use $sudo DEBIAN_FRONTEND=noninteractive apt-get install clearwater-management --yes Errors were encountered while processing: ellis E: Sub-process /usr/bin/dpkg returned an error code (1) 2.in installation of bono: $sudo DEBIAN_FRONTEND=noninteractive apt-get install bono restund --yes dnsmasq: failed to create listening socket for 127.0.0.1: Address already in use dnsmasq: failed to create listening socket for port 53: Address already in use 3.in installation of sprout: $sudo DEBIAN_FRONTEND=noninteractive apt-get install sprout --yes dnsmasq: failed to create listening socket for port 53: Address already in use dnsmasq: failed to create listening socket for port 53: Address already in use dnsmasq: failed to create listening socket for 127.0.0.1: Address already in use dnsmasq: failed to create listening socket for 127.0.0.1: Address already in use $sudo DEBIAN_FRONTEND=noninteractive apt-get install clearwater-management --yes dnsmasq: failed to create listening socket for port 53: Address already in use dnsmasq: failed to create listening socket for 127.0.0.1: Address already in use 4.in installtion of homer: $sudo DEBIAN_FRONTEND=noninteractive apt-get install homer --yes dnsmasq: failed to create listening socket for port 53: Address already in use [fail] invoke-rc.d: initscript dnsmasq, action "start" failed. dnsmasq: failed to create listening socket for 127.0.0.1: Address already in use 5.in installtion of dime: $sudo DEBIAN_FRONTEND=noninteractive apt-get install dime clearwater-prov-tools --yes dnsmasq: failed to create listening socket for port 53: Address already in use dnsmasq: failed to create listening socket for 127.0.0.1: Address already in use 6.in installtion of vellum: * Starting DNS forwarder and DHCP server dnsmasq dnsmasq: failed to create listening socket for port 53: Address already in use * Restarting DNS forwarder and DHCP server dnsmasq dnsmasq: failed to create listening socket for 127.0.0.1: Address already in use [fail] reload: Job is not running: clearwater-monit /usr/share/clearwater/infrastructure/scripts/memcached: line 43: /etc/memcached.conf: No such file or directory reload: Job is not running: clearwater-monit -->I ignore above errors and move on to next step i.e. editing shared_config file and uploading -->shared_config file: [bono]bono@Bono:/etc/clearwater$ cat shared_config ##################################################################### # No Shared Config has been provided # Replace this file with the Shared Configuration for your deployment ##################################################################### # Deployment definitions home_domain=iind.intel.com sprout_hostname=sprout.iind.intel.com sprout_registration_store=vellum.iind.intel.com hs_hostname=hs.iind.intel.com:8888 hs_provisioning_hostname=hs.iind.intel.com:8889 homestead_impu_store=vellum.iind.intel.com ralf_hostname=ralf.iind.intel.com:10888 ralf_session_store=vellum.iind.intel.com xdms_hostname=homer.iind.intel.com:7888 chronos_hostname=vellum.iind.intel.com cassandra_hostname=vellum.iin.intel.com # Email server configuration smtp_smarthost= smtp_username=username smtp_password=password email_recovery_sender=clearwa...@example.org<mailto:email_recovery_sender=clearwa...@example.org> # Keys signup_key=secret turn_workaround=secret ellis_api_key=secret ellis_cookie_key=secret ? What would I use in smtp_smarthost= (is it localhost?) ques: Is home_domain =iind.intel.com is right? (when I ping using #ping ellis it automatically takes like ellis.iind.intel.com) , it basically the intel's domain. -->local_config : (IP ans hostname changed in every node) local_ip=10.224.61.25 public_ip=10.224.61.25 public_hostname=Ellis etcd_cluster="10.224.61.20,10.224.61.21,10.224.61.22,10.224.61.25,10.224.61.48,10.224.61.50" ? After that when I am trying to connect to ellis using http://ellis.iind.intel.com or http://10.224.61.25 it is giving like 502 Bad Gateway nginx /1.4.6 (Ubuntu). I have not did any DNS configuration yet (because I am using the intel domain or do I need to do it... in which node and how). ? I will love to hear your response Thanks Pushpendra
_______________________________________________ Clearwater mailing list Clearwater@lists.projectclearwater.org http://lists.projectclearwater.org/mailman/listinfo/clearwater_lists.projectclearwater.org
