Re: [opencontrail-dev] analytics-api not giving proper results when queried on another IP than 127.0.0.1
Hi Martin, Our analytics design requires redis to be running locally (w/ no sentinel, no clustering) on each of the analytics nodes. And as Pedro stated, we do not depend on redis being persistent - data is recreated when we connect to redis. regards, - Raj On Apr 29, 2015, at 12:23 PM, Martin Loschwitz wrote: > Sorry, > > i just rememberd a discussion Michael Renner recently had with Pedro, in > which Pedro > stated that redis doesn’t have to be persistent anyway. Okay, if that’s the > case, we will > just do away with the idea of sentinel altogether. > > Best regards > Martin > >> Am 29.04.2015 um 21:20 schrieb Martin Loschwitz : >> >> Hallo Raj, >> >> no. We’re running Redis with Sentinel here, and access to Redis is possible >> through an >> HAProxy load balancer that will ensure that nodes always land on the one >> redis node that >> is currently the master of the sentinel cluster. I don’t see how scaling out >> redis for purposes >> of performance and high availability would work in any other way. We’ll need >> the analytics >> api to be fixed to allow to specify an arbitrary ip address to connect to >> redis. Shall I fill a >> bug in Launchpad? >> >> Best regards >> Martin >> >>> Am 29.04.2015 um 21:17 schrieb Raj Reddy : >>> >>> Hi Martin, >>> >>> It appears we are relying on the fact that redis is listening on 0.0.0.0 >>> You can comment the redis.conf bindip parameter or change it to 0.0.0.0 >>> >>> regards, >>> - >>> Raj >> > signature.asc Description: Message signed with OpenPGP using GPGMail ___ Dev mailing list Dev@lists.opencontrail.org http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org
Re: [opencontrail-dev] analytics-api not giving proper results when queried on another IP than 127.0.0.1
Sorry, i just rememberd a discussion Michael Renner recently had with Pedro, in which Pedro stated that redis doesn’t have to be persistent anyway. Okay, if that’s the case, we will just do away with the idea of sentinel altogether. Best regards Martin > Am 29.04.2015 um 21:20 schrieb Martin Loschwitz : > > Hallo Raj, > > no. We’re running Redis with Sentinel here, and access to Redis is possible > through an > HAProxy load balancer that will ensure that nodes always land on the one > redis node that > is currently the master of the sentinel cluster. I don’t see how scaling out > redis for purposes > of performance and high availability would work in any other way. We’ll need > the analytics > api to be fixed to allow to specify an arbitrary ip address to connect to > redis. Shall I fill a > bug in Launchpad? > > Best regards > Martin > >> Am 29.04.2015 um 21:17 schrieb Raj Reddy : >> >> Hi Martin, >> >> It appears we are relying on the fact that redis is listening on 0.0.0.0 >> You can comment the redis.conf bindip parameter or change it to 0.0.0.0 >> >> regards, >> - >> Raj > ___ Dev mailing list Dev@lists.opencontrail.org http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org
Re: [opencontrail-dev] analytics-api not giving proper results when queried on another IP than 127.0.0.1
Hallo Raj, no. We’re running Redis with Sentinel here, and access to Redis is possible through an HAProxy load balancer that will ensure that nodes always land on the one redis node that is currently the master of the sentinel cluster. I don’t see how scaling out redis for purposes of performance and high availability would work in any other way. We’ll need the analytics api to be fixed to allow to specify an arbitrary ip address to connect to redis. Shall I fill a bug in Launchpad? Best regards Martin > Am 29.04.2015 um 21:17 schrieb Raj Reddy : > > Hi Martin, > > It appears we are relying on the fact that redis is listening on 0.0.0.0 > You can comment the redis.conf bindip parameter or change it to 0.0.0.0 > > regards, > - > Raj ___ Dev mailing list Dev@lists.opencontrail.org http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org
Re: [opencontrail-dev] analytics-api not giving proper results when queried on another IP than 127.0.0.1
Hi Martin,
It appears we are relying on the fact that redis is listening on 0.0.0.0
You can comment the redis.conf bindip parameter or change it to 0.0.0.0
regards,
-
Raj
On Apr 29, 2015, at 9:39 AM, Martin Loschwitz wrote:
> Me again,
>
> looking into
> https://github.com/Juniper/contrail-controller/blob/R2.0/src/opserver/opserver.py,
>
> it looks as if 127.0.0.1 is hardcoded there a couple of times?!
>
> Best regards
> Martin
>
>> Am 29.04.2015 um 18:36 schrieb Martin Loschwitz :
>>
>> Hello Raj,
>>
>> thanks for the pointer! Stupid me … so I am one step further now and see, in
>> the webinterface log,
>> this:
>>
>> 2015-04-29T16:28:27.942Z - error: URL
>> [http://77.247.XXX.XXX:8081/analytics/query] returned error ["Failure in
>> connection to the query DB"]
>> 2015-04-29T16:28:27.942Z - error: Error Run Query: REST Server Error:
>> Failure in connection to the query DB
>>
>> At the same time, I see the analytics API doing this:
>>
>> [2015-04-29 18:26:43] "POST /analytics/query HTTP/1.1" 500 172 0.001288
>>
>> In an strace, I see this:
>>
>> [pid 47174] connect(8, {sa_family=AF_INET, sin_port=htons(6379),
>> sin_addr=inet_addr("127.0.0.1")}, 16) = -1 EINPROGRESS (Operation now in
>> progress)
>>
>> So it looks like it is connecting to redis on the local host, albeit having
>> the
>> following in contrail-analytics-api.conf:
>>
>> [REDIS]
>> server = 77.247.XXX.XXX
>> redis_server_port = 6379
>> redis_query_port = 6379
>>
>> What’s wrong here? :(
>>
>> Best regards
>> Martin
>>
>>> Am 29.04.2015 um 18:16 schrieb Raj Reddy :
>>>
>>> by default analytics-api listens at 0.0.0.0:8081 -- both methods should
>>> have worked without any config change..
>>>
>>> You should look at what's the listen ip using
>>> netstat -anp | grep 8081..
>>>
>>> regards,
>>> -
>>> Raj
>>>
>>> On Apr 29, 2015, at 7:08 AM, Martin Gerhard Loschwitz
>>>
>>> wrote:
>>>
Folks,
I see this when trying to query the analytics-api IP address:
cloud14# curl 77.247.XXX.XXX:8081/analytics/tables
curl: (52) Empty reply from server
But when I reconfigure the API to listen on 127.0.0.1 and query that one,
I get:
cloud14# curl 127.0.0.1:8081/analytics/tables
[{"href": "http://127.0.0.1:8081/analytics/table/MessageTable";, "type":
"LOG", "name": "MessageTable"} ...
What's going on here, and how do I fix the issue?
Best regards
Martin
—
Martin Gerhard Loschwitz
OpenStack DACH e.V.
Neuköllner Str. 306
D-12357 Berlin
T: +49 1511 9305928
W: http://www.openstack-dach.org/
___
Dev mailing list
Dev@lists.opencontrail.org
http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org
>>>
>>
>>
>> ___
>> Dev mailing list
>> Dev@lists.opencontrail.org
>> http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org
>
signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Dev mailing list
Dev@lists.opencontrail.org
http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org
Re: [opencontrail-dev] Tunnel issue between agents/MX
Appears to be https://bugs.launchpad.net/juniperopenstack/+bug/1427849. -Nischal On Apr 29, 2015, at 10:17 AM, Nischal Sheth wrote: > > Hi Sylvain, > > Can you check if the agent has the following config object? > > global-vrouter-config:default-global-system-config:default-global-vrouter-config > > You can use something like: > > http://10.84.13.30:8085/Snh_ShowIFMapAgentReq?table_name=global-vrouter-config > > This object has the default encapsulation-priorities, which should be a list > that > consists of MPLSoUDP, MPLSoGRE and VXLAN. > > If the agent doesn't get this config object, it uses it's hard-coded default, > which > is MPLSoUDP. That behavior itself is incorrect (will file a bug), since it > should > choose from the encaps in the route rather than using a hard-coded value. > > The bigger issue though would be why the agent didn't get the > global-vrouter-config. > > -Nischal > > > On Apr 29, 2015, at 1:58 AM, Sylvain Afchain > wrote: > >> Hi All, >> >> We are facing a weird issue on our lab since fews days. >> >> The agents use a MPLSoUDP toward the MXs for the public network while the >> controller is sending XMPP message saying to use GRE (checked with traces). >> It seems that we didn't change anything except playing with the SNAT, thus >> with the service chaining. >> >> We tried to restart the API/Controllers without any success. The fallback >> tunnel protocol set in the agent config file is MPLSoUDP so it seems that is >> the one that is used by the agents. >> >> Checking in the instrospec of the agents we see that we have only MPLSoGRE >> as supported encap and MPLSoUDP as activated encap. >> >> We didn't yet restart the agents since we want to know/understand why we are >> in such situation ? >> >> Any help on this will be welcome... to fix...to troubleshoot :) >> >> Thanks >> >> Sylvain >> >> >> ___ >> Dev mailing list >> Dev@lists.opencontrail.org >> http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org > ___ Dev mailing list Dev@lists.opencontrail.org http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org
Re: [opencontrail-dev] Tunnel issue between agents/MX
Hi Sylvain, Can you check if the agent has the following config object? global-vrouter-config:default-global-system-config:default-global-vrouter-config You can use something like: http://10.84.13.30:8085/Snh_ShowIFMapAgentReq?table_name=global-vrouter-config This object has the default encapsulation-priorities, which should be a list that consists of MPLSoUDP, MPLSoGRE and VXLAN. If the agent doesn't get this config object, it uses it's hard-coded default, which is MPLSoUDP. That behavior itself is incorrect (will file a bug), since it should choose from the encaps in the route rather than using a hard-coded value. The bigger issue though would be why the agent didn't get the global-vrouter-config. -Nischal On Apr 29, 2015, at 1:58 AM, Sylvain Afchain wrote: > Hi All, > > We are facing a weird issue on our lab since fews days. > > The agents use a MPLSoUDP toward the MXs for the public network while the > controller is sending XMPP message saying to use GRE (checked with traces). > It seems that we didn't change anything except playing with the SNAT, thus > with the service chaining. > > We tried to restart the API/Controllers without any success. The fallback > tunnel protocol set in the agent config file is MPLSoUDP so it seems that is > the one that is used by the agents. > > Checking in the instrospec of the agents we see that we have only MPLSoGRE as > supported encap and MPLSoUDP as activated encap. > > We didn't yet restart the agents since we want to know/understand why we are > in such situation ? > > Any help on this will be welcome... to fix...to troubleshoot :) > > Thanks > > Sylvain > > > ___ > Dev mailing list > Dev@lists.opencontrail.org > http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org ___ Dev mailing list Dev@lists.opencontrail.org http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org
Re: [opencontrail-dev] analytics-api not giving proper results when queried on another IP than 127.0.0.1
Me again,
looking into
https://github.com/Juniper/contrail-controller/blob/R2.0/src/opserver/opserver.py,
it looks as if 127.0.0.1 is hardcoded there a couple of times?!
Best regards
Martin
> Am 29.04.2015 um 18:36 schrieb Martin Loschwitz :
>
> Hello Raj,
>
> thanks for the pointer! Stupid me … so I am one step further now and see, in
> the webinterface log,
> this:
>
> 2015-04-29T16:28:27.942Z - error: URL
> [http://77.247.XXX.XXX:8081/analytics/query] returned error ["Failure in
> connection to the query DB"]
> 2015-04-29T16:28:27.942Z - error: Error Run Query: REST Server Error: Failure
> in connection to the query DB
>
> At the same time, I see the analytics API doing this:
>
> [2015-04-29 18:26:43] "POST /analytics/query HTTP/1.1" 500 172 0.001288
>
> In an strace, I see this:
>
> [pid 47174] connect(8, {sa_family=AF_INET, sin_port=htons(6379),
> sin_addr=inet_addr("127.0.0.1")}, 16) = -1 EINPROGRESS (Operation now in
> progress)
>
> So it looks like it is connecting to redis on the local host, albeit having
> the
> following in contrail-analytics-api.conf:
>
> [REDIS]
> server = 77.247.XXX.XXX
> redis_server_port = 6379
> redis_query_port = 6379
>
> What’s wrong here? :(
>
> Best regards
> Martin
>
>> Am 29.04.2015 um 18:16 schrieb Raj Reddy :
>>
>> by default analytics-api listens at 0.0.0.0:8081 -- both methods should have
>> worked without any config change..
>>
>> You should look at what's the listen ip using
>> netstat -anp | grep 8081..
>>
>> regards,
>> -
>> Raj
>>
>> On Apr 29, 2015, at 7:08 AM, Martin Gerhard Loschwitz
>>
>> wrote:
>>
>>> Folks,
>>>
>>> I see this when trying to query the analytics-api IP address:
>>>
>>> cloud14# curl 77.247.XXX.XXX:8081/analytics/tables
>>> curl: (52) Empty reply from server
>>>
>>> But when I reconfigure the API to listen on 127.0.0.1 and query that one, I
>>> get:
>>>
>>> cloud14# curl 127.0.0.1:8081/analytics/tables
>>> [{"href": "http://127.0.0.1:8081/analytics/table/MessageTable";, "type":
>>> "LOG", "name": "MessageTable"} ...
>>>
>>> What's going on here, and how do I fix the issue?
>>>
>>> Best regards
>>> Martin
>>>
>>> —
>>> Martin Gerhard Loschwitz
>>> OpenStack DACH e.V.
>>> Neuköllner Str. 306
>>> D-12357 Berlin
>>>
>>> T: +49 1511 9305928
>>> W: http://www.openstack-dach.org/
>>>
>>> ___
>>> Dev mailing list
>>> Dev@lists.opencontrail.org
>>> http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org
>>
>
>
> ___
> Dev mailing list
> Dev@lists.opencontrail.org
> http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org
___
Dev mailing list
Dev@lists.opencontrail.org
http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org
Re: [opencontrail-dev] analytics-api not giving proper results when queried on another IP than 127.0.0.1
Hello Raj,
thanks for the pointer! Stupid me … so I am one step further now and see, in
the webinterface log,
this:
2015-04-29T16:28:27.942Z - error: URL
[http://77.247.XXX.XXX:8081/analytics/query] returned error ["Failure in
connection to the query DB"]
2015-04-29T16:28:27.942Z - error: Error Run Query: REST Server Error: Failure
in connection to the query DB
At the same time, I see the analytics API doing this:
[2015-04-29 18:26:43] "POST /analytics/query HTTP/1.1" 500 172 0.001288
In an strace, I see this:
[pid 47174] connect(8, {sa_family=AF_INET, sin_port=htons(6379),
sin_addr=inet_addr("127.0.0.1")}, 16) = -1 EINPROGRESS (Operation now in
progress)
So it looks like it is connecting to redis on the local host, albeit having the
following in contrail-analytics-api.conf:
[REDIS]
server = 77.247.XXX.XXX
redis_server_port = 6379
redis_query_port = 6379
What’s wrong here? :(
Best regards
Martin
> Am 29.04.2015 um 18:16 schrieb Raj Reddy :
>
> by default analytics-api listens at 0.0.0.0:8081 -- both methods should have
> worked without any config change..
>
> You should look at what's the listen ip using
> netstat -anp | grep 8081..
>
> regards,
> -
> Raj
>
> On Apr 29, 2015, at 7:08 AM, Martin Gerhard Loschwitz
>
> wrote:
>
>> Folks,
>>
>> I see this when trying to query the analytics-api IP address:
>>
>> cloud14# curl 77.247.XXX.XXX:8081/analytics/tables
>> curl: (52) Empty reply from server
>>
>> But when I reconfigure the API to listen on 127.0.0.1 and query that one, I
>> get:
>>
>> cloud14# curl 127.0.0.1:8081/analytics/tables
>> [{"href": "http://127.0.0.1:8081/analytics/table/MessageTable";, "type":
>> "LOG", "name": "MessageTable"} ...
>>
>> What's going on here, and how do I fix the issue?
>>
>> Best regards
>> Martin
>>
>> —
>> Martin Gerhard Loschwitz
>> OpenStack DACH e.V.
>> Neuköllner Str. 306
>> D-12357 Berlin
>>
>> T: +49 1511 9305928
>> W: http://www.openstack-dach.org/
>>
>> ___
>> Dev mailing list
>> Dev@lists.opencontrail.org
>> http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org
>
___
Dev mailing list
Dev@lists.opencontrail.org
http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org
Re: [opencontrail-dev] analytics-api not giving proper results when queried on another IP than 127.0.0.1
by default analytics-api listens at 0.0.0.0:8081 -- both methods should have
worked without any config change..
You should look at what's the listen ip using
netstat -anp | grep 8081..
regards,
-
Raj
On Apr 29, 2015, at 7:08 AM, Martin Gerhard Loschwitz
wrote:
> Folks,
>
> I see this when trying to query the analytics-api IP address:
>
> cloud14# curl 77.247.XXX.XXX:8081/analytics/tables
> curl: (52) Empty reply from server
>
> But when I reconfigure the API to listen on 127.0.0.1 and query that one, I
> get:
>
> cloud14# curl 127.0.0.1:8081/analytics/tables
> [{"href": "http://127.0.0.1:8081/analytics/table/MessageTable";, "type":
> "LOG", "name": "MessageTable"} ...
>
> What's going on here, and how do I fix the issue?
>
> Best regards
> Martin
>
> —
> Martin Gerhard Loschwitz
> OpenStack DACH e.V.
> Neuköllner Str. 306
> D-12357 Berlin
>
> T: +49 1511 9305928
> W: http://www.openstack-dach.org/
>
> ___
> Dev mailing list
> Dev@lists.opencontrail.org
> http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org
signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Dev mailing list
Dev@lists.opencontrail.org
http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org
[opencontrail-dev] analytics-api not giving proper results when queried on another IP than 127.0.0.1
Folks,
I see this when trying to query the analytics-api IP address:
cloud14# curl 77.247.XXX.XXX:8081/analytics/tables
curl: (52) Empty reply from server
But when I reconfigure the API to listen on 127.0.0.1 and query that one, I get:
cloud14# curl 127.0.0.1:8081/analytics/tables
[{"href": "http://127.0.0.1:8081/analytics/table/MessageTable";, "type": "LOG",
"name": "MessageTable"} ...
What's going on here, and how do I fix the issue?
Best regards
Martin
—
Martin Gerhard Loschwitz
OpenStack DACH e.V.
Neuköllner Str. 306
D-12357 Berlin
T: +49 1511 9305928
W: http://www.openstack-dach.org/
signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Dev mailing list
Dev@lists.opencontrail.org
http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org
[opencontrail-dev] Tunnel issue between agents/MX
Hi All, We are facing a weird issue on our lab since fews days. The agents use a MPLSoUDP toward the MXs for the public network while the controller is sending XMPP message saying to use GRE (checked with traces). It seems that we didn't change anything except playing with the SNAT, thus with the service chaining. We tried to restart the API/Controllers without any success. The fallback tunnel protocol set in the agent config file is MPLSoUDP so it seems that is the one that is used by the agents. Checking in the instrospec of the agents we see that we have only MPLSoGRE as supported encap and MPLSoUDP as activated encap. We didn't yet restart the agents since we want to know/understand why we are in such situation ? Any help on this will be welcome... to fix...to troubleshoot :) Thanks Sylvain ___ Dev mailing list Dev@lists.opencontrail.org http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org
