Re: "Not enough replicas available for query" after reboot
Yes, that works with consistency ALL. I restarted one of the Cassandra instances, and seems it's working again now. I'm not sure what happened. On 4 February 2016 at 23:48, Peddi, Praveen wrote: > Are you able to run queries using cqlsh with consistency ALL? > > On Feb 4, 2016, at 6:32 PM, Flavien Charlon > wrote: > > No, there was no other change. I did run "apt-get upgrade" before > rebooting, but Cassandra has not been upgraded. > > On 4 February 2016 at 22:48, Bryan Cheng wrote: > >> Hey Flavien! >> >> Did your reboot come with any other changes (schema, configuration, >> topology, version)? >> >> On Thu, Feb 4, 2016 at 2:06 PM, Flavien Charlon < >> flavien.char...@gmail.com> wrote: >> >>> I'm using the C# driver 2.5.2. I did try to restart the client >>> application, but that didn't make any difference, I still get the same >>> error after restart. >>> >>> On 4 February 2016 at 21:54, wrote: >>> >>>> What client are you using? >>>> >>>> >>>> >>>> It is possible that the client saw nodes down and has kept them marked >>>> that way (without retrying). Depending on the client, you may have options >>>> to set in RetryPolicy, FailoverPolicy, etc. A bounce of the client will >>>> probably fix the problem for now. >>>> >>>> >>>> >>>> >>>> >>>> Sean Durity >>>> >>>> >>>> >>>> *From:* Flavien Charlon [mailto:flavien.char...@gmail.com] >>>> *Sent:* Thursday, February 04, 2016 4:06 PM >>>> *To:* user@cassandra.apache.org >>>> *Subject:* Re: "Not enough replicas available for query" after reboot >>>> >>>> >>>> >>>> Yes, all three nodes see all three nodes as UN. >>>> >>>> >>>> >>>> Also, connecting from a local Cassandra machine using cqlsh, I can run >>>> the same query just fine (with QUORUM consistency level). >>>> >>>> >>>> >>>> On 4 February 2016 at 21:02, Robert Coli wrote: >>>> >>>> On Thu, Feb 4, 2016 at 12:53 PM, Flavien Charlon < >>>> flavien.char...@gmail.com> wrote: >>>> >>>> My cluster was running fine. I rebooted all three nodes (one by one), >>>> and now all nodes are back up and running. "nodetool status" shows UP for >>>> all three nodes on all three nodes: >>>> >>>> >>>> >>>> -- AddressLoad Tokens OwnsHost ID >>>> Rack >>>> >>>> UN xx.xx.xx.xx331.84 GB 1 ? >>>> d3d3a79b-9ca5-43f9-88c4-c3c7f08ca538 RAC1 >>>> >>>> UN xx.xx.xx.xx317.2 GB 1 ? >>>> de7917ed-0de9-434d-be88-bc91eb4f8713 RAC1 >>>> >>>> UN xx.xx.xx.xx 291.61 GB 1 ? >>>> b489c970-68db-44a7-90c6-be734b41475f RAC1 >>>> >>>> >>>> >>>> However, now the client application fails to run queries on the cluster >>>> with: >>>> >>>> >>>> >>>> Cassandra.UnavailableException: Not enough replicas available for query >>>> at consistency Quorum (2 required but only 1 alive) >>>> >>>> >>>> >>>> Do *all* nodes see each other as UP/UN? >>>> >>>> >>>> >>>> =Rob >>>> >>>> >>>> >>>> >>>> >>>> -- >>>> >>>> The information in this Internet Email is confidential and may be >>>> legally privileged. It is intended solely for the addressee. Access to this >>>> Email by anyone else is unauthorized. If you are not the intended >>>> recipient, any disclosure, copying, distribution or any action taken or >>>> omitted to be taken in reliance on it, is prohibited and may be unlawful. >>>> When addressed to our clients any opinions or advice contained in this >>>> Email are subject to the terms and conditions expressed in any applicable >>>> governing The Home Depot terms of business or client engagement letter. The >>>> Home Depot disclaims all responsibility and liability for the accuracy and >>>> content of this attachment and for any damages or losses arising from any >>>> inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other >>>> items of a destructive nature, which may be contained in this attachment >>>> and shall not be liable for direct, indirect, consequential or special >>>> damages in connection with this e-mail message or its attachment. >>>> >>> >>> >> >
Re: "Not enough replicas available for query" after reboot
Are you able to run queries using cqlsh with consistency ALL? On Feb 4, 2016, at 6:32 PM, Flavien Charlon mailto:flavien.char...@gmail.com>> wrote: No, there was no other change. I did run "apt-get upgrade" before rebooting, but Cassandra has not been upgraded. On 4 February 2016 at 22:48, Bryan Cheng mailto:br...@blockcypher.com>> wrote: Hey Flavien! Did your reboot come with any other changes (schema, configuration, topology, version)? On Thu, Feb 4, 2016 at 2:06 PM, Flavien Charlon mailto:flavien.char...@gmail.com>> wrote: I'm using the C# driver 2.5.2. I did try to restart the client application, but that didn't make any difference, I still get the same error after restart. On 4 February 2016 at 21:54, mailto:sean_r_dur...@homedepot.com>> wrote: What client are you using? It is possible that the client saw nodes down and has kept them marked that way (without retrying). Depending on the client, you may have options to set in RetryPolicy, FailoverPolicy, etc. A bounce of the client will probably fix the problem for now. Sean Durity From: Flavien Charlon [mailto:flavien.char...@gmail.com<mailto:flavien.char...@gmail.com>] Sent: Thursday, February 04, 2016 4:06 PM To: user@cassandra.apache.org<mailto:user@cassandra.apache.org> Subject: Re: "Not enough replicas available for query" after reboot Yes, all three nodes see all three nodes as UN. Also, connecting from a local Cassandra machine using cqlsh, I can run the same query just fine (with QUORUM consistency level). On 4 February 2016 at 21:02, Robert Coli mailto:rc...@eventbrite.com>> wrote: On Thu, Feb 4, 2016 at 12:53 PM, Flavien Charlon mailto:flavien.char...@gmail.com>> wrote: My cluster was running fine. I rebooted all three nodes (one by one), and now all nodes are back up and running. "nodetool status" shows UP for all three nodes on all three nodes: -- AddressLoad Tokens OwnsHost ID Rack UN xx.xx.xx.xx331.84 GB 1 ? d3d3a79b-9ca5-43f9-88c4-c3c7f08ca538 RAC1 UN xx.xx.xx.xx317.2 GB 1 ? de7917ed-0de9-434d-be88-bc91eb4f8713 RAC1 UN xx.xx.xx.xx 291.61 GB 1 ? b489c970-68db-44a7-90c6-be734b41475f RAC1 However, now the client application fails to run queries on the cluster with: Cassandra.UnavailableException: Not enough replicas available for query at consistency Quorum (2 required but only 1 alive) Do *all* nodes see each other as UP/UN? =Rob The information in this Internet Email is confidential and may be legally privileged. It is intended solely for the addressee. Access to this Email by anyone else is unauthorized. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. When addressed to our clients any opinions or advice contained in this Email are subject to the terms and conditions expressed in any applicable governing The Home Depot terms of business or client engagement letter. The Home Depot disclaims all responsibility and liability for the accuracy and content of this attachment and for any damages or losses arising from any inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other items of a destructive nature, which may be contained in this attachment and shall not be liable for direct, indirect, consequential or special damages in connection with this e-mail message or its attachment.
Re: "Not enough replicas available for query" after reboot
No, there was no other change. I did run "apt-get upgrade" before rebooting, but Cassandra has not been upgraded. On 4 February 2016 at 22:48, Bryan Cheng wrote: > Hey Flavien! > > Did your reboot come with any other changes (schema, configuration, > topology, version)? > > On Thu, Feb 4, 2016 at 2:06 PM, Flavien Charlon > wrote: > >> I'm using the C# driver 2.5.2. I did try to restart the client >> application, but that didn't make any difference, I still get the same >> error after restart. >> >> On 4 February 2016 at 21:54, wrote: >> >>> What client are you using? >>> >>> >>> >>> It is possible that the client saw nodes down and has kept them marked >>> that way (without retrying). Depending on the client, you may have options >>> to set in RetryPolicy, FailoverPolicy, etc. A bounce of the client will >>> probably fix the problem for now. >>> >>> >>> >>> >>> >>> Sean Durity >>> >>> >>> >>> *From:* Flavien Charlon [mailto:flavien.char...@gmail.com] >>> *Sent:* Thursday, February 04, 2016 4:06 PM >>> *To:* user@cassandra.apache.org >>> *Subject:* Re: "Not enough replicas available for query" after reboot >>> >>> >>> >>> Yes, all three nodes see all three nodes as UN. >>> >>> >>> >>> Also, connecting from a local Cassandra machine using cqlsh, I can run >>> the same query just fine (with QUORUM consistency level). >>> >>> >>> >>> On 4 February 2016 at 21:02, Robert Coli wrote: >>> >>> On Thu, Feb 4, 2016 at 12:53 PM, Flavien Charlon < >>> flavien.char...@gmail.com> wrote: >>> >>> My cluster was running fine. I rebooted all three nodes (one by one), >>> and now all nodes are back up and running. "nodetool status" shows UP for >>> all three nodes on all three nodes: >>> >>> >>> >>> -- AddressLoad Tokens OwnsHost ID >>> Rack >>> >>> UN xx.xx.xx.xx331.84 GB 1 ? >>> d3d3a79b-9ca5-43f9-88c4-c3c7f08ca538 RAC1 >>> >>> UN xx.xx.xx.xx317.2 GB 1 ? >>> de7917ed-0de9-434d-be88-bc91eb4f8713 RAC1 >>> >>> UN xx.xx.xx.xx 291.61 GB 1 ? >>> b489c970-68db-44a7-90c6-be734b41475f RAC1 >>> >>> >>> >>> However, now the client application fails to run queries on the cluster >>> with: >>> >>> >>> >>> Cassandra.UnavailableException: Not enough replicas available for query >>> at consistency Quorum (2 required but only 1 alive) >>> >>> >>> >>> Do *all* nodes see each other as UP/UN? >>> >>> >>> >>> =Rob >>> >>> >>> >>> >>> >>> -- >>> >>> The information in this Internet Email is confidential and may be >>> legally privileged. It is intended solely for the addressee. Access to this >>> Email by anyone else is unauthorized. If you are not the intended >>> recipient, any disclosure, copying, distribution or any action taken or >>> omitted to be taken in reliance on it, is prohibited and may be unlawful. >>> When addressed to our clients any opinions or advice contained in this >>> Email are subject to the terms and conditions expressed in any applicable >>> governing The Home Depot terms of business or client engagement letter. The >>> Home Depot disclaims all responsibility and liability for the accuracy and >>> content of this attachment and for any damages or losses arising from any >>> inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other >>> items of a destructive nature, which may be contained in this attachment >>> and shall not be liable for direct, indirect, consequential or special >>> damages in connection with this e-mail message or its attachment. >>> >> >> >
Re: "Not enough replicas available for query" after reboot
Hey Flavien! Did your reboot come with any other changes (schema, configuration, topology, version)? On Thu, Feb 4, 2016 at 2:06 PM, Flavien Charlon wrote: > I'm using the C# driver 2.5.2. I did try to restart the client > application, but that didn't make any difference, I still get the same > error after restart. > > On 4 February 2016 at 21:54, wrote: > >> What client are you using? >> >> >> >> It is possible that the client saw nodes down and has kept them marked >> that way (without retrying). Depending on the client, you may have options >> to set in RetryPolicy, FailoverPolicy, etc. A bounce of the client will >> probably fix the problem for now. >> >> >> >> >> >> Sean Durity >> >> >> >> *From:* Flavien Charlon [mailto:flavien.char...@gmail.com] >> *Sent:* Thursday, February 04, 2016 4:06 PM >> *To:* user@cassandra.apache.org >> *Subject:* Re: "Not enough replicas available for query" after reboot >> >> >> >> Yes, all three nodes see all three nodes as UN. >> >> >> >> Also, connecting from a local Cassandra machine using cqlsh, I can run >> the same query just fine (with QUORUM consistency level). >> >> >> >> On 4 February 2016 at 21:02, Robert Coli wrote: >> >> On Thu, Feb 4, 2016 at 12:53 PM, Flavien Charlon < >> flavien.char...@gmail.com> wrote: >> >> My cluster was running fine. I rebooted all three nodes (one by one), and >> now all nodes are back up and running. "nodetool status" shows UP for all >> three nodes on all three nodes: >> >> >> >> -- AddressLoad Tokens OwnsHost ID >> Rack >> >> UN xx.xx.xx.xx331.84 GB 1 ? >> d3d3a79b-9ca5-43f9-88c4-c3c7f08ca538 RAC1 >> >> UN xx.xx.xx.xx317.2 GB 1 ? >> de7917ed-0de9-434d-be88-bc91eb4f8713 RAC1 >> >> UN xx.xx.xx.xx 291.61 GB 1 ? >> b489c970-68db-44a7-90c6-be734b41475f RAC1 >> >> >> >> However, now the client application fails to run queries on the cluster >> with: >> >> >> >> Cassandra.UnavailableException: Not enough replicas available for query >> at consistency Quorum (2 required but only 1 alive) >> >> >> >> Do *all* nodes see each other as UP/UN? >> >> >> >> =Rob >> >> >> >> >> >> -- >> >> The information in this Internet Email is confidential and may be legally >> privileged. It is intended solely for the addressee. Access to this Email >> by anyone else is unauthorized. If you are not the intended recipient, any >> disclosure, copying, distribution or any action taken or omitted to be >> taken in reliance on it, is prohibited and may be unlawful. When addressed >> to our clients any opinions or advice contained in this Email are subject >> to the terms and conditions expressed in any applicable governing The Home >> Depot terms of business or client engagement letter. The Home Depot >> disclaims all responsibility and liability for the accuracy and content of >> this attachment and for any damages or losses arising from any >> inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other >> items of a destructive nature, which may be contained in this attachment >> and shall not be liable for direct, indirect, consequential or special >> damages in connection with this e-mail message or its attachment. >> > >
Re: "Not enough replicas available for query" after reboot
I'm using the C# driver 2.5.2. I did try to restart the client application, but that didn't make any difference, I still get the same error after restart. On 4 February 2016 at 21:54, wrote: > What client are you using? > > > > It is possible that the client saw nodes down and has kept them marked > that way (without retrying). Depending on the client, you may have options > to set in RetryPolicy, FailoverPolicy, etc. A bounce of the client will > probably fix the problem for now. > > > > > > Sean Durity > > > > *From:* Flavien Charlon [mailto:flavien.char...@gmail.com] > *Sent:* Thursday, February 04, 2016 4:06 PM > *To:* user@cassandra.apache.org > *Subject:* Re: "Not enough replicas available for query" after reboot > > > > Yes, all three nodes see all three nodes as UN. > > > > Also, connecting from a local Cassandra machine using cqlsh, I can run the > same query just fine (with QUORUM consistency level). > > > > On 4 February 2016 at 21:02, Robert Coli wrote: > > On Thu, Feb 4, 2016 at 12:53 PM, Flavien Charlon < > flavien.char...@gmail.com> wrote: > > My cluster was running fine. I rebooted all three nodes (one by one), and > now all nodes are back up and running. "nodetool status" shows UP for all > three nodes on all three nodes: > > > > -- AddressLoad Tokens OwnsHost ID > Rack > > UN xx.xx.xx.xx331.84 GB 1 ? > d3d3a79b-9ca5-43f9-88c4-c3c7f08ca538 RAC1 > > UN xx.xx.xx.xx317.2 GB 1 ? > de7917ed-0de9-434d-be88-bc91eb4f8713 RAC1 > > UN xx.xx.xx.xx 291.61 GB 1 ? > b489c970-68db-44a7-90c6-be734b41475f RAC1 > > > > However, now the client application fails to run queries on the cluster > with: > > > > Cassandra.UnavailableException: Not enough replicas available for query at > consistency Quorum (2 required but only 1 alive) > > > > Do *all* nodes see each other as UP/UN? > > > > =Rob > > > > > > -- > > The information in this Internet Email is confidential and may be legally > privileged. It is intended solely for the addressee. Access to this Email > by anyone else is unauthorized. If you are not the intended recipient, any > disclosure, copying, distribution or any action taken or omitted to be > taken in reliance on it, is prohibited and may be unlawful. When addressed > to our clients any opinions or advice contained in this Email are subject > to the terms and conditions expressed in any applicable governing The Home > Depot terms of business or client engagement letter. The Home Depot > disclaims all responsibility and liability for the accuracy and content of > this attachment and for any damages or losses arising from any > inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other > items of a destructive nature, which may be contained in this attachment > and shall not be liable for direct, indirect, consequential or special > damages in connection with this e-mail message or its attachment. >
RE: "Not enough replicas available for query" after reboot
What client are you using? It is possible that the client saw nodes down and has kept them marked that way (without retrying). Depending on the client, you may have options to set in RetryPolicy, FailoverPolicy, etc. A bounce of the client will probably fix the problem for now. Sean Durity From: Flavien Charlon [mailto:flavien.char...@gmail.com] Sent: Thursday, February 04, 2016 4:06 PM To: user@cassandra.apache.org Subject: Re: "Not enough replicas available for query" after reboot Yes, all three nodes see all three nodes as UN. Also, connecting from a local Cassandra machine using cqlsh, I can run the same query just fine (with QUORUM consistency level). On 4 February 2016 at 21:02, Robert Coli mailto:rc...@eventbrite.com>> wrote: On Thu, Feb 4, 2016 at 12:53 PM, Flavien Charlon mailto:flavien.char...@gmail.com>> wrote: My cluster was running fine. I rebooted all three nodes (one by one), and now all nodes are back up and running. "nodetool status" shows UP for all three nodes on all three nodes: -- AddressLoad Tokens OwnsHost ID Rack UN xx.xx.xx.xx331.84 GB 1 ? d3d3a79b-9ca5-43f9-88c4-c3c7f08ca538 RAC1 UN xx.xx.xx.xx317.2 GB 1 ? de7917ed-0de9-434d-be88-bc91eb4f8713 RAC1 UN xx.xx.xx.xx 291.61 GB 1 ? b489c970-68db-44a7-90c6-be734b41475f RAC1 However, now the client application fails to run queries on the cluster with: Cassandra.UnavailableException: Not enough replicas available for query at consistency Quorum (2 required but only 1 alive) Do *all* nodes see each other as UP/UN? =Rob The information in this Internet Email is confidential and may be legally privileged. It is intended solely for the addressee. Access to this Email by anyone else is unauthorized. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. When addressed to our clients any opinions or advice contained in this Email are subject to the terms and conditions expressed in any applicable governing The Home Depot terms of business or client engagement letter. The Home Depot disclaims all responsibility and liability for the accuracy and content of this attachment and for any damages or losses arising from any inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other items of a destructive nature, which may be contained in this attachment and shall not be liable for direct, indirect, consequential or special damages in connection with this e-mail message or its attachment.
Re: "Not enough replicas available for query" after reboot
Yes, all three nodes see all three nodes as UN. Also, connecting from a local Cassandra machine using cqlsh, I can run the same query just fine (with QUORUM consistency level). On 4 February 2016 at 21:02, Robert Coli wrote: > On Thu, Feb 4, 2016 at 12:53 PM, Flavien Charlon < > flavien.char...@gmail.com> wrote: > >> My cluster was running fine. I rebooted all three nodes (one by one), and >> now all nodes are back up and running. "nodetool status" shows UP for all >> three nodes on all three nodes: >> >> -- AddressLoad Tokens OwnsHost ID >> Rack >> UN xx.xx.xx.xx331.84 GB 1 ? >> d3d3a79b-9ca5-43f9-88c4-c3c7f08ca538 RAC1 >> UN xx.xx.xx.xx317.2 GB 1 ? >> de7917ed-0de9-434d-be88-bc91eb4f8713 RAC1 >> UN xx.xx.xx.xx 291.61 GB 1 ? >> b489c970-68db-44a7-90c6-be734b41475f RAC1 >> >> However, now the client application fails to run queries on the cluster >> with: >> >> Cassandra.UnavailableException: Not enough replicas available for query >>> at consistency Quorum (2 required but only 1 alive) >> >> > Do *all* nodes see each other as UP/UN? > > =Rob > >
Re: "Not enough replicas available for query" after reboot
On Thu, Feb 4, 2016 at 12:53 PM, Flavien Charlon wrote: > My cluster was running fine. I rebooted all three nodes (one by one), and > now all nodes are back up and running. "nodetool status" shows UP for all > three nodes on all three nodes: > > -- AddressLoad Tokens OwnsHost ID > Rack > UN xx.xx.xx.xx331.84 GB 1 ? > d3d3a79b-9ca5-43f9-88c4-c3c7f08ca538 RAC1 > UN xx.xx.xx.xx317.2 GB 1 ? > de7917ed-0de9-434d-be88-bc91eb4f8713 RAC1 > UN xx.xx.xx.xx 291.61 GB 1 ? > b489c970-68db-44a7-90c6-be734b41475f RAC1 > > However, now the client application fails to run queries on the cluster > with: > > Cassandra.UnavailableException: Not enough replicas available for query at >> consistency Quorum (2 required but only 1 alive) > > Do *all* nodes see each other as UP/UN? =Rob
