Re: Getting Error while Writing in Multi DC mode when Remote Dc is Down.
The query has QUORUM not LOCAL_QUORUM. So 3 of 5 nodes are required. Maybe 1 node in DRPOCcluster also was temporarily unavailable during that query? 2017-01-23 12:16 GMT+01:00 Abhishek Kumar Maheshwari < abhishek.maheshw...@timesinternet.in>: > Hi All, > > > > I have Cassandra stack with 2 Dc > > > > Datacenter: DRPOCcluster > > > > Status=Up/Down > > |/ State=Normal/Leaving/Joining/Moving > > -- AddressLoad Tokens OwnsHost > ID Rack > > UN 172.29.xx.xxx 88.88 GB 256 ? > b6b8cbb9-1fed-471f-aea9-6a657e7ac80a > 01 > > UN 172.29.xx.xxx 73.95 GB 256 ? > 604abbf5-8639-4104-8f60-fd6573fb2e17 > 03 > > UN 172.29. xx.xxx 66.42 GB 256 ? > 32fa79ee-93c6-4e5b-a910-f27a1e9d66c1 02 > > Datacenter: dc_india > > > > Status=Up/Down > > |/ State=Normal/Leaving/Joining/Moving > > -- AddressLoad Tokens OwnsHost > ID Rack > > DN 172.26. .xx.xxx 78.97 GB 256 ? > 3e8133ed-98b5-418d-96b5-690a1450cd30 RACK1 > > DN 172.26. .xx.xxx 79.18 GB 256 ? > 7d3f5b25-88f9-4be7-b0f5-746619153543 RACK2 > > > > > > I am using below code to connect with java driver: > > > > cluster = Cluster.*builder*().addContactPoints(hostAddresses > ).withRetryPolicy(DefaultRetryPolicy.*INSTANCE*) > >.withReconnectionPolicy(*new* > ConstantReconnectionPolicy(3L)) > >.withLoadBalancingPolicy(*new* > TokenAwarePolicy(*new* DCAwareRoundRobinPolicy.Builder().withLocalDc(" > DRPOCcluster").withUsedHostsPerRemoteDc(2).build())).build(); > > cluster.getConfiguration().getQueryOptions().setConsistencyLevel( > ConsistencyLevel.LOCAL_QUORUM); > > > > hostAddresses is 172.29.xx.xxx . when Dc with IP 172.26. .xx.xxx is > down, we are getting below exception : > > > > > > Exception in thread "main" > com.datastax.driver.core.exceptions.UnavailableException: > Not enough replicas available for query at consistency QUORUM (3 required > but only 2 alive) > >at com.datastax.driver.core.exceptions.UnavailableException.copy( > UnavailableException.java:109) > >at com.datastax.driver.core.exceptions.UnavailableException.copy( > UnavailableException.java:27) > >at com.datastax.driver.core.DriverThrowables.propagateCause( > DriverThrowables.java:37) > >at com.datastax.driver.core.DefaultResultSetFuture. > getUninterruptibly(DefaultResultSetFuture.java:245) > > > > Cassandra version : 3.0.9 > > Datastax Java Driver Version: > > > > > > com.datastax.cassandra > > cassandra-driver- > core > > 3.1.2 > > > > > > > > *Thanks & Regards,* > *Abhishek Kumar Maheshwari* > *+91- 805591 <+91%208%2005591> (Mobile)* > > Times Internet Ltd. | A Times of India Group Company > > FC - 6, Sector 16A, Film City, Noida, U.P. 201301 | INDIA > > *P** Please do not print this email unless it is absolutely necessary. > Spread environmental awareness.* > > > We the soldiers of our new economy, pledge to stop doubting and start > spending, to enable others to go digital, to use less cash. We pledge to > #RemonetiseIndia. Join the Times Network ‘Remonetise India’ movement today. > To pledge for growth, give a missed call on +91 9223515515 > <+91%2092235%2015515>. Visit www.remonetiseindia.com > -- Benjamin Roth Prokurist Jaumo GmbH · www.jaumo.com Wehrstraße 46 · 73035 Göppingen · Germany Phone +49 7161 304880-6 · Fax +49 7161 304880-1 AG Ulm · HRB 731058 · Managing Director: Jens Kammerer
RE: Getting Error while Writing in Multi DC mode when Remote Dc is Down.
Hi Benjamin, I find the issue. while I was making query, I was overriding LOCAL_QUORUM to QUORUM. Also, one more Question, I was able insert data in DRPOCcluster. But when I bring up dc_india DC, data doesn’t seem in dc_india keyspace and column family (I wait near about 30 min)? Thanks & Regards, Abhishek Kumar Maheshwari +91- 805591 (Mobile) Times Internet Ltd. | A Times of India Group Company FC - 6, Sector 16A, Film City, Noida, U.P. 201301 | INDIA P Please do not print this email unless it is absolutely necessary. Spread environmental awareness. From: Benjamin Roth [mailto:benjamin.r...@jaumo.com] Sent: Monday, January 23, 2017 5:05 PM To: user@cassandra.apache.org Subject: Re: Getting Error while Writing in Multi DC mode when Remote Dc is Down. The query has QUORUM not LOCAL_QUORUM. So 3 of 5 nodes are required. Maybe 1 node in DRPOCcluster also was temporarily unavailable during that query? 2017-01-23 12:16 GMT+01:00 Abhishek Kumar Maheshwari mailto:abhishek.maheshw...@timesinternet.in>>: Hi All, I have Cassandra stack with 2 Dc Datacenter: DRPOCcluster Status=Up/Down |/ State=Normal/Leaving/Joining/Moving -- AddressLoad Tokens OwnsHost ID Rack UN 172.29.xx.xxx 88.88 GB 256 ? b6b8cbb9-1fed-471f-aea9-6a657e7ac80a 01 UN 172.29.xx.xxx 73.95 GB 256 ? 604abbf5-8639-4104-8f60-fd6573fb2e17 03 UN 172.29. xx.xxx 66.42 GB 256 ? 32fa79ee-93c6-4e5b-a910-f27a1e9d66c1 02 Datacenter: dc_india Status=Up/Down |/ State=Normal/Leaving/Joining/Moving -- AddressLoad Tokens OwnsHost ID Rack DN 172.26. .xx.xxx 78.97 GB 256 ? 3e8133ed-98b5-418d-96b5-690a1450cd30 RACK1 DN 172.26. .xx.xxx 79.18 GB 256 ? 7d3f5b25-88f9-4be7-b0f5-746619153543 RACK2 I am using below code to connect with java driver: cluster = Cluster.builder().addContactPoints(hostAddresses).withRetryPolicy(DefaultRetryPolicy.INSTANCE) .withReconnectionPolicy(new ConstantReconnectionPolicy(3L)) .withLoadBalancingPolicy(new TokenAwarePolicy(new DCAwareRoundRobinPolicy.Builder().withLocalDc("DRPOCcluster").withUsedHostsPerRemoteDc(2).build())).build(); cluster.getConfiguration().getQueryOptions().setConsistencyLevel(ConsistencyLevel.LOCAL_QUORUM); hostAddresses is 172.29.xx.xxx . when Dc with IP 172.26. .xx.xxx is down, we are getting below exception : Exception in thread "main" com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:109) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:27) at com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37) at com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245) Cassandra version : 3.0.9 Datastax Java Driver Version: com.datastax.cassandra cassandra-driver-core 3.1.2 Thanks & Regards, Abhishek Kumar Maheshwari +91- 805591 (Mobile) Times Internet Ltd. | A Times of India Group Company FC - 6, Sector 16A, Film City, Noida, U.P. 201301 | INDIA P Please do not print this email unless it is absolutely necessary. Spread environmental awareness. We the soldiers of our new economy, pledge to stop doubting and start spending, to enable others to go digital, to use less cash. We pledge to #RemonetiseIndia. Join the Times Network ‘Remonetise India’ movement today. To pledge for growth, give a missed call on +91 9223515515. Visit www.remonetiseindia.com<http://www.remonetiseindia.com> -- Benjamin Roth Prokurist Jaumo GmbH · www.jaumo.com<http://www.jaumo.com> Wehrstraße 46 · 73035 Göppingen · Germany Phone +49 7161 304880-6 · Fax +49 7161 304880-1 AG Ulm · HRB 731058 · Managing Director: Jens Kammerer
Re: Getting Error while Writing in Multi DC mode when Remote Dc is Down.
Sorry for the short answer, I am on the run: I guess your hints expired. Default setting is 3h. If a node is down for a longertime, no hints will be written. Only a repair will help then. 2017-01-23 12:47 GMT+01:00 Abhishek Kumar Maheshwari < abhishek.maheshw...@timesinternet.in>: > Hi Benjamin, > > > > I find the issue. while I was making query, I was overriding LOCAL_QUORUM > to QUORUM. > > > > Also, one more Question, > > > > I was able insert data in DRPOCcluster. But when I bring up dc_india DC, > data doesn’t seem in dc_india keyspace and column family (I wait near about > 30 min)? > > > > > > > > > > *Thanks & Regards,* > *Abhishek Kumar Maheshwari* > *+91- 805591 <+91%208%2005591> (Mobile)* > > Times Internet Ltd. | A Times of India Group Company > > FC - 6, Sector 16A, Film City, Noida, U.P. 201301 | INDIA > > *P** Please do not print this email unless it is absolutely necessary. > Spread environmental awareness.* > > > > *From:* Benjamin Roth [mailto:benjamin.r...@jaumo.com] > *Sent:* Monday, January 23, 2017 5:05 PM > *To:* user@cassandra.apache.org > *Subject:* Re: Getting Error while Writing in Multi DC mode when Remote > Dc is Down. > > > > The query has QUORUM not LOCAL_QUORUM. So 3 of 5 nodes are required. Maybe > 1 node in DRPOCcluster also was temporarily unavailable during that query? > > > > 2017-01-23 12:16 GMT+01:00 Abhishek Kumar Maheshwari timesinternet.in>: > > Hi All, > > > > I have Cassandra stack with 2 Dc > > > > Datacenter: DRPOCcluster > > > > Status=Up/Down > > |/ State=Normal/Leaving/Joining/Moving > > -- AddressLoad Tokens OwnsHost > ID Rack > > UN 172.29.xx.xxx 88.88 GB 256 ? > b6b8cbb9-1fed-471f-aea9-6a657e7ac80a > 01 > > UN 172.29.xx.xxx 73.95 GB 256 ? > 604abbf5-8639-4104-8f60-fd6573fb2e17 > 03 > > UN 172.29. xx.xxx 66.42 GB 256 ? > 32fa79ee-93c6-4e5b-a910-f27a1e9d66c1 02 > > Datacenter: dc_india > > > > Status=Up/Down > > |/ State=Normal/Leaving/Joining/Moving > > -- AddressLoad Tokens OwnsHost > ID Rack > > DN 172.26. .xx.xxx 78.97 GB 256 ? > 3e8133ed-98b5-418d-96b5-690a1450cd30 RACK1 > > DN 172.26. .xx.xxx 79.18 GB 256 ? > 7d3f5b25-88f9-4be7-b0f5-746619153543 RACK2 > > > > > > I am using below code to connect with java driver: > > > > cluster = Cluster.*builder*().addContactPoints(hostAddresses > ).withRetryPolicy(DefaultRetryPolicy.*INSTANCE*) > >.withReconnectionPolicy(*new* > ConstantReconnectionPolicy(3L)) > >.withLoadBalancingPolicy(*new* > TokenAwarePolicy(*new* DCAwareRoundRobinPolicy.Builder().withLocalDc(" > DRPOCcluster").withUsedHostsPerRemoteDc(2).build())).build(); > > cluster.getConfiguration().getQueryOptions().setConsistencyLevel( > ConsistencyLevel.LOCAL_QUORUM); > > > > hostAddresses is 172.29.xx.xxx . when Dc with IP 172.26. .xx.xxx is > down, we are getting below exception : > > > > > > Exception in thread "main" > com.datastax.driver.core.exceptions.UnavailableException: > Not enough replicas available for query at consistency QUORUM (3 required > but only 2 alive) > >at com.datastax.driver.core.exceptions.UnavailableException.copy( > UnavailableException.java:109) > >at com.datastax.driver.core.exceptions.UnavailableException.copy( > UnavailableException.java:27) > >at com.datastax.driver.core.DriverThrowables.propagateCause( > DriverThrowables.java:37) > >at com.datastax.driver.core.DefaultResultSetFuture. > getUninterruptibly(DefaultResultSetFuture.java:245) > > > > Cassandra version : 3.0.9 > > Datastax Java Driver Version: > > > > > > com.datastax.cassandra > > cassandra-driver- > core > > 3.1.2 > > > > > > > > *Thanks & Regards,* > *Abhishek Kumar Maheshwari* > *+91- 805591 <+91%208%2005591> (Mobile)* > > Times Internet Ltd. | A Times of India Group Company > > FC - 6, Sector 16A, Film City, Noida, U.P. 201301 | INDIA > > *P** Please do not print this email unless it is absolutely necessary. > Spread environmental awareness.* > > > > We the soldiers of our new economy, pledge to s
RE: Getting Error while Writing in Multi DC mode when Remote Dc is Down.
Thanks, Benjamin, I found the issue hints was off in Cassandra.yml. Thanks & Regards, Abhishek Kumar Maheshwari +91- 805591 (Mobile) Times Internet Ltd. | A Times of India Group Company FC - 6, Sector 16A, Film City, Noida, U.P. 201301 | INDIA P Please do not print this email unless it is absolutely necessary. Spread environmental awareness. From: Benjamin Roth [mailto:benjamin.r...@jaumo.com] Sent: Monday, January 23, 2017 6:09 PM To: user@cassandra.apache.org Subject: Re: Getting Error while Writing in Multi DC mode when Remote Dc is Down. Sorry for the short answer, I am on the run: I guess your hints expired. Default setting is 3h. If a node is down for a longertime, no hints will be written. Only a repair will help then. 2017-01-23 12:47 GMT+01:00 Abhishek Kumar Maheshwari mailto:abhishek.maheshw...@timesinternet.in>>: Hi Benjamin, I find the issue. while I was making query, I was overriding LOCAL_QUORUM to QUORUM. Also, one more Question, I was able insert data in DRPOCcluster. But when I bring up dc_india DC, data doesn’t seem in dc_india keyspace and column family (I wait near about 30 min)? Thanks & Regards, Abhishek Kumar Maheshwari +91- 805591 (Mobile) Times Internet Ltd. | A Times of India Group Company FC - 6, Sector 16A, Film City, Noida, U.P. 201301 | INDIA P Please do not print this email unless it is absolutely necessary. Spread environmental awareness. From: Benjamin Roth [mailto:benjamin.r...@jaumo.com<mailto:benjamin.r...@jaumo.com>] Sent: Monday, January 23, 2017 5:05 PM To: user@cassandra.apache.org<mailto:user@cassandra.apache.org> Subject: Re: Getting Error while Writing in Multi DC mode when Remote Dc is Down. The query has QUORUM not LOCAL_QUORUM. So 3 of 5 nodes are required. Maybe 1 node in DRPOCcluster also was temporarily unavailable during that query? 2017-01-23 12:16 GMT+01:00 Abhishek Kumar Maheshwari mailto:abhishek.maheshw...@timesinternet.in>>: Hi All, I have Cassandra stack with 2 Dc Datacenter: DRPOCcluster Status=Up/Down |/ State=Normal/Leaving/Joining/Moving -- AddressLoad Tokens OwnsHost ID Rack UN 172.29.xx.xxx 88.88 GB 256 ? b6b8cbb9-1fed-471f-aea9-6a657e7ac80a 01 UN 172.29.xx.xxx 73.95 GB 256 ? 604abbf5-8639-4104-8f60-fd6573fb2e17 03 UN 172.29. xx.xxx 66.42 GB 256 ? 32fa79ee-93c6-4e5b-a910-f27a1e9d66c1 02 Datacenter: dc_india Status=Up/Down |/ State=Normal/Leaving/Joining/Moving -- AddressLoad Tokens OwnsHost ID Rack DN 172.26. .xx.xxx 78.97 GB 256 ? 3e8133ed-98b5-418d-96b5-690a1450cd30 RACK1 DN 172.26. .xx.xxx 79.18 GB 256 ? 7d3f5b25-88f9-4be7-b0f5-746619153543 RACK2 I am using below code to connect with java driver: cluster = Cluster.builder().addContactPoints(hostAddresses).withRetryPolicy(DefaultRetryPolicy.INSTANCE) .withReconnectionPolicy(new ConstantReconnectionPolicy(3L)) .withLoadBalancingPolicy(new TokenAwarePolicy(new DCAwareRoundRobinPolicy.Builder().withLocalDc("DRPOCcluster").withUsedHostsPerRemoteDc(2).build())).build(); cluster.getConfiguration().getQueryOptions().setConsistencyLevel(ConsistencyLevel.LOCAL_QUORUM); hostAddresses is 172.29.xx.xxx . when Dc with IP 172.26. .xx.xxx is down, we are getting below exception : Exception in thread "main" com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:109) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:27) at com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37) at com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245) Cassandra version : 3.0.9 Datastax Java Driver Version: com.datastax.cassandra cassandra-driver-core 3.1.2 Thanks & Regards, Abhishek Kumar Maheshwari +91- 805591 (Mobile) Times Internet Ltd. | A Times of India Group Company FC - 6, Sector 16A, Film City, Noida, U.P. 201301 | INDIA P Please do not print this email unless it is absolutely necessary. Spread environmental awareness. We the soldiers of our new economy, pledge to stop doubting and start spending, to enable others to go digital, to use less cash. We pledge to #RemonetiseIndia. Join the Times Network ‘Remonetise India’ movement today. To pledge for growth, give a missed call on +91 9223515515. Visit www.remonetiseindia.com<http://ww