[jira] [Created] (CASSANDRA-8536) Wrong cluster information and replication
Vova created CASSANDRA-8536:
---
Summary: Wrong cluster information and replication
Key: CASSANDRA-8536
URL: https://issues.apache.org/jira/browse/CASSANDRA-8536
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: CentOS 7 x64
Reporter: Vova
Two machine cluster - Cassandra 2.1.2, GossipingPropertyFileSnitch, one data
center with one rack.
Seed - 10.0.0.2
Node - 10.0.0.3
-start seed
-start node
Run nodetool status on any machine:
{quote}
Datacenter: DC1
===
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens OwnsHost ID
Rack
UN 10.0.0.3 107.15 KB 256 ? ad29cd96-d21e-4d02-94e7-0fd68ef5fbad
RAC1
UN 10.0.0.2 87.73 KB 256 ? c26fdffc-6df5-4d1a-8eda-6d585d2178c1
RAC1
{quote}
-stop both instances
-run seed
-run nodetool status on seed
{quote}
Datacenter: DC1
===
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens OwnsHost ID
Rack
UN 10.0.0.2 113.31 KB 256 ? c26fdffc-6df5-4d1a-8eda-6d585d2178c1
RAC1
{quote}
So no information about node 10.0.0.3 at all.
Actually main problem is not wrong info, but replication/synchronization
problem:
On seed (after restart, when 2nd node is down) create keyspace with replication
factor 2 (strategy doesn't matter), create table, insert something into table:
CREATE KEYSPACE Excelsior WITH
REPLICATION={'class':'SimpleStrategy','replication_factor':2};
CREATE TABLE Excelsior.users (name text PRIMARY KEY, id int);
INSERT INTO excelsior.users (name, id ) VALUES ( 'node',123);
SELECT * FROM excelsior.users;
name | id
--+-
node | 123
(1 rows)
Start node, now nodetool status shows both nodes UN on both machines again.
Now created keyspace and table are seen on node (create was propagated from
seed), but is empty from node point of view:
{quote}
SELECT * FROM excelsior.users;
name | id
--+
(0 rows)
{quote}
I guess synchronization problem probably not different bug, but stems from
wrong cluster information.
Version 2.0.11 works fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8537) ConcurrentModificationException while executing 'nodetool cleanup'
Noureddine Chatti created CASSANDRA-8537: Summary: ConcurrentModificationException while executing 'nodetool cleanup' Key: CASSANDRA-8537 URL: https://issues.apache.org/jira/browse/CASSANDRA-8537 Project: Cassandra Issue Type: Bug Components: Tools Environment: Debian 7.7, Oracle JRE 1.7.0_72 Reporter: Noureddine Chatti Fix For: 2.1.2 After adding a new node to an existing cluster (7 already started nodes), and waiting a few minutes to be sure that data migration to the new node is completed, I began to use the command nodetool cleanup sequentially on each old node. When I issued this command on the third node, after a few minutes I got a ConcurrentModificationException. ~$ nodetool cleanup error: null -- StackTrace -- java.util.ConcurrentModificationException at java.util.ArrayList$Itr.checkForComodification(Unknown Source) at java.util.ArrayList$Itr.next(Unknown Source) at org.apache.cassandra.db.index.SecondaryIndexManager.deleteFromIndexes(SecondaryIndexManager.java:476) at org.apache.cassandra.db.compaction.CompactionManager$CleanupStrategy$Full.cleanup(CompactionManager.java:833) at org.apache.cassandra.db.compaction.CompactionManager.doCleanupOne(CompactionManager.java:704) at org.apache.cassandra.db.compaction.CompactionManager.access$400(CompactionManager.java:97) at org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:370) at org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:267) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8538) Clarify default write time timestamp for CQL insert and update
Jack Krupansky created CASSANDRA-8538: - Summary: Clarify default write time timestamp for CQL insert and update Key: CASSANDRA-8538 URL: https://issues.apache.org/jira/browse/CASSANDRA-8538 Project: Cassandra Issue Type: Improvement Components: Documentation website Reporter: Jack Krupansky Priority: Minor The current CQL spec (and downstream doc) says that the default timestamp for write time for CQL inserts and updates is the current time of the insertion, but that is somewhat vague and non-specific. In particular, is that the time when the coordinator node parses the CQL statement, or... when the row is inserted or updated on the target nodes, or... something else? In particular, if the coordinator doesn't own the token of the primary key, will the owning node set the write time or does the coordinator node do that? Obviously the application can set an explicit TIMESTAMP, but this issue is concerned with the default if that explicit option is not used. Also, will all replicas of the insert or update share the precisely same write time, or will they reflect the actual time when each particular replica row is inserted or updated on each of the replica nodes? Finally, if a batch statement is used to insert or update multiple rows, will they all share the same write time (e.g., the time the batch statement was parsed) or when each replica row is actually inserted or updated on the target (if the coordinator node does not own the token of the partition key) or replica nodes? It would also be helpful if the tracing option was specific as to which time is the official write time for the insert or update. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8534) The default configuration URL does not have the required file:// prefix and throws an exception if cassandra.config is not set.
[ https://issues.apache.org/jira/browse/CASSANDRA-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259428#comment-14259428 ] Mike Adamson commented on CASSANDRA-8534: - If the configURL fails to load as a URL then the code attempts to load the configuration as a resource which does not need the file:// prefix. This code must work because the default launch scripts do not add the cassandra.config system property and they successfully find a cassandra.yaml from the classpath. The default configuration URL does not have the required file:// prefix and throws an exception if cassandra.config is not set. --- Key: CASSANDRA-8534 URL: https://issues.apache.org/jira/browse/CASSANDRA-8534 Project: Cassandra Issue Type: Bug Components: Config, Core Environment: Any Reporter: Andrew Trimble Fix For: 2.1.3 Attachments: error.txt Original Estimate: 1h Remaining Estimate: 1h In the class org.apache.cassandra.config.YamlConfigurationLoader, the DEFAULT_CONFIGURATION is set to cassandra.yaml. This is improperly formatted as it does not contain the prefix file://. If this value is used, a ConfigurationException is thrown (see line 73 of the same class). A solution is to set the cassandra.config system property, but this does not solve the underlying problem. A vanilla Cassandra installation will throw this error. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
cassandra git commit: move null guard to where it's useful
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 b53fefa73 - 12b96915c
move null guard to where it's useful
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/12b96915
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/12b96915
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/12b96915
Branch: refs/heads/cassandra-2.1
Commit: 12b96915c3e37edc1ea6b584ebc406f44426d4ec
Parents: b53fefa
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Sat Dec 27 15:18:59 2014 -0500
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Sat Dec 27 15:18:59 2014 -0500
--
.../cassandra/hadoop/cql3/CqlRecordWriter.java | 26 +++-
1 file changed, 15 insertions(+), 11 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/12b96915/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordWriter.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordWriter.java
b/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordWriter.java
index ff1989d..702cae3 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordWriter.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordWriter.java
@@ -104,23 +104,27 @@ class CqlRecordWriter extends
AbstractColumnFamilyRecordWriterMapString, ByteB
try
{
Cassandra.Client client =
ConfigHelper.getClientFromOutputAddressList(conf);
-client.set_keyspace(ConfigHelper.getOutputKeyspace(conf));
-String user = ConfigHelper.getOutputKeyspaceUserName(conf);
-String password = ConfigHelper.getOutputKeyspacePassword(conf);
-if ((user != null) (password != null))
-AbstractColumnFamilyOutputFormat.login(user, password, client);
-retrievePartitionKeyValidator(client);
-String cqlQuery = CqlConfigHelper.getOutputCql(conf).trim();
-if (cqlQuery.toLowerCase().startsWith(insert))
-throw new UnsupportedOperationException(INSERT with
CqlRecordWriter is not supported, please use UPDATE/DELETE statement);
-cql = appendKeyWhereClauses(cqlQuery);
-
if (client != null)
{
+client.set_keyspace(ConfigHelper.getOutputKeyspace(conf));
+String user = ConfigHelper.getOutputKeyspaceUserName(conf);
+String password = ConfigHelper.getOutputKeyspacePassword(conf);
+if ((user != null) (password != null))
+AbstractColumnFamilyOutputFormat.login(user, password,
client);
+retrievePartitionKeyValidator(client);
+String cqlQuery = CqlConfigHelper.getOutputCql(conf).trim();
+if (cqlQuery.toLowerCase().startsWith(insert))
+throw new UnsupportedOperationException(INSERT with
CqlRecordWriter is not supported, please use UPDATE/DELETE statement);
+cql = appendKeyWhereClauses(cqlQuery);
+
TTransport transport =
client.getOutputProtocol().getTransport();
if (transport.isOpen())
transport.close();
}
+else
+{
+throw new IllegalArgumentException(Invalid configuration
specified + conf);
+}
}
catch (Exception e)
{
[2/2] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/17d9791d Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/17d9791d Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/17d9791d Branch: refs/heads/trunk Commit: 17d9791d63c4bfde657ac1c13b45d1a357ce44bd Parents: b5a232b 12b9691 Author: Dave Brosius dbros...@mebigfatguy.com Authored: Sat Dec 27 15:19:28 2014 -0500 Committer: Dave Brosius dbros...@mebigfatguy.com Committed: Sat Dec 27 15:19:28 2014 -0500 -- .../cassandra/hadoop/cql3/CqlRecordWriter.java | 26 +++- 1 file changed, 15 insertions(+), 11 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/17d9791d/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordWriter.java --
[1/2] cassandra git commit: move null guard to where it's useful
Repository: cassandra
Updated Branches:
refs/heads/trunk b5a232b04 - 17d9791d6
move null guard to where it's useful
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/12b96915
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/12b96915
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/12b96915
Branch: refs/heads/trunk
Commit: 12b96915c3e37edc1ea6b584ebc406f44426d4ec
Parents: b53fefa
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Sat Dec 27 15:18:59 2014 -0500
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Sat Dec 27 15:18:59 2014 -0500
--
.../cassandra/hadoop/cql3/CqlRecordWriter.java | 26 +++-
1 file changed, 15 insertions(+), 11 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/12b96915/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordWriter.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordWriter.java
b/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordWriter.java
index ff1989d..702cae3 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordWriter.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordWriter.java
@@ -104,23 +104,27 @@ class CqlRecordWriter extends
AbstractColumnFamilyRecordWriterMapString, ByteB
try
{
Cassandra.Client client =
ConfigHelper.getClientFromOutputAddressList(conf);
-client.set_keyspace(ConfigHelper.getOutputKeyspace(conf));
-String user = ConfigHelper.getOutputKeyspaceUserName(conf);
-String password = ConfigHelper.getOutputKeyspacePassword(conf);
-if ((user != null) (password != null))
-AbstractColumnFamilyOutputFormat.login(user, password, client);
-retrievePartitionKeyValidator(client);
-String cqlQuery = CqlConfigHelper.getOutputCql(conf).trim();
-if (cqlQuery.toLowerCase().startsWith(insert))
-throw new UnsupportedOperationException(INSERT with
CqlRecordWriter is not supported, please use UPDATE/DELETE statement);
-cql = appendKeyWhereClauses(cqlQuery);
-
if (client != null)
{
+client.set_keyspace(ConfigHelper.getOutputKeyspace(conf));
+String user = ConfigHelper.getOutputKeyspaceUserName(conf);
+String password = ConfigHelper.getOutputKeyspacePassword(conf);
+if ((user != null) (password != null))
+AbstractColumnFamilyOutputFormat.login(user, password,
client);
+retrievePartitionKeyValidator(client);
+String cqlQuery = CqlConfigHelper.getOutputCql(conf).trim();
+if (cqlQuery.toLowerCase().startsWith(insert))
+throw new UnsupportedOperationException(INSERT with
CqlRecordWriter is not supported, please use UPDATE/DELETE statement);
+cql = appendKeyWhereClauses(cqlQuery);
+
TTransport transport =
client.getOutputProtocol().getTransport();
if (transport.isOpen())
transport.close();
}
+else
+{
+throw new IllegalArgumentException(Invalid configuration
specified + conf);
+}
}
catch (Exception e)
{
[jira] [Commented] (CASSANDRA-8281) CQLSSTableWriter close does not work
[
https://issues.apache.org/jira/browse/CASSANDRA-8281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259482#comment-14259482
]
Benyi Wang commented on CASSANDRA-8281:
---
I believe the root cause of this problem is QueryProcessor, who creates a
thread, but never shutdowns it.
{code}
private static final ScheduledExecutorService
evictionCheckTimer=Executors.newScheduledThreadPool(1);
{code}
I used cassandra-all-2.1.2.jar from maven central. If you run the above example
code with {{Config.setClientMode(true)}} In debug mode in eclipse, you will see
the thread created by QueryProcessor keeps running after the program exits
main().
{code}
package org.apache.cassandra.hadoop.cql3;
import org.apache.cassandra.config.Config;
import org.apache.cassandra.io.sstable.CQLSSTableWriter;
public class TWriter2 {
public static void main(String[] args) {
String schema = create table mgindex.test (x int primary key,
y int);
String insert = insert into mgindex.test (x,y) values(?,?);
Config.setClientMode(true);
CQLSSTableWriter w = CQLSSTableWriter.builder()
.inDirectory(/tmp/mgindex/test).forTable(schema)
.using(insert).build();
try {
w.addRow(1, 1);
w.close();
} catch (Exception e) {
System.out.println(e);
}
}
}
{code}
If you run the similar code in a JUnit test, you won't see this problem. I
believe the thread is terminated by the test runner.
CQLSSTableWriter close does not work
Key: CASSANDRA-8281
URL: https://issues.apache.org/jira/browse/CASSANDRA-8281
Project: Cassandra
Issue Type: Bug
Components: API
Environment: Cassandra 2.1.1
Reporter: Xu Zhongxing
Assignee: Benjamin Lerer
Attachments: CASSANDRA-8281.txt
I called CQLSSTableWriter.close(). But the program still cannot exit. But the
same code works fine on Cassandra 2.0.10.
It seems that CQLSSTableWriter cannot be closed, and blocks the program from
exit.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8281) CQLSSTableWriter close does not work
[
https://issues.apache.org/jira/browse/CASSANDRA-8281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259482#comment-14259482
]
Benyi Wang edited comment on CASSANDRA-8281 at 12/27/14 8:59 PM:
-
I believe the root cause of this problem is QueryProcessor, who creates a
thread, but never shutdowns it.
{code}
private static final ScheduledExecutorService
evictionCheckTimer=Executors.newScheduledThreadPool(1);
{code}
This code was introduced in Fix tests by making QP use its own executor
488e775cc1 after 2.1.0 and included in 2.1.1.
I used cassandra-all-2.1.2.jar from maven central. If you run the above example
code with {{Config.setClientMode(true)}} In debug mode in eclipse, you will see
the thread created by QueryProcessor keeps running after the program exits
main().
{code}
package org.apache.cassandra.hadoop.cql3;
import org.apache.cassandra.config.Config;
import org.apache.cassandra.io.sstable.CQLSSTableWriter;
public class TWriter2 {
public static void main(String[] args) {
String schema = create table mgindex.test (x int primary key,
y int);
String insert = insert into mgindex.test (x,y) values(?,?);
Config.setClientMode(true);
CQLSSTableWriter w = CQLSSTableWriter.builder()
.inDirectory(/tmp/mgindex/test).forTable(schema)
.using(insert).build();
try {
w.addRow(1, 1);
w.close();
} catch (Exception e) {
System.out.println(e);
}
}
}
{code}
If you run the similar code in a JUnit test, you won't see this problem. I
believe the thread is terminated by the test runner.
was (Author: bewang.tech):
I believe the root cause of this problem is QueryProcessor, who creates a
thread, but never shutdowns it.
{code}
private static final ScheduledExecutorService
evictionCheckTimer=Executors.newScheduledThreadPool(1);
{code}
I used cassandra-all-2.1.2.jar from maven central. If you run the above example
code with {{Config.setClientMode(true)}} In debug mode in eclipse, you will see
the thread created by QueryProcessor keeps running after the program exits
main().
{code}
package org.apache.cassandra.hadoop.cql3;
import org.apache.cassandra.config.Config;
import org.apache.cassandra.io.sstable.CQLSSTableWriter;
public class TWriter2 {
public static void main(String[] args) {
String schema = create table mgindex.test (x int primary key,
y int);
String insert = insert into mgindex.test (x,y) values(?,?);
Config.setClientMode(true);
CQLSSTableWriter w = CQLSSTableWriter.builder()
.inDirectory(/tmp/mgindex/test).forTable(schema)
.using(insert).build();
try {
w.addRow(1, 1);
w.close();
} catch (Exception e) {
System.out.println(e);
}
}
}
{code}
If you run the similar code in a JUnit test, you won't see this problem. I
believe the thread is terminated by the test runner.
CQLSSTableWriter close does not work
Key: CASSANDRA-8281
URL: https://issues.apache.org/jira/browse/CASSANDRA-8281
Project: Cassandra
Issue Type: Bug
Components: API
Environment: Cassandra 2.1.1
Reporter: Xu Zhongxing
Assignee: Benjamin Lerer
Attachments: CASSANDRA-8281.txt
I called CQLSSTableWriter.close(). But the program still cannot exit. But the
same code works fine on Cassandra 2.0.10.
It seems that CQLSSTableWriter cannot be closed, and blocks the program from
exit.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7808) LazilyCompactedRow incorrectly handles row tombstones
[
https://issues.apache.org/jira/browse/CASSANDRA-7808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259490#comment-14259490
]
Fahd commented on CASSANDRA-7808:
-
We are on Cassandra 1.2.19, but still seeing this error in Hints column family:
ERROR [HintedHandoff:3] 2014-12-27 20:53:16,742 CassandraDaemon.java (line 191)
Exception in thread Thread[HintedHandoff:3,1,main]
java.lang.RuntimeException: java.util.concurrent.ExecutionException:
java.lang.AssertionError: originally calculated column size of 5324 but now it
is 372186
at
org.apache.cassandra.db.HintedHandOffManager.doDeliverHintsToEndpoint(HintedHandOffManager.java:429)
at
org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:280)
at
org.apache.cassandra.db.HintedHandOffManager.access$300(HintedHandOffManager.java:88)
at
org.apache.cassandra.db.HintedHandOffManager$4.run(HintedHandOffManager.java:495)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException: java.lang.AssertionError:
originally calculated column size of 5324 but now it is 372186
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:188)
at
org.apache.cassandra.db.HintedHandOffManager.doDeliverHintsToEndpoint(HintedHandOffManager.java:425)
... 6 more
Caused by: java.lang.AssertionError: originally calculated column size of 5324
but now it is 372186
at
org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:135)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:160)
at
org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:162)
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:58)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60)
at
org.apache.cassandra.db.compaction.CompactionManager$7.runMayThrow(CompactionManager.java:442)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
LazilyCompactedRow incorrectly handles row tombstones
-
Key: CASSANDRA-7808
URL: https://issues.apache.org/jira/browse/CASSANDRA-7808
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Richard Low
Assignee: Richard Low
Fix For: 1.2.19, 2.0.11, 2.1.0
Attachments: 7808-v1.diff
LazilyCompactedRow doesn’t handle row tombstones correctly, leading to an
AssertionError (CASSANDRA-4206) in some cases, and the row tombstone being
incorrectly dropped in others. It looks like this was introduced by
CASSANDRA-5677.
To reproduce an AssertionError:
1. Hack a really small return value for
DatabaseDescriptor.getInMemoryCompactionLimit() like 10 bytes to force large
row compaction
2. Create a column family with gc_grace = 10
3. Insert a few columns in one row
4. Call nodetool flush
5. Delete the row
6. Call nodetool flush
7. Wait 10 seconds
8. Call nodetool compact and it will fail
To reproduce the row tombstone being dropped, do the same except, after the
delete (in step 5), insert a column that sorts before the ones you inserted
in step 3. E.g. if you inserted b, c, d in step 3, insert a now. After the
compaction, which now succeeds, the full row will be visible, rather than
just a.
The problem is two fold. Firstly, LazilyCompactedRow.Reducer.reduce() and
getReduce() incorrectly call container.clear(). This clears the columns (as
intended) but also removes the deletion times from container. This means no
further columns are deleted if they are annihilated by the row tombstone.
Secondly, after the second pass, LazilyCompactedRow.isEmpty() is called which
calls
{{ColumnFamilyStore.removeDeletedCF(emptyColumnFamily,
controller.gcBefore(key.getToken()))}}
which unfortunately removes the last deleted time from emptyColumnFamily if
it is earlier than gcBefore. Since this is only called after the second pass,
the second pass doesn’t remove any columns that are removed by the row
tombstone whereas the first pass removes just the first one.
This is pretty serious - no large rows can ever be compacted and row
tombstones can go missing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8539) Tests Errors: Latest Cassandra trunk
Rekha Joshi created CASSANDRA-8539:
--
Summary: Tests Errors: Latest Cassandra trunk
Key: CASSANDRA-8539
URL: https://issues.apache.org/jira/browse/CASSANDRA-8539
Project: Cassandra
Issue Type: Test
Components: Tests
Reporter: Rekha Joshi
On Mac 10.9.5, Java 1.7, latest cassandra trunk build is good, but tests have
14 errors/203 warnings.Few test error snippets below.
{noformat}
[junit] Testcase:
testRowTombstoneObservedBeforePurging(org.apache.cassandra.db.compaction.CompactionsPurgeTest):
FAILED
[junit] expected:0 but was:1
[junit] junit.framework.AssertionFailedError: expected:0 but was:1
[junit] at
org.apache.cassandra.db.compaction.CompactionsPurgeTest.testRowTombstoneObservedBeforePurging(CompactionsPurgeTest.java:413)
[junit]
[junit]
[junit] Test org.apache.cassandra.db.compaction.CompactionsPurgeTest FAILED
[junit] Testcase:
org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest:testNewRepairedSSTable:
Caused an ERROR
[junit] Timeout occurred. Please note the time in the report does not
reflect the time until the timeout.
[junit] junit.framework.AssertionFailedError: Timeout occurred. Please note
the time in the report does not reflect the time until the timeout.
[junit] at java.lang.Thread.run(Thread.java:745)
[junit]
[junit]
[junit] Test
org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest FAILED
(timeout)
[junit] ERROR 19:15:35 Invocation of function 'cql_test_keyspace.function_10 :
(int, int) - int' failed
[junit] java.lang.RuntimeException: null
[junit] at
org.apache.cassandra.cql3.udf.gen.Ccql_test_keyspacefunction_10_80.executeInternal(Ccql_test_keyspacefunction_10_80.java)
[na:na]
[junit] at
org.apache.cassandra.cql3.udf.gen.Ccql_test_keyspacefunction_10_80.execute(Ccql_test_keyspacefunction_10_80.java)
[na:na]
[junit] at
org.apache.cassandra.cql3.functions.UDAggregate$2.addInput(UDAggregate.java:156)
[main/:na]
[junit] at
org.apache.cassandra.cql3.selection.AggregateFunctionSelector.addInput(AggregateFunctionSelector.java:47)
[main/:na]
[junit] at
org.apache.cassandra.cql3.selection.Selection$SelectionWithProcessing$1.addInputRow(Selection.java:505)
[main/:na]
[junit] at
org.apache.cassandra.cql3.selection.Selection$ResultSetBuilder.newRow(Selection.java:315)
[main/:na]
[junit] at
org.apache.cassandra.cql3.statements.SelectStatement.processColumnFamily(SelectStatement.java:646)
[main/:na]
[junit] at
org.apache.cassandra.cql3.statements.SelectStatement.process(SelectStatement.java:580)
[main/:na]
[junit] at
org.apache.cassandra.cql3.statements.SelectStatement.processResults(SelectStatement.java:251)
[main/:na]
[junit] at
org.apache.cassandra.cql3.statements.SelectStatement.executeInternal(SelectStatement.java:275)
[main/:na]
[junit] at
org.apache.cassandra.cql3.statements.SelectStatement.executeInternal(SelectStatement.java:72)
[main/:na]
[junit] at
org.apache.cassandra.cql3.QueryProcessor.executeOnceInternal(QueryProcessor.java:362)
[main/:na]
[junit] at
org.apache.cassandra.cql3.CQLTester.execute(CQLTester.java:470) [classes/:na]
[junit] at
org.apache.cassandra.cql3.CQLTester.assertInvalidMessage(CQLTester.java:638)
[classes/:na]
[junit] at
org.apache.cassandra.cql3.CQLTester.assertInvalid(CQLTester.java:629)
[classes/:na]
[junit] at
org.apache.cassandra.cql3.AggregationTest.testJavaAggregateFailingFuncs(AggregationTest.java:456)
[classes/:na]
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[na:1.7.0_71]
[junit] at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
~[na:1.7.0_71]
[junit] at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[na:1.7.0_71]
[junit] at java.lang.reflect.Method.invoke(Method.java:606)
~[na:1.7.0_71]
[junit] at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
[junit-4.6.jar:na]
[junit] at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
[junit-4.6.jar:na]
[junit] at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
[junit-4.6.jar:na]
[junit] at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
[junit-4.6.jar:na]
[junit] at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
[junit-4.6.jar:na]
[junit] at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
[junit-4.6.jar:na]
[junit] at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
[junit-4.6.jar:na]
[jira] [Updated] (CASSANDRA-8539) Tests Errors: Latest Cassandra trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-8539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rekha Joshi updated CASSANDRA-8539:
---
Fix Version/s: 3.0
Tests Errors: Latest Cassandra trunk
Key: CASSANDRA-8539
URL: https://issues.apache.org/jira/browse/CASSANDRA-8539
Project: Cassandra
Issue Type: Test
Components: Tests
Reporter: Rekha Joshi
Labels: tests
Fix For: 3.0
On Mac 10.9.5, Java 1.7, latest cassandra trunk build is good, but tests have
14 errors/203 warnings.Few test error snippets below.
{noformat}
[junit] Testcase:
testRowTombstoneObservedBeforePurging(org.apache.cassandra.db.compaction.CompactionsPurgeTest):
FAILED
[junit] expected:0 but was:1
[junit] junit.framework.AssertionFailedError: expected:0 but was:1
[junit] at
org.apache.cassandra.db.compaction.CompactionsPurgeTest.testRowTombstoneObservedBeforePurging(CompactionsPurgeTest.java:413)
[junit]
[junit]
[junit] Test org.apache.cassandra.db.compaction.CompactionsPurgeTest
FAILED
[junit] Testcase:
org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest:testNewRepairedSSTable:
Caused an ERROR
[junit] Timeout occurred. Please note the time in the report does not
reflect the time until the timeout.
[junit] junit.framework.AssertionFailedError: Timeout occurred. Please
note the time in the report does not reflect the time until the timeout.
[junit] at java.lang.Thread.run(Thread.java:745)
[junit]
[junit]
[junit] Test
org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest FAILED
(timeout)
[junit] ERROR 19:15:35 Invocation of function 'cql_test_keyspace.function_10
: (int, int) - int' failed
[junit] java.lang.RuntimeException: null
[junit] at
org.apache.cassandra.cql3.udf.gen.Ccql_test_keyspacefunction_10_80.executeInternal(Ccql_test_keyspacefunction_10_80.java)
[na:na]
[junit] at
org.apache.cassandra.cql3.udf.gen.Ccql_test_keyspacefunction_10_80.execute(Ccql_test_keyspacefunction_10_80.java)
[na:na]
[junit] at
org.apache.cassandra.cql3.functions.UDAggregate$2.addInput(UDAggregate.java:156)
[main/:na]
[junit] at
org.apache.cassandra.cql3.selection.AggregateFunctionSelector.addInput(AggregateFunctionSelector.java:47)
[main/:na]
[junit] at
org.apache.cassandra.cql3.selection.Selection$SelectionWithProcessing$1.addInputRow(Selection.java:505)
[main/:na]
[junit] at
org.apache.cassandra.cql3.selection.Selection$ResultSetBuilder.newRow(Selection.java:315)
[main/:na]
[junit] at
org.apache.cassandra.cql3.statements.SelectStatement.processColumnFamily(SelectStatement.java:646)
[main/:na]
[junit] at
org.apache.cassandra.cql3.statements.SelectStatement.process(SelectStatement.java:580)
[main/:na]
[junit] at
org.apache.cassandra.cql3.statements.SelectStatement.processResults(SelectStatement.java:251)
[main/:na]
[junit] at
org.apache.cassandra.cql3.statements.SelectStatement.executeInternal(SelectStatement.java:275)
[main/:na]
[junit] at
org.apache.cassandra.cql3.statements.SelectStatement.executeInternal(SelectStatement.java:72)
[main/:na]
[junit] at
org.apache.cassandra.cql3.QueryProcessor.executeOnceInternal(QueryProcessor.java:362)
[main/:na]
[junit] at
org.apache.cassandra.cql3.CQLTester.execute(CQLTester.java:470) [classes/:na]
[junit] at
org.apache.cassandra.cql3.CQLTester.assertInvalidMessage(CQLTester.java:638)
[classes/:na]
[junit] at
org.apache.cassandra.cql3.CQLTester.assertInvalid(CQLTester.java:629)
[classes/:na]
[junit] at
org.apache.cassandra.cql3.AggregationTest.testJavaAggregateFailingFuncs(AggregationTest.java:456)
[classes/:na]
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[na:1.7.0_71]
[junit] at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
~[na:1.7.0_71]
[junit] at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[na:1.7.0_71]
[junit] at java.lang.reflect.Method.invoke(Method.java:606)
~[na:1.7.0_71]
[junit] at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
[junit-4.6.jar:na]
[junit] at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
[junit-4.6.jar:na]
[junit] at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
[junit-4.6.jar:na]
[junit] at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
[junit-4.6.jar:na]
[junit] at
[jira] [Updated] (CASSANDRA-8539) Tests Errors: Latest Cassandra trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-8539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rekha Joshi updated CASSANDRA-8539:
---
Labels: tests (was: )
Tests Errors: Latest Cassandra trunk
Key: CASSANDRA-8539
URL: https://issues.apache.org/jira/browse/CASSANDRA-8539
Project: Cassandra
Issue Type: Test
Components: Tests
Reporter: Rekha Joshi
Labels: tests
Fix For: 3.0
On Mac 10.9.5, Java 1.7, latest cassandra trunk build is good, but tests have
14 errors/203 warnings.Few test error snippets below.
{noformat}
[junit] Testcase:
testRowTombstoneObservedBeforePurging(org.apache.cassandra.db.compaction.CompactionsPurgeTest):
FAILED
[junit] expected:0 but was:1
[junit] junit.framework.AssertionFailedError: expected:0 but was:1
[junit] at
org.apache.cassandra.db.compaction.CompactionsPurgeTest.testRowTombstoneObservedBeforePurging(CompactionsPurgeTest.java:413)
[junit]
[junit]
[junit] Test org.apache.cassandra.db.compaction.CompactionsPurgeTest
FAILED
[junit] Testcase:
org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest:testNewRepairedSSTable:
Caused an ERROR
[junit] Timeout occurred. Please note the time in the report does not
reflect the time until the timeout.
[junit] junit.framework.AssertionFailedError: Timeout occurred. Please
note the time in the report does not reflect the time until the timeout.
[junit] at java.lang.Thread.run(Thread.java:745)
[junit]
[junit]
[junit] Test
org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest FAILED
(timeout)
[junit] ERROR 19:15:35 Invocation of function 'cql_test_keyspace.function_10
: (int, int) - int' failed
[junit] java.lang.RuntimeException: null
[junit] at
org.apache.cassandra.cql3.udf.gen.Ccql_test_keyspacefunction_10_80.executeInternal(Ccql_test_keyspacefunction_10_80.java)
[na:na]
[junit] at
org.apache.cassandra.cql3.udf.gen.Ccql_test_keyspacefunction_10_80.execute(Ccql_test_keyspacefunction_10_80.java)
[na:na]
[junit] at
org.apache.cassandra.cql3.functions.UDAggregate$2.addInput(UDAggregate.java:156)
[main/:na]
[junit] at
org.apache.cassandra.cql3.selection.AggregateFunctionSelector.addInput(AggregateFunctionSelector.java:47)
[main/:na]
[junit] at
org.apache.cassandra.cql3.selection.Selection$SelectionWithProcessing$1.addInputRow(Selection.java:505)
[main/:na]
[junit] at
org.apache.cassandra.cql3.selection.Selection$ResultSetBuilder.newRow(Selection.java:315)
[main/:na]
[junit] at
org.apache.cassandra.cql3.statements.SelectStatement.processColumnFamily(SelectStatement.java:646)
[main/:na]
[junit] at
org.apache.cassandra.cql3.statements.SelectStatement.process(SelectStatement.java:580)
[main/:na]
[junit] at
org.apache.cassandra.cql3.statements.SelectStatement.processResults(SelectStatement.java:251)
[main/:na]
[junit] at
org.apache.cassandra.cql3.statements.SelectStatement.executeInternal(SelectStatement.java:275)
[main/:na]
[junit] at
org.apache.cassandra.cql3.statements.SelectStatement.executeInternal(SelectStatement.java:72)
[main/:na]
[junit] at
org.apache.cassandra.cql3.QueryProcessor.executeOnceInternal(QueryProcessor.java:362)
[main/:na]
[junit] at
org.apache.cassandra.cql3.CQLTester.execute(CQLTester.java:470) [classes/:na]
[junit] at
org.apache.cassandra.cql3.CQLTester.assertInvalidMessage(CQLTester.java:638)
[classes/:na]
[junit] at
org.apache.cassandra.cql3.CQLTester.assertInvalid(CQLTester.java:629)
[classes/:na]
[junit] at
org.apache.cassandra.cql3.AggregationTest.testJavaAggregateFailingFuncs(AggregationTest.java:456)
[classes/:na]
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[na:1.7.0_71]
[junit] at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
~[na:1.7.0_71]
[junit] at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[na:1.7.0_71]
[junit] at java.lang.reflect.Method.invoke(Method.java:606)
~[na:1.7.0_71]
[junit] at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
[junit-4.6.jar:na]
[junit] at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
[junit-4.6.jar:na]
[junit] at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
[junit-4.6.jar:na]
[junit] at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
[junit-4.6.jar:na]
[junit] at
[jira] [Comment Edited] (CASSANDRA-7808) LazilyCompactedRow incorrectly handles row tombstones
[
https://issues.apache.org/jira/browse/CASSANDRA-7808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259490#comment-14259490
]
Fahd Siddiqui edited comment on CASSANDRA-7808 at 12/27/14 10:27 PM:
-
We are on Cassandra 1.2.19, but still seeing this error in Hints column family:
{code}
ERROR [HintedHandoff:3] 2014-12-27 20:53:16,742 CassandraDaemon.java (line 191)
Exception in thread Thread[HintedHandoff:3,1,main]
java.lang.RuntimeException: java.util.concurrent.ExecutionException:
java.lang.AssertionError: originally calculated column size of 5324 but now it
is 372186
at
org.apache.cassandra.db.HintedHandOffManager.doDeliverHintsToEndpoint(HintedHandOffManager.java:429)
at
org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:280)
at
org.apache.cassandra.db.HintedHandOffManager.access$300(HintedHandOffManager.java:88)
at
org.apache.cassandra.db.HintedHandOffManager$4.run(HintedHandOffManager.java:495)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException: java.lang.AssertionError:
originally calculated column size of 5324 but now it is 372186
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:188)
at
org.apache.cassandra.db.HintedHandOffManager.doDeliverHintsToEndpoint(HintedHandOffManager.java:425)
... 6 more
Caused by: java.lang.AssertionError: originally calculated column size of 5324
but now it is 372186
at
org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:135)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:160)
at
org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:162)
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:58)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60)
at
org.apache.cassandra.db.compaction.CompactionManager$7.runMayThrow(CompactionManager.java:442)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
{code}
was (Author: fahdsiddiqui):
We are on Cassandra 1.2.19, but still seeing this error in Hints column family:
ERROR [HintedHandoff:3] 2014-12-27 20:53:16,742 CassandraDaemon.java (line 191)
Exception in thread Thread[HintedHandoff:3,1,main]
java.lang.RuntimeException: java.util.concurrent.ExecutionException:
java.lang.AssertionError: originally calculated column size of 5324 but now it
is 372186
at
org.apache.cassandra.db.HintedHandOffManager.doDeliverHintsToEndpoint(HintedHandOffManager.java:429)
at
org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:280)
at
org.apache.cassandra.db.HintedHandOffManager.access$300(HintedHandOffManager.java:88)
at
org.apache.cassandra.db.HintedHandOffManager$4.run(HintedHandOffManager.java:495)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException: java.lang.AssertionError:
originally calculated column size of 5324 but now it is 372186
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:188)
at
org.apache.cassandra.db.HintedHandOffManager.doDeliverHintsToEndpoint(HintedHandOffManager.java:425)
... 6 more
Caused by: java.lang.AssertionError: originally calculated column size of 5324
but now it is 372186
at
org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:135)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:160)
at
org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:162)
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:58)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60)
at
[jira] [Commented] (CASSANDRA-7768) Error when creating multiple CQLSSTableWriters for more than one column family in the same keyspace
[
https://issues.apache.org/jira/browse/CASSANDRA-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259509#comment-14259509
]
Benyi Wang commented on CASSANDRA-7768:
---
I encounter this problem in cassandra-all.2.1.2 if you write two column
families of the same keyspace in one JVM. The following code can reproduce the
error.
{code}
package org.apache.cassandra.hadoop.cql3;
import java.lang.reflect.Field;
import java.util.concurrent.ScheduledExecutorService;
import org.apache.cassandra.config.Config;
import org.apache.cassandra.cql3.QueryProcessor;
import org.apache.cassandra.io.sstable.CQLSSTableWriter;
public class TWriter2 {
public static void main(String[] args) {
String schema = create table mgindex.test (x int primary key,
y int);
String insert = insert into mgindex.test (x,y) values(?,?);
String schema1 = create table mgindex.test1 (x int primary
key, y int);
String insert1 = insert into mgindex.test1 (x,y) values(?,?);
Config.setClientMode(true);
try {
CQLSSTableWriter w = CQLSSTableWriter.builder()
.inDirectory(/tmp/feat).forTable(schema).using(insert)
.build();
w.addRow(1, 1);
w.close();
CQLSSTableWriter w1 = CQLSSTableWriter.builder()
.inDirectory(/tmp/feat).forTable(schema1).using(insert1)
.build();
w1.addRow(4, 4);
w1.close();
} catch (Exception e) {
System.out.println(e);
}
}
}
{code}
You will see this exception when the second {{forTable}} is called.
{code}
Exception in thread main java.lang.ExceptionInInitializerError
at
org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.forTable(CQLSSTableWriter.java:360)
at org.apache.cassandra.hadoop.cql3.TWriter2.main(TWriter2.java:39)
Caused by: java.lang.NullPointerException
at
org.apache.cassandra.config.DatabaseDescriptor.createAllDirectories(DatabaseDescriptor.java:710)
at org.apache.cassandra.db.Keyspace.clinit(Keyspace.java:72)
... 2 more
{code}
Here is how the code fails for the second column family:
# For the first column family: ksm is null, and it will work correctly
# For the second column family: ksm is not null, it goes into the second branch
to check if the table exists. If not, load the this.schema into
Schema.instance.
{code}
// We need to register the keyspace/table metadata through
Schema, otherwise we won't be able to properly
// build the insert statement in using().
KSMetaData ksm =
Schema.instance.getKSMetaData(this.schema.ksName);
if (ksm == null)
{
ksm = KSMetaData.newKeyspace(this.schema.ksName,
AbstractReplicationStrategy.getClass(org.apache.cassandra.locator.SimpleStrategy),
ImmutableMap.of(replication_factor, 1),
true,
Collections.singleton(this.schema));
Schema.instance.load(ksm);
}
else if (Schema.instance.getCFMetaData(this.schema.ksName,
this.schema.cfName) == null)
{
Schema.instance.load(this.schema);
ksm = KSMetaData.cloneWith(ksm,
Iterables.concat(ksm.cfMetaData().values(),
Collections.singleton(this.schema)));
Schema.instance.setKeyspaceDefinition(ksm);
Keyspace.open(ksm.name).initCf(this.schema.cfId,
this.schema.cfName, false);
}
return this;
{code}
The problem happens when Keyspace class is initialized at {{Keyspace.open...}},
{code}
static
{
if (!StorageService.instance.isClientMode())
DatabaseDescriptor.createAllDirectories();
}
{code}
At this time, StorageService is not in client mode, and createAllDirectories
will fail because conf.data_file_directories is null.
I'm not quite sure why {{Keyspace.open(...}} is needed. When I comment out this
line, everything seems working.
Could you reopen this jira and fix it?
Error when creating multiple CQLSSTableWriters for more than one column
family in the same keyspace
---
Key: CASSANDRA-7768
URL: https://issues.apache.org/jira/browse/CASSANDRA-7768
Project: Cassandra
Issue Type: Bug
[jira] [Commented] (CASSANDRA-8281) CQLSSTableWriter close does not work
[
https://issues.apache.org/jira/browse/CASSANDRA-8281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259510#comment-14259510
]
Benyi Wang commented on CASSANDRA-8281:
---
if I invoke {{evictionCheckTimer.shutdown}} using java reflection, the program
will exit successfully. Call this method after {{w.close()}}.
{code}
public static void shutdownQueryProcessorThread()
throws NoSuchFieldException, SecurityException,
IllegalArgumentException, IllegalAccessException {
Field f =
QueryProcessor.class.getDeclaredField(evictionCheckTimer);
f.setAccessible(true);
ScheduledExecutorService evictionCheckTimer =
(ScheduledExecutorService) f
.get(null);
evictionCheckTimer.shutdown();
}
{code}
CQLSSTableWriter close does not work
Key: CASSANDRA-8281
URL: https://issues.apache.org/jira/browse/CASSANDRA-8281
Project: Cassandra
Issue Type: Bug
Components: API
Environment: Cassandra 2.1.1
Reporter: Xu Zhongxing
Assignee: Benjamin Lerer
Attachments: CASSANDRA-8281.txt
I called CQLSSTableWriter.close(). But the program still cannot exit. But the
same code works fine on Cassandra 2.0.10.
It seems that CQLSSTableWriter cannot be closed, and blocks the program from
exit.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8540) CQL: Describe table does not show index interval properties set on create.
[
https://issues.apache.org/jira/browse/CASSANDRA-8540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rekha Joshi updated CASSANDRA-8540:
---
Labels: cql (was: )
CQL: Describe table does not show index interval properties set on create.
--
Key: CASSANDRA-8540
URL: https://issues.apache.org/jira/browse/CASSANDRA-8540
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Rekha Joshi
Labels: cql
Fix For: 3.0
On Mac 10.9.5, Java 1.7, latest cassandra trunk the describe of table does
not show index interval(min/max) properties on a table.
{noformat}
cqlsh:playground CREATE TABLE test (
... key int PRIMARY KEY
... ) WITH bloom_filter_fp_chance = 0.01
... AND caching = '{keys:ALL,
rows_per_partition:NONE}'
... AND comment = ''
... AND compaction = {'min_threshold': '4', 'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32'}
... AND compression = {'sstable_compression':
'org.apache.cassandra.io.compress.LZ4Compressor'}
... AND default_time_to_live = 0
... AND gc_grace_seconds = 864000
... AND max_index_interval = 2048
... AND memtable_flush_period_in_ms = 0
... AND min_index_interval = 128
... AND populate_io_cache_on_flush = false
... AND read_repair_chance = 0.1
... AND speculative_retry = '99.0PERCENTILE';
cqlsh:playground desc table test;
CREATE TABLE test (
key int,
PRIMARY KEY (key)
) WITH
bloom_filter_fp_chance=0.01 AND
caching='{keys:ALL, rows_per_partition:NONE}' AND
comment='' AND
dclocal_read_repair_chance=0.10 AND
gc_grace_seconds=864000 AND
read_repair_chance=0.10 AND
default_time_to_live=0 AND
speculative_retry='99.0PERCENTILE' AND
memtable_flush_period_in_ms=0 AND
compaction={'min_threshold': '4', 'class': 'SizeTieredCompactionStrategy',
'max_threshold': '32'} AND
compression={'sstable_compression': 'LZ4Compressor’};
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8540) CQL: Describe table does not show index interval properties set on create.
[
https://issues.apache.org/jira/browse/CASSANDRA-8540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rekha Joshi updated CASSANDRA-8540:
---
Fix Version/s: 3.0
CQL: Describe table does not show index interval properties set on create.
--
Key: CASSANDRA-8540
URL: https://issues.apache.org/jira/browse/CASSANDRA-8540
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Rekha Joshi
Fix For: 3.0
On Mac 10.9.5, Java 1.7, latest cassandra trunk the describe of table does
not show index interval(min/max) properties on a table.
{noformat}
cqlsh:playground CREATE TABLE test (
... key int PRIMARY KEY
... ) WITH bloom_filter_fp_chance = 0.01
... AND caching = '{keys:ALL,
rows_per_partition:NONE}'
... AND comment = ''
... AND compaction = {'min_threshold': '4', 'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32'}
... AND compression = {'sstable_compression':
'org.apache.cassandra.io.compress.LZ4Compressor'}
... AND default_time_to_live = 0
... AND gc_grace_seconds = 864000
... AND max_index_interval = 2048
... AND memtable_flush_period_in_ms = 0
... AND min_index_interval = 128
... AND populate_io_cache_on_flush = false
... AND read_repair_chance = 0.1
... AND speculative_retry = '99.0PERCENTILE';
cqlsh:playground desc table test;
CREATE TABLE test (
key int,
PRIMARY KEY (key)
) WITH
bloom_filter_fp_chance=0.01 AND
caching='{keys:ALL, rows_per_partition:NONE}' AND
comment='' AND
dclocal_read_repair_chance=0.10 AND
gc_grace_seconds=864000 AND
read_repair_chance=0.10 AND
default_time_to_live=0 AND
speculative_retry='99.0PERCENTILE' AND
memtable_flush_period_in_ms=0 AND
compaction={'min_threshold': '4', 'class': 'SizeTieredCompactionStrategy',
'max_threshold': '32'} AND
compression={'sstable_compression': 'LZ4Compressor’};
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8540) CQL: Describe table does not show index interval properties set on create.
Rekha Joshi created CASSANDRA-8540:
--
Summary: CQL: Describe table does not show index interval
properties set on create.
Key: CASSANDRA-8540
URL: https://issues.apache.org/jira/browse/CASSANDRA-8540
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Rekha Joshi
On Mac 10.9.5, Java 1.7, latest cassandra trunk the describe of table does not
show index interval(min/max) properties on a table.
{noformat}
cqlsh:playground CREATE TABLE test (
... key int PRIMARY KEY
... ) WITH bloom_filter_fp_chance = 0.01
... AND caching = '{keys:ALL,
rows_per_partition:NONE}'
... AND comment = ''
... AND compaction = {'min_threshold': '4', 'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32'}
... AND compression = {'sstable_compression':
'org.apache.cassandra.io.compress.LZ4Compressor'}
... AND default_time_to_live = 0
... AND gc_grace_seconds = 864000
... AND max_index_interval = 2048
... AND memtable_flush_period_in_ms = 0
... AND min_index_interval = 128
... AND populate_io_cache_on_flush = false
... AND read_repair_chance = 0.1
... AND speculative_retry = '99.0PERCENTILE';
cqlsh:playground desc table test;
CREATE TABLE test (
key int,
PRIMARY KEY (key)
) WITH
bloom_filter_fp_chance=0.01 AND
caching='{keys:ALL, rows_per_partition:NONE}' AND
comment='' AND
dclocal_read_repair_chance=0.10 AND
gc_grace_seconds=864000 AND
read_repair_chance=0.10 AND
default_time_to_live=0 AND
speculative_retry='99.0PERCENTILE' AND
memtable_flush_period_in_ms=0 AND
compaction={'min_threshold': '4', 'class': 'SizeTieredCompactionStrategy',
'max_threshold': '32'} AND
compression={'sstable_compression': 'LZ4Compressor’};
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8541) References to non-existent/deprecated CqlPagingInputFormat in code
Rekha Joshi created CASSANDRA-8541: -- Summary: References to non-existent/deprecated CqlPagingInputFormat in code Key: CASSANDRA-8541 URL: https://issues.apache.org/jira/browse/CASSANDRA-8541 Project: Cassandra Issue Type: Bug Components: Hadoop Reporter: Rekha Joshi On Mac 10.9.5, Java 1.7, latest cassandra trunk - References to non-existent/deprecated CqlPagingInputFormat in code. As per Changes.txt/7570 both CqlPagingInputFormat and CqlPagingRecordReader are removed, but lingering references in WordCount,CqlStorage.. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8541) References to non-existent/deprecated CqlPagingInputFormat in code
[ https://issues.apache.org/jira/browse/CASSANDRA-8541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi updated CASSANDRA-8541: --- Attachment: CASSANDRA-8541.txt References to non-existent/deprecated CqlPagingInputFormat in code -- Key: CASSANDRA-8541 URL: https://issues.apache.org/jira/browse/CASSANDRA-8541 Project: Cassandra Issue Type: Bug Components: Hadoop Reporter: Rekha Joshi Labels: hadoop Fix For: 2.0.12 Attachments: CASSANDRA-8541.txt On Mac 10.9.5, Java 1.7, latest cassandra trunk - References to non-existent/deprecated CqlPagingInputFormat in code. As per Changes.txt/7570 both CqlPagingInputFormat and CqlPagingRecordReader are removed, but lingering references in WordCount,CqlStorage.. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7269) Make CqlInputFormat and CqlRecordReader consistent with comments
[
https://issues.apache.org/jira/browse/CASSANDRA-7269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259526#comment-14259526
]
Rekha Joshi commented on CASSANDRA-7269:
This no longer applies.As both CqlPagingRecordReader, CqlPagingInputFormat are
removed/deprecated.
Attached patch to remove unused reference of DEFAULT_CQL_PAGE_LIMIT from
CqlRecordReader.Thanks.
Make CqlInputFormat and CqlRecordReader consistent with comments
Key: CASSANDRA-7269
URL: https://issues.apache.org/jira/browse/CASSANDRA-7269
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Reporter: Jeremy Hanna
Priority: Minor
Labels: hadoop
Both the CqlInputFormat and CqlPagingInputFormat have the following comment:
{code}
/**
...
* the number of CQL rows per page
* CQLConfigHelper.setInputCQLPageRowSize. The default page row size is
1000. You
* should set it to as big as possible, but no bigger. It set the LIMIT
for the CQL
* query, so you need set it big enough to minimize the network overhead,
and also
* not too big to avoid out of memory issue.
...
**/
{code}
The property is used in both classes, but the default is only set to 1000 in
CqlPagingRecordReader explicitly.
We should either make the default part of the CqlConfigHelper so it's set in
both places or update the comments in the CqlInputFormat to say that if it's
not set, it will default to the java driver fetch size which is 5000.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7269) Make CqlInputFormat and CqlRecordReader consistent with comments
[
https://issues.apache.org/jira/browse/CASSANDRA-7269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rekha Joshi updated CASSANDRA-7269:
---
Attachment: CASSANDRA-7269.txt
Make CqlInputFormat and CqlRecordReader consistent with comments
Key: CASSANDRA-7269
URL: https://issues.apache.org/jira/browse/CASSANDRA-7269
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Reporter: Jeremy Hanna
Priority: Minor
Labels: hadoop
Attachments: CASSANDRA-7269.txt
Both the CqlInputFormat and CqlPagingInputFormat have the following comment:
{code}
/**
...
* the number of CQL rows per page
* CQLConfigHelper.setInputCQLPageRowSize. The default page row size is
1000. You
* should set it to as big as possible, but no bigger. It set the LIMIT
for the CQL
* query, so you need set it big enough to minimize the network overhead,
and also
* not too big to avoid out of memory issue.
...
**/
{code}
The property is used in both classes, but the default is only set to 1000 in
CqlPagingRecordReader explicitly.
We should either make the default part of the CqlConfigHelper so it's set in
both places or update the comments in the CqlInputFormat to say that if it's
not set, it will default to the java driver fetch size which is 5000.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8540) CQL: Describe table does not show index interval properties set on create.
[
https://issues.apache.org/jira/browse/CASSANDRA-8540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259527#comment-14259527
]
Rekha Joshi commented on CASSANDRA-8540:
Looking into this.
CQL: Describe table does not show index interval properties set on create.
--
Key: CASSANDRA-8540
URL: https://issues.apache.org/jira/browse/CASSANDRA-8540
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Rekha Joshi
Labels: cql
Fix For: 3.0
On Mac 10.9.5, Java 1.7, latest cassandra trunk the describe of table does
not show index interval(min/max) properties on a table.
{noformat}
cqlsh:playground CREATE TABLE test (
... key int PRIMARY KEY
... ) WITH bloom_filter_fp_chance = 0.01
... AND caching = '{keys:ALL,
rows_per_partition:NONE}'
... AND comment = ''
... AND compaction = {'min_threshold': '4', 'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32'}
... AND compression = {'sstable_compression':
'org.apache.cassandra.io.compress.LZ4Compressor'}
... AND default_time_to_live = 0
... AND gc_grace_seconds = 864000
... AND max_index_interval = 2048
... AND memtable_flush_period_in_ms = 0
... AND min_index_interval = 128
... AND populate_io_cache_on_flush = false
... AND read_repair_chance = 0.1
... AND speculative_retry = '99.0PERCENTILE';
cqlsh:playground desc table test;
CREATE TABLE test (
key int,
PRIMARY KEY (key)
) WITH
bloom_filter_fp_chance=0.01 AND
caching='{keys:ALL, rows_per_partition:NONE}' AND
comment='' AND
dclocal_read_repair_chance=0.10 AND
gc_grace_seconds=864000 AND
read_repair_chance=0.10 AND
default_time_to_live=0 AND
speculative_retry='99.0PERCENTILE' AND
memtable_flush_period_in_ms=0 AND
compaction={'min_threshold': '4', 'class': 'SizeTieredCompactionStrategy',
'max_threshold': '32'} AND
compression={'sstable_compression': 'LZ4Compressor’};
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8541) References to non-existent/deprecated CqlPagingInputFormat in code
[ https://issues.apache.org/jira/browse/CASSANDRA-8541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi updated CASSANDRA-8541: --- Reviewer: Dave Brosius References to non-existent/deprecated CqlPagingInputFormat in code -- Key: CASSANDRA-8541 URL: https://issues.apache.org/jira/browse/CASSANDRA-8541 Project: Cassandra Issue Type: Bug Components: Hadoop Reporter: Rekha Joshi Labels: hadoop Fix For: 2.0.12 Attachments: CASSANDRA-8541.txt On Mac 10.9.5, Java 1.7, latest cassandra trunk - References to non-existent/deprecated CqlPagingInputFormat in code. As per Changes.txt/7570 both CqlPagingInputFormat and CqlPagingRecordReader are removed, but lingering references in WordCount,CqlStorage.. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
cassandra git commit: fix references to defunct CqlPagingInputFormat patch by rjoshi reviewed by dbrosius for CASSANDRA-8541
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.0 f981bd596 - e550ea602
fix references to defunct CqlPagingInputFormat
patch by rjoshi reviewed by dbrosius for CASSANDRA-8541
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e550ea60
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e550ea60
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e550ea60
Branch: refs/heads/cassandra-2.0
Commit: e550ea60212e933f3849a11717ba4ef916fc4aa3
Parents: f981bd5
Author: rekhajoshm dbros...@mebigfatguy.com
Authored: Sat Dec 27 20:23:01 2014 -0500
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Sat Dec 27 20:23:01 2014 -0500
--
examples/hadoop_cql3_word_count/src/WordCount.java | 3 +--
examples/hadoop_cql3_word_count/src/WordCountCounters.java | 3 +--
src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java | 2 +-
3 files changed, 3 insertions(+), 5 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e550ea60/examples/hadoop_cql3_word_count/src/WordCount.java
--
diff --git a/examples/hadoop_cql3_word_count/src/WordCount.java
b/examples/hadoop_cql3_word_count/src/WordCount.java
index 519a98f..6a2f846 100644
--- a/examples/hadoop_cql3_word_count/src/WordCount.java
+++ b/examples/hadoop_cql3_word_count/src/WordCount.java
@@ -26,7 +26,6 @@ import org.apache.cassandra.hadoop.cql3.CqlOutputFormat;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-import org.apache.cassandra.hadoop.cql3.CqlPagingInputFormat;
import org.apache.cassandra.hadoop.cql3.CqlInputFormat;
import org.apache.cassandra.hadoop.ConfigHelper;
import org.apache.cassandra.utils.ByteBufferUtil;
@@ -247,7 +246,7 @@ public class WordCount extends Configured implements Tool
else
{
job.setMapperClass(TokenizerMapper.class);
-job.setInputFormatClass(CqlPagingInputFormat.class);
+job.setInputFormatClass(CqlInputFormat.class);
ConfigHelper.setInputRpcPort(job.getConfiguration(), 9160);
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e550ea60/examples/hadoop_cql3_word_count/src/WordCountCounters.java
--
diff --git a/examples/hadoop_cql3_word_count/src/WordCountCounters.java
b/examples/hadoop_cql3_word_count/src/WordCountCounters.java
index 74de9ab..150d18d 100644
--- a/examples/hadoop_cql3_word_count/src/WordCountCounters.java
+++ b/examples/hadoop_cql3_word_count/src/WordCountCounters.java
@@ -25,7 +25,6 @@ import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.apache.cassandra.hadoop.cql3.CqlConfigHelper;
-import org.apache.cassandra.hadoop.cql3.CqlPagingInputFormat;
import org.apache.cassandra.hadoop.cql3.CqlInputFormat;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.conf.Configured;
@@ -156,7 +155,7 @@ public class WordCountCounters extends Configured
implements Tool
else
{
job.setMapperClass(SumMapper.class);
-job.setInputFormatClass(CqlPagingInputFormat.class);
+job.setInputFormatClass(CqlInputFormat.class);
ConfigHelper.setInputRpcPort(job.getConfiguration(), 9160);
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e550ea60/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
b/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
index 53f3900..6c50ab2 100644
--- a/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
+++ b/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
@@ -74,7 +74,7 @@ public class CqlStorage extends AbstractCassandraStorage
{
super();
this.pageSize = pageSize;
-DEFAULT_INPUT_FORMAT =
org.apache.cassandra.hadoop.cql3.CqlPagingInputFormat;
+DEFAULT_INPUT_FORMAT =
org.apache.cassandra.hadoop.cql3.CqlInputFormat;
DEFAULT_OUTPUT_FORMAT =
org.apache.cassandra.hadoop.cql3.CqlOutputFormat;
}
[2/2] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6124a733 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6124a733 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6124a733 Branch: refs/heads/cassandra-2.1 Commit: 6124a733cb1473f3951f2cae8dcaa9c83b9a3b8d Parents: 12b9691 e550ea6 Author: Dave Brosius dbros...@mebigfatguy.com Authored: Sat Dec 27 20:32:20 2014 -0500 Committer: Dave Brosius dbros...@mebigfatguy.com Committed: Sat Dec 27 20:32:20 2014 -0500 -- examples/hadoop_cql3_word_count/src/WordCount.java | 3 +-- examples/hadoop_cql3_word_count/src/WordCountCounters.java | 3 +-- src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java | 2 +- 3 files changed, 3 insertions(+), 5 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/6124a733/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java --
[1/2] cassandra git commit: fix references to defunct CqlPagingInputFormat patch by rjoshi reviewed by dbrosius for CASSANDRA-8541
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 12b96915c - 6124a733c
fix references to defunct CqlPagingInputFormat
patch by rjoshi reviewed by dbrosius for CASSANDRA-8541
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e550ea60
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e550ea60
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e550ea60
Branch: refs/heads/cassandra-2.1
Commit: e550ea60212e933f3849a11717ba4ef916fc4aa3
Parents: f981bd5
Author: rekhajoshm dbros...@mebigfatguy.com
Authored: Sat Dec 27 20:23:01 2014 -0500
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Sat Dec 27 20:23:01 2014 -0500
--
examples/hadoop_cql3_word_count/src/WordCount.java | 3 +--
examples/hadoop_cql3_word_count/src/WordCountCounters.java | 3 +--
src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java | 2 +-
3 files changed, 3 insertions(+), 5 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e550ea60/examples/hadoop_cql3_word_count/src/WordCount.java
--
diff --git a/examples/hadoop_cql3_word_count/src/WordCount.java
b/examples/hadoop_cql3_word_count/src/WordCount.java
index 519a98f..6a2f846 100644
--- a/examples/hadoop_cql3_word_count/src/WordCount.java
+++ b/examples/hadoop_cql3_word_count/src/WordCount.java
@@ -26,7 +26,6 @@ import org.apache.cassandra.hadoop.cql3.CqlOutputFormat;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-import org.apache.cassandra.hadoop.cql3.CqlPagingInputFormat;
import org.apache.cassandra.hadoop.cql3.CqlInputFormat;
import org.apache.cassandra.hadoop.ConfigHelper;
import org.apache.cassandra.utils.ByteBufferUtil;
@@ -247,7 +246,7 @@ public class WordCount extends Configured implements Tool
else
{
job.setMapperClass(TokenizerMapper.class);
-job.setInputFormatClass(CqlPagingInputFormat.class);
+job.setInputFormatClass(CqlInputFormat.class);
ConfigHelper.setInputRpcPort(job.getConfiguration(), 9160);
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e550ea60/examples/hadoop_cql3_word_count/src/WordCountCounters.java
--
diff --git a/examples/hadoop_cql3_word_count/src/WordCountCounters.java
b/examples/hadoop_cql3_word_count/src/WordCountCounters.java
index 74de9ab..150d18d 100644
--- a/examples/hadoop_cql3_word_count/src/WordCountCounters.java
+++ b/examples/hadoop_cql3_word_count/src/WordCountCounters.java
@@ -25,7 +25,6 @@ import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.apache.cassandra.hadoop.cql3.CqlConfigHelper;
-import org.apache.cassandra.hadoop.cql3.CqlPagingInputFormat;
import org.apache.cassandra.hadoop.cql3.CqlInputFormat;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.conf.Configured;
@@ -156,7 +155,7 @@ public class WordCountCounters extends Configured
implements Tool
else
{
job.setMapperClass(SumMapper.class);
-job.setInputFormatClass(CqlPagingInputFormat.class);
+job.setInputFormatClass(CqlInputFormat.class);
ConfigHelper.setInputRpcPort(job.getConfiguration(), 9160);
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e550ea60/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
b/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
index 53f3900..6c50ab2 100644
--- a/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
+++ b/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
@@ -74,7 +74,7 @@ public class CqlStorage extends AbstractCassandraStorage
{
super();
this.pageSize = pageSize;
-DEFAULT_INPUT_FORMAT =
org.apache.cassandra.hadoop.cql3.CqlPagingInputFormat;
+DEFAULT_INPUT_FORMAT =
org.apache.cassandra.hadoop.cql3.CqlInputFormat;
DEFAULT_OUTPUT_FORMAT =
org.apache.cassandra.hadoop.cql3.CqlOutputFormat;
}
[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/36b287be Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/36b287be Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/36b287be Branch: refs/heads/trunk Commit: 36b287bea4e98cfd737b17a3b3a9391a9654cd72 Parents: 17d9791 6124a73 Author: Dave Brosius dbros...@mebigfatguy.com Authored: Sat Dec 27 20:40:16 2014 -0500 Committer: Dave Brosius dbros...@mebigfatguy.com Committed: Sat Dec 27 20:40:16 2014 -0500 -- examples/hadoop_cql3_word_count/src/WordCount.java | 3 +-- examples/hadoop_cql3_word_count/src/WordCountCounters.java | 3 +-- src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java | 2 +- 3 files changed, 3 insertions(+), 5 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/36b287be/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java --
[2/3] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6124a733 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6124a733 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6124a733 Branch: refs/heads/trunk Commit: 6124a733cb1473f3951f2cae8dcaa9c83b9a3b8d Parents: 12b9691 e550ea6 Author: Dave Brosius dbros...@mebigfatguy.com Authored: Sat Dec 27 20:32:20 2014 -0500 Committer: Dave Brosius dbros...@mebigfatguy.com Committed: Sat Dec 27 20:32:20 2014 -0500 -- examples/hadoop_cql3_word_count/src/WordCount.java | 3 +-- examples/hadoop_cql3_word_count/src/WordCountCounters.java | 3 +-- src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java | 2 +- 3 files changed, 3 insertions(+), 5 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/6124a733/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java --
[1/3] cassandra git commit: fix references to defunct CqlPagingInputFormat patch by rjoshi reviewed by dbrosius for CASSANDRA-8541
Repository: cassandra
Updated Branches:
refs/heads/trunk 17d9791d6 - 36b287bea
fix references to defunct CqlPagingInputFormat
patch by rjoshi reviewed by dbrosius for CASSANDRA-8541
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e550ea60
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e550ea60
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e550ea60
Branch: refs/heads/trunk
Commit: e550ea60212e933f3849a11717ba4ef916fc4aa3
Parents: f981bd5
Author: rekhajoshm dbros...@mebigfatguy.com
Authored: Sat Dec 27 20:23:01 2014 -0500
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Sat Dec 27 20:23:01 2014 -0500
--
examples/hadoop_cql3_word_count/src/WordCount.java | 3 +--
examples/hadoop_cql3_word_count/src/WordCountCounters.java | 3 +--
src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java | 2 +-
3 files changed, 3 insertions(+), 5 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e550ea60/examples/hadoop_cql3_word_count/src/WordCount.java
--
diff --git a/examples/hadoop_cql3_word_count/src/WordCount.java
b/examples/hadoop_cql3_word_count/src/WordCount.java
index 519a98f..6a2f846 100644
--- a/examples/hadoop_cql3_word_count/src/WordCount.java
+++ b/examples/hadoop_cql3_word_count/src/WordCount.java
@@ -26,7 +26,6 @@ import org.apache.cassandra.hadoop.cql3.CqlOutputFormat;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-import org.apache.cassandra.hadoop.cql3.CqlPagingInputFormat;
import org.apache.cassandra.hadoop.cql3.CqlInputFormat;
import org.apache.cassandra.hadoop.ConfigHelper;
import org.apache.cassandra.utils.ByteBufferUtil;
@@ -247,7 +246,7 @@ public class WordCount extends Configured implements Tool
else
{
job.setMapperClass(TokenizerMapper.class);
-job.setInputFormatClass(CqlPagingInputFormat.class);
+job.setInputFormatClass(CqlInputFormat.class);
ConfigHelper.setInputRpcPort(job.getConfiguration(), 9160);
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e550ea60/examples/hadoop_cql3_word_count/src/WordCountCounters.java
--
diff --git a/examples/hadoop_cql3_word_count/src/WordCountCounters.java
b/examples/hadoop_cql3_word_count/src/WordCountCounters.java
index 74de9ab..150d18d 100644
--- a/examples/hadoop_cql3_word_count/src/WordCountCounters.java
+++ b/examples/hadoop_cql3_word_count/src/WordCountCounters.java
@@ -25,7 +25,6 @@ import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.apache.cassandra.hadoop.cql3.CqlConfigHelper;
-import org.apache.cassandra.hadoop.cql3.CqlPagingInputFormat;
import org.apache.cassandra.hadoop.cql3.CqlInputFormat;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.conf.Configured;
@@ -156,7 +155,7 @@ public class WordCountCounters extends Configured
implements Tool
else
{
job.setMapperClass(SumMapper.class);
-job.setInputFormatClass(CqlPagingInputFormat.class);
+job.setInputFormatClass(CqlInputFormat.class);
ConfigHelper.setInputRpcPort(job.getConfiguration(), 9160);
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e550ea60/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
b/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
index 53f3900..6c50ab2 100644
--- a/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
+++ b/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
@@ -74,7 +74,7 @@ public class CqlStorage extends AbstractCassandraStorage
{
super();
this.pageSize = pageSize;
-DEFAULT_INPUT_FORMAT =
org.apache.cassandra.hadoop.cql3.CqlPagingInputFormat;
+DEFAULT_INPUT_FORMAT =
org.apache.cassandra.hadoop.cql3.CqlInputFormat;
DEFAULT_OUTPUT_FORMAT =
org.apache.cassandra.hadoop.cql3.CqlOutputFormat;
}
[jira] [Updated] (CASSANDRA-8541) References to non-existent/deprecated CqlPagingInputFormat in code
[ https://issues.apache.org/jira/browse/CASSANDRA-8541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dave Brosius updated CASSANDRA-8541: Assignee: Rekha Joshi References to non-existent/deprecated CqlPagingInputFormat in code -- Key: CASSANDRA-8541 URL: https://issues.apache.org/jira/browse/CASSANDRA-8541 Project: Cassandra Issue Type: Bug Components: Hadoop Reporter: Rekha Joshi Assignee: Rekha Joshi Labels: hadoop Fix For: 2.0.12 Attachments: CASSANDRA-8541.txt On Mac 10.9.5, Java 1.7, latest cassandra trunk - References to non-existent/deprecated CqlPagingInputFormat in code. As per Changes.txt/7570 both CqlPagingInputFormat and CqlPagingRecordReader are removed, but lingering references in WordCount,CqlStorage.. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
cassandra git commit: simplify:
Repository: cassandra
Updated Branches:
refs/heads/trunk 36b287bea - cfee3da90
simplify:
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cfee3da9
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cfee3da9
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cfee3da9
Branch: refs/heads/trunk
Commit: cfee3da908c685dad3103a497dd70eb9a884df3d
Parents: 36b287b
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Sat Dec 27 21:09:26 2014 -0500
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Sat Dec 27 21:09:26 2014 -0500
--
examples/hadoop_cql3_word_count/src/WordCount.java | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/cfee3da9/examples/hadoop_cql3_word_count/src/WordCount.java
--
diff --git a/examples/hadoop_cql3_word_count/src/WordCount.java
b/examples/hadoop_cql3_word_count/src/WordCount.java
index 6a2f846..3702a2b 100644
--- a/examples/hadoop_cql3_word_count/src/WordCount.java
+++ b/examples/hadoop_cql3_word_count/src/WordCount.java
@@ -37,12 +37,10 @@ import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.Reducer;
-import org.apache.hadoop.mapreduce.Mapper.Context;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
import org.apache.hadoop.util.Tool;
import org.apache.hadoop.util.ToolRunner;
import com.datastax.driver.core.Row;
-import java.nio.charset.CharacterCodingException;
/**
* This counts the occurrences of words in ColumnFamily
@@ -122,7 +120,7 @@ public class WordCount extends Configured implements Tool
public void map(Long key, Row row, Context context) throws
IOException, InterruptedException
{
String value = row.getString(line);
-logger.debug(read {}:{}={} from {}, new Object[] {key, line,
value, context.getInputSplit()});
+logger.debug(read {}:{}={} from {}, key, line, value,
context.getInputSplit());
StringTokenizer itr = new StringTokenizer(value);
while (itr.hasMoreTokens())
{
