[jira] [Commented] (CASSANDRA-7216) Restricted superuser account request
[ https://issues.apache.org/jira/browse/CASSANDRA-7216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14002909#comment-14002909 ] Oded Peer commented on CASSANDRA-7216: -- My problem is with granting the permissions. If the user managing other users can grant any permission to any user what prevents the attacker from creating a new user and granting permissions to read existing keyspaces? Restricted superuser account request Key: CASSANDRA-7216 URL: https://issues.apache.org/jira/browse/CASSANDRA-7216 Project: Cassandra Issue Type: Improvement Reporter: Oded Peer Assignee: Dave Brosius Priority: Minor Fix For: 3.0 Attachments: 7216-POC.txt, 7216.txt I am developing a multi-tenant service. Every tenant has its own user, keyspace and can access only his keyspace. As new tenants are provisioned there is a need to create new users and keyspaces. Only a superuser can issue CREATE USER requests, so we must have a super user account in the system. On the other hand super users have access to all the keyspaces, which poses a security risk. For tenant provisioning I would like to have a restricted account which can only create new users, without read access to keyspaces. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7231) Support more concurrent requests per native transport connection
[ https://issues.apache.org/jira/browse/CASSANDRA-7231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14002917#comment-14002917 ] Sylvain Lebresne commented on CASSANDRA-7231: - I've attached a branch with what I think is the best option. I won't claim using a flag is the best thing ever but I also don't buy that doing otherwise is much better and I continue to think that having this optional is a nice to have for driver upgrade. I'm also not sure what's the version-based option is, since again, this has never meant to be *not* be option based. But if we're talking about growing the current stream byte to 2 bytes, then it will make the frame header 9 bytes instead of 8, which imo is ugly. Granted, the option flag is kind of a hack around that, but well, it still has my preference. But since it's arguably a preference, if it doesn't align with the majority, I'm fine with an alternative provided it's not a pain for drivers that do want/need to support multiple versions of the protocol. Don't wait on me to provide the alternative however. Support more concurrent requests per native transport connection Key: CASSANDRA-7231 URL: https://issues.apache.org/jira/browse/CASSANDRA-7231 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Benedict Assignee: Sylvain Lebresne Priority: Minor Fix For: 2.1.0 Attachments: 7231.txt Right now we only support 127 concurrent requests against a given native transport connection. This causes us to waste file handles opening multiple connections, increases driver complexity and dilutes writes across multiple connections so that batching cannot easily be performed. I propose raising this limit substantially, to somewhere in the region of 16-64K, and that this is a good time to do it since we're already bumping the protocol version. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6950) Secondary index query fails with tc range query when ordered by DESC
[
https://issues.apache.org/jira/browse/CASSANDRA-6950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003026#comment-14003026
]
Sylvain Lebresne commented on CASSANDRA-6950:
-
bq. I'm against changing it for Thrift in 2.0.x. But support it for 2.1, so
long as it's accompanied by a noticeable NEWS.txt entry.
At which point, is there really a point bothering? With thrift, we're in don't
break stuff for people mode after all, not change edge case behavior to be
more logical for the years to come. Besides, I'm still bugged by making a
special case for ReversedType for thrift, it does sound very non-thrift-like to
me, and whether or not it is more logical is debatable imo.
In any case, I tend to agree with Aleksey that no matter what we do for 2.1 we
should preserve the behavior in 2.0, so the currently attached patch stands for
2.0 and I'd prefer we first focus on fixing what's definitively a bug for CQL.
Let's maybe open a separate ticket to debate whether or not changing the thrift
behavior is something we want?
Secondary index query fails with tc range query when ordered by DESC
Key: CASSANDRA-6950
URL: https://issues.apache.org/jira/browse/CASSANDRA-6950
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: RHEL 6.3 virtual guest,
apache-cassandra-2.0.6-SNAPSHOT-src.tar.gz from build #284 (also tried with
2.0.5 with CASSANDRA- patch custom-applied with same result).
Reporter: Andre Campeau
Assignee: Sylvain Lebresne
Fix For: 2.0.8
Attachments: 6950-pycassa-repro.py, 6950.txt
create table test4 ( name text, lname text, tc bigint, record text,
PRIMARY KEY ((name, lname), tc)) WITH CLUSTERING ORDER BY (tc DESC) AND
compaction={'class': 'LeveledCompactionStrategy'};
create index test4_index ON test4(lname);
Populate it with some data and non-zero tc values, then try:
select * from test4 where lname='blah' and tc0 allow filtering;
And, (0 rows) returned, even though there are rows which should be found.
When I create the table using CLUSTERING ORDER BY (tc ASC), the above query
works. Rows are correctly returned based on the range check.
Tried various combinations but with descending order on tc nothing works.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-6950) Secondary index query fails with tc range query when ordered by DESC
[
https://issues.apache.org/jira/browse/CASSANDRA-6950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003032#comment-14003032
]
Aleksey Yeschenko commented on CASSANDRA-6950:
--
bq. But support it for 2.1, so long as it's accompanied by a noticeable
NEWS.txt entry.
Read as not against, as in +0.
Secondary index query fails with tc range query when ordered by DESC
Key: CASSANDRA-6950
URL: https://issues.apache.org/jira/browse/CASSANDRA-6950
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: RHEL 6.3 virtual guest,
apache-cassandra-2.0.6-SNAPSHOT-src.tar.gz from build #284 (also tried with
2.0.5 with CASSANDRA- patch custom-applied with same result).
Reporter: Andre Campeau
Assignee: Sylvain Lebresne
Fix For: 2.0.8
Attachments: 6950-pycassa-repro.py, 6950.txt
create table test4 ( name text, lname text, tc bigint, record text,
PRIMARY KEY ((name, lname), tc)) WITH CLUSTERING ORDER BY (tc DESC) AND
compaction={'class': 'LeveledCompactionStrategy'};
create index test4_index ON test4(lname);
Populate it with some data and non-zero tc values, then try:
select * from test4 where lname='blah' and tc0 allow filtering;
And, (0 rows) returned, even though there are rows which should be found.
When I create the table using CLUSTERING ORDER BY (tc ASC), the above query
works. Rows are correctly returned based on the range check.
Tried various combinations but with descending order on tc nothing works.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-3569) Failure detector downs should not break streams
[
https://issues.apache.org/jira/browse/CASSANDRA-3569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003040#comment-14003040
]
Marcus Eriksson commented on CASSANDRA-3569:
It works perfectly when the TCP connection actually comes back, a few problems
when it doesn't;
On the sending side, I get a proper exception after keepalive_time +
keepalive_probe*keepalive_intvl, but the stream session is not cleared,
nodetool netstats still says we are sending files, but in the logs it says
Session ... is complete and Stream failed.
On the receiving side I get nothing (this might very well be me messing up the
firewalling), stream/repair is just stuck.
The firewall rules I use are;
{code}
iptables -A OUTPUT -d othernodeip -j DROP
iptables -A INPUT -s othernodeip -j DROP
{code}
Failure detector downs should not break streams
---
Key: CASSANDRA-3569
URL: https://issues.apache.org/jira/browse/CASSANDRA-3569
Project: Cassandra
Issue Type: New Feature
Reporter: Peter Schuller
Assignee: Joshua McKenzie
Fix For: 2.1.1
Attachments: 3569-2.0.txt, 3569_v1.txt
CASSANDRA-2433 introduced this behavior just to get repairs to don't sit
there waiting forever. In my opinion the correct fix to that problem is to
use TCP keep alive. Unfortunately the TCP keep alive period is insanely high
by default on a modern Linux, so just doing that is not entirely good either.
But using the failure detector seems non-sensicle to me. We have a
communication method which is the TCP transport, that we know is used for
long-running processes that you don't want to incorrectly be killed for no
good reason, and we are using a failure detector tuned to detecting when not
to send real-time sensitive request to nodes in order to actively kill a
working connection.
So, rather than add complexity with protocol based ping/pongs and such, I
propose that we simply just use TCP keep alive for streaming connections and
instruct operators of production clusters to tweak
net.ipv4.tcp_keepalive_{probes,intvl} as appropriate (or whatever equivalent
on their OS).
I can submit the patch. Awaiting opinions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (CASSANDRA-7269) Make CqlInputFormat and CqlRecordReader consistent with comments
Jeremy Hanna created CASSANDRA-7269:
---
Summary: Make CqlInputFormat and CqlRecordReader consistent with
comments
Key: CASSANDRA-7269
URL: https://issues.apache.org/jira/browse/CASSANDRA-7269
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Reporter: Jeremy Hanna
Priority: Minor
Both the CqlInputFormat and CqlPagingInputFormat have the following comment:
{code}
/**
...
* the number of CQL rows per page
* CQLConfigHelper.setInputCQLPageRowSize. The default page row size is 1000.
You
* should set it to as big as possible, but no bigger. It set the LIMIT for
the CQL
* query, so you need set it big enough to minimize the network overhead, and
also
* not too big to avoid out of memory issue.
...
**/
{code}
The property is used in both classes, but the default is only set to 1000 in
CqlPagingRecordReader explicitly.
We should either make the default part of the CqlConfigHelper so it's set in
both places or update the comments in the CqlInputFormat to say that if it's
not set, it will default to the java driver fetch size which is 5000.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-7270) nodetool netstats broken
[ https://issues.apache.org/jira/browse/CASSANDRA-7270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-7270: --- Attachment: 0001-fix-session-index.patch nodetool netstats broken Key: CASSANDRA-7270 URL: https://issues.apache.org/jira/browse/CASSANDRA-7270 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Priority: Minor Fix For: 2.1 rc1 Attachments: 0001-fix-session-index.patch nodetool netstats throws exception when we have ongoing streams -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (CASSANDRA-7270) nodetool netstats broken
Marcus Eriksson created CASSANDRA-7270: -- Summary: nodetool netstats broken Key: CASSANDRA-7270 URL: https://issues.apache.org/jira/browse/CASSANDRA-7270 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Priority: Minor Fix For: 2.1 rc1 Attachments: 0001-fix-session-index.patch nodetool netstats throws exception when we have ongoing streams -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7267) Embedded sets in user defined data-types are not updating
[
https://issues.apache.org/jira/browse/CASSANDRA-7267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Zimmer updated CASSANDRA-7267:
-
Fix Version/s: (was: 2.1 beta2)
2.1 rc1
Embedded sets in user defined data-types are not updating
-
Key: CASSANDRA-7267
URL: https://issues.apache.org/jira/browse/CASSANDRA-7267
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Thomas Zimmer
Fix For: 2.1 rc1
Hi,
i just played around with Cassandra 2.1.0 beta2 and i might have found an
issue with embedded Sets in User Defined Data Types.
Here is how i can reproduce it:
1.) Create a keyspace test
2.) Create a table like this:
create table songs (title varchar PRIMARY KEY, band varchar, tags
Setvarchar);
3.) Create a udt like this:
create type band_info_type (founded timestamp, members Setvarchar,
description text);
4.) Try to insert data:
insert into songs (title, band, band_info, tags) values ('The trooper', 'Iron
Maiden', {founded:188694000, members: {'Bruce Dickinson', 'Dave Murray',
'Adrian Smith', 'Janick Gers', 'Steve Harris', 'Nicko McBrain'}, description:
'Pure evil metal'}, {'metal', 'england'});
5.) Select the data:
select * from songs;
Returns this:
The trooper | Iron Maiden | {founded: '1970-01-03 05:24:54+0100', members:
{}, description: 'Pure evil metal'} | {'england', 'metal'}
The embedded data-set seems to empty. I also tried updating a row which also
does not seem to work.
Regards,
Thomas
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-6563) TTL histogram compactions not triggered at high Estimated droppable tombstones rate
[
https://issues.apache.org/jira/browse/CASSANDRA-6563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003134#comment-14003134
]
Marcus Eriksson commented on CASSANDRA-6563:
[~pauloricardomg] I think making it configurable is the way to go, calling the
config option something like aggressive_tombstone_compactions or something,
with default off.
TTL histogram compactions not triggered at high Estimated droppable
tombstones rate
-
Key: CASSANDRA-6563
URL: https://issues.apache.org/jira/browse/CASSANDRA-6563
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: 1.2.12ish
Reporter: Chris Burroughs
Assignee: Paulo Ricardo Motta Gomes
Fix For: 1.2.17, 2.0.8
Attachments: 1.2.16-CASSANDRA-6563-v2.txt, 1.2.16-CASSANDRA-6563.txt,
2.0.7-CASSANDRA-6563.txt, patch-v1-iostat.png, patch-v1-range1.png,
patch-v2-range3.png, patched-droppadble-ratio.png, patched-storage-load.png,
patched1-compacted-bytes.png, patched2-compacted-bytes.png,
unpatched-droppable-ratio.png, unpatched-storage-load.png,
unpatched1-compacted-bytes.png, unpatched2-compacted-bytes.png
I have several column families in a largish cluster where virtually all
columns are written with a (usually the same) TTL. My understanding of
CASSANDRA-3442 is that sstables that have a high ( 20%) estimated
percentage of droppable tombstones should be individually compacted. This
does not appear to be occurring with size tired compaction.
Example from one node:
{noformat}
$ ll /data/sstables/data/ks/Cf/*Data.db
-rw-rw-r-- 31 cassandra cassandra 26651211757 Nov 26 22:59
/data/sstables/data/ks/Cf/ks-Cf-ic-295562-Data.db
-rw-rw-r-- 31 cassandra cassandra 6272641818 Nov 27 02:51
/data/sstables/data/ks/Cf/ks-Cf-ic-296121-Data.db
-rw-rw-r-- 31 cassandra cassandra 1814691996 Dec 4 21:50
/data/sstables/data/ks/Cf/ks-Cf-ic-320449-Data.db
-rw-rw-r-- 30 cassandra cassandra 10909061157 Dec 11 17:31
/data/sstables/data/ks/Cf/ks-Cf-ic-340318-Data.db
-rw-rw-r-- 29 cassandra cassandra 459508942 Dec 12 10:37
/data/sstables/data/ks/Cf/ks-Cf-ic-342259-Data.db
-rw-rw-r-- 1 cassandra cassandra 336908 Dec 12 12:03
/data/sstables/data/ks/Cf/ks-Cf-ic-342307-Data.db
-rw-rw-r-- 1 cassandra cassandra 2063935 Dec 12 12:03
/data/sstables/data/ks/Cf/ks-Cf-ic-342309-Data.db
-rw-rw-r-- 1 cassandra cassandra 409 Dec 12 12:03
/data/sstables/data/ks/Cf/ks-Cf-ic-342314-Data.db
-rw-rw-r-- 1 cassandra cassandra31180007 Dec 12 12:03
/data/sstables/data/ks/Cf/ks-Cf-ic-342319-Data.db
-rw-rw-r-- 1 cassandra cassandra 2398345 Dec 12 12:03
/data/sstables/data/ks/Cf/ks-Cf-ic-342322-Data.db
-rw-rw-r-- 1 cassandra cassandra 21095 Dec 12 12:03
/data/sstables/data/ks/Cf/ks-Cf-ic-342331-Data.db
-rw-rw-r-- 1 cassandra cassandra 81454 Dec 12 12:03
/data/sstables/data/ks/Cf/ks-Cf-ic-342335-Data.db
-rw-rw-r-- 1 cassandra cassandra 1063718 Dec 12 12:03
/data/sstables/data/ks/Cf/ks-Cf-ic-342339-Data.db
-rw-rw-r-- 1 cassandra cassandra 127004 Dec 12 12:03
/data/sstables/data/ks/Cf/ks-Cf-ic-342344-Data.db
-rw-rw-r-- 1 cassandra cassandra 146785 Dec 12 12:03
/data/sstables/data/ks/Cf/ks-Cf-ic-342346-Data.db
-rw-rw-r-- 1 cassandra cassandra 697338 Dec 12 12:03
/data/sstables/data/ks/Cf/ks-Cf-ic-342351-Data.db
-rw-rw-r-- 1 cassandra cassandra 3921428 Dec 12 12:03
/data/sstables/data/ks/Cf/ks-Cf-ic-342367-Data.db
-rw-rw-r-- 1 cassandra cassandra 240332 Dec 12 12:03
/data/sstables/data/ks/Cf/ks-Cf-ic-342370-Data.db
-rw-rw-r-- 1 cassandra cassandra 45669 Dec 12 12:03
/data/sstables/data/ks/Cf/ks-Cf-ic-342374-Data.db
-rw-rw-r-- 1 cassandra cassandra53127549 Dec 12 12:03
/data/sstables/data/ks/Cf/ks-Cf-ic-342375-Data.db
-rw-rw-r-- 16 cassandra cassandra 12466853166 Dec 25 22:40
/data/sstables/data/ks/Cf/ks-Cf-ic-396473-Data.db
-rw-rw-r-- 12 cassandra cassandra 3903237198 Dec 29 19:42
/data/sstables/data/ks/Cf/ks-Cf-ic-408926-Data.db
-rw-rw-r-- 7 cassandra cassandra 3692260987 Jan 3 08:25
/data/sstables/data/ks/Cf/ks-Cf-ic-427733-Data.db
-rw-rw-r-- 4 cassandra cassandra 3971403602 Jan 6 20:50
/data/sstables/data/ks/Cf/ks-Cf-ic-437537-Data.db
-rw-rw-r-- 3 cassandra cassandra 1007832224 Jan 7 15:19
/data/sstables/data/ks/Cf/ks-Cf-ic-440331-Data.db
-rw-rw-r-- 2 cassandra cassandra 896132537 Jan 8 11:05
/data/sstables/data/ks/Cf/ks-Cf-ic-447740-Data.db
-rw-rw-r-- 1 cassandra cassandra 963039096 Jan 9 04:59
/data/sstables/data/ks/Cf/ks-Cf-ic-449425-Data.db
-rw-rw-r-- 1 cassandra cassandra 232168351 Jan 9 10:14
/data/sstables/data/ks/Cf/ks-Cf-ic-450287-Data.db
-rw-rw-r-- 1 cassandra cassandra
[jira] [Commented] (CASSANDRA-6851) Improve anticompaction after incremental repair
[ https://issues.apache.org/jira/browse/CASSANDRA-6851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003149#comment-14003149 ] Marcus Eriksson commented on CASSANDRA-6851: The idea is that we can do anticompaction at the same time over several SSTables, if a key X is in all included files, it has been repaired in all those files, and can go into the same repaired output file (re-reading my description I see that it was not very clear, sorry about that) I think what is needed is to pass in more sstables when creating the compaction scanners instead of doing Arrays.asList(sstable); Perhaps a first step could be to always give it two sstables so that we atleast don't increase the number of sstables by doing anticompaction? I guess we could experiment a bit here and find a heuristic that is good for most cases. Improve anticompaction after incremental repair --- Key: CASSANDRA-6851 URL: https://issues.apache.org/jira/browse/CASSANDRA-6851 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Marcus Eriksson Assignee: Russell Alexander Spitzer Priority: Minor Labels: compaction, lhf Fix For: 2.1.1 After an incremental repair we iterate over all sstables and split them in two parts, one containing the repaired data and one the unrepaired. We could in theory double the number of sstables on a node. To avoid this we could make anticompaction also do a compaction, for example, if we are to anticompact 10 sstables, we could anticompact those to 2. Note that we need to avoid creating too big sstables though, if we anticompact all sstables on a node it would essentially be a major compaction. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (CASSANDRA-7271) Bulk data loading in Cassandra causing OOM
Prasanth Gullapalli created CASSANDRA-7271:
--
Summary: Bulk data loading in Cassandra causing OOM
Key: CASSANDRA-7271
URL: https://issues.apache.org/jira/browse/CASSANDRA-7271
Project: Cassandra
Issue Type: Bug
Reporter: Prasanth Gullapalli
I am trying to load data from a csv file into Cassandra table using
SSTableSimpleUnsortedWriter. As the latest maven cassandra dependencies have
some issues with it, I have taken the _next_ beta (rc) version cut as suggested
in CASSANDRA-7218. But after taking it, I am facing issues with bulk data
loading
Here is the piece of code which loads data:
public void loadData(TableDefinition tableDefinition, InputStream
csvInputStream){
createDataInDBFormat(tableDefinition, csvInputStream);
Path dbFilePath = Paths.get(TEMP_DIR, keyspace, tableDefinition.getName());
//BulkLoader.main(new
String[]{-d,localhost,dbFilePath.toUri().getPath()});
try {
JMXServiceURL jmxUrl = new JMXServiceURL(String.format(
service:jmx:rmi:///jndi/rmi://%s:%d/jmxrmi, cassandraHost,
cassandraJMXPort));
JMXConnector connector = JMXConnectorFactory.connect(jmxUrl, new
HashMapString, Object());
MBeanServerConnection mbeanServerConn =
connector.getMBeanServerConnection();
ObjectName name = new
ObjectName(org.apache.cassandra.db:type=StorageService);
StorageServiceMBean storageBean = JMX.newMBeanProxy(mbeanServerConn,
name, StorageServiceMBean.class);
storageBean.bulkLoad(dbFilePath.toUri().getPath());
connector.close();
} catch (IOException | MalformedObjectNameException e) {
e.printStacktrace()
}
FileUtils.deleteQuietly(dbFilePath.toFile());
}
private void createDataInDBFormat(TableDefinition tableDefinition, InputStream
csvInputStream) {
try(Reader reader = new InputStreamReader(csvInputStream)){
String tableName = tableDefinition.getName();
File directory = Paths.get(TEMP_DIR, keyspace, tableName).toFile();
directory.mkdirs();
String yamlPath =
file:\\+CASSANDRA_HOME+File.separator+conf+File.separator+cassandra.yaml;
System.setProperty(cassandra.config, yamlPath);
SSTableSimpleUnsortedWriter writer = new SSTableSimpleUnsortedWriter(
directory, new Murmur3Partitioner(), keyspace,
tableName, AsciiType.instance, null, 10);
long timestamp = System.currentTimeMillis() * 1000;
CSVReader csvReader = new CSVReader(reader);
String[] colValues = null;
ListColumnDefinition columnDefinitions =
tableDefinition.getColumnDefinitions();
while((colValues = csvReader.readNext()) != null){
if(colValues.length != 0){
writer.newRow(bytes(colValues[0]));
for(int index = 1; index colValues.length; index++){
ColumnDefinition columnDefinition =
columnDefinitions.get(index);
writer.addColumn(bytes(columnDefinition.getName()),
bytes(colValues[index]), timestamp);
}
}
}
csvReader.close();
writer.close();
} catch (IOException e) {
e.printStacktrace();
}
}
On trying to run loadData, it is giving me the following exception:
11:23:18.035 [45742123@qtp-1703018180-0] ERROR
com.adaequare.common.config.TransactionPerRequestFilter.doInTransactionWithoutResult
39 - Problem in executing request :
[http://localhost:8081/mapro-engine/rest/masterdata/pumpData]. Cause
:org.glassfish.jersey.server.ContainerException: java.lang.OutOfMemoryError:
Java heap space
javax.servlet.ServletException: org.glassfish.jersey.server.ContainerException:
java.lang.OutOfMemoryError: Java heap space
at
org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:392)
~[jersey-container-servlet-core-2.6.jar:na]
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:381)
~[jersey-container-servlet-core-2.6.jar:na]
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:344)
~[jersey-container-servlet-core-2.6.jar:na]
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:219)
~[jersey-container-servlet-core-2.6.jar:na]
at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
~[jetty-6.1.25.jar:6.1.25]
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
[jetty-6.1.25.jar:6.1.25]
at
com.adaequare.common.config.TransactionPerRequestFilter$1.doInTransactionWithoutResult(TransactionPerRequestFilter.java:37)
~[mapro-commons-1.0.jar:na]
at
org.springframework.transaction.support.TransactionCallbackWithoutResult.doInTransaction(TransactionCallbackWithoutResult.java:34)
[spring-tx-4.0.3.RELEASE.jar:4.0.3.RELEASE]
[jira] [Updated] (CASSANDRA-7271) Bulk data loading in Cassandra causing OOM
[
https://issues.apache.org/jira/browse/CASSANDRA-7271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Gullapalli updated CASSANDRA-7271:
---
Description:
I am trying to load data from a csv file into Cassandra table using
SSTableSimpleUnsortedWriter. As the latest maven cassandra dependencies have
some issues with it, I have taken the _next_ beta (rc) version cut as suggested
in CASSANDRA-7218. But after taking it, I am facing issues with bulk data
loading
Here is the piece of code which loads data:
{code:java}
public void loadData(TableDefinition tableDefinition, InputStream
csvInputStream){
createDataInDBFormat(tableDefinition, csvInputStream);
Path dbFilePath = Paths.get(TEMP_DIR, keyspace, tableDefinition.getName());
//BulkLoader.main(new
String[]{-d,localhost,dbFilePath.toUri().getPath()});
try {
JMXServiceURL jmxUrl = new JMXServiceURL(String.format(
service:jmx:rmi:///jndi/rmi://%s:%d/jmxrmi, cassandraHost,
cassandraJMXPort));
JMXConnector connector = JMXConnectorFactory.connect(jmxUrl, new
HashMapString, Object());
MBeanServerConnection mbeanServerConn =
connector.getMBeanServerConnection();
ObjectName name = new
ObjectName(org.apache.cassandra.db:type=StorageService);
StorageServiceMBean storageBean = JMX.newMBeanProxy(mbeanServerConn,
name, StorageServiceMBean.class);
storageBean.bulkLoad(dbFilePath.toUri().getPath());
connector.close();
} catch (IOException | MalformedObjectNameException e) {
e.printStacktrace()
}
FileUtils.deleteQuietly(dbFilePath.toFile());
}
private void createDataInDBFormat(TableDefinition tableDefinition, InputStream
csvInputStream) {
try(Reader reader = new InputStreamReader(csvInputStream)){
String tableName = tableDefinition.getName();
File directory = Paths.get(TEMP_DIR, keyspace, tableName).toFile();
directory.mkdirs();
String yamlPath =
file:\\+CASSANDRA_HOME+File.separator+conf+File.separator+cassandra.yaml;
System.setProperty(cassandra.config, yamlPath);
SSTableSimpleUnsortedWriter writer = new SSTableSimpleUnsortedWriter(
directory, new Murmur3Partitioner(), keyspace,
tableName, AsciiType.instance, null, 10);
long timestamp = System.currentTimeMillis() * 1000;
CSVReader csvReader = new CSVReader(reader);
String[] colValues = null;
ListColumnDefinition columnDefinitions =
tableDefinition.getColumnDefinitions();
while((colValues = csvReader.readNext()) != null){
if(colValues.length != 0){
writer.newRow(bytes(colValues[0]));
for(int index = 1; index colValues.length; index++){
ColumnDefinition columnDefinition =
columnDefinitions.get(index);
writer.addColumn(bytes(columnDefinition.getName()),
bytes(colValues[index]), timestamp);
}
}
}
csvReader.close();
writer.close();
} catch (IOException e) {
e.printStacktrace();
}
}
{code}
On trying to run loadData, it is giving me the following exception:
11:23:18.035 [45742123@qtp-1703018180-0] ERROR
com.adaequare.common.config.TransactionPerRequestFilter.doInTransactionWithoutResult
39 - Problem in executing request :
[http://localhost:8081/mapro-engine/rest/masterdata/pumpData]. Cause
:org.glassfish.jersey.server.ContainerException: java.lang.OutOfMemoryError:
Java heap space
javax.servlet.ServletException: org.glassfish.jersey.server.ContainerException:
java.lang.OutOfMemoryError: Java heap space
at
org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:392)
~[jersey-container-servlet-core-2.6.jar:na]
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:381)
~[jersey-container-servlet-core-2.6.jar:na]
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:344)
~[jersey-container-servlet-core-2.6.jar:na]
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:219)
~[jersey-container-servlet-core-2.6.jar:na]
at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
~[jetty-6.1.25.jar:6.1.25]
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
[jetty-6.1.25.jar:6.1.25]
at
com.adaequare.common.config.TransactionPerRequestFilter$1.doInTransactionWithoutResult(TransactionPerRequestFilter.java:37)
~[mapro-commons-1.0.jar:na]
at
org.springframework.transaction.support.TransactionCallbackWithoutResult.doInTransaction(TransactionCallbackWithoutResult.java:34)
[spring-tx-4.0.3.RELEASE.jar:4.0.3.RELEASE]
:4.0.3.RELEASE]
at
[jira] [Updated] (CASSANDRA-7271) Bulk data loading in Cassandra causing OOM
[
https://issues.apache.org/jira/browse/CASSANDRA-7271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Gullapalli updated CASSANDRA-7271:
---
Description:
I am trying to load data from a csv file into Cassandra table using
SSTableSimpleUnsortedWriter. As the latest maven cassandra dependencies have
some issues with it, I have taken the _next_ beta (rc) version cut as suggested
in CASSANDRA-7218. But after taking it, I am facing issues with bulk data
loading
Here is the piece of code which loads data:
{code:java}
public void loadData(TableDefinition tableDefinition, InputStream
csvInputStream){
createDataInDBFormat(tableDefinition, csvInputStream);
Path dbFilePath = Paths.get(TEMP_DIR, keyspace, tableDefinition.getName());
//BulkLoader.main(new
String[]{-d,localhost,dbFilePath.toUri().getPath()});
try {
JMXServiceURL jmxUrl = new JMXServiceURL(String.format(
service:jmx:rmi:///jndi/rmi://%s:%d/jmxrmi, cassandraHost,
cassandraJMXPort));
JMXConnector connector = JMXConnectorFactory.connect(jmxUrl, new
HashMapString, Object());
MBeanServerConnection mbeanServerConn =
connector.getMBeanServerConnection();
ObjectName name = new
ObjectName(org.apache.cassandra.db:type=StorageService);
StorageServiceMBean storageBean = JMX.newMBeanProxy(mbeanServerConn,
name, StorageServiceMBean.class);
storageBean.bulkLoad(dbFilePath.toUri().getPath());
connector.close();
} catch (IOException | MalformedObjectNameException e) {
e.printStacktrace()
}
FileUtils.deleteQuietly(dbFilePath.toFile());
}
private void createDataInDBFormat(TableDefinition tableDefinition, InputStream
csvInputStream) {
try(Reader reader = new InputStreamReader(csvInputStream)){
String tableName = tableDefinition.getName();
File directory = Paths.get(TEMP_DIR, keyspace, tableName).toFile();
directory.mkdirs();
String yamlPath =
file:\\+CASSANDRA_HOME+File.separator+conf+File.separator+cassandra.yaml;
System.setProperty(cassandra.config, yamlPath);
SSTableSimpleUnsortedWriter writer = new SSTableSimpleUnsortedWriter(
directory, new Murmur3Partitioner(), keyspace,
tableName, AsciiType.instance, null, 10);
long timestamp = System.currentTimeMillis() * 1000;
CSVReader csvReader = new CSVReader(reader);
String[] colValues = null;
ListColumnDefinition columnDefinitions =
tableDefinition.getColumnDefinitions();
while((colValues = csvReader.readNext()) != null){
if(colValues.length != 0){
writer.newRow(bytes(colValues[0]));
for(int index = 1; index colValues.length; index++){
ColumnDefinition columnDefinition =
columnDefinitions.get(index);
writer.addColumn(bytes(columnDefinition.getName()),
bytes(colValues[index]), timestamp);
}
}
}
csvReader.close();
writer.close();
} catch (IOException e) {
e.printStacktrace();
}
}
{code}
On trying to run loadData, it is giving me the following exception:
{code:xml}
11:23:18.035 [45742123@qtp-1703018180-0] ERROR
com.adaequare.common.config.TransactionPerRequestFilter.doInTransactionWithoutResult
39 - Problem in executing request :
[http://localhost:8081/mapro-engine/rest/masterdata/pumpData]. Cause
:org.glassfish.jersey.server.ContainerException: java.lang.OutOfMemoryError:
Java heap space
javax.servlet.ServletException: org.glassfish.jersey.server.ContainerException:
java.lang.OutOfMemoryError: Java heap space
at
org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:392)
~[jersey-container-servlet-core-2.6.jar:na]
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:381)
~[jersey-container-servlet-core-2.6.jar:na]
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:344)
~[jersey-container-servlet-core-2.6.jar:na]
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:219)
~[jersey-container-servlet-core-2.6.jar:na]
at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
~[jetty-6.1.25.jar:6.1.25]
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
[jetty-6.1.25.jar:6.1.25]
at
com.adaequare.common.config.TransactionPerRequestFilter$1.doInTransactionWithoutResult(TransactionPerRequestFilter.java:37)
~[mapro-commons-1.0.jar:na]
at
org.springframework.transaction.support.TransactionCallbackWithoutResult.doInTransaction(TransactionCallbackWithoutResult.java:34)
[spring-tx-4.0.3.RELEASE.jar:4.0.3.RELEASE]
:4.0.3.RELEASE]
at
[jira] [Created] (CASSANDRA-7272) Add Major Compaction to LCS
T Jake Luciani created CASSANDRA-7272: - Summary: Add Major Compaction to LCS Key: CASSANDRA-7272 URL: https://issues.apache.org/jira/browse/CASSANDRA-7272 Project: Cassandra Issue Type: Improvement Components: Core Reporter: T Jake Luciani Priority: Minor Fix For: 3.0 LCS has a number of minor issues (maybe major depending on your perspective). LCS is primarily used for wide rows so for instance when you repair data in LCS you end up with a copy of an entire repaired row in L0. Over time if you repair you end up with multiple copies of a row in L0 - L5. This can make predicting disk usage confusing. Another issue is cleaning up tombstoned data. If a tombstone lives in level 1 and data for the cell lives in level 5 the data will not be reclaimed from disk until the tombstone reaches level 5. I propose we add a major compaction for LCS that forces consolidation of data to level 5 to address these. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-4718) More-efficient ExecutorService for improved throughput
[ https://issues.apache.org/jira/browse/CASSANDRA-4718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003213#comment-14003213 ] Jonathan Ellis commented on CASSANDRA-4718: --- This looks exactly like I would have predicted: the more disk-bound the workload is, the less the executorservice matters. But when our hot dataset is cacheable (10M and to a lesser degree 100M) -sep is a clear win. This is the scenario that we ought to be optimizing for. More-efficient ExecutorService for improved throughput -- Key: CASSANDRA-4718 URL: https://issues.apache.org/jira/browse/CASSANDRA-4718 Project: Cassandra Issue Type: Improvement Reporter: Jonathan Ellis Assignee: Benedict Priority: Minor Labels: performance Fix For: 2.1.0 Attachments: 4718-v1.patch, E100M_summary_key_s.svg, E10M_summary_key_s.svg, E600M_summary_key_s.svg, PerThreadQueue.java, austin_diskbound_read.svg, aws.svg, aws_read.svg, backpressure-stress.out.txt, baq vs trunk.png, belliotsmith_branches-stress.out.txt, jason_read.svg, jason_read_latency.svg, jason_run1.svg, jason_run2.svg, jason_run3.svg, jason_write.svg, op costs of various queues.ods, stress op rate with various queues.ods, stress_2014May15.txt, stress_2014May16.txt, v1-stress.out Currently all our execution stages dequeue tasks one at a time. This can result in contention between producers and consumers (although we do our best to minimize this by using LinkedBlockingQueue). One approach to mitigating this would be to make consumer threads do more work in bulk instead of just one task per dequeue. (Producer threads tend to be single-task oriented by nature, so I don't see an equivalent opportunity there.) BlockingQueue has a drainTo(collection, int) method that would be perfect for this. However, no ExecutorService in the jdk supports using drainTo, nor could I google one. What I would like to do here is create just such a beast and wire it into (at least) the write and read stages. (Other possible candidates for such an optimization, such as the CommitLog and OutboundTCPConnection, are not ExecutorService-based and will need to be one-offs.) AbstractExecutorService may be useful. The implementations of ICommitLogExecutorService may also be useful. (Despite the name these are not actual ExecutorServices, although they share the most important properties of one.) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7272) Add Major Compaction to LCS
[ https://issues.apache.org/jira/browse/CASSANDRA-7272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-7272: -- Labels: compaction (was: ) Add Major Compaction to LCS -- Key: CASSANDRA-7272 URL: https://issues.apache.org/jira/browse/CASSANDRA-7272 Project: Cassandra Issue Type: Improvement Components: Core Reporter: T Jake Luciani Priority: Minor Labels: compaction Fix For: 3.0 LCS has a number of minor issues (maybe major depending on your perspective). LCS is primarily used for wide rows so for instance when you repair data in LCS you end up with a copy of an entire repaired row in L0. Over time if you repair you end up with multiple copies of a row in L0 - L5. This can make predicting disk usage confusing. Another issue is cleaning up tombstoned data. If a tombstone lives in level 1 and data for the cell lives in level 5 the data will not be reclaimed from disk until the tombstone reaches level 5. I propose we add a major compaction for LCS that forces consolidation of data to level 5 to address these. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7272) Add Major Compaction to LCS
[ https://issues.apache.org/jira/browse/CASSANDRA-7272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003225#comment-14003225 ] T Jake Luciani edited comment on CASSANDRA-7272 at 5/20/14 1:32 PM: This may be accomplished also with CASSANDRA-6851 and CASSANDRA-7019 was (Author: tjake): This may be accomplished also with CASSANDRA-6851 and CASSANDRA-7109 Add Major Compaction to LCS -- Key: CASSANDRA-7272 URL: https://issues.apache.org/jira/browse/CASSANDRA-7272 Project: Cassandra Issue Type: Improvement Components: Core Reporter: T Jake Luciani Priority: Minor Labels: compaction Fix For: 3.0 LCS has a number of minor issues (maybe major depending on your perspective). LCS is primarily used for wide rows so for instance when you repair data in LCS you end up with a copy of an entire repaired row in L0. Over time if you repair you end up with multiple copies of a row in L0 - L5. This can make predicting disk usage confusing. Another issue is cleaning up tombstoned data. If a tombstone lives in level 1 and data for the cell lives in level 5 the data will not be reclaimed from disk until the tombstone reaches level 5. I propose we add a major compaction for LCS that forces consolidation of data to level 5 to address these. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7272) Add Major Compaction to LCS
[ https://issues.apache.org/jira/browse/CASSANDRA-7272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003225#comment-14003225 ] T Jake Luciani commented on CASSANDRA-7272: --- This may be accomplished also with CASSANDRA-6851 and CASSANDRA-7109 Add Major Compaction to LCS -- Key: CASSANDRA-7272 URL: https://issues.apache.org/jira/browse/CASSANDRA-7272 Project: Cassandra Issue Type: Improvement Components: Core Reporter: T Jake Luciani Priority: Minor Fix For: 3.0 LCS has a number of minor issues (maybe major depending on your perspective). LCS is primarily used for wide rows so for instance when you repair data in LCS you end up with a copy of an entire repaired row in L0. Over time if you repair you end up with multiple copies of a row in L0 - L5. This can make predicting disk usage confusing. Another issue is cleaning up tombstoned data. If a tombstone lives in level 1 and data for the cell lives in level 5 the data will not be reclaimed from disk until the tombstone reaches level 5. I propose we add a major compaction for LCS that forces consolidation of data to level 5 to address these. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7262) During streaming: java.lang.AssertionError: Reference counter -1
[
https://issues.apache.org/jira/browse/CASSANDRA-7262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003401#comment-14003401
]
Joshua McKenzie commented on CASSANDRA-7262:
closeSession is called from multiple places that aren't synchronized (while one
is). Another approach to preventing this would be to change all possible entry
points to being synchronized (convict, onError) but that's a higher level of
granularity to take care of the potential race between various completion /
error possibilities.
Having said that, it looks like there's other issues in play here as the
exception posted in the ticket is only the 1st and only occurs once. From the
log attached:
{code:title=other exceptions}
ERROR [CompactionExecutor:382] 2014-05-17 01:17:53,157 CassandraDaemon.java
(line 198) Exception in thread Thread[CompactionExecutor:382,1,main]
java.lang.AssertionError: Reference counter -1 for
/mnt/ssd1/cassandra/data/ldn_production/historical_accounts/ldn_production-historical_accounts-jb-83888-Data.db
at
org.apache.cassandra.io.sstable.SSTableReader.releaseReference(SSTableReader.java:1107)
at
org.apache.cassandra.io.sstable.SSTableReader.releaseReferences(SSTableReader.java:1429)
at
org.apache.cassandra.db.compaction.CompactionController.close(CompactionController.java:207)
at
org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:220)
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:197)
...
{code}
While the reference issue on closeSession() chain could lead us into a
situation where other releases fail to call tidy() etc, given that the
CompactionTask operation leads to the 1st violation in reference count would
imply to me that we have 2 separate issues on this ticket. I haven't
reproduced the compaction reference error nor have I looked deeply into that
yet.
During streaming: java.lang.AssertionError: Reference counter -1
Key: CASSANDRA-7262
URL: https://issues.apache.org/jira/browse/CASSANDRA-7262
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Cassandra 2.07, x86-64 Ubuntu 12.04.4, Oracle java
1.7.0_45
Reporter: Duncan Sands
Assignee: Joshua McKenzie
Priority: Minor
Fix For: 2.0.8, 2.1 rc1
Attachments: 7262_v1.txt, system.log.gz
Got this assertion failure this weekend during repair:
ERROR [STREAM-IN-/192.168.21.14] 2014-05-17 01:17:52,332 StreamSession.java
(line 420) [Stream #3a3ac8a2-dd50-11e3-b3c1-6bf6dccd6457] Streaming error
occurred
java.lang.RuntimeException: Outgoing stream handler has been closed
at
org.apache.cassandra.streaming.ConnectionHandler.sendMessage(ConnectionHandler.java:170)
at
org.apache.cassandra.streaming.StreamSession.receive(StreamSession.java:483)
at
org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:372)
at
org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:289)
at java.lang.Thread.run(Thread.java:744)
ERROR [STREAM-IN-/192.168.21.14] 2014-05-17 01:17:52,350 CassandraDaemon.java
(line 198) Exception in thread Thread[STREAM-IN-/192.168.21.14,5,RMI Runtime]
java.lang.AssertionError: Reference counter -1 for
/mnt/ssd1/cassandra/data/ldn_production/historical_accounts/ldn_production-historical_accounts-jb-79827-Data.db
at
org.apache.cassandra.io.sstable.SSTableReader.releaseReference(SSTableReader.java:1107)
at
org.apache.cassandra.streaming.StreamTransferTask.abort(StreamTransferTask.java:80)
at
org.apache.cassandra.streaming.StreamSession.closeSession(StreamSession.java:322)
at
org.apache.cassandra.streaming.StreamSession.onError(StreamSession.java:425)
at
org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:300)
at java.lang.Thread.run(Thread.java:744)
followed by a few more (the reference counter got down to -3). Got the same
kind of assertion failure on one other node (in a different data centre;
there are 21 nodes altogether distributed over 4 data centres).
I've attached the relevant part of the log. It starts quite a bit before the
assertion failure at the first exception on this node (Cannot proceed on
repair
[jira] [Updated] (CASSANDRA-7262) During streaming: java.lang.AssertionError: Reference counter -1
[ https://issues.apache.org/jira/browse/CASSANDRA-7262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-7262: --- Description: Got this assertion failure this weekend during repair: ERROR [STREAM-IN-/192.168.21.14] 2014-05-17 01:17:52,332 StreamSession.java (line 420) [Stream #3a3ac8a2-dd50-11e3-b3c1-6bf6dccd6457] Streaming error occurred java.lang.RuntimeException: Outgoing stream handler has been closed at org.apache.cassandra.streaming.ConnectionHandler.sendMessage(ConnectionHandler.java:170) at org.apache.cassandra.streaming.StreamSession.receive(StreamSession.java:483) at org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:372) at org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:289) at java.lang.Thread.run(Thread.java:744) ERROR [STREAM-IN-/192.168.21.14] 2014-05-17 01:17:52,350 CassandraDaemon.java (line 198) Exception in thread Thread[STREAM-IN-/192.168.21.14,5,RMI Runtime] java.lang.AssertionError: Reference counter -1 for /mnt/ssd1/cassandra/data/ldn_production/historical_accounts/ldn_production-historical_accounts-jb-79827-Data.db at org.apache.cassandra.io.sstable.SSTableReader.releaseReference(SSTableReader.java:1107) at org.apache.cassandra.streaming.StreamTransferTask.abort(StreamTransferTask.java:80) at org.apache.cassandra.streaming.StreamSession.closeSession(StreamSession.java:322) at org.apache.cassandra.streaming.StreamSession.onError(StreamSession.java:425) at org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:300) at java.lang.Thread.run(Thread.java:744) followed by a few more (the reference counter got down to -3). Got the same kind of assertion failure on one other node (in a different data centre; there are 21 nodes altogether distributed over 4 data centres). I've attached the relevant part of the log. It starts quite a bit before the assertion failure at the first exception on this node (Cannot proceed on repair because a neighbor ... is dead), and finishes a few hours afterwards when the node was restarted. Edit: The following Reference counter assertion failures followed the 1st on a different file and have a different stack trace: ERROR [CompactionExecutor:382] 2014-05-17 01:17:53,157 CassandraDaemon.java (line 198) Exception in thread Thread[CompactionExecutor:382,1,main] java.lang.AssertionError: Reference counter -1 for /mnt/ssd1/cassandra/data/ldn_production/historical_accounts/ldn_production-historical_accounts-jb-83888-Data.db at org.apache.cassandra.io.sstable.SSTableReader.releaseReference(SSTableReader.java:1107) at org.apache.cassandra.io.sstable.SSTableReader.releaseReferences(SSTableReader.java:1429) at org.apache.cassandra.db.compaction.CompactionController.close(CompactionController.java:207) at org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:220) at org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60) at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59) at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:197) ... was: Got this assertion failure this weekend during repair: ERROR [STREAM-IN-/192.168.21.14] 2014-05-17 01:17:52,332 StreamSession.java (line 420) [Stream #3a3ac8a2-dd50-11e3-b3c1-6bf6dccd6457] Streaming error occurred java.lang.RuntimeException: Outgoing stream handler has been closed at org.apache.cassandra.streaming.ConnectionHandler.sendMessage(ConnectionHandler.java:170) at org.apache.cassandra.streaming.StreamSession.receive(StreamSession.java:483) at org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:372) at org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:289) at java.lang.Thread.run(Thread.java:744) ERROR [STREAM-IN-/192.168.21.14] 2014-05-17 01:17:52,350 CassandraDaemon.java (line 198) Exception in thread Thread[STREAM-IN-/192.168.21.14,5,RMI Runtime] java.lang.AssertionError: Reference counter -1 for /mnt/ssd1/cassandra/data/ldn_production/historical_accounts/ldn_production-historical_accounts-jb-79827-Data.db at org.apache.cassandra.io.sstable.SSTableReader.releaseReference(SSTableReader.java:1107) at org.apache.cassandra.streaming.StreamTransferTask.abort(StreamTransferTask.java:80) at org.apache.cassandra.streaming.StreamSession.closeSession(StreamSession.java:322) at
[jira] [Commented] (CASSANDRA-7252) RingCache cannot be configured to use local DC only
[ https://issues.apache.org/jira/browse/CASSANDRA-7252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003408#comment-14003408 ] Robbie Strickland commented on CASSANDRA-7252: -- [~iamaleksey] I see you changed the fix version to 2.0.9. Do you want me to rebase the patch? RingCache cannot be configured to use local DC only --- Key: CASSANDRA-7252 URL: https://issues.apache.org/jira/browse/CASSANDRA-7252 Project: Cassandra Issue Type: Bug Components: Hadoop Reporter: Robbie Strickland Assignee: Jonathan Ellis Labels: patch Fix For: 2.0.9 Attachments: cassandra-2.0.7-7252-2.txt, cassandra-2.0.7-7252.txt RingCache always calls describe_ring, returning the entire cluster. Considering it's used in the context of writing from Hadoop (which is typically in a multi-DC configuration), this is often not desirable behavior. In some cases there may be high-latency connections between the analytics DC and other DCs. I am attaching a patch that adds an optional config value to tell RingCache to use local nodes only, in which case it calls describe_local_ring instead. It also adds helpful failed host information to IOExceptions thrown in AbstractColumnFamilyOutputFormat.createAuthenticatedClient, CqlRecordWriter, and ColumnFamilyRecordWriter. This allows a user to more easily solve related connectivity issues. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-4718) More-efficient ExecutorService for improved throughput
[ https://issues.apache.org/jira/browse/CASSANDRA-4718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003406#comment-14003406 ] Benedict commented on CASSANDRA-4718: - One thing worth mentioning is that the size of the dataset over which this is effective is not necessarily represented accurately by the test, as it was run over a fully-compacted dataset, so the 10M keys would have been randomly distributed across all pages (we select from a prefix of the key range, but the murmur hash will get evenly distributed across the entire dataset once fully compacted). Were this run on a real dataset, with the most recent data being compacted separately to the older data, and the most recent data being hit primarily, there would be greater locality of data access and so any gains should be effective over a larger quantity of data. More-efficient ExecutorService for improved throughput -- Key: CASSANDRA-4718 URL: https://issues.apache.org/jira/browse/CASSANDRA-4718 Project: Cassandra Issue Type: Improvement Reporter: Jonathan Ellis Assignee: Benedict Priority: Minor Labels: performance Fix For: 2.1.0 Attachments: 4718-v1.patch, E100M_summary_key_s.svg, E10M_summary_key_s.svg, E600M_summary_key_s.svg, PerThreadQueue.java, austin_diskbound_read.svg, aws.svg, aws_read.svg, backpressure-stress.out.txt, baq vs trunk.png, belliotsmith_branches-stress.out.txt, jason_read.svg, jason_read_latency.svg, jason_run1.svg, jason_run2.svg, jason_run3.svg, jason_write.svg, op costs of various queues.ods, stress op rate with various queues.ods, stress_2014May15.txt, stress_2014May16.txt, v1-stress.out Currently all our execution stages dequeue tasks one at a time. This can result in contention between producers and consumers (although we do our best to minimize this by using LinkedBlockingQueue). One approach to mitigating this would be to make consumer threads do more work in bulk instead of just one task per dequeue. (Producer threads tend to be single-task oriented by nature, so I don't see an equivalent opportunity there.) BlockingQueue has a drainTo(collection, int) method that would be perfect for this. However, no ExecutorService in the jdk supports using drainTo, nor could I google one. What I would like to do here is create just such a beast and wire it into (at least) the write and read stages. (Other possible candidates for such an optimization, such as the CommitLog and OutboundTCPConnection, are not ExecutorService-based and will need to be one-offs.) AbstractExecutorService may be useful. The implementations of ICommitLogExecutorService may also be useful. (Despite the name these are not actual ExecutorServices, although they share the most important properties of one.) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7241) Pig test fails on 2.1 branch
[ https://issues.apache.org/jira/browse/CASSANDRA-7241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003413#comment-14003413 ] Sylvain Lebresne commented on CASSANDRA-7241: - If others run into that NPE in MiniDFSCluster.startDataNodes, the problem/solution is explained [here|http://stackoverflow.com/questions/17625938/hbase-minidfscluster-java-fails-in-certain-environments]. With that fixed, I do get the same testCqlStorageCompositeKeyTable failure. That said, when starting to look into what's going on, I added one simple log statement in ModificationStatement to log the applied mutation, and somehow that was enough to make the test pass some of the times (not every time, but often enough). So it's a timing issue, probably something that's not really related to CASSANDRA-5417 but was exposed by it because the timing changed. Where to go from there, I'm not sure. I have zero experience with Pig or Hadoop and our integration of it, and I have a hard time wrapping my head around the CqlStorage/AbstractCassandraStorage code tbh, so I'm not sure I'm the most qualified person to track this race down. But given that the pig test seems to be the only ones having problem of that sort, it suggests (though I can't be sure) that said race is more likely in either the pig code or the pig tests than anything else. So I'm going to unassigned myself for now because I just don't have the time and experience to dig into the pig layers. If someone more familiar with those is able to reduce the failure to something Cassandra specific, I'll gladly take a look at that. Pig test fails on 2.1 branch Key: CASSANDRA-7241 URL: https://issues.apache.org/jira/browse/CASSANDRA-7241 Project: Cassandra Issue Type: Bug Reporter: Alex Liu Assignee: Sylvain Lebresne run ant pig-test on cassandra-2.1 branch. There are many tests failed. I trace it a little and find out Pig test fails starts from https://github.com/apache/cassandra/commit/362cc05352ec67e707e0ac790732e96a15e63f6b commit. It looks like storage changes break Pig tests. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7241) Pig test fails on 2.1 branch
[ https://issues.apache.org/jira/browse/CASSANDRA-7241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-7241: Assignee: (was: Sylvain Lebresne) Pig test fails on 2.1 branch Key: CASSANDRA-7241 URL: https://issues.apache.org/jira/browse/CASSANDRA-7241 Project: Cassandra Issue Type: Bug Reporter: Alex Liu run ant pig-test on cassandra-2.1 branch. There are many tests failed. I trace it a little and find out Pig test fails starts from https://github.com/apache/cassandra/commit/362cc05352ec67e707e0ac790732e96a15e63f6b commit. It looks like storage changes break Pig tests. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6146) CQL-native stress
[ https://issues.apache.org/jira/browse/CASSANDRA-6146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003420#comment-14003420 ] Jonathan Ellis commented on CASSANDRA-6146: --- bq. It's done by supplying the path to a yaml file containing the cql3 schema and queries Why wouldn't we just introspect the schema from the cluster? Note also that in most cases we don't actually need to know the schema up front, to run a query (or to process a resultset either). See my first comment: https://issues.apache.org/jira/browse/CASSANDRA-6146?focusedCommentId=13786212page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13786212 CQL-native stress - Key: CASSANDRA-6146 URL: https://issues.apache.org/jira/browse/CASSANDRA-6146 Project: Cassandra Issue Type: New Feature Components: Tools Reporter: Jonathan Ellis Assignee: T Jake Luciani Fix For: 2.1 rc1 Attachments: 6146.txt The existing CQL support in stress is not worth discussing. We need to start over, and we might as well kill two birds with one stone and move to the native protocol while we're at it. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6146) CQL-native stress
[ https://issues.apache.org/jira/browse/CASSANDRA-6146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003491#comment-14003491 ] Benedict commented on CASSANDRA-6146: - bq. Why wouldn't we just introspect the schema from the cluster? Note also that in most cases we don't actually need to know the schema up front, to run a query (or to process a resultset either). Well, you need to define a number of things around the generation of data (and distributions for clustering columns etc.) to populate the schema with, which requires defining the columns anyway. Simply inspecting the schema limits the flexibility (though we could easily inspect the schema for a general purpose do something automatic if we wanted) CQL-native stress - Key: CASSANDRA-6146 URL: https://issues.apache.org/jira/browse/CASSANDRA-6146 Project: Cassandra Issue Type: New Feature Components: Tools Reporter: Jonathan Ellis Assignee: T Jake Luciani Fix For: 2.1 rc1 Attachments: 6146.txt The existing CQL support in stress is not worth discussing. We need to start over, and we might as well kill two birds with one stone and move to the native protocol while we're at it. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6146) CQL-native stress
[ https://issues.apache.org/jira/browse/CASSANDRA-6146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003498#comment-14003498 ] T Jake Luciani commented on CASSANDRA-6146: --- bq. Why wouldn't we just introspect the schema from the cluster? Note also that in most cases we don't actually need to know the schema up front, to run a query (or to process a resultset either). You don't need to specify a schema, but chances are you want to burn through a number of scenarios this allows you to do it in a single well defined way. We could allow you to just specify an existing keyspace and table as well but your queries would fail because the stress tool knows nothing about the shape of your data. You would need to write into the existing schema first. CQL-native stress - Key: CASSANDRA-6146 URL: https://issues.apache.org/jira/browse/CASSANDRA-6146 Project: Cassandra Issue Type: New Feature Components: Tools Reporter: Jonathan Ellis Assignee: T Jake Luciani Fix For: 2.1 rc1 Attachments: 6146.txt The existing CQL support in stress is not worth discussing. We need to start over, and we might as well kill two birds with one stone and move to the native protocol while we're at it. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7270) nodetool netstats broken
[ https://issues.apache.org/jira/browse/CASSANDRA-7270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003555#comment-14003555 ] Joshua McKenzie commented on CASSANDRA-7270: LGTM. +1. nodetool netstats broken Key: CASSANDRA-7270 URL: https://issues.apache.org/jira/browse/CASSANDRA-7270 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Priority: Minor Fix For: 2.1 rc1 Attachments: 0001-fix-session-index.patch nodetool netstats throws exception when we have ongoing streams -- This message was sent by Atlassian JIRA (v6.2#6252)
[05/12] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/57fa76fd Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/57fa76fd Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/57fa76fd Branch: refs/heads/trunk Commit: 57fa76fda2c54c2f17c44e2cd0d0e6517feb833a Parents: 8465a91 1b9f712 Author: Brandon Williams brandonwilli...@apache.org Authored: Tue May 20 10:57:15 2014 -0500 Committer: Brandon Williams brandonwilli...@apache.org Committed: Tue May 20 10:57:15 2014 -0500 -- .../hadoop/cql3/CqlPagingRecordReader.java | 20 ++-- 1 file changed, 18 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/57fa76fd/src/java/org/apache/cassandra/hadoop/cql3/CqlPagingRecordReader.java --
[07/12] git commit: Fix ConfigHelper.setInputColumnFamily with uppercased keyspace names.
Fix ConfigHelper.setInputColumnFamily with uppercased keyspace names.
Patch by Alex Liu, reviewed by pkolackzk for CASSANDRA-7093
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0356ee76
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0356ee76
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0356ee76
Branch: refs/heads/cassandra-2.0
Commit: 0356ee76e4c0b4b863625c5517ae8a4f883766dc
Parents: 1b9f712
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue May 20 11:09:04 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue May 20 11:09:04 2014 -0500
--
.../cassandra/hadoop/cql3/CqlConfigHelper.java | 20 ++--
.../cassandra/hadoop/cql3/CqlRecordReader.java | 8 ++--
2 files changed, 16 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0356ee76/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java
b/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java
index a2cf1e7..63279d1 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java
@@ -370,29 +370,29 @@ public class CqlConfigHelper
conf.set(INPUT_NATIVE_AUTH_PROVIDER, authProvider);
}
-public static void setInputNativeSSLTruststorePath(Configuration conf,
String authProvider)
+public static void setInputNativeSSLTruststorePath(Configuration conf,
String path)
{
-conf.set(INPUT_NATIVE_SSL_TRUST_STORE_PATH, authProvider);
+conf.set(INPUT_NATIVE_SSL_TRUST_STORE_PATH, path);
}
-public static void setInputNativeSSLKeystorePath(Configuration conf,
String authProvider)
+public static void setInputNativeSSLKeystorePath(Configuration conf,
String path)
{
-conf.set(INPUT_NATIVE_SSL_KEY_STORE_PATH, authProvider);
+conf.set(INPUT_NATIVE_SSL_KEY_STORE_PATH, path);
}
-public static void setInputNativeSSLKeystorePassword(Configuration conf,
String authProvider)
+public static void setInputNativeSSLKeystorePassword(Configuration conf,
String pass)
{
-conf.set(INPUT_NATIVE_SSL_KEY_STORE_PASSWARD, authProvider);
+conf.set(INPUT_NATIVE_SSL_KEY_STORE_PASSWARD, pass);
}
-public static void setInputNativeSSLTruststorePassword(Configuration conf,
String authProvider)
+public static void setInputNativeSSLTruststorePassword(Configuration conf,
String pass)
{
-conf.set(INPUT_NATIVE_SSL_TRUST_STORE_PASSWARD, authProvider);
+conf.set(INPUT_NATIVE_SSL_TRUST_STORE_PASSWARD, pass);
}
-public static void setInputNativeSSLCipherSuites(Configuration conf,
String authProvider)
+public static void setInputNativeSSLCipherSuites(Configuration conf,
String suites)
{
-conf.set(INPUT_NATIVE_SSL_CIPHER_SUITES, authProvider);
+conf.set(INPUT_NATIVE_SSL_CIPHER_SUITES, suites);
}
public static void setInputNativeReuseAddress(Configuration conf, String
reuseAddress)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0356ee76/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
b/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
index ff4f1e6..a6514e3 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
@@ -29,7 +29,6 @@ import com.google.common.collect.Maps;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-
import org.apache.cassandra.db.marshal.AbstractType;
import org.apache.cassandra.db.marshal.BytesType;
import org.apache.cassandra.dht.IPartitioner;
@@ -124,7 +123,7 @@ public class CqlRecordReader extends RecordReaderLong, Row
}
if (cluster != null)
-session = cluster.connect(keyspace);
+session = cluster.connect(quote(keyspace));
rowIterator = new RowIterator();
logger.debug(created {}, rowIterator);
}
@@ -486,4 +485,9 @@ public class CqlRecordReader extends RecordReaderLong, Row
return row.getMap(name, keysClass, valuesClass);
}
}
+
+private String quote(String identifier)
+{
+return \ + identifier.replaceAll(\, \\) + \;
+}
}
[06/12] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fef40a5a Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fef40a5a Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fef40a5a Branch: refs/heads/trunk Commit: fef40a5ae03978fa7e27ae020e8ccb4897d8 Parents: 96171d3 57fa76f Author: Brandon Williams brandonwilli...@apache.org Authored: Tue May 20 10:57:33 2014 -0500 Committer: Brandon Williams brandonwilli...@apache.org Committed: Tue May 20 10:57:33 2014 -0500 -- .../hadoop/cql3/CqlPagingRecordReader.java | 20 ++-- 1 file changed, 18 insertions(+), 2 deletions(-) --
[01/12] git commit: Fix cql3 reader when columns is reversed.
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.0 76139e426 - 0356ee76e
refs/heads/cassandra-2.1 8465a91b4 - 86cd42348
refs/heads/trunk 96171d36b - a44ee2ad4
Fix cql3 reader when columns is reversed.
Patch by Alex Liu, reviewed by pkolackzk for CASSANDRA-7100
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1b9f712e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1b9f712e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1b9f712e
Branch: refs/heads/cassandra-2.0
Commit: 1b9f712eb8852068f4e324b9819a957ac7ad4f0e
Parents: 76139e4
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue May 20 10:55:33 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue May 20 10:55:33 2014 -0500
--
.../hadoop/cql3/CqlPagingRecordReader.java | 20 ++--
1 file changed, 18 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1b9f712e/src/java/org/apache/cassandra/hadoop/cql3/CqlPagingRecordReader.java
--
diff --git
a/src/java/org/apache/cassandra/hadoop/cql3/CqlPagingRecordReader.java
b/src/java/org/apache/cassandra/hadoop/cql3/CqlPagingRecordReader.java
index 1492ce0..03d9ae9 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/CqlPagingRecordReader.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlPagingRecordReader.java
@@ -27,13 +27,14 @@ import java.util.*;
import com.google.common.base.Optional;
import com.google.common.collect.AbstractIterator;
import com.google.common.collect.Iterables;
+
import org.apache.cassandra.hadoop.HadoopCompat;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-
import org.apache.cassandra.db.marshal.AbstractType;
import org.apache.cassandra.db.marshal.CompositeType;
import org.apache.cassandra.db.marshal.LongType;
+import org.apache.cassandra.db.marshal.ReversedType;
import org.apache.cassandra.db.marshal.TypeParser;
import org.apache.cassandra.dht.IPartitioner;
import org.apache.cassandra.exceptions.ConfigurationException;
@@ -507,7 +508,7 @@ public class CqlPagingRecordReader extends
RecordReaderMapString, ByteBuffer,
private PairInteger, String whereClause(ListBoundColumn column,
int position)
{
if (position == column.size() - 1 || column.get(position +
1).value == null)
-return Pair.create(position + 2, String.format( AND %s ? ,
quote(column.get(position).name)));
+return Pair.create(position + 2, String.format( AND %s %s ?
, quote(column.get(position).name), column.get(position).reversed ?:
));
PairInteger, String clause = whereClause(column, position + 1);
return Pair.create(clause.left, String.format( AND %s = ? %s,
quote(column.get(position).name), clause.right));
@@ -711,6 +712,20 @@ public class CqlPagingRecordReader extends
RecordReaderMapString, ByteBuffer,
{
partitionBoundColumns.get(0).validator = keyValidator;
}
+
+Column rawComparator = cqlRow.columns.get(3);
+String comparator =
ByteBufferUtil.string(ByteBuffer.wrap(rawComparator.getValue()));
+logger.debug(comparator: {}, comparator);
+AbstractType comparatorValidator = parseType(comparator);
+if (comparatorValidator instanceof CompositeType)
+{
+for (int i = 0; i clusterColumns.size(); i++)
+clusterColumns.get(i).reversed = (((CompositeType)
comparatorValidator).types.get(i) instanceof ReversedType);
+}
+else if (comparatorValidator instanceof ReversedType)
+{
+clusterColumns.get(0).reversed = true;
+}
}
/** check whether current row is at the end of range */
@@ -762,6 +777,7 @@ public class CqlPagingRecordReader extends
RecordReaderMapString, ByteBuffer,
final String name;
ByteBuffer value;
AbstractType? validator;
+boolean reversed = false;
public BoundColumn(String name)
{
[09/12] git commit: Fix ConfigHelper.setInputColumnFamily with uppercased keyspace names.
Fix ConfigHelper.setInputColumnFamily with uppercased keyspace names.
Patch by Alex Liu, reviewed by pkolackzk for CASSANDRA-7093
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0356ee76
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0356ee76
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0356ee76
Branch: refs/heads/cassandra-2.1
Commit: 0356ee76e4c0b4b863625c5517ae8a4f883766dc
Parents: 1b9f712
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue May 20 11:09:04 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue May 20 11:09:04 2014 -0500
--
.../cassandra/hadoop/cql3/CqlConfigHelper.java | 20 ++--
.../cassandra/hadoop/cql3/CqlRecordReader.java | 8 ++--
2 files changed, 16 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0356ee76/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java
b/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java
index a2cf1e7..63279d1 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java
@@ -370,29 +370,29 @@ public class CqlConfigHelper
conf.set(INPUT_NATIVE_AUTH_PROVIDER, authProvider);
}
-public static void setInputNativeSSLTruststorePath(Configuration conf,
String authProvider)
+public static void setInputNativeSSLTruststorePath(Configuration conf,
String path)
{
-conf.set(INPUT_NATIVE_SSL_TRUST_STORE_PATH, authProvider);
+conf.set(INPUT_NATIVE_SSL_TRUST_STORE_PATH, path);
}
-public static void setInputNativeSSLKeystorePath(Configuration conf,
String authProvider)
+public static void setInputNativeSSLKeystorePath(Configuration conf,
String path)
{
-conf.set(INPUT_NATIVE_SSL_KEY_STORE_PATH, authProvider);
+conf.set(INPUT_NATIVE_SSL_KEY_STORE_PATH, path);
}
-public static void setInputNativeSSLKeystorePassword(Configuration conf,
String authProvider)
+public static void setInputNativeSSLKeystorePassword(Configuration conf,
String pass)
{
-conf.set(INPUT_NATIVE_SSL_KEY_STORE_PASSWARD, authProvider);
+conf.set(INPUT_NATIVE_SSL_KEY_STORE_PASSWARD, pass);
}
-public static void setInputNativeSSLTruststorePassword(Configuration conf,
String authProvider)
+public static void setInputNativeSSLTruststorePassword(Configuration conf,
String pass)
{
-conf.set(INPUT_NATIVE_SSL_TRUST_STORE_PASSWARD, authProvider);
+conf.set(INPUT_NATIVE_SSL_TRUST_STORE_PASSWARD, pass);
}
-public static void setInputNativeSSLCipherSuites(Configuration conf,
String authProvider)
+public static void setInputNativeSSLCipherSuites(Configuration conf,
String suites)
{
-conf.set(INPUT_NATIVE_SSL_CIPHER_SUITES, authProvider);
+conf.set(INPUT_NATIVE_SSL_CIPHER_SUITES, suites);
}
public static void setInputNativeReuseAddress(Configuration conf, String
reuseAddress)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0356ee76/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
b/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
index ff4f1e6..a6514e3 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
@@ -29,7 +29,6 @@ import com.google.common.collect.Maps;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-
import org.apache.cassandra.db.marshal.AbstractType;
import org.apache.cassandra.db.marshal.BytesType;
import org.apache.cassandra.dht.IPartitioner;
@@ -124,7 +123,7 @@ public class CqlRecordReader extends RecordReaderLong, Row
}
if (cluster != null)
-session = cluster.connect(keyspace);
+session = cluster.connect(quote(keyspace));
rowIterator = new RowIterator();
logger.debug(created {}, rowIterator);
}
@@ -486,4 +485,9 @@ public class CqlRecordReader extends RecordReaderLong, Row
return row.getMap(name, keysClass, valuesClass);
}
}
+
+private String quote(String identifier)
+{
+return \ + identifier.replaceAll(\, \\) + \;
+}
}
[08/12] git commit: Fix ConfigHelper.setInputColumnFamily with uppercased keyspace names.
Fix ConfigHelper.setInputColumnFamily with uppercased keyspace names.
Patch by Alex Liu, reviewed by pkolackzk for CASSANDRA-7093
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0356ee76
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0356ee76
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0356ee76
Branch: refs/heads/trunk
Commit: 0356ee76e4c0b4b863625c5517ae8a4f883766dc
Parents: 1b9f712
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue May 20 11:09:04 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue May 20 11:09:04 2014 -0500
--
.../cassandra/hadoop/cql3/CqlConfigHelper.java | 20 ++--
.../cassandra/hadoop/cql3/CqlRecordReader.java | 8 ++--
2 files changed, 16 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0356ee76/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java
b/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java
index a2cf1e7..63279d1 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java
@@ -370,29 +370,29 @@ public class CqlConfigHelper
conf.set(INPUT_NATIVE_AUTH_PROVIDER, authProvider);
}
-public static void setInputNativeSSLTruststorePath(Configuration conf,
String authProvider)
+public static void setInputNativeSSLTruststorePath(Configuration conf,
String path)
{
-conf.set(INPUT_NATIVE_SSL_TRUST_STORE_PATH, authProvider);
+conf.set(INPUT_NATIVE_SSL_TRUST_STORE_PATH, path);
}
-public static void setInputNativeSSLKeystorePath(Configuration conf,
String authProvider)
+public static void setInputNativeSSLKeystorePath(Configuration conf,
String path)
{
-conf.set(INPUT_NATIVE_SSL_KEY_STORE_PATH, authProvider);
+conf.set(INPUT_NATIVE_SSL_KEY_STORE_PATH, path);
}
-public static void setInputNativeSSLKeystorePassword(Configuration conf,
String authProvider)
+public static void setInputNativeSSLKeystorePassword(Configuration conf,
String pass)
{
-conf.set(INPUT_NATIVE_SSL_KEY_STORE_PASSWARD, authProvider);
+conf.set(INPUT_NATIVE_SSL_KEY_STORE_PASSWARD, pass);
}
-public static void setInputNativeSSLTruststorePassword(Configuration conf,
String authProvider)
+public static void setInputNativeSSLTruststorePassword(Configuration conf,
String pass)
{
-conf.set(INPUT_NATIVE_SSL_TRUST_STORE_PASSWARD, authProvider);
+conf.set(INPUT_NATIVE_SSL_TRUST_STORE_PASSWARD, pass);
}
-public static void setInputNativeSSLCipherSuites(Configuration conf,
String authProvider)
+public static void setInputNativeSSLCipherSuites(Configuration conf,
String suites)
{
-conf.set(INPUT_NATIVE_SSL_CIPHER_SUITES, authProvider);
+conf.set(INPUT_NATIVE_SSL_CIPHER_SUITES, suites);
}
public static void setInputNativeReuseAddress(Configuration conf, String
reuseAddress)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0356ee76/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
b/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
index ff4f1e6..a6514e3 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java
@@ -29,7 +29,6 @@ import com.google.common.collect.Maps;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-
import org.apache.cassandra.db.marshal.AbstractType;
import org.apache.cassandra.db.marshal.BytesType;
import org.apache.cassandra.dht.IPartitioner;
@@ -124,7 +123,7 @@ public class CqlRecordReader extends RecordReaderLong, Row
}
if (cluster != null)
-session = cluster.connect(keyspace);
+session = cluster.connect(quote(keyspace));
rowIterator = new RowIterator();
logger.debug(created {}, rowIterator);
}
@@ -486,4 +485,9 @@ public class CqlRecordReader extends RecordReaderLong, Row
return row.getMap(name, keysClass, valuesClass);
}
}
+
+private String quote(String identifier)
+{
+return \ + identifier.replaceAll(\, \\) + \;
+}
}
[12/12] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a44ee2ad Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a44ee2ad Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a44ee2ad Branch: refs/heads/trunk Commit: a44ee2ad4a830ac85d87f04efe9741cb0aeafdd1 Parents: fef40a5 86cd423 Author: Brandon Williams brandonwilli...@apache.org Authored: Tue May 20 11:14:30 2014 -0500 Committer: Brandon Williams brandonwilli...@apache.org Committed: Tue May 20 11:14:30 2014 -0500 -- .../cassandra/hadoop/cql3/CqlConfigHelper.java | 20 ++-- .../cassandra/hadoop/cql3/CqlRecordReader.java | 8 ++-- 2 files changed, 16 insertions(+), 12 deletions(-) --
[11/12] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/86cd4234 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/86cd4234 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/86cd4234 Branch: refs/heads/trunk Commit: 86cd423489236adf564c8ce65dd354e92ef8fb90 Parents: 57fa76f 0356ee7 Author: Brandon Williams brandonwilli...@apache.org Authored: Tue May 20 11:14:20 2014 -0500 Committer: Brandon Williams brandonwilli...@apache.org Committed: Tue May 20 11:14:20 2014 -0500 -- .../cassandra/hadoop/cql3/CqlConfigHelper.java | 20 ++-- .../cassandra/hadoop/cql3/CqlRecordReader.java | 8 ++-- 2 files changed, 16 insertions(+), 12 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/86cd4234/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java --
[02/12] git commit: Fix cql3 reader when columns is reversed.
Fix cql3 reader when columns is reversed.
Patch by Alex Liu, reviewed by pkolackzk for CASSANDRA-7100
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1b9f712e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1b9f712e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1b9f712e
Branch: refs/heads/cassandra-2.1
Commit: 1b9f712eb8852068f4e324b9819a957ac7ad4f0e
Parents: 76139e4
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue May 20 10:55:33 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue May 20 10:55:33 2014 -0500
--
.../hadoop/cql3/CqlPagingRecordReader.java | 20 ++--
1 file changed, 18 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1b9f712e/src/java/org/apache/cassandra/hadoop/cql3/CqlPagingRecordReader.java
--
diff --git
a/src/java/org/apache/cassandra/hadoop/cql3/CqlPagingRecordReader.java
b/src/java/org/apache/cassandra/hadoop/cql3/CqlPagingRecordReader.java
index 1492ce0..03d9ae9 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/CqlPagingRecordReader.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlPagingRecordReader.java
@@ -27,13 +27,14 @@ import java.util.*;
import com.google.common.base.Optional;
import com.google.common.collect.AbstractIterator;
import com.google.common.collect.Iterables;
+
import org.apache.cassandra.hadoop.HadoopCompat;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-
import org.apache.cassandra.db.marshal.AbstractType;
import org.apache.cassandra.db.marshal.CompositeType;
import org.apache.cassandra.db.marshal.LongType;
+import org.apache.cassandra.db.marshal.ReversedType;
import org.apache.cassandra.db.marshal.TypeParser;
import org.apache.cassandra.dht.IPartitioner;
import org.apache.cassandra.exceptions.ConfigurationException;
@@ -507,7 +508,7 @@ public class CqlPagingRecordReader extends
RecordReaderMapString, ByteBuffer,
private PairInteger, String whereClause(ListBoundColumn column,
int position)
{
if (position == column.size() - 1 || column.get(position +
1).value == null)
-return Pair.create(position + 2, String.format( AND %s ? ,
quote(column.get(position).name)));
+return Pair.create(position + 2, String.format( AND %s %s ?
, quote(column.get(position).name), column.get(position).reversed ?:
));
PairInteger, String clause = whereClause(column, position + 1);
return Pair.create(clause.left, String.format( AND %s = ? %s,
quote(column.get(position).name), clause.right));
@@ -711,6 +712,20 @@ public class CqlPagingRecordReader extends
RecordReaderMapString, ByteBuffer,
{
partitionBoundColumns.get(0).validator = keyValidator;
}
+
+Column rawComparator = cqlRow.columns.get(3);
+String comparator =
ByteBufferUtil.string(ByteBuffer.wrap(rawComparator.getValue()));
+logger.debug(comparator: {}, comparator);
+AbstractType comparatorValidator = parseType(comparator);
+if (comparatorValidator instanceof CompositeType)
+{
+for (int i = 0; i clusterColumns.size(); i++)
+clusterColumns.get(i).reversed = (((CompositeType)
comparatorValidator).types.get(i) instanceof ReversedType);
+}
+else if (comparatorValidator instanceof ReversedType)
+{
+clusterColumns.get(0).reversed = true;
+}
}
/** check whether current row is at the end of range */
@@ -762,6 +777,7 @@ public class CqlPagingRecordReader extends
RecordReaderMapString, ByteBuffer,
final String name;
ByteBuffer value;
AbstractType? validator;
+boolean reversed = false;
public BoundColumn(String name)
{
[04/12] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/57fa76fd Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/57fa76fd Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/57fa76fd Branch: refs/heads/cassandra-2.1 Commit: 57fa76fda2c54c2f17c44e2cd0d0e6517feb833a Parents: 8465a91 1b9f712 Author: Brandon Williams brandonwilli...@apache.org Authored: Tue May 20 10:57:15 2014 -0500 Committer: Brandon Williams brandonwilli...@apache.org Committed: Tue May 20 10:57:15 2014 -0500 -- .../hadoop/cql3/CqlPagingRecordReader.java | 20 ++-- 1 file changed, 18 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/57fa76fd/src/java/org/apache/cassandra/hadoop/cql3/CqlPagingRecordReader.java --
[10/12] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/86cd4234 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/86cd4234 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/86cd4234 Branch: refs/heads/cassandra-2.1 Commit: 86cd423489236adf564c8ce65dd354e92ef8fb90 Parents: 57fa76f 0356ee7 Author: Brandon Williams brandonwilli...@apache.org Authored: Tue May 20 11:14:20 2014 -0500 Committer: Brandon Williams brandonwilli...@apache.org Committed: Tue May 20 11:14:20 2014 -0500 -- .../cassandra/hadoop/cql3/CqlConfigHelper.java | 20 ++-- .../cassandra/hadoop/cql3/CqlRecordReader.java | 8 ++-- 2 files changed, 16 insertions(+), 12 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/86cd4234/src/java/org/apache/cassandra/hadoop/cql3/CqlRecordReader.java --
[03/12] git commit: Fix cql3 reader when columns is reversed.
Fix cql3 reader when columns is reversed.
Patch by Alex Liu, reviewed by pkolackzk for CASSANDRA-7100
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1b9f712e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1b9f712e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1b9f712e
Branch: refs/heads/trunk
Commit: 1b9f712eb8852068f4e324b9819a957ac7ad4f0e
Parents: 76139e4
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue May 20 10:55:33 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue May 20 10:55:33 2014 -0500
--
.../hadoop/cql3/CqlPagingRecordReader.java | 20 ++--
1 file changed, 18 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1b9f712e/src/java/org/apache/cassandra/hadoop/cql3/CqlPagingRecordReader.java
--
diff --git
a/src/java/org/apache/cassandra/hadoop/cql3/CqlPagingRecordReader.java
b/src/java/org/apache/cassandra/hadoop/cql3/CqlPagingRecordReader.java
index 1492ce0..03d9ae9 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/CqlPagingRecordReader.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlPagingRecordReader.java
@@ -27,13 +27,14 @@ import java.util.*;
import com.google.common.base.Optional;
import com.google.common.collect.AbstractIterator;
import com.google.common.collect.Iterables;
+
import org.apache.cassandra.hadoop.HadoopCompat;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-
import org.apache.cassandra.db.marshal.AbstractType;
import org.apache.cassandra.db.marshal.CompositeType;
import org.apache.cassandra.db.marshal.LongType;
+import org.apache.cassandra.db.marshal.ReversedType;
import org.apache.cassandra.db.marshal.TypeParser;
import org.apache.cassandra.dht.IPartitioner;
import org.apache.cassandra.exceptions.ConfigurationException;
@@ -507,7 +508,7 @@ public class CqlPagingRecordReader extends
RecordReaderMapString, ByteBuffer,
private PairInteger, String whereClause(ListBoundColumn column,
int position)
{
if (position == column.size() - 1 || column.get(position +
1).value == null)
-return Pair.create(position + 2, String.format( AND %s ? ,
quote(column.get(position).name)));
+return Pair.create(position + 2, String.format( AND %s %s ?
, quote(column.get(position).name), column.get(position).reversed ?:
));
PairInteger, String clause = whereClause(column, position + 1);
return Pair.create(clause.left, String.format( AND %s = ? %s,
quote(column.get(position).name), clause.right));
@@ -711,6 +712,20 @@ public class CqlPagingRecordReader extends
RecordReaderMapString, ByteBuffer,
{
partitionBoundColumns.get(0).validator = keyValidator;
}
+
+Column rawComparator = cqlRow.columns.get(3);
+String comparator =
ByteBufferUtil.string(ByteBuffer.wrap(rawComparator.getValue()));
+logger.debug(comparator: {}, comparator);
+AbstractType comparatorValidator = parseType(comparator);
+if (comparatorValidator instanceof CompositeType)
+{
+for (int i = 0; i clusterColumns.size(); i++)
+clusterColumns.get(i).reversed = (((CompositeType)
comparatorValidator).types.get(i) instanceof ReversedType);
+}
+else if (comparatorValidator instanceof ReversedType)
+{
+clusterColumns.get(0).reversed = true;
+}
}
/** check whether current row is at the end of range */
@@ -762,6 +777,7 @@ public class CqlPagingRecordReader extends
RecordReaderMapString, ByteBuffer,
final String name;
ByteBuffer value;
AbstractType? validator;
+boolean reversed = false;
public BoundColumn(String name)
{
[jira] [Updated] (CASSANDRA-7100) Regression in 2.0.x - Cql3 reader returns duplicate rows if the cluster column is reversed
[ https://issues.apache.org/jira/browse/CASSANDRA-7100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-7100: Fix Version/s: 2.1 rc1 2.0.8 Regression in 2.0.x - Cql3 reader returns duplicate rows if the cluster column is reversed -- Key: CASSANDRA-7100 URL: https://issues.apache.org/jira/browse/CASSANDRA-7100 Project: Cassandra Issue Type: Bug Affects Versions: 1.2.6 Reporter: Rohit Rai Assignee: Alex Liu Fix For: 2.0.8, 2.1 rc1 Attachments: 7100-2.0-branch.txt To reproduce it, cqlsh:test select * from wordfreq; title | occurances | word -++--- alex123 | 4 | liu3 alex1 | 23456 | liu2 alex10 | 10 | liu10 alex12 | 34 | liu3 alex | 123456 | liu1 alex | 1000 | liu CREATE TABLE wordfreq ( title text, word text, occurances int, PRIMARY KEY (title,occurances)) WITH CLUSTERING ORDER by (occurances DESC); The hadoop job returns 7 rows instead of 6 rows. I will post a patch soon. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (CASSANDRA-7100) Regression in 2.0.x - Cql3 reader returns duplicate rows if the cluster column is reversed
[ https://issues.apache.org/jira/browse/CASSANDRA-7100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams resolved CASSANDRA-7100. - Resolution: Fixed Committed. Regression in 2.0.x - Cql3 reader returns duplicate rows if the cluster column is reversed -- Key: CASSANDRA-7100 URL: https://issues.apache.org/jira/browse/CASSANDRA-7100 Project: Cassandra Issue Type: Bug Affects Versions: 1.2.6 Reporter: Rohit Rai Assignee: Alex Liu Attachments: 7100-2.0-branch.txt To reproduce it, cqlsh:test select * from wordfreq; title | occurances | word -++--- alex123 | 4 | liu3 alex1 | 23456 | liu2 alex10 | 10 | liu10 alex12 | 34 | liu3 alex | 123456 | liu1 alex | 1000 | liu CREATE TABLE wordfreq ( title text, word text, occurances int, PRIMARY KEY (title,occurances)) WITH CLUSTERING ORDER by (occurances DESC); The hadoop job returns 7 rows instead of 6 rows. I will post a patch soon. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (CASSANDRA-7093) ConfigHelper.setInputColumnFamily incompatible with upper-cased keyspaces since 2.0.7
[ https://issues.apache.org/jira/browse/CASSANDRA-7093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams resolved CASSANDRA-7093. - Resolution: Fixed Fix Version/s: 2.0.8 Committed. ConfigHelper.setInputColumnFamily incompatible with upper-cased keyspaces since 2.0.7 - Key: CASSANDRA-7093 URL: https://issues.apache.org/jira/browse/CASSANDRA-7093 Project: Cassandra Issue Type: Bug Reporter: Maxime Nay Assignee: Alex Liu Fix For: 2.0.8 Attachments: 7093.txt Hi, We have a keyspace starting with an upper-case character: Visitors. We are trying to run a map reduce job on one of the column family of this keyspace. To specify the keyspace it seems we have to use: org.apache.cassandra.hadoop. ConfigHelper.setInputColumnFamily(conf, keyspace, columnFamily); If we do: ConfigHelper.setInputColumnFamily(conf, Visitors, columnFamily); we get: com.datastax.driver.core.exceptions.InvalidQueryException: Keyspace 'visitors' does not exist at com.datastax.driver.core.exceptions.InvalidQueryException.copy(InvalidQueryException.java:35) at com.datastax.driver.core.DefaultResultSetFuture.extractCauseFromExecutionException(DefaultResultSetFuture.java:256) at com.datastax.driver.core.SessionManager.setKeyspace(SessionManager.java:335) ... And if we do: ConfigHelper.setInputColumnFamily(conf, \Visitors\, columnFamily); we get: Exception in thread main java.lang.RuntimeException: InvalidRequestException(why:No such keyspace: Visitors) at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.getRangeMap(AbstractColumnFamilyInputFormat.java:339) at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.getSplits(AbstractColumnFamilyInputFormat.java:125) at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:962) at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:979) ... This is working just fine if the keyspace is lowercase. And it was working just fine with Cassandra 2.0.6. But with Cassandra 2.0.7, and the addition of Datastax's java driver in the dependencies, I am getting this error. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7093) ConfigHelper.setInputColumnFamily incompatible with upper-cased keyspaces since 2.0.7
[ https://issues.apache.org/jira/browse/CASSANDRA-7093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-7093: Fix Version/s: 2.1 rc1 ConfigHelper.setInputColumnFamily incompatible with upper-cased keyspaces since 2.0.7 - Key: CASSANDRA-7093 URL: https://issues.apache.org/jira/browse/CASSANDRA-7093 Project: Cassandra Issue Type: Bug Reporter: Maxime Nay Assignee: Alex Liu Fix For: 2.0.8, 2.1 rc1 Attachments: 7093.txt Hi, We have a keyspace starting with an upper-case character: Visitors. We are trying to run a map reduce job on one of the column family of this keyspace. To specify the keyspace it seems we have to use: org.apache.cassandra.hadoop. ConfigHelper.setInputColumnFamily(conf, keyspace, columnFamily); If we do: ConfigHelper.setInputColumnFamily(conf, Visitors, columnFamily); we get: com.datastax.driver.core.exceptions.InvalidQueryException: Keyspace 'visitors' does not exist at com.datastax.driver.core.exceptions.InvalidQueryException.copy(InvalidQueryException.java:35) at com.datastax.driver.core.DefaultResultSetFuture.extractCauseFromExecutionException(DefaultResultSetFuture.java:256) at com.datastax.driver.core.SessionManager.setKeyspace(SessionManager.java:335) ... And if we do: ConfigHelper.setInputColumnFamily(conf, \Visitors\, columnFamily); we get: Exception in thread main java.lang.RuntimeException: InvalidRequestException(why:No such keyspace: Visitors) at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.getRangeMap(AbstractColumnFamilyInputFormat.java:339) at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.getSplits(AbstractColumnFamilyInputFormat.java:125) at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:962) at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:979) ... This is working just fine if the keyspace is lowercase. And it was working just fine with Cassandra 2.0.6. But with Cassandra 2.0.7, and the addition of Datastax's java driver in the dependencies, I am getting this error. -- This message was sent by Atlassian JIRA (v6.2#6252)
git commit: Change serialization format for UDT
Repository: cassandra Updated Branches: refs/heads/cassandra-2.1 86cd42348 - db9ef0b6d Change serialization format for UDT patch by slebresne; reviewed by iamaleksey for CASSANDRA-7209 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/db9ef0b6 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/db9ef0b6 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/db9ef0b6 Branch: refs/heads/cassandra-2.1 Commit: db9ef0b6db3b3198cc0c67eb00505f0ef53b87e3 Parents: 86cd423 Author: Sylvain Lebresne sylv...@datastax.com Authored: Tue May 13 14:03:54 2014 +0200 Committer: Sylvain Lebresne sylv...@datastax.com Committed: Tue May 20 18:29:52 2014 +0200 -- CHANGES.txt | 1 + doc/native_protocol_v3.spec | 9 +- .../org/apache/cassandra/config/CFMetaData.java | 4 +- .../org/apache/cassandra/config/UTMetaData.java | 16 +- .../org/apache/cassandra/cql3/UserTypes.java| 21 ++- .../cql3/statements/AlterTypeStatement.java | 26 +-- .../cql3/statements/CreateTypeStatement.java| 8 +- .../cql3/statements/DropTypeStatement.java | 19 +- .../cassandra/cql3/statements/Selection.java| 10 +- .../apache/cassandra/db/marshal/UserType.java | 175 +-- .../apache/cassandra/transport/DataType.java| 14 +- 11 files changed, 230 insertions(+), 73 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/db9ef0b6/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index ef678f1..e7e7c7f 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -16,6 +16,7 @@ * remove unused classes (CASSANDRA-7197) * Limit user types to the keyspace they are defined in (CASSANDRA-6643) * Add validate method to CollectionType (CASSANDRA-7208) + * New serialization format for UDT values (CASSANDRA-7209) Merged from 2.0: * Always reallocate buffers in HSHA (CASSANDRA-6285) * (Hadoop) support authentication in CqlRecordReader (CASSANDRA-7221) http://git-wip-us.apache.org/repos/asf/cassandra/blob/db9ef0b6/doc/native_protocol_v3.spec -- diff --git a/doc/native_protocol_v3.spec b/doc/native_protocol_v3.spec index 6662f1c..6719838 100644 --- a/doc/native_protocol_v3.spec +++ b/doc/native_protocol_v3.spec @@ -758,12 +758,13 @@ Table of Contents 7. User defined types - This section describe the serialization format for User defined types (UDT) values. + This section describes the serialization format for User defined types (UDT) values. UDT values are the values of the User Defined Types as defined in section 4.2.5.2. - A UDT value is a [short] n indicating the number of values (field) of UDT values - followed by n elements. Each element is a [short bytes] representing the serialized - field. + A UDT value is composed of successive [bytes] values, one for each field of the UDT + value (in the order defined by the type). A UDT value will generally have one value + for each field of the type it represents, but it is allowed to have less values than + the type has fields. 8. Result paging http://git-wip-us.apache.org/repos/asf/cassandra/blob/db9ef0b6/src/java/org/apache/cassandra/config/CFMetaData.java -- diff --git a/src/java/org/apache/cassandra/config/CFMetaData.java b/src/java/org/apache/cassandra/config/CFMetaData.java index 2df42ae..f13b408 100644 --- a/src/java/org/apache/cassandra/config/CFMetaData.java +++ b/src/java/org/apache/cassandra/config/CFMetaData.java @@ -216,8 +216,8 @@ public final class CFMetaData public static final CFMetaData SchemaUserTypesCf = compile(CREATE TABLE + SystemKeyspace.SCHEMA_USER_TYPES_CF + ( + keyspace_name text, + type_name text, - + column_names listtext, - + column_types listtext, + + field_names listtext, + + field_types listtext, + PRIMARY KEY (keyspace_name, type_name) + ) WITH COMMENT='Defined user types' AND gc_grace_seconds=8640); http://git-wip-us.apache.org/repos/asf/cassandra/blob/db9ef0b6/src/java/org/apache/cassandra/config/UTMetaData.java -- diff
[1/2] git commit: Change serialization format for UDT
Repository: cassandra Updated Branches: refs/heads/trunk a44ee2ad4 - d3e778706 Change serialization format for UDT patch by slebresne; reviewed by iamaleksey for CASSANDRA-7209 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/db9ef0b6 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/db9ef0b6 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/db9ef0b6 Branch: refs/heads/trunk Commit: db9ef0b6db3b3198cc0c67eb00505f0ef53b87e3 Parents: 86cd423 Author: Sylvain Lebresne sylv...@datastax.com Authored: Tue May 13 14:03:54 2014 +0200 Committer: Sylvain Lebresne sylv...@datastax.com Committed: Tue May 20 18:29:52 2014 +0200 -- CHANGES.txt | 1 + doc/native_protocol_v3.spec | 9 +- .../org/apache/cassandra/config/CFMetaData.java | 4 +- .../org/apache/cassandra/config/UTMetaData.java | 16 +- .../org/apache/cassandra/cql3/UserTypes.java| 21 ++- .../cql3/statements/AlterTypeStatement.java | 26 +-- .../cql3/statements/CreateTypeStatement.java| 8 +- .../cql3/statements/DropTypeStatement.java | 19 +- .../cassandra/cql3/statements/Selection.java| 10 +- .../apache/cassandra/db/marshal/UserType.java | 175 +-- .../apache/cassandra/transport/DataType.java| 14 +- 11 files changed, 230 insertions(+), 73 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/db9ef0b6/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index ef678f1..e7e7c7f 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -16,6 +16,7 @@ * remove unused classes (CASSANDRA-7197) * Limit user types to the keyspace they are defined in (CASSANDRA-6643) * Add validate method to CollectionType (CASSANDRA-7208) + * New serialization format for UDT values (CASSANDRA-7209) Merged from 2.0: * Always reallocate buffers in HSHA (CASSANDRA-6285) * (Hadoop) support authentication in CqlRecordReader (CASSANDRA-7221) http://git-wip-us.apache.org/repos/asf/cassandra/blob/db9ef0b6/doc/native_protocol_v3.spec -- diff --git a/doc/native_protocol_v3.spec b/doc/native_protocol_v3.spec index 6662f1c..6719838 100644 --- a/doc/native_protocol_v3.spec +++ b/doc/native_protocol_v3.spec @@ -758,12 +758,13 @@ Table of Contents 7. User defined types - This section describe the serialization format for User defined types (UDT) values. + This section describes the serialization format for User defined types (UDT) values. UDT values are the values of the User Defined Types as defined in section 4.2.5.2. - A UDT value is a [short] n indicating the number of values (field) of UDT values - followed by n elements. Each element is a [short bytes] representing the serialized - field. + A UDT value is composed of successive [bytes] values, one for each field of the UDT + value (in the order defined by the type). A UDT value will generally have one value + for each field of the type it represents, but it is allowed to have less values than + the type has fields. 8. Result paging http://git-wip-us.apache.org/repos/asf/cassandra/blob/db9ef0b6/src/java/org/apache/cassandra/config/CFMetaData.java -- diff --git a/src/java/org/apache/cassandra/config/CFMetaData.java b/src/java/org/apache/cassandra/config/CFMetaData.java index 2df42ae..f13b408 100644 --- a/src/java/org/apache/cassandra/config/CFMetaData.java +++ b/src/java/org/apache/cassandra/config/CFMetaData.java @@ -216,8 +216,8 @@ public final class CFMetaData public static final CFMetaData SchemaUserTypesCf = compile(CREATE TABLE + SystemKeyspace.SCHEMA_USER_TYPES_CF + ( + keyspace_name text, + type_name text, - + column_names listtext, - + column_types listtext, + + field_names listtext, + + field_types listtext, + PRIMARY KEY (keyspace_name, type_name) + ) WITH COMMENT='Defined user types' AND gc_grace_seconds=8640); http://git-wip-us.apache.org/repos/asf/cassandra/blob/db9ef0b6/src/java/org/apache/cassandra/config/UTMetaData.java -- diff --git
[2/2] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d3e77870 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d3e77870 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d3e77870 Branch: refs/heads/trunk Commit: d3e7787064d1ac833541b7c5529912e1c1307b99 Parents: a44ee2a db9ef0b Author: Sylvain Lebresne sylv...@datastax.com Authored: Tue May 20 18:30:06 2014 +0200 Committer: Sylvain Lebresne sylv...@datastax.com Committed: Tue May 20 18:30:06 2014 +0200 -- CHANGES.txt | 1 + doc/native_protocol_v3.spec | 9 +- .../org/apache/cassandra/config/CFMetaData.java | 4 +- .../org/apache/cassandra/config/UTMetaData.java | 16 +- .../org/apache/cassandra/cql3/UserTypes.java| 21 ++- .../cql3/statements/AlterTypeStatement.java | 26 +-- .../cql3/statements/CreateTypeStatement.java| 8 +- .../cql3/statements/DropTypeStatement.java | 19 +- .../cassandra/cql3/statements/Selection.java| 10 +- .../apache/cassandra/db/marshal/UserType.java | 175 +-- .../apache/cassandra/transport/DataType.java| 14 +- 11 files changed, 230 insertions(+), 73 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d3e77870/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d3e77870/src/java/org/apache/cassandra/config/CFMetaData.java --
[jira] [Commented] (CASSANDRA-7023) Nodetool clearsnapshot should not remove all snapshots by default
[ https://issues.apache.org/jira/browse/CASSANDRA-7023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003597#comment-14003597 ] Laura Adney commented on CASSANDRA-7023: Want to add another point (business scenario) to this. When using snapshots for backup, administrators might do the following: Take a snapshot, for tape backups Repairs running at the same time, which adds an additional snapshot Do tape backup Clear snapshot The process runs the risk of removing a snapshot used for repairs as well as adding unwanted space to the backup. The process here is to help manage disk space in a manner that doesn't require a lot of complexity or additional management for snapshots created for repair versus snapshots created for backup. It would streamline the clear snapshot process to be able to exclude those snapshots that repair is using. Nodetool clearsnapshot should not remove all snapshots by default - Key: CASSANDRA-7023 URL: https://issues.apache.org/jira/browse/CASSANDRA-7023 Project: Cassandra Issue Type: New Feature Components: Tools Reporter: Sucwinder Bassi Priority: Minor Running a nodetool clearsnapshot will remove all snapshot files by default. As this is removing data this shouldn't just go away and just remove all snapshots. If you want to remove all snapshots there should be a force option. A list option showing snapshots available would also be helpful which could be piped into the nodetool clearsnapshot command. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7209) Consider changing UDT serialization format before 2.1 release.
[ https://issues.apache.org/jira/browse/CASSANDRA-7209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003607#comment-14003607 ] Sylvain Lebresne commented on CASSANDRA-7209: - Committed with doc typos, thanks. bq. One nit, as mentioned before, is that we don't need to serialize the keyspace name That's true. It's not entirely trivial code-wise to remove it because we need it in DataType#readValue to create a UserType (the AbstractType class) from what we got from the protocol definition and we don't have the context easily available. We could probably make such context available somehow but it's likely painful to refactor. So, since as said by Tyler this is not hugely relevant to performance since we only send that a preparation time (thanks to the skip_metadata), I'd vote for letting it be. Besides, one advantage of leaving it in is that if we ever change our mind about limiting user type to their keyspace of definition, we won't be stuck because of that. Consider changing UDT serialization format before 2.1 release. -- Key: CASSANDRA-7209 URL: https://issues.apache.org/jira/browse/CASSANDRA-7209 Project: Cassandra Issue Type: Bug Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Fix For: 2.1 rc1 Attachments: 0001-7209.txt, 0002-Rename-column_names-types-to-field_names-types.txt, doc-typos.txt The current serialization format of UDT is the one of CompositeType. This was initially done on purpose, so that users that were using CompositeType for values in their thrift schema could migrate smoothly to UDT (it was also convenient code wise but that's a weak point). I'm having serious doubt about this being wise however for 2 reasons: * for each component, CompositeType stores an addition byte (the end-of-component) for reasons that only pertain to querying. This byte is basically wasted for UDT and makes no sense. I'll note that outside the inefficiency, there is also the fact that it will likely be pretty surprising/error-prone for driver authors. * it uses an unsigned short for the length of each component. While it's certainly not advisable in the current implementation to use values too big inside an UDT, having this limitation hard-coded in the serialization format is wrong and we've been bitten by this with collection already which we've had to fix in the protocol v3. It's probably worth no doing that mistake again. Furthermore, if we use an int for the size, we can use a negative size to represent a null value (the main point being that it's consistent with how we serialize values in the native protocol), which can be useful (CASSANDRA-7206). Of course, if we change that serialization format, we'd better do it before the 2.1 release. But I think the advantages outweigh the cons especially in the long run so I think we should do it. I'll try to work out a patch quickly so if you have a problem with the principle of this issue, it would be nice to voice it quickly. I'll note that doing that change will mean existing CompositeType values won't be able to be migrated transparently to UDT. I think this was anecdotal in the first place at best, I don't think using CompositeType for values is that popular in thrift tbh. Besides, if we really really want to, it might not be too hard to re-introduce that compatibility later by having some protocol level trick. We can't change the serialization format without breaking people however. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7248) Tuple type
[
https://issues.apache.org/jira/browse/CASSANDRA-7248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003613#comment-14003613
]
Tyler Hobbs commented on CASSANDRA-7248:
bq. That is, if we do CASSANDRA-6875, we will have TupleType, it will be part
of the native protocol and it will be a type visible to users since they that
will be a type of bind variables. To take the Java driver as example, this
imply that we will need to have a 'tuple type' in DataType and will need to
introduce a new tuple class for tuple values so user can pass them for
CASSANDRA-6875.
Ah, that's a fair point. With the python driver users would be totally unaware
of TupleType, so I didn't think about that. With that in mind, making the
tuple type externally visible seems more reasonable to me.
Tuple type
--
Key: CASSANDRA-7248
URL: https://issues.apache.org/jira/browse/CASSANDRA-7248
Project: Cassandra
Issue Type: Bug
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Labels: cql3
Fix For: 2.1 rc1
For CASSANDRA-6875 we need to be able to talk about tuples values and types
(for prepared variables). Since we need it there, clients will need to
support them anyway and so I think it would be a lot cleaner to start
supporting those more generally. Besides, having tuples is a relatively
simple and natural extension to what we have. I'll note in particular that
tuple have a close relationship to user type in the sense that a tuple will
be really just like an anonymous with no name for the fields and in
particular a tuple value will be the same than a user type value.
The syntax would simply look like that:
{noformat}
CREATE TABLE foo (
k int PRIMARY KEY,
v tupleint, text, float
)
INSERT INTO foo(k, v) VALUES(0, (3, 'bar', 2.1));
{noformat}
We can also add projections in selects if we want:
{noformat}
SELECT v[0], v[2] FROM foo WHERE k = 0;
{noformat}
but that can come later (after all, we still don't have projections for
collections and it's not a big deal).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Assigned] (CASSANDRA-7267) Embedded sets in user defined data-types are not updating
[
https://issues.apache.org/jira/browse/CASSANDRA-7267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne reassigned CASSANDRA-7267:
---
Assignee: Sylvain Lebresne
Embedded sets in user defined data-types are not updating
-
Key: CASSANDRA-7267
URL: https://issues.apache.org/jira/browse/CASSANDRA-7267
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Thomas Zimmer
Assignee: Sylvain Lebresne
Fix For: 2.1 rc1
Hi,
i just played around with Cassandra 2.1.0 beta2 and i might have found an
issue with embedded Sets in User Defined Data Types.
Here is how i can reproduce it:
1.) Create a keyspace test
2.) Create a table like this:
create table songs (title varchar PRIMARY KEY, band varchar, tags
Setvarchar);
3.) Create a udt like this:
create type band_info_type (founded timestamp, members Setvarchar,
description text);
4.) Try to insert data:
insert into songs (title, band, band_info, tags) values ('The trooper', 'Iron
Maiden', {founded:188694000, members: {'Bruce Dickinson', 'Dave Murray',
'Adrian Smith', 'Janick Gers', 'Steve Harris', 'Nicko McBrain'}, description:
'Pure evil metal'}, {'metal', 'england'});
5.) Select the data:
select * from songs;
Returns this:
The trooper | Iron Maiden | {founded: '1970-01-03 05:24:54+0100', members:
{}, description: 'Pure evil metal'} | {'england', 'metal'}
The embedded data-set seems to empty. I also tried updating a row which also
does not seem to work.
Regards,
Thomas
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-6397) removenode outputs confusing non-error
[
https://issues.apache.org/jira/browse/CASSANDRA-6397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003629#comment-14003629
]
Kirk True commented on CASSANDRA-6397:
--
Yes, I agree that's ugly.
My concern is that this change turns an error case (forcing when nothing is
being removed) into a non-error case. Do users of either nodetool or JMX rely
on getting an error? Do they need to know that their request was a no-op when
they ask to force a removenode?
If so, they can check the boolean, kind of like File.delete(). But if you want,
I can just remove the boolean FYI flag altogether.
removenode outputs confusing non-error
--
Key: CASSANDRA-6397
URL: https://issues.apache.org/jira/browse/CASSANDRA-6397
Project: Cassandra
Issue Type: Bug
Components: Tools
Reporter: Ryan McGuire
Assignee: Kirk True
Priority: Trivial
Labels: lhf
Fix For: 2.0.8
Attachments: trunk-6397.txt
*{{nodetool removenode force}}* outputs a slightly confusing error message
when there is nothing for it to do.
* Start a cluster, then kill one of the nodes.
* run *{{nodetool removenode}}* on the node you killed.
* Simultaneously, in another shell, run *{{nodetool removenode force}}*, see
that it outputs a simple message regarding it's status.
* Run *{{nodetool removenode force}}* again after the firsrt removenode
command finishes, you'll see this message and traceback:
{code}
$ ~/.ccm/test/node1/bin/nodetool -p 7100 removenode force
RemovalStatus: No token removals in process.
Exception in thread main java.lang.UnsupportedOperationException: No tokens
to force removal on, call 'removetoken' first
at
org.apache.cassandra.service.StorageService.forceRemoveCompletion(StorageService.java:3140)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:111)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:45)
at
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:235)
at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:250)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:791)
at
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1486)
at
javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:96)
at
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1327)
at
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1419)
at
javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:847)
at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
at sun.rmi.transport.Transport$1.run(Transport.java:177)
at sun.rmi.transport.Transport$1.run(Transport.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
at
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:553)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:808)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:667)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
{code}
Two issues I see with this traceback:
* No tokens to force removal on is telling me the same thing that the
message before it tells me: RemovalStatus: No token removals in process.,
So the entire traceback is redundant.
* call 'removetoken' first - removetoken has been deprecated according to
the message output by removenode, so there is inconsistency in directions to
the user.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7255) Rename nodetool cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-7255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003649#comment-14003649 ] Sylvain Lebresne commented on CASSANDRA-7255: - bq. 'cleanranges' requires cassandra knowledge that would prevent this problem to begin with But at least it's less likely that someone will think it's cleaning up tombstones (and we can probably go with something even more specific like cleantokenranges). I don't think it's about having completely new user figure out what it does precisely just by the name, but rather avoid that people would think it does something it doesn't because the name is too generic. I can buy that cleanup is too generic on that front (but I agree 'trim' is not particularly better). That said, the one downside of changing it is that it might break existing user scripts and require fixing all docs (which will likely generate more confusion than it will fix at least in the short term). I don't have a very strong opinion on whether the benefits of a name change outweighs those initial downsides however. Rename nodetool cleanup --- Key: CASSANDRA-7255 URL: https://issues.apache.org/jira/browse/CASSANDRA-7255 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Tyler Hobbs Priority: Trivial A lot of cassandra users seem to think nodetool cleanup does all kinds of things that it doesn't. For example, you'll find a lot of quotes like this in IRC and on the mailing list: bq. you need to run 'nodetool cleanup' to make sure all tombstones are deleted Renaming it to something clearer could help cut down on the confusion. Perhaps nodetool cleanranges or nodetool trim. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6397) removenode outputs confusing non-error
[
https://issues.apache.org/jira/browse/CASSANDRA-6397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003640#comment-14003640
]
Brandon Williams commented on CASSANDRA-6397:
-
I think we're conflating a couple issues here. I think Ryan's point is there
shouldn't be a stacktrace printed, but if there is a problem we should report
that and exit with a non-zero code. Programmatically though, the boolean is
completely unused as the patch stands, so we don't need that unless it's to
enable solving the first problem.
removenode outputs confusing non-error
--
Key: CASSANDRA-6397
URL: https://issues.apache.org/jira/browse/CASSANDRA-6397
Project: Cassandra
Issue Type: Bug
Components: Tools
Reporter: Ryan McGuire
Assignee: Kirk True
Priority: Trivial
Labels: lhf
Fix For: 2.0.8
Attachments: trunk-6397.txt
*{{nodetool removenode force}}* outputs a slightly confusing error message
when there is nothing for it to do.
* Start a cluster, then kill one of the nodes.
* run *{{nodetool removenode}}* on the node you killed.
* Simultaneously, in another shell, run *{{nodetool removenode force}}*, see
that it outputs a simple message regarding it's status.
* Run *{{nodetool removenode force}}* again after the firsrt removenode
command finishes, you'll see this message and traceback:
{code}
$ ~/.ccm/test/node1/bin/nodetool -p 7100 removenode force
RemovalStatus: No token removals in process.
Exception in thread main java.lang.UnsupportedOperationException: No tokens
to force removal on, call 'removetoken' first
at
org.apache.cassandra.service.StorageService.forceRemoveCompletion(StorageService.java:3140)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:111)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:45)
at
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:235)
at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:250)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:791)
at
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1486)
at
javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:96)
at
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1327)
at
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1419)
at
javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:847)
at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
at sun.rmi.transport.Transport$1.run(Transport.java:177)
at sun.rmi.transport.Transport$1.run(Transport.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
at
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:553)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:808)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:667)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
{code}
Two issues I see with this traceback:
* No tokens to force removal on is telling me the same thing that the
message before it tells me: RemovalStatus: No token removals in process.,
So the entire traceback is redundant.
* call 'removetoken' first - removetoken has been deprecated according to
the message output by removenode, so there is inconsistency in directions to
the user.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7231) Support more concurrent requests per native transport connection
[ https://issues.apache.org/jira/browse/CASSANDRA-7231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003658#comment-14003658 ] Aleksey Yeschenko commented on CASSANDRA-7231: -- My point is that you already have to make similar version-dependent changes in the driver to support native protocol v3 (collections serialization, for one). So changing the id from 1 byte to 2 bytes is nothing. You don't even have to change the ID generator, really, and keep generating [0,127] ids, and only switch to generating 2-byte ones when you feel like (as a driver author). It's an extremely tiny change, and is simpler than flags. And yes, a 9 byte header is uglier than an 8 byte header - but that's not really an argument, if it has no further consequences (: Either way, I can live with the flag-way as well - the main point is that one way or another, the core issue (too many available ids) gets fixed. So I won't fight the flag-way, but it is my preference that we go the fixed-2 bytes way. Feel free to override me if you preference is stronger, or involve the majority, either way it's fine by me. Support more concurrent requests per native transport connection Key: CASSANDRA-7231 URL: https://issues.apache.org/jira/browse/CASSANDRA-7231 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Benedict Assignee: Sylvain Lebresne Priority: Minor Fix For: 2.1.0 Attachments: 7231.txt Right now we only support 127 concurrent requests against a given native transport connection. This causes us to waste file handles opening multiple connections, increases driver complexity and dilutes writes across multiple connections so that batching cannot easily be performed. I propose raising this limit substantially, to somewhere in the region of 16-64K, and that this is a good time to do it since we're already bumping the protocol version. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7255) Rename nodetool cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-7255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003684#comment-14003684 ] Brandon Williams commented on CASSANDRA-7255: - bq. That said, the one downside of changing it is that it might break existing user scripts and require fixing all docs Well, presumably that extends further as we'd rename the jmx method as well, so not just scripts but also anything using jmx. Rename nodetool cleanup --- Key: CASSANDRA-7255 URL: https://issues.apache.org/jira/browse/CASSANDRA-7255 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Tyler Hobbs Priority: Trivial A lot of cassandra users seem to think nodetool cleanup does all kinds of things that it doesn't. For example, you'll find a lot of quotes like this in IRC and on the mailing list: bq. you need to run 'nodetool cleanup' to make sure all tombstones are deleted Renaming it to something clearer could help cut down on the confusion. Perhaps nodetool cleanranges or nodetool trim. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7255) Rename nodetool cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-7255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003776#comment-14003776 ] Tyler Hobbs commented on CASSANDRA-7255: What about hiding cleanup and the old JMX methods but continuing to accept them for a while (maybe with a deprecation warning message)? Rename nodetool cleanup --- Key: CASSANDRA-7255 URL: https://issues.apache.org/jira/browse/CASSANDRA-7255 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Tyler Hobbs Priority: Trivial A lot of cassandra users seem to think nodetool cleanup does all kinds of things that it doesn't. For example, you'll find a lot of quotes like this in IRC and on the mailing list: bq. you need to run 'nodetool cleanup' to make sure all tombstones are deleted Renaming it to something clearer could help cut down on the confusion. Perhaps nodetool cleanranges or nodetool trim. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7011) auth_test system_auth_ks_is_alterable_test dtest hangs in 2.1 and 2.0
[
https://issues.apache.org/jira/browse/CASSANDRA-7011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003784#comment-14003784
]
Aleksey Yeschenko commented on CASSANDRA-7011:
--
[~mshuler] are the tests still hanging?
auth_test system_auth_ks_is_alterable_test dtest hangs in 2.1 and 2.0
-
Key: CASSANDRA-7011
URL: https://issues.apache.org/jira/browse/CASSANDRA-7011
Project: Cassandra
Issue Type: Test
Components: Tests
Reporter: Michael Shuler
Assignee: Aleksey Yeschenko
Fix For: 2.1 rc1
This test hangs forever. When I hit ctl-c after running the test, then the
ccm nodes actually continue running - I think ccm is looking for log lines
that never occur until the test is killed(?).
{noformat}
$ export MAX_HEAP_SIZE=1G; export HEAP_NEWSIZE=256M; ENABLE_VNODES=true
PRINT_DEBUG=true nosetests --nocapture --nologcapture --verbosity=3
auth_test.py:TestAuth.system_auth_ks_is_alterable_test
nose.config: INFO: Ignoring files matching ['^\\.', '^_', '^setup\\.py$']
system_auth_ks_is_alterable_test (auth_test.TestAuth) ... cluster ccm
directory: /tmp/dtest-O3AAJr
^C
{noformat}
Search for (hanging here) below - I typed this prior to hitting ctl-c. Then
the nodes start running again and I see Listening for thrift clients later
on.
{noformat}
mshuler@hana:~$ tail -f /tmp/dtest-O3AAJr/test/node*/logs/system.log
== /tmp/dtest-O3AAJr/test/node1/logs/system.log ==
INFO [MemtableFlushWriter:2] 2014-04-08 16:45:12,599 Memtable.java:344 -
Writing Memtable-schema_columnfamilies@1792243696(1627 serialized bytes, 27
ops, 0%/0% of on/off-heap limit)
INFO [CompactionExecutor:2] 2014-04-08 16:45:12,603 CompactionTask.java:287
- Compacted 4 sstables to
[/tmp/dtest-O3AAJr/test/node1/data/system/schema_columns-296e9c049bec3085827dc17d3df2122a/system-schema_columns-ka-13,].
14,454 bytes to 11,603 (~80% of original) in 105ms = 0.105386MB/s. 7 total
partitions merged to 3. Partition merge counts were {1:1, 2:1, 4:1, }
INFO [MemtableFlushWriter:2] 2014-04-08 16:45:12,668 Memtable.java:378 -
Completed flushing
/tmp/dtest-O3AAJr/test/node1/flush/system/schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697/system-schema_columnfamilies-ka-14-Data.db
(956 bytes) for commitlog position ReplayPosition(segmentId=1396993504671,
position=193292)
INFO [MigrationStage:1] 2014-04-08 16:45:12,669 ColumnFamilyStore.java:853 -
Enqueuing flush of schema_columns: 6806 (0%) on-heap, 0 (0%) off-heap
INFO [MemtableFlushWriter:1] 2014-04-08 16:45:12,670 Memtable.java:344 -
Writing Memtable-schema_columns@352928691(1014 serialized bytes, 21 ops,
0%/0% of on/off-heap limit)
INFO [CompactionExecutor:1] 2014-04-08 16:45:12,672 CompactionTask.java:287
- Compacted 4 sstables to
[/tmp/dtest-O3AAJr/test/node1/data/system/schema_keyspaces-b0f2235744583cdb9631c43e59ce3676/system-schema_keyspaces-ka-17,].
710 bytes to 233 (~32% of original) in 70ms = 0.003174MB/s. 6 total
partitions merged to 3. Partition merge counts were {1:2, 4:1, }
INFO [MemtableFlushWriter:1] 2014-04-08 16:45:12,721 Memtable.java:378 -
Completed flushing
/tmp/dtest-O3AAJr/test/node1/flush/system/schema_columns-296e9c049bec3085827dc17d3df2122a/system-schema_columns-ka-14-Data.db
(435 bytes) for commitlog position ReplayPosition(segmentId=1396993504671,
position=193830)
WARN [NonPeriodicTasks:1] 2014-04-08 16:45:20,566 FBUtilities.java:359 -
Trigger directory doesn't exist, please create it and try again.
INFO [NonPeriodicTasks:1] 2014-04-08 16:45:20,570
PasswordAuthenticator.java:220 - PasswordAuthenticator created default user
'cassandra'
INFO [NonPeriodicTasks:1] 2014-04-08 16:45:21,806 Auth.java:232 - Created
default superuser 'cassandra'
== /tmp/dtest-O3AAJr/test/node2/logs/system.log ==
INFO [InternalResponseStage:4] 2014-04-08 16:45:12,214
ColumnFamilyStore.java:853 - Enqueuing flush of schema_keyspaces: 1004 (0%)
on-heap, 0 (0%) off-heap
INFO [MemtableFlushWriter:1] 2014-04-08 16:45:12,215 Memtable.java:344 -
Writing Memtable-schema_keyspaces@781373873(276 serialized bytes, 6 ops,
0%/0% of on/off-heap limit)
INFO [MemtableFlushWriter:1] 2014-04-08 16:45:12,295 Memtable.java:378 -
Completed flushing
/tmp/dtest-O3AAJr/test/node2/flush/system/schema_keyspaces-b0f2235744583cdb9631c43e59ce3676/system-schema_keyspaces-ka-15-Data.db
(179 bytes) for commitlog position ReplayPosition(segmentId=1396993504760,
position=243552)
INFO [InternalResponseStage:4] 2014-04-08 16:45:12,296
ColumnFamilyStore.java:853 - Enqueuing flush of schema_columnfamilies: 34190
(0%) on-heap, 0 (0%) off-heap
INFO [MemtableFlushWriter:2] 2014-04-08 16:45:12,297 Memtable.java:344 -
Writing
[jira] [Commented] (CASSANDRA-6950) Secondary index query fails with tc range query when ordered by DESC
[
https://issues.apache.org/jira/browse/CASSANDRA-6950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003788#comment-14003788
]
Tyler Hobbs commented on CASSANDRA-6950:
The Thrift behavior is such an uncommon case that I'm not concerned about
fixing it, I just thought it shouldn't get in the way of making our preferred
fix. If you prefer the current patch over changing
{{ExtendedFilter.isSatisfiedBy()}}, then I'm +1 on the patch.
Secondary index query fails with tc range query when ordered by DESC
Key: CASSANDRA-6950
URL: https://issues.apache.org/jira/browse/CASSANDRA-6950
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: RHEL 6.3 virtual guest,
apache-cassandra-2.0.6-SNAPSHOT-src.tar.gz from build #284 (also tried with
2.0.5 with CASSANDRA- patch custom-applied with same result).
Reporter: Andre Campeau
Assignee: Sylvain Lebresne
Fix For: 2.0.8
Attachments: 6950-pycassa-repro.py, 6950.txt
create table test4 ( name text, lname text, tc bigint, record text,
PRIMARY KEY ((name, lname), tc)) WITH CLUSTERING ORDER BY (tc DESC) AND
compaction={'class': 'LeveledCompactionStrategy'};
create index test4_index ON test4(lname);
Populate it with some data and non-zero tc values, then try:
select * from test4 where lname='blah' and tc0 allow filtering;
And, (0 rows) returned, even though there are rows which should be found.
When I create the table using CLUSTERING ORDER BY (tc ASC), the above query
works. Rows are correctly returned based on the range check.
Tried various combinations but with descending order on tc nothing works.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-6950) Secondary index query fails with tc range query when ordered by DESC
[
https://issues.apache.org/jira/browse/CASSANDRA-6950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003794#comment-14003794
]
Aleksey Yeschenko commented on CASSANDRA-6950:
--
Can we have both? Fix it in the logical place, and then break on the thrift
side, to preserve the current behavior? Thus special-casing thrift logic, not
CQL?
Secondary index query fails with tc range query when ordered by DESC
Key: CASSANDRA-6950
URL: https://issues.apache.org/jira/browse/CASSANDRA-6950
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: RHEL 6.3 virtual guest,
apache-cassandra-2.0.6-SNAPSHOT-src.tar.gz from build #284 (also tried with
2.0.5 with CASSANDRA- patch custom-applied with same result).
Reporter: Andre Campeau
Assignee: Sylvain Lebresne
Fix For: 2.0.8
Attachments: 6950-pycassa-repro.py, 6950.txt
create table test4 ( name text, lname text, tc bigint, record text,
PRIMARY KEY ((name, lname), tc)) WITH CLUSTERING ORDER BY (tc DESC) AND
compaction={'class': 'LeveledCompactionStrategy'};
create index test4_index ON test4(lname);
Populate it with some data and non-zero tc values, then try:
select * from test4 where lname='blah' and tc0 allow filtering;
And, (0 rows) returned, even though there are rows which should be found.
When I create the table using CLUSTERING ORDER BY (tc ASC), the above query
works. Rows are correctly returned based on the range check.
Tried various combinations but with descending order on tc nothing works.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7011) auth_test system_auth_ks_is_alterable_test dtest hangs in 2.1 and 2.0
[
https://issues.apache.org/jira/browse/CASSANDRA-7011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003815#comment-14003815
]
Michael Shuler commented on CASSANDRA-7011:
---
This test does still hang (and is being skipped). TBH, I still need to spend
time looking at this, and I'm not sure if it is a Cassandra issue, a test
issue, or ccm.
auth_test system_auth_ks_is_alterable_test dtest hangs in 2.1 and 2.0
-
Key: CASSANDRA-7011
URL: https://issues.apache.org/jira/browse/CASSANDRA-7011
Project: Cassandra
Issue Type: Test
Components: Tests
Reporter: Michael Shuler
Assignee: Aleksey Yeschenko
Fix For: 2.1 rc1
This test hangs forever. When I hit ctl-c after running the test, then the
ccm nodes actually continue running - I think ccm is looking for log lines
that never occur until the test is killed(?).
{noformat}
$ export MAX_HEAP_SIZE=1G; export HEAP_NEWSIZE=256M; ENABLE_VNODES=true
PRINT_DEBUG=true nosetests --nocapture --nologcapture --verbosity=3
auth_test.py:TestAuth.system_auth_ks_is_alterable_test
nose.config: INFO: Ignoring files matching ['^\\.', '^_', '^setup\\.py$']
system_auth_ks_is_alterable_test (auth_test.TestAuth) ... cluster ccm
directory: /tmp/dtest-O3AAJr
^C
{noformat}
Search for (hanging here) below - I typed this prior to hitting ctl-c. Then
the nodes start running again and I see Listening for thrift clients later
on.
{noformat}
mshuler@hana:~$ tail -f /tmp/dtest-O3AAJr/test/node*/logs/system.log
== /tmp/dtest-O3AAJr/test/node1/logs/system.log ==
INFO [MemtableFlushWriter:2] 2014-04-08 16:45:12,599 Memtable.java:344 -
Writing Memtable-schema_columnfamilies@1792243696(1627 serialized bytes, 27
ops, 0%/0% of on/off-heap limit)
INFO [CompactionExecutor:2] 2014-04-08 16:45:12,603 CompactionTask.java:287
- Compacted 4 sstables to
[/tmp/dtest-O3AAJr/test/node1/data/system/schema_columns-296e9c049bec3085827dc17d3df2122a/system-schema_columns-ka-13,].
14,454 bytes to 11,603 (~80% of original) in 105ms = 0.105386MB/s. 7 total
partitions merged to 3. Partition merge counts were {1:1, 2:1, 4:1, }
INFO [MemtableFlushWriter:2] 2014-04-08 16:45:12,668 Memtable.java:378 -
Completed flushing
/tmp/dtest-O3AAJr/test/node1/flush/system/schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697/system-schema_columnfamilies-ka-14-Data.db
(956 bytes) for commitlog position ReplayPosition(segmentId=1396993504671,
position=193292)
INFO [MigrationStage:1] 2014-04-08 16:45:12,669 ColumnFamilyStore.java:853 -
Enqueuing flush of schema_columns: 6806 (0%) on-heap, 0 (0%) off-heap
INFO [MemtableFlushWriter:1] 2014-04-08 16:45:12,670 Memtable.java:344 -
Writing Memtable-schema_columns@352928691(1014 serialized bytes, 21 ops,
0%/0% of on/off-heap limit)
INFO [CompactionExecutor:1] 2014-04-08 16:45:12,672 CompactionTask.java:287
- Compacted 4 sstables to
[/tmp/dtest-O3AAJr/test/node1/data/system/schema_keyspaces-b0f2235744583cdb9631c43e59ce3676/system-schema_keyspaces-ka-17,].
710 bytes to 233 (~32% of original) in 70ms = 0.003174MB/s. 6 total
partitions merged to 3. Partition merge counts were {1:2, 4:1, }
INFO [MemtableFlushWriter:1] 2014-04-08 16:45:12,721 Memtable.java:378 -
Completed flushing
/tmp/dtest-O3AAJr/test/node1/flush/system/schema_columns-296e9c049bec3085827dc17d3df2122a/system-schema_columns-ka-14-Data.db
(435 bytes) for commitlog position ReplayPosition(segmentId=1396993504671,
position=193830)
WARN [NonPeriodicTasks:1] 2014-04-08 16:45:20,566 FBUtilities.java:359 -
Trigger directory doesn't exist, please create it and try again.
INFO [NonPeriodicTasks:1] 2014-04-08 16:45:20,570
PasswordAuthenticator.java:220 - PasswordAuthenticator created default user
'cassandra'
INFO [NonPeriodicTasks:1] 2014-04-08 16:45:21,806 Auth.java:232 - Created
default superuser 'cassandra'
== /tmp/dtest-O3AAJr/test/node2/logs/system.log ==
INFO [InternalResponseStage:4] 2014-04-08 16:45:12,214
ColumnFamilyStore.java:853 - Enqueuing flush of schema_keyspaces: 1004 (0%)
on-heap, 0 (0%) off-heap
INFO [MemtableFlushWriter:1] 2014-04-08 16:45:12,215 Memtable.java:344 -
Writing Memtable-schema_keyspaces@781373873(276 serialized bytes, 6 ops,
0%/0% of on/off-heap limit)
INFO [MemtableFlushWriter:1] 2014-04-08 16:45:12,295 Memtable.java:378 -
Completed flushing
/tmp/dtest-O3AAJr/test/node2/flush/system/schema_keyspaces-b0f2235744583cdb9631c43e59ce3676/system-schema_keyspaces-ka-15-Data.db
(179 bytes) for commitlog position ReplayPosition(segmentId=1396993504760,
position=243552)
INFO [InternalResponseStage:4] 2014-04-08 16:45:12,296
ColumnFamilyStore.java:853 - Enqueuing flush of schema_columnfamilies: 34190
(0%) on-heap, 0 (0%) off-heap
[jira] [Updated] (CASSANDRA-7011) auth_test system_auth_ks_is_alterable_test dtest hangs in 2.1 and 2.0
[
https://issues.apache.org/jira/browse/CASSANDRA-7011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-7011:
-
Assignee: Michael Shuler (was: Aleksey Yeschenko)
auth_test system_auth_ks_is_alterable_test dtest hangs in 2.1 and 2.0
-
Key: CASSANDRA-7011
URL: https://issues.apache.org/jira/browse/CASSANDRA-7011
Project: Cassandra
Issue Type: Test
Components: Tests
Reporter: Michael Shuler
Assignee: Michael Shuler
Fix For: 2.1 rc1
This test hangs forever. When I hit ctl-c after running the test, then the
ccm nodes actually continue running - I think ccm is looking for log lines
that never occur until the test is killed(?).
{noformat}
$ export MAX_HEAP_SIZE=1G; export HEAP_NEWSIZE=256M; ENABLE_VNODES=true
PRINT_DEBUG=true nosetests --nocapture --nologcapture --verbosity=3
auth_test.py:TestAuth.system_auth_ks_is_alterable_test
nose.config: INFO: Ignoring files matching ['^\\.', '^_', '^setup\\.py$']
system_auth_ks_is_alterable_test (auth_test.TestAuth) ... cluster ccm
directory: /tmp/dtest-O3AAJr
^C
{noformat}
Search for (hanging here) below - I typed this prior to hitting ctl-c. Then
the nodes start running again and I see Listening for thrift clients later
on.
{noformat}
mshuler@hana:~$ tail -f /tmp/dtest-O3AAJr/test/node*/logs/system.log
== /tmp/dtest-O3AAJr/test/node1/logs/system.log ==
INFO [MemtableFlushWriter:2] 2014-04-08 16:45:12,599 Memtable.java:344 -
Writing Memtable-schema_columnfamilies@1792243696(1627 serialized bytes, 27
ops, 0%/0% of on/off-heap limit)
INFO [CompactionExecutor:2] 2014-04-08 16:45:12,603 CompactionTask.java:287
- Compacted 4 sstables to
[/tmp/dtest-O3AAJr/test/node1/data/system/schema_columns-296e9c049bec3085827dc17d3df2122a/system-schema_columns-ka-13,].
14,454 bytes to 11,603 (~80% of original) in 105ms = 0.105386MB/s. 7 total
partitions merged to 3. Partition merge counts were {1:1, 2:1, 4:1, }
INFO [MemtableFlushWriter:2] 2014-04-08 16:45:12,668 Memtable.java:378 -
Completed flushing
/tmp/dtest-O3AAJr/test/node1/flush/system/schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697/system-schema_columnfamilies-ka-14-Data.db
(956 bytes) for commitlog position ReplayPosition(segmentId=1396993504671,
position=193292)
INFO [MigrationStage:1] 2014-04-08 16:45:12,669 ColumnFamilyStore.java:853 -
Enqueuing flush of schema_columns: 6806 (0%) on-heap, 0 (0%) off-heap
INFO [MemtableFlushWriter:1] 2014-04-08 16:45:12,670 Memtable.java:344 -
Writing Memtable-schema_columns@352928691(1014 serialized bytes, 21 ops,
0%/0% of on/off-heap limit)
INFO [CompactionExecutor:1] 2014-04-08 16:45:12,672 CompactionTask.java:287
- Compacted 4 sstables to
[/tmp/dtest-O3AAJr/test/node1/data/system/schema_keyspaces-b0f2235744583cdb9631c43e59ce3676/system-schema_keyspaces-ka-17,].
710 bytes to 233 (~32% of original) in 70ms = 0.003174MB/s. 6 total
partitions merged to 3. Partition merge counts were {1:2, 4:1, }
INFO [MemtableFlushWriter:1] 2014-04-08 16:45:12,721 Memtable.java:378 -
Completed flushing
/tmp/dtest-O3AAJr/test/node1/flush/system/schema_columns-296e9c049bec3085827dc17d3df2122a/system-schema_columns-ka-14-Data.db
(435 bytes) for commitlog position ReplayPosition(segmentId=1396993504671,
position=193830)
WARN [NonPeriodicTasks:1] 2014-04-08 16:45:20,566 FBUtilities.java:359 -
Trigger directory doesn't exist, please create it and try again.
INFO [NonPeriodicTasks:1] 2014-04-08 16:45:20,570
PasswordAuthenticator.java:220 - PasswordAuthenticator created default user
'cassandra'
INFO [NonPeriodicTasks:1] 2014-04-08 16:45:21,806 Auth.java:232 - Created
default superuser 'cassandra'
== /tmp/dtest-O3AAJr/test/node2/logs/system.log ==
INFO [InternalResponseStage:4] 2014-04-08 16:45:12,214
ColumnFamilyStore.java:853 - Enqueuing flush of schema_keyspaces: 1004 (0%)
on-heap, 0 (0%) off-heap
INFO [MemtableFlushWriter:1] 2014-04-08 16:45:12,215 Memtable.java:344 -
Writing Memtable-schema_keyspaces@781373873(276 serialized bytes, 6 ops,
0%/0% of on/off-heap limit)
INFO [MemtableFlushWriter:1] 2014-04-08 16:45:12,295 Memtable.java:378 -
Completed flushing
/tmp/dtest-O3AAJr/test/node2/flush/system/schema_keyspaces-b0f2235744583cdb9631c43e59ce3676/system-schema_keyspaces-ka-15-Data.db
(179 bytes) for commitlog position ReplayPosition(segmentId=1396993504760,
position=243552)
INFO [InternalResponseStage:4] 2014-04-08 16:45:12,296
ColumnFamilyStore.java:853 - Enqueuing flush of schema_columnfamilies: 34190
(0%) on-heap, 0 (0%) off-heap
INFO [MemtableFlushWriter:2] 2014-04-08 16:45:12,297 Memtable.java:344 -
Writing Memtable-schema_columnfamilies@2077216447(5746 serialized bytes, 108
ops, 0%/0% of
[jira] [Commented] (CASSANDRA-7011) auth_test system_auth_ks_is_alterable_test dtest hangs in 2.1 and 2.0
[
https://issues.apache.org/jira/browse/CASSANDRA-7011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003820#comment-14003820
]
Aleksey Yeschenko commented on CASSANDRA-7011:
--
Reassigning to you for now, then.
auth_test system_auth_ks_is_alterable_test dtest hangs in 2.1 and 2.0
-
Key: CASSANDRA-7011
URL: https://issues.apache.org/jira/browse/CASSANDRA-7011
Project: Cassandra
Issue Type: Test
Components: Tests
Reporter: Michael Shuler
Assignee: Michael Shuler
Fix For: 2.1 rc1
This test hangs forever. When I hit ctl-c after running the test, then the
ccm nodes actually continue running - I think ccm is looking for log lines
that never occur until the test is killed(?).
{noformat}
$ export MAX_HEAP_SIZE=1G; export HEAP_NEWSIZE=256M; ENABLE_VNODES=true
PRINT_DEBUG=true nosetests --nocapture --nologcapture --verbosity=3
auth_test.py:TestAuth.system_auth_ks_is_alterable_test
nose.config: INFO: Ignoring files matching ['^\\.', '^_', '^setup\\.py$']
system_auth_ks_is_alterable_test (auth_test.TestAuth) ... cluster ccm
directory: /tmp/dtest-O3AAJr
^C
{noformat}
Search for (hanging here) below - I typed this prior to hitting ctl-c. Then
the nodes start running again and I see Listening for thrift clients later
on.
{noformat}
mshuler@hana:~$ tail -f /tmp/dtest-O3AAJr/test/node*/logs/system.log
== /tmp/dtest-O3AAJr/test/node1/logs/system.log ==
INFO [MemtableFlushWriter:2] 2014-04-08 16:45:12,599 Memtable.java:344 -
Writing Memtable-schema_columnfamilies@1792243696(1627 serialized bytes, 27
ops, 0%/0% of on/off-heap limit)
INFO [CompactionExecutor:2] 2014-04-08 16:45:12,603 CompactionTask.java:287
- Compacted 4 sstables to
[/tmp/dtest-O3AAJr/test/node1/data/system/schema_columns-296e9c049bec3085827dc17d3df2122a/system-schema_columns-ka-13,].
14,454 bytes to 11,603 (~80% of original) in 105ms = 0.105386MB/s. 7 total
partitions merged to 3. Partition merge counts were {1:1, 2:1, 4:1, }
INFO [MemtableFlushWriter:2] 2014-04-08 16:45:12,668 Memtable.java:378 -
Completed flushing
/tmp/dtest-O3AAJr/test/node1/flush/system/schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697/system-schema_columnfamilies-ka-14-Data.db
(956 bytes) for commitlog position ReplayPosition(segmentId=1396993504671,
position=193292)
INFO [MigrationStage:1] 2014-04-08 16:45:12,669 ColumnFamilyStore.java:853 -
Enqueuing flush of schema_columns: 6806 (0%) on-heap, 0 (0%) off-heap
INFO [MemtableFlushWriter:1] 2014-04-08 16:45:12,670 Memtable.java:344 -
Writing Memtable-schema_columns@352928691(1014 serialized bytes, 21 ops,
0%/0% of on/off-heap limit)
INFO [CompactionExecutor:1] 2014-04-08 16:45:12,672 CompactionTask.java:287
- Compacted 4 sstables to
[/tmp/dtest-O3AAJr/test/node1/data/system/schema_keyspaces-b0f2235744583cdb9631c43e59ce3676/system-schema_keyspaces-ka-17,].
710 bytes to 233 (~32% of original) in 70ms = 0.003174MB/s. 6 total
partitions merged to 3. Partition merge counts were {1:2, 4:1, }
INFO [MemtableFlushWriter:1] 2014-04-08 16:45:12,721 Memtable.java:378 -
Completed flushing
/tmp/dtest-O3AAJr/test/node1/flush/system/schema_columns-296e9c049bec3085827dc17d3df2122a/system-schema_columns-ka-14-Data.db
(435 bytes) for commitlog position ReplayPosition(segmentId=1396993504671,
position=193830)
WARN [NonPeriodicTasks:1] 2014-04-08 16:45:20,566 FBUtilities.java:359 -
Trigger directory doesn't exist, please create it and try again.
INFO [NonPeriodicTasks:1] 2014-04-08 16:45:20,570
PasswordAuthenticator.java:220 - PasswordAuthenticator created default user
'cassandra'
INFO [NonPeriodicTasks:1] 2014-04-08 16:45:21,806 Auth.java:232 - Created
default superuser 'cassandra'
== /tmp/dtest-O3AAJr/test/node2/logs/system.log ==
INFO [InternalResponseStage:4] 2014-04-08 16:45:12,214
ColumnFamilyStore.java:853 - Enqueuing flush of schema_keyspaces: 1004 (0%)
on-heap, 0 (0%) off-heap
INFO [MemtableFlushWriter:1] 2014-04-08 16:45:12,215 Memtable.java:344 -
Writing Memtable-schema_keyspaces@781373873(276 serialized bytes, 6 ops,
0%/0% of on/off-heap limit)
INFO [MemtableFlushWriter:1] 2014-04-08 16:45:12,295 Memtable.java:378 -
Completed flushing
/tmp/dtest-O3AAJr/test/node2/flush/system/schema_keyspaces-b0f2235744583cdb9631c43e59ce3676/system-schema_keyspaces-ka-15-Data.db
(179 bytes) for commitlog position ReplayPosition(segmentId=1396993504760,
position=243552)
INFO [InternalResponseStage:4] 2014-04-08 16:45:12,296
ColumnFamilyStore.java:853 - Enqueuing flush of schema_columnfamilies: 34190
(0%) on-heap, 0 (0%) off-heap
INFO [MemtableFlushWriter:2] 2014-04-08 16:45:12,297 Memtable.java:344 -
Writing Memtable-schema_columnfamilies@2077216447(5746
[jira] [Commented] (CASSANDRA-7011) auth_test system_auth_ks_is_alterable_test dtest hangs in 2.1 and 2.0
[
https://issues.apache.org/jira/browse/CASSANDRA-7011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003839#comment-14003839
]
Michael Shuler commented on CASSANDRA-7011:
---
adding a few debug lines, this is never getting out of
{{self.cluster.populate(nodes).start()}} in prepare() at:
https://github.com/riptano/cassandra-dtest/blob/master/auth_test.py#L502
auth_test system_auth_ks_is_alterable_test dtest hangs in 2.1 and 2.0
-
Key: CASSANDRA-7011
URL: https://issues.apache.org/jira/browse/CASSANDRA-7011
Project: Cassandra
Issue Type: Test
Components: Tests
Reporter: Michael Shuler
Assignee: Michael Shuler
Fix For: 2.1 rc1
This test hangs forever. When I hit ctl-c after running the test, then the
ccm nodes actually continue running - I think ccm is looking for log lines
that never occur until the test is killed(?).
{noformat}
$ export MAX_HEAP_SIZE=1G; export HEAP_NEWSIZE=256M; ENABLE_VNODES=true
PRINT_DEBUG=true nosetests --nocapture --nologcapture --verbosity=3
auth_test.py:TestAuth.system_auth_ks_is_alterable_test
nose.config: INFO: Ignoring files matching ['^\\.', '^_', '^setup\\.py$']
system_auth_ks_is_alterable_test (auth_test.TestAuth) ... cluster ccm
directory: /tmp/dtest-O3AAJr
^C
{noformat}
Search for (hanging here) below - I typed this prior to hitting ctl-c. Then
the nodes start running again and I see Listening for thrift clients later
on.
{noformat}
mshuler@hana:~$ tail -f /tmp/dtest-O3AAJr/test/node*/logs/system.log
== /tmp/dtest-O3AAJr/test/node1/logs/system.log ==
INFO [MemtableFlushWriter:2] 2014-04-08 16:45:12,599 Memtable.java:344 -
Writing Memtable-schema_columnfamilies@1792243696(1627 serialized bytes, 27
ops, 0%/0% of on/off-heap limit)
INFO [CompactionExecutor:2] 2014-04-08 16:45:12,603 CompactionTask.java:287
- Compacted 4 sstables to
[/tmp/dtest-O3AAJr/test/node1/data/system/schema_columns-296e9c049bec3085827dc17d3df2122a/system-schema_columns-ka-13,].
14,454 bytes to 11,603 (~80% of original) in 105ms = 0.105386MB/s. 7 total
partitions merged to 3. Partition merge counts were {1:1, 2:1, 4:1, }
INFO [MemtableFlushWriter:2] 2014-04-08 16:45:12,668 Memtable.java:378 -
Completed flushing
/tmp/dtest-O3AAJr/test/node1/flush/system/schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697/system-schema_columnfamilies-ka-14-Data.db
(956 bytes) for commitlog position ReplayPosition(segmentId=1396993504671,
position=193292)
INFO [MigrationStage:1] 2014-04-08 16:45:12,669 ColumnFamilyStore.java:853 -
Enqueuing flush of schema_columns: 6806 (0%) on-heap, 0 (0%) off-heap
INFO [MemtableFlushWriter:1] 2014-04-08 16:45:12,670 Memtable.java:344 -
Writing Memtable-schema_columns@352928691(1014 serialized bytes, 21 ops,
0%/0% of on/off-heap limit)
INFO [CompactionExecutor:1] 2014-04-08 16:45:12,672 CompactionTask.java:287
- Compacted 4 sstables to
[/tmp/dtest-O3AAJr/test/node1/data/system/schema_keyspaces-b0f2235744583cdb9631c43e59ce3676/system-schema_keyspaces-ka-17,].
710 bytes to 233 (~32% of original) in 70ms = 0.003174MB/s. 6 total
partitions merged to 3. Partition merge counts were {1:2, 4:1, }
INFO [MemtableFlushWriter:1] 2014-04-08 16:45:12,721 Memtable.java:378 -
Completed flushing
/tmp/dtest-O3AAJr/test/node1/flush/system/schema_columns-296e9c049bec3085827dc17d3df2122a/system-schema_columns-ka-14-Data.db
(435 bytes) for commitlog position ReplayPosition(segmentId=1396993504671,
position=193830)
WARN [NonPeriodicTasks:1] 2014-04-08 16:45:20,566 FBUtilities.java:359 -
Trigger directory doesn't exist, please create it and try again.
INFO [NonPeriodicTasks:1] 2014-04-08 16:45:20,570
PasswordAuthenticator.java:220 - PasswordAuthenticator created default user
'cassandra'
INFO [NonPeriodicTasks:1] 2014-04-08 16:45:21,806 Auth.java:232 - Created
default superuser 'cassandra'
== /tmp/dtest-O3AAJr/test/node2/logs/system.log ==
INFO [InternalResponseStage:4] 2014-04-08 16:45:12,214
ColumnFamilyStore.java:853 - Enqueuing flush of schema_keyspaces: 1004 (0%)
on-heap, 0 (0%) off-heap
INFO [MemtableFlushWriter:1] 2014-04-08 16:45:12,215 Memtable.java:344 -
Writing Memtable-schema_keyspaces@781373873(276 serialized bytes, 6 ops,
0%/0% of on/off-heap limit)
INFO [MemtableFlushWriter:1] 2014-04-08 16:45:12,295 Memtable.java:378 -
Completed flushing
/tmp/dtest-O3AAJr/test/node2/flush/system/schema_keyspaces-b0f2235744583cdb9631c43e59ce3676/system-schema_keyspaces-ka-15-Data.db
(179 bytes) for commitlog position ReplayPosition(segmentId=1396993504760,
position=243552)
INFO [InternalResponseStage:4] 2014-04-08 16:45:12,296
ColumnFamilyStore.java:853 - Enqueuing flush of schema_columnfamilies: 34190
(0%) on-heap, 0
[jira] [Commented] (CASSANDRA-6525) Cannot select data which using WHERE
[
https://issues.apache.org/jira/browse/CASSANDRA-6525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003874#comment-14003874
]
Tyler Hobbs commented on CASSANDRA-6525:
bq. wouldn't make sense to do also invalidate for truncate in
CFS.truncateBlocking, just to be on the safe side?
Truncates don't reset the SSTable generation counter
({{CFS.fileIndexGenerator}}), so new tables will have different generation
numbers (and hence different key cache keys).
Cannot select data which using WHERE
--
Key: CASSANDRA-6525
URL: https://issues.apache.org/jira/browse/CASSANDRA-6525
Project: Cassandra
Issue Type: Bug
Environment: Linux RHEL5
RAM: 1GB
Cassandra 2.0.3
CQL spec 3.1.1
Thrift protocol 19.38.0
Reporter: Silence Chow
Assignee: Tyler Hobbs
Fix For: 2.0.8
Attachments: 6525-2.0.txt, 6981_test.py
I am developing a system on my single machine using VMware Player with 1GB
Ram and 1Gb HHD. When I select all data, I didn't have any problems. But when
I using WHERE and it has just below 10 records. I have got this error in
system log:
{noformat}
ERROR [ReadStage:41] 2013-12-25 18:52:11,913 CassandraDaemon.java (line 187)
Exception in thread Thread[ReadStage:41,5,main]
java.io.IOError: java.io.EOFException
at org.apache.cassandra.db.Column$1.computeNext(Column.java:79)
at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:88)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:37)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:82)
at
org.apache.cassandra.db.filter.QueryFilter$2.getNext(QueryFilter.java:157)
at
org.apache.cassandra.db.filter.QueryFilter$2.hasNext(QueryFilter.java:140)
at
org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:144)
at
org.apache.cassandra.utils.MergeIterator$ManyToOne.init(MergeIterator.java:87)
at org.apache.cassandra.utils.MergeIterator.get(MergeIterator.java:46)
at
org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:120)
at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:72)
at
org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:297)
at
org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:53)
at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1487)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1306)
at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:332)
at
org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:65)
at
org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1401)
at
org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1936)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.io.EOFException
at java.io.RandomAccessFile.readFully(Unknown Source)
at java.io.RandomAccessFile.readFully(Unknown Source)
at
org.apache.cassandra.io.util.RandomAccessReader.readBytes(RandomAccessReader.java:348)
at
org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:392)
at
org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:371)
at
org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:74)
at org.apache.cassandra.db.Column$1.computeNext(Column.java:75)
... 27 more
{noformat}
E.g.
{{SELECT * FROM table;}}
Its fine.
{{SELECT * FROM table WHERE field = 'N';}}
field is the partition key.
Its said Request did not complete within rpc_timeout. in cqlsh
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-6851) Improve anticompaction after incremental repair
[ https://issues.apache.org/jira/browse/CASSANDRA-6851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003896#comment-14003896 ] Russell Alexander Spitzer commented on CASSANDRA-6851: -- Ah I see, that makes a lot more sense. We could always try to do a comparison of bloom filters for sstables, and try to anti-compact pairs with the most similar filters? IE: if dotproduct(bloomfilters) some threshold : place in the same anticompaction group. Improve anticompaction after incremental repair --- Key: CASSANDRA-6851 URL: https://issues.apache.org/jira/browse/CASSANDRA-6851 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Marcus Eriksson Assignee: Russell Alexander Spitzer Priority: Minor Labels: compaction, lhf Fix For: 2.1.1 After an incremental repair we iterate over all sstables and split them in two parts, one containing the repaired data and one the unrepaired. We could in theory double the number of sstables on a node. To avoid this we could make anticompaction also do a compaction, for example, if we are to anticompact 10 sstables, we could anticompact those to 2. Note that we need to avoid creating too big sstables though, if we anticompact all sstables on a node it would essentially be a major compaction. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7023) Nodetool clearsnapshot should not remove all snapshots by default
[ https://issues.apache.org/jira/browse/CASSANDRA-7023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003906#comment-14003906 ] Evin Callahan commented on CASSANDRA-7023: -- Also being affected by this, in the context of repairs. Would love to see a simple option that removes only used-based snapshots or ignores repair based ones, or something more flexible. Nodetool clearsnapshot should not remove all snapshots by default - Key: CASSANDRA-7023 URL: https://issues.apache.org/jira/browse/CASSANDRA-7023 Project: Cassandra Issue Type: New Feature Components: Tools Reporter: Sucwinder Bassi Priority: Minor Running a nodetool clearsnapshot will remove all snapshot files by default. As this is removing data this shouldn't just go away and just remove all snapshots. If you want to remove all snapshots there should be a force option. A list option showing snapshots available would also be helpful which could be piped into the nodetool clearsnapshot command. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6525) Cannot select data which using WHERE
[
https://issues.apache.org/jira/browse/CASSANDRA-6525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003915#comment-14003915
]
Vladimir Kuptsov commented on CASSANDRA-6525:
-
We have a cluster with 5 nodes in one DC and a cluster with two nodes in the
other without a replication between these datacenters. In all DC we use C*
2.0.5.
Today we've found a bug with similar messages but with the different result. We
have dropped and recreated one table in the DC with 5 nodes and just truncated
the same table in another DC.
After ~10 hours we have noticed appearing of the following messages in the
first DC logs:
```
ERROR [ReadStage:231469] 2014-05-20 21:05:20,349 CassandraDaemon.java (line
192) Exception in thread Thread[ReadStage:231469,5,main]
java.io.IOError: java.io.EOFException
at org.apache.cassandra.db.Column$1.computeNext(Column.java:79)
at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:88)
:
```.
For the node, on which this messages started, we found several messages like
INFO [GossipTasks:1] 2014-05-20 21:20:31,864 Gossiper.java (line 863)
InetAddress /10.33.20.91 is now DOWN
INFO [RequestResponseStage:10] 2014-05-20 21:20:32,186 Gossiper.java (line
849) InetAddress /10.33.20.91 is now UP
INFO [GossipTasks:1] 2014-05-20 21:26:51,965 Gossiper.java (line 863)
InetAddress /10.33.20.91 is now DOWN
and finally the node has stopped.
We found such effect only in the DC, where we have dropped and recreated table.
In the DC with truncate everything is OK.
Cannot select data which using WHERE
--
Key: CASSANDRA-6525
URL: https://issues.apache.org/jira/browse/CASSANDRA-6525
Project: Cassandra
Issue Type: Bug
Environment: Linux RHEL5
RAM: 1GB
Cassandra 2.0.3
CQL spec 3.1.1
Thrift protocol 19.38.0
Reporter: Silence Chow
Assignee: Tyler Hobbs
Fix For: 2.0.8
Attachments: 6525-2.0.txt, 6981_test.py
I am developing a system on my single machine using VMware Player with 1GB
Ram and 1Gb HHD. When I select all data, I didn't have any problems. But when
I using WHERE and it has just below 10 records. I have got this error in
system log:
{noformat}
ERROR [ReadStage:41] 2013-12-25 18:52:11,913 CassandraDaemon.java (line 187)
Exception in thread Thread[ReadStage:41,5,main]
java.io.IOError: java.io.EOFException
at org.apache.cassandra.db.Column$1.computeNext(Column.java:79)
at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:88)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:37)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:82)
at
org.apache.cassandra.db.filter.QueryFilter$2.getNext(QueryFilter.java:157)
at
org.apache.cassandra.db.filter.QueryFilter$2.hasNext(QueryFilter.java:140)
at
org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:144)
at
org.apache.cassandra.utils.MergeIterator$ManyToOne.init(MergeIterator.java:87)
at org.apache.cassandra.utils.MergeIterator.get(MergeIterator.java:46)
at
org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:120)
at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:72)
at
org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:297)
at
org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:53)
at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1487)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1306)
at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:332)
at
[jira] [Comment Edited] (CASSANDRA-6525) Cannot select data which using WHERE
[
https://issues.apache.org/jira/browse/CASSANDRA-6525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003915#comment-14003915
]
Vladimir Kuptsov edited comment on CASSANDRA-6525 at 5/20/14 8:16 PM:
--
We have a cluster with 5 nodes in one DC and a cluster with two nodes in the
other without a replication between these datacenters. In all DC we use C*
2.0.5.
Today we've found a bug with similar messages but with the different result. We
have dropped and recreated one table in the DC with 5 nodes and just truncated
the same table in another DC.
After ~10 hours we have noticed appearing of the following messages in the
first DC logs:
[code]
ERROR [ReadStage:231469] 2014-05-20 21:05:20,349 CassandraDaemon.java (line
192) Exception in thread Thread[ReadStage:231469,5,main]
java.io.IOError: java.io.EOFException
at org.apache.cassandra.db.Column$1.computeNext(Column.java:79)
at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:88)
:
[/code]
For the node, on which this messages started, we found several messages like
INFO [GossipTasks:1] 2014-05-20 21:20:31,864 Gossiper.java (line 863)
InetAddress /10.33.20.91 is now DOWN
INFO [RequestResponseStage:10] 2014-05-20 21:20:32,186 Gossiper.java (line
849) InetAddress /10.33.20.91 is now UP
INFO [GossipTasks:1] 2014-05-20 21:26:51,965 Gossiper.java (line 863)
InetAddress /10.33.20.91 is now DOWN
and finally the node has stopped.
We found such effect only in the DC, where we have dropped and recreated table.
In the DC with truncate everything is OK.
was (Author: vkuptcov):
We have a cluster with 5 nodes in one DC and a cluster with two nodes in the
other without a replication between these datacenters. In all DC we use C*
2.0.5.
Today we've found a bug with similar messages but with the different result. We
have dropped and recreated one table in the DC with 5 nodes and just truncated
the same table in another DC.
After ~10 hours we have noticed appearing of the following messages in the
first DC logs:
```
ERROR [ReadStage:231469] 2014-05-20 21:05:20,349 CassandraDaemon.java (line
192) Exception in thread Thread[ReadStage:231469,5,main]
java.io.IOError: java.io.EOFException
at org.apache.cassandra.db.Column$1.computeNext(Column.java:79)
at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:88)
:
```.
For the node, on which this messages started, we found several messages like
INFO [GossipTasks:1] 2014-05-20 21:20:31,864 Gossiper.java (line 863)
InetAddress /10.33.20.91 is now DOWN
INFO [RequestResponseStage:10] 2014-05-20 21:20:32,186 Gossiper.java (line
849) InetAddress /10.33.20.91 is now UP
INFO [GossipTasks:1] 2014-05-20 21:26:51,965 Gossiper.java (line 863)
InetAddress /10.33.20.91 is now DOWN
and finally the node has stopped.
We found such effect only in the DC, where we have dropped and recreated table.
In the DC with truncate everything is OK.
Cannot select data which using WHERE
--
Key: CASSANDRA-6525
URL: https://issues.apache.org/jira/browse/CASSANDRA-6525
Project: Cassandra
Issue Type: Bug
Environment: Linux RHEL5
RAM: 1GB
Cassandra 2.0.3
CQL spec 3.1.1
Thrift protocol 19.38.0
Reporter: Silence Chow
Assignee: Tyler Hobbs
Fix For: 2.0.8
Attachments: 6525-2.0.txt, 6981_test.py
I am developing a system on my single machine using VMware Player with 1GB
Ram and 1Gb HHD. When I select all data, I didn't have any problems. But when
I using WHERE and it has just below 10 records. I have got this error in
system log:
{noformat}
ERROR [ReadStage:41] 2013-12-25 18:52:11,913 CassandraDaemon.java (line 187)
Exception in thread Thread[ReadStage:41,5,main]
java.io.IOError: java.io.EOFException
at org.apache.cassandra.db.Column$1.computeNext(Column.java:79)
at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
[jira] [Comment Edited] (CASSANDRA-6525) Cannot select data which using WHERE
[
https://issues.apache.org/jira/browse/CASSANDRA-6525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003915#comment-14003915
]
Vladimir Kuptsov edited comment on CASSANDRA-6525 at 5/20/14 8:17 PM:
--
We have a cluster with 5 nodes in one DC and a cluster with two nodes in the
other without a replication between these datacenters. In all DC we use C*
2.0.5.
Today we've found a bug with similar messages but with the different result. We
have dropped and recreated one table in the DC with 5 nodes and just truncated
the same table in another DC.
After ~10 hours we have noticed appearing of the following messages in the
first DC logs:
{code}
ERROR [ReadStage:231469] 2014-05-20 21:05:20,349 CassandraDaemon.java (line
192) Exception in thread Thread[ReadStage:231469,5,main]
java.io.IOError: java.io.EOFException
at org.apache.cassandra.db.Column$1.computeNext(Column.java:79)
at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:88)
:
{code}
For the node, on which this messages started, we found several messages like
INFO [GossipTasks:1] 2014-05-20 21:20:31,864 Gossiper.java (line 863)
InetAddress /10.33.20.91 is now DOWN
INFO [RequestResponseStage:10] 2014-05-20 21:20:32,186 Gossiper.java (line
849) InetAddress /10.33.20.91 is now UP
INFO [GossipTasks:1] 2014-05-20 21:26:51,965 Gossiper.java (line 863)
InetAddress /10.33.20.91 is now DOWN
and finally the node has stopped.
We found such effect only in the DC, where we have dropped and recreated table.
In the DC with truncate everything is OK.
was (Author: vkuptcov):
We have a cluster with 5 nodes in one DC and a cluster with two nodes in the
other without a replication between these datacenters. In all DC we use C*
2.0.5.
Today we've found a bug with similar messages but with the different result. We
have dropped and recreated one table in the DC with 5 nodes and just truncated
the same table in another DC.
After ~10 hours we have noticed appearing of the following messages in the
first DC logs:
ERROR [ReadStage:231469] 2014-05-20 21:05:20,349 CassandraDaemon.java (line
192) Exception in thread Thread[ReadStage:231469,5,main]
java.io.IOError: java.io.EOFException
at org.apache.cassandra.db.Column$1.computeNext(Column.java:79)
at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:88)
:
For the node, on which this messages started, we found several messages like
INFO [GossipTasks:1] 2014-05-20 21:20:31,864 Gossiper.java (line 863)
InetAddress /10.33.20.91 is now DOWN
INFO [RequestResponseStage:10] 2014-05-20 21:20:32,186 Gossiper.java (line
849) InetAddress /10.33.20.91 is now UP
INFO [GossipTasks:1] 2014-05-20 21:26:51,965 Gossiper.java (line 863)
InetAddress /10.33.20.91 is now DOWN
and finally the node has stopped.
We found such effect only in the DC, where we have dropped and recreated table.
In the DC with truncate everything is OK.
Cannot select data which using WHERE
--
Key: CASSANDRA-6525
URL: https://issues.apache.org/jira/browse/CASSANDRA-6525
Project: Cassandra
Issue Type: Bug
Environment: Linux RHEL5
RAM: 1GB
Cassandra 2.0.3
CQL spec 3.1.1
Thrift protocol 19.38.0
Reporter: Silence Chow
Assignee: Tyler Hobbs
Fix For: 2.0.8
Attachments: 6525-2.0.txt, 6981_test.py
I am developing a system on my single machine using VMware Player with 1GB
Ram and 1Gb HHD. When I select all data, I didn't have any problems. But when
I using WHERE and it has just below 10 records. I have got this error in
system log:
{noformat}
ERROR [ReadStage:41] 2013-12-25 18:52:11,913 CassandraDaemon.java (line 187)
Exception in thread Thread[ReadStage:41,5,main]
java.io.IOError: java.io.EOFException
at org.apache.cassandra.db.Column$1.computeNext(Column.java:79)
at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
[jira] [Comment Edited] (CASSANDRA-6525) Cannot select data which using WHERE
[
https://issues.apache.org/jira/browse/CASSANDRA-6525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003915#comment-14003915
]
Vladimir Kuptsov edited comment on CASSANDRA-6525 at 5/20/14 8:18 PM:
--
We have a cluster with 5 nodes in one DC and a cluster with two nodes in the
other without a replication between these datacenters. In all DC we use C*
2.0.5.
Today we've found a bug with similar messages but with the different result. We
have dropped and recreated one table in the DC with 5 nodes and just truncated
the same table in another DC.
After ~10 hours we have noticed appearing of the following messages in the
first DC logs:
{code}
ERROR [ReadStage:231469] 2014-05-20 21:05:20,349 CassandraDaemon.java (line
192) Exception in thread Thread[ReadStage:231469,5,main]
java.io.IOError: java.io.EOFException
at org.apache.cassandra.db.Column$1.computeNext(Column.java:79)
at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:88)
:
{code}
For the node, on which this messages started, we found several messages like
{code}
INFO [GossipTasks:1] 2014-05-20 21:20:31,864 Gossiper.java (line 863)
InetAddress /10.33.20.91 is now DOWN
INFO [RequestResponseStage:10] 2014-05-20 21:20:32,186 Gossiper.java (line
849) InetAddress /10.33.20.91 is now UP
INFO [GossipTasks:1] 2014-05-20 21:26:51,965 Gossiper.java (line 863)
InetAddress /10.33.20.91 is now DOWN
{code}
and finally the node has stopped.
We found such effect only in the DC, where we have dropped and recreated table.
In the DC with truncate everything is OK.
was (Author: vkuptcov):
We have a cluster with 5 nodes in one DC and a cluster with two nodes in the
other without a replication between these datacenters. In all DC we use C*
2.0.5.
Today we've found a bug with similar messages but with the different result. We
have dropped and recreated one table in the DC with 5 nodes and just truncated
the same table in another DC.
After ~10 hours we have noticed appearing of the following messages in the
first DC logs:
{code}
ERROR [ReadStage:231469] 2014-05-20 21:05:20,349 CassandraDaemon.java (line
192) Exception in thread Thread[ReadStage:231469,5,main]
java.io.IOError: java.io.EOFException
at org.apache.cassandra.db.Column$1.computeNext(Column.java:79)
at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:88)
:
{code}
For the node, on which this messages started, we found several messages like
INFO [GossipTasks:1] 2014-05-20 21:20:31,864 Gossiper.java (line 863)
InetAddress /10.33.20.91 is now DOWN
INFO [RequestResponseStage:10] 2014-05-20 21:20:32,186 Gossiper.java (line
849) InetAddress /10.33.20.91 is now UP
INFO [GossipTasks:1] 2014-05-20 21:26:51,965 Gossiper.java (line 863)
InetAddress /10.33.20.91 is now DOWN
and finally the node has stopped.
We found such effect only in the DC, where we have dropped and recreated table.
In the DC with truncate everything is OK.
Cannot select data which using WHERE
--
Key: CASSANDRA-6525
URL: https://issues.apache.org/jira/browse/CASSANDRA-6525
Project: Cassandra
Issue Type: Bug
Environment: Linux RHEL5
RAM: 1GB
Cassandra 2.0.3
CQL spec 3.1.1
Thrift protocol 19.38.0
Reporter: Silence Chow
Assignee: Tyler Hobbs
Fix For: 2.0.8
Attachments: 6525-2.0.txt, 6981_test.py
I am developing a system on my single machine using VMware Player with 1GB
Ram and 1Gb HHD. When I select all data, I didn't have any problems. But when
I using WHERE and it has just below 10 records. I have got this error in
system log:
{noformat}
ERROR [ReadStage:41] 2013-12-25 18:52:11,913 CassandraDaemon.java (line 187)
Exception in thread Thread[ReadStage:41,5,main]
java.io.IOError: java.io.EOFException
at org.apache.cassandra.db.Column$1.computeNext(Column.java:79)
at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
[jira] [Comment Edited] (CASSANDRA-6525) Cannot select data which using WHERE
[
https://issues.apache.org/jira/browse/CASSANDRA-6525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003915#comment-14003915
]
Vladimir Kuptsov edited comment on CASSANDRA-6525 at 5/20/14 8:16 PM:
--
We have a cluster with 5 nodes in one DC and a cluster with two nodes in the
other without a replication between these datacenters. In all DC we use C*
2.0.5.
Today we've found a bug with similar messages but with the different result. We
have dropped and recreated one table in the DC with 5 nodes and just truncated
the same table in another DC.
After ~10 hours we have noticed appearing of the following messages in the
first DC logs:
ERROR [ReadStage:231469] 2014-05-20 21:05:20,349 CassandraDaemon.java (line
192) Exception in thread Thread[ReadStage:231469,5,main]
java.io.IOError: java.io.EOFException
at org.apache.cassandra.db.Column$1.computeNext(Column.java:79)
at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:88)
:
For the node, on which this messages started, we found several messages like
INFO [GossipTasks:1] 2014-05-20 21:20:31,864 Gossiper.java (line 863)
InetAddress /10.33.20.91 is now DOWN
INFO [RequestResponseStage:10] 2014-05-20 21:20:32,186 Gossiper.java (line
849) InetAddress /10.33.20.91 is now UP
INFO [GossipTasks:1] 2014-05-20 21:26:51,965 Gossiper.java (line 863)
InetAddress /10.33.20.91 is now DOWN
and finally the node has stopped.
We found such effect only in the DC, where we have dropped and recreated table.
In the DC with truncate everything is OK.
was (Author: vkuptcov):
We have a cluster with 5 nodes in one DC and a cluster with two nodes in the
other without a replication between these datacenters. In all DC we use C*
2.0.5.
Today we've found a bug with similar messages but with the different result. We
have dropped and recreated one table in the DC with 5 nodes and just truncated
the same table in another DC.
After ~10 hours we have noticed appearing of the following messages in the
first DC logs:
[code]
ERROR [ReadStage:231469] 2014-05-20 21:05:20,349 CassandraDaemon.java (line
192) Exception in thread Thread[ReadStage:231469,5,main]
java.io.IOError: java.io.EOFException
at org.apache.cassandra.db.Column$1.computeNext(Column.java:79)
at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:88)
:
[/code]
For the node, on which this messages started, we found several messages like
INFO [GossipTasks:1] 2014-05-20 21:20:31,864 Gossiper.java (line 863)
InetAddress /10.33.20.91 is now DOWN
INFO [RequestResponseStage:10] 2014-05-20 21:20:32,186 Gossiper.java (line
849) InetAddress /10.33.20.91 is now UP
INFO [GossipTasks:1] 2014-05-20 21:26:51,965 Gossiper.java (line 863)
InetAddress /10.33.20.91 is now DOWN
and finally the node has stopped.
We found such effect only in the DC, where we have dropped and recreated table.
In the DC with truncate everything is OK.
Cannot select data which using WHERE
--
Key: CASSANDRA-6525
URL: https://issues.apache.org/jira/browse/CASSANDRA-6525
Project: Cassandra
Issue Type: Bug
Environment: Linux RHEL5
RAM: 1GB
Cassandra 2.0.3
CQL spec 3.1.1
Thrift protocol 19.38.0
Reporter: Silence Chow
Assignee: Tyler Hobbs
Fix For: 2.0.8
Attachments: 6525-2.0.txt, 6981_test.py
I am developing a system on my single machine using VMware Player with 1GB
Ram and 1Gb HHD. When I select all data, I didn't have any problems. But when
I using WHERE and it has just below 10 records. I have got this error in
system log:
{noformat}
ERROR [ReadStage:41] 2013-12-25 18:52:11,913 CassandraDaemon.java (line 187)
Exception in thread Thread[ReadStage:41,5,main]
java.io.IOError: java.io.EOFException
at org.apache.cassandra.db.Column$1.computeNext(Column.java:79)
at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
[jira] [Comment Edited] (CASSANDRA-6525) Cannot select data which using WHERE
[
https://issues.apache.org/jira/browse/CASSANDRA-6525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003915#comment-14003915
]
Vladimir Kuptsov edited comment on CASSANDRA-6525 at 5/20/14 8:18 PM:
--
We have a cluster with 5 nodes in one DC and a cluster with two nodes in the
other without a replication between these datacenters. In all DC we use C*
2.0.5.
Today we've found a bug with similar messages but with the different result. We
have dropped and recreated one table in the DC with 5 nodes and just truncated
the same table in another DC.
After ~10 hours we have noticed appearing of the following messages in the
first DC logs:
{code}
ERROR [ReadStage:231469] 2014-05-20 21:05:20,349 CassandraDaemon.java (line
192) Exception in thread Thread[ReadStage:231469,5,main]
java.io.IOError: java.io.EOFException
at org.apache.cassandra.db.Column$1.computeNext(Column.java:79)
at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:88)
:
{code}
For the node, on which this messages started, we found several messages like on
the other nodes
{code}
INFO [GossipTasks:1] 2014-05-20 21:20:31,864 Gossiper.java (line 863)
InetAddress /10.33.20.91 is now DOWN
INFO [RequestResponseStage:10] 2014-05-20 21:20:32,186 Gossiper.java (line
849) InetAddress /10.33.20.91 is now UP
INFO [GossipTasks:1] 2014-05-20 21:26:51,965 Gossiper.java (line 863)
InetAddress /10.33.20.91 is now DOWN
{code}
and finally the node has stopped.
We found such effect only in the DC, where we have dropped and recreated table.
In the DC with truncate everything is OK.
was (Author: vkuptcov):
We have a cluster with 5 nodes in one DC and a cluster with two nodes in the
other without a replication between these datacenters. In all DC we use C*
2.0.5.
Today we've found a bug with similar messages but with the different result. We
have dropped and recreated one table in the DC with 5 nodes and just truncated
the same table in another DC.
After ~10 hours we have noticed appearing of the following messages in the
first DC logs:
{code}
ERROR [ReadStage:231469] 2014-05-20 21:05:20,349 CassandraDaemon.java (line
192) Exception in thread Thread[ReadStage:231469,5,main]
java.io.IOError: java.io.EOFException
at org.apache.cassandra.db.Column$1.computeNext(Column.java:79)
at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:88)
:
{code}
For the node, on which this messages started, we found several messages like
{code}
INFO [GossipTasks:1] 2014-05-20 21:20:31,864 Gossiper.java (line 863)
InetAddress /10.33.20.91 is now DOWN
INFO [RequestResponseStage:10] 2014-05-20 21:20:32,186 Gossiper.java (line
849) InetAddress /10.33.20.91 is now UP
INFO [GossipTasks:1] 2014-05-20 21:26:51,965 Gossiper.java (line 863)
InetAddress /10.33.20.91 is now DOWN
{code}
and finally the node has stopped.
We found such effect only in the DC, where we have dropped and recreated table.
In the DC with truncate everything is OK.
Cannot select data which using WHERE
--
Key: CASSANDRA-6525
URL: https://issues.apache.org/jira/browse/CASSANDRA-6525
Project: Cassandra
Issue Type: Bug
Environment: Linux RHEL5
RAM: 1GB
Cassandra 2.0.3
CQL spec 3.1.1
Thrift protocol 19.38.0
Reporter: Silence Chow
Assignee: Tyler Hobbs
Fix For: 2.0.8
Attachments: 6525-2.0.txt, 6981_test.py
I am developing a system on my single machine using VMware Player with 1GB
Ram and 1Gb HHD. When I select all data, I didn't have any problems. But when
I using WHERE and it has just below 10 records. I have got this error in
system log:
{noformat}
ERROR [ReadStage:41] 2013-12-25 18:52:11,913 CassandraDaemon.java (line 187)
Exception in thread Thread[ReadStage:41,5,main]
java.io.IOError: java.io.EOFException
at org.apache.cassandra.db.Column$1.computeNext(Column.java:79)
at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
[jira] [Commented] (CASSANDRA-7248) Tuple type
[
https://issues.apache.org/jira/browse/CASSANDRA-7248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003931#comment-14003931
]
Tyler Hobbs commented on CASSANDRA-7248:
Once CASSANDRA-7209 is complete, I'll give this a quick final review and then
we can work on extracting this for CASSANDRA-6875.
Tuple type
--
Key: CASSANDRA-7248
URL: https://issues.apache.org/jira/browse/CASSANDRA-7248
Project: Cassandra
Issue Type: Bug
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Labels: cql3
Fix For: 2.1 rc1
For CASSANDRA-6875 we need to be able to talk about tuples values and types
(for prepared variables). Since we need it there, clients will need to
support them anyway and so I think it would be a lot cleaner to start
supporting those more generally. Besides, having tuples is a relatively
simple and natural extension to what we have. I'll note in particular that
tuple have a close relationship to user type in the sense that a tuple will
be really just like an anonymous with no name for the fields and in
particular a tuple value will be the same than a user type value.
The syntax would simply look like that:
{noformat}
CREATE TABLE foo (
k int PRIMARY KEY,
v tupleint, text, float
)
INSERT INTO foo(k, v) VALUES(0, (3, 'bar', 2.1));
{noformat}
We can also add projections in selects if we want:
{noformat}
SELECT v[0], v[2] FROM foo WHERE k = 0;
{noformat}
but that can come later (after all, we still don't have projections for
collections and it's not a big deal).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-6525) Cannot select data which using WHERE
[
https://issues.apache.org/jira/browse/CASSANDRA-6525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003948#comment-14003948
]
Tyler Hobbs commented on CASSANDRA-6525:
[~vkuptcov] that seems consistent with what I found. I suggest invalidating
your key cache in the problematic DC. You can use {{nodetool
invalidatekeycache}} to do this.
Cannot select data which using WHERE
--
Key: CASSANDRA-6525
URL: https://issues.apache.org/jira/browse/CASSANDRA-6525
Project: Cassandra
Issue Type: Bug
Environment: Linux RHEL5
RAM: 1GB
Cassandra 2.0.3
CQL spec 3.1.1
Thrift protocol 19.38.0
Reporter: Silence Chow
Assignee: Tyler Hobbs
Fix For: 2.0.8
Attachments: 6525-2.0.txt, 6981_test.py
I am developing a system on my single machine using VMware Player with 1GB
Ram and 1Gb HHD. When I select all data, I didn't have any problems. But when
I using WHERE and it has just below 10 records. I have got this error in
system log:
{noformat}
ERROR [ReadStage:41] 2013-12-25 18:52:11,913 CassandraDaemon.java (line 187)
Exception in thread Thread[ReadStage:41,5,main]
java.io.IOError: java.io.EOFException
at org.apache.cassandra.db.Column$1.computeNext(Column.java:79)
at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:88)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:37)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:82)
at
org.apache.cassandra.db.filter.QueryFilter$2.getNext(QueryFilter.java:157)
at
org.apache.cassandra.db.filter.QueryFilter$2.hasNext(QueryFilter.java:140)
at
org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:144)
at
org.apache.cassandra.utils.MergeIterator$ManyToOne.init(MergeIterator.java:87)
at org.apache.cassandra.utils.MergeIterator.get(MergeIterator.java:46)
at
org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:120)
at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:72)
at
org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:297)
at
org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:53)
at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1487)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1306)
at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:332)
at
org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:65)
at
org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1401)
at
org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1936)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.io.EOFException
at java.io.RandomAccessFile.readFully(Unknown Source)
at java.io.RandomAccessFile.readFully(Unknown Source)
at
org.apache.cassandra.io.util.RandomAccessReader.readBytes(RandomAccessReader.java:348)
at
org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:392)
at
org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:371)
at
org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:74)
at org.apache.cassandra.db.Column$1.computeNext(Column.java:75)
... 27 more
{noformat}
E.g.
{{SELECT * FROM table;}}
Its fine.
{{SELECT * FROM table WHERE field = 'N';}}
field is the partition key.
Its said Request did not complete within rpc_timeout. in cqlsh
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (CASSANDRA-7273) expose global ColumnFamily metrics
Richard Wagner created CASSANDRA-7273: - Summary: expose global ColumnFamily metrics Key: CASSANDRA-7273 URL: https://issues.apache.org/jira/browse/CASSANDRA-7273 Project: Cassandra Issue Type: New Feature Reporter: Richard Wagner Priority: Minor It would be very useful to have cassandra expose ColumnFamily metrics that span all column families. A general purpose cassandra monitoring system built up around the current ColumnFamily metrics really only has a couple of choices right now: publish metrics for all column families or fetch metrics for all column families, aggregate them and then publish the aggregated metrics. The first can be quite expensive for the downstream monitoring system and the second is a piece of work that it seems is better pushed into cassandra itself. Perhaps these global ColumnFamily metrics could be published under a name of: org.apache.cassandra.metrics:type=(ColumnFamily|IndexColumnFamily),name=(Metric name) (Same as existing ColumnFamily metrics, but with no keyspace or scope.) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6525) Cannot select data which using WHERE
[
https://issues.apache.org/jira/browse/CASSANDRA-6525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003958#comment-14003958
]
Vladimir Kuptsov commented on CASSANDRA-6525:
-
Yes, it looks like this. We have deleted the data from
/var/lib/cassandra/saved_caches/* and after nodes restarting we don't notice
the mentioned exceptions.
Cannot select data which using WHERE
--
Key: CASSANDRA-6525
URL: https://issues.apache.org/jira/browse/CASSANDRA-6525
Project: Cassandra
Issue Type: Bug
Environment: Linux RHEL5
RAM: 1GB
Cassandra 2.0.3
CQL spec 3.1.1
Thrift protocol 19.38.0
Reporter: Silence Chow
Assignee: Tyler Hobbs
Fix For: 2.0.8
Attachments: 6525-2.0.txt, 6981_test.py
I am developing a system on my single machine using VMware Player with 1GB
Ram and 1Gb HHD. When I select all data, I didn't have any problems. But when
I using WHERE and it has just below 10 records. I have got this error in
system log:
{noformat}
ERROR [ReadStage:41] 2013-12-25 18:52:11,913 CassandraDaemon.java (line 187)
Exception in thread Thread[ReadStage:41,5,main]
java.io.IOError: java.io.EOFException
at org.apache.cassandra.db.Column$1.computeNext(Column.java:79)
at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:88)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:37)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:82)
at
org.apache.cassandra.db.filter.QueryFilter$2.getNext(QueryFilter.java:157)
at
org.apache.cassandra.db.filter.QueryFilter$2.hasNext(QueryFilter.java:140)
at
org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:144)
at
org.apache.cassandra.utils.MergeIterator$ManyToOne.init(MergeIterator.java:87)
at org.apache.cassandra.utils.MergeIterator.get(MergeIterator.java:46)
at
org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:120)
at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:72)
at
org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:297)
at
org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:53)
at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1487)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1306)
at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:332)
at
org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:65)
at
org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1401)
at
org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1936)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.io.EOFException
at java.io.RandomAccessFile.readFully(Unknown Source)
at java.io.RandomAccessFile.readFully(Unknown Source)
at
org.apache.cassandra.io.util.RandomAccessReader.readBytes(RandomAccessReader.java:348)
at
org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:392)
at
org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:371)
at
org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:74)
at org.apache.cassandra.db.Column$1.computeNext(Column.java:75)
... 27 more
{noformat}
E.g.
{{SELECT * FROM table;}}
Its fine.
{{SELECT * FROM table WHERE field = 'N';}}
field is the partition key.
Its said Request did not complete within rpc_timeout. in cqlsh
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7233) Dropping a keyspace fails to purge the Key Cache resulting in SSTable Corruption during searches
[ https://issues.apache.org/jira/browse/CASSANDRA-7233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003982#comment-14003982 ] Tyler Hobbs commented on CASSANDRA-7233: CASSANDRA-6525 will help will this in 2.0. Dropping a keyspace fails to purge the Key Cache resulting in SSTable Corruption during searches Key: CASSANDRA-7233 URL: https://issues.apache.org/jira/browse/CASSANDRA-7233 Project: Cassandra Issue Type: Bug Components: Core Reporter: Vinoo Ganesh Fix For: 1.2.17 Dropping a keyspace fails to purge the Key Cache resulting in SSTable corruption during searches. Dropping a full keyspace (with column families) and re-creating that same keyspace (with column families) without restarting Cassandra causes searches to print out CorruptSSTable messages. This has to do with the fact that keys that reference the deleted data persist in the key cache after the data has been deleted. Could we have the key cache automatically invalidated when for data that is dropped? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7233) Dropping a keyspace fails to purge the Key Cache resulting in SSTable Corruption during searches
[ https://issues.apache.org/jira/browse/CASSANDRA-7233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003982#comment-14003982 ] Tyler Hobbs edited comment on CASSANDRA-7233 at 5/20/14 9:17 PM: - CASSANDRA-6525 will help with this in 2.0. was (Author: thobbs): CASSANDRA-6525 will help will this in 2.0. Dropping a keyspace fails to purge the Key Cache resulting in SSTable Corruption during searches Key: CASSANDRA-7233 URL: https://issues.apache.org/jira/browse/CASSANDRA-7233 Project: Cassandra Issue Type: Bug Components: Core Reporter: Vinoo Ganesh Fix For: 1.2.17 Dropping a keyspace fails to purge the Key Cache resulting in SSTable corruption during searches. Dropping a full keyspace (with column families) and re-creating that same keyspace (with column families) without restarting Cassandra causes searches to print out CorruptSSTable messages. This has to do with the fact that keys that reference the deleted data persist in the key cache after the data has been deleted. Could we have the key cache automatically invalidated when for data that is dropped? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7273) expose global ColumnFamily metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-7273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14003986#comment-14003986 ] Brandon Williams commented on CASSANDRA-7273: - Sounds very close to CASSANDRA-6539 expose global ColumnFamily metrics -- Key: CASSANDRA-7273 URL: https://issues.apache.org/jira/browse/CASSANDRA-7273 Project: Cassandra Issue Type: New Feature Reporter: Richard Wagner Priority: Minor It would be very useful to have cassandra expose ColumnFamily metrics that span all column families. A general purpose cassandra monitoring system built up around the current ColumnFamily metrics really only has a couple of choices right now: publish metrics for all column families or fetch metrics for all column families, aggregate them and then publish the aggregated metrics. The first can be quite expensive for the downstream monitoring system and the second is a piece of work that it seems is better pushed into cassandra itself. Perhaps these global ColumnFamily metrics could be published under a name of: org.apache.cassandra.metrics:type=(ColumnFamily|IndexColumnFamily),name=(Metric name) (Same as existing ColumnFamily metrics, but with no keyspace or scope.) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7273) expose global ColumnFamily metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-7273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14004001#comment-14004001 ] Nick Bailey commented on CASSANDRA-7273: Related: CASSANDRA-6539 expose global ColumnFamily metrics -- Key: CASSANDRA-7273 URL: https://issues.apache.org/jira/browse/CASSANDRA-7273 Project: Cassandra Issue Type: New Feature Reporter: Richard Wagner Priority: Minor It would be very useful to have cassandra expose ColumnFamily metrics that span all column families. A general purpose cassandra monitoring system built up around the current ColumnFamily metrics really only has a couple of choices right now: publish metrics for all column families or fetch metrics for all column families, aggregate them and then publish the aggregated metrics. The first can be quite expensive for the downstream monitoring system and the second is a piece of work that it seems is better pushed into cassandra itself. Perhaps these global ColumnFamily metrics could be published under a name of: org.apache.cassandra.metrics:type=(ColumnFamily|IndexColumnFamily),name=(Metric name) (Same as existing ColumnFamily metrics, but with no keyspace or scope.) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7247) Provide top ten most frequent keys per column family
[ https://issues.apache.org/jira/browse/CASSANDRA-7247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lohfink updated CASSANDRA-7247: - Attachment: patch.txt Provide top ten most frequent keys per column family Key: CASSANDRA-7247 URL: https://issues.apache.org/jira/browse/CASSANDRA-7247 Project: Cassandra Issue Type: Improvement Reporter: Chris Lohfink Assignee: Chris Lohfink Priority: Minor Attachments: patch.txt Since already have the nice addthis stream library, can use it to keep track of most frequent DecoratedKeys that come through the system using StreamSummaries ([nice explaination|http://boundary.com/blog/2013/05/14/approximate-heavy-hitters-the-spacesaving-algorithm/]). Then provide a new metric to access them via JMX. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7247) Provide top ten most frequent keys per column family
[ https://issues.apache.org/jira/browse/CASSANDRA-7247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lohfink updated CASSANDRA-7247: - Attachment: (was: patch.diff) Provide top ten most frequent keys per column family Key: CASSANDRA-7247 URL: https://issues.apache.org/jira/browse/CASSANDRA-7247 Project: Cassandra Issue Type: Improvement Reporter: Chris Lohfink Assignee: Chris Lohfink Priority: Minor Attachments: patch.txt Since already have the nice addthis stream library, can use it to keep track of most frequent DecoratedKeys that come through the system using StreamSummaries ([nice explaination|http://boundary.com/blog/2013/05/14/approximate-heavy-hitters-the-spacesaving-algorithm/]). Then provide a new metric to access them via JMX. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Issue Comment Deleted] (CASSANDRA-7273) expose global ColumnFamily metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-7273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Bailey updated CASSANDRA-7273: --- Comment: was deleted (was: Related: CASSANDRA-6539) expose global ColumnFamily metrics -- Key: CASSANDRA-7273 URL: https://issues.apache.org/jira/browse/CASSANDRA-7273 Project: Cassandra Issue Type: New Feature Reporter: Richard Wagner Priority: Minor It would be very useful to have cassandra expose ColumnFamily metrics that span all column families. A general purpose cassandra monitoring system built up around the current ColumnFamily metrics really only has a couple of choices right now: publish metrics for all column families or fetch metrics for all column families, aggregate them and then publish the aggregated metrics. The first can be quite expensive for the downstream monitoring system and the second is a piece of work that it seems is better pushed into cassandra itself. Perhaps these global ColumnFamily metrics could be published under a name of: org.apache.cassandra.metrics:type=(ColumnFamily|IndexColumnFamily),name=(Metric name) (Same as existing ColumnFamily metrics, but with no keyspace or scope.) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7247) Provide top ten most frequent keys per column family
[ https://issues.apache.org/jira/browse/CASSANDRA-7247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14004005#comment-14004005 ] Chris Lohfink commented on CASSANDRA-7247: -- Added patch that uses the trace executor to track the partition thats updated the most, has the most columns inserted (useful for finding rows that are too wide) and the partitions with slowest insertion times. Will only track if trace probability 0. Provide top ten most frequent keys per column family Key: CASSANDRA-7247 URL: https://issues.apache.org/jira/browse/CASSANDRA-7247 Project: Cassandra Issue Type: Improvement Reporter: Chris Lohfink Assignee: Chris Lohfink Priority: Minor Attachments: jconsole.png, patch.txt Since already have the nice addthis stream library, can use it to keep track of most frequent DecoratedKeys that come through the system using StreamSummaries ([nice explaination|http://boundary.com/blog/2013/05/14/approximate-heavy-hitters-the-spacesaving-algorithm/]). Then provide a new metric to access them via JMX. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7247) Provide top ten most frequent keys per column family
[ https://issues.apache.org/jira/browse/CASSANDRA-7247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lohfink updated CASSANDRA-7247: - Attachment: jconsole.png Provide top ten most frequent keys per column family Key: CASSANDRA-7247 URL: https://issues.apache.org/jira/browse/CASSANDRA-7247 Project: Cassandra Issue Type: Improvement Reporter: Chris Lohfink Assignee: Chris Lohfink Priority: Minor Attachments: jconsole.png, patch.txt Since already have the nice addthis stream library, can use it to keep track of most frequent DecoratedKeys that come through the system using StreamSummaries ([nice explaination|http://boundary.com/blog/2013/05/14/approximate-heavy-hitters-the-spacesaving-algorithm/]). Then provide a new metric to access them via JMX. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7247) Provide top ten most frequent keys per column family
[ https://issues.apache.org/jira/browse/CASSANDRA-7247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14004005#comment-14004005 ] Chris Lohfink edited comment on CASSANDRA-7247 at 5/20/14 9:32 PM: --- Added patch that uses the trace executor to track the partition thats updated the most, has the most columns inserted (useful for finding rows that are too wide) and the partitions with slowest insertion times. Will only track if trace probability 0. Gives (key,count,error) tuples was (Author: cnlwsu): Added patch that uses the trace executor to track the partition thats updated the most, has the most columns inserted (useful for finding rows that are too wide) and the partitions with slowest insertion times. Will only track if trace probability 0. Provide top ten most frequent keys per column family Key: CASSANDRA-7247 URL: https://issues.apache.org/jira/browse/CASSANDRA-7247 Project: Cassandra Issue Type: Improvement Reporter: Chris Lohfink Assignee: Chris Lohfink Priority: Minor Attachments: jconsole.png, patch.txt Since already have the nice addthis stream library, can use it to keep track of most frequent DecoratedKeys that come through the system using StreamSummaries ([nice explaination|http://boundary.com/blog/2013/05/14/approximate-heavy-hitters-the-spacesaving-algorithm/]). Then provide a new metric to access them via JMX. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7248) Tuple type
[
https://issues.apache.org/jira/browse/CASSANDRA-7248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14004015#comment-14004015
]
Tyler Hobbs commented on CASSANDRA-7248:
Overall the patch looks pretty good. I'd like to see some thorough integration
tests (similar to user_types.py in the dtests) and maybe some unit tests on
TupleType.compare() and TupleType.validate().
I'll start extracting/backporting this for 6875.
Tuple type
--
Key: CASSANDRA-7248
URL: https://issues.apache.org/jira/browse/CASSANDRA-7248
Project: Cassandra
Issue Type: Bug
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Labels: cql3
Fix For: 2.1 rc1
For CASSANDRA-6875 we need to be able to talk about tuples values and types
(for prepared variables). Since we need it there, clients will need to
support them anyway and so I think it would be a lot cleaner to start
supporting those more generally. Besides, having tuples is a relatively
simple and natural extension to what we have. I'll note in particular that
tuple have a close relationship to user type in the sense that a tuple will
be really just like an anonymous with no name for the fields and in
particular a tuple value will be the same than a user type value.
The syntax would simply look like that:
{noformat}
CREATE TABLE foo (
k int PRIMARY KEY,
v tupleint, text, float
)
INSERT INTO foo(k, v) VALUES(0, (3, 'bar', 2.1));
{noformat}
We can also add projections in selects if we want:
{noformat}
SELECT v[0], v[2] FROM foo WHERE k = 0;
{noformat}
but that can come later (after all, we still don't have projections for
collections and it's not a big deal).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Assigned] (CASSANDRA-7241) Pig test fails on 2.1 branch
[ https://issues.apache.org/jira/browse/CASSANDRA-7241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams reassigned CASSANDRA-7241: --- Assignee: Brandon Williams Pig test fails on 2.1 branch Key: CASSANDRA-7241 URL: https://issues.apache.org/jira/browse/CASSANDRA-7241 Project: Cassandra Issue Type: Bug Reporter: Alex Liu Assignee: Brandon Williams run ant pig-test on cassandra-2.1 branch. There are many tests failed. I trace it a little and find out Pig test fails starts from https://github.com/apache/cassandra/commit/362cc05352ec67e707e0ac790732e96a15e63f6b commit. It looks like storage changes break Pig tests. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (CASSANDRA-7271) Bulk data loading in Cassandra causing OOM
[
https://issues.apache.org/jira/browse/CASSANDRA-7271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis resolved CASSANDRA-7271.
---
Resolution: Not a Problem
the Unsorted writer has to buffer everything in memory until you are done, to
sort it. Write less rows at a time, or use the Sorted writer instead.
Bulk data loading in Cassandra causing OOM
--
Key: CASSANDRA-7271
URL: https://issues.apache.org/jira/browse/CASSANDRA-7271
Project: Cassandra
Issue Type: Bug
Reporter: Prasanth Gullapalli
I am trying to load data from a csv file into Cassandra table using
SSTableSimpleUnsortedWriter. As the latest maven cassandra dependencies have
some issues with it, I have taken the _next_ beta (rc) version cut as
suggested in CASSANDRA-7218. But after taking it, I am facing issues with
bulk data loading
Here is the piece of code which loads data:
{code:java}
public void loadData(TableDefinition tableDefinition, InputStream
csvInputStream){
createDataInDBFormat(tableDefinition, csvInputStream);
Path dbFilePath = Paths.get(TEMP_DIR, keyspace,
tableDefinition.getName());
//BulkLoader.main(new
String[]{-d,localhost,dbFilePath.toUri().getPath()});
try {
JMXServiceURL jmxUrl = new JMXServiceURL(String.format(
service:jmx:rmi:///jndi/rmi://%s:%d/jmxrmi, cassandraHost,
cassandraJMXPort));
JMXConnector connector = JMXConnectorFactory.connect(jmxUrl, new
HashMapString, Object());
MBeanServerConnection mbeanServerConn =
connector.getMBeanServerConnection();
ObjectName name = new
ObjectName(org.apache.cassandra.db:type=StorageService);
StorageServiceMBean storageBean = JMX.newMBeanProxy(mbeanServerConn,
name, StorageServiceMBean.class);
storageBean.bulkLoad(dbFilePath.toUri().getPath());
connector.close();
} catch (IOException | MalformedObjectNameException e) {
e.printStacktrace()
}
FileUtils.deleteQuietly(dbFilePath.toFile());
}
private void createDataInDBFormat(TableDefinition tableDefinition,
InputStream csvInputStream) {
try(Reader reader = new InputStreamReader(csvInputStream)){
String tableName = tableDefinition.getName();
File directory = Paths.get(TEMP_DIR, keyspace, tableName).toFile();
directory.mkdirs();
String yamlPath =
file:\\+CASSANDRA_HOME+File.separator+conf+File.separator+cassandra.yaml;
System.setProperty(cassandra.config, yamlPath);
SSTableSimpleUnsortedWriter writer = new SSTableSimpleUnsortedWriter(
directory, new Murmur3Partitioner(), keyspace,
tableName, AsciiType.instance, null, 10);
long timestamp = System.currentTimeMillis() * 1000;
CSVReader csvReader = new CSVReader(reader);
String[] colValues = null;
ListColumnDefinition columnDefinitions =
tableDefinition.getColumnDefinitions();
while((colValues = csvReader.readNext()) != null){
if(colValues.length != 0){
writer.newRow(bytes(colValues[0]));
for(int index = 1; index colValues.length; index++){
ColumnDefinition columnDefinition =
columnDefinitions.get(index);
writer.addColumn(bytes(columnDefinition.getName()),
bytes(colValues[index]), timestamp);
}
}
}
csvReader.close();
writer.close();
} catch (IOException e) {
e.printStacktrace();
}
}
{code}
On trying to run loadData, it is giving me the following exception:
{code:xml}
11:23:18.035 [45742123@qtp-1703018180-0] ERROR
com.adaequare.common.config.TransactionPerRequestFilter.doInTransactionWithoutResult
39 - Problem in executing request :
[http://localhost:8081/mapro-engine/rest/masterdata/pumpData]. Cause
:org.glassfish.jersey.server.ContainerException: java.lang.OutOfMemoryError:
Java heap space
javax.servlet.ServletException:
org.glassfish.jersey.server.ContainerException: java.lang.OutOfMemoryError:
Java heap space
at
org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:392)
~[jersey-container-servlet-core-2.6.jar:na]
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:381)
~[jersey-container-servlet-core-2.6.jar:na]
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:344)
~[jersey-container-servlet-core-2.6.jar:na]
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:219)
~[jersey-container-servlet-core-2.6.jar:na]
at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
~[jetty-6.1.25.jar:6.1.25]
at
[jira] [Commented] (CASSANDRA-7262) During streaming: java.lang.AssertionError: Reference counter -1
[ https://issues.apache.org/jira/browse/CASSANDRA-7262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14004042#comment-14004042 ] Jonathan Ellis commented on CASSANDRA-7262: --- ISTM that fixing the root synchronization cause would be a better fix than applying a band-aid to make abort idempotent. During streaming: java.lang.AssertionError: Reference counter -1 Key: CASSANDRA-7262 URL: https://issues.apache.org/jira/browse/CASSANDRA-7262 Project: Cassandra Issue Type: Bug Components: Core Environment: Cassandra 2.07, x86-64 Ubuntu 12.04.4, Oracle java 1.7.0_45 Reporter: Duncan Sands Assignee: Joshua McKenzie Priority: Minor Fix For: 2.0.8, 2.1 rc1 Attachments: 7262_v1.txt, system.log.gz Got this assertion failure this weekend during repair: ERROR [STREAM-IN-/192.168.21.14] 2014-05-17 01:17:52,332 StreamSession.java (line 420) [Stream #3a3ac8a2-dd50-11e3-b3c1-6bf6dccd6457] Streaming error occurred java.lang.RuntimeException: Outgoing stream handler has been closed at org.apache.cassandra.streaming.ConnectionHandler.sendMessage(ConnectionHandler.java:170) at org.apache.cassandra.streaming.StreamSession.receive(StreamSession.java:483) at org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:372) at org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:289) at java.lang.Thread.run(Thread.java:744) ERROR [STREAM-IN-/192.168.21.14] 2014-05-17 01:17:52,350 CassandraDaemon.java (line 198) Exception in thread Thread[STREAM-IN-/192.168.21.14,5,RMI Runtime] java.lang.AssertionError: Reference counter -1 for /mnt/ssd1/cassandra/data/ldn_production/historical_accounts/ldn_production-historical_accounts-jb-79827-Data.db at org.apache.cassandra.io.sstable.SSTableReader.releaseReference(SSTableReader.java:1107) at org.apache.cassandra.streaming.StreamTransferTask.abort(StreamTransferTask.java:80) at org.apache.cassandra.streaming.StreamSession.closeSession(StreamSession.java:322) at org.apache.cassandra.streaming.StreamSession.onError(StreamSession.java:425) at org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:300) at java.lang.Thread.run(Thread.java:744) followed by a few more (the reference counter got down to -3). Got the same kind of assertion failure on one other node (in a different data centre; there are 21 nodes altogether distributed over 4 data centres). I've attached the relevant part of the log. It starts quite a bit before the assertion failure at the first exception on this node (Cannot proceed on repair because a neighbor ... is dead), and finishes a few hours afterwards when the node was restarted. Edit: The following Reference counter assertion failures followed the 1st on a different file and have a different stack trace: ERROR [CompactionExecutor:382] 2014-05-17 01:17:53,157 CassandraDaemon.java (line 198) Exception in thread Thread[CompactionExecutor:382,1,main] java.lang.AssertionError: Reference counter -1 for /mnt/ssd1/cassandra/data/ldn_production/historical_accounts/ldn_production-historical_accounts-jb-83888-Data.db at org.apache.cassandra.io.sstable.SSTableReader.releaseReference(SSTableReader.java:1107) at org.apache.cassandra.io.sstable.SSTableReader.releaseReferences(SSTableReader.java:1429) at org.apache.cassandra.db.compaction.CompactionController.close(CompactionController.java:207) at org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:220) at org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60) at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59) at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:197) ... -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6146) CQL-native stress
[
https://issues.apache.org/jira/browse/CASSANDRA-6146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14004044#comment-14004044
]
Jonathan Ellis commented on CASSANDRA-6146:
---
bq. you need to define a number of things around the generation of data (and
distributions for clustering columns etc.) to populate the schema with, which
requires defining the columns anyway
Okay, but why not define them by asking the cluster?
bq. You would need to write into the existing schema first
I don't see how that follows at all; we have the exact same information in
{{system}} that you get from CREATE TABLE, only already parsed for you.
CQL-native stress
-
Key: CASSANDRA-6146
URL: https://issues.apache.org/jira/browse/CASSANDRA-6146
Project: Cassandra
Issue Type: New Feature
Components: Tools
Reporter: Jonathan Ellis
Assignee: T Jake Luciani
Fix For: 2.1 rc1
Attachments: 6146.txt
The existing CQL support in stress is not worth discussing. We need to
start over, and we might as well kill two birds with one stone and move to
the native protocol while we're at it.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-6851) Improve anticompaction after incremental repair
[ https://issues.apache.org/jira/browse/CASSANDRA-6851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14004048#comment-14004048 ] Jonathan Ellis commented on CASSANDRA-6851: --- Bloom filters aren't composeable that way, but there is a class of algorithms called cardinality estimation that are: CASSANDRA-6474 Improve anticompaction after incremental repair --- Key: CASSANDRA-6851 URL: https://issues.apache.org/jira/browse/CASSANDRA-6851 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Marcus Eriksson Assignee: Russell Alexander Spitzer Priority: Minor Labels: compaction, lhf Fix For: 2.1.1 After an incremental repair we iterate over all sstables and split them in two parts, one containing the repaired data and one the unrepaired. We could in theory double the number of sstables on a node. To avoid this we could make anticompaction also do a compaction, for example, if we are to anticompact 10 sstables, we could anticompact those to 2. Note that we need to avoid creating too big sstables though, if we anticompact all sstables on a node it would essentially be a major compaction. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7247) Provide top ten most frequent keys per column family
[ https://issues.apache.org/jira/browse/CASSANDRA-7247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14004005#comment-14004005 ] Chris Lohfink edited comment on CASSANDRA-7247 at 5/20/14 10:15 PM: Added patch that uses the trace executor to track the partition thats updated the most, has the most columns inserted (useful for finding rows that are too wide) and the partitions with slowest insertion times. Will only track if trace probability 0. Gives (key,count,error) tuples !jconsole.png! was (Author: cnlwsu): Added patch that uses the trace executor to track the partition thats updated the most, has the most columns inserted (useful for finding rows that are too wide) and the partitions with slowest insertion times. Will only track if trace probability 0. Gives (key,count,error) tuples Provide top ten most frequent keys per column family Key: CASSANDRA-7247 URL: https://issues.apache.org/jira/browse/CASSANDRA-7247 Project: Cassandra Issue Type: Improvement Reporter: Chris Lohfink Assignee: Chris Lohfink Priority: Minor Attachments: jconsole.png, patch.txt Since already have the nice addthis stream library, can use it to keep track of most frequent DecoratedKeys that come through the system using StreamSummaries ([nice explaination|http://boundary.com/blog/2013/05/14/approximate-heavy-hitters-the-spacesaving-algorithm/]). Then provide a new metric to access them via JMX. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Issue Comment Deleted] (CASSANDRA-7247) Provide top ten most frequent keys per column family
[
https://issues.apache.org/jira/browse/CASSANDRA-7247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Lohfink updated CASSANDRA-7247:
-
Comment: was deleted
(was: Problem is StreamSummary is not thread safe. There is a
ConcurrentStreamSummary, which I found in this implementation to be ~4x slower
then a synchronized block around the offer of the non-thread safe one.
Concurrent did perform similarly when also wrapped in synchronized block which
I will show below but because it would lose any benefit of being a concurrent
implementation when access is serialized I think the faster impl is best.
Done on 2013 retina MBP with 500gb ssd against trunk:
{code:title=No Changes}
id, ops ,op/s, key/s,mean, med, .95,
.99,.999, max, time, stderr
4 threadCount, 634450, 21692, 21692, 0.2, 0.2, 0.2,
0.2, 0.4, 740.1, 29.2, 0.01188
8 threadCount, 886600, 29762, 29762, 0.3, 0.2, 0.3,
0.4, 1.3, 1007.3, 29.8, 0.01220
16 threadCount, 912050, 29035, 29035, 0.5, 0.3, 0.9,
2.5,11.2, 1393.8, 31.4, 0.01162
24 threadCount, 1022250 , 32681, 32681, 0.7, 0.5, 1.0,
2.9,13.5, 1126.5, 31.3, 0.00923
36 threadCount, 946550, 30900, 30900, 1.2, 0.8, 1.4,
3.0,22.5, 1369.2, 30.6, 0.01089
{code}
{code:title=With Patch}
id, ops ,op/s, key/s,mean, med, .95,
.99,.999, max, time, stderr
4 threadCount, 643900, 21700, 21700, 0.2, 0.2, 0.2,
0.2, 0.9, 941.1, 29.7, 0.01079
8 threadCount, 942100, 32300, 32300, 0.2, 0.2, 0.3,
0.3, 1.2, 849.5, 29.2, 0.01519
16 threadCount, 907400, 30650, 30650, 0.5, 0.3, 0.8,
1.9,10.7, 1124.0, 29.6, 0.01112
24 threadCount, 1026150 , 31753, 31753, 0.7, 0.5, 0.9,
3.3,20.6, 1299.0, 32.3, 0.01295
36 threadCount, 980600, 30077, 30077, 1.2, 0.8, 1.3,
2.7,24.9, 1394.3, 32.6, 0.01747
{code}
{code:title=ConcurrentStreamSummary with sync}
4 threadCount, 494350, 16643, 16643, 0.2, 0.2, 0.3,
0.3, 1.0, 943.6, 29.7, 0.01286
8 threadCount, 812950, 26358, 26358, 0.3, 0.2, 0.3,
0.5, 1.4, 1488.9, 30.8, 0.01909
16 threadCount, 877500, 27396, 27396, 0.6, 0.3, 1.0,
2.2,12.1, 1299.2, 32.0, 0.01824
24 threadCount, 837550, 25345, 25345, 0.9, 0.4, 1.2,
3.7,84.2, 2123.6, 33.0, 0.02437
36 threadCount, 910200, 28008, 28008, 1.3, 0.6, 2.8,
9.2,32.2, 1212.8, 32.5, 0.01654
{code}
{code:title=ConcurentStreamSummary no blocking}
id, ops ,op/s, key/s,mean, med, .95,
.99,.999, max, time, stderr
4 threadCount, 183600,6145,6145, 0.6, 0.6, 0.8,
1.0, 2.6, 354.5, 29.9, 0.01063
8 threadCount, 197200,6593,6593, 1.2, 1.1, 1.4,
1.8, 3.3, 413.5, 29.9, 0.00716
16 threadCount, 203200,6794,6794, 2.3, 2.2, 2.6,
3.5,12.1, 649.1, 29.9, 0.01096
24 threadCount, 198000,6615,6615, 3.6, 3.3, 4.2,
4.9,44.2, 570.4, 29.9, 0.00894
36 threadCount, 199800,6627,6627, 5.4, 4.9, 6.5,
8.0, 110.8, 272.3, 30.1, 0.01452
{code})
Provide top ten most frequent keys per column family
Key: CASSANDRA-7247
URL: https://issues.apache.org/jira/browse/CASSANDRA-7247
Project: Cassandra
Issue Type: Improvement
Reporter: Chris Lohfink
Assignee: Chris Lohfink
Priority: Minor
Attachments: jconsole.png, patch.txt
Since already have the nice addthis stream library, can use it to keep track
of most frequent DecoratedKeys that come through the system using
StreamSummaries ([nice
explaination|http://boundary.com/blog/2013/05/14/approximate-heavy-hitters-the-spacesaving-algorithm/]).
Then provide a new metric to access them via JMX.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Issue Comment Deleted] (CASSANDRA-7247) Provide top ten most frequent keys per column family
[ https://issues.apache.org/jira/browse/CASSANDRA-7247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lohfink updated CASSANDRA-7247: - Comment: was deleted (was: patch against trunk (284d3f7fe0210acfdc9b242c2987124eda5a9c66)) Provide top ten most frequent keys per column family Key: CASSANDRA-7247 URL: https://issues.apache.org/jira/browse/CASSANDRA-7247 Project: Cassandra Issue Type: Improvement Reporter: Chris Lohfink Assignee: Chris Lohfink Priority: Minor Attachments: jconsole.png, patch.txt Since already have the nice addthis stream library, can use it to keep track of most frequent DecoratedKeys that come through the system using StreamSummaries ([nice explaination|http://boundary.com/blog/2013/05/14/approximate-heavy-hitters-the-spacesaving-algorithm/]). Then provide a new metric to access them via JMX. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Issue Comment Deleted] (CASSANDRA-7247) Provide top ten most frequent keys per column family
[ https://issues.apache.org/jira/browse/CASSANDRA-7247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lohfink updated CASSANDRA-7247: - Comment: was deleted (was: Another option might be to spin off this and other metrics into the MiscStage, it only has single thread so no synchronization required and wont be as bad to put additional metrics in there as well for additional visibility like topK size in bytes, worst latencies and such. I wouldn't expect much difference performance-wise with just the one stream summary above since enqueuing onto the LinkedBlockingQueue should have similar locking performance (synchronization on putlock), but then reading of metric would never cause contention (albeit very small) on write path. If theres any interest I can give it a shot though and maybe throw in some additional metrics.) Provide top ten most frequent keys per column family Key: CASSANDRA-7247 URL: https://issues.apache.org/jira/browse/CASSANDRA-7247 Project: Cassandra Issue Type: Improvement Reporter: Chris Lohfink Assignee: Chris Lohfink Priority: Minor Attachments: jconsole.png, patch.txt Since already have the nice addthis stream library, can use it to keep track of most frequent DecoratedKeys that come through the system using StreamSummaries ([nice explaination|http://boundary.com/blog/2013/05/14/approximate-heavy-hitters-the-spacesaving-algorithm/]). Then provide a new metric to access them via JMX. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Issue Comment Deleted] (CASSANDRA-7247) Provide top ten most frequent keys per column family
[ https://issues.apache.org/jira/browse/CASSANDRA-7247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lohfink updated CASSANDRA-7247: - Comment: was deleted (was: Id like to rework this I think to use the trace executor) Provide top ten most frequent keys per column family Key: CASSANDRA-7247 URL: https://issues.apache.org/jira/browse/CASSANDRA-7247 Project: Cassandra Issue Type: Improvement Reporter: Chris Lohfink Assignee: Chris Lohfink Priority: Minor Attachments: jconsole.png, patch.txt Since already have the nice addthis stream library, can use it to keep track of most frequent DecoratedKeys that come through the system using StreamSummaries ([nice explaination|http://boundary.com/blog/2013/05/14/approximate-heavy-hitters-the-spacesaving-algorithm/]). Then provide a new metric to access them via JMX. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6875) CQL3: select multiple CQL rows in a single partition using IN
[
https://issues.apache.org/jira/browse/CASSANDRA-6875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14004088#comment-14004088
]
Tyler Hobbs commented on CASSANDRA-6875:
I've backported TupleType from 7248 and pushed my changes to the same branch.
(I merged in cassandra-2.0, so let me know if you'd like a rebase or anything.)
CQL3: select multiple CQL rows in a single partition using IN
-
Key: CASSANDRA-6875
URL: https://issues.apache.org/jira/browse/CASSANDRA-6875
Project: Cassandra
Issue Type: Bug
Components: API
Reporter: Nicolas Favre-Felix
Assignee: Tyler Hobbs
Priority: Minor
Fix For: 2.0.8
In the spirit of CASSANDRA-4851 and to bring CQL to parity with Thrift, it is
important to support reading several distinct CQL rows from a given partition
using a distinct set of coordinates for these rows within the partition.
CASSANDRA-4851 introduced a range scan over the multi-dimensional space of
clustering keys. We also need to support a multi-get of CQL rows,
potentially using the IN keyword to define a set of clustering keys to
fetch at once.
(reusing the same example\:)
Consider the following table:
{code}
CREATE TABLE test (
k int,
c1 int,
c2 int,
PRIMARY KEY (k, c1, c2)
);
{code}
with the following data:
{code}
k | c1 | c2
---++
0 | 0 | 0
0 | 0 | 1
0 | 1 | 0
0 | 1 | 1
{code}
We can fetch a single row or a range of rows, but not a set of them:
{code}
SELECT * FROM test WHERE k = 0 AND (c1, c2) IN ((0, 0), (1,1)) ;
Bad Request: line 1:54 missing EOF at ','
{code}
Supporting this syntax would return:
{code}
k | c1 | c2
---++
0 | 0 | 0
0 | 1 | 1
{code}
Being able to fetch these two CQL rows in a single read is important to
maintain partition-level isolation.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
