buildbot success in ASF Buildbot on cassandra-trunk
The Buildbot has detected a restored build of cassandra-trunk on ASF Buildbot. Full details are available at: http://ci.apache.org/builders/cassandra-trunk/builds/1047 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: isis_ubuntu Build Reason: Build Source Stamp: [branch cassandra/trunk] 1072854 Blamelist: jbellis Build succeeded! sincerely, -The Buildbot
svn commit: r1072854 - /cassandra/trunk/test/unit/org/apache/cassandra/streaming/StreamingTransferTest.java
Author: jbellis Date: Mon Feb 21 06:23:48 2011 New Revision: 1072854 URL: http://svn.apache.org/viewvc?rev=1072854&view=rev Log: fix build Modified: cassandra/trunk/test/unit/org/apache/cassandra/streaming/StreamingTransferTest.java Modified: cassandra/trunk/test/unit/org/apache/cassandra/streaming/StreamingTransferTest.java URL: http://svn.apache.org/viewvc/cassandra/trunk/test/unit/org/apache/cassandra/streaming/StreamingTransferTest.java?rev=1072854&r1=1072853&r2=1072854&view=diff == --- cassandra/trunk/test/unit/org/apache/cassandra/streaming/StreamingTransferTest.java (original) +++ cassandra/trunk/test/unit/org/apache/cassandra/streaming/StreamingTransferTest.java Mon Feb 21 06:23:48 2011 @@ -183,7 +183,7 @@ public class StreamingTransferTest exten } StreamOutSession session = StreamOutSession.create(keyspace, LOCAL, null); -StreamOut.transferSSTables(session, ssTableReaders, ranges); +StreamOut.transferSSTables(session, ssTableReaders, ranges, OperationType.BOOTSTRAP); session.await();
[jira] Commented: (CASSANDRA-2203) Class to measure real memory allocated for an object
[ https://issues.apache.org/jira/browse/CASSANDRA-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12997281#comment-12997281 ] Jonathan Ellis commented on CASSANDRA-2203: --- (For Cassandra you will want to configure it with omitSharedBufferOverhead.) > Class to measure real memory allocated for an object > > > Key: CASSANDRA-2203 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2203 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Goffinet >Assignee: Chris Goffinet >Priority: Trivial > Attachments: > 0001-Class-to-measure-actual-object-size-for-Hotspot-JVM.patch > > > We wanted to share a class we are using internally to measure actual object > sizes for Hotspot VM. We've been using this on RowCache and Memtables. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2203) Class to measure real memory allocated for an object
[
https://issues.apache.org/jira/browse/CASSANDRA-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12997280#comment-12997280
]
Jonathan Ellis commented on CASSANDRA-2203:
---
I wrote a java agent to do this kind of thing more accurately:
https://github.com/jbellis/jamm/
Includes test suite that starts with some simple stuff, so if your code
disagrees with that it should be possible to track down which is right.
To use it with cassandra, copy the jamm jar into lib/ and add to
cassandra-env.sh,
{noformat}
JVM_OPTS="$JVM_OPTS -javaagent:$CASSANDRA_HOME/lib/jamm-0.2.jar"
{noformat}
> Class to measure real memory allocated for an object
>
>
> Key: CASSANDRA-2203
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2203
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Chris Goffinet
>Assignee: Chris Goffinet
>Priority: Trivial
> Attachments:
> 0001-Class-to-measure-actual-object-size-for-Hotspot-JVM.patch
>
>
> We wanted to share a class we are using internally to measure actual object
> sizes for Hotspot VM. We've been using this on RowCache and Memtables.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (CASSANDRA-2204) JMX agent for easier monitoring through firewall
JMX agent for easier monitoring through firewall Key: CASSANDRA-2204 URL: https://issues.apache.org/jira/browse/CASSANDRA-2204 Project: Cassandra Issue Type: Improvement Reporter: Marcus Better It would be useful to have something like Tomcat's JmxRemoteLifecycleListener, which enables JMX monitoring through firewalls by using configurable fixed port numbers for the RMI connection. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1072849 - in /cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable: SSTableReader.java SSTableWriter.java
Author: jbellis
Date: Mon Feb 21 04:53:38 2011
New Revision: 1072849
URL: http://svn.apache.org/viewvc?rev=1072849&view=rev
Log:
fix build
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTableReader.java?rev=1072849&r1=1072848&r2=1072849&view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
Mon Feb 21 04:53:38 2011
@@ -678,11 +678,4 @@ public class SSTableReader extends SSTab
{
return bloomFilterTracker.getRecentTruePositiveCount();
}
-
-@Override
-protected String toSubString() {
-return ", ifile=" + ifile +
- ", dfile=" + dfile +
- ", maxDataAge=" + maxDataAge;
-}
}
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java?rev=1072849&r1=1072848&r2=1072849&view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java
Mon Feb 21 04:53:38 2011
@@ -230,11 +230,6 @@ public class SSTableWriter extends SSTab
return new Builder(desc);
}
-@Override
-protected String toSubString() {
-return ", dataFile=" + dataFile;
-}
-
/**
* Removes the given SSTable from temporary status and opens it,
rebuilding the
* bloom filter and row index from the data file.
[Cassandra Wiki] Trivial Update of "FrontPage_JP" by MakiWatanabe
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "FrontPage_JP" page has been changed by MakiWatanabe. The comment on this change is: Translate PerformanceTuning_JP. http://wiki.apache.org/cassandra/FrontPage_JP?action=diff&rev1=70&rev2=71 -- * [[MultinodeCluster_JP|マルチノードクラスターの作成]]（翻訳済み） * [[Operations_JP|オペレーション]]（翻訳済み） * [[Embedding_JP|Cassandraの組み込み]]（翻訳済み） - * [[MemtableThresholds|Memtableの閾値]] と [[PerformanceTuning|パフォーマンスチューニング]] + * [[MemtableThresholds|Memtableの閾値]] と [[PerformanceTuning_JP|パフォーマンスチューニング]]（翻訳済み） * [[CassandraHardware_JP|Cassandraで使うハードウェアの注意点]]（翻訳済み） * [[CloudConfig|RackspaceまたはAmazon Web Servicesの構成]] * [[LargeDataSetConsiderations|巨大なデータセットに関する注意点]]
[Cassandra Wiki] Update of "PerformanceTuning_JP" by MakiWatanabe
Dear Wiki user,
You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for
change notification.
The "PerformanceTuning_JP" page has been changed by MakiWatanabe.
The comment on this change is: Translation to Japanese.
http://wiki.apache.org/cassandra/PerformanceTuning_JP?action=diff&rev1=8&rev2=9
--
## page was copied from PerformanceTuning
- = Improving write performance =
- 1. Make sure your commit log and data dirs (sstables) are on different disks.
- 1. There isn't really a step 2. Writes go into the commitlog (fast) and
straight into memory (really fast) and can't really be sped up beyond that.
+ = write性能を向上させる方法 =
+ 1. commit logとデータディレクトリ（sstables）を異なるディスクに配置して下さい。
+ 1. 実はStep 2はありません。Writeはcommit
logに書き込まれ（高速です）、次に直接メモリにロードされます（大変高速です）。この過程を更に高速化する方法は現実的にありません。
- If you're seeing huge latency spikes under write-heavy loads, that's due to
other factors. It's a 'fix what has gone horribly wrong' problem, not a 'tune
for better peak performance' problem.
+
もしwrite中心の負荷で大きな遅延のスパイクが発生しているなら、原因は他の要素です。ピーク性能がまずまずなら、それを更にチューニングするより、本当に性能が悪い部分の改善に注力したほうがいいでしょう。
- = Improving read performance =
- Be sure to review the page on [[MemtableThresholds|Memtable Thresholds]] --
several important configuration and system settings are described there.
+ = read性能を向上する方法 =
+ まず[[MemtableThresholds|Memtable
Thresholds]]のページを確認してください。重要な設定パラメータや、システム設定についていくつか説明されています。
- === Lessen overall system impact of Compactions ===
- To lower compaction priority (thus reducing its impact on the rest of the
system, and making it take longer), add these options to cassandra.in.sh in
0.6.3 or later:
+ === Compactionがシステムに与える影響を抑制する ===
+
Compactionのプライオリティを下げる（システム全体に与える影響が減りますが、Compactionにより長い時間がかかります）には、以下のオプションをcassandra.in.shに追加して下さい（0.6.3以降）。
{{{
-XX:+UseThreadPriorities \
-XX:ThreadPriorityPolicy=42 \
-Dcassandra.compaction.priority=1 \
}}}
- === Useful JVM options ===
+
+ === 有用なJVMオプション ===
{{{
- -XX:+UseCompressedOops # enables compressed references, reducing memory
overhead on 64bit JVMs
+ -XX:+UseCompressedOops # Compressed Referenceを有効にし、64bit JVM上でメモリオーバヘッドを減少させます
}}}
- See http://wikis.sun.com/display/HotSpotInternals/CompressedOops and
http://publib.boulder.ibm.com/infocenter/javasdk/v6r0/topic/com.ibm.java.doc.diagnostics.60/diag/understanding/mm_compressed_references.html
for discussion on the Sun and IBM JVMs, respectively.
- Compressed references (aka OOPs = ordinary object pointers) are not stable
for Sun JVM versions before 6u19.
+ Sun、IBMのJVMについてはそれぞれ次のURLにおける議論を参照して下さい。
+ http://wikis.sun.com/display/HotSpotInternals/CompressedOops
http://publib.boulder.ibm.com/infocenter/javasdk/v6r0/topic/com.ibm.java.doc.diagnostics.60/diag/understanding/mm_compressed_references.html
+ Sun JVM 6u19以前のバージョンではCompressed Reference（OOP=Ordinary Object
Pointersとして知られています）は安定していません。
- == System Settings ==
- todo: describe how top, iostat -x, and JMX stats can help you see what is
making things slow
- For starters, see
"[[http://spyced.blogspot.com/2010/01/linux-performance-basics.html|Linux
Performance Basics for Cassandra]]".
+ == システム設定 ==
+ ToDo: describe how top, iostat -x, and JMX stats can help you see what is
making things slow
+
初心者の方は"[[http://spyced.blogspot.com/2010/01/linux-performance-basics.html|Linux
Performance Basics for Cassandra]]"を参照して下さい。
+
[jira] Updated: (CASSANDRA-2203) Class to measure real memory allocated for an object
[ https://issues.apache.org/jira/browse/CASSANDRA-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Goffinet updated CASSANDRA-2203: -- Attachment: 0001-Class-to-measure-actual-object-size-for-Hotspot-JVM.patch This requires JSR305 and Guava. Contribution is from Attila Szegedi. > Class to measure real memory allocated for an object > > > Key: CASSANDRA-2203 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2203 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Goffinet >Assignee: Chris Goffinet >Priority: Trivial > Attachments: > 0001-Class-to-measure-actual-object-size-for-Hotspot-JVM.patch > > > We wanted to share a class we are using internally to measure actual object > sizes for Hotspot VM. We've been using this on RowCache and Memtables. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (CASSANDRA-2203) Class to measure real memory allocated for an object
Class to measure real memory allocated for an object Key: CASSANDRA-2203 URL: https://issues.apache.org/jira/browse/CASSANDRA-2203 Project: Cassandra Issue Type: Improvement Reporter: Chris Goffinet Assignee: Chris Goffinet Priority: Trivial We wanted to share a class we are using internally to measure actual object sizes for Hotspot VM. We've been using this on RowCache and Memtables. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1072816 - /cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTable.java
Author: jbellis
Date: Mon Feb 21 00:20:48 2011
New Revision: 1072816
URL: http://svn.apache.org/viewvc?rev=1072816&view=rev
Log:
getSimpleName really is better than getName though
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTable.java
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTable.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTable.java?rev=1072816&r1=1072815&r2=1072816&view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTable.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTable.java
Mon Feb 21 00:20:48 2011
@@ -259,7 +259,7 @@ public abstract class SSTable
@Override
public String toString()
{
-return getClass().getName() + "(" +
+return getClass().getSimpleName() + "(" +
"path='" + getFilename() + '\'' +
')';
}
svn commit: r1072815 - in /cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable: Descriptor.java SSTable.java
Author: jbellis
Date: Mon Feb 21 00:15:07 2011
New Revision: 1072815
URL: http://svn.apache.org/viewvc?rev=1072815&view=rev
Log:
revert generic toStrings that made logging worse
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/Descriptor.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTable.java
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/Descriptor.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/Descriptor.java?rev=1072815&r1=1072814&r2=1072815&view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/Descriptor.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/Descriptor.java
Mon Feb 21 00:15:07 2011
@@ -186,20 +186,9 @@ public class Descriptor
}
@Override
-public String toString() {
-return "Descriptor(baseFilename=" + baseFilename() +
- ", directory=" + directory +
- ", ksname='" + ksname + '\'' +
- ", cfname='" + cfname + '\'' +
- ", version='" + version + '\'' +
- ", generation=" + generation +
- ", temporary=" + temporary +
- ", usesOldBloomFilter=" + usesOldBloomFilter +
- ", isLatestVersion=" + isLatestVersion +
- ", hasStringsInBloomFilter=" + hasStringsInBloomFilter +
- ", hasIntRowSize=" + hasIntRowSize +
- ", hasEncodedKeys=" + hasEncodedKeys +
- ")";
+public String toString()
+{
+return baseFilename();
}
@Override
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTable.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTable.java?rev=1072815&r1=1072814&r2=1072815&view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTable.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTable.java
Mon Feb 21 00:15:07 2011
@@ -257,15 +257,10 @@ public abstract class SSTable
}
@Override
-public String toString() {
-return getClass().getSimpleName() + "(descriptor=" + descriptor +
- ", path='" + getFilename() + '\'' +
- ", metadata=" + metadata +
- toSubString() +
- ")";
-}
-
-protected String toSubString() {
-return "";
+public String toString()
+{
+return getClass().getName() + "(" +
+ "path='" + getFilename() + '\'' +
+ ')';
}
}
[jira] Commented: (CASSANDRA-2202) Better error reporting and meaningful toStrings
[ https://issues.apache.org/jira/browse/CASSANDRA-2202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12997235#comment-12997235 ] Hudson commented on CASSANDRA-2202: --- Integrated in Cassandra-0.7 #300 (See [https://hudson.apache.org/hudson/job/Cassandra-0.7/300/]) Better error reporting for sstable problems. Patch by Kjetil Valstadsve, reviewed by brandonwilliams for CASSANDRA-2202. > Better error reporting and meaningful toStrings > --- > > Key: CASSANDRA-2202 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2202 > Project: Cassandra > Issue Type: Improvement > Components: Core >Affects Versions: 0.7.2 >Reporter: Kjetil Valstadsve >Priority: Minor > Labels: exception, logging, stacktrace > Attachments: errorreporting.txt > > > Having been through some trouble with the upgrade to 0.7.2 via 0.7.1, I've > learned the hard way that Cassandra's error reporting can be improved. In > particular, I had issues with deserialization of bloom filters, and with some > corrupt SSTables. > Chatting with driftx on the IRC channel, I complained about lack of error > reporting, and showed him a stacktrace that indicated a corrupted SSTable. I > then added various toString implementations and some new try/catch/rethrow > points in critical control flow, which revealed the name of the SSTable. (It > turned out to be a hints table, allowing driftx to advise me on the next step > - namely delete the local tables and rely on read repair.) > I'm submitting a patch reflecting this work. It addresses these issues, and > is perhaps best taken as a suggested guideline, maybe as an eye-opener. > I generally advocate two practices/good habits wrt. error reporting: > 1) Provide a good toString, reflecting the important state of an instance. > (In this case, file names carry valuable information.) > 2) Trapping and reporting errors is more useful at some points of the control > flow than in others. For instance, at a point in the code where we have > gathered all relevant parameters and are about to embark on major I/O (like > reading a table from disk). In these points, a try/catch/throw can be used > to capture this information. It's ok to catch Exception here, if you ask me, > as long as you wrap it as the cause of a new exception. This new exception > should describe the most important parameters involved in the failed call. > To do this, it should rely on the toStrings from item 1. > 2a) The same goes for log messages. Notice that the utility of a toString > multiplies, as it will emit useful information wherever it's used for logging > and/or exception messages. > By following these points, you will get stack traces partitioned up with a > chain of "Caused by" exceptions, each providing some information from its > relevant layer in the call stack. > I have applied this approach in the attached patch. As a result, we were > able to get the file name from a stacktrace that originally just said > "EOFException", and the problem could be easily fixed. > Caveats: Yes, you can get insanely long exception messages, and/or log > messages. To keep the post-mortem sessions manageable, constant tuning and > re-tuning of toStrings and log/exception messages is needed. However, even a > terrifyingly verbose exception won't be _less_ meaningful to the user than a > nameless EOFException, and it will convey more information to the developers. > (Even if that information boils down to "this toString is way more verbose > than it needs to be".) > In short: This patch helps with detecting corrupt SSTables and bogus bloom > filers, and I hope it inspires someone to give out more informative error > messages as well. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1072774 - in /cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra: db/ db/columniterator/ db/filter/ io/sstable/ io/util/
Author: brandonwilliams
Date: Sun Feb 20 22:18:01 2011
New Revision: 1072774
URL: http://svn.apache.org/viewvc?rev=1072774&view=rev
Log:
Better error reporting for sstable problems.
Patch by Kjetil Valstadsve, reviewed by brandonwilliams for
CASSANDRA-2202.
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Table.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/columniterator/SSTableNamesIterator.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/filter/QueryFilter.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/filter/QueryPath.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/filter/SliceQueryFilter.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/Descriptor.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTable.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTableScanner.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/util/BufferedRandomAccessFile.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/util/MappedFileDataInput.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/util/SegmentedFile.java
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Table.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Table.java?rev=1072774&r1=1072773&r2=1072774&view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Table.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/Table.java
Sun Feb 20 22:18:01 2011
@@ -699,4 +699,9 @@ public class Table
cfs.truncate().get();
logger.debug("Truncation done.");
}
+
+@Override
+public String toString() {
+return getClass().getSimpleName() + "(name='" + name + "')";
+}
}
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/columniterator/SSTableNamesIterator.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/columniterator/SSTableNamesIterator.java?rev=1072774&r1=1072773&r2=1072774&view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/columniterator/SSTableNamesIterator.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/columniterator/SSTableNamesIterator.java
Sun Feb 20 22:18:01 2011
@@ -25,6 +25,7 @@ import java.io.IOException;
import java.nio.ByteBuffer;
import java.util.*;
+import org.apache.cassandra.db.ColumnFamilySerializer;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -108,7 +109,13 @@ public class SSTableNamesIterator extend
// we can stop early if bloom filter says none of the columns actually
exist -- but,
// we can't stop before initializing the cf above, in case there's a
relevant tombstone
-cf =
ColumnFamily.serializer().deserializeFromSSTableNoColumns(ColumnFamily.create(sstable.metadata),
file);
+ColumnFamilySerializer serializer = ColumnFamily.serializer();
+try {
+cf =
serializer.deserializeFromSSTableNoColumns(ColumnFamily.create(sstable.metadata),
file);
+} catch (Exception e) {
+throw new IOException
+(serializer + " failed to deserialize " +
sstable.getColumnFamilyName() + " with " + sstable.metadata + " from " + file,
e);
+}
List filteredColumnNames = new
ArrayList(columns.size());
for (ByteBuffer name : columns)
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/filter/QueryFilter.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/filter/QueryFilter.java?rev=1072774&r1=1072773&r2=1072774&view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/filter/QueryFilter.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/filter/QueryFilter.java
Sun Feb 20 22:18:01 2011
@@ -210,4 +210,13 @@ public class QueryFilter
{
return new QueryFilter(key, path, new NamesQueryFilter(column));
}
+
+@Override
+public String toString() {
+return getClass().getSimpleName() + "(key=" + key +
+ ", path=" + path +
+ (filter == null ? "" : ", filter=" + filter) +
+
[jira] Updated: (CASSANDRA-2202) Better error reporting and meaningful toStrings
[ https://issues.apache.org/jira/browse/CASSANDRA-2202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kjetil Valstadsve updated CASSANDRA-2202: - Attachment: errorreporting.txt Various toStrings and exception rethrows. > Better error reporting and meaningful toStrings > --- > > Key: CASSANDRA-2202 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2202 > Project: Cassandra > Issue Type: Improvement > Components: Core >Affects Versions: 0.7.2 >Reporter: Kjetil Valstadsve >Priority: Minor > Labels: exception, logging, stacktrace > Attachments: errorreporting.txt > > > Having been through some trouble with the upgrade to 0.7.2 via 0.7.1, I've > learned the hard way that Cassandra's error reporting can be improved. In > particular, I had issues with deserialization of bloom filters, and with some > corrupt SSTables. > Chatting with driftx on the IRC channel, I complained about lack of error > reporting, and showed him a stacktrace that indicated a corrupted SSTable. I > then added various toString implementations and some new try/catch/rethrow > points in critical control flow, which revealed the name of the SSTable. (It > turned out to be a hints table, allowing driftx to advise me on the next step > - namely delete the local tables and rely on read repair.) > I'm submitting a patch reflecting this work. It addresses these issues, and > is perhaps best taken as a suggested guideline, maybe as an eye-opener. > I generally advocate two practices/good habits wrt. error reporting: > 1) Provide a good toString, reflecting the important state of an instance. > (In this case, file names carry valuable information.) > 2) Trapping and reporting errors is more useful at some points of the control > flow than in others. For instance, at a point in the code where we have > gathered all relevant parameters and are about to embark on major I/O (like > reading a table from disk). In these points, a try/catch/throw can be used > to capture this information. It's ok to catch Exception here, if you ask me, > as long as you wrap it as the cause of a new exception. This new exception > should describe the most important parameters involved in the failed call. > To do this, it should rely on the toStrings from item 1. > 2a) The same goes for log messages. Notice that the utility of a toString > multiplies, as it will emit useful information wherever it's used for logging > and/or exception messages. > By following these points, you will get stack traces partitioned up with a > chain of "Caused by" exceptions, each providing some information from its > relevant layer in the call stack. > I have applied this approach in the attached patch. As a result, we were > able to get the file name from a stacktrace that originally just said > "EOFException", and the problem could be easily fixed. > Caveats: Yes, you can get insanely long exception messages, and/or log > messages. To keep the post-mortem sessions manageable, constant tuning and > re-tuning of toStrings and log/exception messages is needed. However, even a > terrifyingly verbose exception won't be _less_ meaningful to the user than a > nameless EOFException, and it will convey more information to the developers. > (Even if that information boils down to "this toString is way more verbose > than it needs to be".) > In short: This patch helps with detecting corrupt SSTables and bogus bloom > filers, and I hope it inspires someone to give out more informative error > messages as well. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (CASSANDRA-2202) Better error reporting and meaningful toStrings
Better error reporting and meaningful toStrings --- Key: CASSANDRA-2202 URL: https://issues.apache.org/jira/browse/CASSANDRA-2202 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.7.2 Reporter: Kjetil Valstadsve Priority: Minor Having been through some trouble with the upgrade to 0.7.2 via 0.7.1, I've learned the hard way that Cassandra's error reporting can be improved. In particular, I had issues with deserialization of bloom filters, and with some corrupt SSTables. Chatting with driftx on the IRC channel, I complained about lack of error reporting, and showed him a stacktrace that indicated a corrupted SSTable. I then added various toString implementations and some new try/catch/rethrow points in critical control flow, which revealed the name of the SSTable. (It turned out to be a hints table, allowing driftx to advise me on the next step - namely delete the local tables and rely on read repair.) I'm submitting a patch reflecting this work. It addresses these issues, and is perhaps best taken as a suggested guideline, maybe as an eye-opener. I generally advocate two practices/good habits wrt. error reporting: 1) Provide a good toString, reflecting the important state of an instance. (In this case, file names carry valuable information.) 2) Trapping and reporting errors is more useful at some points of the control flow than in others. For instance, at a point in the code where we have gathered all relevant parameters and are about to embark on major I/O (like reading a table from disk). In these points, a try/catch/throw can be used to capture this information. It's ok to catch Exception here, if you ask me, as long as you wrap it as the cause of a new exception. This new exception should describe the most important parameters involved in the failed call. To do this, it should rely on the toStrings from item 1. 2a) The same goes for log messages. Notice that the utility of a toString multiplies, as it will emit useful information wherever it's used for logging and/or exception messages. By following these points, you will get stack traces partitioned up with a chain of "Caused by" exceptions, each providing some information from its relevant layer in the call stack. I have applied this approach in the attached patch. As a result, we were able to get the file name from a stacktrace that originally just said "EOFException", and the problem could be easily fixed. Caveats: Yes, you can get insanely long exception messages, and/or log messages. To keep the post-mortem sessions manageable, constant tuning and re-tuning of toStrings and log/exception messages is needed. However, even a terrifyingly verbose exception won't be _less_ meaningful to the user than a nameless EOFException, and it will convey more information to the developers. (Even if that information boils down to "this toString is way more verbose than it needs to be".) In short: This patch helps with detecting corrupt SSTables and bogus bloom filers, and I hope it inspires someone to give out more informative error messages as well. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Trivial Update of "FrontPage_JP" by MakiWatanabe
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "FrontPage_JP" page has been changed by MakiWatanabe. The comment on this change is: Translated: Operations. http://wiki.apache.org/cassandra/FrontPage_JP?action=diff&rev1=69&rev2=70 -- * [[StorageConfiguration_JP|Storageの設定]] （翻訳済み） * [[MultinodeCluster_JP|マルチノードクラスターの作成]]（翻訳済み） - * [[Operations_JP|オペレーション]]（翻訳中） + * [[Operations_JP|オペレーション]]（翻訳済み） * [[Embedding_JP|Cassandraの組み込み]]（翻訳済み） * [[MemtableThresholds|Memtableの閾値]] と [[PerformanceTuning|パフォーマンスチューニング]] * [[CassandraHardware_JP|Cassandraで使うハードウェアの注意点]]（翻訳済み）
[Cassandra Wiki] Update of "Operations_JP" by MakiWatanabe
Dear Wiki user,
You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for
change notification.
The "Operations_JP" page has been changed by MakiWatanabe.
The comment on this change is: Translation to Japanese completed.
http://wiki.apache.org/cassandra/Operations_JP?action=diff&rev1=95&rev2=96
--
注意: バージョン0.7より、json2sstableとsstable2jsonはschemaをsystem
tableからロードできるような環境で実行する必要があります。これはcassandra.yamlファイルがclasspath上に存在し、適切なストレージディレクトリを参照するように構成されていることを意味します。
== 監視 ==
+
Cassandraは内部の統計情報をJMXデータとして公開しています。これはJVMの世界では一般的です。程度に差はあるもののOpenNMS、Nagios、MuninでJMXがサポートされています。JMXインターフェースの仕様はJmxInterfaceで解説されています。
- Cassandraは内部の統計情報をJMXデータとして公開しています。これはJVMの世界では一般的です。OpenNMS、Nagios
- Cassandra exposes internal metrics as JMX data. This is a common standard in
the JVM world; OpenNMS, Nagios, and Munin at least offer some level of JMX
support. The specifics of the JMX Interface are documented at JmxInterface.
- Running `nodetool cfstats` can provide an overview of each Column Family, and
important metrics to graph your cluster. Some folks prefer having to deal with
non-jmx clients, there is a JMX-to-REST bridge available at
http://code.google.com/p/polarrose-jmx-rest-bridge/
+ `nodetool
cfstats`を実行するとカラムファミリごとの概略とクラスタの状態を示す主要な統計情報を取得できます。JMX以外のクライアントを使いたい場合には、JMXからRESTへのブリッジが利用可能です。http://code.google.com/p/polarrose-jmx-rest-bridge/
- Important metrics to watch on a per-Column Family basis would be: '''Read
Count, Read Latency, Write Count and Write Latency'''. '''Pending Tasks''' tell
you if things are backing up. These metrics can also be exposed using any JMX
client such as `jconsole`. (See also
[[http://simplygenius.com/2010/08/jconsole-via-socks-ssh-tunnel.html]] for how
to proxy JConsole to firewalled machines.)
+ カラムファミリ単位の主要統計情報には次のような情報が含まれます: '''Read Count、Read Latency、Write Count、Write
Latency'''
+ '''Pending
Tasks'''を見ると滞留しているタスクを知ることができます。これらの統計情報は`jconsole`のようなJMXクライアントでも確認できます。（JConsoleをファイアウォールの背後のマシンにプロキシするためには
[[http://simplygenius.com/2010/08/jconsole-via-socks-ssh-tunnel.html]]を参照して下さい。）
- You can also use jconsole, and the MBeans tab to look at PendingTasks for
thread pools. If you see one particular thread backing up, this can give you an
indication of a problem. One example would be ROW-MUTATION-STAGE indicating
that write requests are arriving faster than they can be handled. A more subtle
example is the FLUSH stages: if these start backing up, cassandra is accepting
writes into memory fast enough, but the sort-and-write-to-disk stages are
falling behind.
+
スレッドプールのPendingTaskはconsoleのMBeansタブでも確認できます。もし特定のスレッドが滞留している場合、何か問題が発生している可能性があります。例えばROW-MUTATION-STAGEで滞留している場合、流入するwriteリクエストが多すぎて処理が追いついていないことを示します。より精妙な例はFLUSHステージです。FLUSHステージで滞留が発生している場合、Cassandraはwriteを十分に速くメモリに読み込んでいますが、ソート・ディスクへの書き込みで遅延が発生していることを示しています。
- If you are seeing a lot of tasks being built up, your hardware or
configuration tuning is probably the bottleneck.
+ 大量のタスクが滞留しているなら、おそらくハードウェアかパラメータチューニングがボトルネックです。
- Running `nodetool tpstats` will dump all of those threads to console if you
don't want to use jconsole. Example:
+ jconsoleを使いたくないのであれば、`nodetool tpstats`を実行することですべてのスレッドをダンプ出力できます。
+ 例:
{{{
Pool NameActive Pending Completed
FILEUTILS-DELETE-POOL 0 0119
@@ -248, +249 @@
HINTED-HANDOFF-POOL 0 0154
}}}
- == Monitoring with MX4J ==
- mx4j provides an HTML and HTTP interface to JMX. Starting from version 0.7.0
cassandra lets you hook up mx4j very easily.
- To enable mx4j on a Cassandra node:
- * Download mx4j-tools.jar from http://mx4j.sourceforge.net/
- * Add mx4j-tools.jar to the classpath (e.g. under lib/)
- * Start cassandra
- * In the log you should see a message such as Http``Atapter started on port
8081
- * To choose a different port (8081 is the default) or a different listen
address (0.0.0.0 is not the default) edit conf/cassandra-env.sh and uncomment
#MX4J_ADDRESS="-Dmx4jaddress=0.0.0.0" and #MX4J_PORT="-Dmx4jport=8081"
+ == MX4Jによる監視 ==
+ mx4jはJMXへのHTMLおよびHTTPインターフェースを提供します。Cassandra 0.7.0よりmx4jをより簡単に統合できます。
+ Cassandraノードでmx4jを有効にする手順は次の通りです:
+ * http://mx4j.sourceforge.net/ から mx4j-tools.jar をダウンロードします。
+ * mx4j-tools.jar をclasspathに追加します。（lib/ の下など）
+ * Cassandraを起動します。
+ * ログにHttp``Atapter started on port 8081のようなメッセージが出力されていることを確認します。
+ *
標準ポート（8081）以外のポートや、標準アドレス（0.0.0.0）以外でlistenしたい場合はconf/cassandra-env.shを編集し、#MX4J_ADDRESS="-Dmx4jaddress=0.0.0.0"
and #MX4J_PORT="-Dmx4jport=8081" がコメントアウトされている部分を戻して下さい。
- Now browse to http://cassandra:8081/ and use the HTML interface.
+ http://cassandra:8081/ にアクセスすればHTMLインターフェースが使用できます。
- If you want XML then add &template=identity to the end of any URL, e.g.
http://cassandra:8081/?&template=identity
+ XMLが必要ならURLの最後に&template=identityを追加して下さい。（例:
http://cassandra:8081/?&template=identity ）
- For more details see https
