[
https://issues.apache.org/jira/browse/CASSANDRA-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Schuller updated CASSANDRA-3417:
--------------------------------------
Attachment: CASSANDRA-3417-tokenmap.txt
{{CASSANDRA\-3417\-tokenmap.txt}} attached.
> InvocationTargetException ConcurrentModificationException at startup
> --------------------------------------------------------------------
>
> Key: CASSANDRA-3417
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3417
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 1.0.0
> Reporter: Joaquin Casares
> Assignee: Peter Schuller
> Priority: Minor
> Fix For: 1.0.8
>
> Attachments: 3417-2.txt, 3417-3.txt, 3417.txt,
> CASSANDRA-3417-tokenmap.txt
>
>
> I was starting up the new DataStax AMI where the seed starts first and 34
> nodes would latch on together. So far things have been working decently for
> launching, but right now I just got this during startup.
> {CODE}
> ubuntu@ip-10-40-190-143:~$ sudo cat /var/log/cassandra/output.log
> INFO 09:24:38,453 JVM vendor/version: Java HotSpot(TM) 64-Bit Server
> VM/1.6.0_26
> INFO 09:24:38,456 Heap size: 1936719872/1937768448
> INFO 09:24:38,457 Classpath:
> /usr/share/cassandra/lib/antlr-3.2.jar:/usr/share/cassandra/lib/avro-1.4.0-fixes.jar:/usr/share/cassandra/lib/avro-1.4.0-sources-fixes.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang-2.4.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.2.jar:/usr/share/cassandra/lib/guava-r08.jar:/usr/share/cassandra/lib/high-scale-lib-1.1.2.jar:/usr/share/cassandra/lib/jackson-core-asl-1.4.0.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.4.0.jar:/usr/share/cassandra/lib/jamm-0.2.5.jar:/usr/share/cassandra/lib/jline-0.9.94.jar:/usr/share/cassandra/lib/joda-time-1.6.2.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.6.jar:/usr/share/cassandra/lib/log4j-1.2.16.jar:/usr/share/cassandra/lib/servlet-api-2.5-20081211.jar:/usr/share/cassandra/lib/slf4j-api-1.6.1.jar:/usr/share/cassandra/lib/slf4j-log4j12-1.6.1.jar:/usr/share/cassandra/lib/snakeyaml-1.6.jar:/usr/share/cassandra/lib/snappy-java-1.0.3.jar:/usr/share/cassandra/apache-cassandra-1.0.0.jar:/usr/share/cassandra/apache-cassandra-thrift-1.0.0.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/java/jna.jar:/etc/cassandra:/usr/share/java/commons-daemon.jar:/usr/share/cassandra/lib/jamm-0.2.5.jar
> INFO 09:24:39,891 JNA mlockall successful
> INFO 09:24:39,901 Loading settings from file:/etc/cassandra/cassandra.yaml
> INFO 09:24:40,057 DiskAccessMode 'auto' determined to be mmap,
> indexAccessMode is mmap
> INFO 09:24:40,069 Global memtable threshold is enabled at 616MB
> INFO 09:24:40,159 EC2Snitch using region: us-east, zone: 1d.
> INFO 09:24:40,475 Creating new commitlog segment
> /raid0/cassandra/commitlog/CommitLog-1319793880475.log
> INFO 09:24:40,486 Couldn't detect any schema definitions in local storage.
> INFO 09:24:40,486 Found table data in data directories. Consider using the
> CLI to define your schema.
> INFO 09:24:40,497 No commitlog files found; skipping replay
> INFO 09:24:40,501 Cassandra version: 1.0.0
> INFO 09:24:40,502 Thrift API version: 19.18.0
> INFO 09:24:40,502 Loading persisted ring state
> INFO 09:24:40,506 Starting up server gossip
> INFO 09:24:40,529 Enqueuing flush of
> Memtable-LocationInfo@1388314661(190/237 serialized/live bytes, 4 ops)
> INFO 09:24:40,530 Writing Memtable-LocationInfo@1388314661(190/237
> serialized/live bytes, 4 ops)
> INFO 09:24:40,600 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-1-Data.db (298 bytes)
> INFO 09:24:40,613 Ec2Snitch adding ApplicationState ec2region=us-east
> ec2zone=1d
> INFO 09:24:40,621 Starting Messaging Service on /10.40.190.143:7000
> INFO 09:24:40,628 Joining: waiting for ring and schema information
> INFO 09:24:43,389 InetAddress /10.194.29.156 is now dead.
> INFO 09:24:43,391 InetAddress /10.85.11.38 is now dead.
> INFO 09:24:43,392 InetAddress /10.34.42.28 is now dead.
> INFO 09:24:43,393 InetAddress /10.77.63.49 is now dead.
> INFO 09:24:43,394 InetAddress /10.194.22.191 is now dead.
> INFO 09:24:43,395 InetAddress /10.34.74.58 is now dead.
> INFO 09:24:43,395 Node /10.34.33.16 is now part of the cluster
> INFO 09:24:43,396 InetAddress /10.34.33.16 is now UP
> INFO 09:24:43,397 Enqueuing flush of Memtable-LocationInfo@1629818866(20/25
> serialized/live bytes, 1 ops)
> INFO 09:24:43,398 Writing Memtable-LocationInfo@1629818866(20/25
> serialized/live bytes, 1 ops)
> INFO 09:24:43,417 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-2-Data.db (74 bytes)
> INFO 09:24:43,418 InetAddress /10.202.67.43 is now dead.
> INFO 09:24:43,419 InetAddress /10.116.215.81 is now dead.
> INFO 09:24:43,420 InetAddress /10.99.39.242 is now dead.
> INFO 09:24:43,421 InetAddress /10.80.110.28 is now dead.
> INFO 09:24:43,422 InetAddress /10.118.233.198 is now dead.
> INFO 09:24:43,423 InetAddress /10.40.177.173 is now dead.
> INFO 09:24:43,424 InetAddress /10.205.23.34 is now dead.
> INFO 09:24:43,425 InetAddress /10.101.41.8 is now dead.
> INFO 09:24:43,669 InetAddress /10.118.230.219 is now dead.
> INFO 09:24:43,670 InetAddress /10.80.41.192 is now dead.
> INFO 09:24:43,671 InetAddress /10.40.22.224 is now dead.
> INFO 09:24:43,672 InetAddress /10.39.107.114 is now dead.
> INFO 09:24:46,164 InetAddress /10.118.185.68 is now dead.
> INFO 09:24:46,166 InetAddress /10.84.205.93 is now dead.
> INFO 09:24:46,167 InetAddress /10.116.134.183 is now dead.
> INFO 09:24:46,670 InetAddress /10.118.179.67 is now dead.
> INFO 09:24:46,671 InetAddress /10.116.241.250 is now dead.
> INFO 09:24:48,441 InetAddress /10.118.94.62 is now dead.
> INFO 09:24:48,442 InetAddress /10.99.86.251 is now dead.
> INFO 09:24:50,176 InetAddress /10.113.42.21 is now dead.
> INFO 09:24:50,177 InetAddress /10.34.159.72 is now dead.
> INFO 09:24:50,178 InetAddress /10.32.79.134 is now dead.
> INFO 09:24:50,179 InetAddress /10.80.210.38 is now dead.
> INFO 09:24:50,180 InetAddress /10.34.70.73 is now dead.
> INFO 09:24:50,181 InetAddress /10.196.79.240 is now dead.
> INFO 09:25:01,713 InetAddress /10.82.210.172 is now dead.
> INFO 09:25:06,202 InetAddress /10.80.110.28 is now UP
> INFO 09:25:06,908 InetAddress /10.99.39.242 is now UP
> INFO 09:25:07,696 InetAddress /10.118.233.198 is now UP
> INFO 09:25:07,697 InetAddress /10.205.23.34 is now UP
> INFO 09:25:08,704 InetAddress /10.194.22.191 is now UP
> INFO 09:25:08,705 InetAddress /10.40.177.173 is now UP
> INFO 09:25:08,706 InetAddress /10.101.41.8 is now UP
> INFO 09:25:09,489 InetAddress /10.202.67.43 is now UP
> INFO 09:25:09,698 InetAddress /10.77.63.49 is now UP
> INFO 09:25:10,628 Joining: getting bootstrap token
> INFO 09:25:10,631 Enqueuing flush of Memtable-LocationInfo@1733057335(36/45
> serialized/live bytes, 1 ops)
> INFO 09:25:10,631 Writing Memtable-LocationInfo@1733057335(36/45
> serialized/live bytes, 1 ops)
> INFO 09:25:10,647 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-3-Data.db (87 bytes)
> INFO 09:25:10,649 Joining: sleeping 30000 ms for pending range setup
> INFO 09:25:10,689 InetAddress /10.85.11.38 is now UP
> INFO 09:25:10,708 InetAddress /10.34.74.58 is now UP
> INFO 09:25:10,912 InetAddress /10.194.29.156 is now UP
> INFO 09:25:11,261 Applying migration bb843dd0-0146-11e1-0000-b877c09da5ff
> Add keyspace: OpsCenter, rep
> strategy:SimpleStrategy{org.apache.cassandra.config.CFMetaData@55e29b99[cfId=1000,ksName=OpsCenter,cfName=pdps,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType,subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=300.0,readRepairChance=0.25,replicateOnWrite=true,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=43200,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.ConcurrentLinkedHashCacheProvider@105585dc,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
>
> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionOptions={}],
>
> org.apache.cassandra.config.CFMetaData@5ec736e4[cfId=1004,ksName=OpsCenter,cfName=rollups86400,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType,subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=50.0,readRepairChance=0.25,replicateOnWrite=true,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=2,maxCompactionThreshold=8,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=43200,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.ConcurrentLinkedHashCacheProvider@68e4e358,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
>
> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionOptions={}],
>
> org.apache.cassandra.config.CFMetaData@b09dc35[cfId=1003,ksName=OpsCenter,cfName=rollups7200,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType,subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=50.0,readRepairChance=0.25,replicateOnWrite=true,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=2,maxCompactionThreshold=8,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=43200,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.ConcurrentLinkedHashCacheProvider@3458213c,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
>
> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionOptions={}],
>
> org.apache.cassandra.config.CFMetaData@5ee04fd[cfId=1002,ksName=OpsCenter,cfName=rollups300,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType,subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=50.0,readRepairChance=0.25,replicateOnWrite=true,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=16,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=43200,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.ConcurrentLinkedHashCacheProvider@4d898115,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
>
> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionOptions={}],
>
> org.apache.cassandra.config.CFMetaData@7e79b177[cfId=1005,ksName=OpsCenter,cfName=events,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType,subcolumncomparator=<null>,comment=OpsCenter
> raw event
> storage,rowCacheSize=0.0,keyCacheSize=50.0,readRepairChance=0.25,replicateOnWrite=true,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=8,maxCompactionThreshold=12,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=43200,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.ConcurrentLinkedHashCacheProvider@67723c7f,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
>
> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionOptions={}],
>
> org.apache.cassandra.config.CFMetaData@540523be[cfId=1006,ksName=OpsCenter,cfName=events_timeline,cfType=Standard,comparator=org.apache.cassandra.db.marshal.LongType,subcolumncomparator=<null>,comment=OpsCenter
> event
> timelines,rowCacheSize=0.0,keyCacheSize=5.0,readRepairChance=0.25,replicateOnWrite=true,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=8,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=0,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.ConcurrentLinkedHashCacheProvider@1d6dba0a,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
>
> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionOptions={}],
>
> org.apache.cassandra.config.CFMetaData@ed0f59e[cfId=1007,ksName=OpsCenter,cfName=settings,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType,subcolumncomparator=<null>,comment=OpsCenter
>
> settings,rowCacheSize=0.0,keyCacheSize=50.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=8,maxCompactionThreshold=12,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=43200,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.ConcurrentLinkedHashCacheProvider@38ad5fab,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
>
> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionOptions={}],
>
> org.apache.cassandra.config.CFMetaData@7e63f09e[cfId=1001,ksName=OpsCenter,cfName=rollups60,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType,subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=50.0,readRepairChance=0.25,replicateOnWrite=true,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=43200,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.ConcurrentLinkedHashCacheProvider@534a55e5,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
>
> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionOptions={}]},
> durable_writes: true
> INFO 09:25:11,273 Enqueuing flush of
> Memtable-Migrations@1767199109(12925/16156 serialized/live bytes, 1 ops)
> INFO 09:25:11,273 Writing Memtable-Migrations@1767199109(12925/16156
> serialized/live bytes, 1 ops)
> INFO 09:25:11,274 Enqueuing flush of Memtable-Schema@1616586953(5820/7275
> serialized/live bytes, 3 ops)
> INFO 09:25:11,358 Completed flushing
> /raid0/cassandra/data/system/Migrations-h-1-Data.db (12989 bytes)
> INFO 09:25:11,358 Writing Memtable-Schema@1616586953(5820/7275
> serialized/live bytes, 3 ops)
> INFO 09:25:11,390 Completed flushing
> /raid0/cassandra/data/system/Schema-h-1-Data.db (5970 bytes)
> INFO 09:25:11,727 InetAddress /10.116.215.81 is now UP
> INFO 09:25:11,744 InetAddress /10.34.42.28 is now UP
> INFO 09:25:11,750 InetAddress /10.40.22.224 is now UP
> INFO 09:25:12,023 InetAddress /10.80.41.192 is now UP
> INFO 09:25:12,712 InetAddress /10.39.107.114 is now UP
> INFO 09:25:12,717 InetAddress /10.118.185.68 is now UP
> INFO 09:25:12,721 InetAddress /10.116.134.183 is now UP
> INFO 09:25:13,322 InetAddress /10.118.230.219 is now UP
> INFO 09:25:13,632 InetAddress /10.84.205.93 is now UP
> INFO 09:25:14,713 InetAddress /10.118.179.67 is now UP
> INFO 09:25:14,717 InetAddress /10.116.241.250 is now UP
> INFO 09:25:17,468 InetAddress /10.34.159.72 is now UP
> INFO 09:25:17,476 InetAddress /10.118.94.62 is now UP
> INFO 09:25:17,480 InetAddress /10.80.210.38 is now UP
> INFO 09:25:17,716 InetAddress /10.32.79.134 is now UP
> INFO 09:25:17,721 InetAddress /10.99.86.251 is now UP
> INFO 09:25:18,717 InetAddress /10.196.79.240 is now UP
> INFO 09:25:18,727 InetAddress /10.34.70.73 is now UP
> INFO 09:25:19,596 InetAddress /10.113.42.21 is now UP
> INFO 09:25:25,750 InetAddress /10.82.210.172 is now UP
> INFO 09:25:37,743 Enqueuing flush of Memtable-LocationInfo@288976631(35/43
> serialized/live bytes, 1 ops)
> INFO 09:25:37,744 Writing Memtable-LocationInfo@288976631(35/43
> serialized/live bytes, 1 ops)
> INFO 09:25:37,764 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-4-Data.db (89 bytes)
> INFO 09:25:37,773 Compacting
> [SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-1-Data.db'),
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-3-Data.db'),
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-4-Data.db'),
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-2-Data.db')]
> INFO 09:25:37,776 Enqueuing flush of Memtable-LocationInfo@1950702248(35/43
> serialized/live bytes, 1 ops)
> INFO 09:25:37,777 Writing Memtable-LocationInfo@1950702248(35/43
> serialized/live bytes, 1 ops)
> INFO 09:25:37,821 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-5-Data.db (89 bytes)
> INFO 09:25:37,869 Compacted to
> [/raid0/cassandra/data/system/LocationInfo-h-6-Data.db,]. 548 to 443 (~80%
> of original) bytes for 3 keys at 0.006500MB/s. Time: 65ms.
> INFO 09:25:38,740 Enqueuing flush of Memtable-LocationInfo@92917455(35/43
> serialized/live bytes, 1 ops)
> INFO 09:25:38,740 Writing Memtable-LocationInfo@92917455(35/43
> serialized/live bytes, 1 ops)
> INFO 09:25:38,757 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-8-Data.db (89 bytes)
> INFO 09:25:38,766 Enqueuing flush of Memtable-LocationInfo@1096488363(35/43
> serialized/live bytes, 1 ops)
> INFO 09:25:38,767 Writing Memtable-LocationInfo@1096488363(35/43
> serialized/live bytes, 1 ops)
> INFO 09:25:38,814 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-9-Data.db (89 bytes)
> INFO 09:25:38,816 Compacting
> [SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-6-Data.db'),
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-9-Data.db'),
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-8-Data.db'),
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-5-Data.db')]
> INFO 09:25:38,823 Enqueuing flush of Memtable-LocationInfo@1734564525(35/43
> serialized/live bytes, 1 ops)
> INFO 09:25:38,823 Writing Memtable-LocationInfo@1734564525(35/43
> serialized/live bytes, 1 ops)
> INFO 09:25:38,893 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-10-Data.db (89 bytes)
> INFO 09:25:38,916 Compacted to
> [/raid0/cassandra/data/system/LocationInfo-h-11-Data.db,]. 710 to 548 (~77%
> of original) bytes for 3 keys at 0.005226MB/s. Time: 100ms.
> INFO 09:25:39,538 Enqueuing flush of Memtable-LocationInfo@811507066(35/43
> serialized/live bytes, 1 ops)
> INFO 09:25:39,539 Writing Memtable-LocationInfo@811507066(35/43
> serialized/live bytes, 1 ops)
> INFO 09:25:39,555 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-13-Data.db (89 bytes)
> INFO 09:25:39,578 Enqueuing flush of Memtable-LocationInfo@1125690366(35/43
> serialized/live bytes, 1 ops)
> INFO 09:25:39,578 Writing Memtable-LocationInfo@1125690366(35/43
> serialized/live bytes, 1 ops)
> INFO 09:25:39,594 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-14-Data.db (89 bytes)
> INFO 09:25:39,596 Compacting
> [SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-11-Data.db'),
>
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-10-Data.db'),
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-14-Data.db'),
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-13-Data.db')]
> INFO 09:25:39,613 Enqueuing flush of Memtable-LocationInfo@1870148830(35/43
> serialized/live bytes, 1 ops)
> INFO 09:25:39,614 Writing Memtable-LocationInfo@1870148830(35/43
> serialized/live bytes, 1 ops)
> INFO 09:25:39,652 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-16-Data.db (89 bytes)
> INFO 09:25:39,692 Compacted to
> [/raid0/cassandra/data/system/LocationInfo-h-15-Data.db,]. 815 to 653 (~80%
> of original) bytes for 3 keys at 0.006487MB/s. Time: 96ms.
> INFO 09:25:39,731 Enqueuing flush of Memtable-LocationInfo@1279866611(35/43
> serialized/live bytes, 1 ops)
> INFO 09:25:39,731 Writing Memtable-LocationInfo@1279866611(35/43
> serialized/live bytes, 1 ops)
> INFO 09:25:39,747 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-18-Data.db (89 bytes)
> INFO 09:25:40,649 Starting to bootstrap...
> INFO 09:25:40,701 Finished streaming session 304272969286 from /10.205.23.34
> INFO 09:25:40,703 Enqueuing flush of Memtable-LocationInfo@1868577756(53/66
> serialized/live bytes, 2 ops)
> INFO 09:25:40,703 Writing Memtable-LocationInfo@1868577756(53/66
> serialized/live bytes, 2 ops)
> INFO 09:25:40,721 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-19-Data.db (163 bytes)
> INFO 09:25:40,722 Compacting
> [SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-19-Data.db'),
>
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-15-Data.db'),
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-18-Data.db'),
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-16-Data.db')]
> INFO 09:25:40,726 Node /10.40.190.143 state jump to normal
> INFO 09:25:40,734 Enqueuing flush of Memtable-LocationInfo@641287650(35/43
> serialized/live bytes, 1 ops)
> INFO 09:25:40,735 Writing Memtable-LocationInfo@641287650(35/43
> serialized/live bytes, 1 ops)
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.commons.daemon.support.DaemonLoader.load(DaemonLoader.java:160)
> Caused by: java.util.ConcurrentModificationException
> at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
> at java.util.HashMap$EntryIterator.next(HashMap.java:834)
> at java.util.HashMap$EntryIterator.next(HashMap.java:832)
> at
> com.google.common.collect.AbstractBiMap$EntrySet$1.next(AbstractBiMap.java:301)
> at
> com.google.common.collect.AbstractBiMap$EntrySet$1.next(AbstractBiMap.java:293)
> at
> org.apache.cassandra.service.StorageService.calculatePendingRanges(StorageService.java:1127)
> at
> org.apache.cassandra.service.StorageService.calculatePendingRanges(StorageService.java:1084)
> at
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:920)
> at
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:805)
> at org.apache.cassandra.gms.Gossiper.doNotifications(Gossiper.java:880)
> at
> org.apache.cassandra.gms.Gossiper.addLocalApplicationState(Gossiper.java:1027)
> at
> org.apache.cassandra.service.StorageService.setToken(StorageService.java:226)
> at
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:573)
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:460)
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:381)
> at
> org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:215)
> at
> org.apache.cassandra.service.AbstractCassandraDaemon.init(AbstractCassandraDaemon.java:238)
> ... 5 more
> Cannot load daemon
> Service exit with a return value of 3
> INFO 09:35:35,156 JVM vendor/version: Java HotSpot(TM) 64-Bit Server
> VM/1.6.0_26
> INFO 09:35:35,159 Heap size: 1936719872/1937768448
> INFO 09:35:35,160 Classpath:
> /usr/share/cassandra/lib/antlr-3.2.jar:/usr/share/cassandra/lib/avro-1.4.0-fixes.jar:/usr/share/cassandra/lib/avro-1.4.0-sources-fixes.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang-2.4.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.2.jar:/usr/share/cassandra/lib/guava-r08.jar:/usr/share/cassandra/lib/high-scale-lib-1.1.2.jar:/usr/share/cassandra/lib/jackson-core-asl-1.4.0.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.4.0.jar:/usr/share/cassandra/lib/jamm-0.2.5.jar:/usr/share/cassandra/lib/jline-0.9.94.jar:/usr/share/cassandra/lib/joda-time-1.6.2.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.6.jar:/usr/share/cassandra/lib/log4j-1.2.16.jar:/usr/share/cassandra/lib/servlet-api-2.5-20081211.jar:/usr/share/cassandra/lib/slf4j-api-1.6.1.jar:/usr/share/cassandra/lib/slf4j-log4j12-1.6.1.jar:/usr/share/cassandra/lib/snakeyaml-1.6.jar:/usr/share/cassandra/lib/snappy-java-1.0.3.jar:/usr/share/cassandra/apache-cassandra-1.0.0.jar:/usr/share/cassandra/apache-cassandra-thrift-1.0.0.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/java/jna.jar:/etc/cassandra:/usr/share/java/commons-daemon.jar:/usr/share/cassandra/lib/jamm-0.2.5.jar
> INFO 09:35:36,626 JNA mlockall successful
> INFO 09:35:36,636 Loading settings from file:/etc/cassandra/cassandra.yaml
> INFO 09:35:36,757 DiskAccessMode 'auto' determined to be mmap,
> indexAccessMode is mmap
> INFO 09:35:36,769 Global memtable threshold is enabled at 616MB
> INFO 09:35:36,811 EC2Snitch using region: us-east, zone: 1d.
> INFO 09:35:37,030 Opening /raid0/cassandra/data/system/Schema-h-1 (5970
> bytes)
> INFO 09:35:37,067 Opening /raid0/cassandra/data/system/Migrations-h-1 (12989
> bytes)
> INFO 09:35:37,075 Opening /raid0/cassandra/data/system/LocationInfo-h-19
> (163 bytes)
> INFO 09:35:37,075 Opening /raid0/cassandra/data/system/LocationInfo-h-18 (89
> bytes)
> INFO 09:35:37,083 Opening /raid0/cassandra/data/system/LocationInfo-h-15
> (653 bytes)
> INFO 09:35:37,085 Opening /raid0/cassandra/data/system/LocationInfo-h-16 (89
> bytes)
> INFO 09:35:37,131 Loading schema version bb843dd0-0146-11e1-0000-b877c09da5ff
> INFO 09:35:37,372 Creating new commitlog segment
> /raid0/cassandra/commitlog/CommitLog-1319794537372.log
> INFO 09:35:37,384 Replaying
> /raid0/cassandra/commitlog/CommitLog-1319793880475.log
> INFO 09:35:37,416 Finished reading
> /raid0/cassandra/commitlog/CommitLog-1319793880475.log
> INFO 09:35:37,422 Enqueuing flush of Memtable-events@1830423861(164/205
> serialized/live bytes, 5 ops)
> INFO 09:35:37,423 Writing Memtable-events@1830423861(164/205 serialized/live
> bytes, 5 ops)
> INFO 09:35:37,424 Enqueuing flush of Memtable-Versions@817138449(83/103
> serialized/live bytes, 3 ops)
> INFO 09:35:37,472 Completed flushing
> /raid0/cassandra/data/OpsCenter/events-h-1-Data.db (230 bytes)
> INFO 09:35:37,479 Writing Memtable-Versions@817138449(83/103 serialized/live
> bytes, 3 ops)
> INFO 09:35:37,497 Completed flushing
> /raid0/cassandra/data/system/Versions-h-1-Data.db (247 bytes)
> INFO 09:35:37,497 Log replay complete, 4 replayed mutations
> INFO 09:35:37,509 Cassandra version: 1.0.0
> INFO 09:35:37,510 Thrift API version: 19.18.0
> INFO 09:35:37,510 Loading persisted ring state
> INFO 09:35:37,528 Starting up server gossip
> INFO 09:35:37,530 Enqueuing flush of Memtable-LocationInfo@1655441108(29/36
> serialized/live bytes, 1 ops)
> INFO 09:35:37,530 Writing Memtable-LocationInfo@1655441108(29/36
> serialized/live bytes, 1 ops)
> INFO 09:35:37,554 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-20-Data.db (80 bytes)
> INFO 09:35:37,555 Ec2Snitch adding ApplicationState ec2region=us-east
> ec2zone=1d
> INFO 09:35:37,562 Compacting
> [SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-16-Data.db'),
>
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-18-Data.db'),
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-19-Data.db'),
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-20-Data.db'),
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-15-Data.db')]
> INFO 09:35:37,566 Starting Messaging Service on /10.40.190.143:7000
> INFO 09:35:37,592 Using saved token 19444706681196483626478548996101040654
> INFO 09:35:37,593 Enqueuing flush of Memtable-LocationInfo@995684858(53/66
> serialized/live bytes, 2 ops)
> INFO 09:35:37,593 Writing Memtable-LocationInfo@995684858(53/66
> serialized/live bytes, 2 ops)
> INFO 09:35:37,616 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-22-Data.db (163 bytes)
> INFO 09:35:37,620 Node /10.40.190.143 state jump to normal
> INFO 09:35:37,639 Bootstrap/Replace/Move completed! Now serving reads.
> INFO 09:35:37,640 Will not load MX4J, mx4j-tools.jar is not in the classpath
> INFO 09:35:37,684 Binding thrift service to /0.0.0.0:9160
> INFO 09:35:37,687 Compacted to
> [/raid0/cassandra/data/system/LocationInfo-h-21-Data.db,]. 1,074 to 799
> (~74% of original) bytes for 4 keys at 0.007620MB/s. Time: 100ms.
> INFO 09:35:37,688 Using TFastFramedTransport with a max frame size of
> 15728640 bytes.
> INFO 09:35:37,692 Using synchronous/threadpool thrift server on /0.0.0.0 :
> 9160
> INFO 09:35:37,695 Listening for thrift clients...
> INFO 09:35:37,706 Node /10.118.230.219 is now part of the cluster
> INFO 09:35:37,707 InetAddress /10.118.230.219 is now UP
> INFO 09:35:37,708 Enqueuing flush of Memtable-LocationInfo@2035487037(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:37,709 Writing Memtable-LocationInfo@2035487037(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:37,725 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-24-Data.db (89 bytes)
> INFO 09:35:37,726 Node /10.34.42.28 is now part of the cluster
> INFO 09:35:37,727 InetAddress /10.34.42.28 is now UP
> INFO 09:35:37,729 Enqueuing flush of Memtable-LocationInfo@321887181(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:37,729 Writing Memtable-LocationInfo@321887181(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:37,747 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-25-Data.db (89 bytes)
> INFO 09:35:37,748 Node /10.77.63.49 has restarted, now UP
> INFO 09:35:37,748 Compacting
> [SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-24-Data.db'),
>
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-22-Data.db'),
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-25-Data.db'),
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-21-Data.db')]
> INFO 09:35:37,748 InetAddress /10.77.63.49 is now UP
> INFO 09:35:37,749 Node /10.77.63.49 state jump to normal
> INFO 09:35:37,750 Node /10.34.70.73 is now part of the cluster
> INFO 09:35:37,750 InetAddress /10.34.70.73 is now UP
> INFO 09:35:37,752 Enqueuing flush of Memtable-LocationInfo@1354749546(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:37,752 Writing Memtable-LocationInfo@1354749546(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:37,789 Compacted to
> [/raid0/cassandra/data/system/LocationInfo-h-26-Data.db,]. 1,140 to 877
> (~76% of original) bytes for 4 keys at 0.020399MB/s. Time: 41ms.
> INFO 09:35:37,801 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-27-Data.db (89 bytes)
> INFO 09:35:37,801 Node /10.99.86.251 is now part of the cluster
> INFO 09:35:37,802 InetAddress /10.99.86.251 is now UP
> INFO 09:35:37,803 Enqueuing flush of Memtable-LocationInfo@793374785(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:37,804 Writing Memtable-LocationInfo@793374785(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:37,825 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-29-Data.db (89 bytes)
> INFO 09:35:37,826 Node /10.202.67.43 has restarted, now UP
> INFO 09:35:37,827 InetAddress /10.202.67.43 is now UP
> INFO 09:35:37,827 Node /10.202.67.43 state jump to normal
> INFO 09:35:37,828 Node /10.116.134.183 is now part of the cluster
> INFO 09:35:37,828 InetAddress /10.116.134.183 is now UP
> INFO 09:35:37,829 Enqueuing flush of Memtable-LocationInfo@1728699027(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:37,830 Writing Memtable-LocationInfo@1728699027(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:37,850 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-30-Data.db (89 bytes)
> INFO 09:35:37,852 Compacting
> [SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-30-Data.db'),
>
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-27-Data.db'),
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-26-Data.db'),
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-29-Data.db')]
> INFO 09:35:37,853 Node /10.118.94.62 is now part of the cluster
> INFO 09:35:37,853 InetAddress /10.118.94.62 is now UP
> INFO 09:35:37,855 Enqueuing flush of Memtable-LocationInfo@2001229122(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:37,855 Writing Memtable-LocationInfo@2001229122(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:37,885 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-31-Data.db (89 bytes)
> INFO 09:35:37,886 Node /10.116.215.81 is now part of the cluster
> INFO 09:35:37,887 InetAddress /10.116.215.81 is now UP
> INFO 09:35:37,888 Enqueuing flush of Memtable-LocationInfo@1748800276(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:37,888 Writing Memtable-LocationInfo@1748800276(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:37,909 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-33-Data.db (89 bytes)
> INFO 09:35:37,910 Node /10.80.110.28 has restarted, now UP
> INFO 09:35:37,911 InetAddress /10.80.110.28 is now UP
> INFO 09:35:37,911 Node /10.80.110.28 state jump to normal
> INFO 09:35:37,912 Node /10.80.210.38 is now part of the cluster
> INFO 09:35:37,912 InetAddress /10.80.210.38 is now UP
> INFO 09:35:37,914 Enqueuing flush of Memtable-LocationInfo@1761382005(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:37,914 Writing Memtable-LocationInfo@1761382005(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:37,925 Compacted to
> [/raid0/cassandra/data/system/LocationInfo-h-32-Data.db,]. 1,144 to 982
> (~85% of original) bytes for 4 keys at 0.014190MB/s. Time: 66ms.
> INFO 09:35:37,927 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-35-Data.db (89 bytes)
> INFO 09:35:37,928 Node /10.40.177.173 has restarted, now UP
> INFO 09:35:37,929 Compacting
> [SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-31-Data.db'),
>
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-32-Data.db'),
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-33-Data.db'),
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-35-Data.db')]
> INFO 09:35:37,929 InetAddress /10.40.177.173 is now UP
> INFO 09:35:37,929 Node /10.40.177.173 state jump to normal
> INFO 09:35:37,930 Node /10.101.41.8 has restarted, now UP
> INFO 09:35:37,931 InetAddress /10.101.41.8 is now UP
> INFO 09:35:37,931 Node /10.101.41.8 state jump to normal
> INFO 09:35:37,931 Node /10.205.23.34 has restarted, now UP
> INFO 09:35:37,932 InetAddress /10.205.23.34 is now UP
> INFO 09:35:37,932 Node /10.205.23.34 state jump to normal
> INFO 09:35:37,933 Node /10.118.185.68 is now part of the cluster
> INFO 09:35:37,933 InetAddress /10.118.185.68 is now UP
> INFO 09:35:37,934 Enqueuing flush of Memtable-LocationInfo@260440278(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:37,935 Writing Memtable-LocationInfo@260440278(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:37,970 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-36-Data.db (89 bytes)
> INFO 09:35:37,971 Node /10.116.241.250 is now part of the cluster
> INFO 09:35:37,972 InetAddress /10.116.241.250 is now UP
> INFO 09:35:37,973 Enqueuing flush of Memtable-LocationInfo@768673839(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:37,974 Writing Memtable-LocationInfo@768673839(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:38,003 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-38-Data.db (89 bytes)
> INFO 09:35:38,004 Node /10.113.42.21 is now part of the cluster
> INFO 09:35:38,005 InetAddress /10.113.42.21 is now UP
> INFO 09:35:38,007 Enqueuing flush of Memtable-LocationInfo@1610335061(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:38,008 Writing Memtable-LocationInfo@1610335061(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:38,014 Compacted to
> [/raid0/cassandra/data/system/LocationInfo-h-37-Data.db,]. 1,249 to 1,087
> (~87% of original) bytes for 4 keys at 0.012196MB/s. Time: 85ms.
> INFO 09:35:38,024 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-40-Data.db (89 bytes)
> INFO 09:35:38,024 Node /10.194.29.156 is now part of the cluster
> INFO 09:35:38,025 Compacting
> [SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-37-Data.db'),
>
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-40-Data.db'),
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-36-Data.db'),
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-38-Data.db')]
> INFO 09:35:38,025 InetAddress /10.194.29.156 is now UP
> INFO 09:35:38,026 Enqueuing flush of Memtable-LocationInfo@1625488363(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:38,027 Writing Memtable-LocationInfo@1625488363(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:38,042 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-41-Data.db (89 bytes)
> INFO 09:35:38,043 Node /10.85.11.38 has restarted, now UP
> INFO 09:35:38,044 InetAddress /10.85.11.38 is now UP
> INFO 09:35:38,044 Node /10.85.11.38 state jump to normal
> INFO 09:35:38,045 Node /10.34.159.72 is now part of the cluster
> INFO 09:35:38,046 InetAddress /10.34.159.72 is now UP
> INFO 09:35:38,047 Enqueuing flush of Memtable-LocationInfo@747881713(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:38,048 Writing Memtable-LocationInfo@747881713(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:38,065 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-42-Data.db (89 bytes)
> INFO 09:35:38,067 Node /10.194.22.191 is now part of the cluster
> INFO 09:35:38,067 InetAddress /10.194.22.191 is now UP
> INFO 09:35:38,069 Enqueuing flush of Memtable-LocationInfo@709926392(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:38,069 Writing Memtable-LocationInfo@709926392(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:38,092 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-44-Data.db (89 bytes)
> INFO 09:35:38,093 Node /10.34.74.58 is now part of the cluster
> INFO 09:35:38,097 InetAddress /10.34.74.58 is now UP
> INFO 09:35:38,098 Enqueuing flush of Memtable-LocationInfo@1356841826(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:38,099 Writing Memtable-LocationInfo@1356841826(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:38,105 Compacting
> [SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-43-Data.db'),
>
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-41-Data.db'),
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-44-Data.db'),
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-42-Data.db')]
> INFO 09:35:38,106 Compacted to
> [/raid0/cassandra/data/system/LocationInfo-h-43-Data.db,]. 1,354 to 1,192
> (~88% of original) bytes for 4 keys at 0.014034MB/s. Time: 81ms.
> INFO 09:35:38,144 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-46-Data.db (89 bytes)
> INFO 09:35:38,145 Node /10.40.22.224 is now part of the cluster
> INFO 09:35:38,146 InetAddress /10.40.22.224 is now UP
> INFO 09:35:38,147 Enqueuing flush of Memtable-LocationInfo@422797318(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:38,148 Writing Memtable-LocationInfo@422797318(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:38,155 Compacted to
> [/raid0/cassandra/data/system/LocationInfo-h-47-Data.db,]. 1,459 to 1,297
> (~88% of original) bytes for 4 keys at 0.024738MB/s. Time: 50ms.
> INFO 09:35:38,164 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-49-Data.db (89 bytes)
> INFO 09:35:38,165 Node /10.32.79.134 is now part of the cluster
> INFO 09:35:38,166 InetAddress /10.32.79.134 is now UP
> INFO 09:35:38,167 Enqueuing flush of Memtable-LocationInfo@1455093129(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:38,168 Writing Memtable-LocationInfo@1455093129(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:38,199 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-50-Data.db (89 bytes)
> INFO 09:35:38,200 Node /10.118.179.67 is now part of the cluster
> INFO 09:35:38,200 Compacting
> [SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-50-Data.db'),
>
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-47-Data.db'),
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-49-Data.db'),
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-46-Data.db')]
> INFO 09:35:38,200 InetAddress /10.118.179.67 is now UP
> INFO 09:35:38,202 Enqueuing flush of Memtable-LocationInfo@1105436908(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:38,202 Writing Memtable-LocationInfo@1105436908(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:38,248 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-51-Data.db (89 bytes)
> INFO 09:35:38,249 Node /10.84.205.93 is now part of the cluster
> INFO 09:35:38,249 InetAddress /10.84.205.93 is now UP
> INFO 09:35:38,251 Enqueuing flush of Memtable-LocationInfo@1306980591(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:38,251 Writing Memtable-LocationInfo@1306980591(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:38,262 Compacted to
> [/raid0/cassandra/data/system/LocationInfo-h-52-Data.db,]. 1,564 to 1,402
> (~89% of original) bytes for 4 keys at 0.021919MB/s. Time: 61ms.
> INFO 09:35:38,294 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-54-Data.db (89 bytes)
> INFO 09:35:38,294 Node /10.34.33.16 has restarted, now UP
> INFO 09:35:38,295 InetAddress /10.34.33.16 is now UP
> INFO 09:35:38,296 Node /10.34.33.16 state jump to normal
> INFO 09:35:38,296 Node /10.39.107.114 is now part of the cluster
> INFO 09:35:38,297 InetAddress /10.39.107.114 is now UP
> INFO 09:35:38,298 Enqueuing flush of Memtable-LocationInfo@1038389338(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:38,299 Writing Memtable-LocationInfo@1038389338(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:38,311 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-55-Data.db (89 bytes)
> INFO 09:35:38,312 Node /10.196.79.240 is now part of the cluster
> INFO 09:35:38,312 Compacting
> [SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-52-Data.db'),
>
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-55-Data.db'),
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-54-Data.db'),
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-51-Data.db')]
> INFO 09:35:38,313 InetAddress /10.196.79.240 is now UP
> INFO 09:35:38,314 Enqueuing flush of Memtable-LocationInfo@1850278722(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:38,315 Writing Memtable-LocationInfo@1850278722(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:38,354 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-56-Data.db (89 bytes)
> INFO 09:35:38,355 Node /10.99.39.242 has restarted, now UP
> INFO 09:35:38,356 InetAddress /10.99.39.242 is now UP
> INFO 09:35:38,356 Node /10.99.39.242 state jump to normal
> INFO 09:35:38,357 Node /10.118.233.198 has restarted, now UP
> INFO 09:35:38,358 InetAddress /10.118.233.198 is now UP
> INFO 09:35:38,358 Node /10.118.233.198 state jump to normal
> INFO 09:35:38,359 Node /10.82.210.172 is now part of the cluster
> INFO 09:35:38,359 InetAddress /10.82.210.172 is now UP
> INFO 09:35:38,364 Enqueuing flush of Memtable-LocationInfo@786665924(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:38,364 Writing Memtable-LocationInfo@786665924(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:38,439 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-58-Data.db (89 bytes)
> INFO 09:35:38,440 Node /10.80.41.192 is now part of the cluster
> INFO 09:35:38,440 InetAddress /10.80.41.192 is now UP
> INFO 09:35:38,442 Enqueuing flush of Memtable-LocationInfo@1647844754(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:38,442 Writing Memtable-LocationInfo@1647844754(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:38,451 Compacted to
> [/raid0/cassandra/data/system/LocationInfo-h-57-Data.db,]. 1,669 to 1,515
> (~90% of original) bytes for 4 keys at 0.010470MB/s. Time: 138ms.
> INFO 09:35:38,459 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-60-Data.db (89 bytes)
> INFO 09:35:38,459 Node /10.76.243.129 is now part of the cluster
> INFO 09:35:38,460 Compacting
> [SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-56-Data.db'),
>
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-58-Data.db'),
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-57-Data.db'),
> SSTableReader(path='/raid0/cassandra/data/system/LocationInfo-h-60-Data.db')]
> INFO 09:35:38,460 InetAddress /10.76.243.129 is now UP
> INFO 09:35:38,462 Enqueuing flush of Memtable-LocationInfo@585652261(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:38,462 Writing Memtable-LocationInfo@585652261(35/43
> serialized/live bytes, 1 ops)
> INFO 09:35:38,478 Completed flushing
> /raid0/cassandra/data/system/LocationInfo-h-61-Data.db (89 bytes)
> INFO 09:35:38,486 Node /10.34.42.28 state jump to normal
> INFO 09:35:38,487 Node /10.34.70.73 state jump to normal
> INFO 09:35:38,488 Node /10.99.86.251 state jump to normal
> INFO 09:35:38,489 Node /10.118.94.62 state jump to normal
> INFO 09:35:38,489 Node /10.80.110.28 state jump to normal
> INFO 09:35:38,490 Node /10.80.210.38 state jump to normal
> INFO 09:35:38,491 Node /10.40.177.173 state jump to normal
> INFO 09:35:38,493 Node /10.101.41.8 state jump to normal
> INFO 09:35:38,493 Node /10.113.42.21 state jump to normal
> INFO 09:35:38,494 Node /10.85.11.38 state jump to normal
> INFO 09:35:38,495 Node /10.34.159.72 state jump to normal
> INFO 09:35:38,496 Node /10.34.74.58 state jump to normal
> INFO 09:35:38,497 Node /10.84.205.93 state jump to normal
> INFO 09:35:38,497 Node /10.118.179.67 state jump to normal
> INFO 09:35:38,498 Node /10.34.33.16 state jump to normal
> INFO 09:35:38,499 Node /10.196.79.240 state jump to normal
> INFO 09:35:38,500 Node /10.118.233.198 state jump to normal
> INFO 09:35:38,501 Node /10.80.41.192 state jump to normal
> INFO 09:35:38,502 Node /10.76.243.129 state jump to normal
> INFO 09:35:38,508 Node /10.118.185.68 state jump to normal
> INFO 09:35:38,524 Node /10.118.230.219 state jump to normal
> INFO 09:35:38,536 Compacted to
> [/raid0/cassandra/data/system/LocationInfo-h-62-Data.db,]. 1,782 to 1,620
> (~90% of original) bytes for 4 keys at 0.020328MB/s. Time: 76ms.
> INFO 09:35:38,537 Node /10.80.110.28 state jump to normal
> INFO 09:35:38,537 Node /10.40.177.173 state jump to normal
> INFO 09:35:38,538 Node /10.101.41.8 state jump to normal
> INFO 09:35:38,539 Node /10.116.241.250 state jump to normal
> INFO 09:35:38,540 Node /10.194.29.156 state jump to normal
> INFO 09:35:38,540 Node /10.34.74.58 state jump to normal
> INFO 09:35:38,541 Node /10.40.22.224 state jump to normal
> INFO 09:35:38,542 Node /10.32.79.134 state jump to normal
> INFO 09:35:38,543 Node /10.39.107.114 state jump to normal
> INFO 09:35:38,543 Node /10.99.39.242 state jump to normal
> INFO 09:35:38,550 Node /10.77.63.49 state jump to normal
> INFO 09:35:38,550 Node /10.34.42.28 state jump to normal
> INFO 09:35:38,551 Node /10.116.134.183 state jump to normal
> INFO 09:35:38,553 Node /10.76.243.129 state jump to normal
> INFO 09:35:38,557 Node /10.202.67.43 state jump to normal
> INFO 09:35:38,558 Node /10.118.94.62 state jump to normal
> INFO 09:35:38,562 Node /10.116.215.81 state jump to normal
> INFO 09:35:38,563 Node /10.80.210.38 state jump to normal
> INFO 09:35:38,564 Node /10.205.23.34 state jump to normal
> INFO 09:35:38,565 Node /10.39.107.114 state jump to normal
> {CODE}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
