Re: Is the secondary index re-built under compaction?
Yes secondary index builds are done via the compaction manager. Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 17/04/2012, at 1:06 PM, Maxim Potekhin wrote: I noticed that nodetool compactionstats shows the building of the secondary index while I initiate compaction. Is this to be expected? Cassandra version 0.8.8. Thank you Maxim
Re: Is the secondary index re-built under compaction?
Thanks Aaaron. Just to be clear, every time I do a compaction, I rebuild all indexes from scratch. Right? Maxim On 4/17/2012 6:16 AM, aaron morton wrote: Yes secondary index builds are done via the compaction manager. Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 17/04/2012, at 1:06 PM, Maxim Potekhin wrote: I noticed that nodetool compactionstats shows the building of the secondary index while I initiate compaction. Is this to be expected? Cassandra version 0.8.8. Thank you Maxim
Re: Is the secondary index re-built under compaction?
No, the indexes are not rebuilt every compaction. Only if you manually rebuild or bootstrap a new node does it use compaction manager to rebuild. On Tue, Apr 17, 2012 at 9:47 AM, Maxim Potekhin potek...@bnl.gov wrote: Thanks Aaaron. Just to be clear, every time I do a compaction, I rebuild all indexes from scratch. Right? Maxim On 4/17/2012 6:16 AM, aaron morton wrote: Yes secondary index builds are done via the compaction manager. Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 17/04/2012, at 1:06 PM, Maxim Potekhin wrote: I noticed that nodetool compactionstats shows the building of the secondary index while I initiate compaction. Is this to be expected? Cassandra version 0.8.8. Thank you Maxim -- http://twitter.com/tjake
Re: Is the secondary index re-built under compaction?
Thanks Jake. Then I am definitely seeing weirdness, as there are tons of pending tasks in compaction stats, and tons of index files created in the data directory. Plus it does tell me that it is building the secondary index, and that seems to be happening at an amazingly glacial pace. I have 2 CFs there, with multiple secondary indexes. I'll try to compact the CF one by one, reboot and see if that helps. Maxim On 4/17/2012 9:53 AM, Jake Luciani wrote: No, the indexes are not rebuilt every compaction. Only if you manually rebuild or bootstrap a new node does it use compaction manager to rebuild. On Tue, Apr 17, 2012 at 9:47 AM, Maxim Potekhin potek...@bnl.gov mailto:potek...@bnl.gov wrote: Thanks Aaaron. Just to be clear, every time I do a compaction, I rebuild all indexes from scratch. Right? Maxim On 4/17/2012 6:16 AM, aaron morton wrote: Yes secondary index builds are done via the compaction manager. Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 17/04/2012, at 1:06 PM, Maxim Potekhin wrote: I noticed that nodetool compactionstats shows the building of the secondary index while I initiate compaction. Is this to be expected? Cassandra version 0.8.8. Thank you Maxim -- http://twitter.com/tjake
Re: Is the secondary index re-built under compaction?
Well, the since the secondary indexes are themselves column families they too are compacted along with everything else. On Tue, Apr 17, 2012 at 10:02 AM, Maxim Potekhin potek...@bnl.gov wrote: Thanks Jake. Then I am definitely seeing weirdness, as there are tons of pending tasks in compaction stats, and tons of index files created in the data directory. Plus it does tell me that it is building the secondary index, and that seems to be happening at an amazingly glacial pace. I have 2 CFs there, with multiple secondary indexes. I'll try to compact the CF one by one, reboot and see if that helps. Maxim On 4/17/2012 9:53 AM, Jake Luciani wrote: No, the indexes are not rebuilt every compaction. Only if you manually rebuild or bootstrap a new node does it use compaction manager to rebuild. On Tue, Apr 17, 2012 at 9:47 AM, Maxim Potekhin potek...@bnl.gov wrote: Thanks Aaaron. Just to be clear, every time I do a compaction, I rebuild all indexes from scratch. Right? Maxim On 4/17/2012 6:16 AM, aaron morton wrote: Yes secondary index builds are done via the compaction manager. Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 17/04/2012, at 1:06 PM, Maxim Potekhin wrote: I noticed that nodetool compactionstats shows the building of the secondary index while I initiate compaction. Is this to be expected? Cassandra version 0.8.8. Thank you Maxim -- http://twitter.com/tjake -- http://twitter.com/tjake
Re: Is the secondary index re-built under compaction?
I understand that indexes are CFs. But the compaction stats says it's building the index, not compacting the corresponding CF. Either that's an ambiguous diagnostic, or indeed something is not right with my rig as of late. Maxim On 4/17/2012 10:05 AM, Jake Luciani wrote: Well, the since the secondary indexes are themselves column families they too are compacted along with everything else. On Tue, Apr 17, 2012 at 10:02 AM, Maxim Potekhin potek...@bnl.gov mailto:potek...@bnl.gov wrote: Thanks Jake. Then I am definitely seeing weirdness, as there are tons of pending tasks in compaction stats, and tons of index files created in the data directory. Plus it does tell me that it is building the secondary index, and that seems to be happening at an amazingly glacial pace. I have 2 CFs there, with multiple secondary indexes. I'll try to compact the CF one by one, reboot and see if that helps. Maxim On 4/17/2012 9:53 AM, Jake Luciani wrote: No, the indexes are not rebuilt every compaction. Only if you manually rebuild or bootstrap a new node does it use compaction manager to rebuild. On Tue, Apr 17, 2012 at 9:47 AM, Maxim Potekhin potek...@bnl.gov mailto:potek...@bnl.gov wrote: Thanks Aaaron. Just to be clear, every time I do a compaction, I rebuild all indexes from scratch. Right? Maxim On 4/17/2012 6:16 AM, aaron morton wrote: Yes secondary index builds are done via the compaction manager. Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 17/04/2012, at 1:06 PM, Maxim Potekhin wrote: I noticed that nodetool compactionstats shows the building of the secondary index while I initiate compaction. Is this to be expected? Cassandra version 0.8.8. Thank you Maxim -- http://twitter.com/tjake -- http://twitter.com/tjake
Re: Is the secondary index re-built under compaction?
Hmm that does sound fishy. When you run show keyspaces from cassandra-cli it shows which indexes are built. Are they marked built in your column family? -Jake On Tue, Apr 17, 2012 at 10:09 AM, Maxim Potekhin potek...@bnl.gov wrote: I understand that indexes are CFs. But the compaction stats says it's building the index, not compacting the corresponding CF. Either that's an ambiguous diagnostic, or indeed something is not right with my rig as of late. Maxim On 4/17/2012 10:05 AM, Jake Luciani wrote: Well, the since the secondary indexes are themselves column families they too are compacted along with everything else. On Tue, Apr 17, 2012 at 10:02 AM, Maxim Potekhin potek...@bnl.gov wrote: Thanks Jake. Then I am definitely seeing weirdness, as there are tons of pending tasks in compaction stats, and tons of index files created in the data directory. Plus it does tell me that it is building the secondary index, and that seems to be happening at an amazingly glacial pace. I have 2 CFs there, with multiple secondary indexes. I'll try to compact the CF one by one, reboot and see if that helps. Maxim On 4/17/2012 9:53 AM, Jake Luciani wrote: No, the indexes are not rebuilt every compaction. Only if you manually rebuild or bootstrap a new node does it use compaction manager to rebuild. On Tue, Apr 17, 2012 at 9:47 AM, Maxim Potekhin potek...@bnl.gov wrote: Thanks Aaaron. Just to be clear, every time I do a compaction, I rebuild all indexes from scratch. Right? Maxim On 4/17/2012 6:16 AM, aaron morton wrote: Yes secondary index builds are done via the compaction manager. Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 17/04/2012, at 1:06 PM, Maxim Potekhin wrote: I noticed that nodetool compactionstats shows the building of the secondary index while I initiate compaction. Is this to be expected? Cassandra version 0.8.8. Thank you Maxim -- http://twitter.com/tjake -- http://twitter.com/tjake -- http://twitter.com/tjake
Re: Is the secondary index re-built under compaction?
Yes. Sorry I didn't mention this, but of course I'm checking on indexes once in a while. So yes, they are marked as built. All of this started happening after a few days of continuous loading process. Since the nodes have good hardware (24 cores + SSD), the apparent load on each node was nothing remarkable, even at 20kHz insertion rate. But maybe I'm being overoptimistic. Maxim On 4/17/2012 10:12 AM, Jake Luciani wrote: Hmm that does sound fishy. When you run show keyspaces from cassandra-cli it shows which indexes are built. Are they marked built in your column family? -Jake On Tue, Apr 17, 2012 at 10:09 AM, Maxim Potekhin potek...@bnl.gov mailto:potek...@bnl.gov wrote: I understand that indexes are CFs. But the compaction stats says it's building the index, not compacting the corresponding CF. Either that's an ambiguous diagnostic, or indeed something is not right with my rig as of late. Maxim On 4/17/2012 10:05 AM, Jake Luciani wrote: Well, the since the secondary indexes are themselves column families they too are compacted along with everything else. On Tue, Apr 17, 2012 at 10:02 AM, Maxim Potekhin potek...@bnl.gov mailto:potek...@bnl.gov wrote: Thanks Jake. Then I am definitely seeing weirdness, as there are tons of pending tasks in compaction stats, and tons of index files created in the data directory. Plus it does tell me that it is building the secondary index, and that seems to be happening at an amazingly glacial pace. I have 2 CFs there, with multiple secondary indexes. I'll try to compact the CF one by one, reboot and see if that helps. Maxim On 4/17/2012 9:53 AM, Jake Luciani wrote: No, the indexes are not rebuilt every compaction. Only if you manually rebuild or bootstrap a new node does it use compaction manager to rebuild. On Tue, Apr 17, 2012 at 9:47 AM, Maxim Potekhin potek...@bnl.gov mailto:potek...@bnl.gov wrote: Thanks Aaaron. Just to be clear, every time I do a compaction, I rebuild all indexes from scratch. Right? Maxim On 4/17/2012 6:16 AM, aaron morton wrote: Yes secondary index builds are done via the compaction manager. Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 17/04/2012, at 1:06 PM, Maxim Potekhin wrote: I noticed that nodetool compactionstats shows the building of the secondary index while I initiate compaction. Is this to be expected? Cassandra version 0.8.8. Thank you Maxim -- http://twitter.com/tjake -- http://twitter.com/tjake -- http://twitter.com/tjake
Re: Is the secondary index re-built under compaction?
How many indexes are there? On Tue, Apr 17, 2012 at 10:16 AM, Maxim Potekhin potek...@bnl.gov wrote: Yes. Sorry I didn't mention this, but of course I'm checking on indexes once in a while. So yes, they are marked as built. All of this started happening after a few days of continuous loading process. Since the nodes have good hardware (24 cores + SSD), the apparent load on each node was nothing remarkable, even at 20kHz insertion rate. But maybe I'm being overoptimistic. Maxim On 4/17/2012 10:12 AM, Jake Luciani wrote: Hmm that does sound fishy. When you run show keyspaces from cassandra-cli it shows which indexes are built. Are they marked built in your column family? -Jake On Tue, Apr 17, 2012 at 10:09 AM, Maxim Potekhin potek...@bnl.govwrote: I understand that indexes are CFs. But the compaction stats says it's building the index, not compacting the corresponding CF. Either that's an ambiguous diagnostic, or indeed something is not right with my rig as of late. Maxim On 4/17/2012 10:05 AM, Jake Luciani wrote: Well, the since the secondary indexes are themselves column families they too are compacted along with everything else. On Tue, Apr 17, 2012 at 10:02 AM, Maxim Potekhin potek...@bnl.govwrote: Thanks Jake. Then I am definitely seeing weirdness, as there are tons of pending tasks in compaction stats, and tons of index files created in the data directory. Plus it does tell me that it is building the secondary index, and that seems to be happening at an amazingly glacial pace. I have 2 CFs there, with multiple secondary indexes. I'll try to compact the CF one by one, reboot and see if that helps. Maxim On 4/17/2012 9:53 AM, Jake Luciani wrote: No, the indexes are not rebuilt every compaction. Only if you manually rebuild or bootstrap a new node does it use compaction manager to rebuild. On Tue, Apr 17, 2012 at 9:47 AM, Maxim Potekhin potek...@bnl.govwrote: Thanks Aaaron. Just to be clear, every time I do a compaction, I rebuild all indexes from scratch. Right? Maxim On 4/17/2012 6:16 AM, aaron morton wrote: Yes secondary index builds are done via the compaction manager. Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 17/04/2012, at 1:06 PM, Maxim Potekhin wrote: I noticed that nodetool compactionstats shows the building of the secondary index while I initiate compaction. Is this to be expected? Cassandra version 0.8.8. Thank you Maxim -- http://twitter.com/tjake -- http://twitter.com/tjake -- http://twitter.com/tjake -- http://twitter.com/tjake
Re: Is the secondary index re-built under compaction?
The offending CF only has one. The other one, that seems to behave well, has nine. Maxim On 4/17/2012 10:20 AM, Jake Luciani wrote: How many indexes are there? On Tue, Apr 17, 2012 at 10:16 AM, Maxim Potekhin potek...@bnl.gov mailto:potek...@bnl.gov wrote: Yes. Sorry I didn't mention this, but of course I'm checking on indexes once in a while. So yes, they are marked as built. All of this started happening after a few days of continuous loading process. Since the nodes have good hardware (24 cores + SSD), the apparent load on each node was nothing remarkable, even at 20kHz insertion rate. But maybe I'm being overoptimistic. Maxim On 4/17/2012 10:12 AM, Jake Luciani wrote: Hmm that does sound fishy. When you run show keyspaces from cassandra-cli it shows which indexes are built. Are they marked built in your column family? -Jake On Tue, Apr 17, 2012 at 10:09 AM, Maxim Potekhin potek...@bnl.gov mailto:potek...@bnl.gov wrote: I understand that indexes are CFs. But the compaction stats says it's building the index, not compacting the corresponding CF. Either that's an ambiguous diagnostic, or indeed something is not right with my rig as of late. Maxim On 4/17/2012 10:05 AM, Jake Luciani wrote: Well, the since the secondary indexes are themselves column families they too are compacted along with everything else. On Tue, Apr 17, 2012 at 10:02 AM, Maxim Potekhin potek...@bnl.gov mailto:potek...@bnl.gov wrote: Thanks Jake. Then I am definitely seeing weirdness, as there are tons of pending tasks in compaction stats, and tons of index files created in the data directory. Plus it does tell me that it is building the secondary index, and that seems to be happening at an amazingly glacial pace. I have 2 CFs there, with multiple secondary indexes. I'll try to compact the CF one by one, reboot and see if that helps. Maxim On 4/17/2012 9:53 AM, Jake Luciani wrote: No, the indexes are not rebuilt every compaction. Only if you manually rebuild or bootstrap a new node does it use compaction manager to rebuild. On Tue, Apr 17, 2012 at 9:47 AM, Maxim Potekhin potek...@bnl.gov mailto:potek...@bnl.gov wrote: Thanks Aaaron. Just to be clear, every time I do a compaction, I rebuild all indexes from scratch. Right? Maxim On 4/17/2012 6:16 AM, aaron morton wrote: Yes secondary index builds are done via the compaction manager. Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 17/04/2012, at 1:06 PM, Maxim Potekhin wrote: I noticed that nodetool compactionstats shows the building of the secondary index while I initiate compaction. Is this to be expected? Cassandra version 0.8.8. Thank you Maxim -- http://twitter.com/tjake -- http://twitter.com/tjake -- http://twitter.com/tjake -- http://twitter.com/tjake
Is the secondary index re-built under compaction?
I noticed that nodetool compactionstats shows the building of the secondary index while I initiate compaction. Is this to be expected? Cassandra version 0.8.8. Thank you Maxim
