date:20171108


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245164#comment-16245164
 ] 

Paulo Motta commented on CASSANDRA-13215:
-

Good job, this is much nicer than having the StorageService manage the disk 
boundaries. Patch and tests LGTM.

Two minor nits:
 * could you make {{getDiskBoundaryValue}} and {{getDiskBoundaries}} static?
 * can you log the actual boundary changes to facilitate debugging?

Probably not a big deal but we will unnecessarily invalidate the disk 
boundaries whenever there is a keyspace change (table creation, drop, add view, 
etc) - rather then when replication settings or local ranges changes, do you 
think we should invalidate the boundaries only when replication settings/local 
range change or not really bother about this?

> Cassandra nodes startup time 20x more after upgarding to 3.x
> 
>
> Key: CASSANDRA-13215
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13215
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
> Environment: Cluster setup: two datacenters (dc-main, dc-backup).
> dc-main - 9 servers, no vnodes
> dc-backup - 6 servers, vnodes
>Reporter: Viktor Kuzmin
>Assignee: Marcus Eriksson
> Fix For: 3.11.x, 4.x
>
> Attachments: simple-cache.patch
>
>
> CompactionStrategyManage.getCompactionStrategyIndex is called on each sstable 
> at startup. And this function calls StorageService.getDiskBoundaries. And 
> getDiskBoundaries calls AbstractReplicationStrategy.getAddressRanges.
> It appears that last function can be really slow. In our environment we have 
> 1545 tokens and with NetworkTopologyStrategy it can make 1545*1545 
> computations in worst case (maybe I'm wrong, but it really takes lot's of 
> cpu).
> Also this function can affect runtime later, cause it is called not only 
> during startup.
> I've tried to implement simple cache for getDiskBoundaries results and now 
> startup time is about one minute instead of 25m, but I'm not sure if it's a 
> good solution.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13948) Reload compaction strategies when JBOD disk boundary changes


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245169#comment-16245169
 ] 

Paulo Motta commented on CASSANDRA-13948:
-

Just a heads up that internal CI looks good for the latest version of the patch 
(there are a bunch of unrelated failures on trunk). I will rebase this on top 
of CASSANDRA-13215 after the first round of review.

> Reload compaction strategies when JBOD disk boundary changes
> 
>
> Key: CASSANDRA-13948
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13948
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Paulo Motta
>Assignee: Paulo Motta
> Fix For: 3.11.x, 4.x
>
> Attachments: debug.log, threaddump-cleanup.txt, threaddump.txt, 
> trace.log
>
>
> The thread dump below shows a race between an sstable replacement by the 
> {{IndexSummaryRedistribution}} and 
> {{AbstractCompactionTask.getNextBackgroundTask}}:
> {noformat}
> Thread 94580: (state = BLOCKED)
>  - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information 
> may be imprecise)
>  - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
> line=175 (Compiled frame)
>  - 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() 
> @bci=1, line=836 (Compiled frame)
>  - 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(java.util.concurrent.locks.AbstractQueuedSynchronizer$Node,
>  int) @bci=67, line=870 (Compiled frame)
>  - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(int) 
> @bci=17, line=1199 (Compiled frame)
>  - java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock() @bci=5, 
> line=943 (Compiled frame)
>  - 
> org.apache.cassandra.db.compaction.CompactionStrategyManager.handleListChangedNotification(java.lang.Iterable,
>  java.lang.Iterable) @bci=359, line=483 (Interpreted frame)
>  - 
> org.apache.cassandra.db.compaction.CompactionStrategyManager.handleNotification(org.apache.cassandra.notifications.INotification,
>  java.lang.Object) @bci=53, line=555 (Interpreted frame)
>  - 
> org.apache.cassandra.db.lifecycle.Tracker.notifySSTablesChanged(java.util.Collection,
>  java.util.Collection, org.apache.cassandra.db.compaction.OperationType, 
> java.lang.Throwable) @bci=50, line=409 (Interpreted frame)
>  - 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.doCommit(java.lang.Throwable)
>  @bci=157, line=227 (Interpreted frame)
>  - 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.commit(java.lang.Throwable)
>  @bci=61, line=116 (Compiled frame)
>  - 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.commit()
>  @bci=2, line=200 (Interpreted frame)
>  - 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.finish()
>  @bci=5, line=185 (Interpreted frame)
>  - 
> org.apache.cassandra.io.sstable.IndexSummaryRedistribution.redistributeSummaries()
>  @bci=559, line=130 (Interpreted frame)
>  - 
> org.apache.cassandra.db.compaction.CompactionManager.runIndexSummaryRedistribution(org.apache.cassandra.io.sstable.IndexSummaryRedistribution)
>  @bci=9, line=1420 (Interpreted frame)
>  - 
> org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(org.apache.cassandra.io.sstable.IndexSummaryRedistribution)
>  @bci=4, line=250 (Interpreted frame)
>  - 
> org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries() 
> @bci=30, line=228 (Interpreted frame)
>  - org.apache.cassandra.io.sstable.IndexSummaryManager$1.runMayThrow() 
> @bci=4, line=125 (Interpreted frame)
>  - org.apache.cassandra.utils.WrappedRunnable.run() @bci=1, line=28 
> (Interpreted frame)
>  - 
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run()
>  @bci=4, line=118 (Compiled frame)
>  - java.util.concurrent.Executors$RunnableAdapter.call() @bci=4, line=511 
> (Compiled frame)
>  - java.util.concurrent.FutureTask.runAndReset() @bci=47, line=308 (Compiled 
> frame)
>  - 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask)
>  @bci=1, line=180 (Compiled frame)
>  - java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run() 
> @bci=37, line=294 (Compiled frame)
>  - 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
>  @bci=95, line=1149 (Compiled frame)
>  - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=624 
> (Interpreted frame)
>  - 
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(java.lang.Runnable)
>  @bci=1, line=81 (Interpreted frame)
>  -

[jira] [Commented] (CASSANDRA-14004) Increase Guava upstream version API compatibility

2017-11-08 Thread Thibault Kruse (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245155#comment-16245155
 ] 

Thibault Kruse commented on CASSANDRA-14004:


fair enough, close if you want. Might still be worth thinking about how to deal 
with future guava incompatibilities for cassandra 4.x +

> Increase Guava upstream version API compatibility
> -
>
> Key: CASSANDRA-14004
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14004
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Thibault Kruse
>Priority: Minor
>
> Using cassandra-all as a java library has the downside that it uses guava 18 
> methods that have been removed from more recent guava versions, causing users 
> to be forced to stick with Guava <= 19.0.
> It would be nice for us if such method calls to guava could be replaced with 
> code of higher compatibility, for Casandra versions as far back as 3.0.x
> As an example replacing code like this
> Iterators.emptyIterator()
> with
> Collections.emptyIterator()
> as done in
> https://github.com/krummas/cassandra/commits/marcuse/guava23
> Ensuring any cassandra-release is API compatible with several guava versions 
> is more painful, possibly a blacklist of method calls can be used via 
> checkstyle to reduce the risk of re-introducing version specific calls to 
> guava methods.
> Related to the Guava upgrade ticket: 
> https://issues.apache.org/jira/browse/CASSANDRA-13997



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14005) CqlBulkRecordWriter swallows exceptions


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vincent Dumont updated CASSANDRA-14005:
---
Description: 
The {{close()}} method in {{CqlBulkRecordWriter}} has the following:

{code:java}
try
{
future.get(1000, TimeUnit.MILLISECONDS);
break;
}
catch (ExecutionException | TimeoutException te)
{
if (null != progress)
progress.progress();
if (null != context)
HadoopCompat.progress(context);
}
catch (InterruptedException e)
{
throw new IOException(e);
}
{code}

And so any exception that happens when uploading SSTables silently gets 
swallowed and not surfaced.

  was:
The close() method in CqlBulkRecordWriter has the following:

{code:java}
try
{
future.get(1000, TimeUnit.MILLISECONDS);
break;
}
catch (ExecutionException | TimeoutException te)
{
if (null != progress)
progress.progress();
if (null != context)
HadoopCompat.progress(context);
}
catch (InterruptedException e)
{
throw new IOException(e);
}
{code}

And so any exception that happens when uploading SSTables silently gets 
swallowed and not surfaced.


> CqlBulkRecordWriter swallows exceptions
> ---
>
> Key: CASSANDRA-14005
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14005
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Vincent Dumont
>
> The {{close()}} method in {{CqlBulkRecordWriter}} has the following:
> {code:java}
> try
> {
> future.get(1000, TimeUnit.MILLISECONDS);
> break;
> }
> catch (ExecutionException | TimeoutException te)
> {
> if (null != progress)
> progress.progress();
> if (null != context)
> HadoopCompat.progress(context);
> }
> catch (InterruptedException e)
> {
> throw new IOException(e);
> }
> {code}
> And so any exception that happens when uploading SSTables silently gets 
> swallowed and not surfaced.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14005) CqlBulkRecordWriter swallows exceptions


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vincent Dumont updated CASSANDRA-14005:
---
Description: 
The close() method in CqlBulkRecordWriter has the following:

{code}
try
{
future.get(1000, TimeUnit.MILLISECONDS);
break;
}
catch (ExecutionException | TimeoutException te)
{
if (null != progress)
progress.progress();
if (null != context)
HadoopCompat.progress(context);
}
catch (InterruptedException e)
{
throw new IOException(e);
}
{/code}

And so any exception that happens when uploading SSTables silently gets 
swallowed and not surfaced.

  was:
The close() method in CqlBulkRecordWriter has the following:

{{try
{
future.get(1000, TimeUnit.MILLISECONDS);
break;
}
catch (ExecutionException | TimeoutException te)
{
if (null != progress)
progress.progress();
if (null != context)
HadoopCompat.progress(context);
}
catch (InterruptedException e)
{
throw new IOException(e);
}}}


And so any exception that happens when uploading SSTables silently gets 
swallowed and not surfaced.


> CqlBulkRecordWriter swallows exceptions
> ---
>
> Key: CASSANDRA-14005
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14005
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Vincent Dumont
>
> The close() method in CqlBulkRecordWriter has the following:
> {code}
> try
> {
> future.get(1000, TimeUnit.MILLISECONDS);
> break;
> }
> catch (ExecutionException | TimeoutException te)
> {
> if (null != progress)
> progress.progress();
> if (null != context)
> HadoopCompat.progress(context);
> }
> catch (InterruptedException e)
> {
> throw new IOException(e);
> }
> {/code}
> And so any exception that happens when uploading SSTables silently gets 
> swallowed and not surfaced.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14005) CqlBulkRecordWriter swallows exceptions


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vincent Dumont updated CASSANDRA-14005:
---
Description: 
The close() method in CqlBulkRecordWriter has the following:

{code:java}
try
{
future.get(1000, TimeUnit.MILLISECONDS);
break;
}
catch (ExecutionException | TimeoutException te)
{
if (null != progress)
progress.progress();
if (null != context)
HadoopCompat.progress(context);
}
catch (InterruptedException e)
{
throw new IOException(e);
}
{code}

And so any exception that happens when uploading SSTables silently gets 
swallowed and not surfaced.

  was:
The close() method in CqlBulkRecordWriter has the following:

{code}
try
{
future.get(1000, TimeUnit.MILLISECONDS);
break;
}
catch (ExecutionException | TimeoutException te)
{
if (null != progress)
progress.progress();
if (null != context)
HadoopCompat.progress(context);
}
catch (InterruptedException e)
{
throw new IOException(e);
}
{/code}

And so any exception that happens when uploading SSTables silently gets 
swallowed and not surfaced.


> CqlBulkRecordWriter swallows exceptions
> ---
>
> Key: CASSANDRA-14005
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14005
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Vincent Dumont
>
> The close() method in CqlBulkRecordWriter has the following:
> {code:java}
> try
> {
> future.get(1000, TimeUnit.MILLISECONDS);
> break;
> }
> catch (ExecutionException | TimeoutException te)
> {
> if (null != progress)
> progress.progress();
> if (null != context)
> HadoopCompat.progress(context);
> }
> catch (InterruptedException e)
> {
> throw new IOException(e);
> }
> {code}
> And so any exception that happens when uploading SSTables silently gets 
> swallowed and not surfaced.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14005) CqlBulkRecordWriter swallows exceptions

Vincent Dumont created CASSANDRA-14005:
--

 Summary: CqlBulkRecordWriter swallows exceptions
 Key: CASSANDRA-14005
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14005
 Project: Cassandra
  Issue Type: Bug
  Components: CQL
Reporter: Vincent Dumont


The close() method in CqlBulkRecordWriter has the following:

{{try
{
future.get(1000, TimeUnit.MILLISECONDS);
break;
}
catch (ExecutionException | TimeoutException te)
{
if (null != progress)
progress.progress();
if (null != context)
HadoopCompat.progress(context);
}
catch (InterruptedException e)
{
throw new IOException(e);
}}}


And so any exception that happens when uploading SSTables silently gets 
swallowed and not surfaced.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14004) Increase Guava upstream version API compatibility


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245117#comment-16245117
 ] 

Jason Brown commented on CASSANDRA-14004:
-

[~tkruse] Sorry, but we are not going to go back and rewrite 3.0/3.11 just for 
guava upgrades. Those branches are essentially in bug-fix mode and not really 
for new features (which is what this ticket amounts to). The database itself is 
stable with the version of guava it ships with, and you will have a very 
difficult time convincing the maintainers to risk instability for the sake of 
external applications that actually pull in the database server code.

> Increase Guava upstream version API compatibility
> -
>
> Key: CASSANDRA-14004
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14004
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Thibault Kruse
>Priority: Minor
>
> Using cassandra-all as a java library has the downside that it uses guava 18 
> methods that have been removed from more recent guava versions, causing users 
> to be forced to stick with Guava <= 19.0.
> It would be nice for us if such method calls to guava could be replaced with 
> code of higher compatibility, for Casandra versions as far back as 3.0.x
> As an example replacing code like this
> Iterators.emptyIterator()
> with
> Collections.emptyIterator()
> as done in
> https://github.com/krummas/cassandra/commits/marcuse/guava23
> Ensuring any cassandra-release is API compatible with several guava versions 
> is more painful, possibly a blacklist of method calls can be used via 
> checkstyle to reduce the risk of re-introducing version specific calls to 
> guava methods.
> Related to the Guava upgrade ticket: 
> https://issues.apache.org/jira/browse/CASSANDRA-13997



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-12245) initial view build can be parallel

[
https://issues.apache.org/jira/browse/CASSANDRA-12245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245074#comment-16245074
]

Paulo Motta edited comment on CASSANDRA-12245 at 11/9/17 1:41 AM:
--

Getting back to this after a while, sorry about the delay. We are really close
now, I think we can wrap up after this last round of review. See follow-up
below:

bq. This makes a lot of sense. I'm worried about creating thousands of tasks
for large datasets if the number of tasks is relative to the amount of data.
Instead, I think we could fix the number of partitions to the higher reasonable
number of parallel tasks, something like a multiple of the number of available
processors. This would provide the desired immediate performance improvement if
the user increases the number of concurrent view builders while keeping the
number of tasks limited, independently of the amount of data. What do you
think? Does it make any sense?

Awesome! One minor thing is that we should probably only split the view build
tasks at all if the base table is larger than a given size (let's say 500MB or
so?), to avoid 4 * num_processor flushes for base tables with negligible size,
WDYT?

bq. One case that we hadn't considered is that if the token ranges change or
are split in a different way when resuming a build then the progress would be
lost, because ViewBuildTask won't found any entry for the new range at
system.view_builds_in_progress. This would be specially true if we split the
ranges by their data size. So, independently of how we finally split the
ranges, I think it makes sense to load all the ranges with any progress from
system.view_builds_in_progress at ViewBuilder before splitting the local
ranges, create a task for those of them that are not already finish, and then
split any remaining uncovered local range. It also has the advantage of
skipping the creation of tasks for already completed ranges. What do you think?

Good idea! This will prevent a node from computing new tasks for already built
subranges if a new node joins the ring during view build. I will add a dtest to
verify that scenario on CASSANDRA-13762.

bq. I have also removed the method SystemKeyspace.beginViewBuild because I
don't see any need of saving a range without progress. Indeed, if the view
build is restarted it is probably better to don't restore the task without
progress and let their tokens to be processed by the split logic.

bq. The first wait does the opposite to _wait_for_view, it waits until there is
some progress. But we can use self._wait_for_view("ks", "t_by_v") here. Did you
mean that?

Yep.

bq. Makes sense. The new test for ViewBuilderTask is here, and I've extended
ViewTest.testViewBuilderResume to run with different number of concurrent view
builders.

Awesome, this looks much better now, thanks!

I had a final skim through the patch and I think we covered everything and this
looks much more robust now. Three minor things before we can close this:
- I noticed we don't stop in-progess view builds when a view is removed, would
you mind adding that?
- ViewBuildExecutor is being constructed with minThreads=1 and
maxPoolSize=concurrent_materialized_view_builders, but according to the
{{DebuggableThreadPoolExecutor}}'s'
[javadoc|https://github.com/apache/cassandra/blob/8b3a60b9a7dbefeecc06bace617279612ec7092d/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java#L33],
this will actually make the executor with size 1 since maxPoolSize is not
supported by {{DebuggableThreadPoolExecutor}} - and even if it were, new
threads would only be created after the queue of the initial threads were full
(which is quite unintuitive), but we actually want the pool to have
concurrent_materialized_view_builders concurrent threads at most, so we should
use the {{threadCount}} constructor instead - at some point we should actually
remove the maximumPoolSize constructors from the TPE's since it can be quite
confusing.
- In the linked dtest branch results,
{{base_replica_repair_with_contention_test}} is showing the following error:
{noformat}
ERROR [InternalResponseStage:1] 2017-10-11 11:03:16,005
CassandraDaemon.java:211 - Exception in thread
Thread[InternalResponseStage:1,5,main]
java.lang.RuntimeException: javax.management.InstanceAlreadyExistsException:
org.apache.cassandra.db:type=Tables,keyspace=ks,table=t
at
org.apache.cassandra.db.ColumnFamilyStore.(ColumnFamilyStore.java:484)
~[main/:na]
at
org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:639)
~[main/:na]
at
org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:613)
~[main/:na]
at
org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:604)
~[main/:na]
at

[jira] [Created] (CASSANDRA-14004) Increase Guava upstream version API compatibility

2017-11-08 Thread Thibault Kruse (JIRA)

Thibault Kruse created CASSANDRA-14004:
--

 Summary: Increase Guava upstream version API compatibility
 Key: CASSANDRA-14004
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14004
 Project: Cassandra
  Issue Type: Improvement
Reporter: Thibault Kruse
Priority: Minor


Using cassandra-all as a java library has the downside that it uses guava 18 
methods that have been removed from more recent guava versions, causing users 
to be forced to stick with Guava <= 19.0.

It would be nice for us if such method calls to guava could be replaced with 
code of higher compatibility, for Casandra versions as far back as 3.0.x

As an example replacing code like this

Iterators.emptyIterator()

with

Collections.emptyIterator()

as done in

https://github.com/krummas/cassandra/commits/marcuse/guava23

Ensuring any cassandra-release is API compatible with several guava versions is 
more painful, possibly a blacklist of method calls can be used via checkstyle 
to reduce the risk of re-introducing version specific calls to guava methods.

Related to the Guava upgrade ticket: 
https://issues.apache.org/jira/browse/CASSANDRA-13997



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12245) initial view build can be parallel

[
https://issues.apache.org/jira/browse/CASSANDRA-12245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245074#comment-16245074
]

Paulo Motta commented on CASSANDRA-12245:
-

Getting back to this after a while, sorry about the delay. We are really close
now, I think we can wrap up after this last round of review. See follow-up
below:

Good idea! This will prevent a node from computing new tasks for already built
subranges if a new node joins the ring during view build. I will add a dtest to
verify that scenario on CASSANDRA-13762.

bq. The first wait does the opposite to _wait_for_view, it waits until there is
some progress. But we can use self._wait_for_view("ks", "t_by_v") here. Did you
mean that?

Yep.

bq. Makes sense. The new test for ViewBuilderTask is here, and I've extended
ViewTest.testViewBuilderResume to run with different number of concurrent view
builders.

Awesome, this looks much better now, thanks!

[jira] [Updated] (CASSANDRA-13849) GossipStage blocks because of race in ActiveRepairService


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Blake Eggleston updated CASSANDRA-13849:

   Resolution: Fixed
Fix Version/s: 4.0
   Status: Resolved  (was: Patch Available)

I ran the trunk unit tests locally and everything passed. The rest of the test 
failures were either flaky tests, or unrelated tests that passed locally. 
Committed as {{49edd70740e2efae3681cb79a391369bfb7de02e}}.

Thanks [~slapukhov]!

> GossipStage blocks because of race in ActiveRepairService
> -
>
> Key: CASSANDRA-13849
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13849
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Tom van der Woerdt
>Assignee: Sergey Lapukhov
>  Labels: patch
> Fix For: 4.0, 3.0.x, 3.11.x
>
> Attachments: CAS-13849.patch, CAS-13849_2.patch, CAS-13849_3.patch
>
>
> Bad luck caused a kernel panic in a cluster, and that took another node with 
> it because GossipStage stopped responding.
> I think it's pretty obvious what's happening, here are the relevant excerpts 
> from the stack traces :
> {noformat}
> "Thread-24004" #393781 daemon prio=5 os_prio=0 tid=0x7efca9647400 
> nid=0xe75c waiting on condition [0x7efaa47fe000]
>java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for  <0x00052b63a7e8> (a 
> java.util.concurrent.CountDownLatch$Sync)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
> at 
> org.apache.cassandra.service.ActiveRepairService.prepareForRepair(ActiveRepairService.java:332)
> - locked <0x0002e6bc99f0> (a 
> org.apache.cassandra.service.ActiveRepairService)
> at 
> org.apache.cassandra.repair.RepairRunnable.runMayThrow(RepairRunnable.java:211)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)   
>   
>   at 
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> at 
> org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$3/1498438472.run(Unknown
>  Source)
> at java.lang.Thread.run(Thread.java:748)
> "GossipTasks:1" #367 daemon prio=5 os_prio=0 tid=0x7efc5e971000 
> nid=0x700b waiting for monitor entry [0x7dfb839fe000]
>java.lang.Thread.State: BLOCKED (on object monitor)
> at 
> org.apache.cassandra.service.ActiveRepairService.removeParentRepairSession(ActiveRepairService.java:421)
> - waiting to lock <0x0002e6bc99f0> (a 
> org.apache.cassandra.service.ActiveRepairService)
> at 
> org.apache.cassandra.service.ActiveRepairService.convict(ActiveRepairService.java:776)
> at 
> org.apache.cassandra.gms.FailureDetector.interpret(FailureDetector.java:306)
> at org.apache.cassandra.gms.Gossiper.doStatusCheck(Gossiper.java:775) 
>   
>  at 
> org.apache.cassandra.gms.Gossiper.access$800(Gossiper.java:67)
> at org.apache.cassandra.gms.Gossiper$GossipTask.run(Gossiper.java:187)
> at 
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at 
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> at 
> org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$3/1498438472.run(Unknown
>  Source)
> at java.lang.Thread.run(Thread.java:748)
> "GossipStage:1" #320 daemon prio=5 os_prio=0 tid=0x7efc5b9f2c00

[3/6] cassandra git commit: Fix repair race that caused gossip to block

Fix repair race that caused gossip to block

Patch by Sergey Lapukhov; Reviewed by Blake Eggleston for CASSANDRA-13849


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/49edd707
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/49edd707
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/49edd707

Branch: refs/heads/trunk
Commit: 49edd70740e2efae3681cb79a391369bfb7de02e
Parents: 58daf13
Author: Sergey Lapukhov 
Authored: Wed Nov 1 11:35:09 2017 +0100
Committer: Blake Eggleston 
Committed: Wed Nov 8 16:38:21 2017 -0800

--
 CHANGES.txt | 1 +
 .../org/apache/cassandra/service/ActiveRepairService.java   | 9 ++---
 2 files changed, 7 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/49edd707/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3f4f3f2..a94dbc9 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.16
+ * Fix repair race that caused gossip to block (CASSANDRA-13849)
  * Tracing interferes with digest requests when using RandomPartitioner 
(CASSANDRA-13964)
  * Add flag to disable materialized views, and warnings on creation 
(CASSANDRA-13959)
  * Don't let user drop or generally break tables in system_distributed 
(CASSANDRA-13813)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/49edd707/src/java/org/apache/cassandra/service/ActiveRepairService.java
--
diff --git a/src/java/org/apache/cassandra/service/ActiveRepairService.java 
b/src/java/org/apache/cassandra/service/ActiveRepairService.java
index b4cea79..adb6fab 100644
--- a/src/java/org/apache/cassandra/service/ActiveRepairService.java
+++ b/src/java/org/apache/cassandra/service/ActiveRepairService.java
@@ -280,7 +280,7 @@ public class ActiveRepairService implements 
IEndpointStateChangeSubscriber, IFai
 return neighbors;
 }
 
-public synchronized UUID prepareForRepair(UUID parentRepairSession, 
InetAddress coordinator, Set endpoints, RepairOption options, 
List columnFamilyStores)
+public UUID prepareForRepair(UUID parentRepairSession, InetAddress 
coordinator, Set endpoints, RepairOption options, 
List columnFamilyStores)
 {
 long timestamp = System.currentTimeMillis();
 registerParentRepairSession(parentRepairSession, coordinator, 
columnFamilyStores, options.getRanges(), options.isIncremental(), timestamp, 
options.isGlobal());
@@ -351,7 +351,7 @@ public class ActiveRepairService implements 
IEndpointStateChangeSubscriber, IFai
 throw new RuntimeException(errorMsg);
 }
 
-public void registerParentRepairSession(UUID parentRepairSession, 
InetAddress coordinator, List columnFamilyStores, 
Collection ranges, boolean isIncremental, long timestamp, boolean 
isGlobal)
+public synchronized void registerParentRepairSession(UUID 
parentRepairSession, InetAddress coordinator, List 
columnFamilyStores, Collection ranges, boolean isIncremental, 
long timestamp, boolean isGlobal)
 {
 if (!registeredForEndpointChanges)
 {
@@ -360,7 +360,10 @@ public class ActiveRepairService implements 
IEndpointStateChangeSubscriber, IFai
 registeredForEndpointChanges = true;
 }
 
-parentRepairSessions.put(parentRepairSession, new 
ParentRepairSession(coordinator, columnFamilyStores, ranges, isIncremental, 
timestamp, isGlobal));
+if (!parentRepairSessions.containsKey(parentRepairSession))
+{
+parentRepairSessions.put(parentRepairSession, new 
ParentRepairSession(coordinator, columnFamilyStores, ranges, isIncremental, 
timestamp, isGlobal));
+}
 }
 
 public Set currentlyRepairing(UUID cfId, UUID 
parentRepairSession)


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7d6474bb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7d6474bb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7d6474bb

Branch: refs/heads/cassandra-3.11
Commit: 7d6474bb4d5e8fe940ad0a75652524ca221dab62
Parents: df37bcb 49edd70
Author: Blake Eggleston 
Authored: Wed Nov 8 16:40:38 2017 -0800
Committer: Blake Eggleston 
Committed: Wed Nov 8 16:42:13 2017 -0800

--
 CHANGES.txt | 1 +
 .../org/apache/cassandra/service/ActiveRepairService.java   | 9 ++---
 2 files changed, 7 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7d6474bb/CHANGES.txt
--
diff --cc CHANGES.txt
index 83015b1,a94dbc9..6bd6de8
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,9 -1,5 +1,10 @@@
 -3.0.16
 +3.11.2
 + * Add asm jar to build.xml for maven builds (CASSANDRA-11193)
 + * Round buffer size to powers of 2 for the chunk cache (CASSANDRA-13897)
 + * Update jackson JSON jars (CASSANDRA-13949)
 + * Avoid locks when checking LCS fanout and if we should defrag 
(CASSANDRA-13930)
 +Merged from 3.0:
+  * Fix repair race that caused gossip to block (CASSANDRA-13849)
   * Tracing interferes with digest requests when using RandomPartitioner 
(CASSANDRA-13964)
   * Add flag to disable materialized views, and warnings on creation 
(CASSANDRA-13959)
   * Don't let user drop or generally break tables in system_distributed 
(CASSANDRA-13813)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7d6474bb/src/java/org/apache/cassandra/service/ActiveRepairService.java
--
diff --cc src/java/org/apache/cassandra/service/ActiveRepairService.java
index fea690e,adb6fab..626aa91
--- a/src/java/org/apache/cassandra/service/ActiveRepairService.java
+++ b/src/java/org/apache/cassandra/service/ActiveRepairService.java
@@@ -283,9 -280,9 +283,9 @@@ public class ActiveRepairService implem
  return neighbors;
  }
  
- public synchronized UUID prepareForRepair(UUID parentRepairSession, 
InetAddress coordinator, Set endpoints, RepairOption options, 
List columnFamilyStores)
+ public UUID prepareForRepair(UUID parentRepairSession, InetAddress 
coordinator, Set endpoints, RepairOption options, 
List columnFamilyStores)
  {
 -long timestamp = System.currentTimeMillis();
 +long timestamp = Clock.instance.currentTimeMillis();
  registerParentRepairSession(parentRepairSession, coordinator, 
columnFamilyStores, options.getRanges(), options.isIncremental(), timestamp, 
options.isGlobal());
  final CountDownLatch prepareLatch = new 
CountDownLatch(endpoints.size());
  final AtomicBoolean status = new AtomicBoolean(true);


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[6/6] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ba5f946e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ba5f946e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ba5f946e

Branch: refs/heads/trunk
Commit: ba5f946e5f6763549e7235b48357a1edd0d67728
Parents: 77a352b 7d6474b
Author: Blake Eggleston 
Authored: Wed Nov 8 16:43:50 2017 -0800
Committer: Blake Eggleston 
Committed: Wed Nov 8 16:58:58 2017 -0800

--
 CHANGES.txt   | 1 +
 .../org/apache/cassandra/service/ActiveRepairService.java | 7 +--
 2 files changed, 6 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ba5f946e/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ba5f946e/src/java/org/apache/cassandra/service/ActiveRepairService.java
--
diff --cc src/java/org/apache/cassandra/service/ActiveRepairService.java
index ab92822,626aa91..62352ad
--- a/src/java/org/apache/cassandra/service/ActiveRepairService.java
+++ b/src/java/org/apache/cassandra/service/ActiveRepairService.java
@@@ -469,9 -354,8 +469,9 @@@ public class ActiveRepairService implem
  throw new RuntimeException(errorMsg);
  }
  
- public void registerParentRepairSession(UUID parentRepairSession, 
InetAddress coordinator, List columnFamilyStores, 
Collection ranges, boolean isIncremental, long repairedAt, 
boolean isGlobal, PreviewKind previewKind)
 -public synchronized void registerParentRepairSession(UUID 
parentRepairSession, InetAddress coordinator, List 
columnFamilyStores, Collection ranges, boolean isIncremental, 
long timestamp, boolean isGlobal)
++public synchronized void registerParentRepairSession(UUID 
parentRepairSession, InetAddress coordinator, List 
columnFamilyStores, Collection ranges, boolean isIncremental, 
long repairedAt, boolean isGlobal, PreviewKind previewKind)
  {
 +assert isIncremental || repairedAt == 
ActiveRepairService.UNREPAIRED_SSTABLE;
  if (!registeredForEndpointChanges)
  {
  Gossiper.instance.register(this);
@@@ -479,9 -363,46 +479,12 @@@
  registeredForEndpointChanges = true;
  }
  
- parentRepairSessions.put(parentRepairSession, new 
ParentRepairSession(coordinator, columnFamilyStores, ranges, isIncremental, 
repairedAt, isGlobal, previewKind));
+ if (!parentRepairSessions.containsKey(parentRepairSession))
+ {
 -parentRepairSessions.put(parentRepairSession, new 
ParentRepairSession(coordinator, columnFamilyStores, ranges, isIncremental, 
timestamp, isGlobal));
++parentRepairSessions.put(parentRepairSession, new 
ParentRepairSession(coordinator, columnFamilyStores, ranges, isIncremental, 
repairedAt, isGlobal, previewKind));
+ }
  }
  
 -public Set currentlyRepairing(UUID cfId, UUID 
parentRepairSession)
 -{
 -Set repairing = new HashSet<>();
 -for (Map.Entry entry : 
parentRepairSessions.entrySet())
 -{
 -Collection sstables = 
entry.getValue().getActiveSSTables(cfId);
 -if (sstables != null && 
!entry.getKey().equals(parentRepairSession))
 -repairing.addAll(sstables);
 -}
 -return repairing;
 -}
 -
 -/**
 - * Run final process of repair.
 - * This removes all resources held by parent repair session, after 
performing anti compaction if necessary.
 - *
 - * @param parentSession Parent session ID
 - * @param neighbors Repair participants (not including self)
 - * @param successfulRanges Ranges that repaired successfully
 - */
 -public synchronized ListenableFuture finishParentSession(UUID 
parentSession, Set neighbors, Collection 
successfulRanges)
 -{
 -List tasks = new ArrayList<>(neighbors.size() + 
1);
 -for (InetAddress neighbor : neighbors)
 -{
 -AnticompactionTask task = new AnticompactionTask(parentSession, 
neighbor, successfulRanges);
 -registerOnFdAndGossip(task);
 -tasks.add(task);
 -task.run(); // 'run' is just sending message
 -}
 -tasks.add(doAntiCompaction(parentSession, successfulRanges));
 -return Futures.successfulAsList(tasks);
 -}
 -
  public ParentRepairSession getParentRepairSession(UUID parentSessionId)
  {
  ParentRepairSession session = 
parentRepairSessions.get(parentSessionId);

[2/6] cassandra git commit: Fix repair race that caused gossip to block

Fix repair race that caused gossip to block

Patch by Sergey Lapukhov; Reviewed by Blake Eggleston for CASSANDRA-13849


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/49edd707
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/49edd707
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/49edd707

Branch: refs/heads/cassandra-3.11
Commit: 49edd70740e2efae3681cb79a391369bfb7de02e
Parents: 58daf13
Author: Sergey Lapukhov 
Authored: Wed Nov 1 11:35:09 2017 +0100
Committer: Blake Eggleston 
Committed: Wed Nov 8 16:38:21 2017 -0800

--
 CHANGES.txt | 1 +
 .../org/apache/cassandra/service/ActiveRepairService.java   | 9 ++---
 2 files changed, 7 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/49edd707/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3f4f3f2..a94dbc9 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.16
+ * Fix repair race that caused gossip to block (CASSANDRA-13849)
  * Tracing interferes with digest requests when using RandomPartitioner 
(CASSANDRA-13964)
  * Add flag to disable materialized views, and warnings on creation 
(CASSANDRA-13959)
  * Don't let user drop or generally break tables in system_distributed 
(CASSANDRA-13813)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/49edd707/src/java/org/apache/cassandra/service/ActiveRepairService.java
--
diff --git a/src/java/org/apache/cassandra/service/ActiveRepairService.java 
b/src/java/org/apache/cassandra/service/ActiveRepairService.java
index b4cea79..adb6fab 100644
--- a/src/java/org/apache/cassandra/service/ActiveRepairService.java
+++ b/src/java/org/apache/cassandra/service/ActiveRepairService.java
@@ -280,7 +280,7 @@ public class ActiveRepairService implements 
IEndpointStateChangeSubscriber, IFai
 return neighbors;
 }
 
-public synchronized UUID prepareForRepair(UUID parentRepairSession, 
InetAddress coordinator, Set endpoints, RepairOption options, 
List columnFamilyStores)
+public UUID prepareForRepair(UUID parentRepairSession, InetAddress 
coordinator, Set endpoints, RepairOption options, 
List columnFamilyStores)
 {
 long timestamp = System.currentTimeMillis();
 registerParentRepairSession(parentRepairSession, coordinator, 
columnFamilyStores, options.getRanges(), options.isIncremental(), timestamp, 
options.isGlobal());
@@ -351,7 +351,7 @@ public class ActiveRepairService implements 
IEndpointStateChangeSubscriber, IFai
 throw new RuntimeException(errorMsg);
 }
 
-public void registerParentRepairSession(UUID parentRepairSession, 
InetAddress coordinator, List columnFamilyStores, 
Collection ranges, boolean isIncremental, long timestamp, boolean 
isGlobal)
+public synchronized void registerParentRepairSession(UUID 
parentRepairSession, InetAddress coordinator, List 
columnFamilyStores, Collection ranges, boolean isIncremental, 
long timestamp, boolean isGlobal)
 {
 if (!registeredForEndpointChanges)
 {
@@ -360,7 +360,10 @@ public class ActiveRepairService implements 
IEndpointStateChangeSubscriber, IFai
 registeredForEndpointChanges = true;
 }
 
-parentRepairSessions.put(parentRepairSession, new 
ParentRepairSession(coordinator, columnFamilyStores, ranges, isIncremental, 
timestamp, isGlobal));
+if (!parentRepairSessions.containsKey(parentRepairSession))
+{
+parentRepairSessions.put(parentRepairSession, new 
ParentRepairSession(coordinator, columnFamilyStores, ranges, isIncremental, 
timestamp, isGlobal));
+}
 }
 
 public Set currentlyRepairing(UUID cfId, UUID 
parentRepairSession)


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[1/6] cassandra git commit: Fix repair race that caused gossip to block

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 58daf1376 -> 49edd7074
  refs/heads/cassandra-3.11 df37bcb7c -> 7d6474bb4
  refs/heads/trunk 77a352b26 -> ba5f946e5


Fix repair race that caused gossip to block

Patch by Sergey Lapukhov; Reviewed by Blake Eggleston for CASSANDRA-13849


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/49edd707
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/49edd707
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/49edd707

Branch: refs/heads/cassandra-3.0
Commit: 49edd70740e2efae3681cb79a391369bfb7de02e
Parents: 58daf13
Author: Sergey Lapukhov 
Authored: Wed Nov 1 11:35:09 2017 +0100
Committer: Blake Eggleston 
Committed: Wed Nov 8 16:38:21 2017 -0800

--
 CHANGES.txt | 1 +
 .../org/apache/cassandra/service/ActiveRepairService.java   | 9 ++---
 2 files changed, 7 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/49edd707/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3f4f3f2..a94dbc9 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.16
+ * Fix repair race that caused gossip to block (CASSANDRA-13849)
  * Tracing interferes with digest requests when using RandomPartitioner 
(CASSANDRA-13964)
  * Add flag to disable materialized views, and warnings on creation 
(CASSANDRA-13959)
  * Don't let user drop or generally break tables in system_distributed 
(CASSANDRA-13813)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/49edd707/src/java/org/apache/cassandra/service/ActiveRepairService.java
--
diff --git a/src/java/org/apache/cassandra/service/ActiveRepairService.java 
b/src/java/org/apache/cassandra/service/ActiveRepairService.java
index b4cea79..adb6fab 100644
--- a/src/java/org/apache/cassandra/service/ActiveRepairService.java
+++ b/src/java/org/apache/cassandra/service/ActiveRepairService.java
@@ -280,7 +280,7 @@ public class ActiveRepairService implements 
IEndpointStateChangeSubscriber, IFai
 return neighbors;
 }
 
-public synchronized UUID prepareForRepair(UUID parentRepairSession, 
InetAddress coordinator, Set endpoints, RepairOption options, 
List columnFamilyStores)
+public UUID prepareForRepair(UUID parentRepairSession, InetAddress 
coordinator, Set endpoints, RepairOption options, 
List columnFamilyStores)
 {
 long timestamp = System.currentTimeMillis();
 registerParentRepairSession(parentRepairSession, coordinator, 
columnFamilyStores, options.getRanges(), options.isIncremental(), timestamp, 
options.isGlobal());
@@ -351,7 +351,7 @@ public class ActiveRepairService implements 
IEndpointStateChangeSubscriber, IFai
 throw new RuntimeException(errorMsg);
 }
 
-public void registerParentRepairSession(UUID parentRepairSession, 
InetAddress coordinator, List columnFamilyStores, 
Collection ranges, boolean isIncremental, long timestamp, boolean 
isGlobal)
+public synchronized void registerParentRepairSession(UUID 
parentRepairSession, InetAddress coordinator, List 
columnFamilyStores, Collection ranges, boolean isIncremental, 
long timestamp, boolean isGlobal)
 {
 if (!registeredForEndpointChanges)
 {
@@ -360,7 +360,10 @@ public class ActiveRepairService implements 
IEndpointStateChangeSubscriber, IFai
 registeredForEndpointChanges = true;
 }
 
-parentRepairSessions.put(parentRepairSession, new 
ParentRepairSession(coordinator, columnFamilyStores, ranges, isIncremental, 
timestamp, isGlobal));
+if (!parentRepairSessions.containsKey(parentRepairSession))
+{
+parentRepairSessions.put(parentRepairSession, new 
ParentRepairSession(coordinator, columnFamilyStores, ranges, isIncremental, 
timestamp, isGlobal));
+}
 }
 
 public Set currentlyRepairing(UUID cfId, UUID 
parentRepairSession)


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7d6474bb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7d6474bb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7d6474bb

Branch: refs/heads/trunk
Commit: 7d6474bb4d5e8fe940ad0a75652524ca221dab62
Parents: df37bcb 49edd70
Author: Blake Eggleston 
Authored: Wed Nov 8 16:40:38 2017 -0800
Committer: Blake Eggleston 
Committed: Wed Nov 8 16:42:13 2017 -0800

--
 CHANGES.txt | 1 +
 .../org/apache/cassandra/service/ActiveRepairService.java   | 9 ++---
 2 files changed, 7 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7d6474bb/CHANGES.txt
--
diff --cc CHANGES.txt
index 83015b1,a94dbc9..6bd6de8
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,9 -1,5 +1,10 @@@
 -3.0.16
 +3.11.2
 + * Add asm jar to build.xml for maven builds (CASSANDRA-11193)
 + * Round buffer size to powers of 2 for the chunk cache (CASSANDRA-13897)
 + * Update jackson JSON jars (CASSANDRA-13949)
 + * Avoid locks when checking LCS fanout and if we should defrag 
(CASSANDRA-13930)
 +Merged from 3.0:
+  * Fix repair race that caused gossip to block (CASSANDRA-13849)
   * Tracing interferes with digest requests when using RandomPartitioner 
(CASSANDRA-13964)
   * Add flag to disable materialized views, and warnings on creation 
(CASSANDRA-13959)
   * Don't let user drop or generally break tables in system_distributed 
(CASSANDRA-13813)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7d6474bb/src/java/org/apache/cassandra/service/ActiveRepairService.java
--
diff --cc src/java/org/apache/cassandra/service/ActiveRepairService.java
index fea690e,adb6fab..626aa91
--- a/src/java/org/apache/cassandra/service/ActiveRepairService.java
+++ b/src/java/org/apache/cassandra/service/ActiveRepairService.java
@@@ -283,9 -280,9 +283,9 @@@ public class ActiveRepairService implem
  return neighbors;
  }
  
- public synchronized UUID prepareForRepair(UUID parentRepairSession, 
InetAddress coordinator, Set endpoints, RepairOption options, 
List columnFamilyStores)
+ public UUID prepareForRepair(UUID parentRepairSession, InetAddress 
coordinator, Set endpoints, RepairOption options, 
List columnFamilyStores)
  {
 -long timestamp = System.currentTimeMillis();
 +long timestamp = Clock.instance.currentTimeMillis();
  registerParentRepairSession(parentRepairSession, coordinator, 
columnFamilyStores, options.getRanges(), options.isIncremental(), timestamp, 
options.isGlobal());
  final CountDownLatch prepareLatch = new 
CountDownLatch(endpoints.size());
  final AtomicBoolean status = new AtomicBoolean(true);


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14003) Correct logger message formatting in SSTableLoader


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245010#comment-16245010
 ] 

Jaydeepkumar Chovatia edited comment on CASSANDRA-14003 at 11/9/17 12:33 AM:
-

Please find patch details here:

|| patch || uTest ||
| [trunk|https://github.com/apache/cassandra/pull/172] | 
[circleci|https://circleci.com/gh/jaydeepkumar1984/cassandra/22]|


was (Author: chovatia.jayd...@gmail.com):
Please find patch her:

|| patch || uTest ||
| [trunk|https://github.com/apache/cassandra/pull/172] | 
[circleci|https://circleci.com/gh/jaydeepkumar1984/cassandra/22]|

> Correct logger message formatting in SSTableLoader
> --
>
> Key: CASSANDRA-14003
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14003
> Project: Cassandra
>  Issue Type: Task
>  Components: Tools
>Reporter: Jaydeepkumar Chovatia
>Assignee: Jaydeepkumar Chovatia
>Priority: Trivial
> Fix For: 4.x
>
>
> Correct logger message formatting inside SSTableLoader.java
> -outputHandler.output(String.format("Streaming relevant part of %sto 
> %s", names(sstables), endpointToRanges.keySet()));
> +outputHandler.output(String.format("Streaming relevant part of %s to 
> %s", names(sstables), endpointToRanges.keySet()));
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14003) Correct logger message formatting in SSTableLoader


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jaydeepkumar Chovatia updated CASSANDRA-14003:
--
Status: Patch Available  (was: Open)

> Correct logger message formatting in SSTableLoader
> --
>
> Key: CASSANDRA-14003
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14003
> Project: Cassandra
>  Issue Type: Task
>  Components: Tools
>Reporter: Jaydeepkumar Chovatia
>Assignee: Jaydeepkumar Chovatia
>Priority: Trivial
> Fix For: 4.x
>
>
> Correct logger message formatting inside SSTableLoader.java
> -outputHandler.output(String.format("Streaming relevant part of %sto 
> %s", names(sstables), endpointToRanges.keySet()));
> +outputHandler.output(String.format("Streaming relevant part of %s to 
> %s", names(sstables), endpointToRanges.keySet()));
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14003) Correct logger message formatting in SSTableLoader


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245010#comment-16245010
 ] 

Jaydeepkumar Chovatia commented on CASSANDRA-14003:
---

Please find patch her:

|| patch || uTest ||
| [trunk|https://github.com/apache/cassandra/pull/172] | 
[circleci|https://circleci.com/gh/jaydeepkumar1984/cassandra/22]|

> Correct logger message formatting in SSTableLoader
> --
>
> Key: CASSANDRA-14003
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14003
> Project: Cassandra
>  Issue Type: Task
>  Components: Tools
>Reporter: Jaydeepkumar Chovatia
>Assignee: Jaydeepkumar Chovatia
>Priority: Trivial
> Fix For: 4.x
>
>
> Correct logger message formatting inside SSTableLoader.java
> -outputHandler.output(String.format("Streaming relevant part of %sto 
> %s", names(sstables), endpointToRanges.keySet()));
> +outputHandler.output(String.format("Streaming relevant part of %s to 
> %s", names(sstables), endpointToRanges.keySet()));
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13987) Multithreaded commitlog subtly changed durability


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-13987:

Status: Patch Available  (was: Open)

> Multithreaded commitlog subtly changed durability
> -
>
> Key: CASSANDRA-13987
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13987
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jason Brown
>Assignee: Jason Brown
> Fix For: 4.x
>
>
> When multithreaded commitlog was introduced in CASSANDRA-3578, we subtly 
> changed the way that commitlog durability worked. Everything still gets 
> written to an mmap file. However, not everything is replayable from the 
> mmaped file after a process crash, in periodic mode.
> In brief, the reason this changesd is due to the chained markers that are 
> required for the multithreaded commit log. At each msync, we wait for 
> outstanding mutations to serialize into the commitlog, and update a marker 
> before and after the commits that have accumluated since the last sync. With 
> those markers, we can safely replay that section of the commitlog. Without 
> the markers, we have no guarantee that the commits in that section were 
> successfully written, thus we abandon those commits on replay.
> If you have correlated process failures of multiple nodes at "nearly" the 
> same time (see ["There Is No 
> Now"|http://queue.acm.org/detail.cfm?id=2745385]), it is possible to have 
> data loss if none of the nodes msync the commitlog. For example, with RF=3, 
> if quorum write succeeds on two nodes (and we acknowledge the write back to 
> the client), and then the process on both nodes OOMs (say, due to reading the 
> index for a 100GB partition), the write will be lost if neither process 
> msync'ed the commitlog. More exactly, the commitlog cannot be fully replayed. 
> The reason why this data is silently lost is due to the chained markers that 
> were introduced with CASSANDRA-3578.
> The problem we are addressing with this ticket is incrementally improving 
> 'durability' due to process crash, not host crash. (Note: operators should 
> use batch mode to ensure greater durability, but batch mode in it's current 
> implementation is a) borked, and b) will burn through, *very* rapidly, SSDs 
> that don't have a non-volatile write cache sitting in front.) 
> The current default for {{commitlog_sync_period_in_ms}} is 10 seconds, which 
> means that a node could lose up to ten seconds of data due to process crash. 
> The unfortunate thing is that the data is still avaialble, in the mmap file, 
> but we can't replay it due to incomplete chained markers.
> ftr, I don't believe we've ever had a stated policy about commitlog 
> durability wrt process crash. Pre-2.0 we naturally piggy-backed off the 
> memory mapped file and the fact that every mutation was acquired a lock and 
> wrote into the mmap buffer, and the ability to replay everything out of it 
> came for free. With CASSANDRA-3578, that was subtly changed. 
> Something [~jjirsa] pointed out to me is that [MySQL provides a way to adjust 
> the durability 
> guarantees|https://dev.mysql.com/doc/refman/5.6/en/innodb-parameters.html#sysvar_innodb_flush_log_at_trx_commit]
>  of each commit in innodb via the {{innodb_flush_log_at_trx_commit}}. I'm 
> using that idea as a loose springboard for what to do here.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13993) Add optional startup delay to wait until peers are ready


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-13993:

Status: Patch Available  (was: Open)

> Add optional startup delay to wait until peers are ready
> 
>
> Key: CASSANDRA-13993
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13993
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Lifecycle
>Reporter: Jason Brown
>Assignee: Jason Brown
>Priority: Minor
> Fix For: 4.x
>
>
> When bouncing a node in a large cluster, is can take a while to recognize the 
> rest of the cluster as available. This is especially true if using TLS on 
> internode messaging connections. The bouncing node (and any clients connected 
> to it) may see a series of Unavailable or Timeout exceptions until the node 
> is 'warmed up' as connecting to the rest of the cluster is asynchronous from 
> the rest of the startup process.
> There are two aspects that drive a node's ability to successfully communicate 
> with a peer after a bounce:
> - marking the peer as 'alive' (state that is held in gossip). This affects 
> the unavailable exceptions
> - having both open outbound and inbound connections open and ready to each 
> peer. This affects timeouts.
> Details of each of these mechanisms are described in the comments below.
> This ticket proposes adding a mechanism, optional and configurable, to delay 
> opening the client native protocol port until some percentage of the peers in 
> the cluster is marked alive and connected to/from. Thus while we potentially 
> slow down startup (delay opening the client port), we alleviate the chance 
> that queries made by clients don't hit transient unavailable/timeout 
> exceptions.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14003) Correct logger message formatting in SSTableLoader

Jaydeepkumar Chovatia created CASSANDRA-14003:
-

 Summary: Correct logger message formatting in SSTableLoader
 Key: CASSANDRA-14003
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14003
 Project: Cassandra
  Issue Type: Task
  Components: Tools
Reporter: Jaydeepkumar Chovatia
Assignee: Jaydeepkumar Chovatia
Priority: Trivial
 Fix For: 4.x


Correct logger message formatting inside SSTableLoader.java

-outputHandler.output(String.format("Streaming relevant part of %sto 
%s", names(sstables), endpointToRanges.keySet()));
+outputHandler.output(String.format("Streaming relevant part of %s to 
%s", names(sstables), endpointToRanges.keySet()));
 




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-9988) Introduce leaf-only iterator


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16244848#comment-16244848
 ] 

Jason Brown edited comment on CASSANDRA-9988 at 11/9/17 12:15 AM:
--

Overall, I think the patch is pretty solid. A couple of comments and nits:

- is {{BTreeSearchIterator}} necessary anymore? It's just an empty interface now
- just to confirm, {{FullBTreeSearchIterator}} is just a rename of the previous 
{{BTreeSearchIterator}} class? My diff'ing seems to confirm that, but I'd like 
the extra level of sanity checking.

nits:
- {{BTreeSet}} switch imports back to original format
- in {{LeafBTreeSearchIterator}}, move {{#compareToFirst}} and {{#searchNext}} 
closer to the methods from where they are called. Or better yet, since each is 
only called from one place, just inline the functionality into those calling 
methods.

My biggest problem is with the microbench. I know this ticket has gone through 
a bunch on contortions over it's life, so maybe a little confusion is 
reasonable. However, what I was really hoping to see was a comparison of the 
leaf vs full (tree) iterators, as that's what this whole ticket is about. I 
took liberty of reworking the mircobench to explicitly test the differences 
bewtween leaf and tree iterators, as well as reduce of benchmark executions 
(due to every value between 0 and 31, inclusive, being a param to the 
{{targetIdx}} field. (I simply removed about 2/3 of those values, as I think we 
only really care about testing reading from the beginning, middle, and end of 
the btree entries; anything else just bloats total test execution time). 
Further, as {{#iteratorTree()}} and {{#multiSearchFound}} don't use the 
{{targetIdx}} param, I think they can be moved to another class as they would 
be executed once for each of the entries in {{targetIdx}} (I'm not awatre of a 
way to ignore that in JMH).

My version of your branch with the updated microbench is 
[here|https://github.com/jasobrown/cassandra/tree/9988]

EDIT: forgot to mention that with the updated microbench, I was able to measure 
that the leaf iterator was anywhere from 3-20% faster, depending on the test. 
So, indeed this is a good patch


was (Author: jasobrown):
Overall, I think the patch is pretty solid. A couple of comments and nits:

- is {{BTreeSearchIterator}} necessary anymore? It's just an empty interface now
- just to confirm, {{FullBTreeSearchIterator}} is just a rename of the previous 
{{BTreeSearchIterator}} class? My diff'ing seems to confirm that, but I'd like 
the extra level of sanity checking.

nits:
- {{BTreeSet}} switch imports back to original format
- in {{LeafBTreeSearchIterator}}, move {{#compareToFirst}} and {{#searchNext}} 
closer to the methods from where they are called. Or better yet, since each is 
only called from one place, just inline the functionality into those calling 
methods.

My biggest problem is with the microbench. I know this ticket has gone through 
a bunch on contortions over it's life, so maybe a little confusion is 
reasonable. However, what I was really hoping to see was a comparison of the 
leaf vs full (tree) iterators, as that's what this whole ticket is about. I 
took liberty of reworking the mircobench to explicitly test the differences 
bewtween leaf and tree iterators, as well as reduce of benchmark executions 
(due to every value between 0 and 31, inclusive, being a param to the 
{{targetIdx}} field. (I simply removed about 2/3 of those values, as I think we 
only really care about testing reading from the beginning, middle, and end of 
the btree entries; anything else just bloats total test execution time). 
Further, as {{#iteratorTree()}} and {{#multiSearchFound}} don't use the 
{{targetIdx}} param, I think they can be moved to another class as they would 
be executed once for each of the entries in {{targetIdx}} (I'm not awatre of a 
way to ignore that in JMH).

My version of your branch with the updated microbench is 
[here|https://github.com/jasobrown/cassandra/tree/9988]

EDIT: forgot to mention that with the updated microbench, the leaf iterator was 
anywhere from 3-20% faster, depending on the test.

> Introduce leaf-only iterator
> 
>
> Key: CASSANDRA-9988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9988
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Benedict
>Assignee: Jay Zhuang
>Priority: Minor
>  Labels: patch
> Fix For: 4.0
>
> Attachments: 9988-3tests.png, 9988-data.png, 9988-result.png, 
> 9988-result2.png, 9988-result3.png, 9988-test-result-expsearch.xlsx, 
> 9988-test-result-raw.png, 9988-test-result.xlsx, 9988-test-result3.png, 
> 9988-trunk-new-update.txt, 9988-trunk-new.txt, trunk-9988.txt
>
>
> In many cases we have small btrees, small enough to fit

[3/3] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

2017-11-08 Thread jasobrown

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/77a352b2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/77a352b2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/77a352b2

Branch: refs/heads/trunk
Commit: 77a352b26bc4237a748ece00757975dc667e2f37
Parents: 6186ad9 df37bcb
Author: Jason Brown 
Authored: Wed Nov 8 15:58:18 2017 -0800
Committer: Jason Brown 
Committed: Wed Nov 8 15:59:30 2017 -0800

--
 CHANGES.txt |  1 +
 build.xml   | 10 ++
 2 files changed, 7 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/77a352b2/CHANGES.txt
--
diff --cc CHANGES.txt
index bb27f2e,83015b1..9d63abd
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,163 -1,5 +1,164 @@@
 +4.0
 + * Allow only one concurrent call to StatusLogger (CASSANDRA-12182)
 + * Refactoring to specialised functional interfaces (CASSANDRA-13982)
 + * Speculative retry should allow more friendly params (CASSANDRA-13876)
 + * Throw exception if we send/receive repair messages to incompatible nodes 
(CASSANDRA-13944)
 + * Replace usages of MessageDigest with Guava's Hasher (CASSANDRA-13291)
 + * Add nodetool cmd to print hinted handoff window (CASSANDRA-13728)
 + * Fix some alerts raised by static analysis (CASSANDRA-13799)
 + * Checksum sstable metadata (CASSANDRA-13321, CASSANDRA-13593)
 + * Add result set metadata to prepared statement MD5 hash calculation 
(CASSANDRA-10786)
 + * Refactor GcCompactionTest to avoid boxing (CASSANDRA-13941)
 + * Expose recent histograms in JmxHistograms (CASSANDRA-13642)
 + * Fix buffer length comparison when decompressing in netty-based streaming 
(CASSANDRA-13899)
 + * Properly close StreamCompressionInputStream to release any ByteBuf 
(CASSANDRA-13906)
 + * Add SERIAL and LOCAL_SERIAL support for cassandra-stress (CASSANDRA-13925)
 + * LCS needlessly checks for L0 STCS candidates multiple times 
(CASSANDRA-12961)
 + * Correctly close netty channels when a stream session ends (CASSANDRA-13905)
 + * Update lz4 to 1.4.0 (CASSANDRA-13741)
 + * Optimize Paxos prepare and propose stage for local requests 
(CASSANDRA-13862)
 + * Throttle base partitions during MV repair streaming to prevent OOM 
(CASSANDRA-13299)
 + * Use compaction threshold for STCS in L0 (CASSANDRA-13861)
 + * Fix problem with min_compress_ratio: 1 and disallow ratio < 1 
(CASSANDRA-13703)
 + * Add extra information to SASI timeout exception (CASSANDRA-13677)
 + * Add incremental repair support for --hosts, --force, and subrange repair 
(CASSANDRA-13818)
 + * Rework CompactionStrategyManager.getScanners synchronization 
(CASSANDRA-13786)
 + * Add additional unit tests for batch behavior, TTLs, Timestamps 
(CASSANDRA-13846)
 + * Add keyspace and table name in schema validation exception 
(CASSANDRA-13845)
 + * Emit metrics whenever we hit tombstone failures and warn thresholds 
(CASSANDRA-13771)
 + * Make netty EventLoopGroups daemon threads (CASSANDRA-13837)
 + * Race condition when closing stream sessions (CASSANDRA-13852)
 + * NettyFactoryTest is failing in trunk on macOS (CASSANDRA-13831)
 + * Allow changing log levels via nodetool for related classes 
(CASSANDRA-12696)
 + * Add stress profile yaml with LWT (CASSANDRA-7960)
 + * Reduce memory copies and object creations when acting on ByteBufs 
(CASSANDRA-13789)
 + * Simplify mx4j configuration (Cassandra-13578)
 + * Fix trigger example on 4.0 (CASSANDRA-13796)
 + * Force minumum timeout value (CASSANDRA-9375)
 + * Use netty for streaming (CASSANDRA-12229)
 + * Use netty for internode messaging (CASSANDRA-8457)
 + * Add bytes repaired/unrepaired to nodetool tablestats (CASSANDRA-13774)
 + * Don't delete incremental repair sessions if they still have sstables 
(CASSANDRA-13758)
 + * Fix pending repair manager index out of bounds check (CASSANDRA-13769)
 + * Don't use RangeFetchMapCalculator when RF=1 (CASSANDRA-13576)
 + * Don't optimise trivial ranges in RangeFetchMapCalculator (CASSANDRA-13664)
 + * Use an ExecutorService for repair commands instead of new 
Thread(..).start() (CASSANDRA-13594)
 + * Fix race / ref leak in anticompaction (CASSANDRA-13688)
 + * Expose tasks queue length via JMX (CASSANDRA-12758)
 + * Fix race / ref leak in PendingRepairManager (CASSANDRA-13751)
 + * Enable ppc64le runtime as unsupported architecture (CASSANDRA-13615)
 + * Improve sstablemetadata output (CASSANDRA-11483)
 + * Support for migrating legacy users to roles has been dropped 
(CASSANDRA-13371)
 + * Introduce error metrics for repair (CASSANDRA-13387)
 + * Refactoring to primitive functional interfaces in AuthCache 
(CASSANDRA-13732)
 + * Update metrics to 3.1.5 (CASSANDRA-13648)
 + *

[2/3] cassandra git commit: Add asm jar to build.xml for maven builds

2017-11-08 Thread jasobrown

Add asm jar to build.xml for maven builds

patch by DOAN DuyHai; reviewed by jasobrown for CASSANDRA-11193


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/df37bcb7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/df37bcb7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/df37bcb7

Branch: refs/heads/trunk
Commit: df37bcb7cba75c3da35e585726c90ad90a454d48
Parents: ab6201c
Author: DOAN DuyHai 
Authored: Wed Nov 8 15:42:15 2017 -0800
Committer: Jason Brown 
Committed: Wed Nov 8 15:56:49 2017 -0800

--
 CHANGES.txt |  1 +
 build.xml   | 10 ++
 2 files changed, 7 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/df37bcb7/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 1269dcf..83015b1 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.11.2
+ * Add asm jar to build.xml for maven builds (CASSANDRA-11193)
  * Round buffer size to powers of 2 for the chunk cache (CASSANDRA-13897)
  * Update jackson JSON jars (CASSANDRA-13949)
  * Avoid locks when checking LCS fanout and if we should defrag 
(CASSANDRA-13930)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/df37bcb7/build.xml
--
diff --git a/build.xml b/build.xml
index 38e8963..2990ca9 100644
--- a/build.xml
+++ b/build.xml
@@ -453,8 +453,9 @@
   
   
   
- 
- 
+  
+  
+  
 
 
 
@@ -637,8 +638,9 @@
 
 
 
-   
-   
+
+
+

[1/3] cassandra git commit: Add asm jar to build.xml for maven builds

2017-11-08 Thread jasobrown

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.11 ab6201c65 -> df37bcb7c
  refs/heads/trunk 6186ad97a -> 77a352b26


Add asm jar to build.xml for maven builds

patch by DOAN DuyHai; reviewed by jasobrown for CASSANDRA-11193


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/df37bcb7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/df37bcb7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/df37bcb7

Branch: refs/heads/cassandra-3.11
Commit: df37bcb7cba75c3da35e585726c90ad90a454d48
Parents: ab6201c
Author: DOAN DuyHai 
Authored: Wed Nov 8 15:42:15 2017 -0800
Committer: Jason Brown 
Committed: Wed Nov 8 15:56:49 2017 -0800

--
 CHANGES.txt |  1 +
 build.xml   | 10 ++
 2 files changed, 7 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/df37bcb7/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 1269dcf..83015b1 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.11.2
+ * Add asm jar to build.xml for maven builds (CASSANDRA-11193)
  * Round buffer size to powers of 2 for the chunk cache (CASSANDRA-13897)
  * Update jackson JSON jars (CASSANDRA-13949)
  * Avoid locks when checking LCS fanout and if we should defrag 
(CASSANDRA-13930)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/df37bcb7/build.xml
--
diff --git a/build.xml b/build.xml
index 38e8963..2990ca9 100644
--- a/build.xml
+++ b/build.xml
@@ -453,8 +453,9 @@
   
   
   
- 
- 
+  
+  
+  
 
 
 
@@ -637,8 +638,9 @@
 
 
 
-   
-   
+
+
+

[jira] [Resolved] (CASSANDRA-11193) Missing binary dependencies for running Cassandra in embedded mode


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown resolved CASSANDRA-11193.
-
   Resolution: Fixed
 Assignee: DOAN DuyHai
 Reviewer: Jason Brown
Fix Version/s: 4.0
   3.11.2

OK, sorry for being a PITA about this, [~vroyer59]. It's a trivial change, and 
safe at that. I'm +1'ing it and have committed as sha 
{{df37bcb7cba75c3da35e585726c90ad90a454d48}}

> Missing binary dependencies for running Cassandra in embedded mode
> --
>
> Key: CASSANDRA-11193
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11193
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Cassandra 3.3
>Reporter: DOAN DuyHai
>Assignee: DOAN DuyHai
>Priority: Minor
> Fix For: 3.11.2, 4.0
>
> Attachments: patch.txt
>
>
> When running Cassandra in embedded mode (pulling the *cassandra-all-3.3.jar* 
> from Maven) and activating *UDF*, I face the following exceptions when trying 
> to create an UDF:
> {noformat}
> 18:13:57.922 [main] DEBUG ACHILLES_DDL_SCRIPT -   SCRIPT : CREATE 
> FUNCTION convertToLong(input text) RETURNS NULL ON NULL INPUT RETURNS bigint 
> LANGUAGE java AS $$return Long.parseLong(input);$$;
> 18:13:57.970 [SharedPool-Worker-1] ERROR o.apache.cassandra.transport.Message 
> - Unexpected exception during request; channel = [id: 0x03f52731, 
> /192.168.1.16:55224 => /192.168.1.16:9240]
> java.lang.NoClassDefFoundError: org/objectweb/asm/ClassVisitor
>   at 
> org.apache.cassandra.cql3.functions.JavaBasedUDFunction.(JavaBasedUDFunction.java:79)
>  ~[cassandra-all-3.3.jar:3.3]
>   at 
> org.apache.cassandra.cql3.functions.UDFunction.create(UDFunction.java:223) 
> ~[cassandra-all-3.3.jar:3.3]
>   at 
> org.apache.cassandra.cql3.statements.CreateFunctionStatement.announceMigration(CreateFunctionStatement.java:162)
>  ~[cassandra-all-3.3.jar:3.3]
>   at 
> org.apache.cassandra.cql3.statements.SchemaAlteringStatement.execute(SchemaAlteringStatement.java:93)
>  ~[cassandra-all-3.3.jar:3.3]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)
>  ~[cassandra-all-3.3.jar:3.3]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:237) 
> ~[cassandra-all-3.3.jar:3.3]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:222) 
> ~[cassandra-all-3.3.jar:3.3]
>   at 
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:115)
>  ~[cassandra-all-3.3.jar:3.3]
>   at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:507)
>  [cassandra-all-3.3.jar:3.3]
>   at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:401)
>  [cassandra-all-3.3.jar:3.3]
>   at 
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_60-ea]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
>  [cassandra-all-3.3.jar:3.3]
>   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [cassandra-all-3.3.jar:3.3]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60-ea]
> Caused by: java.lang.ClassNotFoundException: org.objectweb.asm.ClassVisitor
>   at java.net.URLClassLoader.findClass(URLClassLoader.java:381) 
> ~[na:1.8.0_60-ea]
>   at java.lang.ClassLoader.loadClass(ClassLoader.java:424) 
> ~[na:1.8.0_60-ea]
>   at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) 
> ~[na:1.8.0_60-ea]
>   at java.lang.ClassLoader.loadClass(ClassLoader.java:357) 
> ~[na:1.8.0_60-ea]
>   ... 18 common frames omitted
> {noformat}
>  The stack-trace is quite explicit, some classes from the objectweb/asm are 
> missing. By looking into the {{$CASSANDRA_HOME/lib folder}}:
> {noformat}
>  19:44:07 :/opt/apps/apache-cassandra-3.2/lib]
> % ll
> total 48768
> -rw-r--r--@  1 archinnovinfo  wheel   234K Jan  7 22:42 ST4-4.0.8.jar
> -rw-r--r--@  1 archinnovinfo  wheel85K Jan  7 22:42 airline-0.6.jar
>

[jira] [Comment Edited] (CASSANDRA-9988) Introduce leaf-only iterator


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16244848#comment-16244848
 ] 

Jason Brown edited comment on CASSANDRA-9988 at 11/8/17 10:41 PM:
--

Overall, I think the patch is pretty solid. A couple of comments and nits:

- is {{BTreeSearchIterator}} necessary anymore? It's just an empty interface now
- just to confirm, {{FullBTreeSearchIterator}} is just a rename of the previous 
{{BTreeSearchIterator}} class? My diff'ing seems to confirm that, but I'd like 
the extra level of sanity checking.

nits:
- {{BTreeSet}} switch imports back to original format
- in {{LeafBTreeSearchIterator}}, move {{#compareToFirst}} and {{#searchNext}} 
closer to the methods from where they are called. Or better yet, since each is 
only called from one place, just inline the functionality into those calling 
methods.

My biggest problem is with the microbench. I know this ticket has gone through 
a bunch on contortions over it's life, so maybe a little confusion is 
reasonable. However, what I was really hoping to see was a comparison of the 
leaf vs full (tree) iterators, as that's what this whole ticket is about. I 
took liberty of reworking the mircobench to explicitly test the differences 
bewtween leaf and tree iterators, as well as reduce of benchmark executions 
(due to every value between 0 and 31, inclusive, being a param to the 
{{targetIdx}} field. (I simply removed about 2/3 of those values, as I think we 
only really care about testing reading from the beginning, middle, and end of 
the btree entries; anything else just bloats total test execution time). 
Further, as {{#iteratorTree()}} and {{#multiSearchFound}} don't use the 
{{targetIdx}} param, I think they can be moved to another class as they would 
be executed once for each of the entries in {{targetIdx}} (I'm not awatre of a 
way to ignore that in JMH).

My version of your branch with the updated microbench is 
[here|https://github.com/jasobrown/cassandra/tree/9988]

EDIT: forgot to mention that with the updated microbench, the leaf iterator was 
anywhere from 3-20% faster, depending on the test.


was (Author: jasobrown):
Overall, I think the patch is pretty solid. A couple of comments and nits:

- is {{BTreeSearchIterator}} necessary anymore? It's just an empty interface now
- just to confirm, {{FullBTreeSearchIterator}} is just a rename of the previous 
{{BTreeSearchIterator}} class? My diff'ing seems to confirm that, but I'd like 
the extra level of sanity checking.

nits:
- {{BTreeSet}} switch imports back to original format
- in {{LeafBTreeSearchIterator}}, move {{#compareToFirst}} and {{#searchNext}} 
closer to the methods from where they are called. Or better yet, since each is 
only called from one place, just inline the functionality into those calling 
methods.

My biggest problem is with the microbench. I know this ticket has gone through 
a bunch on contortions over it's life, so maybe a little confusion is 
reasonable. However, what I was really hoping to see was a comparison of the 
leaf vs full (tree) iterators, as that's what this whole ticket is about. I 
took liberty of reworking the mircobench to explicitly test the differences 
bewtween leaf and tree iterators, as well as reduce of benchmark executions 
(due to every value between 0 and 31, inclusive, being a param to the 
{{targetIdx}} field. (I simply removed about 2/3 of those values, as I think we 
only really care about testing reading from the beginning, middle, and end of 
the btree entries; anything else just bloats total test execution time). 
Further, as {{#iteratorTree()}} and {{#multiSearchFound}} don't use the 
{{targetIdx}} param, I think they can be moved to another class as they would 
be executed once for each of the entries in {{targetIdx}} (I'm not awatre of a 
way to ignore that in JMH).

My version of your branch with the updated microbench is 
[here|https://github.com/jasobrown/cassandra/tree/9988]

> Introduce leaf-only iterator
> 
>
> Key: CASSANDRA-9988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9988
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Benedict
>Assignee: Jay Zhuang
>Priority: Minor
>  Labels: patch
> Fix For: 4.0
>
> Attachments: 9988-3tests.png, 9988-data.png, 9988-result.png, 
> 9988-result2.png, 9988-result3.png, 9988-test-result-expsearch.xlsx, 
> 9988-test-result-raw.png, 9988-test-result.xlsx, 9988-test-result3.png, 
> 9988-trunk-new-update.txt, 9988-trunk-new.txt, trunk-9988.txt
>
>
> In many cases we have small btrees, small enough to fit in a single leaf 
> page. In this case it _may_ be more efficient to specialise our iterator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (CASSANDRA-9988) Introduce leaf-only iterator


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-9988:
---
Reviewer: Jason Brown  (was: Anthony Grasso)

> Introduce leaf-only iterator
> 
>
> Key: CASSANDRA-9988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9988
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Benedict
>Assignee: Jay Zhuang
>Priority: Minor
>  Labels: patch
> Fix For: 4.0
>
> Attachments: 9988-3tests.png, 9988-data.png, 9988-result.png, 
> 9988-result2.png, 9988-result3.png, 9988-test-result-expsearch.xlsx, 
> 9988-test-result-raw.png, 9988-test-result.xlsx, 9988-test-result3.png, 
> 9988-trunk-new-update.txt, 9988-trunk-new.txt, trunk-9988.txt
>
>
> In many cases we have small btrees, small enough to fit in a single leaf 
> page. In this case it _may_ be more efficient to specialise our iterator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-9988) Introduce leaf-only iterator


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16244848#comment-16244848
 ] 

Jason Brown commented on CASSANDRA-9988:


Overall, I think the patch is pretty solid. A couple of comments and nits:

- is {{BTreeSearchIterator}} necessary anymore? It's just an empty interface now
- just to confirm, {{FullBTreeSearchIterator}} is just a rename of the previous 
{{BTreeSearchIterator}} class? My diff'ing seems to confirm that, but I'd like 
the extra level of sanity checking.

nits:
- {{BTreeSet}} switch imports back to original format
- in {{LeafBTreeSearchIterator}}, move {{#compareToFirst}} and {{#searchNext}} 
closer to the methods from where they are called. Or better yet, since each is 
only called from one place, just inline the functionality into those calling 
methods.

My biggest problem is with the microbench. I know this ticket has gone through 
a bunch on contortions over it's life, so maybe a little confusion is 
reasonable. However, what I was really hoping to see was a comparison of the 
leaf vs full (tree) iterators, as that's what this whole ticket is about. I 
took liberty of reworking the mircobench to explicitly test the differences 
bewtween leaf and tree iterators, as well as reduce of benchmark executions 
(due to every value between 0 and 31, inclusive, being a param to the 
{{targetIdx}} field. (I simply removed about 2/3 of those values, as I think we 
only really care about testing reading from the beginning, middle, and end of 
the btree entries; anything else just bloats total test execution time). 
Further, as {{#iteratorTree()}} and {{#multiSearchFound}} don't use the 
{{targetIdx}} param, I think they can be moved to another class as they would 
be executed once for each of the entries in {{targetIdx}} (I'm not awatre of a 
way to ignore that in JMH).

My version of your branch with the updated microbench is 
[here|https://github.com/jasobrown/cassandra/tree/9988]

> Introduce leaf-only iterator
> 
>
> Key: CASSANDRA-9988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9988
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Benedict
>Assignee: Jay Zhuang
>Priority: Minor
>  Labels: patch
> Fix For: 4.0
>
> Attachments: 9988-3tests.png, 9988-data.png, 9988-result.png, 
> 9988-result2.png, 9988-result3.png, 9988-test-result-expsearch.xlsx, 
> 9988-test-result-raw.png, 9988-test-result.xlsx, 9988-test-result3.png, 
> 9988-trunk-new-update.txt, 9988-trunk-new.txt, trunk-9988.txt
>
>
> In many cases we have small btrees, small enough to fit in a single leaf 
> page. In this case it _may_ be more efficient to specialise our iterator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13948) Reload compaction strategies when JBOD disk boundary changes


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16244697#comment-16244697
 ] 

Paulo Motta commented on CASSANDRA-13948:
-

bq. Starting a cleanup cancel the ongoing compactions (which is expected from 
my understanding) and then get lost. Not performing any cleanup nor processing 
the pending compactions. 1 thread is using 100% of a core all the time. The log 
doesn't show any error. I've attached a thread dump.

this looks similar to CASSANDRA-13362 and unrelated to this issue. looking at 
the jstack it seems like there is some kind of deadlock on logback, do you have 
enough disk space in your log dir? did you set your log level back to DEBUG 
from TRACE level (which can be quite verbose and generate too many log 
rotations - what may be contributing to the logback problem)?

> Reload compaction strategies when JBOD disk boundary changes
> 
>
> Key: CASSANDRA-13948
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13948
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Paulo Motta
>Assignee: Paulo Motta
> Fix For: 3.11.x, 4.x
>
> Attachments: debug.log, threaddump-cleanup.txt, threaddump.txt, 
> trace.log
>
>
> The thread dump below shows a race between an sstable replacement by the 
> {{IndexSummaryRedistribution}} and 
> {{AbstractCompactionTask.getNextBackgroundTask}}:
> {noformat}
> Thread 94580: (state = BLOCKED)
>  - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information 
> may be imprecise)
>  - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
> line=175 (Compiled frame)
>  - 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() 
> @bci=1, line=836 (Compiled frame)
>  - 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(java.util.concurrent.locks.AbstractQueuedSynchronizer$Node,
>  int) @bci=67, line=870 (Compiled frame)
>  - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(int) 
> @bci=17, line=1199 (Compiled frame)
>  - java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock() @bci=5, 
> line=943 (Compiled frame)
>  - 
> org.apache.cassandra.db.compaction.CompactionStrategyManager.handleListChangedNotification(java.lang.Iterable,
>  java.lang.Iterable) @bci=359, line=483 (Interpreted frame)
>  - 
> org.apache.cassandra.db.compaction.CompactionStrategyManager.handleNotification(org.apache.cassandra.notifications.INotification,
>  java.lang.Object) @bci=53, line=555 (Interpreted frame)
>  - 
> org.apache.cassandra.db.lifecycle.Tracker.notifySSTablesChanged(java.util.Collection,
>  java.util.Collection, org.apache.cassandra.db.compaction.OperationType, 
> java.lang.Throwable) @bci=50, line=409 (Interpreted frame)
>  - 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.doCommit(java.lang.Throwable)
>  @bci=157, line=227 (Interpreted frame)
>  - 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.commit(java.lang.Throwable)
>  @bci=61, line=116 (Compiled frame)
>  - 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.commit()
>  @bci=2, line=200 (Interpreted frame)
>  - 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.finish()
>  @bci=5, line=185 (Interpreted frame)
>  - 
> org.apache.cassandra.io.sstable.IndexSummaryRedistribution.redistributeSummaries()
>  @bci=559, line=130 (Interpreted frame)
>  - 
> org.apache.cassandra.db.compaction.CompactionManager.runIndexSummaryRedistribution(org.apache.cassandra.io.sstable.IndexSummaryRedistribution)
>  @bci=9, line=1420 (Interpreted frame)
>  - 
> org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(org.apache.cassandra.io.sstable.IndexSummaryRedistribution)
>  @bci=4, line=250 (Interpreted frame)
>  - 
> org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries() 
> @bci=30, line=228 (Interpreted frame)
>  - org.apache.cassandra.io.sstable.IndexSummaryManager$1.runMayThrow() 
> @bci=4, line=125 (Interpreted frame)
>  - org.apache.cassandra.utils.WrappedRunnable.run() @bci=1, line=28 
> (Interpreted frame)
>  - 
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run()
>  @bci=4, line=118 (Compiled frame)
>  - java.util.concurrent.Executors$RunnableAdapter.call() @bci=4, line=511 
> (Compiled frame)
>  - java.util.concurrent.FutureTask.runAndReset() @bci=47, line=308 (Compiled 
> frame)
>  - 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask)
>  @bci=1, line=180 (Compiled frame)
>  - java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run() 
> @bci=37,

[jira] [Commented] (CASSANDRA-14002) Don't use SHA256 when building merkle trees

2017-11-08 Thread Michael Kjellman (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16244696#comment-16244696
 ] 

Michael Kjellman commented on CASSANDRA-14002:
--

So +1 from me. Comments look good and I don't even see any code style related 
nits. 50% faster is a bigger win than I expected when I saw it in profiling a 
long long long time ago to be honest... so that's super awesome news! 

Looks like a few repair related dtests did fail but looking at stdout it seems 
that this is weirdness with the Apache Jenkins instance and ccm not starting 
instances cleanly and not any fallout from this change.

> Don't use SHA256 when building merkle trees
> ---
>
> Key: CASSANDRA-14002
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14002
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 4.x
>
>
> We should avoid using SHA-2 when building merkle trees as we don't need a 
> cryptographic hash function for this.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13958) [CQL] Inconsistent handling double dollar sign for strings

2017-11-08 Thread JIRA


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16244660#comment-16244660
 ] 

Michał Szczygieł commented on CASSANDRA-13958:
--

Thank you for the comments [~snazy]. I'm very new to the C* community and 
codebase and I was not aware that test method naming is a strict convention 
(which I personally don't like, because {{@Test}} already says that it's a 
test). Of course I'm going to change it and use the convention.

Regarding the tests and the behavior itself. I think I should change the 
approach and actually stop accepting leading and trailing dollars (or accept at 
most one on each side to be consistent and not break compatibility). Trailing 
is not a problem, but leading dollars may introduce confusion - should 
{{,}} be interpreted as {{$$,$$}} or two empty strings separated by a 
comma?

> [CQL] Inconsistent handling double dollar sign for strings
> --
>
> Key: CASSANDRA-13958
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13958
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Hugo Picado
>Assignee: Michał Szczygieł
>Priority: Minor
>
> Double dollar signs is a [built-in method for escaping columns that may 
> contain single quotes in its 
> content](https://docs.datastax.com/en/cql/3.3/cql/cql_reference/escape_char_r.html).
>  The way this is handled however is not consistent, in the sense that it 
> allows for $ to appear in the middle of the string but not in the last char.
> *Examples:*
> Valid: insert into users(id, name) values(1, $$john$$)
> Inserts the string *john*
> Valid: insert into users(id, name) values(1, $$jo$hn$$)
> Inserts the string *jo$hn*
> Valid: insert into users(id, name) values(1, $$$john$$)
> Inserts the string *$john*
> Invalid: insert into users(id, name) values(1, $$john$$$)
> Fails with:
> {code}
> Invalid syntax at line 1, char 48
>   insert into users(id, name) values(1, $$john$$$);
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14002) Don't use SHA256 when building merkle trees


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16244576#comment-16244576
 ] 

Jason Brown commented on CASSANDRA-14002:
-

bq. we don't need to worry about the upgrade path here because streaming will 
be broken between majors anyways

This is correct, but not because "streaming is broken between majors". There 
are no guarantees that an un-upgraded node would be parse to read sstables 
streamed to it from an upgrade node, as the sstable format may have changed (we 
do not force/guarantee backward compatibility of the sstable format). Thus, we 
don't support streaming between major versions (although there's nothing that 
bluntly states this, at least as far as I've seen).


> Don't use SHA256 when building merkle trees
> ---
>
> Key: CASSANDRA-14002
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14002
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 4.x
>
>
> We should avoid using SHA-2 when building merkle trees as we don't need a 
> cryptographic hash function for this.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14002) Don't use SHA256 when building merkle trees

2017-11-08 Thread Michael Kjellman (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16244559#comment-16244559
 ] 

Michael Kjellman commented on CASSANDRA-14002:
--

awesome! I'm guessing we don't need to worry about the upgrade path here 
because streaming will be broken between majors anyways, right?

> Don't use SHA256 when building merkle trees
> ---
>
> Key: CASSANDRA-14002
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14002
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 4.x
>
>
> We should avoid using SHA-2 when building merkle trees as we don't need a 
> cryptographic hash function for this.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14002) Don't use SHA256 when building merkle trees

2017-11-08 Thread Michael Kjellman (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Kjellman updated CASSANDRA-14002:
-
Reviewer: Michael Kjellman

> Don't use SHA256 when building merkle trees
> ---
>
> Key: CASSANDRA-14002
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14002
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 4.x
>
>
> We should avoid using SHA-2 when building merkle trees as we don't need a 
> cryptographic hash function for this.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13992) Don't send new_metadata_id for conditional updates

2017-11-08 Thread Olivier Michallat (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16244458#comment-16244458
 ] 

Olivier Michallat commented on CASSANDRA-13992:
---

bq. If C* thinks the metadata changed it will set the METADATA_CHANGED flag and 
the driver should need to update it's metadata. TBH this isn't super clear from 
the spec but appears to be what the code achieves here.
That makes sense, and indeed explains why the server ignores SKIP_METADATA.

I've tested your patch against a driver snapshot. The bound statement 
executions now always return the same (empty) digest, but the problem is that 
the initial preparation still returns a non-empty digest. So the driver 
executes with that initial digest and gets METADATA_CHANGED with the empty 
digest every time.
The prepare should also return an empty digest, which I think should be done 
around 
[here|https://github.com/kgreav/cassandra/blob/fa259fd79ea3e0a0fac8583a54c9c76464a653be/src/java/org/apache/cassandra/cql3/QueryProcessor.java#L444].

cc [~ifesdjeen]

> Don't send new_metadata_id for conditional updates
> --
>
> Key: CASSANDRA-13992
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13992
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Olivier Michallat
>Assignee: Kurt Greaves
>Priority: Minor
>
> This is a follow-up to CASSANDRA-10786.
> Given the table
> {code}
> CREATE TABLE foo (k int PRIMARY KEY)
> {code}
> And the prepared statement
> {code}
> INSERT INTO foo (k) VALUES (?) IF NOT EXISTS
> {code}
> The result set metadata changes depending on the outcome of the update:
> * if the row didn't exist, there is only a single column \[applied] = true
> * if it did, the result contains \[applied] = false, plus the current value 
> of column k.
> The way this was handled so far is that the PREPARED response contains no 
> result set metadata, and therefore all EXECUTE messages have SKIP_METADATA = 
> false, and the responses always include the full (and correct) metadata.
> CASSANDRA-10786 still sends the PREPARED response with no metadata, *but the 
> response to EXECUTE now contains a {{new_metadata_id}}*. The driver thinks it 
> is because of a schema change, and updates its local copy of the prepared 
> statement's result metadata.
> The next EXECUTE is sent with SKIP_METADATA = true, but the server appears to 
> ignore that, and still sends the metadata in the response. So each response 
> includes the correct metadata, the driver uses it, and there is no visible 
> issue for client code.
> The only drawback is that the driver updates its local copy of the metadata 
> unnecessarily, every time. We can work around that by only updating if we had 
> metadata before, at the cost of an extra volatile read. But I think the best 
> thing to do would be to never send a {{new_metadata_id}} in for a conditional 
> update.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13985) Support restricting reads and writes to specific datacenters on a per user basis


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16244367#comment-16244367
 ] 

Blake Eggleston commented on CASSANDRA-13985:
-

I agree that tacking this onto the existing permission system is a little 
hacky. Adding this as a separately managed control makes sense. 

The reason I was playing around with extending the existing hooks, vs adding a 
new one was because of ui reasons. Today we have grant and revoke statements, 
8303 will add create/drop restriction statements, this will probably add a 
statement for DCs, and I won’t be surprised if we see a rate/resource 
limitation thing get requested in the future. Depending on how this gets 
implemented, the cql statements behind it will probably overlap with either 
grant/revoke or create/drop restriction.

Maybe this isn’t that big of a deal though. If that’s the case, mapping 
statements to different controls under the hood wouldn't be too difficult.


> Support restricting reads and writes to specific datacenters on a per user 
> basis
> 
>
> Key: CASSANDRA-13985
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13985
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Minor
> Fix For: 4.0
>
>
> There are a few use cases where it makes sense to restrict the operations a 
> given user can perform in specific data centers. The obvious use case is the 
> production/analytics datacenter configuration. You don’t want the production 
> user to be reading/or writing to the analytics datacenter, and you don’t want 
> the analytics user to be reading from the production datacenter.
> Although we expect users to get this right on that application level, we 
> should also be able to enforce this at the database level. The first approach 
> that comes to mind would be to support an optional DC parameter when granting 
> select and modify permissions to roles. Something like {{GRANT SELECT ON 
> some_keyspace TO that_user IN DC dc1}}, statements that omit the dc would 
> implicitly be granting permission to all dcs. However, I’m not married to 
> this approach.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13948) Reload compaction strategies when JBOD disk boundary changes

2017-11-08 Thread Loic Lambiel (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Loic Lambiel updated CASSANDRA-13948:
-
Attachment: threaddump-cleanup.txt

Compactions are now processing as expected with your latest patch :)

However I'm facing an issue with nodetool cleanup, dunno if it is related or 
not.

Starting a cleanup cancel the ongoing compactions (which is expected from my 
understanding) and then get lost. Not performing any cleanup nor processing the 
pending compactions. 1 thread is using 100% of a core all the time. The log 
doesn't show any error. I've attached a thread dump.

I'm happy to open another Jira if it's not related.

> Reload compaction strategies when JBOD disk boundary changes
> 
>
> Key: CASSANDRA-13948
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13948
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Paulo Motta
>Assignee: Paulo Motta
> Fix For: 3.11.x, 4.x
>
> Attachments: debug.log, threaddump-cleanup.txt, threaddump.txt, 
> trace.log
>
>
> The thread dump below shows a race between an sstable replacement by the 
> {{IndexSummaryRedistribution}} and 
> {{AbstractCompactionTask.getNextBackgroundTask}}:
> {noformat}
> Thread 94580: (state = BLOCKED)
>  - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information 
> may be imprecise)
>  - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
> line=175 (Compiled frame)
>  - 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() 
> @bci=1, line=836 (Compiled frame)
>  - 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(java.util.concurrent.locks.AbstractQueuedSynchronizer$Node,
>  int) @bci=67, line=870 (Compiled frame)
>  - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(int) 
> @bci=17, line=1199 (Compiled frame)
>  - java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock() @bci=5, 
> line=943 (Compiled frame)
>  - 
> org.apache.cassandra.db.compaction.CompactionStrategyManager.handleListChangedNotification(java.lang.Iterable,
>  java.lang.Iterable) @bci=359, line=483 (Interpreted frame)
>  - 
> org.apache.cassandra.db.compaction.CompactionStrategyManager.handleNotification(org.apache.cassandra.notifications.INotification,
>  java.lang.Object) @bci=53, line=555 (Interpreted frame)
>  - 
> org.apache.cassandra.db.lifecycle.Tracker.notifySSTablesChanged(java.util.Collection,
>  java.util.Collection, org.apache.cassandra.db.compaction.OperationType, 
> java.lang.Throwable) @bci=50, line=409 (Interpreted frame)
>  - 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.doCommit(java.lang.Throwable)
>  @bci=157, line=227 (Interpreted frame)
>  - 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.commit(java.lang.Throwable)
>  @bci=61, line=116 (Compiled frame)
>  - 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.commit()
>  @bci=2, line=200 (Interpreted frame)
>  - 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.finish()
>  @bci=5, line=185 (Interpreted frame)
>  - 
> org.apache.cassandra.io.sstable.IndexSummaryRedistribution.redistributeSummaries()
>  @bci=559, line=130 (Interpreted frame)
>  - 
> org.apache.cassandra.db.compaction.CompactionManager.runIndexSummaryRedistribution(org.apache.cassandra.io.sstable.IndexSummaryRedistribution)
>  @bci=9, line=1420 (Interpreted frame)
>  - 
> org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(org.apache.cassandra.io.sstable.IndexSummaryRedistribution)
>  @bci=4, line=250 (Interpreted frame)
>  - 
> org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries() 
> @bci=30, line=228 (Interpreted frame)
>  - org.apache.cassandra.io.sstable.IndexSummaryManager$1.runMayThrow() 
> @bci=4, line=125 (Interpreted frame)
>  - org.apache.cassandra.utils.WrappedRunnable.run() @bci=1, line=28 
> (Interpreted frame)
>  - 
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run()
>  @bci=4, line=118 (Compiled frame)
>  - java.util.concurrent.Executors$RunnableAdapter.call() @bci=4, line=511 
> (Compiled frame)
>  - java.util.concurrent.FutureTask.runAndReset() @bci=47, line=308 (Compiled 
> frame)
>  - 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask)
>  @bci=1, line=180 (Compiled frame)
>  - java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run() 
> @bci=37, line=294 (Compiled frame)
>  - 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
>  @bci=95, line=1149 (Compiled frame)

[jira] [Updated] (CASSANDRA-13403) nodetool repair breaks SASI index

2017-11-08 Thread Ludovic Boutros (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ludovic Boutros updated CASSANDRA-13403:

Attachment: CASSANDRA-13403.patch

> nodetool repair breaks SASI index
> -
>
> Key: CASSANDRA-13403
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13403
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
> Environment: 3.10
>Reporter: Igor Novgorodov
>Assignee: Alex Petrov
>  Labels: patch
> Attachments: 3_nodes_compaction.log, 4_nodes_compaction.log, 
> CASSANDRA-13403.patch, testSASIRepair.patch
>
>
> I've got table:
> {code}
> CREATE TABLE cservice.bulks_recipients (
> recipient text,
> bulk_id uuid,
> datetime_final timestamp,
> datetime_sent timestamp,
> request_id uuid,
> status int,
> PRIMARY KEY (recipient, bulk_id)
> ) WITH CLUSTERING ORDER BY (bulk_id ASC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'ALL'}
> AND comment = ''
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '64', 'class': 
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE';
> CREATE CUSTOM INDEX bulk_recipients_bulk_id ON cservice.bulks_recipients 
> (bulk_id) USING 'org.apache.cassandra.index.sasi.SASIIndex';
> {code}
> There are 11 rows in it:
> {code}
> > select * from bulks_recipients;
> ...
> (11 rows)
> {code}
> Let's query by index (all rows have the same *bulk_id*):
> {code}
> > select * from bulks_recipients where bulk_id = 
> > baa94815-e276-4ca4-adda-5b9734e6c4a5;   
> >   
> ...
> (11 rows)
> {code}
> Ok, everything is fine.
> Now i'm doing *nodetool repair --partitioner-range --job-threads 4 --full* on 
> each node in cluster sequentially.
> After it finished:
> {code}
> > select * from bulks_recipients where bulk_id = 
> > baa94815-e276-4ca4-adda-5b9734e6c4a5;
> ...
> (2 rows)
> {code}
> Only two rows.
> While the rows are actually there:
> {code}
> > select * from bulks_recipients;
> ...
> (11 rows)
> {code}
> If i issue an incremental repair on a random node, i can get like 7 rows 
> after index query.
> Dropping index and recreating it fixes the issue. Is it a bug or am i doing 
> the repair the wrong way?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13403) nodetool repair breaks SASI index

2017-11-08 Thread Ludovic Boutros (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ludovic Boutros updated CASSANDRA-13403:

   Labels: patch  (was: )
 Reviewer: Alex Petrov
Reproduced In: 3.11.1, 3.10  (was: 3.10)
   Status: Patch Available  (was: Open)

A patch against the 3.11.1 branch

> nodetool repair breaks SASI index
> -
>
> Key: CASSANDRA-13403
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13403
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
> Environment: 3.10
>Reporter: Igor Novgorodov
>Assignee: Alex Petrov
>  Labels: patch
> Attachments: 3_nodes_compaction.log, 4_nodes_compaction.log, 
> testSASIRepair.patch
>
>
> I've got table:
> {code}
> CREATE TABLE cservice.bulks_recipients (
> recipient text,
> bulk_id uuid,
> datetime_final timestamp,
> datetime_sent timestamp,
> request_id uuid,
> status int,
> PRIMARY KEY (recipient, bulk_id)
> ) WITH CLUSTERING ORDER BY (bulk_id ASC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'ALL'}
> AND comment = ''
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '64', 'class': 
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE';
> CREATE CUSTOM INDEX bulk_recipients_bulk_id ON cservice.bulks_recipients 
> (bulk_id) USING 'org.apache.cassandra.index.sasi.SASIIndex';
> {code}
> There are 11 rows in it:
> {code}
> > select * from bulks_recipients;
> ...
> (11 rows)
> {code}
> Let's query by index (all rows have the same *bulk_id*):
> {code}
> > select * from bulks_recipients where bulk_id = 
> > baa94815-e276-4ca4-adda-5b9734e6c4a5;   
> >   
> ...
> (11 rows)
> {code}
> Ok, everything is fine.
> Now i'm doing *nodetool repair --partitioner-range --job-threads 4 --full* on 
> each node in cluster sequentially.
> After it finished:
> {code}
> > select * from bulks_recipients where bulk_id = 
> > baa94815-e276-4ca4-adda-5b9734e6c4a5;
> ...
> (2 rows)
> {code}
> Only two rows.
> While the rows are actually there:
> {code}
> > select * from bulks_recipients;
> ...
> (11 rows)
> {code}
> If i issue an incremental repair on a random node, i can get like 7 rows 
> after index query.
> Dropping index and recreating it fixes the issue. Is it a bug or am i doing 
> the repair the wrong way?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13403) nodetool repair breaks SASI index

2017-11-08 Thread Ludovic Boutros (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16244283#comment-16244283
 ] 

Ludovic Boutros commented on CASSANDRA-13403:
-

[~ifesdjeen],

adding a flag in the class 'SSTableRewriter' in order to prevent the call to 
the commit function of the shared transaction does the job and my unit test 
works.

But I really don't know if it is the good way to go.

I will attach a small patch just to let you see what I've done.
I don't know what to do for the 'toPrepare' and 'doAbort' functions of the 
'SSTableRewriter'. Currently I've just prevented the call to the shared 
transaction as it's done for the commit function.

Please let me know what you think about these changes.

> nodetool repair breaks SASI index
> -
>
> Key: CASSANDRA-13403
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13403
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
> Environment: 3.10
>Reporter: Igor Novgorodov
>Assignee: Alex Petrov
> Attachments: 3_nodes_compaction.log, 4_nodes_compaction.log, 
> testSASIRepair.patch
>
>
> I've got table:
> {code}
> CREATE TABLE cservice.bulks_recipients (
> recipient text,
> bulk_id uuid,
> datetime_final timestamp,
> datetime_sent timestamp,
> request_id uuid,
> status int,
> PRIMARY KEY (recipient, bulk_id)
> ) WITH CLUSTERING ORDER BY (bulk_id ASC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'ALL'}
> AND comment = ''
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '64', 'class': 
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE';
> CREATE CUSTOM INDEX bulk_recipients_bulk_id ON cservice.bulks_recipients 
> (bulk_id) USING 'org.apache.cassandra.index.sasi.SASIIndex';
> {code}
> There are 11 rows in it:
> {code}
> > select * from bulks_recipients;
> ...
> (11 rows)
> {code}
> Let's query by index (all rows have the same *bulk_id*):
> {code}
> > select * from bulks_recipients where bulk_id = 
> > baa94815-e276-4ca4-adda-5b9734e6c4a5;   
> >   
> ...
> (11 rows)
> {code}
> Ok, everything is fine.
> Now i'm doing *nodetool repair --partitioner-range --job-threads 4 --full* on 
> each node in cluster sequentially.
> After it finished:
> {code}
> > select * from bulks_recipients where bulk_id = 
> > baa94815-e276-4ca4-adda-5b9734e6c4a5;
> ...
> (2 rows)
> {code}
> Only two rows.
> While the rows are actually there:
> {code}
> > select * from bulks_recipients;
> ...
> (11 rows)
> {code}
> If i issue an incremental repair on a random node, i can get like 7 rows 
> after index query.
> Dropping index and recreating it fixes the issue. Is it a bug or am i doing 
> the repair the wrong way?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13985) Support restricting reads and writes to specific datacenters on a per user basis


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Blake Eggleston updated CASSANDRA-13985:

Issue Type: Improvement  (was: Bug)

> Support restricting reads and writes to specific datacenters on a per user 
> basis
> 
>
> Key: CASSANDRA-13985
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13985
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Minor
> Fix For: 4.0
>
>
> There are a few use cases where it makes sense to restrict the operations a 
> given user can perform in specific data centers. The obvious use case is the 
> production/analytics datacenter configuration. You don’t want the production 
> user to be reading/or writing to the analytics datacenter, and you don’t want 
> the analytics user to be reading from the production datacenter.
> Although we expect users to get this right on that application level, we 
> should also be able to enforce this at the database level. The first approach 
> that comes to mind would be to support an optional DC parameter when granting 
> select and modify permissions to roles. Something like {{GRANT SELECT ON 
> some_keyspace TO that_user IN DC dc1}}, statements that omit the dc would 
> implicitly be granting permission to all dcs. However, I’m not married to 
> this approach.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13985) Support restricting reads and writes to specific datacenters on a per user basis


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Blake Eggleston updated CASSANDRA-13985:

Fix Version/s: 4.0

> Support restricting reads and writes to specific datacenters on a per user 
> basis
> 
>
> Key: CASSANDRA-13985
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13985
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Minor
> Fix For: 4.0
>
>
> There are a few use cases where it makes sense to restrict the operations a 
> given user can perform in specific data centers. The obvious use case is the 
> production/analytics datacenter configuration. You don’t want the production 
> user to be reading/or writing to the analytics datacenter, and you don’t want 
> the analytics user to be reading from the production datacenter.
> Although we expect users to get this right on that application level, we 
> should also be able to enforce this at the database level. The first approach 
> that comes to mind would be to support an optional DC parameter when granting 
> select and modify permissions to roles. Something like {{GRANT SELECT ON 
> some_keyspace TO that_user IN DC dc1}}, statements that omit the dc would 
> implicitly be granting permission to all dcs. However, I’m not married to 
> this approach.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13997) Upgrade guava to 23.3


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16244242#comment-16244242
 ] 

Marcus Eriksson commented on CASSANDRA-13997:
-

pushed up a commit to remove the j2obj dep from build.xml

> Upgrade guava to 23.3
> -
>
> Key: CASSANDRA-13997
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13997
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 4.x
>
>
> For 4.0 we should upgrade guava to the latest version
> patch here: https://github.com/krummas/cassandra/commits/marcuse/guava23
> A bunch of quite commonly used methods have been deprecated since guava 18 
> which we use now ({{Throwables.propagate}} for example), this patch mostly 
> updates uses where compilation fails. {{Futures.transform(ListenableFuture 
> ..., AsyncFunction ...}} was deprecated in Guava 19 and removed in 20 for 
> example, we should probably open new tickets to remove calls to all 
> deprecated guava methods.
> Also had to add a dependency on {{com.google.j2objc.j2objc-annotations}}, to 
> avoid some build-time warnings (maybe due to 
> https://github.com/google/guava/commit/fffd2b1f67d158c7b4052123c5032b0ba54a910d
>  ?)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-13997) Upgrade guava to 23.3


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16244242#comment-16244242
 ] 

Marcus Eriksson edited comment on CASSANDRA-13997 at 11/8/17 4:16 PM:
--

pushed up a commit to remove the j2objc dep from build.xml


was (Author: krummas):
pushed up a commit to remove the j2obj dep from build.xml

> Upgrade guava to 23.3
> -
>
> Key: CASSANDRA-13997
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13997
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 4.x
>
>
> For 4.0 we should upgrade guava to the latest version
> patch here: https://github.com/krummas/cassandra/commits/marcuse/guava23
> A bunch of quite commonly used methods have been deprecated since guava 18 
> which we use now ({{Throwables.propagate}} for example), this patch mostly 
> updates uses where compilation fails. {{Futures.transform(ListenableFuture 
> ..., AsyncFunction ...}} was deprecated in Guava 19 and removed in 20 for 
> example, we should probably open new tickets to remove calls to all 
> deprecated guava methods.
> Also had to add a dependency on {{com.google.j2objc.j2objc-annotations}}, to 
> avoid some build-time warnings (maybe due to 
> https://github.com/google/guava/commit/fffd2b1f67d158c7b4052123c5032b0ba54a910d
>  ?)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13997) Upgrade guava to 23.3

2017-11-08 Thread Stefan Podkowinski (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16244236#comment-16244236
 ] 

Stefan Podkowinski commented on CASSANDRA-13997:


Ah, never mind. These dependencies have been introduced in January for v22 
already. Should have clicked that little "..." icon next to the tag for the 
commit. Guess there's no going back then, as they seem to have settled down for 
the [decision|https://github.com/google/guava/issues/2824] to keep the 
dependencies non-optional.

> Upgrade guava to 23.3
> -
>
> Key: CASSANDRA-13997
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13997
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 4.x
>
>
> For 4.0 we should upgrade guava to the latest version
> patch here: https://github.com/krummas/cassandra/commits/marcuse/guava23
> A bunch of quite commonly used methods have been deprecated since guava 18 
> which we use now ({{Throwables.propagate}} for example), this patch mostly 
> updates uses where compilation fails. {{Futures.transform(ListenableFuture 
> ..., AsyncFunction ...}} was deprecated in Guava 19 and removed in 20 for 
> example, we should probably open new tickets to remove calls to all 
> deprecated guava methods.
> Also had to add a dependency on {{com.google.j2objc.j2objc-annotations}}, to 
> avoid some build-time warnings (maybe due to 
> https://github.com/google/guava/commit/fffd2b1f67d158c7b4052123c5032b0ba54a910d
>  ?)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13997) Upgrade guava to 23.3


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16244216#comment-16244216
 ] 

Marcus Eriksson commented on CASSANDRA-13997:
-

bq. I don't think you have to add j2objc-annotations to the pom (build.xml) as 
well, as it's already a transitive guava dependency.
Good point

bq. There are also other dependencies that are no longer optional and not 
everyone seems to be happy about it.
Is there a discussion about this somewhere?

bq. Shouldn't this be reason enough to stick with 23.2 instead 
Couldn't see anything obvious in the diff between 23.2 and .3 that could affect 
us, so if you think that is better, I'm totally ok with that

> Upgrade guava to 23.3
> -
>
> Key: CASSANDRA-13997
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13997
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 4.x
>
>
> For 4.0 we should upgrade guava to the latest version
> patch here: https://github.com/krummas/cassandra/commits/marcuse/guava23
> A bunch of quite commonly used methods have been deprecated since guava 18 
> which we use now ({{Throwables.propagate}} for example), this patch mostly 
> updates uses where compilation fails. {{Futures.transform(ListenableFuture 
> ..., AsyncFunction ...}} was deprecated in Guava 19 and removed in 20 for 
> example, we should probably open new tickets to remove calls to all 
> deprecated guava methods.
> Also had to add a dependency on {{com.google.j2objc.j2objc-annotations}}, to 
> avoid some build-time warnings (maybe due to 
> https://github.com/google/guava/commit/fffd2b1f67d158c7b4052123c5032b0ba54a910d
>  ?)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13996) Close DataInputBuffer in MetadataSerializer


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16244171#comment-16244171
 ] 

Marcus Eriksson commented on CASSANDRA-13996:
-

[~iamaleksey] seems {{@SuppressWarnings("resource")}} silences this warning, 
but its a rather large method so I opted to just close the DIB instead of 
silencing the warnings for the whole thing

> Close DataInputBuffer in MetadataSerializer
> ---
>
> Key: CASSANDRA-13996
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13996
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 4.0
>
>
> eclipse-warnings complains about this, either introduced by CASSANDRA-13321 
> or CASSANDRA-13953
> Patch here: https://github.com/krummas/cassandra/commits/marcuse/closeDIB
> https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/416/
> https://circleci.com/gh/krummas/cassandra/170



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13997) Upgrade guava to 23.3

2017-11-08 Thread Stefan Podkowinski (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16244185#comment-16244185
 ] 

Stefan Podkowinski commented on CASSANDRA-13997:


I don't think you have to add {{j2objc-annotations}} to the pom (build.xml) as 
well, as it's already a transitive guava dependency. There are also other 
dependencies that are [no longer 
optional|https://github.com/google/guava/commit/0e29934933f33379de953727171f7ca1ec616a58]
 and not everyone seems to be happy about it. Shouldn't this be reason enough 
to stick with 23.2 instead for the time being?


> Upgrade guava to 23.3
> -
>
> Key: CASSANDRA-13997
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13997
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 4.x
>
>
> For 4.0 we should upgrade guava to the latest version
> patch here: https://github.com/krummas/cassandra/commits/marcuse/guava23
> A bunch of quite commonly used methods have been deprecated since guava 18 
> which we use now ({{Throwables.propagate}} for example), this patch mostly 
> updates uses where compilation fails. {{Futures.transform(ListenableFuture 
> ..., AsyncFunction ...}} was deprecated in Guava 19 and removed in 20 for 
> example, we should probably open new tickets to remove calls to all 
> deprecated guava methods.
> Also had to add a dependency on {{com.google.j2objc.j2objc-annotations}}, to 
> avoid some build-time warnings (maybe due to 
> https://github.com/google/guava/commit/fffd2b1f67d158c7b4052123c5032b0ba54a910d
>  ?)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13996) Close DataInputBuffer in MetadataSerializer


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-13996:

Resolution: Fixed
Status: Resolved  (was: Ready to Commit)

committed as {{6186ad97a8239c9062363c2a4a23cf31c4e6e3c2}}

> Close DataInputBuffer in MetadataSerializer
> ---
>
> Key: CASSANDRA-13996
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13996
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 4.x
>
>
> eclipse-warnings complains about this, either introduced by CASSANDRA-13321 
> or CASSANDRA-13953
> Patch here: https://github.com/krummas/cassandra/commits/marcuse/closeDIB
> https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/416/
> https://circleci.com/gh/krummas/cassandra/170



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13996) Close DataInputBuffer in MetadataSerializer


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-13996:

 Reviewer: Jason Brown  (was: Aleksey Yeschenko)
Fix Version/s: (was: 4.x)
   4.0

> Close DataInputBuffer in MetadataSerializer
> ---
>
> Key: CASSANDRA-13996
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13996
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 4.0
>
>
> eclipse-warnings complains about this, either introduced by CASSANDRA-13321 
> or CASSANDRA-13953
> Patch here: https://github.com/krummas/cassandra/commits/marcuse/closeDIB
> https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/416/
> https://circleci.com/gh/krummas/cassandra/170



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

cassandra git commit: Close DataInputBuffer to silence eclipse-warnings

2017-11-08 Thread marcuse

Repository: cassandra
Updated Branches:
  refs/heads/trunk 65ff3e6d9 -> 6186ad97a


Close DataInputBuffer to silence eclipse-warnings

Patch by marcuse; reviewed by Jason Brown for CASSANDRA-13996


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6186ad97
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6186ad97
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6186ad97

Branch: refs/heads/trunk
Commit: 6186ad97a8239c9062363c2a4a23cf31c4e6e3c2
Parents: 65ff3e6
Author: Marcus Eriksson 
Authored: Mon Nov 6 10:46:07 2017 +0100
Committer: Marcus Eriksson 
Committed: Wed Nov 8 16:26:33 2017 +0100

--
 .../cassandra/io/sstable/metadata/MetadataSerializer.java  | 6 --
 1 file changed, 4 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6186ad97/src/java/org/apache/cassandra/io/sstable/metadata/MetadataSerializer.java
--
diff --git 
a/src/java/org/apache/cassandra/io/sstable/metadata/MetadataSerializer.java 
b/src/java/org/apache/cassandra/io/sstable/metadata/MetadataSerializer.java
index 9499ff3..c1bbeb0 100644
--- a/src/java/org/apache/cassandra/io/sstable/metadata/MetadataSerializer.java
+++ b/src/java/org/apache/cassandra/io/sstable/metadata/MetadataSerializer.java
@@ -195,8 +195,10 @@ public class MetadataSerializer implements 
IMetadataSerializer
 
 crc.reset(); crc.update(buffer);
 maybeValidateChecksum(crc, in, descriptor);
-
-components.put(type, 
type.serializer.deserialize(descriptor.version, new DataInputBuffer(buffer)));
+try (DataInputBuffer dataInputBuffer = new DataInputBuffer(buffer))
+{
+components.put(type, 
type.serializer.deserialize(descriptor.version, dataInputBuffer));
+}
 }
 
 return components;


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13953) Switch to CRC32 for sstable metadata checksums

2017-11-08 Thread Aleksey Yeschenko (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16244140#comment-16244140
 ] 

Aleksey Yeschenko commented on CASSANDRA-13953:
---

It's a bogus warning, but the fix is underway anyway.

> Switch to CRC32 for sstable metadata checksums
> --
>
> Key: CASSANDRA-13953
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13953
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Fix For: 4.0
>
>
> We should switch to CRC32 for sstable metadata checksumming for consistency 
> with the rest of the code base. There are a few other cleanups that should be 
> done at the same time.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13953) Switch to CRC32 for sstable metadata checksums


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16244127#comment-16244127
 ] 

Marcus Eriksson commented on CASSANDRA-13953:
-

[~spo...@gmail.com] yeah, CASSANDRA-13996

> Switch to CRC32 for sstable metadata checksums
> --
>
> Key: CASSANDRA-13953
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13953
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Fix For: 4.0
>
>
> We should switch to CRC32 for sstable metadata checksumming for consistency 
> with the rest of the code base. There are a few other cleanups that should be 
> done at the same time.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13953) Switch to CRC32 for sstable metadata checksums

2017-11-08 Thread Stefan Podkowinski (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16244124#comment-16244124
 ] 

Stefan Podkowinski commented on CASSANDRA-13953:


Running {{ant eclipse-warnings}} will fail with the following error:

{noformat}
 [java] 1. ERROR in 
/home/spod/git/cassandra-trunk/src/java/org/apache/cassandra/io/sstable/metadata/MetadataSerializer.java
 (at line 199)
 [java] components.put(type, 
type.serializer.deserialize(descriptor.version, new DataInputBuffer(buffer)));
 [java] 
 ^^^
 [java] Potential resource leak: '' may not be 
closed
 [java] --
{noformat}



> Switch to CRC32 for sstable metadata checksums
> --
>
> Key: CASSANDRA-13953
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13953
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Fix For: 4.0
>
>
> We should switch to CRC32 for sstable metadata checksumming for consistency 
> with the rest of the code base. There are a few other cleanups that should be 
> done at the same time.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13985) Support restricting reads and writes to specific datacenters on a per user basis

2017-11-08 Thread Sam Tunnicliffe (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16244121#comment-16244121
 ] 

Sam Tunnicliffe commented on CASSANDRA-13985:
-

Postgres has something vaguely similar in its {{pg_hba.conf}}, which lets you 
restrict whether clients can connect based on various criteria such as user, 
host, ip range, etc. To block a user/role entirely you would just remove all of 
its entries in the config file. It's not directly analogous as postgres also 
supports multiple authn methods and obviously isn't distributed so the 
restrictions apply to the single db server, but it is separating the access to 
the service itself from the specifics of auth(n|z).


> Support restricting reads and writes to specific datacenters on a per user 
> basis
> 
>
> Key: CASSANDRA-13985
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13985
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Minor
>
> There are a few use cases where it makes sense to restrict the operations a 
> given user can perform in specific data centers. The obvious use case is the 
> production/analytics datacenter configuration. You don’t want the production 
> user to be reading/or writing to the analytics datacenter, and you don’t want 
> the analytics user to be reading from the production datacenter.
> Although we expect users to get this right on that application level, we 
> should also be able to enforce this at the database level. The first approach 
> that comes to mind would be to support an optional DC parameter when granting 
> select and modify permissions to roles. Something like {{GRANT SELECT ON 
> some_keyspace TO that_user IN DC dc1}}, statements that omit the dc would 
> implicitly be granting permission to all dcs. However, I’m not married to 
> this approach.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-13985) Support restricting reads and writes to specific datacenters on a per user basis

2017-11-08 Thread Sam Tunnicliffe (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16243985#comment-16243985
 ] 

Sam Tunnicliffe edited comment on CASSANDRA-13985 at 11/8/17 2:19 PM:
--

In general I agree with the aim of this ticket, but I think perhaps we should 
come at it from a different angle.

First, DCs don't fit well into the authz model as it's defined currently, with 
Roles, Resources & Permissions, so we have to extend that model and make DC a 
first class citizen. I don't think this is huge problem in itself, but it 
follows that if we do that we should do the same for Capabilities as proposed 
in CASSANDRA-8303. I appreciate that that ticket is still in discussion and 
also hasn't moved in quite a while, but I do think it's still worth doing and 
plan/hope to get back to it soon. Adding another component to either of those 
models is going to increase complexity which is always best avoided in this 
area. 

Second, I'm not altogether convinced that having a given user's permissions (or 
capabilities) modified by DC is the right way to go. Having varying privileges 
dependent on which coordinator you happen to be connected to seems like it 
could end up being a bit counter-intuitive. My feeling is that it makes more 
sense to control whether a role is permitted to access a location at all. If 
so, then normal permissions should apply. 

Third, I'm not a big fan of the implicit granting by omission, even though it's 
more or less consistent with the way permissions work, it somehow feels kind of 
jarring here (but I'm afraid I'm struggling to articulate exactly why).

These things make me think that this might be better implemented as an entirely 
separate feature, distinct from permissions and capabilities. Rather than 
adding complexity by extending those models we could add a new thing which 
simply bans a user from connecting to nodes in a given DC or blocks execution 
of CQL statements for already connected clients. This would be really simple to 
implement and would provide better separation from the code and concepts of 
permissions and capabilities.


was (Author: beobal):
In general I agree with the aim of this ticket, but I think perhaps we should 
come at it from a different angle.

First, DCs don't fit well into the authz model as it's defined currently, with 
Roles, Resources & Permissions, so we have to extend that model and make DC a 
first class citizen. I don't think this is huge problem in itself, but it 
follows that if we do that we should do the same for Capabilities as proposed 
in CASSANDRA-8303. I appreciate that that ticket is still in discussion and 
also hasn't moved in quite a while, but I do think it's still worth doing and 
plan/hope to get back to it soon. Adding another component to either of those 
models is 

Second, I'm not altogether convinced that having a given user's permissions (or 
capabilities) modified by DC is the right way to go. Having varying privileges 
dependent on which coordinator you happen to be connected to seems like it 
could end up being a bit counter-intuitive. My feeling is that it makes more 
sense to control whether a role is permitted to access a location at all. If 
so, then normal permissions should apply. 

Third, I'm not a big fan of the implicit granting by omission, even though it's 
more or less consistent with the way permissions work, it somehow feels kind of 
jarring here (but I'm afraid I'm struggling to articulate exactly why).

These things make me think that this might be better implemented as an entirely 
separate feature, distinct from permissions and capabilities. Rather than 
adding complexity by extending those models we could add a new thing which 
simply bans a user from connecting to nodes in a given DC or blocks execution 
of CQL statements for already connected clients. This would be really simple to 
implement and would provide better separation from the code and concepts of 
permissions and capabilities.

> Support restricting reads and writes to specific datacenters on a per user 
> basis
> 
>
> Key: CASSANDRA-13985
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13985
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Minor
>
> There are a few use cases where it makes sense to restrict the operations a 
> given user can perform in specific data centers. The obvious use case is the 
> production/analytics datacenter configuration. You don’t want the production 
> user to be reading/or writing to the analytics datacenter, and you don’t want 
> the analytics user to be reading from the production datacenter.
> Although we expect users to get this right on that

[jira] [Commented] (CASSANDRA-13985) Support restricting reads and writes to specific datacenters on a per user basis

2017-11-08 Thread Sam Tunnicliffe (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16243985#comment-16243985
 ] 

Sam Tunnicliffe commented on CASSANDRA-13985:
-

In general I agree with the aim of this ticket, but I think perhaps we should 
come at it from a different angle.

First, DCs don't fit well into the authz model as it's defined currently, with 
Roles, Resources & Permissions, so we have to extend that model and make DC a 
first class citizen. I don't think this is huge problem in itself, but it 
follows that if we do that we should do the same for Capabilities as proposed 
in CASSANDRA-8303. I appreciate that that ticket is still in discussion and 
also hasn't moved in quite a while, but I do think it's still worth doing and 
plan/hope to get back to it soon. Adding another component to either of those 
models is 

Second, I'm not altogether convinced that having a given user's permissions (or 
capabilities) modified by DC is the right way to go. Having varying privileges 
dependent on which coordinator you happen to be connected to seems like it 
could end up being a bit counter-intuitive. My feeling is that it makes more 
sense to control whether a role is permitted to access a location at all. If 
so, then normal permissions should apply. 

Third, I'm not a big fan of the implicit granting by omission, even though it's 
more or less consistent with the way permissions work, it somehow feels kind of 
jarring here (but I'm afraid I'm struggling to articulate exactly why).

These things make me think that this might be better implemented as an entirely 
separate feature, distinct from permissions and capabilities. Rather than 
adding complexity by extending those models we could add a new thing which 
simply bans a user from connecting to nodes in a given DC or blocks execution 
of CQL statements for already connected clients. This would be really simple to 
implement and would provide better separation from the code and concepts of 
permissions and capabilities.

> Support restricting reads and writes to specific datacenters on a per user 
> basis
> 
>
> Key: CASSANDRA-13985
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13985
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Minor
>
> There are a few use cases where it makes sense to restrict the operations a 
> given user can perform in specific data centers. The obvious use case is the 
> production/analytics datacenter configuration. You don’t want the production 
> user to be reading/or writing to the analytics datacenter, and you don’t want 
> the analytics user to be reading from the production datacenter.
> Although we expect users to get this right on that application level, we 
> should also be able to enforce this at the database level. The first approach 
> that comes to mind would be to support an optional DC parameter when granting 
> select and modify permissions to roles. Something like {{GRANT SELECT ON 
> some_keyspace TO that_user IN DC dc1}}, statements that omit the dc would 
> implicitly be granting permission to all dcs. However, I’m not married to 
> this approach.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13996) Close DataInputBuffer in MetadataSerializer

2017-11-08 Thread Aleksey Yeschenko (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16243980#comment-16243980
 ] 

Aleksey Yeschenko commented on CASSANDRA-13996:
---

No harm in it though, sorry. +1 too. I am now curious however if there is an 
annotation that eclipse accepts.

> Close DataInputBuffer in MetadataSerializer
> ---
>
> Key: CASSANDRA-13996
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13996
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 4.x
>
>
> eclipse-warnings complains about this, either introduced by CASSANDRA-13321 
> or CASSANDRA-13953
> Patch here: https://github.com/krummas/cassandra/commits/marcuse/closeDIB
> https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/416/
> https://circleci.com/gh/krummas/cassandra/170



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13996) Close DataInputBuffer in MetadataSerializer

2017-11-08 Thread Aleksey Yeschenko (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-13996:
--
Status: Ready to Commit  (was: Patch Available)

> Close DataInputBuffer in MetadataSerializer
> ---
>
> Key: CASSANDRA-13996
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13996
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 4.x
>
>
> eclipse-warnings complains about this, either introduced by CASSANDRA-13321 
> or CASSANDRA-13953
> Patch here: https://github.com/krummas/cassandra/commits/marcuse/closeDIB
> https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/416/
> https://circleci.com/gh/krummas/cassandra/170



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Resolved] (CASSANDRA-13986) Fix native protocol v5 spec for new_metadata_id position in Rows response

2017-11-08 Thread Alex Petrov (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov resolved CASSANDRA-13986.
-
Resolution: Fixed

> Fix native protocol v5 spec for new_metadata_id position in Rows response
> -
>
> Key: CASSANDRA-13986
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13986
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Olivier Michallat
>Assignee: Alex Petrov
>Priority: Trivial
>
> There's a mistake in the protocol specification for CASSANDRA-10786. In 
> `native_protocol_v5.spec`, section 4.2.5.2:
> {code}
> 4.2.5.2. Rows
>   Indicates a set of rows. The rest of the body of a Rows result is:
> 
>   where:
> -  is composed of:
> 
> [][][?...]
> {code}
> The last line should be:
> {code}
> 
> [][][?...]
> {code}
> That is, if there is both a paging state and a new metadata id, the paging 
> state comes *first*, not second.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13986) Fix native protocol v5 spec for new_metadata_id position in Rows response

2017-11-08 Thread Alex Petrov (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16243800#comment-16243800
 ] 

Alex Petrov commented on CASSANDRA-13986:
-

Ninja-committed to {{trunk}} with 
[65ff3e6d9e15060786fe5fdec92005b9932cab08|https://github.com/apache/cassandra/commit/65ff3e6d9e15060786fe5fdec92005b9932cab08]

> Fix native protocol v5 spec for new_metadata_id position in Rows response
> -
>
> Key: CASSANDRA-13986
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13986
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Olivier Michallat
>Assignee: Alex Petrov
>Priority: Trivial
>
> There's a mistake in the protocol specification for CASSANDRA-10786. In 
> `native_protocol_v5.spec`, section 4.2.5.2:
> {code}
> 4.2.5.2. Rows
>   Indicates a set of rows. The rest of the body of a Rows result is:
> 
>   where:
> -  is composed of:
> 
> [][][?...]
> {code}
> The last line should be:
> {code}
> 
> [][][?...]
> {code}
> That is, if there is both a paging state and a new metadata id, the paging 
> state comes *first*, not second.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

cassandra git commit: Ninja: fix v5 protocol spec, paging state and metadata id order in result metadata.

2017-11-08 Thread ifesdjeen

Repository: cassandra
Updated Branches:
  refs/heads/trunk 976f48fb0 -> 65ff3e6d9


Ninja: fix v5 protocol spec, paging state and metadata id order in result 
metadata.

Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/65ff3e6d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/65ff3e6d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/65ff3e6d

Branch: refs/heads/trunk
Commit: 65ff3e6d9e15060786fe5fdec92005b9932cab08
Parents: 976f48f
Author: Alex Petrov 
Authored: Wed Nov 8 13:13:00 2017 +0100
Committer: Alex Petrov 
Committed: Wed Nov 8 13:13:00 2017 +0100

--
 doc/native_protocol_v5.spec | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/65ff3e6d/doc/native_protocol_v5.spec
--
diff --git a/doc/native_protocol_v5.spec b/doc/native_protocol_v5.spec
index 0addbc4..7f6ba42 100644
--- a/doc/native_protocol_v5.spec
+++ b/doc/native_protocol_v5.spec
@@ -586,7 +586,7 @@ Table of Contents
 
   where:
 -  is composed of:
-
[][][?...]
+
[][][?...]
   where:
 -  is an [int]. The bits of  provides information on the
   formatting of the remaining information. A flag is set if the bit


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13997) Upgrade guava to 23.3


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16243548#comment-16243548
 ] 

Marcus Eriksson commented on CASSANDRA-13997:
-

[~tkruse] 3.11 is mostly in bugfix mode now, could you open a new ticket where 
this could be discussed?

> Upgrade guava to 23.3
> -
>
> Key: CASSANDRA-13997
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13997
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 4.x
>
>
> For 4.0 we should upgrade guava to the latest version
> patch here: https://github.com/krummas/cassandra/commits/marcuse/guava23
> A bunch of quite commonly used methods have been deprecated since guava 18 
> which we use now ({{Throwables.propagate}} for example), this patch mostly 
> updates uses where compilation fails. {{Futures.transform(ListenableFuture 
> ..., AsyncFunction ...}} was deprecated in Guava 19 and removed in 20 for 
> example, we should probably open new tickets to remove calls to all 
> deprecated guava methods.
> Also had to add a dependency on {{com.google.j2objc.j2objc-annotations}}, to 
> avoid some build-time warnings (maybe due to 
> https://github.com/google/guava/commit/fffd2b1f67d158c7b4052123c5032b0ba54a910d
>  ?)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13997) Upgrade guava to 23.3


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-13997:

Status: Patch Available  (was: Open)

Tests look "good" (can't find anything related to the guava upgrade)

> Upgrade guava to 23.3
> -
>
> Key: CASSANDRA-13997
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13997
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 4.x
>
>
> For 4.0 we should upgrade guava to the latest version
> patch here: https://github.com/krummas/cassandra/commits/marcuse/guava23
> A bunch of quite commonly used methods have been deprecated since guava 18 
> which we use now ({{Throwables.propagate}} for example), this patch mostly 
> updates uses where compilation fails. {{Futures.transform(ListenableFuture 
> ..., AsyncFunction ...}} was deprecated in Guava 19 and removed in 20 for 
> example, we should probably open new tickets to remove calls to all 
> deprecated guava methods.
> Also had to add a dependency on {{com.google.j2objc.j2objc-annotations}}, to 
> avoid some build-time warnings (maybe due to 
> https://github.com/google/guava/commit/fffd2b1f67d158c7b4052123c5032b0ba54a910d
>  ?)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14002) Don't use SHA256 when building merkle trees


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-14002:

Status: Patch Available  (was: Open)

Patch for this 
[here|https://github.com/krummas/cassandra/commits/marcuse/nosharepairs] - it 
replaces the SHA-256 with Murmur3.

Murmur3 is only 128 bit though, so the patch instead hashes every value twice 
using 2 different murmur3 instances with different seeds to get the same number 
of bits as SHA-256. The approach used is similar to what guava does in its 
ConcatenatedHashFunction.

In my tests with semi-wide partitions (~100KB mean partition size) this reduces 
the time spent building merkle trees with at least 50%.

https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/422/
https://circleci.com/gh/krummas/cassandra/173

cc [~mkjellman]

> Don't use SHA256 when building merkle trees
> ---
>
> Key: CASSANDRA-14002
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14002
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 4.x
>
>
> We should avoid using SHA-2 when building merkle trees as we don't need a 
> cryptographic hash function for this.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14002) Don't use SHA256 when building merkle trees