[jira] [Commented] (CASSANDRA-3397) Problem markers don't show up in Eclipse

[David Allsopp (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-3397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175577#comment-13175577
 ] 

David Allsopp commented on CASSANDRA-3397:
--

I was just interested in the error markers, thanks - I too found the Ant 
Builder too heavy!

> Problem markers don't show up in Eclipse
> 
>
> Key: CASSANDRA-3397
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3397
> Project: Cassandra
>  Issue Type: Bug
>  Components: Packaging
>Affects Versions: 1.0.0
> Environment: Eclipse
>Reporter: David Allsopp
>Assignee: David Allsopp
>Priority: Minor
>  Labels: ant, eclipse, ide
> Fix For: 1.0.7
>
> Attachments: Cassandra-3397.patch
>
>
> The generated Eclipse files install an Ant Builder to build Cassandra within 
> Eclipse. This appears to mean that the default Java Builder is not present. 
> This means that no problem markers show up in the Problem view or the Package 
> Explorer etc when there are compiler errors or warnings  - you have to study 
> the console output, then navigate manually to the sources of the problems, 
> which is very tedious.
> It seems to be possible to re-install the default Java Builder in parallel 
> with the Ant Builder, getting the best of both worlds. I have documented this 
> on the wiki at http://wiki.apache.org/cassandra/RunningCassandraInEclipse
> I was wondering a) whether this can be done automatically by the 
> generate-eclipse-files Ant target, and b) whether using both Builders will be 
> problem if one is working on any of the generated code (Thrift, CQL etc). The 
> Java Builder can be temporarily disabled if so by unticking it under 
> Properties->Builders...
> See also https://issues.apache.org/jira/browse/CASSANDRA-2854

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3512) Getting Started instructions don't work in README.txt - wrong version of jamm, wrong path

[David Allsopp (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-3512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154278#comment-13154278
 ] 

David Allsopp commented on CASSANDRA-3512:
--

I think the problem is that bin/cassandra.bat is detecting a cassandra.in.sh 
from a previous install of Cassandra. Setting CASSANDRA_INCLUDE to point 
explicitly to bin/cassandra.in.sh fixes the problem.

> Getting Started instructions don't work in README.txt - wrong version of 
> jamm, wrong path
> -
>
> Key: CASSANDRA-3512
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3512
> Project: Cassandra
>  Issue Type: Bug
>  Components: Packaging
>Affects Versions: 1.0.3
> Environment: Ubuntu 11.04
>Reporter: David Allsopp
>Priority: Minor
>
> Download latest release from 
> http://www.apache.org/dyn/closer.cgi?path=/cassandra/1.0.3/apache-cassandra-1.0.3-bin.tar.gz
> Unpack the tarball.
> Follow instructions in README.txt, concluding with:
> {noformat}
> dna@master:~/code/apache-cassandra-1.0.3$ bin/cassandra -f
> Error opening zip file or JAR manifest missing : /lib/jamm-0.2.1.jar
> Error occurred during initialization of VM
> agent library failed to init: instrument
> {noformat}
> Firstly, the version of jamm packaged with Cassandra 1.0.3 is jamm-0.2.5, not 
> jamm-0.2.1. 
> Both bin/cassandra.bat and conf/cassandra-env.sh reference jamm-0.2.5 so not 
> sure where jamm-0.2.1 is being referenced from - nothing obvious using grep.
> Secondly, /lib/jamm-0.2.1.jar is the wrong path - should be set relative to 
> working directory, not filesystem root
> (Incidentally, Cassandra v1.0.3 is still listed as unreleased on JIRA.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2975) Upgrade MurmurHash to version 3

[David Allsopp (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13153858#comment-13153858
 ] 

David Allsopp commented on CASSANDRA-2975:
--

Have just repeated the benchmark with HeapByteBuffer as input rather than 
DirectByteBuffer (using ByteBuffer allocate() rather than allocateDirect()), 
and the performance improvement seems to almost vanish.

The input to MurmurHash within Cassandra seems to be a HeapByteBuffer (based on 
adding a println to the existing MurmurHash2 hash64() method), so the inlining 
is probably of no benefit in practice.

> Upgrade MurmurHash to version 3
> ---
>
> Key: CASSANDRA-2975
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2975
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Brian Lindauer
>Assignee: Brian Lindauer
>Priority: Trivial
>  Labels: lhf
> Fix For: 1.1
>
> Attachments: 
> 0001-Convert-BloomFilter-to-use-MurmurHash-v3-instead-of-.patch, 
> 0002-Backwards-compatibility-with-files-using-Murmur2-blo.patch, 
> Murmur3Benchmark.java
>
>
> MurmurHash version 3 was finalized on June 3. It provides an enormous speedup 
> and increased robustness over version 2, which is implemented in Cassandra. 
> Information here:
> http://code.google.com/p/smhasher/
> The reference implementation is here:
> http://code.google.com/p/smhasher/source/browse/trunk/MurmurHash3.cpp?spec=svn136&r=136
> I have already done the work to port the (public domain) reference 
> implementation to Java in the MurmurHash class and updated the BloomFilter 
> class to use the new implementation:
> https://github.com/lindauer/cassandra/commit/cea6068a4a3e5d7d9509335394f9ef3350d37e93
> Apart from the faster hash time, the new version only requires one call to 
> hash() rather than 2, since it returns 128 bits of hash instead of 64.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2975) Upgrade MurmurHash to version 3

[David Allsopp (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13153112#comment-13153112
 ] 

David Allsopp commented on CASSANDRA-2975:
--

The benchmark does several rounds of warmup for each iteration (i.e. for each 
buffer size from 1 to 32 bytes). 

It reduces the number of iterations as the input buffer size grows, so that 
each run processes a similar number of bytes - though this is probably 
irrelevant since the performance seems fairly constant with respect to buffer 
size.

{noformat}
Running test for buffer lengths from 1 to 32
 *|Ratio: 0.96 for keylength 1 iterations=1
 *|Ratio: 0.95 for keylength 2 iterations=5000
 *|Ratio: 0.95 for keylength 3 iterations=
 *|Ratio: 0.96 for keylength 4 iterations=2500
 *|Ratio: 0.94 for keylength 5 iterations=2000
 *|Ratio: 0.94 for keylength 6 iterations=1666
 *|Ratio: 0.96 for keylength 7 iterations=14285714
 *|Ratio: 0.93 for keylength 8 iterations=1250
* |Ratio: 0.89 for keylength 9 iterations=
 *|Ratio: 0.93 for keylength 10 iterations=1000
 *|Ratio: 0.95 for keylength 11 iterations=9090909
 *|Ratio: 0.95 for keylength 12 iterations=833
 *|Ratio: 0.93 for keylength 13 iterations=7692307
 *|Ratio: 0.90 for keylength 14 iterations=7142857
 *|Ratio: 0.95 for keylength 15 iterations=666
* |Ratio: 0.86 for keylength 16 iterations=625
* |Ratio: 0.87 for keylength 17 iterations=5882352
 *|Ratio: 0.91 for keylength 18 iterations=555
* |Ratio: 0.83 for keylength 19 iterations=5263157
* |Ratio: 0.83 for keylength 20 iterations=500
* |Ratio: 0.80 for keylength 21 iterations=4761904
* |Ratio: 0.88 for keylength 22 iterations=4545454
 *|Ratio: 0.91 for keylength 23 iterations=4347826
 *|Ratio: 0.91 for keylength 24 iterations=416
* |Ratio: 0.88 for keylength 25 iterations=400
 *|Ratio: 0.92 for keylength 26 iterations=3846153
* |Ratio: 0.85 for keylength 27 iterations=3703703
* |Ratio: 0.88 for keylength 28 iterations=3571428
* |Ratio: 0.88 for keylength 29 iterations=3448275
* |Ratio: 0.89 for keylength 30 iterations=333
 *|Ratio: 0.92 for keylength 31 iterations=3225806

Old (ms): 18938
New (ms): 17470
Overall ratio: 0.9224838948146583
{noformat}

i.e. 8% improvement on average.



> Upgrade MurmurHash to version 3
> ---
>
> Key: CASSANDRA-2975
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2975
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Brian Lindauer
>Assignee: Brian Lindauer
>Priority: Trivial
>  Labels: lhf
> Fix For: 1.1
>
> Attachments: 
> 0001-Convert-BloomFilter-to-use-MurmurHash-v3-instead-of-.patch, 
> 0002-Backwards-compatibility-with-files-using-Murmur2-blo.patch, 
> Murmur3Benchmark.java
>
>
> MurmurHash version 3 was finalized on June 3. It provides an enormous speedup 
> and increased robustness over version 2, which is implemented in Cassandra. 
> Information here:
> http://code.google.com/p/smhasher/
> The reference implementation is here:
> http://code.google.com/p/smhasher/source/browse/trunk/MurmurHash3.cpp?spec=svn136&r=136
> I have already done the work to port the (public domain) reference 
> implementation to Java in the MurmurHash class and updated the BloomFilter 
> class to use the new implementation:
> https://github.com/lindauer/cassandra/commit/cea6068a4a3e5d7d9509335394f9ef3350d37e93
> Apart from the faster hash time, the new version only requires one call to 
> hash() rather than 2, since it returns 128 bits of hash instead of 64.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2975) Upgrade MurmurHash to version 3

[David Allsopp (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13151121#comment-13151121
 ] 

David Allsopp commented on CASSANDRA-2975:
--

I really must learn not to post stuff late at night :-(.  The "optimisation" of 
the {{switch-case}} breaks the algorithm because it relies on the fall-through 
behaviour of {{switch-case}} in C and Java. Oh well. The inlining only speeds 
things up a few percent, but might be worthwhile if others see the same 
improvement on other hardware.

> Upgrade MurmurHash to version 3
> ---
>
> Key: CASSANDRA-2975
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2975
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Brian Lindauer
>Assignee: Brian Lindauer
>Priority: Trivial
>  Labels: lhf
> Fix For: 1.1
>
> Attachments: 
> 0001-Convert-BloomFilter-to-use-MurmurHash-v3-instead-of-.patch, 
> 0002-Backwards-compatibility-with-files-using-Murmur2-blo.patch
>
>
> MurmurHash version 3 was finalized on June 3. It provides an enormous speedup 
> and increased robustness over version 2, which is implemented in Cassandra. 
> Information here:
> http://code.google.com/p/smhasher/
> The reference implementation is here:
> http://code.google.com/p/smhasher/source/browse/trunk/MurmurHash3.cpp?spec=svn136&r=136
> I have already done the work to port the (public domain) reference 
> implementation to Java in the MurmurHash class and updated the BloomFilter 
> class to use the new implementation:
> https://github.com/lindauer/cassandra/commit/cea6068a4a3e5d7d9509335394f9ef3350d37e93
> Apart from the faster hash time, the new version only requires one call to 
> hash() rather than 2, since it returns 128 bits of hash instead of 64.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2975) Upgrade MurmurHash to version 3

[David Allsopp (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13151092#comment-13151092
 ] 

David Allsopp commented on CASSANDRA-2975:
--

Having slept on it, I realized that the {{switch-case}} is slow because of the 
{{switch-case}} fallthrough in Java - _i.e. all 15 cases get tested every time_ 
(http://download.oracle.com/javase/tutorial/java/nutsandbolts/switch.html)

Adding a {{break}} to the end of each case fixes this without the verbosity of 
the binary search approach, i.e:

{noformat}
switch (length & 15)
{
case 0: break;
case 1:
k1 ^= ((long) key.get(offset));
k1 *= c1;
k1 = rotl64(k1, 31);
k1 *= c2;
h1 ^= k1;
break;
case 2:
k1 ^= ((long) key.get(offset + 1)) << 8;
break;
...etc...
{noformat}

Combining this with the inlining almost doubles the speed of hashing for keys 
with lengths from 1-32 bytes!

> Upgrade MurmurHash to version 3
> ---
>
> Key: CASSANDRA-2975
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2975
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Brian Lindauer
>Assignee: Brian Lindauer
>Priority: Trivial
>  Labels: lhf
> Fix For: 1.1
>
> Attachments: 
> 0001-Convert-BloomFilter-to-use-MurmurHash-v3-instead-of-.patch, 
> 0002-Backwards-compatibility-with-files-using-Murmur2-blo.patch
>
>
> MurmurHash version 3 was finalized on June 3. It provides an enormous speedup 
> and increased robustness over version 2, which is implemented in Cassandra. 
> Information here:
> http://code.google.com/p/smhasher/
> The reference implementation is here:
> http://code.google.com/p/smhasher/source/browse/trunk/MurmurHash3.cpp?spec=svn136&r=136
> I have already done the work to port the (public domain) reference 
> implementation to Java in the MurmurHash class and updated the BloomFilter 
> class to use the new implementation:
> https://github.com/lindauer/cassandra/commit/cea6068a4a3e5d7d9509335394f9ef3350d37e93
> Apart from the faster hash time, the new version only requires one call to 
> hash() rather than 2, since it returns 128 bits of hash instead of 64.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2975) Upgrade MurmurHash to version 3

[David Allsopp (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150931#comment-13150931
 ] 

David Allsopp commented on CASSANDRA-2975:
--

Does anyone have any data on what the typical key size is (i.e. the average 
input size for the hash)?

I have a couple of optimisations for the MurmurHash3 implementation that I 
think give another 10-40% speedup, particularly for smaller values (e.g. 30% 
speedup for buffer lengths under 256 bytes) and no worse for large values (tens 
of KB). These results were on a AMD Phenom II X6 1055T @ 2.80 GHz, under 64-bit 
Windows 7, Java 1.6.0_27.

Firstly, inline the {rotl64}} calls , e.g. so that
{noformat}
k1 = rotl64(k1, 31);
{noformat}
becomes
{noformat}
k1 = (k1 << 31) | (k1 >>> 33);
{noformat}

Secondly, rather than a large {{switch-case}} to handle the 'tail', use nested 
{{if-else}} to form a simple binary search. Particularly for relatively small 
inputs, handling the 'tail' is a significant part of the computation. E.g:

{noformat}
int ln = length & 15;
if (ln > 8)
  {
 if (ln > 12)
   {
  // etc for cases 13 - 15
   }
 else
   {
  // cases 11 and 12
   }

  }
else
  {
 // etc for cases 1-7
  }
{noformat}

Will try to post a proper benchmark when I've tidied it up (run out of time 
today!) so anyone interested can try it on other hardware...

This latter optimisation is pretty verbose and ugly to look at - it _might_ be 
just as fast, and much more concise, to lookup the offsets and shifts from an 
array, and deal with the special cases 1 and 9 as, well, special cases - but 
haven't benchmarked this alternative yet...

> Upgrade MurmurHash to version 3
> ---
>
> Key: CASSANDRA-2975
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2975
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Brian Lindauer
>Assignee: Brian Lindauer
>Priority: Trivial
>  Labels: lhf
> Fix For: 1.1
>
> Attachments: 
> 0001-Convert-BloomFilter-to-use-MurmurHash-v3-instead-of-.patch, 
> 0002-Backwards-compatibility-with-files-using-Murmur2-blo.patch
>
>
> MurmurHash version 3 was finalized on June 3. It provides an enormous speedup 
> and increased robustness over version 2, which is implemented in Cassandra. 
> Information here:
> http://code.google.com/p/smhasher/
> The reference implementation is here:
> http://code.google.com/p/smhasher/source/browse/trunk/MurmurHash3.cpp?spec=svn136&r=136
> I have already done the work to port the (public domain) reference 
> implementation to Java in the MurmurHash class and updated the BloomFilter 
> class to use the new implementation:
> https://github.com/lindauer/cassandra/commit/cea6068a4a3e5d7d9509335394f9ef3350d37e93
> Apart from the faster hash time, the new version only requires one call to 
> hash() rather than 2, since it returns 128 bits of hash instead of 64.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3017) add a Message size limit

[David Allsopp (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-3017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150408#comment-13150408
 ] 

David Allsopp commented on CASSANDRA-3017:
--

In the latest trunk code, at least, isn't it {{bodySize}} not {{totalSize}} 
that we need to check, to avoid huge buffers? {{totalSize}} is only used to 
skip spare bytes at the end (could check both, of course).

While we're there, should we also check for negative values (does anything 
terribly bad happen if the value is negative?).

Could fix typo in the log message ("Ignorning") while you're there!

> add a Message size limit
> 
>
> Key: CASSANDRA-3017
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3017
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Jonathan Ellis
>Assignee: Ryan King
>Priority: Minor
>  Labels: lhf
> Attachments: 
> 0001-use-the-thrift-max-message-size-for-inter-node-messa.patch
>
>
> We protect the server from allocating huge buffers for malformed message with 
> the Thrift frame size (CASSANDRA-475).  But we don't have similar protection 
> for the inter-node Message objects.
> Adding this would be good to deal with malicious adversaries as well as a 
> malfunctioning cluster participant.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2975) Upgrade MurmurHash to version 3

[David Allsopp (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150393#comment-13150393
 ] 

David Allsopp commented on CASSANDRA-2975:
--

In the attached patch, in MurmurHash.hash3_x64_128(), in the main loop:

{noformat}
for(int i = 0; i < nblocks; i++)
{
int i_8 = i << 4;
...
{noformat}

{{i_8}} is assigned but not used, as far as I can see?

> Upgrade MurmurHash to version 3
> ---
>
> Key: CASSANDRA-2975
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2975
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Brian Lindauer
>Assignee: Brian Lindauer
>Priority: Trivial
>  Labels: lhf
> Fix For: 1.1
>
> Attachments: 
> 0001-Convert-BloomFilter-to-use-MurmurHash-v3-instead-of-.patch, 
> 0002-Backwards-compatibility-with-files-using-Murmur2-blo.patch
>
>
> MurmurHash version 3 was finalized on June 3. It provides an enormous speedup 
> and increased robustness over version 2, which is implemented in Cassandra. 
> Information here:
> http://code.google.com/p/smhasher/
> The reference implementation is here:
> http://code.google.com/p/smhasher/source/browse/trunk/MurmurHash3.cpp?spec=svn136&r=136
> I have already done the work to port the (public domain) reference 
> implementation to Java in the MurmurHash class and updated the BloomFilter 
> class to use the new implementation:
> https://github.com/lindauer/cassandra/commit/cea6068a4a3e5d7d9509335394f9ef3350d37e93
> Apart from the faster hash time, the new version only requires one call to 
> hash() rather than 2, since it returns 128 bits of hash instead of 64.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2274) Restrict Cassandra cluster node joins to a list of named hosts

[David Allsopp (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148539#comment-13148539
 ] 

David Allsopp commented on CASSANDRA-2274:
--

Just a thought - I don't know what the overhead is with encryption enabled, but 
if this is a major issue, then presumably encryption could be enabled with a 
cypher suite that doesn't actually _encrypt_, thus providing just data 
_integrity_ (MD5/SHA checksum) or even a 'null' encryption algorithm? I assume 
(!) that this would provide authentication via the certificates, without the 
encryption overhead.

Listing the supported suites using 
SSLContext.getServerSocketFactory().getSupportedCipherSuites() includes 
SSL_RSA_WITH_NULL_MD5 and SSL_RSA_WITH_NULL_SHA - which are suites used for 
integrity only (checksum). 

> Restrict Cassandra cluster node joins to a list of named hosts
> --
>
> Key: CASSANDRA-2274
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2274
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Affects Versions: 0.7.2
> Environment: All
>Reporter: Andrew Schiefelbein
>
> Because firewalls and employees are not infallible it would be nice to 
> restrict the ability of any node to join a cluster to a list of named hosts 
> in the configuration so that someone would be unable to start a node and 
> replicate all the data locally.  I understand that in order to do this the 
> person must know the seed servers and the cluster name and to extract the 
> data they will need a userid and password but another level of security would 
> be to force them to execute any brute force attack from a locked down server 
> instead of replicating all the data locally.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3462) Determine IP address of Message sender from the socket, rather than trusting the sender

[David Allsopp (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-3462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13146243#comment-13146243
 ] 

David Allsopp commented on CASSANDRA-3462:
--

Ah, yes :-).  Given that the parent issue has been closed there's probably no 
point pursuing this.

Validating forwarded messages end-to-end is possible using a 'chain of trust' 
where each forwarding node vouches for the originator, but it seems the 
inter-node encryption achieves the same goal and more!

> Determine IP address of Message sender from the socket, rather than trusting 
> the sender
> ---
>
> Key: CASSANDRA-3462
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3462
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Core
> Environment: All.
>Reporter: David Allsopp
>Assignee: David Allsopp
>  Labels: authentication, security
> Fix For: 1.0.3
>
> Attachments: Cassandra-3462-v2.patch, Cassandra-3462.patch
>
>
> A prerequisite for preventing malicious nodes from joining a cluster (parent 
> issue https://issues.apache.org/jira/browse/CASSANDRA-2274) is that we can 
> determine the IP of the sender (setting aside the fact that this may be 
> spoofed by a determined attacker). 
> Currently we deserialize the "from" IP address from the incoming message 
> header, using Header.deserialize() and 
> CompactEndpointSerializationHelper.deserialize() i.e. we trust the sender to 
> supply a true IP address.
> We could stop storing the IP address in the message Header at all (saving a 
> small amount of space) and set the 'true' sender IP upon receipt of the 
> message, in org.apache.cassandra.net.IncomingTcpConnection, using 
> socket.getInetAddress().

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2274) Restrict Cassandra cluster node joins to a list of named hosts

[David Allsopp (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13146228#comment-13146228
 ] 

David Allsopp commented on CASSANDRA-2274:
--

One thing that should at least be emphasized in documentation is that the 
security is a bit all-or-nothing.  With encryption enabled I expect the 
security to be solid (though I haven't investigated in detail yet - I'm unsure 
how easy this is to administer, or what the performance implications are).

However, on an unencrypted cluster users may currently have a false sense of 
security - Thrift authentication can be enabled, but since the underlying 
inter-node messages are unauthenticated, I think an attacker can do what they 
like if they have access to the network. Iptables is therefore very important 
if you don't have complete control over who can access your local network. Even 
then, Thrift authentication can still be largely bypassed by anyone who has 
access to any node not blocked by iptables (i.e. any operational cluster node).

> Restrict Cassandra cluster node joins to a list of named hosts
> --
>
> Key: CASSANDRA-2274
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2274
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Affects Versions: 0.7.2
> Environment: All
>Reporter: Andrew Schiefelbein
>
> Because firewalls and employees are not infallible it would be nice to 
> restrict the ability of any node to join a cluster to a list of named hosts 
> in the configuration so that someone would be unable to start a node and 
> replicate all the data locally.  I understand that in order to do this the 
> person must know the seed servers and the cluster name and to extract the 
> data they will need a userid and password but another level of security would 
> be to force them to execute any brute force attack from a locked down server 
> instead of replicating all the data locally.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2274) Restrict Cassandra cluster node joins to a list of named hosts

[David Allsopp (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145909#comment-13145909
 ] 

David Allsopp commented on CASSANDRA-2274:
--

D'oh, I hadn't realized encryption was implemented now! Spent too much time on 
old versions...

@Richard -  Dropping a keyspace is a Thrift operation, and I think you can 
already define different levels of access for 'normal' read/write access versus 
'dangerous' schema operations. 

The crux of the above issue, however, is that inter-node messaging is separate 
from the Thrift communications - so even with Thrift authentication enabled, a 
node can join the cluster and read data. Presumably the inter-node encryption 
fixes this.

Verifying IP addresses might still be useful as per the sub-issue, but it's a 
minor point if proper encryption is in place. 

> Restrict Cassandra cluster node joins to a list of named hosts
> --
>
> Key: CASSANDRA-2274
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2274
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Affects Versions: 0.7.2
> Environment: All
>Reporter: Andrew Schiefelbein
>
> Because firewalls and employees are not infallible it would be nice to 
> restrict the ability of any node to join a cluster to a list of named hosts 
> in the configuration so that someone would be unable to start a node and 
> replicate all the data locally.  I understand that in order to do this the 
> person must know the seed servers and the cluster name and to extract the 
> data they will need a userid and password but another level of security would 
> be to force them to execute any brute force attack from a locked down server 
> instead of replicating all the data locally.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3462) Determine IP address of Message sender from the socket, rather than trusting the sender

[David Allsopp (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-3462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145600#comment-13145600
 ] 

David Allsopp commented on CASSANDRA-3462:
--

The patch above logs a warning and 'repairs' the incorrect IP - but should we 
actually just drop any messages with incorrect IPs on the assumption that they 
come from a hostile (or badly broken) node?

> Determine IP address of Message sender from the socket, rather than trusting 
> the sender
> ---
>
> Key: CASSANDRA-3462
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3462
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Core
>Affects Versions: 1.0.1
> Environment: All.
>Reporter: David Allsopp
>  Labels: authentication, security
> Attachments: Cassandra-3462-v2.patch, Cassandra-3462.patch
>
>
> A prerequisite for preventing malicious nodes from joining a cluster (parent 
> issue https://issues.apache.org/jira/browse/CASSANDRA-2274) is that we can 
> determine the IP of the sender (setting aside the fact that this may be 
> spoofed by a determined attacker). 
> Currently we deserialize the "from" IP address from the incoming message 
> header, using Header.deserialize() and 
> CompactEndpointSerializationHelper.deserialize() i.e. we trust the sender to 
> supply a true IP address.
> We could stop storing the IP address in the message Header at all (saving a 
> small amount of space) and set the 'true' sender IP upon receipt of the 
> message, in org.apache.cassandra.net.IncomingTcpConnection, using 
> socket.getInetAddress().

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2274) Restrict Cassandra cluster node joins to a list of named hosts

[David Allsopp (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144993#comment-13144993
 ] 

David Allsopp commented on CASSANDRA-2274:
--

One additional issue is that currently we deserialize the "from" IP address 
from the incoming message header, using Header.deserialize() and 
CompactEndpointSerializationHelper.deserialize()  i.e. _we trust the sender to 
supply a true IP address_.

> Restrict Cassandra cluster node joins to a list of named hosts
> --
>
> Key: CASSANDRA-2274
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2274
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Affects Versions: 0.7.2
> Environment: All
>Reporter: Andrew Schiefelbein
>
> Because firewalls and employees are not infallible it would be nice to 
> restrict the ability of any node to join a cluster to a list of named hosts 
> in the configuration so that someone would be unable to start a node and 
> replicate all the data locally.  I understand that in order to do this the 
> person must know the seed servers and the cluster name and to extract the 
> data they will need a userid and password but another level of security would 
> be to force them to execute any brute force attack from a locked down server 
> instead of replicating all the data locally.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2274) Restrict Cassandra cluster node joins to a list of named hosts

[David Allsopp (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144834#comment-13144834
 ] 

David Allsopp commented on CASSANDRA-2274:
--

Using a custom PROTOCOL_MAGIC would be a very easy way to improve matters 
substantially. Would be better if it was at least a {{long}}, not an {{int}} 
though, otherwise you could brute force it, given a few weeks.

> Restrict Cassandra cluster node joins to a list of named hosts
> --
>
> Key: CASSANDRA-2274
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2274
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Affects Versions: 0.7.2
> Environment: All
>Reporter: Andrew Schiefelbein
>
> Because firewalls and employees are not infallible it would be nice to 
> restrict the ability of any node to join a cluster to a list of named hosts 
> in the configuration so that someone would be unable to start a node and 
> replicate all the data locally.  I understand that in order to do this the 
> person must know the seed servers and the cluster name and to extract the 
> data they will need a userid and password but another level of security would 
> be to force them to execute any brute force attack from a locked down server 
> instead of replicating all the data locally.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2274) Restrict Cassandra cluster node joins to a list of named hosts

[David Allsopp (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144707#comment-13144707
 ] 

David Allsopp commented on CASSANDRA-2274:
--

>From a quick scan through the code, it looks as though we'd need 
>MessagingService (and possibly JMX) to check the list of IPs or a supplied 
>token, in addition to the existing Thrift authentication.

Currently it appears that (in the absence of a firewall) there is no security 
against reading data from a cluster - the Thrift interface requires 
authentication, but the MessagingService doesn't, so I _think_ you can just 
send Read messages etc to pull data out of a cluster. Is that right, or is 
there something I've missed? 

I think we need to be precise about what threat we are trying to defend 
against, so we can weight up whether it's worth the effort. For example, does 
our hypothetical attacker have root access to a host on the LAN? If so, then 
presumably they may be able to spoof IP addresses, completely bypassing any IP 
checks? What happens if they simply set their IP to the same as that of one of 
the genuine nodes? I assume that you can defend against this with a suitably 
confgured router, but our scenario here seems to be that we don't have control 
over this sort of detail (which makes any kind of security very difficult).  If 
our attacker has the same privileges as us, then IP-based checks are probably 
OK.

Note also that none of the defences discussed so far would defend against 
denial-of-service attacks from an insider on the network (by trying to overload 
or crash nodes in the cluster). 

Physical attacks require physical defences - if untrusted people have access to 
your hardware then it's game over.

> Restrict Cassandra cluster node joins to a list of named hosts
> --
>
> Key: CASSANDRA-2274
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2274
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Affects Versions: 0.7.2
> Environment: All
>Reporter: Andrew Schiefelbein
>
> Because firewalls and employees are not infallible it would be nice to 
> restrict the ability of any node to join a cluster to a list of named hosts 
> in the configuration so that someone would be unable to start a node and 
> replicate all the data locally.  I understand that in order to do this the 
> person must know the seed servers and the cluster name and to extract the 
> data they will need a userid and password but another level of security would 
> be to force them to execute any brute force attack from a locked down server 
> instead of replicating all the data locally.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2274) Restrict Cassandra cluster node joins to a list of named hosts

[David Allsopp (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13142605#comment-13142605
 ] 

David Allsopp commented on CASSANDRA-2274:
--

Good point about storing the settings in the Cassandra storage itself, though 
it looks from https://issues.apache.org/jira/browse/CASSANDRA-3319 as though it 
needs to be a non-system keyspace (?). It feels wrong for it to be a 'normal' 
keyspace though, as it would need to have a predetermined name and schema in 
order for various parts of the server code to use if when authenticating, and 
it would have the same visibility and access as normal data keyspaces (surely 
it should require greater privileges, as for schema modifications and other 
'dangerous' operations?).

> Restrict Cassandra cluster node joins to a list of named hosts
> --
>
> Key: CASSANDRA-2274
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2274
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Affects Versions: 0.7.2
> Environment: All
>Reporter: Andrew Schiefelbein
>
> Because firewalls and employees are not infallible it would be nice to 
> restrict the ability of any node to join a cluster to a list of named hosts 
> in the configuration so that someone would be unable to start a node and 
> replicate all the data locally.  I understand that in order to do this the 
> person must know the seed servers and the cluster name and to extract the 
> data they will need a userid and password but another level of security would 
> be to force them to execute any brute force attack from a locked down server 
> instead of replicating all the data locally.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2274) Restrict Cassandra cluster node joins to a list of named hosts

[David Allsopp (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13141726#comment-13141726
 ] 

David Allsopp commented on CASSANDRA-2274:
--

@Mark - There's a convenience/security trade-off there. Giving each node a 
security token is easier and more flexible than configuring every node with a 
list of all other nodes (it's a bit of a pain to have to update all existing 
nodes each time you want to add a node). However, the security in this scenario 
is to deal with a semi-insider attack. Getting hold of the token might be 
easier for the attacker than having to somehow modify the access list on all 
existing nodes.

However, what stops an attacker from modifying the access list on all existing 
nodes? Only the login/password for those nodes. So in either case the weakest 
link is the protection of a single password/token (maybe multiple passwords if 
you have a different login for each node, but is that commonplace??).

Care would also be needed to make sure the shared secret isn't exposed over the 
network, either as plaintext, or via replay of messages from existing nodes.

So whilst I think your proposal is appealing from the operational efficiency 
point of view, it is probably harder to implement securely.

Trying to protect against an insider who might even have physical access to 
your servers is a very tough game...

> Restrict Cassandra cluster node joins to a list of named hosts
> --
>
> Key: CASSANDRA-2274
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2274
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Affects Versions: 0.7.2
> Environment: All
>Reporter: Andrew Schiefelbein
>
> Because firewalls and employees are not infallible it would be nice to 
> restrict the ability of any node to join a cluster to a list of named hosts 
> in the configuration so that someone would be unable to start a node and 
> replicate all the data locally.  I understand that in order to do this the 
> person must know the seed servers and the cluster name and to extract the 
> data they will need a userid and password but another level of security would 
> be to force them to execute any brute force attack from a locked down server 
> instead of replicating all the data locally.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2335) Windows: Test org.apache.cassandra.streaming.SerializationsTest FAILED

[David Allsopp (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135470#comment-13135470
 ] 

David Allsopp commented on CASSANDRA-2335:
--

Incidentally, the error: {code}artifact:pom doesn't support the "groupId" 
attribute{code}  can be fixed by downloading the latest _Maven Ant Tasks_ 
jarfile from http://maven.apache.org/ant-tasks/download.html - Having the 
latest Maven and the latest Ant isn't enough, in some cases.

> Windows: Test org.apache.cassandra.streaming.SerializationsTest FAILED
> --
>
> Key: CASSANDRA-2335
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2335
> Project: Cassandra
>  Issue Type: Bug
>Affects Versions: 0.7.0
> Environment: Windows
>Reporter: Benjamin Coverston
>Priority: Minor
> Attachments: 2335-windows-serialization-tests.patch, 2335.txt
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2611) static block in AbstractCassandraDaemon makes it difficult to change log4j behavoiur

[David Allsopp (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135461#comment-13135461
 ] 

David Allsopp commented on CASSANDRA-2611:
--

Seems to behave as expected on trunk, i.e. log output is the same before and 
after the change. I was initially concerned that the logger in CassandraDaemon 
is created statically (i.e. before initLog4j is called) but this doesn't seem 
to be an issue. None of the static methods in CassandraDaemon does any logging 
before the constructor is called, so the logger should be configured by the 
time anything calls it.

Note that if log4j.defaultInitOverride is undefined or false, initLog4j() 
doesn't actually do anything, so you could configure logging as you please or 
use the default log4j behaviour, in embedded usage.

> static block in AbstractCassandraDaemon makes it difficult to change log4j 
> behavoiur
> 
>
> Key: CASSANDRA-2611
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2611
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Affects Versions: 0.7.4, 0.7.5, 1.0.0
> Environment: Windows 7
>Reporter: Paul Loy
>Assignee: Tommy Tynjä
>Priority: Minor
>  Labels: daemon, initialisation, log4j
> Attachments: CASSANDRA-2611-test.patch, CASSANDRA-2611.patch
>
>
> We embed Cassandra in our application - mainly because our webservices are 
> such a thin layer on top of Cassandra that it really does not make sense for 
> us to have Cassandra in an external JVM. In 0.7.0 this was all fine. Now 
> upgrading to 0.7.5, there is a static block in AbstractCassandraDaemon. This 
> gets called when the class is loaded causing us issues as we have not 
> generated the log4j.properties file at this point in time.
> Can this not be a protected method that is called when 
> AbstractCassandraDaemon is constructed? That way a) I can control the 
> behaviour and b) my log4j.properties file will have been generated by then.
> Thanks.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2274) Restrict Cassandra cluster node joins to a list of named hosts

[David Allsopp (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13134465#comment-13134465
 ] 

David Allsopp commented on CASSANDRA-2274:
--

OK, I understand now!

Just a few things to clarify my understanding:

If I understand right, the Cassandra authentication is configured on a per-node 
basis (or so it appears from the docs I can find online), and is used to 
control Thrift/CQL client access, but not inter-node messages (hence it is 
possible to add a rogue node as you describe above).

Once this new node starts accepting data, its SSTables will presumably be 
accessible to the attacker, and the subset of data held on that node could then 
be extracted without any userid and password? And, given plenty of time, and 
assuming nobody is paying attention to nodetool etc, one could move this node 
around the ring to get a sampling of the data from around the keyspace.

Finally, if the rogue node is queried via Thrift/CQL, presumably it won't be 
configured to require any userid/password. When it passes on queries for rows 
held on other nodes, will those nodes require credentials as if the rogue node 
were a client contacting them directly? Or would these inter-node queries 
bypass the authentication? If the former, how does that work, since the 
authentication is configured (I think) separately on each client, so the 
credentials coudl be different? If the latter, then the rogue node bypasses all 
authentication.

> Restrict Cassandra cluster node joins to a list of named hosts
> --
>
> Key: CASSANDRA-2274
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2274
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Affects Versions: 0.7.2
> Environment: All
>Reporter: Andrew Schiefelbein
>
> Because firewalls and employees are not infallible it would be nice to 
> restrict the ability of any node to join a cluster to a list of named hosts 
> in the configuration so that someone would be unable to start a node and 
> replicate all the data locally.  I understand that in order to do this the 
> person must know the seed servers and the cluster name and to extract the 
> data they will need a userid and password but another level of security would 
> be to force them to execute any brute force attack from a locked down server 
> instead of replicating all the data locally.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3397) Problem markers don't show up in Eclipse

[David Allsopp (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-3397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13133637#comment-13133637
 ] 

David Allsopp commented on CASSANDRA-3397:
--

Looking at build.xml (generate-eclipse-files target), the .project file is 
generated from a verbatim listing, so adding a section for the Java Builder is 
trivial if that's the way we want to go...

> Problem markers don't show up in Eclipse
> 
>
> Key: CASSANDRA-3397
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3397
> Project: Cassandra
>  Issue Type: Bug
>  Components: Packaging
>Affects Versions: 1.0.0
> Environment: Eclipse
>Reporter: David Allsopp
>Priority: Minor
>  Labels: ant, eclipse, ide
>
> The generated Eclipse files install an Ant Builder to build Cassandra within 
> Eclipse. This appears to mean that the default Java Builder is not present. 
> This means that no problem markers show up in the Problem view or the Package 
> Explorer etc when there are compiler errors or warnings  - you have to study 
> the console output, then navigate manually to the sources of the problems, 
> which is very tedious.
> It seems to be possible to re-install the default Java Builder in parallel 
> with the Ant Builder, getting the best of both worlds. I have documented this 
> on the wiki at http://wiki.apache.org/cassandra/RunningCassandraInEclipse
> I was wondering a) whether this can be done automatically by the 
> generate-eclipse-files Ant target, and b) whether using both Builders will be 
> problem if one is working on any of the generated code (Thrift, CQL etc). The 
> Java Builder can be temporarily disabled if so by unticking it under 
> Properties->Builders...
> See also https://issues.apache.org/jira/browse/CASSANDRA-2854

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2611) static block in AbstractCassandraDaemon makes it difficult to change log4j behavoiur

[David Allsopp (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13133466#comment-13133466
 ] 

David Allsopp commented on CASSANDRA-2611:
--

Is this now resolved? Since 0.8.5 the relevant code in AbstractCassandraDaemon 
is in a static method (called by a static block in the 
org.apache.cassandra.thrift.CassandraDaemon subclass). Or does that just move 
the problem to the subclass if that is used in the embedded system?

> static block in AbstractCassandraDaemon makes it difficult to change log4j 
> behavoiur
> 
>
> Key: CASSANDRA-2611
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2611
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Affects Versions: 0.7.4, 0.7.5
> Environment: Windows 7
>Reporter: Paul Loy
>Priority: Minor
>  Labels: daemon, initialisation, log4j
>
> We embed Cassandra in our application - mainly because our webservices are 
> such a thin layer on top of Cassandra that it really does not make sense for 
> us to have Cassandra in an external JVM. In 0.7.0 this was all fine. Now 
> upgrading to 0.7.5, there is a static block in AbstractCassandraDaemon. This 
> gets called when the class is loaded causing us issues as we have not 
> generated the log4j.properties file at this point in time.
> Can this not be a protected method that is called when 
> AbstractCassandraDaemon is constructed? That way a) I can control the 
> behaviour and b) my log4j.properties file will have been generated by then.
> Thanks.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2854) Java Build Path is broken in Eclipse after running generate-eclipse-files Ant target

[David Allsopp (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13133366#comment-13133366
 ] 

David Allsopp commented on CASSANDRA-2854:
--

I think I have found a workable way to get Eclipse set up reliably, so will 
nuke most of the wiki page 
(http://wiki.apache.org/cassandra/RunningCassandraInEclipse) and add a new 
version. There may well be a neater way, as I'm not yet sure _why_ things break 
when following the current instructions. I suspect it may be due to Eclipse 
compiling things into /bin until it is (later) reconfigured to user 
/build/classes/main.


> Java Build Path is broken in Eclipse after running generate-eclipse-files Ant 
> target
> 
>
> Key: CASSANDRA-2854
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2854
> Project: Cassandra
>  Issue Type: Bug
>  Components: Packaging
>Affects Versions: 0.8.7, 1.0.0
> Environment: Windows 7 64-bit, Eclipse Helios SR1, Subclipse
>Reporter: David Allsopp
>Assignee: David Allsopp
>Priority: Minor
> Fix For: 1.0.1
>
> Attachments: cassandra-2854.patch
>
>
> Following the instructions in 
> http://wiki.apache.org/cassandra/RunningCassandraInEclipse, but checking out 
> v0.8.1:
> After running the 'generate-eclipse-files' Ant target, the build path is set 
> up to include all the libraries in build/lib/jars, but each library has ;C 
> (semicolon, letter C) appended to it, so Eclipse can't find the libraries - 
> there are about 55 errors like:
> Project 'cassandra-0.8.1' is missing required library: 
> '\Users\David\eclipse_workspace\cassandra-0.8.1\build\lib\jars\commons-cli-1.2.jar;C'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2274) Restrict Cassandra cluster node joins to a list of named hosts

[David Allsopp (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13133343#comment-13133343
 ] 

David Allsopp commented on CASSANDRA-2274:
--

Can this not be done simply via iptables firewall settings, rolled out across 
the cluster machines? 

(It's not clear to me why a Cassandra configuration file would be any less 
fallible than a firewall configuration file - and if you have both, then you 
have to remember to update both of them...)

> Restrict Cassandra cluster node joins to a list of named hosts
> --
>
> Key: CASSANDRA-2274
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2274
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Affects Versions: 0.7.2
> Environment: All
>Reporter: Andrew Schiefelbein
>
> Because firewalls and employees are not infallible it would be nice to 
> restrict the ability of any node to join a cluster to a list of named hosts 
> in the configuration so that someone would be unable to start a node and 
> replicate all the data locally.  I understand that in order to do this the 
> person must know the seed servers and the cluster name and to extract the 
> data they will need a userid and password but another level of security would 
> be to force them to execute any brute force attack from a locked down server 
> instead of replicating all the data locally.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2854) Java Build Path is broken in Eclipse after running generate-eclipse-files Ant target

[David Allsopp (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1315#comment-1315
 ] 

David Allsopp commented on CASSANDRA-2854:
--

After running generate-eclipse-files on either OS, some of the jars are 
duplicated, i.e. there are two copies on the build path, sometimes of different 
versions.

e.g commons-lang-2.1 and commons-lang-2.4
and servlet-api-2.5.20081211 (one copy from /lib and one from build/lib/jars)

> Java Build Path is broken in Eclipse after running generate-eclipse-files Ant 
> target
> 
>
> Key: CASSANDRA-2854
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2854
> Project: Cassandra
>  Issue Type: Bug
>  Components: Packaging
>Affects Versions: 0.8.1
> Environment: Windows 7 64-bit, Eclipse Helios SR1, Subclipse
>Reporter: David Allsopp
>Priority: Minor
>
> Following the instructions in 
> http://wiki.apache.org/cassandra/RunningCassandraInEclipse, but checking out 
> v0.8.1:
> After running the 'generate-eclipse-files' Ant target, the build path is set 
> up to include all the libraries in build/lib/jars, but each library has ;C 
> (semicolon, letter C) appended to it, so Eclipse can't find the libraries - 
> there are about 55 errors like:
> Project 'cassandra-0.8.1' is missing required library: 
> '\Users\David\eclipse_workspace\cassandra-0.8.1\build\lib\jars\commons-cli-1.2.jar;C'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2854) Java Build Path is broken in Eclipse after running generate-eclipse-files Ant target

[David Allsopp (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1312#comment-1312
 ] 

David Allsopp commented on CASSANDRA-2854:
--

Wasn't having much success getting Eclipse happy on Windows, so tried on Linux 
but similar problems: after checking out trunk, running build.xml (runs OK) and 
then the generate-eclipse-files target (runs OK), I refresh the project and get:

{noformat}
Errors occurred during the build.
Errors running builder 'Integrated External Tool Builder' on project 
'Cassandra-trunk'.
Variable references non-existent resource : 
${workspace_loc:/cassandra-trunk/build.xml}
Variable references non-existent resource : 
${workspace_loc:/cassandra-trunk/build.xml}
{noformat}

This is easy to fix - the project name I chose in Eclipse (Cassandra-trunk) is 
capitalized but the directory (cassandra-trunk) isn't. Go to 
Project->Properties->Builders, select Cassandra-Ant-Builder, Edit, and fix the 
Build File and Base Directory to match the name in Eclipse. This is probably 
caused by me checking out the code using SVN in the console rather than using 
Subversive within Eclipse...

*However, Eclipse report 2153 errors in the project at this point.*

If someone has this working nicely, could they please update the wiki page 
(http://wiki.apache.org/cassandra/RunningCassandraInEclipse), as it's got very 
outdated and confusing. Alternatively, if most Cassandra folks are using an 
environment other than Eclipse that works better, could you describe it on the 
wiki?

> Java Build Path is broken in Eclipse after running generate-eclipse-files Ant 
> target
> 
>
> Key: CASSANDRA-2854
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2854
> Project: Cassandra
>  Issue Type: Bug
>  Components: Packaging
>Affects Versions: 0.8.1
> Environment: Windows 7 64-bit, Eclipse Helios SR1, Subclipse
>Reporter: David Allsopp
>Priority: Minor
>
> Following the instructions in 
> http://wiki.apache.org/cassandra/RunningCassandraInEclipse, but checking out 
> v0.8.1:
> After running the 'generate-eclipse-files' Ant target, the build path is set 
> up to include all the libraries in build/lib/jars, but each library has ;C 
> (semicolon, letter C) appended to it, so Eclipse can't find the libraries - 
> there are about 55 errors like:
> Project 'cassandra-0.8.1' is missing required library: 
> '\Users\David\eclipse_workspace\cassandra-0.8.1\build\lib\jars\commons-cli-1.2.jar;C'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2854) Java Build Path is broken in Eclipse after running generate-eclipse-files Ant target

[David Allsopp (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127366#comment-13127366
 ] 

David Allsopp commented on CASSANDRA-2854:
--

Will try to take a look. Probably needs to detect the OS in Ant to modify the 
regex passed to split(), e.g.:


   


   


No path.sep set for this OS!

Or (possibly) use unix-style paths in the first place if Ant handles them OK on 
Windows (but not sure where they originate from; this may not be possible)...

> Java Build Path is broken in Eclipse after running generate-eclipse-files Ant 
> target
> 
>
> Key: CASSANDRA-2854
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2854
> Project: Cassandra
>  Issue Type: Bug
>  Components: Packaging
>Affects Versions: 0.8.1
> Environment: Windows 7 64-bit, Eclipse Helios SR1, Subclipse
>Reporter: David Allsopp
>Priority: Minor
>
> Following the instructions in 
> http://wiki.apache.org/cassandra/RunningCassandraInEclipse, but checking out 
> v0.8.1:
> After running the 'generate-eclipse-files' Ant target, the build path is set 
> up to include all the libraries in build/lib/jars, but each library has ;C 
> (semicolon, letter C) appended to it, so Eclipse can't find the libraries - 
> there are about 55 errors like:
> Project 'cassandra-0.8.1' is missing required library: 
> '\Users\David\eclipse_workspace\cassandra-0.8.1\build\lib\jars\commons-cli-1.2.jar;C'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira