Is there any limit in the number of partitions that a table can have

2018-03-07 Thread Javier Pareja
Hello all, I have been trying to find an answer to the following but I have had no luck so far: Is there any limit to the number of partitions that a table can have? Let's say a table has a partition key an no clustering key, is there a recommended limit on the number of values that this partition

Re: Is there any limit in the number of partitions that a table can have

2018-03-07 Thread Rahul Singh
The range is 2*2^63 -- Rahul Singh rahul.si...@anant.us Anant Corporation On Mar 7, 2018, 6:06 AM -0500, Javier Pareja , wrote: > Hello all, > > I have been trying to find an answer to the following but I have had no luck > so far: > Is there any limit to the number of partitions that a table c

Re: Is there any limit in the number of partitions that a table can have

2018-03-07 Thread Javier Pareja
Thank you Rahul, but is it a good practice to use a large range here? Or would it be better to create partitions with more than 1 row (by using a clustering key)? >From a data query point of view I will be accessing the rows by a UID one at a time. F Javier Pareja On Wed, Mar 7, 2018 at 11:12 AM,

Re: Is there any limit in the number of partitions that a table can have

2018-03-07 Thread Jeff Jirsa
There is no limit The token range of murmur3 is 2^64, but Cassandra properly handles token overlaps (we use a key that’s effectively a tuple of the token/hash and the underlying key itself), so having more than 2^64 partitions won’t hurt anything in theory That said, having that many partition

Re: Is there any limit in the number of partitions that a table can have

2018-03-07 Thread Carlos Rolo
Hi Jeff, Could you expand: "Tables without clustering keys are often deceptively expensive to compact, as a lot of work (relative to the other cell boundaries) happens on partition boundaries." This is something I didn't know and highly interesting to know more about! -- Carlos Rolo On Wed, Mar

Re: Is there any limit in the number of partitions that a table can have

2018-03-07 Thread Javier Pareja
Thank you Jeff, So, if I understood your email correctly, there is no restriction but I should be using clustering for performance reasons. I am expecting to store 10B rows per year in this table and each row will have a user defined type with an approx size of 1500 bytes. The access to the data i

Re: Is there any limit in the number of partitions that a table can have

2018-03-07 Thread Jeff Jirsa
On Wed, Mar 7, 2018 at 7:13 AM, Carlos Rolo wrote: > Hi Jeff, > > Could you expand: "Tables without clustering keys are often deceptively > expensive to compact, as a lot of work (relative to the other cell > boundaries) happens on partition boundaries." This is something I didn't > know and high

Re: Is there any limit in the number of partitions that a table can have

2018-03-07 Thread Javier Pareja
Thank you for your time Jeff, very helpful.I couldn't find anything out there about the subject and I suspected that this could be the case. Regarding the clustering key in this case: Back in the RDBMS world, you will always assign a sequential (or as sequential as possible) clustering key to a t

Re: Is there any limit in the number of partitions that a table can have

2018-03-07 Thread Carlos Rolo
Great explanation, thanks Jeff! On 7 Mar 2018 17:49, "Javier Pareja" wrote: > Thank you for your time Jeff, very helpful.I couldn't find anything out > there about the subject and I suspected that this could be the case. > > Regarding the clustering key in this case: > Back in the RDBMS world,

Re: [External] Is there any limit in the number of partitions that a table can have

2018-03-07 Thread Tom van der Woerdt
Hi Javier, When our users ask this question, I tend to answer "keep it above a billion". More partitions is better. I'm not aware of any actual limits on partition count. Practically it's almost always limited by the disk space in a server. Tom van der Woerdt Site Reliability Engineer Booking.c