Re: Question: Large partition warning

[Jeff Jirsa]

On 2017-06-16 10:18 (-0700), "Jeff Jirsa" wrote: 
> 
> 
> On 2017-06-15 16:33 (-0700), kurt greaves  wrote: 
> > fyi ticket already existed for this, I've submitted a patch that fixes this
> > specific issue but it looks like there are a few other properties that will
> > suffer from the same. As I said on the ticket, we should probably fix these
> > up even though setting things this high is generally bad practice. If other
> > people want to weigh in and think the same I'll create another ticket.
> > https://issues.apache.org/jira/browse/CASSANDRA-13172
> > 
> 
> If you kick off tests, I'll review+commit for you (though quick glance, you 
> probably also want to update Config.java to make it a long).

Ignore the comment about Config.java, it's wrong, and your patch looks fine, 
just run tests.

- Jeff

-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

Re: Question: Large partition warning

[Jeff Jirsa]

On 2017-06-15 16:33 (-0700), kurt greaves  wrote: 
> fyi ticket already existed for this, I've submitted a patch that fixes this
> specific issue but it looks like there are a few other properties that will
> suffer from the same. As I said on the ticket, we should probably fix these
> up even though setting things this high is generally bad practice. If other
> people want to weigh in and think the same I'll create another ticket.
> https://issues.apache.org/jira/browse/CASSANDRA-13172
> 

If you kick off tests, I'll review+commit for you (though quick glance, you 
probably also want to update Config.java to make it a long).

- Jeff

-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

Re: Question: Large partition warning

[kurt greaves]

fyi ticket already existed for this, I've submitted a patch that fixes this
specific issue but it looks like there are a few other properties that will
suffer from the same. As I said on the ticket, we should probably fix these
up even though setting things this high is generally bad practice. If other
people want to weigh in and think the same I'll create another ticket.
https://issues.apache.org/jira/browse/CASSANDRA-13172

Re: Question: Large partition warning

[Fay Hou [Data Pipeline & Real-time Analytics] ­]

 you really should keep partition size under 100M

On Wed, Jun 14, 2017 at 7:28 PM, Thakrar, Jayesh <
jthak...@conversantmedia.com> wrote:

> Thank you Kurt - that makes sense.
> Will certainly reduce it to 1024.
>
> Greatly appreciate your quick reply.
>
> Thanks,
>
> Jayesh
>
>
>
> From: kurt greaves
> Sent: Wednesday, June 14, 5:53 PM
> Subject: Re: Question: Large partition warning
> To: Fay Hou [Data Pipeline & Real-time Analytics] ­
> Cc: Thakrar, Jayesh, User
>
>
> Looks like you've hit a bug (not the first time I've seen this in relation
> to C* configs). compaction_large_partition_warning_threshold_mb resolves
> to an int, and in the codebase is represented in bytes. 4096 * 1024 * 1024
> and you've got some serious overflow. Granted, you should have this warning
> set considerably lower than 4096MB anyway. Try set it to 1024MB. Looks like
> this affects multiple versions so I'll create a JIRA/patch soon.
>
>
> Relevant code:
>
>  public static int getCompactionLargePartitionWarningThreshold() { return
> conf.compaction_large_partition_warning_threshold_mb * 1024 * 1024; }
>
>
>
>

Re: Question: Large partition warning

[Thakrar, Jayesh]

Thank you Kurt - that makes sense.
Will certainly reduce it to 1024.

Greatly appreciate your quick reply.

Thanks,

Jayesh



From: kurt greaves
Sent: Wednesday, June 14, 5:53 PM
Subject: Re: Question: Large partition warning
To: Fay Hou [Data Pipeline & Real-time Analytics] ­
Cc: Thakrar, Jayesh, User


Looks like you've hit a bug (not the first time I've seen this in relation to 
C* configs). compaction_large_partition_warning_threshold_mb resolves to an 
int, and in the codebase is represented in bytes. 4096 * 1024 * 1024 and you've 
got some serious overflow. Granted, you should have this warning set 
considerably lower than 4096MB anyway. Try set it to 1024MB. Looks like this 
affects multiple versions so I'll create a JIRA/patch soon.


Relevant code:

 public static int getCompactionLargePartitionWarningThreshold() { return 
conf.compaction_large_partition_warning_threshold_mb * 1024 * 1024; }

RE: Question: Large partition warning

[ZAIDI, ASAD A]

Check partition size of your table’s partitions (  nodetool tablehistograms 
tsg_ae logs_by_user).
You may need to reduce partition size of your table using guidelines given in [ 
http://docs.datastax.com/en/archived/cassandra/2.2/cassandra/planning/planPlanningPartitionSize.html]
 & 
[https://stackoverflow.com/questions/20512710/cassandra-has-a-limit-of-2-billion-cells-per-partition-but-whats-a-partition]




From: Thakrar, Jayesh [mailto:jthak...@conversantmedia.com]
Sent: Wednesday, June 14, 2017 3:14 PM
To: User 
Subject: Question: Large partition warning

We are on Cassandra 2.2.5 and I am constantly seeing warning messages about 
large partitions in system.log even though our setting for partition warning 
threshold is set to 4096 (MB).

WARN  [CompactionExecutor:43180] 2017-06-14 20:02:13,189 
BigTableWriter.java:184 - Writing large partition 
tsg_ae/logs_by_user:114303419784957147 (17187 bytes)
WARN  [CompactionExecutor:43180] 2017-06-14 20:02:13,190 
BigTableWriter.java:184 - Writing large partition 
tsg_ae/logs_by_user:820590487870502244 (2613 bytes)
WARN  [CompactionExecutor:43180] 2017-06-14 20:02:13,191 
BigTableWriter.java:184 - Writing large partition 
tsg_ae/logs_by_user:421586605444858333 (54923 bytes)

Here's what we have in cassandra.yaml.

# Log a warning when compacting partitions larger than this value
compaction_large_partition_warning_threshold_mb: 4096

So wondering if such a warning is normal or is there something wrong in our 
configuration or is it a bug?

I have looked at 
https://issues.apache.org/jira/browse/CASSANDRA-9643.

Any pointers will be greatly appreciated.

Thanks,
Jayesh

Re: Question: Large partition warning

[kurt greaves]

Looks like you've hit a bug (not the first time I've seen this in relation
to C* configs). compaction_large_partition_warning_threshold_mb resolves to
an int, and in the codebase is represented in bytes. 4096 * 1024 * 1024 and
you've got some serious overflow. Granted, you should have this warning set
considerably lower than 4096MB anyway. Try set it to 1024MB. Looks like
this affects multiple versions so I'll create a JIRA/patch soon.

Relevant code:
 public static int getCompactionLargePartitionWarningThreshold() { return
conf.compaction_large_partition_warning_threshold_mb * 1024 * 1024; }

Re: Question: Large partition warning

[Fay Hou [Data Pipeline & Real-time Analytics] ­]

 nodetool tablehistograms  tsg_ae/logs_by_user will give you an idea about
the estimate of the partition size. It is recommended that the partition
size is not over 100MB

Large partitions are also creating heap pressure during compactions, which
will issue warnings in the logs (look for "large partition").

Thanks,

Fay

On Wed, Jun 14, 2017 at 1:13 PM, Thakrar, Jayesh <
jthak...@conversantmedia.com> wrote:

> We are on Cassandra 2.2.5 and I am constantly seeing warning messages
> about large partitions in system.log even though our setting for partition
> warning threshold is set to 4096 (MB).
>
>
>
> WARN  [CompactionExecutor:43180] 2017-06-14 20:02:13,189
> BigTableWriter.java:184 - Writing large partition 
> tsg_ae/logs_by_user:114303419784957147
> (17187 bytes)
>
> WARN  [CompactionExecutor:43180] 2017-06-14 20:02:13,190
> BigTableWriter.java:184 - Writing large partition 
> tsg_ae/logs_by_user:820590487870502244
> (2613 bytes)
>
> WARN  [CompactionExecutor:43180] 2017-06-14 20:02:13,191
> BigTableWriter.java:184 - Writing large partition 
> tsg_ae/logs_by_user:421586605444858333
> (54923 bytes)
>
>
>
> Here's what we have in cassandra.yaml.
>
>
>
> # Log a warning when compacting partitions larger than this value
>
> compaction_large_partition_warning_threshold_mb: 4096
>
>
>
> So wondering if such a warning is normal or is there something wrong in
> our configuration or is it a bug?
>
>
>
> I have looked at https://issues.apache.org/jira/browse/CASSANDRA-9643.
>
>
>
> Any pointers will be greatly appreciated.
>
>
>
> Thanks,
>
> Jayesh
>
>
>