[
https://issues.apache.org/jira/browse/CASSANDRA-15608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17048173#comment-17048173
]
rsasupport commented on CASSANDRA-15608:
----------------------------------------
Any recommended changes to be done??
Please find the JVM.options below:
###########################################################################
# jvm.options #
# #
# - all flags defined here will be used by cassandra to startup the JVM #
# - one flag should be specified per line #
# - lines that do not start with '-' will be ignored #
# - only static flags are accepted (no variables or parameters) #
# - dynamic flags will be appended to these on cassandra-env #
###########################################################################
######################
# STARTUP PARAMETERS #
######################
# Uncomment any of the following properties to enable specific startup
parameters
# In a multi-instance deployment, multiple Cassandra instances will
independently assume that all
# CPU processors are available to it. This setting allows you to specify a
smaller set of processors
# and perhaps have affinity.
#-Dcassandra.available_processors=number_of_processors
# The directory location of the cassandra.yaml file.
#-Dcassandra.config=directory
# Sets the initial partitioner token for a node the first time the node is
started.
#-Dcassandra.initial_token=token
# Set to false to start Cassandra on a node but not have the node join the
cluster.
#-Dcassandra.join_ring=true|false
# Set to false to clear all gossip state for the node on restart. Use when you
have changed node
# information in cassandra.yaml (such as listen_address).
#-Dcassandra.load_ring_state=true|false
# Enable pluggable metrics reporter. See Pluggable metrics reporting in
Cassandra 2.0.2.
#-Dcassandra.metricsReporterConfigFile=file
# Set the port on which the CQL native transport listens for clients. (Default:
9042)
#-Dcassandra.native_transport_port=port
# Overrides the partitioner. (Default:
org.apache.cassandra.dht.Murmur3Partitioner)
#-Dcassandra.partitioner=partitioner
# To replace a node that has died, restart a new node in its place specifying
the address of the
# dead node. The new node must not have any data in its data directory, that
is, it must be in the
# same state as before bootstrapping.
#-Dcassandra.replace_address=listen_address or broadcast_address of dead node
# Allow restoring specific tables from an archived commit log.
#-Dcassandra.replayList=table
# Allows overriding of the default RING_DELAY (30000ms), which is the amount of
time a node waits
# before joining the ring.
#-Dcassandra.ring_delay_ms=ms
# Set the port for the Thrift RPC service, which is used for client
connections. (Default: 9160)
#-Dcassandra.rpc_port=port
# Set the SSL port for encrypted communication. (Default: 7001)
#-Dcassandra.ssl_storage_port=port
# Enable or disable the native transport server. See start_native_transport in
cassandra.yaml.
# cassandra.start_native_transport=true|false
# Enable or disable the Thrift RPC server. (Default: true)
#-Dcassandra.start_rpc=true/false
# Set the port for inter-node communication. (Default: 7000)
#-Dcassandra.storage_port=port
# Set the default location for the trigger JARs. (Default: conf/triggers)
#-Dcassandra.triggers_dir=directory
# For testing new compaction and compression strategies. It allows you to
experiment with different
# strategies and benchmark write performance differences without affecting the
production workload.
#-Dcassandra.write_survey=true
# To disable configuration via JMX of auth caches (such as those for
credentials, permissions and
# roles). This will mean those config options can only be set (persistently) in
cassandra.yaml
# and will require a restart for new values to take effect.
#-Dcassandra.disable_auth_caches_remote_configuration=true
# To disable dynamic calculation of the page size used when indexing an entire
partition (during
# initial index build/rebuild). If set to true, the page size will be fixed to
the default of
# 10000 rows per page.
#-Dcassandra.force_default_indexing_page_size=true
########################
# GENERAL JVM SETTINGS #
########################
# enable assertions. highly suggested for correct application functionality.
-ea
# enable thread priorities, primarily so we can give periodic tasks
# a lower priority to avoid interfering with client workload
-XX:+UseThreadPriorities
# allows lowering thread priority without being root on linux - probably
# not necessary on Windows but doesn't harm anything.
# see http://tech.stolsvik.com/2010/01/linux-java-thread-priorities-workar
-XX:ThreadPriorityPolicy=42
# Enable heap-dump if there's an OOM
-XX:+HeapDumpOnOutOfMemoryError
# Per-thread stack size.
-Xss256k
# Larger interned string table, for gossip's benefit (CASSANDRA-6410)
-XX:StringTableSize=1000003
# Make sure all memory is faulted and zeroed on startup.
# This helps prevent soft faults in containers and makes
# transparent hugepage allocation more effective.
-XX:+AlwaysPreTouch
# Disable biased locking as it does not benefit Cassandra.
-XX:-UseBiasedLocking
# Enable thread-local allocation blocks and allow the JVM to automatically
# resize them at runtime.
-XX:+UseTLAB
-XX:+ResizeTLAB
-XX:+UseNUMA
# http://www.evanjones.ca/jvm-mmap-pause.html
-XX:+PerfDisableSharedMem
# Prefer binding to IPv4 network intefaces (when net.ipv6.bindv6only=1). See
# http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6342561 (short version:
# comment out this entry to enable IPv6 support).
-Djava.net.preferIPv4Stack=true
### Debug options
# uncomment to enable flight recorder
#-XX:+UnlockCommercialFeatures
#-XX:+FlightRecorder
# uncomment to have Cassandra JVM listen for remote debuggers/profilers on port
1414
#-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=1414
# uncomment to have Cassandra JVM log internal method compilation (developers
only)
#-XX:+UnlockDiagnosticVMOptions
#-XX:+LogCompilation
#################
# HEAP SETTINGS #
#################
# Heap size is automatically calculated by cassandra-env based on this
# formula: max(min(1/2 ram, 1024MB), min(1/4 ram, 8GB))
# That is:
# - calculate 1/2 ram and cap to 1024MB
# - calculate 1/4 ram and cap to 8192MB
# - pick the max
#
# For production use you may wish to adjust this for your environment.
# If that's the case, uncomment the -Xmx and Xms options below to override the
# automatic calculation of JVM heap memory.
#
# It is recommended to set min (-Xms) and max (-Xmx) heap sizes to
# the same value to avoid stop-the-world GC pauses during resize, and
# so that we can lock the heap in memory on startup to prevent any
# of it from being swapped out.
-Xms32G
-Xmx32G
# Young generation size is automatically calculated by cassandra-env
# based on this formula: min(100 * num_cores, 1/4 * heap size)
#
# The main trade-off for the young generation is that the larger it
# is, the longer GC pause times will be. The shorter it is, the more
# expensive GC will be (usually).
#
# It is not recommended to set the young generation size if using the
# G1 GC, since that will override the target pause-time goal.
# More info: http://www.oracle.com/technetwork/articles/java/g1gc-1984535.html
#
# The example below assumes a modern 8-core+ machine for decent
# times. If in doubt, and if you do not particularly want to tweak, go
# 100 MB per physical CPU core.
#-Xmn800M
###################################
# EXPIRATION DATE OVERFLOW POLICY #
###################################
# Defines how to handle INSERT requests with TTL exceeding the maximum
supported expiration date:
# * REJECT: this is the default policy and will reject any requests with
expiration date timestamp after 2038-01-19T03:14:06+00:00.
# * CAP: any insert with TTL expiring after 2038-01-19T03:14:06+00:00 will
expire on 2038-01-19T03:14:06+00:00 and the client will receive a warning.
# * CAP_NOWARN: same as previous, except that the client warning will not be
emitted.
#
#-Dcassandra.expiration_date_overflow_policy=REJECT
#################
# GC SETTINGS #
#################
### CMS Settings
#-XX:+UseParNewGC
#-XX:+UseConcMarkSweepGC
#-XX:+CMSParallelRemarkEnabled
#-XX:SurvivorRatio=4
#-XX:MaxTenuringThreshold=1
#-XX:CMSInitiatingOccupancyFraction=30
#-XX:+UseCMSInitiatingOccupancyOnly
#-XX:CMSWaitDuration=100000
#-XX:+CMSParallelInitialMarkEnabled
#-XX:+CMSEdenChunksRecordAlways
# some JVMs will fill up their heap when accessed via JMX, see CASSANDRA-6541
#-XX:+CMSClassUnloadingEnabled
### G1 Settings (experimental, comment previous section and uncomment section
below to enable)
## Use the Hotspot garbage-first collector.
-XX:+UseG1GC
#
# Have the JVM do less remembered set work during STW, instead
## preferring concurrent GC. Reduces p99.9 latency.
#-XX:G1RSetUpdatingPauseTimePercent=5
#
## Main G1GC tunable: lowering the pause target will lower throughput and vise
versa.
## 200ms is the JVM default and lowest viable setting
## 1000ms increases throughput. Keep it smaller than the timeouts in
cassandra.yaml.
-XX:MaxGCPauseMillis=200
## Optional G1 Settings
# Save CPU time on large (>= 16GB) heaps by delaying region scanning
# until the heap is 70% full. The default in Hotspot 8u40 is 40%.
#-XX:InitiatingHeapOccupancyPercent=70
# For systems with > 8 cores, the default ParallelGCThreads is 5/8 the number
of logical cores.
# Otherwise equal to the number of cores when 8 or less.
# Machines with > 10 cores should try setting these to <= full cores.
-XX:ParallelGCThreads=16
# By default, ConcGCThreads is 1/4 of ParallelGCThreads.
# Setting both to the same value can reduce STW durations.
#-XX:ConcGCThreads=10
### GC logging options -- uncomment to enable
-XX:+PrintGCDetails
-XX:+PrintGCDateStamps
-XX:+PrintHeapAtGC
-XX:+PrintTenuringDistribution
-XX:+PrintGCApplicationStoppedTime
-XX:+PrintPromotionFailure
-XX:PrintFLSStatistics=1
#-Xloggc:/var/log/cassandra/gc.log
-XX:+UseGCLogFileRotation
-XX:NumberOfGCLogFiles=10
-XX:GCLogFileSize=10M
> full repair getting failed for 1 large table version 3.11.2
> -----------------------------------------------------------
>
> Key: CASSANDRA-15608
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15608
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Repair, Consistency/Streaming
> Reporter: rsasupport
> Priority: Urgent
> Attachments: system.log
>
>
> nodetool repair -pr test table1
> [2020-02-28 17:56:00,192] Starting repair command #1
> (7a634f80-5a25-11ea-8b5a-3b5edc400697), repairing keyspace test with repair
> options (parallelism: parallel, primary range: true, incremental: true, job
> threads: 1, ColumnFamilies: [table1], dataCenters: [], hosts: [], # of
> ranges: 256, pull repair: false)
> [2020-02-28 17:56:17,173] Repair session 7ae020a1-5a25-11ea-8b5a-3b5edc400697
> for range [(-1398494450650640175,-1398484603620354999]] finished (progress:
> 1%)
> [2020-02-28 17:56:22,633] Repair session 7ad3c491-5a25-11ea-8b5a-3b5edc400697
> for range [(-2055080187802596770,-2055069591393124910]] finished (progress:
> 2%)
> [2020-02-28 17:56:23,888] Repair session 7ace4651-5a25-11ea-8b5a-3b5edc400697
> for range [(-5916076129714268351,-5916054112636649354]] finished (progress:
> 2%)
> [2020-02-28 17:56:27,675] Repair session 7ad83161-5a25-11ea-8b5a-3b5edc400697
> for range [(3262419392837600448,3262453314317655259]] finished (progress: 3%)
> [2020-02-28 17:56:31,674] Repair session 7ad32851-5a25-11ea-8b5a-3b5edc400697
> for range [(2146167582342729236,2146207858477993492]] finished (progress: 3%)
> [2020-02-28 17:56:52,964] Repair session 7af15eb0-5a25-11ea-8b5a-3b5edc400697
> for range [(-7088663015651548617,-7088601571353808807]] finished (progress:
> 3%)
> [2020-02-28 17:57:03,237] Repair session 7acf30b2-5a25-11ea-8b5a-3b5edc400697
> for range [(-5392084277480362928,-5391992322619247178]] finished (progress:
> 4%)
> [2020-02-28 17:57:08,937] Repair session 7ada5443-5a25-11ea-8b5a-3b5edc400697
> for range [(-2247846360993532740,-2247745485112718002]] finished (progress:
> 4%)
> [2020-02-28 17:57:14,102] Repair session 7ae1a740-5a25-11ea-8b5a-3b5edc400697
> for range [(4804514170425290662,4804611019437533765]] finished (progress: 5%)
> [2020-02-28 17:57:18,190] Repair session 7ae6fe71-5a25-11ea-8b5a-3b5edc400697
> for range [(4989724097640378549,4989803009764075856]] finished (progress: 5%)
> [2020-02-28 17:57:27,549] Repair session 7acfccf1-5a25-11ea-8b5a-3b5edc400697
> for range [(4512226173347132723,4512312572318114003]] finished (progress: 5%)
> [2020-02-28 17:57:37,534] Repair session 7ae1ce50-5a25-11ea-8b5a-3b5edc400697
> for range [(7146381406763943813,7146516394221309657]] finished (progress: 6%)
> Feb 28, 2020 5:59:00 PM ClientCommunicatorAdmin Checker-run
> WARNING: Failed to check the connection: java.net.SocketTimeoutException:
> Read timed out
> [2020-02-28 17:59:36,419] Repair session 7aeacf01-5a25-11ea-8b5a-3b5edc400697
> for range [(4717535938324360043,4717683452072792471]] finished (progress: 6%)
> Current setup Details:
> Allocated Memory: 128gb
> No of nodes:12 DC1 and 12 DC2
> |*Sr No*|*Parameter Name*|*Old value*|*New Value*|*File Name*|
> |1|Min Heap Size -Xms|-Xms32G|-Xms64G|jvm.options|
> |2|Max Heap Size -Xmx|-Xms32G|-Xms64G|jvm.options|
> |3|Young generation size -Xmn|#-Xmn800M|-Xmn3072M|jvm.options|
> |4|GC SETTINGS|CMS|G1GC|jvm.options|
> |5|Disk Read_ahead_kb|4096|64|/sys/class/block/sdd/queue/read_ahead_kb|
> |6|Disk Rotational|1|1|/sys/class/block/sdd/queue/rotational|
> |7|block devices attributes|8192|128|blockdev --setra 128 /dev/sdd (To set
> it)|
> |8|commitlog_segment_size_in_mb|32|64|cassandra.yaml|
> |9|memtable_heap_space_in_mb|1/4 the size of the heap|8192|cassandra.yaml|
> |10|memtable_offheap_space_in_mb|1/4 the size of the heap|8192|cassandra.yaml|
> |11|phi_convict_threshold|#8|12|cassandra.yaml|
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
