[ https://issues.apache.org/jira/browse/CASSANDRA-15953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17162977#comment-17162977 ]
Sam Tunnicliffe commented on CASSANDRA-15953: --------------------------------------------- {quote}Given there is really no point of comparing system tables {quote} It isn't just that, {{system.batches}} uses {{LocalPartitioner}} so if you run an auto-discovery diff without excluding {{system}}, you get a bunch of errors like: {code} com.datastax.driver.core.exceptions.CodecNotFoundException: Codec not found for requested operation: [timeuuid <-> java.lang.Long] at com.datastax.driver.core.CodecRegistry.notFound(CodecRegistry.java:806) at com.datastax.driver.core.CodecRegistry.createCodec(CodecRegistry.java:649) at com.datastax.driver.core.CodecRegistry.findCodec(CodecRegistry.java:631) at com.datastax.driver.core.CodecRegistry.codecFor(CodecRegistry.java:556) at com.datastax.driver.core.AbstractGettableByIndexData.codecFor(AbstractGettableByIndexData.java:80) at com.datastax.driver.core.BoundStatement.bind(BoundStatement.java:203) at com.datastax.driver.core.DefaultPreparedStatement.bind(DefaultPreparedStatement.java:134) at org.apache.cassandra.diff.DiffCluster.fetchPartitionKeys(DiffCluster.java:120) at org.apache.cassandra.diff.DiffCluster.getPartitionKeys(DiffCluster.java:110) at org.apache.cassandra.diff.Differ.diffTable(Differ.java:202) at org.apache.cassandra.diff.Differ.run(Differ.java:177) at org.apache.cassandra.diff.DiffJob.lambda$run$f7b4d595$1(DiffJob.java:173) at org.apache.spark.api.java.JavaPairRDD$$anonfun$toScalaFunction$1.apply(JavaPairRDD.scala:1040) at scala.collection.Iterator$$anon$11.next(Iterator.scala:409) at scala.collection.Iterator$class.foreach(Iterator.scala:893) at scala.collection.AbstractIterator.foreach(Iterator.scala:1336) at scala.collection.TraversableOnce$class.reduceLeft(TraversableOnce.scala:185) at scala.collection.AbstractIterator.reduceLeft(Iterator.scala:1336) at org.apache.spark.rdd.RDD$$anonfun$reduce$1$$anonfun$14.apply(RDD.scala:1015) at org.apache.spark.rdd.RDD$$anonfun$reduce$1$$anonfun$14.apply(RDD.scala:1013) at org.apache.spark.SparkContext$$anonfun$33.apply(SparkContext.scala:2130) at org.apache.spark.SparkContext$$anonfun$33.apply(SparkContext.scala:2130) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87) at org.apache.spark.scheduler.Task.run(Task.scala:109) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) {code} > Support fetching all user tables to compare in Cassandra-diff > ------------------------------------------------------------- > > Key: CASSANDRA-15953 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15953 > Project: Cassandra > Issue Type: Improvement > Components: Tool/diff > Reporter: Yifan Cai > Assignee: Yifan Cai > Priority: Normal > Labels: pull-request-available > > The spark diff job may fail to launch with kernel error "E2BIG: Argument list > too long", when passing a large list of keyspace table list to compare. > Proposing a mode to fetch all user tables from the clusters to be compared. > When the mode is on, the spark job ignores the parameter "keyspace_tables". -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org