frankgh commented on code in PR #45:
URL:
https://github.com/apache/cassandra-analytics/pull/45#discussion_r1540121145
##########
cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/CassandraBulkWriterContext.java:
##########
@@ -87,11 +87,21 @@ private CassandraBulkWriterContext(@NotNull BulkSparkConf
conf,
Set<String> udts = CqlUtils.extractUdts(keyspaceSchema, keyspace);
ReplicationFactor replicationFactor =
CqlUtils.extractReplicationFactor(keyspaceSchema, keyspace);
int indexCount = CqlUtils.extractIndexCount(keyspaceSchema, keyspace,
table);
- CqlTable cqlTable = bridge.buildSchema(createTableSchema, keyspace,
replicationFactor, partitioner, udts, null, indexCount);
+ CqlTable cqlTable = bridge().buildSchema(createTableSchema, keyspace,
replicationFactor, partitioner, udts, null, indexCount);
TableInfoProvider tableInfoProvider = new
CqlTableInfoProvider(createTableSchema, cqlTable);
TableSchema tableSchema = initializeTableSchema(conf, dfSchema,
tableInfoProvider, lowestCassandraVersion);
- schemaInfo = new CassandraSchemaInfo(tableSchema);
+ schemaInfo = new CassandraSchemaInfo(tableSchema, udts, cqlTable);
+ }
+
+ @Override
+ public synchronized CassandraBridge bridge()
+ {
+ if (this.bridge == null)
+ {
+ this.bridge =
CassandraBridgeFactory.get(this.clusterInfo.getLowestCassandraVersion());
Review Comment:
On a smaller cluster maybe not, but I think it can add up on a big Cassandra
cluster when running a large spark job. I think there's a tradeoff here. In the
case where that one instance coming back with a lower version, the instance
would probably miss the write (as it would be unable to import the SStables)
and the data would need to be repaired. So the question is whether we want so
save some REST endpoint requests on all the job runs vs save on repair cost on
a very unlikely scenario that would very rarely happen.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
