phet commented on a change in pull request #3438:
URL: https://github.com/apache/gobblin/pull/3438#discussion_r764494243
##########
File path:
gobblin-modules/gobblin-sql/src/main/java/org/apache/gobblin/writer/commands/MySqlBufferedInserter.java
##########
@@ -84,6 +96,13 @@ protected void initializeBatch(String databaseName, String
table)
+ " due to # of params limitation " + this.maxParamSize + " , # of
columns: " + this.columnNames.size());
}
this.batchSize = actualBatchSize;
+
+ // Use separate insertion statement if replacements are allowed
Review comment:
comment could potentially live in `createInsertStatementStr` javadoc
##########
File path:
gobblin-modules/gobblin-sql/src/main/java/org/apache/gobblin/writer/commands/PostgresWriterCommands.java
##########
@@ -51,7 +51,11 @@
private final JdbcBufferedInserter jdbcBufferedWriter;
private final Connection conn;
- public PostgresWriterCommands(State state, Connection conn) {
+ public PostgresWriterCommands(State state, Connection conn, boolean
overwriteRecords) throws IllegalArgumentException {
+ if (overwriteRecords) {
+ throw new IllegalArgumentException("Replace existing records is not
supported in PostgresWriterCommands");
Review comment:
somewhat esoteric, but canonical practice might recommend
`UnsupportedOperationException` in this situation - see, e.g.:
https://stackoverflow.com/a/49749857
##########
File path:
gobblin-modules/gobblin-sql/src/test/java/org/apache/gobblin/writer/MySqlBufferedInserterTest.java
##########
@@ -63,7 +93,7 @@ public void testMySqlBufferedInsert() throws SQLException {
}
inserter.flush();
- verify(conn, times(2)).prepareStatement(anyString());
+ verify(conn, times(2)).prepareStatement(matches("REPLACE INTO .*"));
Review comment:
ideally we'd have one, but pragmatically I understand the imperative to
ship these changes, so up to you whether we leave this as tech debt IoU to
return to
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
