[kudu-CR] Add error handling to Backup and Restore
Hello Kudu Jenkins, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10941 to look at the new patch set (#15). Change subject: Add error handling to Backup and Restore .. Add error handling to Backup and Restore A backup/restore will be attempted on every,regardless of any one table failure. If any one table fails the job will exit with a non-zero code. If a table is successfully backed up in the current job or the table metadata has a timestamp equaling the user-supplied timestamp, the table will be marked successful. If a table fails to back up any partial data will be removed. If a table fails to restore and a restore Kudu table has already been created the `-restore` table will not be removed. This should probably be done in a later commit. A list of failed/success full backups or restores will be printed at the end of each job. Spark SaveMode overwrite/append semantics are not handled as part of this commit. Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 --- M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala 5 files changed, 384 insertions(+), 113 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/41/10941/15 -- To view, visit http://gerrit.cloudera.org:8080/10941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 Gerrit-Change-Number: 10941 Gerrit-PatchSet: 15 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tony Foerster
[kudu-CR] Add error handling to Backup and Restore
Hello Kudu Jenkins, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10941 to look at the new patch set (#14). Change subject: Add error handling to Backup and Restore .. Add error handling to Backup and Restore A backup/restore will be attempted on every,regardless of any one table failure. If any one table fails the job will exit with a non-zero code. If a table is successfully backed up in the current job or the table metadata has a timestamp equaling the user-supplied timestamp, the table will be marked successful. If a table fails to back up any partial data will be removed. If a table fails to restore and a restore Kudu table has already been created the `-restore` table will not be removed. This should probably be done in a later commit. A list of failed/success full backups or restores will be printed at the end of each job. Spark SaveMode overwrite/append semantics are not handled as part of this commit. Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 --- M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala 5 files changed, 384 insertions(+), 116 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/41/10941/14 -- To view, visit http://gerrit.cloudera.org:8080/10941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 Gerrit-Change-Number: 10941 Gerrit-PatchSet: 14 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tony Foerster
[kudu-CR] Add error handling to Backup and Restore
Hello Kudu Jenkins, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10941 to look at the new patch set (#13). Change subject: Add error handling to Backup and Restore .. Add error handling to Backup and Restore A backup/restore will be attempted on every,regardless of any one table failure. If any one table fails the job will exit with a non-zero code. If a table is successfully backed up in the current job or the table metadata has a timestamp equaling the user-supplied timestamp, the table will be marked successful. If a table fails to back up any partial data will be removed. If a table fails to restore and a restore Kudu table has already been created the `-restore` table will not be removed. This should probably be done in a later commit. A list of failed/success full backups or restores will be printed at the end of each job. Spark SaveMode overwrite/append semantics are not handled as part of this commit. Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 --- M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala 5 files changed, 343 insertions(+), 110 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/41/10941/13 -- To view, visit http://gerrit.cloudera.org:8080/10941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 Gerrit-Change-Number: 10941 Gerrit-PatchSet: 13 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tony Foerster
[kudu-CR] Add error handling to Backup and Restore
Hello Kudu Jenkins, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10941 to look at the new patch set (#12). Change subject: Add error handling to Backup and Restore .. Add error handling to Backup and Restore A backup/restore will be attempted on every,regardless of any one table failure. If any one table fails the job will exit with a non-zero code. If a table is successfully backed up in the current job or the table metadata has a timestamp equaling the user-supplied timestamp, the table will be marked successful. If a table fails to back up any partial data will be removed. If a table fails to restore and a restore Kudu table has already been created the `-restore` table will not be removed. This should probably be done in a later commit. A list of failed/success full backups or restores will be printed at the end of each job. Spark SaveMode overwrite/append semantics are not handled as part of this commit. Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 --- M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala 5 files changed, 342 insertions(+), 109 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/41/10941/12 -- To view, visit http://gerrit.cloudera.org:8080/10941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 Gerrit-Change-Number: 10941 Gerrit-PatchSet: 12 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tony Foerster
[kudu-CR] Add error handling to Backup and Restore
Hello Kudu Jenkins, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10941 to look at the new patch set (#11). Change subject: Add error handling to Backup and Restore .. Add error handling to Backup and Restore A backup/restore will be attempted on every,regardless of any one table failure. If any one table fails the job will exit with a non-zero code. If a table is successfully backed up in the current job or the table metadata has a timestamp equaling the user-supplied timestamp, the table will be marked successful. If a table fails to back up any partial data will be removed. If a table fails to restore and a restore Kudu table has already been created the `-restore` table will not be removed. This should probably be done in a later commit. A list of failed/success full backups or restores will be printed at the end of each job. Spark SaveMode overwrite/append semantics are not handled as part of this commit. Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 --- M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala 5 files changed, 343 insertions(+), 109 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/41/10941/11 -- To view, visit http://gerrit.cloudera.org:8080/10941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 Gerrit-Change-Number: 10941 Gerrit-PatchSet: 11 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tony Foerster
[kudu-CR] Add error handling to Backup and Restore
Hello Kudu Jenkins, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10941 to look at the new patch set (#10). Change subject: Add error handling to Backup and Restore .. Add error handling to Backup and Restore A backup/restore will be attempted on every table. If any one table fails the job will exit with a non-zero code. If a table is successfully backed up in the current job or the table metadata has a timestamp up to the current user supplied timestamp, the table will be marked successful. If a table fails to back up any partial data will be removed. If a table fails to restore and a restore Kudu table has already been created, it will not be removed. A list of failed/success full backups or restores will be printed at the end of each job. Spark SaveMode overwrite/append semantics are not handled as part of this commit. Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 --- M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala 5 files changed, 344 insertions(+), 114 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/41/10941/10 -- To view, visit http://gerrit.cloudera.org:8080/10941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 Gerrit-Change-Number: 10941 Gerrit-PatchSet: 10 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tony Foerster
[kudu-CR] Add error handling to Backup and Restore
Tony Foerster has restored this change. ( http://gerrit.cloudera.org:8080/10941 ) Change subject: Add error handling to Backup and Restore .. Restored Restoring after fixes -- To view, visit http://gerrit.cloudera.org:8080/10941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: restore Gerrit-Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 Gerrit-Change-Number: 10941 Gerrit-PatchSet: 9 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tony Foerster
[kudu-CR] Add error handling to Backup and Restore
Tony Foerster has abandoned this change. ( http://gerrit.cloudera.org:8080/10941 ) Change subject: Add error handling to Backup and Restore .. Abandoned I was considering this a sort of incremental change, but looking at it I think it needs more work. I'll open a new request after reworking. -- To view, visit http://gerrit.cloudera.org:8080/10941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 Gerrit-Change-Number: 10941 Gerrit-PatchSet: 9 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tony Foerster
[kudu-CR] KUDU-2095 - Add scanner `keepAlive` RPC call to Java API
Tony Foerster has abandoned this change. ( http://gerrit.cloudera.org:8080/7749 ) Change subject: KUDU-2095 - Add scanner `keepAlive` RPC call to Java API .. Abandoned Keepalive shouldn't rely on getting the same replica from the cached RemoteTablet. -- To view, visit http://gerrit.cloudera.org:8080/7749 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: Iba647e7b5ed4ee8d223d387b6656d3fb02c41713 Gerrit-Change-Number: 7749 Gerrit-PatchSet: 9 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Tony Foerster
[kudu-CR] KUDU-2524: temporarily disable scalafmt in gradle build
Tony Foerster has posted comments on this change. ( http://gerrit.cloudera.org:8080/11101 ) Change subject: KUDU-2524: temporarily disable scalafmt in gradle build .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/11101 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30cf5cfaf72b65c0178bc951e4abb75da0ad7069 Gerrit-Change-Number: 11101 Gerrit-PatchSet: 1 Gerrit-Owner: Adar Dembo Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tony Foerster Gerrit-Comment-Date: Wed, 01 Aug 2018 18:47:01 + Gerrit-HasComments: No
[kudu-CR] [Java] Minor fixes to the Scalafmt changes
Tony Foerster has posted comments on this change. ( http://gerrit.cloudera.org:8080/11089 ) Change subject: [Java] Minor fixes to the Scalafmt changes .. Patch Set 3: Code-Review+1 +1 on max columns 100, can't believe I missed that. -- To view, visit http://gerrit.cloudera.org:8080/11089 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ied557d2ab501f43288e7edae2874ba123ab036bc Gerrit-Change-Number: 11089 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Tony Foerster Gerrit-Comment-Date: Tue, 31 Jul 2018 21:37:06 + Gerrit-HasComments: No
[kudu-CR] KUDU-2095 - Add scanner `keepAlive` RPC call to Java API
Tony Foerster has posted comments on this change. ( http://gerrit.cloudera.org:8080/7749 ) Change subject: KUDU-2095 - Add scanner `keepAlive` RPC call to Java API .. Patch Set 9: Code-Review-1 (1 comment) http://gerrit.cloudera.org:8080/#/c/7749/9/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java File java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java: http://gerrit.cloudera.org:8080/#/c/7749/9/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java@1008 PS9, Line 1008: final ReplicaSelection replicaSelection = scanner.getReplicaSelection(); : final ServerInfo info = : tablet.getReplicaSelectedServerInfo(replicaSelection); > I think the server is always the same but this behavior has been marked as Doesn't sound like relying on getting the same replica is the way to go, I'll look into keeping the replica data with a scanner. -- To view, visit http://gerrit.cloudera.org:8080/7749 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iba647e7b5ed4ee8d223d387b6656d3fb02c41713 Gerrit-Change-Number: 7749 Gerrit-PatchSet: 9 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Tony Foerster Gerrit-Comment-Date: Tue, 31 Jul 2018 20:15:45 + Gerrit-HasComments: Yes
[kudu-CR] [Java] Minor fixes to the Scalafmt changes
Tony Foerster has posted comments on this change. ( http://gerrit.cloudera.org:8080/11089 ) Change subject: [Java] Minor fixes to the Scalafmt changes .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/11089 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ied557d2ab501f43288e7edae2874ba123ab036bc Gerrit-Change-Number: 11089 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Tony Foerster Gerrit-Comment-Date: Tue, 31 Jul 2018 18:29:03 + Gerrit-HasComments: No
[kudu-CR] KUDU-2095 - Add scanner `keepAlive` RPC call to Java API
Tony Foerster has posted comments on this change. ( http://gerrit.cloudera.org:8080/7749 ) Change subject: KUDU-2095 - Add scanner `keepAlive` RPC call to Java API .. Patch Set 9: (5 comments) http://gerrit.cloudera.org:8080/#/c/7749/9/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java File java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java: http://gerrit.cloudera.org:8080/#/c/7749/9/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java@1006 PS9, Line 1006: > nit: double space Done http://gerrit.cloudera.org:8080/#/c/7749/9/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java@1007 PS9, Line 1007: defaultAdminOperationTimeoutMs > this seems like an odd choice of timeout. I think this defaults to be prett I'm not sure exactly what's reasonable here? I set it to a constant 10 seconds. I also considered setting it to some division of defaultAdminOperationTimeoutMs, that way it would be somewhat configurable (but also more complicated maybe than it needs to be). http://gerrit.cloudera.org:8080/#/c/7749/9/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java@1008 PS9, Line 1008: final ReplicaSelection replicaSelection = scanner.getReplicaSelection(); : final ServerInfo info = : tablet.getReplicaSelectedServerInfo(replicaSelection); > is this guaranteed to be the same replica that the scanner is already open I think the server is always the same but this behavior has been marked as a bug in the meantime: https://github.com/cloudera/kudu/blob/master/java/kudu-client/src/main/java/org/apache/kudu/client/RemoteTablet.java#L191 I opened a pull request for this here: https://gerrit.cloudera.org/#/c/11088/ I kept the replica returned non-random per RemoteTablet instance, this way I won't have to keep track of replicas within the scanner. I think this is ok because it's unlikely (let me know if this is a bad assumption) for any client to read from the same tablet concurrently. http://gerrit.cloudera.org:8080/#/c/7749/9/java/kudu-client/src/main/java/org/apache/kudu/client/KuduScanner.java File java/kudu-client/src/main/java/org/apache/kudu/client/KuduScanner.java: http://gerrit.cloudera.org:8080/#/c/7749/9/java/kudu-client/src/main/java/org/apache/kudu/client/KuduScanner.java@75 PS9, Line 75: SetFaultTolerant > this is the C++ API name I shamelessly stole this from the C++ docs obviously. I need to double check that the behavior with isFaultTolerant is the same. http://gerrit.cloudera.org:8080/#/c/7749/9/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java File java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java: http://gerrit.cloudera.org:8080/#/c/7749/9/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java@203 PS9, Line 203: 50% of the scanner ttl. > I dont see anywhere setting a short TTL on this test, am I missing somethin Not missing anything I removed mistakenly. I had been doing this for the entire test like https://gerrit.cloudera.org/#/c/7749/6/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java I also tried just spawning another minicluster https://gerrit.cloudera.org/#/c/7749/3/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java I am thinking I will move this to a new test class. -- To view, visit http://gerrit.cloudera.org:8080/7749 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iba647e7b5ed4ee8d223d387b6656d3fb02c41713 Gerrit-Change-Number: 7749 Gerrit-PatchSet: 9 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Tony Foerster Gerrit-Comment-Date: Tue, 31 Jul 2018 15:59:57 + Gerrit-HasComments: Yes
[kudu-CR] KUDU-2348 return random replica from RemoteTablet
Tony Foerster has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11088 Change subject: KUDU-2348 return random replica from RemoteTablet .. KUDU-2348 return random replica from RemoteTablet RemoteTablet is returning a non-random replica based on the hash of the tablet servers when no local replica exists and a leader isn't requested. This commit changes uses a random seed to return a random replica per RemoteTablet instance. RemoteTablet instances are kept in the TabletLocationsCache for 1 hour by default. When the cache is invalidated and a new RemoteReplica is created, a new random replica will be returned. It's possible to return a completely random replica, but this would make implementing features like a scanner keepalive more difficult. Change-Id: I6d0b0f92dafc56786c2e915f88daf83afedc8f90 --- M java/kudu-client/src/main/java/org/apache/kudu/client/RemoteTablet.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestRemoteTablet.java 2 files changed, 41 insertions(+), 24 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/88/11088/1 -- To view, visit http://gerrit.cloudera.org:8080/11088 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I6d0b0f92dafc56786c2e915f88daf83afedc8f90 Gerrit-Change-Number: 11088 Gerrit-PatchSet: 1 Gerrit-Owner: Tony Foerster
[kudu-CR] Scala code formatting with Scalafmt
Tony Foerster has posted comments on this change. ( http://gerrit.cloudera.org:8080/11030 ) Change subject: Scala code formatting with Scalafmt .. Patch Set 13: Code-Review+1 (1 comment) http://gerrit.cloudera.org:8080/#/c/11030/11/build-support/jenkins/build-and-test.sh File build-support/jenkins/build-and-test.sh: http://gerrit.cloudera.org:8080/#/c/11030/11/build-support/jenkins/build-and-test.sh@392 PS11, Line 392: GRADLE_FLAGS="$GRADLE_FLAGS --console=plain --no-daemon --continue -DrerunFailingTestsCount=3" > I think you can define this in the gradle build itself by adding this to gr Done -- To view, visit http://gerrit.cloudera.org:8080/11030 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iac96383d88394084e19712177d05f9fc63de766c Gerrit-Change-Number: 11030 Gerrit-PatchSet: 13 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Tony Foerster Gerrit-Comment-Date: Sat, 28 Jul 2018 02:15:43 + Gerrit-HasComments: Yes
[kudu-CR] Add kudu-backup module to Maven build
Tony Foerster has posted comments on this change. ( http://gerrit.cloudera.org:8080/11059 ) Change subject: Add kudu-backup module to Maven build .. Patch Set 4: (2 comments) http://gerrit.cloudera.org:8080/#/c/11059/3/java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala File java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala: http://gerrit.cloudera.org:8080/#/c/11059/3/java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala@226 PS3, Line 226: Base64.encodeBase64String(value.asInstanceOf[Array[Byte]]) > I think this should be `Base64.encodeBase64String(value.asInstanceOf[Array[ Done http://gerrit.cloudera.org:8080/#/c/11059/3/java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala@243 PS3, Line 243: case Type.BINARY => Base64.decodeBase64(value.asInstanceOf[Array[Byte]]) > I think this should be `Base64.decodeBase64(value)` Done -- To view, visit http://gerrit.cloudera.org:8080/11059 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia1b34b733d8d85106060068e611be7eecc0717f2 Gerrit-Change-Number: 11059 Gerrit-PatchSet: 4 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tony Foerster Gerrit-Comment-Date: Thu, 26 Jul 2018 19:23:47 + Gerrit-HasComments: Yes
[kudu-CR] Add kudu-backup module to Maven build
Hello Kudu Jenkins, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11059 to look at the new patch set (#4). Change subject: Add kudu-backup module to Maven build .. Add kudu-backup module to Maven build Change Base64 encode/decode to java 7 compatible versions to fix sniffer errors. Change-Id: Ia1b34b733d8d85106060068e611be7eecc0717f2 --- M java/kudu-backup/pom.xml M java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala M java/pom.xml 3 files changed, 13 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/59/11059/4 -- To view, visit http://gerrit.cloudera.org:8080/11059 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia1b34b733d8d85106060068e611be7eecc0717f2 Gerrit-Change-Number: 11059 Gerrit-PatchSet: 4 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tony Foerster
[kudu-CR] Add kudu-backup module to Maven build
Tony Foerster has posted comments on this change. ( http://gerrit.cloudera.org:8080/11059 ) Change subject: Add kudu-backup module to Maven build .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/11059/2/java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala File java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala: http://gerrit.cloudera.org:8080/#/c/11059/2/java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala@23 PS2, Line 23: import javax.xml.bind.DatatypeConverter > Maybe use import org.apache.commons.net.util.Base64 since we use that elsew Done -- To view, visit http://gerrit.cloudera.org:8080/11059 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia1b34b733d8d85106060068e611be7eecc0717f2 Gerrit-Change-Number: 11059 Gerrit-PatchSet: 2 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tony Foerster Gerrit-Comment-Date: Thu, 26 Jul 2018 19:03:53 + Gerrit-HasComments: Yes
[kudu-CR] Add kudu-backup module to Maven build
Hello Kudu Jenkins, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11059 to look at the new patch set (#3). Change subject: Add kudu-backup module to Maven build .. Add kudu-backup module to Maven build Change Base64 encode/decode to java 7 compatible versions to fix sniffer errors. Change-Id: Ia1b34b733d8d85106060068e611be7eecc0717f2 --- M java/kudu-backup/pom.xml M java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala M java/pom.xml 3 files changed, 13 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/59/11059/3 -- To view, visit http://gerrit.cloudera.org:8080/11059 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia1b34b733d8d85106060068e611be7eecc0717f2 Gerrit-Change-Number: 11059 Gerrit-PatchSet: 3 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tony Foerster
[kudu-CR] Scala code formatting with Scalafmt
Hello Mike Percy, Kudu Jenkins, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11030 to look at the new patch set (#13). Change subject: Scala code formatting with Scalafmt .. Scala code formatting with Scalafmt Scalafmt https://scalameta.org/scalafmt/#Scalafmt is a code formatting tool, scalafmt.conf is the main configuration file. Scalafmt is added using the Gradle Scalafmt plugin: https://github.com/alenkacz/gradle-scalafmt and Maven scalafmt plugin: https://github.com/SimonJPegg/mvn_scalafmt. The plugin is configured to run on compile or can invoked with 'gradle scalafmt' or 'gradle testScalafmt' to format test code. It will run during the verification stage of a Maven build. Set gradle endoding globally because 'testScalafmt' target would otherwise choke on some special characters. Change-Id: Iac96383d88394084e19712177d05f9fc63de766c --- M java/.gitignore A java/.scalafmt.conf M java/buildSrc/build.gradle M java/gradle.properties M java/gradle/quality.gradle M java/kudu-backup/pom.xml M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupRDD.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestoreOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala M java/kudu-spark-tools/pom.xml M java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/ImportExportFiles.scala M java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/IntegrationTestBigLinkedList.scala M java/kudu-spark/pom.xml M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduRDD.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduWriteOptions.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/OperationType.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/SparkUtil.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/package.scala M java/pom.xml 24 files changed, 1,117 insertions(+), 767 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/30/11030/13 -- To view, visit http://gerrit.cloudera.org:8080/11030 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iac96383d88394084e19712177d05f9fc63de766c Gerrit-Change-Number: 11030 Gerrit-PatchSet: 13 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Tony Foerster
[kudu-CR] KUDU-2095 - Add scanner `keepAlive` RPC call to Java API
Hello Kudu Jenkins, Grant Henke, Todd Lipcon, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/7749 to look at the new patch set (#9). Change subject: KUDU-2095 - Add scanner `keepAlive` RPC call to Java API .. KUDU-2095 - Add scanner `keepAlive` RPC call to Java API - Adds keepAlive methods to sync and async clients - Methods return a (Deferred) Status object Where possible the behavior mimics the C++ client. Publicly accessible methods are available in AsyncKuduScanner and KuduScanner. A call to KeepAlive will return a Status object which will be OK if - The scanner has been initialized - The scanner is active and has rows left to return - The scanner is between tablets It will return a non-ok status when - The scanner has not yet been initialized - The scanner has no more rows Change-Id: Iba647e7b5ed4ee8d223d387b6656d3fb02c41713 --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduScanner.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduScanner.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java 4 files changed, 190 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/49/7749/9 -- To view, visit http://gerrit.cloudera.org:8080/7749 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iba647e7b5ed4ee8d223d387b6656d3fb02c41713 Gerrit-Change-Number: 7749 Gerrit-PatchSet: 9 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Tony Foerster
[kudu-CR] Add kudu-backup module to Maven build
Tony Foerster has posted comments on this change. (
http://gerrit.cloudera.org:8080/11059 )
Change subject: Add kudu-backup module to Maven build
..
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/11059/2/java/kudu-backup/pom.xml
File java/kudu-backup/pom.xml:
http://gerrit.cloudera.org:8080/#/c/11059/2/java/kudu-backup/pom.xml@172
PS2, Line 172:
com.google.protobuf:protoc:${protobuf.version}:exe:${os.detected.classifier}
Used the protobuf version for protoc, same as the gradle build.
--
To view, visit http://gerrit.cloudera.org:8080/11059
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1b34b733d8d85106060068e611be7eecc0717f2
Gerrit-Change-Number: 11059
Gerrit-PatchSet: 2
Gerrit-Owner: Tony Foerster
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tony Foerster
Gerrit-Comment-Date: Thu, 26 Jul 2018 17:00:53 +
Gerrit-HasComments: Yes
[kudu-CR] Add kudu-backup module to Maven build
Tony Foerster has posted comments on this change. ( http://gerrit.cloudera.org:8080/11059 ) Change subject: Add kudu-backup module to Maven build .. Patch Set 2: (1 comment) Fixed formatting in second push. http://gerrit.cloudera.org:8080/#/c/11059/1/java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala File java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala: http://gerrit.cloudera.org:8080/#/c/11059/1/java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala@13 PS1, Line 13: * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. oops, formatted. Will fix -- To view, visit http://gerrit.cloudera.org:8080/11059 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia1b34b733d8d85106060068e611be7eecc0717f2 Gerrit-Change-Number: 11059 Gerrit-PatchSet: 2 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tony Foerster Gerrit-Comment-Date: Thu, 26 Jul 2018 16:59:46 + Gerrit-HasComments: Yes
[kudu-CR] Add kudu-backup module to Maven build
Hello Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11059 to look at the new patch set (#2). Change subject: Add kudu-backup module to Maven build .. Add kudu-backup module to Maven build Change Base64 encode/decode to java 7 compatible versions to fix sniffer errors. Change-Id: Ia1b34b733d8d85106060068e611be7eecc0717f2 --- M java/kudu-backup/pom.xml M java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala M java/pom.xml 3 files changed, 12 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/59/11059/2 -- To view, visit http://gerrit.cloudera.org:8080/11059 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia1b34b733d8d85106060068e611be7eecc0717f2 Gerrit-Change-Number: 11059 Gerrit-PatchSet: 2 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Kudu Jenkins
[kudu-CR] Add kudu-backup module to Maven build
Tony Foerster has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11059 Change subject: Add kudu-backup module to Maven build .. Add kudu-backup module to Maven build Change Base64 encode/decode to java 7 compatible versions to fix sniffer errors. Change-Id: Ia1b34b733d8d85106060068e611be7eecc0717f2 --- M java/kudu-backup/pom.xml M java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala M java/pom.xml 3 files changed, 108 insertions(+), 44 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/59/11059/1 -- To view, visit http://gerrit.cloudera.org:8080/11059 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ia1b34b733d8d85106060068e611be7eecc0717f2 Gerrit-Change-Number: 11059 Gerrit-PatchSet: 1 Gerrit-Owner: Tony Foerster
[kudu-CR] Scala code formatting with Scalafmt
Hello Mike Percy, Kudu Jenkins, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11030 to look at the new patch set (#12). Change subject: Scala code formatting with Scalafmt .. Scala code formatting with Scalafmt Scalafmt is added using the Gradle Scalafmt plugin: https://github.com/alenkacz/gradle-scalafmt. Scalafmt https://scalameta.org/scalafmt/#Scalafmt is a code formatting tool that runs from Gradle or as an IDE plugin. The .scalafmt.conf is the main configuration file for scalafmt. The plugin is configured to run on compile or can invoked with 'gradle scalafmt' or 'gradle testScalafmt' to format test code. Set gradle endoding globally because 'testScalafmt' target would otherwise choke on some special characters. Change-Id: Iac96383d88394084e19712177d05f9fc63de766c --- M java/.gitignore M java/buildSrc/build.gradle M java/gradle.properties M java/gradle/quality.gradle M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupRDD.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestoreOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala M java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/ImportExportFiles.scala M java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/IntegrationTestBigLinkedList.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduRDD.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduWriteOptions.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/OperationType.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/SparkUtil.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/package.scala 19 files changed, 1,067 insertions(+), 767 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/30/11030/12 -- To view, visit http://gerrit.cloudera.org:8080/11030 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iac96383d88394084e19712177d05f9fc63de766c Gerrit-Change-Number: 11030 Gerrit-PatchSet: 12 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Tony Foerster
[kudu-CR] Scala code formatting with Scalafmt
Tony Foerster has posted comments on this change. ( http://gerrit.cloudera.org:8080/11030 ) Change subject: Scala code formatting with Scalafmt .. Patch Set 11: I reset/reformatted all files with the new scalafmt settings. -- To view, visit http://gerrit.cloudera.org:8080/11030 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iac96383d88394084e19712177d05f9fc63de766c Gerrit-Change-Number: 11030 Gerrit-PatchSet: 11 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Tony Foerster Gerrit-Comment-Date: Thu, 26 Jul 2018 15:11:23 + Gerrit-HasComments: No
[kudu-CR] Scala code formatting with Scalafmt
Hello Mike Percy, Kudu Jenkins, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11030 to look at the new patch set (#11). Change subject: Scala code formatting with Scalafmt .. Scala code formatting with Scalafmt Scalafmt is added using the Gradle Scalafmt plugin: https://github.com/alenkacz/gradle-scalafmt. Scalafmt https://scalameta.org/scalafmt/#Scalafmt is a code formatting tool that runs from Gradle or as an IDE plugin. The .scalafmt.conf is the main configuration file for scalafmt. The plugin is configured to run on compile or can invoked with 'gradle scalafmt' or 'gradle testScalafmt' to format test code. Added gradle encoding argument because 'testScalafmt' target would otherwise choke on some special characters. Change-Id: Iac96383d88394084e19712177d05f9fc63de766c --- M build-support/jenkins/build-and-test.sh M java/.gitignore M java/buildSrc/build.gradle M java/gradle/quality.gradle M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupRDD.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestoreOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala M java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/ImportExportFiles.scala M java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/IntegrationTestBigLinkedList.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduRDD.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduWriteOptions.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/OperationType.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/SparkUtil.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/package.scala 19 files changed, 1,066 insertions(+), 768 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/30/11030/11 -- To view, visit http://gerrit.cloudera.org:8080/11030 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iac96383d88394084e19712177d05f9fc63de766c Gerrit-Change-Number: 11030 Gerrit-PatchSet: 11 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Tony Foerster
[kudu-CR] Scala code formatting with Scalafmt
Hello Mike Percy, Kudu Jenkins, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11030 to look at the new patch set (#10). Change subject: Scala code formatting with Scalafmt .. Scala code formatting with Scalafmt Scalafmt is added using the Gradle Scalafmt plugin: https://github.com/alenkacz/gradle-scalafmt. Scalafmt https://scalameta.org/scalafmt/#Scalafmt is a code formatting tool that runs from Gradle or as an IDE plugin. The .scalafmt.conf is the main configuration file for scalafmt. The plugin is configured to run on compile or can invoked with 'gradle scalafmt' or 'gradle testScalafmt' to format test code. Added gradle encoding argument because 'testScalafmt' target would otherwise choke on some special characters. Change-Id: Iac96383d88394084e19712177d05f9fc63de766c --- M build-support/jenkins/build-and-test.sh M java/.gitignore A java/.scalafmt.conf M java/buildSrc/build.gradle M java/gradle/quality.gradle M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupRDD.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestoreOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala M java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/ImportExportFiles.scala M java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/IntegrationTestBigLinkedList.scala M java/kudu-spark-tools/src/test/scala/org/apache/kudu/spark/tools/ITBigLinkedListTest.scala M java/kudu-spark-tools/src/test/scala/org/apache/kudu/spark/tools/TestImportExportFiles.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduRDD.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduWriteOptions.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/OperationType.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/SparkUtil.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/package.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/KuduContextTest.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/TestContext.scala 26 files changed, 1,632 insertions(+), 1,015 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/30/11030/10 -- To view, visit http://gerrit.cloudera.org:8080/11030 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iac96383d88394084e19712177d05f9fc63de766c Gerrit-Change-Number: 11030 Gerrit-PatchSet: 10 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Tony Foerster
[kudu-CR] Scala code formatting with Scalafmt
Hello Mike Percy, Kudu Jenkins, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11030 to look at the new patch set (#9). Change subject: Scala code formatting with Scalafmt .. Scala code formatting with Scalafmt Scalafmt is added using the Gradle Scalafmt plugin: https://github.com/alenkacz/gradle-scalafmt. Scalafmt https://scalameta.org/scalafmt/#Scalafmt is a code formatting tool that runs from Gradle or as an IDE plugin. The .scalafmt.conf is the main configuration file for scalafmt. The plugin is configured to run on compile or can invoked with 'gradle scalafmt' or 'gradle testScalafmt' to format test code. Added gradle encoding argument because 'testScalafmt' target would otherwise choke on some special characters. Change-Id: Iac96383d88394084e19712177d05f9fc63de766c --- M build-support/jenkins/build-and-test.sh M java/.gitignore A java/.scalafmt.conf M java/buildSrc/build.gradle M java/gradle/quality.gradle M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupRDD.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestoreOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala M java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/ImportExportFiles.scala M java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/IntegrationTestBigLinkedList.scala M java/kudu-spark-tools/src/test/scala/org/apache/kudu/spark/tools/ITBigLinkedListTest.scala M java/kudu-spark-tools/src/test/scala/org/apache/kudu/spark/tools/TestImportExportFiles.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduRDD.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduWriteOptions.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/OperationType.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/SparkUtil.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/package.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/KuduContextTest.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/TestContext.scala 26 files changed, 1,632 insertions(+), 1,015 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/30/11030/9 -- To view, visit http://gerrit.cloudera.org:8080/11030 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iac96383d88394084e19712177d05f9fc63de766c Gerrit-Change-Number: 11030 Gerrit-PatchSet: 9 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Tony Foerster
[kudu-CR] Fix flaky 'Random Backup and Restore' test
Tony Foerster has abandoned this change. ( http://gerrit.cloudera.org:8080/10902 ) Change subject: Fix flaky 'Random Backup and Restore' test .. Abandoned This is a bad workaround and the core issue should be fixed. -- To view, visit http://gerrit.cloudera.org:8080/10902 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: I51a6dbeeb064dff157609fdbbafe505668eb2b26 Gerrit-Change-Number: 10902 Gerrit-PatchSet: 2 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tony Foerster
[kudu-CR] Scala code formatting with Scalafmt
Hello Mike Percy, Kudu Jenkins, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11030 to look at the new patch set (#8). Change subject: Scala code formatting with Scalafmt .. Scala code formatting with Scalafmt Scalafmt is added using the Gradle Scalafmt plugin: https://github.com/alenkacz/gradle-scalafmt. Scalafmt https://scalameta.org/scalafmt/#Scalafmt is a code formatting tool that runs from Gradle or as an IDE plugin. The .scalafmt.conf is the main configuration file for scalafmt. The plugin is configured to run on compile or can invoked with 'gradle scalafmt' or 'gradle testScalafmt' to format test code Change-Id: Iac96383d88394084e19712177d05f9fc63de766c --- M java/.gitignore A java/.scalafmt.conf M java/buildSrc/build.gradle M java/gradle/quality.gradle M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupRDD.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestoreOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala M java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/ImportExportFiles.scala M java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/IntegrationTestBigLinkedList.scala M java/kudu-spark-tools/src/test/scala/org/apache/kudu/spark/tools/ITBigLinkedListTest.scala M java/kudu-spark-tools/src/test/scala/org/apache/kudu/spark/tools/TestImportExportFiles.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduRDD.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduWriteOptions.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/OperationType.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/SparkUtil.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/package.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/KuduContextTest.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/TestContext.scala 25 files changed, 1,631 insertions(+), 1,014 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/30/11030/8 -- To view, visit http://gerrit.cloudera.org:8080/11030 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iac96383d88394084e19712177d05f9fc63de766c Gerrit-Change-Number: 11030 Gerrit-PatchSet: 8 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Tony Foerster
[kudu-CR] Scala code formatting with Scalafmt
Hello Mike Percy, Kudu Jenkins, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11030 to look at the new patch set (#7). Change subject: Scala code formatting with Scalafmt .. Scala code formatting with Scalafmt Scalafmt is added using the Gradle Scalafmt plugin: https://github.com/alenkacz/gradle-scalafmt. Scalafmt https://scalameta.org/scalafmt/#Scalafmt is a code formatting tool that runs from Gradle or as an IDE plugin. The .scalafmt.conf is the main configuration file for scalafmt. The plugin is configured to run on compile or can invoked with 'gradle scalafmt' or 'gradle testScalafmt' to format test code Change-Id: Iac96383d88394084e19712177d05f9fc63de766c --- M java/.gitignore A java/.scalafmt.conf M java/buildSrc/build.gradle M java/gradle/quality.gradle M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupRDD.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestoreOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala M java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/ImportExportFiles.scala M java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/IntegrationTestBigLinkedList.scala M java/kudu-spark-tools/src/test/scala/org/apache/kudu/spark/tools/ITBigLinkedListTest.scala M java/kudu-spark-tools/src/test/scala/org/apache/kudu/spark/tools/TestImportExportFiles.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduRDD.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduWriteOptions.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/OperationType.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/SparkUtil.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/package.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/KuduContextTest.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/TestContext.scala 25 files changed, 1,631 insertions(+), 1,014 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/30/11030/7 -- To view, visit http://gerrit.cloudera.org:8080/11030 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iac96383d88394084e19712177d05f9fc63de766c Gerrit-Change-Number: 11030 Gerrit-PatchSet: 7 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Tony Foerster
[kudu-CR] KUDU-2095 - Add scanner `keepAlive` RPC call to Java API
Hello Kudu Jenkins, Todd Lipcon, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/7749 to look at the new patch set (#8). Change subject: KUDU-2095 - Add scanner `keepAlive` RPC call to Java API .. KUDU-2095 - Add scanner `keepAlive` RPC call to Java API - Adds keepAlive methods to sync and async clients - Methods return a (Deferred) Status object Where possible the behavior mimics the C++ client. Publicly accessible methods are available in AsyncKuduScanner and KuduScanner. A call to KeepAlive will return a Status object which will be OK if - The scanner has been initialized - The scanner is active and has rows left to return - The scanner is between tablets It will return a non-ok status when - The scanner has not yet been initialized - The scanner has no more rows Change-Id: Iba647e7b5ed4ee8d223d387b6656d3fb02c41713 --- A java/kudu-client/out/test/resources/log4j.properties A java/kudu-client/out/test/resources/test-key-and-cert.jks M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduScanner.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduScanner.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java 6 files changed, 212 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/49/7749/8 -- To view, visit http://gerrit.cloudera.org:8080/7749 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iba647e7b5ed4ee8d223d387b6656d3fb02c41713 Gerrit-Change-Number: 7749 Gerrit-PatchSet: 8 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Tony Foerster
[kudu-CR] Scala code formatting with Scalafmt
Tony Foerster has posted comments on this change. (
http://gerrit.cloudera.org:8080/11030 )
Change subject: Scala code formatting with Scalafmt
..
Patch Set 6:
Hmm, Jenkins is failing but the same target (:kudu-spark:testScalafmt) passes
locally for me, it's failing here
14:48:06 Execution failed for task ':kudu-spark:testScalafmt'.
14:48:06 > :628: error: unclosed character literal
14:48:06 val chars = List('a', 'b', '???', Char.MaxValue, '\u')
That line shows up differently in my editor
val chars = List('a', 'b', '乕', Char.MaxValue, '\u')
Maybe Jenkins is using a different character encoding?
--
To view, visit http://gerrit.cloudera.org:8080/11030
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iac96383d88394084e19712177d05f9fc63de766c
Gerrit-Change-Number: 11030
Gerrit-PatchSet: 6
Gerrit-Owner: Tony Foerster
Gerrit-Reviewer: Grant Henke
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy
Gerrit-Reviewer: Tony Foerster
Gerrit-Comment-Date: Wed, 25 Jul 2018 01:23:52 +
Gerrit-HasComments: No
[kudu-CR] Scala code formatting with Scalafmt
Tony Foerster has posted comments on this change. ( http://gerrit.cloudera.org:8080/11030 ) Change subject: Scala code formatting with Scalafmt .. Patch Set 6: (2 comments) http://gerrit.cloudera.org:8080/#/c/11030/6/java/.gitignore File java/.gitignore: http://gerrit.cloudera.org:8080/#/c/11030/6/java/.gitignore@39 PS6, Line 39: **/out let me know if you want this as part of a different commit. http://gerrit.cloudera.org:8080/#/c/11030/3/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/SparkUtil.scala File java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/SparkUtil.scala: http://gerrit.cloudera.org:8080/#/c/11030/3/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/SparkUtil.scala@25 PS3, Line 25: case Type.BOOL => BooleanType > You may need to undo the changes and then re-run the formatting. My underst Check out patch set 6, it's fixed there. The formatter will un-align, this patch set was not reading .scalafmt.conf and was instead using the default. -- To view, visit http://gerrit.cloudera.org:8080/11030 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iac96383d88394084e19712177d05f9fc63de766c Gerrit-Change-Number: 11030 Gerrit-PatchSet: 6 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Tony Foerster Gerrit-Comment-Date: Tue, 24 Jul 2018 22:05:43 + Gerrit-HasComments: Yes
[kudu-CR] Scala code formatting with Scalafmt
Hello Mike Percy, Kudu Jenkins, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11030 to look at the new patch set (#5). Change subject: Scala code formatting with Scalafmt .. Scala code formatting with Scalafmt Scalafmt is added using the Gradle Scalafmt plugin: https://github.com/alenkacz/gradle-scalafmt. Scalafmt https://scalameta.org/scalafmt/#Scalafmt is a code formatting tool that runs from Gradle or as an IDE plugin. The .scalafmt.conf is the main configuration file for scalafmt. The plugin is configured to run on compile or can invoked with 'gradle scalafmt' or 'gradle testScalafmt' to format test code Change-Id: Iac96383d88394084e19712177d05f9fc63de766c --- M java/.gitignore A java/.scalafmt.conf M java/buildSrc/build.gradle M java/gradle/quality.gradle M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupRDD.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestoreOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala M java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/ImportExportFiles.scala M java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/IntegrationTestBigLinkedList.scala M java/kudu-spark-tools/src/test/scala/org/apache/kudu/spark/tools/ITBigLinkedListTest.scala M java/kudu-spark-tools/src/test/scala/org/apache/kudu/spark/tools/TestImportExportFiles.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduRDD.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduWriteOptions.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/SparkUtil.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/package.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/KuduContextTest.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/TestContext.scala 24 files changed, 1,247 insertions(+), 720 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/30/11030/5 -- To view, visit http://gerrit.cloudera.org:8080/11030 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iac96383d88394084e19712177d05f9fc63de766c Gerrit-Change-Number: 11030 Gerrit-PatchSet: 5 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Tony Foerster
[kudu-CR] Scala code formatting with Scalafmt
Hello Mike Percy, Kudu Jenkins, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11030 to look at the new patch set (#4). Change subject: Scala code formatting with Scalafmt .. Scala code formatting with Scalafmt Scalafmt is added using the Gradle Scalafmt plugin: https://github.com/alenkacz/gradle-scalafmt. Scalafmt https://scalameta.org/scalafmt/#Scalafmt is a code formatting tool that runs from Gradle or as an IDE plugin. The .scalafmt.conf is the main configuration file for scalafmt. The plugin is configured to run on compile or can invoked with 'gradle scalafmt' or 'gradle testScalafmt' to format test code Change-Id: Iac96383d88394084e19712177d05f9fc63de766c --- M java/.gitignore A java/.scalafmt.conf M java/buildSrc/build.gradle M java/gradle/quality.gradle M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupRDD.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestoreOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala M java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/ImportExportFiles.scala M java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/IntegrationTestBigLinkedList.scala M java/kudu-spark-tools/src/test/scala/org/apache/kudu/spark/tools/ITBigLinkedListTest.scala M java/kudu-spark-tools/src/test/scala/org/apache/kudu/spark/tools/TestImportExportFiles.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduRDD.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduWriteOptions.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/SparkUtil.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/package.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/KuduContextTest.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/TestContext.scala 24 files changed, 1,248 insertions(+), 720 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/30/11030/4 -- To view, visit http://gerrit.cloudera.org:8080/11030 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iac96383d88394084e19712177d05f9fc63de766c Gerrit-Change-Number: 11030 Gerrit-PatchSet: 4 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Tony Foerster
[kudu-CR] Scala code formatting with Scalafmt
Tony Foerster has posted comments on this change. ( http://gerrit.cloudera.org:8080/11030 ) Change subject: Scala code formatting with Scalafmt .. Patch Set 3: Code-Review-1 (1 comment) It's still aligning case arrows, please ignore this until I can get that figured out http://gerrit.cloudera.org:8080/#/c/11030/3/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/SparkUtil.scala File java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/SparkUtil.scala: http://gerrit.cloudera.org:8080/#/c/11030/3/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/SparkUtil.scala@25 PS3, Line 25: case Type.BOOL=> BooleanType Not sure why it's still aligning case arrows, checking that. -- To view, visit http://gerrit.cloudera.org:8080/11030 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iac96383d88394084e19712177d05f9fc63de766c Gerrit-Change-Number: 11030 Gerrit-PatchSet: 3 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Tony Foerster Gerrit-Comment-Date: Tue, 24 Jul 2018 19:53:21 + Gerrit-HasComments: Yes
[kudu-CR] Scala code formatting with Scalafmt
Tony Foerster has posted comments on this change. ( http://gerrit.cloudera.org:8080/11030 ) Change subject: Scala code formatting with Scalafmt .. Patch Set 3: (7 comments) http://gerrit.cloudera.org:8080/#/c/11030/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11030/2//COMMIT_MSG@9 PS2, Line 9: Scalafmt is added using the Gradle Scalafmt plugin: > Have you considered using the spotless plugin? I saw that Spotless would do scalafmt, but also does a lot more, so I went with the simpler solution. If we want to add Java formatting as well Spotless might be the way to go, I don't have any experience with it. I'd feel better about doing the commits for scala/java style separately. If spotless was added in the future it should be trivial to use it for scalafmt. http://gerrit.cloudera.org:8080/#/c/11030/2/java/.scalafmt.conf File java/.scalafmt.conf: http://gerrit.cloudera.org:8080/#/c/11030/2/java/.scalafmt.conf@1 PS2, Line 1: rewrite.rules = [SortImports, sortModifiers, prefercurlyfors, AvoidInfix, expandimportselectors] > I like `expandimportselectors` because it results in less change collisions Done http://gerrit.cloudera.org:8080/#/c/11030/2/java/.scalafmt.conf@2 PS2, Line 2: align = none > +1 this setting is pretty aggressive I don't mind the aligned cases (as long as it's done automatically for me), but two on one I'll let it go :) http://gerrit.cloudera.org:8080/#/c/11030/2/java/.scalafmt.conf@9 PS2, Line 9: newlines.alwaysBeforeTopLevelStatements = true > Here is an example of what this setting can do: https://github.com/olafurpg Agreed, the formatting can get a little too vertical. http://gerrit.cloudera.org:8080/#/c/11030/2/java/gradle/quality.gradle File java/gradle/quality.gradle: http://gerrit.cloudera.org:8080/#/c/11030/2/java/gradle/quality.gradle@25 PS2, Line 25: apply plugin: "scalafmt" > nit: newline Done http://gerrit.cloudera.org:8080/#/c/11030/2/java/kudu-backup/out/test/resources/log4j.properties File java/kudu-backup/out/test/resources/log4j.properties: http://gerrit.cloudera.org:8080/#/c/11030/2/java/kudu-backup/out/test/resources/log4j.properties@1 PS2, Line 1: > OK I've done this many times now. I added the 'out' dirs to the .gitignore, let me know if you'd rather it be its own commit. http://gerrit.cloudera.org:8080/#/c/11030/2/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/package.scala File java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/package.scala: http://gerrit.cloudera.org:8080/#/c/11030/2/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/package.scala@24 PS2, Line 24: * Adds a method, `kudu`, to DataFrameReader that allows you to read Kudu tables using > Why is this using the Scala style comment? Odd, none of them changed to javadoc. Since they all seem to be scaladoc style in the first place I'll switch to that for now. Or maybe it's just broken. -- To view, visit http://gerrit.cloudera.org:8080/11030 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iac96383d88394084e19712177d05f9fc63de766c Gerrit-Change-Number: 11030 Gerrit-PatchSet: 3 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Tony Foerster Gerrit-Comment-Date: Tue, 24 Jul 2018 19:25:21 + Gerrit-HasComments: Yes
[kudu-CR] Scala code formatting with Scalafmt
Hello Mike Percy, Kudu Jenkins, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11030 to look at the new patch set (#3). Change subject: Scala code formatting with Scalafmt .. Scala code formatting with Scalafmt Scalafmt is added using the Gradle Scalafmt plugin: https://github.com/alenkacz/gradle-scalafmt. Scalafmt https://scalameta.org/scalafmt/#Scalafmt is a code formatting tool that runs from Gradle or as an IDE plugin. The .scalafmt.conf is the main configuration file for scalafmt. The plugin is configured to run on compile or can invoked with 'gradle scalafmt' or 'gradle testScalafmt' to format test code Change-Id: Iac96383d88394084e19712177d05f9fc63de766c --- M java/.gitignore A java/.scalafmt.conf M java/buildSrc/build.gradle M java/gradle/quality.gradle M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupRDD.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestoreOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala M java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/ImportExportFiles.scala M java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/IntegrationTestBigLinkedList.scala M java/kudu-spark-tools/src/test/scala/org/apache/kudu/spark/tools/ITBigLinkedListTest.scala M java/kudu-spark-tools/src/test/scala/org/apache/kudu/spark/tools/TestImportExportFiles.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduRDD.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduWriteOptions.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/SparkUtil.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/package.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/KuduContextTest.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/TestContext.scala 24 files changed, 1,249 insertions(+), 720 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/30/11030/3 -- To view, visit http://gerrit.cloudera.org:8080/11030 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iac96383d88394084e19712177d05f9fc63de766c Gerrit-Change-Number: 11030 Gerrit-PatchSet: 3 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy
[kudu-CR] Scala code formatting with Scalafmt
Hello Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11030 to look at the new patch set (#2). Change subject: Scala code formatting with Scalafmt .. Scala code formatting with Scalafmt Scalafmt is added using the Gradle Scalafmt plugin: https://github.com/alenkacz/gradle-scalafmt. Scalafmt https://scalameta.org/scalafmt/#Scalafmt is a code formatting tool that runs from Gradle or as an IDE plugin. The .scalafmt.conf is the main configuration file for scalafmt. Scalafmt can be invoked with 'gradle scalafmt' or 'gradle testScalafmt' to format test code Change-Id: Iac96383d88394084e19712177d05f9fc63de766c --- A java/.scalafmt.conf M java/buildSrc/build.gradle M java/gradle/quality.gradle A java/kudu-backup/out/test/resources/log4j.properties M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupRDD.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestoreOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala A java/kudu-spark-tools/out/test/resources/TestImportExportFiles.csv A java/kudu-spark-tools/out/test/resources/log4j.properties M java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/ImportExportFiles.scala M java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/IntegrationTestBigLinkedList.scala M java/kudu-spark-tools/src/test/scala/org/apache/kudu/spark/tools/TestImportExportFiles.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduRDD.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduWriteOptions.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/SparkUtil.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/package.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/KuduContextTest.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/TestContext.scala 25 files changed, 1,273 insertions(+), 721 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/30/11030/2 -- To view, visit http://gerrit.cloudera.org:8080/11030 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iac96383d88394084e19712177d05f9fc63de766c Gerrit-Change-Number: 11030 Gerrit-PatchSet: 2 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Kudu Jenkins
[kudu-CR] Scala code formatting with Scalafmt
Tony Foerster has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11030 Change subject: Scala code formatting with Scalafmt .. Scala code formatting with Scalafmt Scalafmt https://scalameta.org/scalafmt/#Scalafmt is a code formatting tool that runs from Gradle or as an IDE plugin. The .scalafmt.conf is the main configuration file for scalafmt. Scalafmt can be invoked with 'gradle scalafmt' or 'gradle testScalafmt' to format test code Change-Id: Iac96383d88394084e19712177d05f9fc63de766c --- A java/.scalafmt.conf M java/buildSrc/build.gradle M java/gradle/quality.gradle A java/kudu-backup/out/test/resources/log4j.properties M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupRDD.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestoreOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala A java/kudu-spark-tools/out/test/resources/TestImportExportFiles.csv A java/kudu-spark-tools/out/test/resources/log4j.properties M java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/ImportExportFiles.scala M java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/IntegrationTestBigLinkedList.scala M java/kudu-spark-tools/src/test/scala/org/apache/kudu/spark/tools/TestImportExportFiles.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduRDD.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduWriteOptions.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/SparkUtil.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/package.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/KuduContextTest.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/TestContext.scala 25 files changed, 1,273 insertions(+), 721 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/30/11030/1 -- To view, visit http://gerrit.cloudera.org:8080/11030 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Iac96383d88394084e19712177d05f9fc63de766c Gerrit-Change-Number: 11030 Gerrit-PatchSet: 1 Gerrit-Owner: Tony Foerster
[kudu-CR] Add error handling to Backup and Restore
Tony Foerster has posted comments on this change. ( http://gerrit.cloudera.org:8080/10941 ) Change subject: Add error handling to Backup and Restore .. Patch Set 9: Build failure is not related to this changeset (OptionalSSL/TestRpc.TestClientConnectionMetrics/0) -- To view, visit http://gerrit.cloudera.org:8080/10941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 Gerrit-Change-Number: 10941 Gerrit-PatchSet: 9 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tony Foerster Gerrit-Comment-Date: Mon, 23 Jul 2018 20:17:35 + Gerrit-HasComments: No
[kudu-CR] Add error handling to Backup and Restore
Hello Kudu Jenkins, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10941 to look at the new patch set (#9). Change subject: Add error handling to Backup and Restore .. Add error handling to Backup and Restore A backup/restore will be attempted on every table. If any one table fails the job will exit with a non-zero code. If a table is successfully backed up in the current job or the table metadata has a timestamp up to the current user supplied timestamp, the table will be marked successful. If a table fails to back up any partial data will be removed. If a table fails to restore and a restore Kudu table has already been created, it will not be removed. Spark SaveMode overwrite/append semantics are not handled as part of this commit. Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 --- M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/TestContext.scala 5 files changed, 346 insertions(+), 145 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/41/10941/9 -- To view, visit http://gerrit.cloudera.org:8080/10941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 Gerrit-Change-Number: 10941 Gerrit-PatchSet: 9 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tony Foerster
[kudu-CR] Add error handling to Backup and Restore
Tony Foerster has posted comments on this change. (
http://gerrit.cloudera.org:8080/10941 )
Change subject: Add error handling to Backup and Restore
..
Patch Set 8:
(8 comments)
http://gerrit.cloudera.org:8080/#/c/10941/6/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala
File java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala:
http://gerrit.cloudera.org:8080/#/c/10941/6/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala@35
PS6, Line 35: private val cause: Throwable
= null)
> nit: Just None.orNull is a bit unusual. Just null should work.
Done
http://gerrit.cloudera.org:8080/#/c/10941/6/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala@46
PS6, Line 46: // TODO: Make parallel so each table isn't process serially?
> nit: s/vailid/valid
Done
http://gerrit.cloudera.org:8080/#/c/10941/6/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala@57
PS6, Line 57: // Backup for this timestamp has already been
completed.
> I think we should only consider the case where getToMs == options.timestamp
Done
http://gerrit.cloudera.org:8080/#/c/10941/6/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala@111
PS6, Line 111: val out = fs.create(hPath, /* overwrite= */ false)
> nit: extra spaces
Done
http://gerrit.cloudera.org:8080/#/c/10941/6/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala
File java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala:
http://gerrit.cloudera.org:8080/#/c/10941/6/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala@90
PS6, Line 90: val conf = session.sparkContext.hadoopConfiguration
> nit: extra spaces. I think a formatter is aligning the =. This should be fi
right it's my formatter...
http://gerrit.cloudera.org:8080/#/c/10941/6/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala@103
PS6, Line 103:context: KuduContext):
Try[KuduTable] = Try {
> nit: explicit return type.
?? This has an explicit return type
http://gerrit.cloudera.org:8080/#/c/10941/6/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala@115
PS6, Line 115: def main(args: Array[String]): Unit = {
> Where is this used? We definitely don't want the restore process dropping t
It was used when removing partial "${tableName}-restore" tables. I've removed
that part of the code for now but this was left behind. Fixed.
http://gerrit.cloudera.org:8080/#/c/10941/6/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala@136
PS6, Line 136:
> nit: s/note/not
Done
--
To view, visit http://gerrit.cloudera.org:8080/10941
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691
Gerrit-Change-Number: 10941
Gerrit-PatchSet: 8
Gerrit-Owner: Tony Foerster
Gerrit-Reviewer: Grant Henke
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tony Foerster
Gerrit-Comment-Date: Mon, 23 Jul 2018 19:36:21 +
Gerrit-HasComments: Yes
[kudu-CR] Add error handling to Backup and Restore
Hello Kudu Jenkins, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10941 to look at the new patch set (#8). Change subject: Add error handling to Backup and Restore .. Add error handling to Backup and Restore A backup/restore will be attempted on every table. If any one table fails the job will exit with a non-zero code. If a table is successfully backed up in the current job or the table metadata has a timestamp up to the current user supplied timestamp, the table will be marked successful. If a table fails to back up any partial data will be removed. If a table fails to restore and a restore Kudu table has already been created, it will not be removed. Spark SaveMode overwrite/append semantics are not handled as part of this commit. Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 --- A java/kudu-backup/out/test/resources/log4j.properties M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/TestContext.scala 6 files changed, 369 insertions(+), 145 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/41/10941/8 -- To view, visit http://gerrit.cloudera.org:8080/10941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 Gerrit-Change-Number: 10941 Gerrit-PatchSet: 8 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tony Foerster
[kudu-CR] Add error handling to Backup and Restore
Hello Kudu Jenkins, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10941 to look at the new patch set (#7). Change subject: Add error handling to Backup and Restore .. Add error handling to Backup and Restore A backup/restore will be attempted on every table. If any one table fails the job will exit with a non-zero code. If a table is successfully backed up in the current job or the table metadata has a timestamp up to the current user supplied timestamp, the table will be marked successful. If a table fails to back up any partial data will be removed. If a table fails to restore and a restore Kudu table has already been created, it will not be removed. Spark SaveMode overwrite/append semantics are not handled as part of this commit. Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 --- M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/TestContext.scala 5 files changed, 354 insertions(+), 153 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/41/10941/7 -- To view, visit http://gerrit.cloudera.org:8080/10941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 Gerrit-Change-Number: 10941 Gerrit-PatchSet: 7 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tony Foerster
[kudu-CR] Add error handling to Backup and Restore
Hello Kudu Jenkins, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10941 to look at the new patch set (#6). Change subject: Add error handling to Backup and Restore .. Add error handling to Backup and Restore A backup/restore will be attempted on every table. If any one table fails the job will exit with a non-zero code. If a table is successfully backed up in the current job or the table metadata has a timestamp up to the current user supplied timestamp, the table will be marked successful. If a table fails to back up any partial data will be removed. If a table fails to restore and a restore Kudu table has already been created, it will not be removed. Spark SaveMode overwrite/append semantics are not handled as part of this commit. Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 --- M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/TestContext.scala 5 files changed, 368 insertions(+), 159 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/41/10941/6 -- To view, visit http://gerrit.cloudera.org:8080/10941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 Gerrit-Change-Number: 10941 Gerrit-PatchSet: 6 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tony Foerster
[kudu-CR] Add error handling to Backup and Restore
Hello Kudu Jenkins, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10941 to look at the new patch set (#5). Change subject: Add error handling to Backup and Restore .. Add error handling to Backup and Restore A backup/restore will be attempted on every table. If any one table fails job will exit with a non-zero code. If a table is successfully backed up in the current job or the table metadata has a timestamp up to the current user supplied timestamp, the table will be marked successful. If a table fails to back up any partial data will be removed. If a table fails to restore and a restore table has already been created, it will not be removed. At the end of the job if any table has failed to backup/restore an exception is thrown and the job exits with a non-zero code. Spark SaveMode overwrite/append semantics are not handled as part of this commit. Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 --- M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/TestContext.scala 5 files changed, 368 insertions(+), 159 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/41/10941/5 -- To view, visit http://gerrit.cloudera.org:8080/10941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 Gerrit-Change-Number: 10941 Gerrit-PatchSet: 5 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tony Foerster
[kudu-CR] Add error handling to KuduBackup
Tony Foerster has posted comments on this change. (
http://gerrit.cloudera.org:8080/10941 )
Change subject: Add error handling to KuduBackup
..
Patch Set 4: Code-Review-1
(16 comments)
http://gerrit.cloudera.org:8080/#/c/10941/2/java/kudu-backup/src/main/scala/org/apache/kudu/backup/BackupStatus.scala
File java/kudu-backup/src/main/scala/org/apache/kudu/backup/BackupStatus.scala:
http://gerrit.cloudera.org:8080/#/c/10941/2/java/kudu-backup/src/main/scala/org/apache/kudu/backup/BackupStatus.scala@5
PS2, Line 5:
> Since their isn't a ton of custom functionality perhaps using Scala's built
I've removed this for now, will tackle it at a later time. Currently just throw
an exception and tell the user to look back through the logs for failure.
http://gerrit.cloudera.org:8080/#/c/10941/2/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala
File java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala:
http://gerrit.cloudera.org:8080/#/c/10941/2/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala@39
PS2, Line 39: @InterfaceStability.Unstable
> Is there a reason you moved main to the top of the class? I think we should
I've got the 'scalafmt' plugin in intellij and I did the formatting out of
reflex. I'll fix the formatting/order
http://gerrit.cloudera.org:8080/#/c/10941/2/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala@56
PS2, Line 56: .map { metadata =>
> Nit. Use a more specific exception.
Done
http://gerrit.cloudera.org:8080/#/c/10941/2/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala@57
PS2, Line 57: if (metadata.getToMs >= options.timestampMs) {
> Instead of "at least one". Include the count that failed out of the total s
Done
http://gerrit.cloudera.org:8080/#/c/10941/2/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala@70
PS2, Line 70: }
> I am not sure this is how we will handle incremental. I am thinking we woul
I've changed this to check the timestamp in the metadata to see if the current
timestamp is already backed up. If it is, mark it success. If it's not, the
incremental can be handled there.
If we failed when the metadata exists, and in a yarn job some tables succeeded
and there was a retry, the tables that succeeded the first time would be marked
failed on the retry.
http://gerrit.cloudera.org:8080/#/c/10941/2/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala@74
PS2, Line 74: t: String,
> Why did this context get move here? I think it can be shared for all uses i
Done
http://gerrit.cloudera.org:8080/#/c/10941/2/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala@87
PS2, Line 87: }
> Maybe info level
Done
http://gerrit.cloudera.org:8080/#/c/10941/2/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala@96
PS2, Line 96: log.error(s"Failed to remove path: $tablePath", ex)
> Nit: add explicit return type
merged
http://gerrit.cloudera.org:8080/#/c/10941/2/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala@127
PS2, Line 127: session.sqlContext.createDataFrame(rdd,
sparkSchema(table.getSchema))
> Now that we do this 3 line preamble to get the filesystem and hpath a few t
It's used twice, once in restore and once in backup (third time was removed).
I'd like to leave it for now since the utility class + number of parameters in
the method would be a little more complex than the few lines it takes right now.
http://gerrit.cloudera.org:8080/#/c/10941/2/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala@135
PS2, Line 135: }
> What is this for? It looks like it writes the metadata.
removed.
http://gerrit.cloudera.org:8080/#/c/10941/4/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala
File java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala:
http://gerrit.cloudera.org:8080/#/c/10941/4/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala@21
PS4, Line 21: import java.nio.file.{ Path, Paths }
Is there a standard formatting tool you guys use? This is inconsistent with the
other files, I'll fix this.
http://gerrit.cloudera.org:8080/#/c/10941/1/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala
File java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala:
PS1:
Ignore this file. Restore will be more consistent with Backup.
http://gerrit.cloudera.org:8080/#/c/10941/2/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala
File java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala:
http://gerrit.cloudera.org:8080/#/c/10941/2/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala@66
PS2, Line 66: val writeOptions =
> Should the be
[kudu-CR] Add error handling to KuduBackup
Hello Kudu Jenkins, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10941 to look at the new patch set (#4). Change subject: Add error handling to KuduBackup .. Add error handling to KuduBackup KuduBackup will attempt to backup every table. If any one table fails the entire job will fail. If a table is was already backed up to the specified timestamp or the table is backed up, the table is marked successful. If a table fails to back up any partial data will be removed. At the end of the job if any table has failed to backup/restore an exception is thrown. Spark SaveMode overwrite/append semantics are not handled as part of this commit. Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 --- M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/TestContext.scala 5 files changed, 335 insertions(+), 126 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/41/10941/4 -- To view, visit http://gerrit.cloudera.org:8080/10941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 Gerrit-Change-Number: 10941 Gerrit-PatchSet: 4 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tony Foerster
[kudu-CR] Add error handling to KuduBackup
Hello Kudu Jenkins, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10941 to look at the new patch set (#3). Change subject: Add error handling to KuduBackup .. Add error handling to KuduBackup KuduBackup will attempt to backup every table. If any one table fails the entire job will fail. If a table is was already backed up to the specified timestamp or the table is backed up, the table is marked successful. If a table fails to back up any partial data will be removed. At the end of the job if any table has failed to backup/restore an exception is thrown. Spark SaveMode overwrite/append semantics are not handled as part of this commit. Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 --- A java/kudu-backup/out/test/resources/log4j.properties M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/TestContext.scala 6 files changed, 358 insertions(+), 126 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/41/10941/3 -- To view, visit http://gerrit.cloudera.org:8080/10941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 Gerrit-Change-Number: 10941 Gerrit-PatchSet: 3 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tony Foerster
[kudu-CR] WIP Add error handling to KuduBackup DONT BUILD KuduBackup will attempt to backup every table. If any one table fails the entire job will fail.
Hello Kudu Jenkins, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10941 to look at the new patch set (#2). Change subject: WIP Add error handling to KuduBackup DONT_BUILD KuduBackup will attempt to backup every table. If any one table fails the entire job will fail. .. WIP Add error handling to KuduBackup DONT_BUILD KuduBackup will attempt to backup every table. If any one table fails the entire job will fail. A backup attempt can have 2 results: - Success - table was backed up or an existing backup is in the output dir. Incremental backup to be implemented - Failure - table failed to backup. Any partial data will be removed At the end of the job lists of successes and failures are printed out. An exception is thrown if there are any table failures. TODO - - if 'overwrite' flag is specified, overwrite the 'metadata' folder. Not sure yet how to handle this on retries or if it's even a good idea. - more tests Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 --- A java/kudu-backup/out/test/resources/log4j.properties A java/kudu-backup/src/main/scala/org/apache/kudu/backup/BackupStatus.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala 6 files changed, 232 insertions(+), 56 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/41/10941/2 -- To view, visit http://gerrit.cloudera.org:8080/10941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 Gerrit-Change-Number: 10941 Gerrit-PatchSet: 2 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tony Foerster
[kudu-CR] WIP/repro reproduce 1-off errors in TestKuduBackup
Hello Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10959 to look at the new patch set (#2). Change subject: WIP/repro reproduce 1-off errors in TestKuduBackup .. WIP/repro reproduce 1-off errors in TestKuduBackup Three of 4 tests in TestKuduBackup are failing with one off errors, the restored table is 1 smaller than the original table. Ignore the Impala test, that one is failing because I'm asking for the lastPropagatedTimestamp but no records were ever inserted. Change-Id: I7f829065878873568d17ec5e54126da3f6716485 --- A java/kudu-backup/out/test/resources/log4j.properties M java/kudu-backup/src/main/protobuf/backup.proto M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupRDD.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala 6 files changed, 39 insertions(+), 14 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/59/10959/2 -- To view, visit http://gerrit.cloudera.org:8080/10959 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I7f829065878873568d17ec5e54126da3f6716485 Gerrit-Change-Number: 10959 Gerrit-PatchSet: 2 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Kudu Jenkins
[kudu-CR] WIP/repro reproduce 1-off errors in TestKuduBackup
Tony Foerster has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10959 Change subject: WIP/repro reproduce 1-off errors in TestKuduBackup .. WIP/repro reproduce 1-off errors in TestKuduBackup Three of 4 tests in TestKuduBackup are failing with one off errors, the restored table is 1 smaller than the original table. Ignore the Impala test, that one is failing because I'm asking for the lastPropagatedTimestamp but no records were ever inserted. NO_BUILD Change-Id: I7f829065878873568d17ec5e54126da3f6716485 --- A java/kudu-backup/out/test/resources/log4j.properties M java/kudu-backup/src/main/protobuf/backup.proto M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupRDD.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/TableMetadata.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala 6 files changed, 39 insertions(+), 14 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/59/10959/1 -- To view, visit http://gerrit.cloudera.org:8080/10959 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I7f829065878873568d17ec5e54126da3f6716485 Gerrit-Change-Number: 10959 Gerrit-PatchSet: 1 Gerrit-Owner: Tony Foerster
[kudu-CR] WIP Add error handling to KuduBackup DONT BUILD KuduBackup will attempt to backup every table. If any one table fails the entire job will fail. All exceptions are logged when they happen.
Tony Foerster has posted comments on this change. ( http://gerrit.cloudera.org:8080/10941 ) Change subject: WIP Add error handling to KuduBackup DONT_BUILD KuduBackup will attempt to backup every table. If any one table fails the entire job will fail. All exceptions are logged when they happen. .. Patch Set 1: Ignore the 'KuduRestore' changes, that's even more half-baked than the KuduBackup (and not tested) -- To view, visit http://gerrit.cloudera.org:8080/10941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 Gerrit-Change-Number: 10941 Gerrit-PatchSet: 1 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tony Foerster Gerrit-Comment-Date: Fri, 13 Jul 2018 14:43:19 + Gerrit-HasComments: No
[kudu-CR] Change yetus dependency to compile in spark modules
Tony Foerster has posted comments on this change. ( http://gerrit.cloudera.org:8080/10927 ) Change subject: Change yetus dependency to compile in spark modules .. Patch Set 2: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/10927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I16867a75b7f29f57c557449f94e52439cb9642c9 Gerrit-Change-Number: 10927 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Tony Foerster Gerrit-Comment-Date: Fri, 13 Jul 2018 14:42:13 + Gerrit-HasComments: No
[kudu-CR] WIP Add error handling to KuduBackup DONT BUILD KuduBackup will attempt to backup every table. If any one table fails the entire job will fail. All exceptions are logged when they happen.
Tony Foerster has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10941 Change subject: WIP Add error handling to KuduBackup DONT_BUILD KuduBackup will attempt to backup every table. If any one table fails the entire job will fail. All exceptions are logged when they happen. .. WIP Add error handling to KuduBackup DONT_BUILD KuduBackup will attempt to backup every table. If any one table fails the entire job will fail. All exceptions are logged when they happen. A backup attempt can have 3 results: - Success - table was backed up - Exists - table backup 'metadata' directory already existed. Assuming this was a retry. - Failure - table failed to backup. All data At the end of the job lists of failures, successes, and existing backups are printed out. A lot of this code is still assuming the 'batch' backup mode. I think a lot of this will change when we start doing the incremental backups. TODO - - if 'overwrite' flag is specified, overwrite the 'metadata' folder. Not sure yet how to handle this on retries or if it's even a good idea. Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 --- M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala 4 files changed, 208 insertions(+), 46 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/41/10941/1 -- To view, visit http://gerrit.cloudera.org:8080/10941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I104fd604e12b70fcff9beace71ed4ef96b54d691 Gerrit-Change-Number: 10941 Gerrit-PatchSet: 1 Gerrit-Owner: Tony Foerster
[kudu-CR] Change yetus dependency to compile in spark modules
Tony Foerster has posted comments on this change. ( http://gerrit.cloudera.org:8080/10927 ) Change subject: Change yetus dependency to compile in spark modules .. Patch Set 1: Code-Review+1 (1 comment) http://gerrit.cloudera.org:8080/#/c/10927/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/10927/1//COMMIT_MSG@13 PS1, Line 13: change to scope : to compile Extra 'to' -- To view, visit http://gerrit.cloudera.org:8080/10927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I16867a75b7f29f57c557449f94e52439cb9642c9 Gerrit-Change-Number: 10927 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Tony Foerster Gerrit-Comment-Date: Thu, 12 Jul 2018 19:40:47 + Gerrit-HasComments: Yes
[kudu-CR] Test for backup/restore multiple tables
Tony Foerster has posted comments on this change. (
http://gerrit.cloudera.org:8080/10899 )
Change subject: Test for backup/restore multiple tables
..
Patch Set 2:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/10899/1/java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala
File
java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala:
http://gerrit.cloudera.org:8080/#/c/10899/1/java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala@97
PS1, Line 97: num
> Nit: maybe rename this to "numRows" to assign it some kind of semantic mean
Done
http://gerrit.cloudera.org:8080/#/c/10899/1/java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/TestContext.scala
File
java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/TestContext.scala:
http://gerrit.cloudera.org:8080/#/c/10899/1/java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/TestContext.scala@111
PS1, Line 111: val tableOptions: CreateTableOptions = {
: val bottom = schema.newPartialRow() // Unbounded.
: val middle = schema.newPartialRow()
: middle.addInt("key", 50)
> Could you parameterize this so that createTable() can also be used to creat
I couldn't parameterize it without ending up back at the KuduClient createTable
signature
kuduClient.createTable(tableName, schema, tableOptions)
I made the CreateTableOptions public and just used that.
http://gerrit.cloudera.org:8080/#/c/10899/1/java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/TestContext.scala@137
PS1, Line 137: targetTable: Kud
> This is now shadowing the existing class-level 'table' variable; could you
Done
--
To view, visit http://gerrit.cloudera.org:8080/10899
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I439e0d1c6904395ad382e3f059846b3b03a79af4
Gerrit-Change-Number: 10899
Gerrit-PatchSet: 2
Gerrit-Owner: Tony Foerster
Gerrit-Reviewer: Adar Dembo
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tony Foerster
Gerrit-Comment-Date: Thu, 12 Jul 2018 13:53:25 +
Gerrit-HasComments: Yes
[kudu-CR] Test for backup/restore multiple tables
Hello Kudu Jenkins, Adar Dembo, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10899 to look at the new patch set (#2). Change subject: Test for backup/restore multiple tables .. Test for backup/restore multiple tables This is a prerequisite for testing parallel backup/restore and error handling. - Extracted `createTable` method so it can be re-used - Added `tableName` argument to `insertRows` so it can be re-used Change-Id: I439e0d1c6904395ad382e3f059846b3b03a79af4 --- M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/KuduContextTest.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/KuduRDDTest.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/TestContext.scala 5 files changed, 44 insertions(+), 20 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/99/10899/2 -- To view, visit http://gerrit.cloudera.org:8080/10899 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I439e0d1c6904395ad382e3f059846b3b03a79af4 Gerrit-Change-Number: 10899 Gerrit-PatchSet: 2 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Kudu Jenkins
[kudu-CR] Fix flaky 'Random Backup and Restore' test
Tony Foerster has posted comments on this change. ( http://gerrit.cloudera.org:8080/10902 ) Change subject: Fix flaky 'Random Backup and Restore' test .. Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/10902/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/10902/1//COMMIT_MSG@21 PS1, Line 21: Not sure of the root cause, I'd expect the timetamp obtained : from the Kudu client to work. > I would too, but only if there were rows actually written; otherwise there' The row counts are always off by one: org.apache.kudu.backup.TestKuduBackup > Random Backup and Restore FAILED java.lang.AssertionError: expected:<191> but was:<190> http://gerrit.cloudera.org:8080/#/c/10902/1/java/kudu-backup/out/test/resources/log4j.properties File java/kudu-backup/out/test/resources/log4j.properties: PS1: > What does this file do? I'm asking because it's in a non-standard Maven loc My mistake, this should not have been committed. I'm not sure why the non-standard 'out' location, I thought it just hadn't been added to the .gitignore. Not sure what command I ran to create it. -- To view, visit http://gerrit.cloudera.org:8080/10902 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I51a6dbeeb064dff157609fdbbafe505668eb2b26 Gerrit-Change-Number: 10902 Gerrit-PatchSet: 1 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tony Foerster Gerrit-Comment-Date: Thu, 12 Jul 2018 13:29:10 + Gerrit-HasComments: Yes
[kudu-CR] Fix flaky 'Random Backup and Restore' test
Hello Kudu Jenkins, Adar Dembo, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10902 to look at the new patch set (#2). Change subject: Fix flaky 'Random Backup and Restore' test .. Fix flaky 'Random Backup and Restore' test Add time to the snapshot timestamp to make sure all records written to test table are read by the backup job. Job would fail ~1/3 of the time before. I also tried using a timestamp obtained from the KuduClient: ``` val snapshotTimestampMicros = HybridTimeUtil.HTTimestampToPhysicalAndLogical(kuduClient.getLastPropagatedTimestamp)(0) val snapshotTimestampMillis = snapshotTimestampMicros / 1000 ``` The above code would produce something within 1ms of the System timestamp. Not sure of the root cause, I'd expect the timetamp obtained from the Kudu client to work. Change-Id: I51a6dbeeb064dff157609fdbbafe505668eb2b26 --- M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala 1 file changed, 5 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/02/10902/2 -- To view, visit http://gerrit.cloudera.org:8080/10902 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I51a6dbeeb064dff157609fdbbafe505668eb2b26 Gerrit-Change-Number: 10902 Gerrit-PatchSet: 2 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Kudu Jenkins
[kudu-CR] Fix flaky 'Random Backup and Restore' test
Tony Foerster has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10902 Change subject: Fix flaky 'Random Backup and Restore' test .. Fix flaky 'Random Backup and Restore' test Add time to the snapshot timestamp to make sure all records written to test table are read by the backup job. Job would fail ~1/3 of the time before. I also tried using a timestamp obtained from the KuduClient: ``` val snapshotTimestampMicros = HybridTimeUtil.HTTimestampToPhysicalAndLogical(kuduClient.getLastPropagatedTimestamp)(0) val snapshotTimestampMillis = snapshotTimestampMicros / 1000 ``` The above code would produce something withing 1ms of the System timestamp. Not sure of the root cause, I'd expect the timetamp obtained from the Kudu client to work. Change-Id: I51a6dbeeb064dff157609fdbbafe505668eb2b26 --- A java/kudu-backup/out/test/resources/log4j.properties M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala 2 files changed, 28 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/02/10902/1 -- To view, visit http://gerrit.cloudera.org:8080/10902 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I51a6dbeeb064dff157609fdbbafe505668eb2b26 Gerrit-Change-Number: 10902 Gerrit-PatchSet: 1 Gerrit-Owner: Tony Foerster
[kudu-CR] Test for backup/restore multiple tables
Tony Foerster has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10899 Change subject: Test for backup/restore multiple tables .. Test for backup/restore multiple tables This is a prerequisite for testing parallel backup/restore and error handling. - Extracted `createTable` method so it can be re-used - Added `tableName` argument to `insertRows` so it can be re-used Change-Id: I439e0d1c6904395ad382e3f059846b3b03a79af4 --- M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/KuduContextTest.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/KuduRDDTest.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/TestContext.scala 5 files changed, 40 insertions(+), 16 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/99/10899/1 -- To view, visit http://gerrit.cloudera.org:8080/10899 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I439e0d1c6904395ad382e3f059846b3b03a79af4 Gerrit-Change-Number: 10899 Gerrit-PatchSet: 1 Gerrit-Owner: Tony Foerster
[kudu-CR] Read url-encoded table name on restore
Tony Foerster has posted comments on this change. ( http://gerrit.cloudera.org:8080/10880 ) Change subject: Read url-encoded table name on restore .. Patch Set 6: The build failure doesn't look to be related to the test (ClientTest.TestFailedDnsResolution) -- To view, visit http://gerrit.cloudera.org:8080/10880 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib028b9239c9e4aa71084757eb2dc16a3068de929 Gerrit-Change-Number: 10880 Gerrit-PatchSet: 6 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tony Foerster Gerrit-Comment-Date: Mon, 09 Jul 2018 18:06:54 + Gerrit-HasComments: No
[kudu-CR] Read url-encoded table name on restore
Tony Foerster has posted comments on this change. (
http://gerrit.cloudera.org:8080/10880 )
Change subject: Read url-encoded table name on restore
..
Patch Set 6:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/10880/6/java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala
File
java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala:
http://gerrit.cloudera.org:8080/#/c/10880/6/java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala@59
PS6, Line 59: test("Simple Backup and Restore Table Name With Special
Characters") {
I pulled the impala special character/url encoding test into its own test, the
Impala-style table name wasn't actually being created.
--
To view, visit http://gerrit.cloudera.org:8080/10880
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib028b9239c9e4aa71084757eb2dc16a3068de929
Gerrit-Change-Number: 10880
Gerrit-PatchSet: 6
Gerrit-Owner: Tony Foerster
Gerrit-Reviewer: Adar Dembo
Gerrit-Reviewer: Grant Henke
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tony Foerster
Gerrit-Comment-Date: Mon, 09 Jul 2018 15:05:39 +
Gerrit-HasComments: Yes
[kudu-CR] Read url-encoded table name on restore
Hello Kudu Jenkins, Adar Dembo, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10880 to look at the new patch set (#6). Change subject: Read url-encoded table name on restore .. Read url-encoded table name on restore Kudu table names are being url-encoded on backup. Read the url-encoded table names on restore. Change-Id: Ib028b9239c9e4aa71084757eb2dc16a3068de929 --- M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala 3 files changed, 21 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/80/10880/6 -- To view, visit http://gerrit.cloudera.org:8080/10880 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib028b9239c9e4aa71084757eb2dc16a3068de929 Gerrit-Change-Number: 10880 Gerrit-PatchSet: 6 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tony Foerster
[kudu-CR] Read url-encoded table name on restore
Hello Kudu Jenkins, Adar Dembo, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10880 to look at the new patch set (#5). Change subject: Read url-encoded table name on restore .. Read url-encoded table name on restore Kudu table names are being url-encoded on backup. Read the url-encoded table names on restore. Change-Id: Ib028b9239c9e4aa71084757eb2dc16a3068de929 --- M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala 3 files changed, 8 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/80/10880/5 -- To view, visit http://gerrit.cloudera.org:8080/10880 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib028b9239c9e4aa71084757eb2dc16a3068de929 Gerrit-Change-Number: 10880 Gerrit-PatchSet: 5 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tony Foerster
[kudu-CR] Read url-encoded table name on restore
Tony Foerster has posted comments on this change. (
http://gerrit.cloudera.org:8080/10880 )
Change subject: Read url-encoded table name on restore
..
Patch Set 4:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/10880/3/java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala
File
java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala:
http://gerrit.cloudera.org:8080/#/c/10880/3/java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala@44
PS3, Line 44: test("Simple Backup and Restore") {
> What's the effect of this override? What different code path is being teste
The override seems unnecessary. I moved the impala table name string into a
single test and added a comment
--
To view, visit http://gerrit.cloudera.org:8080/10880
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib028b9239c9e4aa71084757eb2dc16a3068de929
Gerrit-Change-Number: 10880
Gerrit-PatchSet: 4
Gerrit-Owner: Tony Foerster
Gerrit-Reviewer: Adar Dembo
Gerrit-Reviewer: Grant Henke
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tony Foerster
Gerrit-Comment-Date: Fri, 06 Jul 2018 18:59:56 +
Gerrit-HasComments: Yes
[kudu-CR] Read url-encoded table name on restore
Hello Kudu Jenkins, Adar Dembo, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10880 to look at the new patch set (#4). Change subject: Read url-encoded table name on restore .. Read url-encoded table name on restore Kudu table names are being url-encoded on backup. Read the url-encoded table names on restore. Change-Id: Ib028b9239c9e4aa71084757eb2dc16a3068de929 --- M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala 3 files changed, 8 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/80/10880/4 -- To view, visit http://gerrit.cloudera.org:8080/10880 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib028b9239c9e4aa71084757eb2dc16a3068de929 Gerrit-Change-Number: 10880 Gerrit-PatchSet: 4 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins
[kudu-CR] Read url-enconded hdfs path on restore
Hello Kudu Jenkins, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10880 to look at the new patch set (#2). Change subject: Read url-enconded hdfs path on restore .. Read url-enconded hdfs path on restore Backup locations are url encoded, while KuduRestore was trying to read the unmodified path. Change-Id: Ib028b9239c9e4aa71084757eb2dc16a3068de929 --- M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackup.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/TestContext.scala 4 files changed, 8 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/80/10880/2 -- To view, visit http://gerrit.cloudera.org:8080/10880 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib028b9239c9e4aa71084757eb2dc16a3068de929 Gerrit-Change-Number: 10880 Gerrit-PatchSet: 2 Gerrit-Owner: Tony Foerster Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins
