Clemens Valiente has posted comments on this change. ( http://gerrit.cloudera.org:8080/8921 )
Change subject: KUDU-2249 give the TableRecordReader their own KuduClient to use. ...................................................................... Patch Set 1: (1 comment) Hi David, I discussed this on the dev mailing list here: http://mail-archives.apache.org/mod_mbox/kudu-dev/201712.mbox/%3CAM0PR0502MB405140DD23048A17522BB55C9C070%40AM0PR0502MB4051.eurprd05.prod.outlook.com%3E The problem is the getInputSplit() closing the client that the Reader want to use. There's no real clean way of sharing the client and closing it properly. Due to the mapreduce architecture, each tablet usually will read from a separate Map Container and thus need its own client anyway. This just fixes a bug in the rare scenario that the getinputsplit and record reader are executed in one container. http://gerrit.cloudera.org:8080/#/c/8921/1/java/kudu-mapreduce/src/main/java/org/apache/kudu/mapreduce/KuduTableInputFormat.java File java/kudu-mapreduce/src/main/java/org/apache/kudu/mapreduce/KuduTableInputFormat.java: http://gerrit.cloudera.org:8080/#/c/8921/1/java/kudu-mapreduce/src/main/java/org/apache/kudu/mapreduce/KuduTableInputFormat.java@253 PS1, Line 253: private KuduClient getKuduClient() { : : String masterAddresses = conf.get(MASTER_ADDRESSES_KEY); : this.operationTimeoutMs = conf.getLong(OPERATION_TIMEOUT_MS_KEY, : AsyncKuduClient.DEFAULT_OPERATION_TIMEOUT_MS); : KuduClient kuduClient = new KuduClient.KuduClientBuilder(masterAddresses) : .defaultOperationTimeoutMs(operationTimeoutMs) : .build(); : KuduTableMapReduceUtil.importCredentialsFromCurrentSubject(kuduClient); : return kuduClient; : } > Kudu clients cache state internally, like tablet locations. Changing this t Hi David, I discussed this on the dev mailing list here: http://mail-archives.apache.org/mod_mbox/kudu-dev/201712.mbox/%3CAM0PR0502MB405140DD23048A17522BB55C9C070%40AM0PR0502MB4051.eurprd05.prod.outlook.com%3E The problem is the getInputSplit() closing the client that the Reader want to use. There's no real clean way of sharing the client and closing it properly. Due to the mapreduce architecture, each tablet usually will read from a separate Map Container and thus need its own client anyway. This just fixes a bug in the rare scenario that the getinputsplit and record reader are executed in one container. -- To view, visit http://gerrit.cloudera.org:8080/8921 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I24f45ee9253790c5348cabd0afe6c6a4b6d3f3d4 Gerrit-Change-Number: 8921 Gerrit-PatchSet: 1 Gerrit-Owner: Clemens Valiente <clemens.valie...@gmail.com> Gerrit-Reviewer: Clemens Valiente <clemens.valie...@gmail.com> Gerrit-Reviewer: David Ribeiro Alves <davidral...@gmail.com> Gerrit-Reviewer: Kudu Jenkins Gerrit-Comment-Date: Wed, 03 Jan 2018 08:39:45 +0000 Gerrit-HasComments: Yes
