dlmarion commented on issue #3499: URL: https://github.com/apache/accumulo/issues/3499#issuecomment-1598944945
I asked @dtspence to update https://github.com/dtspence/accumulo-jmh-test with the following: 1. you are creating 3 tablet servers, one test table, and 10,000 files. However, you are not adding splits to the test table, so it's very possible that there is only one metadata table tablet. You may want to consider setting the metadata table split size to something low so that it splits and adding splits to the test table. Maybe 100 splits with 100 files each? 2. if you make the number of files configurable, then you can run several times with different inputs (e.g. 100, 1000, 10000) to see what the curve looks like. 3. I created a non-streaming implementation of the getReferences() method (below). Can you also add a test for this method? ``` public Iterator<Reference> getReferenceIter() { final IteratorChain<Reference> iterChain = new IteratorChain<>(); Iterable<TabletMetadata> tmIter = null; if (level == Ample.DataLevel.ROOT) { tmIter = Collections.singleton(context.getAmple().readTablet(RootTable.EXTENT, DIR, FILES, SCANS)); } else { tmIter = TabletsMetadata.builder(context).scanTable(level.metaTable()) .checkConsistency().fetch(DIR, FILES, SCANS).build(); } tmIter.forEach(tabletMetadata -> { int fileCount = tabletMetadata.getFiles().size() + tabletMetadata.getScans().size() + ((tabletMetadata.getDirName() == null) ? 0 : 1); List<Reference> referencedFiles = new ArrayList<>(fileCount); tabletMetadata.getFiles().forEach(stf -> referencedFiles.add(new ReferenceFile(tabletMetadata.getTableId(), stf.getMetaUpdateDelete()))); tabletMetadata.getScans().forEach(stf -> referencedFiles.add(new ReferenceFile(tabletMetadata.getTableId(), stf.getMetaUpdateDelete()))); if (tabletMetadata.getDirName() != null) { referencedFiles.add(new ReferenceDirectory(tabletMetadata.getTableId(), tabletMetadata.getDirName())); } iterChain.addIterator(referencedFiles.iterator()); }); List<ReferenceFile> ssrefs = context.getAmple().getScanServerFileReferences() .map(sfr -> new ReferenceFile(sfr.getTableId(), sfr.getPathStr())).collect(Collectors.toList()); iterChain.addIterator(ssrefs.iterator()); return iterChain; } ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
