Dear all, Pseudo-distributed mode is still used since I am still coding.
When scanning a table, I noticed that a single thread was much faster than each one in a multi-threads module. For example, the following method can be done in 2 or 3ms with a single thread. If 30 threads execute the method together, it takes each thread about 150ms to execute. Each of the threads can get a HTableInterface from HTablePool. So I think the performance should not be so low. Maybe the pseudo-distributed mode causes the problem? Thanks so much! Best regards, Bing public Set<String> GetOutgoingHHNeighborKeys(String hubKey, String groupKey, int timingScale) { List<Filter> hhNeighborFilterList = new ArrayList<Filter>(); SingleColumnValueFilter hubKeyFilter = new SingleColumnValueFilter(NeighborStructure.HUB_HUB_NEIGHBOR_FAMILY, NeighborStructure.HUB_HUB_NEIGHBOR_HUB_KEY_COLUMN, CompareFilter.CompareOp.EQUAL, new SubstringComparator(hubKey)); hubKeyFilter.setFilterIfMissing(true); hhNeighborFilterList.add(hubKeyFilter); SingleColumnValueFilter groupKeyFilter = new SingleColumnValueFilter(NeighborStructure.HUB_HUB_NEIGHBOR_FAMILY, NeighborStructure.HUB_HUB_NEIGHBOR_GROUP_KEY_COLUMN, CompareFilter.CompareOp.EQUAL, new SubstringComparator(groupKey)); groupKeyFilter.setFilterIfMissing(true); hhNeighborFilterList.add(groupKeyFilter); SingleColumnValueFilter timingScaleFilter = new SingleColumnValueFilter(NeighborStructure.HUB_HUB_NEIGHBOR_FAMILY, NeighborStructure.HUB_HUB_NEIGHBOR_TIMING_SCALE_COLUMN, CompareFilter.CompareOp.EQUAL, new BinaryComparator(Bytes.toBytes(timingScale))); timingScaleFilter.setFilterIfMissing(true); hhNeighborFilterList.add(timingScaleFilter); FilterList hhNeighborFilter = new FilterList(hhNeighborFilterList); Scan scan = new Scan(); scan.setFilter(hhNeighborFilter); scan.setCaching(Parameters.CACHING_SIZE); scan.setBatch(Parameters.BATCHING_SIZE); Set<String> neighborKeySet = Sets.newHashSet(); String qualifier; try { ResultScanner scanner = this.neighborTable.getScanner(scan); for (Result result : scanner) { for (KeyValue kv : result.raw()) { qualifier = Bytes.toString(kv.getQualifier()); if (qualifier.equals(NeighborStructure.HUB_HUB_NEIGHBOR_NODE_KEY_STRING_COLUMN)) { neighborKeySet.add(Bytes.toString(kv.getValue())); } } } scanner.close(); } catch (IOException e) { e.printStackTrace(); } return neighborKeySet; }