Review Request 19821: TAJO-717: Improve file splitting for large number of splits

Sat, 29 Mar 2014 03:52:29 -0700 

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19821/
-----------------------------------------------------------

Review request for Tajo.


Bugs: TAJO-717
    https://issues.apache.org/jira/browse/TAJO-717


Repository: tajo


Description
-------

In currently, The storageManager invoke the getFileBlockStorageLocations() per 
input path, it occurred too many rpc to the associated datanodes


Diffs
-----

  
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/Repartitioner.java
 b2adaa4a2623b49022f4dac716dbf5457e93b71d 
  tajo-storage/pom.xml 9f144bb206820d23238532f158b511974f518592 
  
tajo-storage/src/main/java/org/apache/tajo/storage/AbstractStorageManager.java 
a7ed9817749787bfddcb719913ab03021aa19c6c 
  tajo-storage/src/main/java/org/apache/tajo/storage/CSVFile.java 
116e25ce6df8eb5d8c70ef68e2d78a17dca1ef5b 
  tajo-storage/src/main/java/org/apache/tajo/storage/fragment/FileFragment.java 
ea8bf9f16b93ac5d8f32bcbadd5e4359dac8ad9b 
  tajo-storage/src/main/java/org/apache/tajo/storage/rcfile/RCFile.java 
2fd34554c2905ecb593555dd463a7e8290e4d0e6 
  tajo-storage/src/test/java/org/apache/tajo/storage/TestStorageManager.java 
083670a27545c95b56dd2bece8d1c0620593a6f8 

Diff: https://reviews.apache.org/r/19821/diff/


Testing
-------


Thanks,

Jinho Kim

Reply via email to