Github user hyunsik commented on a diff in the pull request:
https://github.com/apache/tajo/pull/232#discussion_r20282144
--- Diff:
tajo-storage/src/main/java/org/apache/tajo/storage/StorageManager.java ---
@@ -76,20 +98,169 @@
private static final Map<Class<?>, Constructor<?>> CONSTRUCTOR_CACHE =
new ConcurrentHashMap<Class<?>, Constructor<?>>();
- protected abstract void storageInit() throws IOException ;
- public abstract void createTable(TableDesc tableDesc) throws IOException;
+ public StorageManager(StoreType storeType) {
+ this.storeType = storeType;
+ }
+
+ /**
+ * Initialize storage manager.
+ * @throws IOException
+ */
+ protected abstract void storageInit() throws IOException;
+
+ /**
+ * This method is called after executing "CREATE TABLE" statement.
+ * If a storage is a file based storage, a storage manager may create
directory.
+ *
+ * @param tableDesc Table description which is created.
+ * @param ifNotExists Creates the table only when the table does not
exist.
+ * @throws IOException
+ */
+ public abstract void createTable(TableDesc tableDesc, boolean
ifNotExists) throws IOException;
+
+ /**
+ * This method is called after executing "DROP TABLE" statement with the
'PURGE' option
+ * which is the option to delete all the data.
+ *
+ * @param tableDesc
+ * @throws IOException
+ */
public abstract void purgeTable(TableDesc tableDesc) throws IOException;
- public abstract List<Fragment> getSplits(String fragmentId, TableDesc
tableDesc) throws IOException;
+ /**
+ * Returns the splits that will serve as input for the scan tasks. The
+ * number of splits matches the number of regions in a table.
+ * @param fragmentId The table name or previous ExecutionBlockId
+ * @param tableDesc The table description for the target data.
+ * @param scanNode The logical node for scanning.
+ * @return The list of input fragments.
+ * @throws IOException
+ */
+ public abstract List<Fragment> getSplits(String fragmentId, TableDesc
tableDesc,
+ ScanNode scanNode) throws
IOException;
+
+ /**
+ * It returns the splits that will serve as input for the non-forward
query scanner such as 'select * from table1'.
+ * The result list should be small. If there is many fragments for
scanning, TajoMaster uses the paging navigation.
+ * @param tableDesc The table description for the target data.
+ * @param currentPage The current page number within the entire list.
+ * @param numFragments The number of fragments in the result.
+ * @return The list of input fragments.
+ * @throws IOException
+ */
+ public abstract List<Fragment> getNonForwardSplit(TableDesc tableDesc,
int currentPage, int numFragments)
+ throws IOException;
+
+ /**
+ * It returns the storage property.
+ * @return The storage property
+ */
+ public abstract StorageProperty getStorageProperty();
+
+ /**
+ * Release storage manager resource
+ */
+ public abstract void closeStorageManager();
+
+ /**
+ * It is called by a Repartitioner for range shuffling when the
SortRangeType of SortNode is USING_STORAGE_MANAGER.
+ * In general Repartitioner determines the partition range using
previous output statistics data.
+ * In the special cases, such as HBase Repartitioner uses the result of
this method.
+ *
+ * @param queryContext The current query context which contains query
properties.
+ * @param tableDesc The table description for the target data.
+ * @param inputSchema The input schema
+ * @param sortSpecs The sort specification that contains the sort column
and sort order.
+ * @return The list of sort ranges.
+ * @throws IOException
+ */
+ public abstract TupleRange[] getInsertSortRanges(OverridableConf
queryContext, TableDesc tableDesc,
+ Schema inputSchema,
SortSpec[] sortSpecs,
+ TupleRange dataRange)
throws IOException;
+
+ /**
+ * This method is called before executing 'INSERT' or 'CREATE TABLE as
SELECT'.
+ * In general Tajo creates the target table after finishing the final
sub-query of CATS.
+ * But In the special cases, such as HBase INSERT or CAST query uses the
target table information.
+ * That kind of the storage should implements the logic related to
creating table in this method.
+ *
+ * @param node The child node of the root node.
+ * @throws IOException
+ */
+ public abstract void beforeInsertOrCATS(LogicalNode node) throws
IOException;
+
+ /**
+ * It is called when the query failed.
+ * Each storage manager should implement to be processed when the query
fails in this method.
+ *
+ * @param node The child node of the root node.
+ * @throws IOException
+ */
+ public abstract void queryFailed(LogicalNode node) throws IOException;
--- End diff --
I'd like to suggest ```rollbackOutputCommit```.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
