[
https://issues.apache.org/jira/browse/CARBONDATA-297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15565878#comment-15565878
]
ASF GitHub Bot commented on CARBONDATA-297:
-------------------------------------------
Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/229#discussion_r82830846
--- Diff:
processing/src/main/java/org/apache/carbondata/processing/newflow/DataLoadProcessorStep.java
---
@@ -0,0 +1,40 @@
+package org.apache.carbondata.processing.newflow;
+
+import java.util.Iterator;
+
+import
org.apache.carbondata.processing.newflow.exception.CarbonDataLoadingException;
+
+/**
+ * This base interface for data loading. It can do transformation jobs as
per the implementation.
+ *
+ */
+public interface DataLoadProcessorStep {
+
+ /**
+ * The output meta for this step. The data returns from this step is as
per this meta.
+ * @return
+ */
+ DataField[] getOutput();
+
+ /**
+ * Intialization process for this step.
+ * @param configuration
+ * @param child
+ * @throws CarbonDataLoadingException
+ */
+ void intialize(CarbonDataLoadConfiguration configuration,
DataLoadProcessorStep child) throws
+ CarbonDataLoadingException;
+
+ /**
+ * Tranform the data as per the implemetation.
+ * @return Iterator of data
+ * @throws CarbonDataLoadingException
+ */
+ Iterator<Object[]> execute() throws CarbonDataLoadingException;
--- End diff --
I think it is better to return array of Iterator. Like
`Iterator<Object[]>[] execute()`
So that multiple blocks can be parallely processed to improve the
performance. Other wise lot of btrees need to be created at the end of data
loading.
@jackylk what do you think?
> 2. Add interfaces for data loading.
> -----------------------------------
>
> Key: CARBONDATA-297
> URL: https://issues.apache.org/jira/browse/CARBONDATA-297
> Project: CarbonData
> Issue Type: Sub-task
> Reporter: Ravindra Pesala
> Assignee: Ravindra Pesala
> Fix For: 0.2.0-incubating
>
>
> Add the major interface classes for data loading so that the following jiras
> can use this interfaces to implement it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
