[
https://issues.apache.org/jira/browse/GOBBLIN-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenqin Wang resolved GOBBLIN-1323.
----------------------------------
Resolution: Fixed
Issue resolved by pul request #3156
https://github.com/apache/incubator-gobblin/pull/3156
> make columnListInQuery in RestApiExtractor overridable in child class for
> Dynamics Extractor
> --------------------------------------------------------------------------------------------
>
> Key: GOBBLIN-1323
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1323
> Project: Apache Gobblin
> Issue Type: Bug
> Components: gobblin-core
> Reporter: Wenqin Wang
> Assignee: Abhishek Tiwari
> Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Recently we've discovered that in our Dynamics ingestion pipeline, we would
> not remove invalid fields in our query before pulling data for dynamics
> entities, and therefore gobblin ingestion would fail due to field not found
> error. This feature is implemented for Salesforce in RestApiExtrator.java,
> and the logic is specifically tailored for Salesforce. Therefore i'm
> proposing some changes and hope to get Gobblin dev's guidance and reviews.
> For Dynamics, to enable this feature, here is the solution that I think works
> the best:
> since the query looks different from Salesforce query, we need to change the
> way we extract fields from client-specified-query. I will make a protected
> method in RestApiExtractor.java to wrap this Utils.getColumnListFromQuery,
> and override it in DynamicsExtractor.java (the child class) to write specific
> logic for dynamics use case. After that, we can override the buildDataQuery
> method in DynamicsExtractor.java ourselves to replace user-specified-query
> with validated fields. This way it would not affect other extractors.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
