drcrallen commented on issue #6910: Bring GCP functionality to core functionality URL: https://github.com/apache/incubator-druid/issues/6910#issuecomment-457410561 Sure, the pain points are that I have extensions that do things like password providers using KMS, and things like the watermark extension. These use GCP clients which require a lot of duplicates of libraries somewhere in the class loaders. This makes Hadoop style indexing have problems because all of the libs get jumbled together when the hadoop jobs are run. Here is a rough example of the module I'm hoping can be in stock druid: ```java public class GcpModule implements DruidModule { @Override public List<? extends Module> getJacksonModules() { return ImmutableList.of(); } @Override public void configure(Binder binder) { // NOOP } @Provides @LazySingleton public HttpRequestInitializer getHttpRequestInitializer( HttpTransport transport, JsonFactory factory ) { try { return GoogleCredential .getApplicationDefault(transport, factory) .createScoped(Collections.singleton("https://www.googleapis.com/auth/cloud-platform";)); } catch (IOException e) { throw new RuntimeException("Unable to build authentication", e); } } @Provides @LazySingleton public HttpTransport getHttpTransport() { return new NetHttpTransport.Builder() .build(); } @Provides @LazySingleton public JsonFactory getJsonFactory() { return JacksonFactory.getDefaultInstance(); } } ``` This would allow the GCP api client, which has transitive guava and jackson dependencies, to be controlled at the master pom level rather than having their lib versions buried down in the extension. That makes guava and jackson version conflicts easier to manage, and allows extensions in house to not have to load the same jars multiple times.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
