djkevincr commented on issue #161: GORA-565: Enable Spark in Unit Tests URL: https://github.com/apache/gora/pull/161#issuecomment-487840780 ``` public <K, V extends Persistent> Configuration generateOutputConf(DataStore<K, V> dataStore) throws IOException { Configuration hadoopConf = ((Configurable) dataStore).getConf(); // Configuration hadoopConf = new Configuration(); - previously it was GoraMapReduceUtils.setIOSerializations(hadoopConf, true); Job job = Job.getInstance(hadoopConf); return generateOutputConf(job, dataStore.getClass(), dataStore.getKeyClass(), dataStore.getPersistentClass()); } ``` To properly fix the issue, I think GoraSparkEngine class method generateOutputConf should be changed to the above. ( Check java single line comment ) Basically we should reuse the Hadoop conf created at GoraMongodbTestDriver through out entire tests. Initial test startup phase, we set these mongo server details to Hadoop conf created. I am not sure whether there exist a case which returns dataStore conf as null. ( Eg:- de serilizations ) so we can do something similar to below. This code is extreacted from the GoraSparkEngine class method initialize. Can you please address the changes and update the PR? ``` if ((dataStore instanceof Configurable) && ((Configurable) dataStore).getConf() != null) { hadoopConf = ((Configurable) dataStore).getConf(); } else { hadoopConf = new Configuration(); } ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
