> On Jan. 21, 2016, 3:30 a.m., Colin Ma wrote: > > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/hdfsWriter/GenericHdfsWriter.java, > > line 29 > > <https://reviews.apache.org/r/42327/diff/7/?file=1202469#file1202469line29> > > > > Is it possible to pass the LoaderContext instead of Schema? > > Schema is only used for parquetWriter, if add more writer in the > > future, all the specific parameters can be stored in LoaderContext.
im not sure if this is something we need to do yet. i generally like to avoid the context objects because they add a layer of complexity. i would wait to see if this argument list gets out of hand and then we could use it. > On Jan. 21, 2016, 3:30 a.m., Colin Ma wrote: > > connector/connector-hdfs/pom.xml, line 79 > > <https://reviews.apache.org/r/42327/diff/7/?file=1202464#file1202464line79> > > > > The dependency can be add to pom.xml and the version can be added as > > ${parquet.version}. > > > > connector/connector-hdfs/pom.xml and test/pom.xml can just use the > > dependency without version. thanks! > On Jan. 21, 2016, 3:30 a.m., Colin Ma wrote: > > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java, > > line 34 > > <https://reviews.apache.org/r/42327/diff/7/?file=1202465#file1202465line34> > > > > For these org.apache.haddop.mapred.*, is it possible to use > > org.apache.haddop.mapreduce.* ? thanks! > On Jan. 21, 2016, 3:30 a.m., Colin Ma wrote: > > connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/AVROIntermediateDataFormat.java, > > line 66 > > <https://reviews.apache.org/r/42327/diff/7/?file=1202475#file1202475line66> > > > > It can be deleted. thanks! > On Jan. 21, 2016, 3:30 a.m., Colin Ma wrote: > > test/src/test/java/org/apache/sqoop/integration/connector/hdfs/ParquetTest.java, > > line 60 > > <https://reviews.apache.org/r/42327/diff/7/?file=1202477#file1202477line60> > > > > This method should be only for fromParquetTest, move this into > > fromParquetTest will be better. thanks! > On Jan. 21, 2016, 3:30 a.m., Colin Ma wrote: > > test/src/test/java/org/apache/sqoop/integration/connector/hdfs/ParquetTest.java, > > line 63 > > <https://reviews.apache.org/r/42327/diff/7/?file=1202477#file1202477line63> > > > > This test is not in group "slow", is it necessary to add (alwaysRun = > > true)? thanks! - Abraham ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/42327/#review115545 ----------------------------------------------------------- On Jan. 21, 2016, 7:06 p.m., Abraham Fine wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/42327/ > ----------------------------------------------------------- > > (Updated Jan. 21, 2016, 7:06 p.m.) > > > Review request for Sqoop. > > > Bugs: SQOOP-2788 > https://issues.apache.org/jira/browse/SQOOP-2788 > > > Repository: sqoop-sqoop2 > > > Description > ------- > > read and write parquet in hdfsconnector > > > Diffs > ----- > > connector/connector-hdfs/pom.xml 599631418ca63cc43d645c1ee1e7a73dc824b313 > > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java > a813c479a07d68e14ed49936f642e762e5b37437 > > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java > 774221aaf5c8cdb8d26ca108fae239598b42229b > > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartition.java > 644de60581faf90ceb2fcef8d3e0544067791fcc > > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToFormat.java > 27d121f529ecb4d5bd79e2b8c74ab8f7cc15fb10 > > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/hdfsWriter/GenericHdfsWriter.java > 2ccccc4a94a582c8b47ccdefa523d1fd1632e627 > > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/hdfsWriter/HdfsParquetWriter.java > PRE-CREATION > > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/hdfsWriter/HdfsSequenceWriter.java > 75c2e7ef192d7d9628e622cc3c5ef176e33a73d0 > > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/hdfsWriter/HdfsTextWriter.java > 78cf9732fdb89689b04d43e4af70ca5a43732dbf > > connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java > 11fcef2a38209c79928f582cf8aa03e889247f22 > > connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopAvroUtils.java > 985149cbb0d28b55a19d17076d996364d7f2ae90 > > connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/AVROIntermediateDataFormat.java > d78fa8b72ecfe62eeec240e01597e7f2a7e4dd76 > pom.xml cb8a973abc96af1de905cebd80d30177cbaf1cb4 > test/pom.xml 644a9c7dbc746d4a3268532bdcf0babd4faaafba > > test/src/test/java/org/apache/sqoop/integration/connector/hdfs/ParquetTest.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/42327/diff/ > > > Testing > ------- > > integration tests pass > > > Thanks, > > Abraham Fine > >
