----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18139/ -----------------------------------------------------------
Review request for DataFu. Repository: datafu Description ------- This migrates the build system from Ant to Gradle. The JAR that is generated should be exactly the same as that generated by Ant. I've moved all the Pig UDF source code to datafu-pig. So it may look like a lot of changes, but these are just file moves. This should give us a good foundation to start adding new projects, like hive, crunch, etc. Here are the things you want to pay attention to: 1) gradlew is a wrapper script that runs gradle. I just copied this from Samza. 2) in the root directory, build.gradle, settings.gradle, and gradle.properties provide basic config and setup 3) all code under src and test was moved to datafu-pig 4) the gradle directory has some common configuration 5) in the datafu-directory, the build.gradle has the equivalent of the original build.xml I've left the old build.xml file for reference, but I want to remove this and any other remnants of the old system. Diffs ----- .gitignore 81e9ae8 .project 022f4a7 HEADER PRE-CREATION README.md 8b3726c build-plugin/.gitignore PRE-CREATION build-plugin/build.gradle PRE-CREATION build-plugin/src/main/java/org/adrianwalker/multilinestring/EcjMultilineProcessor.java PRE-CREATION build-plugin/src/main/java/org/adrianwalker/multilinestring/JavacMultilineProcessor.java PRE-CREATION build-plugin/src/main/java/org/adrianwalker/multilinestring/Multiline.java PRE-CREATION build-plugin/src/main/java/org/adrianwalker/multilinestring/MultilineProcessor.java PRE-CREATION build-plugin/src/main/resources/META-INF/services/javax.annotation.processing.Processor PRE-CREATION build.gradle PRE-CREATION datafu-pig/.gitignore PRE-CREATION datafu-pig/build.gradle PRE-CREATION datafu-pig/src/main/java/datafu/pig/bags/AppendToBag.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/bags/BagConcat.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/bags/BagGroup.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/bags/BagLeftOuterJoin.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/bags/BagSplit.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/bags/CountEach.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/bags/DistinctBy.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/bags/EmptyBagToNull.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/bags/EmptyBagToNullFields.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/bags/Enumerate.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/bags/FirstTupleFromBag.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/bags/NullToEmptyBag.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/bags/PrependToBag.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/bags/ReverseEnumerate.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/bags/UnorderedPairs.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/bags/package-info.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/geo/HaversineDistInMiles.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/geo/package-info.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/hash/MD5.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/hash/SHA.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/hash/package-info.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/linkanalysis/PageRank.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/linkanalysis/PageRankImpl.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/linkanalysis/ProgressIndicator.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/linkanalysis/package-info.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/random/RandInt.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/random/RandomUUID.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/random/package-info.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/sampling/Reservoir.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/sampling/ReservoirSample.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/sampling/SampleByKey.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/sampling/ScoredTuple.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/sampling/SimpleRandomSample.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/sampling/SimpleRandomSampleWithReplacementElect.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/sampling/SimpleRandomSampleWithReplacementVote.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/sampling/WeightedReservoirSample.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/sampling/WeightedSample.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/sampling/package-info.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/sessions/SessionCount.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/sessions/Sessionize.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/sessions/package-info.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/sets/SetDifference.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/sets/SetIntersect.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/sets/SetOperationsBase.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/sets/SetUnion.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/sets/package-info.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/stats/DoubleVAR.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/stats/FloatVAR.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/stats/HyperLogLogPlusPlus.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/stats/IntVAR.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/stats/LongVAR.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/stats/MarkovPairs.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/stats/Median.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/stats/Quantile.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/stats/QuantileUtil.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/stats/StreamingMedian.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/stats/StreamingQuantile.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/stats/VAR.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/stats/WilsonBinConf.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/stats/entropy/ChaoShenEntropyEstimator.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/stats/entropy/EmpiricalEntropyEstimator.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/stats/entropy/Entropy.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/stats/entropy/EntropyEstimator.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/stats/entropy/EntropyUtil.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/stats/entropy/StreamingCondEntropy.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/stats/entropy/StreamingEntropy.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/stats/package-info.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/text/opennlp/CachedFile.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/text/opennlp/POSTag.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/text/opennlp/SentenceDetect.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/text/opennlp/TokenizeME.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/text/opennlp/TokenizeSimple.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/text/opennlp/TokenizeWhitespace.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/urls/UserAgentClassify.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/urls/package-info.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/util/AliasableEvalFunc.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/util/Assert.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/util/AssertUDF.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/util/BoolToInt.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/util/Coalesce.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/util/ContextualEvalFunc.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/util/DataFuException.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/util/FieldNotFound.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/util/In.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/util/InUDF.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/util/IntToBool.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/util/SimpleEvalFunc.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/util/TransposeTupleToBag.java PRE-CREATION datafu-pig/src/main/java/datafu/pig/util/package-info.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/PigTests.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/bags/BagTests.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/geo/GeoTests.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/hash/HashTests.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/linkanalysis/PageRankImplTests.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/linkanalysis/PageRankTests.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/random/NumberTests.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/random/UUIDTests.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/sampling/SamplingTests.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/sampling/SimpleRandomSampleTest.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/sampling/SimpleRandomSampleTestOld.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/sampling/SimpleRandomSampleWithReplacementTest.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/sampling/WeightedReservoirSamplingTests.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/sessions/SessionTests.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/sets/SetTests.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/stats/EstimationTests.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/stats/MarkovPairTests.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/stats/QuantileTests.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/stats/VARTests.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/stats/WilsonBinConfTests.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/stats/entropy/AbstractEntropyTests.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/stats/entropy/EntropyTests.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/stats/entropy/StreamingChaoShenEntropyTests.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/stats/entropy/StreamingEmpiricalCondEntropyTests.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/stats/entropy/StreamingEmpiricalEntropyTests.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/text/NLPTests.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/urls/UserAgentTest.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/util/AssertTests.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/util/CoalesceTests.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/util/InTests.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/util/IntBoolConversionPigTests.java PRE-CREATION datafu-pig/src/test/java/datafu/test/pig/util/TransposeTest.java PRE-CREATION gradle.properties PRE-CREATION gradle/buildscript.gradle PRE-CREATION gradle/dependency-versions.gradle PRE-CREATION plugin/java/org/adrianwalker/multilinestring/EcjMultilineProcessor.java 7dbfe9a plugin/java/org/adrianwalker/multilinestring/JavacMultilineProcessor.java 39aa24e plugin/java/org/adrianwalker/multilinestring/Multiline.java c29d3ef plugin/java/org/adrianwalker/multilinestring/MultilineProcessor.java 9abdba5 settings.gradle PRE-CREATION src/java/datafu/pig/bags/AppendToBag.java 55c9e76 src/java/datafu/pig/bags/BagConcat.java 290f44b src/java/datafu/pig/bags/BagGroup.java 7b6c2e2 src/java/datafu/pig/bags/BagLeftOuterJoin.java ba6bc11 src/java/datafu/pig/bags/BagSplit.java 74d39d0 src/java/datafu/pig/bags/CountEach.java cfb0152 src/java/datafu/pig/bags/DistinctBy.java 0bbb0e6 src/java/datafu/pig/bags/EmptyBagToNull.java 5524ad5 src/java/datafu/pig/bags/EmptyBagToNullFields.java 6933c28 src/java/datafu/pig/bags/Enumerate.java 8a0d072 src/java/datafu/pig/bags/FirstTupleFromBag.java 1f24984 src/java/datafu/pig/bags/NullToEmptyBag.java 09fffb3 src/java/datafu/pig/bags/PrependToBag.java 9292871 src/java/datafu/pig/bags/ReverseEnumerate.java c86ffcf src/java/datafu/pig/bags/UnorderedPairs.java a1d149e src/java/datafu/pig/bags/package-info.java 214b837 src/java/datafu/pig/geo/HaversineDistInMiles.java d1e3988 src/java/datafu/pig/geo/package-info.java 12b27b0 src/java/datafu/pig/hash/MD5.java b7b51da src/java/datafu/pig/hash/SHA.java ff859e5 src/java/datafu/pig/hash/package-info.java 320a029 src/java/datafu/pig/linkanalysis/PageRank.java 80ff567 src/java/datafu/pig/linkanalysis/PageRankImpl.java 5d0b932 src/java/datafu/pig/linkanalysis/ProgressIndicator.java 3dc54c5 src/java/datafu/pig/linkanalysis/package-info.java 2c6c078 src/java/datafu/pig/random/RandInt.java de89c4a src/java/datafu/pig/random/RandomUUID.java d63f4cf src/java/datafu/pig/random/package-info.java 8c7750d src/java/datafu/pig/sampling/Reservoir.java 09b4c95 src/java/datafu/pig/sampling/ReservoirSample.java 48deaad src/java/datafu/pig/sampling/SampleByKey.java 90ea576 src/java/datafu/pig/sampling/ScoredTuple.java c793584 src/java/datafu/pig/sampling/SimpleRandomSample.java 8e8debf src/java/datafu/pig/sampling/SimpleRandomSampleWithReplacementElect.java a59816a src/java/datafu/pig/sampling/SimpleRandomSampleWithReplacementVote.java 598e58c src/java/datafu/pig/sampling/WeightedReservoirSample.java 92af6a3 src/java/datafu/pig/sampling/WeightedSample.java a77164d src/java/datafu/pig/sampling/package-info.java 0c7d8bd src/java/datafu/pig/sessions/SessionCount.java fe68888 src/java/datafu/pig/sessions/Sessionize.java 52d159b src/java/datafu/pig/sessions/package-info.java 686fbf0 src/java/datafu/pig/sets/SetDifference.java b6469e9 src/java/datafu/pig/sets/SetIntersect.java 0ba586e src/java/datafu/pig/sets/SetOperationsBase.java c9997f8 src/java/datafu/pig/sets/SetUnion.java 49fd99c src/java/datafu/pig/sets/package-info.java 2c5b087 src/java/datafu/pig/stats/DoubleVAR.java ab8eed2 src/java/datafu/pig/stats/FloatVAR.java 8a697e0 src/java/datafu/pig/stats/HyperLogLogPlusPlus.java 0ebd94f src/java/datafu/pig/stats/IntVAR.java 0f34d8e src/java/datafu/pig/stats/LongVAR.java 561f15a src/java/datafu/pig/stats/MarkovPairs.java d3d7632 src/java/datafu/pig/stats/Median.java e33a84e src/java/datafu/pig/stats/Quantile.java 6fd42d3 src/java/datafu/pig/stats/QuantileUtil.java c6fd36a src/java/datafu/pig/stats/StreamingMedian.java c4c3be4 src/java/datafu/pig/stats/StreamingQuantile.java e4a65b4 src/java/datafu/pig/stats/VAR.java 6f22f25 src/java/datafu/pig/stats/WilsonBinConf.java 1448611 src/java/datafu/pig/stats/entropy/ChaoShenEntropyEstimator.java bfb7398 src/java/datafu/pig/stats/entropy/EmpiricalEntropyEstimator.java 3f0acab src/java/datafu/pig/stats/entropy/Entropy.java a074bc0 src/java/datafu/pig/stats/entropy/EntropyEstimator.java 336ef86 src/java/datafu/pig/stats/entropy/EntropyUtil.java 685f3e2 src/java/datafu/pig/stats/entropy/StreamingCondEntropy.java 67ad4d3 src/java/datafu/pig/stats/entropy/StreamingEntropy.java 707e690 src/java/datafu/pig/stats/package-info.java 4c100e7 src/java/datafu/pig/text/opennlp/CachedFile.java 5832c81 src/java/datafu/pig/text/opennlp/POSTag.java fb17c63 src/java/datafu/pig/text/opennlp/SentenceDetect.java 50537fd src/java/datafu/pig/text/opennlp/TokenizeME.java f1f4257 src/java/datafu/pig/text/opennlp/TokenizeSimple.java cea48b4 src/java/datafu/pig/text/opennlp/TokenizeWhitespace.java 8efafb0 src/java/datafu/pig/urls/UserAgentClassify.java 4040fea src/java/datafu/pig/urls/package-info.java 332beda src/java/datafu/pig/util/AliasableEvalFunc.java ee2c3f3 src/java/datafu/pig/util/Assert.java a258915 src/java/datafu/pig/util/AssertUDF.java 16f9247 src/java/datafu/pig/util/BoolToInt.java 5ec8d3a src/java/datafu/pig/util/Coalesce.java f8e25f4 src/java/datafu/pig/util/ContextualEvalFunc.java c534b77 src/java/datafu/pig/util/DataFuException.java 0066aa8 src/java/datafu/pig/util/FieldNotFound.java d624007 src/java/datafu/pig/util/In.java 0667914 src/java/datafu/pig/util/InUDF.java 5057285 src/java/datafu/pig/util/IntToBool.java d00e297 src/java/datafu/pig/util/SimpleEvalFunc.java 2d262b4 src/java/datafu/pig/util/TransposeTupleToBag.java f8a39df src/java/datafu/pig/util/package-info.java cce46a0 test.sh f017e58 test/pig/datafu/test/pig/PigTests.java 9b4eddd test/pig/datafu/test/pig/bags/BagTests.java c9e3f63 test/pig/datafu/test/pig/geo/GeoTests.java 39c7b61 test/pig/datafu/test/pig/hash/HashTests.java ba19344 test/pig/datafu/test/pig/linkanalysis/PageRankImplTests.java 01d540f test/pig/datafu/test/pig/linkanalysis/PageRankTests.java 371d2cc test/pig/datafu/test/pig/random/NumberTests.java 867c27c test/pig/datafu/test/pig/random/UUIDTests.java e199760 test/pig/datafu/test/pig/sampling/SamplingTests.java 9209133 test/pig/datafu/test/pig/sampling/SimpleRandomSampleTest.java bd34881 test/pig/datafu/test/pig/sampling/SimpleRandomSampleTestOld.java 15e1fd6 test/pig/datafu/test/pig/sampling/SimpleRandomSampleWithReplacementTest.java 338b3e7 test/pig/datafu/test/pig/sampling/WeightedReservoirSamplingTests.java 9bab8a0 test/pig/datafu/test/pig/sessions/SessionTests.java 084f970 test/pig/datafu/test/pig/sets/SetTests.java de6aa51 test/pig/datafu/test/pig/stats/EstimationTests.java 258f320 test/pig/datafu/test/pig/stats/MarkovPairTests.java b2f618e test/pig/datafu/test/pig/stats/QuantileTests.java 1814016 test/pig/datafu/test/pig/stats/VARTests.java 776ed9e test/pig/datafu/test/pig/stats/WilsonBinConfTests.java 34e0a48 test/pig/datafu/test/pig/stats/entropy/AbstractEntropyTests.java 6e512d4 test/pig/datafu/test/pig/stats/entropy/EntropyTests.java 8ef94c3 test/pig/datafu/test/pig/stats/entropy/StreamingChaoShenEntropyTests.java 77a8e8b test/pig/datafu/test/pig/stats/entropy/StreamingEmpiricalCondEntropyTests.java 151bf8b test/pig/datafu/test/pig/stats/entropy/StreamingEmpiricalEntropyTests.java a98d79f test/pig/datafu/test/pig/text/NLPTests.java 372b17d test/pig/datafu/test/pig/urls/UserAgentTest.java 82515ad test/pig/datafu/test/pig/util/AssertTests.java bf558a7 test/pig/datafu/test/pig/util/CoalesceTests.java 0dc0f4b test/pig/datafu/test/pig/util/InTests.java a21a0b2 test/pig/datafu/test/pig/util/IntBoolConversionPigTests.java 4017867 test/pig/datafu/test/pig/util/TransposeTest.java f131cd0 Diff: https://reviews.apache.org/r/18139/diff/ Testing ------- You can generate the datafu-pig JAR with: ./gradlew assemble All the tests should pass: ./gradlew test You can generate the eclipse files with: ./gradlew eclipse The tests should pass in eclipse too. Thanks, Matthew Hayes
