-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24215/
-----------------------------------------------------------
Review request for DataFu.
Repository: datafu
Description
-------
Added the Gradle bootstrap task to be used when building from the source
release. This is from SAMZA-283.
Created a sourceRelease task to build the source tarball (also from SAMZA-283).
There is also a signSourceRelease task to build a signed version. The source
release does not include the gradle jar. This is why the bootstrap task is
necessary.
Created RELEASE.md for release instructions on how to create the source
tarball. This is mostly derived from SAMZA-283.
Did some preparation for a binary release. The artifacts datafu-pig and
datafu-hourglass can be installed to the local maven repo using the 'install'
task.
I also fixed all the javadoc issues, which caused build errors when I added the
nexus plugin. These were numerous.
Note: This includes a lot of Gradle build code taken from Apache Samza related
to SAMZA-283.
Diffs
-----
README.md 1b3c821
RELEASE.md PRE-CREATION
bootstrap.gradle PRE-CREATION
build-plugin/build.gradle eab96e0
build.gradle 9ed6eb1
datafu-hourglass/build.gradle 9cc2b99
datafu-hourglass/src/main/java/datafu/hourglass/avro/AvroDateRangeMetadata.java
0136ee9
datafu-hourglass/src/main/java/datafu/hourglass/avro/AvroKeyWithMetadataOutputFormat.java
b37c559
datafu-hourglass/src/main/java/datafu/hourglass/avro/AvroKeyWithMetadataRecordWriter.java
0e61d87
datafu-hourglass/src/main/java/datafu/hourglass/fs/PathUtils.java f9580e7
datafu-hourglass/src/main/java/datafu/hourglass/jobs/AbstractJob.java 2d242eb
datafu-hourglass/src/main/java/datafu/hourglass/jobs/AbstractNonIncrementalJob.java
13db6a4
datafu-hourglass/src/main/java/datafu/hourglass/jobs/AbstractPartitionCollapsingIncrementalJob.java
0ca4e53
datafu-hourglass/src/main/java/datafu/hourglass/jobs/AbstractPartitionPreservingIncrementalJob.java
39c41f8
datafu-hourglass/src/main/java/datafu/hourglass/jobs/ExecutionPlanner.java
ce1c1d6
datafu-hourglass/src/main/java/datafu/hourglass/jobs/FileCleaner.java 7248969
datafu-hourglass/src/main/java/datafu/hourglass/jobs/PartitionCollapsingExecutionPlanner.java
34368f7
datafu-hourglass/src/main/java/datafu/hourglass/jobs/PartitionCollapsingIncrementalJob.java
68e776a
datafu-hourglass/src/main/java/datafu/hourglass/jobs/PartitionPreservingExecutionPlanner.java
6ac55a8
datafu-hourglass/src/main/java/datafu/hourglass/jobs/PartitionPreservingIncrementalJob.java
99aba47
datafu-hourglass/src/main/java/datafu/hourglass/jobs/StagedOutputJob.java
5178133
datafu-hourglass/src/main/java/datafu/hourglass/mapreduce/CollapsingCombiner.java
60ce4fe
datafu-hourglass/src/main/java/datafu/hourglass/mapreduce/CollapsingMapper.java
7c6c5a2
datafu-hourglass/src/main/java/datafu/hourglass/mapreduce/CollapsingReducer.java
86aa66c
datafu-hourglass/src/main/java/datafu/hourglass/mapreduce/DistributedCacheHelper.java
78c8911
datafu-hourglass/src/main/java/datafu/hourglass/mapreduce/PartitioningMapper.java
d17385d
datafu-hourglass/src/main/java/datafu/hourglass/mapreduce/PartitioningReducer.java
1ce4995
datafu-hourglass/src/main/java/datafu/hourglass/model/KeyValueCollector.java
4d37a85
datafu-hourglass/src/main/java/datafu/hourglass/model/Mapper.java 902478a
datafu-pig/build.gradle e21a5b1
datafu-pig/src/main/java/datafu/pig/bags/BagGroup.java 409709c
datafu-pig/src/main/java/datafu/pig/bags/CountEach.java cfb0152
datafu-pig/src/main/java/datafu/pig/bags/DistinctBy.java a79e4de
datafu-pig/src/main/java/datafu/pig/bags/Enumerate.java 8a0d072
datafu-pig/src/main/java/datafu/pig/bags/PrependToBag.java 9292871
datafu-pig/src/main/java/datafu/pig/bags/ReverseEnumerate.java c86ffcf
datafu-pig/src/main/java/datafu/pig/bags/UnorderedPairs.java a1d149e
datafu-pig/src/main/java/datafu/pig/geo/HaversineDistInMiles.java d1e3988
datafu-pig/src/main/java/datafu/pig/hash/lsh/CosineDistanceHash.java 1664362
datafu-pig/src/main/java/datafu/pig/hash/lsh/L1PStableHash.java 2dd42cd
datafu-pig/src/main/java/datafu/pig/hash/lsh/L2PStableHash.java 588f199
datafu-pig/src/main/java/datafu/pig/hash/lsh/LSHFamily.java 394109c
datafu-pig/src/main/java/datafu/pig/hash/lsh/cosine/HyperplaneLSH.java
deff98e
datafu-pig/src/main/java/datafu/pig/hash/lsh/cosine/package-info.java 2e44920
datafu-pig/src/main/java/datafu/pig/hash/lsh/interfaces/LSH.java 69d4043
datafu-pig/src/main/java/datafu/pig/hash/lsh/interfaces/LSHCreator.java
e3dc5b4
datafu-pig/src/main/java/datafu/pig/hash/lsh/interfaces/Sampler.java 0c57a0d
datafu-pig/src/main/java/datafu/pig/hash/lsh/interfaces/package-info.java
5357473
datafu-pig/src/main/java/datafu/pig/hash/lsh/metric/Cosine.java 22222a1
datafu-pig/src/main/java/datafu/pig/hash/lsh/metric/L1.java 311c2ed
datafu-pig/src/main/java/datafu/pig/hash/lsh/metric/L2.java d01f9fb
datafu-pig/src/main/java/datafu/pig/hash/lsh/metric/MetricUDF.java da00a60
datafu-pig/src/main/java/datafu/pig/hash/lsh/metric/package-info.java df8dbc7
datafu-pig/src/main/java/datafu/pig/hash/lsh/p_stable/AbstractStableDistributionFunction.java
0f3ba94
datafu-pig/src/main/java/datafu/pig/hash/lsh/p_stable/L1LSH.java 79bf7e5
datafu-pig/src/main/java/datafu/pig/hash/lsh/p_stable/L2LSH.java d18b189
datafu-pig/src/main/java/datafu/pig/hash/lsh/p_stable/package-info.java
ec9c313
datafu-pig/src/main/java/datafu/pig/hash/lsh/package-info.java 045ed0d
datafu-pig/src/main/java/datafu/pig/hash/lsh/util/DataTypeUtil.java da30179
datafu-pig/src/main/java/datafu/pig/linkanalysis/PageRank.java 80ff567
datafu-pig/src/main/java/datafu/pig/linkanalysis/PageRankImpl.java 5d0b932
datafu-pig/src/main/java/datafu/pig/random/RandInt.java de89c4a
datafu-pig/src/main/java/datafu/pig/sampling/SampleByKey.java 90ea576
datafu-pig/src/main/java/datafu/pig/sampling/SimpleRandomSample.java 8e8debf
datafu-pig/src/main/java/datafu/pig/sampling/SimpleRandomSampleWithReplacementVote.java
598e58c
datafu-pig/src/main/java/datafu/pig/sampling/WeightedReservoirSample.java
92af6a3
datafu-pig/src/main/java/datafu/pig/sessions/Sessionize.java 52d159b
datafu-pig/src/main/java/datafu/pig/sets/SetOperationsBase.java c9997f8
datafu-pig/src/main/java/datafu/pig/stats/Median.java e33a84e
datafu-pig/src/main/java/datafu/pig/stats/Quantile.java 6fd42d3
datafu-pig/src/main/java/datafu/pig/stats/QuantileUtil.java c6fd36a
datafu-pig/src/main/java/datafu/pig/stats/StreamingMedian.java c4c3be4
datafu-pig/src/main/java/datafu/pig/stats/StreamingQuantile.java e4a65b4
datafu-pig/src/main/java/datafu/pig/stats/VAR.java 6f22f25
datafu-pig/src/main/java/datafu/pig/stats/WilsonBinConf.java 1448611
datafu-pig/src/main/java/datafu/pig/stats/entropy/CondEntropy.java 26b743e
datafu-pig/src/main/java/datafu/pig/stats/entropy/EmpiricalCountEntropy.java
388b80f
datafu-pig/src/main/java/datafu/pig/stats/entropy/Entropy.java 9dfff1a
datafu-pig/src/main/java/datafu/pig/util/AliasableEvalFunc.java ee2c3f3
datafu-pig/src/main/java/datafu/pig/util/AssertUDF.java 16f9247
datafu-pig/src/main/java/datafu/pig/util/Coalesce.java f8e25f4
datafu-pig/src/main/java/datafu/pig/util/ContextualEvalFunc.java c534b77
datafu-pig/src/main/java/datafu/pig/util/DataFuException.java 0066aa8
datafu-pig/src/main/java/datafu/pig/util/InUDF.java 5057285
datafu-pig/src/main/java/datafu/pig/util/TransposeTupleToBag.java f8a39df
datafu-pig/src/test/java/datafu/test/pig/hash/lsh/LSHPigTest.java ac3e409
datafu-pig/src/test/java/datafu/test/pig/hash/lsh/LSHTest.java be64bc8
datafu-pig/src/test/java/datafu/test/pig/util/Base64Test.java 99af987
gradle.properties 33df918
gradle/buildscript.gradle 225e0a8
gradle/dependency-versions.gradle eb24e4a
gradle/release.gradle PRE-CREATION
Diff: https://reviews.apache.org/r/24215/diff/
Testing
-------
See the README.md and RELEASE.md. I followed the steps here and verified
everything looked good.
In particular:
I ran the 'signSourceRelease' task and verified that this creates a tarball
with a corresponding signature in build/distribution. I was able to extract
the files from the archive, bootstrap the Gradle wrapper, and then build the
code.
I was able to run the 'install' task to build the code and install the
artifacts in the local Maven repository. I looked at the POM files generated
for datafu-pig and datafu-hourglass and confirmed these looked correct.
Thanks,
Matthew Hayes
