[jira] [Updated] (KAFKA-1171) Gradle build for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-1171: - Fix Version/s: 0.8.1 0.8.1.1 Gradle build for Kafka -- Key: KAFKA-1171 URL: https://issues.apache.org/jira/browse/KAFKA-1171 Project: Kafka Issue Type: Improvement Components: packaging Affects Versions: 0.8.1, 0.9.0 Reporter: David Arthur Assignee: David Arthur Priority: Blocker Fix For: 0.8.1, 0.8.1.1 Attachments: 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, kafka-1171_v10.patch, kafka-1171_v11.patch, kafka-1171_v12.patch, kafka-1171_v13.patch, kafka-1171_v14.patch, kafka-1171_v15.patch, kafka-1171_v6.patch, kafka-1171_v7.patch, kafka-1171_v8.patch, kafka-1171_v9.patch We have previously discussed moving away from SBT to an easier-to-comprehend-and-debug build system such as Ant or Gradle. I put up a patch for an Ant+Ivy build a while ago[1], and it sounded like people wanted to check out Gradle as well. 1. https://issues.apache.org/jira/browse/KAFKA-855 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1171) Gradle build for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1171: --- Attachment: kafka-1171_v15.patch Attach patch v15. Fixed a few more things. 1. Added tasks to build test jars. 2. Changed kafka-run-class.sh to pick up the right version of kafka and kafka-perf jar based on the scala version. Also included the jar from all projects (examples, clients, etc) in the script. 3. Changed unit test reporting. Now the test task prints out the name of each of the tests. If there is a failure, it prints out a longer version of the stack trace. 4. Figured out a way to force re-running a unit test w/o code change. Documented in README. 5. Changed tasks from *_all to *All. 6. Documented all changes in README. Also, 7. ./gradlew build is equivalent to ./gradlew test. I just documented the latter since it's more intuitive. 8. Those tests failures from 2.10.1 seem to be transient. I ran those a couple of times in both mac and linux and they all pass. Gradle build for Kafka -- Key: KAFKA-1171 URL: https://issues.apache.org/jira/browse/KAFKA-1171 Project: Kafka Issue Type: Improvement Components: packaging Affects Versions: 0.8.1, 0.9.0 Reporter: David Arthur Assignee: David Arthur Priority: Blocker Attachments: 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, kafka-1171_v10.patch, kafka-1171_v11.patch, kafka-1171_v12.patch, kafka-1171_v13.patch, kafka-1171_v14.patch, kafka-1171_v15.patch, kafka-1171_v6.patch, kafka-1171_v7.patch, kafka-1171_v8.patch, kafka-1171_v9.patch We have previously discussed moving away from SBT to an easier-to-comprehend-and-debug build system such as Ant or Gradle. I put up a patch for an Ant+Ivy build a while ago[1], and it sounded like people wanted to check out Gradle as well. 1. https://issues.apache.org/jira/browse/KAFKA-855 -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1171) Gradle build for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-1171: - Priority: Blocker (was: Major) Gradle build for Kafka -- Key: KAFKA-1171 URL: https://issues.apache.org/jira/browse/KAFKA-1171 Project: Kafka Issue Type: Improvement Components: packaging Affects Versions: 0.8.1, 0.9.0 Reporter: David Arthur Assignee: David Arthur Priority: Blocker Attachments: 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, kafka-1171_v10.patch, kafka-1171_v11.patch, kafka-1171_v12.patch, kafka-1171_v13.patch, kafka-1171_v14.patch, kafka-1171_v6.patch, kafka-1171_v7.patch, kafka-1171_v8.patch, kafka-1171_v9.patch We have previously discussed moving away from SBT to an easier-to-comprehend-and-debug build system such as Ant or Gradle. I put up a patch for an Ant+Ivy build a while ago[1], and it sounded like people wanted to check out Gradle as well. 1. https://issues.apache.org/jira/browse/KAFKA-855 -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1171) Gradle build for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1171: --- Attachment: kafka-1171_v13.patch Attach patch v13. This fixes all known issues. 1. Added hadoop-producer and hadoop-consumer projects. 2. Fixed maven publishing to use project name. kafka.jar is still published under kafka, same as we have in maven now. Other projects like hadoop-consumer will now be called kafka-hadoop-consumer to make it clear. Since those jars are not widely used, changing the name is probably ok. 3. Renamed the *_all tasks a bit and the new tasks are summarized in README. 4. For copyDependantLibs, I kept the name. The old assemblyDependency target in sbt builds a fat jar. Here, the jars are copied individually and not assembled together, which is clearer. 5. ./gradlew is probably used to distinguish it from the locally install gradle executable. Future stuff: 1. I think we should keep the core jar name kafka until the old producer/consumer code is phased out, at which point, we can rename it to kafka-server. Before that, since it has the old client, we should leave it in the old coordinate so that an application can dedup the jars correctly if multiple versions of kafka jar are dragged in through transitive dependencies. Gradle build for Kafka -- Key: KAFKA-1171 URL: https://issues.apache.org/jira/browse/KAFKA-1171 Project: Kafka Issue Type: Improvement Components: packaging Affects Versions: 0.8.1, 0.9.0 Reporter: David Arthur Assignee: David Arthur Attachments: 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, kafka-1171_v10.patch, kafka-1171_v11.patch, kafka-1171_v12.patch, kafka-1171_v13.patch, kafka-1171_v6.patch, kafka-1171_v7.patch, kafka-1171_v8.patch, kafka-1171_v9.patch We have previously discussed moving away from SBT to an easier-to-comprehend-and-debug build system such as Ant or Gradle. I put up a patch for an Ant+Ivy build a while ago[1], and it sounded like people wanted to check out Gradle as well. 1. https://issues.apache.org/jira/browse/KAFKA-855 -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1171) Gradle build for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1171: --- Attachment: kafka-1171_v14.patch Attach patch v14 to fix a typo in README. Gradle build for Kafka -- Key: KAFKA-1171 URL: https://issues.apache.org/jira/browse/KAFKA-1171 Project: Kafka Issue Type: Improvement Components: packaging Affects Versions: 0.8.1, 0.9.0 Reporter: David Arthur Assignee: David Arthur Attachments: 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, kafka-1171_v10.patch, kafka-1171_v11.patch, kafka-1171_v12.patch, kafka-1171_v13.patch, kafka-1171_v14.patch, kafka-1171_v6.patch, kafka-1171_v7.patch, kafka-1171_v8.patch, kafka-1171_v9.patch We have previously discussed moving away from SBT to an easier-to-comprehend-and-debug build system such as Ant or Gradle. I put up a patch for an Ant+Ivy build a while ago[1], and it sounded like people wanted to check out Gradle as well. 1. https://issues.apache.org/jira/browse/KAFKA-855 -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1171) Gradle build for Kafka
[
https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Arthur updated KAFKA-1171:
Attachment: 0001-Adding-basic-Gradle-build.patch
* Adding upload_* tasks for the different Scala versions
* Add subproject name to the upload artifact
* Tested uploading all artifacts to Nexus
* Tested pulling down artifacts via Ivy
* Fixed javadoc error (remove package.html from util dir)
Still seeing a few issues (non-blockers):
* Gradle warnings
{code}
The TaskContainer.add() method has been deprecated and is scheduled to be
removed in Gradle 2.0. Please use the create() method instead.
Building project 'core' with Scala version 2.8.0
The scalaTools configuration has been deprecated and is scheduled to be removed
in Gradle 2.0. Typically, usages of 'scalaTools' can simply be removed, and the
Scala tools libraries to be used will be inferred from the Scala library found
on the regular (compile) class path. In some cases, it may be necessary to
additionally configure the 'scalaClasspath' property of ScalaCompile and
ScalaDoc tasks.
{code}
* Some of our custom tasks don't show up in the tasks output
Other questions:
* Publishing docs/sources jars?
* How to produce a release dist with different Scala versions? Right now it
picks 2.8.0, maybe this should default to our recommended Scala version since
this is probably how people will run Kafka brokers.
*
Gradle build for Kafka
--
Key: KAFKA-1171
URL: https://issues.apache.org/jira/browse/KAFKA-1171
Project: Kafka
Issue Type: Improvement
Components: packaging
Affects Versions: 0.8.1, 0.9.0
Reporter: David Arthur
Assignee: David Arthur
Attachments: 0001-Adding-basic-Gradle-build.patch,
0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch,
0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch,
0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch,
kafka-1171_v6.patch, kafka-1171_v7.patch, kafka-1171_v8.patch,
kafka-1171_v9.patch
We have previously discussed moving away from SBT to an
easier-to-comprehend-and-debug build system such as Ant or Gradle. I put up a
patch for an Ant+Ivy build a while ago[1], and it sounded like people wanted
to check out Gradle as well.
1. https://issues.apache.org/jira/browse/KAFKA-855
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (KAFKA-1171) Gradle build for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1171: --- Attachment: kafka-1171_v10.patch Attach patch v10. 1. added uploadCoreArchives_all task 2. added releaseTarGz_all task 3. updated README 4. added Apache license header 5. rebased and made fixes to use the new version of snappy. David Arthur, I didn't incorporate your latest changes. Could you apply what's not already covered on top of v10 and upload a new patch? Thanks. Gradle build for Kafka -- Key: KAFKA-1171 URL: https://issues.apache.org/jira/browse/KAFKA-1171 Project: Kafka Issue Type: Improvement Components: packaging Affects Versions: 0.8.1, 0.9.0 Reporter: David Arthur Assignee: David Arthur Attachments: 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, kafka-1171_v10.patch, kafka-1171_v6.patch, kafka-1171_v7.patch, kafka-1171_v8.patch, kafka-1171_v9.patch We have previously discussed moving away from SBT to an easier-to-comprehend-and-debug build system such as Ant or Gradle. I put up a patch for an Ant+Ivy build a while ago[1], and it sounded like people wanted to check out Gradle as well. 1. https://issues.apache.org/jira/browse/KAFKA-855 -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1171) Gradle build for Kafka
[
https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jun Rao updated KAFKA-1171:
---
Attachment: kafka-1171_v11.patch
Attach patch v11.
1. Included all but one fixes from David Arthur on top of v9.
2. Added a new project clients for the new producer. To build clients jar and
run unit tests, do
./gradlew clients:jar
./gradlew clients:test
3. Removed scalaTools from build.gradle to get rid of the warning.
4. Updated README.md.
The patch is ready to be checked in (will remove sbt stuff during checking in).
David Arthur,
I didn't take your following change since it publishes the kafka jar under
kafka-core, which is different from the 0.8.0 release. I haven't figured out
how to use the modified project name, but will do. However, this is not a
blocker since we only publish one jar to maven now.
+pom.artifactId = kafka-${project.name}_${scalaVersion}
Gradle build for Kafka
--
Key: KAFKA-1171
URL: https://issues.apache.org/jira/browse/KAFKA-1171
Project: Kafka
Issue Type: Improvement
Components: packaging
Affects Versions: 0.8.1, 0.9.0
Reporter: David Arthur
Assignee: David Arthur
Attachments: 0001-Adding-basic-Gradle-build.patch,
0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch,
0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch,
0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch,
kafka-1171_v10.patch, kafka-1171_v11.patch, kafka-1171_v6.patch,
kafka-1171_v7.patch, kafka-1171_v8.patch, kafka-1171_v9.patch
We have previously discussed moving away from SBT to an
easier-to-comprehend-and-debug build system such as Ant or Gradle. I put up a
patch for an Ant+Ivy build a while ago[1], and it sounded like people wanted
to check out Gradle as well.
1. https://issues.apache.org/jira/browse/KAFKA-855
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (KAFKA-1171) Gradle build for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1171: --- Attachment: kafka-1171_v12.patch Did a google search on scalatest and found the answer right in this jira. Apparently, Chris Freeman has posted the answer earlier. Attach patch v12. 1. Unit tests should work with scala 2.10* now. 2. Added the project example. Todo: 1. Need to figure out how to add the two subprojects under contrib. Gradle build for Kafka -- Key: KAFKA-1171 URL: https://issues.apache.org/jira/browse/KAFKA-1171 Project: Kafka Issue Type: Improvement Components: packaging Affects Versions: 0.8.1, 0.9.0 Reporter: David Arthur Assignee: David Arthur Attachments: 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, kafka-1171_v10.patch, kafka-1171_v11.patch, kafka-1171_v12.patch, kafka-1171_v6.patch, kafka-1171_v7.patch, kafka-1171_v8.patch, kafka-1171_v9.patch We have previously discussed moving away from SBT to an easier-to-comprehend-and-debug build system such as Ant or Gradle. I put up a patch for an Ant+Ivy build a while ago[1], and it sounded like people wanted to check out Gradle as well. 1. https://issues.apache.org/jira/browse/KAFKA-855 -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1171) Gradle build for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1171: --- Attachment: kafka-1171_v8.patch Attach patch v8. 1. ./gradlew idea works now. I still couldn't get it compile in Intellij. However, this is due to an existing problem that we have two versions of Annotations.scala (one for 2.8 and another for 2.9+). TODO: 1. The dependencies in the pom file is still a bit weird. Unlike runtime dependencies, pom only includes direct dependencies. However, it's not clear why the following dependencies are listed since they are test dependencies only. 'org.easymock:easymock:3.0' 'org.objenesis:objenesis:1.2' 'org.scalatest:scalatest:1.2' Gradle build for Kafka -- Key: KAFKA-1171 URL: https://issues.apache.org/jira/browse/KAFKA-1171 Project: Kafka Issue Type: Improvement Components: packaging Affects Versions: 0.8.1, 0.9.0 Reporter: David Arthur Assignee: David Arthur Attachments: 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, kafka-1171_v6.patch, kafka-1171_v7.patch, kafka-1171_v8.patch We have previously discussed moving away from SBT to an easier-to-comprehend-and-debug build system such as Ant or Gradle. I put up a patch for an Ant+Ivy build a while ago[1], and it sounded like people wanted to check out Gradle as well. 1. https://issues.apache.org/jira/browse/KAFKA-855 -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1171) Gradle build for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1171: --- Attachment: kafka-1171_v9.patch Attach patch v9. The pom dependency seems to work since the included test dependencies are marked with test. 1. Added groovy script to simplify cross compile for jar_all and build_all. There are no known existing issues. Please verify. Joe Stein, could you verify the pom file? Gradle build for Kafka -- Key: KAFKA-1171 URL: https://issues.apache.org/jira/browse/KAFKA-1171 Project: Kafka Issue Type: Improvement Components: packaging Affects Versions: 0.8.1, 0.9.0 Reporter: David Arthur Assignee: David Arthur Attachments: 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, kafka-1171_v6.patch, kafka-1171_v7.patch, kafka-1171_v8.patch, kafka-1171_v9.patch We have previously discussed moving away from SBT to an easier-to-comprehend-and-debug build system such as Ant or Gradle. I put up a patch for an Ant+Ivy build a while ago[1], and it sounded like people wanted to check out Gradle as well. 1. https://issues.apache.org/jira/browse/KAFKA-855 -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1171) Gradle build for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1171: --- Attachment: kafka-1171_v7.patch Attach patch v7. Additionally fixes since v6. 1. defaulted the build to scala 2.8.0. 2. fixed the build artifact names to be consistent with those produced in sbt. 3. added the task jar_all to build the jar for all scala versions. 4. fixed the runtime dependency; now both copyDependantLibs and releaseTarGz generate the correct runtime dependent jars. 5. included David's fix on maven; pom file can be generated now. 6. fixed README.md. TODO: 1. Task idea fails with: Execution failed for task ':ideaProject'. Cannot infer Scala class path because no Scala library Jar was found on class path: [/Users/jrao/.gradle/caches/artifacts-24/filestore/net.sf.jopt-simple/jopt-simple/3.2/jar/d625f12ba08083c8c16dcedd5396ec730e9e77ab/jopt-simple-3.2.jar] 2. The dependencies listed in pom file is a bit different from runtime dependencies. The dependencies in the pom file (./gradlew :core:uploadArchives) include: artifactIdeasymock/artifactId artifactIdjopt-simple/artifactId artifactIdjunit/artifactId artifactIdmetrics-annotation/artifactId artifactIdmetrics-core/artifactId artifactIdobjenesis/artifactId artifactIdscala-library/artifactId artifactIdscalatest/artifactId artifactIdsnappy-java/artifactId artifactIdzkclient/artifactId artifactIdzookeeper/artifactId Runtime dependencies (./gradlew copyDependantLibs) are: ./core/build/dependant-libs-2.8.0/jopt-simple-3.2.jar ./core/build/dependant-libs-2.8.0/log4j-1.2.15.jar ./core/build/dependant-libs-2.8.0/metrics-annotation-2.2.0.jar ./core/build/dependant-libs-2.8.0/metrics-core-2.2.0.jar ./core/build/dependant-libs-2.8.0/scala-library-2.8.0.jar ./core/build/dependant-libs-2.8.0/slf4j-api-1.7.2.jar ./core/build/dependant-libs-2.8.0/snappy-java-1.0.4.1.jar ./core/build/dependant-libs-2.8.0/zkclient-0.3.jar ./core/build/dependant-libs-2.8.0/zookeeper-3.3.4.jar Gradle build for Kafka -- Key: KAFKA-1171 URL: https://issues.apache.org/jira/browse/KAFKA-1171 Project: Kafka Issue Type: Improvement Components: packaging Affects Versions: 0.8.1, 0.9.0 Reporter: David Arthur Assignee: David Arthur Attachments: 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, kafka-1171_v6.patch, kafka-1171_v7.patch We have previously discussed moving away from SBT to an easier-to-comprehend-and-debug build system such as Ant or Gradle. I put up a patch for an Ant+Ivy build a while ago[1], and it sounded like people wanted to check out Gradle as well. 1. https://issues.apache.org/jira/browse/KAFKA-855 -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1171) Gradle build for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1171: --- Attachment: (was: kafka-1171_v6.patch) Gradle build for Kafka -- Key: KAFKA-1171 URL: https://issues.apache.org/jira/browse/KAFKA-1171 Project: Kafka Issue Type: Improvement Components: packaging Affects Versions: 0.8.1, 0.9.0 Reporter: David Arthur Assignee: David Arthur Attachments: 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch We have previously discussed moving away from SBT to an easier-to-comprehend-and-debug build system such as Ant or Gradle. I put up a patch for an Ant+Ivy build a while ago[1], and it sounded like people wanted to check out Gradle as well. 1. https://issues.apache.org/jira/browse/KAFKA-855 -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1171) Gradle build for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1171: --- Attachment: kafka-1171_v6.patch Attaching v6 patch again with the gradle binary jar included. Joe, Could you add your change on top of v6? When generating the patch, make sure you use the --binary in git diff so that the binary jar is included. Gradle build for Kafka -- Key: KAFKA-1171 URL: https://issues.apache.org/jira/browse/KAFKA-1171 Project: Kafka Issue Type: Improvement Components: packaging Affects Versions: 0.8.1, 0.9.0 Reporter: David Arthur Assignee: David Arthur Attachments: 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, kafka-1171_v6.patch We have previously discussed moving away from SBT to an easier-to-comprehend-and-debug build system such as Ant or Gradle. I put up a patch for an Ant+Ivy build a while ago[1], and it sounded like people wanted to check out Gradle as well. 1. https://issues.apache.org/jira/browse/KAFKA-855 -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1171) Gradle build for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1171: - Attachment: 0001-Adding-basic-Gradle-build.patch patched the build to sign the jars, the pom during uploadArchives stage passes with sigs but pom is blank https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.9.1/0.8.1/kafka_2.9.1-0.8.1.pom will look into that next Gradle build for Kafka -- Key: KAFKA-1171 URL: https://issues.apache.org/jira/browse/KAFKA-1171 Project: Kafka Issue Type: Improvement Components: packaging Affects Versions: 0.8.1, 0.9.0 Reporter: David Arthur Assignee: David Arthur Attachments: 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, kafka-1171_v6.patch We have previously discussed moving away from SBT to an easier-to-comprehend-and-debug build system such as Ant or Gradle. I put up a patch for an Ant+Ivy build a while ago[1], and it sounded like people wanted to check out Gradle as well. 1. https://issues.apache.org/jira/browse/KAFKA-855 -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1171) Gradle build for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1171: --- Attachment: kafka-1171_v6.patch I made a pass of this. Attach the latest patch v6, which includes: 1. added a new task for copying all dependent jars locally and fixed Kafka script to pick up those jars 2. added a new task for building the release tar gz file 3. fixed the name of the built artifacts (with kafka prefix and scala version) 4. fixed READE with the new commands. 5. added the license HEADER for build to work. TODO: 1. Currently, we default to build with scala 2.9.1. So the script only works if you build by specifying scala 2.8.0. Will look into how to default to build with scala 2.8.0. 2. Task idea fails with: Execution failed for task ':ideaProject'. Cannot infer Scala class path because no Scala library Jar was found on class path: [/Users/jrao/.gradle/caches/artifacts-24/filestore/net.sf.jopt-simple/jopt-simple/3.2/jar/d625f12ba08083c8c16dcedd5396ec730e9e77ab/jopt-simple-3.2.jar] 3. There is a slight difference in artifact names. Currently, we have kafka_2.8.0-0.8.1.jar With this patch, it's changed to kafka-2.8.0-0.8.1.jar Will look into how to fix it. Gradle build for Kafka -- Key: KAFKA-1171 URL: https://issues.apache.org/jira/browse/KAFKA-1171 Project: Kafka Issue Type: Improvement Components: packaging Affects Versions: 0.8.1, 0.9.0 Reporter: David Arthur Assignee: David Arthur Attachments: 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, kafka-1171_v6.patch We have previously discussed moving away from SBT to an easier-to-comprehend-and-debug build system such as Ant or Gradle. I put up a patch for an Ant+Ivy build a while ago[1], and it sounded like people wanted to check out Gradle as well. 1. https://issues.apache.org/jira/browse/KAFKA-855 -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1171) Gradle build for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-1171: Attachment: 0001-Adding-basic-Gradle-build.patch Updating patch to include two missing .gradle files Gradle build for Kafka -- Key: KAFKA-1171 URL: https://issues.apache.org/jira/browse/KAFKA-1171 Project: Kafka Issue Type: Improvement Components: packaging Affects Versions: 0.8.1, 0.9.0 Reporter: David Arthur Assignee: David Arthur Attachments: 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch We have previously discussed moving away from SBT to an easier-to-comprehend-and-debug build system such as Ant or Gradle. I put up a patch for an Ant+Ivy build a while ago[1], and it sounded like people wanted to check out Gradle as well. 1. https://issues.apache.org/jira/browse/KAFKA-855 -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (KAFKA-1171) Gradle build for Kafka
[
https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Arthur updated KAFKA-1171:
Attachment: 0001-Adding-basic-Gradle-build.patch
Updating patch:
* Added perf subproject
* Use Zinc for incremental compilation
* Compile with Scala 2.8.0, 2.8.2, 2.9.1, 2.9.2, 2.10.0
* Test passing except for 2.10.0
* Added license checking (taken from Samza)
* Added Maven publishing, {{./gradlew uploadArtifacts}} see gradle.properties
for Maven URL and auth
Compile and test with build target: {{./gradlew -PscalaVersion=2.9.2 build}}
Gradle build for Kafka
--
Key: KAFKA-1171
URL: https://issues.apache.org/jira/browse/KAFKA-1171
Project: Kafka
Issue Type: Improvement
Components: packaging
Affects Versions: 0.8.1, 0.9.0
Reporter: David Arthur
Assignee: David Arthur
Attachments: 0001-Adding-basic-Gradle-build.patch,
0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch,
0001-Adding-basic-Gradle-build.patch
We have previously discussed moving away from SBT to an
easier-to-comprehend-and-debug build system such as Ant or Gradle. I put up a
patch for an Ant+Ivy build a while ago[1], and it sounded like people wanted
to check out Gradle as well.
1. https://issues.apache.org/jira/browse/KAFKA-855
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Updated] (KAFKA-1171) Gradle build for Kafka
[
https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Arthur updated KAFKA-1171:
Attachment: 0001-Adding-basic-Gradle-build.patch
Basic parameterized build is working. Added in Zinc compiler
Run it like: {{./gradlew -PscalaVersion=2.9.1 clean core:test}}
Gradle build for Kafka
--
Key: KAFKA-1171
URL: https://issues.apache.org/jira/browse/KAFKA-1171
Project: Kafka
Issue Type: Improvement
Components: packaging
Affects Versions: 0.8.1, 0.9.0
Reporter: David Arthur
Assignee: David Arthur
Attachments: 0001-Adding-basic-Gradle-build.patch,
0001-Adding-basic-Gradle-build.patch, 0001-Adding-basic-Gradle-build.patch
We have previously discussed moving away from SBT to an
easier-to-comprehend-and-debug build system such as Ant or Gradle. I put up a
patch for an Ant+Ivy build a while ago[1], and it sounded like people wanted
to check out Gradle as well.
1. https://issues.apache.org/jira/browse/KAFKA-855
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Updated] (KAFKA-1171) Gradle build for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-1171: Attachment: (was: KAFKA-1171.diff) Gradle build for Kafka -- Key: KAFKA-1171 URL: https://issues.apache.org/jira/browse/KAFKA-1171 Project: Kafka Issue Type: Improvement Components: packaging Affects Versions: 0.8.1, 0.9.0 Reporter: David Arthur Assignee: David Arthur Attachments: 0001-Adding-basic-Gradle-build.patch We have previously discussed moving away from SBT to an easier-to-comprehend-and-debug build system such as Ant or Gradle. I put up a patch for an Ant+Ivy build a while ago[1], and it sounded like people wanted to check out Gradle as well. 1. https://issues.apache.org/jira/browse/KAFKA-855 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (KAFKA-1171) Gradle build for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-1171: Attachment: KAFKA-1171.patch Gradle build for Kafka -- Key: KAFKA-1171 URL: https://issues.apache.org/jira/browse/KAFKA-1171 Project: Kafka Issue Type: Improvement Components: packaging Affects Versions: 0.8.1, 0.9.0 Reporter: David Arthur Assignee: David Arthur Attachments: 0001-Adding-basic-Gradle-build.patch, KAFKA-1171.patch, KAFKA-1171.patch We have previously discussed moving away from SBT to an easier-to-comprehend-and-debug build system such as Ant or Gradle. I put up a patch for an Ant+Ivy build a while ago[1], and it sounded like people wanted to check out Gradle as well. 1. https://issues.apache.org/jira/browse/KAFKA-855 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (KAFKA-1171) Gradle build for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-1171: Attachment: KAFKA-1171.patch Gradle build for Kafka -- Key: KAFKA-1171 URL: https://issues.apache.org/jira/browse/KAFKA-1171 Project: Kafka Issue Type: Improvement Components: packaging Affects Versions: 0.8.1, 0.9.0 Reporter: David Arthur Assignee: David Arthur Attachments: 0001-Adding-basic-Gradle-build.patch, KAFKA-1171.patch, KAFKA-1171.patch We have previously discussed moving away from SBT to an easier-to-comprehend-and-debug build system such as Ant or Gradle. I put up a patch for an Ant+Ivy build a while ago[1], and it sounded like people wanted to check out Gradle as well. 1. https://issues.apache.org/jira/browse/KAFKA-855 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (KAFKA-1171) Gradle build for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-1171: Attachment: (was: KAFKA-1171.patch) Gradle build for Kafka -- Key: KAFKA-1171 URL: https://issues.apache.org/jira/browse/KAFKA-1171 Project: Kafka Issue Type: Improvement Components: packaging Affects Versions: 0.8.1, 0.9.0 Reporter: David Arthur Assignee: David Arthur Attachments: 0001-Adding-basic-Gradle-build.patch We have previously discussed moving away from SBT to an easier-to-comprehend-and-debug build system such as Ant or Gradle. I put up a patch for an Ant+Ivy build a while ago[1], and it sounded like people wanted to check out Gradle as well. 1. https://issues.apache.org/jira/browse/KAFKA-855 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (KAFKA-1171) Gradle build for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-1171: Attachment: KAFKA-1171.diff Here is a very simple Gradle project that compiles and tests the core part of the Kafka source tree. Notable things missing: * Multiple versions of Scala (http://issues.gradle.org/browse/GRADLE-2478) * Other sub-modules (perf, contrib, etc) * Release packaging * Other stuff I'm forgetting at the moment As I understand it, all of these things are possible with existing Gradle plugins except for multiple Scala versions - couldn't find an answer there. Compared with the Ant build, it is much less verbose, but less flexible. Doing things like cross-targeting Scala is pretty cut-and-dry with Ant. Setting up this simple project up didn't take me very long (~1.5 hours) but I did run into some bad documentation from Gradle's website and some unhelpful errors. Here are two of the more fun errors I encountered: https://gist.github.com/mumrah/6de7be3d6640059904b5 The error messaging gives me pause, b/c (in my experience) it's almost immediately obvious what went wrong with your build in Ant. Also notice that Gradle is calling the ant scalac target. Looking forward hear to what others think! Gradle build for Kafka -- Key: KAFKA-1171 URL: https://issues.apache.org/jira/browse/KAFKA-1171 Project: Kafka Issue Type: Improvement Components: packaging Affects Versions: 0.8.1, 0.9.0 Reporter: David Arthur Assignee: David Arthur Attachments: KAFKA-1171.diff We have previously discussed moving away from SBT to an easier-to-comprehend-and-debug build system such as Ant or Gradle. I put up a patch for an Ant+Ivy build a while ago[1], and it sounded like people wanted to check out Gradle as well. 1. https://issues.apache.org/jira/browse/KAFKA-855 -- This message was sent by Atlassian JIRA (v6.1#6144)
