[jira] [Work logged] (BEAM-4488) Contributor's guide needs to document how to install Python SDK from source
[
https://issues.apache.org/jira/browse/BEAM-4488?focusedWorklogId=110298&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-110298
]
ASF GitHub Bot logged work on BEAM-4488:
Author: ASF GitHub Bot
Created on: 08/Jun/18 22:03
Start Date: 08/Jun/18 22:03
Worklog Time Spent: 10m
Work Description: asfgit closed pull request #464: [BEAM-4488],
[BEAM-4489], [BEAM-4490] Improvements to contribution guide
URL: https://github.com/apache/beam-site/pull/464
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/src/contribute/index.md b/src/contribute/index.md
index de269f1d3..49c107bae 100644
--- a/src/contribute/index.md
+++ b/src/contribute/index.md
@@ -19,8 +19,14 @@ The Apache Beam community welcomes contributions from anyone!
There are lots of opportunities:
- - write new examples
+ - ask or answer questions on [u...@beam.apache.org]({{ site.baseurl
+}}/community/contact-us/) or
+[stackoverflow](https://stackoverflow.com/questions/tagged/apache-beam)
+ - review proposed design ideas on [d...@beam.apache.org]({{ site.baseurl
+}}/community/contact-us/)
- improve the documentation
+ - contribute [bug
reports](https://issues.apache.org/jira/projects/BEAM/issues)
+ - write new examples
- add new user-facing libraries (new statistical libraries, new IO connectors,
etc)
- improve your favorite language SDK (Java, Python, Go, etc)
@@ -28,36 +34,44 @@ There are lots of opportunities:
Cloud Dataflow, etc)
- work on the core programming model (what is a Beam pipeline and how does it
run?)
+ - improve the developer experience on Windows
Most importantly, if you have an idea of how to contribute, then do it!
For a list of open starter tasks, check
[https://s.apache.org/beam-starter-tasks](https://s.apache.org/beam-starter-tasks).
-And, of course, we would love if you [contact us]({{ site.baseurl
-}}/community/contact-us/) and introduce yourself.
-
## Contributing code
+Discussons about contributing code to beam happens on the [dev@ mailing
list]({{ site.baseurl
+}}/community/contact-us/). Introduce yourself!
+
+Questions can be asked on the [#beam channel of the ASF slack]({{ site.baseurl
+}}/community/contact-us/). Introduce yourself!
+
Coding happens at
[https://github.com/apache/beam](https://github.com/apache/beam). To
contribute, follow the usual GitHub process: fork the repo, make your changes,
-and open a pull request. If you are unfamiliar with this workflow, GitHub
-maintains these helpful guides:
+and open a pull request and @mention a reviewer. If you have more than one
commit
+in your change, you many be asked to rebase and squash the commits.
+If you are unfamiliar with this workflow, GitHub maintains these helpful
guides:
- [Git Handbook](https://guides.github.com/introduction/git-handbook/)
- [Forking a repository](https://guides.github.com/activities/forking/)
-If your change is large, it is a good idea to [discuss it on the dev@ mailing
list]({{
-site.baseurl }}/community/contact-us/) (you may be asked to create a design doc
+If your change is large or it is your first change, it is a good idea to
+[discuss it on the dev@ mailing list]({{ site.baseurl }}/community/contact-us/)
+
+For large changes (you may be asked to create a design doc
([template](https://s.apache.org/beam-design-doc-template),
-[examples](https://s.apache.org/beam-design-docs))). You will also
-need to submit a signed [Individual Contributor License
+[examples](https://s.apache.org/beam-design-docs))).
+
+Documentation happens at
[https://github.com/apache/beam-site](https://github.com/apache/beam-site)
+and contributions are welcome.
+
+Large contributions require a signed [Individual Contributor License
Agreement](https://www.apache.org/licenses/icla.pdf) (ICLA) to the Apache
-Software Foundation (ASF). The purpose of this agreement is to clearly define
-the terms under which intellectual property has been contributed to the ASF and
-thereby allow us to defend the project should there be a legal dispute
-regarding the software at some future time.
+Software Foundation (ASF).
If you are contributing a `PTransform` to Beam, we have an extensive
[PTransform Style Guide]({{ site.baseurl }}/contribute/ptransform-style-guide).
@@ -66,6 +80,11 @@ If you are contributing a `PTransform` to Beam, we have an
extensive
We use Gradle to orchestrate building and testing.
+You do not need to install gradle, but you do need a Java SDK installed.
+You can develop on Linux, macOS, or Microsoft Windows. There have been
+issues noted when developing using Windows; feel free to contribute fixes
+to make it easier.
+
The entir
[jira] [Work logged] (BEAM-4488) Contributor's guide needs to document how to install Python SDK from source
[ https://issues.apache.org/jira/browse/BEAM-4488?focusedWorklogId=110295&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-110295 ] ASF GitHub Bot logged work on BEAM-4488: Author: ASF GitHub Bot Created on: 08/Jun/18 21:59 Start Date: 08/Jun/18 21:59 Worklog Time Spent: 10m Work Description: pabloem commented on issue #464: [BEAM-4488], [BEAM-4489], [BEAM-4490] Improvements to contribution guide URL: https://github.com/apache/beam-site/pull/464#issuecomment-395903298 @asfgit merge This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 110295) Time Spent: 2h (was: 1h 50m) > Contributor's guide needs to document how to install Python SDK from source > --- > > Key: BEAM-4488 > URL: https://issues.apache.org/jira/browse/BEAM-4488 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Pablo Estrada >Assignee: Alan Myrvold >Priority: Major > Time Spent: 2h > Remaining Estimate: 0h > > Currently it is not documented, though it does appear in the quickstart -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4488) Contributor's guide needs to document how to install Python SDK from source
[ https://issues.apache.org/jira/browse/BEAM-4488?focusedWorklogId=110270&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-110270 ] ASF GitHub Bot logged work on BEAM-4488: Author: ASF GitHub Bot Created on: 08/Jun/18 20:58 Start Date: 08/Jun/18 20:58 Worklog Time Spent: 10m Work Description: alanmyrvold commented on a change in pull request #464: [BEAM-4488], [BEAM-4489], [BEAM-4490] Improvements to contribution guide URL: https://github.com/apache/beam-site/pull/464#discussion_r194179795 ## File path: src/contribute/index.md ## @@ -215,6 +301,12 @@ Easy to use Java 8 DSL for the Beam Java SDK. Provides a high-level abstraction - JIRA: [dsl-euphoria](https://issues.apache.org/jira/browse/BEAM-4366?jql=project%20%3D%20BEAM%20AND%20component%20%3D%20dsl-euphoria) / [BEAM-3900](https://issues.apache.org/jira/browse/BEAM-3900) - Contact: [David Moravek](mailto:david.mora...@gmail.com) +### Improving the contributor experience + +Making it easier to write code, run tests, and release. Investigating using docker for jenkins builds, automating the release process, and improving the reliability of tests. + +Ideas and help welcome! Contact: [Alan Myrvold](mailto:amyrv...@google.com), [Mark Liu](mailto:mark...@google.com), [Yifan Zou](mailto:yifan...@google.com) Review comment: All of the other sections for Works in Progress list personal contacts. We could create separate mailing lists like other projects do; several apache projects have more granular mailing lists, http://mail-archives.apache.org/mod_mbox/ but I don't think we have the volume to justify splitting off the dev list into works in progress lists. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 110270) Time Spent: 1h 50m (was: 1h 40m) > Contributor's guide needs to document how to install Python SDK from source > --- > > Key: BEAM-4488 > URL: https://issues.apache.org/jira/browse/BEAM-4488 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Pablo Estrada >Assignee: Alan Myrvold >Priority: Major > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently it is not documented, though it does appear in the quickstart -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4488) Contributor's guide needs to document how to install Python SDK from source
[ https://issues.apache.org/jira/browse/BEAM-4488?focusedWorklogId=110224&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-110224 ] ASF GitHub Bot logged work on BEAM-4488: Author: ASF GitHub Bot Created on: 08/Jun/18 18:51 Start Date: 08/Jun/18 18:51 Worklog Time Spent: 10m Work Description: aaltay commented on a change in pull request #464: [BEAM-4488], [BEAM-4489], [BEAM-4490] Improvements to contribution guide URL: https://github.com/apache/beam-site/pull/464#discussion_r194150629 ## File path: src/contribute/index.md ## @@ -215,6 +301,12 @@ Easy to use Java 8 DSL for the Beam Java SDK. Provides a high-level abstraction - JIRA: [dsl-euphoria](https://issues.apache.org/jira/browse/BEAM-4366?jql=project%20%3D%20BEAM%20AND%20component%20%3D%20dsl-euphoria) / [BEAM-3900](https://issues.apache.org/jira/browse/BEAM-3900) - Contact: [David Moravek](mailto:david.mora...@gmail.com) +### Improving the contributor experience + +Making it easier to write code, run tests, and release. Investigating using docker for jenkins builds, automating the release process, and improving the reliability of tests. + +Ideas and help welcome! Contact: [Alan Myrvold](mailto:amyrv...@google.com), [Mark Liu](mailto:mark...@google.com), [Yifan Zou](mailto:yifan...@google.com) Review comment: Not an objection, but a question. Is there a more generic contact we can use instead of direct personal contact? The reason is we would not like to hot-spot these people forever, and we would like to exclude others. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 110224) Time Spent: 1h 40m (was: 1.5h) > Contributor's guide needs to document how to install Python SDK from source > --- > > Key: BEAM-4488 > URL: https://issues.apache.org/jira/browse/BEAM-4488 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Pablo Estrada >Assignee: Alan Myrvold >Priority: Major > Time Spent: 1h 40m > Remaining Estimate: 0h > > Currently it is not documented, though it does appear in the quickstart -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4488) Contributor's guide needs to document how to install Python SDK from source
[ https://issues.apache.org/jira/browse/BEAM-4488?focusedWorklogId=110183&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-110183 ] ASF GitHub Bot logged work on BEAM-4488: Author: ASF GitHub Bot Created on: 08/Jun/18 17:37 Start Date: 08/Jun/18 17:37 Worklog Time Spent: 10m Work Description: alanmyrvold commented on issue #464: [BEAM-4488], [BEAM-4489], [BEAM-4490] Improvements to contribution guide URL: https://github.com/apache/beam-site/pull/464#issuecomment-395833919 @aaltay Any comments? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 110183) Time Spent: 1.5h (was: 1h 20m) > Contributor's guide needs to document how to install Python SDK from source > --- > > Key: BEAM-4488 > URL: https://issues.apache.org/jira/browse/BEAM-4488 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Pablo Estrada >Assignee: Alan Myrvold >Priority: Major > Time Spent: 1.5h > Remaining Estimate: 0h > > Currently it is not documented, though it does appear in the quickstart -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4488) Contributor's guide needs to document how to install Python SDK from source
[ https://issues.apache.org/jira/browse/BEAM-4488?focusedWorklogId=109855&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-109855 ] ASF GitHub Bot logged work on BEAM-4488: Author: ASF GitHub Bot Created on: 07/Jun/18 20:46 Start Date: 07/Jun/18 20:46 Worklog Time Spent: 10m Work Description: alanmyrvold commented on a change in pull request #464: [BEAM-4488], [BEAM-4489], [BEAM-4490] Improvements to contribution guide URL: https://github.com/apache/beam-site/pull/464#discussion_r193883951 ## File path: src/contribute/index.md ## @@ -78,29 +97,90 @@ necessary things to run those tests. For example: $ ./gradlew -p sdks/java/io/cassandra check $ ./gradlew -p runners/flink check -### Testing the Python SDK +You can see what build tasks are available with + +$ ./gradlew tasks + +or for a module, + +$ ./gradlew sdks/java/io/cassandra tasks + +### Developing with an IDE + +Generate an IDEA project .ipr file with: + +$ ./gradlew idea + +### Pull requests + +When your change is ready to be reviewed and merged, create a pull request. +Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, Review comment: done This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 109855) Time Spent: 1h 10m (was: 1h) > Contributor's guide needs to document how to install Python SDK from source > --- > > Key: BEAM-4488 > URL: https://issues.apache.org/jira/browse/BEAM-4488 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Pablo Estrada >Assignee: Alan Myrvold >Priority: Major > Time Spent: 1h 10m > Remaining Estimate: 0h > > Currently it is not documented, though it does appear in the quickstart -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4488) Contributor's guide needs to document how to install Python SDK from source
[
https://issues.apache.org/jira/browse/BEAM-4488?focusedWorklogId=109856&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-109856
]
ASF GitHub Bot logged work on BEAM-4488:
Author: ASF GitHub Bot
Created on: 07/Jun/18 20:46
Start Date: 07/Jun/18 20:46
Worklog Time Spent: 10m
Work Description: alanmyrvold commented on a change in pull request #464:
[BEAM-4488], [BEAM-4489], [BEAM-4490] Improvements to contribution guide
URL: https://github.com/apache/beam-site/pull/464#discussion_r193883969
##
File path: src/contribute/index.md
##
@@ -78,29 +97,90 @@ necessary things to run those tests. For example:
$ ./gradlew -p sdks/java/io/cassandra check
$ ./gradlew -p runners/flink check
-### Testing the Python SDK
+You can see what build tasks are available with
+
+$ ./gradlew tasks
+
+or for a module,
+
+$ ./gradlew sdks/java/io/cassandra tasks
+
+### Developing with an IDE
+
+Generate an IDEA project .ipr file with:
+
+$ ./gradlew idea
+
+### Pull requests
+
+When your change is ready to be reviewed and merged, create a pull request.
+Format the pull request title like [BEAM-XXX] Fixes bug in
ApproximateQuantiles,
+where you replace BEAM-XXX with the appropriate JIRA issue.
+This will automatically link the pull request to the issue.
+
+Pull requests can only be merged by a [beam committer]
+(http://people.apache.org/phonebook.html?unix=beam).
+To find a committer for your area, look for similar code merges or ask on
+[d...@beam.apache.org]({{ site.baseurl }}/community/contact-us/)
+
+Use @mention in the pull request to notify the reviewer.
+
+The pull request and any changes pushed to it will trigger precommit jobs.
+If a test fails and appears unrelated to your change, you can cause tests
+to be re-run by adding a single line comment on your PR
+
+ retest this please
+
+There are other trigger phrases for post-commit tests found in
+.testinfra/jenkins, but use these sparingly because postcommit
+tests consume shared development resources.
+
+### Developing with the Python SDK
+
+Gradle can build and test python, and is used by the Jenkins jobs, so needs to
+be maintained.
You can directly use the Python toolchain instead of having Gradle orchestrate
-it. This may be faster for you. We recommend setting up a virtual environment
-before testing your code.
+it, which may be faster for you, but it is your preference.
+If you do want to use Python tools directly, we recommend setting up a virtual
+environment before testing your code.
If you update any of the [cythonized](http://cython.org) files in Python SDK,
you must install the `cython` package before running following command to
properly test your code.
The following commands should be run in the `sdks/python` directory.
-This command runs all Python tests.
+This installs Python from source and includes the test and gcp dependencies.
+
+On macOS/Linix:
+
+$ virtualenv env
+$ . ./env/bin/activate
+(env) $ pip install .[gcp,test]
-$ python setup.py nosetests
+On Windows:
+
+> c:\Python27\python.exe -m virtualenv
+> env\Scripts\activate
+(env) > pip install .[gcp,test]
+
+This command runs all Python tests. The nose dependency is installed by [test]
in pip install.
+
+(env) $ python setup.py nosetests
You can use following command to run a single test method.
-$ python setup.py nosetests --tests :.
+(env) $ python setup.py nosetests --tests :.
+
+For example:
+(env) $ python setup.py nosetests --tests
apache_beam.io.textio_test:TextSourceTest.test_progress
-Example:
-$ python setup.py nosetests --tests
apache_beam.io.textio_test:TextSourceTest.test_progress
+You can deactivate the virtualenv when done.
-To check just for lint errors, run the following command.
+(env) $ deactivate
+$
+
+To check just for Python lint errors, run the following command.
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 109856)
Time Spent: 1h 20m (was: 1h 10m)
> Contributor's guide needs to document how to install Python SDK from source
> ---
>
> Key: BEAM-4488
> URL: https://issues.apache.org/jira/browse/BEAM-4488
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py-core
>Reporter: Pablo Estrada
>Assignee: Alan Myrvold
>Priority: Major
> Time Spent: 1h 20m
> Remainin
[jira] [Work logged] (BEAM-4488) Contributor's guide needs to document how to install Python SDK from source
[
https://issues.apache.org/jira/browse/BEAM-4488?focusedWorklogId=109854&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-109854
]
ASF GitHub Bot logged work on BEAM-4488:
Author: ASF GitHub Bot
Created on: 07/Jun/18 20:46
Start Date: 07/Jun/18 20:46
Worklog Time Spent: 10m
Work Description: alanmyrvold commented on a change in pull request #464:
[BEAM-4488], [BEAM-4489], [BEAM-4490] Improvements to contribution guide
URL: https://github.com/apache/beam-site/pull/464#discussion_r193883932
##
File path: src/contribute/index.md
##
@@ -19,45 +19,59 @@ The Apache Beam community welcomes contributions from
anyone!
There are lots of opportunities:
- - write new examples
+ - ask or answer questions on [u...@beam.apache.org]({{ site.baseurl
+}}/community/contact-us/) or
+[stackoverflow](https://stackoverflow.com/questions/tagged/apache-beam)
+ - review proposed design ideas on [d...@beam.apache.org]({{ site.baseurl
+}}/community/contact-us/)
- improve the documentation
+ - contribute bug reports
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 109854)
Time Spent: 1h (was: 50m)
> Contributor's guide needs to document how to install Python SDK from source
> ---
>
> Key: BEAM-4488
> URL: https://issues.apache.org/jira/browse/BEAM-4488
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py-core
>Reporter: Pablo Estrada
>Assignee: Alan Myrvold
>Priority: Major
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Currently it is not documented, though it does appear in the quickstart
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-4488) Contributor's guide needs to document how to install Python SDK from source
[
https://issues.apache.org/jira/browse/BEAM-4488?focusedWorklogId=109633&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-109633
]
ASF GitHub Bot logged work on BEAM-4488:
Author: ASF GitHub Bot
Created on: 07/Jun/18 04:31
Start Date: 07/Jun/18 04:31
Worklog Time Spent: 10m
Work Description: pabloem commented on a change in pull request #464:
[BEAM-4488], [BEAM-4489], [BEAM-4490] Improvements to contribution guide
URL: https://github.com/apache/beam-site/pull/464#discussion_r193622173
##
File path: src/contribute/index.md
##
@@ -19,45 +19,59 @@ The Apache Beam community welcomes contributions from
anyone!
There are lots of opportunities:
- - write new examples
+ - ask or answer questions on [u...@beam.apache.org]({{ site.baseurl
+}}/community/contact-us/) or
+[stackoverflow](https://stackoverflow.com/questions/tagged/apache-beam)
+ - review proposed design ideas on [d...@beam.apache.org]({{ site.baseurl
+}}/community/contact-us/)
- improve the documentation
+ - contribute bug reports
Review comment:
Maybe link Jira? : ) https://issues.apache.org/jira/projects/BEAM/issues
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 109633)
Time Spent: 50m (was: 40m)
> Contributor's guide needs to document how to install Python SDK from source
> ---
>
> Key: BEAM-4488
> URL: https://issues.apache.org/jira/browse/BEAM-4488
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py-core
>Reporter: Pablo Estrada
>Assignee: Alan Myrvold
>Priority: Major
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Currently it is not documented, though it does appear in the quickstart
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-4488) Contributor's guide needs to document how to install Python SDK from source
[ https://issues.apache.org/jira/browse/BEAM-4488?focusedWorklogId=109632&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-109632 ] ASF GitHub Bot logged work on BEAM-4488: Author: ASF GitHub Bot Created on: 07/Jun/18 04:31 Start Date: 07/Jun/18 04:31 Worklog Time Spent: 10m Work Description: pabloem commented on a change in pull request #464: [BEAM-4488], [BEAM-4489], [BEAM-4490] Improvements to contribution guide URL: https://github.com/apache/beam-site/pull/464#discussion_r193622481 ## File path: src/contribute/index.md ## @@ -78,29 +97,90 @@ necessary things to run those tests. For example: $ ./gradlew -p sdks/java/io/cassandra check $ ./gradlew -p runners/flink check -### Testing the Python SDK +You can see what build tasks are available with + +$ ./gradlew tasks + +or for a module, + +$ ./gradlew sdks/java/io/cassandra tasks + +### Developing with an IDE + +Generate an IDEA project .ipr file with: + +$ ./gradlew idea + +### Pull requests + +When your change is ready to be reviewed and merged, create a pull request. +Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, Review comment: Maybe add backticks or quotes for the title of the Pull Request? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 109632) Time Spent: 40m (was: 0.5h) > Contributor's guide needs to document how to install Python SDK from source > --- > > Key: BEAM-4488 > URL: https://issues.apache.org/jira/browse/BEAM-4488 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Pablo Estrada >Assignee: Alan Myrvold >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > Currently it is not documented, though it does appear in the quickstart -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4488) Contributor's guide needs to document how to install Python SDK from source
[
https://issues.apache.org/jira/browse/BEAM-4488?focusedWorklogId=109631&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-109631
]
ASF GitHub Bot logged work on BEAM-4488:
Author: ASF GitHub Bot
Created on: 07/Jun/18 04:31
Start Date: 07/Jun/18 04:31
Worklog Time Spent: 10m
Work Description: pabloem commented on a change in pull request #464:
[BEAM-4488], [BEAM-4489], [BEAM-4490] Improvements to contribution guide
URL: https://github.com/apache/beam-site/pull/464#discussion_r193622831
##
File path: src/contribute/index.md
##
@@ -78,29 +97,90 @@ necessary things to run those tests. For example:
$ ./gradlew -p sdks/java/io/cassandra check
$ ./gradlew -p runners/flink check
-### Testing the Python SDK
+You can see what build tasks are available with
+
+$ ./gradlew tasks
+
+or for a module,
+
+$ ./gradlew sdks/java/io/cassandra tasks
+
+### Developing with an IDE
+
+Generate an IDEA project .ipr file with:
+
+$ ./gradlew idea
+
+### Pull requests
+
+When your change is ready to be reviewed and merged, create a pull request.
+Format the pull request title like [BEAM-XXX] Fixes bug in
ApproximateQuantiles,
+where you replace BEAM-XXX with the appropriate JIRA issue.
+This will automatically link the pull request to the issue.
+
+Pull requests can only be merged by a [beam committer]
+(http://people.apache.org/phonebook.html?unix=beam).
+To find a committer for your area, look for similar code merges or ask on
+[d...@beam.apache.org]({{ site.baseurl }}/community/contact-us/)
+
+Use @mention in the pull request to notify the reviewer.
+
+The pull request and any changes pushed to it will trigger precommit jobs.
+If a test fails and appears unrelated to your change, you can cause tests
+to be re-run by adding a single line comment on your PR
+
+ retest this please
+
+There are other trigger phrases for post-commit tests found in
+.testinfra/jenkins, but use these sparingly because postcommit
+tests consume shared development resources.
+
+### Developing with the Python SDK
+
+Gradle can build and test python, and is used by the Jenkins jobs, so needs to
+be maintained.
You can directly use the Python toolchain instead of having Gradle orchestrate
-it. This may be faster for you. We recommend setting up a virtual environment
-before testing your code.
+it, which may be faster for you, but it is your preference.
+If you do want to use Python tools directly, we recommend setting up a virtual
+environment before testing your code.
If you update any of the [cythonized](http://cython.org) files in Python SDK,
you must install the `cython` package before running following command to
properly test your code.
The following commands should be run in the `sdks/python` directory.
-This command runs all Python tests.
+This installs Python from source and includes the test and gcp dependencies.
+
+On macOS/Linix:
+
+$ virtualenv env
+$ . ./env/bin/activate
+(env) $ pip install .[gcp,test]
-$ python setup.py nosetests
+On Windows:
+
+> c:\Python27\python.exe -m virtualenv
+> env\Scripts\activate
+(env) > pip install .[gcp,test]
+
+This command runs all Python tests. The nose dependency is installed by [test]
in pip install.
+
+(env) $ python setup.py nosetests
You can use following command to run a single test method.
-$ python setup.py nosetests --tests :.
+(env) $ python setup.py nosetests --tests :.
+
+For example:
+(env) $ python setup.py nosetests --tests
apache_beam.io.textio_test:TextSourceTest.test_progress
-Example:
-$ python setup.py nosetests --tests
apache_beam.io.textio_test:TextSourceTest.test_progress
+You can deactivate the virtualenv when done.
-To check just for lint errors, run the following command.
+(env) $ deactivate
+$
+
+To check just for Python lint errors, run the following command.
Review comment:
We have `tox` commands that run lint tasks. I am not sure, but maybe it'd be
worth mentioning them
tox -e py27-lint # For python 2.7
tox -e py3-lint# For python 3
tox -e py27-lint3 # For python 2-3 compatibility
All of these run in precommits
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 109631)
Time Spent: 0.5h (was: 20m)
> Contributor's guide needs to document how to install Python SDK from source
> ---
>
> Key: BEAM-4488
> URL: https://iss
[jira] [Work logged] (BEAM-4488) Contributor's guide needs to document how to install Python SDK from source
[ https://issues.apache.org/jira/browse/BEAM-4488?focusedWorklogId=109583&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-109583 ] ASF GitHub Bot logged work on BEAM-4488: Author: ASF GitHub Bot Created on: 07/Jun/18 00:30 Start Date: 07/Jun/18 00:30 Worklog Time Spent: 10m Work Description: alanmyrvold commented on issue #464: [BEAM-4488], [BEAM-4489], [BEAM-4490] Improvements to contribution guide URL: https://github.com/apache/beam-site/pull/464#issuecomment-395255310 @aaltay @pabloem PTAL This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 109583) Time Spent: 20m (was: 10m) > Contributor's guide needs to document how to install Python SDK from source > --- > > Key: BEAM-4488 > URL: https://issues.apache.org/jira/browse/BEAM-4488 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Pablo Estrada >Assignee: Alan Myrvold >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Currently it is not documented, though it does appear in the quickstart -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4488) Contributor's guide needs to document how to install Python SDK from source
[ https://issues.apache.org/jira/browse/BEAM-4488?focusedWorklogId=109582&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-109582 ] ASF GitHub Bot logged work on BEAM-4488: Author: ASF GitHub Bot Created on: 07/Jun/18 00:29 Start Date: 07/Jun/18 00:29 Worklog Time Spent: 10m Work Description: alanmyrvold opened a new pull request #464: [BEAM-4488], [BEAM-4489], [BEAM-4490] Improvements to contribution guide URL: https://github.com/apache/beam-site/pull/464 [BEAM-4488] How to install Python SDK from source [BEAM-4489] Document expectations for tools to use with Python [BEAM-4490] Document the pull request workflow This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 109582) Time Spent: 10m Remaining Estimate: 0h > Contributor's guide needs to document how to install Python SDK from source > --- > > Key: BEAM-4488 > URL: https://issues.apache.org/jira/browse/BEAM-4488 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Pablo Estrada >Assignee: Alan Myrvold >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Currently it is not documented, though it does appear in the quickstart -- This message was sent by Atlassian JIRA (v7.6.3#76005)
