bamaer opened a new pull request, #8331:
URL: https://github.com/apache/hop/pull/8331
## What
Adds a **Git Input** transform and a **Git connection** metadata type: read
commits, commit
files, issues, pull requests, issue comments, PR comments and issue events
from GitHub
(cloud and Enterprise), GitLab, Bitbucket, Forgejo and Gitea — or straight
from a local
clone with JGit, no credentials needed.
Every provider is normalised to the same output row, so a pipeline written
against GitHub
keeps working against GitLab. Timestamps come out as real Date values.
Fixes #8314
## Where
`plugins/misc/git`, alongside the existing Git perspective, following the
`plugins/misc/mail`
precedent of a metadata connection type shipping with its transforms. That
module already
declares JGit at the version this needs, so no new module and no assembly
wiring.
## Changes outside the plugin
- One new `HopMetadataPropertyType` constant (`GIT_CONNECTION`) in `core`
- `json-simple` (provided — already shipped via hop-core) and WireMock
(test) in the plugin pom
- **No new runtime dependencies, so no LICENSE/NOTICE changes**
- Docs: `pipeline/transforms/gitinput.adoc` and
`metadata-types/git-connection.adoc`, plus nav
and index entries
## Testing
- 291 unit tests, incl. WireMock coverage of all four provider clients
- 2 integration tests in `integration-tests/transforms`, run against the
live GitHub API.
`0111` pins commits to an immutable tag, giving a genuine golden data set
at one request;
`0112` asserts the row contract for pull requests, which are a moving
target.
## Known limitations
- **Bitbucket is implemented from API documentation and captured payloads,
not exercised
against a live instance.** Reports from a Bitbucket user welcome.
- **The integration tests authenticate anonymously**, deliberately: this is
a public repo and
repo secrets are widely readable. They are therefore subject to GitHub's
60 requests/hour
anonymous limit. A rate-limited CI run looks like a ~3 minute pause
followed by an HTTP 403,
not a functional failure.
- Repository search in the Browse dialog is server-side on GitLab and
Bitbucket, client-side
on GitHub and Forgejo, so Browse fetches the full list on those two.
## Follow-ups (not in this PR)
- Server-side repository search for GitHub (`/search/repositories`) and
Forgejo
(`/repos/search`) — removes the full-list fetch. Note GitHub's search has
its own rate pool.
- `ISSUE_COMMENTS` / `PR_COMMENTS` / `ISSUE_EVENTS` for Gitea and Forgejo,
which currently
report the type as unsupported rather than returning nothing.
- An SWTBot test driving the transform dialog; there is no precedent for one
in the repo today.
------------------------
Thank you for your contribution! Follow this checklist to help us
incorporate your contribution quickly and easily:
- [x] Run `mvn clean install apache-rat:check` to make sure basic checks
pass. A more thorough check will be performed on your pull request
automatically.
- [x] If you have a group of commits related to the same change, please
squash your commits into one and force push your branch using `git rebase -i`.
- [x] Mention the appropriate issue in your description (for example:
`addresses #123`), if applicable.
To make clear that you license your contribution under the [Apache License
Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0)
you have to acknowledge this by using the following check-box.
- [x] I hereby declare this contribution to be licensed under the [Apache
License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0)
- [ ] In any other case, please file an [Apache Individual Contributor
License Agreement](https://www.apache.org/licenses/icla.pdf).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
