Thanks for the interest!
I will try to provide a detailed writeup about the project.
Generally, the idea is to replace current communication layer by a pure
message bus implementation. Below you can find some things from the top of
my head:
- All communications between master and agent happen over the message bus
- Jenkins agents and masters can connect to a message bus.
- The topology is TBD, it may be either embedded message bus (Jenkins
master as a host) or an external service. The latter one is preferable
IMHO
- Jenkins agents query connection metadata over the bus.
- Current implementations query metadata from an HTTP/HTTPs endpoint
before connection. Getting rid of that would help
- Command transport for message bus is implemented (as Jesse
mentioned).
- It would include transparent support of command invocation, RMI,
classloading and data streaming
- We utilize message bus features to make connections over
Remoting more stable
- Fault tolerance
- The messaging layer survives short network outages
- Nice2Have: We use multiple message bus hosts so they do not
become a bottleneck
- Traffic/Request prioritization
- Currently we have an issue with logging/file transfer messaging
overloading the channel and blocking system commands. We could use
message
bus features to prevent that
- TBD: More efficient data streaming
- Implementation:
- Executable for the agent side
- Jenkins plugin, which provides the master-side logic and serves the
agent executable
- We have a reference implementation for demos
- E.g: a set of Docker images, which bring up a Jenkins cluster with a
master and several agents (e.g. with Kubernetes or Docker Compose)
- It may be somehow aligned with other ongoing activities (e.g. the
demo could be built on the top of EverGreen distro)
<https://github.com/jenkinsci/jep/tree/master/jep/300>
- TBD: a PoC for Master <=> Master communication
- In Jenkins project we have some master => master plugins like
Parameterized Remote Trigger. We could try to use a message bus to
deliver
messages between masters
- TBD: Streaming agent logs to a log storage over the message bus
- TBD: Remoting autoupdate over message bus
- Message bust engine could be a bootstrap library, which loads
Remoting library from the master after establishing the connection.
- In Jenkins project we often see issues with update Remoting
versions on the agent side. Although it would be great to solve the issue
on the Remoting side, usage of such bootstrap could be a quick win
Best regards,
Oleg
вторник, 13 февраля 2018 г., 16:53:00 UTC+1 пользователь Jesse Glick
написал:
>
> On Tue, Feb 13, 2018 at 9:49 AM, Pham Vu Tuan <phamvu...@gmail.com
> <javascript:>> wrote:
> > I am quite interested in project
> > "Jenkins Remoting over Message Bus/Queue" and want to understand more
> about
> > the project.
>
> Probably Oleg will give you some depth but to get started
> understanding the background, look at subclasses of:
>
>
> https://github.com/jenkinsci/remoting/blob/master/src/main/java/hudson/remoting/CommandTransport.java
>
>
--
You received this message because you are subscribed to the Google Groups
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-dev/23b3c653-bdab-4173-a043-ac270b5f7961%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
