package ui templ About() {
Converge is a utility for troubleshooting builds on continuous integration servers. It solves a common problem where the cause of job failure is difficult to determine. This is complicated further by the fact that build jobs are usually run on a build farm where there is no access to the build agents or in more modern envrionments when jobs are run in ephemeral containers.
With Converge it is possible to get remote shell access to such jobs. This works by configuring the build job to connect to a Converge server using an agent program. The agent program can be downloaded from within the CI job using curl or wget. Next, an end-user can connect to the Converge server, a rendez-vous server, that connects the client and server together based on a common identifier specified by both client and server.
The basic principle of converge is described below. Access to a running remote continous integration job is usually not possible without a lot of access to the backend environment where jobs are running. However, the job can connect to a server running outside, and so can the client.
wsproxy
that is used as a proxy command for SSH allowing SSH to connect to the websocket. Using wsproxy, the
rendez-vous id is passed to the server. There are a few special situations:
The setup is such that the connection from client (end-user) to server (agent on CI job) is end-to-end encrypted. The Converge server itself is no more than a bitpipe which pumps data between client and agent.
Using authorized keys is a secure way of connecting. When running the agent, the authorized keys
must be put in a file, allowing only the designated users to connect. The file containing authorized keys
can also be edited during a session with the agent, allowing more people to be added when required without
having to start over again.
Using authorized keys is made easy through the
usage page, which provides the exact commands to execute based
on the target environment. If users are hesitant to use their public key it is also possible
to generate a separate ssh key-pair using ssh-keygen
and use that instead.
To be able to use Converge, you must already have access to the configuration of a build job. Having that access means it is possible to execute any command on a build agent. The Converge agent is started by the build job and does not have any additional rights compared to what you could script in the continous integration job definition.
Converge does not provide any stealth features to hide it. The public sessions page show all agents and clients including details about the clients and the agents. The idea is that it should be light-weight and easy to use. There is no reason to hide the fact that someone is debugging a continuous integration job. Also, all sessions are logged,both using standard kubernetes tooling such as (fluentbit/filebeat, and loki/elasticsearch depending on the environment). This logging includes only the details about the sessions, but not what the user is doing inside a session. Also, Converge provides a prometheus metrics endpoint which allows user sessions to be tracked over time after the fact. Thie data is also made accessible using a grafana dashboard.
Both ssh and sftp are supported. Multiple concurrent sessions to same agent are allowed as well as multiple agents are also allowed.
There is a timeout mechanism in the agent such that jobs do not hang indefinitely waiting for a connection. This mechanism is useful to make sure build agents do not keep build agents occupied for a long time. By default, the agent exits with status 0 when the last client exits after logging in. The timeout is an inactivity timeout. Activity is detected as follows:
ls
command
in an sftp session will also lead to activity since the server will output the result of the command. When the user touches a .hold file, the agent keeps waiting for connections even after the last client logs out, taking into account the timeout. By default the agent exits when the last user has logged out.
The agent supports a --shells command-line option by which a comma-separated
list of shells can be prepended to the default search path for shells, e.g.
--shells zsh,csh,sh
(linux) or cmd,powershell
for
windows.
The agent sets an
The user will get notifications from the agent any time something important happens such as the session being close to timeout.
Using available existing tools such as breakpoint in combination with a websocket tunneling tool such as wstunnel a similar solution can be obtained. There are however some problems with these solutions that converge is trying to address:
wsproxy
is provided that allows SSH to connect using websockets.
wsproxy
. This is an SSH proxy command that communicates with converge
and provides additional information to the user.