Converge is a utility for troubleshooting builds on continuous integration servers. It solves a common problem where the cause of job failure is difficult to determine. This is complicated further by the fact that build jobs are usually run on a build farm where there is no access to the build agents or in more modern envrionments when jobs are run in ephemeral containers.
With Converge it is possible to get remote shell access to such jobs. This works by configuring the build job to connect to a Converge server using an agent program. The agent program can be downloaded from within the CI job using curl or wget. Next, an end-user can connect to the Converge server, a rendez-vous server, that connects the client and server together.
The setup is such that the connection from client (end-user) to server (agent on CI job) is end-to-end encrypted. The Converge server itself is no more than a bitpipe which pumps data between client and agent.
Both ssh and sftp are supported. Multiple shells are also allowed.
There is a timeout mechanism in the agent such that jobs do not hang indefinitely waiting for a connection. This mechanism is useful to make sure build agents do not keep build agents occupied for a long time. By default, the agent exits with status 0 when the first client exits after logging in. This behavior as well as general expiry can be controlled from within a shell session by touching a .hold file. After logging in, the user can control expiry of the session as instructed by messages in the ssh session. When the timeout of a session is near the user is informed about this with messages in the shell.
In a continous integration job, download the agent, chmod it and run it.
# linux
curl http{{.secure}}://{{.host}}/docs/agent > agent
chmod 755 agent
./agent ws{{.secure}}://{{.host}}/agent/ID
# windows
curl http{{.secure}}://{{.host}}/docs/agent.exe > agent.exe
agent ws{{.secure}}://{{.host}}/agent/ID
Above, ID is a unique id for the job. This should not conflict with other ids. This connects the agent to the converge server. Clients can now connect to the Converge server to establish a connection to the CI job through converge.
NOTE: When running the agent on windows, an exit of the remote session using exit in powershell or command prompt does not terminate the shell completely. To terminate the client ssh session must be killed by closing the terminal. Cleanup of remote processes on the agent appears to work properly despite this problem. It is just that exit of the windows shell (powershell or command prompt) is not detected properly.
wsproxy is a command that can be used as a proxy command for SSH which performs the connection to the
remote server. This command needs to be downloaded only once (see downloads below). It does not depend on
the converge implementation but only on the websocket standards. Other tools that
provide a mapping of stdio to a websocket can also be used instead of wsproxy.
Next step is to run a local SSH or SFTP client:
ssh -oServerAliveInterval=10 -oProxyCommand="wsproxy ws{{.secure}}://{{.host}}/client/ID" abc@localhost
sftp -oServerAliveInterval=10 -oProxyCommand="wsproxy ws{{.secure}}://{{.host}}/client/ID" abc@localhost
abc is a fixed user defined by converge. It has a very exciting password.
This option is less convenient than the proxy command because it requires two separate commands to execute.
Local clients can connect using regular ssh and sftp commands through a tunnel that translates a local TCP port to a websocket connection in converge. See the downloads section. This runs a local client that allows SSH to port 10000 and connects to converge using a websocket connection.
Next step is to run a local SSH of SFTP client:
ssh -oServerAliveInterval=10 -p 10000 abc@localhost
sftp -oServerAliveInterval=10 -oPort=10000 abc@localhost
abc is a fixed user defined by converge. It has a very exciting password.
| Component | Purpose | Linux | Windows |
|---|---|---|---|
| agent | The agent to run inside aa CI job | agent | agent.exe |
| wsproxy | SSH proxy command that can be directly used by ssh | wsproxy | wsproxy.exe |
| tcptows | TCP to WS tunnel for allowing regular SSH and SFTP clients to connect to converge | tcptows | tcptows.exe |