package templates templ Index(secure string, host string, username string) { Converge

About

Converge is a utility for troubleshooting builds on continuous integration servers. It solves a common problem where the cause of job failure is difficult to determine. This is complicated further by the fact that build jobs are usually run on a build farm where there is no access to the build agents or in more modern envrionments when jobs are run in ephemeral containers.

With Converge it is possible to get remote shell access to such jobs. This works by configuring the build job to connect to a Converge server using an agent program. The agent program can be downloaded from within the CI job using curl or wget. Next, an end-user can connect to the Converge server, a rendez-vous server, that connects the client and server together.

The setup is such that the connection from client (end-user) to server (agent on CI job) is end-to-end encrypted. The Converge server itself is no more than a bitpipe which pumps data between client and agent.

Both ssh and sftp are supported. Multiple shells are also allowed.

There is a timeout mechanism in the agent such that jobs do not hang indefinitely waiting for a connection. This mechanism is useful to make sure build agents do not keep build agents occupied for a long time. By default, the agent exits with status 0 when the first client exits after logging in. This behavior as well as general expiry can be controlled from within a shell session by touching a .hold file. After logging in, the user can control expiry of the session as instructed by messages in the ssh session. When the timeout of a session is near the user is informed about this with messages in the shell.

Usage

Continous integration jobs

In a continous integration job, download the agent, chmod it and run it. 

{`
         # linux
         `}curl http{secure}://{host}/docs/agent > agent{`
         chmod 755 agent
         `}./agent --id ID ws{secure}://{host}{`

         # windows
         `}curl http{secure}://{host}/docs/agent.exe > agent.exe{`
         `}agent --id ID ws{secure}://{host}{`
        `}

Above, ID is a unique id for the job, the so-called rendez-cous ID. This should not conflict with IDs used by other agents. The ID is used for a rendez-vous between the end-user on a local system and the continuous integration job running on a build agent. If you don't specify an id, a random id will be generated. The agent to the converge server and tells it the ID. Clients can now connect to the Converge server to establish a connection to the CI job through converge by also specifying the same ID. Communication between end-user and agent is encrypted using SSH and the rendez-vous server is unable to read the contents. The rendez-vous server is nothing more then a glorified bit pipe, simply transferring data between end-user SSH client and the agent which runs an embedded SSH server.

NOTE: When running the agent on windows, an exit of the remote session using exit in powershell or command prompt does not terminate the shell completely. To terminate the client ssh session must be killed by closing the terminal. Cleanup of remote processes on the agent appears to work properly despite this problem. It is just that exit of the windows shell (powershell or command prompt) is not detected properly.

The agent has more options, download the agent and run it without arguments to see all options.

Local clients: using ssh with a proxy command

wsproxy is a command that can be used as a proxy command for SSH which performs the connection to the remote server. This command needs to be downloaded only once (see downloads below). It does not depend on the converge implementation but only on the websocket standards. Other tools that provide a mapping of stdio to a websocket can also be used instead of wsproxy.

Next step is to run a local SSH or SFTP client: 

                                                                                                                            {`
        `}ssh -oServerAliveInterval=10 -oProxyCommand="wsproxy ws{secure}://{host}/client/ID"  { username }{"@localhost"}   {`
        `}sftp -oServerAliveInterval=10 -oProxyCommand="wsproxy ws{secure}://{host}/client/ID" { username }{"@localhost"}   {`
        `}

Local clients: using SSH with a local TCP forwarding proxy

This option is less convenient than the proxy command because it requires two separate commands to execute.

Local clients can connect using regular ssh and sftp commands through a tunnel that translates a local TCP port to a websocket connection in converge. See the downloads section. This runs a local client that allows SSH to port 10000 and connects to converge using a websocket connection.

Next step is to run a local SSH of SFTP client: 

                                                           {`
        `}ssh -oServerAliveInterval=10 -p 10000 { username }{"@localhost"}          {`
        `}sftp -oServerAliveInterval=10 -oPort=10000 { username }{"@localhost"}     {`
        `}

Authentication

The { username } user above the Converge server and communicated to the agent when the agent is started. This is the username that must be used when setting up an ssh connection. Another way to authenticate is through an .authorized_keys file in the same directory as where the agent is started. This can be setup as follows before starting the agent: 

                                         {`
        `}# linux                                                 {`
        `}echo "ssh-rsa dkddkdkkk a@b.c" > .authorized_keys       {`
        `}echo "ssh-rsa adfadjfdf d@e.f" >> .authorized_keys      {`
        `}                                                        {`
        `}# windows                                               {`
        `}echo ssh-rsa dkddkdkkk a@b.c > .authorized_keys         {`
        `}echo ssh-rsa adfadjfdf d@e.f >> .authorized_keys

Note that on windows you should not used quotes.

Downloads

Component Purpose Linux Windows
agent The agent to run inside aa CI job agent agent.exe
wsproxy SSH proxy command that can be directly used by ssh wsproxy wsproxy.exe
tcptows TCP to WS tunnel for allowing regular SSH and SFTP clients to connect to converge tcptows tcptows.exe
}