Building a Concourse CI VM on Ubuntu
Recently, I’ve found myself needing a Concourse CI system. I struggled with the documentation on concourse.ci, couldn’t find any comprehensive build guides. Knew for certain I wasn’t going to use VirtualBox. So, having worked it out; thought I’d share what I went through to get to a working system.
Discovered that the CentOS version I was using previously did not have a compatible Linux kernel version. CentOS 7.2 uses kernel 3.10, Concourse requires 3.19+. So, I’m starting with a freshly-deployed Ubuntu Server 16.04 LTS this time.
Not a lot we have to do, but still pretty important:
- Make sure port for concourse is open
sudo ufw allow 8080
sudo ufw status
sudo ufw disable
I disabled the firewall on ubuntu because it was preventing the concourse worker and concourse web from communicating.
- Update and make sure wget is installed
apt-get install wget
Concourse expects to use a postgresql database, I don’t have one standing by, so let’s install it.
- Pretty straightforward on Ubuntu too:
apt-get install postgresql postgresql-contrib
Enter y to install the bits. On Ubuntu, we don’t have to take extra steps to configure the service.
- Ok, now we have to create an account and a database for concourse. First, lets create the linux account. I’m calling mine “concourse” because I’m creative like that.
- Next, we create the account (aka “role” or “user”) in postgres via the createuser command. In order to do this, we have to switch to the postgres account, do that with sudo:
sudo -i -u postgres
Now, while in as postgres we can use the createuser command
You’ll enter the name of the account, and answer a couple of special permissions questions.
- While still logged in as postgres, run this command to create a new database for concourse. I’m naming my database “concourse” – my creativity is legendary. Actually, I think it makes life easier if the role and database are named the same
- Test by switching users to the concourse account and making sure it can run psql against the concourse databaseWhile in psql, use this command to set the password for the account in postgress
ALTER ROLE concourse WITH PASSWORD 'changeme';
- Type \q to exit psql
Ok, we have a running postgresql service and and account to be used for concourse. Let’s go.
- Create a folder for concourse. I used /concourse, but you can use /var/lib/whatever/concourse if you feel like it.
- Download the binary from concourse.ci/downloads.html into your /concourse folder using wget or transfer via scp.
- Create a symbolic link named “concourse” to the file you downloaded and make it executable
ln -s ./concourse_linux_amd64 ./concourse
chmod +x ./concourse_linux_amd64
- Create keys for concourse
mkdir -p keys/web keys/worker
ssh-keygen -t rsa -f ./keys/web/tsa_host_key -N ”
ssh-keygen -t rsa -f ./keys/web/session_signing_key -N ”
ssh-keygen -t rsa -f ./keys/worker/worker_key -N ”
cp ./keys/worker/worker_key.pub ./keys/web/authorized_worker_keys
cp ./keys/web/tsa_host_key.pub ./keys/worker
- Create start-up script for Concourse. Save this as /concourse/start.sh:
/concourse/concourse web \
–basic-auth-username myuser \
–basic-auth-password mypass \
–session-signing-key /concourse/keys/web/session_signing_key \
–tsa-host-key /concourse/keys/web/tsa_host_key \
–tsa-authorized-keys /concourse/keys/web/authorized_worker_keys \
–external-url http://192.168.103.81:8080 \
/concourse/concourse worker \
–work-dir /opt/concourse/worker \
–tsa-host 127.0.0.1 \
–tsa-public-key /concourse/keys/worker/tsa_host_key.pub \
The items in red should definitely be changed for your environment. “external_url” uses the IP address of the VM its running on. and the username and password values in the postgres-data-source should reflect what you set up earlier. Save the file and be sure to set it as executable (
chmod +x ./start.sh)
- Run the script “./start.sh”. You should see several lines go by concerning worker-collectors and builder-reapers.
- If you instead see a message about authentication, you’ll want to make sure that 1) the credentials in the script are correct, 2) the account has not had it’s password set in linux or in postgres
- If you instead see a message about the connection not accepting SSL, be sure that the connection string in the script includes “?sslmode=disable” after the database name
- Test by pointing a browser at the value you assigned to the external_url. You should see “no pipelines configured”. You can login using the basic-auth username and password you specified in the startup script.
- Back in your SSH session, you can kill it with <CRTL>+C
Now we just have to make sure that concourse starts when the system reboots. I am certain that there are better/safer/more reliable ways to do this, but here’s what I did:
Use nano or your favorite text editor to add “/concourse/start.sh” to /etc/rc.local ABOVE the line that reads “exit 0”
Now, reboot your VM and retest the connectivity to the concourse page.
EMC ECS Community Edition project for how to start the script on boot.
Mitchell Anicas’ very helpful post on setting up postgres on Ubuntu.
Concourse.ci for some wholly inadequate documentation
Alfredo Sánchez for bringing the issue with Concourse and CentOS to my attention