First, some background – I promise to keep it short. You should never have credentials in a public github repo. Probably not good to have them in a private repo either. At Pivotal, the github client is configured with credalert which complains when I try to push credentials to github. I maintain compliance, I needed a way to update the stuff in the repo and have my credentials too. Concourse supports a couple of AWS credential managers, Vault and Credhub. Since CredHub is built-in to ops-manager-deployed BOSH director, we don’t have to spin anything else up.
The simplified diagram shows how this will work. CredHub is on the BOSH director, so it’ll need to be reachable from the Concourse Web ATS service and anywhere the credhub cli will be used. If your BOSH director is behind a NAT, you may want to configure a DNAT, so it can be reached.
In this case, we’re using a “management/infrastructure” Operations Manager and BOSH director to deploy and manage concourse and minio. The pipelines on concourse will be used to deploy and maintain other foundations in the environment.
Configure UAA
- Logon to the ops manager and navigate to status to record the IP address of the BOSH director. If your BOSH director is behind a NAT, locate it’s DNAT instead.
- Navigate to the credentials tab. We’re going to need the uaa_login_client_credentials password and the uaa_admin_client_credentials password.
- While here, save the ops manager root ca to your computer. From the installation dashboard, click on your name in the upper right, select Settings. Then click Advanced and Download Root CA.
- SSH into your ops manager: ubuntu@<ops manager name or IP>
- Set uaac target
uaac target https://<IP of BOSH director>:8443 –ca-cert /var/tempest/workspaces/default/root_ca_certificate
- Login to uaac – ok, this gets awkward
uaac token owner get login -s <uaa_login_client_credentials>
- Replace <uaa_login_client_credentials> with the value you saved
- When prompted for a username enter admin
- For password enter the uaa_admin_client_password value you saved
- You should see “Successfully fetched token…”
- Create a uaac client for concourse to use with credhub
uaac client add –name concourse-to-credhub –authorized-grant-types client_credentials –authorities credhub.read,credhub.write –access-token-validity 30 –secret MySecretPassword
Please replace MySecretPassword with something else
- Create a uaac user for use with the CredHub cli
uaac user add credhub –email credhub@whatever.com -p MySecretPassword
Try out Credhub cli
-
- Download and install the credhub cli. On mac, you can use brew install credhub
- From a terminal/command line run this to point the cli to the credhub instance on the BOSH director:
credhub api -s <IP of BOSH director>:8844 –ca-cert ./root_ca_certificate
- Replace <IP of BOSH director> with the name or reachable IP of the director
- root_ca_certificate is the root CA from ops manager you downloaded earlier
- Login to credhub:
credhub login -u credhub -p MySecretPassword
User and pass are from the User we added to uaa earlier
- Set a test value:
credhub –type:value –name=/testval –value=hello
Here’s we’re setting a key (aka credential) with the name /testval to the value “hello”. Note that all the things stored in credhub start with a slash and that there are several types of credentials that can be stored, the simplest being “value”
- Get our value:
credhub –name /testval
This will return the metadata for our key/credential
Configuring Concourse to use CredHub
Concourse TSA must be configured to look to credhub as a credential manager. I’m using BOSH-deployed concourse, so I’ll simply update the deployment manifest with the new params. if you’re using concourse via docker-compose, you’ll want to update the yml with the additional params as described here.
For concourse deployed via BOSH and using concouse-bosh-deployment, we’ll include the /operations/credhub.yml file and the additional params. For me this looks like
bosh -e core deploy -d concourse concourse.yml \
-l ../versions.yml \
–vars-store cluster-creds.yml \
-o operations/static-web.yml \
-o operations/basic-auth.yml \
-o operations/scale.yml \
-o operations/privileged-http.yml \
-o operations/credhub.yml \
–var web_ip=192.168.100.205 \
–var external_url=http://concourse.ragazzilab.com \
–var network_name=INFRA \
–var web_vm_type=small.disk \
–var db_vm_type=small.disk \
–var azs=[BOSH] \
–var db_persistent_disk_type=10240 \
–var worker_vm_type=concourse.worker \
–var deployment_name=concourse \
–var local_user.username=myuser \
–var local_user.password=mypass \
–var web_instances=1 \
–var worker_instances=1 \
–var syslog_address=syslog.ragazzilab.com \
–var syslog_port=514 \
–var syslog_permitted_peer=syslog.ragazzilab.com \
–var credhub_url=”https://192.168.100.200:8844 ” \
–var credhub_client_id=concourse-to-credhub \
–var credhub_client_secret=MySecretPassword \
–var credhub_ca_cert=”$(cat root_ca_certificate)”
Test a pipeline
-
- Use credhub cli to create a value
credhub set –name /concourse/main/hello-credhub/hello –value World
Concourse has a default pattern for looking up interpolation values. It’s /concourse/<team name>/<pipeline name>/<key>
- Get the test pipeline from here.
jobs:
– name: hello-credhub
plan:
– do:
– task: hello-credhub
config:
platform: linux
image_resource:
type: docker-image
source:
repository: ubuntu
run:
path: sh
args:
– -exc
– |
echo “Hello $WORLD_PARAM”
params:
WORLD_PARAM: ((hello)) - Use fly to set the test pipeline
fly -t concourse login -c http://concourse -u myuser -p mypass -n main
fly -t core sp -p hello-credhub -c hello-credhub.yml - Run the test pipeline in concourse. If all goes well, it should say Hello World”
- Use credhub cli to create a value
This should be the last “how to install concourse” post; With this, I think I’ve covered all the interesting ways to install it. Using BOSH is by-far my favorite approach. After this, I hope to post more related to the use of concourse and pipelines.
Let’s deploy something with BOSH!
While in psql, use this command to set the password for the account in postgress
Brian Ragazzi 