Using CheatSheets To Apply Best Practices

CheatSheet: Concourse

CheatSheet: Concourse


linkedin
github
slack

PRs Welcome

File me Issues or star this repo.

1.1 Concourse Concepts

Name Summary
fly Pipelines are configured entirely via fly CLI
Teams Every pipeline and one-off build belongs to a team.
Targets Alias of Concourse endpoints, which composes of server url, credentials, etc
Pipelines A pipeline is the result of configuring Jobs and Resources together.
Jobs Jobs determine the actions of your pipeline,
Resource Allow you to define inputs and outputs to jobs in a pipeline
Tasks The smallest configurable unit in a Concourse pipeline is a single task.
Reference Link: Concourse Concepts, Link: concoursetutorial.com

1.2 Concourse Resources

Name Summary
file-downloader-resource enable downloading pivotal product files and stemcells
gcs-resource: google bucket  
pool-resource Atomically manages external environments
github-release  
git-release tracks commits in a branch of a Git repository
docker-image-resource: docker image  
Step: tags Use tag to match workers
bbl (bosh-bootloader) Command line utility for standing up a BOSH director on any IAAS
Reference GitHub: pcf-pipelines

1.3 Concourse Examples

Name Summary
Hello world pipeline example: helloworld.yml
Pass output to input across different tasks example: tasks-output-input.yml
Trigger pipeline test every 5 minutes example: run-test-regularly.yml
git-resource example: example-git.yml
concourse-curl-resource: http download link example: url-download.yml

1.4 Concourse Containers

Name Summary
Login to container fly -t $target hijack -u $job_url
Run command without sudden exit nohup SOMECOMMAND &; tail -f nohup.out
Hijack to containers with a successful run By design when task has passed, the container will be deleted

1.5 Concourse Pipeline

Name Summary
List all authenticated end points fly targets
Get fly version fly --version
fly login fly -t $target login -c $concourse_url -n $team_name
Get pipeline definition fly -t $target get-pipeline -p $job_name > job.yaml
Update pipeline fly -t $target set-pipeline -p $pipeline_name -c pipeline.yml -l credentials.yml
Update pipeline fly -t $target set-pipeline -p $pipeline_name -c pipeline.yml -v env1=var1
Destroy pipeline fly -t $target destroy-pipeline -p $pipeline_name
Destroy pipeline without confirmation fly -t $target destroy-pipeline -p $pipeline_name -n
Format pipeline fly format-pipeline -c pipeline.yml
Validate pipeline fly validate-pipeline -c pipeline.yml

concourse-sample.png

1.6 Concourse Resource

Name Summary
github-release resource get the old tag fly -t $target check-resource -r $pipeline-name/$resource-name -f tag:$tag
bosh-io-stemcell resource get the old version fly -t $target check-resource -r $pipeline-name/ubuntu-xenial-stemcell -f version:170.25
github release use a specific tag github-release-specific-tag.md

1.7 Concourse Security

Name Summary
Fly login to the default main team fly -t $target login -n main -c $concourse_url
Fly login to a different team fly -t $target login -n $team_name -c $concourse_url
List all teams fly -t $target teams, fly -t $target teams -d
Fly logout all targets fly logout --all
Install ansible vault in mac brew install ansible
Create a team fly -t $target set-team -n $team_name --github-org $my_org
Create a team fly -t $target set-team -n $team_name --allow-all-users
Destroy a team fly -t $target destroy-team --team-name $team_name
Login to a team fly -t $target set-team -n $team_name
Fly conf file ~/.flyrc
Reference After login to the main team, you can update other teams. Link: Concourse Team

1.8 Concourse Workers

Name Summary
List all concourse workers fly -t $target workers
List all concourse containers fly -t $target containers

concourse-design.png

1.9 Concourse Opportunities

Name Summary
fly set-pipeline run into “error: forbidden” Need to logout from both fly client (fly logout --all) and Concourse GUI
List team details for all targets fly -t $target teams -d, ~/.flyrc
Concourse UI runs into a blank page If pipeline has recursive dependencies, Concourse UI may have issue to run its javascript code
Split one pipeline yaml into small reusable pieces Not supported. An uncomfortable workaround: generating yaml with other tools. Link: GitHub
Add a readonly member to one team Not supported. Concourse only support authentication control, but not authorization control
Does Concourse have REST API? Not supported so far. Link: GitHub, Link: Concourse REST endpoints
Reference concourse-tutorial, concourse-pipeline-samples

1.10 fly Online Usage

bash-3.2$ fly --help
error: Usage:
  fly [OPTIONS] <command>

Application Options:
  -t, --target=              Concourse target name
  -v, --version              Print the version of Fly and exit
      --verbose              Print API requests and responses
      --print-table-headers  Print table headers even for redirected output

Help Options:
  -h, --help                 Show this help message

Available commands:
  abort-build        Abort a build (aliases: ab)
  builds             List builds data (aliases: bs)
  check-resource     Check a resource (aliases: cr)
  checklist          Print a Checkfile of the given pipeline (aliases: cl)
  containers         Print the active containers (aliases: cs)
  destroy-pipeline   Destroy a pipeline (aliases: dp)
  destroy-team       Destroy a team and delete all of its data (aliases: dt)
  execute            Execute a one-off build using local bits (aliases: e)
  expose-pipeline    Make a pipeline publicly viewable (aliases: ep)
  format-pipeline    Format a pipeline config (aliases: fp)
  get-pipeline       Get a pipeline's current configuration (aliases: gp)
  help               Print this help message
  hide-pipeline      Hide a pipeline from the public (aliases: hp)
  hijack             Execute a command in a container (aliases: intercept, i)
  jobs               List the jobs in the pipelines (aliases: js)
  login              Authenticate with the target (aliases: l)
  logout             Release authentication with the target (aliases: o)
  order-pipelines    Orders pipelines (aliases: op)
  pause-job          Pause a job (aliases: pj)
  pause-pipeline     Pause a pipeline (aliases: pp)
  pause-resource     Pause a resource (aliases: pr)
  pipelines          List the configured pipelines (aliases: ps)
  prune-worker       Prune a stalled, landing, landed, or retiring worker (aliases: pw)
  rename-pipeline    Rename a pipeline (aliases: rp)
  rename-team        Rename a team (aliases: rt)
  set-pipeline       Create or update a pipeline's configuration (aliases: sp)
  set-team           Create or modify a team to have the given credentials (aliases: st)
  status             Login status
  sync               Download and replace the current fly from the target (aliases: s)
  targets            List saved targets (aliases: ts)
  teams              List the configured teams (aliases: t)
  trigger-job        Start a job in a pipeline (aliases: tj)
  unpause-job        Unpause a job (aliases: uj)
  unpause-pipeline   Un-pause a pipeline (aliases: up)
  unpause-resource   Unpause a resource (aliases: ur)
  validate-pipeline  Validate a pipeline config (aliases: vp)
  volumes            List the active volumes (aliases: vs)
  watch              Stream a build's output (aliases: w)
  workers            List the registered workers (aliases: ws)

1.11 fly set-pipeline Online Usage

> fly  set-pipeline --help
error: Usage:
  fly [OPTIONS] set-pipeline [set-pipeline-OPTIONS]

Application Options:
  -t, --target=                     Concourse target name
  -v, --version                     Print the version of Fly and exit
      --verbose                     Print API requests and responses
      --print-table-headers         Print table headers even for redirected output

Help Options:
  -h, --help                        Show this help message

[set-pipeline command options]
      -n, --non-interactive         Skips interactions, uses default values
          --no-color                Disable color output
          --check-creds             Validate credential variables against credential manager
      -p, --pipeline=               Pipeline to configure
      -c, --config=                 Pipeline configuration file
      -v, --var=[NAME=STRING]       Specify a string value to set for a variable in the pipeline
      -y, --yaml-var=[NAME=YAML]    Specify a YAML value to set for a variable in the pipeline
      -l, --load-vars-from=         Variable flag that can be used for filling in template values in configuration from a YAML file


Leave a Reply

Your email address will not be published. Required fields are marked *