Tony Kay
2020-03-10 70782bba2ff827debcd486c1b1e2f3e44d4611ff
New Ansible Workshop Config (ansible-workshops) with new networking options (#1285)

* cloned linklight config as part of refactor and cleanup

* Removed old var files from old cisco workshop

* Refactored all ansible workshop var files

* Renamed common var file ansible-workshops

* Add workshop_type to sample_vars ansible-workshops

* removed references to env_secret_vars.yml from ansible_workshops config

* Updated env_type in new ansible-workshops config

* Upated workshop clone repo path to reflect new env_type

* Refactoring ansible-workshop config

* More refactoring and cleanup of ansible-workshops config

* Updates to README ansible-workshops

* Updated docs for ansible-workshops config

* More cleanup to ansible-workshops config

* Travis syntax check changes to accomodate move from linklight towards ansible-workshops
20 files added
1 files modified
482 ■■■■■ changed files
ansible/configs/ansible-workshops/README.adoc 128 ●●●●● patch | view | raw | blame | history
ansible/configs/ansible-workshops/ansible.cfg 21 ●●●●● patch | view | raw | blame | history
ansible/configs/ansible-workshops/destroy_env.yml 9 ●●●●● patch | view | raw | blame | history
ansible/configs/ansible-workshops/env_vars.yml 39 ●●●●● patch | view | raw | blame | history
ansible/configs/ansible-workshops/post_infra.yml 36 ●●●●● patch | view | raw | blame | history
ansible/configs/ansible-workshops/post_software.yml 24 ●●●●● patch | view | raw | blame | history
ansible/configs/ansible-workshops/pre_infra.yml 70 ●●●●● patch | view | raw | blame | history
ansible/configs/ansible-workshops/pre_software.yml 14 ●●●●● patch | view | raw | blame | history
ansible/configs/ansible-workshops/sample_vars.yml 25 ●●●●● patch | view | raw | blame | history
ansible/configs/ansible-workshops/software.yml 19 ●●●●● patch | view | raw | blame | history
ansible/configs/ansible-workshops/vars/README.adoc 4 ●●●● patch | view | raw | blame | history
ansible/configs/ansible-workshops/vars/common_all_ansible_workshop_vars.yml 9 ●●●●● patch | view | raw | blame | history
ansible/configs/ansible-workshops/vars/f5_tower_workshop_vars.yml 9 ●●●●● patch | view | raw | blame | history
ansible/configs/ansible-workshops/vars/networking_arista_tower_workshop_vars.yml 8 ●●●●● patch | view | raw | blame | history
ansible/configs/ansible-workshops/vars/networking_cisco_tower_workshop_vars.yml 8 ●●●●● patch | view | raw | blame | history
ansible/configs/ansible-workshops/vars/networking_juniper_tower_workshop_vars.yml 8 ●●●●● patch | view | raw | blame | history
ansible/configs/ansible-workshops/vars/networking_multivendor_tower_workshop_vars.yml 8 ●●●●● patch | view | raw | blame | history
ansible/configs/ansible-workshops/vars/rhel_tower_workshop_vars.yml 7 ●●●●● patch | view | raw | blame | history
ansible/configs/ansible-workshops/vars/security_tower_workshop_vars.yml 15 ●●●●● patch | view | raw | blame | history
ansible/configs/ansible-workshops/vars/windows_tower_workshop_vars.yml 11 ●●●●● patch | view | raw | blame | history
tests/static/syntax-check.sh 10 ●●●● patch | view | raw | blame | history
ansible/configs/ansible-workshops/README.adoc
New file
@@ -0,0 +1,128 @@
= Ansible Workshops Config
Ansible Workshops is an example of an existing deployer being wrapped by Ansible.
link:https://github.com/ansible/workshops.git[Ansible Workshops] is capable of deploying multiple different types of Ansible based workshops:
* RHEL - 4 Linux nodes
* F5
* Networking - a 4 switch/router lab available in 4 different configurations
** multivendor (2 x Arista, Cisco, Juniper)
** 4 x Arista
** 4 x Cisco
** 4 x Juniper
* Windows
* Security
The `ansible-workshops` deployer is called by first running a fairly typical `Ansible Agnostic Deployer`
`pre_infra.yml` but at the end of that playbook:
Cloning the `ansible-workshops` repo to a specfic commit, this is currently hard coded into the deployer script
and executing the cloned provisioioning playbook. `ansible-workshops` specfic vars are made availble via:
* common_all_ansible_workshop_vars.yml
And one of:
* f5_tower_workshop_vars.yml
* networking_arista_tower_workshop_vars.yml
* networking_cisco_tower_workshop_vars.yml
* networking_juniper_tower_workshop_vars.yml
* networking_multivendor_tower_workshop_vars.yml
* rhel_tower_workshop_vars.yml
* security_tower_workshop_vars.yml
* windows_tower_workshop_vars.yml
== How the Different Labs get Configured
For deployment via RHPDS or similar deployer script will substitute the value of
`student_workloads` into a call to the relevant lab var file e.g.
Deployer Usage:
[source,bash]
----
$ mydeploy-script.sh --student_workloads=f5_tower_workshop --more_args ...
----
[source,bash]
----
ansible-playbook main.yml \
  -e ANSIBLE_REPO_PATH=${ANSIBLE_REPO_PATH} \
  -e guid=${GUID} \
  ..... args omitted for brevity
  -e @configs/ansible-workshops/common_all_ansible_workshop_vars.yml
  -e @configs/ansible-workshops/${STUDENT_WORKLOAD}_vars.yml
----
== Set up your "Secret" variables
* You may need to provide some credentials for deployments to work
* Create a file called "env_secret_vars.yml" and put it in the
 ./ansible/configs/CONFIGNAME/ directory.
** At this point this file *has to be created* even if no vars from it are used.
* You can choose to provide these values as extra vars (-e "var=value") in the
 command line if you prefer not to keep sensitive information in a file.
.Example contents of "Secret" Vars file
----
foo: bar
----
== Running Ansible Playbook
You can run the playbook with the following arguments
[NOTE]
====
Also it sets an ANSIBLE_CONFIG var to load a specfic set of vars for these labs.
This is the HIGHEST precedence for `ansible.cfg` so will mask any other
`ansible.cfg`
====
=== New sample vars format
[source,bash]
----
AWS_PROFILE=ansiblebu
ANSIBLE_CONFIG=configs/ansible-workshops/ansible.cfg
ANSIBLE_REPO_PATH=$(pwd)
ansible-playbook main.yml                                                   \
  -e ANSIBLE_REPO_PATH=${ANSIBLE_REPO_PATH}                                 \
  -e @configs/ansible-workshops/sample_vars.yml                             \
  -e @configs/ansible-workshops/vars/common_all_ansible_workshop_vars.yml   \
  -e @configs/ansible-workshops/vars/f5_tower_workshop_vars.yml             \
  -e @~/secret.yml                                                          \
  --skip-tags=deploy_infrastructure,pre_software_tasks,deploy_software,post_software,email \
  -vv
----
== Deploying other Ansible Workshops
=== To Delete an environment
This deletes the Ansible Engine Lab, notes below for the Networking labs
[source,bash]
----
GUID=td08
ENV_TYPE=ansible-workshops
EC2_NAME_PREFIX=${GUID}
EC2_REGION=us-east-1
ANSIBLE_REPO_PATH=$(pwd)
ansible-playbook ${ANSIBLE_REPO_PATH}/configs/${ENV_TYPE}/destroy_env.yml \
  -e ANSIBLE_REPO_PATH=${ANSIBLE_REPO_PATH} \
  -e ec2_name_prefix=${GUID} \
  -e ec2_region=${EC2_REGION} \
  -e @configs/ansible-workshops/vars/common_all_ansible_workshop_vars.yml   \
  -e @configs/ansible-workshops/vars/f5_tower_workshop_vars.yml             \
----
ansible/configs/ansible-workshops/ansible.cfg
New file
@@ -0,0 +1,21 @@
[defaults]
inventory               = hosts
forks                   = 50
host_key_checking       = False
retry_files_enabled     = False
no_target_syslog        = False
callback_whitelist      = time
[ssh_connection]
scp_if_ssh              = True
# persistent_connection vars are for Cisco Routers etc
# issues seen with timeouts when deploying regionally
# e.g. ap-southeast-1 etc. Also used by original
# Ansible Workshops
[persistent_connection]
command_timeout         = 1000
connect_timeout         = 1000
connect_retry_timeout   = 1000
ansible/configs/ansible-workshops/destroy_env.yml
New file
@@ -0,0 +1,9 @@
- name: Delete Infrastructure
  hosts: localhost
  connection: local
  gather_facts: False
  become: no
  vars_files:
    - "./env_vars.yml"
- import_playbook: "{{ ANSIBLE_REPO_PATH }}/workdir/{{ env_type }}/provisioner/teardown_lab.yml"
ansible/configs/ansible-workshops/env_vars.yml
New file
@@ -0,0 +1,39 @@
####### VARIABLES YOU SHOULD CONFIGURE FOR YOUR DEPLOYEMNT
####### OR PASS as "-e" args to ansible-playbook command
project_tag: "{{ env_type }}-{{ guid }}"
#
#      Ansible Workshops comes with its OWN deployer which AAD simply wraps
#      Hence many of the variables typically used are redundant
###### Variables from the Ansible Workshops AWS Provisioner:
ec2_key_name: toktest           # SSH key in AWS to put in all the instances
ec2_region: us-east-1           # region where the nodes will live
ec2_az: us-east-1a              # the availability zone
ec2_name_prefix: "{{ guid }}"   # name prefix for all the VMs
student_total: 1                # amount of work benches to provision
## Optional Variables
#networking: true               # workshop is put into networking mode, uses two Cisco IOS-XE devices
# Obsolete - password set externally to agnosticd
# admin_password: ansible         # password used for student account on control node
localsecurity: false            # skips firewalld installation and SE Linux when false
create_login_page: false
email: no                       # <OPTIONAL> Set this if you wish to disable email
users:
  - name: tony kay
    username: tok
    email: tok@redhat.com
### END OF Ansible Workshops AWS Provisioner Variables
#
use_own_key: true
env_authorized_key: "{{guid}}key"
set_env_authorized_key: true
ansible/configs/ansible-workshops/post_infra.yml
New file
@@ -0,0 +1,36 @@
---
- name: Step 002 Post Infrastructure
  hosts: localhost
  connection: local
  become: false
  vars_files:
    - "./env_vars.yml"
  tags:
    - step002
    - post_infrastructure
  tasks:
  - name: Gather EC2 facts
    ec2_instance_facts:
      region: "{{ aws_region_final|d(aws_region) | default(region) | default('us-east-1')}}"
      filters:
        instance-state-name: running
        "tag:Workshop": "{{ guid }}"
    register: r_ec2_facts
  - name: Print our new instances
    debug:
      var: item.instance_id
      verbosity: 2
    loop: "{{ r_ec2_facts['instances'] }}"
  - name: Tag all machines with owner
    ec2_tag:
      region: "{{ aws_region_final | d(aws_region) | d(region) | d('us-east-1') }}"
      resource: "{{ item.instance_id }}"
      state: present
      tags:
        Owner: "{{ user_owner }}"
        Email: "{{ user_email }}"
        Workshop_type: "{{ workshop_type }}"
    loop: "{{ r_ec2_facts['instances'] }}"
ansible/configs/ansible-workshops/post_software.yml
New file
@@ -0,0 +1,24 @@
---
- name: Step 00xxxxx post software
  hosts: support
  gather_facts: False
  become: yes
  vars_files:
    - "{{ ANSIBLE_REPO_PATH }}/configs/{{ env_type }}/env_vars.yml"
  tasks:
    - debug:
        msg: "Post-Software tasks Started"
- name: PostSoftware flight-check
  hosts: localhost
  connection: local
  gather_facts: false
  become: false
  vars_files:
    - "{{ ANSIBLE_REPO_PATH }}/configs/{{ env_type }}/env_vars.yml"
  tags:
    - post_flight_check
  tasks:
    - debug:
        msg: "Post-Software checks completed successfully"
...
ansible/configs/ansible-workshops/pre_infra.yml
New file
@@ -0,0 +1,70 @@
- name: Step 000 Pre Infrastructure
  hosts: localhost
  connection: local
  gather_facts: false
  become: false
  vars_files:
    - "{{ ANSIBLE_REPO_PATH }}/configs/{{ env_type }}/env_vars.yml"
  tags:
    - step001
    - pre_infrastructure
    - generate_env_keys
  tasks:
  - debug:
      msg: "Step 000 Pre Infrastructure"
  - name: Generate SSH keys when set_env_authorized_key
    block:
    - name: Generate SSH keys
      shell: ssh-keygen -b 2048 -t rsa -f "{{ ANSIBLE_REPO_PATH }}/workdir/{{ env_authorized_key }}" -q -N ""
      args:
        creates: "{{ ANSIBLE_REPO_PATH }}/workdir/{{ env_authorized_key }}"
    - name: fix permission
      file:
        path: "{{ ANSIBLE_REPO_PATH }}/workdir/{{ env_authorized_key }}"
        mode: 0400
    - name: Generate SSH pub key
      shell: ssh-keygen -y -f "{{ ANSIBLE_REPO_PATH }}/workdir/{{ env_authorized_key }}" > "{{ ANSIBLE_REPO_PATH }}/workdir/{{ env_authorized_key }}.pub"
      args:
        creates: "{{ ANSIBLE_REPO_PATH }}/workdir/{{ env_authorized_key }}.pub"
    when: set_env_authorized_key | bool
- name: PreSoftware flight-check
  hosts: localhost
  connection: local
  gather_facts: false
  become: false
  vars_files:
    - "{{ ANSIBLE_REPO_PATH }}/configs/{{ env_type }}/env_vars.yml"
  tags:
    - flight_check
  tasks:
    - debug:
        msg: "Pre-Software checks completed successfully"
- import_playbook: "{{ ANSIBLE_REPO_PATH }}/workdir/{{ env_type }}/provisioner/provision_lab.yml"
  tags:
   - ansible_workshops_provision
- name: PostSoftware flight-check
  hosts: localhost
  connection: local
  gather_facts: false
  become: false
  vars_files:
    - "{{ ANSIBLE_REPO_PATH }}/configs/{{ env_type }}/env_vars.yml"
  tags:
    - post_flight_check
  tasks:
    - when: create_login_page
      name: Output user.info string for email when a workshop, create_login_page set to true, S3 bucket
      debug:
        msg: "user.info: The list of VMs for this workshop is available at: http://{{ guid }}.{{ workshop_dns_zone }}"
    - debug:
        msg: "Post-Software checks completed successfully"
ansible/configs/ansible-workshops/pre_software.yml
New file
@@ -0,0 +1,14 @@
---
- name: PreSoftware flight-check
  hosts: localhost
  connection: local
  gather_facts: false
  become: false
  vars_files:
    - "{{ ANSIBLE_REPO_PATH }}/configs/{{ env_type }}/env_vars.yml"
  tags:
    - flight_check
  tasks:
    - debug:
        msg: "Pre-Software checks completed successfully"
...
ansible/configs/ansible-workshops/sample_vars.yml
New file
@@ -0,0 +1,25 @@
# sample vars configuration file
#
# This file is passed to ansible-playbook to set key vars which need to be set
# and typically customized for a sucessful deployment.
#
# Usage: ansible-playbook main.yml -e @configs/ansible-workshops/sample_vars.yml
#
# Ideally make and keep a copy OUTSIDE your repo, especially if using Cloud Credentials
# Credentials can also be set seperately i.e. ~/secrets.yml and passed in with
guid: ans-workshop-00
GUID: "{{ guid }}"
env_type: ansible-workshops
project_tag: "{{ env_type }}-{{ guid }}"
cloud_provider: ec2
ec2_key_name: "{{ guid }}-link"
ec2_region: us-east-1
aws_region: "{{ ec2_region }}"
ec2_name_prefix: "{{ GUID }}"
student_total: 1
software_to_deploy: none
workshop_type: rhel
ansible/configs/ansible-workshops/software.yml
New file
@@ -0,0 +1,19 @@
---
- name: Step 00xxxxx software
  hosts: localhost
  gather_facts: False
  become: false
  tasks:
    - debug:
        msg: "Software tasks Started"
- name: Software flight-check
  hosts: localhost
  connection: local
  gather_facts: false
  become: false
  tags:
    - post_flight_check
  tasks:
    - debug:
        msg: "Software checks completed successfully"
ansible/configs/ansible-workshops/vars/README.adoc
New file
@@ -0,0 +1,4 @@
== Overview
vars files for the Ansible Workshops drawing heavily on the examples given in the link:https://github.com/ansible/workshops/tree/master/provisioner/sample_workshops[Ansible Workshop Sample Vars directory]
ansible/configs/ansible-workshops/vars/common_all_ansible_workshop_vars.yml
New file
@@ -0,0 +1,9 @@
---
# All Ansible Workshops via RHPDS consume this file first
workshop_dns_zone:      rhdemo.io     # Domain for S3 bucket
dns_type:               aws
create_login_page:      true          # Creates S3 bucket with labs and login details
localsecurity:          false
email:                  no
...
ansible/configs/ansible-workshops/vars/f5_tower_workshop_vars.yml
New file
@@ -0,0 +1,9 @@
---
# 2020-03-10 Ansible Workshop F5 vars
xrdp:             false            # install xrdp with xfce for graphical interface
towerinstall:     true            # Installs latest Tower
autolicense:      true
workshop_type:    f5
...
ansible/configs/ansible-workshops/vars/networking_arista_tower_workshop_vars.yml
New file
@@ -0,0 +1,8 @@
---
# 2020-03-10 Arista Tower workshop vars
autolicense:          true
towerinstall:         true
workshop_type:        networking
network_type:         arista
...
ansible/configs/ansible-workshops/vars/networking_cisco_tower_workshop_vars.yml
New file
@@ -0,0 +1,8 @@
---
# 2020-03-10 Cisco Tower workshop vars
autolicense:          true
towerinstall:         true
workshop_type:        networking
network_type:         cisco
...
ansible/configs/ansible-workshops/vars/networking_juniper_tower_workshop_vars.yml
New file
@@ -0,0 +1,8 @@
---
# 2020-03-10 Juniper Tower workshop vars
autolicense:          true
towerinstall:         true
workshop_type:        networking
network_type:         juniper
...
ansible/configs/ansible-workshops/vars/networking_multivendor_tower_workshop_vars.yml
New file
@@ -0,0 +1,8 @@
---
# 2020-03-10 Multivendor Tower workshop vars
autolicense:          true
towerinstall:         true
workshop_type:        networking
#network_type:         multivendor
...
ansible/configs/ansible-workshops/vars/rhel_tower_workshop_vars.yml
New file
@@ -0,0 +1,7 @@
---
# 2020-03-10 Ansible Workshop RHEL with Tower
autolicense:          true
towerinstall:         true                  # Installs latest Tower
workshop_type:        rhel
...
ansible/configs/ansible-workshops/vars/security_tower_workshop_vars.yml
New file
@@ -0,0 +1,15 @@
---
workshop_type: security
security_console: 'qradar'
## Optional Variables
create_login_page: true
autolicense: false
towerinstall: true
dns_type: aws
## Should be sourced from BASH script
#admin_password: 'dynamic from BASH'                 # password used for student account on control node
#windows_password: 'dynamic from BASH'
ansible/configs/ansible-workshops/vars/windows_tower_workshop_vars.yml
New file
@@ -0,0 +1,11 @@
---
workshop_type:    windows
instance_loc:     ec2
doubleup:         false     # creates 2 windows instances per student instead of default 1
# Tower vars
towerinstall:     true     # Installs latest Tower
autolicense:      true
...
tests/static/syntax-check.sh
@@ -26,11 +26,11 @@
    fi
    env_type=$(egrep ^env_type: ${i}|cut -d' ' -f 2)
    # Linklight needs to be downloaded
    if [ "${env_type}" = linklight ]; then
        if [ ! -d ${ansible_path}/workdir/linklight ]; then
            echo "Download linklight"
            git clone https://github.com/ansible/workshops.git ${ansible_path}/workdir/linklight
    # Ansible Workshops AKA as Linklight needs to be downloaded
    if [ "${env_type}" = linklight ] || [ "${env_type}" = ansible-workshops ]; then
        if [ ! -d ${ansible_path}/workdir/${env_type} ]; then
            echo "Download ${env_type}"
            git clone https://github.com/ansible/workshops.git ${ansible_path}/workdir/${env_type}
        fi
        touch $(dirname "${i}")/env_secret_vars.yml
        extra_args=(