Add new groovy files for 3 cloud native CIs and Babylon empty-config CI (#1...

Dibyendu Jana

2020-02-27 33fc99134cf9441f64c8ad17eadc55b88e3fcee8

Add new groovy files for 3 cloud native CIs and Babylon empty-config CI (#1004)

* Added new updated file for Auto Environment Test for Workshop Integreatly.

* Added variable definition to get integreatly version from parameter.

* Fixing typo error for integreatly version.

* Increased timeout from 100 minutes to 120 minutes as deployment is taking time more than 1 hour 45 minutes.

* Updated after incorporating review comment from Noelle.

* AutoJenkins test commit for groovy on Ansible-Network-Automation-Workshop lab

* Making NA region as default.

* Fixing typo issue on variable student_workloads

* Updated groovy to accommodate Ansible F5 Automation Workshop

* Updated groovy to accommodate linklight_engine_tower

* Renamed groovy file as same file is used for three ansible automation workshop

* Fixing email validation issue

* Updated timeout for ansible automation linklight workshops and fix web app location for integreatly

* Created different groovy file for 3 different Ansible job.

* Updated gptezabbixalert@redhat.com on failure

* Added groovy script for OCP Workshops. Thanks Chandrabose for sharing 4.0 workshop script.

* Updated groovy script with latest CF input for OCP Workshops

* Fixing missing comma in the script for OpenShift Workshop groovy

* Increased duration on checking last mail.

* Correct email logic

* Fix the second email text checking

* Skip the updated mail checking as poll_email is not coded to exit for this case.

* Fixing array index out of range for Openshift Master Console URL.

* Fixing email issue - http not https.

* Updated OpenShift Workshop. to OpenShift Workshop

* Groovy scripts on eight ansible catalog entry.

* Updated logic last email checking

* fixing email text

* Corrected output from email.

* Updated email text fix to all ansible labs

* Fixing email text checking.

* Added stage to test SSH

* Correcting SSH test changes

* Added log for SSH command test.

* Added groovy script for OpenShift Implementation 3.11 Lab.

* Added script for OPENTLC OpenShift 3.11 Shared Access.

* Updated openshift location and remove ssh

* Fixing email text check

* Added code trace error

* Fix for email text checking logic.

* Adding Jenkins job for OpenShift-4-Shared-Access and OpenShift-4-Client-VM

* Adding groovy for four ocp related labs on OPENTLC.

* Added code to catch exception on email text checking.

* Remove ssh check as it should be failing.

* Modification aws install vm

* Updating correct text checking with regex and fix messages

* Updated runtime as per CF entry

* remove ssh on failure

* Code to test SSH with password

* Fix variable definition

* Testing error scenario

* Remove ssh password as of now

* Remove SSH testing and remove more log

* Remove SSH testing

* Passing as cfparams.

* Increase timeout to wait for last mail

* Added script for openshift-service-mesh-in-action

* Added corrected repo link and added first mail checking logic

* fFixed email check and added ssh testing

* Remove ssh oc command as it is not in ansible env

* Adding groovy for HA labs

* Adding groovy script for OCP and Container Storage for Admins

* Adding OCP operator labs groovy script

* Updated CI values

* Removed unused parameter

* Increased duration of wait for last mail

* Adding script for Fuse Foundations Part 1 - Fuse Online.

* Updated city and salesforce value.

* Updated new cloudforms-oob url

* Fixing email text checking

* Updated timeout for last email 90 minutes.

* Fixing ocp related auto test job after updated migrating to agnosticv

* Increased the last mail receiving duration limit.

* Increased 60 minutes.

* Adding groovy script for Babylon empty-config test

* Groovy files for could-native CI Tests and email changes on others

* Babylon test script moved to babylon repository

* Updating name of CI, integreatly version and waiting completion mail duration

* Adding script for OpenShift Storage Lab

* Testing script for OpenShift 3.11 Shared Environment and OpenShift 4 Shared Environment

* Updating code for recent email text checking.

* Code for many testing scriots

* Adding notes variable

* email text checking for rhpam-7-order-it-hardware-demo

* Email text checking fix for ocp gpu one

* email text checking

* Skipping first mail receiving check.

* Skipping the the first mail check

* skipping the first mail check

* Email fix for ansible tower implementation 3.3

* Fixing duration

* Increase timeout to max 150 as it does not affect to complete early if mail receive early

* Increase timeout to max 150 as it does not affect to complete job early if mail receive early

* Skip first mail logic because sometimes the completed email arrives before the 'has started' email

* Removed typo spaces

* Skipping started mail check

* Added nodes parameter

* Skip first mail checking step

* Fixing email text and adding CI entry

* Fixing email text regex

* Fixing typo error

* Updating notification email to myself so that zabbix alert should not triggger while developing the script

* Fixing single mail receiving issue for some RHEL CIs

* single mail update

* Fix step error

* Testing single mail logic.

* Remove guid from poll email as it is not present in mail subject for this CI

* Just passed one space for GUID as GUID is not present in email subject

* Added GUID as we fix email subject for all Ravello CIs

* Fixing email text

* Fixing regex for new Ravello email format.

* Fixing syntex error for escape character

* Trying to fix email checking logic

* Adding sleep for 40 minutes to complete deployment

* Removed space

* Finalised the script and updated notification mail id.

* Updating new logic for CIs

* Updating working code for CI

* Testing sleep script

* Identifying email text issue why = is add after each line

* Testing regex

* Testing regex

* Fixing regex to have more value using multiline regex

* Removed multiline regex

* Parse more regex

* combining multiline host url into one.

* Updated logic of Ravello CIs for email checking for combining two regex value into one for external host

* Displaying email to fix fully.

* Fix all for 10 ravello CIs

* Removed space at the end of the external host

* Update integreatly-workshop.groovy

* Updating config changes as per CF

* Updating email to test the changes.

* Updated email text logic.

* Updating most failing CI with new email. TASK0662875.

* OCP service mesh is decommissioned, now OCP4 service mesh is there. So updating with new details.

* Updated and defined variables names

* Fixing environment parameter

* Updated email address as per new need. TASK0662875.

* Updating this as of now to fix the CI for newly upgraded CI

* Removed commented code as of now.

* Fixed wrong choice variable name.

* Updated logic.

Co-authored-by: bosebc <42863563+bosebc@users.noreply.github.com>

 tests/jenkins/3scale-api-security-with-oidc.groovy

copy from tests/jenkins/openshift-service-mesh-lab.groovy
copy to tests/jenkins/3scale-api-security-with-oidc.groovy
File was copied from tests/jenkins/openshift-service-mesh-lab.groovy
@@ -2,13 +2,13 @@
// CloudForms
def opentlc_creds = 'b93d2da4-c2b7-45b5-bf3b-ee2c08c6368e'
def opentlc_admin_creds = '73b84287-8feb-478a-b1f2-345fd0a1af47'
def cf_uri = 'https://labs.opentlc.com'
def cf_group = 'opentlc-access-cicd'
def cf_uri = 'https://rhpds.redhat.com'
def cf_group = 'rhpds-access-cicd'
// IMAP
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -20,20 +20,25 @@
// state variables
def guid=''
def openshift_location = ''
//def ssh_location = ''
//def ssh_p = ''


// Catalog items
def choices = [
    'OPENTLC OpenShift Labs / OpenShift Service Mesh Lab',
    'DevOps Team Development / DEV - OpenShift Service Mesh Lab',
    'Multi-product Demos / 3Scale API Security with OIDC',
].join("\n")

def region_choice = [
    'na_sandboxes_gpte',
    'apac_sandboxes_gpte',
    'emea_sandboxes_gpte',
    'global_gpte',
].join("\n")

def nodes_choice = [
    '3',
    '1',
    '2',    
    '4',
    '5',
    '6',
    '7',
    '8',
].join("\n")

pipeline {
@@ -56,8 +61,13 @@
        )
        choice(
            choices: region_choice,
            description: 'Region',
            description: 'Catalog item',
            name: 'region',
        )
        choice(
            choices: nodes_choice,
            description: 'Number of Nodes',
            name: 'nodes',
        )
    }

@@ -77,6 +87,15 @@
                    def catalog = params.catalog_item.split(' / ')[0].trim()
                    def item = params.catalog_item.split(' / ')[1].trim()
                    def region = params.region.trim()
                    def nodes = params.nodes.trim()
                    def cfparams = [
                        'check=t',
                        'expiration=14',
                        'runtime=8',
                        'quotacheck=t',
                        "region=${region}",
                        "nodes=${nodes}",
                    ].join(',').trim()
                    echo "'${catalog}' '${item}'"
                    guid = sh(
                        returnStdout: true,
@@ -85,7 +104,7 @@
                          -c '${catalog}' \
                          -i '${item}' \
                          -G '${cf_group}' \
                          -d 'expiration=6,runtime=8,region=${region}'
                          -d '${cfparams}' \
                        """
                    ).trim()

@@ -93,31 +112,30 @@
                }
            }
        }
        /* Skip this step because sometimes the completed email arrives
         before the 'has started' email */
        // Skip this step because sometimes the completed email arrives
        // before the 'has started' email
        stage('Wait for first email') {
            environment {
                credentials=credentials("${imap_creds}")
            }
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'


                sh """./tests/jenkins/downstream/poll_email.py \
                    --server '${imap_server}' \
                    --guid ${guid} \
                    --timeout 20 \
                    --timeout 30 \
                    --filter 'has started'"""
            }
        }
        
        stage('Wait for last email and parse SSH location') {

        stage('Wait for last email and parse OpenShift and App location') {
            environment {
                credentials=credentials("${imap_creds}")
            }
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'

                script {
@@ -127,26 +145,15 @@
                          ./tests/jenkins/downstream/poll_email.py \
                          --server '${imap_server}' \
                          --guid ${guid} \
                          --timeout 100 \
                          --timeout 150 \
                          --filter 'has completed'
                        """
                    ).trim()

                    try {
                        def m = email =~ /Openshift Master Console: (.*)/
                        openshift_location = m[0][1]
                        echo "Openshift Master Console: ${openshift_location}"
                    
//                        m = email =~ /This cluster has authentication enabled. (.*)/
//                        echo "Cluster authentication:  ${m[0][1]}"
                      
//                        m = email =~ /SSH Access: (.*)/
//                        ssh_location = m[0][1]
//                        echo "SSH Access: ${ssh_location}"
						
//                        m = email =~ /SSH password: (.*)/
//                        ssh_p =​ m[0][1]
//                        echo "SSH password: ${ssh_p}"
                        def m = email =~ /Connect to the shared cluster by pointing your web browser to <a href="(https:\/\/master\.[^"]+)/
                        openshift_location = m[0][1]
                        echo "openshift_location = '${openshift_location}'"
                    } catch(Exception ex) {
                        echo "Could not parse email:"
                        echo email
@@ -156,20 +163,6 @@
                }
            }
        }

//        stage('SSH') {
//            steps {
//                withCredentials([
//                    sshUserPrivateKey(
//                        credentialsId: ssh_creds,
//                        keyFileVariable: 'ssh_key',
//                        usernameVariable: 'ssh_username')
//                ]) {
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} w"
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} oc version"
//                }
//            }
//        }

        stage('Confirm before retiring') {
            when {
@@ -220,7 +213,7 @@
        }
        stage('Wait for deletion email') {
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'

                withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
@@ -249,6 +242,21 @@
                export DEBUG=true
                ./opentlc/delete_svc_guid.sh '${guid}'
                """
            }

            /* Print ansible logs */
            withCredentials([
                string(credentialsId: ssh_admin_host, variable: 'ssh_admin'),
                sshUserPrivateKey(
                    credentialsId: ssh_creds,
                    keyFileVariable: 'ssh_key',
                    usernameVariable: 'ssh_username')
            ]) {
                sh("""
                    ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_admin} \
                    "bin/logs.sh ${guid}" || true
                """.trim()
                )
            }

            withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
@@ -282,4 +290,4 @@
            }
        }
    }
}
}

 tests/jenkins/ansible-advanced-homework.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gptezabbixalert@redhat.com'
def notification_email = 'gpteinfrasev2@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key

 tests/jenkins/ansible-advanced-openstack.groovy

copy from tests/jenkins/openshift-service-mesh-lab.groovy
copy to tests/jenkins/ansible-advanced-openstack.groovy
File was copied from tests/jenkins/openshift-service-mesh-lab.groovy
@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -19,21 +19,18 @@

// state variables
def guid=''
def openshift_location = ''
//def ssh_location = ''
//def ssh_p = ''
def external_host = ''


// Catalog items
def choices = [
    'OPENTLC OpenShift Labs / OpenShift Service Mesh Lab',
    'DevOps Team Development / DEV - OpenShift Service Mesh Lab',
    'OPENTLC Automation / Ansible Advanced - OpenStack',
].join("\n")

def region_choice = [
    'na_sandboxes_gpte',
    'apac_sandboxes_gpte',
    'emea_sandboxes_gpte',
    'na',
    'apac',
    'emea',
].join("\n")

pipeline {
@@ -56,7 +53,7 @@
        )
        choice(
            choices: region_choice,
            description: 'Region',
            description: 'Catalog item',
            name: 'region',
        )
    }
@@ -77,6 +74,14 @@
                    def catalog = params.catalog_item.split(' / ')[0].trim()
                    def item = params.catalog_item.split(' / ')[1].trim()
                    def region = params.region.trim()
                    def cfparams = [
                        'check=t',
                        'expiration=7',
                        'runtime=8',
                        "region=${region}",
                        'quotacheck=t'
                    ].join(',').trim()

                    echo "'${catalog}' '${item}'"
                    guid = sh(
                        returnStdout: true,
@@ -85,7 +90,7 @@
                          -c '${catalog}' \
                          -i '${item}' \
                          -G '${cf_group}' \
                          -d 'expiration=6,runtime=8,region=${region}'
                          -d '${cfparams}' \
                        """
                    ).trim()

@@ -93,26 +98,9 @@
                }
            }
        }
        /* Skip this step because sometimes the completed email arrives
         before the 'has started' email */
        stage('Wait for first email') {
            environment {
                credentials=credentials("${imap_creds}")
            }
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                    branch: 'development'


                sh """./tests/jenkins/downstream/poll_email.py \
                    --server '${imap_server}' \
                    --guid ${guid} \
                    --timeout 20 \
                    --filter 'has started'"""
            }
        }
        
        stage('Wait for last email and parse SSH location') {
        // This kind of CI send only one mail
        stage('Wait to receive and parse email') {
            environment {
                credentials=credentials("${imap_creds}")
            }
@@ -127,26 +115,16 @@
                          ./tests/jenkins/downstream/poll_email.py \
                          --server '${imap_server}' \
                          --guid ${guid} \
                          --timeout 100 \
                          --filter 'has completed'
                          --timeout 30 \
                          --filter 'is building'
                        """
                    ).trim()

                    try {
                        def m = email =~ /Openshift Master Console: (.*)/
                        openshift_location = m[0][1]
                        echo "Openshift Master Console: ${openshift_location}"
                    
//                        m = email =~ /This cluster has authentication enabled. (.*)/
//                        echo "Cluster authentication:  ${m[0][1]}"
                      
//                        m = email =~ /SSH Access: (.*)/
//                        ssh_location = m[0][1]
//                        echo "SSH Access: ${ssh_location}"
						
//                        m = email =~ /SSH password: (.*)/
//                        ssh_p =​ m[0][1]
//                        echo "SSH password: ${ssh_p}"
                        def m = email =~ /External Hostname<\/TH><TD>(.*)/
                        def mm = email =~ /(.*)<\/TD><\/TR><TR><TH>Internal Hostname/
                        external_host = m[0][1].replaceAll("=","") + mm[0][1].replaceAll(" ","")
                        echo "External-Host='${external_host}'"
                    } catch(Exception ex) {
                        echo "Could not parse email:"
                        echo email
@@ -156,20 +134,13 @@
                }
            }
        }

//        stage('SSH') {
//            steps {
//                withCredentials([
//                    sshUserPrivateKey(
//                        credentialsId: ssh_creds,
//                        keyFileVariable: 'ssh_key',
//                        usernameVariable: 'ssh_username')
//                ]) {
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} w"
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} oc version"
//                }
//            }
//        }
        
        stage ('Wait to complete provision') {
            steps {
                echo "Wait for 30 minutes for deployment to complete"
                sleep 1800 // seconds
            }
        }

        stage('Confirm before retiring') {
            when {
@@ -220,7 +191,7 @@
        }
        stage('Wait for deletion email') {
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'

                withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
@@ -251,6 +222,21 @@
                """
            }

            /* Print ansible logs */
            withCredentials([
                string(credentialsId: ssh_admin_host, variable: 'ssh_admin'),
                sshUserPrivateKey(
                    credentialsId: ssh_creds,
                    keyFileVariable: 'ssh_key',
                    usernameVariable: 'ssh_username')
            ]) {
                sh("""
                    ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_admin} \
                    "bin/logs.sh ${guid}" || true
                """.trim()
                )
            }

            withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
                mail(
                    subject: "${env.JOB_NAME} (${env.BUILD_NUMBER}) failed GUID=${guid}",

 tests/jenkins/ansible-advanced-three-tier-app.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gptezabbixalert@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key

 tests/jenkins/ansible-advanced-tower.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gptezabbixalert@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -136,9 +136,9 @@
                    ).trim()

                    try {
                        def m = email =~ /Login is ssh (.*)/
                        ssh_location = m[0]
                        echo "User instructed: '${ssh_location}'"
                        def m = email =~ /<pre>. *ssh -i [^ ]+ *([^ <]+?) *<\/pre>/
                        ssh_location = m[0][1]
                        echo "SSH command: '${ssh_location}'"
                    } catch(Exception ex) {
                        echo "Could not parse email:"
                        echo email

 tests/jenkins/ansible-f5-automation-workshop.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gptezabbixalert@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key

 tests/jenkins/ansible-for-windows.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gptezabbixalert@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key

 tests/jenkins/ansible-foundations-engine-lab.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gptezabbixalert@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key

 tests/jenkins/ansible-foundations-networking-lab.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gptezabbixalert@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key

 tests/jenkins/ansible-foundations-tower-lab.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gptezabbixalert@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key

 tests/jenkins/ansible-implementation.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gptezabbixalert@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key

 tests/jenkins/ansible-linux-automation.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gptezabbixalert@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key

 tests/jenkins/ansible-network-automation-workshop.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gptezabbixalert@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key

 tests/jenkins/ansible-tower-implementation-3.3.groovy

copy from tests/jenkins/openshift-service-mesh-lab.groovy
copy to tests/jenkins/ansible-tower-implementation-3.3.groovy
File was copied from tests/jenkins/openshift-service-mesh-lab.groovy
@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -19,21 +19,15 @@

// state variables
def guid=''
def openshift_location = ''
//def ssh_location = ''
//def ssh_p = ''

def external_host=''

// Catalog items
def choices = [
    'OPENTLC OpenShift Labs / OpenShift Service Mesh Lab',
    'DevOps Team Development / DEV - OpenShift Service Mesh Lab',
    'OPENTLC Automation / Ansible Tower Implementation 3.3',
].join("\n")

def region_choice = [
    'na_sandboxes_gpte',
    'apac_sandboxes_gpte',
    'emea_sandboxes_gpte',
    'global_gpte',
].join("\n")

pipeline {
@@ -56,7 +50,7 @@
        )
        choice(
            choices: region_choice,
            description: 'Region',
            description: 'Catalog item',
            name: 'region',
        )
    }
@@ -77,6 +71,14 @@
                    def catalog = params.catalog_item.split(' / ')[0].trim()
                    def item = params.catalog_item.split(' / ')[1].trim()
                    def region = params.region.trim()
                    def cfparams = [
                        'check=t',
                        'expiration=7',
                        'runtime=4',
                        "region=${region}",
                        'quotacheck=t'
                    ].join(',').trim()

                    echo "'${catalog}' '${item}'"
                    guid = sh(
                        returnStdout: true,
@@ -85,7 +87,7 @@
                          -c '${catalog}' \
                          -i '${item}' \
                          -G '${cf_group}' \
                          -d 'expiration=6,runtime=8,region=${region}'
                          -d '${cfparams}' \
                        """
                    ).trim()

@@ -93,26 +95,9 @@
                }
            }
        }
        /* Skip this step because sometimes the completed email arrives
         before the 'has started' email */
        stage('Wait for first email') {
            environment {
                credentials=credentials("${imap_creds}")
            }
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                    branch: 'development'


                sh """./tests/jenkins/downstream/poll_email.py \
                    --server '${imap_server}' \
                    --guid ${guid} \
                    --timeout 20 \
                    --filter 'has started'"""
            }
        }
        
        stage('Wait for last email and parse SSH location') {
		
        // This kind of CI send only one mail
        stage('Wait to receive and parse email') {
            environment {
                credentials=credentials("${imap_creds}")
            }
@@ -127,26 +112,16 @@
                          ./tests/jenkins/downstream/poll_email.py \
                          --server '${imap_server}' \
                          --guid ${guid} \
                          --timeout 100 \
                          --filter 'has completed'
                          --timeout 30 \
                          --filter 'is building'
                        """
                    ).trim()

                    try {
                        def m = email =~ /Openshift Master Console: (.*)/
                        openshift_location = m[0][1]
                        echo "Openshift Master Console: ${openshift_location}"
                    
//                        m = email =~ /This cluster has authentication enabled. (.*)/
//                        echo "Cluster authentication:  ${m[0][1]}"
                      
//                        m = email =~ /SSH Access: (.*)/
//                        ssh_location = m[0][1]
//                        echo "SSH Access: ${ssh_location}"
						
//                        m = email =~ /SSH password: (.*)/
//                        ssh_p =​ m[0][1]
//                        echo "SSH password: ${ssh_p}"
                        def m = email =~ /External Hostname<\/TH><TD>(.*)/
                        def mm = email =~ /(.*)<\/TD><\/TR><TR><TH>Internal IP Address/
                        external_host = m[0][1].replaceAll("=","") + mm[0][1].replaceAll(" ","")
                        echo "External-Host='${external_host}'"
                    } catch(Exception ex) {
                        echo "Could not parse email:"
                        echo email
@@ -156,20 +131,13 @@
                }
            }
        }

//        stage('SSH') {
//            steps {
//                withCredentials([
//                    sshUserPrivateKey(
//                        credentialsId: ssh_creds,
//                        keyFileVariable: 'ssh_key',
//                        usernameVariable: 'ssh_username')
//                ]) {
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} w"
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} oc version"
//                }
//            }
//        }
        
        stage ('Wait to complete provision') {
            steps {
                echo "Wait for 30 minutes for deployment to complete"
                sleep 1800 // seconds
            }
        }

        stage('Confirm before retiring') {
            when {
@@ -220,7 +188,7 @@
        }
        stage('Wait for deletion email') {
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'

                withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
@@ -251,6 +219,21 @@
                """
            }

            /* Print ansible logs */
            withCredentials([
                string(credentialsId: ssh_admin_host, variable: 'ssh_admin'),
                sshUserPrivateKey(
                    credentialsId: ssh_creds,
                    keyFileVariable: 'ssh_key',
                    usernameVariable: 'ssh_username')
            ]) {
                sh("""
                    ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_admin} \
                    "bin/logs.sh ${guid}" || true
                """.trim()
                )
            }

            withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
                mail(
                    subject: "${env.JOB_NAME} (${env.BUILD_NUMBER}) failed GUID=${guid}",

 tests/jenkins/azure-red-hat-openshift.groovy

copy from tests/jenkins/openshift-service-mesh-lab.groovy
copy to tests/jenkins/azure-red-hat-openshift.groovy
File was copied from tests/jenkins/openshift-service-mesh-lab.groovy
@@ -2,13 +2,13 @@
// CloudForms
def opentlc_creds = 'b93d2da4-c2b7-45b5-bf3b-ee2c08c6368e'
def opentlc_admin_creds = '73b84287-8feb-478a-b1f2-345fd0a1af47'
def cf_uri = 'https://labs.opentlc.com'
def cf_group = 'opentlc-access-cicd'
def cf_uri = 'https://rhpds.redhat.com'
def cf_group = 'rhpds-access-cicd'
// IMAP
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'djana@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -20,20 +20,17 @@
// state variables
def guid=''
def openshift_location = ''
//def ssh_location = ''
//def ssh_p = ''


// Catalog items
def choices = [
    'OPENTLC OpenShift Labs / OpenShift Service Mesh Lab',
    'DevOps Team Development / DEV - OpenShift Service Mesh Lab',
    'Workshops / Azure Red Hat OpenShift',
    'Development / DEV - Azure Red Hat OpenShift',
].join("\n")

def region_choice = [
    'na_sandboxes_gpte',
    'apac_sandboxes_gpte',
    'emea_sandboxes_gpte',
    'azure_eastus2',
    'azure_southeastasia',
    'azure_westeurope',
].join("\n")

pipeline {
@@ -56,7 +53,7 @@
        )
        choice(
            choices: region_choice,
            description: 'Region',
            description: 'Catalog item',
            name: 'region',
        )
    }
@@ -77,6 +74,16 @@
                    def catalog = params.catalog_item.split(' / ')[0].trim()
                    def item = params.catalog_item.split(' / ')[1].trim()
                    def region = params.region.trim()
                    def cfparams = [
                        'expiration=3',
                        'runtime=72',
                        "region=${region}",
                        'city=jenkins',
                        'salesforce=gptejen',
                        'check=t',
                        'quotacheck=t',
                        'notes=devops_automation_jenkins',
                    ].join(',').trim()
                    echo "'${catalog}' '${item}'"
                    guid = sh(
                        returnStdout: true,
@@ -85,7 +92,7 @@
                          -c '${catalog}' \
                          -i '${item}' \
                          -G '${cf_group}' \
                          -d 'expiration=6,runtime=8,region=${region}'
                          -d '${cfparams}' \
                        """
                    ).trim()

@@ -93,31 +100,30 @@
                }
            }
        }
        /* Skip this step because sometimes the completed email arrives
         before the 'has started' email */
        // Skip this step because sometimes the completed email arrives
        // before the 'has started' email
        stage('Wait for first email') {
            environment {
                credentials=credentials("${imap_creds}")
            }
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'


                sh """./tests/jenkins/downstream/poll_email.py \
                    --server '${imap_server}' \
                    --guid ${guid} \
                    --timeout 20 \
                    --timeout 30 \
                    --filter 'has started'"""
            }
        }
        
        stage('Wait for last email and parse SSH location') {

        stage('Wait for last email and parse OpenShift and App location') {
            environment {
                credentials=credentials("${imap_creds}")
            }
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'

                script {
@@ -127,26 +133,15 @@
                          ./tests/jenkins/downstream/poll_email.py \
                          --server '${imap_server}' \
                          --guid ${guid} \
                          --timeout 100 \
                          --timeout 150 \
                          --filter 'has completed'
                        """
                    ).trim()

                    try {
                        def m = email =~ /Openshift Master Console: (.*)/
                        openshift_location = m[0][1]
                        echo "Openshift Master Console: ${openshift_location}"
                    
//                        m = email =~ /This cluster has authentication enabled. (.*)/
//                        echo "Cluster authentication:  ${m[0][1]}"
                      
//                        m = email =~ /SSH Access: (.*)/
//                        ssh_location = m[0][1]
//                        echo "SSH Access: ${ssh_location}"
						
//                        m = email =~ /SSH password: (.*)/
//                        ssh_p =​ m[0][1]
//                        echo "SSH password: ${ssh_p}"
                        def m = email =~ /Openshift Master Console: (http:\/\/[^ \n]+)/
                        openshift_location = m[0][1]
                        echo "openshift_location = '${openshift_location}'"
                    } catch(Exception ex) {
                        echo "Could not parse email:"
                        echo email
@@ -156,20 +151,6 @@
                }
            }
        }

//        stage('SSH') {
//            steps {
//                withCredentials([
//                    sshUserPrivateKey(
//                        credentialsId: ssh_creds,
//                        keyFileVariable: 'ssh_key',
//                        usernameVariable: 'ssh_username')
//                ]) {
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} w"
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} oc version"
//                }
//            }
//        }

        stage('Confirm before retiring') {
            when {
@@ -220,7 +201,7 @@
        }
        stage('Wait for deletion email') {
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'

                withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
@@ -249,6 +230,21 @@
                export DEBUG=true
                ./opentlc/delete_svc_guid.sh '${guid}'
                """
            }

            /* Print ansible logs */
            withCredentials([
                string(credentialsId: ssh_admin_host, variable: 'ssh_admin'),
                sshUserPrivateKey(
                    credentialsId: ssh_creds,
                    keyFileVariable: 'ssh_key',
                    usernameVariable: 'ssh_username')
            ]) {
                sh("""
                    ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_admin} \
                    "bin/logs.sh ${guid}" || true
                """.trim()
                )
            }

            withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
@@ -282,4 +278,4 @@
            }
        }
    }
}
}

 tests/jenkins/ccn-roadshow-for-dev-track.groovy

copy from tests/jenkins/openshift-service-mesh-lab.groovy
copy to tests/jenkins/ccn-roadshow-for-dev-track.groovy
File was copied from tests/jenkins/openshift-service-mesh-lab.groovy
@@ -2,13 +2,13 @@
// CloudForms
def opentlc_creds = 'b93d2da4-c2b7-45b5-bf3b-ee2c08c6368e'
def opentlc_admin_creds = '73b84287-8feb-478a-b1f2-345fd0a1af47'
def cf_uri = 'https://labs.opentlc.com'
def cf_group = 'opentlc-access-cicd'
def cf_uri = 'https://rhpds.redhat.com'
def cf_group = 'rhpds-access-cicd'
// IMAP
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev2@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -20,20 +20,21 @@
// state variables
def guid=''
def openshift_location = ''
//def ssh_location = ''
//def ssh_p = ''


// Catalog items
def choices = [
    'OPENTLC OpenShift Labs / OpenShift Service Mesh Lab',
    'DevOps Team Development / DEV - OpenShift Service Mesh Lab',
    'Workshops / CCN Roadshow for Dev Track',
    'DevOps Team Development Catalog / DEV - CCN Roadshow for Dev Track',
].join("\n")

def region_choice = [
    'na_sandboxes_gpte',
    'apac_sandboxes_gpte',
    'emea_sandboxes_gpte',
    'na_gpte',
    'apac_gpte',
    'emea_gpte',
].join("\n")

def course_module_list_choice = [
    'm1,m2,m3,m4',
].join("\n")

pipeline {
@@ -56,8 +57,13 @@
        )
        choice(
            choices: region_choice,
            description: 'Region',
            description: 'Catalog item',
            name: 'region',
        )
        choice(
            choices: course_module_list_choice,
            description: 'Catalog item',
            name: 'course_module_list',
        )
    }

@@ -77,6 +83,20 @@
                    def catalog = params.catalog_item.split(' / ')[0].trim()
                    def item = params.catalog_item.split(' / ')[1].trim()
                    def region = params.region.trim()
                    def course_module_list = params.course_module_list.trim()
                    def cfparams = [
                        'check=t',
                        'salesforce=gptejen',
                        'city=jenkins',
                        'notes=devops_automation_jenkins',
                        'use_letsencrypt=f',
                        'expiration=2',
                        'runtime=24',
                        'quotacheck=t',
                        'users=2',
                        "region=${region}",
                        "course_module_list=${course_module_list}",
                    ].join(',').trim()
                    echo "'${catalog}' '${item}'"
                    guid = sh(
                        returnStdout: true,
@@ -85,7 +105,7 @@
                          -c '${catalog}' \
                          -i '${item}' \
                          -G '${cf_group}' \
                          -d 'expiration=6,runtime=8,region=${region}'
                          -d '${cfparams}' \
                        """
                    ).trim()

@@ -93,31 +113,30 @@
                }
            }
        }
        /* Skip this step because sometimes the completed email arrives
         before the 'has started' email */
        // Skip this step because sometimes the completed email arrives
        // before the 'has started' email
        stage('Wait for first email') {
            environment {
                credentials=credentials("${imap_creds}")
            }
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'


                sh """./tests/jenkins/downstream/poll_email.py \
                    --server '${imap_server}' \
                    --guid ${guid} \
                    --timeout 20 \
                    --timeout 30 \
                    --filter 'has started'"""
            }
        }
        
        stage('Wait for last email and parse SSH location') {

        stage('Wait for last email and parse OpenShift and App location') {
            environment {
                credentials=credentials("${imap_creds}")
            }
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'

                script {
@@ -127,26 +146,15 @@
                          ./tests/jenkins/downstream/poll_email.py \
                          --server '${imap_server}' \
                          --guid ${guid} \
                          --timeout 100 \
                          --timeout 150 \
                          --filter 'has completed'
                        """
                    ).trim()

                    try {
                        def m = email =~ /Openshift Master Console: (.*)/
                        openshift_location = m[0][1]
                        echo "Openshift Master Console: ${openshift_location}"
                    
//                        m = email =~ /This cluster has authentication enabled. (.*)/
//                        echo "Cluster authentication:  ${m[0][1]}"
                      
//                        m = email =~ /SSH Access: (.*)/
//                        ssh_location = m[0][1]
//                        echo "SSH Access: ${ssh_location}"
						
//                        m = email =~ /SSH password: (.*)/
//                        ssh_p =​ m[0][1]
//                        echo "SSH password: ${ssh_p}"
                        def m = email =~ /Openshift Master Console: (http:\/\/[^ \n]+)/
                        openshift_location = m[0][1]
                        echo "openshift_location = '${openshift_location}'"
                    } catch(Exception ex) {
                        echo "Could not parse email:"
                        echo email
@@ -156,20 +164,6 @@
                }
            }
        }

//        stage('SSH') {
//            steps {
//                withCredentials([
//                    sshUserPrivateKey(
//                        credentialsId: ssh_creds,
//                        keyFileVariable: 'ssh_key',
//                        usernameVariable: 'ssh_username')
//                ]) {
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} w"
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} oc version"
//                }
//            }
//        }

        stage('Confirm before retiring') {
            when {
@@ -220,7 +214,7 @@
        }
        stage('Wait for deletion email') {
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'

                withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
@@ -249,6 +243,21 @@
                export DEBUG=true
                ./opentlc/delete_svc_guid.sh '${guid}'
                """
            }

            /* Print ansible logs */
            withCredentials([
                string(credentialsId: ssh_admin_host, variable: 'ssh_admin'),
                sshUserPrivateKey(
                    credentialsId: ssh_creds,
                    keyFileVariable: 'ssh_key',
                    usernameVariable: 'ssh_username')
            ]) {
                sh("""
                    ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_admin} \
                    "bin/logs.sh ${guid}" || true
                """.trim()
                )
            }

            withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
@@ -282,4 +291,4 @@
            }
        }
    }
}
}

 tests/jenkins/cloud-native-dev-with-eclipse-vertx.groovy

copy from tests/jenkins/openshift-service-mesh-lab.groovy
copy to tests/jenkins/cloud-native-dev-with-eclipse-vertx.groovy
File was copied from tests/jenkins/openshift-service-mesh-lab.groovy
@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -20,20 +20,37 @@
// state variables
def guid=''
def openshift_location = ''
//def ssh_location = ''
//def ssh_p = ''


// Catalog items
def choices = [
    'OPENTLC OpenShift Labs / OpenShift Service Mesh Lab',
    'DevOps Team Development / DEV - OpenShift Service Mesh Lab',
    'OPENTLC Middleware Solutions Labs / Cloud-Native Dev with Eclipse Vert.x',
    'Middleware Preprod / Cloud-Native Dev with Eclipse Vert.x ',
].join("\n")

def se_version_choice = [
    '311',
].join("\n")

def region_choice = [
    'na_sandboxes_gpte',
    'apac_sandboxes_gpte',
    'emea_sandboxes_gpte',
    'global_gpte',
].join("\n")

def nodes_choice = [
    '3',
    '1',
    '2',    
    '4',
    '5',
    '6',
    '7',
    '8',
].join("\n")

def environment_choice = [
    'PROD',
    'TEST',
    'DEV',
].join("\n")

pipeline {
@@ -55,9 +72,24 @@
            name: 'catalog_item',
        )
        choice(
            choices: se_version_choice,
            description: 'Catalog item',
            name: 'se_version',
        )
        choice(
            choices: nodes_choice,
            description: 'Number of Nodes',
            name: 'nodes',
        )
        choice(
            choices: region_choice,
            description: 'Region',
            name: 'region',
        )
        choice(
            choices: environment_choice,
            description: 'Environment',
            name: 'environment',
        )
    }

@@ -76,7 +108,10 @@
                script {
                    def catalog = params.catalog_item.split(' / ')[0].trim()
                    def item = params.catalog_item.split(' / ')[1].trim()
                    def se_version = params.se_version.trim()
                    def nodes = params.nodes.trim()
                    def region = params.region.trim()
                    def environment = params.environment.trim()
                    echo "'${catalog}' '${item}'"
                    guid = sh(
                        returnStdout: true,
@@ -85,7 +120,7 @@
                          -c '${catalog}' \
                          -i '${item}' \
                          -G '${cf_group}' \
                          -d 'expiration=6,runtime=8,region=${region}'
                          -d 'check=t,expiration=7,runtime=8,quotacheck=t,region=${region},nodes=${nodes},environment=${environment},se_version=${se_version}'
                        """
                    ).trim()

@@ -94,15 +129,12 @@
            }
        }
        /* Skip this step because sometimes the completed email arrives
         before the 'has started' email */
         before the 'has started' email
        stage('Wait for first email') {
            environment {
                credentials=credentials("${imap_creds}")
            }
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                    branch: 'development'


                sh """./tests/jenkins/downstream/poll_email.py \
                    --server '${imap_server}' \
@@ -111,7 +143,7 @@
                    --filter 'has started'"""
            }
        }
        
        */
        stage('Wait for last email and parse SSH location') {
            environment {
                credentials=credentials("${imap_creds}")
@@ -127,26 +159,16 @@
                          ./tests/jenkins/downstream/poll_email.py \
                          --server '${imap_server}' \
                          --guid ${guid} \
                          --timeout 100 \
                          --timeout 40 \
                          --filter 'has completed'
                        """
                    ).trim()

                    try {
                        def m = email =~ /Openshift Master Console: (.*)/
                        openshift_location = m[0][1]
                        echo "Openshift Master Console: ${openshift_location}"
                    
//                        m = email =~ /This cluster has authentication enabled. (.*)/
//                        echo "Cluster authentication:  ${m[0][1]}"
                      
//                        m = email =~ /SSH Access: (.*)/
//                        ssh_location = m[0][1]
//                        echo "SSH Access: ${ssh_location}"
						
//                        m = email =~ /SSH password: (.*)/
//                        ssh_p =​ m[0][1]
//                        echo "SSH password: ${ssh_p}"
                        def m = email =~ /Connect to the shared cluster by pointing your web browser to <a href="(https:\/\/master\.[^"]+)/
                        openshift_location = m[0]
                        echo "openshift_location = '${openshift_location}'"

                    } catch(Exception ex) {
                        echo "Could not parse email:"
                        echo email
@@ -156,20 +178,6 @@
                }
            }
        }

//        stage('SSH') {
//            steps {
//                withCredentials([
//                    sshUserPrivateKey(
//                        credentialsId: ssh_creds,
//                        keyFileVariable: 'ssh_key',
//                        usernameVariable: 'ssh_username')
//                ]) {
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} w"
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} oc version"
//                }
//            }
//        }

        stage('Confirm before retiring') {
            when {
@@ -251,6 +259,21 @@
                """
            }

            /* Print ansible logs */
            withCredentials([
                string(credentialsId: ssh_admin_host, variable: 'ssh_admin'),
                sshUserPrivateKey(
                    credentialsId: ssh_creds,
                    keyFileVariable: 'ssh_key',
                    usernameVariable: 'ssh_username')
            ]) {
                sh("""
                    ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_admin} \
                    "find deployer_logs -name '*${guid}*log' | xargs cat"
                """.trim()
                )
            }

            withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
                mail(
                    subject: "${env.JOB_NAME} (${env.BUILD_NUMBER}) failed GUID=${guid}",

 tests/jenkins/cloud-native-dev-with-spring-boot.groovy

copy from tests/jenkins/openshift-service-mesh-lab.groovy
copy to tests/jenkins/cloud-native-dev-with-spring-boot.groovy
File was copied from tests/jenkins/openshift-service-mesh-lab.groovy
@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -20,20 +20,37 @@
// state variables
def guid=''
def openshift_location = ''
//def ssh_location = ''
//def ssh_p = ''


// Catalog items
def choices = [
    'OPENTLC OpenShift Labs / OpenShift Service Mesh Lab',
    'DevOps Team Development / DEV - OpenShift Service Mesh Lab',
    'OPENTLC Middleware Solutions Labs / Cloud-Native Dev with Spring Boot',
    'Middleware Preprod / Cloud-Native Dev with Spring Boot ',
].join("\n")

def se_version_choice = [
    '311',
].join("\n")

def region_choice = [
    'na_sandboxes_gpte',
    'apac_sandboxes_gpte',
    'emea_sandboxes_gpte',
    'global_gpte',
].join("\n")

def nodes_choice = [
    '3',
    '1',
    '2',    
    '4',
    '5',
    '6',
    '7',
    '8',
].join("\n")

def environment_choice = [
    'PROD',
    'TEST',
    'DEV',
].join("\n")

pipeline {
@@ -55,9 +72,24 @@
            name: 'catalog_item',
        )
        choice(
            choices: se_version_choice,
            description: 'Catalog item',
            name: 'se_version',
        )
        choice(
            choices: nodes_choice,
            description: 'Number of Nodes',
            name: 'nodes',
        )
        choice(
            choices: region_choice,
            description: 'Region',
            name: 'region',
        )
        choice(
            choices: environment_choice,
            description: 'Environment',
            name: 'environment',
        )
    }

@@ -76,7 +108,10 @@
                script {
                    def catalog = params.catalog_item.split(' / ')[0].trim()
                    def item = params.catalog_item.split(' / ')[1].trim()
                    def se_version = params.se_version.trim()
                    def nodes = params.nodes.trim()
                    def region = params.region.trim()
                    def environment = params.environment.trim()
                    echo "'${catalog}' '${item}'"
                    guid = sh(
                        returnStdout: true,
@@ -85,7 +120,7 @@
                          -c '${catalog}' \
                          -i '${item}' \
                          -G '${cf_group}' \
                          -d 'expiration=6,runtime=8,region=${region}'
                          -d 'check=t,expiration=7,runtime=8,quotacheck=t,region=${region},nodes=${nodes},environment=${environment},se_version=${se_version}'
                        """
                    ).trim()

@@ -94,15 +129,12 @@
            }
        }
        /* Skip this step because sometimes the completed email arrives
         before the 'has started' email */
         before the 'has started' email
        stage('Wait for first email') {
            environment {
                credentials=credentials("${imap_creds}")
            }
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                    branch: 'development'


                sh """./tests/jenkins/downstream/poll_email.py \
                    --server '${imap_server}' \
@@ -111,7 +143,7 @@
                    --filter 'has started'"""
            }
        }
        
        */
        stage('Wait for last email and parse SSH location') {
            environment {
                credentials=credentials("${imap_creds}")
@@ -127,26 +159,16 @@
                          ./tests/jenkins/downstream/poll_email.py \
                          --server '${imap_server}' \
                          --guid ${guid} \
                          --timeout 100 \
                          --timeout 40 \
                          --filter 'has completed'
                        """
                    ).trim()

                    try {
                        def m = email =~ /Openshift Master Console: (.*)/
                        openshift_location = m[0][1]
                        echo "Openshift Master Console: ${openshift_location}"
                    
//                        m = email =~ /This cluster has authentication enabled. (.*)/
//                        echo "Cluster authentication:  ${m[0][1]}"
                      
//                        m = email =~ /SSH Access: (.*)/
//                        ssh_location = m[0][1]
//                        echo "SSH Access: ${ssh_location}"
						
//                        m = email =~ /SSH password: (.*)/
//                        ssh_p =​ m[0][1]
//                        echo "SSH password: ${ssh_p}"
                        def m = email =~ /Connect to the shared cluster by pointing your web browser to <a href="(https:\/\/master\.[^"]+)/
                        openshift_location = m[0]
                        echo "openshift_location = '${openshift_location}'"

                    } catch(Exception ex) {
                        echo "Could not parse email:"
                        echo email
@@ -156,20 +178,6 @@
                }
            }
        }

//        stage('SSH') {
//            steps {
//                withCredentials([
//                    sshUserPrivateKey(
//                        credentialsId: ssh_creds,
//                        keyFileVariable: 'ssh_key',
//                        usernameVariable: 'ssh_username')
//                ]) {
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} w"
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} oc version"
//                }
//            }
//        }

        stage('Confirm before retiring') {
            when {
@@ -251,6 +259,21 @@
                """
            }

            /* Print ansible logs */
            withCredentials([
                string(credentialsId: ssh_admin_host, variable: 'ssh_admin'),
                sshUserPrivateKey(
                    credentialsId: ssh_creds,
                    keyFileVariable: 'ssh_key',
                    usernameVariable: 'ssh_username')
            ]) {
                sh("""
                    ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_admin} \
                    "find deployer_logs -name '*${guid}*log' | xargs cat"
                """.trim()
                )
            }

            withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
                mail(
                    subject: "${env.JOB_NAME} (${env.BUILD_NUMBER}) failed GUID=${guid}",

 tests/jenkins/cloud-native-development-with-thorntail.groovy

copy from tests/jenkins/openshift-service-mesh-lab.groovy
copy to tests/jenkins/cloud-native-development-with-thorntail.groovy
File was copied from tests/jenkins/openshift-service-mesh-lab.groovy
@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -20,20 +20,37 @@
// state variables
def guid=''
def openshift_location = ''
//def ssh_location = ''
//def ssh_p = ''


// Catalog items
def choices = [
    'OPENTLC OpenShift Labs / OpenShift Service Mesh Lab',
    'DevOps Team Development / DEV - OpenShift Service Mesh Lab',
    'OPENTLC Middleware Solutions Labs / Cloud-Native Development with Thorntail',
    'Middleware Preprod / Cloud-Native Development with Thorntail ',
].join("\n")

def se_version_choice = [
    '311',
].join("\n")

def region_choice = [
    'na_sandboxes_gpte',
    'apac_sandboxes_gpte',
    'emea_sandboxes_gpte',
    'global_gpte',
].join("\n")

def nodes_choice = [
    '3',
    '1',
    '2',    
    '4',
    '5',
    '6',
    '7',
    '8',
].join("\n")

def environment_choice = [
    'PROD',
    'TEST',
    'DEV',
].join("\n")

pipeline {
@@ -55,9 +72,24 @@
            name: 'catalog_item',
        )
        choice(
            choices: se_version_choice,
            description: 'Catalog item',
            name: 'se_version',
        )
        choice(
            choices: nodes_choice,
            description: 'Number of Nodes',
            name: 'nodes',
        )
        choice(
            choices: region_choice,
            description: 'Region',
            name: 'region',
        )
        choice(
            choices: environment_choice,
            description: 'Environment',
            name: 'environment',
        )
    }

@@ -76,7 +108,10 @@
                script {
                    def catalog = params.catalog_item.split(' / ')[0].trim()
                    def item = params.catalog_item.split(' / ')[1].trim()
                    def se_version = params.se_version.trim()
                    def nodes = params.nodes.trim()
                    def region = params.region.trim()
                    def environment = params.environment.trim()
                    echo "'${catalog}' '${item}'"
                    guid = sh(
                        returnStdout: true,
@@ -85,7 +120,7 @@
                          -c '${catalog}' \
                          -i '${item}' \
                          -G '${cf_group}' \
                          -d 'expiration=6,runtime=8,region=${region}'
                          -d 'check=t,expiration=7,runtime=8,quotacheck=t,region=${region},nodes=${nodes},environment=${environment},se_version=${se_version}'
                        """
                    ).trim()

@@ -94,15 +129,12 @@
            }
        }
        /* Skip this step because sometimes the completed email arrives
         before the 'has started' email */
         before the 'has started' email
        stage('Wait for first email') {
            environment {
                credentials=credentials("${imap_creds}")
            }
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                    branch: 'development'


                sh """./tests/jenkins/downstream/poll_email.py \
                    --server '${imap_server}' \
@@ -111,7 +143,7 @@
                    --filter 'has started'"""
            }
        }
        
        */
        stage('Wait for last email and parse SSH location') {
            environment {
                credentials=credentials("${imap_creds}")
@@ -127,26 +159,16 @@
                          ./tests/jenkins/downstream/poll_email.py \
                          --server '${imap_server}' \
                          --guid ${guid} \
                          --timeout 100 \
                          --timeout 40 \
                          --filter 'has completed'
                        """
                    ).trim()

                    try {
                        def m = email =~ /Openshift Master Console: (.*)/
                        openshift_location = m[0][1]
                        echo "Openshift Master Console: ${openshift_location}"
                    
//                        m = email =~ /This cluster has authentication enabled. (.*)/
//                        echo "Cluster authentication:  ${m[0][1]}"
                      
//                        m = email =~ /SSH Access: (.*)/
//                        ssh_location = m[0][1]
//                        echo "SSH Access: ${ssh_location}"
						
//                        m = email =~ /SSH password: (.*)/
//                        ssh_p =​ m[0][1]
//                        echo "SSH password: ${ssh_p}"
                        def m = email =~ /Connect to the shared cluster by pointing your web browser to <a href="(https:\/\/master\.[^"]+)/
                        openshift_location = m[0]
                        echo "openshift_location = '${openshift_location}'"

                    } catch(Exception ex) {
                        echo "Could not parse email:"
                        echo email
@@ -156,20 +178,6 @@
                }
            }
        }

//        stage('SSH') {
//            steps {
//                withCredentials([
//                    sshUserPrivateKey(
//                        credentialsId: ssh_creds,
//                        keyFileVariable: 'ssh_key',
//                        usernameVariable: 'ssh_username')
//                ]) {
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} w"
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} oc version"
//                }
//            }
//        }

        stage('Confirm before retiring') {
            when {
@@ -251,6 +259,21 @@
                """
            }

            /* Print ansible logs */
            withCredentials([
                string(credentialsId: ssh_admin_host, variable: 'ssh_admin'),
                sshUserPrivateKey(
                    credentialsId: ssh_creds,
                    keyFileVariable: 'ssh_key',
                    usernameVariable: 'ssh_username')
            ]) {
                sh("""
                    ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_admin} \
                    "find deployer_logs -name '*${guid}*log' | xargs cat"
                """.trim()
                )
            }

            withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
                mail(
                    subject: "${env.JOB_NAME} (${env.BUILD_NUMBER}) failed GUID=${guid}",

 tests/jenkins/fuse-foundations-part1-fuse-online.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'djana@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key

 tests/jenkins/fuse-on-openshift.groovy

File was renamed from tests/jenkins/openshift-service-mesh-lab.groovy
@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'djana@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -20,20 +20,31 @@
// state variables
def guid=''
def openshift_location = ''
//def ssh_location = ''
//def ssh_p = ''


// Catalog items
def choices = [
    'OPENTLC OpenShift Labs / OpenShift Service Mesh Lab',
    'DevOps Team Development / DEV - OpenShift Service Mesh Lab',
    'OPENTLC Middleware Solutions Labs / Fuse on OpenShift',
    'Middleware Preprod / Fuse on OpenShift - Preprod',
    'Shared Cluster Development / DEV Fuse on OpenShift',
].join("\n")

def region_choice = [
    'na_sandboxes_gpte',
    'apac_sandboxes_gpte',
    'emea_sandboxes_gpte',
    'na',
    'emea',
    'latam',
    'apac',
].join("\n")

def nodes_choice = [
    '3',
    '1',
    '2',    
    '4',
    '5',
    '6',
    '7',
    '8',
].join("\n")

pipeline {
@@ -59,6 +70,11 @@
            description: 'Region',
            name: 'region',
        )
        choice(
            choices: nodes_choice,
            description: 'Number of Nodes',
            name: 'nodes',
        )
    }

    stages {
@@ -77,6 +93,7 @@
                    def catalog = params.catalog_item.split(' / ')[0].trim()
                    def item = params.catalog_item.split(' / ')[1].trim()
                    def region = params.region.trim()
                    def nodes = params.nodes.trim()
                    echo "'${catalog}' '${item}'"
                    guid = sh(
                        returnStdout: true,
@@ -85,7 +102,7 @@
                          -c '${catalog}' \
                          -i '${item}' \
                          -G '${cf_group}' \
                          -d 'expiration=6,runtime=8,region=${region}'
                          -d 'check=t,expiration=7,runtime=8,quotacheck=t,region=${region},nodes=${nodes}'
                        """
                    ).trim()

@@ -93,6 +110,7 @@
                }
            }
        }
        
        /* Skip this step because sometimes the completed email arrives
         before the 'has started' email */
        stage('Wait for first email') {
@@ -127,49 +145,25 @@
                          ./tests/jenkins/downstream/poll_email.py \
                          --server '${imap_server}' \
                          --guid ${guid} \
                          --timeout 100 \
                          --timeout 150 \
                          --filter 'has completed'
                        """
                    ).trim()

                    try {
                        def m = email =~ /Openshift Master Console: (.*)/
                        openshift_location = m[0][1]
                        echo "Openshift Master Console: ${openshift_location}"
                    
//                        m = email =~ /This cluster has authentication enabled. (.*)/
//                        echo "Cluster authentication:  ${m[0][1]}"
                      
//                        m = email =~ /SSH Access: (.*)/
//                        ssh_location = m[0][1]
//                        echo "SSH Access: ${ssh_location}"
						
//                        m = email =~ /SSH password: (.*)/
//                        ssh_p =​ m[0][1]
//                        echo "SSH password: ${ssh_p}"
                        def m = email =~ /Connect to the shared cluster by pointing your web browser to <a href="(.*)"/
                        openshift_location = m[0]
                        echo "openshift_location = ${openshift_location}"
                    } catch(Exception ex) {
                        echo "Could not parse email:"
                        echo email
                        echo ex.toString()
                        throw ex
                    }

                }
            }
        }

//        stage('SSH') {
//            steps {
//                withCredentials([
//                    sshUserPrivateKey(
//                        credentialsId: ssh_creds,
//                        keyFileVariable: 'ssh_key',
//                        usernameVariable: 'ssh_username')
//                ]) {
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} w"
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} oc version"
//                }
//            }
//        }

        stage('Confirm before retiring') {
            when {
@@ -251,6 +245,21 @@
                """
            }

            /* Print ansible logs */
            withCredentials([
                string(credentialsId: ssh_admin_host, variable: 'ssh_admin'),
                sshUserPrivateKey(
                    credentialsId: ssh_creds,
                    keyFileVariable: 'ssh_key',
                    usernameVariable: 'ssh_username')
            ]) {
                sh("""
                    ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_admin} \
                    "find deployer_logs -name '*${guid}*log' | xargs cat"
                """.trim()
                )
            }

            withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
                mail(
                    subject: "${env.JOB_NAME} (${env.BUILD_NUMBER}) failed GUID=${guid}",

 tests/jenkins/implementing-proactive-security-ocp.groovy

copy from tests/jenkins/openshift-service-mesh-lab.groovy
copy to tests/jenkins/implementing-proactive-security-ocp.groovy
File was copied from tests/jenkins/openshift-service-mesh-lab.groovy
@@ -2,13 +2,13 @@
// CloudForms
def opentlc_creds = 'b93d2da4-c2b7-45b5-bf3b-ee2c08c6368e'
def opentlc_admin_creds = '73b84287-8feb-478a-b1f2-345fd0a1af47'
def cf_uri = 'https://labs.opentlc.com'
def cf_group = 'opentlc-access-cicd'
def cf_uri = 'https://rhpds.redhat.com'
def cf_group = 'rhpds-access-cicd'
// IMAP
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -20,20 +20,16 @@
// state variables
def guid=''
def openshift_location = ''
//def ssh_location = ''
//def ssh_p = ''


// Catalog items
def choices = [
    'OPENTLC OpenShift Labs / OpenShift Service Mesh Lab',
    'DevOps Team Development / DEV - OpenShift Service Mesh Lab',
    'Red Hat Summit 2019 / Implementing Proactive Security OCP',
].join("\n")

def region_choice = [
    'na_sandboxes_gpte',
    'apac_sandboxes_gpte',
    'emea_sandboxes_gpte',
    'na_gpte',
    'apac_gpte',
    'emea_gpte',
].join("\n")

pipeline {
@@ -56,7 +52,7 @@
        )
        choice(
            choices: region_choice,
            description: 'Region',
            description: 'Catalog item',
            name: 'region',
        )
    }
@@ -77,6 +73,13 @@
                    def catalog = params.catalog_item.split(' / ')[0].trim()
                    def item = params.catalog_item.split(' / ')[1].trim()
                    def region = params.region.trim()
                    def cfparams = [
                        'expiration=2',
                        'runtime=8',
                        "region=${region}",
                        'users=2',
                        'check=t',
                    ].join(',').trim()
                    echo "'${catalog}' '${item}'"
                    guid = sh(
                        returnStdout: true,
@@ -85,7 +88,7 @@
                          -c '${catalog}' \
                          -i '${item}' \
                          -G '${cf_group}' \
                          -d 'expiration=6,runtime=8,region=${region}'
                          -d '${cfparams}' \
                        """
                    ).trim()

@@ -93,31 +96,30 @@
                }
            }
        }
        /* Skip this step because sometimes the completed email arrives
         before the 'has started' email */
        // Skip this step because sometimes the completed email arrives
        // before the 'has started' email
        stage('Wait for first email') {
            environment {
                credentials=credentials("${imap_creds}")
            }
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'


                sh """./tests/jenkins/downstream/poll_email.py \
                    --server '${imap_server}' \
                    --guid ${guid} \
                    --timeout 20 \
                    --timeout 30 \
                    --filter 'has started'"""
            }
        }
        
        stage('Wait for last email and parse SSH location') {

        stage('Wait for last email and parse OpenShift and App location') {
            environment {
                credentials=credentials("${imap_creds}")
            }
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'

                script {
@@ -127,26 +129,15 @@
                          ./tests/jenkins/downstream/poll_email.py \
                          --server '${imap_server}' \
                          --guid ${guid} \
                          --timeout 100 \
                          --timeout 150 \
                          --filter 'has completed'
                        """
                    ).trim()

                    try {
                        def m = email =~ /Openshift Master Console: (.*)/
                        openshift_location = m[0][1]
                        echo "Openshift Master Console: ${openshift_location}"
                    
//                        m = email =~ /This cluster has authentication enabled. (.*)/
//                        echo "Cluster authentication:  ${m[0][1]}"
                      
//                        m = email =~ /SSH Access: (.*)/
//                        ssh_location = m[0][1]
//                        echo "SSH Access: ${ssh_location}"
						
//                        m = email =~ /SSH password: (.*)/
//                        ssh_p =​ m[0][1]
//                        echo "SSH password: ${ssh_p}"
                        def m = email =~ /Openshift Master Console: (https:\/\/master\.[^ ]+)/
                        openshift_location = m[0][1]
                        echo "openshift_location = '${openshift_location}'"
                    } catch(Exception ex) {
                        echo "Could not parse email:"
                        echo email
@@ -156,20 +147,6 @@
                }
            }
        }

//        stage('SSH') {
//            steps {
//                withCredentials([
//                    sshUserPrivateKey(
//                        credentialsId: ssh_creds,
//                        keyFileVariable: 'ssh_key',
//                        usernameVariable: 'ssh_username')
//                ]) {
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} w"
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} oc version"
//                }
//            }
//        }

        stage('Confirm before retiring') {
            when {
@@ -220,7 +197,7 @@
        }
        stage('Wait for deletion email') {
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'

                withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
@@ -249,6 +226,21 @@
                export DEBUG=true
                ./opentlc/delete_svc_guid.sh '${guid}'
                """
            }

            /* Print ansible logs */
            withCredentials([
                string(credentialsId: ssh_admin_host, variable: 'ssh_admin'),
                sshUserPrivateKey(
                    credentialsId: ssh_creds,
                    keyFileVariable: 'ssh_key',
                    usernameVariable: 'ssh_username')
            ]) {
                sh("""
                    ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_admin} \
                    "bin/logs.sh ${guid}" || true
                """.trim()
                )
            }

            withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
@@ -282,4 +274,4 @@
            }
        }
    }
}
}

 tests/jenkins/integreatly-workshop.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gptezabbixalert@redhat.com'
def notification_email = 'djana@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -25,26 +25,24 @@

// Catalog items
def choices = [
    'Workshops / Integreatly Workshop',
    'Workshops / RHMI (Integreatly) Workshop',
    'DevOps Team Testing Catalog / TEST - Integreatly Workshop',
    'DevOps Team Development Catalog / DEV - Integreatly Workshop',
].join("\n")

def ocprelease_choice = [
    '3.11.104',
    '3.11.43',
].join("\n")

def ig_version_choice = [
    '1.5.1',
    '1.5.0',
    '1.4.1',
def app_version_choice = [
    '1.6.0',
    '1.5.2',
].join("\n")

def region_choice = [
    'na_openshiftbu',
    'apac_openshift_bu',
    'emea_openshiftbu',
    'gpte_na',
    'gpte_emea',
    'gpte_apac',
].join("\n")

pipeline {
@@ -71,9 +69,9 @@
            name: 'ocprelease',
        )
        choice(
            choices: ig_version_choice,
            choices: app_version_choice,
            description: 'Catalog item',
            name: 'ig_version',
            name: 'app_version',
        )
        choice(
            choices: region_choice,
@@ -97,20 +95,20 @@
                script {
                    def catalog = params.catalog_item.split(' / ')[0].trim()
                    def item = params.catalog_item.split(' / ')[1].trim()
                    def ig_version = params.ig_version.trim()
                    def app_version = params.app_version.trim()
                    def ocprelease = params.ocprelease.trim()
                    def region = params.region.trim()
                    def cfparams = [
                        'expiration=2',
                        'runtime=8',
                        'expiration=3',
                        'runtime=36',
                        "region=${region}",
                        'city=jenkins',
                        'salesforce=gptejen',
                        'users=2',
                        'users=1',
                        'check=t',
                        'check2=t',
                        'quotacheck=t',
                        "app_version=${ig_version}",
                        "app_version=${app_version}",
                        "ocprelease=${ocprelease}",
                        'notes=devops_automation_jenkins',
                    ].join(',').trim()
@@ -164,7 +162,7 @@
                          ./tests/jenkins/downstream/poll_email.py \
                          --server '${imap_server}' \
                          --guid ${guid} \
                          --timeout 120 \
                          --timeout 150 \
                          --filter 'has completed'
                        """
                    ).trim()

 tests/jenkins/ocp-and-container-storage-for-admins.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gptezabbixalert@redhat.com'
def notification_email = 'gpteinfrasev2@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key

 tests/jenkins/ocp-clientvm.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key

 tests/jenkins/ocp-workload-iot-demo.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// state variables

 tests/jenkins/ocp-workshop-azure.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key

 tests/jenkins/ocp-workshop-integreatly.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key

 tests/jenkins/openshift-3-operators-lab.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -162,7 +162,7 @@
                }
            }
        }

        /* Skipping it as of now
        stage('SSH') {
            steps {
                withCredentials([
@@ -175,7 +175,7 @@
                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} oc version"
                }
            }
        }
        } */

        stage('Confirm before retiring') {
            when {

 tests/jenkins/openshift-3.11-shared-environment.groovy

copy from tests/jenkins/openshift-service-mesh-lab.groovy
copy to tests/jenkins/openshift-3.11-shared-environment.groovy
File was copied from tests/jenkins/openshift-service-mesh-lab.groovy
@@ -2,13 +2,13 @@
// CloudForms
def opentlc_creds = 'b93d2da4-c2b7-45b5-bf3b-ee2c08c6368e'
def opentlc_admin_creds = '73b84287-8feb-478a-b1f2-345fd0a1af47'
def cf_uri = 'https://labs.opentlc.com'
def cf_group = 'opentlc-access-cicd'
def cf_uri = 'https://rhpds.redhat.com'
def cf_group = 'rhpds-access-cicd'
// IMAP
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -20,20 +20,25 @@
// state variables
def guid=''
def openshift_location = ''
//def ssh_location = ''
//def ssh_p = ''


// Catalog items
def choices = [
    'OPENTLC OpenShift Labs / OpenShift Service Mesh Lab',
    'DevOps Team Development / DEV - OpenShift Service Mesh Lab',
    'OpenShift Demos / OpenShift 3.11 Shared Environment',
].join("\n")

def region_choice = [
    'na_sandboxes_gpte',
    'apac_sandboxes_gpte',
    'emea_sandboxes_gpte',
    'rhpds1',
].join("\n")

def nodes_choice = [
    '3',
    '1',
    '2',    
    '4',
    '5',
    '6',
    '7',
    '8',
].join("\n")

pipeline {
@@ -56,8 +61,13 @@
        )
        choice(
            choices: region_choice,
            description: 'Region',
            description: 'Catalog item',
            name: 'region',
        )
        choice(
            choices: nodes_choice,
            description: 'Number of Nodes',
            name: 'nodes',
        )
    }

@@ -77,6 +87,15 @@
                    def catalog = params.catalog_item.split(' / ')[0].trim()
                    def item = params.catalog_item.split(' / ')[1].trim()
                    def region = params.region.trim()
                    def nodes = params.nodes.trim()
                    def cfparams = [
                        'check=t',
                        'expiration=14',
                        'runtime=8',
                        'quotacheck=t',
                        "region=${region}",
                        "nodes=${nodes}",
                    ].join(',').trim()
                    echo "'${catalog}' '${item}'"
                    guid = sh(
                        returnStdout: true,
@@ -85,7 +104,7 @@
                          -c '${catalog}' \
                          -i '${item}' \
                          -G '${cf_group}' \
                          -d 'expiration=6,runtime=8,region=${region}'
                          -d '${cfparams}' \
                        """
                    ).trim()

@@ -93,31 +112,30 @@
                }
            }
        }
        /* Skip this step because sometimes the completed email arrives
         before the 'has started' email */
        // Skip this step because sometimes the completed email arrives
        // before the 'has started' email
        stage('Wait for first email') {
            environment {
                credentials=credentials("${imap_creds}")
            }
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'


                sh """./tests/jenkins/downstream/poll_email.py \
                    --server '${imap_server}' \
                    --guid ${guid} \
                    --timeout 20 \
                    --timeout 30 \
                    --filter 'has started'"""
            }
        }
        
        stage('Wait for last email and parse SSH location') {

        stage('Wait for last email and parse OpenShift and App location') {
            environment {
                credentials=credentials("${imap_creds}")
            }
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'

                script {
@@ -127,26 +145,15 @@
                          ./tests/jenkins/downstream/poll_email.py \
                          --server '${imap_server}' \
                          --guid ${guid} \
                          --timeout 100 \
                          --timeout 30 \
                          --filter 'has completed'
                        """
                    ).trim()

                    try {
                        def m = email =~ /Openshift Master Console: (.*)/
                        openshift_location = m[0][1]
                        echo "Openshift Master Console: ${openshift_location}"
                    
//                        m = email =~ /This cluster has authentication enabled. (.*)/
//                        echo "Cluster authentication:  ${m[0][1]}"
                      
//                        m = email =~ /SSH Access: (.*)/
//                        ssh_location = m[0][1]
//                        echo "SSH Access: ${ssh_location}"
						
//                        m = email =~ /SSH password: (.*)/
//                        ssh_p =​ m[0][1]
//                        echo "SSH password: ${ssh_p}"
                        def m = email =~ /Connect to the shared cluster by pointing your web browser to <a href="(https:\/\/master\.[^"]+)/
                        openshift_location = m[0][1]
                        echo "openshift_location = '${openshift_location}'"
                    } catch(Exception ex) {
                        echo "Could not parse email:"
                        echo email
@@ -156,20 +163,6 @@
                }
            }
        }

//        stage('SSH') {
//            steps {
//                withCredentials([
//                    sshUserPrivateKey(
//                        credentialsId: ssh_creds,
//                        keyFileVariable: 'ssh_key',
//                        usernameVariable: 'ssh_username')
//                ]) {
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} w"
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} oc version"
//                }
//            }
//        }

        stage('Confirm before retiring') {
            when {
@@ -220,7 +213,7 @@
        }
        stage('Wait for deletion email') {
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'

                withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
@@ -249,6 +242,21 @@
                export DEBUG=true
                ./opentlc/delete_svc_guid.sh '${guid}'
                """
            }

            /* Print ansible logs */
            withCredentials([
                string(credentialsId: ssh_admin_host, variable: 'ssh_admin'),
                sshUserPrivateKey(
                    credentialsId: ssh_creds,
                    keyFileVariable: 'ssh_key',
                    usernameVariable: 'ssh_username')
            ]) {
                sh("""
                    ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_admin} \
                    "bin/logs.sh ${guid}" || true
                """.trim()
                )
            }

            withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
@@ -282,4 +290,4 @@
            }
        }
    }
}
}

 tests/jenkins/openshift-4-aws-install-vm.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -125,7 +125,7 @@
                          ./tests/jenkins/downstream/poll_email.py \
                          --server '${imap_server}' \
                          --guid ${guid} \
                          --timeout 20 \
                          --timeout 60 \
                          --filter 'has completed'
                        """
                    ).trim()

 tests/jenkins/openshift-4-client-vm.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key

 tests/jenkins/openshift-4-configuration-lab.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev2@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key

 tests/jenkins/openshift-4-installation-lab.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -125,7 +125,7 @@
                          ./tests/jenkins/downstream/poll_email.py \
                          --server '${imap_server}' \
                          --guid ${guid} \
                          --timeout 20 \
                          --timeout 60 \
                          --filter 'has completed'
                        """
                    ).trim()

 tests/jenkins/openshift-4-service-mesh-lab.groovy

copy from tests/jenkins/openshift-service-mesh-lab.groovy
copy to tests/jenkins/openshift-4-service-mesh-lab.groovy
File was copied from tests/jenkins/openshift-service-mesh-lab.groovy
@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'djana@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -20,20 +20,26 @@
// state variables
def guid=''
def openshift_location = ''
//def ssh_location = ''
//def ssh_p = ''
def ssh_location = ''
// def ssh_p = ''


// Catalog items
def choices = [
    'OPENTLC OpenShift Labs / OpenShift Service Mesh Lab',
    'DevOps Team Development / DEV - OpenShift Service Mesh Lab',
    'OPENTLC OpenShift 4 Labs / OpenShift 4 Service Mesh Lab',
    'DevOps Team Development / PREPROD - OpenShift Service Mesh Lab',
].join("\n")

def region_choice = [
    'na_sandboxes_gpte',
    'apac_sandboxes_gpte',
    'emea_sandboxes_gpte',
].join("\n")

def environment_choice = [
    'PROD',
    'TEST',
    'DEV',
].join("\n")

pipeline {
@@ -59,6 +65,11 @@
            description: 'Region',
            name: 'region',
        )
        choice(
            choices: environment_choice,
            description: 'Environment',
            name: 'environment',
        )
    }

    stages {
@@ -71,12 +82,13 @@
            /* This step use the order_svc_guid.sh script to order
             a service from CloudForms */
            steps {
                git url: 'https://github.com/redhat-gpte-devopsautomation/cloudforms-oob'
                git url: 'https://github.com/redhat-cop/agnosticd'

                script {
                    def catalog = params.catalog_item.split(' / ')[0].trim()
                    def item = params.catalog_item.split(' / ')[1].trim()
                    def region = params.region.trim()
                    def environment = params.environment.trim()
                    echo "'${catalog}' '${item}'"
                    guid = sh(
                        returnStdout: true,
@@ -85,7 +97,7 @@
                          -c '${catalog}' \
                          -i '${item}' \
                          -G '${cf_group}' \
                          -d 'expiration=6,runtime=8,region=${region}'
                          -d 'check=t,expiration=7,runtime=10,region=${region},environment=${environment}'
                        """
                    ).trim()

@@ -136,17 +148,11 @@
                        def m = email =~ /Openshift Master Console: (.*)/
                        openshift_location = m[0][1]
                        echo "Openshift Master Console: ${openshift_location}"
                    
//                        m = email =~ /This cluster has authentication enabled. (.*)/
//                        echo "Cluster authentication:  ${m[0][1]}"
                      
//                        m = email =~ /SSH Access: (.*)/
//                        ssh_location = m[0][1]
//                        echo "SSH Access: ${ssh_location}"
						
//                        m = email =~ /SSH password: (.*)/
//                        ssh_p =​ m[0][1]
//                        echo "SSH password: ${ssh_p}"
                        m = email =~ /SSH Access: (.*)/
                        ssh_location = m[0][1]
                        echo "SSH Access: ${ssh_location}"

                    } catch(Exception ex) {
                        echo "Could not parse email:"
                        echo email
@@ -156,20 +162,20 @@
                }
            }
        }

//        stage('SSH') {
//            steps {
//                withCredentials([
//                    sshUserPrivateKey(
//                        credentialsId: ssh_creds,
//                        keyFileVariable: 'ssh_key',
//                        usernameVariable: 'ssh_username')
//                ]) {
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} w"
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} oc version"
//                }
//            }
//        }
        /*
        stage('SSH') {
            steps {
                withCredentials([
                    sshUserPrivateKey(
                        credentialsId: ssh_creds,
                        keyFileVariable: 'ssh_key',
                        usernameVariable: 'ssh_username')
                ]) {
                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} w"
                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} oc version"
                }
            }
        }*/

        stage('Confirm before retiring') {
            when {
@@ -251,6 +257,21 @@
                """
            }

            /* Print ansible logs */
            withCredentials([
                string(credentialsId: ssh_admin_host, variable: 'ssh_admin'),
                sshUserPrivateKey(
                    credentialsId: ssh_creds,
                    keyFileVariable: 'ssh_key',
                    usernameVariable: 'ssh_username')
            ]) {
                sh("""
                    ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_admin} \
                    "bin/logs.sh ${guid}" || true
                """.trim()
                )
            }

            withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
                mail(
                    subject: "${env.JOB_NAME} (${env.BUILD_NUMBER}) failed GUID=${guid}",
@@ -282,4 +303,4 @@
            }
        }
    }
}
}

 tests/jenkins/openshift-4-shared-access.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key

 tests/jenkins/openshift-4-shared-environment.groovy

copy from tests/jenkins/openshift-service-mesh-lab.groovy
copy to tests/jenkins/openshift-4-shared-environment.groovy
File was copied from tests/jenkins/openshift-service-mesh-lab.groovy
@@ -2,13 +2,13 @@
// CloudForms
def opentlc_creds = 'b93d2da4-c2b7-45b5-bf3b-ee2c08c6368e'
def opentlc_admin_creds = '73b84287-8feb-478a-b1f2-345fd0a1af47'
def cf_uri = 'https://labs.opentlc.com'
def cf_group = 'opentlc-access-cicd'
def cf_uri = 'https://rhpds.redhat.com'
def cf_group = 'rhpds-access-cicd'
// IMAP
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev2@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -20,20 +20,27 @@
// state variables
def guid=''
def openshift_location = ''
//def ssh_location = ''
//def ssh_p = ''


// Catalog items
def choices = [
    'OPENTLC OpenShift Labs / OpenShift Service Mesh Lab',
    'DevOps Team Development / DEV - OpenShift Service Mesh Lab',
    'OpenShift Demos / OpenShift 4 Shared Environment',
    'DevOps Shared Cluster Testing / Testing - OpenShift 4 Shared Environment',
    'DevOps Shared Cluster Development / DEV - OpenShift 4 Shared Environment',
].join("\n")

def region_choice = [
    'na_sandboxes_gpte',
    'apac_sandboxes_gpte',
    'emea_sandboxes_gpte',
    'rhpds1',
].join("\n")

def nodes_choice = [
    '3',
    '1',
    '2',    
    '4',
    '5',
    '6',
    '7',
    '8',
].join("\n")

pipeline {
@@ -56,8 +63,13 @@
        )
        choice(
            choices: region_choice,
            description: 'Region',
            description: 'Catalog item',
            name: 'region',
        )
        choice(
            choices: nodes_choice,
            description: 'Number of Nodes',
            name: 'nodes',
        )
    }

@@ -77,6 +89,15 @@
                    def catalog = params.catalog_item.split(' / ')[0].trim()
                    def item = params.catalog_item.split(' / ')[1].trim()
                    def region = params.region.trim()
                    def nodes = params.nodes.trim()
                    def cfparams = [
                        'check=t',
                        'expiration=14',
                        'runtime=8',
                        'quotacheck=t',
                        "region=${region}",
                        "nodes=${nodes}",
                    ].join(',').trim()
                    echo "'${catalog}' '${item}'"
                    guid = sh(
                        returnStdout: true,
@@ -85,7 +106,7 @@
                          -c '${catalog}' \
                          -i '${item}' \
                          -G '${cf_group}' \
                          -d 'expiration=6,runtime=8,region=${region}'
                          -d '${cfparams}' \
                        """
                    ).trim()

@@ -93,31 +114,30 @@
                }
            }
        }
        /* Skip this step because sometimes the completed email arrives
         before the 'has started' email */
        // Skip this step because sometimes the completed email arrives
        // before the 'has started' email
        stage('Wait for first email') {
            environment {
                credentials=credentials("${imap_creds}")
            }
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'


                sh """./tests/jenkins/downstream/poll_email.py \
                    --server '${imap_server}' \
                    --guid ${guid} \
                    --timeout 20 \
                    --timeout 30 \
                    --filter 'has started'"""
            }
        }
        
        stage('Wait for last email and parse SSH location') {

        stage('Wait for last email and parse OpenShift and App location') {
            environment {
                credentials=credentials("${imap_creds}")
            }
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'

                script {
@@ -127,26 +147,15 @@
                          ./tests/jenkins/downstream/poll_email.py \
                          --server '${imap_server}' \
                          --guid ${guid} \
                          --timeout 100 \
                          --timeout 30 \
                          --filter 'has completed'
                        """
                    ).trim()

                    try {
                        def m = email =~ /Openshift Master Console: (.*)/
                        openshift_location = m[0][1]
                        echo "Openshift Master Console: ${openshift_location}"
                    
//                        m = email =~ /This cluster has authentication enabled. (.*)/
//                        echo "Cluster authentication:  ${m[0][1]}"
                      
//                        m = email =~ /SSH Access: (.*)/
//                        ssh_location = m[0][1]
//                        echo "SSH Access: ${ssh_location}"
						
//                        m = email =~ /SSH password: (.*)/
//                        ssh_p =​ m[0][1]
//                        echo "SSH password: ${ssh_p}"
                        def m = email =~ /Openshift Master Console: (http:\/\/[^ \n]+)/
                        openshift_location = m[0][1]
                        echo "openshift_location = '${openshift_location}'"
                    } catch(Exception ex) {
                        echo "Could not parse email:"
                        echo email
@@ -156,20 +165,6 @@
                }
            }
        }

//        stage('SSH') {
//            steps {
//                withCredentials([
//                    sshUserPrivateKey(
//                        credentialsId: ssh_creds,
//                        keyFileVariable: 'ssh_key',
//                        usernameVariable: 'ssh_username')
//                ]) {
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} w"
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} oc version"
//                }
//            }
//        }

        stage('Confirm before retiring') {
            when {
@@ -220,7 +215,7 @@
        }
        stage('Wait for deletion email') {
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'

                withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
@@ -249,6 +244,21 @@
                export DEBUG=true
                ./opentlc/delete_svc_guid.sh '${guid}'
                """
            }

            /* Print ansible logs */
            withCredentials([
                string(credentialsId: ssh_admin_host, variable: 'ssh_admin'),
                sshUserPrivateKey(
                    credentialsId: ssh_creds,
                    keyFileVariable: 'ssh_key',
                    usernameVariable: 'ssh_username')
            ]) {
                sh("""
                    ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_admin} \
                    "bin/logs.sh ${guid}" || true
                """.trim()
                )
            }

            withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
@@ -282,4 +292,4 @@
            }
        }
    }
}
}

 tests/jenkins/openshift-4-workshop.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gptezabbixalert@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key

 tests/jenkins/openshift-4.2-workshop.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gptezabbixalert@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key

 tests/jenkins/openshift-disconnected-ha-lab.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'djana@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key

 tests/jenkins/openshift-gpu-enabled-workload-demo-beta.groovy

copy from tests/jenkins/openshift-service-mesh-lab.groovy
copy to tests/jenkins/openshift-gpu-enabled-workload-demo-beta.groovy
File was copied from tests/jenkins/openshift-service-mesh-lab.groovy
@@ -2,13 +2,13 @@
// CloudForms
def opentlc_creds = 'b93d2da4-c2b7-45b5-bf3b-ee2c08c6368e'
def opentlc_admin_creds = '73b84287-8feb-478a-b1f2-345fd0a1af47'
def cf_uri = 'https://labs.opentlc.com'
def cf_group = 'opentlc-access-cicd'
def cf_uri = 'https://rhpds.redhat.com'
def cf_group = 'rhpds-access-cicd'
// IMAP
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -20,20 +20,15 @@
// state variables
def guid=''
def openshift_location = ''
//def ssh_location = ''
//def ssh_p = ''


// Catalog items
def choices = [
    'OPENTLC OpenShift Labs / OpenShift Service Mesh Lab',
    'DevOps Team Development / DEV - OpenShift Service Mesh Lab',
    'OpenShift Demos / OpenShift GPU-enabled Workload Demo Beta',
    'DevOps Team Development Catalog / DEV-OpenShift GPU-enabled Workload Demo',
].join("\n")

def region_choice = [
    'na_sandboxes_gpte',
    'apac_sandboxes_gpte',
    'emea_sandboxes_gpte',
    'na_gpte',
].join("\n")

pipeline {
@@ -56,7 +51,7 @@
        )
        choice(
            choices: region_choice,
            description: 'Region',
            description: 'Catalog item',
            name: 'region',
        )
    }
@@ -77,6 +72,16 @@
                    def catalog = params.catalog_item.split(' / ')[0].trim()
                    def item = params.catalog_item.split(' / ')[1].trim()
                    def region = params.region.trim()
                    def cfparams = [
                        'expiration=2',
                        'runtime=8',
                        "region=${region}",
                        'city=jenkins',
                        'salesforce=gptejen',
                        'check=t',
                        'check2=t',
                        'notes=devops_automation_jenkins',
                    ].join(',').trim()
                    echo "'${catalog}' '${item}'"
                    guid = sh(
                        returnStdout: true,
@@ -85,7 +90,7 @@
                          -c '${catalog}' \
                          -i '${item}' \
                          -G '${cf_group}' \
                          -d 'expiration=6,runtime=8,region=${region}'
                          -d '${cfparams}' \
                        """
                    ).trim()

@@ -93,31 +98,30 @@
                }
            }
        }
        /* Skip this step because sometimes the completed email arrives
         before the 'has started' email */
        // Skip this step because sometimes the completed email arrives
        // before the 'has started' email
        stage('Wait for first email') {
            environment {
                credentials=credentials("${imap_creds}")
            }
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'


                sh """./tests/jenkins/downstream/poll_email.py \
                    --server '${imap_server}' \
                    --guid ${guid} \
                    --timeout 20 \
                    --timeout 30 \
                    --filter 'has started'"""
            }
        }
        
        stage('Wait for last email and parse SSH location') {

        stage('Wait for last email and parse OpenShift and App location') {
            environment {
                credentials=credentials("${imap_creds}")
            }
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'

                script {
@@ -127,26 +131,15 @@
                          ./tests/jenkins/downstream/poll_email.py \
                          --server '${imap_server}' \
                          --guid ${guid} \
                          --timeout 100 \
                          --timeout 150 \
                          --filter 'has completed'
                        """
                    ).trim()

                    try {
                        def m = email =~ /Openshift Master Console: (.*)/
                        openshift_location = m[0][1]
                        echo "Openshift Master Console: ${openshift_location}"
                    
//                        m = email =~ /This cluster has authentication enabled. (.*)/
//                        echo "Cluster authentication:  ${m[0][1]}"
                      
//                        m = email =~ /SSH Access: (.*)/
//                        ssh_location = m[0][1]
//                        echo "SSH Access: ${ssh_location}"
						
//                        m = email =~ /SSH password: (.*)/
//                        ssh_p =​ m[0][1]
//                        echo "SSH password: ${ssh_p}"
                        def m = email =~ /The OpenShift console for your environment is available at <a href="(https:\/\/master\.[^"]+)/
                        openshift_location = m[0][1]
                        echo "openshift_location = '${openshift_location}'"
                    } catch(Exception ex) {
                        echo "Could not parse email:"
                        echo email
@@ -156,20 +149,6 @@
                }
            }
        }

//        stage('SSH') {
//            steps {
//                withCredentials([
//                    sshUserPrivateKey(
//                        credentialsId: ssh_creds,
//                        keyFileVariable: 'ssh_key',
//                        usernameVariable: 'ssh_username')
//                ]) {
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} w"
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} oc version"
//                }
//            }
//        }

        stage('Confirm before retiring') {
            when {
@@ -220,7 +199,7 @@
        }
        stage('Wait for deletion email') {
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'

                withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
@@ -251,6 +230,21 @@
                """
            }

            /* Print ansible logs */
            withCredentials([
                string(credentialsId: ssh_admin_host, variable: 'ssh_admin'),
                sshUserPrivateKey(
                    credentialsId: ssh_creds,
                    keyFileVariable: 'ssh_key',
                    usernameVariable: 'ssh_username')
            ]) {
                sh("""
                    ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_admin} \
                    "bin/logs.sh ${guid}" || true
                """.trim()
                )
            }

            withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
                mail(
                    subject: "${env.JOB_NAME} (${env.BUILD_NUMBER}) failed GUID=${guid}",

 tests/jenkins/openshift-ha-lab.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -152,7 +152,7 @@
                          ./tests/jenkins/downstream/poll_email.py \
                          --server '${imap_server}' \
                          --guid ${guid} \
                          --timeout 40 \
                          --timeout 60 \
                          --filter 'has completed'
                        """
                    ).trim()

 tests/jenkins/openshift-implementation-lab.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -149,7 +149,7 @@
                          ./tests/jenkins/downstream/poll_email.py \
                          --server '${imap_server}' \
                          --guid ${guid} \
                          --timeout 40 \
                          --timeout 90 \
                          --filter 'has completed'
                        """
                    ).trim()

 tests/jenkins/openshift-operators-lab.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -19,7 +19,7 @@

// state variables
def guid=''
def ssh_location = ''
def openshift_location = ''


// Catalog items
@@ -155,9 +155,9 @@
                    ).trim()

                    try {
                        def m = email =~ /<pre>. *ssh -i [^ ]+ *([^ <]+?) *<\/pre>/
                        ssh_location = m[0][1]
                        echo "ssh_location = '${ssh_location}'"
                        def m = email =~ /Openshift Master Console: (.*)/
                        openshift_location = m[0][1]
                        echo "openshift_location = '${openshift_location}'"
                    } catch(Exception ex) {
                        echo "Could not parse email:"
                        echo email
@@ -169,6 +169,7 @@
            }
        }

        /* Skipping it as of now
        stage('SSH') {
            steps {
                withCredentials([
@@ -181,7 +182,7 @@
                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} oc version"
                }
            }
        }
        } */

        stage('Confirm before retiring') {
            when {

 tests/jenkins/openshift-service-mesh-in-action.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gptezabbixalert@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key

 tests/jenkins/openshift-shared-access.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key

 tests/jenkins/openshift-storage-lab.groovy

copy from tests/jenkins/openshift-service-mesh-lab.groovy
copy to tests/jenkins/openshift-storage-lab.groovy
File was copied from tests/jenkins/openshift-service-mesh-lab.groovy
@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -19,21 +19,31 @@

// state variables
def guid=''
def openshift_location = ''
//def ssh_location = ''
//def ssh_p = ''
def ssh_location = ''


// Catalog items
def choices = [
    'OPENTLC OpenShift Labs / OpenShift Service Mesh Lab',
    'DevOps Team Development / DEV - OpenShift Service Mesh Lab',
    'OPENTLC OpenShift Labs / OpenShift Storage Lab',
    'DevOps Team Development / DEV OCP Storage Lab',	
].join("\n")

def region_choice = [
    'na_sandboxes_gpte',
    'apac_sandboxes_gpte',
    'emea_sandboxes_gpte',
    'na',
    'emea',
    'latam',
    'apac',
].join("\n")

def nodes_choice = [
    '2',
    '1',
    '3',    
    '4',
    '5',
    '6',
    '7',
    '8',
].join("\n")

pipeline {
@@ -59,6 +69,11 @@
            description: 'Region',
            name: 'region',
        )
        choice(
            choices: nodes_choice,
            description: 'Number of Nodes',
            name: 'nodes',
        )
    }

    stages {
@@ -77,6 +92,7 @@
                    def catalog = params.catalog_item.split(' / ')[0].trim()
                    def item = params.catalog_item.split(' / ')[1].trim()
                    def region = params.region.trim()
                    def nodes = params.nodes.trim()
                    echo "'${catalog}' '${item}'"
                    guid = sh(
                        returnStdout: true,
@@ -85,7 +101,7 @@
                          -c '${catalog}' \
                          -i '${item}' \
                          -G '${cf_group}' \
                          -d 'expiration=6,runtime=8,region=${region}'
                          -d 'expiration=7,runtime=8,region=${region},nodes=${nodes}'
                        """
                    ).trim()

@@ -93,6 +109,7 @@
                }
            }
        }
        
        /* Skip this step because sometimes the completed email arrives
         before the 'has started' email */
        stage('Wait for first email') {
@@ -133,43 +150,33 @@
                    ).trim()

                    try {
                        def m = email =~ /Openshift Master Console: (.*)/
                        openshift_location = m[0][1]
                        echo "Openshift Master Console: ${openshift_location}"
                    
//                        m = email =~ /This cluster has authentication enabled. (.*)/
//                        echo "Cluster authentication:  ${m[0][1]}"
                      
//                        m = email =~ /SSH Access: (.*)/
//                        ssh_location = m[0][1]
//                        echo "SSH Access: ${ssh_location}"
						
//                        m = email =~ /SSH password: (.*)/
//                        ssh_p =​ m[0][1]
//                        echo "SSH password: ${ssh_p}"
                        def m = email =~ /<pre>. *ssh -i [^ ]+ *([^ <]+?) *<\/pre>/
                        ssh_location = m[0][1]
                        echo "ssh_location = '${ssh_location}'"
                    } catch(Exception ex) {
                        echo "Could not parse email:"
                        echo email
                        echo ex.toString()
                        throw ex
                    }

                }
            }
        }

//        stage('SSH') {
//            steps {
//                withCredentials([
//                    sshUserPrivateKey(
//                        credentialsId: ssh_creds,
//                        keyFileVariable: 'ssh_key',
//                        usernameVariable: 'ssh_username')
//                ]) {
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} w"
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} oc version"
//                }
//            }
//        }
        /* Skipping it as of now
        stage('SSH') {
            steps {
                withCredentials([
                    sshUserPrivateKey(
                        credentialsId: ssh_creds,
                        keyFileVariable: 'ssh_key',
                        usernameVariable: 'ssh_username')
                ]) {
                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} w"
                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} oc version"
                }
            }
        } */

        stage('Confirm before retiring') {
            when {
@@ -251,6 +258,21 @@
                """
            }

            /* Print ansible logs */
            withCredentials([
                string(credentialsId: ssh_admin_host, variable: 'ssh_admin'),
                sshUserPrivateKey(
                    credentialsId: ssh_creds,
                    keyFileVariable: 'ssh_key',
                    usernameVariable: 'ssh_username')
            ]) {
                sh("""
                    ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_admin} \
                    "find deployer_logs -name '*${guid}*log' | xargs cat"
                """.trim()
                )
            }

            withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
                mail(
                    subject: "${env.JOB_NAME} (${env.BUILD_NUMBER}) failed GUID=${guid}",

 tests/jenkins/openshift-workshop-deployer.groovy

copy from tests/jenkins/openshift-service-mesh-lab.groovy
copy to tests/jenkins/openshift-workshop-deployer.groovy
File was copied from tests/jenkins/openshift-service-mesh-lab.groovy
@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -19,21 +19,50 @@

// state variables
def guid=''
def openshift_location = ''
//def ssh_location = ''
//def ssh_p = ''
def ssh_location = ''


// Catalog items
def choices = [
    'OPENTLC OpenShift Labs / OpenShift Service Mesh Lab',
    'DevOps Team Development / DEV - OpenShift Service Mesh Lab',
    'OPENTLC OpenShift Labs / OpenShift Workshop Deployer',
].join("\n")

def ocprelease_choice = [
    '3.11.43',
    '3.11.129',
    '3.11.59',
    '3.11.16',
    '3.10.34',
    '3.10.14',
    '3.9.40',
    '3.7.23',
].join("\n")

def region_choice = [
    'na_sandboxes_gpte',
    'apac_sandboxes_gpte',
    'emea_sandboxes_gpte',
    'na',
    'emea',
    'latam',
    'apac',
].join("\n")

def nodes_choice = [
    '3',
    '1',
    '2',    
    '4',
    '5',
    '6',
    '7',
    '8',
    '9',
    '10',
    '15',
].join("\n")

def environment_choice = [
    'PROD',
    'TEST',
    'DEV',
].join("\n")

pipeline {
@@ -55,9 +84,25 @@
            name: 'catalog_item',
        )
        choice(
            choices: ocprelease_choice,
            description: 'Catalog item',
            name: 'ocprelease',
        )

        choice(
            choices: region_choice,
            description: 'Region',
            name: 'region',
        )
        choice(
            choices: nodes_choice,
            description: 'Number of Nodes',
            name: 'nodes',
        )
        choice(
            choices: environment_choice,
            description: 'Environment',
            name: 'environment',
        )
    }

@@ -76,7 +121,10 @@
                script {
                    def catalog = params.catalog_item.split(' / ')[0].trim()
                    def item = params.catalog_item.split(' / ')[1].trim()
                    def ocprelease = params.ocprelease.trim()
                    def region = params.region.trim()
                    def nodes = params.nodes.trim()
                    def environment = params.environment.trim()
                    echo "'${catalog}' '${item}'"
                    guid = sh(
                        returnStdout: true,
@@ -85,7 +133,7 @@
                          -c '${catalog}' \
                          -i '${item}' \
                          -G '${cf_group}' \
                          -d 'expiration=6,runtime=8,region=${region}'
                          -d 'expiration=7,runtime=8,ocprelease=${ocprelease},region=${region},nodes=${nodes},environment=${environment}'
                        """
                    ).trim()

@@ -93,14 +141,13 @@
                }
            }
        }
        /* Skip this step because sometimes the completed email arrives
         before the 'has started' email */
        
        stage('Wait for first email') {
            environment {
                credentials=credentials("${imap_creds}")
            }
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'


@@ -110,14 +157,14 @@
                    --timeout 20 \
                    --filter 'has started'"""
            }
        }
        
        stage('Wait for last email and parse SSH location') {
        }	

        stage('Wait for last email and parse') {
            environment {
                credentials=credentials("${imap_creds}")
            }
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'

                script {
@@ -127,26 +174,15 @@
                          ./tests/jenkins/downstream/poll_email.py \
                          --server '${imap_server}' \
                          --guid ${guid} \
                          --timeout 100 \
                          --timeout 150 \
                          --filter 'has completed'
                        """
                    ).trim()

                    try {
                        def m = email =~ /Openshift Master Console: (.*)/
                        openshift_location = m[0][1]
                        echo "Openshift Master Console: ${openshift_location}"
                    
//                        m = email =~ /This cluster has authentication enabled. (.*)/
//                        echo "Cluster authentication:  ${m[0][1]}"
                      
//                        m = email =~ /SSH Access: (.*)/
//                        ssh_location = m[0][1]
//                        echo "SSH Access: ${ssh_location}"
						
//                        m = email =~ /SSH password: (.*)/
//                        ssh_p =​ m[0][1]
//                        echo "SSH password: ${ssh_p}"
                        def m = email =~ /<pre>. *ssh -i [^ ]+ *([^ <]+?) *<\/pre>/
                        ssh_location = m[0][1]
                        echo "SSH command: '${ssh_location}'"
                    } catch(Exception ex) {
                        echo "Could not parse email:"
                        echo email
@@ -156,20 +192,6 @@
                }
            }
        }

//        stage('SSH') {
//            steps {
//                withCredentials([
//                    sshUserPrivateKey(
//                        credentialsId: ssh_creds,
//                        keyFileVariable: 'ssh_key',
//                        usernameVariable: 'ssh_username')
//                ]) {
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} w"
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} oc version"
//                }
//            }
//        }

        stage('Confirm before retiring') {
            when {
@@ -251,6 +273,21 @@
                """
            }

            /* Print ansible logs */
            withCredentials([
                string(credentialsId: ssh_admin_host, variable: 'ssh_admin'),
                sshUserPrivateKey(
                    credentialsId: ssh_creds,
                    keyFileVariable: 'ssh_key',
                    usernameVariable: 'ssh_username')
            ]) {
                sh("""
                    ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_admin} \
                    "find deployer_logs -name '*${guid}*log' | xargs cat"
                """.trim()
                )
            }

            withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
                mail(
                    subject: "${env.JOB_NAME} (${env.BUILD_NUMBER}) failed GUID=${guid}",

 tests/jenkins/openshift-workshop.groovy

@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gptezabbixalert@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -154,7 +154,7 @@
                          ./tests/jenkins/downstream/poll_email.py \
                          --server '${imap_server}' \
                          --guid ${guid} \
                          --timeout 60 \
                          --timeout 150 \
                          --filter 'has completed'
                        """
                    ).trim()

 tests/jenkins/openstack-10-concepts-and-architecture.groovy

copy from tests/jenkins/openshift-service-mesh-lab.groovy
copy to tests/jenkins/openstack-10-concepts-and-architecture.groovy
File was copied from tests/jenkins/openshift-service-mesh-lab.groovy
@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -19,21 +19,16 @@

// state variables
def guid=''
def openshift_location = ''
//def ssh_location = ''
//def ssh_p = ''
def external_host = ''


// Catalog items
def choices = [
    'OPENTLC OpenShift Labs / OpenShift Service Mesh Lab',
    'DevOps Team Development / DEV - OpenShift Service Mesh Lab',
    'OPENTLC Cloud Infrastructure Labs / OpenStack 10 Concepts and Architecture',
].join("\n")

def region_choice = [
    'na_sandboxes_gpte',
    'apac_sandboxes_gpte',
    'emea_sandboxes_gpte',
    'global_gpte',
].join("\n")

pipeline {
@@ -56,7 +51,7 @@
        )
        choice(
            choices: region_choice,
            description: 'Region',
            description: 'Catalog item',
            name: 'region',
        )
    }
@@ -77,6 +72,14 @@
                    def catalog = params.catalog_item.split(' / ')[0].trim()
                    def item = params.catalog_item.split(' / ')[1].trim()
                    def region = params.region.trim()
                    def cfparams = [
                        'check=t',
                        'expiration=7',
                        'runtime=4',
                        "region=${region}",
                        'quotacheck=t',
                    ].join(',').trim()

                    echo "'${catalog}' '${item}'"
                    guid = sh(
                        returnStdout: true,
@@ -85,7 +88,7 @@
                          -c '${catalog}' \
                          -i '${item}' \
                          -G '${cf_group}' \
                          -d 'expiration=6,runtime=8,region=${region}'
                          -d '${cfparams}' \
                        """
                    ).trim()

@@ -93,26 +96,9 @@
                }
            }
        }
        /* Skip this step because sometimes the completed email arrives
         before the 'has started' email */
        stage('Wait for first email') {
            environment {
                credentials=credentials("${imap_creds}")
            }
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                    branch: 'development'


                sh """./tests/jenkins/downstream/poll_email.py \
                    --server '${imap_server}' \
                    --guid ${guid} \
                    --timeout 20 \
                    --filter 'has started'"""
            }
        }
        
        stage('Wait for last email and parse SSH location') {
        // This kind of CI send only one mail
        stage('Wait to receive and parse email') {
            environment {
                credentials=credentials("${imap_creds}")
            }
@@ -127,26 +113,16 @@
                          ./tests/jenkins/downstream/poll_email.py \
                          --server '${imap_server}' \
                          --guid ${guid} \
                          --timeout 100 \
                          --filter 'has completed'
                          --timeout 30 \
                          --filter 'is building'
                        """
                    ).trim()

                    try {
                        def m = email =~ /Openshift Master Console: (.*)/
                        openshift_location = m[0][1]
                        echo "Openshift Master Console: ${openshift_location}"
                    
//                        m = email =~ /This cluster has authentication enabled. (.*)/
//                        echo "Cluster authentication:  ${m[0][1]}"
                      
//                        m = email =~ /SSH Access: (.*)/
//                        ssh_location = m[0][1]
//                        echo "SSH Access: ${ssh_location}"
						
//                        m = email =~ /SSH password: (.*)/
//                        ssh_p =​ m[0][1]
//                        echo "SSH password: ${ssh_p}"
                        def m = email =~ /External Hostname<\/TH><TD>(.*)/
                        def mm = email =~ /(.*)<\/TD><\/TR><TR><TH>Internal Hostname/
                        external_host = m[0][1].replaceAll("=","") + mm[0][1].replaceAll(" ","")
                        echo "External-Host='${external_host}'"
                    } catch(Exception ex) {
                        echo "Could not parse email:"
                        echo email
@@ -156,20 +132,13 @@
                }
            }
        }

//        stage('SSH') {
//            steps {
//                withCredentials([
//                    sshUserPrivateKey(
//                        credentialsId: ssh_creds,
//                        keyFileVariable: 'ssh_key',
//                        usernameVariable: 'ssh_username')
//                ]) {
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} w"
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} oc version"
//                }
//            }
//        }
        
        stage ('Wait to complete provision') {
            steps {
                echo "Wait for 30 minutes for deployment to complete"
                sleep 1800 // seconds
            }
        }

        stage('Confirm before retiring') {
            when {
@@ -220,7 +189,7 @@
        }
        stage('Wait for deletion email') {
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'

                withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
@@ -251,6 +220,21 @@
                """
            }

            /* Print ansible logs */
            withCredentials([
                string(credentialsId: ssh_admin_host, variable: 'ssh_admin'),
                sshUserPrivateKey(
                    credentialsId: ssh_creds,
                    keyFileVariable: 'ssh_key',
                    usernameVariable: 'ssh_username')
            ]) {
                sh("""
                    ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_admin} \
                    "bin/logs.sh ${guid}" || true
                """.trim()
                )
            }

            withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
                mail(
                    subject: "${env.JOB_NAME} (${env.BUILD_NUMBER}) failed GUID=${guid}",

 tests/jenkins/openstack-13-foundations.groovy

copy from tests/jenkins/openshift-service-mesh-lab.groovy
copy to tests/jenkins/openstack-13-foundations.groovy
File was copied from tests/jenkins/openshift-service-mesh-lab.groovy
@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -19,21 +19,16 @@

// state variables
def guid=''
def openshift_location = ''
//def ssh_location = ''
//def ssh_p = ''
def external_host = ''


// Catalog items
def choices = [
    'OPENTLC OpenShift Labs / OpenShift Service Mesh Lab',
    'DevOps Team Development / DEV - OpenShift Service Mesh Lab',
    'OPENTLC Cloud Infrastructure Labs / OpenStack 13 Foundations',
].join("\n")

def region_choice = [
    'na_sandboxes_gpte',
    'apac_sandboxes_gpte',
    'emea_sandboxes_gpte',
    'global_gpte',
].join("\n")

pipeline {
@@ -56,7 +51,7 @@
        )
        choice(
            choices: region_choice,
            description: 'Region',
            description: 'Catalog item',
            name: 'region',
        )
    }
@@ -77,6 +72,14 @@
                    def catalog = params.catalog_item.split(' / ')[0].trim()
                    def item = params.catalog_item.split(' / ')[1].trim()
                    def region = params.region.trim()
                    def cfparams = [
                        'check=t',
                        'expiration=7',
                        'runtime=8',
                        "region=${region}",
                        'quotacheck=t',
                    ].join(',').trim()

                    echo "'${catalog}' '${item}'"
                    guid = sh(
                        returnStdout: true,
@@ -85,7 +88,7 @@
                          -c '${catalog}' \
                          -i '${item}' \
                          -G '${cf_group}' \
                          -d 'expiration=6,runtime=8,region=${region}'
                          -d '${cfparams}' \
                        """
                    ).trim()

@@ -93,26 +96,9 @@
                }
            }
        }
        /* Skip this step because sometimes the completed email arrives
         before the 'has started' email */
        stage('Wait for first email') {
            environment {
                credentials=credentials("${imap_creds}")
            }
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                    branch: 'development'


                sh """./tests/jenkins/downstream/poll_email.py \
                    --server '${imap_server}' \
                    --guid ${guid} \
                    --timeout 20 \
                    --filter 'has started'"""
            }
        }
        
        stage('Wait for last email and parse SSH location') {
        // This kind of CI send only one mail
        stage('Wait to receive and parse email') {
            environment {
                credentials=credentials("${imap_creds}")
            }
@@ -127,26 +113,16 @@
                          ./tests/jenkins/downstream/poll_email.py \
                          --server '${imap_server}' \
                          --guid ${guid} \
                          --timeout 100 \
                          --filter 'has completed'
                          --timeout 30 \
                          --filter 'is building'
                        """
                    ).trim()

                    try {
                        def m = email =~ /Openshift Master Console: (.*)/
                        openshift_location = m[0][1]
                        echo "Openshift Master Console: ${openshift_location}"
                    
//                        m = email =~ /This cluster has authentication enabled. (.*)/
//                        echo "Cluster authentication:  ${m[0][1]}"
                      
//                        m = email =~ /SSH Access: (.*)/
//                        ssh_location = m[0][1]
//                        echo "SSH Access: ${ssh_location}"
						
//                        m = email =~ /SSH password: (.*)/
//                        ssh_p =​ m[0][1]
//                        echo "SSH password: ${ssh_p}"
                        def m = email =~ /External Hostname<\/TH><TD>(.*)/
                        def mm = email =~ /(.*)<\/TD><\/TR><TR><TH>Internal Hostname/
                        external_host = m[0][1].replaceAll("=","") + mm[0][1].replaceAll(" ","")
                        echo "External-Host='${external_host}'"
                    } catch(Exception ex) {
                        echo "Could not parse email:"
                        echo email
@@ -156,20 +132,13 @@
                }
            }
        }

//        stage('SSH') {
//            steps {
//                withCredentials([
//                    sshUserPrivateKey(
//                        credentialsId: ssh_creds,
//                        keyFileVariable: 'ssh_key',
//                        usernameVariable: 'ssh_username')
//                ]) {
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} w"
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} oc version"
//                }
//            }
//        }
        
        stage ('Wait to complete provision') {
            steps {
                echo "Wait for 30 minutes for deployment to complete"
                sleep 1800 // seconds
            }
        }

        stage('Confirm before retiring') {
            when {
@@ -220,7 +189,7 @@
        }
        stage('Wait for deletion email') {
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'

                withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
@@ -251,6 +220,21 @@
                """
            }

            /* Print ansible logs */
            withCredentials([
                string(credentialsId: ssh_admin_host, variable: 'ssh_admin'),
                sshUserPrivateKey(
                    credentialsId: ssh_creds,
                    keyFileVariable: 'ssh_key',
                    usernameVariable: 'ssh_username')
            ]) {
                sh("""
                    ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_admin} \
                    "bin/logs.sh ${guid}" || true
                """.trim()
                )
            }

            withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
                mail(
                    subject: "${env.JOB_NAME} (${env.BUILD_NUMBER}) failed GUID=${guid}",

 tests/jenkins/openstack-13-implementation.groovy

copy from tests/jenkins/openshift-service-mesh-lab.groovy
copy to tests/jenkins/openstack-13-implementation.groovy
File was copied from tests/jenkins/openshift-service-mesh-lab.groovy
@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -19,21 +19,16 @@

// state variables
def guid=''
def openshift_location = ''
//def ssh_location = ''
//def ssh_p = ''
def external_host = ''


// Catalog items
def choices = [
    'OPENTLC OpenShift Labs / OpenShift Service Mesh Lab',
    'DevOps Team Development / DEV - OpenShift Service Mesh Lab',
    'OPENTLC Cloud Infrastructure Labs / OpenStack 13 Implementation',
].join("\n")

def region_choice = [
    'na_sandboxes_gpte',
    'apac_sandboxes_gpte',
    'emea_sandboxes_gpte',
    'global_gpte',
].join("\n")

pipeline {
@@ -56,7 +51,7 @@
        )
        choice(
            choices: region_choice,
            description: 'Region',
            description: 'Catalog item',
            name: 'region',
        )
    }
@@ -77,6 +72,14 @@
                    def catalog = params.catalog_item.split(' / ')[0].trim()
                    def item = params.catalog_item.split(' / ')[1].trim()
                    def region = params.region.trim()
                    def cfparams = [
                        'check=t',
                        'expiration=7',
                        'runtime=12',
                        "region=${region}",
                        'quotacheck=t',
                    ].join(',').trim()

                    echo "'${catalog}' '${item}'"
                    guid = sh(
                        returnStdout: true,
@@ -85,7 +88,7 @@
                          -c '${catalog}' \
                          -i '${item}' \
                          -G '${cf_group}' \
                          -d 'expiration=6,runtime=8,region=${region}'
                          -d '${cfparams}' \
                        """
                    ).trim()

@@ -93,26 +96,9 @@
                }
            }
        }
        /* Skip this step because sometimes the completed email arrives
         before the 'has started' email */
        stage('Wait for first email') {
            environment {
                credentials=credentials("${imap_creds}")
            }
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                    branch: 'development'


                sh """./tests/jenkins/downstream/poll_email.py \
                    --server '${imap_server}' \
                    --guid ${guid} \
                    --timeout 20 \
                    --filter 'has started'"""
            }
        }
        
        stage('Wait for last email and parse SSH location') {
		
        // This kind of CI send only one mail
        stage('Wait to receive and parse email') {
            environment {
                credentials=credentials("${imap_creds}")
            }
@@ -127,26 +113,16 @@
                          ./tests/jenkins/downstream/poll_email.py \
                          --server '${imap_server}' \
                          --guid ${guid} \
                          --timeout 100 \
                          --filter 'has completed'
                          --timeout 30 \
                          --filter 'is building'
                        """
                    ).trim()

                    try {
                        def m = email =~ /Openshift Master Console: (.*)/
                        openshift_location = m[0][1]
                        echo "Openshift Master Console: ${openshift_location}"
                    
//                        m = email =~ /This cluster has authentication enabled. (.*)/
//                        echo "Cluster authentication:  ${m[0][1]}"
                      
//                        m = email =~ /SSH Access: (.*)/
//                        ssh_location = m[0][1]
//                        echo "SSH Access: ${ssh_location}"
						
//                        m = email =~ /SSH password: (.*)/
//                        ssh_p =​ m[0][1]
//                        echo "SSH password: ${ssh_p}"
                        def m = email =~ /External Hostname<\/TH><TD>(.*)/
                        def mm = email =~ /(.*)<\/TD><\/TR><TR><TH>Internal Hostname/
                        external_host = m[0][1].replaceAll("=","") + mm[0][1].replaceAll(" ","")
                        echo "External-Host='${external_host}'"
                    } catch(Exception ex) {
                        echo "Could not parse email:"
                        echo email
@@ -156,20 +132,13 @@
                }
            }
        }

//        stage('SSH') {
//            steps {
//                withCredentials([
//                    sshUserPrivateKey(
//                        credentialsId: ssh_creds,
//                        keyFileVariable: 'ssh_key',
//                        usernameVariable: 'ssh_username')
//                ]) {
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} w"
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} oc version"
//                }
//            }
//        }
        
        stage ('Wait to complete provision') {
            steps {
                echo "Wait for 30 minutes for deployment to complete"
                sleep 1800 // seconds
            }
        }

        stage('Confirm before retiring') {
            when {
@@ -220,7 +189,7 @@
        }
        stage('Wait for deletion email') {
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'

                withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
@@ -251,6 +220,21 @@
                """
            }

            /* Print ansible logs */
            withCredentials([
                string(credentialsId: ssh_admin_host, variable: 'ssh_admin'),
                sshUserPrivateKey(
                    credentialsId: ssh_creds,
                    keyFileVariable: 'ssh_key',
                    usernameVariable: 'ssh_username')
            ]) {
                sh("""
                    ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_admin} \
                    "bin/logs.sh ${guid}" || true
                """.trim()
                )
            }

            withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
                mail(
                    subject: "${env.JOB_NAME} (${env.BUILD_NUMBER}) failed GUID=${guid}",

 tests/jenkins/rhel-7-implementation-lab.groovy

copy from tests/jenkins/openshift-service-mesh-lab.groovy
copy to tests/jenkins/rhel-7-implementation-lab.groovy
File was copied from tests/jenkins/openshift-service-mesh-lab.groovy
@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -19,21 +19,16 @@

// state variables
def guid=''
def openshift_location = ''
//def ssh_location = ''
//def ssh_p = ''
def external_host = ''


// Catalog items
def choices = [
    'OPENTLC OpenShift Labs / OpenShift Service Mesh Lab',
    'DevOps Team Development / DEV - OpenShift Service Mesh Lab',
    'OPENTLC Datacenter Infrastructure Labs / RHEL 7 Implementation Lab',		
].join("\n")

def region_choice = [
    'na_sandboxes_gpte',
    'apac_sandboxes_gpte',
    'emea_sandboxes_gpte',
    'global_gpte',
].join("\n")

pipeline {
@@ -85,7 +80,7 @@
                          -c '${catalog}' \
                          -i '${item}' \
                          -G '${cf_group}' \
                          -d 'expiration=6,runtime=8,region=${region}'
                          -d 'check=t,expiration=7,runtime=4,region=${region},quotacheck=t'
                        """
                    ).trim()

@@ -93,26 +88,9 @@
                }
            }
        }
        /* Skip this step because sometimes the completed email arrives
         before the 'has started' email */
        stage('Wait for first email') {
            environment {
                credentials=credentials("${imap_creds}")
            }
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                    branch: 'development'


                sh """./tests/jenkins/downstream/poll_email.py \
                    --server '${imap_server}' \
                    --guid ${guid} \
                    --timeout 20 \
                    --filter 'has started'"""
            }
        }
        
        stage('Wait for last email and parse SSH location') {
        // This kind of CI send only one mail
        stage('Wait to receive and parse email') {
            environment {
                credentials=credentials("${imap_creds}")
            }
@@ -127,26 +105,16 @@
                          ./tests/jenkins/downstream/poll_email.py \
                          --server '${imap_server}' \
                          --guid ${guid} \
                          --timeout 100 \
                          --filter 'has completed'
                          --timeout 30 \
                          --filter 'is building'
                        """
                    ).trim()

                    try {
                        def m = email =~ /Openshift Master Console: (.*)/
                        openshift_location = m[0][1]
                        echo "Openshift Master Console: ${openshift_location}"
                    
//                        m = email =~ /This cluster has authentication enabled. (.*)/
//                        echo "Cluster authentication:  ${m[0][1]}"
                      
//                        m = email =~ /SSH Access: (.*)/
//                        ssh_location = m[0][1]
//                        echo "SSH Access: ${ssh_location}"
						
//                        m = email =~ /SSH password: (.*)/
//                        ssh_p =​ m[0][1]
//                        echo "SSH password: ${ssh_p}"
                        def m = email =~ /External Hostname<\/TH><TD>(.*)/
                        def mm = email =~ /(.*)<\/TD><\/TR><TR><TH>Internal Hostname/
                        external_host = m[0][1].replaceAll("=","") + mm[0][1].replaceAll(" ","")
                        echo "External-Host='${external_host}'"
                    } catch(Exception ex) {
                        echo "Could not parse email:"
                        echo email
@@ -156,20 +124,13 @@
                }
            }
        }

//        stage('SSH') {
//            steps {
//                withCredentials([
//                    sshUserPrivateKey(
//                        credentialsId: ssh_creds,
//                        keyFileVariable: 'ssh_key',
//                        usernameVariable: 'ssh_username')
//                ]) {
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} w"
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} oc version"
//                }
//            }
//        }
        
        stage ('Wait to complete provision') {
            steps {
                echo "Wait for 30 minutes for deployment to complete"
                sleep 1800 // seconds
            }
        }

        stage('Confirm before retiring') {
            when {
@@ -251,6 +212,21 @@
                """
            }

            /* Print ansible logs */
            withCredentials([
                string(credentialsId: ssh_admin_host, variable: 'ssh_admin'),
                sshUserPrivateKey(
                    credentialsId: ssh_creds,
                    keyFileVariable: 'ssh_key',
                    usernameVariable: 'ssh_username')
            ]) {
                sh("""
                    ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_admin} \
                    "find deployer_logs -name '*${guid}*log' | xargs cat"
                """.trim()
                )
            }

            withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
                mail(
                    subject: "${env.JOB_NAME} (${env.BUILD_NUMBER}) failed GUID=${guid}",

 tests/jenkins/rhel-8-new-features-for-exp-admins.groovy

copy from tests/jenkins/openshift-service-mesh-lab.groovy
copy to tests/jenkins/rhel-8-new-features-for-exp-admins.groovy
File was copied from tests/jenkins/openshift-service-mesh-lab.groovy
@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -19,21 +19,16 @@

// state variables
def guid=''
def openshift_location = ''
//def ssh_location = ''
//def ssh_p = ''
def external_host = ''


// Catalog items
def choices = [
    'OPENTLC OpenShift Labs / OpenShift Service Mesh Lab',
    'DevOps Team Development / DEV - OpenShift Service Mesh Lab',
    'OPENTLC Datacenter Infrastructure Labs / RHEL 8 New Features For Exp. Admins',		
].join("\n")

def region_choice = [
    'na_sandboxes_gpte',
    'apac_sandboxes_gpte',
    'emea_sandboxes_gpte',
    'global_gpte',
].join("\n")

pipeline {
@@ -85,7 +80,7 @@
                          -c '${catalog}' \
                          -i '${item}' \
                          -G '${cf_group}' \
                          -d 'expiration=6,runtime=8,region=${region}'
                          -d 'check=t,expiration=7,runtime=4,region=${region},quotacheck=t'
                        """
                    ).trim()

@@ -93,26 +88,9 @@
                }
            }
        }
        /* Skip this step because sometimes the completed email arrives
         before the 'has started' email */
        stage('Wait for first email') {
            environment {
                credentials=credentials("${imap_creds}")
            }
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                    branch: 'development'


                sh """./tests/jenkins/downstream/poll_email.py \
                    --server '${imap_server}' \
                    --guid ${guid} \
                    --timeout 20 \
                    --filter 'has started'"""
            }
        }
        
        stage('Wait for last email and parse SSH location') {
        // This kind of CI send only one mail
        stage('Wait to receive and parse email') {
            environment {
                credentials=credentials("${imap_creds}")
            }
@@ -127,26 +105,16 @@
                          ./tests/jenkins/downstream/poll_email.py \
                          --server '${imap_server}' \
                          --guid ${guid} \
                          --timeout 100 \
                          --filter 'has completed'
                          --timeout 30 \
                          --filter 'is building'
                        """
                    ).trim()

                    try {
                        def m = email =~ /Openshift Master Console: (.*)/
                        openshift_location = m[0][1]
                        echo "Openshift Master Console: ${openshift_location}"
                    
//                        m = email =~ /This cluster has authentication enabled. (.*)/
//                        echo "Cluster authentication:  ${m[0][1]}"
                      
//                        m = email =~ /SSH Access: (.*)/
//                        ssh_location = m[0][1]
//                        echo "SSH Access: ${ssh_location}"
						
//                        m = email =~ /SSH password: (.*)/
//                        ssh_p =​ m[0][1]
//                        echo "SSH password: ${ssh_p}"
                        def m = email =~ /External Hostname<\/TH><TD>(.*)/
                        def mm = email =~ /(.*)<\/TD><\/TR><TR><TH>Internal IP Address/
                        external_host = m[0][1].replaceAll("=","") + mm[0][1].replaceAll(" ","")
                        echo "External-Host='${external_host}'"
                    } catch(Exception ex) {
                        echo "Could not parse email:"
                        echo email
@@ -156,20 +124,13 @@
                }
            }
        }

//        stage('SSH') {
//            steps {
//                withCredentials([
//                    sshUserPrivateKey(
//                        credentialsId: ssh_creds,
//                        keyFileVariable: 'ssh_key',
//                        usernameVariable: 'ssh_username')
//                ]) {
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} w"
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} oc version"
//                }
//            }
//        }
        
        stage ('Wait to complete provision') {
            steps {
                echo "Wait for 30 minutes for deployment to complete"
                sleep 1800 // seconds
            }
        }

        stage('Confirm before retiring') {
            when {
@@ -251,6 +212,21 @@
                """
            }

            /* Print ansible logs */
            withCredentials([
                string(credentialsId: ssh_admin_host, variable: 'ssh_admin'),
                sshUserPrivateKey(
                    credentialsId: ssh_creds,
                    keyFileVariable: 'ssh_key',
                    usernameVariable: 'ssh_username')
            ]) {
                sh("""
                    ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_admin} \
                    "find deployer_logs -name '*${guid}*log' | xargs cat"
                """.trim()
                )
            }

            withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
                mail(
                    subject: "${env.JOB_NAME} (${env.BUILD_NUMBER}) failed GUID=${guid}",

 tests/jenkins/rhpam-7-order-it-hardware-demo.groovy

copy from tests/jenkins/openshift-service-mesh-lab.groovy
copy to tests/jenkins/rhpam-7-order-it-hardware-demo.groovy
File was copied from tests/jenkins/openshift-service-mesh-lab.groovy
@@ -2,13 +2,13 @@
// CloudForms
def opentlc_creds = 'b93d2da4-c2b7-45b5-bf3b-ee2c08c6368e'
def opentlc_admin_creds = '73b84287-8feb-478a-b1f2-345fd0a1af47'
def cf_uri = 'https://labs.opentlc.com'
def cf_group = 'opentlc-access-cicd'
def cf_uri = 'https://rhpds.redhat.com'
def cf_group = 'rhpds-access-cicd'
// IMAP
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -20,20 +20,27 @@
// state variables
def guid=''
def openshift_location = ''
//def ssh_location = ''
//def ssh_p = ''


// Catalog items
def choices = [
    'OPENTLC OpenShift Labs / OpenShift Service Mesh Lab',
    'DevOps Team Development / DEV - OpenShift Service Mesh Lab',
    'Middleware Solutions Demos / RHPAM 7: Order IT Hardware Demo',
    'DevOps Shared Cluster Testing / Testing - Shared PAM Order IT Hardware',
    'DevOps Shared Cluster Development / DEV - PAM Order IT Hardware',
].join("\n")

def region_choice = [
    'na_sandboxes_gpte',
    'apac_sandboxes_gpte',
    'emea_sandboxes_gpte',
    'global_gpte',
].join("\n")

def nodes_choice = [
    '3',
    '1',
    '2',    
    '4',
    '5',
    '6',
    '7',
    '8',
].join("\n")

pipeline {
@@ -56,8 +63,13 @@
        )
        choice(
            choices: region_choice,
            description: 'Region',
            description: 'Catalog item',
            name: 'region',
        )
        choice(
            choices: nodes_choice,
            description: 'Number of Nodes',
            name: 'nodes',
        )
    }

@@ -77,6 +89,15 @@
                    def catalog = params.catalog_item.split(' / ')[0].trim()
                    def item = params.catalog_item.split(' / ')[1].trim()
                    def region = params.region.trim()
                    def nodes = params.nodes.trim()
                    def cfparams = [
                        'check=t',
                        'expiration=14',
                        'runtime=8',
                        'quotacheck=t',
                        "region=${region}",
                        "nodes=${nodes}",
                    ].join(',').trim()
                    echo "'${catalog}' '${item}'"
                    guid = sh(
                        returnStdout: true,
@@ -85,7 +106,7 @@
                          -c '${catalog}' \
                          -i '${item}' \
                          -G '${cf_group}' \
                          -d 'expiration=6,runtime=8,region=${region}'
                          -d '${cfparams}' \
                        """
                    ).trim()

@@ -93,31 +114,30 @@
                }
            }
        }
        /* Skip this step because sometimes the completed email arrives
         before the 'has started' email */
        // Skip this step because sometimes the completed email arrives
        // before the 'has started' email
        stage('Wait for first email') {
            environment {
                credentials=credentials("${imap_creds}")
            }
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'


                sh """./tests/jenkins/downstream/poll_email.py \
                    --server '${imap_server}' \
                    --guid ${guid} \
                    --timeout 20 \
                    --timeout 30 \
                    --filter 'has started'"""
            }
        }
        
        stage('Wait for last email and parse SSH location') {

        stage('Wait for last email and parse OpenShift and App location') {
            environment {
                credentials=credentials("${imap_creds}")
            }
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'

                script {
@@ -127,26 +147,15 @@
                          ./tests/jenkins/downstream/poll_email.py \
                          --server '${imap_server}' \
                          --guid ${guid} \
                          --timeout 100 \
                          --timeout 150 \
                          --filter 'has completed'
                        """
                    ).trim()

                    try {
                        def m = email =~ /Openshift Master Console: (.*)/
                        openshift_location = m[0][1]
                        echo "Openshift Master Console: ${openshift_location}"
                    
//                        m = email =~ /This cluster has authentication enabled. (.*)/
//                        echo "Cluster authentication:  ${m[0][1]}"
                      
//                        m = email =~ /SSH Access: (.*)/
//                        ssh_location = m[0][1]
//                        echo "SSH Access: ${ssh_location}"
						
//                        m = email =~ /SSH password: (.*)/
//                        ssh_p =​ m[0][1]
//                        echo "SSH password: ${ssh_p}"
                        def m = email =~ /Connect to the shared cluster by pointing your web browser to <a href="(https:\/\/master\.[^"]+)/
                        openshift_location = m[0][1]
                        echo "openshift_location = '${openshift_location}'"
                    } catch(Exception ex) {
                        echo "Could not parse email:"
                        echo email
@@ -156,20 +165,6 @@
                }
            }
        }

//        stage('SSH') {
//            steps {
//                withCredentials([
//                    sshUserPrivateKey(
//                        credentialsId: ssh_creds,
//                        keyFileVariable: 'ssh_key',
//                        usernameVariable: 'ssh_username')
//                ]) {
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} w"
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} oc version"
//                }
//            }
//        }

        stage('Confirm before retiring') {
            when {
@@ -220,7 +215,7 @@
        }
        stage('Wait for deletion email') {
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'

                withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
@@ -249,6 +244,21 @@
                export DEBUG=true
                ./opentlc/delete_svc_guid.sh '${guid}'
                """
            }

            /* Print ansible logs */
            withCredentials([
                string(credentialsId: ssh_admin_host, variable: 'ssh_admin'),
                sshUserPrivateKey(
                    credentialsId: ssh_creds,
                    keyFileVariable: 'ssh_key',
                    usernameVariable: 'ssh_username')
            ]) {
                sh("""
                    ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_admin} \
                    "bin/logs.sh ${guid}" || true
                """.trim()
                )
            }

            withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
@@ -282,4 +292,4 @@
            }
        }
    }
}
}

 tests/jenkins/rhv-4.0-foundations-lab.groovy

copy from tests/jenkins/openshift-service-mesh-lab.groovy
copy to tests/jenkins/rhv-4.0-foundations-lab.groovy
File was copied from tests/jenkins/openshift-service-mesh-lab.groovy
@@ -8,7 +8,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -19,21 +19,16 @@

// state variables
def guid=''
def openshift_location = ''
//def ssh_location = ''
//def ssh_p = ''
def external_host = ''


// Catalog items
def choices = [
    'OPENTLC OpenShift Labs / OpenShift Service Mesh Lab',
    'DevOps Team Development / DEV - OpenShift Service Mesh Lab',
    'OPENTLC Datacenter Infrastructure Labs / RHV 4.0 Foundations Lab',		
].join("\n")

def region_choice = [
    'na_sandboxes_gpte',
    'apac_sandboxes_gpte',
    'emea_sandboxes_gpte',
    'global_gpte',
].join("\n")

pipeline {
@@ -85,7 +80,7 @@
                          -c '${catalog}' \
                          -i '${item}' \
                          -G '${cf_group}' \
                          -d 'expiration=6,runtime=8,region=${region}'
                          -d 'check=t,expiration=7,runtime=4,region=${region},quotacheck=t'
                        """
                    ).trim()

@@ -93,26 +88,9 @@
                }
            }
        }
        /* Skip this step because sometimes the completed email arrives
         before the 'has started' email */
        stage('Wait for first email') {
            environment {
                credentials=credentials("${imap_creds}")
            }
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                    branch: 'development'


                sh """./tests/jenkins/downstream/poll_email.py \
                    --server '${imap_server}' \
                    --guid ${guid} \
                    --timeout 20 \
                    --filter 'has started'"""
            }
        }
        
        stage('Wait for last email and parse SSH location') {
        // This kind of CI send only one mail
        stage('Wait to receive and parse email') {
            environment {
                credentials=credentials("${imap_creds}")
            }
@@ -127,26 +105,16 @@
                          ./tests/jenkins/downstream/poll_email.py \
                          --server '${imap_server}' \
                          --guid ${guid} \
                          --timeout 100 \
                          --filter 'has completed'
                          --timeout 30 \
                          --filter 'is building'
                        """
                    ).trim()

                    try {
                        def m = email =~ /Openshift Master Console: (.*)/
                        openshift_location = m[0][1]
                        echo "Openshift Master Console: ${openshift_location}"
                    
//                        m = email =~ /This cluster has authentication enabled. (.*)/
//                        echo "Cluster authentication:  ${m[0][1]}"
                      
//                        m = email =~ /SSH Access: (.*)/
//                        ssh_location = m[0][1]
//                        echo "SSH Access: ${ssh_location}"
						
//                        m = email =~ /SSH password: (.*)/
//                        ssh_p =​ m[0][1]
//                        echo "SSH password: ${ssh_p}"
                        def m = email =~ /External Hostname<\/TH><TD>(.*)/
                        def mm = email =~ /(.*)<\/TD><\/TR><TR><TH>Internal Hostname/
                        external_host = m[0][1].replaceAll("=","") + mm[0][1].replaceAll(" ","")
                        echo "External-Host='${external_host}'"
                    } catch(Exception ex) {
                        echo "Could not parse email:"
                        echo email
@@ -156,20 +124,13 @@
                }
            }
        }

//        stage('SSH') {
//            steps {
//                withCredentials([
//                    sshUserPrivateKey(
//                        credentialsId: ssh_creds,
//                        keyFileVariable: 'ssh_key',
//                        usernameVariable: 'ssh_username')
//                ]) {
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} w"
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} oc version"
//                }
//            }
//        }
        
        stage ('Wait to complete provision') {
            steps {
                echo "Wait for 30 minutes for deployment to complete"
                sleep 1800 // seconds
            }
        }

        stage('Confirm before retiring') {
            when {
@@ -251,6 +212,21 @@
                """
            }

            /* Print ansible logs */
            withCredentials([
                string(credentialsId: ssh_admin_host, variable: 'ssh_admin'),
                sshUserPrivateKey(
                    credentialsId: ssh_creds,
                    keyFileVariable: 'ssh_key',
                    usernameVariable: 'ssh_username')
            ]) {
                sh("""
                    ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_admin} \
                    "find deployer_logs -name '*${guid}*log' | xargs cat"
                """.trim()
                )
            }

            withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
                mail(
                    subject: "${env.JOB_NAME} (${env.BUILD_NUMBER}) failed GUID=${guid}",

 tests/jenkins/shared_developer_env_RHPDS.groovy

@@ -7,7 +7,7 @@
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// state variables

 tests/jenkins/the-definitive-rhel-8-hands-on-lab.groovy

copy from tests/jenkins/openshift-service-mesh-lab.groovy
copy to tests/jenkins/the-definitive-rhel-8-hands-on-lab.groovy
File was copied from tests/jenkins/openshift-service-mesh-lab.groovy
@@ -2,13 +2,13 @@
// CloudForms
def opentlc_creds = 'b93d2da4-c2b7-45b5-bf3b-ee2c08c6368e'
def opentlc_admin_creds = '73b84287-8feb-478a-b1f2-345fd0a1af47'
def cf_uri = 'https://labs.opentlc.com'
def cf_group = 'opentlc-access-cicd'
def cf_uri = 'https://rhpds.redhat.com'
def cf_group = 'rhpds-access-cicd'
// IMAP
def imap_creds = 'd8762f05-ca66-4364-adf2-bc3ce1dca16c'
def imap_server = 'imap.gmail.com'
// Notifications
def notification_email = 'gucore@redhat.com'
def notification_email = 'gpteinfrasev3@redhat.com'
def rocketchat_hook = '5d28935e-f7ca-4b11-8b8e-d7a7161a013a'

// SSH key
@@ -19,21 +19,26 @@

// state variables
def guid=''
def openshift_location = ''
//def ssh_location = ''
//def ssh_p = ''

def external_host = ''

// Catalog items
def choices = [
    'OPENTLC OpenShift Labs / OpenShift Service Mesh Lab',
    'DevOps Team Development / DEV - OpenShift Service Mesh Lab',
    'Red Hat Summit 2019 / The Definitive RHEL 8 Hands-On Lab',
].join("\n")

def region_choice = [
    'na_sandboxes_gpte',
    'apac_sandboxes_gpte',
    'emea_sandboxes_gpte',
    'rhpds1',
].join("\n")

def nodes_choice = [
    '3',
    '1',
    '2',    
    '4',
    '5',
    '6',
    '7',
    '8',
].join("\n")

pipeline {
@@ -56,8 +61,13 @@
        )
        choice(
            choices: region_choice,
            description: 'Region',
            description: 'Catalog item',
            name: 'region',
        )
        choice(
            choices: nodes_choice,
            description: 'Number of Nodes',
            name: 'nodes',
        )
    }

@@ -77,6 +87,18 @@
                    def catalog = params.catalog_item.split(' / ')[0].trim()
                    def item = params.catalog_item.split(' / ')[1].trim()
                    def region = params.region.trim()
                    def nodes = params.nodes.trim()
                    def cfparams = [
                        'check=t',
                        'check2=t',
                        "region=${region}",
                        'expiration=14',
                        'runtime=8',
                        'city=jenkins',
                        'salesforce=gptejen',
                        'quotacheck=t',
                        "nodes=${nodes}",
                    ].join(',').trim()
                    echo "'${catalog}' '${item}'"
                    guid = sh(
                        returnStdout: true,
@@ -85,7 +107,7 @@
                          -c '${catalog}' \
                          -i '${item}' \
                          -G '${cf_group}' \
                          -d 'expiration=6,runtime=8,region=${region}'
                          -d '${cfparams}' \
                        """
                    ).trim()

@@ -93,26 +115,9 @@
                }
            }
        }
        /* Skip this step because sometimes the completed email arrives
         before the 'has started' email */
        stage('Wait for first email') {
            environment {
                credentials=credentials("${imap_creds}")
            }
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                    branch: 'development'


                sh """./tests/jenkins/downstream/poll_email.py \
                    --server '${imap_server}' \
                    --guid ${guid} \
                    --timeout 20 \
                    --filter 'has started'"""
            }
        }
        
        stage('Wait for last email and parse SSH location') {
        // This kind of CI send only one mail
        stage('Wait to receive and parse email') {
            environment {
                credentials=credentials("${imap_creds}")
            }
@@ -127,26 +132,15 @@
                          ./tests/jenkins/downstream/poll_email.py \
                          --server '${imap_server}' \
                          --guid ${guid} \
                          --timeout 100 \
                          --filter 'has completed'
                          --timeout 30 \
                          --filter 'is building'
                        """
                    ).trim()

                    try {
                        def m = email =~ /Openshift Master Console: (.*)/
                        openshift_location = m[0][1]
                        echo "Openshift Master Console: ${openshift_location}"
                    
//                        m = email =~ /This cluster has authentication enabled. (.*)/
//                        echo "Cluster authentication:  ${m[0][1]}"
                      
//                        m = email =~ /SSH Access: (.*)/
//                        ssh_location = m[0][1]
//                        echo "SSH Access: ${ssh_location}"
						
//                        m = email =~ /SSH password: (.*)/
//                        ssh_p =​ m[0][1]
//                        echo "SSH password: ${ssh_p}"
                        def m = email =~ /External Hostname<\/TH><TD>(.*) </
                        external_host = m[0][1]
                        echo "External-Host='${external_host}'"
                    } catch(Exception ex) {
                        echo "Could not parse email:"
                        echo email
@@ -156,20 +150,13 @@
                }
            }
        }

//        stage('SSH') {
//            steps {
//                withCredentials([
//                    sshUserPrivateKey(
//                        credentialsId: ssh_creds,
//                        keyFileVariable: 'ssh_key',
//                        usernameVariable: 'ssh_username')
//                ]) {
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} w"
//                    sh "ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_location} oc version"
//                }
//            }
//        }
        
        stage ('Wait to complete provision') {
            steps {
                echo "Wait for 30 minutes for deployment to complete"
                sleep 1800 // seconds
            }
        }

        stage('Confirm before retiring') {
            when {
@@ -220,7 +207,7 @@
        }
        stage('Wait for deletion email') {
            steps {
                git url: 'https://github.com/redhat-cop/agnosticd',
                git url: 'https://github.com/sborenst/ansible_agnostic_deployer',
                    branch: 'development'

                withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
@@ -249,6 +236,21 @@
                export DEBUG=true
                ./opentlc/delete_svc_guid.sh '${guid}'
                """
            }

            /* Print ansible logs */
            withCredentials([
                string(credentialsId: ssh_admin_host, variable: 'ssh_admin'),
                sshUserPrivateKey(
                    credentialsId: ssh_creds,
                    keyFileVariable: 'ssh_key',
                    usernameVariable: 'ssh_username')
            ]) {
                sh("""
                    ssh -o StrictHostKeyChecking=no -i ${ssh_key} ${ssh_admin} \
                    "bin/logs.sh ${guid}" || true
                """.trim()
                )
            }

            withCredentials([usernameColonPassword(credentialsId: imap_creds, variable: 'credentials')]) {
@@ -282,4 +284,4 @@
            }
        }
    }
}
}