RedHatTraining/rht-labs-docs.git

wmcdonald404

2018-06-08 853f0c525a5523182c0e6d4f9289b059b388c86e

commit \| author \| age
ed818c	1	# The Non-Functionals Strike back
64c32e	2	> In this exercise we explore the non-functional side of testing.
6b9713	3
D	4	![death-star-vent](../images/exercise5/death-star-vent.jpeg)
542e51	5
D	6	## Exercise Intro
5e7a31	7	Non-functional testing provides valuable insights into code quality and application performance. Often overlooked but usually one of the most essential types of testing, non-functional testing types can include, but are not limited to
542e51	8	- Performance Testing
D	9	- Security testing
	10	- Static Code analysis
	11	- Vulnerability scanning
	12
5e7a31	13	There are many tools out there for supporting these testing types but often they are left to the end of a delivery. Many traditional projects will leave Performance testing or security sign off to a few weeks before Go Live. This raises the question of what do we do if things do not pass these tests? Do we hold off the Go Live or accept the risk? In most cases we can learn earlier if things will be show stoppers and more importantly we can automate them.
542e51	14
853f0c	15	For example; imagine a developer called `Sam` has checked in some poorly performing function into an API that spikes its response time. Let's call this `Sam Code`. This `Sam Code` may not be caught by our Unit Tests but could have very big impact on application usability. Before building code on top this and it becoming more of an issue at the end of a project; we could code and capture metrics around API response time and track these over time and hopefully spot regressions earlier.
542e51	16
cc6b97	17	Another one of the age old questions is "How do we know we're testing enough?". Well the simple answer is you can never do enough testing! But how do we know we are testing the right things? Code coverage metrics can be run on our application while running the tests. They can identify what files are being tested and a line by line of how many times a test executes against a block of code. Reporting these metrics in our pipeline gives a greater handle on the quality of our testing.
542e51	18
cc6b97	19	Static code analysis can provide great insights into the quality of the code we've written. By analysing code without executing it bugs and common gotchas can be identified. Code complexity can be assessed using tooling such as SonarQube. Linting of code is a useful thing for non compiled languages such as JavaScript and can be executed in a build to provide some feedback.
542e51	20
D	21	Integrating these tools into the developer workflow can greatly improve quality and readability of code. Putting them in the pipeline ensures they are executed - removing the _"It works on my machine"_ view some may take.
	22
	23	#### Why run non functional tests?
	24	- Early identify common code gotchas and pitfalls
	25	- Tighten our feedback loop by executing non functional tests earlier in a project lifecycle
	26	- Improve Code readability with agreeing a coding standard and then codifying it.
	27	- Improve trust in the code quality and bring together security teams with dev teams earlier
	28	- They can be automated; so why not run them as part of a build!
	29
5a16fd	30	_____
D	31
	32	## Learning Outcomes
	33	As a learner you will be able to
ea88bb	34	- Create additional Jenkins stages to scan for security vulnerabilities in the Apps
ed818c	35	- Assess test quality by producing coverage reports as part of a build
ea88bb	36	- Improve code readability with linting
D	37	- Do some light performance testing to monitor throughput of APIs
5a16fd	38
D	39	## Tools and Frameworks
867471	40	> Below is a collection of the new frameworks and tools that will be used in this exercise
5a16fd	41
ea88bb	42	1. [eslint](https://eslint.org/) - ESLint is an open source JavaScript linting utility originally created by Nicholas C. Zakas in June 2013. Code linting is a type of static analysis that is frequently used to find problematic patterns or code that doesn’t adhere to certain style guidelines. There are code linters for most programming languages, and compilers sometimes incorporate linting into the compilation process.
cc6b97	43	1. [Zed Attack Proxy](https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project) - The OWASP Zed Attack Proxy (ZAP) is one of the world’s most popular free security tools and is actively maintained by hundreds of international volunteers*. It can help you automatically find security vulnerabilities in your web applications while you are developing and testing your applications. Its also a great tool for experienced pentesters to use for manual security testing.
D	44	1. [Arachni Crawler](http://www.arachni-scanner.com/) - Arachni is a feature-full, modular, high-performance Ruby framework aimed towards helping penetration testers and administrators evaluate the security of modern web applications. It is free, with its source code public and available for review. It is versatile enough to cover a great deal of use cases, ranging from a simple command line scanner utility, to a global high performance grid of scanners, to a Ruby library allowing for scripted audits, to a multi-user multi-scan web collaboration platform. In addition, its simple REST API makes integration a cinch.
542e51	45	1. [stryker](http://stryker-mutator.io/) - Mutation testing! What is it? Bugs, or mutants, are automatically inserted into your production code. Your tests are run for each mutant. If your tests fail then the mutant is killed. If your tests passed, the mutant survived. The higher the percentage of mutants killed, the more effective your tests are. It's really that simple.
5a16fd	46
D	47	## Big Picture
	48	This exercise begins cluster containing blah blah
	49
	50	_____
	51
	52	## 10,000 Ft View
ed818c	53	> This lesson will use the Exercise 4's Zap Slave and Arachni scanner to improve the pipeline. Linting will be included in the build and code coverage too.
5a16fd	54
ea88bb	55	2. Add a parallel stage after the e2e tests on the front end to run OWASP Zap and Arachni against the deployed apps.
5a16fd	56
cc6b97	57	2. Add Code Coverage reporting to the build for gaining greater insight into test improvements.
ea88bb	58
542e51	59	2. Add `npm run lint` to the Frontend and report the result using the Checkstyle Plugin in Jenkins.
ea88bb	60
D	61	2. Create a new Jenkins job to run some light performance testing against the API layer using the perf tests tasks.
5a16fd	62
D	63	## Step by Step Instructions
ea88bb	64	> This is a well structured guide with references to exact filenames and indications as to what should be done.
5a16fd	65
5e7a31	66	### Part 1 - Add Security scanning to the pipeline
e33cae	67	> _In this exercise the first of our non-functional testing is explored in the form of some security scanning. We will add the scans to our Jenkinsfile and have them run as new stages_
5a16fd	68
e33cae	69	2. Open the `todolist-fe` application's `Jenkinsfile` in your favourite editor. The file is stored in the root of the project.
D	70
853f0c	71	2. The file is laid out with a collection of stages that correspond to each part of our build as seen below. We will create a new stage to execute in parallel.
c452bb	72	![stages](../images/exercise5/stages.png)
e33cae	73
5e7a31	74	2. Create a new Parallel Stage called `security scanning` underneath the `stage("e2e test") { }` section as shown below. The contents of the `e2e test` have been removed for simplicity.
e33cae	75	```groovy
D	76	stage("e2e test") {
	77	// ... stuff in here ....
	78	}
	79	stage("security scanning") {
	80	parallel {
	81	stage('OWASP Scan') {
	82
	83	}
	84	stage('Arachni Scan') {
	85
	86	}
	87	}
	88	}
5a16fd	89	```
e33cae	90
ed818c	91	2. Let's start filling out the configuration for the OWASP Zap scan first. We will set the label to our slave created in previous exercise and a `when` condition of the master or develop branch.
e33cae	92	```groovy
D	93	stage('OWASP Scan') {
	94	agent {
	95	node {
	96	label "jenkins-slave-zap"
	97	}
	98	}
	99	when {
	100	expression { GIT_BRANCH ==~ /(.master\|.develop)/ }
	101	}
	102	}
5a16fd	103	```
e33cae	104
ac41ef	105	2. Add a `step` with a `sh` command to run the tool by passing in the URL of the app we're going to test.
e33cae	106	```groovy
D	107	stage('OWASP Scan') {
ac41ef	108	agent {
D	109	node {
	110	label "jenkins-slave-zap"
	111	}
e33cae	112	}
ac41ef	113	when {
D	114	expression { GIT_BRANCH ==~ /(.master\|.develop)/ }
	115	}
	116	steps {
	117	sh '''
	118	/zap/zap-baseline.py -r index.html -t ${E2E_TEST_ROUTE} \|\| return_code=$?
	119	echo "exit value was - " $return_code
	120	'''
	121	}
e33cae	122	}
D	123	```
	124
853f0c	125	2. Finally add the reporting for Jenkins in `post` hook of our Declarative Pipeline. This is to report the findings of the scan in Jenkins as an HTML report.
e33cae	126	```groovy
D	127	stage('OWASP Scan') {
	128	agent {
	129	node {
	130	label "jenkins-slave-zap"
	131	}
	132	}
	133	when {
	134	expression { GIT_BRANCH ==~ /(.master\|.develop)/ }
	135	}
	136	steps {
	137	sh '''
ac41ef	138	/zap/zap-baseline.py -r index.html -t http://${E2E_TEST_ROUTE} \|\| return_code=$?
D	139	echo "exit value was - " $return_code
e33cae	140	'''
D	141	}
	142	post {
	143	always {
	144	// publish html
	145	publishHTML target: [
	146	allowMissing: false,
	147	alwaysLinkToLastBuild: false,
	148	keepAll: true,
	149	reportDir: '/zap/wrk',
	150	reportFiles: 'index.html',
	151	reportName: 'Zap Branniscan'
	152	]
	153	}
	154	}
	155	}
	156	```
	157
ed818c	158	2. Let's add our Arachni Scan to the second part of the parallel block. The main difference between these sections is Jenkins will report an XML report too for failing the build accordingly. Below is the snippet for the Arachni scanning.
e33cae	159	```groovy
D	160	stage('Arachni Scan') {
	161	agent {
	162	node {
	163	label "jenkins-slave-arachni"
	164	}
	165	}
	166	when {
	167	expression { GIT_BRANCH ==~ /(.master\|.develop)/ }
	168	}
	169	steps {
	170	sh '''
ccc1ac	171	/arachni/bin/arachni http://${E2E_TEST_ROUTE} --report-save-path=arachni-report.afr
e33cae	172	/arachni/bin/arachni_reporter arachni-report.afr --reporter=xunit:outfile=report.xml --reporter=html:outfile=web-report.zip
D	173	unzip web-report.zip -d arachni-web-report
	174	'''
	175	}
	176	post {
	177	always {
	178	junit 'report.xml'
	179	publishHTML target: [
	180	allowMissing: false,
	181	alwaysLinkToLastBuild: false,
	182	keepAll: true,
	183	reportDir: 'arachni-web-report',
	184	reportFiles: 'index.html',
	185	reportName: 'Arachni Web Crawl'
	186	]
	187	}
	188	}
	189	}
	190	```
	191
db90ef	192	2. With this config in place run a build on Jenkins. To do this; commit your code (from your terminal):
D	193	```bash
58480f	194	git add .
D	195	```
	196	```bash
	197	git commit -m "ADD - security scanning tools to pipeline"
	198	```
	199	```bash
	200	git push
db90ef	201	```
D	202
ac41ef	203	2. Check out the Blue Ocean Jenkins view for how the parallel stage is viewed!
D	204	![jenkins-parallel](../images/exercise5/jenkins-parallel.png)
	205
db90ef	206	2. Once the Jobs have completed; navigate to the Jobs status and see the scores. You can find the graphs and test reports on overview of the Job. Explore the results!
c452bb	207	![report-arachni](../images/exercise5/report-arachni.png)
D	208	![jenkins-arachni](../images/exercise5/jenkins-arachni.png)
	209
	210	<p class="tip">
ac41ef	211	NOTE - your build may have failed because of the a security failure but the reports should still be generated, it is OK to proceed with the next exercise!
c452bb	212	</p>
5a16fd	213
a82977	214	2. TODO - add solution for failing Security scans!
D	215
ea88bb	216	### Part 2 - Add Code Coverage & Linting to the pipeline
887bbe	217	> _Let's continue to enhance our pipeline with some non-functional testing. Static code analysis and testing coverage reports can provide a useful indicator on code quality and testing distribution_
5a16fd	218
887bbe	219	3. Coverage reports are already being generated as part of the tests. We can have Jenkins produce a HTML report showing in detail where our testing is lacking. Open the `todolist-fe` in your favourite editor.
D	220
5e7a31	221	3. Open the `Jenkinsfile` in the root of the project; move to the `stage("node-build"){ ... }` section. In the `post` section add a block for producing a `HTML` report as part of our builds. This is all that is needed for Jenkins to report the coverage stats.
887bbe	222	```groovy
D	223	// Post can be used both on individual stages and for the entire build.
	224	post {
	225	always {
	226	archive "**"
	227	junit 'test-report.xml'
	228	// publish html
	229	publishHTML target: [
	230	allowMissing: false,
	231	alwaysLinkToLastBuild: false,
	232	keepAll: true,
	233	reportDir: 'reports/coverage',
	234	reportFiles: 'index.html',
	235	reportName: 'Code Coverage'
	236	]
	237	}
	238	```
	239
5e7a31	240	3. To get the linting working; we will add a new step to our `stage("node-build"){ }` section to lint the JavaScript code. Continuing in the `Jenkinsfile`, After the `npm install`; add a command to run the linting.
887bbe	241	```groovy
D	242	echo '### Install deps ###'
	243	sh 'npm install'
	244	echo '### Running linting ###'
ac41ef	245	sh 'npm run lint'
887bbe	246	```
D	247
	248	3. Save the `Jenkinsfile` and commit it to trigger a build with some more enhancements.
	249	```bash
58480f	250	git add .
D	251	```
	252	```bash
	253	git commit -m "ADD - linting and coverage to the pipeline"
	254	```
	255	```bash
	256	git push
887bbe	257	```
D	258
ac41ef	259	3. When the build has completed; fix the linting errors if there are any and commit your changes. Look in Jenkins log for what the issue might be....
D	260	![linting-issue](../images/exercise5/linting-issue.png)
	261
5e7a31	262	3. To view the coverage graph; go to the job's build page and open the `Code Coverage` report from the nav bar on the side.
c26359	263	<p class="tip">
A	264	NOTE - Sometimes this won't display on the `yourjenkins.com/job/todolist-fe/job/branch/` sidebar, click on an individual build in the build history and it should appear on the side navbar.
	265	</p>
ac41ef	266	![report-location](../images/exercise5/report-location.png)
D	267
5e7a31	268	3. Open the report to drill down into detail of where testing coverage could be improved!
64c32e	269	![report-coverage](../images/exercise5/report-coverage.png)
D	270	<p class="tip">
5e7a31	271	NOTE - a good practice for teams is to try and increase the code coverage metrics over the life of a project. Teams will often start low and use practices such as retrospective to increase the quality at specific times.
64c32e	272	</p>
D	273
887bbe	274	3. (Optional Step) - Install the Checkstyle plugin; and add `checkstyle pattern: 'eslint-report.xml'` below the `publishHTML` block to add reporting to Jenkins!
D	275
64c32e	276	### Part 3 - Nightly light performance testing
5e7a31	277	> _In this exercise, we will execute the light performance tasks in our API to collect data about throughput time in hopes if the API ever has some `Sam` quality code checked in, we will spot it_
887bbe	278
5e7a31	279	An arbitrary value for the APIs to respond in has been chosen. It is set in the `todolist-api/tasks/perf-test.js` file. In this exercise we will get Jenkins to execute the tests and fail based on the score set there!
64c32e	280
a82977	281	4. Create a new Item on Jenkins, `nightly-perf-test` and make it a freestyle job.
D	282	![new-job](../images/exercise5/new-job.png)
	283
	284	4. Set the `label` on `Restrict where this project can be run` to `jenkins-slave-npm` one used by the build jobs previously.
	285	![slave-label](../images/exercise5/slave-label.png)
64c32e	286
D	287	4. In the SCM section; set the project to use the `todolist-api` git project. Set the credentials accordingly.
a82977	288	![git-settings](../images/exercise5/git-settings.png)
D	289
	290	4. Set the build to execute each night; for example 0300 in the morning. Hit `Build periodically` on the Build Triggers section and set it to `H 3 * * *`.
	291	![build-schedule](../images/exercise5/build-schedule.png)
	292
	293	4. Set the `Color ANSI Console Output` on the Build Environment section.
64c32e	294
45eb81	295	4. Click `add build step` and select `execute shell` and add the following to it, replacing `<YOUR_NAME>` as expected. We will just test the `create` and `show` API for the moment. We are grabbing the response code of the perf-test to keep Jenkins running both shells steps and then exiting with whichever fails:
64c32e	296	```bash
45eb81	297	export E2E_TEST_ROUTE=todolist-api-<YOUR_NAME>-dev.apps.lader.rht-labs.com
64c32e	298	npm install
D	299	set +e
	300	npm run perf-test:create
	301	rc1=$?
	302	npm run perf-test:show
	303	rc2=$?
	304	set -e
	305	exit $(($rc1 \| $rc2))
	306	```
	307
	308	4. On the Post Build actions section we will plot the data from the perf tests in Jenkins. Add a `Post-build Action > Plot Build Data`.
	309
5e7a31	310	4. On the new dialog, name the Plot group e.g. `benchmark-tests` and add `create-api` as the Plot title. Set the `Number of Builds to Include` to a large number like `100`. Set the Data Series file to be `reports/server/perf/create-perf-score.csv` and mark the `Load data from CSV field` checkbox. Apply those changes
64c32e	311	![jenkins-plot](../images/exercise5/jenkins-plot.png)
D	312
c26359	313	4. Hit `Add Plot` to add another. Set Plot group to `benchmark-tests` again but this time setting the Plot title to `show-api`. Set the Data Series file to be `reports/server/perf/show-perf-score.csv` and mark the `Load data from CSV` radio button. Save those changes and run the job (Job could take a while to execute!).
64c32e	314
D	315	4. Run it a few times to start to generate the data points on the plot. The `bench-tests` plot is available on the job's homepage
	316	![result-plot](../images/exercise5/result-plot.png)
5a16fd	317
D	318	_____
	319
	320	## Extension Tasks
	321	> _Ideas for go-getters. Advanced topic for doers to get on with if they finish early. These will usually not have a solution and are provided for additional scope._
	322
887bbe	323	- Enhance the `todolist-api` with the security scanning tools as you've done for the `todolist-api`
D	324	- Enhance the `todolist-api` with the coverage reporting as you've done for `todolist-api`
ea88bb	325	- Add Black Duck or other package scanning tooling for our NodeJS app
D	326	- Add Container Vulnerability scanning tooling to the pipeline
64c32e	327	- Add `Stryker` to create mutants and do additional non functional testing of the App
887bbe	328	- Add the Checkstyle plugin to Jenkins for reporting scores
5a16fd	329
D	330	## Additional Reading
	331	> List of links or other reading that might be of use / reference for the exercise
	332
4f0295	333	## Slide Links
RH	334
	335	- [Intro](https://docs.google.com/presentation/d/1YQ0hUV3o7DW8O40SiI-BQZXCOSVeQGjo2iTxCL2GZfk/)
	336	- [Wrap-up](https://docs.google.com/presentation/d/102hRHDlC9PUIsMs3m1fZy8QUaB5UKzBlhBPdehRWw38/)
5e7a31	337	- [All Material](https://drive.google.com/drive/folders/1seT0V3ABHNonvtFvORNt836NgSeYPuWW)