Security and Compliance Testing

Security and compliance testing identifies vulnerabilities, policy violations, and licence risks in application code before deployment to production environments. These automated checks integrate into continuous integration pipelines to provide rapid feedback and enforce security gates that prevent flawed code from progressing through deployment stages.

The testing types covered here operate at different points in the development lifecycle. Static analysis examines source code without execution, catching vulnerabilities during development. Dynamic analysis tests running applications, discovering issues that only manifest at runtime. Dependency scanning evaluates third-party libraries for known vulnerabilities. Secret detection prevents credentials from entering version control. Compliance scanning verifies licence compatibility and policy adherence.

Prerequisites

SAST (Static Application Security Testing): Analysis of source code, bytecode, or binary code to identify security vulnerabilities without executing the application.
DAST (Dynamic Application Security Testing): Testing of running applications by simulating attacks against exposed interfaces to discover runtime vulnerabilities.
SCA (Software Composition Analysis): Identification and evaluation of third-party components, libraries, and dependencies for known vulnerabilities and licence compliance.
CVE (Common Vulnerabilities and Exposures): Standardised identifier for publicly known security vulnerabilities, formatted as CVE-YEAR-NUMBER.

Before configuring security testing, verify the following requirements are satisfied.

Requirement	Specification	Verification command
Git repository access	Read access to target repositories	`git clone --depth 1 <repo-url>`
CI runner resources	Minimum 4GB RAM, 2 CPU cores	Check runner configuration
Network access	Outbound HTTPS to vulnerability databases	`curl -I https://nvd.nist.gov`
Container runtime	Docker 20.10+ or Podman 4.0+	`docker --version`
Target application	Deployable to test environment	Application-specific

Install the core open source tooling. These tools provide comprehensive coverage without licensing costs.

# Semgrep for SAST
pip install semgrep --break-system-packages

# Verify installation
semgrep --version
# Expected: semgrep 1.45.0 or higher

# Trivy for container and dependency scanning
curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh | sh -s -- -b /usr/local/bin v0.48.0

# Verify installation
trivy --version
# Expected: Version: 0.48.0

# Gitleaks for secret detection
curl -sSfL https://github.com/gitleaks/gitleaks/releases/download/v8.18.0/gitleaks_8.18.0_linux_x64.tar.gz | tar xz -C /usr/local/bin gitleaks

# Verify installation
gitleaks version
# Expected: gitleaks version 8.18.0

# OWASP ZAP for DAST (Docker-based)
docker pull ghcr.io/zaproxy/zaproxy:stable

Create a baseline configuration directory structure in your repository:

.security/
├── semgrep.yml           # SAST rules configuration
├── trivy.yml             # Vulnerability scanning config
├── gitleaks.toml         # Secret detection patterns
├── zap-baseline.conf     # DAST scan configuration
└── allowed-licences.txt  # Approved licence list

Static Application Security Testing

SAST examines source code to identify vulnerabilities such as injection flaws, authentication weaknesses, and insecure data handling. Run SAST early in development to catch issues before they reach code review.

Configure Semgrep rules

Create the Semgrep configuration file at .security/semgrep.yml:

rules:
  # Include standard security rulesets
  - p/security-audit
  - p/secrets
  - p/owasp-top-ten

  # Language-specific rules
  - p/python
  - p/javascript
  - p/typescript

# Paths to exclude from scanning
exclude:
  - "test/"
  - "tests/"
  - "*_test.py"
  - "*.test.js"
  - "node_modules/"
  - "vendor/"
  - ".git/"

# Severity threshold for CI failure
severity:
  - ERROR
  - WARNING

Execute SAST scan

Navigate to the repository root directory:

   cd /path/to/repository

Run Semgrep with the security configuration:

   semgrep scan --config .security/semgrep.yml --json --output semgrep-results.json .

The scan produces JSON output containing identified issues with file locations, severity levels, and remediation guidance.

Generate a human-readable report:

   semgrep scan --config .security/semgrep.yml --text --output semgrep-report.txt .

Review the summary output:

   Findings:

     src/auth/login.py
       security.python.django-sql-injection
         SQL injection vulnerability detected
         Line 45: cursor.execute(f"SELECT * FROM users WHERE id = {user_id}")
         Severity: ERROR
         Fix: Use parameterised queries

For CI integration, use the exit code to gate pipeline progression:

   semgrep scan --config .security/semgrep.yml --error .
   echo $?
   # Exit code 0: No issues found
   # Exit code 1: Issues detected at configured severity

Interpret SAST findings

Semgrep categorises findings by severity and confidence. The severity indicates potential impact if exploited. The confidence indicates how likely the finding represents a true vulnerability rather than a false positive.

+------------------------------------------------------------------+
|                    SAST FINDING TRIAGE                           |
+------------------------------------------------------------------+
|                                                                  |
|                         Finding                                  |
|                            |                                     |
|                            v                                     |
|                   +------------------+                           |
|                   |    Severity?     |                           |
|                   +--------+---------+                           |
|                            |                                     |
|             +--------------+-------------+                       |
|             |              |             |                       |
|             v              v             v                       |
|           ERROR         WARNING        INFO                      |
|             |              |             |                       |
|             v              v             v                       |
|         +---------+   +---------+   +---------+                  |
|         |  Block  |   | Review  |   | Track   |                  |
|         |  merge  |   | before  |   | in      |                  |
|         |         |   | merge   |   | backlog |                  |
|         +----+----+   +----+----+   +---------+                  |
|              |             |                                     |
|              v             v                                     |
|            +------------------+                                  |
|            | True positive?   |                                  |
|            +--------+---------+                                  |
|                     |                                            |
|               +-----+-----+                                      |
|               |           |                                      |
|              Yes          No                                     |
|               |           |                                      |
|               v           v                                      |
|           +-------+   +--------+                                 |
|           | Fix   |   |Add to  |                                 |
|           | code  |   |ignore  |                                 |
|           +-------+   |with    |                                 |
|                       |comment |                                 |
|                       +--------+                                 |
+------------------------------------------------------------------+

To suppress a false positive, add an inline comment:

# Reason: Query uses validated enum value, not user input
cursor.execute(f"SELECT * FROM reports WHERE type = {ReportType.MONTHLY.value}")

Document all suppressions in a tracking file for audit purposes.

Dynamic Application Security Testing

DAST tests running applications by sending malicious requests and analysing responses. Unlike SAST, DAST discovers vulnerabilities that emerge from runtime behaviour, configuration issues, and server-side processing.

Configure OWASP ZAP

Create the ZAP configuration file at .security/zap-baseline.conf:

# ZAP Baseline Scan Configuration

# Target URL (override in CI)
target=http://localhost:8080

# Spider configuration
spider.maxduration=5
spider.maxdepth=5

# Active scan settings
activescan.policy=Default Policy

# Alert thresholds
alertthreshold=Low

# Output format
outputformat=json

# Excluded URLs (authentication endpoints, healthchecks)
exclude=.*logout.*,.*healthz.*,.*metrics.*

Execute DAST scan

DAST requires a running application instance. Deploy the application to a test environment before scanning.

Start the target application in a test environment:

   # Example: Docker-based application
   docker run -d --name app-under-test -p 8080:8080 your-application:latest

   # Wait for application readiness
   until curl -sf http://localhost:8080/healthz; do sleep 2; done

Run the ZAP baseline scan:

   docker run --rm --network host \
     -v $(pwd)/.security:/zap/wrk:rw \
     ghcr.io/zaproxy/zaproxy:stable \
     zap-baseline.py \
     -t http://localhost:8080 \
     -c zap-baseline.conf \
     -J zap-results.json \
     -r zap-report.html

The baseline scan completes in 2 to 5 minutes for most applications.

For authenticated scanning, provide a context file with session information:

   docker run --rm --network host \
     -v $(pwd)/.security:/zap/wrk:rw \
     ghcr.io/zaproxy/zaproxy:stable \
     zap-full-scan.py \
     -t http://localhost:8080 \
     -n auth-context.context \
     -J zap-auth-results.json

Review the JSON output for findings:

   {
     "alerts": [
       {
         "pluginid": "10038",
         "alertRef": "10038",
         "alert": "Content Security Policy (CSP) Header Not Set",
         "name": "Content Security Policy (CSP) Header Not Set",
         "riskcode": "2",
         "confidence": "3",
         "riskdesc": "Medium (High)",
         "desc": "Content Security Policy (CSP) is an added layer of security...",
         "instances": [
           {
             "uri": "http://localhost:8080/",
             "method": "GET"
           }
         ],
         "solution": "Ensure that your web server, application server, load balancer..."
       }
     ]
   }

Stop and remove the test application:

   docker stop app-under-test && docker rm app-under-test

DAST risk classifications

ZAP assigns risk codes that determine required response:

Risk code	Classification	CI gate action	Resolution timeframe
3	High	Block deployment	Before merge
2	Medium	Block deployment	Within 5 working days
1	Low	Warning only	Within 30 days
0	Informational	Log only	At discretion

Dependency vulnerability scanning

Third-party dependencies introduce vulnerabilities outside your direct control. Dependency scanning identifies known CVEs in libraries and frameworks, enabling remediation through updates or mitigations.

Configure Trivy

Create the Trivy configuration at .security/trivy.yml:

# Trivy vulnerability scanner configuration

# Severity levels to report
severity:
  - CRITICAL
  - HIGH
  - MEDIUM

# Skip vulnerability database update (use cached for CI speed)
skip-db-update: false

# Ignore unfixed vulnerabilities (optional, for pragmatic minimum)
ignore-unfixed: false

# Output format
format: json

# Exit code on findings
exit-code: 1

# Vulnerability types to scan
vuln-type:
  - os
  - library

# Security checks to run
security-checks:
  - vuln
  - secret
  - config

Scan application dependencies

For Node.js projects, scan the package lock file:

   trivy fs --config .security/trivy.yml --output trivy-deps.json package-lock.json

For Python projects, scan requirements files:

   trivy fs --config .security/trivy.yml --output trivy-deps.json requirements.txt

For multi-language projects, scan the entire filesystem:

   trivy fs --config .security/trivy.yml --output trivy-deps.json .

Review identified vulnerabilities:

   {
     "Results": [
       {
         "Target": "package-lock.json",
         "Type": "npm",
         "Vulnerabilities": [
           {
             "VulnerabilityID": "CVE-2024-21538",
             "PkgName": "cross-spawn",
             "InstalledVersion": "7.0.3",
             "FixedVersion": "7.0.5",
             "Severity": "HIGH",
             "Title": "Regular Expression Denial of Service",
             "Description": "Versions of the package cross-spawn before 7.0.5..."
           }
         ]
       }
     ]
   }

Update vulnerable dependencies:

   # Node.js
   npm update cross-spawn

   # Or for specific version
   npm install cross-spawn@7.0.5

Scan container images

Container images include operating system packages and application dependencies. Scan images before pushing to registries and before deployment.

Scan a local image:

   trivy image --config .security/trivy.yml --output trivy-image.json your-application:latest

Scan an image in a remote registry:

   trivy image --config .security/trivy.yml ghcr.io/your-org/your-application:v1.2.3

For base image selection, compare vulnerability counts:

   # Compare Alpine vs Debian base images
   trivy image --severity HIGH,CRITICAL alpine:3.19
   trivy image --severity HIGH,CRITICAL debian:bookworm-slim

Select base images with fewer vulnerabilities and active maintenance.

Secret detection

Secrets committed to version control persist in repository history even after removal. Secret detection scans code and commit history to identify credentials, API keys, and tokens before they become security incidents.

Configure Gitleaks

Create the Gitleaks configuration at .security/gitleaks.toml:

# Gitleaks configuration

title = "Organisation Secret Detection Rules"

[extend]
# Use default rules as base
useDefault = true

# Additional custom patterns
[[rules]]
id = "organisation-api-key"
description = "Organisation internal API key"
regex = '''ORG_[A-Z0-9]{32}'''
secretGroup = 0
entropy = 3.5

[[rules]]
id = "database-connection-string"
description = "Database connection string with credentials"
regex = '''(postgres|mysql|mongodb)://[^:]+:[^@]+@[^/]+/'''
secretGroup = 0

# Paths to exclude
[allowlist]
paths = [
  '''\.security/gitleaks\.toml''',
  '''test/fixtures/.*''',
  '''.*_test\.go''',
  '''.*\.test\.js'''
]

# Known false positives
regexes = [
  '''EXAMPLE_.*''',
  '''test-api-key-.*'''
]

Execute secret scan

Scan the current working directory:

   gitleaks detect --config .security/gitleaks.toml --source . --report-format json --report-path gitleaks-results.json

Exit code 0 indicates no secrets found. Exit code 1 indicates detected secrets.

Scan git history for previously committed secrets:

   gitleaks detect --config .security/gitleaks.toml --source . --log-opts="--all" --report-format json --report-path gitleaks-history.json

This scan examines all commits in all branches. Duration scales with repository history size.

Use pre-commit hooks to prevent secret commits:

   # Install pre-commit hook
   gitleaks protect --config .security/gitleaks.toml --staged

Add to .git/hooks/pre-commit:

   #!/bin/bash
   gitleaks protect --config .security/gitleaks.toml --staged

When secrets are detected, respond immediately:

   # Rotate the compromised credential (external to this procedure)

   # Remove from git history using git-filter-repo
   pip install git-filter-repo --break-system-packages
   git filter-repo --invert-paths --path path/to/file-with-secret

   # Force push to remote (coordinate with team)
   git push --force-with-lease origin main

Compliance scanning

Compliance scanning verifies that code and dependencies meet organisational policies for software licences, security configurations, and regulatory requirements.

Licence compliance

Open source licences impose obligations that range from attribution to source code disclosure. Verify all dependencies use approved licences before inclusion.

Create the approved licence list at .security/allowed-licences.txt:

MIT
Apache-2.0
BSD-2-Clause
BSD-3-Clause
ISC
0BSD
CC0-1.0
Unlicense
MPL-2.0
LGPL-2.1-only
LGPL-3.0-only

Scan for licence compliance:

# Node.js projects
npx license-checker --json --out licence-report.json

# Filter for disallowed licences
npx license-checker --failOn "GPL-3.0;AGPL-3.0;SSPL-1.0"

# Python projects
pip install pip-licenses --break-system-packages
pip-licenses --format=json --output-file=licence-report.json

# Check against allowed list
pip-licenses --fail-on="GPL-3.0;AGPL-3.0"

Infrastructure as Code security

IaC templates for Terraform, CloudFormation, and Kubernetes contain security configurations that require validation.

Scan Terraform configurations:

   trivy config --config .security/trivy.yml --output trivy-iac.json ./terraform/

Scan Kubernetes manifests:

   trivy config --config .security/trivy.yml --output trivy-k8s.json ./kubernetes/

Review misconfigurations:

   {
     "Results": [
       {
         "Target": "terraform/main.tf",
         "Misconfigurations": [
           {
             "Type": "Terraform Security Check",
             "ID": "AVD-AWS-0086",
             "Title": "S3 bucket does not have encryption enabled",
             "Description": "S3 Buckets should be encrypted to protect data...",
             "Severity": "HIGH",
             "Resolution": "Add server-side encryption configuration"
           }
         ]
       }
     ]
   }

CI pipeline integration

Integrate security testing into CI pipelines to enforce security gates automatically. The following configuration demonstrates GitHub Actions integration; adapt for GitLab CI, Azure DevOps, or other platforms.

name: Security Testing

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  sast:
    name: Static Analysis
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Run Semgrep
        uses: returntocorp/semgrep-action@v1
        with:
          config: .security/semgrep.yml
          generateSarif: true

      - name: Upload SARIF
        uses: github/codeql-action/upload-sarif@v2
        with:
          sarif_file: semgrep.sarif

  secrets:
    name: Secret Detection
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Run Gitleaks
        uses: gitleaks/gitleaks-action@v2
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

  dependencies:
    name: Dependency Scanning
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Run Trivy
        uses: aquasecurity/trivy-action@master
        with:
          scan-type: 'fs'
          scan-ref: '.'
          severity: 'CRITICAL,HIGH'
          exit-code: '1'

  container:
    name: Container Scanning
    runs-on: ubuntu-latest
    needs: [sast, secrets, dependencies]
    steps:
      - uses: actions/checkout@v4

      - name: Build image
        run: docker build -t app:${{ github.sha }} .

      - name: Scan image
        uses: aquasecurity/trivy-action@master
        with:
          image-ref: 'app:${{ github.sha }}'
          severity: 'CRITICAL,HIGH'
          exit-code: '1'

  dast:
    name: Dynamic Analysis
    runs-on: ubuntu-latest
    needs: [container]
    steps:
      - uses: actions/checkout@v4

      - name: Start application
        run: |
          docker run -d --name app -p 8080:8080 app:${{ github.sha }}
          sleep 10

      - name: Run ZAP
        uses: zaproxy/action-baseline@v0.10.0
        with:
          target: 'http://localhost:8080'
          rules_file_name: '.security/zap-baseline.conf'

This pipeline executes security checks in parallel where possible, with container and DAST stages waiting for earlier stages to complete. The pipeline fails if any security gate identifies HIGH or CRITICAL issues.

+--------------------------------------------------------------------+
|                     CI SECURITY PIPELINE                           |
+--------------------------------------------------------------------+
|                                                                    |
|  +--------+     +----------+     +--------------+                  |
|  | Push   |---->| Checkout |---->| Parallel     |                  |
|  | Event  |     | Code     |     | Scans        |                  |
|  +--------+     +----------+     +------+-------+                  |
|                                         |                          |
|                        +----------------+----------------+         |
|                        |                |                |         |
|                        v                v                v         |
|                   +----+----+     +-----+-----+    +-----+-----+   |
|                   | SAST    |     | Secrets   |    | Dependency|   |
|                   | Semgrep |     | Gitleaks  |    | Trivy     |   |
|                   +----+----+     +-----+-----+    +-----+-----+   |
|                        |                |                |         |
|                        +----------------+----------------+         |
|                                         |                          |
|                                         v                          |
|                                  +------+-------+                  |
|                                  | All passed?  |                  |
|                                  +------+-------+                  |
|                                         |                          |
|                              +----------+----------+               |
|                              |                     |               |
|                             Yes                    No              |
|                              |                     |               |
|                              v                     v               |
|                       +------+-------+      +------+-------+       |
|                       | Build        |      | Fail         |       |
|                       | Container    |      | Pipeline     |       |
|                       +------+-------+      +--------------+       |
|                              |                                     |
|                              v                                     |
|                       +------+-------+                             |
|                       | Scan         |                             |
|                       | Container    |                             |
|                       +------+-------+                             |
|                              |                                     |
|                              v                                     |
|                       +------+-------+                             |
|                       | Deploy to    |                             |
|                       | Test Env     |                             |
|                       +------+-------+                             |
|                              |                                     |
|                              v                                     |
|                       +------+-------+                             |
|                       | DAST         |                             |
|                       | ZAP Scan     |                             |
|                       +------+-------+                             |
|                              |                                     |
|                              v                                     |
|                       +------+-------+                             |
|                       | Gate         |                             |
|                       | Decision     |                             |
|                       +--------------+                             |
|                                                                    |
+--------------------------------------------------------------------+

Verification

After configuring security testing, verify the complete pipeline functions correctly.

Trigger a test run with a known vulnerability to confirm detection:

# Create a test file with a deliberate SQL injection vulnerability
cat > test-vuln.py << 'EOF'
def get_user(user_id):
    query = f"SELECT * FROM users WHERE id = {user_id}"
    return execute_query(query)
EOF

# Run SAST
semgrep scan --config .security/semgrep.yml test-vuln.py
# Expected: Finding reported for SQL injection

# Clean up
rm test-vuln.py

Verify secret detection with a test credential:

# Create a test file with a fake secret
echo 'API_KEY="sk-test-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"' > test-secret.txt

# Run Gitleaks
gitleaks detect --source . --no-git
# Expected: Finding reported for API key pattern

# Clean up
rm test-secret.txt

Confirm CI integration by checking pipeline execution:

# Push a test branch
git checkout -b security-test
git commit --allow-empty -m "Test security pipeline"
git push origin security-test

# Verify pipeline triggered in CI platform
# Check job statuses: sast, secrets, dependencies should complete

Validate DAST configuration by scanning a known vulnerable application:

# Use OWASP WebGoat as test target
docker run -d --name webgoat -p 8080:8080 webgoat/webgoat

# Run ZAP baseline
docker run --rm --network host ghcr.io/zaproxy/zaproxy:stable \
  zap-baseline.py -t http://localhost:8080/WebGoat

# Expected: Multiple findings across risk levels
# Clean up
docker stop webgoat && docker rm webgoat

Troubleshooting

Symptom	Cause	Resolution
Semgrep exits with code 2	Configuration syntax error	Run `semgrep --validate --config .security/semgrep.yml` to identify malformed YAML
Semgrep reports “no targets found”	Incorrect path or exclusion pattern too broad	Verify repository path and review exclude patterns in configuration
Trivy reports “failed to download vulnerability database”	Network connectivity or rate limiting	Use `--skip-db-update` with pre-cached database or configure proxy with `HTTPS_PROXY`
Trivy scan times out	Large image or slow network	Increase timeout with `--timeout 30m` or scan filesystem instead of image
Gitleaks reports “could not determine git root”	Running outside Git repository	Ensure `.git` directory exists or use `--no-git` for filesystem-only scan
ZAP cannot connect to target	Application not running or wrong port	Verify application is accessible with `curl http://localhost:8080` before starting scan
ZAP scan runs indefinitely	Spider configured with no depth limit	Set `spider.maxdepth=5` and `spider.maxduration=5` in configuration
CI pipeline times out	DAST scan exceeds job limit	Use baseline scan instead of full scan; set scan duration limits
False positives flood results	Default rules too aggressive	Create allowlist entries for known safe patterns; document exclusions
Different results locally vs CI	Tool version mismatch	Pin tool versions in CI configuration; update local tools to match
Container scan finds OS vulnerabilities	Base image outdated	Update to latest patch version of base image; rebuild container
Licence scan fails on internal packages	Private packages missing licence metadata	Add licence field to internal package.json or setup.py
Secret detected in test fixtures	Test data contains realistic patterns	Add test fixture paths to Gitleaks allowlist; use obviously fake values
SARIF upload fails	Invalid SARIF format or size exceeds limit	Validate SARIF with `sarif validate`; filter results to reduce size
Pipeline passes but vulnerabilities exist	Severity threshold misconfigured	Verify `--severity` and `--exit-code` flags match policy requirements