Quick answer: CI/CD pipeline penetration testing is a security service that examines, through an attacker's eyes, the flaws of an organization's software build and deployment chain, that is the automated process in which code is produced and released to production. The most critical risk of this chain is this: an attacker who takes over the chain can push malicious code to the production environment without writing a single line of code. The most common flaws are secrets leaked into the chain, insecure runners, unvetted dependencies and overly privileged deployment keys. DSET runs this test with its local AI engine KAOS and verifies every finding with a working proof.

Modern software is produced by code a developer writes passing through an automated chain that builds, tests and releases it. This chain, that is the continuous integration and continuous deployment chain, has direct write access to one of the organization's most sensitive assets, the production environment. For this reason the chain itself is a high impact attack surface that is often overlooked. This article explains what CI/CD pipeline penetration testing is, which flaws it targets and the DSET and KAOS approach.

What is CI/CD pipeline penetration testing

CI/CD pipeline penetration testing tests the flaws of the software build and deployment chain through controlled attacks. The test covers each of the parts that make up the chain: the code repository and its access rights, the build scripts and the machines that run them, the sources from which dependencies are pulled, the secrets defined in the chain and the deployment rights to the production environment. The goal is to prove whether an attacker can take over one of these parts and from there reach the production environment.

Which flaws it targets

The flaws of the CI/CD chain often stem from broad rights granted for convenience and secrets left in the chain.

Flaw class What it means Typical result
Leaked secret A key or password stays clear in a chain log or variable Credential takeover
Poisoned build step The attacker adds a malicious step to the build script Malicious code to production
Insecure runner The build machine is shared and not isolated Cross job compromise
Unvetted dependency The chain pulls a malicious package Supply chain attack
Overly privileged deployment The deployment key has more access than needed Wide blast radius

The essence of the table is this: because the CI/CD chain has write access to the production environment, a small flaw in the chain can turn directly into the highest impact. For this reason the security of the chain is as important as the security of the application itself. For the risk of poisoned dependencies supply chain attacks and software dependency security is complementary, and for the flaws of the code pulled into the chain source code security review and SCA.

Relation to DevSecOps

DevSecOps is the approach that aims to embed security tests inside the software development chain. CI/CD pipeline penetration testing touches two legs of this approach. First, it checks whether the automated test steps that should be embedded in the chain are set up correctly and whether they can be bypassed. Second, it tests the chain itself as an attack target. For the difference between automated security testing and deep manual testing, the SAST, DAST, IAST and SCA code security testing selection matrix is a good starting point.

Why secret management is critical

The CI/CD chain carries many secrets to connect to production databases, cloud providers and external services. When a developer writes these secrets in clear in a configuration file or an environment variable for convenience, anyone who reaches the chain's logs can see them. For this reason an important part of the test is examining how the secrets defined in the chain are stored, whether they leak into logs and whether the rights are bounded correctly.

When is it needed

CI/CD pipeline penetration testing should be done when a new deployment chain is set up, when the chain architecture changes and at regular intervals. It is especially critical for chains that deploy automatically to production and are accessed by many developers. The scope must be drawn clearly before the test, for which the penetration testing preparation and scoping guide is the reference point.

The KAOS and DSET approach

DSET runs CI/CD pipeline penetration testing with its local AI engine KAOS. KAOS examines the chain configuration for leaked secrets and excessive rights, tests the build steps against poisoning, vets the pulled dependencies and reports every flaw it finds with a working proof, under the principle of proof based verification. The difference is this: it does not settle for saying a configuration is theoretically risky, it proves safely that the risk actually reaches the production environment. After a finding is verified, retest and remediation verification is applied to see that the fix works.

Frequently asked questions

Does CI/CD chain testing replace application penetration testing? No, it complements it. Application penetration testing examines the flaws of the released software. CI/CD chain testing examines the process in which that software is produced and released. The two target different layers; both are needed for a complete security picture.

I use a cloud based chain, is a test still needed? Yes. A cloud based chain delegates part of the infrastructure to the provider, but the configuration of the chain, the defined secrets, the access rights and the build scripts are your responsibility. Most attacks happen in this layer that is your responsibility, so being cloud based does not make the test unnecessary.

Will the test affect my production deployment? In a well planned test no. Which chain, which environment and which steps will be tested are set in advance by contract. When possible the test is run on a non production copy of the chain, so the live deployment is not affected.

Sources

To examine your software build and deployment chain with a proof based test, contact DSET. We provide CI/CD pipeline and DevSecOps security testing and consulting from our Ankara Hacettepe Teknokent laboratory.