TL;DR: A CAIQ documents your cloud security controls; a penetration test checks whether they hold up under testing. CSA STAR does not prescribe one universal testing schedule, but CCM v4.1 includes TVM-07 on periodic independent third-party penetration testing. The findings and remediation records can support your assessment and customer reviews.
Introduction
Your SaaS platform has a CSA STAR Registry listing, and you’ve filled out a security questionnaire for a potential customer. Is that enough to convince them your cloud service is secure?
Not always. The customer may also want to see the results of a penetration test to check whether someone could break into your systems or access data they shouldn’t. The CAIQ tells customers what security controls you have in place. A penetration test checks whether an attacker could find and exploit weaknesses in your systems.
What Is CSA STAR?
CSA STAR is a cloud-specific assurance programme; ISO/IEC 27001 is a general information security management standard that STAR Certification builds on. The technical backbone of the programme is the Cloud Controls Matrix, or CCM. Think of the CCM as a structured set of security controls for cloud providers, covering areas such as data encryption, access management, and vulnerability management.
The latest version of the Cloud Controls Matrix, CCM v4.1, was released in January 2026. It includes 207 controls across 17 security domains, while the accompanying CAIQ v4.1 contains 283 questions. Level 1 is the Cloud Security Alliance self-assessment option. The provider completes the STAR Level 1 CAIQ and submits it to the public STAR Registry. The CAIQ contains questions about whether specific controls are in place and how they are implemented. Level 2 provides independent assurance through either STAR Certification, which builds on ISO/IEC 27001, or STAR Attestation, which uses the SOC 2 framework alongside the CCM.
The difference is that a CSA STAR assessment documents or evaluates security controls, while a penetration test actively probes a defined system for exploitable weaknesses. The two can support each other, but they do not serve the same purpose.
| STAR level | What is assessed | Who assesses it | Where penetration testing fits |
| Level 1: Self-Assessment | The provider completes the CAIQ and submits its control responses to the STAR Registry. | The cloud provider completes the self-assessment; no independent third-party audit is inherent to this route. | The provider documents how it meets the applicable penetration-testing control. Test results can support its CAIQ responses. |
| Level 2: STAR Certification | CCM controls alongside an ISO/IEC 27001 information-security management system. | An accredited third-party certification body. | Test reports, remediation records, and retest evidence can support the assessment of applicable controls. |
| Level 2: STAR Attestation | CCM controls alongside a SOC 2 examination. | An independent CPA firm. | Test findings can provide supporting technical evidence, but do not replace the attestation process. |
Why a CAIQ Alone Is Not Enough
Completing the CSA CAIQ (Consensus Assessments Initiative Questionnaire) is useful work. It makes a cloud provider think carefully about which controls are in place, who is responsible for them, and what customers expect. Nobody should dismiss it. There is, however, a difference between describing a control and showing that it holds up when someone tries to break it.
What a Questionnaire Cannot Verify
When a security team fills out a CAIQ, they’re writing about their environment as they understand it. They might mark “yes” to whether encryption is applied in transit and at rest, and technically, they’d be right. Encryption is configured. If an API has no rate limit on login attempts, an attacker could try many passwords or stolen credentials without being slowed down. Encryption alone would not stop this, and the CAIQ does not test whether an attacker can exploit the weakness.
Cloud security incidents also involve issues that a questionnaire cannot independently verify. These include weaknesses in API access controls, compromised identities, misconfigured cloud services, and flaws in third-party integrations. A CAIQ can help document the controls intended to manage these risks, but the answers alone cannot prove that those controls work as expected.
Verizon’s 2026 Data Breach Investigations Report found that credential abuse accounted for 13% of known initial access vectors in non-Error, non-Misuse breaches, while vulnerability exploitation accounted for 31%. These findings support testing identity and access controls alongside exposed applications and infrastructure. Providers such as AWS, Azure, and Google Cloud secure parts of the underlying infrastructure, while customers remain responsible for certain configurations, identities, applications, and data, depending on the service they use. A completed questionnaire cannot, by itself, prove that these responsibilities have been handled correctly.
A US healthcare company or financial services firm reviewing a cloud provider may request security questionnaires, independent assurance, or penetration-test results. A provider that already holds SOC 2 or ISO/IEC 27001 may still pursue CSA STAR to show cloud-specific security assurance.
What a Real Penetration Test Adds
Manual Testing vs. Automated Scanning
An automated scan can find known vulnerabilities, but is that enough to tell you how an attacker could get into your cloud environment? Some weaknesses only become clear when someone tests how different parts of your system work together.
| Automated scanning | Manual penetration testing |
| Finds known vulnerabilities and common security issues. | Tests whether an attacker could exploit weaknesses in your systems. |
| May miss risks that involve several weaknesses working together. | Checks whether those weaknesses could be combined to gain unauthorised access. |
| Helps your team spot issues that need attention. | Helps your team understand what an attacker could access or do. |
For example, a storage bucket may not be open to the public, but an integration with excessive permissions could still expose its data. Manual testing can help reveal how an attacker might misuse those permissions.
Always remember that penetration testing, cloud configuration reviews, and detection testing are different activities. A penetration test tries to exploit weaknesses, a configuration review checks your security settings, and detection testing checks whether your monitoring tools and response team can spot suspicious activity. You can include all three in one engagement, but make sure the agreed scope covers what you need.
Areas to Include in the Test
For a cloud provider working through a CSA STAR assessment, the test needs to cover the kinds of attacks that cloud environments actually face:
- Cloud configuration testing, where included, to check for exposed storage, overly permissive roles, and other insecure settings.
- Identity and access management testing, including credentials, tokens, and access privileges. Google Cloud reported that identity compromise underpinned 83% of compromises in its analysis of cloud intrusions in H2 2025.
- API testing that looks at whether authentication can be bypassed, whether one customer can access another’s data, and whether limits on how often an API can be called are missing.
- Third-party integration review, particularly CI/CD pipelines (the systems that automatically deploy your code), OAuth connections, and other places where an external party has access to your environment.
The test report matters as much as the test itself. A useful penetration test report explains what was found, how the weakness could affect the business, and what the engineering team can do to fix it. A retest can then confirm whether the fix worked, rather than relying only on a note that says “resolved.” That remediation and retest record can provide useful audit evidence.
How Penetration Testing Strengthens CSA STAR
The CCM organises cloud security into 17 domains. Several domains, including vulnerability management, application and interface security, identity and access management, and infrastructure, include controls that penetration testing can help assess. During a Level 2 assessment, an auditor reviews the controls and supporting evidence rather than relying only on written answers.
For organisations pursuing CSA STAR certification, penetration-test findings can provide supporting technical evidence, alongside other records the auditor reviews.
Take IAM as an example. Your CAIQ might state that access policies follow least privilege, meaning users and systems only get the permissions they need to do their specific job, nothing more. An auditor reviewing that control may look for configurations and other evidence that support the claim, rather than relying on the written policy alone. Documented findings and retest results can support that evidence, although they do not replace the auditor’s full control assessment.
Enterprise clients may also request penetration-test results during vendor reviews. A current report can provide technical evidence and help answer their security team’s questions.
What the Test Should Cover
Not every penetration test covers the same systems. For a CSA STAR assessment, the scope needs to match your cloud environment. A network test alone may not reveal exposed cloud storage or weaknesses that let one customer access another’s data.
The scope should cover:
- Cloud configuration and infrastructure across your provider environment
- Identity and access management, including service accounts, role assignments, and federated access (where one system trusts an identity verified by another)
- APIs, web applications, and any mobile surfaces the platform exposes
- Network segmentation and exposure to the public internet
- Data storage permissions, along with encryption settings and how data is protected at rest and in transit
- Logging and monitoring, if detection testing is included in the engagement
- Third-party integrations, especially where an external system or partner has access to your environment
Why Third-Party Access Matters
Third-party integrations also deserve attention. If an integration has more access than it needs, your team should review and manage the permissions it controls under the shared responsibility model. Google Cloud’s H1 2026 Threat Horizons report describes an incident in which a compromised npm package stole a developer’s GitHub token. The attacker then abused an OIDC trust relationship between GitHub Actions and AWS to obtain full AWS administrator access in under 72 hours. The case shows how a stolen developer token and overly permissive trust between systems can give attackers access to cloud resources.
CSA STAR Penetration Testing Checklist
Before engaging a testing firm, agree on the scope, responsibilities, and evidence you need:
| # | Action | Owner | Deliverable |
| 1 | Define the cloud environment, services, shared-responsibility boundaries, and testing windows. | Cloud or engineering lead | Approved scope document |
| 2 | Map critical assets, customer-facing systems, data flows, and third-party access paths. | Security lead with engineering support | Asset and access-path map |
| 3 | Confirm coverage for cloud configurations, IAM, APIs, applications, infrastructure, and integrations. | Security lead and testing provider | Agreed test plan |
| 4 | Agree on report format, severity ratings, evidence requirements, and remediation guidance. | Security lead and testing provider | Reporting requirements |
| 5 | Assign an owner and due date to each finding, then schedule retesting for resolved issues. | Security lead and finding owners | Remediation tracker |
| 6 | Keep reports and remediation evidence organised to reduce avoidable follow-up during audits and customer reviews. | Compliance or GRC lead | Audit-evidence pack |
Common Mistakes to Avoid
A common mistake is treating a completed CAIQ as proof that every control works as intended.
Other mistakes include:
- Relying entirely on automated scanning. Scanners find known vulnerabilities, but they may miss attack paths that combine weaknesses, such as a billing flaw that lets one customer access another’s records.
- Leaving identity out of scope. Stolen credentials and excessive permissions can give attackers access to cloud systems, so IAM should be part of a cloud-focused assessment.
- Skipping retesting leaves the team relying on the developer’s claim that the fix worked. Retesting checks whether the weakness has actually been resolved.
- Finally, excluding third-party integrations from scope can leave excessive access unchecked. If an integration has more permissions than it needs, review who manages those permissions and include the access path in testing where it falls within the agreed scope.
How Qualysec Helps
If you’re preparing for a Cloud Security Alliance CSA STAR certification assessment, you may need more than completed questionnaires to explain how your cloud environment holds up under testing. Qualysec can help assess your cloud infrastructure, APIs, applications, and other systems within an agreed scope.
- Human-Led, AI-Powered testing that combines automated checks with expert review.
- Testing across cloud, API, web, mobile, and infrastructure, based on your agreed scope.
- Clear findings and remediation guidance to help your technical team understand what needs fixing.
- Retesting options to check whether reported weaknesses have been resolved.
- Technical evidence your team can organise for customer reviews and audit discussions.
Contact Qualysec to discuss the scope of your cloud penetration test.
Conclusion
A penetration test does not replace the CAIQ or the CSA STAR assessment process. It provides technical evidence of weaknesses that could affect the systems protecting your cloud services.
If you are preparing for a CSA STAR assessment, plan testing early enough to review the findings, fix confirmed weaknesses, and retest where needed. Keeping the report and remediation records organised can also help when customers or assessors request supporting evidence.
FAQs
1. Does CSA STAR require penetration testing?
CCM v4.1 includes control TVM-07, which addresses periodic independent third-party penetration testing. CSA STAR does not prescribe one universal testing schedule for every provider. The applicable control requirements and the provider’s testing programme should be considered when preparing supporting evidence.
2. What is the difference between CAIQ and penetration testing?
The CAIQ documents what security controls you say you have. A penetration test actively tests a system for weaknesses an attacker could exploit. They serve different purposes and work better together.
3. Can penetration testing help with CSA STAR Level 2?
Yes. A penetration test can provide technical evidence of exploitable weaknesses. Findings, remediation records, and retest results can support the assessment, but they do not replace the auditor’s review of the applicable controls.
4. How often should a cloud provider run penetration testing?
The frequency depends on your risk, contractual obligations, and applicable requirements. Annual testing and testing after major changes may be appropriate, but CSA STAR does not universally prescribe this schedule. Allow time to fix and retest findings before an audit.
5. Is penetration testing required for CSA STAR Level 2?
CSA STAR Level 2 does not prescribe one universal penetration-test format or fixed testing frequency. Under CCM v4.1, control TVM-07 addresses periodic penetration testing by independent third parties. Providers should document their testing process and retain findings and remediation records to support assessment of the applicable control.
6. CSA STAR vs ISO 27001: which do I need?
ISO/IEC 27001 provides a general framework for managing information security, while CSA STAR adds cloud-specific assurance through the Cloud Controls Matrix (CCM). STAR Certification combines ISO/IEC 27001 with the CCM. Your choice depends on customer requirements, your service scope, and the assurance your buyers expect.
7. Does the CAIQ require penetration test evidence?
The CAIQ does not require a penetration-test report to be attached to every submission. Providers should answer applicable questions accurately and retain evidence that supports their responses, including test reports and remediation records where appropriate.
