Adobe Security Testing Reports: Expanding and Evolving Penetration Testing to Combat Adversarial Interests | by Renae Kang | Sep, 2024

By Dana Pirvu, Manager, Penetration Testing, Adobe

As adoption of generative AI (GenAI) continues to grow, adversarial interest in compromising the security posture of all applications and services — whether they leverage GenAI or not — is increasing as well. Companies that develop and deliver the complex software on which so much of the world’s economy depends must focus their efforts on defending against these attacks.

At Adobe, we believe the best way to measure the security posture of a product is by thinking like an adversary. That’s why we’ve incorporated this tenet into the foundation of the Adobe Security Testing Reports, where we rigorously test each of our products from an adversary-aware perspective and measure them against proven, exploitable threats. The resulting product-specific reports provide a transparent view into the wide range of security testing Adobe conducts, which helps us build trust with our customers and partners. In this blog, I will explain how Adobe puts together our annual Security Testing Reports.

Adobe’s annual security testing regime includes five (5) different types of tests:

• Outsourced (or Third-Party) Penetration Testing: Bypassing user access control restrictions and/or gaining privileged access to infrastructure through exploitation of applications and network-related vulnerabilities from an adversarial perspective, outsourced pen tests are externally facing from outside Adobe’s network and provide an objective view of Adobe products’ security posture.
• Internal Penetration Testing: Employing manual testing techniques and automated tools to evaluate internal systems, applications, and networks, internal pen tests are performed in a staged environment inside Adobe’s network and simulate real-world attacks to measure the effectiveness of existing security measures;
• Red Team Testing: Challenging our security teams’ cyber defenses using customized toolkits, Adobe Security’s Red Team employs offensive techniques to evaluate our preparedness to defend against various real-world adversaries;
• Bug Bounty: Collaborating with a global community of external security researchers and ethical hackers helps Adobe provide an extra layer of protection to customers by finding and reporting undiscovered vulnerabilities to our product security teams before real-world adversaries can exploit them.
• Customer Penetration Testing: In some exceptional circumstances, Adobe is able to accommodate customer-led penetration testing. Any findings uncovered from these tests provide an extra input to our security teams.

Adobe Security Testing Reports help achieve consistency in testing across our products, including those with GenAI features, by using the same methodologies, guidelines, industry standards, and best practices to test all products. Adobe then does not publish the results of the Security Testing Reports until all identified vulnerabilities have been addressed and either fixed or mitigated.

In addition to the independent, third-party testing firm’s attestation, each Adobe Security Testing Report also includes:

• Testing Scope: Specifies the solution components included in the assessment.
• Testing Approach: Outlines the methodology, guidelines, industry standards and best practices followed, including frameworks like the Open Worldwide Application Security Project (OWASP) Top 10 or the SysAdmin, Audit, Network, and Security (SANS) Top 25 to identify critical vulnerabilities.
• Testing Methodology: Describes the assessment techniques and methodology followed, such as black-box testing or grey-box testing with source code review.
• Test Results: Summarizes the vulnerabilities identified during the testing process.
• Finding Summaries: Lists the exploitable vulnerabilities discovered during the assessment.
• Finding Resolution: Provides the status of each identified vulnerability, such as whether it has been fixed or mitigated.
• Finding Severity: Assigns a risk rating to each vulnerability based on CVSS 3.1 scoring.
• Security Testing Categories: Details the defined policies, standards, and solutions, along with specific tests designed to achieve control adherence and resilience.

Adobe views the Adobe Security Testing Reports as an ever-evolving source of customer trust, which is why we continuously work to improve both their content and transparency. One example of these improvements is the expansion of the outsourced pen testing component. Earlier this year, we expanded our testing methodology to include a hybrid testing approach, combining “Grey Box” as well as “Authenticated Black Box” testing. Grey-box or source code-assisted tests involve sharing detailed information with testers, such as functionality documentation, user roles, and source code, to ensure a thorough assessment. In authenticated black-box testing, testers are provided with a full list of externally facing URLs/domains and testing credentials to evaluate the application’s security. This addition enables us to conduct comprehensive third-party testing that uses a hybrid testing methodology to verify both test coverage and completeness.

We base our improvements for the Security Testing Reports not only on new threats and updated industry best practices, but also on feedback we receive directly from customers and stakeholders. In the most recent iteration, we incorporated customer-recommended features, including the severity rating and testing source for each finding, as well as a list of all testing sources included in the report.

The Adobe Security Testing Reports are instrumental in helping our customers meet their own internal compliance and regulatory efforts. We often receive feedback that these reports have been valuable supplementary documentation that help customers assess Adobe’s offerings against their own security requirements for application and network testing. Those customers operating in highly regulated industries have been at the forefront of this feedback as a beneficiary of Adobe’s investment in developing comprehensive testing reports alongside other formal certifications found in the Adobe Trust Center.

More importantly, the Adobe Security Testing Reports also play a crucial role in providing our customers with enhanced visibility into our testing capabilities and the efforts we implement to ensure that our products are as resilient and secure as possible against adversarial attacks. We believe that providing this level of transparency enhances the trust our customers place in us, and we will continue to refine and expand these efforts to uphold our commitment to create safer digital experiences for all.