What's the best way to balance penetration test depth and breadth for optimal coverage and efficiency?

For every PT project, we start with an initial meeting to: 1. Define the scope of the PT - number of IP addresses, domains / URLs, list of devices / hosts / VMs / DBs / etc that is to be included. 2. Define the limitations of the PT - range of IPs not included in the testing. Mitigation limitation - to provide suggestions, but not full coding or full remediation process. 3. Define the limitations of the PT methodology. What methodology is being used in the PT, based on NIST / OWASP / OSSTMM / PTES / SANS ? 4. List down the software and/or hardware to be used. 5. Duration, timeframe & milestones of the PT when the scope has been determined. Create a schedule. This should all be discussed & written in the PT agreement before testing starts.

In it's simplest form, I would start by meeting with the respective line of business leaders and technical subject matter experts. Step one is to define the scope which may result is a wide-focus. Step two is to prioritize the work. Step three is to break up the work into digestible chunks into a road map. Depending on the number of resources assigned, my approach would be to not only test but have a plans for a technical team to solution fixes as the vulnerabilities are found. Architects would incorporate long term solutions to solve hardware and software infrastructure issues and course correct. This thought process would be taken as the product is being built, so a security first mindset is adopted and delivered.

Whenever possible, it's important to define "The Crown Jewels", which are the most important assets of the company. Some points that might be taken into account when defining these assets: - Which applications hold critical data like PII, financial information etc. - Which systems are critical to sustain the business. Another point to consider is regarding the time in which the tests will be executed and the impact they may cause like resource consumption, network traffic, possibility of causing DoS etc.

Threat intelligence is a great input to define scope. Understand the threat actors relevant to the organization and the kind of attacks they perform could be useful to prioritize the red team engagement scope

To kick off a penetration testing project, the initial crucial task is to clearly outline its scope. This involves establishing the boundaries and objectives in line with the organization's business goals, risk tolerance, and compliance needs. Specify the target systems, networks, and applications, and outline the chosen testing methods, tools, and techniques. It's vital that the scope is unambiguous, practical, and has consensus among stakeholders, including the testing team, client, and any third parties involved.

Prioritization is often a nuanced task that combines quantitative metrics with qualitative insights. In my role as a CISO, I find that threat modeling frameworks like STRIDE or DREAD can offer a structured way to assess the risk level of different targets. However, these frameworks don't capture the full picture; they should be supplemented with real-world insights and intelligence like threat feeds, incident reports, and industry benchmarks. Additionally, consider involving business stakeholders in this step. They can provide unique perspectives on the criticality of certain assets, which could be different from a purely technical viewpoint.

With the scope set, the next step is prioritizing targets by assessing the assets and vulnerabilities with the greatest risk or value to the organization. This prioritization ensures efforts concentrate on the most critical areas, avoiding unnecessary testing. Consider factors like breach likelihood and impact, vulnerability exposure and exploitability, data sensitivity, and the complexity and cost of remediation. This strategic approach enhances efficiency and focuses resources where they matter most.

Once "The Crown Jewels" have been defined, they should be tested before any other systems. Depending on the number of systems considered as Crown Jewels, they could be prioritized according to risk, impact in case of compromising, kind of information hold/handled by them etc.

To make a pragmatic scope definition define only the "crown jewels" and validate full impact after the exercise. Sometimes is more useful to cut an engagement short if highest value target is compromised and use remaining hours in other scenario. This works with the assumption that if highest value target is compromised all other assets in scope can be compromised

Following target prioritization, meticulous planning of the testing process is essential. Organize phases—stages and steps—aligned with the scope, objectives, and available resources. Common phases include reconnaissance, scanning, exploitation, post-exploitation, and reporting. Each phase must have a well-defined purpose, output, and deliverable, with clear start and end points. This structured approach ensures a systematic and comprehensive penetration testing process.

Planning the phases of a penetration test is a critical step that goes beyond mere procedure. Each phase—reconnaissance, scanning, exploitation, post-exploitation, and reporting—has its unique intricacies. Reconnaissance sets the stage by providing context, while scanning and exploitation leverage both automated and manual techniques for a comprehensive assessment. Post-exploitation clarifies the real-world impact, answering questions like "What data can I access?" Reporting is the endpoint, and it should provide actionable insights rather than just a list of vulnerabilities. Overall, the planning phase is your blueprint for a successful, informative penetration test.

Selecting the right tools for a penetration test is a pivotal decision that should align with the defined scope and targeted assets. In my experience, diversifying your toolkit is crucial for robust testing. Commercial tools often offer reliability and customer support, while open-source tools provide flexibility and can be customized for specific use-cases. Popular tools like Nmap for network scanning, Metasploit for exploitation, and Burp Suite for web application testing have stood the test of time. However, the landscape is ever-evolving, and staying current on emerging tools is important for maintaining a cutting-edge approach.

Now comes the tool selection phase, where you choose the software and hardware to bolster your testing endeavors. Align your tool choices with the test's scope, targets, and phases, considering your skills and preferences. Ensure compatibility, reliability, and currency of the tools. Opt for a mix of commercial, open-source, and custom-made tools based on needs and budget. Examples of widely used penetration testing tools include Nmap, Metasploit, Burp Suite, Wireshark, and Kali Linux.

The toolkit that will be needed during a pentest depends on the kind of the target (web application, mobile application, server etc.) and, of curse, the preferences of the pentester. Also, although we can say something about the tools that we will use during the beginning of the tests, it is not rare the cases when we need to add a tool or another into our belt as we face opportunities like a probable CVE or challenges like a WAF that need to be bypassed. Finally, it usually helps if the pentester is able to develop its own tools whenever the situation demands it, regardless of programming languages.

This is the step where skill and experience of the team really makes a difference. A more mature and skilled team uses same tools but in the right moment and develop it's own tools and techniques according to their experience

In this open space, consider sharing anecdotes, insights, or examples that don't neatly fit into prior sections. Perhaps recount a unique testing challenge, an unexpected discovery during a penetration test, or a valuable lesson learned from a past experience. This platform allows for the exchange of diverse perspectives and knowledge beyond the structured elements of the penetration testing process.

What often goes unspoken in the discourse about penetration testing is the importance of 'soft skills' alongside technical expertise. Over my years as a CISO, I've come to realize that effective communication, especially during the reporting phase, is invaluable. Translating technical findings into business risks that can be easily understood by executive leadership helps ensure that the insights lead to meaningful action. Another under-discussed factor is the psychological impact of penetration tests on staff. Employees may view it as a challenge to their competence, and hence it's crucial to frame the test as a constructive, learning exercise rather than an audit.

Think about a Bug Bounty to compliment your Penetration Testing program. Not only will this run 'constantly' but it may unearth things outside of your Penetration Testing scope that real world attackers are looking to exploit.

Strangely enough, I would start with looking at my resources (PenTesters and Tooling) in order to determine what an achievable work tempo would be for them and to assess whether I need to engage more resources or not.