4 Pentest Paradoxes You Must Get Right

October 25, 2021
Ken Ballard

Penetration testing is the best way to know whether your network or applications have exploitable vulnerabilities. A pentest goes beyond a simple vulnerability scan to determine if a vulnerability can truly be used by a skilled hacker. It is a very real, in-your-face test, which is why every best practice and security regulation requires pentesting done at least annually.  

But, finding a good penetration tester can be a project all by itself.  A good penetration test is a mix of skill, creativity, budget, and rules of engagement.  For too many companies, they feel like finding a penetration testing company is a real gamble.

To help you find the perfect balance(ideally with Security Pursuit), we’ve identified 4 paradoxes or dichotomies that you have to get right.  Get them wrong and you are likely to be disappointed in the results.

4 Penetration Testing Paradoxes

Paradox 1: Creative But Standardized

People who become ethical hackers and engage in penetration testing as their career tend to be quite methodical and systematic. That alone is a required skill, whether you are a professional pentester or a real hacker. One neuro scientific study into the mind of a hacker found a strong positive association between an individual's drive to build and understand systems and their hacking skills. When evaluating penetration testers, it’s vital to seek out professionals who pride themselves on applying best practice techniques using a systemized approach.

A strong degree of standardization is particularly relevant in cases where pentests are mandated by regulatory compliance.In such cases, you need the testing process to stand up to an audit as a solid test that demonstrates compliance. In such cases, a pentest comes down the the final report, which must convince the reader that the pentest meets expected quality standards.

The paradox here is that you also want creativity.  That creative spark is what keeps real hackers ahead of the industry. The best pentesters don’t just follow established processes and checklists—they also allow for room to adapt to the structures and intricacies of your network and find new vulnerabilities to exploit. This is almost always done on the fly, as opportunities are discovered.

The best pentesters can see a vulnerability, quickly determine if/how to exploit it and penetrate your system within the rules-of-engagement.  The skills you seek are not all technical. They are also creative. 

Questions to Ask:

·      Tell me about your process from beginning to end.  From engagement to reporting.

·      What best practices to you like to rely on?

·      What are some creative things you’ve done in a pentest?

Paradox 2: Automated But Manual

Without automation, you would be paying a pentester to manually look for every known vulnerability. It would cost a fortune!  This is why penetration testing is so heavily dependent on automation. It’s also why hackers use the same tools.  Automation tools are used to rapidly scan network systems and applications for vulnerabilities and even run exploits (if you let them).  Artificial intelligence is often incorporated into pentesting tools to allow them to learn and get better over time as they learn and improve from different simulated attacks.  

From the perspective of your business seeking to evaluate a penetration test company, automation makes the process much more efficient and gives more value for money. It’s worthwhile from a business standpoint to get testers who leverage automation as much as possible.

The paradox here is that despite the undoubted benefits of automation, fully automated testing is a bad idea. For one thing, letting a tool automatically run an exploit can cause serious damage to your systems. Human manual confirmation followed by careful testing is often the safest approach.

In addition, humans are still better at seeing the nuances of a unique situation. They can combine several vulnerabilities into a successful compromise. Depending on your scope, your pentest may also include social engineering, cross network layers, and leverage things that a tool simply can’t.  After all, when genuine hackers infiltrate your network, there are real people conducting those attacks and they dynamically adjust their techniques based on what they see on your network. No level of automation can realistically aim to replicate the human element of hacking.

Questions to Ask:

·      Tell me how you use tools to find vulnerabilities

·      Do you let tools exploit

·      How and when do you manually test a vulnerability

Paradox 3: Aggressive But Realistic

Companies do pentests because they want to thoroughly test their systems, service providers and staff.  Otherwise, what’s the point?  As your security improves, you want to test more aggressively, pushing your security controls, people and partners more and more.  Because of this, a penetration test is central to continually improving your security. 

Plenty of pentesters have equal or better skills than serious hackers. Obviously, criminals don’t work according to any rules you may give your pentest provider. So, be careful putting too many specific constraints on your vendor. Hackers have no remorse for the damage they cause, and they won’t be gentle. Your penetration test should emulate that aggressive attitude as much as possible.

The paradox here is that you need the test to be realistic. You want to simulate what a hacker would do.  Hackers look for opportunities to exploit, but they don’t take every opportunity to the absolute edge of reality.  When a vulnerability becomes too hard to exploit, they move on and try something quicker and easier. And, so should your pentester.  When viewed through this lens, it’s important that you pentester has rules of engagement and doesn’t just test vulnerabilities because they are there or because they are interesting to them.

Questions to ask:

·      What rules of engagement do you like to operate under

·      What’s the most underhanded thing you’ve done during a pentest

·      How do you balance time with exploiting a time-consuming vulnerability?

Paradox 4: Cheap But Good

Everything in cybersecurity is constrained by budget.  This is why you have a budget and scope for penetration testing.  Even when you have a line item for penetration testing on your budget, your overall budget is constrained. It can be tempting to go cheaper on your pentest and use the money to fund something else. It’s tempting to think of pentesting as a commodity service and go with someone cheap.

The paradox is that a good pentest will cost a decent amount of money.  It’s a skill issue, and it’s a scope issue. Penetration testers are often among the highest paid professionals in cybersecurity because the skills are so difficult to master. Also, running a vulnerability scan, then poking at the system for 30-minutes probably isn’t going to reveal much. But, spending a lot of money doesn’t guarantee a good pentest, much less good value.

When you look at vendors, you will get a feel for a good budget.  Some pentesters will be cheaper, some will be more, but you should clarify the scope and then question how they arrived at their fee.  

Questions to ask:

·      Given our scope, what do you think a good budget is?

·      You seem cheaper/more expensive than others.  Can you explain why?

·      What aren’t you doing in this pentest that you think would be appropriate?

Security Pursuit Penetration Testing

Security Pursuit offers a complete suite of Penetration Testing services.  We are excellent and can help you balance a thorough test that meets your needs and budget. Email us today at

join our email list