Should penetration tests be conducted outside business hours?

Penetration testing comes with a degree of inherent risk, which may prompt the question of whether it is better to do the test after hours instead of during business hours (normal operating or work hours).  Both options have advantages and disadvantages, but the short answer is that the quality of testing is almost always better when performed during business hours. Let's break down the implications of both options, so that you can make an informed decision about whether your tests should be conducted during normal operating times.

In Favor of Testing After-Hours

The main argument for after-hours testing is that the business is less likely to be impacted by the testing activity. While professional penetration testers will use their best judgement to minimize the likelihood of a production outage, it is undeniable that the risk of such an event still exists. If the tester finds that they have caused an issue during the night, they can attempt to alert an on-call point of contact. This approach will provide a window of time for the business to fix any issues before the business is up and running on the following day. This also means that as an organization running an after-hours test, you also need to have a point-of-contact that would be available to the penetration testers in these off-hours. Failing to provide one will negate the intended benefit of after hours testing.

In Favor of Testing During Business Hours

By testing during business hours, the security, infrastructure, development, and operations teams are better positioned to respond to an outage promptly.  In many organizations, a production outage is more likely to be detected during the day, whereas a production outage occurring outside regular work times may not be detected until the following business day.  Furthermore, testing during business hours will provide an opportunity for the business to collaborate with the tester in real-time, which can make it much easier to correlate specific testing activities to production issues. This also means that the primary point of contact is keeping normal work hours, which is less disruptive to their other duties and work-life balance.

In addition to leaving the business better-equipped to detect and respond to any issues that occur, daytime testing will provide a higher quality test. The testers will typically be better rested and more alert. This makes them better able to notice important details. There are also a number of attacks that depend on the activity of other accounts on the network. The majority of these attacks are used to move between systems on the network.  Scheduling the test during routine business activities provides this type of traffic in a natural and organic way. When testing is performed after everyone else has stopped for the day, the penetration testers are being artificially limited, which can slow down their progress. One more factor is that if the testers lose access, for example by locking out their account, they are unable to proceed until their access is restored.  This can be fixed quickly during the day, with a responsive support contact.  A tester working at night will often be impacted for the remainder of that night's testing time until access is restored the following day.  Since testing is usually either timeboxed or billed by the time allocated, that loss of access will be a costly mistake that lowers the value of the penetration test to you as a client and should be avoided.

To Summarize

A penetration test performed during business hours will allow the tester to move faster and smarter, which facilitates a better test. Even if there's a system in the engagement scope that is known to be particularly fragile, it is better to test during business hours. For the fragile system, the best option is real-time collaboration directly between the tester and the appropriate technical leads from the business. This will allow them to coordinate careful tests, it makes it possible to avoid attacking during events like scheduled batch jobs, and it positions the business to respond to any issues quickly. The business certainly has the right to insist that testing is scheduled for after hours, but that is rarely the best way to mitigate the risk posed by the testing activity.