Grey Box Penetration Testing

Types of Testing
There are many variants of testing techniques.  The following list provides a general overview of a few:

• Black box – In this type of assessment, the testers are not given any details about the systems in question.  No credentials, no architectural diagrams.  This type of testing is used to simulate an external attacker with no inside knowledge.
• Grey box – This type of assessment has many definitions to many people.   It is in between black box and white box testing.  In this scenario, the tester may receive architectural diagrams, credentials, demonstrations of the application, communication with the target, and much more. 
• White box – In this type of assessment, the tester is given a lot of information about the application.  This will include credentials, architectural diagrams, source code, and any other information that will help get a full view of the system.  There is nothing hidden from the tester for this assessment.

Grey Box Testing
As I mentioned in the list above, grey box testing is really in a “grey” area in the testing world.   There are no strict constraints on what it does or does not have access to.  In most testing scenarios, grey box testing is the preferred method.  This is because the tester is not given everything (making them really try to break things), while giving him access to more of the application.  Grey box tests can require very little information to perform.  A tester really just needs to know the target URL(s) and have some credentials to access the application.  Additionally, architecture diagrams or other information can be provided if needed.

The Benefits
So why grey box?  Well, there are a few reasons we prefer this method.   First, black box techniques will still be used during the assessment.  So even though a target and credentials are provided, the tester will still perform recon about the target gathering as much information as possible (as if no information was provided).  Attempts will be made to bypass login forms and other access controls without using the credentials.   In essence, you are getting a black box test with the grey box test.  

Second, credentials increase the surface area of the test.  If an application has a great authentication mechanism and no credentials are provided, how much of the application will actually get tested?  Maybe the Login form, maybe some dirbuster attempts, but it could be a missed opportunity.  In addition, this assumes that only external attackers will attempt access to your application.  What about the valid user, who has credentials?  What can they do?   We must not overlook internal users as a source of malicious activity.  By having credentials, the tester has the ability to test the application from a valid user perspective.   This perspective is going to be about 95% of your application’s surface area.  OK, I made that number up, but it should be about everything except for the authentication pages (which there usually are not that many of). 

The common request for credentials is 2 users per role.  Why?  This allows testing authorization functionality with an application.   Can a read-only user access admin-only functionality?   Can one user access data or functionality only available to a different user?   This is the type of testing that gets included when we have these credentials.  This is important as data loss is a big concern. 

Testing as an internal user allows testing business functionality, since there is now easy access into the application.  Most penetration tests are relatively short in duration.  It is not uncommon to see testers given a week or two to test an application.   A real attacker can spend months or years attacking an application.  If the tester is spending 80% of that time trying to just bypass authentication then there is no time to test the rest of the application.  We want our client’s to get the most out of the testing time we have for their applications.

Third,  we have an open line of communication during the assessment to better understand what is going on.  The idea is to analyze the risk the application presents to the company.  In a black box test, we may identify something that we think looks suspicious, but may actually be nothing.  In grey box testing, we communicate with the client throughout the engagement.  If we see something that looks suspicious, we can reach out to the client and discuss it.  This can help remove findings from a report if it is really nothing.  There are occasions where it actually adds findings to the report, but that is good as the right risks are being identified.

Why not White Box?
There is nothing wrong with performing a white box assessment.  This type of assessment will give you the best view into the system and should produce the most valid findings.  It can be much more costly though, or even inhibit the test deepening on the amount of information provided and the size of the application.  If information overload happens, it can be difficult to manage all that information and perform all the testing in the given time frame.   White box tests help get that last niche of attackers, the internal IT personnel (past and present).  This is usually a small group of people that may not warrant the full white box assessment.

Concerns
I have had clients concerned about the resources needed to create credentials for a grey box test.   Depending on the application and how the user roles are set up, this can be time consuming.  One alternative to the client creating all of the accounts is that they can provide administrator accounts and then the tester can create the other needed accounts.

Conclusion
Hopefully, the points made here are clear and you can see why grey box testing is preferred and the benefits it provides.   Sure, there are plenty of instances where black box and white box are appropriate techniques, but it is important to work with the client to understand what they are really looking for.  It is important for Secure Ideas to understand the needs of the client and provide them with the appropriate recommendations. 

James Jardine is a Principal Security Consultant with Secure Ideas.  If you are in need of a penetration test or other security consulting services you can contact him at james@secureideas.com or visit the Secure Ideas – Professionally Evil site for services provided.