What is pen-testing and how does it keep your business secure?

Penetration testing defined

Penetration testing is a form of ethical hacking, where organisations ask a trusted expert to try to penetrate (access) their network and systems.  A penetration test plays a crucial role in identifying vulnerabilities on your critical infrastructure – systems, networks, and web applications. It allows you to identify and fix holes in your security before cyber-criminals and hackers can find and exploit them.

There are three different types of penetration testing: internal and external network penetration tests and web application penetration tests.

External network testing

An external network penetration test replicates the actions of a cyber-criminal or hacker who is attempting to access your systems across the internet, from outside of your network.

Internal network testing

An internal network penetration test replicates the actions of a cyber-criminal or hacker who is attempting to access your systems from inside your network. This might be by hacking your wi-fi, by plugging a laptop into a network port in a meeting room, or breaking into your server room and installing a device.  Depending on the agreement with the penetration tester, they may send one or more people to your premise to attempt to gain physical access through social engineering.  A more cost-effective solution is to send you a small device which you plug into your network which the tester can then access remotely in order to perform the testing.

Web application testing

Web application penetration testing focuses on finding security flaws in a particular application in order to access or change data from the application or use it in order to pivot into another connected system which stores even more valuable data.

Why Penetration testing Matters

There are generally two approaches people take to penetration testing. The first is compliance based.  Compliance-focused organisations need to get the penetration test done in order to tick a box and achieve compliance with a certain regulation or policy.  This may be PCI-DSS,  or the organisations own Information Security policy – often under the security standard ISO 27001.  The risk in the compliance-based approach is that it can foster a ‘do the least work necessary in order to comply’ attitude.

The second approach is security based.  Security-focused organisations see protecting their data and their clients’ data as their primary objective, and any compliance requirements are simply a useful tool to use along the way.

The United States Postal Service has recently fixed a bug in a publicly available API which allowed, for more than a year, users to view and sometimes modify the account details of 60 million customers (Reported in Krebs on Security).  The flaw was a failure to implement basic ownership validation for the logged in user when querying records – a clear example of the OWASP Broken Access Control risk which would have been picked up by any competent penetration tester.

It is not clear why this aspect of the system was not correctly tested, but it is easy to imagine similar mistakes happening in compliance based organisations who take the view that because a certain API or web app is not processing cardholder data it is not required to be penetration tested for PCI-DSS compliance.  A salutary reminder that in the world of cyber-security being standards compliant is not the same as being secure.

OWASP Broken Access Control explained

The OWASP Top 10 list of web application vulnerabilities is a very useful resource which details the most common flaws in web application design and implementation. This list, often called the OWASP Top 10 for short, is updated regularly and is the starting point for any good penetration test for web applications.

The Broken Access Control is a common flaw in many web app and API designs. It works like this…

Imagine a simple web app for an online shop that allows users to create an account to store their address details to simplify checkout information.  When a user visits the web page they will have to login with a username and password – this is authentication.  It proves you are a valid user of the web application.

Once successfully logged in, the user can then view or edit their saved details- and often the URL for the web app will look something like this:

In this example the edit webpage has been called with an id=157 which tells the web app to find record number 157 and display its contents.  The fact that we want to view record 157 was obtained during the login process as it is the record number for the user who just logged in.

So far all is correct, the user has logged in and viewed their own record.

But what happens if the user edits the URL in their browser and changed the record number in the id =157 to another number such as id =256?

The web app will try to display record number 256 on the screen – which belongs to another customer.

What should happen in this situation is: when the web app retrieves record 256 it should check that record belongs to the currently logged in user.  That information is almost certainly in the database – customer account numbers are usually a key in the database. The important question is: does the web app validate this data ownership or not?

A mistake that is often made is to implement security only at the login stage of the web app. Login security asks the question: is this user allowed to login and use the web app.  This process is known as Authentication.

Access Control, on the other hand, asks a subtly different question every time the database is accessed: Is this logged in user allowed to access this particular record?

In the example above, a correctly implemented Access Control function would have returned an error when trying to retrieve record id =256 because it does not belong to the logged in user.

This is the difference between authentication and access control: authentication answers the question ‘is this user allowed to access the application’  whereas access control answers the questions ‘is this user allowed to access this particular record from the database’

The data breach at the US Postal Service was due to broken access control – a malicious user was able to change the record number in an API request and retrieve records that belonged to other customers.

Penetration testing ensures that your web application or API correctly implements access control and other necessary security requirements such as those found in the OWASP top 10.

Vulnerability Scanning and Penetration Testing – the dynamic duo

Penetration testing is an essential part of your organisations cyber defence strategy, in partnership with vulnerability scanning.   Vulnerability scanning is mandated by PCI-DSS (for criteria 11 – Regularly test security systems and processes) and is a sensible and cost-effective choice for any organisation which makes its systems or API accessible to third parties – whether on the internet or across VPN and closed networks.

Vulnerability Scanning is an automatic process which targets a set of IP addresses or domain names in order to identify any known vulnerabilities or configuration errors in the systems that it discovers.  For cyber-criminals targeting your systems, the path of least resistance is usually to identify the systems and software that you use.  Once the attackers knows the software and versions in use, it is trivial for them to identify any known vulnerabilities that they can exploit.  For example, if you run your website on Apache version 2.4.32 and there is a known vulnerability in that software, the first step of any attack is to identify the version of Apache you are running and then attempt to exploit the known vulnerabilities in that version.

How do the hackers know which version of Apache you are running?  Unless you take steps to hide it, webservers transmit information in headers and metadata which can be used to infer the exact version of the software in use.

So you can think of vulnerability scanning as a safe way to duplicate the first stage of a cyber-attack and identity those parts of your infrastructure which attackers are most likely to target.  The benefit of using vulnerability scanning software is that it will tell you, for every server and device that it scans, which items of software are out of date and need patching or are misconfigured and so are revealing more information than they should.

It is good practice to run a vulnerability scan on a regular basis and any identified issues then become the to-do list for your system administrators to rectify before the next scan.

Vulnerability scanning will help identify those parts of your systems and network that are vulnerable to attack and are placing your organisation at risk.  However, the vulnerability scan will not show you the implications of the vulnerability nor will it show you every vulnerability.  This is where penetration testing comes in.

To use an analogy, vulnerability scanning will point out that the lock on your bathroom window is broken whereas a penetration test will demonstrate that someone can climb in through that window, gain access to your office and then steal cash from your safe.

Penetration testing is a human driven process – there is no other way to duplicate the sneaky and intuitive moves of the cyber-criminal than to use an equally sneaky and intuitive ethical hacker working on your behalf.  Although the penetration tester will use a variety of automated tools and scripts to increase the speed and effectiveness of their work, there is also a fair amount of manual work involved as the tester seeks to try every trick an attacker would attempt.

It is important to ensure that the penetration tester you select has both the technical skills and certifications required to assure you of their technical ability and ethical standing before they’re trusted with the work.  Choose only Crest or Tigerscheme  certified penetration testers.

Penetration testers work within clearly defined terms of reference which are agreed with you before the test commences.  This ensures that the test can be safely conducted without risking your live services or data.  When agreeing the terms of reference, it is important to ensure that the scope of the testing will not only meet any compliance needs (such as those mandated by your PCI-DSS level or Information Security Policy) but also include all parts of your infrastructure that are exposed to potential attack.  The penetration tester should offer advice on how to define the scope of the test and after the initial ‘footprint’ exercise will suggest any additional systems and devices they have identified that you may wish to add to the scope of the test.

After the completion of the penetration test, the penetration tester should then provide a detailed report including both an executive summary which explains the business risks and implications of the findings and a more detailed technical section which your developers and system administrators can use to correct the flaws and vulnerabilities identified during the test.   Once the issues have been corrected, the same penetration tester should conduct a second more targeted test to verify that all the issues listed in the report have been fixed.  Be sure to confirm this re-test is included in the service offered by your chosen penetration tester.

Penetration testing is a requirement of PCI criteria 11- ‘Regularly test security systems and processes’ and it will help demonstrate compliance with several other PCI criteria such as 6 – ‘Develop and maintain secure systems and applications’ and 7 – ‘Restrict access to the cardholder data’.  It can also help demonstrate compliance with criteria 10 – ‘Track and monitor all access to network resources and cardholder data’ by showing how the penetration test shows up in the various system logs and can be identified and tracked.

Beware of compliance blinkers

When defining the scope of penetration testing and vulnerability scans, beware of allowing legitimate compliance requirements to blinker your view of other security risks to your business.

For example, if you are a PCI-DSS compliant organisation, you are required to perform a penetration test on the in-scope cardholder data environment at least once per year.  You may also have systems that are outside of the PCI scope, such as an HR system used for payroll, which are also essential for your business. That HR system may have a portal available on the internet where your employees can view their HR records and book holidays.  This system is not in-scope for PCI but it is essential for your business – assuming your staff want to be paid each month.  A compliance based approach to penetration testing would limit the scope of the test to just those systems within the PCI scope – leaving the HR system untested.  A security based approach would also consider other essential systems such as the HR system and add them to the scope of the penetration test, because the objective is to ensure the whole business is secure against attack.

Risk Management and Insurance

Regular penetration testing and vulnerability scanning will give you a high degree of confidence that your systems and network are secure against cyber-attack.  It will provide concrete evidence to your risk management or audit committee that the necessary steps to mitigate the risk of cyber-attack have been taken.

In the rapidly maturing cyber-insurance world, some providers are now offering premium discounts if a regular penetration testing regime is in place. While other insurers may not call out penetration testing and vulnerability scanning by name in the policy, standard clauses, such as this one for Hiscox’s cyber insurance listed on their website, begs the question – is not performing a penetration test going to be considered failing to identify pre-existing problems that you: ‘ought reasonably to have known about.’

What is not covered … Pre Existing problems…anything likely to lead to a claim, loss, breach, privacy investigation, illegal threat or interruption, which you knew or ought reasonably to have known about before we agreed to insure you.

How often should penetration testing be performed?

PCI requires internal and external penetration tests are performed at least once per year and also after any significant change to the systems or infrastructure i.e. the network no longer looks like the one that was tested.  This could include the addition of new web servers, network segments or even a major operating system or web application upgrade.  What can be harder to quantify is the cumulative effect of several minor changes which together now mean the network has evolved enough to justify another penetration test to ensure new vulnerabilities have not been introduced.  An open discussion with your penetration testing partner of choice will help to manage these risks and ensure a cost-effective testing regime is created that suits the precise needs of your organisation.

ISO 27001 control objective A12.6 (Technical Vulnerability Management) requires that ’information about technical vulnerabilities of information systems being used shall be obtained in a timely fashion, the organisation’s exposure to such vulnerabilities evaluated and appropriate measures taken to address the associated risk’.  Penetration testing is a necessary part of this process in order to identify the technical vulnerabilities.  Although the ISO standard does not mandate an annual test, given the rapid rate of change in the cyber-threat landscape, it is hard to imagine an environment where it would be prudent to leave more than 12 months between penetration tests.  Even though your network or systems may not have changed significantly in those 12 months, the state of the art for attacks will have evolved and previously undiscovered vulnerabilities in the system you use may have been identified which can now be exploited.

In summary

Penetration testing is an essential part of your organisation’s cyber defences. It enables you to identify and fix vulnerabilities that could otherwise be exploited by cyber-criminals and ensure compliance with PCI-DSS and ISO 27001 standards.  Because cyber-criminals are continually developing new attack vectors and vulnerabilities are being discovered in existing systems, penetration testing should be conducted on a regular basis even if your network and systems have not changed significantly.