Security updates and patches can literally be thing of nightmares for many Systems Administrators. To patch or not to patch – that is always the question. From a security perspective, security patches should always be applied to increase the organisation’s resilience to hackers and malware, but with many organisations lacking IT resources and having ever-decreasing maintenance windows in customer Service Level Agreements (SLAs), patching is very often something which falls by the wayside.

Missing software patches continue to be the most common vulnerability that our consultants identify on penetration tests and it continues to be the easiest way for an attacker or malware to gain administrative access to an organisations infrastructure. The System Administrator’s job is never done – especially when software vendors are discovering security flaws in their products and issuing fixes on a monthly basis. Keeping track of all the software fixes and versions to ensure they get installed in good time can become a full-time job for larger networks.  This article provides some advice and guidelines to help you avoid being overwhelmed and keep on top of your software patching.

Why do we need software patching ?

A “patch” is a new version of an existing software program that fixes coding flaws that are contained in previous versions. No software is perfect and all software contains mistakes introduced when it was originally written or introduced during later enhancements.  If those coding errors can be taken advantage of in order to get the software to do something it was not designed to do this is called a vulnerability.  Cyber-criminals and security researchers are always looking for previously undiscovered vulnerabilities; the criminals want to exploit them, the researchers want to fix them.

Often, the vulnerability that is discovered has been in the software for many years – but has only just been ‘discovered’ now.  While vulnerabilities may have only recently been discovered by security researchers or the software vendor, it is possible that criminals have known about the vulnerability for some time and have been exploiting it all along.

The January patch releases from Microsoft included a patch for Microsoft Exchange Server (CVE-2019-0586), which related to versions going back to Exchange 2013 and up to Exchange 2019.  This particular vulnerability would allow an attacker to create a specially-crafted email and send it to the Exchange server and then, in the words of Microsoft: “An attacker who successfully exploited the vulnerability could run arbitrary code in the context of the System user. An attacker could then install programs; view, change, or delete data; or create new accounts.”

Every time a developer makes a change to their software, there is the possibility that they will make some mistakes in the code or busines logic of the application and introduce new vulnerabilities – even if the software appears to be working correctly and is delivered to customers.

So not only are vulnerabilities being found in old software all the time, each new version of software can introduce new vulnerabilities as well. As a result, vendors issue patches to their software and firmware for their devices on a regular basis – usually once a month or when a critical vulnerability is identified.

Why can patching be problematic ?

This monthly stream of patches can be a problem for System Administrators – especially if the installation of the patch requires a server or device to be restarted as this may cause a service outage for a number of minutes or hours while the affected service comes back online. Also, the patches themselves may introduce changes in the behaviour of the software that causes previously reliable systems to encounter errors; therefore, some form of testing may be required before the patch is installed widely across the network.

Compliance regimes (such as PCI-DSS or ISO27001), require a pro-active approach to patching security vulnerabilities, with system administrators expected to actively evaluate new patches and decide which ones to install.  Even on modest networks, this can be a complicated and time consuming job every month.

Use policy to ease decision making

It is good practice to define a policy that specifies how vulnerabilities are fixed and how patches are applied and managed. Regimes like PCI-DSS, ISO27001 or Cyber Essentials require a Vulnerability Management Policy to be in place. This will define the various categories of patches which are to be installed and a process to identify vulnerabilities that have been announced but no patches yet exist. Vulnerability scanning is an excellent way in which businesses of all sizes can identify vulnerabilities in their network infrastructure and feed the results of their vulnerability scans into their patching cycle.

Vendors categorise their patches when they are released, with Microsoft being a typical example of this.  Microsoft publish their new patches on the first Tuesday of each month (commonly known as Patch Tuesday).  Each patch is assigned a category based on the level of risk it presents to an organisation if exploited by an attacker. Patches are categorised as Low, Moderate, Important or Critical. Systems Administrators should decide, based on their business needs and risk profile, which level of patch should always be installed.  For many organisations, Important and Critical is a suitable choice for patches that should be installed as a matter of urgency.

How to schedule patching to minimise risk to critical systems

One approach to manage the monthly patching cycle more efficiently is to use network segmentation and automation.  A good design practice for computer networks is to segment the network into different subnets (areas) that reflect the relative value and risk of the systems and data on each subnet.  A similar approach can be used for patching. Patch the lowest value systems first and then gradually working across the network applying the patches to increasingly higher value systems.  This means that by the time the patches are applied to the highest value systems, the patches have been in use on other servers for a couple of weeks and there is a higher level of confidence that there will be no unanticipated side-effects.

Since most vendors release patches on a monthly cycle, a 4 week roll out schedule can be helpful.

For example:

For lower value servers – perhaps weeks 1 to 3, patches can be configured to be applied automatically in accordance with the organisation’s Vulnerability Management Policy. For the highest value systems, you may need to apply patches manually in order to avoid service outages (e.g. servers may need to be temporarily configured out of processing pools to be patched and then returned to live service).  This approach means that by the time patches are applied to the highest value customer-facing systems, they have been in use on development and testing systems for 2 or 3 weeks giving the opportunity for any problems to be discovered.

Use patch management applications to protect network bandwidth

Some patches can be quite large in size, running to many megabytes (and sometimes gigabytes) in size. For large networks, to have dozens of servers and hundreds of desktop devices all downloading the same large files at the same time can seriously impact the available network bandwidth and possible incur large data transfer charges if you have metered connections.  Using a patch management system such as the Windows Server Update Service (also known as WSUS), can help significantly by downloading a single copy of the required updates and then hosting it locally within your network for your other devices to download and install. It is also recommended for organisations to invest in 3rd-party patching applications which can be deployed across an entire infrastructure to ensure applications like Oracle Java, Adobe Reader, Microsoft Office and other typical business applications are kept up-to-date.

Patching for desktop devices

Patching for desktop devices is equally important in order to protect the network’s integrity but is often hard to do as end-users can interrupt the installation of the patches by clicking Ignore or Cancel on any confirmation prompts if given the opportunity.

With Email emerging as the primary attack vector for cyber-criminals against businesses, it is vitally important that every user’s computer has up to date security patches installed.  Consultancy practice Proofpoint recently published a report claiming 91% of targeted attacks start with an email containing a combination of links to a phishing website or malware within an attachment.  The malware in the attachment can only function by exploiting vulnerabilities present on the users workstation, so a key defence is to reduce the number of vulnerabilities by ensuring the latest security patches are installed on every PC and laptop throughout the organisation.

An effective patching strategy for desktop devices requires three things:

• Automation
  Automatic downloading and installation of available patches on a regular basis will keep most of the fleet of devices up to date. WSUS can do this job for Windows devices.
• Education
  End-users need education and regular reminders to allow patches to install when prompted on their device. The same education sessions can also teach your users how to spot and avoid emails containing phishing links and malware loaded attachments.
• Reporting
  System Administrators need a reporting system so they can easily monitor the number of devices in their estate which have the latest patches applied to them and identify any devices which are missing patches and take steps to rectify the situation.  WSUS provides a ‘Missing Patches’ report which is a great way to view a snapshot of the devices that are awaiting security updates to be applied to them.

Executive and VIP users – patching even more important

In many organisations, the greatest challenge can come when trying to ensure patches are installed promptly on the devices of senior executives and VIP users.  It may be tempting to leave them to last or even wait for their monthly call to the helpdesk to ask for their password to be reset and then deal with the patching backlog.  However, consider that your senior executives and VIP users are the highest profile users in your organisation and are therefore the most likely to be the target of a spear-phishing attack aimed at them as individuals.  Given that the primary attack vector against these users is by email and malware loaded attachments, it is important that these user’s device are as up-to-date as possible to provide the greatest protection.  Consider prioritising these users to be patched first each month or provide a valet service and visit them in person each month to install the updates during a suitable long lunch break.