Create a Security Operations Center on Your Campus

Higher education institutions have a reputation in the Dark Net (frequented by hackers) as having soft, gooey centers perfectly suited for easy mass exploitation. The types of abuse we experience as a result include constant phishing attempts, compromised accounts and systems, illegal downloading of journals, abuse of virtual private networks (VPNs), hijacking of our networks to send spam, unapproved hosting of spamvertising (illegally placed advertising on legitimate sites), use of our systems in distributed denial-of-service (DDoS) attacks, and theft of our intellectual property. In dealing with all these abuses, cybersecurity professionals in a higher education institution must also find a balance between privacy, security, and efficacy while remaining sensitive to the political culture of the institution.

Higher education will continue to be targeted because it has large volumes of public address space, high bandwidth, and valuable information combined with an often weak security posture. In addition, most institutions are highly decentralized, freely open environments catering to research, and subject to political pressures that can prevent good information security practices — all substantial challenges to security. Fortunately, the federal government and many higher education leaders have recognized that colleges and universities can no longer ignore these threats.

This article shares a strategy to pursue, plan, implement, and execute creation of a formal security operations center (SOC) for a higher education institution. The goals are to address cybersecurity operational needs, minimize costs related to cybersecurity, and protect institutional assets using centralized IT security and computer network defense capabilities.

Why Have a SOC?

Campuses should invest in a dedicated central IT SOC to help the institution's IT staff respond to pervasive and growing cybersecurity threats by

• identifying and tracking institutional assets and data,
• identifying and neutralizing cybersecurity threats and vulnerabilities,
• defending and protecting critical assets, and
• providing cybersecurity metrics to the institution and leadership.

These capabilities help ensure the confidentiality, integrity, and availability of systems vital to your institution's mission. They also help you address both quantitative and qualitative factors such as protecting your institutional reputation and individual privacy.

Where to Start

Before dedicating more resources to a SOC, it's vital for cybersecurity professionals to inform leadership of the amount of time, effort, and money already spent by their institutions on cybersecurity issues. Some institutions might have experienced a costly data breach and notification process, reputational hit, or the loss of availability (and productivity) of critical systems and consequently have these metrics readily available. Institutions that have not experienced these interruptions and paid for mitigation might be less hesitant to dedicate additional resources to prevent and detect these types of activities. In either case, both scenarios begin by gathering cybersecurity metrics with the goal of determining if the cost of owning and operating a SOC is less than the costs associated with cybersecurity incidents for the institution.

All About the Metrics

One source of metrics, vulnerability scans, identify vulnerabilities in institutional systems. You can use open-source tools or pay an external vendor and coordinate a mass scan. Additionally, identifying a summary of cybersecurity incidents and the number of staff hours they've occupied provides a good view into the time lost by IT professionals. As for the network costs, every malicious byte traveling down those leased Internet connections costs money.

Other metrics include time spent dealing with compromised accounts, phishing, and spam messages. Identifying these numbers requires investigative work, or identifying and contacting the correct persons and compiling the numbers for management. With a source of metrics most institutions can quickly identify most costs that can be saved by having a central security operations group to mitigate those costs.

Data Is King

Before a SOC can function properly, they must know which data and systems to monitor and protect. This requires significant effort to identify and classify data and systems, as not all warrant the same protection levels. Data classification helps drive risk-based decisions and allows data owners to identify the systems holding and processing sensitive data that warrant protection. Other systems do not hold sensitive data but are critical to an institution's mission because they have high availability requirements. A few examples of sensitive data include research data, controlled unclassified information, HIPAA, payment card industry (PCI), protected health information (PHI), and FERPA data, as well as any data in enterprise credential stores or shared fate systems. A few examples of systems with high availability requirements include industrial control systems (ICS), identity and access management (IAM), mail, domain name system (DNS), web servers, network time protocol (NTP), financial, communication, and safety and health systems.

Metrics and Data Underpin a SOC

With metrics gathered and systems and data classified, the purpose of a SOC becomes more apparent. A SOC is a central authority that knows the threats to an institution and which systems and data are important. It has systems, procedures, and trained resources to respond to cybersecurity threats. SOC staff know who to contact in the event of security incidents, what threats to share with IT stakeholders in the institution, and work needed to prevent and detect threats to minimize the costs of cybersecurity incidents. A SOC functions much like a security guard for an institution's systems and data, keeping or kicking threats out, and decreasing costs to IT stakeholders and the institution so they can focus on their missions. The SOC also provides information related to compliance and risk groups at an institution. A SOC can provide a central point for pooling the knowledge and information from subject matter experts' expertise and providing value by sharing which threats are valid for their institution.

Host or Outsource a SOC?

Decentralized environments in an organization might view IT security responsibilities as someone else's problem or as a secondary responsibility. Without a central authority to claim them, risks can go unchecked. While considering a SOC to address these risks, institutions may find it advisable to outsource SOC functions to a managed security services provider (MSSP). Despite the benefits, many institutions lack the security program maturity and knowledge to outsource these tasks successfully. In either case, before outsourcing, the institutions will have to perform the same tasks to create a SOC as needed to hire an external provider.

Start Small

Start by defining the SOC's mission, charter, objectives, and responsibilities. To begin, the SOC's primary mission is to identify and neutralize vulnerabilities, and to monitor and detect security events from logs sent to tools such as a SIEM by critical assets. The SOC is also responsible for maintaining and managing the SIEM and its related components. Additionally, the SOC may be responsible for managing vulnerability scanners, the file integrity monitoring systems, and any related log systems. The SOC should track and manage adverse security events, and security incidents or threats affecting all stakeholders on the network, and assist when their expertise is requested by stakeholders.

A SOC is responsible for defining and communicating SOC responsibilities and roles and maintaining the confidentiality, integrity, and availability of systems under their direct control or through partnerships. The SOC should coordinate and collaborate with units external to themselves whose systems are identified as critical systems to ensure that these systems meet equal or greater cybersecurity protections comparable to those protections offered by a SOC. The SOC should maintain a registry of systems and assets and audit the registry on at least a biannual basis to ensure their accuracy.

The SOC should be granted authority through the appropriate leadership channels, such as from the information security manager, chief information security officer, chief information officer, chancellor, and vice provost. The authority granted allows the SOC staff to engage faculty, staff, students, and affiliates (such as directors, deans, chancellors, provosts, campus unit IT Staff, student organizations, students, unit leaders, and managers) and to provide guidance, request information related to IT security events, specify security controls and processes to implement, and ensure that adverse events are responded to in a timely manner. The authority does not permit the SOC to specify technologies, people, or processes used to meet these goals. Units should be encouraged to use their own set of security controls and processes if the protection is of equal or greater levels to those that can be provided by a central group.

The SOC staff should have well-defined working hours, such as all business days, five days a week, Monday through Friday, between the hours of 0800 through 1800 excluding holidays and campus closures. Because adverse security incidents do not follow staff hours of operation, the staff should have at least one member on call who can escalate issues to the appropriate channels. The SOC should evaluate hours of operation on a yearly basis to determine if they are sufficient to meet SOC objectives and mission. SOC staff should be housed in a single secure location where they can readily communicate and collaborate during critical security incidents. The staff will work with relevant security systems housed in secure locations, with accurate documentation of systems' locations.

Organization and Structure of the SOC

Because the SOC monitors critical systems for a (presumably) large organization, it belongs in a physically secure SOC facility with additional physical protections, where reasonable. This might mean officers/guards, building alarms, CCTV, controlled physical access, strategically placed lighting, vehicle barriers, and numerous other security considerations. However, the core systems and assets initially being monitored might not require such a significant financial commitment, and space and funding issues often make addressing some of these risks too challenging. You can start a SOC with a few dedicated core people and look into incorporating trusted student assistants interested in IT security.

The SOC often performs all three critical roles (control, monitoring, operations), which are usually handled by different units of people. A new SOC should focus on the state of the institution's security with compliancy testing for campus PCI units and units subject to state, federal, or regulatory IT security compliancy cases (such as HIPAA, FERPA, PHI, etc.). Ideally it will conduct penetration testing, vulnerability testing, and access control list reviews for enterprise systems. As for monitoring, the SOC should focus on events and the appropriate responses, with log monitoring in some sort of log central collector, often SIEMs, along with their administration and incident response. Finally, the SOC should assist institutional units responsible for operational functions focusing on operational security administration, such as identity and access management and key management (SSL certificates), providing best suggestions, default configurations, central firewall administration, and any other historic roles related to IT security operational functions. The initial SOC will primarily focus on monitoring and control and work in tandem with, rather than independently of, the network operations center (NOC) to maintain separation of duties.

The recommended structure for the personnel within the SOC builds onto the existing structure of an existing IT security team. Typically it exhibits a hierarchy beginning with a chief information security officer (CISO) and/or information security officer (ISO), full-time security personnel, and student assistants. Effectively implementing a SOC requires granting it the authority to perform its functions through written policies from the CISO and ISO and other required leadership approvals. Also important are communications to the stakeholders who would interact with or might be contacted by the SOC.

SOC Communications

SOC communication must occur on three fronts: intra-institutional communication across any central IT groups, internal communication within the SOC group, and external communication with outside parties and other stakeholders. Since the SOC will be monitoring critical systems under the direct or indirect control of sometimes unknown or unavailable individuals throughout the institution, a broad communications plan is necessary in order to bridge the divide between detection of an adverse event and communication with relevant parties.

Whichever communication method you already use, document the process and determine how to improve it. Traditionally, personal contacts have served as the point of contact for incidents related to an IT system or network. Lists of IT contacts might include names, e-mail addresses, unit names, and network IP ranges for people who perhaps have claimed ownership of a contiguous block of the institution's IP addresses. Document a method for all parties involved to use when communicating following identification of an adverse event. Sometimes the assets are owned and operated by faculty, graduate students, staff, or undergraduates who are the only authorized system administrators of the system(s) in question, and they should be included in your documentation.

Creating a formal communication plan using specific security points of contact offers one way to implement and communicate with stakeholders regarding computer security events and incidents. These communication structures are sometimes the most difficult part of running a SOC and can hamper effective performance of detection and remediation steps. As an example, to identify events might require finding and communicating with the data custodian or system administrator for a system in an academic unit demonstrating adverse events, and they will need to know how to reach the person who can physically interact with a system.

External communications include obtaining and sharing threat information or threat feeds with the technical community, other higher education organizations, or computer security–related groups, specifically other higher-ed teams, EDUCAUSE, REN-ISAC, the FBI, health centers, SANS, vulnerability scanning groups, secure configuration assessment groups, and vendors whose tools your institution uses, as well as other external law enforcement and security businesses. The IT security team must participate with these groups in order to stay apprised of common threats to higher education or to disseminate relative threat information to the campus community, and also to maintain a community of sharing and openness with your stakeholders and industry practitioners.

Because each college and university has its own technological culture and varying levels of acceptance of suites of technologies, some threats are more serious to one institution than others. For example, if you have a high user base of a particular web server technology for which a new vulnerability is currently being exploited in the wild, then you need to immediately disseminate this information to the campus technical community. The IT security team should use various e-mail lists and official communications channels to share information with the campus community. These e-mail lists — documented and maintained for all SOC members to access — should be reviewed periodically for accuracy. Additionally, it helps to have a specific e-mail list for the campus to use to report security-related incidents or adverse events to the SOC team (for example, [email protected]).

Incident Response Procedures

The SOC will require well-established incident response procedures in order to succeed in its mission to identify, detect, and stop malicious activities on the campus network. Part of the incident response procedure is identifying what classifies as an incident or an event. This is also important for identifying metrics used to report and learn about the types and frequency of attacks affecting the network in order to invest in technology and establish processes to address the most severe and critical attacks. For events and incident classification you could use the NIST classifications based on NIST Special Publication 800-61 Rev 2.

Some common university computer security incidents include:

• Automated notifications sent to POCs for systems under their care.
• E-mails sent tricking a user into opening a virus-infected file that results in a malware package executing.
• Phishing or spear-phishing e-mails sent with the intent of tricking a user into giving their credentials by clicking on a masked link to a website that looks similar to a university login page in order to harvest their credentials.
• Use of botnets to perform DDoS attacks against web or other electronic resources.
• Compromised account credentials used to send spam using university resources.
• E-mails sent with masked links exploiting vulnerabilities in a web browser or web browser plugin to infect a system and communicate with external hosts.
• A virus executed via trickery through e-mail or carefully placed removable media, which results in an attacker obtaining or encrypting data and holding the data ransom (ransomware).
• Data leaking through legitimate software used in an illegitimate manner due to vulnerabilities, exploits, or misconfigurations.

The SOC team should maintain updated incident response procedures on the chosen documentation website for the various types of incidents that affect the campus. These procedures should include the processes, people, and tools needed for identifying individuals involved with adverse events or incidents, the tools used to investigate incidents, and the processes required to obtain, maintain, and create an effective incident response procedure.

Monitoring Processes

SOC monitoring processes should be updated and maintained in standard operating procedures. Monitoring includes functions such as threat monitoring, adverse activity detection, and reviewing relevant threats. The first, and primary, source of threat monitoring is for SOC members to log in and review incidents in your chosen tracking system. The ticket tool used by a SOC allows staff to triage, track, and process incidents. It also allows analysts to  process, act on,  identify trends , and create reports related to security events, along with incidents observed by the institutional community. In some cases a ticketing system might be recommended for reporting a security incident, or perhaps routing incidents through an existing IT help desk. To aid in collecting accurate metrics, all incidents, adverse events, and SOC-related investigations should use the chosen system unless otherwise instructed by campus legal counsel, the CIO, the CISO, or another authoritative party.

A SIEM system configured to collect, normalize, aggregate, and apply severity ratings to the logs sent by assets is the second monitoring source and a key tool for a SOC. These logs can be used to correlate events across multiple log feeds from various sources such as firewalls, operating systems, and authentication systems, and then notify security analysts when specific use cases are observed (such as brute force attacks, spam, compromised accounts, or attempts to attack critical assets). The SIEM should be monitored or configured to identify or report incidents to all team members.

The third monitoring source includes reviewing e-mails sent to [email protected], [email protected], [email protected], [email protected], and the other e-mail lists commonly or optionally used by IT security staff. The primary e-mail addresses used by the SOC for computer security incidents should be commonly known and communicated.

Investigation Process

SOC-specific investigation processes should be created, updated, and maintained in all agreed areas, such as the standard operating procedures. The SOC investigates any reported adverse events, security incidents, abnormalities in traffic or logs from critical systems or mission-critical units' systems, and any other noted or reported incidents that could affect the reputation, confidentiality, integrity, or availability of systems at the institution. For the most current processes and procedures, SOC members should refer to the agreed repository.

When multiple events are observed and require investigations, critical systems and mission-critical units' systems must be a priority. To support these investigations, the SOC must maintain a relationship with the NOC and should have access to NOC tools such as network flow systems, NOC systems, DHCP or DNS systems, and other core networking systems. During an investigation SOC members must leverage this relationship to ensure that they do not perform adverse actions which could exacerbate any problems affecting the campus' network infrastructure, core DNS, and network flow systems, among others. Investigations requiring access to tools maintained or primarily used by other parties must include the key staff who maintain these tools, including investigations involving nondisclosure agreements, legally binding agreements, or law enforcement.

Leveraging an organization's existing tools can challenge SOC staff at the start. However, a SOC will find network flow information invaluable and should filter it to be actionable and usable for traffic analysis in investigations. Also, include analyzing SIEM log analysis, local log analysis, identity and access management system access logs, system administrator and subject matter expert knowledge, and other stakeholders' knowledge of the systems identified as involved with adverse events or incidents. The SIEM log analysis is the primary investigative tool used by most SOCs, followed by network flow analysis and then specific system and application access logs. In cases where log entries are discovered through normal tools, but more information is required to assist in an investigation, the SOC should work with system administrators and may request they inquire into application-specific logs or system logs or configure them to feed into a central logging solution.

Notification Processes

The notification processes define to whom, how, and when the SOC sends notifications when adverse events or security incidents are identified or reported to them. If possible a registry of important systems, data, and regulated data should be created and points of contact kept up-to-date for efficient notification.

A registry of systems and owners can be a website used to fill in assigned roles and responsibilities; function as a register for computing assets, networks, and data; and allow choosing a single point of contact. This system can run in tandem with existing contact methods until judged mature. The registry would include information such as an asset name, IP address or IP address range, IT contact, and expected level of data sensitivity according to chosen categorizations. It would include a system owner and an alternate contact person.

To begin, describe your current communication model, and then decide how to transmit it to a network of assets and people who handle those assets. Notifications sent from the SOC should use a single format or template to notify parties of adverse events and security incidents. E-mails should be signed, preferably. To familiarize the community with new notification methods, maintain standardized formats in the SOC team's documentation space and share them with the community. Define how you will communicate, such as by sending e-mails from a specific signed address with S/MIME or PGP. Call contacts from a specific phone number. Define how notifications can be sent to the SOC team for inquiries or to report adverse events or security incidents. Use whatever means of communication is most convenient for all parties.

Notifications sent by the SOC should include as much relevant information as possible. For example, if adverse events are observed in traffic, then the relevant traffic flow logs should be included: the timestamps, IP addresses involved, ports used, and media access control (MAC) address, if available. At a minimum, notifications should include the observed logs that generate the notification, and must have a timestamp and relevant usernames, IPs, or MAC addresses to allow the SPOC to identify the system. In certain cases, such as VPN concentrators, the SOC may have to work with the NOC and the sole point of contact to track down the system observed in adverse events. If possible, use tools such as endpoint agents to assist with identifying the source of a system.

Reporting Processes

The institution should have some form of documentation with requirements for reporting security events to and from all stakeholders. These reports will contain information regarding the number and types of security incidents, what actions were taken, where additional measures should be taken to increase IT security based on trends and patterns observed across campus, and provide metrics to senior management to aid in budgetary decision-making processes. In lower level reports specify a method of normalizing the data. For example, you can use the Vocabulary for Event Recording and Incident Sharing (VERIS) framework. Reports will be tailored to reflect the different audiences that will receive them. These audiences include the CISO, information security manager, CIO, chancellor, provost, college deans, division deans, IT directors, ISOs, and IT professionals throughout an institution. Include report processes in your documentation to evaluate the usefulness of the reports.

Prepare to Be Ready

The most important factor for any institution is the need to be ready to face increasingly complicated and more debilitating threats. A SOC can provide the central expertise and coordination functions not typically available in higher ed. Properly implemented, a SOC meets the institution's security needs to address, recognize, and prevent cybersecurity threats in a cost-effective and efficient matter. Today we have more resources available for colleges and universities and other institutions who manage their own SOC or who have outsourced functions for a SOC, some of which describe how a SOC can be implemented and function. I welcome feedback and suggestions to improve on the ideas in this article; contact me at wafischer (at) ucdavis.edu. Continual improvement should always be a goal of any unit or organization. Best wishes with your SOC, and good luck in the cyber world!

---

Wayne Fischer is a senior information security analyst in the Information Security Office at the University of California, Davis.