Building Secure and Reliable Systems

Heather Adkins, Betsy Beyer, Paul Blankinship, Piotr Lewandowski, Ana Oprea, Adam Stubblefield

O'Reilly Media, Inc.

Security and Reliability are not features which users would request, but they should be top priorities for any organisation.

When designing for reliability, you need to assume something will go wrong at some point, the systems should either fail safe (default to open), or for security issues, fail securely (closed off). In a fire, you want your smart door to fail and remain open so you can escape, if someone is trying to break into the building they could start a fire to open all the doors. These could be seen as two sides of security and reliability.

Both security and reliability are concerned about:

  • Confidentiality
  • Integrity
  • Availability

Both are also emergent properties, which should be “built in” not “bolted on” as my security professor would say. While it’s difficult to predict future security catastrophes, you can try to understand your adversaries’ motivations beforehand. These could be motivated by fun, fame, activism, financial gain, coercion, manipulation, espionage, and destruction. The attacker could also fall into different profiles of hobbyist, researchers, governments and law enforcement, activists, criminal actors, or insiders.

The strategies to consider for these risks are:

  • Least privilege: granting the fewest privileges necessary to perform the job duties.
  • Zero trust: design automated or proxy mechanisms to prevent broader access that could cause harm.
  • Multi-party authorization: ensure more than one person is required to authorize sensitive actions.
  • Business justifications: require employees to document reason for accessing sensitive data or systems.
  • Auditing and detection: reviewing all access logs and justifications.
  • Recoverability: the ability to recover systems after a destructive action.

To prevent humans from making mistakes or internal attacks, you can use safe proxies such as requiring every change in production to be made by automation (instead of humans). Instead of talking to the systems directly, the users talk via a proxy which checks permissions and executes the commands. The proxy also logs all requests and commands, to allow auditing. Rate limiting can also be applied to the proxy to limit the blast radius of a mistake, or include a manual approval service by another group of humans.

human (or) automation -> proxy (logs) -> approval (optional) -> executes actions on target system applications

A Breakglass mechanism can also be included to help recover from emergencies and bypass the proxies or authorization system, the word comes from the term “break glass in case of emergency.”

When designing features or products, you might need to make design trade-offs to ensure the system is secure and reliable. You will need to find a balance between functional or feature requirements that solves the user problem, and non-functional requirements which might include controlling access to restricted data, uptime, and even developer constraints such as development efficiency and deployment velocity. The software architecture should also consider:

  • Scalability
  • Redundancy and reliability
  • Dependency considerations
  • Data integrity
  • SLA requirements
  • Security and privacy considerations

Because security and reliability need to be “built in” the system must be designed with these concerns in mind. The best way to solve these issues might be to never hold sensitive user data in the first place, and instead rely on third-party services to handle payment processing, user authentication, etc. which might have better safety mechanisms built in.

While these trade-offs might first appear in direct conflict, with careful planning and implementation you can create systems that reduce risk, and reduce the lifetime cost of maintaining the systems. Once the system is live, reliability and security are no longer optional… and adding these requirements on later can create significant difficulty.

The early history of the internet, and the design and evolution of the underlying protocols such as IP, TCP, DNS, and BGP, offers an interesting perspective on this topic. Reliability—in particular, survivability of the network even in the face of outages of nodes and reliability of communications despite failure-prone links—were explicit and high-priority design goals of the early precursors of today’s internet, such as ARPANET. Security, however, is not mentioned much in early internet papers and documentation. Early networks were essentially closed, with nodes operated by trusted research and government institutions. But in today’s open internet, this assumption does not hold at all—many types of malicious actors are participating in the network.

And so, we have numerous security issues across the entire internet which cause widespread trouble.

The next chapters cover different aspects, so we’ll go through them one by one, these are:

  • Design for Least Privilege
  • Design for Understandability
  • Design for A Changing Landscape
  • Design for Resilience
  • Design for Recovery

Design for Least Privilege

The principle of least privilege says users should have the minimum amount of access needed to accomplish a task, regardless of whether the access is from humans or systems. Having unnecessary privilege leads to a growing surface area for possible mistakes, bugs, or compromise. This is also important because although we might trust the humans on our team, their accounts might be compromised, giving access to some unknown party.

Another design principle to incorporate is zero-trust networking – in which the user’s network location doesn’t grant privileged access. The same can be said about zero-touch interfaces, in which proxies are used, or that humans only have access to production environments through tooling and automation.

Some best practices to follow are using small functional APIs, like the unix philosophy to make each program do one thing well – and prefer to build a new tool rather than adding on features to existing well-established systems. Create breakglass mechanism, for use in case of an emergency. Implement auditing systems and strategies for when the systems fail. Implement testing of the systems with least privilege, and testing the least privilege access itself. Create records of diagnosis of how and why access is revoked or denied, so that can be granted temporarily if necessary, or prevent it from being granted again when the privilege has been revoked on purpose. Also ensure the language to describe policies is clear, so it can be updated easily, or when changes are made, it is clear what has changed or what access has been given or revoked.

Design for Understandability

In order to have confidence in your system, you need to manage your system’s complexity. You must be able to meaningfully reason about and understand the system, its components, and their interactions. A complex system is more difficult to reason about when it encounters specially crafted malicious inputs.

Designing a system with understandability takes considerable effort. (sidenote: with AI we now have zero understandability about how the code works… unfortunately.) A typical system should have some security or authorization features that are always true, these are called system invariants and would include traits such as rate-limiting, load balancing, graceful degradation, protection from sql injection, protection from cross-site scripting, logging, etc. If these invariants are not included then the system has a potential security weakness or vulnerability.

Structuring systems into layers and components is key to managing complexity, making use of design patterns can also make it easier to reason about the systems.

Design for A Changing Landscape

The ability to adapt quickly and implement features or bug fixes to your systems, depends on the tools and approaches you use like automated testing, frequent builds and releases, using micro-services for reliability, etc. The ability to respond quickly to a changing landscape of vulnerabilities, expectations, changing regulations and risks is vital. (sidenote: while AI does make changes easier at the trade-off of understandability, it could also lead to systems that are self-repairing, in which they have their own immune system to respond to anomalies and update their code on the fly.)

The best practices to follow when releasing changes should include the following characteristics:

  • Incremental: small standalone change
  • Documented: describe the how and the why
  • Tested: create confidence the changes work as expected
  • Isolated: use feature flags to isolate changes and avoid incompatibility with previous versions
  • Qualified: follow the correct procedure for releasing to production
  • Staged: roll out changes gradually to smaller subsets of users

This should be followed as closely as possible, when possible, but sometimes you need to deal with short-term zero-day vulnerabilities that fix issues immediately.

Design for Resilience

Resilient systems are able to withstand unusual circumstances that stress your system, and keep the system running when you face multiple simultaneous incidents, even if the system is only partially available during that time. Systems that are resilient are able to delay or withstand breakage.

Following the best practices highlighted earlier in the book, the author suggests that each layer in the system should be independently resilient, prioritise features to determine which ones are critical and require more resources to defend. Compartmentalise the system and add redundancy to defend against local failures… and reduce the reaction time by automating resilience and safety mechanisms. The author provides some examples like throttling, load shedding, load balancing and graceful shutdowns, etc.

Design for Recovery

The types of failures determine how you may recover from them, but a well designed system can prepare you and provide enough flexibility when needed.

  • Random Errors: random errors might include hardware failure or bit flips from cosmic rays, or unpredictable events like earthquakes.
  • Accidental Errors: all humans make mistakes, these are errors caused by humans with good intentions.
  • Software Errors: all software has bugs or unexpected states–even well designed software.
  • Malicious Actions: human action to infiltrate a system.

The design principles for recovery the author recommends are:

  • Design to deploy new releases as quickly as possible (guarded by policies)
  • Avoid systems that rely on time as a form of state, as mismatches between time cause issues during recovery, like expired certificates.
  • Avoid or prevent unauthorized rollbacks, as attackers might roll-back patches and continue their attack.
    • Create a Deny list to prevent known bad versions of code from being deployed
    • Pin software to version numbers that are known to be secure
    • Rotate signing keys
  • Use revocation mechanisms to prevent certain code or functions from being executed (can lead to more complexity if the attacker has access to this.)
  • Keep track of known good states in the system so you can recover to them.
  • Use testing and continuous validation.
  • Design for emergency access.
  • Use access controls (that don’t have a single point of failure.)
  • Create emergency communication channels.
  • Create good habits for responders, like having regular on-call staff, relevant documentation, practice responding to emergencies.

The last part of the book focuses on the implementation of these mechanisms in the software development lifecycle, which includes a lot of case studies and code examples, doing risk analysis and setting up an incident response team.