Outcome Oriented Security

by Eleanor Saitta

Outcome Oriented Security - RubyConf AU 2014

In this presentation, Eleanor Saitta, a commercial security consultant, discusses the significance of high-level concepts in security, particularly focusing on security architecture and user-centric security strategies in the software development process. She reflects on her transition towards emphasizing security for high-risk users and challenges the misconception that having no obvious low-level vulnerabilities implies a secure system.

Key Points Discussed:

Definition of Security: Security is defined not just by the absence of specific bugs (e.g., cross-site scripting, SQL injections) but by the capability of users to achieve their goals amidst potential threats.
Importance of Security Objectives: Security objectives articulate what matters in terms of application requirements. Writing these objectives down is vital to guide the development process effectively.
Threat Modeling: The construction of a threat model provides a structured way to analyze human activities and security relevant features of the system. It serves to transparently define threats and necessary security responses.
Process of Threat Modeling: Saitta outlines a six-step iterative process to develop a threat model, which includes identifying actors, assets, actions, threats, and security objectives, thereby visualizing the data flow within the application.
Hazard and Operability (HAZOP) Analysis: This encourages proactive questioning of potential failures at every step in a system’s processes, allowing identification of security weaknesses.
High-Risk User Engagement: A deeper understanding of the unique security needs of high-risk users plays a crucial role in building secure applications. These users often face various challenges, hence developers should create intuitive security mechanisms that accommodate these situations.
Evolving Security Norms: The talk stresses that traditional audit and compliance measures are insufficient—security design must be proactive and holistic, engaging all stakeholders to improve system resilience.

Conclusions and Takeaways:

Developers carry the responsibility of instilling a security-conscious culture within their teams while actively involving users in the design process to better meet their security needs.
Systems must not only be functional but should also consider usability in security tools to ensure overall effectiveness.
Collaboration across various perspectives within the security community is essential to foster an inclusive environment that adequately addresses the diverse needs of users.

The presentation concludes with a call for reflection on inclusivity within the community to enhance collective understanding of users' needs in the ever-evolving technological landscape.

00:00:09.639 So, as he said, I've been working mostly as a commercial security consultant for about a decade. I started with very low-level code reviews and gradually worked my way up. Right now, I focus on security architecture, protocol design, development process re-engineering, and security strategy work.

00:00:15.559 This talk will focus on some higher-level concepts in the security stack, which is crucial for every development shop. I hope it will be useful for all of you. Over the past couple of years, I've transitioned from commercial work to focusing mainly on security for high-risk users.

00:00:28.679 This primarily involves looking at security software and the tools designed to keep people safe online—though, unfortunately, that’s mostly a lie. But we try our best. Now, I have 59 slides to cover in about 40 minutes, so let's jump right in.

00:00:44.800 What is security? Many individuals believe that if they don't have any cross-site scripting bugs, CSRF bugs, or SQL injection vulnerabilities, they are secure. However, that’s not the case. Security is not merely a property of the software at a low level; it isn't specifically about resisting certain classes of bugs, nor does it depend on your testing regime.

00:01:09.000 Security is about whether the humans using your system can achieve their goals in the presence of potentially hostile actors. This notion applies across various contexts, whether it’s a Rails application or the checkout process at a grocery store. If someone can exploit a vulnerability that allows them to drive a tank through it, then you have serious problems.

00:01:24.520 To determine whether you are secure, you first need to understand what your users want. This turns out to be a complicated prospect. We use security objectives to articulate what people actually care about, and I’ll talk more about what a security objective is.

00:01:39.600 Security objectives define what we genuinely care about at the requirements level. When evaluating an application before writing code, someone must clarify its intended functionality. Without knowing what the application is supposed to do, coding cannot responsibly commence.

00:01:46.560 Security objectives guide us in planning and documenting what the application should achieve. They take the form of responses to attempts by bad actors to compromise something significant to us; in these scenarios, our system should respond in a sensible, pre-defined manner.

00:02:06.799 One significant issue is that very few people actually write out security objectives, which is tragic. In the last decade, I've probably gone through 100 to 200 requirements documents. The better ones typically have a section labeled ‘security,’ which I often skip because they rarely communicate anything useful about the system.

00:02:20.239 Instead, they generally state that systems should be secure against certain types of threats without clarifying what ‘secure’ means within the business context. Consequently, we often need to gather smart people around a table to brainstorm risks and what could potentially go wrong, which lacks structured guidance.

00:02:39.800 Ideally, we require a systematic approach to making this brainstorming effective and formal. Threat models provide a framework for accomplishing this. A threat model is a comprehensive and human-readable model that delineates human activities, priorities, and the security relevant features of the system.

00:02:52.519 To explain in simpler terms, a threat model must allow us to reason about scenarios and assess validity. It’s also crucial that the model conveys a clear understanding of when we've finished the analysis, as meetings among smart people can often lead to unfinished business disguised as ‘done’ simply because it’s time to celebrate with a beer.

00:03:39.800 The system we create should help us establish whether we have genuinely completed the analysis. Our threat model needs to be understandable and convey information to non-specialists. It should allow us to ask questions about the system's resilience to variations in circumstances.

00:04:30.000 First, we must understand the human activities and priorities that inform security decisions. It’s not just about technical specifications; the model must encompass what people care about at a fundamental level. Additionally, we need to include all relevant technical features of the system.

00:05:39.919 There needs to be a structured scope for what we're addressing, such as clearly defining our web application versus elements like a DNS server, which doesn’t require an extensive threat model. Moving forward, I’ll outline how to actually create a threat model.

00:06:03.800 The process consists of six steps, much like any other systematic approach. It’s recursive and iterative, evolving into a living document. We begin with our actors, assets, and actions, followed by identifying threats to the system prior to defining our security objectives.

00:06:54.880 From there, we proceed to develop data flow diagrams and explore detailed use cases to understand the architecture better before analyzing potential failure scenarios. Let's begin with actors. Generally, actors are humans, but they can also be autonomous systems or even rogue AIs.

00:07:45.960 What we care about, however, are the assets—things that we value. For example, a password isn't an asset unless you are a password management system; rather, we value the control over accounts it provides. We create a framework to define what actors can do to specific assets using a controlled vocabulary.

00:08:45.400 Thus, certain actions like ‘create,’ ‘read,’ ‘update,’ and ‘delete’ become relevant depending on the context, allowing us to effectively reason about human business rules within the system. Once we've established these parameters, we can move forward to formally outline threats and draft security objectives.

00:09:57.760 In my perspective, a threat can be defined as any negative outcome that undermines the human-driven process supported by the system. If I can hinder someone from doing something they are permitted to do, or perform actions that they are not allowed to, that's a threat.

00:10:14.760 From there, we build a matrix that ranks negative outcomes according to their importance rather than who causes them, which limits bias. Once we understand which negative outcomes we want to mitigate, we can begin constructing our list of security objectives.

00:10:26.840 As we lay that out, we will say that certain malicious actions, like an unauthorized user accessing content they shouldn’t, must be thwarted by the system. This context allows technical teams to align their security checks with business rules.

00:10:48.080 Next comes data flow diagrams, which offer a visual understanding of architectural structures and interconnections. They allow analysis of trust boundaries, showing how different system components validate information flow between users, servers, and data stores.

00:11:26.960 These diagrams should enable us to think critically about threat modeling and help visualize various interactions. With this clarified architecture in place, we can analyze use cases to understand actual workflows and more deeply examine how the system operates.

00:12:10.680 Use cases allow us to capture key states, including preconditions that must be fulfilled prior to successfully completing tasks within the system. Understanding state changes in users is essential for modeling workflows, enabling us to track transitions and avoid overlooked security checks.

00:12:44.560 By examining how intended actions materialize, we can investigate additional workflows and identify potential business rule violations. This leads us back to threat modeling as it helps analyze failure modes, ensuring that we've considered comprehensive security perspectives.

00:13:28.320 Scaling into the concept of hazard and operability (HAZOP) analysis, we ask, 'What could go wrong at every step in our process?' This formal modeling enables thorough examination of how functions could fail and which security objectives may be at risk.

00:14:16.760 The result is not just awareness of security weaknesses, but effective identification of those weaknesses captured during architectural reviews. This approach emphasizes that security is not merely about exposing low-level bugs like cross-site scripting or SQL injection, but a holistic view.

00:14:56.640 Thus, venturing into a robust threat model requires commitment and structured methodical thinking. While daunting, if teams adopt this practice, they'll find they can make significant strides in enhancing the resilience of their applications.

00:15:49.519 A key benefit of a well-implemented threat model is that it can define and drive security protocols throughout the life cycle of the software development process. This integration encompasses security testing and ensures uniformity in understanding security implications at the technical and business levels.

00:16:23.679 Ultimately, it's about recognizing that every development team and individual has a role in amplifying system security. By shaping a security-conscious culture, layered throughout development and final deployment, we can make substantial progress on tangible outcomes.

00:17:58.079 Reflecting on outcome-oriented security, we acknowledge that incidents and exploitations aren't merely inconveniences. Rather, they often have serious ramifications that can deeply affect users' well-being or safety, especially in high-risk environments. Consequently, security developers must understand their customers' unique security concerns.

00:19:30.560 Customers may have a distinct set of security needs that diverges significantly from what service providers envision, so it is critical to engage with users directly to uncover those. In many cases, we’ve historically offloaded certain security responsibilities to users, which has resulted in inadvertent vulnerabilities.

00:20:29.080 The premise of thorough understanding entails direct engagement with high-risk users, ensuring software and services cater to their specific security needs. By recognizing the diverse environments in which users operate and the challenges they face, developers can better tailor their solutions.

00:21:09.760 Consequently, as we survey our established practices, there's significant room for enhancing usability within security mechanisms. Tools must not only promise security but do so in a manner that users can adopt without confusion or frustration.

00:21:43.920 High-risk users often operate under conditions of duress, fatigue, or haste, making it imperative that security mechanisms are intuitive and easily navigable. Ensuring that security measures accommodate users in such states will bolster their effectiveness.

00:22:08.800 In conclusion, we recognize that almost all users could potentially be high-risk at some point, paying attention to the collective security shared by all applications. Historic lessons inform us that security principles must evolve continually to foster resilience in users and systems.

00:22:57.560 Developers therefore have a profound responsibility to construct resilient and secure systems, encouraging practices that not only deliver functional software but also safeguard users’ potential vulnerabilities against external threats.

00:23:37.440 The established norms of assurance in security must shift; mere audits and controls no longer suffice. In today's landscape, proactive security design should be the goal, striving for a holistic approach where risks and mitigations are modeled before they become critical.

00:24:23.760 A commitment to this adaptation can lead to significant advantages in outcomes. The security community must lead the charge by collaborating and sharing knowledge as they advance towards achieving common goals.

00:25:11.680 In empowering users and incorporating their perspectives into the development process, we can foster environments that equitably address their needs, ultimately leading to a more secure and trustworthy technological landscape.

00:26:00.560 Ultimately, collaborative development will ensure that systems appreciate the world we live in and can effectively manage the complexities inherent in that reality, rather than relying on problematic assumptions.

00:26:46.960 Lastly, I urge everyone to reflect critically on how we engage with inclusivity in our community. The lack of diverse perspectives can lead to impaired understanding of users' needs. Showcasing experienced voices from varied backgrounds will enrich our collective output.

00:27:35.760 Here's a list of individuals whose expertise could have profoundly impacted the discussions shared today.

00:28:02.000 Are there any questions?

00:28:14.000 Thank you very much, Eleanor. That was an engaging talk. Are there any questions for Eleanor?

00:28:46.320 Just a quick comment: the threat modeling tool I work with is part of a project called Trike, which you can find more information about at octotrik.org. I'll tweet the URL and the presentation deck later today.

00:29:12.840 My Twitter handle is @dxing, so feel free to reach out for more information.

00:29:27.240 Do you find that systems developed using agile methodologies have worse security problems compared to the old waterfall models?

00:29:49.960 Honestly, they’re about the same. The security issues are more closely related to the team’s discipline, the freshness of the code, the frameworks used, and their understanding of security and business requirements. As projects transition to higher-level languages, like Ruby or Node.js, critical issues tend to diminish.

00:30:42.920 However, this doesn’t imply that significant risks can't exist; they just often require more deliberate effort to exploit compared to lower-level languages like C++.

00:31:06.120 Thank you very much, Eleanor, for that fantastic talk. Let's give Eleanor a big round of applause.

RubyConf AU 2014