Software Architecture
Engineering Engineering: More than the sum of our parts
Edit
#engineering-culture
#team-building
#leadership
#software-architecture
#productivity
#mentorship
#organizational-skills
Summarized using AI

Engineering Engineering: More than the sum of our parts

Vietor Davis • April 17, 2018 • Pittsburgh, PA

In the talk "Engineering Engineering: More than the sum of our parts," Vietor Davis addresses the complexities and intricacies inherent in engineering organizations, proposing a nuanced approach to managing them. Rather than seeking a one-size-fits-all solution, Davis emphasizes the importance of context in navigating the challenges engineers face. Here are the key points discussed throughout the talk:

  • Engineering Complexity: Engineering is defined as both an art and a skill. The field is laden with complexities, such as scope creep and unknowns, which complicate the management of engineering organizations.

  • People Management: The speaker highlights the complexity of individuals within teams, emphasizing that understanding each person's background and motivations is essential for successful engineering management.

  • Contextual Solutions: Davis stresses that there is no single correct way to manage engineering teams. Success depends on the specific context, including team composition, products, and organizational culture.

  • Structure vs. Execution: The right amount of organizational structure varies; while it can support novices, overly rigid structures can become hindrances once teams mature.

  • Trade-offs in Engineering: The challenges engineers face require balancing structure and productivity, with the recognition that different teams might benefit from varying degrees of oversight and freedom.

  • Organizational Pathologies: Common dysfunctions in organizations, such as excessive outsourcing, death marches, and stack ranking, are rooted in short-term pressures that yield immediate benefits but ultimately harm long-term health and productivity.

  • Framework for Understanding: Davis proposes a toolkit for addressing these issues by identifying root causes and pressures that lead to negative outcomes, encouraging proactive measures rather than reactive fixes.

  • Building Trust: The crux of effective engineering management lies in fostering trust within teams, enabling individuals to make decisions and execute effectively without constant oversight.

  • Long-term Vision: Addressing these challenges requires not just immediate actions but a sustained commitment to building healthier organizational practices and structures.

In conclusion, successful engineering management requires a flexible, context-sensitive approach. Davis encourages engineering leaders to be aware of the nuances at play in their organizations and to work towards creating environments that empower engineers to thrive. By understanding the roots of organizational challenges and taking a holistic view, teams can move beyond mere survival to effective achievement and innovation in their practices.

Engineering Engineering: More than the sum of our parts
Vietor Davis • April 17, 2018 • Pittsburgh, PA

Engineering Engineering: More than the sum of our parts by Vietor Davis

Applying the tried and true answers of “It depends.”, “Maybe?”, and “It works for me, what if you try …” to our engineering organizations themselves turns out to work better than trying to find the One True Way. We all know that every engineering project is full of trade-offs, caveats, surprising complexities, and hidden depths; so it should be no surprise that we, the engineers who build such systems, are at least as complicated.

This is a sponsored talk by Procore.

RailsConf 2018

00:00:10 All right, I think we'll get started. We are going hard on time here, and I see a good crowd for this sponsored talk.
00:00:16 This talk is titled 'Engineering Engineering: More than the sum of our parts.' It focuses on the idea of being more than just the sum of our parts.
00:00:23 Engineering involves the action of working artfully to bring something valuable into existence. Whether it is software engineering, chemical engineering, process engineering, or mechanical engineering, it encompasses the art and skill of achieving intended results.
00:00:37 So, engineering engineering refers to the use of our engineering skills to create the structure and environment that allows for a successful engineering organization.
00:00:50 Who am I? I'm a guy who's been working in the tech sector, getting paid to do computer stuff since the late '90s when I’d rather be doing that than my homework.
00:01:03 I’ve been involved with startups for about 17 years, having founded a few and worked in many others. Currently, I serve as a director of engineering at Procore, which is still technically a startup.
00:01:15 So, what exactly is Procore? Procore is a suite of tools designed for managing construction projects. Projects like convention centers—though not this particular convention center.
00:01:28 Procore has been built on Rails since 2006, boasting over a thousand employees. Our headquarters is located in Santa Barbara, California. Technically, we are headquartered in Carpinteria, which is just a few minutes’ drive south of Santa Barbara.
00:01:42 As the director of engineering at Procore, it means I am essentially an engineer who engineers engineering. And that's a challenging task.
00:01:54 But why is it hard? Engineering management appears straightforward at first glance. We're all excited to be here, learning and honing our craft.
00:02:06 This is one of the best jobs in the history of humankind! We believe that self-motivation is important, and that control is not the answer.
00:02:13 Most engineers simply want to do good work, and therefore, it seems that engineering management should be easy. Some companies have attempted to remove all structure and let engineers figure things out on their own.
00:02:27 However, they have had mixed results. So, what’s the caveat? If all incentives seem aligned, why does it feel so challenging?
00:02:38 One of the main challenges is intrinsic complexity associated with engineering. The field is replete with scope creep, shifting requirements, unknown unknowns, and libraries that we thought would work until we are deep into GitHub trying to fix issues.
00:02:48 Everything must be right—correct punctuation, every character in hundreds of thousands of lines of code. Being 'correct' does not have a definitive standard; rather, it encompasses many interpretations.
00:03:00 Now, combine this complexity with the unique intricacies of human beings. We are inherently complicated creatures, each with our own experiences and inner thoughts.
00:03:13 At this conference, you got up this morning. You have your own thoughts from last night, where you traveled from, and everything that shapes your journey.
00:03:27 Now, consider that everyone else here has a similar internal richness. This, fundamentally, makes collaborating even harder. When you combine complex engineering with complex people, everything is magnified.
00:03:43 If we think people can just navigate through this effortlessly because we’re flexible and resilient, we are ignoring the very real emotions that everyone experiences.
00:03:59 If we want to engineer an organization, we must accommodate the complexities of both engineering and human factors.
00:04:09 What is the right way to achieve this? Perhaps I’ve convinced you that this is a difficult challenge, but what methods exist to solve it?
00:04:23 There are multiple correct implementations, but no absolute right way. And just like when programming, there isn't a single correct way to manage an engineering organization.
00:04:38 This is, in a sense, a never-ending problem that needs to be continuously addressed. You bring your best tools and skills to tackle it.
00:04:50 Engineering organizations deal with people, making the task of management exceedingly difficult. Approaching these challenges with sufficient rigor is essential because expecting trivial problems will yield trivial outcomes.
00:05:03 Why isn't there a definitive solution for engineering? Why can't we just find best practices and implement them?
00:05:18 It all comes down to the context in which we operate.
00:05:24 The right approach for a team or organization depends on a wide array of contextual factors: who the individuals are, their skills, motives, what they're working on, and the customers.
00:05:41 For instance, how should one run a sprint? It seems like a straightforward question, but the answer is—it depends. The tools you choose may matter significantly based on the context.
00:06:01 Should a sprint be weekly, bi-weekly, or monthly? How does that align with the project and goals? Do you have release schedules to consider?
00:06:15 Depending on how you're delivering value and what cadence you're aiming for, you may need different approaches.
00:06:26 This is the reality of engineering structures: they can be designed to support direct control over team processes and outputs.
00:06:39 However, while we can control our structure by implementing processes, it's essential to understand that the ultimate goal—execution—is about accomplishing what's needed at the right time to minimize collateral damage.
00:06:55 Different teams require different structures that serve their unique needs. A new team, unfamiliar with each other, will benefit from a supportive scaffold.
00:07:09 As this new team gains collaboration and experience, excessive structure can become cumbersome.
00:07:25 The right balance of structure will vary based on the context. A common analogy is asking whether you can run faster with or without crutches.
00:07:37 The answer highlights a fundamental truth: it completely depends on individual circumstances.
00:07:50 If embracing this uncertainty, it leads us to other questions like—how can we organize ourselves effectively?
00:08:01 What about the type of organization we want to work in? I trust that we want to embody values of trust, integrity, innovation, passion, and fairness.
00:08:15 Conversely, we would likely prefer to avoid organizations characterized by authoritarian control and apathy.
00:08:29 We generally agree that these positive values are critical for long-term success. So why are certain environments a challenge to maintain?
00:08:40 If we consider why certain negative environments arise, we must consider the incentives driving them.
00:08:54 As engineers, we need to think about how our organizations can be resilient against negative pressures.
00:09:07 In some instances, we might adopt practices that create short-term benefits, despite their long-term repercussions.
00:09:23 We need to understand these practices and why they emerge.
00:09:33 Organizations often resort to undesirable behaviors as knee-jerk reactions to immediate pressures.
00:09:46 For example, authoritarian control can produce quickly obvious results, but in the long run, it’s detrimental.
00:10:01 The key is understanding the benefit each behavior aims to achieve, so we can work against them.
00:10:17 Understanding the pressures that lead to negative outcomes allows us to address root causes of problems.
00:10:26 Let’s delve into this further by discussing some examples of organizational issues.
00:10:35 First, destructive outsourcing. It’s sometimes referred to as the iron lung of software. It guarantees that things will get done, but at what cost?
00:10:45 Outsourcing often removes the thinking from the internal processes, reducing accountability and limiting opportunities for discovery.
00:10:56 While outsourcing can be useful, it can also create problems where results are merely approximate.
00:11:06 When teams have to adopt outsourced work, they often incur hidden costs that can destabilize internal dynamics.
00:11:21 To prevent excessive outsourcing, we should focus on improving our internal team composition, ensuring we have necessary resources.
00:11:34 We can also improve our documentation to ensure accountability, setting clear expectations.
00:11:47 Another pathologous practice is the notorious 'death march.' It illustrates the misconception that working harder means achieving more.
00:12:01 However, we are not simply typists; we are thinkers. Long hours often degrade long-term productivity.
00:12:14 The underlying pressures that cause these death marches stem from unrealistic expectations and release deadlines.
00:12:26 To move past this, we need to create a sustainable long-term vision and balance our workloads.
00:12:35 Moving on to the concept of stack ranking—an approach perceived to optimize performance.
00:12:50 However, it misaligns incentives towards individual outcomes rather than team collaboration, ultimately leading to detrimental results.
00:13:05 Stack ranking might seem attractive, but it shifts focus away from cultivating trust within teams.
00:13:18 Instead, we should facilitate performance management across the board, aligning team goals to avoid the pitfalls of individual glory.
00:13:30 Another issue that arises is the excessive commitment to clients. This negatively impacts the organization’s ability to strategize long-term.
00:13:45 These commitments can seem beneficial in generating immediate revenue but may hinder more substantial growth opportunities.
00:13:56 Revenue uncertainty drives organizations to prioritize quick fixes over sustainable solutions.
00:14:09 To combat this, organizations can begin to assess the actual costs of client commitments versus perceived benefits.
00:14:23 By understanding root causes behind these pressures, we can better shape our organizations for future success.
00:14:38 It's essential to foster a culture that values trust. That's possibly one of the toughest challenges for organizations.
00:14:50 Trust emerges when individuals in an organization know they can act without needing permission.
00:15:01 As engineers, we must create stable solutions that balance investment with output.
00:15:15 Remember, what seems right is likely to be context-dependent. It is not about a universal solution but about finding what works in your specific setting.
00:15:27 Understanding the pressures that create negative behaviors is essential if we want to address them effectively moving forward.
00:15:40 Together, we can build healthy organizations that are well-intentioned and overcome their challenges.
00:15:51 Thank you for your time. We are engineers, and we can engineer our engineering organizations to foster growth and resilience.
Vietor Davis
@vietord
Explore all talks recorded at RailsConf 2018
+94