Suggest modification to this talk

Title

Description

Date

Summarized using AI?

If this talk's summary was generated by AI, please check this box. A "Summarized using AI" badge will be displayed in the summary tab to indicate that the summary was generated using AI.

Show "Summarized using AI" badge on summary page

Summary

Markdown supported

In the talk titled "Story of a Haemorrhage," Anne Sophie Rouaux, a back-end Ruby developer at Kiss Kiss Bang Bang, shares her experiences dealing with flaky tests within their testing framework, primarily using RSpec and CircleCI. The presentation brings attention to the challenges posed by flaky tests and their impact on development efficiency.

### Key Points Discussed:
- **Flaky Tests Crisis**: Anne Sophie describes the initial challenge they faced with test execution times averaging 30 to 45 minutes, which prompted the adoption of parallel testing through CircleCI, reducing execution time to approximately 7 minutes.
- **Denial and Discovery**: Following improvements, she encounters unexpected test failures unrelated to recent code changes, revealing a staggering 70% failure rate among jobs, leading her to a rigorous investigation of flaky tests.
- **Reproducing Failures**: She emphasizes the importance of replicating test failures locally using the failed job’s test list and seed number, an essential troubleshooting strategy.
- **Bleeding Data Problem**: Rouaux identifies that the core issue was due to 'bleeding' data, which she investigated by examining the use of the 'database_cleaner' gem and its cleaning strategies.
- **Testing Strategies**: The talk explores the implications of using 'let' versus 'before(:all)' in tests. It highlights how using 'before(:all)' could lead to persistent data between tests, exacerbating the flaky test issue especially noted after switching to parallel test runs.
- **Actionable Solutions**: The solutions proposed include randomizing test execution order and utilizing an 'after(:all)' hook to properly clean up test data. She also suggests reconsidering the need for 'database_cleaner' when transactional tests are employed effectively.

### Important Conclusions:
- Emphasizes reading documentation and understanding the testing framework thoroughly.
- Highlights the risk of misusing shared states in tests and the importance of proper data management techniques.
- Encourages a collaborative approach to problem-solving in testing instead of assigning blame.
- Suggests that removing database_cleaner led to performance improvements, reinforcing the importance of optimization in test environments.

In conclusion, the talk serves as both an informative guide and a cautionary tale for developers managing flaky tests. By sharing her journey, Anne Sophie underlines how critical it is to understand testing dependencies and strategies to maintain a reliable and efficient codebase.

Cancel