RubyKaigi 2017

Mapping your world with Ruby

RubyKaigi2017
http://rubykaigi.org/2017/presentations/matschaffer.html

In the wake of the March 2011 earthquake, many noticed a lack of good environmental data regarding radiation. The Safecast project was born from that need and our Ruby-based infrastructure how is home to nearly 70 million data points.

In this talk we'll go over the basics of the project, what we've learned over the last 6 years of running a volunteer-based Ruby project, and our plans for future expansion into tracking both radiation and air quality data.

RubyKaigi 2017

00:00:00.260 Okay, so we're going to start the first session of this room. We're starting with Mat Schaefer, who is mapping your world with Ruby. He's from Gifu in Japan, and he lives and works there. He works for Elastic, which is right. So, this talk is about his volunteer work.
00:00:21.210 Exactly, it's called Safecast. Thank you, Tomas. Is the mic okay? Everyone here good? Thanks for coming. I know it was a short session after lunch, but that's okay. I’m sure there will be video or something like that.
00:00:35.219 So, thank you for coming today. I'm going to talk about mapping your world through Ruby. As you just heard, my name is Mat Schaefer. I'm a site reliability engineer, mainly working for Elastic, which is also a sponsor here today. My volunteer work is for an organization called Safecast, and that's what I'm going to talk about today. It's where I get to use a lot of my day-to-day Ruby.
00:00:53.579 I'll start with a question: are you safe? It's hard to know for sure, right? It's a complicated question. This is a picture I took just an hour ago on Twitter. The conference room where we sit is reading 0.1 microsieverts per hour, which is the ambient radiation level in that room.
00:01:12.330 The radiation levels were recorded as I tried to take a taxi from Hiroshima station. I didn't have a Geiger counter, but as you can see, these are all the radiation levels recorded while I was walking to the conference today. These readings are part of a much larger data set around the Hiroshima area. It was reading 0.17 microsieverts.
00:01:35.369 It's all color-coded to indicate lower levels in blue. There's a little pink spot here that I plan to check out later, but it’s not too high. These are all the ambient radiation readings around the Hiroshima area.
00:01:53.040 This is also from the Safecast website. If you want to take a look on your laptops or something, you'll find PM2.5 readings around the Boston area. PM2.5 levels are often discussed in the news; is it low, or is it high? This is still new information to me. I can't tell you if it's particularly low or high. I know a little about radiation data, but PM2.5 and air data are areas I'm still learning about.
00:02:31.970 We have a dashboard that we recently set up, showing one sensor recording different PM2.5 readings around the area. All the data you've seen is gathered by a group I volunteer for called Safecast, and what we’re trying to do is answer the question: 'Are you safe?'. One of the best ways to answer that question is through data; you can’t know for sure if you're safe without knowing these levels.
00:03:05.300 These are all factors in your environment that you can't see or feel, but they do impact you and your loved ones. First off, I’m going to talk a little bit about the Safecast project, then I'll discuss how Ruby fits into it. This is RubyKaigi, so I want to spend some time talking about how we use Ruby at Safecast, and share some learnings from my four years of volunteer software work with Safecast.
00:03:37.760 I've learned a few things that could benefit others who might be interested in volunteering for a project like this. I would encourage some of you to consider getting involved with Safecast or a similar project and doing some volunteer software development. I'll share a bit about why I do it and how you might get involved if you're interested.
00:04:04.190 Safecast was started on March 11, 2011, following the Tohoku earthquake, which affected many people in this room. At that time, I was living in San Jose. My wife is from Gifu, and she has family members in Japan. One of the first questions we began asking was, 'Is everyone okay?'. Our family was worried: 'Is everyone safe?'. The people who started Safecast were immediately affected. Peter Franken, one of the main figures in the project, had family just north of the earthquake's epicenter.
00:05:01.490 When the news of the quake struck, the first thing he wanted to do was help. But then the nuclear disaster followed, separating him from his wife's hometown. The first thing they started digging into was what they could do, as they were left wondering about the safety of their loved ones, which was a concern shared by me and my family back in the US.
00:05:41.389 In the early days after the Tohoku earthquake and the Fukushima nuclear disaster, many people were left wondering about the radiation levels. Is it safe? This is an image taken from NHK news on April 22, roughly a month after the quake and the disaster. You'll notice that there's a prefecture missing on this map: Fukushima. The actual accident was north of the areas displayed in the news.
00:06:20.479 There was no data for the area directly affected by the incident, which was a significant problem. So, in those early days, Peter Franken, Shawn Bonner, and Joey Ito had a Skype call to ask, 'What can we do to get data?'. How can we find out if our families are safe? Should we leave Japan? Should we get our families out, or can we stay?
00:06:55.460 The first thing they did was look for data—was it available somewhere they could gather and share with the public? They quickly discovered that, as shown in the NHK image, there really wasn’t much data available. A few organizations were monitoring radiation, but the information was sparse, varied, and not easy to work with.
00:07:54.310 They quickly abandoned the idea of gathering existing data and thought, 'What if we could crowdsource this safety data?'. They wanted to get the community involved in measuring radiation themselves. They discussed the possibility of providing Geiger counters to volunteers to measure levels in different locations. However, they quickly encountered another problem: there were no Geiger counters available. After the accident, the global stock of Geiger counters was exhausted within roughly 24 hours.
00:08:49.480 It was tough to obtain a Geiger counter, which was very expensive. Even today, off-the-shelf Geiger counters can cost seven to eight hundred dollars. In the wake of the disaster, the prices were significantly marked up, making it challenging to crowdsource safety data.
00:09:35.640 With major concerns for family and friends, the clock was ticking. The individuals on that initial Skype call had loved ones close to the incident, with no certainty of their safety. They could only rely on the sparse information provided by the news. They decided to expand their efforts.
00:10:52.710 They began brainstorming on how to collect meaningful data, facing the challenge of limited sensor availability and sparse data reporting by others. Eventually, they adapted the plan, leveraging duct tape—usually used for many things— to secure a Geiger counter to a car window.
00:11:55.640 They used this modified approach for a truck driver involved in the relief efforts in the Fukushima area. They asked the driver to tape a Geiger counter to his window and take pictures of the readings while traveling, allowing them to gather GPS coordinates and radiation measurements along his route.
00:12:37.450 Although this method seemed rudimentary, it worked! They created an initial map where individuals could click on locations to get radiation measurements. This provided momentum to the Safecast project, inspiring them to find ways to gather more comprehensive data. By accumulating measurements, they could analyze the radiation levels along the entire route driven by volunteers.
00:13:29.750 With the limitations of taking photos of Geiger counters, they began searching for a better solution. They decided to automate the data collection process by connecting a laptop and a sensor in a weatherproof box. Rather than using a camera and taking intermittent readings, they aimed for real-time measurements, with the laptop collecting data from the Geiger counter every five seconds.
00:14:16.850 And thus, they attached multiple sensors to vehicles that drove through the Fukushima area, as some of the original volunteers did this work without knowing what they would encounter. They ventured into regions near the Fukushima reactor, using their sensors to compile readings systematically.
00:15:41.430 These initial efforts paved the way for the crowdsourced approach. The team provided the necessary components to volunteers while encouraging them to strap new sensors to their cars and drive around Japan, reviving their original plan of crowdsourcing radiation data.
00:16:11.910 This innovative approach was successful. They were able to measure radiation existing throughout whole ignore fields, and continued developing the initial idea into what became a more polished product.
00:16:43.420 The journey didn’t stop there—over the years, there was continuous growth in both the Safecast community and the development of devices and user engagement. The project also bridged beyond Japan, expanding its functionality to more data sources and types.
00:17:45.210 As data was collected, the organization transitioned into developing other sensing devices such as the B Gaggie and the Solar Cast. These became instrumental tools that informed the public regarding radiation and PM2.5 levels. With participants contributing their measurements across different locations globally, the database steadily expanded.
00:18:16.270 By making these advancements, community engagement grew steadily. Volunteers played a role in gathering data while exploring creative methods. Presently, the Safecast project has developed a robust point casting device that runs continuously, providing reliable readings.
00:19:01.320 The continuous data collection from fixed locations addresses a growing need around radiation measurements. The project evolved with robust methodologies to measure radiation levels through secure installations in readily accessible locations.
00:19:54.610 Additionally, they plan to implement strategies through mobile development to enhance the data gathered and make an API accessible to everyone. Central to these efforts are the community members, whose passion drives the success of Safecast, helping ensure that data remains accessible and transparent.
00:20:50.420 Through it all, the volunteers were key players, actively participating in installations and meetings to maximize the impact of the project. As the project expanded, volunteers enjoyed hands-on experiences gathering data and engaging with their surroundings.
00:21:21.870 The developments supported the understanding of environmental radiation, and the project became a staple for public engagement, encouraging individuals to be part of the growing movement for data transparency. Volunteer contributions defined the trajectory of Safecast.
00:22:07.510 After six years, we have about seventy-five million data points available and growing. As of yesterday, we've been able to map out distinct regions and track changes over time. It's imperative for me as a site reliability engineer to help maintain all these influxes of data as the project continues to expand.
00:23:03.990 The project aims to address a broad array of environmental concerns. Participation across diverse backgrounds has contributed to creating a wealth of information that benefits everyone.
00:23:39.920 Ruby is at the core of Safecast—its infrastructure relies on a solid, reliable backbone to manage extensive data and processes effectively. In managing community projects, tools developed with Ruby have fortified our impact and helped facilitate the project’s goals.
00:24:15.520 The API, the central element of our infrastructure, allows for a seamless integration of various sensors while the active management of databases ensures that all contributions remain accessible. Community engagement has remained the guiding principle throughout the endeavor.
00:25:00.870 The solar cast, a newer addition, is being designed for broader accessibility and usability. They focus on making it easy for anyone to engage with environmental measurements, fostering a culture of participation while generating vital data.
00:25:51.390 In discussing collaborative tools, the volunteers have pipelined their experiences, helping to clarify best practices and showcase how varied tools can be adapted for robust results. Our adaptability helps drive growth.
00:27:00.560 Ruby also supported device integration as we ramped up efforts to increase participation. We've developed numerous repositories, which have streamlined contributions while maximizing our outreach.
00:27:52.990 Involving numerous volunteers has created a unique network, connecting well-rounded enthusiasts toward a shared cause. Engaging with our community amplifies our visibility while making data reports widely accessible.
00:29:05.160 While Ruby has its limitations, particularly regarding performance, the benefits of its versatility and usability have provided the foundation for developing impactful solutions. The project thrives on volunteer engagement, and ensuring smooth operations fuels ongoing success.
00:30:12.130 As we continue moving forward with the Safecast project, we remain aware of the upcoming upgrades and improvements necessary to maintain robust database hygiene. Our tested protocols for development ensure we attract meaningful contributions that resonate with volunteers.
00:31:20.350 It is essential as we engage data contributors that we also focus on expanding our learning—fostering an environment where volunteers can build knowledge and skills while making meaningful impact. Our network will thrive by harnessing its talent and passion as we look toward future enhancements.
00:32:40.000 In summary, Safecast is about people making a difference through a shared commitment to data transparency. It's not necessary to be an expert to participate; it’s more about bringing your passion to the table. We encourage you to look for opportunities that inspire you, and organizational support will help you along the way.
00:33:43.159 As we've established many accomplishments with our structure, we have a ways to go in ensuring our data remains cohesive while continually expanding our reach for the future. To that end, we encourage everyone to become involved, following their passions to contribute toward community well-being.
00:34:50.690 Now before I conclude, I want to highlight some remarkable individuals that have been pivotal in creating Safecast.
00:35:00.000 It’s amazing to have shared this journey with the volunteers building Safecast from the ground up, paving the way for a collaborative network that's central to our mission. This has created an enriching environment for so many people.
00:36:09.550 It has been exciting to see this project evolve while tackling safety and data transparency head-on. Please consider becoming a part of this mission; there's room for everyone to contribute.
00:37:00.000 As always, my contact information is available, and I encourage any questions. I look forward to seeing more individuals join us as we move forward with Safecast.
00:38:19.411 Thank you for your time and attention, and I hope you enjoyed my talk.
00:39:46.690 Now, does anyone have a question for me, or would you like to chat afterwards?
00:40:00.000 Thank you, and I hope to see you around!