00:00:00.000
It seems like it's about that time, so let's get started.
00:00:05.540
The next speaker is Mai Nguyen. I don't know much about wine, but I love drinking and eating, so I'm very much looking forward to her presentation.
00:00:12.000
The title is "Food, Wine, and Machine Learning: Teaching a Bot to Taste." Konnichiwa!
00:00:24.240
Thank you so much for having me. I'm so excited to be here, and considering you have so many choices of other talks to go to, thank you for choosing to come to mine.
00:00:37.770
I'm Mai. At first, I want to say a little bit about myself to introduce myself. I started working in Ruby on Rails back in 2006 at a startup in Washington, D.C., and I continued to work in Rails until 2010. During this time, I became interested in learning more about wine.
00:00:58.559
Food has always been a passion of mine, so I took a break from technology to travel and study winemaking. I obtained a master's degree in winemaking at the University of Adelaide in Australia. After I graduated, I decided to pursue a career in winemaking.
00:01:20.310
I worked in wineries in Australia, France, California, and New Zealand. That's what brought me to New Zealand. The thing about working in the wine industry is that it's agricultural; things are seasonal and repetitive.
00:01:39.360
There’s a cycle and a method you use to make wine, and the wines must be consistent for the brand you're working for. I missed using different parts of my brain and learning new things daily.
00:01:57.030
So, last year, I decided to move to Wellington, which is where most of the technology firms in New Zealand are. I'm now a senior developer at Loyalty NZ, and I’m very thankful for their support, as I wouldn't be here today without it.
00:02:21.360
You might ask, how did I get into machine learning? Well, it's quite simple. People ask me about wine all the time, and while I love discussing it, sometimes I just want to relax with my friends and family.
00:02:33.240
There are lots of wines available, and no one seems to know what they should be drinking. They always ask for advice and recommendations, and I thought, there must be a way to automate this process.
00:02:54.620
So, just a little overview of machine learning: if you're not familiar with it. Machine learning is a branch of artificial intelligence. Some people view it as a buzzword that doesn't mean anything, but to me, it means making computers do intelligent things.
00:03:18.030
Machine learning involves making decisions based on models created from training data—it's not just explicitly coded rules. Within machine learning, there's deep learning, which I think of as layered neural networks.
00:03:35.820
This is essentially machine learning on top of machine learning, and it relies on a ton of data.
00:03:41.400
Natural language processing also plays a key role; this can be based on machine learning or explicitly coded logic. Now that we understand where machine learning fits in the AI landscape, let’s look into what it really is.
00:04:04.500
Machine learning does not rely on coded rules; instead, it creates its own models from training data. Training data can be supervised, meaning it contains the correct answers for the machine learning algorithm to predict or unsupervised, where no right answers exist.
00:04:31.590
The unsupervised approach asks the algorithm to find hidden structures and patterns in the data. There are various algorithms for many different kinds of problems, but I won't delve into all of them—this is only a 40-minute talk.
00:04:49.470
So what can machine learning truly do for me? You can think of it as taking any infinite combinations of inputs, feeding them into your machine, and receiving a finite answer in various forms: a boolean value, a number, a category, etc.
00:05:10.680
Common use cases for machine learning include classification, such as spam filtering, categorizing sentiment, recognizing fraud, ad targeting, and medical diagnosis. Another popular use case is recommendations, familiar to all of us, especially with product recommendations.
00:05:38.610
This technique can also be applied to job recruiting, dating websites, and content suggestions (think Netflix). Additionally, machine learning can be used for predictions, such as forecasting demand, predicting stock market movements, weather forecasts, and even sports outcomes.
00:06:02.370
It can be immensely helpful in inventory and asset management. Lastly, another compelling use case is imputation, which refers to using machine learning to fill in missing values in datasets.
00:06:20.970
Some basic insights about machine learning that might not be immediately apparent include that accuracy improves as you collect more data. You can also automate learning as answers are validated, allowing for faster processing depending on the algorithm used.
00:07:05.400
It can be customized based on your own data and scaled accordingly. But why aren't more people using it? There are various reasons that, in my opinion, aren’t valid.
00:07:17.280
One common excuse is: 'I like Ruby and I don’t want to write Python.' I sympathize, as I don’t know Python myself, but I'm still able to use machine learning; that's a terrible excuse.
00:07:48.000
Another reason is people feel they don't have time to learn these algorithms. Yes, we are busy. However, there are experts who have spent significant time developing these algorithms, so let them do their job while we consume their results.
00:08:02.990
Among Ruby resources, we have many algorithms and tools imported to Ruby—not a complete set, but a good amount available on RubyGems. Just yesterday, Kenta Marotta gave a fantastic talk about Pike, which allows you to call Python functions from Ruby.
00:08:39.560
While you don't have to code in Python, you will still need to read their documentation. Additionally, there are repositories on GitHub listing Ruby machine learning gems and other natural language processing tools that are constantly updated.
00:09:06.660
If you checked out machine learning a year or two ago, you might find that the tools you were seeking have now been ported over to Ruby, allowing you a chance to explore machine learning within your stack.
00:09:13.310
Moreover, many cloud-based learning APIs and services allow you to leverage someone else’s work, eliminating the need for deep data science knowledge. These services can help you avoid worrying about infrastructure and DevOps concerns.
00:09:32.980
BigML is one of the first cloud-based services for machine learning. Amazon has also entered this space with multiple services. Microsoft and Google Cloud offer machine learning solutions as well. If you have any interest in TensorFlow, a project exists to port it to Ruby.
00:10:02.530
There is also a wealth of natural language processing APIs and services available, like Wit.ai, which is well known for chatbot applications, now owned by Facebook.
00:10:16.640
The Microsoft Language API and Google Cloud Speech also provide great resources for voice and text processing needs. We all know about Amazon’s Alexa, and IBM Watson is included as a player, albeit at a higher price point.
00:10:36.850
Now that we know we have tools at our disposal, how do we actually use them? Where do we start? The first step is to formulate the question we want to answer.
00:11:01.590
This is crucial as it will guide all the subsequent decisions. If you’re unsure about the question, also consider the data you might have access to. After all, without data, using machine learning to solve problems is essentially impossible.
00:11:24.690
Regarding data, you can use custom data that your company has gathered, or you can tap into free public data sources. There are numerous sources for public data; for example, the UC Irvine Machine Learning Repository and Kaggle, which hosts a lot of data science competitions and offers plenty of free datasets.
00:11:50.400
Government resources also provide census information and geographical data, among other data types. Regarding good data characteristics: number one, it should be representative of future data.
00:12:30.720
If you are trying to predict something, the data must relate closely to the current conditions to ensure your model is trained effectively. Secondly, it must be complete, meaning there should be no blanks.
00:13:06.300
If your data is collected via surveys and some participants choose not to answer questions, you need to handle that missing data; otherwise, it will introduce noise into your machine learning process.
00:13:35.180
This data should maintain relevant features while minimizing noise. For instance, if you have a dataset of user information, user IDs may not contribute to predicting user behavior; hence, they’re irrelevant.
00:13:53.600
Then, you need enough data; the more you have, the better. When working with machine learning, you'll spend a considerable amount of time on feature engineering.
00:14:12.120
Feature engineering is the process of translating your existing inputs into predictive features you can use effectively.
00:14:30.740
For instance, if you're collecting birthdates, you might not need the exact date; rather, you should focus on their age range (like 20 to 30) and adjust your data accordingly to reflect that.
00:15:11.370
Again, a key point: it’s essential to fill in missing data. Strategies include imputing data based on adjacent values, such as filling gaps in time series data.
00:15:37.860
Also, you can use machine learning to help make educated guesses about missing data. It's important to use relevant inputs.
00:15:55.370
The beauty of machine learning lies in its ability to identify hidden patterns that may not be obvious to humans. Therefore, trial and error is often necessary to determine which pieces of data are critical.
00:16:24.300
Now that we have good data, let’s discuss what to do with it. First, divide your data into two segments: around 80% for training and the remaining 20% for testing your models.
00:16:54.630
After preparing your training dataset through feature engineering, it then goes through the model for training. The test dataset also undergoes similar processing before being used for prediction and evaluation.
00:17:10.210
Evaluation means comparing your model’s predictions against known answers. If the predictions are inaccurate, you will need to adjust either your model or gather more training data, which could involve identifying missed edge cases.
00:17:50.000
This iterative process of training, evaluating, and optimizing continues until you achieve an acceptable accuracy for your model.
00:18:13.010
Predicting with a trained model is straightforward: for new data that you want to predict, channel it through the same feature engineering pipeline, prepare it accordingly, and send it off to the model.
00:18:36.990
Now that you understand the process of creating better models and working with machine learning, keep in mind there are instances when machine learning may not be appropriate to use.
00:19:12.980
One scenario is when the rules are well-defined and finite. For example, if you sell school uniforms, you shouldn't need machine learning to figure out you should advertise to parents of school-age children.
00:19:42.370
Another situation where machine learning might not be suitable is when you require extremely high accuracy. You don’t want to risk life and death decisions based on predictions.
00:20:09.370
Additionally, if data is scarce or difficult to access, that could also signify that machine learning isn't the best route to pursue.
00:20:38.340
There are challenges with machine learning; for example, mistakes in training data can be hard to spot. If the training data is based on voluntary surveys, how do you ensure respondents are truthful?
00:21:06.420
It's a challenge if you expect 100% accuracy—as that's nearly impossible. Testing can get complicated, especially when there are many edge cases that your training data must represent.
00:21:25.030
If the topic you are focusing on evolves significantly, your historical data may not apply to current conditions. An excellent example of this is housing prices—inflation or market trends could skew your predictions.
00:21:53.400
Lastly, biases in training data can excessively amplify impacts. For instance, an infamous case with facial recognition software showed that excluding specific demographic data led to significant inaccuracies.
00:22:14.990
Determining whether your machine learning predictions are successful can also be complex. For example, when Netflix recommends a movie, how do they confirm that it was a good suggestion?
00:22:41.760
Perhaps the user didn’t click on it because they had already seen it, which makes it difficult to assess the guidance without clear feedback.
00:23:02.840
That brings us to a practical example that summarizes many of these challenges. I created a project called the Happy Wine Bot.
00:23:13.860
The problem I was addressing is that many average consumers find wine intimidating and struggle to select wines that pair well with their meals.
00:23:38.330
My solution was to build a chatbot that would be available online at any time to educate, entertain, and suggest wines that match with various foods.
00:23:57.400
To construct this bot, it needs to converse with users effectively and respond to inquiries. It must comprehend both the tastes and flavors of food, as well as those of wine.
00:24:15.340
Furthermore, it should also evaluate food and wine combinations for optimal pairings.
00:24:31.680
To enable conversational abilities in the bot, I first experimented with AI/ML, particularly a gem called Program R, which stands for artificial intelligence markup language.
00:24:54.650
It's essentially an XML-based vocabulary using regular expressions for recognizing phrases, but managing extensive vocabularies via XML files becomes cumbersome.
00:25:14.080
Afterward, I explored various natural language processing APIs and eventually settled on Wit.ai due to its cost—it's free!
00:25:36.650
I integrated it with Twitter, using a polling mechanism to check for tweets related to my account.
00:25:49.620
Additionally, I integrated it with Facebook using a webhook—a push mechanism that allowed it to react to input swiftly.
00:26:05.740
Next, I had to invest considerable effort into developing the chat content that would effectively engage and respond to user questions.
00:26:25.470
So, the Happy Wine Bot is capable of conversing and interacting about wines!
00:26:39.270
Now, let’s discuss taste. We understand that taste involves more than just our tongues; basic tastes include sweet, salty, sour, bitter, and umami—fatty is also recognized as a newer taste.
00:26:56.000
Your tongue can also sense physical sensations such as texture and weight, while pain receptors are activated when consuming spicy foods.
00:27:10.950
Moreover, our noses play a significant role—some studies suggest we can detect anywhere from 10,000 to 1 trillion different smells.
00:27:20.450
These tastes and scents combine to create flavors, informing our brain about what we are consuming, whether it be food or wine.
00:27:34.680
What factors influence food taste? Ingredients play a significant role, including herbs, spices, and proteins like fish, chicken, or beef. The cooking method also contributes to flavor.
00:27:54.030
Grilling gives charred flavors, and slow cooking can make food more tender. These ingredients and methods impact taste, flavors, textures, intensity, and overall experience.
00:28:19.930
When teaching the bot about food, I decided to utilize recipe data focusing on ingredients and methods as training data.
00:28:47.230
For this, I scraped relevant recipe sections from various online recipe sites, ensuring my data were as clean and complete as possible.
00:29:09.940
The flavors and taste attributes derived from these recipes become the features that I want the bot to predict.
00:29:26.410
I generated supervised training data from my own cooking experience and aimed to classify each flavor or feature—one model for sweet, another for salty, and so on.
00:29:45.830
I specifically chose features with interactions relating to wine, identifying over 50 characteristics per recipe. I indeed took some liberties to simplify the process.
00:30:04.410
For instance, when evaluating a hamburger’s flavor profile, we analyze the ingredients and cooking methods to understand its nuances.
00:30:21.090
We'd highlight tastes like salty, sweet, sour, bitter, umami, and textures in our evaluation.
00:30:39.560
Now that we understand how to train the bot regarding food, let’s transition to wine. Various features contribute to how wine tastes: the climate where grapes are grown, the grape varieties, and viticultural practices.
00:30:55.410
Moreover, winemaking decisions possibly affect flavor, such as barrel choice or yeast selection, and of course, the wine’s age matters too.
00:31:13.470
These elements influence viscosity, flavor intensity, tannins, acidity, sweetness, bitterness, and aroma.
00:31:31.480
To equip the bot with knowledge about wine, I needed credible data sources. Yet, wine data can be inconsistent—reviews may be incomplete, subjective, or vary widely based on individual tastes.
00:31:51.340
Therefore, I opted to zero in on general categories of wine, leveraging my own experience and some wine literature to classify over 100 types of wine with corresponding taste profiles.
00:32:07.560
The key attribute classifications, totaling over 40 taste and flavor characteristics, help dictate how food and wine interact.
00:32:28.000
Now, how do we evaluate food and wine combinations to achieve good matches? The concept of pairing involves understanding the role wine plays when combined with food.
00:32:51.360
In some cases, wine elevates the food without altering its essence; other times, the food will enhance the wine. The best pairing occurs when both elements elevate each other.
00:33:10.340
Sommeliers often use strategies like matching flavor intensities—pairing a bold red wine with rich steak, for example.
00:33:33.740
Alternatively, complementing flavors could involve pairing sweet foods with sweet wines since pairing a sweet dish with a dry wine can result in a harsh taste.
00:33:50.090
Another approach is to match or contrast flavors and textures. A sparkling wine generally complements fried dishes well due to the delicious carbonation.
00:34:06.400
Additionally, considering the origins of the cuisine can enhance pairings, as there's a saying in wine circles: 'what grows together goes together'.
00:34:19.820
Lastly, one wants to avoid problematic combinations. Pairing a hot, spicy dish with a high-alcohol wine can exacerbate the heat.
00:34:30.550
So how do we instill this knowledge into the bot? Recognizing that food and wine pairing is fundamentally rules-based—and those rules are generally well-defined—I relied on my prior understanding and knowledge.
00:34:52.060
The bot is designed using a weighted scoring engine to rank and match various wines with corresponding dishes.
00:35:07.800
Here's a demo of the bot! This is the bot on Facebook, and we're clicking on 'Get Started' to initiate our interaction.
00:35:31.600
Once we say hello, the bot responds promptly because it's live and ready to interact with users by discussing wine.
00:35:50.290
You can even ask it for wine match suggestions. If you input something like 'karaage,' a type of fried chicken, it’ll provide recommended wine pairings.
00:36:10.480
After processing your request, the bot presents a list of suggested wines, along with links to further details or other matching options.
00:36:35.340
For instance, karaage pairs nicely with sparkling wines due to its fried nature or a light-bodied red like Pinot Noir for chicken.
00:36:54.950
It provides general advice for wine pairings and offers links for more suggestions outside of Facebook.
00:37:14.290
Building this bot was an exhaustive learning process. I realized that generating data manually is time-consuming, and I often needed to refine the input.
00:37:34.500
Codifying recipes and flavors required considerable effort, as well as curating a list of diverse wines to boost our MVP.
00:37:54.290
I also discovered the importance of having abundant data as using limited data for training presented accuracy challenges.
00:38:06.680
Moreover, bots must contend with users experimenting with spellings and variations that also hinder accuracy.
00:38:30.990
To summarize some takeaways: machine learning is capable of addressing varied tasks and holds potential within any Ruby stack; quantity and quality of data are crucial.
00:39:05.199
So, if you have been considering machine learning, I encourage you to give it a try.
00:39:17.169
Thank you so much!
00:39:37.110
Thank you, Mai! Unfortunately, there's no time left for questions. However, feel free to ask her afterwards. Thank you!