by: Yeeva Chunnanond

Data is dirty. Meaning, the process of gathering and analyzing data is not as straightforward as we often think, nor does it yield the concrete answers that many people tend to expect. This is what competing in MTFC taught me.

Our group chose our research topic specifically because we thought it would be “easy to find data online.” We were wrong. We quickly learned that on the Internet, it’s rare to find data neatly packaged in an Excel file. Sometimes, it was a question of format—one of our group members had to run a PDF through Adobe Acrobat to convert it into a usable format (a neat trick to know, but seriously, who puts a data table in a PDF?). Other times, it was a question of what data could—or, rather, couldn’t—tell us about the real world. Many things we thought could be quantified couldn’t (or hadn’t been). For example, how do you index the flammability of a neighborhood? Is it enough to consider meteorological trends, or should you also account for the flammability of infrastructure and characteristics of the natural environment? Then, how do you measure those things? Has anyone ever measured them before? We had to make a lot of compromises on our initial data schematic once we were knee-deep in the actual research.

Another example: when we opened the can of GIS worms. You will likely encounter data stored in a Geographic Information System (GIS) at some point while competing in MTFC, especially if your research question requires you to treat regions as data points. The catch is that GIS data has a special way of being stored, viewed, and edited, requiring GIS software and someone who knows how to use it. Our team was lucky enough to have a connection to someone in the GIS field who could help us “beat the data until it confessed.” Eventually, we successfully flattened our geospatial data into a neat table and were even able to feed it into Python MATLAB code to create a slick animated map. But it was humbling to realize how many types of data exist and how ill-equipped we were to deal with some of them.

Knowing what went into reaching our project’s conclusion made me trust it way less. Often, even after scouring webpages and bibliographies, we couldn’t pinpoint the exact methodologies our data sources used to gather information, which meant we couldn’t identify the ways the data might be skewed. There were also many factors we wanted to account for but couldn’t, either because we didn’t have access to the data or because it simply didn’t exist. Only after completing the project did I fully realize how incomplete our considerations were. (With some of the p-values we ended up with, I personally would have had an easier time believing a qualitative conclusion over the quantitative one we came to.) Things turned out far less cut-and-dried than I had expected.

Part of this is because we’re used to squeaky-clean, sparkling data from fairytale math problems, and part of it is that we were (and still are) novices in this field. I’m fine with that—it’s okay to be a noob. However, the most pressing implication of this experience is that it revealed just how flimsy a quantitative conclusion can be. People often treat conclusions drawn from “cold, hard data” as more ironclad than they actually are. And the only type of person who can figure out that the data isn’t cold or hard—and rarely ever is—is a fellow mathematician who has traveled the same dirty path as us.