Measuring Effectiveness of Policy: Designing Evaluation Methodology for the Baltimore City EPS Foam Ban

Written by Molly Cook, Graduate student with the Johns Hopkins Bloomberg School of Public Health and School of NursingThe Baltimore City Council and Mayor Pugh signed a foam food container ban into law in April of this year. This is exciting news for many reasons, but for me it became a summer’s obsession. As part of my public health nursing master’s curriculum, my capstone practicum assignment was to design an evaluation plan for the Baltimore City Expanded Polystyrene (EPS) Food Container Ban. After meeting with Ashley Van Stone, executive director, and Claire Jordan, advocacy and outreach manager, I settled in to learn everything I could find out about expanded polystyrene foam and environmental policy evaluation.I searched for hours through literature and policy documents, called experts from non-profits and the Bloomberg School of Public Health, and attended city-wide meetings on implementation and coalition development. The most important takeaway from my research and investigation is that evaluation should be built into the design of programs and policies from their conception. Baltimore did not build evaluation criteria into the ban’s conception, but the long implementation period provides enough time to establish baseline data and develop a plan to track it over time.The second most important takeaway? Policy evaluation is extremely rare, and environmental policy evaluation is even rarer. This makes sense when you consider the level of science, budget, capacity, and political will required to measure and track changes across large geographic areas, and somehow attribute them to any single policy initiative. This is likely why there were no EPS foam ban evaluations available for me to use as models. Building a plan from scratch was intimidating, but it also felt more valuable.The goal of the ban is to reduce toxic litter and waste in a way that equitably distributes both burdens and benefits. This requires both process (is the ban implemented and enforced well) and impact evaluation (does it work to reduce litter and waste). For the process evaluation, I designed a pre/post-test survey that can be distributed before all business and organizations have switched to non-foam alternative containers, and redistributed a few months after enforcement. The survey addresses anticipated challenges and potential benefits that consistently arise as arguments for and against the ban in city halls and state legislatures. It also accounts for the size of the business or organization to help Trash Free Maryland understand the nature of the burden of switching to alternatives for everything from churches or hospitals, to food trucks or local or chain restaurants.I was able to select appropriate indicators to measure: amount of litter in neighborhoods and the harbor, microplastic water pollutants, amount of trash that goes to incineration, and microplastic marine life pollutants that are attributable to EPS foam. Some of these are easier to measure than others, and the above criteria are listed in order from highest priority to lowest based on evidential support and organizational capacity in Baltimore. It is also important to note that EPS foam should be measured by volume rather than mass in litter cleanups and trash collection.The highlight of the summer lay unexpectedly in the trash and debris of Baltimore’s harbor. I went on a trash trawl captained by The National Aquarium along with Ashley from Trash Free Maryland, and staff from  Blue Water Baltimore's Waterkeeper. Trash Free Maryland drafted a repeatable protocol, borrowed a trash trawl from NOAA, and headed out to collect trash in a net, as part of an effort to establish baseline data regarding EPS foam litter in the city prior to the ban taking effect. We also scooped up an appalling amount of large-sized trash along our route that did not factor into our data.The data includes anything that our 333-micron net caught. It can now be separated as shown and analyzed to determine how much of the trash is attributable to EPS Foam. Trash Free Maryland can repeat the trawl annually to determine if the foam ban is reducing harbor EPS foam waste. To deepen baseline data, Trash Free Maryland is working through its Trash Free Baltimore Collaborative to connect with partners city-wide on counting foam in their upcoming efforts, to inform the baseline data.To say I learned a lot does not capture the breadth and value of what I gained from 12 short weeks with Trash Free Maryland. I learned about environmental science, plastics, environmental policy, coalition building, leadership, evaluation, collaboration, and communication. I will never embark on a programmatic or policy intervention without first considering, “how will we even know if this is working?” I will never look at a piece of foam the same. I swim in the ocean with newfound respect for our influence over its health. I look forward to hearing the results from this evaluation of the effort that Baltimore City is making over the tiny corner of our planet they can control.