By Jay P. Greene
Over the last several years, my colleagues and I have produced a number of studies on what students learn from culturally enriching field trips, including to see live theater or to tour an art museum. In general, we have found that students randomly assigned to these field trip demonstrate higher levels of tolerance and empathy and learn important academic content.
We’d like to see more researchers conduct similar studies so that we can explore more related questions and see if our results can be replicated. But when I say this to colleagues, funders, and others in the art world, they imagine that it is really difficult and expensive to pull off these experiments and doubt they can be widely conducted by others.
On one level they are right – these experiments can be difficult and do require serious funding. But compared to how much is spent on other education studies or is spent on arts programming, these studies can be a bargain. In addition, arts organizations are essentially doing these experiments all of the time when they offer their programs to some schools and students and not others. The only problem is that they are not structuring the offer of programs to some and not others as true experiments and they are not bothering to collect data on outcomes.
With just a little bit of careful thought and some increased expense, we could have many more experiments on how students are affected by these arts activities. Here is a list of some of the key steps organizations could take to get new experiments off the ground:
- Right now most school visits to museums and theaters occurs on a first-come, first-served basis. There are a given number of dates and times for visits and schools that submit their requests first get those slots until they are all filled. In addition, there are often limited subsidies available to help defray the costs of these visits. Again, eligible schools that ask first usually get those subsidies until the funds are gone. Instead, museums and theaters could make subsidies and slots available by lottery. If by chance one school gets to visit the museum or theater this semester and another has to come next semester, then you have the makings of a randomized experiment. By comparing students from schools that get to visit to those that, by chance, have to wait until later, you know that any observed differences are likely the result of the visit and not pre-existing differences among thong students.
- If cultural institutions could restructure their school programs so that the scheduling and availability were done by lottery, then all we have to do is collect data and we have experimental results. Museums and theaters are generally not well-equipped to design and administer surveys to collect these data, but plenty of university researchers are. Cultural institutions should contact neighboring universities to see if they can find researchers who would be willing and able to help conduct this type of study. Having the researchers involved in designing and conducting the lottery is also probably a good idea. Researchers could administer surveys to students in treatment schools both before and after their visits and to students in control schools at around the same times even though they didn’t have a visit in the interim. Researchers could also get some administrative data from schools, including grades, test scores, attendance, etc…
- None of this should be prohibitively expensive. Of course, researchers would have to be paid for their time and the time of assistants or graduate students to collect, analyze, and enter data, but most of these folks are relatively cheap and the time required is moderate. In addition, there are costs associated with travelling to schools and printing surveys, but again these are not big expenses.
So cultural institutions basically have it within their grasp to conduct many more experimental studies of what students learn from their programs. They just have to restructure their scheduling process to have the dates of visits allocated by lottery, they have to find some willing researchers, and they have to expend a modest amount of money.
If it’s not that hard, why don’t we have a lot more experimental studies like this? First, the importance of conducting experimental studies to demonstrate with confidence how students are affected by arts activities is just not in the training or experience of most people who work in these organizations. They just don’t know that they could or should be doing experimental studies. Hopefully, our research begins to break that ice.
Second, many people in cultural organizations do not believe that what they do for students can be measured by researchers. Of course, social scientists can never fully capture what people get out of arts activities, but we can catch a glimpse, however imperfectly, with the measures and tools at our disposal. Our past work has documented some of those benefits and, if successfully replicated, future work could do more.
Third, some people in arts advocacy do not see why it is necessary to demonstrate benefits with experimental studies. They think the benefits should be fairly obvious and that non-experimental evidence has been just fine for proving the point. The problem with this view is that the benefits of the arts are not obvious to everyone and not everyone is persuaded by less rigorous research. Given that the arts are under some pressure, it is important that its defenders reach beyond those who already believe or are willing to believe almost any evidence of benefits, and try to persuade those who are open to becoming more supportive if presented with rigorous evidence. Increasingly policymakers, educational leaders, and foundation officials are accustomed to seeing rigorous evidence before they throw their support behind something.
Cultural institutions have an opportunity to produce a lot more of this type of evidence. We are launching a series of new studies as part of the National Endowment for the Arts Research Lab that I direct at the University of Arkansas, but we could use the help of more museums and researchers all over the country in producing more of this work.