Tag Archives: nabp

2014 PCOA Heat Map

A heat map is a visual representation of data, utilizing colors to represent certain values. The assessment team decided a few years ago that a heat map would be a good way to take the complex set of PCOA data provided by NABP, and provide a quick visual overview. We found this colorful visualization to be a useful way to compare our own PCOA results against other schools in the nation. You can read more about “information aesthetics” (defined by Lev Manovich) here: Infosthetics.com.

The idea here, is that by giving a visual representation of data, we can engage our stakeholders, while helping them understand precisely what this data means. As assessment professionals, we often find ourselves mired in the same types of data reports, absorbed often in personality-less information. Our hope is, through the idea of ‘showing’ data, rather than just providing it, we can give our faculty and students important information in a more engaging way.

The College administered the PCOA (Pharmacy Curriculum Outcomes Assessment) examination to P1-P4 pharmacy students in January for the third consecutive year. As stated on NABP’s web site, “The PCOA is a comprehensive tool for colleges and schools of pharmacy to use as they assess student performance in the curricula.” Every year, after this examination is administered at SJU, the assessment team generates a heat map to visualize how our students have fared against the national reference group. This heat map takes advantage of the conditional formatting/color scales in Excel, and allows us to see the overall visual difference between % correct scores for SJU and the national reference. Ideally, this heat map should give a visualization of our curriculum. More specifically, the heat map provides the difference between the college and the national number for the mean percent correct scores for each topic and subtopic within the exam. For example, our P1 students achieved a ‘2’ in microbiology this year, meaning that their microbiology mean percent correct score was 2 points above the national number. The conditional formatting tool in excel takes whatever selection you choose, and conditionally formats it based on the existing numbers. For our purposes, we conditionally format each separate section, so that we could look at the ‘heat’ of individual areas. Each blocked group has been conditionally formatted separately, which explains why one zero may be yellow, while another zero is green.

The PCOA heat map is one way for us to look at our PCOA scores, and gives us something to compare against nationally. As ACPE considers whether or not to include the PCOA in the new standards, it is important for us to be prepared, and utilize the PCOA as a validated assessment tool to compare our curriculum to those across the country. I have attached a Sample School Heatmap, drawn from data provided by NABP on their published ‘Sample Score Report’. This should give a good idea of what we are looking at on our own heat map.

Leave a comment

Filed under Methods, Outcomes Assessment