The Office of Assessment is utilizing the Summer of 2016 to upgrade software and hardware and train faculty in new testing modalities. Recently, the Office completed a hardware and software upgrade to the college’s scantron station. While the hardware upgrade will allow for a more efficient process of grading, the software upgrade of ParScore will allow for more robust data analysis and a heightened experience for faculty utilizing the Office to grade their multiple choice examinations. Data integrity will be enhanced as well, as the new software and hardware combination will allow for an easier backup of data and smoother migrations if necessary in the future.
At the same time, the Office has embarked on the ExamSoft User Development Initiative. This initiative consists of monthly workshops given by the Office of Assessment for faculty interested in ExamSoft. Ms. Carla Hernandez presented on June 28th to a group of faculty, training them in many faculty basics of the ExamSoft portal. This presentation was well received, and will be followed by the continuation of monthly workshops, including another in August. Faculty that attended (as well as all others that have previously used ExamSoft) are now being surveyed on their attitude toward ExamSoft and computer-based testing as a whole. As the Office continues to assist the college in this important initiative, we plan on continuing to train students as well. The Office is planning on assisting faculty in mock examinations and training sessions to new student users in Fall 2016.
Compared to traditional paper testing, Examsoft offers an engaging resource for faculty to share assessment information with students following exams. Feedback received from students via the Examsoft Experience Survey (disseminated by the Office of Assessment) after the Fall 2015 semester demonstrated that 78% of respondents from the College of Pharmacy & Health Sciences were interested in receiving more individualized reports on their performance. Use of the Strength & Opportunity Report can impact student learning as it provides exam takers with a visual aid to identify areas of strength and areas of concern following an assessment. Reviewing the Strength & Opportunity Report can help a student prioritize studying as they can gauge their performance broken down by category.
Faculty in the College has the option to share the Strength & Opportunity Report with every student in the course by posting it directly to their Examsoft Portals after each assessment. Others have also used it during one-on-one student meetings to offer other strategies for improvement. Resources, such as the Strength & Opportunity Report, are encouraged as it enhances student engagement and can help meet the ultimate goal of improving student performance.
With our PharmD Self-Study report now complete and transparency of the utmost importance, we have opened the final report up to all of our stakeholders. Students, alumni and preceptors were sent the final report via DropBox, attached to a bitly link (the Dean sent the report directly to the College’s faculty). Bitly is a web-based link management platform that allows users to shorten URLs for easier sharing of web-based information. A very nice feature of bitly is the ability to track the number of clicks a specific link has received. Within the first two hours, we saw that over 275 clicks of the PharmD Self-Study had been made via bitly! This is an exciting tracking method that has allowed us to confirm that our outreach initiatives are actually reaching our audience. We intend to continue utilizing this type of tracking in the future, as we work toward ever-increasing transparency.
The ACPE site visit for the PharmD program is quickly approaching. A site team will visit the college and meet with all stakeholders including students, faculty, administrators, and preceptors between April 5th and April 7th. The schedule of meetings for these days will be finalized in the next week. These meetings will provide ACPE with information regarding the strengths and weaknesses of the program along with how shortcomings are being addressed. A mock site visit took place in January, which served as a useful way to prepare for the upcoming visit, allowing all stakeholders to become familiar with the process. All participants of the self-study process had the opportunity to ask questions of the mock site-team member and become acquainted with how the actual visit will unfold. The written self-study report is in the final stages of editing and has been submitted to the University administration for review. The Steering Committee will soon review the document prior to its final submission to ACPE in March, ahead of the visit.
The College of Pharmacy and Health Sciences continues toward the completion of a self-study document geared toward ACPE and reaccreditation of our PharmD program. The Steering Committee completed work this week on the latest round of edits, which included clarification of documentation and appendices. Yesterday, the report was moved into the hands of Faculty Council for review and approval. This document and its appendices have been uploaded to the shared Faculty Council Drive for easy access, and transparency. Over the course of the next few weeks, the faculty will review this massive document, making comments, and suggesting edits to the College’s response to the standards. Once these edits are incorporated, the Steering Committee will provide the report to the upper administration of the University for final approval, before sending it to ACPE. Our site visit will take place in the Spring.
Following the initial roll out of Examsoft earlier this year, the computer-based testing platform has been implemented in the Fall 2015 semester for third year Radiologic Sciences courses, Physician Assistants’ incoming didactic courses, and select PharmD program courses. SofTest exam administration is still being utilized in PA clinical year coursework. Based on feedback from the pilot period’s cohort, the Office of Assessment provided student training sessions for the didactic PA students and incoming professional Radiologic Sciences students during their respective program orientations. A separate training opportunity was offered to students of the two PHR 4109 sections utilizing SofTest. Mock exam(s) were administered to familiarize students with SofTest’s interface, testing protocol, and to identify any machines/technical issues which may be preventable in future testing environments. Softest exam administrations are already underway for the Fall 2015 semester and its focus remains on faculty training and student engagement.
Though it has been sometime in the making the Federal College Scorecard site is up and running.
The site uses a combination of Federal data sources including loan and tax return data. The upside of this is the documentation that goes with the site here, contains a wealth of information on how the analysis was done, what data sources were used, and instructions how to tap into (or download) the data using a pretty straightforward API.
There is much discussion concerning the pros and cons of using the data this way. Each position is worth a read. You can read a sample of the concerns from the New York Times, Wall Street Journal, and the Chronicle of Higher Education. The issues raised include fears that this represents another step in the view that Higher Ed Institutions can be viewed as and shopped for. While there is lots of data present here, there is also the underlying worry that the data is incomplete, or provides a skewed view of the institutions.
As with any new data source it is worth a look, especially with the open nature and full descriptions of the data used.
Site – College Scorecard
SJU Information – St. John’s University
There are also a number of nice US Department of Education Blog posts available on their HomeRoom Blog. For College Scorecard specific posts, have a look here.
This type of data is definitely the direction that the new knowledge based society is taking us in. It provides a snapshot of how students fair after school and it deeply lists the data sources that were used. Incomplete or not, there is only going to be more data like this released and used.