The Office of Assessment (OA) presented its Assessment Workshop at the Physician Assistant’s Didactic Orientation on August 30th, 2016 where Dr. S. William Zito, Dean of Assessment, discussed the importance of Assessment at the College. The OA administered a Pre-Assessment Workshop survey to the incoming didactic students one week prior to the Assessment Workshop (receiving a 64% response rate) and its results were then shared with students. Following the Assessment Workshop, a Post-Assessment Workshop Survey was sent out. After being open 5 days, the survey closed with a 35% response rate. In summary, results of both surveys show that after the Assessment Workshop:
- Students’ awareness of the assessment process in the College more than doubled from 41% to 92%
- A higher percentage of students are willing to participate in assessment
- 88% of students now know what the College of Pharmacy & Health Sciences (CPHS) assessment plan is (compared to 28% in the Pre-Workshop Survey)
- Almost 100% of respondents now know that the CPHS has an Assessment Office and a separate Assessment Committee
- 92% of respondents agree that students are involved in improving the culture of assessment, compared to 54% in the Pre-Workshop survey
The Office of Assessment will continue promoting the Culture of Assessment by continuing to participate in future student orientations and in any other opportunities to share the importance of Assessment.
Although all of us in the College of Pharmacy & Health Sciences looked forward to utilizing the new features of SofTest Version 12, the Office of Assessment was notified by Examsoft that all clients must temporarily rollback from SofTest Version 12 (v12) to SofTest Version 11 (v11) while continued improvements are made to the v12 platform. All faculty and student Examsoft users must use SofTest v11 until otherwise instructed to update. The College has been using SofTest v11 since rolling out Examsoft in early 2015, so there will be little to no impact on the student experience this Fall 2016 semester.
If needed, please click here for the links to reinstall SofTest v11 and to access Examsoft’s FAQ section regarding their decision to rollback. Examsoft’s Tech Support can also assist with the installation process at 1-888-792-3926.
If faculty have any questions on Softest v11 use, please reach out to us at the Office of Assessment. We expect an update from Examsoft on SofTest v12’s re-release in the near future and look forward to utilizing this newest version once it is ready.
Examsoft announced its first official, non-beta release of SofTest Version 12 (v12) this past month. All St. John’s University College of Pharmacy & Health Sciences Examsoft student users can expect to use this new version during the fall 2016 semester. SofTest v12 will be available for installation August 8th. As a student, if you are unsure whether your course will be utilizing Softest, please contact your instructor for confirmation. Resources on SofTest v12 use are now available at the Examsoft University site (click here to view). The minimum system requirements to use Softest v12 are listed here.
SofTest v12 contains a variety of improvements. Its interface has been updated with a simpler, more modern layout. Windows and Mac machines’ have a more consistent layout and each includes improved word-processor options for essay questions. Based on student feedback, Examsoft is offering more features during exam administrations and exam reviews. Exam reviews will now allow exam takers to review their notes (written during the exam) at the time of the review. Exam Takers can also view which text was highlighted and which questions they flagged (during the exam’s administration) at the time of the review. The ability to cross out answer choices, highlight in different colors, and view Quick References Guides directly from the SofTest application is another improvement. Other features in SofTest v12, such as zooming in/out on a question stem, suspending an exam, or use of a download password will be released by the end of 2016.
Exam Takers within the St. John’s University College of Pharmacy & Health Sciences will receive communication from the Office of Assessment’s Ms. Carla Hernandez in early August with instructions on how to install SofTest v12 and other pertinent information for its use in the upcoming fall semester. SofTest version 11 will no longer be supported as of September 8th, 2016 and exam takers will be expected to uninstall the application from their machines.
Following the release of SofTest v12, Examsoft also anticipates an update to the Examsoft Online Portal in 2017. Faculty and administrators will be alerted once more information is available.
The Office of Assessment is utilizing the Summer of 2016 to upgrade software and hardware and train faculty in new testing modalities. Recently, the Office completed a hardware and software upgrade to the college’s scantron station. While the hardware upgrade will allow for a more efficient process of grading, the software upgrade of ParScore will allow for more robust data analysis and a heightened experience for faculty utilizing the Office to grade their multiple choice examinations. Data integrity will be enhanced as well, as the new software and hardware combination will allow for an easier backup of data and smoother migrations if necessary in the future.
At the same time, the Office has embarked on the ExamSoft User Development Initiative. This initiative consists of monthly workshops given by the Office of Assessment for faculty interested in ExamSoft. Ms. Carla Hernandez presented on June 28th to a group of faculty, training them in many faculty basics of the ExamSoft portal. This presentation was well received, and will be followed by the continuation of monthly workshops, including another in August. Faculty that attended (as well as all others that have previously used ExamSoft) are now being surveyed on their attitude toward ExamSoft and computer-based testing as a whole. As the Office continues to assist the college in this important initiative, we plan on continuing to train students as well. The Office is planning on assisting faculty in mock examinations and training sessions to new student users in Fall 2016.
Compared to traditional paper testing, Examsoft offers an engaging resource for faculty to share assessment information with students following exams. Feedback received from students via the Examsoft Experience Survey (disseminated by the Office of Assessment) after the Fall 2015 semester demonstrated that 78% of respondents from the College of Pharmacy & Health Sciences were interested in receiving more individualized reports on their performance. Use of the Strength & Opportunity Report can impact student learning as it provides exam takers with a visual aid to identify areas of strength and areas of concern following an assessment. Reviewing the Strength & Opportunity Report can help a student prioritize studying as they can gauge their performance broken down by category.
Faculty in the College has the option to share the Strength & Opportunity Report with every student in the course by posting it directly to their Examsoft Portals after each assessment. Others have also used it during one-on-one student meetings to offer other strategies for improvement. Resources, such as the Strength & Opportunity Report, are encouraged as it enhances student engagement and can help meet the ultimate goal of improving student performance.
With our PharmD Self-Study report now complete and transparency of the utmost importance, we have opened the final report up to all of our stakeholders. Students, alumni and preceptors were sent the final report via DropBox, attached to a bitly link (the Dean sent the report directly to the College’s faculty). Bitly is a web-based link management platform that allows users to shorten URLs for easier sharing of web-based information. A very nice feature of bitly is the ability to track the number of clicks a specific link has received. Within the first two hours, we saw that over 275 clicks of the PharmD Self-Study had been made via bitly! This is an exciting tracking method that has allowed us to confirm that our outreach initiatives are actually reaching our audience. We intend to continue utilizing this type of tracking in the future, as we work toward ever-increasing transparency.
The ACPE site visit for the PharmD program is quickly approaching. A site team will visit the college and meet with all stakeholders including students, faculty, administrators, and preceptors between April 5th and April 7th. The schedule of meetings for these days will be finalized in the next week. These meetings will provide ACPE with information regarding the strengths and weaknesses of the program along with how shortcomings are being addressed. A mock site visit took place in January, which served as a useful way to prepare for the upcoming visit, allowing all stakeholders to become familiar with the process. All participants of the self-study process had the opportunity to ask questions of the mock site-team member and become acquainted with how the actual visit will unfold. The written self-study report is in the final stages of editing and has been submitted to the University administration for review. The Steering Committee will soon review the document prior to its final submission to ACPE in March, ahead of the visit.