The Office of Assessment is in the midst of our freshman assessment orientations. These short workshops have proven each year to be great opportunities for the Office of Assessment to go into freshman classrooms to introduce ourselves, and what we do. They also provide us with a nice forum to introduce ExamSoft as the college’s computer-based testing software, allowing students dedicated time to ask questions about its utility while being trained. With assessment being an area that often limits us to indirect student interaction, it is always a really great start to each academic year to be able to have face-to-face time with our students. Feedback from these meetings is typically positive, and tends to leave students in a better state of understanding concerning assessment and what it means for our college.
The ExamSoft Faculty Experience
The Office of Assessment within our college takes the faculty experience in computer-based testing through ExamSoft, very seriously. We are always working hard toward helping our faculty achieve their exam administration goals, by giving them all the information and training they need to make their exam building and administration experience go smoothly.
Once a year, we ask all faculty that have ExamSoft accounts as well as those that have shown interest in using the software, about their experience with it. Recently, we completed our annual survey, and the results continue to show that our work in this arena is paying off. We achieved a 41% response rate to the survey, receiving 48 responses out of 118 invitations sent. Below are some of the highlights:
- 87% had positive experience with Examsoft during the spring 2018 semester.
- 89% strongly/agree to using Examsoft in more courses.
- 88% used reporting tools during the Spring 2018 semester to assess student and exam performance
- 75% are interested in utilizing ExamSoft to collect documentation for accreditation by mapping curriculum and syllabus/teaching objectives to specific student learning outcomes.
- 75% are interested in attending future Examsoft workshops led by the Office of Assessment.
We continue to be motivated to work hard to push the envelope with computer-based testing in our college, as we move away from using paper in higher education. With positive survey results like those seen above, we can draw on the positive experiences and attitudes of faculty to continue on and look forward to making change not only within the realms of assessment and data analysis, but also concerning exam administration at-large.
SJU Faculty Technology Forum
The Office of Assessment will be presenting on the college’s IPE efforts utilizing ExamSoft this Monday, October 23rd at the SJU Faculty Tech Forum. The SJU Tech Forum is an annual opportunity for stakeholders at SJU to take a look at technologies that faculty are currently using in their classes and colleges. The forum is a wonderful opportunity for us to highlight some of what we do in terms of assessment and technology, and we look forward to sharing our future plans for ExamSoft and IPE. This presentation will particularly focus on how we are using computer-based testing software to measure, track and report our data, ultimately leading toward greater tracking of programmatic goals and outcomes.
Office of Assessment visits Examsoft’s Assessment on Tour
The College of Pharmacy & Health Sciences has been utilizing the ExamSoft platform since January 2015 and has seen its use expand greatly across the College’s four programs- Pharmacy, Physician Assistant, Radiologic Sciences, and Clinical Lab Sciences. As it is adopted by more faculty and used in other courses, the Office of Assessment (OA) continues to immerse itself in computer-based testing while also adopting our own best practices to introduce initiatives that suit the unique needs of our student and faculty users.
ExamSoft recently visited New York City to host its Assessment On Tour (AOT) series and provided insightful presentations from other schools utilizing the platform. Soon after Carla Hernandez, a member of the College’s Office of Assessment, was invited to speak at October 2017’s AOT at the University of North Carolina- Charlotte campus. Carla shared St. John’s University’s College of Pharmacy & Health Sciences’ experience with ExamSoft over the last 2 ½ years. She specifically presented the benefits of analyzing student assessment data through ExamSoft’s reporting tools. The OA aims to continue working with all stakeholders to improve the long-term rollout of ExamSoft at the College.
Examsoft’s Assessment Conference 2017
The Assessment Team just got back from Denver, Colorado after taking part in Examsoft’s Assessment Conference 2017. The conference included an opportunity for us to present on a recent interprofessional education (IPE) project. On top of that, we also had the chance to hear from fellow universities and learn more about the upcoming release of Examsoft products.
On behalf of the Assessment Team, Carla Hernandez, Anthony Marziliano, and Dr. Marc Gillespie presented “Identifying Interprofessional Education Opportunities (IPE) Using ExamSoft Categories that Bridge Goals and Outcomes of Healthcare Programs”. In an effort to summarize our project, interprofessional education is when students from two or more professions learn about, from, and with each other to enable effective collaboration and improve health outcomes (World Health Organization, 2010). Earlier this year we began reviewing national exam blueprints from each of the College’s professional programs to identify overlapping content areas between them. We were seeking to identify content areas across all programs that are targets for IPE. Along with the expertise of our program directors and mind-mapping software, we were able to refine and ultimately identify 21 content areas ideal for IPE.
Pairing this analysis with Examsoft’s ability to categorize test items & run longitudinal analyses, we are able to assess the effectiveness of IPE activities administered through the platform. As for next steps, the Assessment Team met with the College’s IPE Committee, IPE Director, and other faculty to present our findings. We look forward to continuing discussions with key stakeholders on how we can contribute to the College’s continued efforts of expanding IPE development & assessment.
Overall, Examsoft’s Assessment Conference 2017 was well worth the trip- an invitation to share our analysis, participate in engaging sessions, and the chance to network with fellow assessment folks. We look forward to next year’s conference. #Assess2017
2017 Examsoft Assessment Conference
The Office of Assessment is proud to announce that we have been invited to deliver a 1-hour presentation titled, “Identifying Interprofessional Education Opportunities (IPE) Using ExamSoft Categories that Bridge Goals and Outcomes of Healthcare Programs” at the 2017 Examsoft Assessment Conference (EAC) in Denver, Colorado this June. For more information about the EAC sessions, please visit their site here. The College of Pharmacy and Health Science’s Assessment Team looks forward to connecting with colleagues from other universities nation-wide to discuss how their assessment efforts drive change at their institutions.
Students & Softest: “How do I prepare for exam day?”
Whether students are sitting for their first exam via SofTest or they’ve been using it for multiple semesters, understanding exam day protocol is key to having a smooth exam (and not adding to those exam day jitters). SofTest is an Examsoft application used by the College of Pharmacy & Health Sciences to administer computer-based assessments. To promote consistency across students of all programs, the Office of Assessment has outlined the College’s Softest exam-day process:
What should you do prior to exam day? Ask yourself the following questions:
“Is my laptop in good shape?” As a student, it is up to you to ensure your laptop is in proper working order and meets SofTest minimum system requirements- click here for specifics. If your laptop is having technical problems prior to using SofTest, it may not be fit for exam day use. Alert your professor immediately if this is the case. They may be able to request a loaner laptop on your behalf with sufficient notice.
“Are there any pesky computer updates I should take care of before the exam?” Ensure that all necessary updates are installed on your laptop before exam day, so that there’s no delaying the start of your exam.
“Is my exam available for download before exam day?” Professors tend to make exam files available for download sometime before exam day, so students are advised to download it at home the night before exam day. Remember that internet connectivity is required to download the exam file! Click here to learn more about downloading an exam file.
What should you do on exam day?
- Starting with the most obvious, bring your laptop & laptop charger on exam day. We recommend your machine be fully charged, just in case there is a temporary power outage.
- Anti-virus applications may effect SofTest performance during exams. When taking a secure exam, disable antivirus software(s) before taking an exam & re-enable it after exiting your exam.
- Put away cell phones, smart watches, notes, etc. Follow any other security protocols announced by your professor. During an exam, cell phones and any other forms of wearable technology left ‘on your person’ will not be tolerated (this includes your pocket, your shoes, etc.) – place them away from your desk.
- Open SofTest, click ‘Take an Exam’ and select today’s exam. Await instruction from the proctor to enter the Assessment Password and start the exam once you get permission. Good luck!
- Once you finish your exam and exit Softest, you must show evidence of the green ‘Congratulations’ screen to a proctor before leaving the exam room. This is a crucial step.
As an exam taker, is there anything else I should know?
- Once in a blue moon during an exam, your SofTest may freeze or experience a technical issue. First step: remain calm. If your computer is unresponsive for more than two minutes, please alert your professor by raising your hand using two fingers. Most computer freezes usually just require a hard reset (hold power button down until laptop shuts off and then turn it back on). SofTest will automatically open up & you’ll continue right where you left off. No need to worry about losing your answers because SofTest saves them every 60 seconds. If you have any concerns about the issue you experienced mid-exam, contact Examsoft Tech Support after your exam to have the SofTest issue further investigated.
- Your answer file has not been received by your professor unless you both see the green ‘congratulations’ screen. If your laptop has trouble reaching it, first check your internet connectivity. Many times your laptop has not yet reconnected to wi-fi and you simply need to try uploading your answer file again once it connects.
- Lastly, it is best to contact Examsoft’s Tech Support if you ever have any issues using SofTest. Their team is best suited to troubleshoot the issue and provide you with a solution quickly. They can be reached at 1-866-429-8889.
Providing our students with the resources to feel confident taking computer-based exams is a top priority. Stay tuned for the Office of Assessment’s End-of-Semester Examsoft /SofTest Survey and provide the College with your feedback and suggestions.
Graduating Student Exit Interviews
The College of Pharmacy and Health Sciences is once again in the throes of collecting Exit Interview data as we approach graduation. These faculty/administrator-led interviews allow us to hear directly from the best source of information about our programs – the student.
The Office of Assessment (OA) is currently collecting graduate student exit interview data for the 5th consecutive year. As this program has transitioned from a pilot to a yearly occurrence, the OA continues to work with faculty to conduct meaningful and deep interviews of students within our college that have finished their MS or PhD in any one of our graduate programs. Along with the long-standing PharmD Exit Interview process, the OA and Committee on Assessment and Outcomes takes pride in being able to provide qualitative student-generated data back to program directors and departments, so that change can be made within programs. We often hear that this information informs our yearly assessment matrix process, as programs monitor their learning goals and outcomes. Real change starts with data collection, and these interviews allow us to work toward closing the loop.
This year’s PharmD Exit Interview will kick off on March 3rd.
Examsoft User Development Initiative
As Examsoft use has continued to expand each semester at the College of Pharmacy & Health Sciences, the Office of Assessment launched the Examsoft User Development Initiative (ESUDI) during the Summer of 2016 based upon a recommendation from Dean Digate. The goal of the ESUDI is to provide hands-on faculty training workshops, as well as, other resources to ensure all users (new and returning) within the College are confident in Examsoft use. We aim to provide basic tutorials on how to use the Examsoft Portal and in-depth discussions with faculty (of all disciplines) on how they can utilize the reporting features to better track student learning outcomes and further engage their students throughout the semester.
The first ESUDI session was held June 28th, 2016. The session was well received and the Office of Assessment followed this up with a second summer session on August 18th, 2016. We will continue offering monthly workshops discussing different topics during the Fall 2016 semester available to all Examsoft Faculty users in the College. Future topics to be discussed will include Examsoft’s Reporting Tools; SofTest: The Student’s Perspective; Strategies for Team Taught Courses; Tracking Student Performance & Learning Outcomes via Categories; and many more workshop tailored to faculty’s needs here at the College.
The Office of Assessment will also be utilizing Blackboard tools to provide faculty with other interactive resources. Further communication will be provided once ready for faculty use.
Examsoft faculty users can expect their invitations to the Fall 2016 workshops shortly with specific time/location information. We hope to see you all there!