Psychological assessment has been regarded an essential competency domain in professional psychology. Influenced by the competency-based movement, researchers and educators start to redesign new measurement tools for assessing psychological assessment competencies (PACs) of psychologists from entry to practice. Compared with physicians and nurses, license examinations of professional psychologists still heavily rely on the paper-pencil tests to measure professional competencies of psychologists. In the past five decades, an objective structured clinical examination (OSCE) has become one of the most commonly used measurement tools to assess professional knowledge, clinical skills and professional attitudes of healthcare professionals at the same time. Several meta-analysis studies have confirmed that an OSCE is not only a reliable and valid measurement tool for assessing clinical competencies of healthcare professionals, but also a good teaching aid to facilitate a wide range of clinical skill training courses. 

In Taiwan, OSCEs have been widely used to assess professional competencies of physicians and nursing practitioners, however, its application in professional psychologists is still very limited. Lan et al. (2011) proposed four possible factors affecting the OSCE application in professional psychology. First of all, competencies in professional psychology (e.g., rapport building skills, expressions of empathy) are more abstract and difficult to measure than those in medicine and nursing. Second, it requires longer time and more effort to develop and administer a psychology OSCE than a medical OSCE or a nursing OSCE. Third, faculty members and practicing psychologists need to receive further training (e.g., taking an OSCE rater training workshop) to ensure that they reach inter-rater agreement before scoring student or participant performance on a psychology OSCE. Last but not least, the major obstacle to adopting OSCEs in professional psychology is high cost of running an OSCE on a regular basis. Compared with the resource input of a medical OSCE, the resource input of a psychology OSCE is relatively insufficient. If the cost of running a psychology OSCE could be effectively reduced, it would help to promote the OSCE application in professional psychology. 

To overcome the aforementioned problem, the virtual reality (VR) technology was applied in this study to develop a VR version of the Intake OSCE (VR-intake OSCE) to serve this need. The purpose of the present study was to examine the feasibility of the VR technology in assess competencies in psychological assessment through the viewpoints of students and psychologists. The present study comprises three phases: the testing phase 1, the testing phase 2 and the formal study phase. 

In the testing phase 1, six psychologists from master education to practice were hired as mock test takers to examine the versatility of the intake OSCE in assessing PACs of psychologists from counseling and clinical psychology. The intake OSCE was developed by Lan et al. (2015) to examine psychologists’ PACs when conducting an intake interview. This test consists of a standardized patient test (SP test), a follow-up short answer test (S test) and a multiple-choice test. In the testing phase 1, only the SP test and the S test were selected to examine the versatility of the intake OSCE in assessing PACs of psychologists from counseling and clinical psychology. No significant score differences were found between clinical psychologists and counseling psychologists. This result confirmed that the intake OSCE could be used to assess psychologists in both fields. 

In the testing phase 2, the same mock test takers (from the testing phase 1) were hired to evaluate the initial version of the VR-intake OSCE, which consists of a virtual standardized patient test (VSP test) and a follow-up S test. To administer a VSP test on the VR equipment, a new VSP system (called VR psychological assessment room) was developed in the present study to serve this need. By adopting the automatic speech recognition technology, the design of the VSP system allows participants to conduct an intake interview with the VSP through a free questioning format, and collect assessment related information (e.g., personal medical history, stressors, and social support) in order to generate a differential diagnosis of the VSP and also provide an initial treatment plan. All mock test takers first took the VR-intake OSCE, and then filled out the post-test questionnaire. This post-test questionnaire was designed to evaluate the quality of the initial version of the VR-intake OSCE and its test administration procedure. 

Responses provided by mock test takers were summarized as the followings. First of all, mock test takers considered the VR-intake OSCE did not only measure their PACs, but also their abilities to use the VR equipment and the VSP system. For psychologists with little or no VR experience, their test scores might be affected by unfamiliar use of the VR equipment and the VSP system. Second, a mock test taker reported that she felt dizzy and discomfort similar to motion sickness after taking the VSP test, and then affected her performance on the follow-up S test. Third, most of the mock test takers agreed that the VR-intake OSCE could be used to replace some assessment functions of the intake OSCE when its VSP system has been fully developed. 

Based on responses provided by mock test takers, the test administration procedure of the VR-intake OSCE in the formal study phase was revised as the followings. First, an additional practice VSP test would be provided to allow participants to familiarize the VR equipment and the VSP system before taking the VR-intake OSCE. Second, an intermission session between the VSP test and the follow-up S test would be extended from 5 minutes to 10 minutes to help participants to relieve possible dizziness and discomfort before taking the S test. 

In the formal study phase, a sample of 37 participants was recruited on a voluntary basis. All participants were divided into three groups (i.e., clinical student group, counseling student group, and counseling psychologist group) to examine the feasibility of the VR technology in assessing PACs of psychologists and the design of the VSP system. All participants first took the VR-intake OSCE, and then filled out a post-test questionnaire, which was designed to examine the feasibility of the VR technology in assessing PACs and the design of the VSP system. 

Results of the formal study revealed that the VR-intake OSCE was more feasible in assessing cognitive aspects of PACs (e.g., differential diagnosis and integrated inference) than behavioral aspects of PACs (e.g., communication skills and professional attitudes). In terms of the design of the VSP system, participants gave the “user interface” design and the “help function” design the highest evaluations. They also suggested that the designs of “voice recognition” and “interactivity” needed further improvement. 

From the viewpoints of students and psychologists, the application of VR technology in assessing PACs of psychologists from entry to practice is feasible. The cost-effective effect of using the VR-intake OSCE to replace the intake OSCE can be further investigated after its VSP system has reached a mature state. In the near future, researchers and educators who are willing to devote themselves to the VR application in psychology OSCEs can develop different types of VSP systems in order to assess a wide range of professional competencies (e.g., intervention, consultation, and supervision/teaching). They can also apply the VR technology to different types of clinical training courses to enhance the training effects and enrich in-class activities.