Journal of Pharmacy Practice and Pharmaceutical Sciences

Review Article

A Narrative Overview of the Effectiveness of E-learning in Pharmacy Education

Conor O’Hare1and Briegeen Girvin1*

1School of Pharmacy, Queen’s University, Belfast, UK

Received: 31 August, 2018

Accepted: 24 October, 2018

Version of Record Online: 19 November, 2018

Citation

O’Hare C, Girvin B (2018) A Narrative Overview of the Effectiveness of E-Learning in Pharmacy Education. J Pharm Pract Pharm Sci 2018(1): 01-09.

Correspondence should be addressed to

Briegeen Girvin, UK

E-mail: b.girvin@qub.ac.uk

DOI: https://doi.org/10.33513/PPPS/1801-03

Copyright

Copyright © 2018 Briegeen Girvin et al. This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and work is properly cited.


Abstract

In response to the increasing use of a range of technologies within pharmacy education, this review aimed to establish the effectiveness and acceptance of e-learning within pharmacy education and to identify limitations within the research carried out since 2013. The e-learning interventions studied were defined as any online or computer-based technology* used to deliver pharmacy education to pharmacy students or qualified pharmacists in a remote (out of classroom) setting. Evidence of short-term effectiveness of e-learning was found and a small number of studies provided evidence of long-term effectiveness. In comparisons, e-learning was demonstrated to be at least as effective as traditional face-to-face teaching methods and superior to no training at all. Six studies also demonstrated that e-learning could result in a change of pharmacists’ practice, while no evidence was found that e-learning could lead to patient benefit or improvements in care. E-learning also appears to be an engaging learning method, which is generally well received among participants. Despite a significant increase in the number of publications investigating e-learning within pharmacy education in the past five years, further research is still required to address limitations within the current literature and to fully establish the effectiveness of e-learning within pharmacy education.

*excluding technologies that are recreational in nature, e.g. computer games.

Introduction

E-learning can be simply defined as learning which is facilitated by technology. This can be either synchronous or asynchronous, and includes both fully online learning and also blended learning, where e-learning is used in combination with traditional face-to-face teaching [1]. Educators appear to be drawn to the unique advantages afforded by e-learning [2,3] and the development of new and effective technologies is driving its increased adoption within higher education. This is reflected by the large volume of e-learning literature in recent years [1,4]. Within pharmacy education, e-learning has been integrated into undergraduate programmes, pre-registration training and continuing education and it appears reasonable to assume that this trend will continue. With decreasing resources in pharmacy education, e-learning may be seen as an efficient method to deliver pharmacy undergraduate curricula to free up time for ‘active’ learning strategies. There are potentially numerous educational and individual benefits from e-learning - flexibility being high up on the list. For example, pharmacists can take part in e-learning at home or from their workplace, there is less time and cost involved as there is no need to travel and there is more choice as online events are not restricted by location.

There have been huge technological advancements in the past 5 years and marked changes predicted in how people will work in the future [5]. The growing trend towards more remote and more flexible working comes with a requirement for new knowledge and skills and more flexible and accessible methods of learning for the acquisition of these skills [5]. It is therefore timely to examine the literature relating to the effectiveness of e-learning in pharmacy education in the five-year period since the publication of the only other review into this area by Salter and colleagues [6]. This review concluded that further research was required to establish the long-term effectiveness of e-learning and whether e-learning could result in changes in pharmacists’ practice or patient benefit. Therefore, establishing whether any progress had been made in these areas was of particular importance. In the review by Salter and colleagues [6], e-learning interventions were evaluated according to an adaptation of Kirkpatrick’s four levels of training evaluation model [7], which may be applied to higher education for the purposes of assessing educational effectiveness [8]. The model appraises training programmes on the basis of four potential outcomes:

  1. Reaction, which relates to participants’ satisfaction and opinions.
  2. Learning, which measures an actual or perceived change in attitudes, knowledge or skills of participants.
  3. Behaviour, which assesses the application of knowledge or skills gained from the programme.
  4. Results, which evaluates the extent to which the specific goals are achieved, as a result of the programme [8].

It was our aim to review the literature published since the Salter review [6] and apply the same classification of effectiveness (the adapted Kirkpatrick’s four level model) used by Salter and colleagues to subsequent studies around e-learning in pharmacy education.

Method

The purpose of this literature review was to provide a summary of, and identify gaps within, the literature around the effectiveness of e-learning in pharmacy education between 2013 and 2018. This time period was chosen, as a systematic review has previously examined the literature relating to the effectiveness of e-learning in pharmacy education before 2013 (search date of 4th June 2013)[6].

The definition of e-learning intervention and the search process, including databases used and search terms are detailed in table 1. Studies returned by this search were then reviewed to determine if they met pre-defined inclusion criteria.

Inclusion and Exclusion Criteria

Any study designed to evaluate the effectiveness of e-learning within the pharmacy profession was considered for inclusion in this review. Participants had to be pharmacy students, pharmacists or pre-registration pharmacists. Studies had to measure participants’ perceptions of an e-learning intervention and/or the effect of an e-learning intervention on participants’ knowledge, skills or confidence. Ideally studies would also have investigated the impact of the e-learning intervention on participants’ practice and/or if the e-learning intervention resulted in patient benefit or an improved quality of care. Studies involving other health professionals/students, other pharmacy staff, faculty members or patients were not included, with the exception of one study where a control group of medical students received no training [9]. Studies using technologies that were based solely in a classroom (not remote) setting were excluded. Studies utilising technologies that are recreational in nature, such as computer games, were not included in the review. Studies which examined ways to improve e-learning, factors affecting the success of e-learning or opinions of e-learning in general (not a specific intervention), were not included. Studies which used technology that was not online or computer-based, such as high-fidelity simulations, were also not included. Studies that were unpublished or were not in English, were excluded from the review.

The full text of each included study was subjected to an in-depth review. During this review, key information was extracted from each study including the number of participants, topic, type of e-learning intervention, whether or not a control group was included, type of assessment of effectiveness and classification of effectiveness according to an adapted model of Kirkpatrick’s four levels of training evaluation model. The limitations of the various studies were identified and summarised.

Bibliographic databases

Medline, Web of Knowledge, ScienceDirect, Google scholar

Search terms

Online pharmacy education, virtual pharmacy education, online pharmacy learning, virtual pharmacy learning, pharmacy e-learning

Search date

5thto 9thMarch 2018

Participants

Pharmacists, Pharmacy students, pre-registration pharmacists (9241 in total).

Study locations were international.

E-learning intervention

Any online or computer-based technology* used to deliver pharmacy education to pharmacy students or qualified pharmacists in a remote (out of classroom) setting.

 

*excluding technologies that are recreational in nature e.g. computer games.

Outcomes measured (Kirkpatrick’s model [8])

1.       Reaction

 

2.       Learning

 

3.       Behaviour

 

 

4.       Results

 

 

Learners’ views about the e-learning program, including experiences and satisfaction with the topic and e-learning technology.

 

Change in attitudes, knowledge or skills after training.

 

Transfer of knowledge to the workplace (includes willingness to apply learning in the workplace)

 

Changes in organisational practice (e.g., delivery of care) and patient outcomes as a result of the program.

Year of publication

2013 - 2018 (excluding any studies already reviewed by Salter and colleagues [6])

Table 1: Search process and inclusion criteria.

Results

The initial search process returned 2207 studies, which were then screened according to the inclusion and exclusion criteria and duplicate studies were excluded. The literature search flow diagram is shown in figure 1. Fifty-six studies were subsequently deemed suitable for inclusion in the review. Table 2 (Appendix) provides a summary of the studies, including their location and their effectiveness according to Kirkpatrick’s four levels of training evaluation model [8]. The majority of studies (70%) were conducted in the United States of America. Studies varied widely in terms of their participants, the type of e-learning intervention and the measures of effectiveness.

a-narrative-overview-of-the-effectiveness-of-E-learning-in-pharmacy-education

Figure 1: Literature search flow diagram.

Participants

Eight (14.3%) studies involved pharmacists [10-17], one (1.8%) involved pre-registration pharmacists [18], forty-six (82.1%) involved pharmacy undergraduate students [19-64] and one (1.8%) involved both pharmacists and pharmacy students [9]. The number of participants ranged from eight in the smallest study [34], up to seven hundred and ninety-two in the largest [13].

Intervention

Studies included a wide variety of topics covering many aspects of both undergraduate and postgraduate pharmacy education. The most common topics were ambulatory care, pharmaceutical calculations and pharmaceutical compounding.

A diverse range of technologies were used as the e-learning intervention, but the two most common were online virtual patient cases and online courses. In twelve (21.4%) studies, [22,25,33,34,36,52-55,57,58,63] e-learning interventions were employed alongside face-to-face teaching or group sessions, as part of a blended approach. Twenty-three (41.1%) studies [9,11,13,15,19,22,28,30-33,35,42-45,49,52-54,58,59,63] directly compared e-learning to face-to-face learning or no training at all.

Outcomes

Forty-one (73.2%) studies [10-14,16,17,19-22,25-30,33,35,37-41,43,44,46-48,50,52,55-64] assessed participants’ satisfaction with and opinions of the e-learning intervention (Reaction). This was assessed subjectively through pre-, mid- and post-course surveys or post-course qualitative interviews. In all but one study [55], participants reacted positively to the e-learning intervention. In this study, Khanova et al. [55] examined the delivery of a blended pharmacotherapy course to pharmacy students. Online learning modules were completed prior to face-to-face sessions, which were then used for discussions and active learning, rather than delivery of course material. Students expressed dissatisfaction with the self-directed nature of the course, feeling that it led to the role of lecturers being excessively diminished. The positive reaction in the remaining studies was primarily satisfaction with the effectiveness of the e-learning methods and quality of the learning of experience provided.Studies also identified a number of advantages of e-learning, which include increased convenience [10,12,17,21,28,29,38,52,56], the ability to protect ‘in class time’ [20,33,35,50,55,57,63], the empowerment of students to take responsibility for their own learning [17,20,33,35,50,56,57,63] and the opportunity for students to learn at their own pace [12,17,21,28,29,52,56,58,60].

Forty-nine (87.5%) studies [6,9-20,22-25,27-37,41-43,45-54,56-59,61-64] assessed the effectiveness of the e-learning intervention in terms of Learning and/or Behaviour. The e-learning intervention was deemed to be effective if the performance of the intervention group increased significantly post-intervention; and/or was at least as good as that of the control group post-intervention. In thirty-eight (67.9%) studies, [9,10,13,15,18-20,22,24,25,27,28,30-37,41-43,45-49,51-54,56-58,61-63] changes in participants’ attitudes, knowledge or skills (Learning) were assessed objectively. Objective assessments included overall course grades; mock clinic visits; and pre-, mid- and post- intervention knowledge tests. Thirty-five (62.5%) of these studies [9,10,13,15,18-20,22,24,25,27,28,30-33,35-37,41,43,45-49,51-54,59-58,61,63] found the e-learning intervention to be effective in increasing the knowledge or skills of the participants. In 2 of the 3 studies that did not have a positive outcome, Weaver et al., [34] and DeMella et al., [62] examined the delivery of a blended health policy course and an online ethics course, respectively, to pharmacy students. In both studies students completed pre- and post-course assessments and there was not a statistically significant difference between pre- and post-intervention performance. However, DeMella et al., [62] did find that students felt more confident managing ethical dilemmas following completion of the e-learning intervention. In the third study that did not have a positive outcome, McCabe et al., [42] compared the face-to-face and online delivery of a self-care and non-prescription medications course to pharmacy students. Students completed pre- and post-course knowledge tests. Students in the face-to-face cohort scored higher than those in the online cohort and there was no significant difference between the pre- and post-intervention scores of the online cohort.

Learning was also assessed subjectively, through pre-, mid- and post-intervention surveys in twenty-six (46.4%) studies [11,12,14-19,23,25,29,30,35,37,41-43,45,46,50,52,56,57,59,62,64] and all but two [19,42] of these studies reported that the e-learning intervention was effective. Taglieri et al., [19] investigated the effect of completion of online virtual patient cases on student performance and confidence in mock clinic visits. Despite improved performance in mock clinic visits compared to a control group who did not complete the virtual cases, there was no change in student confidence post-intervention. Four (7.1%) studies [9,12,18,51] demonstrated that the effect of the e-learning intervention could be maintained beyond the short-term.

Twenty-three (41.1%) studies [9,11,13,15,19,22,28,30-33,35,42-45,49,52-54,58,59,63] compared e-learning to face-to-face training or no training, with twenty (35.7%) [9,13,15,19,22,28,30-33,35,43,44,49,52-54,58,59,63] finding it to be at least as effective and well accepted as face-to-face learning. Buxton et al., [11] compared an online synchronous lecture on drug diversion to the same lecture, delivered face-to-face. Pharmacists in both groups reported significant learning, but those who attended the face-to-face lecture reported a higher degree of learning. Al-Dahir et al., [45] compared the effect of online virtual patient cases, completed individually, on pharmacy students’ knowledge of atrial fibrillation against the effect of paper-based cases, completed as teams. Students who completed the paper-based cases outperformed those who completed the online virtual patient cases in a post-intervention assessment. One study also demonstrated that e-learning was superior to no training at all [9].

In six (10.7%) studies, [11,12,16,23,37,59] participants indicated their intention to change their practice in response to knowledge or skills gained from the e-learning intervention (Behaviour). In one study [14] ,participants reported an actual change in their practice post-intervention. Behaviour was assessed subjectively, through mid- and post-intervention surveys.

Limitations of the Studies

Of the fifty-six studies that were reviewed, all were limited by methodological flaws. Thirty (53.6%) studies [11,12,14-17,19,20,22,27-33,35-37,43,46,47,49,52-54,56,58,63,64] did not establish the baseline knowledge, confidence or skills of participants and this is significant in two regards: It does not allow accurate comparison between groups, as differences in baseline knowledge may be responsible for differences in final knowledge; and it does not allow the impact of the intervention in increasing students’ performance to be quantified. Three (5.4%) [25,55,57] of the nine studies that reviewed blended learning methods, failed to assess the effectiveness of the e-learning intervention independently of the face-to-face teaching. Twenty-six (46.4%) studies [11,12,14-19,23,25,29,30,35,37,41-43,45,46,50,52,56,57,59,62,64] measured the effectiveness of e-learning intervention on the basis of self-reported data. These findings are therefore vulnerable to response bias as well as recall bias, when participants were required to make a post-intervention assessment of their pre-intervention skills, knowledge or confidence [65,66]. Several studies were limited by their small sample size [11,16,29,30,34,36,59,61] and only four studies [9,32,43,57] reported calculation of sufficient power to detect a statistically significant difference. Twelve studies (21.4%) had a poor response rate (less than 60%) [9,10,12,14,18,19,22,39,40,46,60,62] and four (7.1%) studies [26,29,37,53] did not report response rate. A potential explanation for poor survey response rates is that those participants who were not satisfied with the e-learning intervention were less likely to reply to the survey [67]. This seems plausible given that all but one study [55], found that participants had a positive opinion of e-learning.

Fourteen (25%) studies [9-14,16,38,41,44,47,49,52,54] involved elective courses and therefore, there was a risk of selection bias. Participants in these studies were a self-selected group and it is likely that they already had a positive opinion of e-learning and a high degree of interest in the course content. Thirty-three (58.9%) studies [10,12,14-16,18,19,21-23,25,27,32,34-39,44-46,48,49,53-55,58-60,62,66,67] did not include a control group and in all studies which did have a control group, the nature of e-learning meant that it was not possible to blind participants to whether they were in the control or the intervention group. For the same reason, it was also not possible to blind researchers to group allocation. In four (7.1%) studies [9,11,13,15] participants were not randomly assigned to the intervention or control group, but instead on the basis of their location. In five (8.9%) studies [22,30,31,35,54] the control group consisted of participants who had not been randomly assigned but instead had completed the face-to-face equivalent of the intervention in a previous year. A failure to blind researchers or participants, to include a control group and/or to randomise participants to the control group means that study outcomes cannot be attributed solely to the intervention, as other factors may have impacted on the results [68]. This is particularly significant for studies which relied on self-reported data to measure the effect of the intervention. The methodological flaws of studies were not always identified by researchers and threaten both the internal and external validity of their findings.

Strengths of the Studies

Despite the limitations of the studies, many have been properly designed so as to reduce the risk of bias and secure the validity and reliability of the results. Twenty-six (46.4%) studies [9,10,13,18,21,23-26,34,38-42,44,45,48,50,51,55,57,59-62] established baseline knowledge of participants. Six (10.7%) [22,33,53,54,58,63] of the nine studies reviewing blended learning methods assessed the e-learning intervention independently of the face-to-face teaching method. Forty-eight (85.7%) studies had a sufficiently large sample size and thirty-one (55.4%) studies [9,10,13,20-22,24,26-28,31-34,36,38-40,44,47-49,51,53-55,58,60,61,63] objectively assessed the effectiveness of the e-learning intervention. In forty (71.4%) studies [11,13,15-17,20,21,23-25,27,28,30-36,38,41-45,47-52,54-59,61,63,64] assessment completion or survey response rate was either compulsory (100%) or was sufficiently high (more than 60%) for the results to be considered valid. In fourteen (25%) studies [19,28,32,33,42-45,49,52,53,58,59,63] there was a control group and a random allocation process. Therefore, it seems reasonable to consider the overall conclusions reached by the studies as legitimate.

Discussion

The current literature supports the short-term effectiveness of e-learning for pharmacy education and also suggests that the effect of the e-learning intervention could be maintained several months post-intervention. E-learning appears to be at least as effective as face-to-face learning and superior to no training at all. However, further research is required to fully establish the long-term effectiveness of e-learning within pharmacy education.

Participant satisfaction with the quality of the learning experience is of fundamental importance when evaluating the effectiveness of any teaching method for pharmacy education [7,8]. The results of this review suggest that e-learning satisfies this minimum requirement and that, when compared to more traditional teaching methods, e-learning offers increased convenience, [10,12,17,21,28,29,38,52,56] the opportunity for self-directed [17,20,33,35,50,56,57,63] and self-paced learning [12,17,21,28,29,52,56,58,60] and the ability to use class time more effectively [20,33,35,50,55,57,63]. These unique advantages clearly establish a role for e-learning within pharmacy education, but it is the disadvantages of e-learning that help to define what this role may be. In the vast majority of studies, participants did not prefer e-learning over face-to-face methods, and concerns exist around the impact of reduced face-to-face interaction [10,17,21,38,39,55,56,58,62] and the diminished role of faculty member [17,21,55,62]. Therefore, it appears that the current role of e-learning within pharmacy education may be as part of a blended approach, so as to supplement, rather than replace face-to-face interactions.

E-learning was found to be a well-accepted and effective learning method in all but five [11,34,42,45,55] of the fifty-six studies reviewed. In these five studies, failure to be effective or well-accepted occurred regardless of topic or delivery method. In three [34,42,45] of the five studies, methodological flaws may instead explain either the superior performance of the control [42,45] or the failure of the intervention to result in a significant change in a measure of effectiveness [34]. These include flawed study design introducing differences between the control and intervention groups [45]; a significantly higher baseline knowledge in the control group [42], greater incentive for the control group to learn course content [42]; loss to follow up in the control group of greater than 50% [42]; and insufficient sample size [34]. In one of these studies [11], e-learning did result in a significant improvement in the skills and/or knowledge of the participants, but the e-learning intervention was outperformed by the control and thus, could not be considered to be effective. In the one other study [55], participants felt that the role of e-learning was too great and led to the course becoming excessively self-directed.

There was some evidence that e-learning interventions could result in an intention to change practice (outcome level 3 of Kirkpatrick’s model) and these studies deserve some further examination as possible predictors for success of e-learning. As would be expected, these studies involved mostly qualified pharmacists [11,12,14,16] with only one study involving pharmacy interns [59] and two studies involving pharmacy undergraduates [23,37]. These seven studies [11,12,14,16,23,37,59] involved several different delivery methods of the e-learning intervention and thus, no relationship could be established between delivery method and success. However, common to all of these studies was that the topic of e-learning was pharmacy practice. This most likely explains why these studies found evidence that e-learning interventions could result in an intention to change practice, as such topics are immediately applicable and more obviously relevant to the workplace.Only one study reported an actual change in practice rather than an intention to change practice [14]. In that study, pharmacists reported that completion of an online education program led to improved patient interactions, more effective inter-professional working and enhanced performance of pharmacy-specific tasks [14]. No studies attempted to objectively measure patient benefit or improvements in care.

In the five years since the only other previous review into the effectiveness of e-learning within pharmacy education [6], there has been continued adoption of e-learning within both undergraduate and postgraduate pharmacy education and a large increase in the volume of literature relating to this topic. However, the evidence base to further our understanding of the effectiveness of e-learning for pharmacy education is limited. There still exists insufficient evidence to establish the impact of e-learning on long-term learning and changes in the practice of pharmacists; and a complete lack of evidence required to determine if an e-learning intervention can lead to an increase in the knowledge or skills of pharmacists sufficiently, so as to result in patient benefit or improved quality of care. The same methodological weaknesses that were identified in the research prior to 2013 are still present in the more recent studies [6].

This review had several strengths. The search process was completed using four bibliographic databases resulting in a comprehensive literature review since 2013. Pre-defined inclusion and exclusion criteria were implemented in order to reduce the likelihood that bias from the researcher would affect the decision to include or exclude studies. Each e-learning intervention was evaluated according to Kirkpatrick’s four levels of training evaluation model. This model allows for effectiveness of each intervention to be established objectively, on the basis of study outcomes.

This study had a number of limitations. Unlike the previous review by Salter and colleagues [6], this was not a systematic review and studies were not assessed for quality. A narrative overview is not subject to the same rigorous methodology as a systematic review. Despite the use of specific search terms and pre-defined inclusion criteria, the decision on whether to include or exclude individual studies may still have been affected by bias on the part of the researchers. Therefore, a relationship between an individual study’s quality and its findings cannot be established, nor can the impact of this on the validity of the overall conclusions be determined. All studies that were reviewed had methodological flaws and the overall conclusions of this review will have to be considered in this light.

Conclusion

This review found e-learning to be both a well-accepted and effective teaching method (albeit in the short term), across a wide range of topics and technologies. For these reasons, it is likely that e-learning will continue to be integrated into all levels of pharmacy education. It appears that the role of e-learning within pharmacy education is in combination with face-to-face teaching, and that achieving the correct balance between these two pedagogical approaches is crucial to ensuring its successful use. The review exposed that within the current research, there is a need for high quality studies and that there are a number of directions for further research. In particular, there is a need for more studies to determine whether e-learning is effective in the long term and whether e-learning results in changes to practice and improvements in patient outcomes. Addressing these areas will provide a comprehensive understanding of the effectiveness of e-learning within pharmacy education.

References

  1. Garrison DR (2011) E-Learning in the 21st Century: A Frame work for Research and Practice. 2nd edition, Taylor& Francis,New York, USA.
  2. Sun P, Tsai RJ, Finger G, Chen Y,Yeh D (2008) What drives a successful e-Learning? An empirical investigation of the critical factors influencing learner satisfaction. Comput Educ 50: 1183-1202.
  3. Arkorful V, Abaidoo N (2015) The role of e-learning, advantages and disadvantages of its adoption in higher education.Int J InstrTechnol Distance Learn 12: 29-36.
  4. Noesgaard SS, Ørngreen R (2015) The Effectiveness of E-Learning: An Explorative and Integrative Review of the Definitions, Methodologies and Factors that Promote e-Learning Effectiveness. Electron J e-Learning 13: 278-290.
  5. Shank P(2016) How will we work. How will your job change?.
  6. Salter SM, Karia A, Sanfilippo FM, Clifford RM (2014) Effectiveness of E-learning in Pharmacy Education. Am J Pharm Educ 78: 1-12.
  7. Kirkpatrick D (1996) Great Ideas Revisited. Techniques for Evaluating Training Programs. Revisiting Kirkpatrick’s Four-Level Model. Train Dev 50: 54-60.
  8. Praslova L (2010) Adaptation of Kirkpatrick’s four level model of training criteria to assessment of learning outcomes and program evaluation in Higher Education. Educ Assessment Eval Account 22: 215-225.
  9. Salter SM, Vale S, Sanfilippo FM, Loh R, Clifford RM (2014) Long-term Effectiveness of Online Anaphylaxis Education for Pharmacists. Am J Pharm Educ 78: 01-11.
  10. Nesterowicz K, Librowski T, Edelbring S (2014) Validating e-learning in continuing pharmacy education: user acceptance and knowledge change. BMC Med Educ 14: 33.
  11. Buxton EC, De Muth JE (2013) Pharmacists’ perceptions of a live continuing education program comparing distance learning versus local learning. Res Soc Adm Pharm 9: 230-235.
  12. Cox CD, Cheon J, Crooks SM, Lee J, Curtis JD (2017) Use of Entertainment Elements in an Online Video Mini-Series to Train Pharmacy Preceptors. Am J Pharm Educ 81: 1-12.
  13. Nesterowicz K, Fereshtehnejad S (2015) E-Learning in Continuing Pharmacy Education is effective and just as accepted as on-site learning. Pharm Educ 15: 22-26.
  14. Farrell B, Jennings B, Ward N, ZeniMarks P, Kennie N, et al. (2013) Evaluation of a pilot e-learning primary health care skills training program for pharmacists. Curr Pharm Teach Learn 5: 580-592.
  15. Franklin AS, Markowsky S, De Leo J, Normann S, Black E (2016) Using Team-based Learning to Teach a Hybrid Pharmacokinetics Course Online and in Class. Am J Pharm Educ 80: 171.
  16. Moczygemba LR, Pierce AL, Dang A, Emberley P, Czar MJ, et al. (2017) The ADAPT online education program: A tool for practicing pharmacists delivering patient-centered care. J Am Pharm Assoc 57: 601-607.
  17. Garrison GD, Baia P, Canning JE, Strang AF (2015) An Asynchronous Learning Approach for the Instructional Component of a Dual-Campus Pharmacy Resident Teaching Program. Am J Pharm Educ 79: 29.
  18. Zlotos L, Power A, Hill D, Chapman P (2016) A Scenario-Based Virtual Patient Program to Support Substance Misuse Education. Am J Pharm Educ 80: 48.
  19. Taglieri CA, Crosby SJ, Zimmerman K, Schneider T, Patel DK (2017) Evaluation of the Use of a Virtual Patient on Student Competence and Confidence in Performing Simulated Clinic Visits. Am J Pharm Educ 81: 87.
  20. Stamper BD, Buhler AV, Harrelson JP,Roberts SC, Malhotra A, et al. (2017) Forecasting academic success through implementation of an online prerequisite review tutorials program for first year pharmacy students. Curr Pharm Teach Learn 9: 261-271.
  21. Kratochwill L, Almagambetova E, Corigliano A,Birnie C, Chablani L (2016) The Use of Podcast as a Teaching Tool for a Pharmacy Compounding Pre-Laboratory Lecture: A Survey Based Study the Use of Podcast as a Teaching Tool for a Pharmacy Compounding Pre-pre-laboratory lecture: A survey-based study. Pharm Educ 16: 38-41.
  22. Hughes PJ, Waldrop B, Chang J (2016) Student perceptions of and performance in a blended foundational drug information course. Curr Pharm Teach Learn 8: 59-363.
  23. Hincapie AL, Cutler TW, Fingado AR (2016) Incorporating Health Information Technology and Pharmacy Informatics in a Pharmacy Professional Didactic Curriculum -with a Team-based Learning Approach. Am J Pharm Educ 80: 1-11.
  24. Harris IM, Reidt SL, Lounsbery JL, Moon J, Pereira CR, et al. (2016) Assessment of core knowledge during ambulatory care advanced pharmacy practice experiences using online modules and pre- andpost-testing. Curr Pharm Teach Learn 8: 213-219.
  25. Harrap N, Usman S, Mcloughlin C,Orwell S, Harris SA, et al. (2016) Using a blended approach to support calculation skills of pharmacy students. Pharm Educ 16: 189-198.
  26. Camacho ME, Carrión MD, Chayah M, Campos JM (2016) The use of wiki to promote students’ learning in higher education (Degree in Pharmacy). Int JEducTechnol High Educ 13: 23.
  27. Zlotos L, Thompson ID, Boyter AC (2015) Integration of an Online Simulated Prescription Analysis into Undergraduate Pharmacy Teaching Using Supplemental and Replacement Models. Am J Pharm Educ 79: 37.
  28. Phillips JA (2015) Replacing traditional live lectures with online learning modules: Effects on learning and student perceptions. Curr Pharm Teach Learn 7: 738-744.
  29. Menendez E, Balisa-rocha B, Jabbur-lopes M,Costa W, Nascimento JR, et al. (2015) Using a virtual patient system for the teaching of pharmaceutical care. Int J Med Inform 84: 640-646.
  30. Fuji KT, Galt KA (2015) An Online Health Informatics Elective Course for Doctor of Pharmacy Students. Am J Pharm Educ 79: 41.
  31. Smith MA, Waite LH (2017) Utilization of a virtual patient for advanced assessment of student performance in pain management. Curr Pharm Teach Learn 9: 893-897.
  32. Baumann-birkbeck L, Karaksha A, Anoopkumar-dukie S, Grant G, Davey A, et al. (2015) Bene fits of e-learning in chemotherapy pharmacology education. Curr Pharm Teach Learn 7: 106-111.
  33. Wong TH, Ip EJ, Lopes I, Rajagopalan V (2014) Pharmacy Students’ Performance and Perceptions in a Flipped Teaching Pilot on Cardiac Arrhythmias. Am J Pharm Educ 78: 185.
  34. Weaver SB, Oji V, Ettienne E, Stolpe S, Maneno M (2014) Hybrid e-learning approach to health policy. Curr Pharm Teach Learn 6: 313-322.
  35. Mclaughlin JE, Roth MT, Glatt DM,Gharkholonarehe N, Davidson CA, et al. (2014) The Flipped Classroom: A Course Redesign to Foster Learning and Engagement in a Health Professions School. Acad Med 89: 236-243.
  36. Leong C, Louizos C, Frankel G,Ng S, Iacovides H, et al. (2014) A Physical Assessment Skills Module on Vital Signs. Am J Pharm Educ 78: 01-07.
  37. Addo-atuah J, Dutta A, Kovera C (2014) A Global Health Elective Course in a PharmD Curriculum. Am J Pharm Educ 78: 187.
  38. Wang T, Wang F, Shi L (2013) The use of microblog-based case studies in a pharmacotherapy introduction class in China. BMC Med Educ 13: 120.
  39. Karaksha A, Grant G, Anoopkumar-dukie S, Nirthanan SN, Davey AK (2013) Student Engagement in Pharmacology Courses Using Online Learning Tools. Am J Pharm Educ 77: 1-10.
  40. Gonzalvo JD, Chomicki JR, Chu MA, Frail CK (2013) Incorporation of online clinical topic modules in an introduction to advanced pharmacy practice experience course. Curr Pharm Teach Learn 5: 438-443.
  41. Douglass MA, Casale JP, Skirvin JA, Divall MV (2013) A Virtual Patient Software Program to Improve Pharmacy Student Learning in a Comprehensive Disease Management Course. Am J Pharm Educ 77: 172
  42. McCabe C, Smith MG,Ferreri SP (2017) Comparison of Flipped Model to Traditional Classroom Learning in a Professional Pharmacy Course. Educ Sci 7: 73.
  43. Barnett SG, Gallimore CE, Pitterle M,Morrill J (2016) Impact of a Paper vs Virtual Simulated Patient Case on Student-Perceived Confidence and Engagement. Am J Pharm Educ 80: 16.
  44. El-magboub A, Haworth IS, Sutch BT, Romero RM (2016) Evaluation of in-class and online discussion meetings in a biopharmaceutics problem-based learning class. Curr Pharm Teach Learn 8: 811-820.
  45. Al-dahir S, Bryant K, Kennedy KB, Robinson DS (2014) Online Virtual-Patient Cases Versus Traditional Problem-Based Learning in Advanced Pharmacy Practice Experiences. Am J Pharm Educ 78: 76.
  46. Pate KA, Pate AN, Sampognaro LA, Brady JH, Caldwell DJ (2017) Design, implementation, and evaluation of an online elective course on current topics in pharmacy. Curr Pharm Teach Learn 9: 528-536.
  47. Kolluru S, VarugheseJT (2017) Structured academic discussions through an online education-specific platform to improve Pharm. D. students learning outcomes. Curr Pharm Teach Learn 9: 230-236.
  48. Smith MA, Siemianowski LA, Benedict N (2016) Virtual Patient Case Sharing Across Two Schools of Pharmacy. Am J Pharm Educ 80: 153.
  49. Gossenheimer AN, Carneiro ML, Bem T,de Castro MS (2017) Impact of distance education on academic performance in a pharmaceutical care course. PLoS One 12: 01-11.
  50. Mclaughlin JE, Griffin LM, Esserman DA, Davidson CA, Glatt DM, et al. (2013) Pharmacy Student Engagement, Performance, and Perception in a Flipped Satellite Class room. Am J Pharm Educ 77: 196.
  51. Bykhovsky M, Meier MF, Mager NA (2014) Impact of an Online Educational Program on Pharmacy Students’ Knowledge of Folic Acid and Neural Tube Defects. J Pharm Pract 27: 513-517.
  52. Porter AL, Pitterle ME, Hayney MS (2014) Comparison of Online Versus Class room Delivery of an Immunization Elective Course. Am J Pharm Educ 78: 96.
  53. Jesus Â, Gomes MJ, Cruz A (2017) Blended versus face-to-face: comparing student performance in a therapeutics class. IET Softw 11: 135-140.
  54. Hernick M (2015) Test-Enhanced Learning in an Immunology and Infectious Disease Medicinal Chemistry / Pharmacology Course. Am J Pharm Educ 79: 97.
  55. Khanova J, Mclaughlin JE, Rhoney DH, Roth MT, Harris S (2015) Student Perceptions of a Flipped Pharmacotherapy Course. Am J Pharm Educ 79: 140.
  56. King AE, Egras AM (2015) Required Online Course with a Public Health Focus for Third Professional Year Pharmacy Students. Am J Pharm Educ 79: 68.
  57. Mclaughlin JE, Gharkholonarehe N, Khanova J, Deyo ZM, Rodgers JE, et al. (2015) The Impact of Blended Learning on Student Performance in a Cardiovascular Pharmacotherapy Course. Am J Pharm Educ 79: 24.
  58. Wanat MA, Tucker AM, Coyle EA (2016) A Critical Care Hybrid Online Elective Course for Third-Year Pharmacy Students. Am J Pharm Educ 80: 154.
  59. Yeh Y, Chen H, Cheng K, Hou S,HsuanYen Y (2014) Evaluating an online pharmaceutical education system for pharmacy interns in critical care. Comput Methods Programs Biomed 113: 682-689.
  60. Flood M, Hayden JC, Bourke B, Gallagher PJ, Maher S (2017) Design and Evaluation of Video Podcasts for Providing Online Feedback on Formative Pharmaceutical Calculations Assessments. Am J Pharm Educ 81: 6400.
  61. Isaacs AN, Nisly S, Walton A (2017) Student- generated e- learning for clinical education. Clin Teach 14: 129-133.
  62. Demella C, Donohoe K, Morgan L, Phipps L, Drisaldi A,et al. (2017) Implementation of an online ethics course into a pharmacy curriculum. Curr Pharm Teach Learn 9: 835-840.
  63. Prescott WA, Woodruff A, Prescott GM, Albanese N, Bernhardi C, et al. (2016) Introduction and Assessment of a Blended-Learning Model to Teach Patient Assessment in a Doctor of Pharmacy Program. Am J Pharm Educ 80: 176.
  64. Park HL, Shrewsbury RP (2016) Student Evaluation of Online Pharmaceutical Compounding Videos. Am J Pharm Educ 80: 30.
  65. Short ME, Goetzel RZ, Pei X,Tabrizi MJ, Ozminkowski RJ, et al. (2009) How Accurate are Self-Reports? An Analysis of Self-Reported Healthcare Utilization and Absence When Compared to Administrative Data. JOccup Environ Med 51: 786-796.
  66. Rosenman R, Tennekoon V, Hill LG (2011) Measuring bias in self-reported data. IntJ BehavHealthc Res 2: 320-332.
  67. Kelley K, Clark B, Brown V, Sitzia J (2003) Good practice in the conduct and reporting of survey research. Int J Qual Health Care 15: 261-266.
  68. Sibbald B, Roland M (1998) Why are randomised controlled trials important? BMJ 316: 201.
 

Authors (Year of Publication), Country

Participants

Topic

E-learning intervention

Assessment

Control Group(s)

Outcomes measured

Buxton et al.[11], USA

29 pharmacists

Drug diversion

Online synchronous lecture

Post-intervention survey

14 of the participants attended same lecture face-to-face

1, 2, 3

Douglass et al. [41], USA

135 pharmacy students

Comprehensive Disease Management

Online virtual patient cases

Assignment,

Pre- and post-intervention knowledge tests

 

Post-intervention survey

None

1, 2

Farrell et al. [14], Canada

75pharmacists

Adapting pharmacists’ skills and approaches to maximize

patients’ drug therapy effectiveness (ADAPT)

Online e-learning programme

Mid- and post-intervention survey

None

1, 2, 3

Gonzalvo et al. [40], USA

159 pharmacy students

Introduction

to advanced pharmacy practice experience

Online modules

Post-intervention survey

Post-rotation year survey

None

1

Karaksha et al. [39], Australia

 

79 pharmacy students

Pharmacology

Online interactive teaching resources

Pre- and post- intervention surveys.

None

1

McLaughlin et al. [50], USA

22 pharmacy students

Basic pharmaceutics

Online synchronous lectures and online modules

Pre- and post- intervention surveys.

None

1,2

Wang et al. [38], China

126 pharmacy students

Pharmacotherapy

Microblog-based case studies

Post-intervention survey

None

1

Addo-Atuah et al. [37], USA

240 pharmacy students

Global Health

Online course

Individual and team readiness tests.

Mid- and post-intervention knowledge tests

Group. assignments

Pre- and post-intervention surveys.

None

1, 2, 3

Al-Dahir et al. [45], USA

119 pharmacy students

Atrial fibrillation

Online virtual patient cases, completed individually.

Pre- and post-intervention knowledge tests.

60 of the participants completed paper-based cases as teams, in a face-to-face session.

2

Bykhovsky et al.

[51], USA

 

133 pharmacy students

Folic acid and neural tube defects

Online video

Pre-intervention, immediately post-intervention and nine months post-intervention knowledge tests

None

2

Leong et al. [36], Canada

48 pharmacy students

Physical Assessment Skills

Blended course; Three online, voiced-over PowerPoint lectures; And face-to-face skills workshop

Post-intervention knowledge test.

 

Post-intervention skills test.

None

2

McLaughlin et al. [35], USA

 

315 pharmacy students

Pharmaceutics

Online course, delivered through pre-recorded lectures

Post-intervention knowledge test.

 

Pre- and post-intervention surveys.

153 of the participants completed the course face-to-face, the year previously

1, 2

Nesterowicz et al.  [10], Poland

553 pharmacists

Antibiotic and chemotherapy of infectious diseases.

 

Effectiveness and safety of new classes of antidiabetic drugs

Two online courses, delivered via lecture slides.

First e-course: Pre- and post-intervention knowledge tests

 

Second e-course: Post-intervention survey

None

1, 2

Porter et al. [52], USA

 

140 pharmacy students

Immunisation

Blended course; pre-recorded online lectures and face-to-face sessions

Overall course grade, final examination and in-class quizzes.

 

Pre- and Post-intervention survey.

69 of the participants completed the course face-to-face

1, 2

Salter et al. [9], Australia

383 pharmacists, pharmacy students, and medical students(control)

Anaphylaxis

Online course, delivered via lecture slides.

Pre-intervention, immediately post-intervention, three months post-intervention and seven months post-intervention knowledge tests

220 of the participants received face-to-face lectures and 106 of the participants(medical students) received no training

2

Weaver et al. [34], USA

8 pharmacy students

Health policy

Blended course; face-to-face classes; online modules lectures; and online discussion forum

Pre- and post-intervention critical thinking skills tests

None

2

Wong et al. [33], USA

206 pharmacy students

Cardiac arrhythmias

Blended course; pre-recorded lectures; And three face-to-face classes

Post-intervention knowledge test

 

Post-intervention survey

105 of the participants received only face-to-face lectures and classes.

1, 2

Yeh et al. [59], Taiwan

23 pharmacy interns

Clinical pharmacy internship

Online training programme and face-to-face group discussions and presentations

Pre- and Post-interventionknowledge, attitude and practice surveys (KAP questionnaire)

12 of the participants received face-to-face teaching only

2, 3

Baumann-Birkbeck

et al. [32], Australia

77 pharmacy students

Chemotherapeutic pharmacology

Suite of 65 e-learning tools and face-to-face teaching

Post-intervention knowledge test

52 of the participants received face-to-face teaching, without access to e-learning tools

2

Fuiji and Galt [30], USA

39 pharmacy students

Health informatics

Online course, delivered via lecture slides with pre-recorded voice over.

Overall course grade, achieved from completion of assignments and quizzes throughout course.

Post-intervention, qualitative assessment of efficacy.

Four of the participants completed the course when delivered face-to-face, a year previously.

1, 2

Garrison et al. [17], USA

 

13 pharmacy residents

Pedagogy instruction

Online course

Post-intervention surveys

None

1, 2

Hernick[54], USA

146 pharmacy students

Immunology and infectious disease medicinal

chemistry and pharmacology

Blended course; optional online modules and face-to-face teaching

Final examination and post-intervention knowledge tests

73 of the participants completed the course face-to-face without access to the online modules, a year previously.

2

Khanova et al. [55], USA

134 pharmacy students

Pharmacotherapy

Blended course; online modules and face-to-face teaching

Pre- and post-intervention surveys

None

1

King and Egras[56], USA

22 pharmacy students

Public Health

Online Modules

Overall course grade

 

Post-intervention survey

None

1, 2

McLaughlin et al. [57], USA

95 pharmacy students

Cardiovascular pharmacotherapy

Blended course; online modules and face-to-face teaching

Final examination

 

Pre and post-intervention knowledge tests

 

Post-intervention survey

None

1, 2

Menendez et al. [29], Brazil

31 pharmacy students

Communication skills

Online virtual patient cases

Post-intervention survey

None

1, 2

Nesterowicz et al. [13], Poland

792 pharmacists

Monitoring of hypertension patients.

Online continuing education course

Pre- and post-intervention knowledge tests

 

Post-intervention survey

251 of the participants attended the same course face-to-face

1, 2

Phillips [28], USA

164 pharmacy students

Advanced ambulatory care

Online interactive learning module

Post-intervention knowledge test

 

Post-intervention survey

82 of the participants attended a lecture face-to-face

1, 2

Zlotos et al. [27], Scotland

 

272 pharmacy students, divided into two cohorts

Pharmacy practice

Online simulated prescription analysis tool (SCRIPT)

Post-intervention knowledge tests

 

Post-intervention interview of 18 students (stopped once no new themes emerged)

None

1, 2

Barnett et al. [43], USA

134 pharmacy students

Osteoarthritis

Online virtual patient cases

Post-intervention knowledge test (SOAP note)

 

Pre- and post-intervention surveys

53 of the participants  completed paper-based cases, in a face-to-face session

1, 2

Camacho et al. [26], Spain

200 pharmacy students

Organic and Pharmaceutical Chemistry

Online wiki, developed by small groups of students

Post-intervention acceptance survey

None

1

El-Magboub et al. [44], USA

185 pharmacy students

Biopharmaceutics

Online group discussion meetings through videoconference, led by a facilitator

Post-intervention survey

All participants also completed the same discussion meetings face-to-face

1

Franklin et al. [15], USA

222 pharmacists

Phenytoin pharmacokinetics

Online team-based learning session

Post-intervention individual and team readiness tests

Post-intervention knowledge & teamwork self-assessment survey.

Two cohorts attended the same session delivered face-to-face

2

Harrap et al. [25], England

189pharmacy students

Pharmaceutical   calculations

Blended course; Online modules, featuring worked examples and practice questions; and face-to-face teaching.

Pre- and post-intervention knowledge tests

 

Post-intervention survey

None

1, 2

Harris et al. [24], USA

70 pharmacy students

Ambulatory care

Online course

Pre- and post-intervention knowledge tests

None

2

Hincapie et al. [23], USA

100 pharmacy students

Pharmacy informatics

Online course

Pre and post-intervention knowledge self-assessment and perceptions survey

None

1, 2, 3

Hughes et al. [22], USA

248 pharmacy students

Foundational drug information

Blended course; Online module delivered in the form of narrated videos and a weekly face-to-face laboratory session.

Mid-course grades and post-intervention knowledge test

 

Pre- and post-intervention survey

121 of the participants completed the same course a year previously, delivered face-to-face.

1, 2

Kratochwill et al. [21], USA

232 pharmacy students

Pharmaceutical Compounding

Podcast used to replace a pre-laboratory

lecture

Post-intervention survey

None

1

Parket al. [64], USA

391 pharmacy students

Pharmaceutical Compounding

Online teaching videos

Post-intervention survey

None

1, 2

Prescott et al. [63], USA

Course 1: 256 pharmacy Students

 

Course 2: 253 pharmacy students

Patient assessment

Two blended courses, consisting of an online module and face-to-face classes.

Overall course grade

 

Post-intervention survey

Course 1: 108 of the participants received face-to-face teaching only.

 

Course 2: 97 of the participants received face-to-face teaching only.

1, 2

Smith et al. [48], USA

131 pharmacy students

Critical care therapeutics

Online virtual patient cases

Pre- and post-intervention knowledge tests

 

Post-intervention survey

None

1, 2

Wanat et al. [58], USA

105 pharmacy students

Critical care

Blended course; pre-recorded online lectures and face-to-face classes.

Final examination

Post-intervention survey

54 of the participants received face-to-face teaching only

1, 2

Zlotos et al. [18], Scotland

106 pre-registration pharmacists

Substance misuse

education

Online virtual patient cases

Pre- , immediately post- and 6-months post-intervention knowledge test and perceived confidence survey

None

2

Cox et al. [12], USA

202 pharmacists

Pharmacy preceptor training

Online training video mini-series

3 months post-intervention knowledge survey

None

1, 2, 3

DeMella et al. [62], USA

134 pharmacy students

Ethics

Four module, online course

Pre- and post-course knowledge tests

 

Post-course survey

None

1, 2

Flood et al. [60], Ireland

162pharmacy students

Feedback on pharmaceutical

calculations examination

Video Podcasts

Post-intervention survey

None

1

Gossenheimer et al. [49], Brazil

82 pharmacy students

Pharmaceutical Care

Two module, online course

Overall course grade

Students were divided into two groups. Groups were taught one module face-to-face and one module online.

2

Isaacs et al. [61], USA

31 pharmacy students

Ambulatory care

Three online modules, developed by students

Pre- and post-intervention knowledge tests

 

post-intervention survey

None

1, 2

Jesus et al. [53], Portugal

110 pharmacy students

Therapeutics

Blended course;

Online virtual patient cases and face-to-face classes

Post-intervention knowledge test

54 of the students received face-to-face teaching only

2

Kolluru and Varughes[47], USA

88 pharmacy students

Pharmacotherapy

Online academic discussion

Final examination Post-intervention survey

None

1, 2

McCabe et al. [42], USA

237 pharmacy students

Self-care and non-prescription medications

Online course

Pre- and post-intervention knowledge test and self-assessment survey

78 of the participants were delivered the course through  face-to-face teaching

2

Moczygemba et al. [16], USA

23 pharmacists

ADAPT

Online e-learning program

Mid- and post-intervention survey

None

1, 2, 3

Pate et al. [46], USA

122 pharmacy students

Current Topics in Professional Pharmacy

Online course with small group discussion via message board

Pre- and post-intervention surveys

 

Post-intervention knowledge test

None

1, 2

Smith and Waite [31], USA

344 pharmacy students

Pain management

Online virtual patient cases

Post-intervention knowledge test

188 of the participants,  completed paper-based cases, a year previously

2

Stamper et al. [20], USA

306 pharmacy students

Basic biology, chemistry, and physiology for pharmacy

Online tutorials

Mid- and post-intervention knowledge tests

 

Post-intervention survey

None

1, 2

Taglieri et al. [19], USA

281 pharmacy students

Skills required for performing Clinic Visits

Online virtual patient cases to prepare students for mock clinic visits

Student performance in mock clinic visits

Pre- and post-intervention surveys

140 of the participants did not complete the virtual patient cases before the mock clinic visit

1, 2

Table 2: Summary of Studies included in the Narrative Overview of the Effectiveness of E-learning in Pharmacy Education.

Key:

  1. Reaction = Learners’ views about the e-learning program, including experiences and satisfaction with the topic and e-learning technology,
  2. Learning = Change in attitudes, knowledge or skills after training,
  3. Behaviour = Transfer of knowledge to the workplace (includes willingness to apply learning in the workplace),
  4. Results = Changes in organisational practice (e.g. delivery of care) and patient outcomes as a result of the program.
Ocimum Scientific Publishers

This work is licensed under a Creative Commons Attribution 4.0 International License.  Creative Commons License

Copyright © 2019 - All Rights Reserved - ocimumpublishers.com