Design of a digital logbook – user-friendliness, practical relevance and usefulness as a training tool, using the example of the block clinical placement in general practice at the Mannheim Faculty of Medicine
Maryna Gornostayeva 1Katrin Schüttpelz-Brauns 2
Nia A. Ruseva 1
Kristina Hoffmann 1
1 Heidelberg University, Mannheim Medical Faculty, Centre for Preventive Medicine and Digital Health (CPD), Division of General Medicine, Mannheim, Germany
2 Heidelberg University, Mannheim Medical Faculty, Division of Student Affairs and Teaching Development, Department of Medical Education Research, Mannheim, Germany
Abstract
Objective: In medical curricula, logbooks serve to structure community-based clinical placements by defining learning objectives and content as well as documenting students’ clinical activities. Yet, studies on the use of logbooks indicate a low level of motivation to use them among students. The aim of the project was to redesign an existing logbook so that it is user-friendly for both students and supervising physicians and relevant for routine general practice.
Project description: For this purpose, the paper-based logbook for the clinical placement in general practice was redesigned into an electronic format and revised in collaboration with stakeholders. In the process, generic tasks and categories of typical patient cases from general practice were designed in a way to facilitate flexible case selection based on the practice focus and/or students’ own interests. Compulsory e-logbook training sessions for training GP practices were organised. The achievement of objectives was assessed using online surveys.
Results: Overall, the digital logbook proved to be user-friendly and well-suited to provide a realistic representation of day-to-day general practice. However, the e-logbook evaluation as a training tool yielded mixed results. The main issues identified were limited internet or computer access in some academic teaching practices, as well as, in some cases, insufficient support from the supervising physicians.
Conclusion: The e-logbook is a suitable tool for supporting clinical placements. However, it requires that the necessary technical resources are available to students at their clinical placements. Supervising physicians should receive ongoing training regarding the content and their role as supervisor, that should be integrated into the ongoing quality assurance process.
Keywords
medical education, practical education, general practice, clinical placement, logbook, e-logbook
1. Introduction
Logbooks are widely used in medical education worldwide. They serve to structure and standardise community-based clinical placements, amongst other things by defining learning objectives and content, as well as documenting students’ clinical activities and, not infrequently, as assessment records [1], [2], [3], [4], [5], [6], [7], [8], [9]. Logbooks facilitate communication between students and supervising physicians and provide a transparent overview of the teaching and learning process [10], [11], [12]. Logbooks also serve as a tool for curriculum evaluation [3], [13], [14], [15].
At Mannheim Medical Faculty, logbooks are used, for example, in the two-week clinical placement in general practice. This placement, as defined in the Medical Licensing Regulations [https://www.gesetze-im-internet.de/_appro_2002/BJNR240500002.html], takes the form of one-on-one supervision in approximately 120 general medical practices that are certified as academic teaching practices. As these practices differ in terms of their organisation and areas of specialisation, the paper-based logbook is intended to provide a uniform structure and ensure the standardised educational content during the placement. Logbooks define the learning objectives and tasks to be completed by students. To prevent students from merely ticking off tasks [1], [11], [16], [17], portfolio components are integrated into the logbook. In portfolios, students collect evidence of their tasks completed, analyse and reflect on their experiences, and have these assessed by their supervising physicians [18], [19], [20], [21]. The integration of portfolio components into logbooks can promote active engagement with learning objectives and content, particularly through feedback from supervising physicians [11], [22]. For this reason, logbook tasks in the two-week clinical placement should be documented, reflected upon in a written form following guiding questions, and discussed with the supervising physicians. Assessment is carried out by the supervising physicians using a structured appraisal form assessing both the student’s practical performance and their logbook documentation. The submitted logbook is checked by the teaching co-ordination team and is a prerequisite for course completion.
Regular evaluations of the two-week clinical placement by the dean’s office revealed that the paper-based logbook met with limited acceptance among the students, partly due to the time it took to complete. This finding is consistent with experiences at other sites [3], [4], [5], [10], [11], [12], [23], [24]. The students found the logbook too long, the tasks too time-consuming and overly prescriptive, and wished to be able to select suitable patient cases for their own work. They considered the paper format inappropriate in the digital age and proposed an electronic format, which generally appears to be preferred by students due to simpler and faster data entry and location-independent access [15], [25], [26]. Furthermore, there is evidence in the literature that training institutions could also benefit from a digital format through cost savings, reduced space requirements, improved readability and analysis of the collected data, and optimisation of the feedback process [15], [25], [27], [26], [28], [29].
These findings prompted a review of the existing paper-based logbook and a redesign into an electronic format in such a way that
- it is user-friendly,
- they are relevant to routine general practice and feasible in everyday situations, and
- the logbook is used as a training tool.
The revision took place in the winter semester of 2022/2023. Although the process and the results described in this project report are based on the example of the clinical placement in general practice, the approach is intended to be transferable across medical disciplines and departments.
2. Project description
The process was based on the twelve tips for the successful implementation of logbooks in clinical training (see table 1 [Tab. 1]) [9].
Table 1: Twelve tips for the successful implementation of logbooks in clinical training [9]
2.1. Project team and participating groups
In addition to the core team from the Division of General Medicine, the Digital Education Team at the Mannheim Medical Faculty was involved in the development and implementation of the logbook. The core team consisted of two research assistants, the teaching co-ordinators, the supervising physicians involved in curriculum development, a medical student assistant, and the head of the teaching department. The study dean’s office and the programme committee endorsed the change. Representatives from all stakeholder groups were thus involved (tip 2).
2.2. Revision of content
In light of recent developments in medical education [30], [31] (tip 1), the learning objectives (see attachment 1 [Att. 1]) and tasks were revised. The number of tasks was reduced, and the specific requirements regarding clinical presentations, reasons for consultation, and examination methods, previously set out in the paper-based logbook (e.g. history taking and physical examination of patients with lower back pain), were removed (see table 2 [Tab. 2]).
The changes were discussed by the supervising physicians from the core team, adapted according to their comments and subsequently revised in collaboration with the medical student, who had already completed this two-week clinical placement.
The portfolio components from the paper-based logbook were retained to enhance students’ reflective processes and learning outcomes [11], [22], [32]. Integrated tools, such as a self-assessment form for practical competencies and guiding questions, were designed to support the dialogue between students and supervising physicians [10], [11], [12]. Supervising physicians are encouraged to set aside sufficient time for discussions and feedback (tips 7 and 8).
Information about the two-week clinical placement, a glossary of relevant terminology, and supplementary educational resources were incorporated (tip 3). In line with a learning spiral approach, the assignments were linked to relevant course materials on the faculty’s internal learning management system, Moodle (tip 6).
To successfully complete the two-week clinical placement, students must work on at least four of the eight patient cases and all generic tasks (see table 2 [Tab. 2]). The range of tasks is intended to facilitate flexible task selection based on the practice focus and/or students’ own interests and thus foster their intrinsic motivation [33].
2.3. Technical implementation
Criticism of the paper format prompted the decision to convert the logbook into an electronic format, thereby enabling students, amongst other things, to enter data more easily and quickly [15], [25], [27], [26]. The e-logbook was integrated into the Moodle platform, which is used regularly by students and is also available to the academic teaching practices. Using an existing platform not only saves costs but also has clear advantages in terms of data protection, as processes for handling personal data are already in place (tips 4 and 5).
The information required for the two-week clinical placement was structured using a modular layout (see figure 1 [Fig. 1]).
Figure 1: Sections of the e-logbook in the learning management system
Each assignment must be completed electronically by students using a form and can be edited as often as required. The final version is submitted following feedback from the supervising physician (tip 5). The submitted assignments are immediately visible to the teaching co-ordinator (tip 9) and are checked to ensure that the course completion requirements are met. For assessment, which takes place in the final supervisory meeting with the supervising physician, a structured appraisal form is available for download in the e-logbook.
To minimise the effort involved in updating the e-logbook, links to the central Moodle page for general practice have been set up for all scheduled dates and deadlines; this page is routinely maintained by the teaching co-ordinators (tip 9).
2.4. Implementation process
Each semester, approximately 140 students complete the two-week clinical placement, divided into four groups at four different times. The launch of the e-logbook took place in the winter semester 2022/2023. At the start of the module, each group received an introduction to the e-logbook, and was requested to submit feedback on the e-logbook via the feedback function. To ensure successful long-term implementation, the e-logbook introduction was integrated as a regular element within the module’s information session for students (tip 11).
Support from supervising physicians is crucial for the successful use of logbooks [1], [9], [11], [12], [24], [32]. Accordingly, the e-logbook was presented to the supervising physicians during a quality improvement meeting organised for all academic teaching practices at the beginning of the semester. In addition, a personalised letter was sent to them by post containing information on the redesign, a brief printed version of the e-logbook, as well as Moodle login details for the full electronic version. Those who had not previously supervised a two-week clinical placement also received an individual introduction via video conference (tips 11 and 12). However, experience has shown that a one-off introduction is not sufficient for the successful use of the e-logbook, and that ongoing supervision of the academic teaching practices is required [9], [24]. As frequent refreshers for approximately 120 supervising physicians are not feasible, mandatory e-logbook training is provided as part of the initial certification and every three years as part of the recertification process for academic teaching practices (tips 11 and 12).
After completion of the first group’s two-week clinical placement, we reviewed students’ feedback in the e-logbook and had a group discussion with supervising physicians in the core team. Based on this, technical issues were resolved and some tasks were clarified (tip 10). For ongoing quality assurance, additional items to evaluate the e-logbook were incorporated into the regular student evaluations that are carried out every two years by the study dean’s office (tip 10). Involving the study dean’s office in the process facilitated the incorporation of these items into the standard evaluation (tip 1).
3. Evaluation
In the summer semester of 2023, the e-logbook was evaluated as part of a focused evaluation [https://www.umm.uni-heidelberg.de/studium/qualitaetsentwicklung/evaluationen/]. Students and supervising physicians were surveyed using online questionnaires developed in-house within the EvaSys teaching evaluation system. The evaluation was planned and carried out in accordance with Heidelberg University’s evaluation regulations, in collaboration with the faculty’s internal quality management Tteam for studies and teaching and the department of medical education research.
In the online surveys, usability was assessed via closed-ended questions (seven items in the student version and eight items in the version for supervising physicians). The items were drawn up based on the “system usability scale” [34]. The practical relevance and feasibility of the tasks in day-to-day clinical practice were assessed using “yes/no” questions for each e-logbook task. The usefulness as a training tool was assessed using closed-ended questions (seven questions in the student version and five questions in the version for supervising physicians). We chose these items based on the intended uses of logbooks in training [15], [22]. Both questionnaires also included fields for free-text comments on all three areas. For the supervising physicians, data was also collected on their supervision experiences in the clinical placement. Invitations to participate were sent by email to 118 students who undertook the two-week clinical placement in the summer semester of 2023, as well as to 72 supervising physicians.
The quantitative data from the online survey were entered into SPSS and analysed descriptively. The open-text responses were analysed inductively using qualitative content analysis according to Mayring [35]. A research assistant reviewed all the data and developed the coding categories and descriptions. In the next step, the open-ended responses were independently assigned to these categories by the same research assistant and a second research assistant. The results were then compared, and category definitions discussed, and adjusted accordingly.
4. Results
Keeping the e-logbook has been compulsory for all students since its implementation in the winter semester 2022/23. It serves as a prerequisite for completing the general practice clinical placement module. Based on the evaluation feedback, technical issues were resolved, tasks adapted and individual task descriptions clarified.
4.1. Participation in the evaluation
56 (47%) students and 27 (38%) supervising physicians took part in the online surveys. 70% of these supervising physicians have been involved in student education for more than five years and 63% have completed further training in teaching. 59% stated that they had supervised more than seven students in the clinical placement to date.
4.2. Usability of the e-logbook
The analysis of usability revealed a Cronbach’s alpha of 0.843 in the questionnaire for students and 0.933 for supervising physicians. In the assessment, the mean score for both students and physicians was 2.4 on a Likert scale of 1-5 (1=strongly agree, 5=strongly disagree). The students were particularly critical of the time required to complete the e-logbook (mean=3.3; SD=1.32). On average, the supervising physicians considered the time invested in discussing the e-logbook with the student to be appropriate (mean=2.5; SD=0.99). The individual items are listed in attachment 2 [Att. 2], table S1.
Examples of the students’ free-text responses and comments are presented in table 3 [Tab. 3].
Table 3: Students’ comments on the user-friendliness of the e-logbook
There were a few brief free-text comments from the supervising physicians, such as “well thought out” and “quite good”. One comment noted that proofreading had been easier before the introduction of electronic documentation.
4.3. Practical relevance and feasibility of tasks in clinical practice
Feedback from supervising physicians and students on the practical applicability of the generic tasks and the patient cases is presented in table 4 [Tab. 4] and table 5 [Tab. 5].
Table 4: Online survey of supervising physicians and students on the practical applicability of the e-logbook tasks (generic tasks)
Table 5: Online survey of supervising physicians and students on the feasibility of e-logbook tasks (patient cases)
Almost all supervising physicians rated the tasks as practical and easy to carry out. In the open-ended comments, some supervising physicians noted that patient care in collaboration with other healthcare professions rarely occurred during the two-week clinical placement. In the students’ comments, it was mentioned several times that this hardly ever took place or that the task was unclear. Some students found it difficult to carry out lifestyle counselling without an established doctor-patient relationship. One of the supervising physicians remarked that the students were “more likely to observe” these situations. The “no” responses regarding home visits and video consultations were justified on the grounds that practices do not carry out these services and, with regard to palliative care cases, that they either do not provide palliative care or that no palliative care cases occurred during the two-week clinical placement period. With regard to the tasks writing a medical report, preparing a medication plan and issuing a prescription, the students reported that they encountered technical difficulties with formatting and saving.
4.4. Usefulness of the e-logbook as a training tool
The following Cronbach’s alpha values were obtained for the usefulness of the e-logbook as a training tool: 0.946 in the questionnaire for students and 0.842 for the supervising physicians. The ratings, on a Likert scale of 1-5 (1=strongly agree, 5=strongly disagree), were in the moderate range for students (mean=3.1; SD=1.15) and in the middle range for supervising physicians (mean=2.6; SD=0.66). Approximately 50% of students disagreed with the statements regarding the targeted development and improvement of practical skills using the e-logbook and regarding support in communicating with the supervising physician. In contrast, almost half of the supervising physicians rated these positively. Half of the students and more than half of the supervising physicians found the e-logbook helpful for the reflection process. The individual items are presented in attachment 2 [Att. 2], table S2.
In their free-text comments, students repeatedly noted that the e-logbook was a valuable tool for preparing for and supporting the two-week clinical placement, and that it had served as a guide through the training period. The structured documentation of tasks had encouraged an in-depth and thorough engagement with the patient cases. However, there were also a few comments from students regarding time-consuming, laborious and unnecessary documentation.
5. Discussion
The transition from a paper-based to a digital logbook, together with the revision of its content, aimed to create a user-friendly training tool with practical tasks that can be easily implemented in the day-to-day general practice.
The evaluation showed that this goal has been achieved. Both students and supervising physicians found the e-logbook easy to use. Most students expressed satisfaction with its implementation in Moodle. They were able to save their drafts and edit or correct them later. Thanks to integrated links, additional educational resources and various examination methods were immediately available, supporting the completion of tasks. Similar findings have been reported elsewhere, indicating that the electronic format facilitates easy data entry and storage, as well as a straightforward access to information, which also increases students’ motivation to use the system [15], [26], [35]. From an organisational perspective, using Moodle saved the cost and effort of purchasing a new software licence and having to re-address the resulting issues regarding data protection, copyright and technical support (tips 4 and 5) [9].
However, the Moodle-based solution also presented challenges. Without a computer or internet access in general practices, students had to document everything by hand and later transfer it to the e-logbook. In some cases, they lacked important data, which made completing the tasks more difficult. This may explain the students’ critical assessment of the time required to complete the e-logbook. Driessen et al. and Paydar et al. also report that limited computer access in the workplace undermines the benefits of a digital format [26], [36]. Furthermore, if the e-logbook was completed outside the practice, there was less opportunity for students and supervising physicians to work together on patient cases. If this exchange is lacking, there is a risk that students will not perceive the keeping of logbooks as very productive [10], [12], [15].
Based on the evaluation feedback, the e-logbook tasks were easy to implement in day-to-day general practice. The continuous involvement of supervising physicians and students in shaping the content right from the start of the process (tip 2) was crucial for developing practice-oriented tasks [9]. Exceptions were the generic tasks relating to lifestyle counselling and collaboration with other health professions. These were perceived as difficult to implement by some supervising physicians and by around one third of the students. However, as these are compulsory tasks, this feedback is of particular importance for further module development.
Students frequently commented positively on the opportunity to select patient cases to work on according to the practice’s medical specialisation and/or their personal interests. However, there were also a few students who felt they had spent too much time searching for suitable cases and, as a result, had participated less in the day-to-day practice routine. Pathil et al. describe the challenge of finding suitable patients as a common logistical problem in medical education, though this is not attributable to the use of logbooks [22].
Regarding the benefits of the e-logbook as a training tool, several students commented that the e-logbook had provided structure to the two-week clinical placement – partly through the tasks – and had defined the objectives.
The tasks encouraged students to examine the supplementary educational resources in greater detail and promoted dialogue with the supervising physicians. This feedback is consistent with previous findings regarding the use of logbooks to structure clinical placements, define objectives and training content [1], [2], [3], [4], [5], [6], [7], [8], [9], as well as to support communication between supervising physicians and students [9], [10], [11], [12]. By documenting their work using guiding questions, the students learnt to present the cases in a structured manner and, through the subsequent feedback from supervising physicians, they gained a better insight into patients’ medical histories and general practice care. Other studies have also shown that reflecting on the cases and receiving feedback from supervising physicians play an important role in engaging with learning objectives and content, and in enhancing the learning outcome [11], [22], [26]. The e-logbook introduction sessions for supervising physicians (tip 11) with a focus on, amongst other things, how to give feedback thus appeared to be beneficial [9]. Nevertheless, the quantitative results of the student survey on the usefulness of the e-logbook as a training tool were, on average (mean=3.1), rather moderate, with a standard deviation of 1.15 indicating a highly heterogeneous range of opinions amongst the students. One reason for the low level of approval could be a lack of, or limited, access to the e-logbook during the two-week clinical placement , which may have led to a lack of exchange with the supervising physicians whilst working on the tasks. Consequently, supervisory meetings could not be carried out optimally. In their systematic review, Paydar et al. highlight the ability to provide constructive feedback in a timely manner as a key advantage of electronic logbooks [26]. However, this advantage can only be utilised if the tasks are completed in the teaching practices.
5.1. Limitations
During the period of the focused evaluation, only 72 out of 120 active teaching practices were involved in block clinical placements and invited to participate in the evaluation. As supervising physicians have a significant influence on students’ acceptance and use of logbooks, it would be important to involve all supervising physicians. The 27 who took part in the online survey rarely used the free-text fields, meaning that the qualitative analysis was based mainly on the students’ comments. Conducting feedback sessions in small groups with the supervising physicians on several occasions would have been a more advantageous survey method for the analysis.
6. Conclusion
The e-logbook provides students with a useful tool for documenting community-based clinical placements. The integrated portfolio components encourage reflection and feedback, thereby supporting the learning process. By integrating the e-logbook into Moodle, it can be used immediately as a template within the faculty for similar short-term placements in other departments or clinical internships, such as the practical year. And it is also suitable for other educational institutions with similar learning platforms. Time and technical resources during the clinical placement, as well as the supervising physicians’ engagement with the students’ e-logbook documentation, are essential in order to reap the benefits of the digital and interactive logbook. Specifically, for general practice clinical rotations, the availability of an internet-enabled workplace is an indispensable prerequisite for recertification or new certification as an academic teaching practice. For supervising physicians, training sessions must be provided on an ongoing basis regarding the objectives and content of the e-logbook, feedback and reflection processes, and the necessary technical and spatial infrastructure at the training site.
Authors’ ORCIDs
- Katrin Schüttpelz-Brauns: [0000-0001-9004-0724]
- Kristina Hoffmann: [0000-0001-7237-670X]
Acknowledgements
The authors thank Ana Maria Bordes and Katja Götze from the Quality Management Team for Studies and Teaching at the Mannheim Medical Faculty for their advice and for conducting the online surveys of students and supervising physicians, as well as Melina Studier and Ursula Goldberger for their editorial support. We thank Robyn May for proof-reading the English version of the manuscript.
Competing interests
The authors declare that they have no competing interests.
References
[1] Busemann A, von Bernstorff W, Heidecke C. Konsequenzen aus der Evaluation eines chirurgischen PJ-Logbuchs [Consequences drawn from the evaluation of logbook-based surgical training for final year students]. Zentralbl Chir. 2011;137(2):165-172. DOI: 10.1055/s-0031-1271384[2] Coates WC. Emergency Medicine Subinternship: Can We Provide a Standard Clinical Experience? Acad Emerg Med. 2003;10(10):1138-1141. DOI: 10.1111/j.1553-2712.2003.tb00589.x
[3] Dolmans D, Schmidt A, van der Beek J, Beintema M, Grever WJ. Does a student log provide a means to better structure clinical education? Med Educ. 1999;33(2):89-94. DOI: 10.1046/j.1365-2923.1999.00285.x
[4] Offergeld C, Neudert M, Zahnert T, Fischer M, Günther J, Giesler M. Einsatz und Akzeptanz des PJ‑HNO‑Basis-Logbuchs an deutschen Universitätskliniken und akademischen Lehrkrankenhäusern [Use and acceptance of the ORL logbook for final year students in German university and academic teaching hospitals]. HNO. 2020;68(4):248-256. DOI: 10.1007/s00106-019-00744-9
[5] Wolfgarten E, Mönig SP, Fetzner UK, Schröder W, Bollschweiler E. Erfahrungen mit einem fächerübergreifenden Logbuch für Medizinstudenten im Chirurgischen Tertial des Praktischen Jahres am Uniklinikum Köln [Experience with an interdisciplinary surgical handbook for medical students]. Zentralbl Chir. 2012;137(2):180-186. DOI: 10.1055/s-0031-1283810
[6] Boehm B, Liebhardt H, Fegert J. Neustrukturierung des Praktischen Jahres: Zufriedene Studierende in Ulm. Dtsch Arztebl. 2008;21:1153-1154. Zugänglich unter/available from: https://api.aerzteblatt.de/pdf/105/21/a1153.pdf
[7] Froehlich S, Kasch R, Schwanitz P, Schulz AP, Walcher F, Mittelmeier M, Obertacke U. Logbuch der erreichbaren Lernziele für die speziellen Wahlfachinhalte Orthopädie/Unfallchirurgie im Praktischen Jahr (PJ) des Medizinstudium [Logbook of learning targets for special educational skills in orthopaedic and trauma surgery for undergraduate medical training]. Z Orthop Unfall. 2013;151(6):610-663. DOI: 10.1055/s-0033-1351065
[8] Huenges B, Gulich M, Böhme K, Fehr F, Streitlein-Böhme I, Rüttermann V, Baum E, Niebling WB, Rusche H. Recommendations for Undergraduate Training in the Primary Care Sector – Position Paper of the GMA-Primary Care Committee. GMS Z Med Ausbild. 2014;31(3):Doc35. DOI: 10.3205/zma000927
[9] Schüttpelz-Brauns K, Narciss E, Schneyinck C, Böhme K, Brüstle P, Mau-Holzmann U, Lammerding-Koeppel M, Obertacke U. Twelve tips for successfully implementing logbooks in clinical training. Med Teach. 2016;38(6):564-569. DOI: 10.3109/0142159x.2015.1132830
[10] Jacobs JC, Bolhuis S, Bulte J, Laan R, Holdrinet RS. Starting learning in medical practice: an evaluation of a new Introductory Clerkship. Med Teach. 2005;27(5):408-414. DOI: 10.1080/01421590500087001
[11] Kadmon M, Roth S, Porsche M, Schürer S, Engel E, Kadmon G. Das interaktive Chirurgische Logbuch im Praktischen Jahr: Eine mehrjährige Retrospektive. GMS Z Med Ausbild. 2009;26(2):Doc22. DOI: 10.3205/zma000614
[12] Kraus B, Jünger J, Schrauth M, Weyrich P, Herzog W, Zipfel S, Nikendei C. Logbücher im klinisch-praktischen Einsatz: Profitieren die Studenten?: Eine Bestandsaufnahme bei PJ-Studenten der Inneren Medizin. GMS Z Med Ausbild. 2007;24(2):Doc112. Zugänglich unter/available from: https://www.egms.de/static/de/journals/zma/2007-24/zma000406.shtml
[13] Chiesa, D, Escalante R, van Wyk J, Bollela V. Evaluating logbooks to improve clerkship learning experiences. Med Educ. 2013;11(47):1122-1123. DOI: 10.1111/medu.12344
[14] Cottrell S, Smith T, Ferrari N, Cannarella Lorenzetti, Jacques M, Shumway J. A generalizability study of student ratings in logbooks to assess the educational process of clinical learning. Acad Med. 2010;85(7):1237-1241. DOI: 10.1097/acm.0b013e3181e00712
[15] Denton GD, DeMott C, Pangaro LN, Hemmer PA. Literature reviews: narrative review: use of student-generated Logbooks in undergraduate medical education. Teach learn Med. 2006;18(2):153-164. DOI: 10.1207/s15328015tlm1802_11
[16] Herbstreit S, Hille L, Rademacher F, Burggraf M, Mester B, Dudda M. Portfoliobasiertes Lernen in der Chirurgie: Kompetenzen „sichtbar“ machen. Portfolio-based learning in surgery: Making competencies "visible". Chirurgie (Heidelb). 2023;94(3):256-264. DOI: 10.1007/s00104-022-01758-w
[17] Tschudi P, Bally K, Isler R. One-on-one tutorials in private practices and clinics: four years of experience in Basel, Switzerland. Med Teach. 2003;25(5):537-543. DOI: 10.1080/0142159031000136734
[18] Amsellem-Ouazana D, Van Pee D, Godin V. Use of portfolios as a learning and assessment tool in a surgical practical session of urology during undergraduate medical training. Med Teach. 2006;28(4):356-359. DOI: 10.1080/01421590600628100
[19] Buckley S, Coleman J, Davison I, Khan KS, Zamora, J, Malick S, Morley D, Pollard D, Ashcroft T, Popovic C, Sayers J. The educational effects of portfolios on undergraduate student learning: a Best Evidence Medical Education (BEME) systematic review. BEME Guide No. 11. Med Teach. 2009;31(4):282-298. DOI: 10.1080/01421590902889897
[20] Driessen E, van Tartwijk J, van der Vleuten C, Wass V. Portfolios in medical education: why do they meet with mixed success? A systematic review. Med Educ. 2007;41(12):1224-1233. DOI: 10.1111/j.1365-2923.2007.02944.x
[21] Haffling AC, Beckmann A, Pahlmblad A, Endgren G. Students' reflections in a portfolio pilot: highlighting professional issues. Med Teach. 2010;32(12):e532-e540. DOI: 10.3109/0142159X.2010.509420
[22] Patil NG, Lee P. Interactive logbooks for medical students: are they useful? Med Educ. 2002; 36(7):672-677. DOI: 10.1046/j.1365-2923.2002.01163.x
[23] McKavanagh P, Boohan M, Savage M, McCluskey D, McKeown, P. Evaluation of a final year work-shadowing attachment. Ulster Med J. 2012;81(2):83-88.
[24] Jud SM, Cupisti S, Frobenius W, Benn S, Winkler A, Antoniadis S, Beckmann MW, Heindl F. Logbooks alone are not enough: initial experience with implementing a logbook for medical students in a clinical internship in gynecology and obstetrics. Eur J Med Res. 2020;25(1):15. DOI: 10.1186/s40001-020-00413-6
[25] Gummersbach E, Abholz HH, Altiner A, Stock K, Schürer-Maly C. Online-Logbuch im Hausarztpraktikum – Bericht über die erste Nutzung in Deutschland. ZFA (Stuttgart). 2010;86(2):73-79. DOI: 10.3238/zfa.2010.0073
[26] Paydar S, Esmaeeli E, Ameri F, Sabahi A, Meraji M. Investigating the advantages and disadvantages of electronic logbooks for education goals promotion in medical sciences students: A systematic review. Health Sci Rep. 2023;6(12):e1776. DOI: 10.1002/hsr2.1776
[27] Kunz AM, Pfadenhauer M. One Campus – Many Ways to Go?! A methodological comparison of paper-pencil and electronic logbooks when exploring students’ patterns of spatial use. J New Front Spat Concept. 2014;6:21-27. DOI: 10.5445/KSP/1000040032
[28] Perella P, Palmer E, Conway R, Wong DJN. A retrospective analysis of case-load and supervision from a large anaesthetic logbook database. Anaesthesia. 2019;74:1524–1533. DOI: 10.1111/anae.14843
[29] McVeigh, Williams R, Pangado L. An Internet home page to log students' patient contacts. Acad Med. 1997;72(5):417-418. DOI: 10.1097/00001888-199705000-00040
[30] Fischer MR, Bauer D, Mohn K. Finally finished! National Competence Based Catalogues of Learning Objectives for Undergraduate Medical Education (NKLM) and Dental Education (NKLZ) ready for trial. GMS Z Med Ausbild. 2015;32(3):Doc35. DOI: 10.3205/zma000977
[31] Berberat PO, Rotthoff T, Baerwald C, Ehrhardt M, Huenges B, Johannink J, Narciss E, Obertacke U, Peters H, Kadmon M. Entrustable Professional Activities in final year undergraduate medical training – advancement of the final year training logbook in Germany. GMS J Med Educ. 2019;36(6):Doc70. DOI: 10.3205/zma001278
[32] Wimmers PF, Schmidt HG, Splinter TAW. Influence of clerkship experiences on clinical competence. Med Educ. 2006;40(5):450-458. DOI: 10.1111/j.1365-2929.2006.02447.x
[33] Ryan RM, Deci EL. Intrinsic and extrinsic motivations. Classic definitions and new directions. Contemp Educ Psychol. 2000;25:54-67. DOI: 10.1006/ceps.1999.1020
[34] Brooke J. SUS-A quick and dirty usability scale. In: Jordan PJ, Thomas B, Weerdmeester BA, McClelland IL, editors. Usability evaluation in industry. London: Taylor & Francis; 1996. p. 4-7.
[35] Mayring P, Fenzl T. Qualitative Inhaltsanalyse. In: Baur N, Blasius J, editors. Handbuch Methoden der empirischen Sozialforschung. Wiesbaden: Springer VS; 2019. p. 33-648. DOI: 10.1007/978-3-658-21308-4_42
[36] Driessen EW, Muijtjens AM, van Tartwijk J, van der Vleuten CP. Web- or paper-based portfolios: is there a difference? Med Educ. 2007;41(11):1067-1073. DOI: 10.1111/j.1365-2923.2007.02859.x




