Peer to peer review and mentoring

Posted on by niche_admin in Uncategorized

Medical and nursing staff in the UK now have to undergo annual appraisals, usually with a peer who coaches them through their appraisal paperwork, discussing issues and highlights of the year with them and helping them to develop themselves as a health professional.  Although there’s always a mad rush at the end of the year to get all the relevant bits of paper uploaded to one’s appraisal file in time for the allotted meeting, the process if done well encourages the health worker to focus on their goals for the next year, helps to prevent burn out and allows us all the space to reflect on our own practice.

Agnes, instructor in Liberia

This process is in its infancy in Liberia and Cameroon.  Jarlath has put together a draft form which gathers information about skills and confidence decay, provides a method of assessing someone’s on-going competence and allows a structure for peer mentoring.  We are not quite sure how this will work in the field but will be rolling it out over the next few years in Cameroon while we work out how to help support local instructors in the long term.

Click here for a preview.

 

Evaluation – how to avoid the “not knowing what you don’t know” trap

Posted on by niche_admin in Uncategorized
Dr Jarlath O’Donohoe instructs on life support courses in many different countries (he has lots of embroidered shirts to prove it).  He is the educator on our Generic Instructor Courses and has been doing a lot of work on our course evaluation forms.  Here are some recent thoughts from him on “pre-post evaluation”:
“No one likes to waste their time. In developing country health services this is even more important since human resources are so scarce. Equally important is to avoid thinking you are wasting your time when you are not. In evaluating a training exercise it seems obvious that asking questions before and then after the training will help identify what is worth doing and what is not worth doing.  
However, like in so many other spheres of life, what is obvious is not always true and what is true is not always obvious. It turns out (Academic Emergency Medicine: Educational Advance: Bhanj F, Gottesman  de Grave W.  The Retrospective Pre–Post: A Practical Method to Evaluate Learning from an Educational Program, Feb 2012) that asking questions at both the beginning and end may fail to identify useful learning. The example given is of someone who thinks himself quite knowledgeable at the beginning of a course and scores himself 7/10.  Then, having learned a lot more about the topic, he again scores himself 7/10 at the end.  There has been a lot of learning and the 10 at the end of the course is a much bigger 10 and therefore the 7 is a much bigger 7.  This can not be shown statistically.
The term the authors use is retrospective pre-post (RPP) evaluation. Experience has shown some people scoring themselves as highly capable at the beginning of our training sessions but who are not able to do things like bag and mask ventilation. So we have moved to an RPP approach to evaluation.”
This entails giving our learners just one questionnaire at the end of the course which asks them how confident they felt before the course in certain skills and areas of knowledge and how confident they feel at the end of the course.  It seems that recall bias might be a lesser evil than “not knowing what you don’t know”.

Outcome measures

Posted on by niche_admin in Uncategorized

The ultimate outcome measurement for the Newborn Care Course project would of course be a reduction in neonatal mortality in the areas where we work.  There are so many confounding factors in any clean data that is actually collected that it is almost impossible to prove that one intervention like this has any statistically significant effect on neonatal mortality.  But our funders always ask for outcome measures.  This year in Cameroon we changed the feedback form a bit, bringing it more into line with the template suggested by the UK’s Royal College of Paediatrics and Child Health.  This has allowed us to measure pre- and post- course confidence in the main areas identified by WHO as contributing to newborn deaths.  Here are the results from last month’s course:

 

The challenge now, of course, is to keep that confidence up going forwards.