Development of the Teacher Feedback Observation Scheme: evaluating the quality of feedback in peer groups

Research suggests that feedback is an essential element in learning. This study focuses on feedback that teachers provide in reciprocal peer groups to improve their performance in the classroom. The Teacher Feedback Observation Scheme (TFOS) was developed to identify feedback patterns, which approaches feedback as a multidimensional process. The TFOS helps acquire insights into the effectiveness of feedback, and provides information regarding the situations in which possible interventions can be undertaken if feedback is declining and becoming ineffective. This may especially be necessary when the communication of feedback is mediated by information and communications technology (ICT). The TFOS was piloted using videotaped sessions of three face-to-face groups, as well as one virtual group, using discussion wikis. All four groups of teachers used the Video Intervision Peer-coaching (VIP) procedure. The findings reveal that feedback in the virtual group was less effective than it was in the face-to-face groups. In addition, ineffective feedback patterns in the face-to-face groups transitioned into more effective feedback patterns. The TFOS appears to be adept at identifying feedback patterns in peer groups.


Introduction
Numerous studies indicate that feedback is an important learning tool and an essential element in learning (Hattie and Timperley 2007;Hattie 2009). Most studies focus on feedback given to students and its effect on student learning. However, these studies rarely consider the role of teachers (Scheeler, Ruhl, and McAfee 2004). This is changing as society is struggling with impending shortages of highly qualified teachers, which has led to a growth in attention to teacher professional development (OECD 2002). In 2006, the Dutch Ministry of Education enacted the Law on Professions in Education to address teacher shortages in the Netherlands. This law elaborates on which competencies teachers should possess and how continuous professional development can contribute to these competencies.
This study focuses on the feedback that teachers give to one another in peer groups as part of their professional development activities, which are oriented towards the improvement of teachers' class performance. The goals of this study are: (1) to investigate the feedback process; (2) to determine whether the feedback given is effective; and (3) to explore which interventions could be implemented if feedback becomes less effective.
This approach differs from that of Williams et al. (2008), who developed a feedback cycle for the communication between on-campus and community-based teacher training programmes. Their feedback cycle focuses on local networking between organisations, whereas this paper's approach focuses on the feedback among teachers. Moreover, this paper's approach differs from most feedback studies, which usually compare one-dimensional differences in feedback characteristics (e.g. immediate versus delayed; Scheeler and Lee 2002), in that this study aims to classify feedback processes into effective and ineffective patterns of feedback, thereby approaching feedback as a multidimensional process.
To study the process of feedback, it is necessary to have an instrument that can analyse the social interaction during the feedback episodes. In particular, the instrument must be capable of identifying patterns of feedback that correspond to certain feedback acts, such as posing questions. The present feedback literature, however, does not report the existence of such an instrument. Therefore, it was decided to develop the Teacher Feedback Observation Scheme (TFOS) to meet these needs. By using the TFOS, it becomes possible to accomplish this study's goals.
This article begins with an overview of the feedback literature, providing insights into the characteristics, conditions, and effects of feedback. Informed by these insights, the first part of the TFOS was constructed. To test the potential of the TFOS in peer groups of teachers, a particular method for peer-coaching was applied, namely the Video Intervision Peer-coaching (VIP) procedure (Jeninga 2003). The application of the VIP procedure influenced the construction of the second part of the TFOS. Before presenting the TFOS, this article discusses the feedback literature as well as the VIP procedure. Then, the methodology of the pilot testing is explained. Finally, the results of the pilot testing are presented and discussed. The results are presented as an illustration of the usability of the TFOS; therefore, only preliminary conclusions can be drawn.
Overview of the feedback literature As was made clear in the Introduction, research on feedback mainly focuses on the learning of students (Hattie and Timperley 2007;Hattie 2009). Consequently, definitions of feedback reflect this focus. For example, Hattie and Timperley define feedback within the context of student learning as 'information provided by an agent regarding aspects of one's performance or understanding ' (2007, 81). However, if feedback is connected with teachers' learning through non-instructional professional development activities, feedback between learners can be defined as 'information that allows for comparison between an actual and a desired outcome' (Mory 2003, 746). Regardless of the definition used, feedback consists of at least one of the following four elements: (1) data on the actual performance of the learners (2) data on the standard of the performance (3) a mechanism for comparing the actual performance and the standard performance (4) a mechanism that can be used to close the gap between the actual and standard performance.
Hattie and Timperley's definition is based upon the first element, whereas Mory's combines the first three elements.
The question regarding what is effective feedback, and consequently ineffective feedback, can be answered by synthesising feedback literature into six dimensions. First, feedback can be directed at the task or goal (Black and Wiliam 1998a) or at learners and their characteristics. Task-or goal-directed feedback is more effective than person-directed feedback (Hattie and Timperley 2007). Second, feedback can be directed at a specific aspect (Mory 2003) or at a general aspect. Specific feedback is more effective than general feedback, although general advice on how to improve one's actions in the future is effective (Black and Wiliam 1998b). Third, feedback can be detailed or vague. Feedback that focuses on specific details is more effective than vague feedback (Scheeler, Ruhl, and McAfee 2004). Fourth, feedback can be corrective (i.e. saying something is wrong and providing a specification of what is wrong and what to do to correct it) or non-corrective (i.e. saying something is wrong without further specification; Scheeler, Ruhl, and McAfee 2004). Corrective feedback is believed to be more effective than non-corrective feedback. Fifth, feedback can be positive or negative. Although some researchers argue that feedback should be positive (Scheeler, Ruhl, and McAfee 2004), others argue that negative feedback can motivate learners (Schelfhout, Dochy, and Janssens 2004), and some even argue that feedback is more effective when it is balanced between positive and negative comments (Weaver 2006). Sixth, the timing of feedback can be either delayed or immediate. Immediate feedback is considered to be more effective than delayed feedback (Mory 2003).
The Video Intervision Peer-coaching procedure The VIP procedure (Jeninga 2003) emphasises reciprocal feedback in a peer group, usually consisting of three teachers. The VIP procedure defines two roles for the teachers, namely that of the coached teacher (CT) and that of the peer coach (PC). Teachers switch between these roles. During each turn, there is one CT, which implies that the other two teachers are PCs. By switching roles, each teacher will be the CT once and the PC twice.
The VIP procedure can be regarded as a practical realisation of the theoretical concept, Visible Learning. This concept contains six signposts (Hattie 2009, 238-9). Hattie argues that despite Visible Learning being focused on student learning, the concept is also applicable to teacher learning. Next, the VIP procedure will be described, after which an illustration of how the Visible Learning signposts are transferred to teacher learning in the VIP procedure will be presented.
The VIP procedure consists of four main cyclic steps (Jeninga 2003). In the first step, teachers decide which teaching behaviours they want to improve, and they then videotape these specific teaching behaviours. In the second step, teachers meet in the first VIP session. Each teacher gets their turn as the CT, while the two other teachers are PCs. The CTs briefly introduce the teaching behaviours that they want to improve and show the associated video excerpt. Subsequently, the PCs use solution-focused thinking (Jackson and McKergow 2002), which aids the CTs in proposing a solution to tackle their teaching behaviours. At the end of each teacher's turn, the goals and actions are recorded in an Action Improvement Plan. In the third step, teachers practise their formulated actions, and videotape their altered teaching behaviours again, which are hopefully improved. In the fourth step, teachers meet in a second VIP session, and all teachers take their turn again. The CTs elaborate on their altered behaviour, and show the newly made video excerpt. The CTs provide feedback, after which the PCs provide their feedback. Next, the CTs evaluate their own behaviour. The CTs give themselves a grade that expresses their satisfaction with respect to the extent that they have reached their goal, and they provide some explanation for this. In addition, the CTs are asked what they can do to raise this grade (e.g. giving an 8 instead of a 7). The feedback and evaluation are recorded in the Action Improvement Plan. Finally, the CTs decide whether they are interested in examining another teaching behaviour or in further improving the current teaching behaviour. In the first case, the cycle begins again; in the latter case, their goals and actions are readdressed, reformulated or adapted, as if they were in the second step of the VIP procedure. The CTs then move to the third step.
Usually, the peer group is guided by a process supervisor, whose task is to facilitate the teachers in the VIP sessions. Process supervisors act as chairmen during the sessions by modelling coaching behaviours and reflecting on the teachers' coaching behaviours (Jeninga 2003). The process by which the Visible Learning signposts (Hattie 2009) are transferred to the VIP procedure is shown in Table 1.

The Teacher Feedback Observation Scheme
The TFOS was developed to analyse the process of giving feedback, given that feedback literature does not report the availability of such an instrument. Based on the dimensions describing the effectiveness of feedback (see above: Overview of the feedback literature), the TFOS considers the characteristics, conditions and effects of feedback. Five of the six dimensions that describe effectiveness of feedback are incorporated in the TFOS: (1) goal-directedness versus person-directedness (2) specific versus general (3) detailed versus vague (4) corrective versus non-corrective (5) positive versus negative.
If feedback is goal directed, specific, detailed, corrective, and balanced between positive and negative comments, then it is more effective than feedback that is person directed, general, vague, non-corrective, and either too positive or too negative. The dimension of timing was not included in the TFOS for two reasons: first, in face-to-face settings, feedback is always communicated during the VIP session, and adequate timing of feedback is difficult to observe. Second, in the virtual setting, it was not possible to detect when messages were posted, because of technical issues.
Given the application of the VIP procedure in this study, two additional aspects are included in the TFOS. First, several types of questions are scored. The VIP procedure is based upon solution-focused thinking (Jackson and McKergow 2002), which assumes that individuals can find their own solutions to their problems. This is facilitated by PCs who mainly ask open-ended, solution-focused questions. The PCs also facilitate the CT in attaining a clear picture of their goals and concrete actions by asking clarifying questions, and through continuous questioning. Therefore, solution-focused thinking emphasises that judging and evocative questions do Teachers need to be direct, influential, caring, and actively engaged in teaching and learning.
Teachers are placed in the centre of the process.
Their own teaching behaviours are discussed, as well as how they can improve these behaviours.
Teachers decide which teaching behaviours they want to improve and then act upon this. The PCs and process supervisor influence and support the CT in their learning process.
Teachers need to be aware of what each student is thinking and knows, to construct meaningful experiences in the light of this knowledge, and have proficient knowledge and understanding of their content to provide meaningful and appropriate feedback, such that each student moves progressively through the curriculum levels.
Teachers consciously engage in their goals and actions, which allows for meaningful experiences. These experiences are videotaped and commented on by CTs and PCs. Because of the cyclic workflow, the teachers will progressively move through their self-selected goals.
Teachers need to know the learning intentions and success criteria of their lessons, know how well they are attaining these criteria for all students, and know where to go next in light of the gap between students' current knowledge and understanding and the success criteria of 1) 'Where are you going', 2) 'How are you going', and 3) 'Where to next'.
The goal and actions of the teachers are explicitly formulated, addressing the gap that they experience between their current position and their desired position. The three questions are addressed in the Action Improvement Plan. The goals of the teachers are formulated. In addition, the Action Improvement Plan guides the teachers during their progress by formulating the actions and evaluating their performance of the actions. During the evaluation, the Action Improvement Plan reinforces the teachers to think about alternative improvement actions that could possibly be better than the current one.
Teachers need to move from a single idea to multiple ideas, and to relate and then extend these ideas, such that learners construct and reconstruct knowledge and ideas. It is not the knowledge or ideas, but the learner's construction of this knowledge and these ideas that is critical.
Constructing knowledge is thinking of alternatives, thinking of criticisms, proposing experimental tests, deriving one object from another, proposing a problem, proposing a solution, and criticizing this solution (Bereiter 2002). These kinds of knowledge construction are all facilitated in the VIP procedure.
School leaders and teachers need to create school, staffroom, and classroom environments in which error is welcomed as a learning opportunity, discarding incorrect knowledge and understanding is welcomed, and participants can feel safe to learn, re-learn, and explore knowledge and understanding.
The VIP procedure is meant to be a safe environment. It is one of the tasks of the process supervisor to create such an environment.
not fit into this process. Second, the VIP procedure aids teachers to find solutions to their problems. In other words, the teachers are actually engaged in problem solving. PCs can provide feedback on problem solving by providing hints or tips and asking guiding questions (Smith and Ragan 1993). The purpose of the TFOS is to identify feedback patterns. To detect these patterns, White's stages (2009) as well as Miles and Huberman's (1994) suggestions were used. The section on data analysis will provide an explanation as to how this was performed.

Method
Participants Thirteen teachers (five male, eight female) participated in the pilot study. They were assigned to three face-to-face groups and one virtual group. Table 2 shows the demographics and other characteristics of the teachers, as well as to which group they were assigned. All teachers were given a fictitious name. The name of each group consisted of the first letters of these fictitious names. Teachers in the face-toface groups worked at the same school, and the teachers in the virtual group were from three different schools.
A process supervisor facilitated the face-to-face groups. The process supervisor (male, aged 53) had previous experience in this position, was unfamiliar to the participants at the start of the study, and had no affiliations with the school. Between sessions, he was available to the participants for support and questions.

Procedure
All teachers had a face-to-face introduction session so that they could familiarise themselves with the VIP procedure. The virtual group used a Moodle environment that contained discussion wikis. These discussion wikis were formatted according to the structure of the Action Improvement Plan. The content of the discussions wikis was scored according to the TFOS. The face-to-face groups participated in three VIP sessions at their school. These sessions were videotaped, transcribed, and scored using the TFOS.

Data analysis
To investigate the interrater reliability of the TFOS, 2 independent researchers scored one randomly chosen session. Cohen's kappa was used to determine the interrater reliability. Cohen's kappa expresses the extent to which the descriptions of the several dimensions and elements are univocally interpreted. The interrater reliability (Landis and Koch 1977) varied between 0.410 and 1.000, averaging 0.756, indicating that scoring using the TFOS is substantially reliable. Table 3 provides an overview of the elements each utterance is scored upon according to the TFOS, their source and the Cohen's kappa.
Each session and wiki was scored using the TFOS. In addition, the transcripts and wikis of each teacher's turn at being the CT were divided according to White's stages (2009). White developed a quality feedback process model, consisting of three stages: Observational stage, Analysis stage and Reflective stage. The Observational stage 'is derived from lecturers observing students while they are teaching on  practicum ' (2009, 128). In the VIP procedure, this stage is addressed in the second and fourth steps, when the peer groups watch and discuss the video excerpt. In the Analysis stage, the lecturers coach the students in formulating goals and actions to improve their practices. In the VIP procedure, this stage is mainly addressed in the second step. The Reflection stage consists of a debriefing session, in which the lecturers provide written and oral feedback to the students regarding the actions that they initiated. The VIP's fourth step is similar to this stage. The divided transcripts were placed into matrices, combining several matrices to be used in qualitative data analysis (Miles and Huberman 1994). By combining White's stages and Miles and Huberman's matrices, several patterns of feedback were discovered.

Results
The TFOS detects effective and ineffective feedback patterns. These are defined both in the literature review regarding feedback and in the review of the VIP procedure. Effective feedback is goal directed, specific, detailed, corrective, and balanced between positive and negative comments. Also, effective feedback includes asking open-ended, solution-focused and clarifying questions. Ineffective feedback is person directed, general, vague, non-corrective, and either too positive or too negative. In addition, ineffective feedback includes hinting, judging and asking evocative questions.
This section is comprised of three parts. First, effective patterns of feedback are described. Then, ineffective patterns of feedback are discussed, including how these can be turned into effective patterns. Finally, the role of the process supervisor is addressed.

Effective patterns of feedback
In the Observation stage, three patterns were discovered. First, the PCs and process supervisor tended to ask clarifying questions that elicited the CTs to elaborate on their video excerpts, such that the CTs' feedback became more goal-directed, specific, and detailed. Second, if the CTs performed their actions well, then the PCs and process supervisor would provide plentiful positive feedback in the fourth step of the VIP procedure. Third, if the CTs were in the fourth step of the VIP procedure (i.e. discussing the changed behaviour), before watching the video excerpt, the process supervisor would ask the CTs to repeat their goals and actions.
In the Analysis stage, three patterns emerged. The first pattern emerged in the third session of the DEFG group. Diana (PC) provided two hints that might help Gerald (CT). However, both times Gerald explained why these hints were not useful. Then, Diana remained silent during the rest of Gerald's turn. The second pattern exemplified how Steve (the process supervisor) guided the CTs in formulating their goals and actions, and hinted at what to videotape for the next session. Steve used the coaching technique of 'listening, summarizing and continuous questioning', by having the CTs repeat what they intended to do, and asking questions such as, 'What helps you to actually do this?' or 'How are you going to remember to implement these actions?' This led the CTs to become more goal directed, specific and detailed. In the third pattern, the CTs mentioned a situation that was similar to the behaviour to be improved. The PCs and process supervisor then asked questions that led the CTs to elaborate on the situation and on what the CTs' actions were. These actions were transferred to their recent behaviour. In this pattern, the same coaching techniques as described in the former pattern were implemented.
An example of this pattern can be found in the DEFG group's first session. Diana (CT) videotaped a disruptive class with many disruptive students. She had trouble with her classroom management. Her PCs and Steve then asked her whether she had less disruptive classes in which her classroom management was better, and what she did differently in those classes. These actions were then recorded in Diana's Action Improvement Plan.
In the Reflection stage, three patterns emerged. In some situations, the CTs immediately wanted to initiate a new goal with a new video excerpt. Then, Steve interrupted by asking the CTs to reflect on their former goal and to fill in the Action Improvement Plan. In other words, Steve ensured that the Reflection stage was attended. Second, it appeared that the grade the CTs assigned did not depend on the length of the Reflective stage, the amount of positive feedback provided before they gave the grade, or whether a CT immediately wanted to initiate a new goal. The third pattern only emerged within the second and sometimes the third peer coaching sessions, in which the video with changed teacher behaviour was shown and discussed (i.e. the fourth step of the VIP procedure). To guide the discussion about this changed teacher behaviour, the Action Improvement Plan was used. Following this Action Improvement Plan, the CTs provided themselves with feedback, then received feedback from their colleagues and, finally, evaluated the changed behaviour. After the evaluation, a new goal was formulated by the CTs, and their PCs applied solution-focused thinking. Steve steered this process, mainly by asking guiding questions.

Ineffective patterns of feedback
In the KLM group (the virtual group), it appeared that interaction occurred only once. The CTs mainly filled out their Action Improvement Plan. The PCs sometimes posted a reaction; however, there was only one case in which the CT actually responded. In addition, feedback was provided in less effective ways than in the face-to-face groups. In particular, many hints were provided, many evocative questions were posed, and judging appeared frequently. To address the third aim of this study, the TFOS examined hinting, evocative questions and judging within the faceto-face groups, as well as whether these ineffective feedback patterns evolved into more effective feedback patterns.
Hinting occurred 43 times across the face-to-face sessions. Specifically, hinting occurred seven times in the Observation stage, 32 times in the Analysis stage and four times in the Reflection stage. In most cases, CTs turned hinting into more effective patterns of feedback. The CTs tended to agree with the hinting and then elaborated on the matter, sometimes followed by a discussion between the CTs and the PCs as to the content of the hinting. If the CTs did not agree on the hinting, they explained why. The PCs, who did not provide a hint, also turned hinting into more effective feedback, by being positive. Finally, the process supervisor was also involved in this process. Either he entered the next step in the processfor example, writing actions down in the Action Improvement Planor he explained what coaching is.
Judging occurred 56 times across the face-to-face sessions. Specifically, judging occurred nine times in the Observation stage, 33 times in the Analysis stage and 14 times in the Reflective stage. In approximately half the cases, the CTs judged themselves. In the remaining cases, the judging was transformed into more effective ways of providing feedback. Several times, the CTs agreed and explained their thoughts about the contents of the judging from their PCs. In a few cases, the CTs weakened the judging. Furthermore, the PCs and the process supervisor, who did not do the judging, provided positive feedback. Finally, the process supervisor entered a new step in the process; for example, asking the CTs to describe their planned actions. The judging remained ineffective in only one case. In the second session of the DEFG group, Eric (PC) judged Diana's (CT) efforts in improving her classroom management. Then, Eric explained his own experience regarding temporarily expelling students from the classroom. He would have been more productive by asking an open-ended, solution-focused question.
Evocative questions were posed 10 times across the face-to-face sessions. Specifically, two evocative questions were posed during the Observation stage, six during the Analysis stage and two during the Reflection stage. In most cases, the CTs provided an answer to the question. The process supervisor sometimes summarised what was said or turned to a next step in the process. Only in one case, in the second session of the DEFG group, was an evocative question posed to Gerald (CT) by Eric (PC), which led Eric to answer the question himself. In sum, the TFOS indicated that in almost all cases, the apparently ineffective patterns of hinting, judging and evocative questions were indeed turned into effective feedback patterns.
Differing ineffective feedback patterns emerged during the Analysis stage. In two cases, one in which Diana was CT and one in which Gerald was CT (DEFG group's second session), Eric's (PC) input had an awkward timing when providing feedback. In both cases, Eric did not say much during the turns. At the end of the turn, when Steve had already suggested giving somebody else a turn as CT, Eric interrupted and started asking questions. Another example of an ineffective feedback pattern was from Hedwig, who frequently tended to finish Isabelle's or Joanna's sentences in their turns as CTs. Finally, Steve slipped into an ineffective feedback pattern once. In the third session of the HIJ group, at the end of Hedwig's turn as CT, Steve stated that there were only two minutes left in her turn. Then, Steve elaborated on his own experience as a teacher.
The process supervisor An experienced process supervisor facilitated the face-to-face groups. The TFOS shows how he steered the process by asking guiding questions, how he reflected explicitly on the coaching behaviour of the participants, and how he contributed when feedback tended to become less effective. Steve (i.e. the process supervisor) led the VIP sessions by asking guiding questions, for instance: 'Can you say once more what it is about?' and 'Could you now record on your Action Improvement Plan what you are planning to do, in detail, when you go to teach that lesson?' An example of how Steve reflected on the coaching behaviour and provided feedback is found in the third session of the HIJ group. During Hedwig's turn as the CT, Steve was silent, while Isabelle and Joanna coached Hedwig. After about 15 minutes, he interrupted the process by summarising what had been said: 'During the story, you [Hedwig] mentioned three things that you think are important, that you want to work on'. Steve repeated these three things, and then reflected on the coaching and feedback process: 'That question that Joanna posed made you think real hard about how you could try to perform those things'. During the remaining part of Hedwig's turn, Steve steered the conversation in such a manner that these three things were all specified, so that Hedwig could record them in the Action Improvement Plan.
Steve intervened when feedback tended to become less effective, as is shown by an example of the ABC group's first session. In Ann's turn at being the CT, Chris (PC) summarised and analysed Ann's goal: '[…] that is another moment when you need to start again. That costs loads of energy, while it may be better that before you start to give the instruction, you should …'. Then, Steve interrupted: 'Hang on a minute. Your analysis is fine, but she [Ann] needs to draw the conclusion herself'. Subsequently, Steve took over the coaching and steered the process in such a manner that Ann drew the conclusion for herself. Steve also interrupted during less effective feedback, then stimulated the PC to try again in a more effective way: That is that you try, by asking questions, to aid the other in thinking how to do better the next time. Chris: Oh, yeah. I should have asked a question. So, how would you …

Discussion
The purpose of this study was to investigate the feedback process, to determine what effective feedback is, and to explore what interventions could be possible if feedback becomes less effective in face-to-face settings and in virtual settings. Based upon the literature review, an instrument for observing feedback was developed: the TFOS. The TFOS was tested using videotaped sessions of three face-to-face groups and using one virtual group's wikis. All groups used the VIP (Jeninga 2003) procedure, which is, theoretically, a practical realisation of the theoretical concept of Visible Learning (see Table 1) (Hattie 2009). The data showed that, in practice, the VIP procedure realises the six signposts of Visible Learning. Therefore, the VIP procedure, at least in face-to-face settings, is a context in which teachers can provide effective feedback to their colleagues. The TFOS supported this insight, and can be used in other contexts in which teachers provide feedback to each other, so that feedback patterns and their effectiveness in these contexts can be investigated. The pilot testing of the TFOS provided insights into the usefulness of the TFOS, as well as preliminary insights into feedback and feedback patterns between teachers.

Usefulness of the Teacher Feedback Observation Scheme
The TFOS identifies five dimensions of feedback, two elements of feedback on problem solving and five types of questions, thereby reflecting the feedback literature. The interrater reliability was substantial, although future studies may need to consider some elements for which Cohen's kappa was low (i.e. the dimensions of positive versus negative and corrective versus non-corrective, and hints). The TFOS, combined with stages in the feedback process cycle (White 2009), was able to categorise effective and ineffective feedback patterns in face-to-face groups, as well as the virtual group. To better understand feedback patterns, the TFOS should include other elements that emerged frequently during the sessions of the face-to-face groups. First, student-directedness and video-directedness can be added to the dimension goal-directedness versus person-directedness. This addition appears to be context specific for the VIP procedure. Second, summarising and acknowledging are both coaching skills that can positively affect the feedback process and should be added (Gallacher 1997). Third, finishing a CT's statement and providing an example of one's own classroom can be included, as they both emerged frequently. Nevertheless, it may be questioned whether these former activities stimulate or hamper the process. Both seem to contradict solutionfocused thinking (Jackson and McKergow 2002), which may cause ineffective feedback patterns. Another issue is that the TFOS only observes feedback. To better understand why some patterns are effective or not, the participants could be interviewed regarding their experiences during providing and receiving feedback. Thus, the TFOS will be extended with a short questionnaire about how the feedback from each session was experienced, as well as including an interview.
This pilot study indicates that feedback effectiveness depends primarily not on how feedback messages are categorised, but rather on patterns or chains of combinations of the feedback dimensions, feedback on problem solving and types of questions. Studies examining patterns of feedback are mainly in the field of second language acquisition (Ashwell 2000;Ferreira, Moore, and Mellish 2007). Future research should investigate alternative methods of understanding the patterns found and why some are more effective than others. This knowledge can be used to instruct teachers as how to provide more effective feedback to one another.
Feedback and feedback patterns: results of the pilot study The preliminary results indicate that the feedback process in the face-to-face groups was more effective than in the virtual group. A process supervisor guided the faceto-face groups in contrast to the virtual group. He steered the process by posing guiding questions, by modelling coaching behaviours (i.e. posing solution-focused, clarifying, open-ended questions) and by reflecting explicitly on the participants' coaching behaviours. Process supervisors can stimulate effective feedback if they perform their role as intended. The results concerning feedback and feedback patterns cannot be generalised, owing to the small research population and to the fact that the conclusions are preliminary, because the TFOS was piloted.
Several patterns of feedback emerged in the face-to-face groups. If ineffective patterns of feedback occurred in the face-to-face groups, the participants would transform them into more effective patterns of feedback. Interactions in the virtual group did not occur and, therefore, such patterns could not arise. One possible explanation for the lack of social interaction in the virtual group is evident from the field of computer-mediated communication. According to computer-mediatedcommunication researchers, face-to-face conversations differ from conversations mediated by information and communication technology (ICT), in that ICT limits the transfer of socio-emotional cues. These cues are necessary to create, sustain and reinforce a positive group climate, in which a sense of community and committed social relationships exist. Feelings of community can increase the flow of information between group members, commitment to group goals, cooperation among the group members and satisfaction with group efforts (Kreijns, Kirschner, and Jochems 2003). Committed social relationships are a significant contributor to the effectiveness of information exchange (Warketin, Sayeed, and Hightower 1997). In sum, if social interaction and feedback is to occur in virtual groups, it is important to consider the limitations of ICT. Following Danchak, Walther, and Swan (2001), social awareness tools and training on writing virtual messages could be necessary to provide effective virtual feedback. Future research could explore whether social awareness tools and training on writing virtual messages stimulate effective feedback.
Future research could also investigate the learning outcomes of VIP participants in terms of their behaviour in the classroom. Learning outcomes are a key issue in feedback literature (Mory 2003). Research has investigated how several types of feedback influence learning outcomes in different types of learning. It would be interesting to investigate whether effective and ineffective patterns of feedback influence the teachers' behaviours in the classroom. This would provide additional insight into the effectiveness of types of feedback patterns among teachers.

Conclusion
This study was conducted to expand further the knowledge on feedback processes between teachers (Scheeler, Ruhl, and McAfee 2004), with the aim of improving these processes. The literature review shows that feedback is a complex process with many confounding variables. This may explain the lack of an instrument that evaluates the quality of feedback. By constructing and validating such an instrument (i.e. the TFOS), important insights regarding effective feedback among teachers were found and directions for future research were established. In sum, the TFOS is suited to observe and categorise effective and ineffective feedback patterns in different settings (face-to-face and virtual). These insights lead to further improvement of teacher learning and teacher professional development. For instance, a recommendation for practice is that if teachers and schools choose to apply the VIP procedure, it requires a proficient process supervisor.
Furthermore, the results clearly indicate that the effectiveness of feedback does not primarily depend on how feedback messages are characterised, as is postulated in many feedback studies. The effectiveness of feedback depends on patterns or chains of interactions between providers and receivers, thereby organising feedback into a multidimensional process. The pilot testing of the TFOS provides a solid argument for investigating alternative methodologies examining the effectiveness of emerging feedback patterns.