Annual EMSC Grantee Meeting

Measuring Change After Educational Interventions:

How Educational Consultants Can Improve

June 20 – 22, 2006

DONNA JEFFE: and in itself, although it can be. Increasing or maintaining an individual self-esteem and pleasure are also strong motivators for adult learning. Okay.

Now, in developing educational, our behavioral interventions, it is important for us to remember one thing. Most of the people who begin to target our interventions on, don’t want to change. They don’t want to do what we ask them to do. Okay. Because -- so, this is a similar situation like kids in school, you know, we’re telling them what we want them to do. And so, a line of our interventions is going to have to involve a change in values, and attitudes, and knowledge. And, the best way to motivate adult learners is to enhance their reasons for learning, and decrease their barriers to learning. And, having more than one reason to learn is better than having only one reason to do so. And, I’ll show you in a minute how the trans-theoretical model, has incorporated this behavioral maxim into their model, through the use of pros and cons.

The curriculum designer, as I said, must make an ascertainment about whether their intervention are the curriculum, is in concert with, or in conflict with, the attitudes and the behaviors of the learner. Okay, so, as I said, some instruction must be designed to change values and attitudes. Adults need to understand why it’s important to change their behavior. And so, especially when we’re dealing with people who may come from different walks of life, who had different educational backgrounds, and who have different job descriptions, we need to consider how to anchor our instruction, so that it is perceived as relevant to the broadest number of people possible.

Adult learners like self-directed and self-designed learning projects; they like to use more than one medium. They like to use the Internet, for example, and they like to set their own pace at learning, which can be very challenging in a research setting, where we’re constrained by the time frame for our research. They want – they see a need for practical use, and how-to information, and these are important reasons for adult learning.

Self-direction, however, does not mean in isolation. It’s important to incorporate other people into the project that serve as resources, or guides, or mentors. We use peer coaches, and life coaches at Washington University, and a number of projects that have been very helpful in terms of delivering interventions to people. And then, lectures and short seminars also are found to be valuable to adult learners.

So, in summary, motivation is the key to adult learning, and to increase motivation we need to enhance the reasons for learning, and decrease barriers to learning. When confronted with life change events, adult seek out learning experiences to help them cope, and they use a variety of media and strategies, and adult learners like to set their own pace. Now, several of these concepts, as I’ve said, have been incorporated into one of the most frequently used models and behavior change, and that ‘s the trans-theoretical model. And this has been used in the medical setting, and in the community. And, I’ve included some readings. There was supposed to be a hand-out, and I don’t know if she put them in the back, but at some point, I hope that you will get a hand-out to the slide, and at least you’ll have the readings to the slide. Okay.

What happened to my objectives? I’m going to address the second two objectives now, to describe how to effectively change behavior through educational interventions, and talking about the two models, the trans-theoretical model, and the theory of reason to action, which is now called the theory of planned behavior. The trans-theoretical model of behavior change can help improve our understanding of how people change their behaviors. This is based on 20 years of experience at the Cancer Research Prevention Center at the University of Rhode Island under the leadership of Jim Prochaska, who is a psychotherapist. Now Jim Prochaska, over the years, noticed that people he saw for therapy were at different stages of readiness, either to accept more and engage in more healthy behaviors and also quit unhealthy behaviors. He was very interested in understanding what works for changing behavior for people who are at different stages of behavior change. Now behavioral scientists and educational scientists can contribute to efforts to change behavior by assessing relationships among knowledge, attitudes and behavior, and also by utilizing behavioral and theoretical models that can explain why some people change and other people do not.

There are four elements of the trans-theoretical model of behavior change. And first and foremost, it’s important to remember that behavior change is a process. It doesn’t just occur. You don’t just tell somebody what they’re supposed to do and then they do it, because that doesn’t happen. And so, the integral part of this model is the stages of change. And each of these other components, the decisional balance, which is the pros and cons which I talked about, self-efficacy and processes of change, which is what people use when they’re trying to change their behavior, are all related to the stages of change and the outcome of interest. There’s five stages in this model: pre-contemplation, contemplation, preparation, action and maintenance. And I’ll talk about each one a little. And people can move through the stages either toward a desirable stage like preparation or action or maintenance, or they can relapse to a less desirable stage, like from action to contemplation. The first stage is pre-contemplation, and this is when people are not thinking about changing their behavior.

There are two subgroups: the “I can’t” and the “I won’t” group. And the people who are in the “I can’t” group are people who may be unaware of the problem or the benefits or the risks of the behavior, or life is just too complicated for them right now and they can’t think about making a change. The “I won’t” group are people who know and are aware of benefits and risks of changing their behavior, but they don’t think it’s important enough to change. Generally, you inquire about a person’s readiness to change and you ask them, “Are you thinking about changing in the next six months?” And it’s very important to be extremely specific about the behavior that you’re talking about, and about the time frame that you’re talking about. So you don’t want to just ask them, “Are you thinking about exercise in the next six months?” Define what you mean by exercise. Do you mean walking everyday for 15 minutes during your lunch period, or do you mean jogging four times a week for at least a half hour? Be very, very specific about the behavior and the time frame.

Now the trans-theoretical model is used a lot in trying; it’s self-directed behavior change for individuals who are either a lot of times trying to exercise, quit smoking, things like that. And they generally use the six-month time period, but you can use any time period that makes sense for the behavior that you’re trying to get people to engage in. Contemplation is when people are thinking about changing their behavior in the next six months. If they say they’re thinking about it, then they’re in contemplation. And empirically, people who are in contemplation are much more likely to actually go ahead and engage in a behavior than people who are in pre-contemplation, people who are not thinking about changing. Preparation is when people are planning to start the behavior in 30 days, according to the definition of the behavior and they’ve taken some steps to do it, okay? So it’s important that they’ve done something that will put them in preparation. If they’re trying to quit smoking, setting a quick date would be an example of an action that they’ve taken in order to get them into preparation.

Now, many intervention programs are implicitly designed for people who are in preparation. So if you want to have a successful intervention to get at more than just the people who are in preparation, you’re going to have to design your intervention that takes into consideration people who are also in pre-contemplation and contemplation. A person who is in the action stage has already begun doing the behavior for some amount of time and this is usually less than six months because if you’ve been engaging in the behavior for six months then they’re in the maintenance stage.

Why is stage important? Well, as I said, you want to tailor your intervention so it’s going to reach the broadest number of people. That it has been shown in smoking cessation, exercise, dietary change, and screening mammography that if you tailor your intervention for people at different stages, you’ll have greater success. In progressing even one stage, is shown to be associated with actually engaging in a behavior.

So these next few slides are going to show you why it’s important to consider stage. These are data from two population-based samples of smokers in California and Rhode Island. And as you can see, most of the people are in pre-contemplation and contemplation, not in preparation. So if you’re going to design your intervention for people in preparation, you’re going to miss 70 or 80 percent of the population. This pretty much shows the same thing from some studies using data from the Harvard community health plan, which is one of the largest health plans in Rhode Island and Eastern Massachusetts in the 1990s. Most people here for each of these types of study were in pre-contemplation or contemplation.

This kind of gives you an idea of who signs up for a study and then who goes on to participate in the study. And twice as many people who are in contemplation and preparation, the 65 percent signed up to participate in this study compared to to pre-contemplators and of those, five times as many people in contemplation and in preparation actually showed up for the intervention compared to pre-contemplators and five times as many completed this study compared to pre-contemplators.

And this slide shows how progress even at one stage to progress through one stage is associated with action at six months so people who were in pre-contemplation at baseline and then still in pre-contemplation at one month, only three percent were in action but people who progress to contemplation, seven percent were in action at six months and then the same thing with contemplation. If they stayed in contemplation, only 20 percent moved but if they had moved to preparation at one month, 41 percent were in action at six months.

Now the decisional balance component to this model talks about the pros or benefits and the cons or barriers to participating or engaging in a target behavior. And a salient pro can outweigh the cons of behaving. This graph is a very common graph that you see in a lot of studies. It doesn’t really matter what the behavior is. You’re going to see this kind of relationship. At pre-contemplation, the pros are lower than the cons, okay? The cons outweigh the pros in the early stages. And as you move to preparation and action and maintenance, then the pros start to outweigh the cons. Self-efficacy is another component of the trans-theoretical model and this is simple the belief in ones ability to engage in this specific behavior. And is especially important to consider engagement in the behavior in challenging situation. So, within the trans-theoretical model, self-efficacy is operationalized as both confidence in performing a behavior and temptation to relapse or not be able to perform the behavior. And there is different implications for people at different stages.

So for people who where in pre-contemplation, self-efficacy can indicate either an unwillingness to engage in the behavior or inability to engage in the behavior, whereas for people who are in action, they are more able to measure their level of confidence because they’re already performing the behavior. Higher confidence levels are associated with progress to action and higher temptation levels are associated with relapse and these questions are questionnaire type items and they’re scaled usually on one to five or on one to six scale. And this graph shows how confidence increases as you move forward through the stages. Now people also use processes of change, I think.

And there are two groups of processes of change. This is what people use actually to help themselves engage in behaviors. And their two components, one is more associated with people in early stages and they’re more cognitive or emotional like getting information or appealing to one’s emotions, perceptions of one’s own self, engaging in a behavior or acting as a role model or realizing social norms. Look at all the gyms around and see it must be important to exercise because everybody is doing it. The second component of processes of change are more behavioral and they involve actually making a commitment to change, setting queues or reminders, appreciating the rewards of behaving in a certain way enlisting social support for engaging in a behavior and substituting bad behaviors for good behaviors and these are more associated with the later stages.

So how can we use this model to help people change? Well first, we want to stage people. We ask specifically what they did, okay? And we write it down and we walk through the process with the person. We’re going to define the behavior specifically and together with the individual, you’re going to compare what they do with what the experts say they should be doing. And then help them stage themselves on the behavior that you’re talking about. You can engage in a decisional balance exercise. Find the salient pro. Give them a list of benefits of engaging in a behavior and challenge the person to see whether or not a benefit pays off for them. And then, look at the cons honestly. Which are excuses for not engaging in a behavior and which can be overcomed by a salient pro. Which are real barriers? There’s a lot of institutional barriers to engaging in a behavior and that involves an entirely different kind of intervention where you’re going to have to intervene at the system level or institutional level and it doesn’t make any difference how much somebody wants to do something if there’s a lot of institutional barriers. Self-efficacy exercises are a lot of times for relapse prevention.

And it’s a good idea to talk about set challenging situations and what might keep them from engaging in a behavior and try to come up with strategies ahead of time to help them deal with these situations before they come up. And then go through the process exercises for the early stages, the people who are pre-contemplators and contemplators. And these are the five processes of change. And you can actually use these processes of change to design your educational intervention. And then for the process exercise for people in preparation, action and maintenance are more in terms of setting a commitment to change and substituting behaviors and getting social support. So this is an example of a survey that we used for a study at Washington University and we were trying to increase medical students’ use of universal precautions in the mid-1990s. And so we gave students in the experimental group a survey before the educational intervention and then a year later. And the educational intervention was administered during the clerkship orientation, before their third year clerkships began. And we actually taught them how to safely perform procedures that are associated with needle stick injuries. And we showed them how to wear double gloves properly, protective eyewear, and how to dispose of sharp instruments appropriately. Now, we also included a control group of students that we surveyed at the same time.

But these were a group of students who had finished their third year clerkships at their baseline measure. And then we surveyed them again one year later after they had finished their second year of clerkships before graduation from medical school. And what you can see here, this is a graph of the stages of readiness to comply with double gloving after one year of clinical training, okay? So that is the follow up measure for the experimental group but the baseline measure for the control group of students. And you can see the control group of students, which is the dotted line, there are many more pre-contemplators in that group at their baseline measure and there are far fewer students who are in maintenance at their baseline measure. This is a graph of their stages of readiness to comply with double gloving at the one-year follow up. So now this is after the experimental group had one year of clinical clerkship training and the control group had had two years of clinical clerkship training. And you can see still that the control groups there are many more pre-contemplators in the control group. But after this year there were more students in the control group who were in action and in maintenance even though the difference between the experimental and the control group was statistically significant. And the reason for this is because we also tried to address institutional barriers at the same time and we delivered a hospital-wide intervention at the same time.

We strategically placed posters and reminders in the operating room and in the emergency department to remind people to use double gloves and wear protective eyewear. So we know that these students in the control group were exposed to some of the hospital-wide intervention. Interestingly also, we’ve also observed actual behavior in the operating room. And there was an increase in the use of double gloves among the students during their surgery clerkships in the experimental group. So we validated our stages of change measures with the actual engagement in the behavior of interest. And this slide then shows also that progress at even one stage--it might have been more than one stage, students in the experimental group were much more likely to progress through the stages than the control group. And the control group were much more likely to relapse overtime.

UNKNOWN SPEAKER: Excuse me. I’m sorry if I missed this but what was the prevention for the experimental group?

DONNA JEFFE: The experimental group, we delivered lecture and we showed them how to perform procedures like drawing blood and IV insertions, things like that. And we also showed them how to properly wear double gloves and wear protective eyewear and dispose of sharps. So, they not only had a didactic intervention but they had hands-on training. Okay?

Okay, now I’d like to switch to our EMSC targeted issues grant. We’re doing a Pediatric Emergency Medicine Fellows conference. David Jaffe of Washington University is the PI and Jane Nap at Children’s Mercy Hospital in Kansas City is the co-PI and me. And David and I are not related. And we always have to say that even though I like him a lot, okay. Excuse me. Let me get a drink of water.

The objectives of our grant are to train Pediatric Emergency Medicine fellows in areas of identified need that will give them the knowledge, attitudes and skills to further their careers and foster scholarship, leadership and partnership in EMSC. And we’re doing this by planning and implementing an annual conference for three years. We’re developing annual conference curricula and we’re evaluating the conference. In planning the curriculum, we are basing all the content on the EMSC five-year plan and looking at the major themes of scholarship, leadership and partnership. So, we went to promote knowledge about and interest in research among Pediatric Emergency Medicine physicians, prepare trainees for leadership roles in EMSC and present trainees with possible opportunities for partnering with state and local and community groups to improve injury prevention and emergency services. And at all times during the planning of the conference, we asked ourselves how are we going to evaluate this component of the curriculum. So, normally evaluations of educational programs and curricula use tests of knowledge or questionnaires that measure attitudes or confidence and also direct observations of behaviors.

Some outcomes cannot be measured directly especially for something like what we’re doing for a conference where you’re going to measure pre and post conference knowledge, attitudes and behaviors. So, we had a lot of behaviors that we’re going to try to measure over the long term but we couldn’t do it initially right at the end of conference so we are using the conceptual framework of Fishbein and Ajzen’s theory of reasoned action which is now also called the theory of planned behavior. And the theory of reasoned action distinguishes between beliefs, subjective norms, attitudes, intentions and behaviors. The important thing about each of these is to look at the relationship between each of these variable. This is a very simple schema of the theory. And you can see the relationship between information, knowledge and beliefs, which influence attitudes, which in turn influences intentions and then, intentions influences behavior. And there’s a feedback mechanism, where behavior then will influence people’s knowledge or beliefs and this is also called learning. And their reaction here is that there’s also a feedback to attitudes, when you engage in behaviors it might have an impact on what you’re attitudes are toward that behavior.

Beliefs are the fundamental building blocks of the theory of recent action. And as I said before you learn or form beliefs by direct observations of behaviors, receiving information or making inferences. And the totality of beliefs is the informational base that ultimately determines a person’s attitudes, intentions and engagement and behavior. Most people hold both positive and negative beliefs about a behavior and so therefore people’s attitudes are based on all of the people, all of a person’s belief about an object or behavior.

Attitudes correspond to the total affect associated with their beliefs. And attitudes are related to people’s intentions to engage in specific behavior. Subjective norms are beliefs of a more normative nature and this component takes into consideration the influence that the social environment has on a person’s beliefs. It is actually when other people think that you should do and mainly people who are important to you because we don’t always care about what everybody thinks that we should do. But people who are important to us, we do care about what they think and so this constitutes the subjective norm component of this model. It’s a major determinant of intention to perform a behavior just like attitudes are. And this is a more complex model of the behavior where you can see that beliefs are associated with the attitudes on the top of the model and normative beliefs about a behavior are associated with the subjective norm component. Now both attitudes and the subjective norms are associated with someone’s intention to perform the behavior and in this model they take into consideration the relative importance of what your own personal attitude is toward a behavior and what the subjective norm is. So, maybe the subjective norm component will weigh more heavily than your own attitude toward the behavior. And then intention is directly related to the engagement in a behavior.

So, as I said, attitudes are related to intentions, subjective norms are related to intentions and then intention is related to the specific behavior. Intention is the immediate determinant of behavior and it is the best predictor of a behavior. So, in our grant we had several behavioral objectives, membership in professional societies, developing mentoring relationships, submitting grant applications in EMSC, conducting your own research in EMSC, collaborating with other trainees or mentors and publications. And some of these we obviously cannot measure right after the conference so we measure intentions.

Some results, well for the 2005 Fellows Conference there were 92 fellows that attended the conference and these represented almost 30 percent of all pediatric emergency medicine fellows in the United States. We ask them to complete questionnaires both before the conference and after the conference and had very good response rates and 88 percent of the attendees completed both the pre- and post-conference questionnaires. And then proportions of first, second and third year fellows are shown at the bottom of the slide. Most of the attendees were first and second year fellows.

Each speaker was asked to provide two objectives and two knowledge items that were formatted in a true-false format. So, these are session specific knowledge items and they pertained only to that speaker’s talk. And we summed the correct answers for all the items and they could have a total score ranging from zero to 24. So, now these knowledge questions pertained to the 2005 conference and they’re not the same knowledge question for the 2006 conference because those had completely different set of knowledge questions. What we observed at the end of the conference that there was a significant increase in their conference specific knowledge at the end of the conference. But there was no difference between the groups depending on their fellowship year but just an increase in knowledge.

UNKOWN SPEAKER: (Inaudible) questions exactly a number of preimposed test?

DONNA JEFFE: Yes. Yes.

UNKOWN SPEAKER: And what was the maximum score of the (inaudible)?

DONNA JEFFE: Twenty-four. That’s it. Twenty-four. Okay? Okay. Now, we also measured on these surveys: confidence and subjective norms. We used five-point scales and this range from none at all to extremely. And there were 15 items that measured confidence and subjective norms; ten items for confidence and five items for subjective norms. We ran a factor--principle components factor analysis and thankfully two factors emerged from the data. So, it validated our--the face validity at least was substantiated by the factor analysis with very high internal consistency for each of the items on these measures. So, higher scores on confidence means they’re more confident and subjective norm meant greater agreement with institutional support of research at their institution and the importance of research at their institution. And for confidence we saw that there was an increase from pre-deposed conference in the confidence scores and these confidence items were about making presentations, conducting research and accepting future leadership positions in EMSC. And interestingly, there was also a difference between groups. So, third year fellows tended to be more confident than second year fellows and first year fellows. But we do not see nor did we expect to see a change in the subjective norm component because we didn’t intervene on the institutional support. And so, we didn’t see any change over time in that measure. This just gives you an example of the questionnaire and how we measured intentions to change.

There were seven intention items on the questionnaire. These are just three: intention to continue in EMSC research beyond my fellowship, intention to submit a grant application in the next three years and intention to participate national collaborative EMSC in research networks in the next three years. Now, when you’re talking about intention the closer you are to--when you want them to engage in the behavior, the better predictor your intention will be. Three years is a long time but three years was really the soonest that we could expect some of these people to engage in some of these behaviors. So, we had to be realistic in our expectations in terms of asking the fellows about their participation in research or submitting a grant proposal. So, we used three years for the measure of intention.

And I’d also like to draw your intention to the I.D. number at the top of the page in case you’re ever trying to do a preimposed evaluations and you want to keep individual person’s identities to link--you want to link individual people. But you are not going to include their names or another identifying number where you know who the person is. You can use something like this, which will allow you to link preimposed data but it will keep the person anonymous. So, we just use the day of birth, the first three letters of the mother’s maiden name, and the last four digits of their social security number. And pretty much everyone has no problem with giving us this information because we have no way of finding out who they are.

UNKNOWN SPEAKER: Did you combine like the three things in one I.D. number?

DONNA JEFFE: Yeah. And then in that way, it allowed us to look at the pre and the post conference data.

UNKNOWN SPEAKER: Did you ever not get to match on that or they were never (inaudible) or they were good about remembering the reproduced stuff? (Inaudible).

DONNA JEFFE: Well, these things are picked because they don’t change. Sometimes people will pick like beeper numbers or phone numbers especially in an educational setting. If you’re doing this with students in schools, phone numbers change or phone numbers get turned off so there’s lot of things. The social security number, these kids don’t know what that is. We use something else.

UNKNOWN SPEAKER: Why aren’t the answers to these questions yes or no?

DONNA JEFFE: Why aren’t they?

UNKNOWN SPEAKER: Yes. Because like it seems to me those are yes or no questions.

DONNA JEFFE: Because we wanted to measure the strength of their intention.

UNKNOWN SPEAKER: Okay. (Inaudible)

DONNA JEFFE: So, that way you get some variation.

UNKNOWN SPEAKER: I mean you can reduce this to yes and no but if you get down towards yes and no, you can’t (inaudible)

DONNA JEFFE: You can’t look at correlations.

UNKNOWN SPEAKER: But it just seems like those--and I’m just trying to put all of this theoretical framework that your presenting and in my head I can’t go further.

DONNA JEFFE: Well, it’s a lot of information.

UNKNOWN SPEAKER: (Inaudible) I intend to do something that’s either yes or no so you put an (inaudible) so you could do some statistics (inaudible)

DONNA JEFFE: It’s very commonly used as a scale and you could do yes or no but you know.

UNKNOWN SPEAKER: Okay.

DONNA JEFFE: Sure.

UNKNOWN SPEAKER: I think frequently what happens is people eventually can kind of (inaudible) for their analysis anyways. So, I think the question is, where is the cut-off point that you guys did to the economy.

DONNA JEFFE: Well then, you have to look at your data.

UNKNOWN SPEAKER: Right. And that’s why having more data is better equivalent than having less.

UNKNOWN SPEAKER: I guess if I intend to do something it’s either a yes or a no.

UNKNOWN SPEAKER: Well, what happens when they go, “Ahh?”

DONNA JEFFE: Yeah. It depends.

UNKNOWN SPEAKER: It’s not anybody who is extremely or not all with yourself.

DONNA JEFFE: Right. Yeah.

UNKNOWN SPEAKER: Some of us (inaudible), I’m sorry my (inaudible). My level of creation is put (inaudible).

UNKNOWN SPEAKER: I know (inaudible)

UNKNOWN SPEAKER: Because it can change. (Inaudible) intention.

DONNA JEFFE: It can change. Right. And this is the same thing with like the stages of change. You might be at one stage today and then something happens and you go back. Well, the same thing could happen with intentions.

UNKNOWN SPEAKER: I think basically, what you’re saying is--being very important for most of us who are doing--about the education in our attempt to find changed behavior. Now, my question to you is if I am doing an intervention to change people behavior, is it adequate enough to come at the end and measure their intention and say, “Okay, well, here’s the group that had the intervention. Here’s the group that didn’t and look at the difference in their intention over the next X number of years or whatever. Or should I be looking at more of concrete measures that actually give me whether that intervention have resulted in a change of behavior or not?”

DONNA JEFFE: Well, you’re getting ahead a little bit but you’re absolutely right. And the thing about both of these models is that they’re based on years and years and years of research and they have validated. So, their stages have been validated against actual engagement in behavior and intentions have been validated against actual engagement in behavior in the future. And so my purpose here really is to talk to you about using these models because especially for us at the end of the conference we can’t see if they have more publications at the end of the conference because they won’t, I mean it’s just a few days.

UNKNOWN SPEAKER: But, are you going to spend any (inaudible)?

DONNA JEFFE: Yes, yes. And we are collecting--we are developing a database to collect those actual behaviors where we can, okay.

UNKNOWN SPEAKER: One more?

DONNA JEFFE: Go ahead.

UNKNOWN SPEAKER: Well, about the scale, I’m seeing more coverage on the right side of the scale for those who have the intention and those who don’t or are not sure yet. I don’t see much representation there. For example, the pre-contemplation or contemplation stages of someone who doesn’t know yet whether yes or no; he can’t decide which way of the fence they’re going to jump. I mean the scale doesn’t (inaudible)--

DONNA JEFFE: So--

UNKNOWN SPEAKER: If they were somewhere in the negative side, with one or two options and then a new (inaudible). And then one or two options in the positive. What will give you more--?

DONNA JEFFE: Well, I think that they--using the scale, we try to do that. So you’re saying not at all to somewhat is not only--

UNKNOWN SPEAKER: Yeah, shortly from no to--then gradually he has--looks more in it. Yeah. Looks like more leading to the positive.

UNKNOWN SPEAKER: Right.

DONNA JEFFE: Well--okay. And I don’t know what--the distribution of the data were off the top of my head to see how the data were distributed.

UNKNOWN SPEAKER: I mean, if someone who didn’t make up their mind as not at all who says, “Well, maybe it is (inaudible) period of three years, it’s not going to give us (inaudible). I don’t know what I’m getting for dinner tonight.” So, some people may not have that decision ready.

DONNA JEFFE: Yeah. Okay.

DAVE: I know you’re going to the next level, but I think one of the (inaudible) answers about scaling is that there is very practical answer that allows you to have room in finding relations (inaudible). And I think for the very practical measure of point of view by giving people scale and allowing them to spread their opinions or range of strength of that opinion about their intentions. It allows you to measure changes that you might not be able to measure with one dichotomous yes or no (inaudible). And so, I think it’s a very practical (inaudible)--I think you’ve seen me included that in the next slide (inaudible).

DONNA JEFFE: Yeah, we’ll never going to get there. Okay. As I said there were seven intentions, and the two intentions changed significantly. And that was the intention to establish national mentoring relationships and intention to continue in EMSC research after their fellowship. And I want to point out that these two were a focus of the conference. So, we really focused on these. And the fellows--so, there were 92 fellows but there is over 20 faculty people at these conferences and there’s a lot of opportunities for the fellows to interact one on one with the faculty at the conference in this really--it helps tremendously. We eat together and we have different sessions that they can go to and meet with after dinner and so--I’m not surprised to see that that first one changed and the continuing in EMSC research--this is a major focus of the conference because this was--one of the goals is to increase interest in research in EMSC. And then there were some marginal differences, improvements and intentions for participating in national collaborations, submitting grant applications in the next three years and assuming leadership roles in EMSC. And then two did not change significantly and these were two partnering intentions and partnering was not really a focus of the 2005 conference but it was more--excuse me, a focus of the 2006 conference. So we’ll see if we’ve had an impact for the 2006 conference.

This just shows how confidence and subjective norms are related to each of the behavioral intentions and they are very highly associated with the intentions. And then we conducted a regression of intentions to continue in the research. And we controlled for their pre-conference intention because we knew that pre-conference intentions was going to correlate with post-conference intentions. And then we entered confidence, knowledge, subjective norms and the fellowship year in as a block in the next step. And the only other variable that was associated with the post-conference intention to continue in research past your fellowship was post-conference confidence. We looked at joining organizations. Eighty-four percent of the fellows had belonged to at least one organization that is associated with pediatric emergency medicine at the beginning of the conference. And then 55 percent had said that they intended to join one or more conferences before the conference and that jumped to 72 percent at the end of the conference. And then 48 percent of the fellows reported that they had established national professional mentoring relationships with faculty at the conference, which was really a great outcome. And 25 percent had already established plans to collaborate with other attendees on research.

Now, limitations--there is a tremendous self-selection bias in our sample. So, although we included about 30 percent of all pediatric emergency medicine fellows attended the conference, these are a highly self-selected group of fellows because they were interested in research and they were interested in the goals of the conference. We used the intentions as the best proxy measure of behavior. And then this was also a single conference program, first in the series of three, and most fellows are not going to be able to attend all three conferences so they’re not going to get all of the content that we’re going to plan.

So, we saw that the 2005 Pediatric Emergency Medicine Fellows’ Conference was associated with knowledge, confidence and intentions to continue in EMSC research, join collaborative research networks, submit grant applications, establish national mentoring relationships and assume leadership roles in EMSC.

I have some readings. Did you get handouts? Yes. Okay, good.

Now, there are some similarities and differences in the two models that I’ve presented to you today. Both of the models have multi-components and while the components are not exactly the same, they are multi-faceted so they’re complicated. And these stages of change and intentions to change are two good proxy measures of behavior change that you can use when you cannot measure the outcome of interest in the timeframe that you want to measure it. Both of these models are very behavior-specific. And greater motivation to engage in target behaviors corresponds to later stages in the trans-theoretical model and having greater intention to engage in a behavior according to the theory of recent action. But there are some differences as well. Staging is categorical and the intention is usually measured as a continuous variable. So, we’re looking at the strength of their intention to engage in a specific behavior. And what you want to consider is to choose a model according to the feasibility of use and the research objective.

So, for the trans-theoretical model, you might want to develop tailored interventions according to the person’s stage and measure a change in stage because a change in stage itself is associated with actual engagement in the behavior. We have lots of data to show that. And then for the theory of recent action, you want to describe the relationships among the variables in the model and you can measure change in intervention over time. In the remaining amount of time, in addition to answering questions, you might want to consider for your own questions: what behaviors do you want to change, what are you thinking about behaviorally in your research, how might you consider principles of adult learning in the design of your intervention, and how might you employ the trans-theoretical model or the theory of recent action in your research. And thank you very much. And any other questions?