Is my child's plan actually being done as written? Treatment integrity, for families (audio overview)

A free audio overview for families, derived from Special Learning's course "Ensuring Treatment Integrity" (presented by Christine Austin and Erin Lombard). It is a synthetic voice, not a person, summarizing one session of the course; it is not the session itself.

Download the MP3

Episode notes

• Free tools and guides, no account needed: special-learning.com/free-resources
• The ABCs of Autism guide, free and written for families: store.special-learning.com/free-abcs-of-autism.html
• The full session, and the rest of the series, is in Special Learning's Video CE Library, written for professionals: special-learning.com/for-behavior-analysts

Transcript

This is a free audio overview from Special Learning. It's a synthetic voice, not a person, and it's summarizing one session of a training course. It isn't the session itself.

The session is called "Ensuring Treatment Integrity." It's taught by Christine Austin and Erin Lombard, two behavior analysts who ran clinical operations and consulting at a center-based autism program. It was recorded for people who supervise the staff working directly with children. But the question underneath it is one families ask all the time, and rarely out loud: is my child's plan actually being done the way it was written?

Here is the session's starting point, and it's blunt. When a child isn't making progress, or a problem behavior isn't coming down, the supervisors say there are two questions to ask before any other. Is the data accurate? And is the plan being carried out the way the staff were trained to carry it out? Until those two are answered, the presenters say, none of the other questions matter, because you'd be reasoning from misinformation.

The first half of the session is about measurement. In this field, the presenters say, data is how every decision gets made. You measure before treatment starts, during it, and afterward, so you can tell whether to continue something or stop it. And the very first thing one presenter does when a graph looks wild, with numbers jumping all over the place, is not to change the plan. It's to check whether the person collecting the numbers is counting what she trained them to count.

The session walks through the ways behavior gets counted, and the reasoning behind each is something a parent can follow. You count how many times something happens when it has a clear start and stop, like a child getting out of a chair. You time how long something lasts when counting makes no sense, like a tantrum that's five minutes on Monday and two hours on Tuesday. One number tells you nothing there. The duration tells you everything. You can time how long it takes a child to begin something after being asked, and the presenters describe a young man who took forty-five minutes to get out of bed, brought down over time to one minute, with nothing but gentle changes to the room, because a physical prompt would have made things worse.

There's a practical thread running through all of it. A method has to be manageable for the person using it, or the numbers won't be real. The presenters describe a child hurting himself two hundred times in a three hour morning and say plainly that nobody could have written that down with a pencil while also keeping him safe. So they use counters, stopwatches, even a strip of masking tape on a leg in a family's home when there was nothing else. And they say the numbers should always come with a time frame. One hundred and fifty times in three hours is a different thing from one hundred and fifty times in a day, and the people reading the report, including parents, deserve to know which it was.

They also describe what a written definition of a behavior looks like when it's done well. Not just "aggression," but exactly what counts and what doesn't, with the ways it usually shows up listed out, so that two different people watching the same moment would write down the same thing. There's a small story here. A child finished a task, got a little reward, sat for a bit, and wandered away from the table. Staff were about to record it as running away. The supervisor's reading was different. He was bored, and nobody had told him what came next. That's not the behavior the plan was written for. The point isn't the label. It's that a definition that's too loose puts noise in the numbers, and then the numbers can't be trusted.

That leads to the idea the presenters call the heart of the session. Two people watch the same child at the same time, each records what they see, and then they compare. The session calls it interobserver agreement. If the two records match, you can trust the definition and trust the data. If they don't, you've found the problem before it found you. The presenters say they do this from a new staff member's first days, and keep doing it, because it's far easier to build a good habit than to undo a bad one.

Then the session turns to the plan itself. It quotes a definition of treatment integrity as the consistent and accurate implementation of the plan as it was designed. And it says something families may find useful to hear from professionals. A beautifully written plan changes nothing if it isn't carried out as written. When a plan looks like it's failing, the presenters' first move is the simplest explanation: go and watch, and find out whether it's being done as designed. Often it's that simple.

The second half is about how staff get trained, and the session is candid about what doesn't work on its own. Handing someone a manual doesn't work on its own. Showing them once doesn't work on its own. Standing beside them and correcting them doesn't work on its own, and can be overwhelming. What the research and their own experience support is a package: written guidance, a demonstration, video, practice with feedback, and then someone watching and checking over time. The presenters describe their own first week for new staff, day by day, and the pattern is the same one used with children. Show it, practice it together, then step back gradually, with someone still checking the data.

Two details from that part are worth repeating. First, the presenters say the plan should travel with the child, so that whoever is with him knows exactly what to do when something happens. Second, they say a supervisor should implement the plan first, with the staff watching, before asking anyone else to. If you're willing to do it, they say, your staff will be willing to do it.

They set a bar for staff, too. Eighty percent accuracy, checked by direct observation, with formal reviews on a schedule and informal looks every week. And they're specific about feedback: begin with praise, keep it about three parts praise to one part correction, and make it about the exact thing the person did.

The session ends on the people doing the work, and it doesn't pretend the work is easy. Staff turnover, injuries, plans that seem to make things worse before they get better, children who have grown bigger than the adults around them. And then a list of ways to keep a team going, one of which is written for families as much as for staff. Ask the people who spend the day with the child what they think, because they are the ones who know the child best. The presenters say that about staff, and they say it about parents in the same breath.

Two things the session includes that this overview leaves out. The presenters describe the software their program uses to organize plans and data, and they offer a trial of it. That was true at the time of recording and isn't part of the ideas here. And the session touches on emergency safety procedures for staff, which is a training matter for professionals and not something a summary should try to explain.

None of this is a way to grade the people working with your child. It's a description of what good practice looks like from the inside, so that when you sit in a meeting and hear the words data, fidelity, or agreement, you know what's being talked about, and what a fair question sounds like. The words for these things differ between countries and services. The idea underneath them doesn't. A plan is only as good as the way it's carried out, and it's reasonable to want to know.

This is general information for families, not clinical advice, and not a treatment recommendation for any individual child.

If you want something to use today rather than something to buy, Special Learning's free tools and guides are at special-learning.com/free-resources, no account needed.

The full session, and the rest of the series, is in Special Learning's Video CE Library; the link is in the episode notes.

This has been a free audio overview from Special Learning.