sos-transcript_working-group-on-integrating-technology-10-20-2017

1 Working Group: Integrating Technology October 20, 2017 DR FARRIS: So, we are missing our two healthcare perspectives here, but I think there are other healthcare perspectives in the room, so I think that we can make sure that is reflected In my view, the purpose of this discussion is for us to be able to provide some recommendations moving forward about how we can improve integration of mobile health into I'm going to call it, systems So, it might be the EMR It might be social service systems It might be educational systems So, while we didn't present an educational system perspective this morning, we had a question about it Paula is here, who can, I think, help us with that view, as well So, if you guys are comfortable with that question, then what I would like to pose is, from your perspective, given what we presented this morning, what are those next steps in important research questions or best practices questions from your experience, from your view? Everyone understand the question? DR FAIRMAN: I can get it started One of the things that we looked at doing - because I did have the opportunity to look at the commercialization of technology, and understanding who the stakeholders are and who would potentially be buying this One of the biggest things is, well, how much does this cost? So, there's the cost of the technology, itself, but there is the cost of using the technology And part of that is understanding how much time is going to need to go into the workflow, right? So, ideally, we're saying that we're saving time, right, but you're going to have to spend some time using the technology as well And so, workflow analysis needs to be a big part of that Workflow analysis is tied to what the cost is of personnel to integrate the technology into their day And whether that personnel is a teacher, whether that personnel is a physician, a case manager, whoever it might be, that has a dollar amount that goes along with it So, we have to be prepared to answer those questions DR SMITH: The workflow is really important I know at the University of Michigan, we were trying to implement telemedicine in the physician activities, and they have their existing workflow So, we always tried to figure out, okay, how can we fit this in Initially, telemedicine tended to take a lot more time because they're getting used to it And then later on, I know with tele-epilepsy consultations, they were able to be more efficient, you know, and they were able to actually save time So, ultimately, it comes down to time and money and work If it's more work, they're not going to want to embrace it DR KOHLER: I was going to point out just a challenge Because I really appreciated Sarah's comment when Seth said, well, how about getting this in health classes How we move these things into everyday practice and education? And so, there, in working with NIDRR and NIDILRR, then I'm sure you appreciate the challenge because we are funded to research, right? Now, they have this technology transfer piece that they have added to it, which is fantastic, but there's an issue that we continue to deal with funded by the U.S Department of Education, Rehabilitation Services Administration, Office of Special Education Programs, as well as IES Back in the administration before the Obama administration, there was a big push on reading Of course, the No Child Left Behind Act and all of that kind of stuff came about So, the administration was working with some reading researchers, and they funded all these grants to improve reading in various contexts The problem was, to get the grant, you had to use this particular reading - commercially available reading curriculum That was a huge conflict of interest These were the people driving the decisions by the administration, funding was attached to it, and there was a huge blowup So, one of the things that we're charged - every time I go to the feds to work on something, we sign - when we are doing research, we sign that we are not proposing any particular commercially available product And then, our other challenge is when we put things on our website and we roll out to this educational community at the state and local level, then, again, sponsoring commercially available things without the evidence of working, like that IES proposes, Institute for Educational Science – so, that's a challenge So, I think when I worked - I was associate vice president for research at Western Michigan University before I took this current position and we funded internally the work of faculty Most universities this So, how you evaluate the proposals that come from engineering, which they wanted to know whether this works or not? That's what their project was We're developing this and we want to know whether or not this works Researchers were proposing a study and theirs was all based on methodology So, was your methodology going to be sound and how were you going to control your variables and all this other stuff So, those are the two worlds that we're talking about here, but they're in this context of moving things into practice within the public or policy arena, commercial We can't support commercial things So, that's our challenge as we talk about moving these into various areas, I think Or that's one of our challenges It’s not the only one Educating educators to use the stuff is another huge challenge DR FARRIS: So, we think that NIDILRR does a good, acceptable, whatever the right word is, in terms of at least incorporating this notion of tech transfer, allowing us to test commercially available products? Do I hear that? DR KOHLER: It's a great that it is there now because that's an improvement, definitely DR FARRIS: Are you advocating that on occasion we need to be testing commercially available products, I guess? That is a question for the audience DR KOHLER: That doesn't necessarily mean randomized control trials because you are never going to get that very often in educational settings because it's just about impossible DR FAIRMAN: I think the best-case scenario is a partnership with industry So, when it's only coming from the academic side or only coming from the industry side, the conflict of interest often gets in the way So, when you have a partnership, you can have the researcher on the academic side ensure that there's no sort of moving the technology forward without evidence, right, and at the same time, you still have stakeholders who are able to kind of protect and also make sure that there's a place for this in the market So, if you have that partnership, that's the best possible scenario It's not always easy to achieve, but that’s really what is necessary because there are often barriers in between academics being able to move into commercialization, which is what we spent the last hour or so with the last panel talking about DR KOHLER: Another issue is, as Seth said, there's individual education programs So, get it in the IEP Assistive technology is a huge place within IEPs But then the question always comes up, who is going to pay for it? So many times, it's putting as a burden on the parents When you talk about working with parents and then their children, then that's hugely important because the parents have got to bring the need for this assistive technology if no one else is doing that, into this IEP context At the same time, there has to be - the burden of cost, if we are talking about things that aren’t 99 cent apps, need to be shared or paid by insurance or whomever Because it can't be just placed on the parents DR LORD: I just wanted to add to the question about testing, testing apps, I think, and getting back to sort of are you testing to see if it works or are you testing, which I think is more helpful, is testing use cases - testing in terms of implementation science and testing using sort of principles of implementation science, so that you really can understand when you introduce something, like you - take a couple use cases where you introduce some assistive technology into an IEP, and then really study it with the facilitators and the barriers so that you can come up with some guidelines and some standardization around implementation That can then help sort of create a foundation Broaden the implementation that ultimately gets to policy folks Hopefully, you're then able to sort of see more of a system-level impact But you need to be able to measure it in order to be able to then demonstrate it DR KOHLER: Our whole center is based on the concept of implementation science It is really important Yes DR NEWMAN: Would you guys be able to explain what implementation science is? Maybe everybody else knows I’m not familiar with that term DR LORD: It's really sort of the, you know, taking a scientific approach, if you will, or a more rigorous approach to implementation, to studying how an intervention or a technology, in this case, actually is integrated and used in a system of care And looking at sort of different, you know thinking about barriers and facilitators like what helps and what gets in the way, from a different sort of level of analysis So, what is it about the intervention that maybe it's not a good fit for your client population? Maybe it's not a good fit for your educators Maybe the attitudes of your educators is what is getting in the way of the system being used? Maybe it's an organizational thing, there's no leadership support And so on, maybe there's no money to pay for it So, it's really just taking a rigorous sort of scientific approach to studying implementation DR KOHLER: Yes, so we intervention research, which everybody here is familiar with So, then the whole concept of implementation research is how you move these scientifically proven interventions into practice? With our center, for example, we work with state departments of education And so, you look at things like doing an infrastructure analysis What is in place? What policies are in place? What the data say about this particular context? And believe me, the state of Michigan differs very differently from Delaware from California So, understanding those contexts Then things like readiness, resources So, if you're going to implement say a full out technology adaptation within a state, focus on a particular - you know, this intervention that we have proven works, then there are a number of variables that help you understand what you have to to move that implementation forward And there are some folks at University of North Carolina, Dean Fixsen's group, which have done a lot of work on understanding these various variables We have our own model, but we have shown over and over again that it works It is not the Fixsen model, but it is implementation science DR FARRIS: Mark, we have folks in family medicine and in the VA at the U of M who are quite accomplished in this area DR LORD: Laura Damschroeder is the one 10 DR FARRIS: Yes She is the woman She is the one DR NEWMAN: I am familiar with all of the concepts I just hadn’t heard that label before In software engineering, all of those things are done In business IT, they all have different names for it DR LORD: It is sort of implicit in your discipline, right? DR NEWMAN: Well, no Not really But yeah, it is sort of a different – when you spell it out, I’m like, oh, yeah, okay, I can see what that level of analysis is DR FARRIS: It is just getting different views around the table, right? And I think the concepts are the same, but the labels may be different DR NEWMAN: I like it because it has science We say implementation all the time Usually, science isn’t part of it DR LORD: That’s why it is really important to kind of shift the model So, I really appreciated the earlier comments about shifting a model from sort of a medical model, and it's the same in my field I'm a psychologist When you think about intervention development and sort of the treatment development, it would go from kind of your basic science and move through 25 EPIC go back and forth – within and coming out of EPIC for research purposes or whatever, people will tell you it can take years to get that implemented DR NEWMAN: I have had the pleasure of helping with a project that is not directly related to my research, but it’s an EPIC integration It is literally – the technical difficulty is the kind of thing you could in a weekend It mean it’s like getting some data out of EPIC – and it has been two years DR FARRIS: It’s the policies DR NEWMAN: It is the policies It is getting the IT – there are like three different subcomponents of the IT people that need to be on the same page They refuse to speak normal English, so I can’t really communicate with them If I say something the wrong way, it takes like three weeks It’s unbelievable I did want to say – so, on the one hand, I absolutely hear what you're saying There are these massive institutional barriers I think it might be a little bit dangerous to overidealize normal software development and recognize that there is – I mean, for one thing, researchers in computer science and things like that are not pushing their - I mean, they could get their stuff out to like a handful of users, but they are not - 26 most of them aren't like launching their software to lots of people and supporting it for lots of time That is a hard thing no matter what And so, it's, you know, to idealize – we talked about the Facebook example I mean, yeah, like Facebook, it's very easy for them to innovate and put new stuff out Part of it is because they're not regulated and a lot of people think they should be regulated more than they are There's becoming more and more evidence that letting Facebook just whatever the hell it wants is kind of a problem for lots of things And I think if you look at like software development - people that work in like aeronautics or automotive industries that are much more highly regulated, it takes a pretty long time to get innovations into those systems I think just having a little bit more of a nuance, because it might help us understand what exactly we can in - I know I'm very new to all of this I'm just shocked at everything I see about like health technology and EHRs and how the torpor is amazing, the sluggishness It's amazing It's fascinating to me It is really fascinating because coming from the mobile like let's launch an app It's like really, really different I 27 still don't understand all the ways in which it's different But I kind of think we need – personally, I'm trying to understand in a much more nuanced like, okay, what is it and where are the avenues where we could actually - where could we actually get some flexibility into the system rather than sort of saying, well, it's so much better over on the other – you know, the green pastures of Silicon Valley or something like that? There are some nice things about that, but it's not uniform across all industries and application areas DR MANN: EPIC and any vendor - the reason it has taken two years is your institution It is not EPIC, fundamentally The reality is, you have to be a bit like a mosquito and just keep looking for the hole in the mesh to get through because these institutions grew up with this very torpor-ous approach I don’t know the right word So, yeah, there is like multiple level systems and stuff and so, you are disrupting it But there is usually someone who will let you it That's happening everywhere You will see some places are further along that disruption and some are less so The vendors are really responding to it So, with EPIC’s App Orchard that just got released last month, 28 these vendors are starting to open up their gates a little bit, and it's really on us, now, as the institutions, as the investigators, to not have given up from maybe spending the past ten years being frustrated and still kind of throw ourselves into the breach again I think, you know, hopefully, those times will take less It won’t take two or three years It will be maybe six to twelve months You’re right, the auto industry and aerospace is a good example It is a regulated environment You want to not kill people It will be slower than it would be to throw up another food delivery service app, fine, but it will be faster than the 20 years it took to put EHRs in, let’s hope DR DE RUYTER: I want to echo what Devin said with EHRs being in the medical space But I think part of the problem, drilling down a little bit, does come from the C-suite, in terms of releasing or giving access to the data A lot of the data is there It is that I think there is concern in terms of how it's going to be used, and different ways that it might be used within an organization that further confounds the interpretation of that data DR FARRIS: Do you think that’s legal? 29 DR DE RUYTER: Sure it is The hospital owns the record They can what they want with it DR FARRIS: It is primarily legal DR MANN: She means the concerns DR FARRIS: Yes, the concerns Apologize The Csuite concerns CYA? DR DE RUYTER: I'm not sure how to answer that I have my own opinion, but I'm not sure I think the other issue that we haven't talked about is the flip side of that, and that's with the data going into the portal There is tons of data going into the portal, but that data never goes into the EHR or rarely goes into the EHR You have to ask the question why is that? If you're a physician and you're seeing patients every 15 minutes, you don't have time to look at 150, 200 inputs of data into a portal of different patients There's also the risk of malpractice Data may be going in there and if you miss something, then there is the risk of that coming back at you That's an issue that I think needs to be addressed DR FARRIS: I know John wants to add DR BUTZER: A couple of comments One, the usability I can't emphasize that too much I mean, it needs to be front and center The electronic record, there 30 are - we work around that and in selected areas, we can present data outside It's a nightmare with the Excel spreadsheets and all of the things, but if you get something really important, you can access the medical assistant or the nurse or whoever is putting the patient in the room You can a supplement sort of thing And it's actually not too bad because if it's in there with the record, the patient – you know, they will put a cover on it and the patient will always look at it and then they're sharing the data and it's kind of an engagement kind of thing as a practical level So, that's good I would caution, moving to a different topic, several times people say there's all that data in there A couple of comments on the data One has to with medical coding And this started with, we need to know what diseases are in the hospital and so we have codes and then you can look at the diseases and look for patterns of care This goes back to the '20s And then the government came in and said we'll pay you by the codes As soon as you tie payment to an activity, you bias the data I think the New England Journal last week or the week before, there was an article 31 about the uselessness of the codes anymore because they're all biased towards how you maximize payment In rehab, the same thing with the FIM measure We've said for years one of the goals of rehab is increasing function FIM is the Functional Independence Measure, and it was designed to measure what people could functionally Medicare now pays you by your FIM data And a recent study showed FIM could be measured at discharge from acute care and there's bias to have that high because we don't want to send anybody home unless they can function or go to the next level, and then it's measured again when you come to acute rehab That's the same day, usually a few hours later, and there was a 20 percent drop because the acute rehab area is paid by the improvement of the FIM So, they have an - and I don't think most people were dishonest They were looking at it within the government rules It's advantageous to us to score it as low as you can so that we can show improvement within the rules It's probably how most of us our income tax You know, what are the rules? We want to minimize it So, I want to be careful to say I'm not - I mean, there are crooks out there, but most people aren't really crooks; they're just bringing this bias together 32 So, these biases are baked into some of this data Just a word of caution when you think if the machine learning people would just analyze the data and tell us what it is, and my – no understanding of machine learning, but the algorithms have to be built on data, the data is constantly renewing So, there's a maintenance and change to all of that and it's not going to be magic You know, and it all is going to start with the data You better understand the limitations and the biases of the data I think, if I'm okay – DR FARRIS: Yes Keep going DR BUTZER: For the future, if you're looking at - and I have total respect for the education and employment and enthusiastic about that, but my comments are directed to health In no way I want to say that's not maybe more important, but just to qualify what I'm talking about If we go back to the value equation, what we in therapy is pretty much what we did in therapy in the 1930s or '40s I know there's a couple of OTs and others here But everybody is respected for their professional opinion and there's very little standardized outcome So, we have wanted to three standardized measures of 33 motility, ten-meter walk, six-minute walk, and there was some talk about that, and then the Berg The Berg is a balance scale and it can predict a lot of things There is a lot of good research about it So, we want to the Berg on every admission Well, the Berg had sub scales So, we go to the medical record people I'm trying to tie this back to the record We talk to our therapists This is science We use the action to whatever model and we translate that They get all excited They're going to measure the Berg Everybody is going to the Berg How we get it in the record? Well, there's this one field and I guess we could spare up for that Well, there's sub scales I want to know what the sub scales – well, we can't put that in there Maybe we could get you one method We’ll petitioner Cerner and then you go all down the line So, then you fight to put it in the medical record or you put it in an Excel spreadsheet and give it to the people in some other way? So, some of the barriers But I think data at every step of the process, if we could have an app that we could put on our patients that would give us a Berg equivalent so that that therapist doesn’t have to spend 20 minutes doing the test 34 before and after So, measurement during the rehab process And then in population health, we've been measuring when they're discharged because that's when we have them Some come back to the clinic; some don't come back to the clinic What population health is going to want to know is what's happening at home I would emphasize, can we get quality of life functional measurements from the home translated into a digestible form in the medical record? That would be the big vision from the practitioner point of view And in no way - obviously employment, huge fan of employment I mean, that's a focus For a long time, and even for us old-timers, even Freud said the purpose of life was love and work So, the concept of work is not brand new but there's new good evidence for that I must have offended somebody DR FARRIS: No, you were profound Any final comments? We may have finished early Seth, based on your discussion yesterday, you think there's anything else we need to - in terms of how you ran yours and getting recommendations out of it Do you have any advice to our panel about how we should have done it or another question we should have posed? 35 DR WARSCHAUSKY: No (Laughter) DR WARSCHAUSKY: We didn't know that we were going to be speaking this morning So, I assume that two things will happen to you You'll be able to distill what was said and there's transcripts to look up any detail DR FARRIS: Any other burning comments that you want to make about mobile health and getting it into various systems? We've certainly heard about data quality, particularly when it's tied to payment We've heard about trying to harness the date that is there, that exists currently to identify some useful activities or useful trends from it What else have we heard? We've heard about barriers to physically getting the data And then we talked quite a bit about this balance between randomized trials, the business interest, and how developing mobile health technologies really need to pull these together so that when we begin development, we're thinking implementation from day one And who is the champion, where is it going to go, how can we get it there, and making sure that we're involving users of many varieties and ensuring accessibility and user experience or usability as we move forward 36 DR MANN: I'll say one last thing My wife is a teacher, so I was thinking the teacher example And it really is the same as a clinical example, which is, I guess my parting chat would be sort of as you're developing and if you really have in your diagram the clinician or teacher in there, then you need to have them in your development team You literally need to put as much energy into understanding what happens when you dump the data on their door as you have in what does it take to get the data from wherever the user is If that's all equal-handed, the teachers will be bought in, the clinicians will be bought in, and it will actually be useful DR RAFI: And one thing to add to what you just put out there about, well, all of those things, is even if you have all the things you just said and the funding and the project is set up so that you get one shot – so, you’ve got everybody in the gang, we're making the thing, and now it's out in the world and it’s done and all the funding is gone and there's no more time to spend on this, then you can never learn So, I think there also needs to be a requiring to pare down – narrow down on just the skeleton of whatever it is, plan to get that out, and then use follow on funding to learn The learning is happening 37 after it's out in the world, not formulate your wonderful idea and get it out and then wonder what happened next DR KOHLER: I was just going to add to John's point about the data and understanding what's there and what's not there because that is a huge issue in education Our center, we analyze all the graduation data, all the drop out data, all the content of the IEP data, and all the post-school outcomes data that is reported by all the states that flows up from all the local districts in the country and territories There's great variation across that Aggregating it and disaggregating it are extremely challenging because of the very things that you pointed out DR LOVE: One last thing I would say I am going to dash out, but I want to make sure that we engage institutional leadership whenever you're designing, too, because I think that if you ensure that buy in prior to deciding you want to wholescale change, I think it's much more easily adaptable Just my experience DR FARRIS: Final comment DR BUTZER: I’d like to make a comment about the C-suite You know, I used to wear an administrative hat as well And I think one of the mistakes that the research 38 community generally makes is that they are waiting for the proof, waiting for the big success, and so on You really need to have an ongoing relationship with the C-suite And you want to get tied in to the highest level in the organization The research department should report to the CEO, and you should have a friend on the board of trustees, if you can swing that And you need to be intentional and smart and political about how you all of that, but it shouldn't be, I'm above that, I'm in research, I'm not tainted by politics and the money and the business You've got to engage DR FAIRMAN: One of the things that Devin said struck me as particularly poignant He mentioned that maybe we get a grant, we say we're going to this, but then we this other project, then we focus on that I think rather than looking at it that way, we could really reach out to people fund us and say these are the things that we need to have in our grants, these are the things that we need to have in our proposals, all these stakeholders are important and that's what's really going to force us to that kind of work If our funding source tells us that this is what it needs to include, then that's what we're going to 39 If not, we're going to a piece of it, right, we’re going to some part We are going to what we can afford to But if all those pieces are in there from our funding source, then it is going to be more realistically implemented DR FARRIS: Okay Thanks, everyone I probably have 15 minutes to summarize, I suspect, what we've covered So, when I try to that, and I haven't done it correctly, just jump up and say, no, she didn't that correctly, here was the real view Okay? That would be great Thank you all (Working Group adjourned.)