skip to Main Content

Planning for Inpatient Bed Capacity During COVID 19 Webinar Transcript

SOFIA DEMARCO:. Hey, hi, everyone. Let me start by introducing today’s speaker, who is Mike Draugelis. Mike is the Chief Data. Scientist at Penn Medicine and was formerly a. Senior Strategic partner and Chief Data Scientist at Lockheed Martin. Since then he’s been applying advanced predictive solutions in health as the leader of. Penn Medicine’s Predictive Healthcare team for the past six-plus years now. I’ll introduce myself as well my name is Sofia DeMarco. I’m a Director of Product. Management here at LeanTaaS. I’ve been working in hospital operations for about the past seven years. And we have one more member of the LeanTaaS team with us today. 

PALLABI SANYAL-DEY: Hi,. I’m Pallabi Sanyal-Dey. I’m a physician and I just joined as the Director of Client Services for the best product, UCSF faculty and was Director of. Clinical Operations at San Francisco General for the past two years. 

SOFIA DEMARCO: So we’re going to jump in with a couple of quick introductory questions here, and then the plan is to spend the bulk of today’s webinar actually walking through the model life that Mike’s team has built. So they’ve created an online tool that’s publicly accessible to help model,. COVID-19 demand in hospitals. So, it’s just a few, quick intro questions and then we’ll get into the model itself. To get us started, Mike why did you build this model? Who built it? And who’s using it right now? 

MICHAEL DRAUGELIS:. Yeah, thanks, Sofia. First of all, thanks for having me and I’m happy for the opportunity to reach out and get feedback and ultimately to keep improving how we use these tools and others use tools like these. Yeah, so back in early. March, March maybe 11 or 12 are CMO reached out to us as they had been in the midst of. COVID planning for some time. And asked our data science team to provide some type of forecasting tool that can help balance scenarios for planning, supplies, staff and just a regional collaboration. And my team, the science team, we’ve been around for six years we tend to build predictive models, machine learning, AI models for operations and health systems and we’re not usually called on to create or expand understandings of epidemiology and certainly not in operations. But let me ask this question, there’s a huge history of models in epidemiology to track the spread of epidemics and pandemics. And people are all much more aware of those and that feels now more than we were a couple of months ago. But what we did was take this epidemiological model and work with a open-source community Code for Philly to just in a couple of days deploy this model. 

And we tweaked the inputs to this traditional epidemiology model so that local planners in the region, as well as health systems, could take observation that they’re seeing on the ground and adjust it to what they believe they’re health system will feel. And since we deployed that we’ve gotten a lot of great feedback, it’s been used around the world, by thousands of people and continually getting input, collaboration from others. And there’s other models popping up and we’re kind of connected to those teams and comparing notes. One thing that is unusual for these models in epidemiology they’re usually used retrospectively to try to understand what happened. This is probably the first time these models are used in this way, so broadly to kind of forecast what to expect and to plan around at this kind of scale. 

SOFIA DEMARCO: OK, fantastic and you guys have been tracking the usage on this tool right? What have you seen in terms of people logging in? 

MICHAEL DRAUGELIS: Yeah, no it’s been pretty amazing, overwhelming experience, humbling. We are giving responses from. Portugal and Saudi Arabia and just looking at the Google Analytics, it’s clearly being used pretty strongly over in 20 different countries and definitely thousands of users. Most impactfully for us and our team is reaching out to other states and health systems and getting feedback from them and so it’s been really nice to have at least a normalized tool and a way to talk about the COVID pandemic in a kind of standard way with our other collaborators. Yeah, so we have this slide here, the SIR model. You know there’s been a lot of press on these types of models. The SIR model really is a process model of spread. So meaning that it actually emulates the process of disease spread where you start with a susceptible population. You have a number of– a number of people that are infected, and are infectious and there’s an interaction between these two people with a chance of spread and that played out over time on a daily basis, constitutes the rate of spread in your population and then, of course, that’s kind of balanced by the time or the duration that the person is infectious and then becomes recovered. And the SIR model is simple in that it assumes that every person in your community has an equal chance of meeting any other person in the community and that everyone will be infected at some point. And so those are broad assumptions, that obviously aren’t true, but what we’re searching for is a useful model for planning as opposed to kind of threading the needle. And, you know, the key thing to remember is, unfortunately, there is so much unknown about this because of the lack of surveillance testing. 

People that are asymptomatic or not sick or mild, it’s hard to really understand the doubling rate of infectious spread and then it’s also hard to understand the actual implications of certain social distancing policies. So it’s all to say that when you have large uncertainty in these parameters, a simple model can– when it allows you to quickly enter data can allow you to bound these scenarios. So for us at Penn, we put our optimistic and pessimistic parameters in to kind of bound and we look at a variation of expecting peak demand takeover patients within a month, sometimes within a couple of weeks. But the key is that you’re entering it with your best estimates of these parameters that are unknown and taking in a bounding constraint. And it’s worth saying that there are other models out there. Absolutely and some that are higher fidelity. There’s one well-publicized model from the University of Washington. That’s a different approach. It is a parametric model based on what happened in China. And so the assumption there is that the disease and interaction of people are going to be similar to China and that the social distancing function that China put in place could be replicated here in the United States. 

These are all assumptions that are just that, assumptions and large unknowns. And so for health systems and regional planners what do you do with this large uncertainty? And the best you can hope for right now is just a bound again, bound for scenarios with some relevant models that are validated under their assumptions. And then parameters that you’re always seeking to update and understand and collaborate and share with one another and with that your health system can start making some key decision- 


MICHAEL DRAUGELIS:. Managing the [? costly– ?] 

SOFIA DEMARCO: To find somebody who needs to plan around. COVID-19 demand right now, can I use this model to reliably tell me what’s going to happen two days from now? 

MICHAEL DRAUGELIS: Right. That’s a great question. It’s not something you can use to say we’re going to get 10 new patients tomorrow, five new patients tomorrow. But it is a tool that can give you orders of magnitude. It can tell you the size of the peak need and it can tell you the range in which you will start to feel that. One value for this model is to give you a sense of what exponential growth feels like, it’s not an intuitive thing. So when you know, your health system knows that you’re going to have the tsunami coming towards you, this huge peak that model can tell you that you should you can so expect a few weeks of calm, of normal operations, and so that’s what this model is really intended to do to layer in different assumptions and to give a sense of when and how big that is going to hit your health system.

 SOFIA DEMARCO: OK at this point it would be great for us to get into the model. So for those of you who are signed into the webinar right now you might have seen this link on the page where you entered in. We’ll also be distributing it after the fact to make sure you’ve got access to the model. You can log in using this URL. It’s publicly available, there’s no log-in or download needed to use it at all. And there are descriptions of the parameters right there available on the web page, also tips for estimating the inputs that you’re going to need to provide to it. One other note is the team who built this at Penn Medicine are continuing to re-evaluate the parameters the model uses as new research emerges. 

So they’ve made a few updates even within the past days and week. It’s a team of data scientists and engineers who have continued to look at the newest research that’s coming out right now, and make sure that this model is doing everything that it can to represent the situation accurately. Bear with me for one second. I’m going to share my screen. I had a little hiccup with that earlier in the presentation today. I’m going to go ahead and share and move into the model itself at this website. 

MICHAEL DRAUGELIS: That’s the challenges of a home office that we’re all experiencing, I think.

 [SOFIA LAUGHS] SOFIA DEMARCO:. OK, go ahead Mike. Take us through the model here. What are we seeing? And how do we think about the parameters outlined on the left? 

MICHAEL DRAUGELIS:. Yeah, that’s great. Can we just orient you to, to what I’m seeing? So when you log onto the tool you’ll see a main splash page. It just has a title of the tool [? chime, ?] and it gets right into a description of a scenario that it’s attempting to run, and it’s going to express how many patients, sorry, how many people it thinks are infected in your region and then it’s going to define how many patients are hospitalized, based on the hospitalization rate and regional account of population size of your region. And then that’s all connected to the market share. So it’s trying to be really explicit about how it’s layering in its assumptions. 

Let me back into that for a second. So one piece that we had to accommodate for our planning was to use these general population models and to kind of hone in what that– what we think that means for our health system. So the way that we did that if– I’m now panning over to the left panel. You would enter there if you’re a health system or if even if you’re a state, you put in the region or your market region that you think you’re your health system services. 

And so for Pennsylvania,. Penn Medicine health system, we have [? four-county ?] region for our three downtown hospitals. So that population if you just use US census for those counties comes up to 3.6 million. And then you can do analysis based on bed count of your hospitals against the total regions or county bed count to arrive at those market share. We looked at two methods of doing that, we got a number from our finance teams when they think about market share and then we just [? add ?] that count. And so far two downtown hospitals they came off to be 15-18%. 

And then when you enter in, every day you have a current set of positive COVID patients that are hospitalized, that could be in the ICU, vented or in the ward. That number of currently hospitalized patients is the number that you would enter into that box and the reason that that’s so important is the lack of true surveillance testing of our communities means that we don’t really have a good estimate of how many people are truly infected. And when you have a SIR model that’s a critical number to have, because that’s the driving force in that curve to know how many people are infected, and that number can be comprised of infected people that are asymptomatic or mild to severe and critical so it’s really everybody. So the way that we back into that number of total infected as you take the current number of hospitalizations and we do analysis based on publications as to how– what is the rate of infected people that get hospitalized at any current moment? and so our best estimate of that is 2-2.5%. 

And then you say,. OK, well, now I understand how many patients are infected based on the current number of hospitalized patients, but you have to scale that by your market share and that’s at 15%. So by just doing the arithmetic on those parameters, you can back into the total number of infected patients. And that’s a key initial condition in the model that you need to put in aside from the original population. One thing also to note, when you put in the original population that’s the initial condition of your susceptible population. So with those two pieces of information that we have are susceptible and infected, and the model is kind of ready to start stepping through that timescale [? 10 ?] series. So if we scroll down a little bit more on the parameters, you’ll notice that we had this category spread and contact parameters and what– you have two options here. We provide an option to fit your observed data to derive the doubling time. The doubling time is the observed time it takes to double the number of infections of your population. 

And so as you move forward in the spread, doubling time has less meaning because you’re no longer in this exponential curve. But in the beginning, it’s a very good indicator of the rate of spread of the disease. So use that doubling time to back into what you may become familiar with, which is R0. The R0 is a good characteristic. For every infected person how many people will they in fact during their period of infection? Or infectious state. And so with the doubling time and this notion of recovery time how long the person is in this infectious state, we can back into this R0 value of in this case it’s 3.65. And- 

SOFIA DEMARCO: Can. I just stop you now, can I ask a couple of questions on this? OK, so I’m looking at the first parameters we talked about how important it is to capture the current hospitalized patients. Now for this, this would mean within my hospital, right? 

MICHAEL DRAUGELIS: Yeah, that’s within actually, it’s within whatever region that you’re interested in. So for Penn Medicine, we have three hospitals that we think serve this one market region, that we think we have 15% on. So that number 69 at the time that we published this represented the number of [? hotspots ?] COVID patients at those three downtown hospitals. If you were Governor of Pennsylvania and you’re concerned about Pennsylvania then you would enter in the current number of hospitalised patients in Pennsylvania and your market share might be 100% and then population in Pennsylvania population. 

SOFIA DEMARCO: So it’s really whatever the scope is that I’m trying to model for and then that should match up with how I hold the market share. If I’m talking about just my own hospital or maybe I’m in a health [? system, ?]. I’ve got a group of hospitals that I’m managing,. I would be looking at that corresponding market share. OK, and then to get this count in the way that the model needs it, should I be entering in all my PUI,. Patients Under Investigation? Should I be entering in any patients who are actually in the emergency department right now and likely have COVID-19? Who should I consider? 

MICHAEL DRAUGELIS: Yeah, that’s a great question, it’s a really great question. I can tell you that in the. Pennsylvania health system we’re entering just our positive number because we’re trying to anchor that in some level of, some level of certainty. We do discuss internally whether we should start looking at patients under investigation. The reason we don’t is the varying degrees of response in test time across our region, varies quite a bit. So what we’re doing is selecting the actual number of hospitalized patients and just updating that. And we found that the model performs pretty well for various reasons detailed reasons. But it performs pretty well if you use just the actual positive hospitalizations. 

SOFIA DEMARCO: OK, got it. Back, to you then we talked about the doubling time, using that based on actual observed cases and how the model uses that. 

MICHAEL DRAUGELIS: That’s right, that’s right. So as I said the data– the number of infectious people are derived from number of hospitalized, observed hospitalized patients. And so one thing you can do is to– as [? doctors ?] as I know the date of the first hospitalization case. If that’s indeed true which for us it is, you can click that. And Sofia are you showing your screen by the way? That- 

SOFIA DEMARCO: I’m sharing my screen. I’m not sure exactly what’s being pushed out right now let me actually try and try to do that once again, as I think the view isn’t showing everything that it should. I’m going to try that took it one more time from my side. Sorry folks for this technical difficulty on that. I’m actually going to end the screen share and try it again. See if we can folks the proper view here. 

MICHAEL DRAUGELIS: OK. So I’ll just talk to that field. And by the way, anyone listening on can kind of walk through this themselves with that, with that app. Yeah, so you click that value you can put into the day of your first hospitalization. For Penn that was March 16 and then what will happen is the model will fit the SIR curve to your observations and it will back into what it thinks you’re doubling time is. And that’s– that way you can kind of make sure that the model is actually fitting here your true observations. Below that- 

SOFIA DEMARCO: OK, so that means 


SOFIA DEMARCO: It [? finds ?] that I do now the date of the first hospitalized case then I don’t need to know the doubling time? 

MICHAEL DRAUGELIS: Yeah, you don’t need to know it. I think that as an analyst running those there should be a sense of what is being published and what’s being reported regionally as to what the doubling time is. The reason for that is a little technical but let me just try to summarize. As early in the spread because of the lag in incubation of the virus and lag in time of someone becoming sick enough to seek out hospital care, that lag kind of creates a coiled up observation of a number of people that are coming into your health system. It appears that you have a faster doubling time if your observation is hospitalizations. In fact, the observed doubling time could be between two and three, 3.5. And so when you fit your curve to that doubling time this model will project out pretty well, what you should expect in the next week or two based on your observations. But overall for this curve to project out overall, that doubling time is fairly aggressive and over time that will converge your observations of hospitalizations, that doubling time of hospitalizations will converge to the doubling time of the infection spread. 

And so what we do in Penn, is we do two scenarios, aggressive scenario where we fit the doubling time to the first observation by clicking that button and putting in March 16 and you’ll get typically an aggressive number. And then right now we’re using a doubling time of five as a conservative number and the reason we use five is that drives down our R0 value to 3.0, which is just is based on a recent publication from Imperial College, March 26, based on data from the US and UK looking at what are some estimates of R0 are. And so there’s two approaches that we take in modeling that’s why there’s two options here. One is to say I know the first date of hospitalization and I want to fit my curve to that so I can project out well in the next couple weeks. For say you have the ED department that’s trying to man up, you know staff up. And then if you want to try to estimate where you think the peak would be and it’s magnitude to try to use some more regionally reported transmission time of the infection.

 And those are two good kind of rules of thumb to use as you kind of set your scenarios 

SOFIA DEMARCO: OK got it. Within that, if I know the first date that’s going to give me the more aggressive scenario? 

MICHAEL DRAUGELIS: That’s right, [? we do more. ?] If you know that first date it’s going to be the more aggressive doubling time scenario and we’re actually writing a blog post on this to kind of share that out. And see what I’m going to do actually I’m just going to walk through this maybe with everyone on the phone and I actually don’t see your screen that you’re sharing but I’m going to kind of walk down and be really descriptive as I’m walking through what I’m seeing. So we have then social distancing. This is a big, big question in the spread of course. And we use as a benchmark two sources whereas again a publication from. Imperial College, a scenario– modeling scenarios [? we are going to ?] use a higher fidelity model where they could put in a lot of specifics about how people interact. 

And they looked at the reduction in the peak census when they say close schools down or when they close non-essential businesses or then when they add more broad social distancing measures like shelter in place. And what we did for our model is we kind of build up what we thought that might be based on some psychology papers that are published and we reference in our blog post about the types of interactions that people have and what if we reduce those based on social policies. And what we came up with was something that fit well and compared well with this other model, which was by closing down schools that had a reduction in social contact by 5%. 

Remember in that case you’re still going to the bars, your kids are still playing with their kids, you’re going to work, all those things. So 5% may give you an intuition around that. If you close non-essential businesses that’s 8%. If you start to layer in stringent social reductions only then it gets into 15%. You add all those things up and you can get near to social distancing reduction of 30%. We have a blog post on this on our website that’s linked on the top of the tool there. And so what we put in for our social distancing bounds are between 30% and 40%. 40% is very aggressive. I can tell you just my personal feeling is that we are in an aggressive place. I feel like I haven’t left my house in a few days. So that’s kind of a bounding, again bounding scenarios. Now one thing that we’re constantly looking for is how to validate the actual reduction. What’s the truth on the ground? Hoe much do people really reduce their interactions with one another? 

And that’s a big, big question and an unknown right now. So again it highlights the need to kind of bound assumptions there being optimistic and pessimistic. Severity parameters, so these are the rates of total infections in your populations at any point in time. Of everyone infected how many of them will be hospitalized? And we have a blog post kind of driving why we think 2.5 is a valid number. I can tell you it’s really driven largely by the number of asymptomatic patients. There’s a lot of publications, several publications stating that of symptomatic patients 5% are hospitalized. And there’s a lot of evidence now showing that there’s large proportions of people that are asymptomatic or mild and never present. So from there, we have [? IT percent, ?] 0.75, that’s derived from 30% of hospitalized patients are in critical care. And then we have vented patients at 0.5 that’s derived from between 2/3 and 70% of patients are in critical care are vented. That was a recent paper published a couple of days ago from Seattle. Infectious days this one is one that we haven’t moved around a lot although I think there’s lots of reasons to be skeptical of this number. This is the number of days that someone is infected and how long they’re infectious, how long they can spread this disease. 

This is a number that was provided by AJ back in late February, and we’re now currently looking heavily for better numbers on number data, someone who is actually truly infectious. The next three [? columns ?] are ones that we get lots of questions on. And I think we can continue to improve how we prompted for this information. But what this is intended to represent is, what is the average hospital length of stay for everybody in the hospital? That’s people that are in the ward only, in the ICU, and then go to the ward, on vent and so on. What is the average? Average days in ICU, that’s for someone that is in the ICU. How many days do they spend in it? That would not account for their entire time in the ICU. 

And then days on vents, what is the total number of days a patient has on vent? I can tell you that there are papers and material that you can draw on to push in these numbers here, but we intend to use 8, 14 and 15. And those numbers are related to one another based on the distribution of acuity of your patients, and then your assumptions on how long people after vent stay in the ICU and after ICU go to the ward. All those things are connected. That’s another area where we’re going to just try to clarify that, but with some arithmetic and a spreadsheet, it’s possible to pull those numbers out yourself. So I am- 

SOFIA DEMARCO: [INAUDIBLE] part of that Mike? These are not– the length of stay numbers, I’ve got three of them, hospital, ICU, and vents. Those are for my entire hospital or set of hospitals that I’m running this for, not just for my COVID-19 patients? 

MICHAEL DRAUGELIS: Sorry, thanks for asking about that. Actually, these are absolutely just COVID-19 patients. Just for COVID-19 patients. That’s a great question. 

SOFIA DEMARCO: So if. I’m wanting to get– if I wanted to get these numbers, I should be looking at my hospital or set of hospitals data. And sets for my. COVID-19 population. 


SOFIA DEMARCO: [INAUDIBLE] MICHAEL DRAUGELIS: And so in our blog post we’re going to reference analysis based on patients that we feel are similar representatives of COVID-19 patients along with citing current publication and then kind of where we ended up in selecting that. But the best data for this is going to be retrospectively looking back and understanding you know what was the pattern for these patients? Yeah, so that’s kind of the run-through. There’s a few display parameters below. There’s a number of days to forecast out, and then you can set your axis on the graph to a static value. And the reason that that’s so important actually in reporting, and generating reports to distribute, so you want to show scenarios– you might show relative scale of change that those scenarios. And again because the parameters are so uncertain, that’s one of the big values as you can reflect the directional change of assuming certain social distancing factors or length of stay changes from one scenario to the next. And so keeping that y-axis allows [? viewer ?] to kind of quickly visualize that. 

And then down below there’s a current date, default is today, and, of course, maybe you’ll notice that it’s not today. We’re rolling that fix probably as we speak so that updates to the actual current day but you can’t go in manually and select that to whatever day you want to pretend it is and so typically in a report you make that the current day of the report to show your current hospitals. I’ll also say one way, like an expert way of using this is we sometimes say select kind of the middle of the pandemic, as we’ve experienced it. Our first hospitalizations were on the 16th. Sometimes we put in there March 21 and we fit that to our past seven or so days and we look how well does it project out where we actually are right now. And that helps us kind of get an intuition in a sense of how well we’re setting the parameters up and how well we can interpret the results of the simulation. 

SOFIA DEMARCO: OK, that seems like a really important one. So usually we would set this to today we can also use this date parameter to help sanity check the results. Is that right? 

MICHAEL DRAUGELIS: That’s right that’s exactly right. Yeah. 

SOFIA DEMARCO: OK, so I can set it to a past date and say based– if I run it for a past date, do the results I get at that point actually match up with my observed growth pattern, my observed count? 

MICHAEL DRAUGELIS: That’s right. So just a note when you do that if you’re going to put in a week you know last week’s date, they need to go up to the top and make sure that you put in current hospitalizations as it was a week ago. So those two pieces of data are aligned and that’s one of the key pieces that the model uses to fit the parameters to. 

SOFIA DEMARCO: Got it. What about on the number of days– the number of days to project? How hard does it make sense to look out? And also wondering how often does it make sense to rerun this model? When should people think of rerunning the model? 

MICHAEL DRAUGELIS: Yes, so we forecast out enough to see and observe the peak. And the reason we do that is not to just run one scenario with your best settings and say ah the peaks going to happen on May 12th, that’s not the reason. The reason is to run bounded scenarios so that you can see the range in which a possible peak could occur. And so that’s the primary driver of the days that you select, it’s really to just explore the potential scenarios that you should plan for as a health system or region. Sofia do you want me just to talk to the displays a little bit? 

SOFIA DEMARCO: Absolutely. 

MICHAEL DRAUGELIS: All right. OK, so yes I’m just scrolling through myself here again. And now I’m looking at new admissions. There’s two key pieces of graphs and table data. It’s new admissions and census. So one thing that’s important to note is the meaning of the data here. I’m looking at the graph and hovering over the red line and it has ventilated patients. So what that means is you know I might– I’m looking at a particular day May 9th near the peak of the curve of the settings and it’s May 12th, sorry. And it says admitted 60 [? keys ?] and it says ventilated. So what that means is on that day our health system is forecasted to admit 60 patients that would need a mechanical vent. 

If I move up a little bit and I look at critical care, it says 90. And what that means is on that day I would expect to admit 90 patients I would give critical care, which includes those vented patients. And as I scroll you know move all the way up, mouse over to the top on the blue, it says 300 patients and what that means is on that day I would expect to admit 300 patients including the critical care patients and the ventilator patients. These are not some of– these don’t sum up to the total number, but they’re cumulative.

 SOFIA DEMARCO: OK, so these are cumulative. Now, s I’m showing different numbers than you are because I’ve set different parameters in here. So let’s not confuse folks. The peak here in the blue that we’re seeing is 208 on May 25th for the parameters that I ran. And that’s the total, the blue line includes the orange and the red so you don’t need to sum these up. And then- 


SOFIA DEMARCO: Mike would this be for all patients in the hospital? Is this focus specifically on COVID-19 patients? How do we think about that? 

MICHAEL DRAUGELIS: Yeah, thanks for driving up that. So yeah it says the projected number for daily COVID-19 admissions. So we’re not providing parameters for the user to enter in their current hospital census but really expecting that you would build a kind of layer that in yourself. 

SOFIA DEMARCO: OK, got it. So if I’m seeing my peak here as 208 a total hospitalized that’s for COVID-19. So anybody else I’m expecting in my census I should mentally add those folks in as well. 

MICHAEL DRAUGELIS: That’s right. Just to add some quick analysis from Penn, we’ve noticed that we’re very near to the base, baseline census of non-COVID patients, where it looks like we’re about 60% full across on average are our health [INAUDIBLE] 60% and 70% full. So we’re kind of mentally layering that in as what our capacity is. So what you’ll see below that just for ease of reporting is just [? intact ?] with the peak values are in each of those categories for admissions. You can download the data in GFC format for spreadsheet analysis. That’ll download an everyday value. It’s unfortunately, it’s in decimal form. we’re showing like half of a patient. We’ll update that as well to round down to a whole number, a whole number patient there. But if you expand the table you can see kind of what that would look like. And you’ll notice something that could potentially be a little confusing. It’s showing a negative number in there. And that’s a function of the fit of the algorithm we have that’s basically taking in your observed day, admissions and if you’re putting in you know the first day of admissions it’s going to fit to that day and it’s going to count backwards from your current day zero to get to that might be in this case, I’m looking at three weeks back. 

If you don’t connect to your– if you don’t enter in your first day of admissions so as I said for Penn that was March 16, if I don’t enter that and instead I put in my doubling time what the model will do is it’ll pick its own first day of hospitalization. And so you’ll still see these negative numbers and they’ll show– it just shows a day that it needed to pick to stick to your observed hospitalizations today with your entered doubling time. 

SOFIA DEMARCO: OK so for these negative days, that would essentially mean if I see negative seven that’s seven days before the start date that I picked at the top where we enter the current date. So seven days before that. I can use it, or can. I use these numbers to do some sanity checking on the model results? See if I’ve entered my parameters well. 

MICHAEL DRAUGELIS: Yeah. You’re right, so these numbers on hospitalized ICU vented these are the numbers that appear in that curve. And so those you can interpret them as on that day listed, I’m looking at March 20 or sorry. April 5th here on my parameters app and you would say 25.2. What that’s saying is that there will be 25 admitted patients into the hospital and to the right of that as the ICU column. They’ll tell you how many patients will be admitted to the ICU and then to write it out as ventilated, how many patients should be ventilated. One note here, our ED departments have you’ve been using this quite a bit to try to understand how many patients will be, will arrive at the ED and ultimately be admitted. And so the ED department’s really focused on these admission numbers. 

If you scroll down to the admitted patients census for me, you know internally to health systems we often talk about census, a number of patients, and we apply that to different categories of patients. So with a census, of our you know heart failure patients, in this case, census of COVID-19 patients. I think it’s a bit of a term of art for internal health systems at least it is at Penn. Medicine, but just a acknowledge that might not be a common term used for folks in the word census. But what we mean here is that in this graph that we’re showing that the number of beds being taken up when we showed– I’m going to go from the blue down to red. So looking at blue at the peak number of patients that are requiring beds, COVID-19 patients that require beds, and moving my mouse over to the orange that’s showing me how many patients are requiring a critical care bed, COVID-19 patients. And then below that’s showing me how many patients require mechanical beds, COVID-19 patients. And then below that is just as a quick summary of what those peaks are over those times. And again you can download the Excel spreadsheet and you can expand the table to show that in tabular form. 

SOFIA DEMARCO: OK so the quick version of the difference between the top chart and the second chart. Sums it up- 


SOFIA DEMARCO: We’ve got that new admissions versus admitted. 

MICHAEL DRAUGELIS: That’s right. And then those new admissions, the model drives out from those new admissions that what the SRI model creates is new admissions. That’s all based on the number of new infections. And then the admitted patients census is driven out by those new admission numbers and so the census takes those new admissions and those length of stay values that you entered and then basically marches out the number of beds that are required and handles the discharge as those days that you’ve entered come to fruition. And so it is kind of keeping track of people in, coming in, and out of the hospital and those acuity levels. 

SOFIA DEMARCO: OK, perfect. And with that, I think we’ve done the walkthrough here in detail so great at this point to start opening it up for questions from folks. For those of you that have been entering questions in, we’re seeing those pop up for us [INAUDIBLE].. So let me go ahead and, we did have some a couple of points within this about how the results are being used for capacity planning at Penn Medicine. 


SOFIA DEMARCO: I think you we should go ahead into the Q&A though. I think we’ve had a couple of questions that have popped up in here, while we’re still looking kind of fresh having just looked at the model, let’s go through these and then we can come back to how this model is being used by Penn as well. All right so the first question that I’m seeing in here. Doubling time is dynamic through the whole pandemic. The initial doubling time is usually very small, usually less than one as it takes very little time to double the cases. Why do you think putting five days in the initial doubling time is legitimate? What is- 



MICHAEL DRAUGELIS:. That’s a great question and thank you for whoever asked that. Exactly so, right. So what we see– you’re right about doubling time being very small or very rapid initially in the spread. And particularly you know I think that’s represented also in the time of incubation and time to become severely sick and seek hospital care. And so that’s what we’re observing right now and in a lot of our health systems. We’re observing doubling time of 2.5, of 3. And so what we’re doing is using those observed doubling time values based on those hospitalizations to forecast out what we thinks possible in the next two to three weeks. Acknowledging absolutely this kind of person that– this is a question acknowledging that they’re doubling time is a dynamic thing that’s moving on, based on how people are social distancing and as the disease spreads through. We use that doubling time and under initial conditions to set our beta or to set R0 value. And so that will adjust as the SIR model moves forward and you have more people that are infected you have more people that are recovered. 

That is not a maintained doubling rate. And if it were the curve would look just completely exponential and it wouldn’t have this kind of bell-shaped curve. The other side of it the answer to this, which is a great one too like why five? It’s a great, great question. So five for us is coming from a recent publication by Imperial College that is stating a R0 value of 3. And so in our simulation when we set or adjust our parameters to see an R0 of three, we have to set our initial doubling time of five. So what we know by doing that is that, that doubling time of five is not going to represent our early spread very well for all the things that I just discussed and the point that the questioner listed. 

But what it could indicate is a better appreciation for when and magnitude of the peak. And so we’re internally running these two scenarios that we view as one as aggressive and one as conservative to try to assess, you know when and to what magnitude are our teams need to be ready. And in both of those cases, by the way, in the model you can enter in social distancing factors on top of both of those assumptions. 

SOFIA DEMARCO: OK so kind of following up on that, does that mean doubling time should be a future estimation, not current doubling time. In other words, how should we choose the right doubling time to calibrate the model to reality? 

MICHAEL DRAUGELIS: Right, right. So really it’s looking at adjusting that initial– setting the doubling time based on initial spread conditions and then adjusting the social distancing as you’re moving through the disease spread and fitting those post values as you get new observations. 

SOFIA DEMARCO: One other really kind of down in the details about the modeling itself and appropriateness of how it’s being done. What about an SEIR model? Is that something that would be appropriate with eventually even be more appropriate? 

MICHAEL DRAUGELIS: It is. So this gets back to the uncertainty of parameters and still trying to make better and better higher fidelity models. So an SEIR model, great question, E stands for exposed and it kind of represents the incubation period of the infection. And certainly if we had that in this model, by adding that in it would account a little better for the lag in doubling time that we’re just now talking about, for one. But then your patient or your person would move from exposed to infected and in that model when you layer in hospitalization as soon as someone is in the infected state they’re now a candidate to be hospitalized which isn’t exactly true either. 

There’s a distribution of time that it takes someone to become infected and then become severely sick and seek hospitalization or hospital care. And so there’s some fidelity issues there and it’s not going to change the magnitude of the impact here. It’s not going to change the core assumptions of the model, which is that everyone kind of moves around and can connect into one another with an equal chance of spread. There are other models that are higher fidelity even so but again there’s always uncertainties in the parameters that you’re feeding it. 

And so in each model each one you choose to use you have to as a health system planner select your boundaries in each case. Each one is founded in first principles and epidemiological study and has legitimacy and use, but you have to understand its assumptions and be able to bound those on your estimates and parameters you’re putting in.

 SOFIA DEMARCO: OK, this question is about the recovery time. So we talked a little bit about that 14 days as the input is being used right now. As the default in the model which, of course, anybody can change when they’re running the model if they want to see results based on a different recovery time estimate. Can you talk a little bit about that estimate right now. How legitimate we think that it is? And if there’s any value to potentially running scenarios with a shortened time. 

MICHAEL DRAUGELIS: Yeah, yeah I highly suspect that 14 values is too aggressive in terms of disease spread and that the number should come down. That’s something that our school’s medicine team is looking into right now. And by doing that what that will affect your R0, not the doubling time. But it will affect the actual number of patients that someone can infect during their infectious period and so that’ll change the volume of the peak as you run your scenario through, it’ll reduce that volume. 

SOFIA DEMARCO: For all of those spread related parameters, are those things that we should expect to have different values place to place? Or are those things where if somebody who’s got the right answer that can be quote unquote “right answer” that can be then applied across the board for anybody trying to run this model? 

MICHAEL DRAUGELIS: Yeah, yeah. I think some yes, and some no. There’s no– it’s kind of a spectrum. Right? So I think we look at acuity in the United States as it is probably a normalized view of who should be hospitalized, who should be vented, who needs critical care. So those proportions of total infected patients are probably pretty standard around US care. Probably length of stay is pretty standard. So I think those numbers are things that we collectively communicate to one another can share and converge on. In terms of these infection– and by the way, I think the infectious period as well, but in terms of R0 and doubling time even with, in our own region it’s varying as we observe it differently from our suburban hospitals to our city hospitals. So you know the doubling time and R0 value you know New York. City is going to be different than in. Marquette, Michigan. And those are the aspects that you have to keep a close eye on.

 SOFIA DEMARCO: OK. With that maybe we can talk a little bit more about the practical aspects of using this model. So let’s turn to how the results are being used for capacity planning at Penn Medicine right now. 

MICHAEL DRAUGELIS:. Yeah, absolutely. Let me just say kind of really briefly what our process is. So we have analysts– we have six hospitals, we have an analyst leader at each one. We meet every day at 11:00. We can essentially distribute the parameters we’re using, we hear feedback from those teams and how the planners are using the data. We kind of agree on the parameters of the day and the goals of the report and then we meet again at 2:30 and consolidate that and send it out. So by having that collaboration across our region, we’re able to take some of these outcomes or decisions across Penn which Sofia is showing now. But so something we’re able to do is decide when we’re going start converting some departments to critical care areas. We’re able to– when we have pretty wide assumptions we’re able to determine when we might move some PPE supplies from one health system to another. 

We had one kind of odd event where we’re building a hospital right now and all construction then you know commanded to stop. But in Philadelphia we’re able to communicate these surge numbers early on and get approval to continue building the hospital. The one that I’d like to really highlight is the last one, the relief to the critical care teams. It’s terrifying to see this coming and to not know when your life’s going to change as a care provider. And so one thing this tool helps with is to one, give a sense of yes what’s coming and how can we prepare. But as we were using this early on we could give our different health systems a window to say you know don’t go 100 miles an hour right now. Take time to rest and plan, for the next two to four weeks and we’ll be ready and ready to go when we need to surge kind of starts to hit. 

SOFIA DEMARCO: Perfect. We would like to wrap up here with a couple of notes on how to get help using this model, right now. So we are going to be publishing a summary of this webinar which includes some links for reference. So, of course, the link to the model itself.

 The team at Penn. Medicine has also prepared some pretty robust documentation that’s available from within the tool so you can click on the little i next to any of the parameters set in there, to get more information about that parameter, how it can be estimated, with helpful links to blog posts as well. There’s also a Slack channel available where if you’d like to ask questions over chat the code for Philly team has a bunch of knowledgeable volunteers who are manning that. You can also see other questions that have been asked recently and they’re helpful. There are resources available on YouTube as well where the team has made short videos answering common questions on the model and how to use it. So that is a link we’ll make available in the summary also. It’s also possible to ask questions by email on this. 

So there is an email address that directly goes to the teams who built this model. For that allow a one-week turnaround, but if you’ve got questions really in the weeds on the model and how it functions that could be a good option for you. There’s also the possibility of getting help just navigating all those resources that are out there and getting answers to questions that may be specific to your institution if you email we can help you with that, and point you back to. Mike’s team if needed, but otherwise help you navigate the resources. Some of the parameters that were there in the model, you need regional information. So there are a couple of links here that we’ve compiled on getting those. If you need to figure out how to pull original population numbers or how to estimate your own hospital market share, these could be helpful for you and we’ll publish those as well. 

And there’s one question that I wanted to get to you that came into the chat here and we do have just a minute left. But if we could just get to this one. It says we are currently using this model and doing daily calibration and found using doubling time 10.5 is a match to the current census and running total. Do you think using daily calibration to adjust the doubling time to predict peak and volume is a legitimate way to do prediction? 

MICHAEL DRAUGELIS: We– we’re doing daily adjustments based on observations that we’re getting and that’s because we’re getting so much more information every day, particularly in terms of new hospitalizations and new understandings of acuity levels. And so for us, yes things are moving that fast where we need to do it daily and we are looking at the forecasted outcomes from the tool and comparing that against observations to make updates to a model. We have a blog post on doing just that and it sounds like perhaps this person might be doing the same kind of thing. It’s with just simple spreadsheet estimates along with using [? Chime. ?]. And you can kind of converge to some parameters that you think best fit what you’re seeing regionally or in your health system. 

SOFIA DEMARCO:. Great and with that I think we are at our time. So thanks very much, Mike for walking through all of that. Thanks, to all of you who joined today. If you’d like to be notified when the recording of this webinar and other materials are available there’s an email address here that you can contact or you can visit and click on the link for this webinar. In addition, if you have a comment or idea for future webinars that you’d like to hear you can send that to Thanks very much everybody. 


Back To Top