Medicaid
Article

Did we undercount Medicaid numbers? Four examples of data limitation.

Emily Campbell
Chief Executive Officer
Additional Contributors
No items found.
March 20, 2023
Read time:
Download Fact Sheets
Register now
Share this resource
Subscribe to our Newsletter
By subscribing you agree to with our Privacy Policy.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Download this as a PDF

The Ohio Legislative District Fact Sheets are the most comprehensive and reliable collection of data points on conditions facing people in each Ohio Senate and Ohio House district in the state that we are able to produce. Still, every statistic carries some level of uncertainty and these Fact Sheets are no exception.

 It is important to understand the limitations of the data which underpin any report so that the end user can interpret the information appropriately.

The Ohio Department of Medicaid dashboard Who We Served – Annual Timeline shows over 3.4 million Ohioans served by Medicaid in 2021. This is a significantly higher number than the 2.1 million Ohioans listed on our State of Ohio Legislative Fact Sheet. How can the gap be so large? Several factors contribute to what we believe is an undercount in Medicaid figures included in our Fact Sheet. Even though we know there are probably many more Ohioans enrolled in Medicaid, these remain the best estimates for the time and the geography.  

In order to be able to produce consistent information for all 132 Ohio General Assembly Districts, we had to use the U.S. Census Bureau’s American Community Survey 2017-2021 five-year estimates. There is simply no other source that provides reliable figures on a variety of topics for these small geographies. There are several aspects of this data source that contribute to the seemingly low number of Ohioans who are reported to have Medicaid health coverage.

Administrative data is the most accurate

We strongly suggest that advocates and policymakers trust the numbers which are provided by the Ohio Department of Medicaid. They are in the best position to know the TRUE number of Ohioans enrolled in the programs they administer. Agencies have some information on every single person who is enrolled in their programs. This makes administrative data incredibly accurate, if you can get it. Although great strides have been made toward open data, sometimes data is trapped inside agencies. It may be difficult to get out of program databases in a usable format. It is usually subject to confidentiality protections which limit what agencies can share. Even so, for straight caseload data, it’s the unquestioned best.

 We strongly suggest that advocates and policymakers trust the numbers which are provided by the Ohio Department of Medicaid.

Administrative data is a byproduct of the work of agencies, not something created for research purposes. There is some information that agencies simply don’t collect. For example: the SNAP application does not include a question about race or ethnicity, so data from the Ohio Department of Job and Family Services about enrollees can’t tell us the racial makeup of who receives SNAP in Ohio.

 There are many important research questions that administrative data can’t answer.

On the other hand, data from the U.S. Census Bureau is collected exclusively to provide information about people and communities. It is provided on a consistent basis in a useable format which allows for analysis by small geography (like legislative district) and for various population groups. But it also has limitations. The American Community Survey (ACS) on which Community Solutions often relies is collected via survey (as the name suggests). The complex process to turn survey responses into counts for whole communities has some uncertainty, expressed as margins of error. It’s also self-reported, which introduces questions that we’ll explore in more depth below.

 ACS data is our gold standard for timely, reliable, consistent data on a variety of topics that can be examined in greater depth.

Timeframes may not line up

While Ohio Department of Medicaid can provide an up-to-the-minute number of people enrolled in their programs, there’s always a delay of about a year in the release of the ACS data.

 The data on which our Legislative Fact Sheets is built is the latest available, but it is from the ACS five-year estimates from 2017-2021.

The data on which our Legislative Fact Sheets is built is the latest available, but it is from the ACS five-year estimates from 2017-2021. For smaller geographies, the Census Bureau groups several years of surveys together to produce an estimate of what the number is today. This makes the data they report much more reliable but dampens year-over-year changes. For example, there was a large jump in the number of Ohioans who received Medicaid when Medicaid Expansion went into full effect in 2014 (below), which is reflected in the data from the Department of Medicaid, but not as much in the ACS 5-year estimates.  

People may not say they have Medicaid (or may not know!)

The precise wording of a question or way information is collected can influence what a data point expresses. Self-reported data isn’t inherently bad, but it does introduce the possibility of human error from the very beginning as different people may interpret questions differently. The ACS questions are routinely tested to reduce the chance of misinterpretation or response bias, but they can never be completely eliminated because people aren’t perfect. Below is the actual health insurance question as it appears on the ACS questionnaire.

 Self-reported data does introduce the possibility of human error from the very beginning as different people may interpret questions differently.

Immediately we see that the question asks about CURRENT coverage. Medicaid’s annual report indicates the total number of Ohioans served, which could be more than those on the program at any given time.  

People often mix up MedicAID and MediCARE, which is probably why the ACS questionnaire describes the two programs in their question. We also wonder if everyone who is on Medicaid realizes that’s where their coverage is coming from. According to Ohio Department of Medicaid, 90 percent of Ohioans receive coverage through a Managed Care Organization, including AmeriHealthCaritas, Anthem, Buckeye, CareSource, Humana, Molina, and UnitedHealthcare. These company names may be more familiar to the average Ohioan than “Medicaid.”

 According to Ohio Department of Medicaid, 90 percent of Ohioans receive coverage through a Managed Care Organization, including AmeriHealthCaritas, Anthem, Buckeye, CareSource, Humana, Molina, and UnitedHealthcare. These company names may be more familiar to the average Ohioan than “Medicaid.”

On surveys Community Solutions conducted, we began to notice that people were indicating “Other” as their health insurance and writing down CareSource or another MCO. We suspect the same thing happens in the ACS.

We never mess with the source

It’s very rare that we are able to get all the data we hope to see reported exactly as we would like to use it. We are limited to what data is available. When the health insurance data is reported in ACS (Table B27010), they list out types of coverage alone as well as the most common combinations, including Medicare and Medicaid coverage together. But there are a substantial number of people who have “Other coverage combinations.” We can’t be sure if Medicaid is part of the mix or not, so rather than guessing, we leave those individuals out of the Medicaid count.

 We are confident that at least that many people have Medicaid, but there could be many more.

If we know that the Medicaid numbers on our Legislative District Fact Sheets are almost certainly too low, why don’t we adjust them?  

Good data sources have clearly outlined methodology, allowing someone using the data to understand the limitations and decide how best to use the data. Any adjustments that we make introduce even more uncertainty as they require researchers to make additional assumptions. For example: without deep analysis we don’t know whether the Medicaid undercount is evenly distributed across the state or if it impacts higher and lower-income areas differently.  

The statisticians and data scientists at the Census Bureau carefully test any adjustment to ensure they aren’t introducing undue data bias into the statistics they report. We trust them.

So, why do we bother to include such a flawed number?

 Slightly flawed data is almost always better than no data.

Policymakers, advocates, and community members are hungry for information they can use to try to understand conditions in their communities. Community Solutions works hard to provide data that meets our standards and can be useful and used to advance conversations about pressing issues facing Ohioans. Otherwise, we’re all just guessing.

Download Fact Sheets

Ohio Statewide Data

Download

Wood County

Download

Wyandot County

Download

Williams County

Download

Washington County

Download

Vinton County

Download

Wayne County

Download

Warren County

Download

Van Wert County

Download

Union County

Download

Tuscarawas County

Download

Stark County

Download

Summit County

Download

Trumbull County

Download

Shelby County

Download

Seneca County

Download

Scioto County

Download

Ross County

Download

Sandusky County

Download

Richland County

Download

Putnam County

Download

Huron County

Download

Portage County

Download

Preble County

Download

Pike County

Download

Pickaway County

Download

Perry County

Download

Noble County

Download

Paulding County

Download

Ottawa County

Download

Morrow County

Download

Muskingum County

Download

Morgan County

Download

Montgomery County

Download

Meigs County

Download

Monroe County

Download

Miami County

Download

Mercer County

Download

Marion County

Download

Madison County

Download

Medina County

Download

Mahoning County

Download

Lucas County

Download

Lorain County

Download

Logan County

Download

Licking County

Download

Lawrence County

Download

Lake County

Download

Holmes County

Download

Jackson County

Download

Knox County

Download

Jefferson County

Download

Hocking County

Download

Henry County

Download

Highland County

Download

Harrison County

Download

Hancock County

Download

Hardin County

Download

Greene County

Download

Geauga County

Download

Guernsey County

Download

Hamilton County

Download

Gallia County

Download

Fayette County

Download

Fulton County

Download

Franklin County

Download

Fairfield County

Download

Erie County

Download

Darke County

Download

Defiance County

Download

Coshocton County

Download

Delaware County

Download

Cuyahoga County

Download

Crawford County

Download

Columbiana County

Download

Clinton County

Download

Clermont County

Download

Clark County

Download

Champaign County

Download

Carroll County

Download

Athens County

Download

Ashtabula County

Download

Brown County

Download

Butler County

Download

Belmont County

Download

Auglaize County

Download

Ashland County

Download

Allen County

Download

Adams County

Download

Lake County

Download

Geauga County

Download

Cuyahoga County

Download

All Municipalities Geauga County

Download

All Municipalities Lake County

Download

All Municipalities Cuyahoga County

Download

Thompson

Download

South Russell

Download

Russell

Download

Parkman

Download
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Download report

Subscribe to our newsletter

5 Things you need to know arrives on Mondays with the latest articles, events, and advocacy developments in Ohio

Explore the fact sheets

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique.

No Related Fact Sheets

Explore Topics

Browse articles, research, and testimony.