What Do We Learn About Population Disparities from Matching?
Silber JH, Rosenbaum PR, Clark AS, Giantonio BJ, Ross RN, Teng Y, Wang M, Niknam BA, Ludwig JM, Wang W, Even-Shoshan O, Fox KR.
Characteristics associated with differences in survival among black and white women with breast cancer. JAMA. 2013 Jul 24;310(4):389-97. PMID: 23917289
This is a very good paper for a clinical journal, and these authors clearly know what they're doing, statistically. The authors show that disparities in provision of medical care matter almost not at all. By the time women present for care, the disparities are already set. This is a somewhat surprising result, given all the attention we put into care and into researching disparities in who gets what care. But for the champions of social determinants of health, it must feel like something of a vindication, since it indicates that inequality out there in the world is what drives disparities. By the time people show up at the doctor's office, the damage is already done.
Still, it struck me as odd that the accompanying commentary still focused a lot on medical care, especially on data for medical interventions that the authors didn't have available (e.g. dose). That is, the message of the paper is that medical care disparities are almost irrelevant to the racial gap, and the commentator argues that perhaps this is due, at least in part, to incomplete data on the medical intervention details.
Despite the clear technical mastery, I have some lingering discomfort with what we get out of the matching strategy, conceptually. The authors argue (p. 395):
"One important strength of our study was that 99,898 white patients were used as potential controls
for 7,375 black patients. This allowed us to achieve very close matches, generally avoiding the need
for model-based analyses. A model fitted to 99,898 whites and 7,375 blacks would be a model that
mostly describes what happens to whites."
I see what they are saying, but I can also see the flip side with respect to generalizability. By throwing away 90% of the white data, the results are no longer representative of what happens in the real world population, but rather in a highly selected population in which we look at only those white patients who present in ways that make them most similar to black patients. Who are those 10% of whites who present at the same late stage and with the same comorbid conditions, etc as blacks? One could guess that they'd be a lot poorer, for one thing, but the authors don’t have individual SES information.
One would obviously expect, therefore, that in the broader, unselected population, the disparities would be much worse. And indeed, this is what they find (i.e. the disparities are much reduced in the matched data). For example, in the analysis data set, the authors matched on characteristics such as comorbid conditions. If a black woman had diabetes, then they had to also find a white women with diabetes, etc. So in these matched data sets, the breast cancer survival experience of white women looked much more like the experience of black women. No surprise. But if something like diabetes is much more prevalent in black women than in white women, the newly defined target population based on measured characteristics of the black population will obscure that real-world inequality. What is left in the sample being analyzed is just the inequality in unmeasured characteristics. As such, this is just a fancy version of the longstanding game of adjusting for some measured factors and seeing how much disparity is left over. The left over part is due to unmeasured things, and so you can't say much about it. The commentator lists a bunch unmeasured medical care things, perhaps because she is an MD, but someone else could speculate about racism or genes or anything else. This therefore seems to me to be somewhat unsatisfying in that it doesn't describe the real world (since it is pruned drastically through matching) and it matches only as far as you can go in these admittedly crude data (for example, very limited data on SES). So what do we get out of this? The authors demonstrate that in the matched sample (which I repeat is not like the real world) the black women do get significantly less intervention, but that this doesn't matter very much for their survival. So does this mean we should stop worrying about discrimination and access to care, etc? I am really not sure how to take this, but it is not clear to me that it is all that reassuring.
Characteristics associated with differences in survival among black and white women with breast cancer. JAMA. 2013 Jul 24;310(4):389-97. PMID: 23917289
This is a very good paper for a clinical journal, and these authors clearly know what they're doing, statistically. The authors show that disparities in provision of medical care matter almost not at all. By the time women present for care, the disparities are already set. This is a somewhat surprising result, given all the attention we put into care and into researching disparities in who gets what care. But for the champions of social determinants of health, it must feel like something of a vindication, since it indicates that inequality out there in the world is what drives disparities. By the time people show up at the doctor's office, the damage is already done.
Still, it struck me as odd that the accompanying commentary still focused a lot on medical care, especially on data for medical interventions that the authors didn't have available (e.g. dose). That is, the message of the paper is that medical care disparities are almost irrelevant to the racial gap, and the commentator argues that perhaps this is due, at least in part, to incomplete data on the medical intervention details.
Despite the clear technical mastery, I have some lingering discomfort with what we get out of the matching strategy, conceptually. The authors argue (p. 395):
"One important strength of our study was that 99,898 white patients were used as potential controls
for 7,375 black patients. This allowed us to achieve very close matches, generally avoiding the need
for model-based analyses. A model fitted to 99,898 whites and 7,375 blacks would be a model that
mostly describes what happens to whites."
I see what they are saying, but I can also see the flip side with respect to generalizability. By throwing away 90% of the white data, the results are no longer representative of what happens in the real world population, but rather in a highly selected population in which we look at only those white patients who present in ways that make them most similar to black patients. Who are those 10% of whites who present at the same late stage and with the same comorbid conditions, etc as blacks? One could guess that they'd be a lot poorer, for one thing, but the authors don’t have individual SES information.
One would obviously expect, therefore, that in the broader, unselected population, the disparities would be much worse. And indeed, this is what they find (i.e. the disparities are much reduced in the matched data). For example, in the analysis data set, the authors matched on characteristics such as comorbid conditions. If a black woman had diabetes, then they had to also find a white women with diabetes, etc. So in these matched data sets, the breast cancer survival experience of white women looked much more like the experience of black women. No surprise. But if something like diabetes is much more prevalent in black women than in white women, the newly defined target population based on measured characteristics of the black population will obscure that real-world inequality. What is left in the sample being analyzed is just the inequality in unmeasured characteristics. As such, this is just a fancy version of the longstanding game of adjusting for some measured factors and seeing how much disparity is left over. The left over part is due to unmeasured things, and so you can't say much about it. The commentator lists a bunch unmeasured medical care things, perhaps because she is an MD, but someone else could speculate about racism or genes or anything else. This therefore seems to me to be somewhat unsatisfying in that it doesn't describe the real world (since it is pruned drastically through matching) and it matches only as far as you can go in these admittedly crude data (for example, very limited data on SES). So what do we get out of this? The authors demonstrate that in the matched sample (which I repeat is not like the real world) the black women do get significantly less intervention, but that this doesn't matter very much for their survival. So does this mean we should stop worrying about discrimination and access to care, etc? I am really not sure how to take this, but it is not clear to me that it is all that reassuring.