James P. Scanlan, Attorney at Law

Adjustment Issues

(Nov. 2, 2009; rev. Sept. 12, 2010, without outdated links replaced May 16, 2021)

This is a preliminary vignette regarding adjustment issues.  A number of comments listed in Section D of the Measuring Health Disparities (MHD) page involve adjustment issues.[1-4]  Some of the issues addressed in those comments may eventually be treated here along with additional issues.  Five issues are addressed immediately below:  (1) adjustment of changes in absolute differences to take confounders into account by means of logistic regression; (2) adjustment of differences between composition of the sample and composition of the population at large; (3) choice of reference group profile; (4) appraising the size of a reduction in a disparity achieved by an adjustment for risk factors; (5) weighted average disparity compared with adjusting underlying rates.

1.  Adjustment of changes in absolute differences to take confounders into account by means of logistic regression

The way that absolute differences and differences measured by odds ratios change systematically in opposite directions is discussed on the Measuring Health Disparities page, on the Scanlan’s Rule page, and in many other places.  There have been situations where changes in healthcare disparities over time have been measured by absolute differences, after which a logistic regression is conducted in order to determine whether confounders have had a role in the observed pattern. 

Table 1 below is an abbreviated version of Table 1 in reference 5 (BSPS 2006 Table 1), which shows various measures at various cut points where the difference between the mean scores of two groups is .5 standard deviations.

Suppose that there occurred an overall change in prevalence such as reflected by a movement from point J to point K.  One would find that the absolute difference between rates decreased from 19.05 to 16.65 percentage points.  Suppose then that one were to conduct a logistic regression to determine whether confounders had some role in the observed changes.  Suppose also that in fact confounders played no role whatever.  The analysis using logistic regression, which would be based on odds ratios with and without adjustment for confounders, would find that the difference as measured by odds ratios increased from 2.24 to 2.31.  Thus, one would wrongly conclude that adjustment for confounders had caused the pattern of change to be reversed when in fact the confounders had no role and both the decrease in the absolute difference and the increase in the odds ratio were functions of changes in overall prevalence. 

2.  Adjustment of differences between composition of the sample and composition of the population at large

Reference 1(Comment on Schulman NEJM 1999 )  involves a study that caused a good deal of controversy, mainly concerning the fact that odds ratios were presented in the media as if they reflected relative risk, but also regarding the fact that the results were presented as if blacks and women were referred for catheterization at lower rates than whites and men when in fact white men, black men, and white women were all referred at exactly the same rate.  Only black women were referred at lower rates.  The various referral rates were:  white men 90.6%; white women 90.6%;  black men 90.6%;  black women 78.8% (reported as 78.8%, actually 78.9%).   A number of problems with this study are discussed in the comment including the curious adjustment for perceptions of physicians in analyzing differences in perceptions.

The instant point, however, involves an adjustment issue that was not only overlooked by the twelve authors of the study but also by those who criticized it.  The study was based on physician recommendations where 720 physicians reviewed profiles and videotapes of symptoms presented by 2 members of each of the four race/gender groups (90 recommendations for each race/gender group).  The reported racial difference, which was presented in terms of an odds ratio, was based on the 90.6% referral rate for white men and women combined and the 84.7% rate for black men and women combined; and the reported gender difference, which also was presented in terms of an odds ratio, was based on the 90.6% figure for white and black men combined and the 84.7% rate for white and black women combined. 
By conducting the same number of analyses of black and white subjects the study in effect substantially over-sampled blacks.  Oversampling in this manner is a legitimate way of enhancing the power of the study to detect differences.  But the broad purpose of this study, and its implications as they would be reported in the media, did not involve the disparities in the treatment of the 8 subjects, but disparities in the larger populations that the 8 subjects were intended to represent.  In circumstances such as here where the differences were driven solely by the results of a subgroup, that fact has important implications for the calculation of the differences in the larger population.
That is, assuming that it makes sense to present differences between men and women generally when only black women differ from any other group (which it probably does not), one needs still to take into account that in the United States at large, there are approximately 6.6 white women for every black woman. Thus, an appropriately weighted female referral rate would have been 89.1 percent, hardly distinguishable from the aggregated male figure (90.6%).  The same reasoning applies to the male rate.  But since the black male rate was the same as the white male rate, a weighted average referral rate for men would be the same as the unweighted average.

This reasoning also applies to the overall racial differences, which should be adjusted for differences in the gender composition of the two racial groups.  Since women comprise a slightly higher proportion of American blacks than of American whites, an appropriate adjustment would slightly increase the black-white difference.  But the increase is not substantial enough to warrant addressing the matter in any detail.
3.  Choice of reference group profile

In his classic Introduction to Epidemiology, in discussing an adjustment for different risk profiles of two groups, Gary Freedman suggests that the adjustment may be just as well based on the profile of both groups combined as on the profile of either group being compared.  That is, in the former approach, the risk profile of the entire group is attributed to each of the two groups; in the latter, the risk profile of one group is attributed to the other.

I favor an adjustment based on the profile of the advantaged group (which also is usually the larger group) rather than the entire group because adjustment based on the combined risk profile would allow the result to be influenced by the comparative size of the advantaged and disadvantaged groups.  Consider a situation where one is adjusting a disparity between whites and minority group A (whom whites outnumber 3 to1) and minority group B (whom whites outnumber 6 to 1).  If groups A and B could have exactly the same risk profiles and exactly the same rates within each risk profile, adjustments of the white-Group A differences according to the profile of whites and group A combined and adjustment of the white-Group B difference according to the risk profile of whites and Group B combined would result in different adjustment to the two sets of rate differences (with a smaller adjustment in the case of the white-group A difference).  Adjustment according to the white risk profile would result in the same adjustment to the disparity between the white rates and the rates of each group, which seems more in keeping with the purpose of the adjustment.[i]

Further, in references 2 (May 2006 Comment on Lynch JECH 2006)  and 3 (Comment on Khang Heart 2008)  below I explained that what the authors maintained was a substantially different adjustment for absolute and relative differences was simply a result of the fact that, rather than employing a standard adjustment for differing risk profiles, the authors had examined the consequences of eliminating all risk factors.  I explained that standard adjustment according to the profile of one group or the other would have the same proportionate effect on the absolute difference between rates as on the relative difference between rates. 

But a 2009 article[ii] that made points about the subject of reference 2 similar to those I had made in reference 2 carried out a standard adjustment for risk factors that used the risk profile of the total group as a reference group.  Review of that article led me to recognize that the proportionate effects of standard adjustment on relative and absolute differences are the same only when the profile of one of the groups being compared is used as the reference profile.  The proportionate reductions can differ when the adjustment attributes the profile of the two groups combined to each of the two groups, as discussed in reference 4 (Nov 2009 Comment on Lynch JECH 2006).  In addition, while adjustment according to the profile of either group will effect the same proportionate reduction in the relative difference in experiencing an outcome as in the relative difference in avoiding the outcome, the proportionate reductions in the two relative differences can differ when the combined profile is attributed to each group.  

4.  Appraising the size of a reduction in a disparity achieved by an adjustment for risk factors. 

Suppose the rates at which AG and DG experience an adverse outcome are 10% and 20%.  Suppose also that adjustment according to what I indicated is my preferred technique resulted in an adjusted rate for DG of 15%.  This would reflect a 50% reduction in the absolute difference between rates and a 50% reduction in both the relative difference in experiencing the outcome and the relative difference in avoiding the outcome.  But according to the approach on the Solutions sub-page of MHD. the initial disparity was .45 standard deviations and the adjusted disparity was .25 standard deviations.  Hence, with respect to what I maintain is the only meaningful indicator of the comparative status of the two groups, the disparity was decreased by only 44.4%.  But see note iv infra.

5.  Weighted average disparity compared with adjusting underlying rates

A number of works listed in Section A of the Measuring Health Disparities page address perceptions about racial differences in mortgage rejection rates, pointing out, for example, that banks that are most responsive to pressure to relax lending criteria will tend to have large rejection rate disparities (but small acceptance rate disparities) and that rejection rate disparities will tend to be high among high income groups (though acceptance rate disparities will tend to be low among such groups.[6,7]  See also Section B.2a of the Scanlan’s Rule page of jpscanlan.com.

One of the more comprehensive treatments of these issues may be found in an unpublished 1997 article styled “Confusion over Credit Discrimination.”   That article treats at greater length the study discussed in reference 6 that formed the basis for the suit against NationsBank discussed in that reference.  In doing so, the 1997 unpublished article treated what it termed an adjustment issue, noting that, rather than adjusting one or both group’s rates to take into account differing risk profiles reflected by different distribution in income categories (the subject of Section 3 supra), the study adjusted the size of the relative difference in mortgage rejection rates by weighting the relative difference in rejection rates in each income category by the number of blacks in each category.  The article went on explain that because blacks were disproportionately concentrated in the low income categories where rejection rates tended to be high and relative differences between rejection rates tended to be low, the adjustment tended to reduce the rejection rate disparity.  But had the study relied on acceptance rate rather than rejection rates, the adjustment technique would have increased rather than decreased the disparity as a result of the same disproportionate concentration of blacks in the low income categories where acceptance rate disparities tend to be large. [iii]

The point concerning an opposite effect with regard to acceptance rate disparities holds, at least in the main.  But what I failed to recognize at the time of that article, as well as when drafting earlier versions of this item, is that the procedure was not really adjusting for differing black/white distributions by income category.  Rather, it was providing an average disparity weighted according to the way blacks were distributed across income categories.  And whatever the effect of the weighting procedure, such effect occurred irrespective of any difference in the black and white distributions according to income category.  That is, the effect, which was driven by the black distribution, would have been the same regardless of whether the black and white distributions differed.  Thus, the procedure did not do any of the things that standard adjustment for differing distributions typically would do.   

Though the procedure was not accomplishing the adjustment for income that the study maintained it was accomplishing, that does not necessarily mean that the procedure did not produce some useful information (though I am not yet sure whether it does or does not).  But if a weighted aggregated disparity is useful information, the measure of the disparity within each category ought to be that discussed on Solutions sub-page of MHD (which is not affected by the overall prevalence of rejection/acceptance within the category).  See Section 4 supra.  The extent to which the adjustment would make any difference would turn on whether there are real differences in disparities within income categories (as distinguished from those that are functions of differing overall rejection/acceptance rates within each category).   Thus, if the strength of the forces causing the disparities are the same in each income category, the adjustment would have no effect at all.[iv]   See discussion on the Comparing Averages sub-page of the Scanlan’s Rule page of jpscanlan.com.

References:

1.  A study with a variety of problems.  Journal Review June 2, 2007 (responding to  Schulman KA, Berlin JA, Harless, et al.  The effect of race and sex on physicians’ recommendations for cardiac catheterization.  N Engl J Med 1999;340:618-26):

http://jpscanlan.com/images/Schulman_NEJM_1999.pdf

2.  Understanding social gradients in adverse health outcomes within high and low risk populations.  J Epidemiol Community Health May 18, 2006 (responding to Lynch J, Davey Smith G, Harper S, Bainbridge K. Explaining the social gradient in coronary heart disease: comparing relative and absolute risk approaches. J Epidemiol Community Health 2006:60:436-441): http://jpscanlan.com/images/First_Comment_on_Lynch_et_al._JECH_2006.pdf

3.  Study shows different adjustment approaches rather than different relative and absolute perspectives.  Journal Review May 1, 2008 (responding to Khang YH, Lynch JW, Jung-Choi K, Cho HJ. Explaining age-specific inequalities in mortality from all causes, cardiovascular disease, and ischaemic heart disease among South Korean public servants: relative and absolute perspectives. Heart 2008;94:75-82): http://jpscanlan.com/images/Khang_Heart_2009.pdf

4.  Effects of standard adjustment approaches on relative and absolute inequalities.  J Epidemiol and Community Health Nov. 2, 2009 (following up on item 2): http://www.jpscanlan.com/images/Second_Comment_on_Lynch_et_al._JECH_2006.pdf

5.  The Misinterpretation of Health Inequalities in the United Kingdom, presented at the British Society for Populations Studies Conference 2006, Southampton, England, Sept. 18-20, 2006:  http://www.jpscanlan.com/images/BSPS_2006_Complete_Paper.pdf

6.  When statistics lie. Legal Times Jan. 1, 1996: http://jpscanlan.com/images/When_Statistics_Lie.pdf

7.  Race and mortality.  Society 2000;37(2):19-35 (reprinted in Current 2000 (Feb)):

 http://www.jpscanlan.com/images/Race_and_Mortality.pdf