Back to Table of Contents

A. Disclaimer Effect

An important standard for an effective scheme is that the disclaimers (language or grades) have an impact on ratings of scientific evidence. For a given product, ratings of the level of scientific evidence for unqualified claims should be greater than those for qualified claims, i.e., those bearing disclaimers indicating that they are not supported by the highest level of scientific evidence. Thus, in an effective disclaimer scheme, the differences between unqualified claims and qualified claims should be negative and statistically significant.

Figure 1 displays the means for ratings on scientific evidence for each of the three schemes for foods. Figure 2 displays the means for dietary supplements. Ratings under the current scheme of SSA claims and allowed word-only QHCs (Scheme 1) are shown in red, ratings under a scheme where SSA claims are written using qualifying language (Scheme 2) are shown in black, ratings of a grading scheme (Scheme 3) are shown in blue, and ratings for unqualified claims, i.e., those without a disclaimer, are shown in green. For a given claim, any deviation of each of the three schemes from the unqualified claim indicates the impact of disclaimers on ratings for level of scientific evidence. In each Figure, claims are ordered by intended level of scientific evidence from strongest (A₍₁₎, A₍₂₎) on the left to weakest (D₍₁₎, D₍₂₎) on the right. 

Figure 1. Mean Ratings of Scientific Evidence of Claims for Foods (1=weak, 5=strong)

Figure 2. Means of Ratings of Scientific Evidence of Claims for Dietary Supplements (1=weak, 5=strong)

The graphs reveal that the disclaimers appear to have an effect. Some of these effects may be appropriate, in that the disclaimers reduce the ratings of scientific evidence, and others may not be appropriate, in that the disclaimers increase ratings of scientific evidence supporting the claims.

The results of planned comparisons of ratings are shown in Table 4 (for foods) and Table 5 (for dietary supplements). For foods, it appears that Schemes 1 and 2 display appropriate disclaimer effects, with A-level qualified claims not significantly different from unqualified claims (Scheme 2) and claims of all other levels eliciting significant and negative impacts on ratings for scientific evidence. Many of the effects in Scheme 3 are appropriate. However, in this scheme, the B-level claim elicits no significant difference in ratings between an unqualified and a qualified claim.

Table 2. Differences in Mean Ratings of Scientific Evidence: Qualified against Unqualified Claims for Foods

*In Scheme 1, A₍₁₎ and A₍₂₎ are unqualified (SSA) claims, thus no comparisons are made.

For dietary supplements, the results are mixed. In the word-only Schemes 1 and 2, the disclaimer effect is neither readily apparent nor consistent. Only the A₍₂₎-level and D₍₁₎-level claims show any statistically significant differences between unqualified and qualified claims. In Scheme 3, all disclaimer effects are significant or marginally significant. However, the disclaimer effects on the A-level and B-level claims are positive, meaning that the disclaimer increases, rather than decreases (the intended effect), the ratings of scientific evidence.

Table 3. Differences in Mean Ratings of Scientific Evidence: Qualified against Unqualified Claims for Dietary Supplement

*In Scheme 1, A₍₁₎ and A₍₂₎ are unqualified (SSA) claims, thus no comparisons are made.

B. Distinct Levels of Scientific Evidence

An effective scheme for conveying scientific evidence enables consumers to discern distinctions between the various levels of scientific evidence. Using the levels of claim shown in Table 1, this implies that ratings for scientific evidence should follow a pattern of {A₍₁₎, A₍₂₎} > B > C > {D₍₁₎, D₍₂₎}.

In Figure 1, mean ratings of scientific evidence for claims on foods for the three schemes exhibit downward sloping patterns, an appropriate trait for an effective scheme as claims have been ordered by strength of scientific evidence from strongest to weakest.

In addition, a scheme that effectively conveys distinct levels of scientific evidence will elicit statistically significant and positive differences in mean ratings between one level and the adjacent level immediately below it. Table 4 shows the results of planned comparisons for ratings between adjacent levels of scientific evidence (e.g., A vs. B) for foods. The results indicate that each of the three schemes exhibits some ability to convey distinct levels of scientific evidence for the claims. Scheme 1, the current scheme employed by FDA, is somewhat effective at conveying distinct levels, with most differences between ratings being positive and significant (p<0.05). However, the difference between A-level and B-level claims are not particularly sharp, with one difference (A₍₁₎ vs. B) being only marginally significant. There also appears to be large variations in magnitude and significance of the differences between C-level and D-level claims. 

Table 4. Differences in Mean Ratings of for Scientific Evidence: Adjacent Levels of Qualified Claims for Foods

Scheme 2 shows little to no improvement on this standard over Scheme 1. Although one difference becomes sharper (A₍₂₎ vs. B), another difference becomes less significant (B vs. C). Scheme 3 appears to be the most effective at conveying distinct levels of scientific evidence, with most differences significant (p<0.001). However, it is clear from the lack of significance for A₍₂₎ vs. B that a grading scheme may not be consistent in its ability to convey substantive differences between a claim that meets a standard of significant scientific agreement and one that does not.

Similar results for dietary supplements are shown in Figure 2 and Table 5. Again, the means show generally downward sloping trends for the three schemes. However, statistical tests indicate that the schemes vary in their effectiveness for conveying levels of scientific evidence. Scheme 1 conveys no statistically significant distinction between A and B-level claims; Scheme 2 conveys a marginally significant distinction between A and B-level claims. Both schemes convey only a marginally significant distinction between B and C-level claims. In the context of dietary supplements, this implies that these two schemes essentially convey two levels of scientific evidence (A, B, and C being one and D being the other) rather than four and do not convey substantive differences between a claim that meets a standard of SSA and one that does not (i.e., B-level claim). Scheme 3, however, shows robust effectiveness at conveying four distinct levels of scientific evidence for claims on dietary supplements.

Table 5. Differences in Mean Ratings of Scientific Evidence: Adjacent Levels of Qualified Claims for Dietary Supplements

C. Perceived Health Benefits and Other Attributes of a Product

The effect of disclaimer schemes on respondent perceptions of a product’s health benefits should be consistent with the effect on perceptions of level of scientific evidence. Three aspects of the product’s health benefits are measured: (1) the likelihood that the product will reduce the risk of the disease mentioned in the claim (i.e., heart disease or cancer); (2) the likelihood that the product will reduce the risk of a disease not mentioned in the claim (e.g., osteoporosis); and (3) the importance of the product for a healthy diet. Other, more global perceptions that are not necessarily related to the healthfulness of the product are also examined; these are product quality and safety.

Health claims convey to consumers information about specific potential health benefits of consuming a product. For the two first measures, respondents are asked questions related to those specific benefits. As such, ratings on the likelihood of those benefits should be related to the strength of scientific evidence concerning the relationship between product consumption and disease risk. Disclaimers on qualified health claims should mitigate the expected impact of the unqualified claims on ratings of the likelihood of reducing the risk of the disease mentioned in the claim. Additionally, disclaimers should not increase ratings of the likelihood of reducing the risk of diseases not mentioned in the claim. Therefore, it is appropriate to compare product ratings of health benefits between a qualified claim and an unqualified claim within each scheme.

For other global product attributes that are not related to the nature of the health claim, such as importance of a product to a healthy diet, product quality and safety, respondent perceptions should not be dependent on the presence of a health claim or disclaimers on those claims. Thus comparisons of product ratings are made between each claim condition and a no claim control. This approach facilitates a comparison of the impact of the three schemes on general perceptions of products bearing claims and disclaimers.

Likelihood of Reducing Risk of Disease Mentioned in Claim

Respondents to the study were asked to rate the likelihood that consumption of the product would reduce the risk of the disease mentioned in the claim: heart disease for A, B, and C-level claims and cancer for the D-level claims, respectively. Results of planned comparisons between each claim and an unqualified claim on this measure for foods are shown in Table 6.

 Table 6. Differences in Mean Ratings of Likelihood of Reducing Risk of Claim Disease: Qualified Claims against Unqualified Claims for Foods

*In Scheme 1, A₍₁₎ and A₍₂₎ are unqualified (SSA) claims, thus no comparisons are made.

A scheme with a parallel effect on ratings of specific health benefits and ratings of scientific evidence should exhibit significant and negative differences for claims bearing disclaimers. In Schemes 1 and 2, only D-level disclaimers exhibit significant and negative effects on ratings of the likelihood that the product will reduce the risk of the disease. In Scheme 3, a D-level disclaimer has milder and inconsistent negative effects on this rating. However, in Scheme 3, a B-level disclaimer has a highly significant and positive impact on this rating relative to that of an unqualified claim. Here, the disclaimer increases, rather than reduces, perceived likelihood that the product will reduce the risk of a disease mentioned in a claim.

Similar results are found within dietary supplements (shown in Table 7). In addition, A-level disclaimers appear to have a positive, though marginal, impact on this measure in Scheme 2, but a strong positive impact in Scheme 3.

Table 7. Differences in Mean Ratings of Likelihood of Reducing Risk of Claim Disease: Qualified Claims against Unqualified Claims for Dietary Supplements

*In Scheme 1, A₍₁₎ and A₍₂₎ are unqualified (SSA) claims, thus no comparisons are made.

Likelihood of Reducing Risk of Disease Not Mentioned in Claim

Respondents to the study were also asked to rate the likelihood that consumption of the product would reduce the risk of diseases not mentioned in the claim: cancer and osteoporosis for A, B, and C-level claims, and heart disease and osteoporosis for the D-level claims, respectively. Ratings for diseases within a given product category (food or dietary supplement) were highly correlated, thus an average rating across both non-claim diseases for each product was calculated. Planned comparison tests of ratings between qualified and unqualified claims for foods are shown in Table 8.

Table 8. Differences in Mean Ratings of Likelihood of Reducing Risk of Non-Claim Diseases: Qualified Claims against Unqualified Claims for Foods

*In Scheme 1, A₍₁₎ and A₍₂₎ are unqualified (SSA) claims, thus no comparisons are made.

In Schemes 1 and 2, none of the disclaimers has a significant impact on ratings of the likelihood that the product will reduce the risk of a disease not mentioned in a claim. In Scheme 3, all disclaimers, with exception of the B-level grade disclaimer, have significant and negative impacts on the ratings. As stated earlier, effective disclaimers should not lead to an increase in ratings on the likelihood that the product will reduce the risk of diseases not mentioned in the claim. In Scheme 3, it appears that disclaimers reduce the likelihood that consumers make inappropriate inferences about products reducing the risk of diseases beyond those mentioned in the claim.

Table 9. Differences in Mean Ratings of Likelihood of Reducing Risk of Non-Claim Diseases: Qualified Claims against Unqualified Claims for Dietary Supplements

*In Scheme 1, A₍₁₎ and A₍₂₎ are unqualified (SSA) claims, thus no comparisons are made.

For dietary supplements, only D-level disclaimers in Schemes 1 and 2 have marginal impacts on ratings of the likelihood that the product will reduce the risk of diseases not mentioned in the claim. In Scheme 3, both B and D-level disclaimers significantly reduce ratings on this measure below that of the unqualified claim. Similarly with foods, Scheme 3 disclaimers reduce likelihood that consumer make inappropriate inferences about dietary supplements reducing the risk of diseases beyond those mentioned in the claim.

Importance to a Healthy Diet

The means for ratings of the food products’ importance to a healthy diet are presented graphically in Figure 3. Visually, it appears that each scheme is equivalent and has minimal impact on the ratings as compared to the no claim control.

Figure 3. Means of Ratings of Importance to a Healthy Diet for Foods (1=not important, 5=very important)

The results of planned comparison tests presented in Table 10 confirm this observation. One claim (D₍₂₎ in Schemes 1 and 2) has a significant impact on the ratings. Although this impact is consistent with a low level of scientific evidence for the health claim, the rating is for a global product assessment, which should not be affected by the claims and disclaimers.

Table 10. Differences in Mean Ratings of Importance to a Healthy Diet: Health Claims against No Claim Controls for Foods

The means for ratings of the dietary supplement products’ importance to a healthy diet are presented graphically in Figure 4. Unlike foods, it appears that each scheme may have some impact on the ratings.

Figure 4. Means of Ratings of Importance to a Healthy Diet for Dietary Supplements (1=not important, 5=very important)

The results of planned comparison tests presented in Table 11 show that, in Schemes 1 and 2, the C-level claim elicits significantly higher ratings on this measure than the no claim control. This finding is surprising, as one might expect a concurrent finding of significant and positive impacts on this measure from higher level (A and B) claims. In Scheme 3, only A-level claims elicit significantly positive differences in ratings for the importance to a healthy diet.

Table 11. Differences in Mean Ratings of Importance to a Healthy Diet: Health Claims against No Claim Controls for Dietary Supplements

Other Product Attributes

In addition to the three questions pertaining to health benefits of the products, respondents were also asked to rate the product in terms of quality and safety. Ratings of products on these measures should not be affected by claims and disclaimers about diet-disease relationships.

Product Quality

The results of planned comparisons of ratings on product quality for various schemes between conditions bearing a claim and no claim controls are shown in Table 12 for foods and Table 13 for dietary supplements. The results show that all of the tested schemes have some effect on ratings of product quality, especially for dietary supplements.

For foods, across the three schemes, D-level claims have a significant and negative impact on most ratings of product quality as compared to the no claim control. The impact is not consistent in Schemes 1 and 2 in that only one of the claims (D₍₂₎) exhibits any effect on ratings. The effect of the D-level claim on ratings of quality is mitigated somewhat in Scheme 3, with smaller effects of lesser statistical significance. The effect in Scheme 3 is also consistent across both D-level claims.

Table 12. Differences in Mean Ratings of Product Quality: Health Claims against No Claim Controls for Foods

For dietary supplements, the impacts on product quality are mixed. Negative effects are mostly significant for D-level claims across the three schemes. In contrast to the effect on foods, Schemes 1 and 2 have consistent effects across both D-level claims, whereas Scheme 3 has an impact on only one of the D-claims. Additionally, there are several significant and positive effects of claims on ratings of product quality. A-level claims in Schemes 2 and 3 elicit higher ratings for product quality than no claim controls. C-level claims in Schemes 1 and 2 also elicit higher ratings on product quality, however there is no concurrent and parallel effect from A and B-level claims.

Table 13. Differences in Mean Ratings of Product Quality: Health Claims against No Claim Controls for Dietary Supplements

Product Safety

The results of planned comparisons of ratings on product safety for various schemes between conditions bearing a claim and no claim controls are shown in Table 14 for foods and Table 15 for dietary supplements. In general, the results show that none of the schemes has a substantial impact on ratings of product safety, except for the A₍₂₎ claim in Scheme 3 on a dietary supplement.

Table 14. Differences in Mean Ratings for Product Safety: Health Claims against No Claim Controls for Foods

Table 15. Differences in Mean Ratings of Product Safety: Health Claims against No Claim Controls for Dietary Supplements

Additional Disclaimer Language

In addition to the grade scheme disclaimers, an additional disclaimer was tested stating that the grades for scientific evidence were “not ratings for the product quality safety, or overall healthfulness.” The addition of this disclaimer has no statistically significant effect on any of the ratings, including scientific evidence, quality, or health benefits.