Does Classification of Composites for Network Meta-analyses Lead to Erroneous Conclusions?
Composites can be classified differently, according to manufacturer information, filler particle size, resin-monomer base, or viscosity, for example. Using clinical trial data, network meta-analyses aim to rank different composite material classes. Dentists then use these ranks to decide whether to use specific materials. Alternatively, annual failure rates (AFRs) of materials can be assessed, not requiring any classification for synthesis. It is unclear whether different classification systems lead to different rankings of the same material (ie, erroneous conclusions). We aimed to evaluate the agreement of material rankings between different classification systems. A systematic review was performed via MEDLINE, Cochrane Central Register of Controlled Trials, and EMBASE. Randomized controlled trials published from 2005-2015 that investigated composite restorations placed in load-bearing cavitated lesions in permanent teeth were included. Network meta-analyses were performed to rank combinations of composite classes (according to manufacturer, filler particle size, resin-monomers, viscosity) and adhesives. Material combinations were additionally ranked using AFRs. A total of 42 studies (6088 restorations, 2325 patients) were included. The ranking of most material class combinations showed significant agreement between classifications (R2 ranged between 0.03 and 0.56). Comparing material combinations using AFRs had low precision and agreement with other systems. AFRs were significantly correlated with follow-up periods of trials. There was high agreement between rankings of identical materials in different classification systems. Such rankings thus allow cautious deductions as to the performance of a specific material. Syntheses based on AFRs might lead to erroneous results because AFRs are determined by follow-up periods and have low precision.SUMMARY
Objectives:
Methods:
Results:
Conclusion:
INTRODUCTION
Placing restorations is still the most frequently performed and expensive dental treatment.1-3 Nowadays, the majority of these restorations are adhesively placed composites, with a vast number of materials from numerous manufacturers being available to dentists. These composites are tested clinically by a similarly large number of controlled trials. Although dentists will not be able to evaluate the findings of each trial, they usually require some kind of synthesis of study findings that summarizes the results and gives guidance as to which materials to use under certain conditions.
In such syntheses, outcomes of trials are usually compared by either pairwise meta-analysis (comparing, for example, the number of failures in one group vs the other and pooling the resulting risk ratios across trials) or via network meta-analysis using direct (ie, evidence stemming from comparisons made within a trial) and indirect evidence (ie, evidence stemming from comparisons between different trials), thereby allowing comparison of more than two groups.4,5 The result is a ranking, either of two comparators (“A is superior to B in condition C”) or multiple groups (“A has a lower risk of failure than B, which has a lower risk than C in condition D” and so on). Given the number of materials available, such syntheses require some kind of classification of materials both for statistical evaluation and for interpretation (comparing 20 or more material brands without summarizing them in classes will likely not allow robust synthesis and will also prohibit easy interpretation of findings).
For dental composites, a number of classification systems are available. Composites might be classified according to manufacturers' information (with some products being actively separated from the “field,” such as ormocers or siloranes). Alternatively, they might be classified on the basis of their filler particle size, their resin-monomer base, or their amount of filler particles, which is also correlated with viscosity. For syntheses, the same material can thus belong to different classes, and could—on the basis of these different classifications—be ranked differently. That could mean the same material (product) is found suitable by one analysis but not another, depending on the group of materials in which it is classified.6 It is thus unclear whether deductions can be made from such class rankings with regard to the performance of a specific material/product.
An alternative to the described meta-analytic syntheses is the comparison of annual failure rates (AFRs). This avoids classification, but pools findings from possibly very different trials (follow-ups, patient groups) via AFRs, which could heavily distort the results. For example, certain materials might have been used only in short-term trials in low-risk patients, whereas others have been investigated over longer periods in high-risk patients. This is likely to lead to higher AFRs in the latter group but does not necessarily prove this material to be performing worse than the material in the former. Whereas this might also be the case in pairwise or network meta-analysis, the risk of distortion is lower here, given that comparisons are made within trials first (yielding risk ratios, as described) before pooling.
In summary, using different classification systems of dental composites might lead to different results and possibly erroneous conclusions. Our aim was to assess the agreement of materials' rankings with regard to risk of failure based on meta-analyses using different classification systems. Moreover, we aimed to evaluate whether such rankings are reflected by AFRs, and whether AFRs are precise (ie, reproducible in different trials) or depend on other factors (such as a follow-up period). Our research question was “For identical material (product) combinations, what is the agreement in rankings yielded by network meta-analyses using different composite classifications, and how do these rankings agree with rankings based on AFRs?” We hypothesized that there was no significant agreement of rankings yielded by different classification systems.
METHODS
Study Design
We built on data previously collected as part of a systematic review of randomized trials on dental restoration materials including composites.6,7 The therein-investigated composites were classified and combinations of composite classes and adhesives submitted to network meta-analyses, yielding a ranking of each material class combination according to its risk of failure. We then compared how identical material (product) combinations were ranked using different classification systems and investigated the agreement of rankings. We further compared these rankings with rankings based on AFRs. Our analysis exemplarily focuses on composites placed in load-bearing cavitated lesions of permanent teeth.
Systematic Review
The performed systematic review had included randomized, controlled clinical trials (RCT) published from 2005-2015 comparing the survival of two or more different composite and/or adhesive materials. RCTs were excluded if they compared different treatment techniques (eg, conventional vs atraumatic restorative treatment), not materials, or placed restorative materials or adhesives as a sealant or for orthodontic bonding, not a restoration. To ensure uniform randomization across trials, we only included trials reporting on survival of composite restorations placed in adult or adolescent (ie, children with permanent teeth) patients in need of restoration of load-bearing cavities in posterior permanent teeth.
The Cochrane Central Register of Controlled Trials, MEDLINE (via PubMed) and EMBASE (via OVID) were searched on March 2, 2015. The search strategy and screening procedure is shown in appendix Figure S1. Screening, inclusion, and data extraction had been performed independently by two reviewers. Consensus was obtained by discussion. Full details regarding the performed review can be found elsewhere.6,7
Classification Systems
Composite materials are placed using adhesives. Therefore, all trials essentially compared combinations of adhesives and composites. Adhesives were classified as follows: 1) 4- or 3-step etch-and-rinse adhesives (3ER); 2) 2-step etch-and-rinse adhesives (2ER); 3) 2-step self-etch adhesives (2SE); 4) one-step self-etch adhesives (1SE). For composites, four classification systems were used, based on
-
Manufacturers' classification. This system had been used in previous publications on this issue6,7 and categorized composites as 1) conventional composites (CC); 2) ormocer composites; 3) bulk-fill composites (both flowable and packable bulk fills); and 4) siloranes.
-
Filler composition (ie, hybrid or single particle size) according to the manufacturer's information: 1) microhybrid, 2) nanohybrid, 3) microfilled, and 4) nanofilled.
-
Resin-monomer base regarding the main component of the resin matrix: 1) conventional monomers (eg, urethane dimethacrylate, bisphenol A glycol dimethacrylate, triethylene glycol dimethacrylate), 2) ormocers, and 3) siloranes.
-
Viscosity according to the manufacturer's classification: 1) composites with conventional viscosity, 2) packable composites, and 3) flowable composites. Where composites were neither classified as packable nor as flowable, we assumed they had conventional viscosity.
If two composite materials had been used in the same cavity (as for some bulk fills, with bulk material being covered by another composite), the material within the focus of the study (in this case, always the bulk fill) was used for classification. Combining adhesive and composite material allowed the classification of each placed material combination (eg, a microhybrid composite placed using a 3ER). Given the different number of classes, the classification systems were differently granular (eg, the filler-based classification included 13 class combinations, whereas the classification according to the resin component included only seven combinations). The full list of assessed materials and their classification can be found in appendix Table S1.
In addition, composites were not classified at all, and instead combinations of each composite with the used adhesive system class were constructed (eg, Tetric Ceram placed using a 3ER). This resulted in 71 unique combinations of composite product and adhesive class, which were used for analyses based on AFRs.
Data Syntheses
Our outcome parameter for syntheses was risk of failure (ie, restorations needing any restorative reintervention—that is, replacement or repair). That included retention loss, but also any United States Public Health Services (USPHS) ratings of charlie or delta, for example. Our analysis only accounted for restorations that were followed up, with risk of failure per study being derived as events (failures) per total sample followed. The unit of analysis was restorations. Note that most trials were clustered (ie, patients contributed more than one lesion), which usually calls for some adjustment because resulting confidence intervals of effect estimates (such as risk ratios) are otherwise artificially narrow. We did not perform such adjustment, because our aim was not to compare materials but to assess the agreement between rankings yielded by different classification systems. Moreover, we did not present confidence or credible intervals, because we focused on rankings.
Therefore, network meta-analyses (NMA) were used for estimating class combination ranks, with Bayesian random-effects models and a Markov chain Monte Carlo simulation being conducted via the Bayesian software package GeMTC 0.6 and WinBugs8 implemented in R 3.0.3 (R Foundation, Vienna, Austria). For each classification system, one separate NMA was performed. Binomial likelihood was used to model the data.9,10 To fit the model, we used a noninformative prior for the basic parameters from a normal distribution N(0,104) and a flat prior U(0,2) for the random-effects standard deviation. The convergence was assessed on the basis of the Brooks-Gelman-Rubin criteria11 and inspection of trace plots. The first 50,000 iterations were discarded as “burn-in” and then a further 50,000 iterations were undertaken for two chains at thinning intervals of five. Different class combinations were ranked according to their probability of having the lowest vs the highest risk of failure,12 and the average rank was calculated. The surface under the cumulative ranking (SUCRA) line was plotted and the area under the plot (SUCRA value) calculated. SUCRA values were eventually used to rank class combinations (eg, from 1 to 13 for the filler-based classification system). Loop inconsistency (ie, the difference between direct and indirect estimates for different treatments within a loop) was evaluated by the Inconsistency Factor (IF) for the loop.13,14 Within each loop, the IF value is defined as IF = Edirect − Eindirect (E: estimate). We rejected the null hypothesis that evidence is consistent (
) when the IF was significantly greater or smaller than 0.6 Note that usually, performing pairwise meta-analyses is recommended alongside network meta-analyses. No such pairwise meta-analyses were performed, given that we were not interested in determining material efficacy but to assess agreement of material combinations.
In addition to NMA, material combinations were ranked according to their AFRs. Materials with AFRs of 0 were all ranked first, resulting in 62 ranks of the 71 individual material combinations.
Assessment of Agreement
Agreement of rankings of the same composite-adhesive combination was assessed twofold: First, graphical assessment using heat maps was performed, assigning a color code to each rank, homogeneously distributing different red and green hues between the lowest and the highest ranks. This allowed us to overcome the issue of a different number of ranks in different classification systems. Similarly, statistical assessment using the Kendall correlation coefficient allowed us to evaluate agreement regardless of the absolute number of ranks. A possible association of AFRs with follow-up periods was assessed using Pearson correlation.
RESULTS
Search and Studies
In the original review, 114 studies (147 articles) were included. From these, we used 42 studies (with 103 comparator groups), solely focusing on composites being placed in load-bearing cavities of permanent teeth. In three comparator groups, a combination of two materials was used for placement of the restorations. After a mean follow-up period of 42 months (range, 12 to 120 months), 4820 restorations had been assessed (Figure S1, Table S2).
Ranking of Different Material Class Combinations
Using manufacturers' classification of composites, nine different material class combinations were assessed (Figure 1a). When ranking them according to their probability of failure (Figure 1b) and the resulting SUCRA value, CC placed using 3ER or 2SE were ranked highest, whereas siloranes placed using 2SE showed the lowest ranking (Figure 1c). There was statistical inconsistency in one assessed loop (finding 2ER/CC nonsignificantly inferior to 1SE/CC using direct comparison [ie, comparisons within a trial], and vice versa using indirect comparison [ie, comparisons between different trials]; the level of inconsistency was limited with p=0.04). Because these material combinations were ranked fourth and fifth (ie, very similarly), the resulting uncertainty was limited.
Citation: Operative Dentistry 43, 2; 10.2341/16-344-LIT
Classifying composites according to their filler particle size (Figure 2a) and ranking them according to risk of failure (Figure 2b), nanohybrids placed using 2SE or 1SE as well as nanofilled composites placed using 2SE were ranked highest, whereas microhybrids placed using 2SE were ranked lowest (Figure 2c). There was no statistical inconsistency.
Citation: Operative Dentistry 43, 2; 10.2341/16-344-LIT
Classification according to the main resin component (Figure 3a) and subsequent ranking (Figure 3b) found composites containing conventional resins placed with 3ER or 2SE ranked highest; silorane resin composites placed using their 2SE adhesive were ranked lowest (Figure 3c). Again, there was no statistical inconsistency.
Citation: Operative Dentistry 43, 2; 10.2341/16-344-LIT
Classification according to viscosity found packable or conventionally viscous composites placed using 3ER ranked highest, and conventionally viscous or packable composites placed with 2ER on the lowest rank (Figure 4a-c). There was statistical inconsistency (p=0.01), with direct evidence finding conventionally viscous composites placed using 1SE nonsignificantly more suitable than those placed using 2SE, whereas indirect evidence found the opposite. Given that those class combinations were ranked seventh and eighth, the impact of this inconsistency was limited.
Citation: Operative Dentistry 43, 2; 10.2341/16-344-LIT
Agreement of Rankings
Using a heat map (Figure 5), we found materials being ranked highly in one classification system to be ranked similarly in other systems in most cases (details can be found in the Appendix Table S3). One notable exception was composites classified as microhybrids (for example, Filtek Z250) placed using 2SE (for example, Clearfil SE Bond); these were ranked poorly, whereas the same material combination (Z250 + Clearfil SE) was ranked highly in other classifications. The reason for this discrepancy was that microhybrids placed with 2SE included silorane-based composites, which performed rather poorly and lowered the ranking of this class combination.
Citation: Operative Dentistry 43, 2; 10.2341/16-344-LIT
The relatively high agreement was reflected in correlation coefficients as well (Table 1), which found a significant association (p<0.05) of ranks yielded by different classification systems with only one exception (agreement between classification according to the manufacturer and classification according to viscosity). However, R2-values were low for most of the groups, indicating low confidence for the found agreements between classification systems.
Limited agreement was found when comparing NMA rankings with AFR rankings (Figure 5), with AFR-based ranks mostly not showing significant association to ranks yielded by other classification (p>0.05) except for a borderline-significant agreement between AFR ranking and NMA rankings according to filler (p=0.04). When evaluating AFRs, it was apparent that the same material combinations could have been ranked very differently depending on the study (for example, Surefil placed using Prime+Bond NT was ranked first as well as 55th), highlighting the limited precision of AFR-based ranking. For material combinations that had been investigated in a minimum of three studies, we assessed the association between AFR and follow-up, and found significant associations for nine of 12 material combinations (the mean correlation coefficient between follow-up and AFR was 0.45).
DISCUSSION
Dentists will use syntheses of original data or their derivatives (such as guidelines) for clinical decision-making rather than consulting all available individual trials. These syntheses usually build on some kind of classification system both for statistical pooling and ease of interpretation. Using the resulting ranking of a specific material class, dentists might decide to use or not to use a specific material (product). On the basis of our findings, this is largely justified, because a material (product) combination that is ranked highly in one classification system is likely to be ranked highly in another classification system and vice versa. Given our findings, we refute our hypothesis of no significant agreement between classification systems.
There are several possible reasons underlying this agreement. First, it might be that there were too few materials in each class, leading to one or few materials dominating each class, with limited risk of distortion via pooling materials. Second, it is possible that material properties such as filler particle size or resin-base (which are reflected by classifications) are indeed the true drivers of performance: This would be encouraging, given that these properties can be technically altered and are the focus of large parts of dental materials science. If material properties are in fact determining a material's performance, the present review would allow the assessment of which combinations are better suited than others for restoring load-bearing cavities in permanent teeth. Third, it can be assumed that the used adhesive partially determined the ranking of a class combination. Because the adhesive system classes remained stable regardless of the applied composite classification, the found agreement might partially stem from there. We expect a combination of these factors to contribute to the recorded agreement. For other medical disciplines, it can be shown that variability in definitions of treatment classifications (ie, variations due to differences in dose or administration routes in trials on drugs) can affect the ranking of the resulting treatment nodes within NMA.15,16 However, the agreement of material class rankings yielded by different tested classification systems was relatively high within our study, as long as ranking was not based on AFRs.
Ranking materials based on AFRs did not correlate significantly with most other rankings. That could mean that most other rankings do not reflect the true performance of a material, as AFRs might be expected to do. However, when closely inspecting AFRs of identical material combinations, we found these to vary widely (up to 5.5% for the same material). This highlights the limited precision of AFRs and AFR-based rankings. Moreover, we found AFRs to correlate significantly with a trial's follow-up period: Longer trials recorded significantly more failures, which increases AFRs compared with short-term trials. Although we did not assess the possible underlying reasons (eg, higher risk of failure later in a restoration's lifetime, long-term trials being from different settings or in different patients), it is clear that pooling such AFRs might not only reflect a material's performance but also the design and conduct of the trial. On the basis of these findings, we cannot recommend continuing to synthesize data on clinical restoration performance using AFRs.
This study has several limitations. First, a number of confounders (such as the adhesive system used) might affect a material's performance and could artificially increase agreement between classes. However, even if this were the case, dentists indeed place combinations of composite and adhesive; any classification that truly predicts the performance of such combination is thus clinically relevant. Moreover, this element of bias would be present in AFR-based rankings, too (where it did not seem to increase agreement). Second, given the limited differences in SUCRA values between many material class combinations, there seems to be a relatively high degree of uncertainty. Several confounders (such as differences between batches of the same material, inclusion of studies that used material combinations) might have contributed to this uncertainty. However, some materials seem to be more suitable than others. For example, and regardless of the classification systems, composites containing ormocers and siloranes were ranked lower, whereas nanohybrids were generally ranked high. It should be noted that despite the discussed uncertainty, we did not find statistical inconsistency to greatly affect our findings. This confirms that the constructed networks were relatively robust and our findings not mere statistical artefacts. Third, we included only randomized trials. Whereas the internal validity of such trials is higher than that of observational trials (mainly due to reduced selection bias), they are usually performed under controlled and somewhat artificial conditions and suffer from limited follow-up periods. Future analyses might consider including controlled nonrandomized trials. Moreover, we included only recently published trials, with an artificial inclusion cutoff of 10 years (2005-2015). That was justified because many materials were only recently available, and pooling these with much older materials might lead to distortion. However, one should note that some included more recent trials followed up older materials after 10 years or longer. Almost half of the included trials had a follow-up period of two years or less. Given that such short-term failures are often related to retention loss, which itself is associated with the adhesive bond, it is likely that the adhesive had a significant influence on the ranking of the class combinations in this study. Long-term studies (which were rare in our review), in contrast, might show 1) a higher rate of complications and 2) different types of complications.17 In this case, the adhesive as determinant for the material class ranking might play a more limited role. Last, the assessed outcome parameter was failure. It is unclear whether using other components of available outcome measures like the USPHS criteria (such as surface properties, or secondary caries) will affect ranking agreements.
CONCLUSIONS
On the basis of our findings, the agreement of material class rankings yielded by different classification systems was relatively high. Nevertheless, interpretation of material class performances should be made with caution, because the same material might be ranked differently depending on the classification system used. Comparing materials solely on the basis of AFRs might lead to erroneous results, given that AFRs are determined by follow-up periods rather than material performance.
Results if composites are classified according to manufacturers as conventional composites (CC), bulk fills (BF), ormocers (OR), siloranes (SI). Adhesives were categorized as follows: 3ER, 4- or 3-step etch-and-rinse; 2ER, 2-step etch-and-rinse; 2SE or 1SE, 2- or 1-step self-etch. (a): Networks of composite adhesive class combinations. The size of the nodes is proportional to the number of placed restorations. The width of the lines is proportional to the number of trials comparing the connected treatments. (b): Ranking of composite adhesive class combinations according to their chance of restoration survival. (c): Mean rank, surface under the cumulative ranking (SUCRA) value and resulting overall rank of different class combinations.
Results if composites are classified according to filler particle size: Microhybrids (MiHy), nanohybrids (NaHy), microfilled composites (MiCo) or nanofilled composites (NaCo). Adhesives were categorized as follows: 3ER, 4 or 3-step etch-and-rinse; 2ER, 2-step etch-and-rinse; 2- or 1SE, 2- or 1-step self-etch. (a) Networks of composite adhesive class combinations. (b) Ranking. (c) Mean rank, surface under the cumulative ranking (SUCRA) value and resulting overall rank of different class combinations.
Results if composites are classified according to resin-monomer base: Conventional monomers (Con), ormocers (OR), siloranes (SI). Adhesives were categorized as follows: 3ER, 4 or 3-step etch-and-rinse; 2ER, 2-step etch-and-rinse; 2- or 1SE, 2- or 1-step self-etch. (a) Networks of composite adhesive class combinations. (b) Ranking. (c) Mean rank, surface under the cumulative ranking (SUCRA) value and resulting overall rank of different class combinations.
Results if composites are classified according to their viscosity: Conventional viscosity (CoV), packable (Pac), flowable (Flo). Adhesives were categorized as follows: 3ER, 4 or 3-step etch-and-rinse; 2ER, 2-step etch-and-rinse; 2- or 1SE, 2- or 1-step self-etch. (a) Networks of composite adhesive class combinations. (b) Ranking. (c) Mean rank, surface under the cumulative ranking (SUCRA) value and resulting overall rank of different class combinations.
Heat map of agreement of ranks. Combinations of composite and adhesive classes were used to yield ranks. Green indicates a class combination was ranked high, whereas red indicates a low rank. Four composite classification systems were assessed: manufacturer classification (M), or according to filler particle size (F), resin-monomer base (R), viscosity (V). In addition, rankings based on annual failure rates (AFR) were estimated. Details can be found in Appendix Table S3.
Contributor Notes