(1) Criteria for reference selection

References that satisfied all of the following 3 criteria were included:

1. 1) References using Kampo formulations approved for manufacture and sale in Japan
   (excluding in-house formulations such as decoctions, because of unknown quality of the medicines used)
2. 2) Randomized controlled trials (R CTs), quasi-randomized controlled trials (quasi-RCTs), crossover trials, and meta-analyses
   (including some with randomization procedure not fully indicated. Crossover trials are regarded as RCT)
3. 3) References published in or after 1986
   Those published in or after 1986 obviously using formulations of 1985 or previous quality, which differs from the current quality levels, were excluded throughout the study period.

(2) Search and screening

Searches were performed using the two databases listed below, with additional reports collected by hand searches. Screening was performed in 2 steps: first, the references that obviously did not satisfy the criteria were excluded by the search staff; then, the remaining references were reviewed to finally select the ones to undergo the process of structured abstract preparation as described below.

1) The Cochrane CENTRAL (C)

2) Ichushi Web (I)

3) Hand searches conducted at the Japan Kampo Medicines Manufacturers Association (N)

(3) Preparation of structured abstracts

The references satisfying the inclusion criteria were compiled as structured abstracts (SA). Studies on SA started in the 1980s. Here, an 8-item structured abstract format of RCTs, as proposed by Altman, et al. and currently used worldwide, was employed.

Altman DG, Gardner MJ. More informative abstracts. Ann Inter Med 1987; 107(5): 790-1.

Aoki T. Kozoka shoroku no kisochisiki (Basic understanding in structured abstracts). In: Tsutani K, Yamazaki S, Sakamaki H (eds). EBM no tameno joho senryaku – ebidensu wo tsukuru, tsutaeru, tsukau – (Information strategies for EBM – generate, transfer and use evidence–). Tokyo: Chugai-igakusha; 2000: 82-93 (in Japanese).

Here, 8 items are referred to as, 1) objectives, 2) design, 3) setting, 4) participants, 5) intervention, 6) main outcome measures, 7) main results, and 8) conclusions.

These 8 items are widely used in medical journals such as JAMA and secondary information journals such as Evidence Based Medicine, ACP Journal Club, as well as in secondary information journals on traditional medicine and complementary and alternative medicine, represented by Focus on Alternative and Complementary Therapies (FACT). The acupuncture part of the journal is available in Japanese (Tsutani K [supervise-trans]. Hari no ebidensu [Evidence for acupuncture – abstracts of articles on clinical evaluation of acupuncture–]. Yokosuka: Ido-no-Nippon-sha; 2003 [in Japanese]. Thereafter, serialized in the journal “IDO-NO NIPPON [The Japanese Journal of Acupuncture & Manual Therapies]” [in Japanese]).

If one SA has been based on multiple references, all the bibliographic information for references used for the SA appears at the top of the SA, in order of publication, with the principle reference appearing in bold type.

Regarding 5) intervention, since the quality may differ among manufacturers, the brand name indicated in the original article was to be used as a rule. When the brand name was changed after the issue of the article for such reason as a change in manufacturer name, the brand name indicated in the article was used.

In the structured abstracts in the “Evidence Reports of Kampo Treatment”, in addition to the above-mentioned 8 worldwide standard items, the following 4 items are included: 9) from the Kampo medicine perspective; 10) safety assessment in the article; 11) abstractor’s comments; and 12) abstractor and date. These are described below:

9) “From the Kampo medicine perspective” means how the unique diagnosis system of Kampo medicine is used. This is applied to 2 processes: design of a clinical trial and analysis after completion of the study. With RCT, this can be referred to as pre-randomization and post-randomization. In the former process of designing clinical trial design, “sho (証, pattern)” of Kampo medicine is indicated in the entry criteria and exclusion criteria of participants in the protocol in a manner that participating investigators can understand. The latter process involves stratified analysis, in which       participants are stratified by “sho” (with sho or without sho), as well as by age and sex, etc. However, the stratified analysis is associated with “inference multiplicity”; that is, repeated testing of many strata produces false positive results, which indicates a difference when actually there is no difference. Among post-hoc approaches are adjustment for covariates.

10) Safety assessment in the article” was incorporated since not only efficacy but also safety should always be considered in Kampo medicine as well. Here, the expression of “safety assessment in the article” was used rather than mere “safety assessment” because “safety assessment” is frequently misunderstood. RCTs usually use efficacy-related endpoints to determine appropriate sample sizes, and are not intended to assess safety. For instance, when no adverse drug reactions occurred among 100 subjects receiving a Kampo medicine, the Kampo medicine is apt to be considered “safe because there are no adverse drug reactions when used in as many as 100 subjects.” Certainly, point estimation yields an incidence of 0%; however, interval estimation yields a 95% confidence interval (CI) for the incidence of 0-3%. Especially when a serious adverse drug reaction may infrequently occur, safety should be judged in consideration of the number of participants in the clinical trial in the article. From this perspective, the expression of “safety assessment in the article” was used.

Reports on Kampo medicines sometime use expressions such as “30 patients received the treatment, and had no adverse reactions”. However, caution is required here in two respects.

First, an adverse drug reaction is different from an adverse event. An adverse event (AE) means “any untoward medical occurrence, whether or not related to the medical product,” and an adverse drug reaction (ADR) means an AE with “causal relationship to the medical product that cannot be ruled out”.

Second, for probabilistic reasons, a confidence interval (CI) is necessary, and usually a 95% CI is used. Its upper and lower limits are referred to as confidence limits. For example, if 10 patients are treated and no adverse events occur, the 95% CI is 0–26%; if 20 patients are treated, the 95% CI is 0–14%; if 50 patients, 0–6%; if 100 patients, 0–3%; and if 500 patients, 0–0.6%. When the sample size is greater than 50, the rule of three can be used to give approximate results. The lower confidence limit is always zero, while the upper confidence limit is [1-0.051/n]. A detailed explanation of the above can be found in the reference cited below.

Tsutani K. Kenkyu dezain no kiso (Research design basics). In: Tsutani K et al. (eds). EBM no tameno joho senryaku – ebidensu wo tsukuru, tsutaeru, tsukau – (Information strategies for EBM – generate, transfer, and use evidence –). Tokyo: Chugai-igakusha; 2000: 26-47. (in Japanese)

The above example assumed no occurrence of adverse events in any patients. Generally, however, when “adverse events occurred in m of n patients”, the 95% CI can be calculated based on binomial distribution using the equation below. This equation, which utilizes normal approximation of binominal distribution, produces larger error with smaller sample size (n < 25).

95% confidence interval for event incidence=p±1.96√(p(1-p) /n), where p=m/n.

Although normal approximation was used for calculating the confidence interval in the above equation, there are other calculation systems readily available on the Internet.

In this evidence report, the description in “safety assessment included in the article” is standardized as follows:

1) Without indication

When safety assessment was not performed or indicated, “none” was indicated.

2) With indication

When safety assessment was performed, even if only slightly, and revealed no adverse drug reactions, that effect was indicated. When adverse drug reactions were specifically indicated, the abstractor in charge indicated this according to the expression used in the reference. When the number of patients with adverse drug reactions was specified in the reference, it was indicated. Note that some indications of adverse drug reactions are not unified.

11) Abstractor’s comments” refers to objective comments on a reference presented as a structured abstract. This helps busy readers not used to critical appraisal correctly and easily judge the value of the article. Abstractors were selected such that the abstractor did not belong to the same group of the authors concerned or have a master-student relationship, in consideration of conflict of interest (COI). How comments should be made was most actively discussed among members of the task force. With the aim of improving and standardizing comments, the 2nd Workshop of the Task Force for Evidence Report “To prepare appropriate comments” was held in conjunction with the Japan Society for Oriental Medicine 58th Annual Meeting held in Hiroshima on 17 Jun. 17, 2007.

Tsuruoka H, Okabe T, Tsutani K. Kampoyaku RCT no kozokashoroku niokeru komento kisai no kaizen － Dai 2-kai ebidensu repoto tasuku fosu wakushoppu hokoku －(Improvement of indication of comments in structured abstracts of RCTs on Kampo medicine－the Report of the 2nd Workshop of the Task Force for Evidence Report－. Nihon Toyo Igaku Zasshi [Kampo Medicine] 2009: 60: 177-84 (in Japanese).

12) Abstractor and date” is intended to clarify the responsibility, which also concerns the abovementioned conflict of interest, and to show the temporary relationship between the comments and related studies and in consideration of the possibility of correction at a later date. When a structured abstract was revised, the date of revision was added.

Structured abstract preparation tasks were assigned to Task Force members in consideration of their specialties. However, since the specialties of members did not cover the whole field, some abstracts were prepared by non-specialized members.

Bibliographic items were indicated in the Vancouver style as a rule, with some modifications, including that the number of authors listed shall be up to 3 and that the name of a journal shall not be abbreviated.

Structured abstracts were arranged in the order of ICD10 (Version 2003) code of diseases. Those with the same code were arranged by date of publication of the main reference evaluated. When more than one ICD code was possible, the one seeming to be generally more understandable was selected. Similarly, excluded references were arranged in the order of ICD10 code. The names of Kampo formulae, etc. that could not be written in Chinese characters for daily use in Japan were written in Katakana. The names of ICD code diseases differ from general names and were therefore read as shown in Table 2 to indicate them in this report.

In preparing structured abstracts, to maintain the quality, a Structured Abstract Preparation Manual was prepared, distributed to Task Force members, and updated as appropriate.

PageTop

• Notes on the current version
• History of version upgrades
• Excecutive summary
• 1. Lists of structured abstracts
• 2. Background
• 3. Purpose
• 4. Steps for development of structured abstracts
• 5. Included and excluded references
• 6. Relation to other projects
• 7. Lists of excluded references
• 8. Conflict of interests
• 9. Organization
• 10. Acknowledgements
• 11. Contact point
• Full Text (PDF)

• released 30 Jun 2022