Errors and Omissions in Experimental Trials - Part 1.
PHILIP R. N. SUTTON
D.D.Sc (Melb), L.D.S. (Vic.)
Senior Research Fellow, Department of Oral Medicine and Surgery
Dental School, University of Melbourne
MELBOURNE UNIVERSITY PRESS
First published in 1959, Second edition, enlarged, 1960
Printed and bound in Australia by Melbourne University Press, Parkville N.2, Victoria.
Registered in Australia for transmission by post as a book
London and New York: Cambridge University Press
PREFACE TO THE SECOND EDITION
Soon after the publication of the first edition of this monograph, in September 1959; copies were sent, by the Australian Dental Association, to the workers in charge of all the studies considered. As a result, critical reviews were published in the February 1960 issue of the Australian Dental Journal. The New Zealand Dental Journal of January 1960 also contained a critical review. These have not indicated the necessity for any modifications in Parts One and Two which are, therefore, reprinted unchanged. However, in this edition a Part Three has been added in which these criticisms are reprinted, at length, and some comments made. It is again stressed that in this book consideration is limited to some aspects of five crucial experimental trials of artificial fluoridation. Results reported from "naturally fluoridated" areas are not considered.
P.R.N.S., Dental School, University of Melbourne, June 1960
PREFACE TO THE FIRST EDITION
Endorsements of the process of the mechanical addition of fluorides to public water supplies, with the aim of reducing the incidence of dental caries, rely mainly on the results published from five trials which were set up to test, primarily, the efficacy of this process.
Important deficiencies in the methods used were revealed during a preliminary investigation of reports of these trials. Therefore this study was undertaken in an attempt to answer the question: Can the claims of considerable dental benefits as a result of artificial fluoridation be regarded as established, or are they based on an unsound foundation.
P.R.N.S., Dental School, University of Melbourne, February 1959.
Members of the Department of Statistics, University of Melbourne, have given most careful consideration to all the statistical matters mentioned in Part One of this monograph they have checked the computations in Part Two and have also given advice regarding statistical matters mentioned in Part Three. Their assistance is gratefully acknowledged. Part One was published in the Medical Journal of Australia, 1 February 1958, pages 139-40. 1 should like to express my thanks to my co-author, and to the Editor of the journal for permission to reprint the paper and also to the Editors of the Australian Dental Journal and the New Zealand Dental Journal for permission to reprint the book reviews shown in Part Three. Extracts from Part Two were presented at the fifteenth Australian Dental Association Congress, Adelaide, 23-7 February 1959. Professor Sir Arthur Amies and Dr Paul Pincus have suggested improvements to the draft of Part Two, and Miss H. N. Rankine, the Librarian of the Dental School of the University of Melbourne, has given valuable assistance. This investigation has been supported by grants from the University of Melbourne Research Fund.
SOME STATISTICAL OBSERVATIONS ON FLUORIDATION TRIALS*
The suggestion that domestic water supplies should be-fluoridated, with the aim of partially preventing the development of dental caries, has gained wide support, and moves are being made in Australia for the widespread introduction of this measure. Much confusion of thought clouds the issue of the desirability, the method of action and the safety of this process. This uncertainty is reflected in two recent events. In November 1956, a Reference Committee of the American Medical Association (1957) stated that "there is a definite need for a re-evaluation of the problem of fluoridation",† and in March 1957, after a public hearing, the proposal to fluoridate the water supply of New York was not put into practice (Nesin, B.C., personal communication, 1957).
Apart from these considerations, an examination reveals that there are aspects that call for a very careful appraisal of the figures presented in the reports of the experimental trials which have been conducted in Brantford, Canada, and in Grand Rapids, Newburgh and Evanston, U.S.A., and upon the results of which proposals to fluoridate domestic water are almost entirely based.
A preliminary survey of the methods used, of the published figures and of the method of their presentation discloses some disturbing facts. Some of these are as follows. (i) In the clinical examinations no attempt was made to devise a randomization procedure, which would have eliminated bias on the part of the examiners. However, the necessity for such a precaution was recognized by Ast, Bushel, Wachs and Chase (1955) in the Newburgh-Kingston trial, when they instituted a combined clinical and X-ray study eight years after the commencement of the ten-year investigation. (ii) No estimate was made of variability between examiners, although in some studies several operators were employed, some being changed from year to year (Blayney and Tucker, 1948; Arnold, Dean and Knutson, 1953); some of the examinations in Kingston were made by two dental hygienists (Ast, Finn and McCaffrey, 1950). Furthermore, there appears to be no estimate of variability within the examiner-that is, the variability of individual examiners from inspection to inspection. (iii) The importance of random variation in the DMF rate (decayed-missing-filled permanent teeth rate) does not appear to have been recognized, or else it has been ignored. (iv) Bias is suggested by the presentation of some results, so that the casual reader may be misled (Ontario Department of Health, 1956).
*Reprinted from a paper by Philip R. N. Sutton. D.D.Sc. (Melb.), L.D.S. (Vic.) and Arthur B. P Amies, C.M.G., D.D.Sc. (Melb.), F.R.C.S. (Edin.), F.R.A.C.S., originally published in the Medical Journal of Australia, 1 February 1958. † In December 1957, the American Medical Association endorsed the principle of fluoridation, but that decision cannot affect the facts which have been stated in this paper.
The following observations will serve as illustrations.
Whilst we do not question the integrity of workers in this field, it must be pointed out that the evidence tendered in favour of fluoridation reveals two disturbing features. The first is that what must be essentially a statistical study does not appear to have been planned as such. The second is that even when sufficient information is presented, no comprehensive attempt at statistical evaluation has been considered.
It is possible that a case for fluoridation can be solidly based, but until adequate statistical treatment of all the pertinent factors has been carried out and this would be quite a major undertaking the question should not be regarded as settled. In the meantime, claims concerning the amount of caries reduction are open to doubt.