This is all a bit vague, but hopefully it will make some sense at some stage…

Is EBM underpinned by an outdated philosophy of science?

(…because frequentist statistics is underpinned by an outdated philosophy of science)

By ‘outdated philosophy of science’ I think I mean ‘irrationalist’ (following - for the moment - Dorling). I am not happy or clear yet on what terminology I want/need to use here, but I am thinking of the the ways that science can respond to the problem of induction and the Duhem problem. By outdated I perhaps unkindly mean Kuhn, Lakatos, Feyerabend and their very different ways of embracing, lamenting or avoiding by methodological fiat the problems raised for providing a rational story for the progression of scientific conjectures.. (apologies, I realise this requires much clarification.. i will continue to use ‘scare’ quotes to highlight my awkwardness in using the term irrationalist).. by what I would mean by non outdated philosophy of science, I am afraid I might even be more vague.. I have in mind the benefits of bayesian analysis but hope that a story could be told that would be consistent (in most senses anyway) with Van Fraassen, Hacking, Cartwright and others..

There seems to be two ways that this argument might be made (A and B). (A) The first focuses on whether frequentist statistics are underpinned by an ‘irrationalist’ picture of science: (1) all inferences which are made with respect to hypotheses in EBM (or more accurately - in the large scale drug RCTs in which I am interested) are made with frequentist inferences - as all inferences are essentially frequentist, frequentist analysis is central to EBM. (2) Frequentist analysis is underpinned by an outdated philosophy of science. I am not too sure how controversial this is. I need to look into how well this argument can be made. Goodman’s Annals of Internal Med papers discussed that (i) Fisher proposed p-values “as an informal index to be used as a measure of discrepancy between the data and the null hypothesis. Fisher suggested that it be used as part of the fluid, non-quantifiable process of drawing conclusions from observations, a process that included combining the p-value in some unspecified way with background information”; and (ii) with respect to Neyman Pearson hypothesis testing - the outcome of the hypothesis test is a behaviour not and inference.. Goodman quoting from NP.. “But we may look at the purpose of tests from another viewpoint. Without hoping to know whether each separate hypothesis is true or false, we may search for rules to govern our behaviour with regard to them, in following which we insure that , in the long run of experience, we shall not often be wrong.” The suggestion here (also in much need of work) is that frequentist methods are ‘irrationalist’ with respect to science - in the sense the both seem to reject the possibility of saying anything positive about a given hypothesis (or the probability of a given hypothesis) (3) Therefore EBM is underpinned by an outdated philosophy of science

2. This second is, I think, interesting but I am not sure whether I am framing it in a useful way yet. It is the idea that EBM’s hierarchy of evidence (with the results of RCTs held as giving greater access to truth than causal scientific stories) implicitly embraces the idea that the induction of causal scientific stories is inherently unreliable or not possible (i.e. ‘irrationalist’). Epistemic privilege is given to the frequentist outcomes of RCTs as if they are deductively valid (which is clearly problematic). Causal scientific stories as the core of science is subordinated. EBM urges decision-making based on the ‘significant’ results of RCTs rather than explicitly taking into account background information and the scientific story.