Several influential characterizations of paternalism or its distinctive wrongness emphasize a belief or judgement that it typically involves—namely, the judgement that the paternalized is likely to act irrationally, or some such. But it’s not clear what about such a belief can be morally objectionable if it has the right epistemic credentials (if it is true, say, and is best supported by the evidence). In this paper, I elaborate on this point, placing it in the context of the relevant epistemological discussions. I explain how evidentialism is opposed to such thoughts; I show that possible ways of rejecting evidentialism (along lines analogous to those of pragmatic encroachment) won’t work; and I sketch an account of the wrongness of paternalism that doesn’t depend on any flaw in the belief about others’ likely behaviour.