From AMS Glossary
The extent to which a statement is considered reliable.
This is a subjective factor, as opposed to probability, which is a mathematical likelihood of occurrence, based on previous experience. For example, a forecaster may state that there is a 50% probability of rain, and assign a certainty factor of 95% to that statement. There are several methods of combining certainty factors to arrive at an overall certainty. The most common of these is the Dempster–Shafer rule (Shafer 1976).
Shafer, G. 1976. A Mathematical Theory of Evidence. Princeton University Press.