The basic problem we study in probability is:
Given a data generating process, what are the properties of the outcomes?
The basic problem of statistical inference is the inverse of probability:
Given the outcomes, what can we say about the process that generated the data?
Book reading notes: All of Statistics, by Larry Wasserman