Phylogenetic modelling of evolutionary processes across sites from sequence alignments has garnered increasing attention over the last few decades. One approach adopts the view that the heterogeneity across observations is a result of the data set having been emitted from several different models, each drawn from a distribution. Finite mixture models provide discretizations of the unknown distribution into a set of sub-models, or components. Choosing a level of discretization is done from a set of likelihood-based model comparisons. We use Bayesian cross-validation to compare a range of finite mixture models, along with the infinite mixture modelling approach known as `CAT', and gamma-distributed rates across sites approach. Using simulations and real alignments, our findings indicate that the improvement in model-fit from finite mixture models is attained when the number of components is between 20 and 60. The magnitude of improvement is dependant on whether or not the gamma approach is invoked.