Stephan Poppe (University of Leipzig)

Species Sampling Processes: predicting the unpredictable and estimating measures of diversity

The sampling of species problem relates to the issue of how to infer the relative species abundances from finite data, when many species occurring in the population are not present in the sample. Although these abundances can be seen to be the ultimate measure of the diversity in a population, there is also some interest in estimating particular summarizing diversity indexes such as  the Shannon index and the actual number of occurring species in the population.

The invocation of finite-dimensional symmetrical Dirichlet priors yields Bayesian statistical models which allow for a straight-forward inference of these abundances in case of a prefixed space of species, whereas their infinite-dimensional counterpart in form of the Pitman-Yor-process allows predictions for the case of previously unknown species.

The adoption of these models is usually mainly due to their analytical simplicity and the conjugacy of the underlying priors, but can also be justified from such predictive principles as exchangeability and Johnson’s sufficientness postulate. Hence, entertaining a dualistic point of view, i.e. looking at both the statistical model and the associated prediction rules, we potentially gain a better understanding of the inductive inference procedures set up. This becomes in particular important as the Dirichlet prior and the Pitman-Yor process do not lead to sensible estimates of neither the Shannon index nor the number of occurring species.

In my talk I will review several “classical” inductive characterizations of prediction rules of  authors such as Johnson, Carnap, Hintikka, Niiniluoto, Kuipers and Zabell. I will also show how the recently introduced species sampling model of Gnedin and Pitman fits in those inductive frameworks. If time permits, I will also shortly demonstrate their utility in improving the estimation of diversity indexes.