Seeking Relationships in Big Data: A Bayesian Perspective

Authors

  • Nozer Singpurwalla The City University of Hong Kong Hong Kong, China

DOI:

https://doi.org/10.55630/sjc.2014.8.97-110

Keywords:

Association, Correlation, Dependence, Mutual Information, Prediction, Regression, Retrospective Data

Abstract

The real purpose of collecting big data is to identify causality
in the hope that this will facilitate credible predictivity . But the search
for causality can trap one into infinite regress, and thus one takes refuge in
seeking associations between variables in data sets. Regrettably, the mere
knowledge of associations does not enable predictivity. Associations need to
be embedded within the framework of probability calculus to make coherent
predictions. This is so because associations are a feature of probability
models, and hence they do not exist outside the framework of a model.
Measures of association, like correlation, regression, and mutual information
merely refute a preconceived model. Estimated measures of associations do
not lead to a probability model; a model is the product of pure thought. This
paper discusses these and other fundamentals that are germane to seeking
associations in particular, and machine learning in general.

Downloads

Published

2015-04-07

Issue

Section

Articles