To Aggregate or not to aggregate : that is the question

  1. (PDF, 291 KB)
AuthorSearch for: ; Search for: ; Search for:
Proceedings titleProceedings
ConferenceACM SIGMIS International Conference on Knowledge Discovery and Information Retrieval (KDIR), Paris, France, October 26-29, 2011
SubjectData pre-processing; Aggregation; Gaussian distribution; L'evy distribution
AbstractConsider a scenario where one aims to learn models from data being characterized by very large fluctuations that are neither attributable to noise nor outliers. This may be the case, for instance, when examining supermarket ketchup sales, predicting earthquakes and when conducting financial data analysis. In such a situation, the standard central limit theorem does not apply, since the associated Gaussian distribution exponentiallysuppresses large fluctuations. In this paper, we argue that, in many cases, the incorrect assumption leads to misleading and incorrect data mining results. We illustrate this argument against synthetic data, and show some results against stock market data.
Publication date
AffiliationNational Research Council Canada; NRC Institute for Information Technology
Peer reviewedNo
NPARC number18608249
Export citationExport as RIS
Report a correctionReport a correction
Record identifier749f5106-d975-441a-b258-9776bfd387e8
Record created2011-09-21
Record modified2016-05-09
Bookmark and share
  • Share this page with Facebook (Opens in a new window)
  • Share this page with Twitter (Opens in a new window)
  • Share this page with Google+ (Opens in a new window)
  • Share this page with Delicious (Opens in a new window)