Nizamettin AYDIN, Gökmen ALTAY, Zeyneb KURT

Comprehensive review of association estimators for the inference of gene networks

Gene network inference (GNI) algorithms allow us to explore the vast amount of interactions among the molecules in cells. In almost all GNI algorithms the main process is to estimate association scores among the variables of the dataset. However, there is no commonly accepted estimator to compute association scores for the current GNI algorithms. In this paper the association estimators that might be used in GNI applications are reviewed. The aim is to prepare a comprehensive and comparative review of all the important association estimators available in the literature. We performed this main aim by presenting, classifying, comparing, and discussing them to reveal which association estimator is more suitable for use in GNI applications by considering only the information available in the literature. Twenty-seven different estimators from various areas are investigated. The estimators were compared according to the GNI performances in the literature. The most promising association estimators for the GNI applications are suggested. As a result of the study, we identified eight promising methods for effective use in GNI. We expect this study to assist many researchers before using those estimators in their own GNI studies.

PDF

___

[1] Mattiussi V, Tumminello M, Iori G, Mantegna RN. Comparing correlation matrix estimators via KullbackLeibler divergence. Available at Social Science Research Network (SSRN) 2011: 77-104.
[2] Rogers LCG, Zhou F. Estimating correlation from high, low, opening and closing prices. Annals of Applied Probability 2008; 18: 813-823.
[3] Lindskog F. Linear Correlation Estimation. RiskLab research papers, 11 Dec. 2000.
[4] Neemuchwala H, Hero AO. Image registration in high dimensional feature space. Proc. of SPIE Conference on Electronic Imaging, San Jose, Jan. 2005.
[5] Hero A, Ma B, Michel O, Gorman J. Applications of entropic spanning graphs. IEEE Signal Processing Magazine 2002; 19: 85-95.
[6] P´oczos B, Kirshner S, Szepesv´ari C. REGO: Rank-based estimation of Renyi information using Euclidean graph optimization. Journal of Machine Learning Research - Proceedings 2010; 9: 605-612.
[7] Brunelli R, Messelodi S. Robust estimation of correlation with applications to computer vision. Pattern Recognition 1995; 28: 833-841.
[8] Shan C, Gong S, McOwan PW. Conditional mutual information based boosting for facial expression recognition. In British Machine Vision Conference (BMVC), Oxford, UK, Sep. 2005.
[9] Lee WCY. An extended correlation function of two random variables applied to mobile radio transmission. Bell Sys Tech Journal 1969; 48: 3423-3440.
[10] Barcel´o-Llad´o JE, P´erez AM, Seco-Granados G. Enhanced correlation estimators for distributed source coding in large wireless sensor networks. IEEE Sensors Journal 2012; 12: 2799-2806.
[11] Kerscher M, Szapudi I, Szalay AS. A comparison of estimators for the two-point correlation function. The Astrophysical Journal 2000; 535: L13-L16.
[12] Habib E, Krajewski WF, Ciach GJ. Estimation of rainfall interstation correlation. Journal of Hydrometeorology 2001; 2: 621-629.
[13] Moon YI, Rajagopalan B, Lall U. Estimation of mutual information using kernel density estimators. Physical Review E 1995; 52: 2318-2321.
[14] Olsen C, Meyer PE, Bontempi G. On the impact of entropy estimation on transcriptional regulatory network inference based on mutual information. EURASIP Journal on Bioinformatics and Systems Biology 2009; 2009(308959).
[15] Simoes RM, Emmert-Streib F. Influence of statistical estimators of mutual information and data heterogeneity on the inference of gene regulatory networks. PLoS ONE 2011; 6(11).
[16] Daub CO, Steuer R, Selbig J, Kloska S. Estimating mutual information using B-spline functions - an improved similarity measure for analysing gene expression data. BMC Bioinformatics 2004; 5(118).
[17] Margolin AA, Nemenman I, Basso K, Wiggins C, Stolovitzky G, Favera, RD, Califano A. ARACNE: an algorithm for the reconstruction of gene regulatory networks in a mammalian cellular context. BMC Bioinformatics 2006; 7(Suppl 1): S7.
[18] Butte AJ, Kohane LS. Mutual information relevance networks: functional genomic clustering using pairwise entropy measurements. Pacific Symposium on Biocomputing, Hawaii, vol. 5, pp. 418-429, 49 Jan. 2000.
[19] Altay G, Emmert-Streib F. Inferring the conservative causal core of gene regulatory networks. BMC Systems Biology 2010; 4(132).
[20] Altay G, Asim M, Markowetz F, Neal DE. Differential C3NET reveals disease networks of direct physical interactions. BMC Bioinformatics 2011; 12(296).
[21] Altay G, Emmert-Streib F. Revealing differences in gene network inference algorithms on the network-level by ensemble methods. Bioinformatics 2010; 26: 1738-1744.
[22] Walters-Williams J, Li Y. Estimation of mutual information: a survey. Lecture Notes in Computer Science Rough Sets and Knowledge Technology 2009; 5589: 389-396.
[23] Hausser J, Strimmer K. Entropy inference and the James-Stein estimator, with application to nonlinear gene association networks. Journal of Machine Learning Research 2009; 10: 1469-1484.
[24] Reshef DN, Reshef YA, Finucane HK, Grossman SR, McVean G, Turnbaugh PJ, Lander ES, Mitzenmacher M, Sabeti PC. Detecting novel associations in large data sets. Science 2011; 334(6062): 1518-1524.
[25] Paninski L. Estimation of entropy and mutual information. Neural Computation 2003; 15: 1191-1253.
[26] Vu VQ, Yu B, Kass RE. Coverage-adjusted entropy estimation. Statistics in Medicine 2007; 26: 4039-4060.
[27] Horvath S. Weighted Network Analysis: Applications in Genomics and Systems Biology, 1st Edition, Berlin, Germany: Springer, 2011.
[28] Meyer PE, Kontos K, Bontempi G. Biological network inference using redundancy analysis. Proceedings of the 1st International Conference on Bioinformatics Research and Development (BIRD07) Lecture Notes in Computer Science; 1214 March 2007; Berlin, Germany; 4414: 16-27.
[29] Federer A. Estimating networks using mutual information. Master Thesis, Swiss Federal Institute of Technology Zurich Department of Mathematics 2011.
[30] de la Fuente A, Bing N, Hoeschele I, Mendes P. Discovery of meaningful associations in genomic data using partial correlation coefficients. Bioinformatics 2004; 20: 3565-3574.
[31] C¸ akır T, Hendriks MMWB, Westerhuis JA, Smilde AK. Metabolic network discovery through reverse engineering of metabolome data. Metabolomics 2009; 5: 318-329.
[32] Cukierski WJ, Nandy K, Gudla P, Meaburn KJ, Misteli T, Foran DJ, Lockett SJ. Ranked retrieval of segmented nuclei for objective assessment of cancer gene repositioning. BMC Bioinformatics 2012; 13: 232.
[33] Zhang G, Su Z. Inferences from structural comparison: flexibility, secondary structure wobble and sequence alignment optimization. BMC Bioinformatics 2012; 13(Suppl 15): S12.
[34] Ray A, Lindahl E, Wallner B. Improved model quality assessment using ProQ2. BMC Bioinformatics 2012; 13: 224.
[35] Malovini A, Barbarini N, Bellazzi R, De Michelis F. Hierarchical Naive Bayes for genetic association studies. BMC Bioinformatics 2012; 13(Suppl 14): S6.
[36] Qiu P, Zhang L. Identification of markers associated with global changes in DNA methylation regulation in cancers. BMC Bioinformatics 2012; 13(Suppl 13): S7.
[37] Mahanta P, Ahmed HA, Bhattacharyya DK, Kalita JK. An effective method for network module extraction from microarray data. BMC Bioinformatics 2012; 13(Suppl 13): S4.
[38] Lin HC, Goldstein S, Mendelowitz L, Zhou S, Wetzel J, Schwartz DC, Pop M. AGORA: Assembly Guided by Optical Restriction Alignment. BMC Bioinformatics 2012; 13: 189.
[39] Wu C, Zhu J, Zhang X. Integrating gene expression and protein-protein interaction network to prioritize cancerassociated genes. BMC Bioinformatics 2012; 13: 182.
[40] Gonnet GH. Surprising results on phylogenetic tree building methods based on molecular sequences. BMC Bioinformatics 2012; 13: 148.
[41] Collingridge PW, Kelly S. MergeAlign: improving multiple sequence alignment performance by dynamic reconstruction of consensus multiple sequence alignments. BMC Bioinformatics 2012; 13: 117.
[42] Ayadi W, Elloumi M, Hao JK. Pattern-driven neighborhood search for biclustering of microarray data. BMC Bioinformatics 2012; 13(Suppl 7):S11.
[43] Van den Bulcke T, Van Leemput K, Naudts B, Van Remortel P, Ma H, Verschoren A, De Moor B, Marchal K. SynTReN: a generator of synthetic gene expression data for design and analysis of structure learning algorithms. BMC Bioinformatics 2006; 7(42).
[44] Kendall M. A new measure of rank correlation. Biometrika 1938; 30: 81-89.
[45] Datta S, Pihur V, Datta S. An adaptive optimal ensemble classifier via bagging and rank aggregation with applications to high dimensional data. BMC Bioinformatics 2012; 11: 427.
[46] Yu H, Kim T, Oh J, Ko I, Kim S, Han WS. Enabling multi-level relevance feedback on PubMed by integrating rank learning into DBMS. BMC Bioinformatics 2010; 11(Suppl 2): S6.
[47] Freilich S, Goldovsky L, Gottlieb A, Blanc E, Tsoka S, Ouzounis CA. Stratification of co-evolving genomic groups using ranked phylogenetic profiles. BMC Bioinformatics 2009; 10: 355.
[48] Pihur V, Datta S, Datta S. RankAggreg: an R package for weighted rank aggregation. BMC Bioinformatics 2009; 10: 62.
[49] Pyysalo S, Airola A, Heimonen J, Bj¨orne J, Ginter F, Salakoski T. Comparative analysis of five protein-protein interaction corpora. BMC Bioinformatics 2008; 9(Suppl 3): S6.
[50] Tien YJ, Lee YS, Wu HM, Chen CH. Methods for simultaneously identifying coherent local clusters with smooth global patterns in gene expression profiles. BMC Bioinformatics 2008; 9: 155.
[51] Casadei D. Estimating the selection efficiency. Journal of Instrumentation 2012; 7: 8021.
[52] Veitch J, Mandel I, Aylott B, Farr B, Raymond V, Rodriguez C, van der Sluys M, Kalogera V, Vecchio A. Estimating parameters of coalescing compact binaries with proposed advanced detector networks. Physical Review D 2012; 85(9).
[53] Molladavoudi S, Zainuddin H, Chan KT. Jensen-Shannon divergence and nonlinear quantum dynamics. Physics Letters A 2012; 376(26-27): 1955-1961.
[54] Reginatto M, Hall MJW. Quantum theory from the geometry of evolving probabilities. 31st International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering; AIP Conference Proceedings, Waterloo, Ontario, Canada, 916 July 2011; 1443: 96-103.
[55] Genoni MG, Olivares S, Brivio D, Cialdi S, Cipriani D, Santamato A, Vezzoli S, Paris MGA. Optical interferometry in the presence of large phase diffusion. Physical Review A 2012; 85: 043817-1-043817-5.
[56] Svihl´ık J, Fliegel K, Kukal J, Jerhotov´a E, P´ata P, V´ıtek S, Koten P. Estimation of nonGaussian noise parameters ˇ in the wavelet domain using the moment-generating function. Journal of Electronic Imaging 2012; 21: 023025-15.
[57] Tuyl F, Gerlach R, Mengersen K. A comparison of BayesLaplace, Jeffreys, and other priors. The American Statistician 2008; 62: 40-44.
[58] Geisser S. On prior distributions for binary trials. The American Statistician 1984; 38: 244-247.
[59] Dempster AP. Upper and lower probability inferences based on a sample from a finite univariate population. Biometrika 1967; 54: 515-528.
[60] Perks W. Some observations on inverse probability including a new indifference rule. J Inst Actuaries 1947; 73: 285-334.
[61] Stambaugh RF. Predictive regressions. Journal of Financial Economics 1999; 54: 375-421.
[62] Bernardo JM. Reference posterior distributions for Bayesian inference. Journal of the Royal Statistical Society. Series B (Methodological) 1979; 41: 113-147.
[63] Kass RE, Wasserman L. The selection of prior distributions by formal rules. Journal of the American Statistical Association 1996; 91: 1343-1370.
[64] Good IJ. Maximum entropy for hypothesis formulation, especially for multidimensional contingency tables. The Annals of Mathematical Statistics 1963; 34: 911-934.
[65] Phillips PCB. To criticize the critics: an objective bayesian analysis of stochastic trends. Journal of Applied Econometrics 1991; 6: 333-364.
[66] Trybula S. Some problems of simultaneous minimax estimation. The Annals of Mathematical Statistics 1958; 29: 245-253.
[67] Fienberga SE, Holland PW. Simultaneous estimation of multinomial cell probabilities. Journal of the American Statistical Association 1973; 68: 683-691.
[68] Wu TD, Nevill-Manning CG, Brutlag DL. Minimal-risk scoring matrices for sequence analysis. Journal of Computational Biology 1999; 6: 219-235.
[69] Braess D, Dette H. The asymptotic minimax risk for the estimation of constrained binomial and multinomial probabilities. Sankhy¯a: The Indian Journal of Statistics (2003-2007) 2004; 66: 707-732.
[70] Wilming N, Betz T, Kietzmann TC, K¨onig P. Measures and limits of models of fixation selection. PLoS ONE 2011; 6(8).
[71] Allen DE, McAleer M, Powell RJ, Kumar-Singh A. A NonParametric and Entropy Based Analysis of the Relationship between the VIX and S&P 500. Social Science Research Network 2012 [Working Paper or Technical Report](Unpublished).
[72] Jost L. Partitioning diversity into independent alpha and beta components. Ecology 2007; 88: 2427-2439.
[73] De Filippo C, Cavalieri D, Di Paola M, Ramazzotti M, Poullet JB, Massart S, Collini S, Pieraccini G, Lionetti P. Impact of diet in shaping gut microbiota revealed by a comparative study in children from Europe and rural Africa. Proceedings of the National Academy of Sciences of the United States of America (PNAS) 2010; 107:14691-14696.
[74] Mao CX, Colwell RK. Estimation of species richness: mixture models, the role of rare species, and inferential challenges. Ecology 2005; 86: 1143-1153.
[75] K´ery M, Schmid H. Estimating species richness: calibrating a large avian monitoring programme. Journal of Applied Ecology 2006; 43: 101-110.
[76] Victor JD. Approaches to information-theoretic analysis of neural activity. Biological Theory 2006; 1: 302-316.
[77] Wieczorkowski R, Grzegorzewski P. Entropy estimators-improvements and comparisons. Communications in Statistics - Simulation and Computation 1999; 28: 541-567.
[78] Zahl S. Jackknifing an index of diversity. Ecology 1977; 58: 907-913.
[79] Moddemeijer R. A statistic to estimate the variance of the histogram-based mutual information estimator based on dependent pairs of observations. Signal Processing 1999; 75: 51-63.
[80] Wu EHC, Yu PLH, Li WK. A smoothed bootstrap test for independence based on mutual information. Computational Statistics & Data Analysis 2009; 53: 2524-2536.
[81] Schlogl A, Keinrath C, Scherer R, Furtscheller P. Information transfer of an EEG-based brain computer interface. First International IEEE EMBS Conference on Neural Engineering 2003; 2022 March, pp. 641-644.
[82] Saha S, Dey KN, Dasgupta R, Ghose A, Mullick K. Missing value estimation in DNA microarrays using b-splines. Journal of Medical and Bioengineering 2013; 2: 88-92.
[83] Steuer R, Kurths J, Daub CO, Weise J, Selbig J. The mutual information: detecting and evaluating dependencies between variables. Bioinformatics 2002; S231-S240.
[84] Luan Y, Li H. Clustering of time-course gene expression data using a mixed-effects model with B-splines. Bioinformatics 2003; 19: 474-482.
[85] Li H, Sun Y, Zhan M. Analysis of gene coexpression by B-spline based CoD estimation. EURASIP Journal on Bioinformatics and Systems Biology 2007; 2007: 49478.
[86] Xu S. Time-course microarray data analysis. In: Principles of Statistical Genomics. Springer: New York, NY, USA, 2013. pp. 365-382.
[87] Storey JD, Xiao W, Leek JT, Tompkins RG, Davis RW. Significance analysis of time course microarray experiments. Proceedings of the National Academy of Sciences of the United States of America 2005; 102(35): 12837-12842.
[88] Chang DTH, Ou YY, Hung HG, Yang MH, Chen CY, Oyang YJ. Prediction of protein secondary structures with a novel kernel density estimator. BMC Research Notes 2008; 1:51.
[89] Chang DTH, Wang CC, Chen JW. Using a kernel density estimation based classifier to predict species-specific microRNA precursors. BMC Bioinformatics 2008; 9(Suppl 12): S2.
[90] Murakami Y, Mizuguchi K. Applying the Na¨ıve Bayes classifier with kernel density estimation to the prediction of proteinprotein interaction sites. Bioinformatics 2010; 26: 1841-1848.
[91] Liao JG, Lin Y, Selvanayagam ZE, Shih WJ. A mixture model for estimating the local false discovery rate in DNA microarray analysis. Bioinformatics 2004; 20: 2694-2701.
[92] Hausser J. Improving Entropy Estimation and the Inference of Genetic Regulatory Networks. M.S. thesis. Department of Biosciences, National Institute of Applied Sciences of Lyon, Cedex, France; 2006.
[93] King BM, Tidor B. MIST: Maximum Information Spanning Trees for dimension reduction of biological data sets. Bioinformatics 2009; 25: 1165-1172.
[94] Margolin AA, Nemenman I, Basso K, Wiggins C, Stolovitzky G, Favera RD, Califano A. Technical Report: Parameter Estimation for the ARACNE Algorithm. Tech Rep nprot. 2006; 106-S2.
[95] Duin RPW. On the choice of smoothing parameter for Parzen estimators of probability density functions. IEEE T Comput 1976; C-25: 1175-1179.
[96] Koontz WLG, Fukunaga K. Asymptotic analysis of a nonparametric clustering technique. IEEE T Comput 1972; C-21: 967-974.
[97] Kraskov A, St¨ogbauer H, Grassberger P. Estimating mutual information. Phys Rev E 2004; 83(1).
[98] Suzuki T, Sugiyama M, Kanamori T, Sese J. Mutual information estimation reveals global associations between stimuli and biological processes. BMC Bioinformatics 2009; 10 (Suppl 1): S52.
[99] Numata J, Ebenh¨oh O, Knapp E-W. Measuring correlations in metabolomic networks with mutual information. Genome Inform 2008; 20: 112-122.
[100] Papana A, Kugiumtzis D. Evaluation of mutual information estimators on nonlinear dynamic systems. Nonlinear Phenomena in Complex Systems 2008; 11: 225-232.
[101] Pereda E, Quiroga RQ, Bhattacharya J. Nonlinear multivariate analysis of neurophysiological signals. Progress in Neurobiology 2005; 77: 1-37.
[102] St¨ogbauer H, Kraskov A, Astakhov SA, Grassberger P. Least-dependent-component analysis based on mutual information. Phys. Rev. E 2004; 70(6): 066123-1-066123-17.
[103] Rossi F, Lendasse A, Fran¸cois D, Wertz V, Verleysen M. Mutual information for the selection of relevant variables in spectrometric nonlinear modeling. Chemometrics and Intelligent Laboratory Systems 2006; 80: 215-226.
[104] Kraskov A, St¨ogbauer H, Andrzejak RG, Grassberger P. Hierarchical clustering using mutual information. EPL (Europhysics Letters) 2005; 70(2).
[105] Fukumizu K, Gretton A, Sun X, Sch¨olkopf B. Kernel Measures of Conditional Dependence. In: Advances in Neural Information Processing Systems 20: Proceedings of the 2007 Conference, MIT Press, Cambridge, MA, USA, pp. 489-496.
[106] Singh H, Misra N, Hnizdo V, Fedorowicz A, Demchuk E. Nearest neighbor estimates of entropy. American J of Math and Manag Sciences 2003; 23: 301-321.
[107] Hnizdo V, Darian E, Fedorowicz, Demchuk E, Li S, Singh H. Nearest-neighbor nonparametric method for estimating the configurational entropy of complex molecules. J Comput Chem 2007; 28: 655-668.
[108] Numata J, Wan M, Knapp EW. Conformational entropy of biomolecules: beyond the quasi-harmonic approximation. Genome Informatics 2007; 18: 192-205.
[109] Kozachenko LF, Leonenko NN. Sample estimates of entropy of a random vector. Problems of Information Transmission 1987; 23: 95-101.
[110] Mnatsakanov RM, Misra N, Li S, Harner EJ. Kn -nearest neighbor estimators of entropy. Mathematical Methods of Statistics 2008; 17: 261-277.
[111] Li S, Mnatsakanov RM, Andrew ME. K-nearest neighbor based consistent entropy estimation for hyperspherical distributions. Entropy 2011; 13: 650-667.
[112] Rahvar ARA, Ardakani M. Boundary effect correction in k-nearest-neighbor estimation. Phys. Rev. E 2011; 83: 051121-1-051121-8.
[113] Mnatsakanov RM, Li S, Harner EJ. Estimation of multivariate shannon entropy using moments. Australian & New Zealand Journal of Statistics 2011; 53: 271-288.
[114] Sch¨afer J, Strimmer K. An empirical Bayes approach to inferring large-scale gene association networks. Bioinformatics (Oxford, England) 2005; 21: 754-764.
[115] Zhang X, Zhao XM, He K, Lu L, Cao Y, Liu J, Hao JK, Liu ZP, Chen L. Inferring gene regulatory networks from gene expression data by PC-algorithm based on conditional mutual information. Bioinformatics 2011; 28: 98-104.
[116] Meyer PE, Lafitte F, Bontempi G. minet: A R/Bioconductor package for inferring large transcriptional networks using mutual information. BMC Bioinformatics 2008; 9: 461.
[117] Kim DC, Wang X, Yang CR, Gao J. Learning biological network using mutual information and conditional independence. BMC Bioinformatics 2010; 11(Suppl 3):S9.
[118] Liang K-C, Wang X. Gene regulatory network reconstruction using conditional mutual information. EURASIP Journal on Bioinformatics and Systems Biology 2008; 2008: 253894.
[119] Chaitankar V, Ghosh P, Perkins EJ, Gong P, Zhang C. Time lagged information theoretic approaches to the reverse engineering of gene regulatory networks. BMC Bioinformatics 2010; 11: S19.
[120] Szekely G, Rizzo M. Brownian distance covariance. The Annals of Applied Statistics 2009; 3: 1236-1265.
[121] Szekely G, Rizzo M, Bakirov N. Measuring and testing independence by correlation of distances. The Annals of Statistics 2007; 35: 2769-2794.
[122] Sejdinovic D, Gretton A, Sriperumbudur B, Fukumizu K. Hypothesis testing using pairwise distances and associated kernels. Proceedings of the 29th International Conference on Machine Learning, Edinburgh, Scotland, UK, 2012.
[123] Sejdinovic D, Sriperumbudur B, Gretton A, Fukumizu K. Equivalence of distance-based and RKHS-based statistics in hypothesis testing, in review. [arXiv].
[124] Grothe O, Schmid F, Schnieders J, Segers J. Measuring association between random vectors. Submitted to Computational Statistics & Data Analysis, in review.
[125] Zhou Z. Measuring nonlinear dependence in time-series, a distance correlation approach. Journal of Time Series Analysis 2012; 33: 438-457.
[126] Heller R, Heller Y, Gorfine MA. A consistent multivariate test of association based on ranks of distances, Technical Report, June 1, 2012.
[127] Simon N, Tibshirani R. Comment on Detecting Novel Associations in Large Data Sets. Science Dec, 2011
[128] K¨uffner R, Petri T, Tavakkolkhah P, Windhager L, Zimmer R. Inferring gene regulatory networks by ANOVA. Bioinformatics 2012; 28: 1376-1382.
[129] Bharathi A, Natarajan AM. Cancer classification of bioinformatics data using ANOVA. International Journal of Computer Theory and Engineering 2010; 2(3).
[130] Krawetz S. Bioinformatics for Systems Biology, 2nd ed., 2009, Springer.
[131] De Haan JR, Wehrens R, Bauerschmidt S, Piek E, van Schaik RC, Buydens LMC. Interpretation of ANOVA models for microarray data using PCA. Bioinformatics 2006; 23: 184-190.
[132] Nueda MJ, Conesa A, Westerhuis JA, Hoefsloot HCJ, Smilde AK, Tal´on M, Ferrer A. Discovering gene expression patterns in time course microarray experiments by ANOVA-SCA. Bioinformatics 2007; 23: 1792-1800.
[133] Smilde AK, Jansen JJ, Hoefsloot HCJ, Lamers R-JAN, van der Greef J, Timmerman ME. ANOVA-simultaneous component analysis (ASCA): a new tool for analyzing designed metabolomics data. Bioinformatics 2005; 21: 3043-3048.
[134] Stolovitzky G, Monroe D, Califano A. Dialogue on Reverse-Engineering Assessment and Methods: The DREAM of High-Throughput Pathway Inference, Annals of the New York Academy of Sciences 2007; 1115: 11-22.
[135] Marbach D, Costello JC, K¨uffner R, Vega NM, Prill RJ, Camacho DM, Allison KR, The DREAM5 Consortium, Kellis M, Collins JJ, et al. Wisdom of crowds for robust gene network inference. Nature Methods 2012; 9: 796-804.
[136] Faith JJ, Driscoll ME, Fusaro VA, Cosgrove EJ, Hayete B, Juhn FS, Schneider SJ, Gardner TS. Many Microbe Microarrays Database: uniformly normalized Affymetrix compendia with structured experimental metadata. Nucleic Acids Res 2008; 36(Database issue): D866D870.
[137] Van Hulle MM. Edgeworth approximation of multivariate differential entropy. Neural Computation 2005; 17: 1903-1910.
[138] Harremoes P. Maximum entropy and the Edgeworth expansion. IEEE Information Theory Workshop 2005, 29 Aug.-1 Sept. Rotorua, New Zealand.
[139] Zeng J, Xie L, Kruger U, Gao C. A nonGaussian regression algorithm based on mutual information maximization. Chemometrics and Intelligent Laboratory Systems 2012; 111: 1-19. [140] Comon P. Independent component analysis, a new concept? Signal Processing 1994; 36: 287-314.
[141] Amari S, Cichocki A, Yang HH. A New Learning Algorithm for Blind Signal Separation. in Advances in Neural Information Processing Systems 1996; 8: 757-763, MIT Press.
[142] Huber P. Projection pursuit. The Annals of Statistics 1985; 13: 435-475.
[143] Hlav´aˇckov´a-Schindler K, Paluˇs M, Vejmelka M, Bhattacharya J. Causality detection based on information-theoretic approaches in time series analysis. Physics Reports 2007; 441: 1-46.
[144] Suzuki T, Sugiyama M, Sese J, Kanamori T. Approximating mutual information by maximum likelihood density ratio estimation. The Journal of Machine Learning Research (JMLR): Workshop and Conference Proceedings 2008; 4: 5-20.
[145] Sugiyama M, Kanamori T, Suzuki T, Hido S, Sese J, Takeuchi I, Wang L. A Density-ratio framework for statistical data processing. IPSJ Transactions on Computer Vision and Applications 2009; 1: 183-208.
[146] Hido S, Tsuboi Y, Kashima H, Sugiyama M, Kanamori T. Statistical outlier detection using direct density ratio estimation. Knowledge and Information Systems 2011; 26: 309-336.
[147] Sugiyama M, Suzuki T. Least-Squares Independence Test. IEICE TRANSACTIONS on Information and Systems 2011; E94-D(6): 1333-1336.
[148] Altay G. Empirically determining the sample size for large-scale gene network inference algorithms. IET Systems Biology 2012; 6: 35-63.
[149] Scutari M. Learning Bayesian Networks with the bnlearn R Package. Journal of Statistical Software 2010; 35: 1-22.
[150] L`ebre S. Inferring dynamic genetic networks with low order independencies. Statistical Applications in Genetics and Molecular Biology 2009; 8: 1-39.
[151] Heckerman D. A tutorial on learning with Bayesian networks. Microsoft Research. Technical Report MSR-TR-95- 06. Redmond, Washington, Mar. 1995.
[152] Perrin BE, Ralaivola L, Mazurie A, Bottani S, Mallet J, dAlch´e-Buc F. Gene networks inference using dynamic Bayesian networks. Bioinformatics 2003; 19: ii138-ii148.