Inverse Ising techniques to infer underlying mechanisms from data

doi:10.1088/1674-1056/ab8da6

Abstract

As a problem in data science the inverse Ising (or Potts) problem is to infer the parameters of a Gibbs-Boltzmann distributions of an Ising (or Potts) model from samples drawn from that distribution. The algorithmic and computational interest stems from the fact that this inference task cannot be carried out efficiently by the maximum likelihood criterion, since the normalizing constant of the distribution (the partition function) cannot be calculated exactly and efficiently. The practical interest on the other hand flows from several outstanding applications, of which the most well known has been predicting spatial contacts in protein structures from tables of homologous protein sequences. Most applications to date have been to data that has been produced by a dynamical process which, as far as it is known, cannot be expected to satisfy detailed balance. There is therefore no a priori reason to expect the distribution to be of the Gibbs-Boltzmann type, and no a priori reason to expect that inverse Ising (or Potts) techniques should yield useful information. In this review we discuss two types of problems where progress nevertheless can be made. We find that depending on model parameters there are phases where, in fact, the distribution is close to Gibbs-Boltzmann distribution, a non-equilibrium nature of the under-lying dynamics notwithstanding. We also discuss the relation between inferred Ising model parameters and parameters of the underlying dynamics.

Keywords: inverse Ising problem kinetic Ising model statistical genetics fitness reconstruction

Received: 09 March 2020
Revised: 09 March 2020
Accepted manuscript online:

PACS:	02.50.Tt	(Inference methods)
	05.40.-a	(Fluctuation phenomena, random processes, noise, and Brownian motion)
	05.45.Tp	(Time series analysis)
	05.90.+m	(Other topics in statistical physics, thermodynamics, and nonlinear dynamical systems)

Fund:

Project supported partially by the National Natural Science Foundation of China (Grant No. 11705097), the Natural Science Foundation of Jiangsu Province of China (Grant No. BK20170895), the Jiangsu Government Scholarship for Overseas Studies of 2018 and Scientific Research Foundation of Nanjing University of Posts and Telecommunications, China (Grant No. NY217013), and the Foundation for Polish Science through TEAM-NET Project (Grant No. POIR.04.04.00-00-17C1/18-00).

Corresponding Authors: Hong-Li Zeng, Erik Aurell
E-mail: hlzeng@njupt.edu.cn;eaurell@kth.se

Cite this article:

Hong-Li Zeng(曾红丽), Erik Aurell Inverse Ising techniques to infer underlying mechanisms from data 2020 Chin. Phys. B 29 080201

[1]	Mézard M, Parisi G and Virasoro M A 1987 Spin Glass Theory and beyond:An Introduction to the Replica Method and Its Applications (Singapore:World Scientific)
[2]	Fischer K and Hertz J A 1991 Spin Glasses (Cambridge:Cambridge University Press)
[3]	Mézard M and Montanari A 2009 Information, Physics, and Computation (Oxford:Oxford University Press)
[4]	Schneidman E, Berry M J, Segev R and Bialek W 2006 Nature 440 1007
[5]	Roudi Y, Aurell E and Hertz J A 2009 Front. Comput. Neurosci. 3 22
[6]	Nguyen H C, Zecchina R and Berg J 2017 Adv. Phys. 66 197
[7]	Wainwright M J and Jordan M I 2007 Found. Trends Mach. Learn. 1 1
[8]	Darmois G 1935 C. R. Acad. Sci. Paris 200 1265
[9]	Koopman B 1936 Trans. Am. Math. Soc. 39 399
[10]	Jaynes E T 1957 Phys. Rev. 106 620
[11]	Aurell E 2016 PLOS Comput. Biol. 12 1
[12]	van Nimwegen E 2016 PLOS Comput. Biol. 12 e1004726
[13]	Amari S I, Barndorff-Nielsen O E, Kass R E, Lauritzen S L and Rao C R 1987 Differential Geometry in Statistical Inference in Lecture Notes-Monograph Series Vol. 10 Chap. 2
[14]	Amari S I and Nagaoka H 2000 Methods Inf. Geom. Translations Mathematical Monographs V. 191 (New York:American Mathematical Society)
[15]	Kampen N V 2007 Stochastic Processes in Physics and Chemistry 3rd edn (Amerstamdam:Elsevier)
[16]	Kuramoto Y 1984 Chemical Oscillations, Waves, and Turbulence (Springer Series in Synergistics) (Berlin:Springer)
[17]	Goldbeter A 2010 Biochemical Oscillations and Cellular Rhythms:The Molecular Bases of Periodic and Chaotic Behaviour (Cambridge:Cambridge University Press)
[18]	Papadimitriou C H 1991 Proceedings 32nd Annual Symposium of Foundations of Computer Science October 1-4, 1991 San Juan, Puerto Rico, USA, pp. 163-169
[19]	Selman B, Kautz H and Cohen B 1996 Local Search Strategies for Satisfiability Testing in DIMACS Series in Discrete Mathematics and Theoretical Computer Science Vol. 26
[20]	Barthel W, Hartmann A K and Weigt M 2003 Phys. Rev. E 67 066104
[21]	Aurell E, Gordon U and Kirkpatrick S 2004 Comparing Beliefs, Surveys and Random Walks in Neural Information Processing Systems 2004 (Vancouver, Canada)
[22]	Seitz S, Alava M and Orponen P 2005 J. Stat. Mech.:Theory Exp. 2005 P06006
[23]	Alava M, Ardelius J, Aurell E, Kaski P, Krishnamurthy S, Orponen P and Seitz S 2008 Proc. Natl. Acad. Sci. USA 105 15253
[24]	Kautz H and Selman B 2007 Disc. Appl. Math. 155 1514
[25]	Lemoy R, Alava M and Aurell E 2015 Phys. Rev. E 91 013305
[26]	Aurell E, Domínguez E, Machado D and Mulet R 2019 Phys. Rev. Lett. 123 230602
[27]	Kree R and Zippelius A 1987 Phys. Rev. A 36 4421
[28]	Braunstein A, Ramezanpour A, Zecchina R and Zhang P 2011 Phys. Rev. E 83 056114
[29]	Nguyen H C and Berg J 2012 Phys. Rev. Lett. 109 050602
[30]	Bertini L, De Sole A, Gabrielli D, Jona-Lasinio G and Landim C 2002 J. Stat. Phys. 107 635
[31]	Derrida B 2007 J. Stat. Mech.:Theory Exp. 2007 P07023
[32]	Dettmer S L, Nguyen H C and Berg J 2016 Phys. Rev. E 94 052116
[33]	Berg J 2017 J. Stat. Mech.:Theory Exp. 2017 083402
[34]	Dettmer S L and Berg J 2018 J. Stat. Mech.:Theory Exp. 2018 023403
[35]	Tkačik G, Marre O, Amodei D, Schneidman E, Bialek W and Berry Michael J n 2014 PLOS Comput. Biol. 10 e1003408
[36]	Cocco S, Feinauer C, Figliuzzi M, Monasson R and Weigt M 2018 Rep. Prog. Phys. 81 032601
[37]	Parisi G 1986 J. Phys. A:Math. Gen. 19 L675
[38]	Roudi Y, Tyrcha J and Hertz J 2009 Phys. Rev. E 79 051915
[39]	Ackley D H, Hinton G E and Sejnowski T J 1985 Cogn. Sci. 9 147
[40]	Kappen H and Rodríguez F B 1998 Neural Comput. 10 1137
[41]	Thouless D J, Anderson P W and Palmer R G 1977 Philos. Mag. 35 593
[42]	Mézard M and Mora T 2009 J. Physiol.-Paris 103 107
[43]	Weigt M, White R A, Szurmant H, Hoch J A and Hwa T 2009 Proc. Natl. Acad. Sci. USA 106 67
[44]	Ricci-Tersenghi F 2012 J. Stat. Mech.:Theory Exp. 2012 P08015
[45]	Besag J 1975 J. R. Stat. Soc. D 24 179
[46]	Morcos F, Pagnani A, Lunt B, Bertolino A, Marks D S, Sander C, Zecchina R, Onuchic J N, Hwa T and Weigt M 2011 Proc. Natl. Acad. Sci. USA 108 E1293
[47]	Marks D S, Colwell L J, Sheridan R, Hopf T A, Pagnani A, Zecchina R and Sander C 2011 PLoS ONE 6 e28766
[48]	Jones D T, Buchan D W A, Cozzetto D and Pontil M 2012 Bioinformatics 28 184
[49]	Andreatta M, Laplagne S, Li S C and Smale S 2014 arXiv 1311.1301v3
[50]	Kamisetty H, Ovchinnikov S and Baker D 2013 Proc. Natl. Acad. Sci. 110 15674
[51]	Ekeberg M, Lövkvist C, Lan Y, Weigt M and Aurell E 2013 Phys. Rev. E 87 012707
[52]	Ekeberg M, Hartonen T and Aurell E 2014 J. Comput. Phys. 276 341
[53]	Ravikumar P, Wainwright M J and Lafferty J D 2010 Ann. Stat. 38 1287
[54]	Bento J and Montanari A 2009 Proceedings of the 22nd International Conference on Neural Information Processing Systems NIPS-09(Red Hook, NY, USA:Curran Associates Inc.) pp. 1303-1311
[55]	Lokhov A Y, Vuffray M, Misra S and Chertkov M 2018 Sci. Adv. 4 e1700791
[56]	Santhanam N P and Wainwright M J 2012 IEEE Trans. Inf. Theory 58 4117
[57]	Vuffray M, Misra S, Lokhov A and Chertkov M 2016 Advances in Neural Information Processing Systems 29 edited by Lee D D, Sugiyama M, Luxburg U V, Guyon I and Garnett R (Curran Associates, Inc.) pp. 2595-2603
[58]	Goel S, Kane D M and Klivans A R 2019 Proceedings of the Thirty-Second Conference on Learning Theory in Proceedings of Machine Learning Research edited by Beygelzimer A and Hsu D (Phoenix, USA:PMLR) Vol. 99 pp. 1449-1469
[59]	Vuffray M, Misra S and Lokhov A Y 2019 Efficient learning of discrete graphical models arXiv:1902.00600
[60]	Xu Y, Aurell E, Corander J and Kabashima Y 2017 arXiv:1704.01459[physics.data-an]
[61]	Xu Y, Puranen S, Corander J and Kabashima Y 2018 Phys. Rev. E 97 062112
[62]	Stein R R, Marks D S and Sander C 2015 PLOS Comput. Biol. 11 e1004182
[63]	Glauber R J 1963 J. Math. Phys. 4 294
[64]	Suzuki M and Kubo R 1968 J. Phys. Soc. Jpn. 24 51
[65]	Gillespie D T 1977 J. Phys. Chem. 81 2340
[66]	Sherrington D and Kirkpatrick S 1975 Phys. Rev. Lett. 35 1792
[67]	Crisanti A and Sompolinsky H 1987 Phys. Rev. A 36 4922
[68]	Dijkstra E W 1974 Commun. ACM 17 643
[69]	Aurell E 2013 J. Phys. Conf. 473 012017
[70]	Decelle A and Zhang P 2015 Phys. Rev. E 91 052136
[71]	Shlens J, Field G, Gauthier J, Grivich M, Petrusca D, Sher A, Litke A and Chichilnisky E 2006 J. Neurosci. 26 8254
[72]	Cocco S, Leibler S and Monasson R 2009 Proc. Natl. Acad. Sci. USA 106 14058
[73]	Roudi Y and Hertz J 2011 Phys. Rev. Lett. 106 048702
[74]	Roudi Y and Hertz J 2011 J. Stat. Mech.:Theory Exp. 2011 P03031
[75]	Mézard M and Sakellariou J 2011 J. Stat. Mech.:Theory Exp. 2011 L07001
[76]	Zhang P 2012 J. Stat. Phys. 148 502
[77]	Kappen H J and Spanjers J J 2000 Phys. Rev. E 61 5658
[78]	Aurell E and Mahmoudi H 2012 Phys. Rev. E 85 031119
[79]	Pillow J W, Shlens J, Paninski L, Sher A, Litke A M, Chichilnisky E and Simoncelli E P 2008 Nature 454 995
[80]	Mastromatteo I and Marsili M 2011 J. Stat. Mech.:Theory Exp. 2011 P10012
[81]	Zeng H L, Aurell E, Alava M and Mahmoudi H 2011 Phys. Rev. E 83 041135
[82]	Zeng H L, Alava M, Aurell E, Hertz J and Roudi Y 2013 Phys. Rev. Lett. 110 210601
[83]	Kipnis C and Landim C 1999 Scaling Limits of Interacting Particle Systems (Berlin:Springer-Verlag) Vol. 320
[84]	Wainwright M J, Ravikumar P and Lafferty J D 2007 Adv. Neural. Inf. Process. Syst. 19 1465
[85]	Zeng H L, Hertz J and Roudi Y 2014 Phys. Scr. 89 105002
[86]	Zeng H L, Lemoy R and Alava M 2014 J. Stat. Mech.:Theory Exp. 2014 P07008
[87]	Roudi Y, Dunn B and Hertz J 2015 Curr. Opin. Neurobiology 32 38
[88]	Cocco S, Monasson R, Posani L and Tavoni G 2017 Curr. Opin. Struct. Biol. 3 103
[89]	Huang H 2017 J. Stat. Mech.:Theory Exp. 2017 033501
[90]	Poli D, Pastore V P, Martinoia S and Massobrio P 2016 J. Neural. Eng. 13 026023
[91]	Latimer K W, Rieke F and Pillow J W 2019 ELife 8 e47012
[92]	Sadeghi K and Berry M J 2020 BioRxiv
[93]	Hoang D T, Song J, Periwal V and Jo J 2019 Phys. Rev. E 99 023311
[94]	Bacry E, Mastromatteo I and Muzy J F 2015 Market Microstruct. Liquidity 01 1550005
[95]	Ma J, Wang L and Wang T 2015 The 27th Chinese Control and Decision Conference (2015 CCDC) (IEEE) May 23-25, 2015, Qingdao, China, pp. 238-243
[96]	Borysov S S, Roudi Y and Balatsky A V 2015 Eur. Phys. J. B 88 321
[97]	Zarinelli E, Treccani M, Farmer J D and Lillo F 2015 Market Microstruct. Liquidity 01 1550004
[98]	Li S, He J and Song K 2016 Entropy 18 331
[99]	Fan Y, Yu G, He Z, Yu H, Bai R, Yang L and Wu D 2017 Entropy 19 51
[100]	Zhao L, Bao W and Li W 2018 J. Phys. Conf. 1113 012009
[101]	Becker A P 2018 Maximum entropy and network approaches to systemic risk and foreign exchange, PhD thesis (Boston:Boston University)
[102]	Alossaimy A N M and Stemler T 2019 Using Complex Networks to Uncover Interaction in Stock Markets, PhD thesis (The University of Western Australia)
[103]	Bucci F, Benzaquen M, Lillo F and Bouchaud J P 2019 arXiv:1901.05332
[104]	Hoffmann T, Peel L, Lambiotte R and Jones N S 2020 Sci. Adv. 6 eaav1478
[105]	Ikeda Y and Takeda H 2020 arXiv:2001.04097
[106]	Segev R, Puchalla J and Berry M J 2006 J. Neurophysiol. 95 2277
[107]	Mastromatteo I and Marsili M 2011 J. Stat. Mech.:Theory Exp. 2011 P10012
[108]	Tanaka T 2000 Neural Comput. 12 1951
[109]	Bury T 2013 Eur. Phys. J. B 86 1
[110]	Bouchaud J P and Potters M 2003 Theory of Financial Risk and Derivative Pricing:From Statistical Physics to Risk Management (Cambridge:Cambridge University Press)
[111]	Mantegna R N and Stanley H E 2003 An Introduction to Econophysics:Correlations and Complexity in Finance (Cambridge:Cambridge University Press)
[112]	Biely C and Thurner S 2008 Quant. Finance 8 705
[113]	Zeng H L 2014 Connectivity Inference with Asynchronously Updated Kinetic Ising Models, PhD thesis (Finland:Aalto University)
[114]	Kullmann L, Kertész J and Kaski K 2002 Phys. Rev. E 66 026125
[115]	Mann J K, Barton J P,Ferguson A L,Omarjee S, Walker B D, Chakraborty A and Ndung'u T 2014 PLoS Comput. Biol. 10 e1003776
[116]	Phillips P C 2008 Nat. Rev. Genet. 9 855
[117]	Gueudré T, Baldassi C, Zamparo M, Weigt M and Pagnani A 2016 Proc. Natl. Acad. Sci. USA 113 12186
[118]	Uguzzoni G, John Lovis S, Oteri F, Schug A, Szurmant H and Weigt M 2017 Proc. Natl. Acad. Sci. USA 114 E2662-E2671
[119]	Pfam 32.02018 http://pfam.xfam.org/
[120]	El-Gebali S, Mistry J, Bateman A, Eddy S R, Luciani A, Potter S C, Qureshi M, Richardson L J, Salazar G A, Smart A, Sonnhammer E L, Hirsh L, Paladin L, Piovesan D, Tosatto S C and Finn R D 2019 Nucleic Acids Res. 47 D427
[121]	Ovchinnikov S, Park H, Varghese N, Huang P S, Pavlopoulos G A, Kim D E, Kamisetty H, Kyrpides N C and Baker D 2017 Science 355 294
[122]	Michel M, Menéndez Hurtado D, Uziela K and Elofsson A 2017 Bioinformatics 33 i23
[123]	Ovchinnikov S, Park H, Kim D E, DiMaio F and Baker D 2018 Proteins 86 113
[124]	Senior A W, Jumper J, Hassabis D and Kohli P 2020 AlphaFold:Using AI for Scientific Discovery
[125]	Senior A W, Evans R, Jumper J, Kirkpatrick J, Sifre L, Green T, Qin C, Ž ídek A, Nelson A W, Bridgland A et al. 2020 Nature 577 706
[126]	De Leonardis E, Lutz B, Ratz S, Simona C, Monasson R, Weigt M and Schug A 2015 Nucleic Acids Res. 43 10444
[127]	Weinreb C, Riesselman A J, Ingraham J B, Gross T, Sander C and Marks D S 2016 Cell 165 963
[128]	Ferguson A L, Mann J K, Omarjee S, Ndung'u T, Walker B D and Chakraborty A K 2013 Immunity 38 606
[129]	Shekhar K, Ruberman C F, Ferguson A L, Barton J P, Kardar M and Chakraborty A K 2013 Phys. Rev. E 88 062705
[130]	Louie R H Y, Kaczorowski K J, Barton J P, Chakraborty A K and McKay M R 2018 Proc. Natl. Acad. Sci. USA 115 E564
[131]	Figliuzzi M, Jacquier H, Schug A, Tenaillon O and Weigt M 2016 Mol. Biol. Evol. 33 268
[132]	Hopf T A, Ingraham J B, Poelwijk F J, Scharfe C P I, Springer M, Sander C and Marks D S 2017 Nat. Biotechnol. 35 128
[133]	Couce A, Caudwell L V, Feinauer C, Hindré T, Feugeas J P, Weigt M, Lenski R E, Schneider D and Tenaillon O 2017 Proc. Natl. Acad. Sci. USA 114 E9026-E9035
[134]	Skwark M J, Croucher N J, Puranen S, Chewapreecha C, Pesonen M, Xu Y Y, Turner P, Harris S R, Beres S B, Musser J M, Parkhill J, Bentley S D, Aurell E and Corander J 2017 PLoS Genet. 13 e1006508
[135]	Schubert B, Maddamsetti R, Nyman J, Farhat M R and Marks D S 2018 BioRxiv 325993
[136]	Puranen S, Pesonen M, Pensar J, Xu Y Y, Lees J A, Bentley S D, Croucher N J and Corander J 2018 Microb. Genom. 4
[137]	Gao C Y, Zhou H J and Aurell E 2018 Phys. Rev. E 98 032407
[138]	Pensar J, Puranen S, Arnold B, MacAlasdair N, Kuronen J, Tonkin-Hill G, Pesonen M, Xu Y, Sipola A, Sánchez-Busó L, Lees J A, Chewapreecha C, Bentley S D, Harris S R, Parkhill J, Croucher N J and Corander J 2019 Nucleic Acids Res. 47 e112
[139]	Hakenbeck R, Brückner R, Denapaite D and Maurer P 2012 Future Microbiol. 7 395
[140]	Fisher R A 1922 P. Roy. Soc. Edinb. 42 321
[141]	Fisher R 1930 The Genetical Theory of Natural Selection (Oxford:Clarendon)
[142]	Kolmogorov A N 1935 Dokl. Akad. Nauk. SSSR 3 129
[143]	Peliti L 1997 arXiv:cond-mat/9712027
[144]	Blythe R A and McKane A J 2007 J. Stat. Mech.:Theory Exp. 2007 P07018
[145]	Chaguza C, Andam C P, Harris S R et al. 2016 MBio 7 e01053
[146]	Eigen M 1971 Naturwissenschaften 58 465
[147]	Eigen M 2002 Proc. Natl. Acad. Sci. USA 99 13374
[148]	Maynard Smith J 1982 Evolution and the Theory of Games (Cambridge:Cambridge University Press)
[149]	Nowak M A and Sigmund K 2004 Science 303 793
[150]	Claussen J C and Traulsen A 2008 Phys. Rev. Lett. 100 058104
[151]	Wang Z, Xu B and Zhou H J 2015 Sci. Rep. 4 5830
[152]	Liao M J, Din M O, Tsimring L and Hasty J 2019 Science 365 1045
[153]	Shahshahani S 1979 A New Mathematical Framework for the Study of Linkage and Selection (New York:American Mathematical Society)
[154]	Bürger R 2000 The Mathematical Theory of Selection, Recombination, and Mutations (New York:Wiley) Vol. 228
[155]	Svirezhev Y and Passekov V 2012 Fundamentals of Mathematical Evolutionary Genetics (Berlin:Springer) Vol. 22
[156]	Huillet T E 2017 J. Stat. Phys. 168 15
[157]	Aurell E, Ekeberg M and Koski T 2019 arXiv:1906.00716
[158]	Neher R A and Shraiman B I 2011 Rev. Mod. Phys. 83 1283
[159]	Gao C Y, Cecconi F, Vulpiani A, Zhou H J and Aurell E 2019 Phys. Biol. 16 026002
[160]	Yahara K, Didelot X, Ansari M A, Sheppard S K and Falush D 2014 Mol. Biol. Evol. 31 1593
[161]	Chewapreecha C, Harris S R, Croucher N J et al. 2014 Nat. Genet. 46 305
[162]	Kimura M 1956 Evolution 10 278
[163]	Kimura M 1964 J. Appl. Probab. 1 177
[164]	Kimura M 1965 Genetics 52 875
[165]	Neher R A and Shraiman B I 2009 Proc. Natl. Acad. Sci. USA 106 6866
[166]	Hardy G H, et al. 1908 Science 28 49
[167]	Weinberg W 1908 Jahresh. Ver. Vaterl. Naturkd. Württemb 64 368
[168]	Neher R and Zanini F 2012 FFPopSim
[169]	Zeng H L and Aurell E 2020 Phys. Rev. E 101 052409
[170]	Thornton K R 2014 Genetics 198 157
[171]	Arnold B, Sohail M, Wadsworth C, Corander J, Hanage W P, Sunyaev S and Grad Y H 2020 Mol. Biol. Evol. 37 417
[172]	Thorell K, Yahara K, Berthenet E et al. 2017 PLoS Genet. 13 e1006730

[1]	Dynamic compensation temperature in kinetic spin-5/2 Ising model on hexagonal lattice Ümüt Temizer, Ayşegül Özkılıç. Chin. Phys. B, 2013, 22(3): 037501.
[2]	Kinetic Ising model in a time-dependent oscillating external magnetic field: effective-field theory Bayram Deviren, Osman Canko, and Mustafa Keskin. Chin. Phys. B, 2010, 19(5): 050518.

No Suggested Reading articles found!

Viewed

Full text

Abstract

Cited

Metrics
Related Articles

Inverse Ising techniques to infer underlying mechanisms from data

Cite this article:

Online attention