{"product_id":"information-theory-meets-power-laws-isbn-9781119625278","title":"Information Theory Meets Power Laws","description":"\u003cp\u003e\u003cb\u003eDiscover new theoretical connections between stochastic phenomena and the structure of natural language with this powerful volume!\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e\u003ci\u003eInformation Theory Meets Power Laws: Stochastic Processes and Language Models\u003c\/i\u003e presents readers with a novel subtype of a probabilistic approach to language, which is based on statistical laws of texts and their analysis by means of information theory. The distinguished author insightfully and rigorously examines the linguistic and mathematical subject matter while eschewing needlessly abstract and superfluous constructions.\u003c\/p\u003e \u003cp\u003eThe book begins with a less formal treatment of its subjects in the first chapter, introducing its concepts to readers without mathematical training and allowing those unfamiliar with linguistics to learn the book’s motivations. Despite its inherent complexity, \u003ci\u003eInformation Theory Meets Power Laws: Stochastic Processes and Language Models \u003c\/i\u003eis a surprisingly approachable treatment of idealized mathematical models of human language.\u003c\/p\u003e \u003cp\u003eThe author succeeds in developing some of the theory underlying fundamental stochastic and semantic phenomena, like strong nonergodicity, in a way that has not previously been seriously attempted. In doing so, he covers topics including:\u003c\/p\u003e \u003cul\u003e \u003cli\u003eZipf’s and Herdan’s laws for natural language\u003c\/li\u003e \u003cli\u003ePower laws for information, repetitions, and correlations\u003c\/li\u003e \u003cli\u003eMarkov, finite-state,and Santa Fe processes\u003c\/li\u003e \u003cli\u003eBayesian and frequentist  interpretations of probability\u003c\/li\u003e \u003cli\u003eErgodic decomposition, Kolmogorov complexity, and universal coding\u003c\/li\u003e \u003cli\u003eTheorems about facts and words\u003c\/li\u003e \u003cli\u003eInformation measures for fields\u003c\/li\u003e \u003cli\u003eRényi entropies, recurrence times, and subword complexity\u003c\/li\u003e \u003cli\u003eAsymptotically mean stationary processes\u003c\/li\u003e \u003c\/ul\u003e \u003cp\u003eWritten primarily for mathematics graduate students and professionals interested in information theory or discrete stochastic processes, \u003ci\u003eInformation Theory Meets Power Laws: Stochastic Processes and Language Models \u003c\/i\u003ealso belongs on the bookshelves of doctoral students and researchers in artificial intelligence, computational and quantitative linguistics as well as physics of complex systems.\u003c\/p\u003e \u003cp\u003ePreface ix\u003c\/p\u003e \u003cp\u003eAcknowledgments xiii\u003c\/p\u003e \u003cp\u003eBasic Notations xv\u003c\/p\u003e \u003cp\u003e\u003cb\u003e1 Guiding Ideas 1\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e1.1 The Motivating Question 1\u003c\/p\u003e \u003cp\u003e1.2 Further Questions About Texts 5\u003c\/p\u003e \u003cp\u003e1.3 Zipf’s and Herdan’s Laws 8\u003c\/p\u003e \u003cp\u003e1.4 Markov and Finite-State Processes 14\u003c\/p\u003e \u003cp\u003e1.5 More General Stochastic Processes 20\u003c\/p\u003e \u003cp\u003e1.6 Two Interpretations of Probability 23\u003c\/p\u003e \u003cp\u003e1.7 Insights from Information Theory 25\u003c\/p\u003e \u003cp\u003e1.8 Estimation of Entropy Rate 28\u003c\/p\u003e \u003cp\u003e1.9 Entropy of Natural Language 30\u003c\/p\u003e \u003cp\u003e1.10 Algorithmic Information Theory 35\u003c\/p\u003e \u003cp\u003e1.11 Descriptions of a Random World 37\u003c\/p\u003e \u003cp\u003e1.12 Facts and Words Related 43\u003c\/p\u003e \u003cp\u003e1.13 Repetitions and Entropies 47\u003c\/p\u003e \u003cp\u003e1.14 Decay of Correlations 52\u003c\/p\u003e \u003cp\u003e1.15 Recapitulation 54\u003c\/p\u003e \u003cp\u003e\u003cb\u003e2 Probabilistic Preliminaries \u003c\/b\u003e\u003cb\u003e57\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e2.1 Probability Measures 59\u003c\/p\u003e \u003cp\u003e2.2 Product Measurable Spaces 63\u003c\/p\u003e \u003cp\u003e2.3 Discrete Random Variables 65\u003c\/p\u003e \u003cp\u003e2.4 From IID to Finite-State Processes 68\u003c\/p\u003e \u003cp\u003eProblems 73\u003c\/p\u003e \u003cp\u003e\u003cb\u003e3 Probabilistic Toolbox \u003c\/b\u003e\u003cb\u003e77\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e3.1 Borel 𝜎-Fields and a Fair Coin 79\u003c\/p\u003e \u003cp\u003e3.2 Integral and Expectation 83\u003c\/p\u003e \u003cp\u003e3.3 Inequalities and Corollaries 87\u003c\/p\u003e \u003cp\u003e3.4 Semidistributions 92\u003c\/p\u003e \u003cp\u003e3.5 Conditional Probability 94\u003c\/p\u003e \u003cp\u003e3.6 Modes of Convergence 101\u003c\/p\u003e \u003cp\u003e3.7 Complete Spaces 103\u003c\/p\u003e \u003cp\u003eProblems 106\u003c\/p\u003e \u003cp\u003e\u003cb\u003e4 Ergodic Properties \u003c\/b\u003e\u003cb\u003e109\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e4.1 Plain Relative Frequency 111\u003c\/p\u003e \u003cp\u003e4.2 Birkhoff Ergodic Theorem 116\u003c\/p\u003e \u003cp\u003e4.3 Ergodic and Mixing Criteria 119\u003c\/p\u003e \u003cp\u003e4.4 Ergodic Decomposition 125\u003c\/p\u003e \u003cp\u003eProblems 128\u003c\/p\u003e \u003cp\u003e\u003cb\u003e5 Entropy and Information \u003c\/b\u003e\u003cb\u003e131\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e5.1 Shannon Measures for Partitions 133\u003c\/p\u003e \u003cp\u003e5.2 Block Entropy and Its Limits 139\u003c\/p\u003e \u003cp\u003e5.3 Shannon Measures for Fields 145\u003c\/p\u003e \u003cp\u003e5.4 Block Entropy Limits Revisited 155\u003c\/p\u003e \u003cp\u003e5.5 Convergence of Entropy 159\u003c\/p\u003e \u003cp\u003e5.6 Entropy as Self-Information 160\u003c\/p\u003e \u003cp\u003eProblems 163\u003c\/p\u003e \u003cp\u003e\u003cb\u003e6 Equipartition and Universality \u003c\/b\u003e\u003cb\u003e167\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e6.1 SMB Theorem 169\u003c\/p\u003e \u003cp\u003e6.2 Universal Semidistributions 171\u003c\/p\u003e \u003cp\u003e6.3 PPM Probability 172\u003c\/p\u003e \u003cp\u003e6.4 SMB Theorem Revisited 178\u003c\/p\u003e \u003cp\u003e6.5 PPM-based Statistics 180\u003c\/p\u003e \u003cp\u003eProblems 186\u003c\/p\u003e \u003cp\u003e\u003cb\u003e7 Coding and Computation \u003c\/b\u003e\u003cb\u003e189\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e7.1 Elements of Coding 191\u003c\/p\u003e \u003cp\u003e7.2 Kolmogorov Complexity 197\u003c\/p\u003e \u003cp\u003e7.3 Algorithmic Coding Theorems 207\u003c\/p\u003e \u003cp\u003e7.4 Limits of Mathematics 215\u003c\/p\u003e \u003cp\u003e7.5 Algorithmic Randomness 220\u003c\/p\u003e \u003cp\u003eProblems 225\u003c\/p\u003e \u003cp\u003e\u003cb\u003e8 Power Laws for Information \u003c\/b\u003e\u003cb\u003e229\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e8.1 Hilberg Exponents 231\u003c\/p\u003e \u003cp\u003e8.2 Second Order SMB Theorem 238\u003c\/p\u003e \u003cp\u003e8.3 Probabilistic and Algorithmic Facts 241\u003c\/p\u003e \u003cp\u003e8.4 Theorems About Facts and Words 248\u003c\/p\u003e \u003cp\u003eProblems 255\u003c\/p\u003e \u003cp\u003e\u003cb\u003e9 Power Laws for Repetitions \u003c\/b\u003e\u003cb\u003e259\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e9.1 Rényi–Arimoto Entropies 261\u003c\/p\u003e \u003cp\u003e9.2 Generalized Entropy Rates 266\u003c\/p\u003e \u003cp\u003e9.3 Recurrence Times 268\u003c\/p\u003e \u003cp\u003e9.4 Subword Complexity 272\u003c\/p\u003e \u003cp\u003e9.5 Two Maximal Lengths 280\u003c\/p\u003e \u003cp\u003e9.6 Logarithmic Power Laws 284\u003c\/p\u003e \u003cp\u003eProblems 289\u003c\/p\u003e \u003cp\u003e\u003cb\u003e10 AMS Processes \u003c\/b\u003e\u003cb\u003e291\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e10.1 AMS and Pseudo AMS Measures 293\u003c\/p\u003e \u003cp\u003e10.2 Quasiperiodic Coding 295\u003c\/p\u003e \u003cp\u003e10.3 Synchronizable Coding 298\u003c\/p\u003e \u003cp\u003e10.4 Entropy Rate in the AMS Case 301\u003c\/p\u003e \u003cp\u003eProblems 304\u003c\/p\u003e \u003cp\u003e\u003cb\u003e11 Toy Examples \u003c\/b\u003e\u003cb\u003e307\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e11.1 Finite and Ultrafinite Energy 309\u003c\/p\u003e \u003cp\u003e11.2 Santa Fe Processes and Alike 315\u003c\/p\u003e \u003cp\u003e11.3 Encoding into a Finite Alphabet 323\u003c\/p\u003e \u003cp\u003e11.4 Random Hierarchical Association 334\u003c\/p\u003e \u003cp\u003e11.5 Toward Better Models 345\u003c\/p\u003e \u003cp\u003eProblems 348\u003c\/p\u003e \u003cp\u003eFuture Research 349\u003c\/p\u003e \u003cp\u003eBibliography 351\u003c\/p\u003e \u003cp\u003eIndex 365\u003c\/p\u003e  \u003cp\u003e\u003cb\u003eŁUKASZ DĘBOWSKI, P\u003csmall\u003eH\u003c\/small\u003eD,\u003c\/b\u003e works at the Institute of Computer Science of the Polish Academy of Sciences in Poland. His doctorate is in mathematics and computer science and his primary research focus is in the areas of information theory and discrete stochastic processes. He is also interested in the theoretical properties of statistical and neural language models.   \u003c\/p\u003e\u003cp\u003e\u003cb\u003eDiscover new theoretical connections between stochastic phenomena and the structure of natural language with this powerful volume\u003c\/b\u003e \u003c\/p\u003e\u003cp\u003e\u003ci\u003eInformation Theory Meets Power Laws: Stochastic Processes and Language Models\u003c\/i\u003e presents readers with a novel subtype of a probabilistic approach to language, which is based on statistical laws of texts and their analysis by means of information theory. The distinguished author insightfully and rigorously examines the linguistic and mathematical subject matter while eschewing needlessly abstract and superfluous constructions. \u003c\/p\u003e\u003cp\u003eThe book begins with a less formal treatment of its subjects in the first chapter, introducing its concepts to readers without mathematical training and allowing those unfamiliar with linguistics to learn the book's motivations. Despite its inherent complexity, \u003ci\u003eInformation Theory Meets Power Laws: Stochastic Processes and Language Models\u003c\/i\u003e is a surprisingly approachable treatment of idealized mathematical models of human language. \u003c\/p\u003e\u003cp\u003eThe author succeeds in developing some of the theory underlying fundamental stochastic and semantic phenomena, like strong nonergodicity, in a way that has not previously been attempted. In doing so, he covers topics including: \u003c\/p\u003e\u003cul\u003e \u003cli\u003eZipf's and Herdan's laws for natural language\u003c\/li\u003e \u003cli\u003ePower laws for information, repetitions, and correlations\u003c\/li\u003e \u003cli\u003eMarkov, finite-state, and Santa Fe processes\u003c\/li\u003e \u003cli\u003eBayesian and frequentist interpretations of probability\u003c\/li\u003e \u003cli\u003eErgodic decomposition, Kolmogorov complexity, and universal coding\u003c\/li\u003e \u003cli\u003eTheorems about facts and words\u003c\/li\u003e \u003cli\u003eInformation measures for fields\u003c\/li\u003e \u003cli\u003eRényi entropies, recurrence times, and subword complexity\u003c\/li\u003e \u003cli\u003eAsymptotically mean stationary processes\u003c\/li\u003e \u003c\/ul\u003e \u003cp\u003eWritten primarily for mathematics graduate students and professionals interested in information theory or discrete stochastic processes, \u003ci\u003eInformation Theory Meets Power Laws: Stochastic Processes and Language Models\u003c\/i\u003e also belongs on the bookshelves of doctoral students and researchers in artificial intelligence, computational and quantitative linguistics as well as physics of complex systems.\u003c\/p\u003e","brand":"Wiley","offers":[{"title":"Default Title","offer_id":47989423341797,"sku":"NP9781119625278","price":120.95,"currency_code":"USD","in_stock":false}],"thumbnail_url":"\/\/cdn.shopify.com\/s\/files\/1\/1842\/7735\/files\/9781119625278.jpg?v=1761784046","url":"https:\/\/k12savings.com\/products\/information-theory-meets-power-laws-isbn-9781119625278","provider":"K12savings","version":"1.0","type":"link"}