{"product_id":"optimal-learning-isbn-9780470596692","title":"Optimal Learning","description":"\u003cb\u003eLearn the science of collecting information to make effective decisions\u003c\/b\u003e  \u003cp\u003eEveryday decisions are made without the benefit of accurate information. \u003ci\u003eOptimal Learning\u003c\/i\u003e develops the needed principles for gathering information to make decisions, especially when collecting information is time-consuming and expensive. Designed for readers with an elementary background in probability and statistics, the book presents effective and practical policies illustrated in a wide range of applications, from energy, homeland security, and transportation to engineering, health, and business.\u003c\/p\u003e \u003cp\u003eThis book covers the fundamental dimensions of a learning problem and presents a simple method for testing and comparing policies for learning. Special attention is given to the knowledge gradient policy and its use with a wide range of belief models, including lookup table and parametric and for online and offline problems. Three sections develop ideas with increasing levels of sophistication:\u003c\/p\u003e \u003cul\u003e \u003cli\u003e\n\u003cb\u003eFundamentals\u003c\/b\u003e explores fundamental topics, including adaptive learning, ranking and selection, the knowledge gradient, and bandit problems\u003c\/li\u003e \u003cli\u003e\n\u003cb\u003eExtensions and Applications\u003c\/b\u003e features coverage of linear belief models, subset selection models, scalar function optimization, optimal bidding, and stopping problems\u003c\/li\u003e \u003cli\u003e\n\u003cb\u003eAdvanced Topics\u003c\/b\u003e explores complex methods including simulation optimization, active learning in mathematical programming, and optimal continuous measurements\u003c\/li\u003e \u003c\/ul\u003e \u003cp\u003eEach chapter identifies a specific learning problem, presents the related, practical algorithms for implementation, and concludes with numerous exercises. A related website features additional applications and downloadable software, including MATLAB and the Optimal Learning Calculator, a spreadsheet-based package that provides an introduc­tion to learning and a variety of policies for learning.\u003c\/p\u003e  Preface xv  \u003cp\u003eAcknowledgments xix\u003c\/p\u003e \u003cp\u003e1 The challenges of learning 1\u003c\/p\u003e \u003cp\u003e1.1 Learning the best path 2\u003c\/p\u003e \u003cp\u003e1.2 Areas of application 4\u003c\/p\u003e \u003cp\u003e1.3 Major problem classes 12\u003c\/p\u003e \u003cp\u003e1.4 The different types of learning 13\u003c\/p\u003e \u003cp\u003e1.5 Learning from different communities 16\u003c\/p\u003e \u003cp\u003e1.6 Information collection using decision trees 18\u003c\/p\u003e \u003cp\u003e1.6.1 A basic decision tree 18\u003c\/p\u003e \u003cp\u003e1.6.2 Decision tree for offline learning 20\u003c\/p\u003e \u003cp\u003e1.6.3 Decision tree for online learning 21\u003c\/p\u003e \u003cp\u003e1.6.4 Discussion 25\u003c\/p\u003e \u003cp\u003e1.7 Website and downloadable software 26\u003c\/p\u003e \u003cp\u003e1.8 Goals of this book 26\u003c\/p\u003e \u003cp\u003eProblems 28\u003c\/p\u003e \u003cp\u003e2 Adaptive learning 31\u003c\/p\u003e \u003cp\u003e2.1 The frequentist view 32\u003c\/p\u003e \u003cp\u003e2.2 The Bayesian view 33\u003c\/p\u003e \u003cp\u003e2.2.1 The updating equations for independent beliefs 34\u003c\/p\u003e \u003cp\u003e2.2.2 The expected value of information 36\u003c\/p\u003e \u003cp\u003e2.2.3 Updating for correlated normal priors 38\u003c\/p\u003e \u003cp\u003e2.2.4 Bayesian updating with an uninformative prior 41\u003c\/p\u003e \u003cp\u003e2.3 Updating for non-Gaussian priors 42\u003c\/p\u003e \u003cp\u003e2.3.1 The gamma-exponential model 43\u003c\/p\u003e \u003cp\u003e2.3.2 The gamma-Poisson model 44\u003c\/p\u003e \u003cp\u003e2.3.3 The Pareto-uniform model 45\u003c\/p\u003e \u003cp\u003e2.3.4 Models for learning probabilities* 46\u003c\/p\u003e \u003cp\u003e2.3.5 Learning an unknown variance* 49\u003c\/p\u003e \u003cp\u003e2.4 Monte Carlo simulation 51\u003c\/p\u003e \u003cp\u003e2.5 Why does it work?* 54\u003c\/p\u003e \u003cp\u003e2.5.1 Derivation of ~_ 54\u003c\/p\u003e \u003cp\u003e2.5.2 Derivation of Bayesian updating equations for independent beliefs 55\u003c\/p\u003e \u003cp\u003e2.6 Bibliographic notes 57\u003c\/p\u003e \u003cp\u003eProblems 57\u003c\/p\u003e \u003cp\u003e3 The economics of information 61\u003c\/p\u003e \u003cp\u003e3.1 An elementary information problem 61\u003c\/p\u003e \u003cp\u003e3.2 The marginal value of information 65\u003c\/p\u003e \u003cp\u003e3.3 An information acquisition problem 68\u003c\/p\u003e \u003cp\u003e3.4 Bibliographic notes 70\u003c\/p\u003e \u003cp\u003eProblems 70\u003c\/p\u003e \u003cp\u003e4 Ranking and selection 71\u003c\/p\u003e \u003cp\u003e4.1 The model 72\u003c\/p\u003e \u003cp\u003e4.2 Measurement policies 75\u003c\/p\u003e \u003cp\u003e4.2.1 Deterministic vs. sequential policies 75\u003c\/p\u003e \u003cp\u003e4.2.2 Optimal sequential policies 76\u003c\/p\u003e \u003cp\u003e4.2.3 Heuristic policies 77\u003c\/p\u003e \u003cp\u003e4.3 Evaluating policies 81\u003c\/p\u003e \u003cp\u003e4.4 More advanced topics* 83\u003c\/p\u003e \u003cp\u003e4.4.1 An alternative representation of the probability space 83\u003c\/p\u003e \u003cp\u003e4.4.2 Equivalence of using true means and sample estimates 84\u003c\/p\u003e \u003cp\u003e4.5 Bibliographic notes 85\u003c\/p\u003e \u003cp\u003eProblems 85\u003c\/p\u003e \u003cp\u003e5 The knowledge gradient 89\u003c\/p\u003e \u003cp\u003e5.1 The knowledge gradient for independent beliefs 90\u003c\/p\u003e \u003cp\u003e5.1.1 Computation 91\u003c\/p\u003e \u003cp\u003e5.1.2 Some properties of the knowledge gradient 93\u003c\/p\u003e \u003cp\u003e5.1.3 The four distributions of learning 94\u003c\/p\u003e \u003cp\u003e5.2 The value of information and the S-curve effect 95\u003c\/p\u003e \u003cp\u003e5.3 Knowledge gradient for correlated beliefs 98\u003c\/p\u003e \u003cp\u003e5.4 The knowledge gradient for some non-Gaussian distributions 103\u003c\/p\u003e \u003cp\u003e5.4.1 The gamma-exponential model 104\u003c\/p\u003e \u003cp\u003e5.4.2 The gamma-Poisson model 107\u003c\/p\u003e \u003cp\u003e5.4.3 The Pareto-uniform model 108\u003c\/p\u003e \u003cp\u003e5.4.4 The beta-Bernoulli model 109\u003c\/p\u003e \u003cp\u003e5.4.5 Discussion 111\u003c\/p\u003e \u003cp\u003e5.5 Relatives of the knowledge gradient 112\u003c\/p\u003e \u003cp\u003e5.5.1 Expected improvement 113\u003c\/p\u003e \u003cp\u003e5.5.2 Linear loss* 114\u003c\/p\u003e \u003cp\u003e5.6 Other issues 116\u003c\/p\u003e \u003cp\u003e5.6.1 Anticipatory vs. experiential learning 117\u003c\/p\u003e \u003cp\u003e5.6.2 The problem of priors 118\u003c\/p\u003e \u003cp\u003e5.6.3 Discussion 120\u003c\/p\u003e \u003cp\u003e5.7 Why does it work?* 121\u003c\/p\u003e \u003cp\u003e5.7.1 Derivation of the knowledge gradient formula 121\u003c\/p\u003e \u003cp\u003e5.8 Bibliographic notes 125\u003c\/p\u003e \u003cp\u003eProblems 126\u003c\/p\u003e \u003cp\u003e6 Bandit problems 139\u003c\/p\u003e \u003cp\u003e6.1 The theory and practice of Gittins indices 141\u003c\/p\u003e \u003cp\u003e6.1.1 Gittins indices in the beta-Bernoulli model 142\u003c\/p\u003e \u003cp\u003e6.1.2 Gittins indices in the normal-normal model 145\u003c\/p\u003e \u003cp\u003e6.1.3 Approximating Gittins indices 147\u003c\/p\u003e \u003cp\u003e6.2 Variations of bandit problems 148\u003c\/p\u003e \u003cp\u003e6.3 Upper confidence bounding 149\u003c\/p\u003e \u003cp\u003e6.4 The knowledge gradient for bandit problems 151\u003c\/p\u003e \u003cp\u003e6.4.1 The basic idea 151\u003c\/p\u003e \u003cp\u003e6.4.2 Some experimental comparisons 153\u003c\/p\u003e \u003cp\u003e6.4.3 Non-normal models 156\u003c\/p\u003e \u003cp\u003e6.5 Bibliographic notes 157\u003c\/p\u003e \u003cp\u003eProblems 157\u003c\/p\u003e \u003cp\u003e7 Elements of a learning problem 163\u003c\/p\u003e \u003cp\u003e7.1 The states of our system 164\u003c\/p\u003e \u003cp\u003e7.2 Types of decisions 166\u003c\/p\u003e \u003cp\u003e7.3 Exogenous information 167\u003c\/p\u003e \u003cp\u003e7.4 Transition functions 168\u003c\/p\u003e \u003cp\u003e7.5 Objective functions 168\u003c\/p\u003e \u003cp\u003e7.5.1 Designing versus controlling 168\u003c\/p\u003e \u003cp\u003e7.5.2 Measurement costs 170\u003c\/p\u003e \u003cp\u003e7.5.3 Objectives 170\u003c\/p\u003e \u003cp\u003e7.6 Evaluating policies 175\u003c\/p\u003e \u003cp\u003e7.7 Discussion 177\u003c\/p\u003e \u003cp\u003e7.8 Bibliographic notes 178\u003c\/p\u003e \u003cp\u003eProblems 178\u003c\/p\u003e \u003cp\u003e8 Linear belief models 181\u003c\/p\u003e \u003cp\u003e8.1 Applications 182\u003c\/p\u003e \u003cp\u003e8.1.1 Maximizing ad clicks 182\u003c\/p\u003e \u003cp\u003e8.1.2 Dynamic pricing 184\u003c\/p\u003e \u003cp\u003e8.1.3 Housing loans 184\u003c\/p\u003e \u003cp\u003e8.1.4 Optimizing dose response 185\u003c\/p\u003e \u003cp\u003e8.2 A brief review of linear regression 186\u003c\/p\u003e \u003cp\u003e8.2.1 The normal equations 186\u003c\/p\u003e \u003cp\u003e8.2.2 Recursive least squares 187\u003c\/p\u003e \u003cp\u003e8.2.3 A Bayesian interpretation 188\u003c\/p\u003e \u003cp\u003e8.2.4 Generating a prior 189\u003c\/p\u003e \u003cp\u003e8.3 The knowledge gradient for a linear model 191\u003c\/p\u003e \u003cp\u003e8.4 Application to drug discovery 192\u003c\/p\u003e \u003cp\u003e8.5 Application to dynamic pricing 196\u003c\/p\u003e \u003cp\u003e8.6 Bibliographic notes 200\u003c\/p\u003e \u003cp\u003eProblems 200\u003c\/p\u003e \u003cp\u003e9 Subset selection problems 203\u003c\/p\u003e \u003cp\u003e9.1 Applications 205\u003c\/p\u003e \u003cp\u003e9.2 Choosing a subset using ranking and selection 206\u003c\/p\u003e \u003cp\u003e9.2.1 Setting prior means and variances 207\u003c\/p\u003e \u003cp\u003e9.2.2 Two strategies for setting prior covariances 208\u003c\/p\u003e \u003cp\u003e9.3 Larger sets 209\u003c\/p\u003e \u003cp\u003e9.3.1 Using simulation to reduce the problem size 210\u003c\/p\u003e \u003cp\u003e9.3.2 Computational issues 212\u003c\/p\u003e \u003cp\u003e9.3.3 Experiments 213\u003c\/p\u003e \u003cp\u003e9.4 Very large sets 214\u003c\/p\u003e \u003cp\u003e9.5 Bibliographic notes 216\u003c\/p\u003e \u003cp\u003eProblems 216\u003c\/p\u003e \u003cp\u003e10 Optimizing a scalar function 219\u003c\/p\u003e \u003cp\u003e10.1 Deterministic measurements 219\u003c\/p\u003e \u003cp\u003e10.2 Stochastic measurements 223\u003c\/p\u003e \u003cp\u003e10.2.1 The model 223\u003c\/p\u003e \u003cp\u003e10.2.2 Finding the posterior distribution 224\u003c\/p\u003e \u003cp\u003e10.2.3 Choosing the measurement 226\u003c\/p\u003e \u003cp\u003e10.2.4 Discussion 229\u003c\/p\u003e \u003cp\u003e10.3 Bibliographic notes 229\u003c\/p\u003e \u003cp\u003eProblems 229\u003c\/p\u003e \u003cp\u003e11 Optimal bidding 231\u003c\/p\u003e \u003cp\u003e11.1 Modeling customer demand 233\u003c\/p\u003e \u003cp\u003e11.1.1 Some valuation models 233\u003c\/p\u003e \u003cp\u003e11.1.2 The logit model 234\u003c\/p\u003e \u003cp\u003e11.2 Bayesian modeling for dynamic pricing 237\u003c\/p\u003e \u003cp\u003e11.2.1 A conjugate prior for choosing between two demand curves 237\u003c\/p\u003e \u003cp\u003e11.2.2 Moment matching for non-conjugate problems 239\u003c\/p\u003e \u003cp\u003e11.2.3 An approximation for the logit model 242\u003c\/p\u003e \u003cp\u003e11.3 Bidding strategies 244\u003c\/p\u003e \u003cp\u003e11.3.1 An idea from multi-armed bandits 245\u003c\/p\u003e \u003cp\u003e11.3.2 Bayes-greedy bidding 245\u003c\/p\u003e \u003cp\u003e11.3.3 Numerical illustrations 247\u003c\/p\u003e \u003cp\u003e11.4 Why does it work?* 251\u003c\/p\u003e \u003cp\u003e11.4.1 Moment matching for Pareto prior 251\u003c\/p\u003e \u003cp\u003e11.4.2 Approximating the logistic expectation 252\u003c\/p\u003e \u003cp\u003e11.5 Bibliographic notes 253\u003c\/p\u003e \u003cp\u003eProblems 254\u003c\/p\u003e \u003cp\u003e12 Stopping problems 255\u003c\/p\u003e \u003cp\u003e12.1 Sequential probability ratio test 255\u003c\/p\u003e \u003cp\u003e12.2 The secretary problem 260\u003c\/p\u003e \u003cp\u003e12.2.1 Setup 261\u003c\/p\u003e \u003cp\u003e12.2.2 Solution 263\u003c\/p\u003e \u003cp\u003e12.3 Bibliographic notes 266\u003c\/p\u003e \u003cp\u003eProblems 266\u003c\/p\u003e \u003cp\u003e13 Active learning in statistics 269\u003c\/p\u003e \u003cp\u003e13.1 Deterministic policies 270\u003c\/p\u003e \u003cp\u003e13.2 Sequential policies for classification 274\u003c\/p\u003e \u003cp\u003e13.2.1 Uncertainty sampling 274\u003c\/p\u003e \u003cp\u003e13.2.2 Query by committee 275\u003c\/p\u003e \u003cp\u003e13.2.3 Expected error reduction 276\u003c\/p\u003e \u003cp\u003e13.3 A variance minimizing policy 277\u003c\/p\u003e \u003cp\u003e13.4 Mixtures of Gaussians 279\u003c\/p\u003e \u003cp\u003e13.4.1 Estimating parameters 280\u003c\/p\u003e \u003cp\u003e13.4.2 Active learning 281\u003c\/p\u003e \u003cp\u003e13.5 Bibliographic notes 283\u003c\/p\u003e \u003cp\u003e14 Simulation optimization 285\u003c\/p\u003e \u003cp\u003e14.1 Indifference zone selection 287\u003c\/p\u003e \u003cp\u003e14.1.1 Batch procedures 288\u003c\/p\u003e \u003cp\u003e14.1.2 Sequential procedures 290\u003c\/p\u003e \u003cp\u003e14.1.3 The 0-1 procedure: connection to linear loss 291\u003c\/p\u003e \u003cp\u003e14.2 Optimal computing budget allocation 292\u003c\/p\u003e \u003cp\u003e14.2.1 Indifference-zone version 293\u003c\/p\u003e \u003cp\u003e14.2.2 Linear loss version 294\u003c\/p\u003e \u003cp\u003e14.2.3 When does it work? 295\u003c\/p\u003e \u003cp\u003e14.3 Model-based simulated annealing 296\u003c\/p\u003e \u003cp\u003e14.4 Other areas of simulation optimization 298\u003c\/p\u003e \u003cp\u003e14.5 Bibliographic notes 299\u003c\/p\u003e \u003cp\u003e15 Learning in mathematical programming 301\u003c\/p\u003e \u003cp\u003e15.1 Applications 303\u003c\/p\u003e \u003cp\u003e15.1.1 Piloting a hot air balloon 303\u003c\/p\u003e \u003cp\u003e15.1.2 Optimizing a portfolio 308\u003c\/p\u003e \u003cp\u003e15.1.3 Network problems 309\u003c\/p\u003e \u003cp\u003e15.1.4 Discussion 313\u003c\/p\u003e \u003cp\u003e15.2 Learning on graphs 313\u003c\/p\u003e \u003cp\u003e15.3 Alternative edge selection policies 316\u003c\/p\u003e \u003cp\u003e15.4 Learning costs for linear programs* 317\u003c\/p\u003e \u003cp\u003e15.5 Bibliographic notes 324\u003c\/p\u003e \u003cp\u003e16 Optimizing over continuous measurements 325\u003c\/p\u003e \u003cp\u003e16.1 The belief model 327\u003c\/p\u003e \u003cp\u003e16.1.1 Updating equations 328\u003c\/p\u003e \u003cp\u003e16.1.2 Parameter estimation 330\u003c\/p\u003e \u003cp\u003e16.2 Sequential kriging optimization 332\u003c\/p\u003e \u003cp\u003e16.3 The knowledge gradient for continuous parameters* 334\u003c\/p\u003e \u003cp\u003e16.3.1 Maximizing the knowledge gradient 334\u003c\/p\u003e \u003cp\u003e16.3.2 Approximating the knowledge gradient 335\u003c\/p\u003e \u003cp\u003e16.3.3 The gradient of the knowledge gradient 336\u003c\/p\u003e \u003cp\u003e16.3.4 Maximizing the knowledge gradient 338\u003c\/p\u003e \u003cp\u003e16.3.5 The KGCP policy 339\u003c\/p\u003e \u003cp\u003e16.4 Efficient global optimization 340\u003c\/p\u003e \u003cp\u003e16.5 Experiments 341\u003c\/p\u003e \u003cp\u003e16.6 Extension to higher dimensional problems 342\u003c\/p\u003e \u003cp\u003e16.7 Bibliographic notes 343\u003c\/p\u003e \u003cp\u003e17 Learning with a physical state 345\u003c\/p\u003e \u003cp\u003e17.1 Introduction to dynamic programming 347\u003c\/p\u003e \u003cp\u003e17.1.1 Approximate dynamic programming 348\u003c\/p\u003e \u003cp\u003e17.1.2 The exploration vs. exploitation problem 350\u003c\/p\u003e \u003cp\u003e17.1.3 Discussion 351\u003c\/p\u003e \u003cp\u003e17.2 Some heuristic learning policies 352\u003c\/p\u003e \u003cp\u003e17.3 The local bandit approximation 353\u003c\/p\u003e \u003cp\u003e17.4 The knowledge gradient in dynamic programming 355\u003c\/p\u003e \u003cp\u003e17.4.1 Generalized learning using basis functions 355\u003c\/p\u003e \u003cp\u003e17.4.2 The knowledge gradient 358\u003c\/p\u003e \u003cp\u003e17.4.3 Experiments 361\u003c\/p\u003e \u003cp\u003e17.5 An expected improvement policy 363\u003c\/p\u003e \u003cp\u003e17.6 Bibliographic notes 364\u003c\/p\u003e \u003cp\u003eIndex 379\u003c\/p\u003e  \u003cp\u003e“He concludes, \"This book collects a number of interesting ideas in optimal learning, allows for connections to be made across disciplines, and is a welcome addition to my bookshelf.”  (\u003ci\u003eInforms Journal on Computing\u003c\/i\u003e, 1 October 2012)\u003c\/p\u003e \u003cp\u003e \u003c\/p\u003e  \u003cp\u003eWARREN B. POWELL, PhD, is Professor of Operations Research and Financial Engineering at Princeton University, where he is founder and Director of CASTLE Laboratory, a research unit that works with industrial partners to test new ideas found in operations research. The recipient of the 2004 INFORMS Fellow Award, Dr. Powell is the author of Approximate Dynamic Programming: Solving the Curses of Dimensionality, Second Edition (Wiley).\u003c\/p\u003e \u003cp\u003eILYA O. RYZHOV, PhD, is Assistant Professor in the Department of Decision, Operations, and Information Technologies at the Robert H. Smith School of Business at the University of Maryland. He has made fundamental contributions to bridge the fields of ranking and selection with multiarmed bandits and optimal learning with mathematical programming.\u003c\/p\u003e  \u003cp\u003eLearn the science of collecting information to make effective decisions\u003c\/p\u003e \u003cp\u003eEveryday decisions are made without the benefit of accurate information. Optimal Learning develops the needed principles for gathering information to make decisions, especially when collecting information is time-consuming and expensive. Designed for readers with an elementary background in probability and statistics, the book presents effective and practical policies illustrated in a wide range of applications, from energy, homeland security, and transportation to engineering, health, and business.\u003c\/p\u003e \u003cp\u003eThis book covers the fundamental dimensions of a learning problem and presents a simple method for testing and comparing policies for learning. Special attention is given to the knowledge gradient policy and its use with a wide range of belief models, including lookup table and parametric and for online and offline problems. Three sections develop ideas with increasing levels of sophistication:\u003c\/p\u003e \u003cul\u003e \u003cli\u003e \u003cp\u003eFundamentals explores fundamental topics, including adaptive learning, ranking and selection, the knowledge gradient, and bandit problems\u003c\/p\u003e \u003c\/li\u003e \u003cli\u003e \u003cp\u003eExtensions and Applications features coverage of linear belief models, subset selection models, scalar function optimization, optimal bidding, and stopping problems\u003c\/p\u003e \u003c\/li\u003e \u003cli\u003e \u003cp\u003eAdvanced Topics explores complex methods including simulation optimization, active learning in mathematical programming, and optimal continuous measurements\u003c\/p\u003e \u003c\/li\u003e \u003c\/ul\u003e \u003cp\u003eEach chapter identifies a specific learning problem, presents the related, practical algorithms for implementation, and concludes with numerous exercises. A related website features additional applications and downloadable software, including MATLAB® and the Optimal Learning Calculator, a spreadsheet-based package that provides an introduction to learning and a variety of policies for learning.\u003c\/p\u003e","brand":"Wiley","offers":[{"title":"Default Title","offer_id":47989721825509,"sku":"NP9780470596692","price":136.95,"currency_code":"USD","in_stock":false}],"thumbnail_url":"\/\/cdn.shopify.com\/s\/files\/1\/1842\/7735\/files\/9780470596692.jpg?v=1761785246","url":"https:\/\/k12savings.com\/es\/products\/optimal-learning-isbn-9780470596692","provider":"K12savings","version":"1.0","type":"link"}