{"product_id":"an-introduction-to-optimization-isbn-9781119877639","title":"An Introduction to Optimization","description":"\u003cb\u003eAn Introduction to Optimization\u003c\/b\u003e \u003cp\u003e\u003cb\u003eAccessible introductory textbook on optimization theory and methods, with an emphasis on engineering design, featuring MATLAB\u003csup\u003e®\u003c\/sup\u003e exercises and worked examples\u003c\/b\u003e \u003c\/p\u003e\u003cp\u003eFully updated to reflect modern developments in the field, the Fifth Edition of \u003ci\u003eAn Introduction to Optimization\u003c\/i\u003e fills the need for an accessible, yet rigorous, introduction to optimization theory and methods, featuring innovative coverage and a straightforward approach. The book begins with a review of basic definitions and notations while also providing the related fundamental background of linear algebra, geometry, and calculus.  \u003c\/p\u003e\u003cp\u003eWith this foundation, the authors explore the essential topics of unconstrained optimization problems, linear programming problems, and nonlinear constrained optimization. In addition, the book includes an introduction to artificial neural networks, convex optimization, multi-objective optimization, and applications of optimization in machine learning. \u003c\/p\u003e\u003cp\u003eNumerous diagrams and figures found throughout the book complement the written presentation of key concepts, and each chapter is followed by MATLAB\u003csup\u003e®\u003c\/sup\u003e exercises and practice problems that reinforce the discussed theory and algorithms. \u003c\/p\u003e\u003cp\u003eThe Fifth Edition features a new chapter on Lagrangian (nonlinear) duality, expanded coverage on matrix games, projected gradient algorithms, machine learning, and numerous new exercises at the end of each chapter.  \u003c\/p\u003e\u003cp\u003e\u003ci\u003eAn Introduction to Optimization\u003c\/i\u003e includes information on: \u003c\/p\u003e\u003cul\u003e\n\u003cli\u003eThe mathematical definitions, notations, and relations from linear algebra, geometry, and calculus used in optimization\u003c\/li\u003e \u003cli\u003eOptimization algorithms, covering one-dimensional search, randomized search, and gradient, Newton, conjugate direction, and quasi-Newton methods\u003c\/li\u003e \u003cli\u003eLinear programming methods, covering the simplex algorithm, interior point methods, and duality \u003c\/li\u003e\n\u003cli\u003eNonlinear constrained optimization, covering theory and algorithms, convex optimization, and Lagrangian duality\u003c\/li\u003e \u003cli\u003eApplications of optimization in machine learning, including neural network training, classification, stochastic gradient descent, linear regression, logistic regression, support vector machines, and clustering.\u003c\/li\u003e\n\u003c\/ul\u003e \u003cp\u003e\u003ci\u003eAn Introduction to Optimization\u003c\/i\u003e is an ideal textbook for a one- or two-semester senior undergraduate or beginning graduate course in optimization theory and methods. The text is also of value for researchers and professionals in mathematics, operations research, electrical engineering, economics, statistics, and business. \u003c\/p\u003e\u003cp\u003ePreface xv\u003c\/p\u003e \u003cp\u003eAbout the Companion Website xviii\u003c\/p\u003e \u003cp\u003e\u003cb\u003ePart I Mathematical Review 1\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e\u003cb\u003e1 Methods of Proof and Some Notation 3\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e1.1 Methods of Proof 3\u003c\/p\u003e \u003cp\u003e1.2 Notation 5\u003c\/p\u003e \u003cp\u003eExercises 5\u003c\/p\u003e \u003cp\u003e\u003cb\u003e2 Vector Spaces and Matrices 7\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e2.1 Vector and Matrix 7\u003c\/p\u003e \u003cp\u003e2.2 Rank of a Matrix 11\u003c\/p\u003e \u003cp\u003e2.3 Linear Equations 16\u003c\/p\u003e \u003cp\u003e2.4 Inner Products and Norms 18\u003c\/p\u003e \u003cp\u003eExercises 20\u003c\/p\u003e \u003cp\u003e\u003cb\u003e3 Transformations 23\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e3.1 Linear Transformations 23\u003c\/p\u003e \u003cp\u003e3.2 Eigenvalues and Eigenvectors 24\u003c\/p\u003e \u003cp\u003e3.3 Orthogonal Projections 26\u003c\/p\u003e \u003cp\u003e3.4 Quadratic Forms 27\u003c\/p\u003e \u003cp\u003e3.5 Matrix Norms 32\u003c\/p\u003e \u003cp\u003eExercises 35\u003c\/p\u003e \u003cp\u003e\u003cb\u003e4 Concepts from Geometry 39\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e4.1 Line Segments 39\u003c\/p\u003e \u003cp\u003e4.2 Hyperplanes and Linear Varieties 39\u003c\/p\u003e \u003cp\u003e4.3 Convex Sets 41\u003c\/p\u003e \u003cp\u003e4.4 Neighborhoods 43\u003c\/p\u003e \u003cp\u003e4.5 Polytopes and Polyhedra 44\u003c\/p\u003e \u003cp\u003eExercises 45\u003c\/p\u003e \u003cp\u003e\u003cb\u003e5 Elements of Calculus 47\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e5.1 Sequences and Limits 47\u003c\/p\u003e \u003cp\u003e5.2 Differentiability 52\u003c\/p\u003e \u003cp\u003e5.3 The Derivative Matrix 54\u003c\/p\u003e \u003cp\u003e5.4 Differentiation Rules 57\u003c\/p\u003e \u003cp\u003e5.5 Level Sets and Gradients 58\u003c\/p\u003e \u003cp\u003e5.6 Taylor Series 61\u003c\/p\u003e \u003cp\u003eExercises 65\u003c\/p\u003e \u003cp\u003e\u003cb\u003ePart II Unconstrained Optimization 67\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e\u003cb\u003e6 Basics of Set-Constrained and Unconstrained Optimization 69\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e6.1 Introduction 69\u003c\/p\u003e \u003cp\u003e6.2 Conditions for Local Minimizers 70\u003c\/p\u003e \u003cp\u003eExercises 78\u003c\/p\u003e \u003cp\u003e\u003cb\u003e7 One-Dimensional Search Methods 87\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e7.1 Introduction 87\u003c\/p\u003e \u003cp\u003e7.2 Golden Section Search 87\u003c\/p\u003e \u003cp\u003e7.3 Fibonacci Method 91\u003c\/p\u003e \u003cp\u003e7.4 Bisection Method 97\u003c\/p\u003e \u003cp\u003e7.5 Newton’s Method 98\u003c\/p\u003e \u003cp\u003e7.6 Secant Method 101\u003c\/p\u003e \u003cp\u003e7.7 Bracketing 103\u003c\/p\u003e \u003cp\u003e7.8 Line Search in Multidimensional Optimization 103\u003c\/p\u003e \u003cp\u003eExercises 105\u003c\/p\u003e \u003cp\u003e\u003cb\u003e8 Gradient Methods 109\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e8.1 Introduction 109\u003c\/p\u003e \u003cp\u003e8.2 Steepest Descent Method 110\u003c\/p\u003e \u003cp\u003e8.3 Analysis of Gradient Methods 117\u003c\/p\u003e \u003cp\u003eExercises 126\u003c\/p\u003e \u003cp\u003e\u003cb\u003e9 Newton’s Method 133\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e9.1 Introduction 133\u003c\/p\u003e \u003cp\u003e9.2 Analysis of Newton’s Method 135\u003c\/p\u003e \u003cp\u003e9.3 Levenberg–Marquardt Modification 138\u003c\/p\u003e \u003cp\u003e9.4 Newton’s Method for Nonlinear Least Squares 139\u003c\/p\u003e \u003cp\u003eExercises 142\u003c\/p\u003e \u003cp\u003e\u003cb\u003e10 Conjugate Direction Methods 145\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e10.1 Introduction 145\u003c\/p\u003e \u003cp\u003e10.2 Conjugate Direction Algorithm 146\u003c\/p\u003e \u003cp\u003e10.2.1 Basic Conjugate Direction Algorithm 146\u003c\/p\u003e \u003cp\u003e10.3 Conjugate Gradient Algorithm 151\u003c\/p\u003e \u003cp\u003e10.4 Conjugate Gradient Algorithm for Nonquadratic Problems 154\u003c\/p\u003e \u003cp\u003eExercises 156\u003c\/p\u003e \u003cp\u003e\u003cb\u003e11 Quasi-Newton Methods 159\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e11.1 Introduction 159\u003c\/p\u003e \u003cp\u003e11.2 Approximating the Inverse Hessian 160\u003c\/p\u003e \u003cp\u003e11.3 Rank One Correction Formula 162\u003c\/p\u003e \u003cp\u003e11.4 DFP Algorithm 166\u003c\/p\u003e \u003cp\u003e11.5 BFGS Algorithm 170\u003c\/p\u003e \u003cp\u003eExercises 173\u003c\/p\u003e \u003cp\u003e\u003cb\u003e12 Solving Linear Equations 179\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e12.1 Least-Squares Analysis 179\u003c\/p\u003e \u003cp\u003e12.2 Recursive Least-Squares Algorithm 187\u003c\/p\u003e \u003cp\u003e12.3 Solution to a Linear Equation with Minimum Norm 190\u003c\/p\u003e \u003cp\u003e12.4 Kaczmarz’s Algorithm 191\u003c\/p\u003e \u003cp\u003e12.5 Solving Linear Equations in General 194\u003c\/p\u003e \u003cp\u003eExercises 201\u003c\/p\u003e \u003cp\u003e\u003cb\u003e13 Unconstrained Optimization and Neural Networks 209\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e13.1 Introduction 209\u003c\/p\u003e \u003cp\u003e13.2 Single-Neuron Training 211\u003c\/p\u003e \u003cp\u003e13.3 Backpropagation Algorithm 213\u003c\/p\u003e \u003cp\u003eExercises 222\u003c\/p\u003e \u003cp\u003e\u003cb\u003e14 Global Search Algorithms 225\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e14.1 Introduction 225\u003c\/p\u003e \u003cp\u003e14.2 Nelder–Mead Simplex Algorithm 225\u003c\/p\u003e \u003cp\u003e14.3 Simulated Annealing 229\u003c\/p\u003e \u003cp\u003e14.3.1 Randomized Search 229\u003c\/p\u003e \u003cp\u003e14.3.2 Simulated Annealing Algorithm 229\u003c\/p\u003e \u003cp\u003e14.4 Particle Swarm Optimization 231\u003c\/p\u003e \u003cp\u003e14.4.1 Basic PSO Algorithm 232\u003c\/p\u003e \u003cp\u003e14.4.2 Variations 233\u003c\/p\u003e \u003cp\u003e14.5 Genetic Algorithms 233\u003c\/p\u003e \u003cp\u003e14.5.1 Basic Description 233\u003c\/p\u003e \u003cp\u003e14.5.1.1 Chromosomes and Representation Schemes 234\u003c\/p\u003e \u003cp\u003e14.5.1.2 Selection and Evolution 234\u003c\/p\u003e \u003cp\u003e14.5.2 Analysis of Genetic Algorithms 238\u003c\/p\u003e \u003cp\u003e14.5.3 Real-Number Genetic Algorithms 243\u003c\/p\u003e \u003cp\u003eExercises 244\u003c\/p\u003e \u003cp\u003e\u003cb\u003ePart III Linear Programming 247\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e\u003cb\u003e15 Introduction to Linear Programming 249\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e15.1 Brief History of Linear Programming 249\u003c\/p\u003e \u003cp\u003e15.2 Simple Examples of Linear Programs 250\u003c\/p\u003e \u003cp\u003e15.3 Two-Dimensional Linear Programs 256\u003c\/p\u003e \u003cp\u003e15.4 Convex Polyhedra and Linear Programming 258\u003c\/p\u003e \u003cp\u003e15.5 Standard Form Linear Programs 260\u003c\/p\u003e \u003cp\u003e15.6 Basic Solutions 264\u003c\/p\u003e \u003cp\u003e15.7 Properties of Basic Solutions 267\u003c\/p\u003e \u003cp\u003e15.8 Geometric View of Linear Programs 269\u003c\/p\u003e \u003cp\u003eExercises 273\u003c\/p\u003e \u003cp\u003e\u003cb\u003e16 Simplex Method 277\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e16.1 Solving Linear Equations Using Row Operations 277\u003c\/p\u003e \u003cp\u003e16.2 The Canonical Augmented Matrix 283\u003c\/p\u003e \u003cp\u003e16.3 Updating the Augmented Matrix 284\u003c\/p\u003e \u003cp\u003e16.4 The Simplex Algorithm 285\u003c\/p\u003e \u003cp\u003e16.5 Matrix Form of the Simplex Method 291\u003c\/p\u003e \u003cp\u003e16.6 Two-Phase Simplex Method 294\u003c\/p\u003e \u003cp\u003e16.7 Revised Simplex Method 297\u003c\/p\u003e \u003cp\u003eExercises 301\u003c\/p\u003e \u003cp\u003e\u003cb\u003e17 Duality 309\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e17.1 Dual Linear Programs 309\u003c\/p\u003e \u003cp\u003e17.2 Properties of Dual Problems 316\u003c\/p\u003e \u003cp\u003e17.3 Matrix Games 321\u003c\/p\u003e \u003cp\u003eExercises 324\u003c\/p\u003e \u003cp\u003e\u003cb\u003e18 Nonsimplex Methods 331\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e18.1 Introduction 331\u003c\/p\u003e \u003cp\u003e18.2 Khachiyan’s Method 332\u003c\/p\u003e \u003cp\u003e18.3 Affine Scaling Method 334\u003c\/p\u003e \u003cp\u003e18.3.1 Basic Algorithm 334\u003c\/p\u003e \u003cp\u003e18.3.2 Two-Phase Method 337\u003c\/p\u003e \u003cp\u003e18.4 Karmarkar’s Method 339\u003c\/p\u003e \u003cp\u003e18.4.1 Basic Ideas 339\u003c\/p\u003e \u003cp\u003e18.4.2 Karmarkar’s Canonical Form 339\u003c\/p\u003e \u003cp\u003e18.4.3 Karmarkar’s Restricted Problem 341\u003c\/p\u003e \u003cp\u003e18.4.4 From General Form to Karmarkar’s Canonical Form 342\u003c\/p\u003e \u003cp\u003e18.4.5 The Algorithm 345\u003c\/p\u003e \u003cp\u003eExercises 349\u003c\/p\u003e \u003cp\u003e\u003cb\u003e19 Integer Linear Programming 351\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e19.1 Introduction 351\u003c\/p\u003e \u003cp\u003e19.2 Unimodular Matrices 351\u003c\/p\u003e \u003cp\u003e19.3 The Gomory Cutting-Plane Method 358\u003c\/p\u003e \u003cp\u003eExercises 366\u003c\/p\u003e \u003cp\u003e\u003cb\u003ePart IV Nonlinear Constrained Optimization 369\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e\u003cb\u003e20 Problems with Equality Constraints 371\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e20.1 Introduction 371\u003c\/p\u003e \u003cp\u003e20.2 Problem Formulation 373\u003c\/p\u003e \u003cp\u003e20.3 Tangent and Normal Spaces 374\u003c\/p\u003e \u003cp\u003e20.4 Lagrange Condition 379\u003c\/p\u003e \u003cp\u003e20.5 Second-Order Conditions 387\u003c\/p\u003e \u003cp\u003e20.6 Minimizing Quadratics Subject to Linear Constraints 390\u003c\/p\u003e \u003cp\u003eExercises 394\u003c\/p\u003e \u003cp\u003e\u003cb\u003e21 Problems with Inequality Constraints 399\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e21.1 Karush–Kuhn–Tucker Condition 399\u003c\/p\u003e \u003cp\u003e21.2 Second-Order Conditions 406\u003c\/p\u003e \u003cp\u003eExercises 410\u003c\/p\u003e \u003cp\u003e\u003cb\u003e22 Convex Optimization Problems 417\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e22.1 Introduction 417\u003c\/p\u003e \u003cp\u003e22.2 Convex Functions 419\u003c\/p\u003e \u003cp\u003e22.3 Convex Optimization Problems 426\u003c\/p\u003e \u003cp\u003e22.4 Semidefinite Programming 431\u003c\/p\u003e \u003cp\u003e22.4.1 Linear Matrix Inequalities and Their Properties 431\u003c\/p\u003e \u003cp\u003e22.4.2 LMI Solvers 435\u003c\/p\u003e \u003cp\u003e22.4.2.1 Finding a Feasible Solution Under LMI Constraints 436\u003c\/p\u003e \u003cp\u003e22.4.2.2 Minimizing a Linear Objective Under LMI Constraints 438\u003c\/p\u003e \u003cp\u003e22.4.2.3 Minimizing a Generalized Eigenvalue Under LMI Constraints 440\u003c\/p\u003e \u003cp\u003eExercises 442\u003c\/p\u003e \u003cp\u003e\u003cb\u003e23 Lagrangian Duality 449\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e23.1 Overview 449\u003c\/p\u003e \u003cp\u003e23.2 Notation 449\u003c\/p\u003e \u003cp\u003e23.3 Primal–Dual Pair 450\u003c\/p\u003e \u003cp\u003e23.4 General Duality Properties 451\u003c\/p\u003e \u003cp\u003e23.4.1 Convexity of Dual Problem 451\u003c\/p\u003e \u003cp\u003e23.4.2 Primal Objective in Terms of Lagrangian 451\u003c\/p\u003e \u003cp\u003e23.4.3 Minimax Inequality Chain 452\u003c\/p\u003e \u003cp\u003e23.4.4 Optimality of Saddle Point 452\u003c\/p\u003e \u003cp\u003e23.4.5 Weak Duality 453\u003c\/p\u003e \u003cp\u003e23.4.6 Duality Gap 453\u003c\/p\u003e \u003cp\u003e23.5 Strong Duality 454\u003c\/p\u003e \u003cp\u003e23.5.1 Strong Duality ⇔ Minimax Equals Maximin 454\u003c\/p\u003e \u003cp\u003e23.5.2 Strong Duality ⇒ Primal Unconstrained Minimization 455\u003c\/p\u003e \u003cp\u003e23.5.3 Strong Duality ⇒ Optimality 455\u003c\/p\u003e \u003cp\u003e23.5.4 Strong Duality ⇒ KKT (Including Complementary Slackness) 455\u003c\/p\u003e \u003cp\u003e23.5.5 Strong Duality ⇒ Saddle Point 456\u003c\/p\u003e \u003cp\u003e23.6 Convex Case 456\u003c\/p\u003e \u003cp\u003e23.6.1 Convex Case: KKT ⇒ Strong Duality 456\u003c\/p\u003e \u003cp\u003e23.6.2 Convex Case: Regular Optimal Primal ⇒ Strong Duality 457\u003c\/p\u003e \u003cp\u003e23.6.3 Convex Case: Slater’s Condition ⇒ Strong Duality 457\u003c\/p\u003e \u003cp\u003e23.7 Summary of Key Results 457\u003c\/p\u003e \u003cp\u003eExercises 458\u003c\/p\u003e \u003cp\u003e\u003cb\u003e24 Algorithms for Constrained Optimization 459\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e24.1 Introduction 459\u003c\/p\u003e \u003cp\u003e24.2 Projections 459\u003c\/p\u003e \u003cp\u003e24.3 Projected Gradient Methods with Linear Constraints 462\u003c\/p\u003e \u003cp\u003e24.4 Convergence of Projected Gradient Algorithms 465\u003c\/p\u003e \u003cp\u003e24.4.1 Fixed Points and First-Order Necessary Conditions 466\u003c\/p\u003e \u003cp\u003e24.4.2 Convergence with Fixed Step Size 468\u003c\/p\u003e \u003cp\u003e24.4.3 Some Properties of Projections 469\u003c\/p\u003e \u003cp\u003e24.4.4 Armijo Condition 470\u003c\/p\u003e \u003cp\u003e24.4.5 Accumulation Points 471\u003c\/p\u003e \u003cp\u003e24.4.6 Projections in the Convex Case 472\u003c\/p\u003e \u003cp\u003e24.4.7 Armijo Condition in the Convex Case 474\u003c\/p\u003e \u003cp\u003e24.4.8 Convergence in the Convex Case 480\u003c\/p\u003e \u003cp\u003e24.4.9 Convergence Rate with Line-Search Step Size 481\u003c\/p\u003e \u003cp\u003e24.5 Lagrangian Algorithms 483\u003c\/p\u003e \u003cp\u003e24.5.1 Lagrangian Algorithm for Equality Constraints 484\u003c\/p\u003e \u003cp\u003e24.5.2 Lagrangian Algorithm for Inequality Constraints 486\u003c\/p\u003e \u003cp\u003e24.6 Penalty Methods 489\u003c\/p\u003e \u003cp\u003eExercises 495\u003c\/p\u003e \u003cp\u003e\u003cb\u003e25 Multiobjective Optimization 499\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e25.1 Introduction 499\u003c\/p\u003e \u003cp\u003e25.2 Pareto Solutions 499\u003c\/p\u003e \u003cp\u003e25.3 Computing the Pareto Front 501\u003c\/p\u003e \u003cp\u003e25.4 From Multiobjective to Single-Objective Optimization 505\u003c\/p\u003e \u003cp\u003e25.5 Uncertain Linear Programming Problems 508\u003c\/p\u003e \u003cp\u003e25.5.1 Uncertain Constraints 508\u003c\/p\u003e \u003cp\u003e25.5.2 Uncertain Objective Function Coefficients 511\u003c\/p\u003e \u003cp\u003e25.5.3 Uncertain Constraint Coefficients 513\u003c\/p\u003e \u003cp\u003e25.5.4 General Uncertainties 513\u003c\/p\u003e \u003cp\u003eExercises 513\u003c\/p\u003e \u003cp\u003e\u003cb\u003ePart V Optimization in Machine Learning 517\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e\u003cb\u003e26 Machine Learning Problems and Feature Engineering 519\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e26.1 Machine Learning Problems 519\u003c\/p\u003e \u003cp\u003e26.1.1 Data with Labels and Supervised Learning 519\u003c\/p\u003e \u003cp\u003e26.1.2 Data Without Labels and Unsupervised Learning 521\u003c\/p\u003e \u003cp\u003e26.2 Data Normalization 522\u003c\/p\u003e \u003cp\u003e26.3 Histogram of Oriented Gradients 524\u003c\/p\u003e \u003cp\u003e26.4 Principal Component Analysis and Linear Autoencoder 526\u003c\/p\u003e \u003cp\u003e26.4.1 Singular Value Decomposition 526\u003c\/p\u003e \u003cp\u003e26.4.2 Principal Axes and Principal Components of a Data Set 527\u003c\/p\u003e \u003cp\u003e26.4.3 Linear Autoencoder 529\u003c\/p\u003e \u003cp\u003eExercises 530\u003c\/p\u003e \u003cp\u003e\u003cb\u003e27 Stochastic Gradient Descent Algorithms 537\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e27.1 Stochastic Gradient Descent Algorithm 537\u003c\/p\u003e \u003cp\u003e27.2 Stochastic Variance Reduced Gradient Algorithm 540\u003c\/p\u003e \u003cp\u003e27.3 Distributed Stochastic Variance Reduced Gradient 542\u003c\/p\u003e \u003cp\u003e27.3.1 Distributed Learning Environment 542\u003c\/p\u003e \u003cp\u003e27.3.2 SVRG in Distributed Optimization 543\u003c\/p\u003e \u003cp\u003e27.3.3 Communication Versus Computation 545\u003c\/p\u003e \u003cp\u003e27.3.4 Data Security 545\u003c\/p\u003e \u003cp\u003eExercises 546\u003c\/p\u003e \u003cp\u003e\u003cb\u003e28 Linear Regression and Its Variants 553\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e28.1 Least-Squares Linear Regression 553\u003c\/p\u003e \u003cp\u003e28.1.1 A Linear Model for Prediction 553\u003c\/p\u003e \u003cp\u003e28.1.2 Training the Model 554\u003c\/p\u003e \u003cp\u003e28.1.3 Computing Optimal ̂w 554\u003c\/p\u003e \u003cp\u003e28.1.4 Optimal Predictor and Performance Evaluation 555\u003c\/p\u003e \u003cp\u003e28.1.5 Least-Squares Linear Regression for Data Sets with Vector Labels 556\u003c\/p\u003e \u003cp\u003e28.2 Model Selection by Cross-Validation 559\u003c\/p\u003e \u003cp\u003e28.3 Model Selection by Regularization 562\u003c\/p\u003e \u003cp\u003eExercises 564\u003c\/p\u003e \u003cp\u003e\u003cb\u003e29 Logistic Regression for Classification 569\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e29.1 Logistic Regression for Binary Classification 569\u003c\/p\u003e \u003cp\u003e29.1.1 Least-Squares Linear Regression for Binary Classification 569\u003c\/p\u003e \u003cp\u003e29.1.2 Logistic Regression for Binary Classification 570\u003c\/p\u003e \u003cp\u003e29.1.3 Interpreting Logistic Regression by Log Error 572\u003c\/p\u003e \u003cp\u003e29.1.4 Confusion Matrix for Binary Classification 573\u003c\/p\u003e \u003cp\u003e29.2 Nonlinear Decision Boundary via Linear Regression 575\u003c\/p\u003e \u003cp\u003e29.2.1 Least-Squares Linear Regression with Nonlinear Transformation 576\u003c\/p\u003e \u003cp\u003e29.2.2 Logistic Regression with Nonlinear Transformation 578\u003c\/p\u003e \u003cp\u003e29.3 Multicategory Classification 580\u003c\/p\u003e \u003cp\u003e29.3.1 One-Versus-All Multicategory Classification 580\u003c\/p\u003e \u003cp\u003e29.3.2 Softmax Regression for Multicategory Classification 581\u003c\/p\u003e \u003cp\u003eExercises 584\u003c\/p\u003e \u003cp\u003e\u003cb\u003e30 Support Vector Machines 589\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e30.1 Hinge-Loss Functions 589\u003c\/p\u003e \u003cp\u003e30.1.1 Geometric Interpretation of the Linear Model 589\u003c\/p\u003e \u003cp\u003e30.1.2 Hinge Loss for Binary Data Sets 590\u003c\/p\u003e \u003cp\u003e30.1.3 Hinge Loss for Multicategory Data Sets 592\u003c\/p\u003e \u003cp\u003e30.2 Classification by Minimizing Hinge Loss 593\u003c\/p\u003e \u003cp\u003e30.2.1 Binary Classification by Minimizing Average Hinge Loss 593\u003c\/p\u003e \u003cp\u003e30.2.2 Multicategory Classification by Minimizing E hww or E hcs 594\u003c\/p\u003e \u003cp\u003e30.3 Support Vector Machines for Binary Classification 596\u003c\/p\u003e \u003cp\u003e30.3.1 Hard-Margin Support Vector Machines 596\u003c\/p\u003e \u003cp\u003e30.3.2 Support Vectors 598\u003c\/p\u003e \u003cp\u003e30.3.3 Soft-Margin Support Vector Machines 599\u003c\/p\u003e \u003cp\u003e30.3.4 Connection to Hinge-Loss Minimization 602\u003c\/p\u003e \u003cp\u003e30.4 Support Vector Machines for Multicategory Classification 602\u003c\/p\u003e \u003cp\u003e30.5 Kernel Trick 603\u003c\/p\u003e \u003cp\u003e30.5.1 Kernels 603\u003c\/p\u003e \u003cp\u003e30.5.2 Kernel Trick 604\u003c\/p\u003e \u003cp\u003e30.5.3 Learning with Kernels 605\u003c\/p\u003e \u003cp\u003e30.5.3.1 Regularized Logistic Regression with Nonlinear Transformation for Binary Classification 605\u003c\/p\u003e \u003cp\u003e30.5.3.2 Regularized Hinge-Loss Minimization for Binary Classification 606\u003c\/p\u003e \u003cp\u003eExercises 607\u003c\/p\u003e \u003cp\u003e\u003cb\u003e31 K-Means Clustering 611\u003c\/b\u003e\u003c\/p\u003e \u003cp\u003e31.1 K-Means Clustering 611\u003c\/p\u003e \u003cp\u003e31.2 K-Means++ forCenterInitialization 615\u003c\/p\u003e \u003cp\u003e31.3 Variants of K-Means Clustering 617\u003c\/p\u003e \u003cp\u003e31.3.1 K-Means Clustering Based on 1-Norm Regularization 617\u003c\/p\u003e \u003cp\u003e31.3.2 PCA-Guided K-Means Clustering 619\u003c\/p\u003e \u003cp\u003e31.4 Image Compression by Vector Quantization and K-Means Clustering 622\u003c\/p\u003e \u003cp\u003eExercises 623\u003c\/p\u003e \u003cp\u003eReferences 627\u003c\/p\u003e \u003cp\u003eIndex 635\u003c\/p\u003e  \u003cp\u003e\u003cb\u003eEdwin K. P. Chong, PhD,\u003c\/b\u003e is Professor and Head of Electrical and Computer Engineering and Professor of Mathematics at Colorado State University. He is a Fellow of the IEEE and AAAS, and was Senior Editor of the IEEE Transactions on Automatic Control.  \u003c\/p\u003e\u003cp\u003e\u003cb\u003eWu-Sheng Lu, PhD,\u003c\/b\u003e is Professor Emeritus of Electrical and Computer Engineering at the University of Victoria, Canada. He is a Fellow of the IEEE and former Associate Editor of the IEEE Transactions on Circuits and??Systems. \u003c\/p\u003e\u003cp\u003e\u003cb\u003eStanislaw H. Żak, PhD,\u003c\/b\u003e is Professor in the School of Electrical and Computer Engineering at Purdue University. He is former Associate Editor of Dynamics and Control and the IEEE Transactions on Neural Networks.   \u003c\/p\u003e\u003cp\u003e\u003cb\u003eAccessible introductory textbook on optimization theory and methods, with an emphasis on engineering design, featuring MATLAB\u003csup\u003e®\u003c\/sup\u003e exercises and worked examples\u003c\/b\u003e \u003c\/p\u003e\u003cp\u003eFully updated to reflect modern developments in the field, the Fifth Edition of \u003ci\u003eAn Introduction to Optimization\u003c\/i\u003e fills the need for an accessible, yet rigorous, introduction to optimization theory and methods, featuring innovative coverage and a straightforward approach. The book begins with a review of basic definitions and notations while also providing the related fundamental background of linear algebra, geometry, and calculus.  \u003c\/p\u003e\u003cp\u003eWith this foundation, the authors explore the essential topics of unconstrained optimization problems, linear programming problems, and nonlinear constrained optimization. In addition, the book includes an introduction to artificial neural networks, convex optimization, multi-objective optimization, and applications of optimization in machine learning. \u003c\/p\u003e\u003cp\u003eNumerous diagrams and figures found throughout the book complement the written presentation of key concepts, and each chapter is followed by MATLAB\u003csup\u003e®\u003c\/sup\u003e exercises and practice problems that reinforce the discussed theory and algorithms. \u003c\/p\u003e\u003cp\u003eThe Fifth Edition features a new chapter on Lagrangian (nonlinear) duality, expanded coverage on matrix games, projected gradient algorithms, machine learning, and numerous new exercises at the end of each chapter.  \u003c\/p\u003e\u003cp\u003e\u003ci\u003eAn Introduction to Optimization\u003c\/i\u003e includes information on: \u003c\/p\u003e\u003cul\u003e\n\u003cli\u003eThe mathematical definitions, notations, and relations from linear algebra, geometry, and calculus used in optimization\u003c\/li\u003e \u003cli\u003eOptimization algorithms, covering one-dimensional search, randomized search, and gradient, Newton, conjugate direction, and quasi-Newton methods\u003c\/li\u003e \u003cli\u003eLinear programming methods, covering the simplex algorithm, interior point methods, and duality \u003c\/li\u003e\n\u003cli\u003eNonlinear constrained optimization, covering theory and algorithms, convex optimization, and Lagrangian duality\u003c\/li\u003e \u003cli\u003eApplications of optimization in machine learning, including neural network training, classification, stochastic gradient descent, linear regression, logistic regression, support vector machines, and clustering.\u003c\/li\u003e\n\u003c\/ul\u003e \u003cp\u003e\u003ci\u003eAn Introduction to Optimization\u003c\/i\u003e is an ideal textbook for a one- or two-semester senior undergraduate or beginning graduate course in optimization theory and methods. The text is also of value for researchers and professionals in mathematics, operations research, electrical engineering, economics, statistics, and business.\u003c\/p\u003e","brand":"Wiley","offers":[{"title":"Default Title","offer_id":47988728627429,"sku":"NP9781119877639","price":102.0,"currency_code":"USD","in_stock":false}],"thumbnail_url":"\/\/cdn.shopify.com\/s\/files\/1\/1842\/7735\/files\/9781119877639.jpg?v=1761781356","url":"https:\/\/k12savings.com\/products\/an-introduction-to-optimization-isbn-9781119877639","provider":"K12savings","version":"1.0","type":"link"}