Probability, Random Processes and Estimation Theory for Engineers, Stark & Woods
Probability, Random Processes, and Estimation Theory for Engineers by Henry Stark and John W. Woods Prentice Hall, 1986, 013711706X, Hardcover without dust jacket, VG condition, no underlining, no highlighting, 430 pages. In this timely book, fundamental principles and developing skills are introduced as a needed foundation for more advanced material in communications, signal processing, controls, robotics, large-scale systems, and physics-related phenomena. Among its many features, the book: •devotes three chapters to an in-depth study of probability and random variables •discusses random vectors, covariance matrices, eigenvalue problems, maximization of quadratic forms, and transformation of random vectors •includes applications to pattern recognition, linear systems, parameter estimation, and decision/detection theory •devotes an entire chapter to the theory of random discrete-time (space) sequences •covers random processes, mean-square calculus, stationary processes and sequences, including an introduction to discrete-time Martingale theory •contains two chapters on estimation theory, including parameter estimation, the Gauss-Markov theorem, estimation of random variables, recursive estimation with the Kalman filter, and Wiener filters/predictors for discrete and continuous time. CONTENTS Preface xi 1 INTRODUCTION TO PROBABILITY 1 1.1Introduction: Why Study Probability? 1 1.2 The Different Kinds of Probability 2 1.3Sets, Fields, and Events5 1.4 Axiomatic Definition of Probability9 1.5Joint, Conditional, and Total Probabilities; Independence11 1.6 Bayes’ Theorem and Applications16 1.7 Combinatorics18 1.8Bernoulli Trials12 1.9 Asymptotic Behavior of the Binomial Law: The Poisson Law 25 1.10 Derivation of the Poisson Probability Law 28 1.11 Summary 31 Problems 31 References 35 2 RANDOM VARIABLES 37 2.1Introduction37 2.2 Definition of a Random Variable 38 2.3 Probability Distribution Function 41 2.4 Probability Density Function (pdf) 43 2.5 Continuous, Discrete, and Mixed Random Variables 49 2.6 The Dirac Delta Function 52 2.7 Conditional and Joint Distributions and Densities56 2.8 A Function of a Random Variable (FRV) 69 2.9 Solving Problems of the Type Y=g (X) 73 2.10 Solving Problems of the Type Z= g (X,Y) 81 2.11 Solving Problems of the Type V=g (X,Y), W= h (X,Y) 95 2.12 Summary 104 Problems 104 References 109 Additional Reading110 3 AVERAGES 111 3.1Expected Value of a Random Variable111 3.2 Conditional Expectations117 3.3 Moments 124 3.4 Chebyshev and Schwarz Inequalities 132 3.5 Moment-Generating Functions 136 3.6 Characteristic Functions139 3.7 Summary 146 Problems 147 References151 4 VECTOR RANDOM VARIABLES 152 4.1Joint Distribution and Densities152 4.2 Expectation Vectors and Covariance Matrices156 4.3 Properties of Covariance Matrices158 4.4 Simultaneous Diagonalizations of Two Covariance Matrices161 4.5 The Multidimensional Gaussian Law 170 4.6 Characteristic Functions of Random Vectors177 4.7 Summary 181 Problems 182 References185 5 ESTIMATION AND DECISION THEORY I 187 5.1Parameter Estimation188 5.2 Estimation of Vector Means and Covariance Matrices192 5.3 Maximum Likelihood Parameter Estimation195 5.4 Linear Estimation of Vector Parameters198 5.5 Optimum Properties of Least-Squares Estimators The Gauss-Markov Theorem 201 5.6 Estimation of Random Variables 205 5.7 Decision Theory: The Bayesian Approach 209 5.8 Summary 217 Problems 218 References 221 6 RANDOM SEQUENCES 222 6.1 Basic Concepts 222 6.2 Basic Principles of Discrete-Time Linear Systems 235 6.3 Random Sequences and Linear Systems 239 6.4 Convergence of Random Sequences 6.5 Laws of Large Numbers 253 6.6 Summary 257 Problems 257 References 262 7 RANDOM PROCESSES 263 7.1Definitions264 7.2 Some Important Random Processes 267 7.3 Linear Systems with Random Inputs 281 7.4 Classification of Random Processes 285 7.5 Summary 290 Problems 290 References 295 8 MEAN-SQUARE CALCULUS 296 8.1Continuity and Derivatives 296 8.2 Stochastic Integrals 308 8.3 Stochastic Differential Equations 311 8.4 Ergodicity 316 8.5 Karhunen-Loeve Expansion 323 8.6 Summary 327 App Integral Equations 328 Problems 332 References 336 9 STATIONARY PROCESSES AND SEQUENCES 337 9.1Power Spectral Density 338 9.2 More on White Noise 343 9.3 Stationary Process and Linear Systems 345 9.4 WSS Random Sequences 352 9.5 Vector Processes and State Equations 354 9.6 Representation of Bandlimited and Periodic Processes 360 9.7 Summary 367 App Residue Method for Inverse Fourier Transformation 368 Problems 374 References 379 10 ESTIMATION THEORY II 380 10.1 The Conditional Mean Revisited 380 10.2 Orthogonality and Linear Estimation 382 10.3 Innovations Sequences and Prediction 388 10.4 Kalman Predictor and Filter 393 10.5 Wiener Filters for Random Sequences 401 10.6 Linear Estimation for Random Processes 404 10.7 Summary 418 Problems 419 References 421 Index 423 nthdegree books
Probability, Random Processes and Estimation Theory for Engineers, Stark & Woods
Probability, Random Processes, and Estimation Theory for Engineers by Henry Stark and John W. Woods Prentice Hall, 1986, 013711706X, Hardcover without dust jacket, VG condition, no underlining, no highlighting, 430 pages. In this timely book, fundamental principles and developing skills are introduced as a needed foundation for more advanced material in communications, signal processing, controls, robotics, large-scale systems, and physics-related phenomena. Among its many features, the book: •devotes three chapters to an in-depth study of probability and random variables •discusses random vectors, covariance matrices, eigenvalue problems, maximization of quadratic forms, and transformation of random vectors •includes applications to pattern recognition, linear systems, parameter estimation, and decision/detection theory •devotes an entire chapter to the theory of random discrete-time (space) sequences •covers random processes, mean-square calculus, stationary processes and sequences, including an introduction to discrete-time Martingale theory •contains two chapters on estimation theory, including parameter estimation, the Gauss-Markov theorem, estimation of random variables, recursive estimation with the Kalman filter, and Wiener filters/predictors for discrete and continuous time. CONTENTS Preface xi 1 INTRODUCTION TO PROBABILITY 1 1.1Introduction: Why Study Probability? 1 1.2 The Different Kinds of Probability 2 1.3Sets, Fields, and Events5 1.4 Axiomatic Definition of Probability9 1.5Joint, Conditional, and Total Probabilities; Independence11 1.6 Bayes’ Theorem and Applications16 1.7 Combinatorics18 1.8Bernoulli Trials12 1.9 Asymptotic Behavior of the Binomial Law: The Poisson Law 25 1.10 Derivation of the Poisson Probability Law 28 1.11 Summary 31 Problems 31 References 35 2 RANDOM VARIABLES 37 2.1Introduction37 2.2 Definition of a Random Variable 38 2.3 Probability Distribution Function 41 2.4 Probability Density Function (pdf) 43 2.5 Continuous, Discrete, and Mixed Random Variables 49 2.6 The Dirac Delta Function 52 2.7 Conditional and Joint Distributions and Densities56 2.8 A Function of a Random Variable (FRV) 69 2.9 Solving Problems of the Type Y=g (X) 73 2.10 Solving Problems of the Type Z= g (X,Y) 81 2.11 Solving Problems of the Type V=g (X,Y), W= h (X,Y) 95 2.12 Summary 104 Problems 104 References 109 Additional Reading110 3 AVERAGES 111 3.1Expected Value of a Random Variable111 3.2 Conditional Expectations117 3.3 Moments 124 3.4 Chebyshev and Schwarz Inequalities 132 3.5 Moment-Generating Functions 136 3.6 Characteristic Functions139 3.7 Summary 146 Problems 147 References151 4 VECTOR RANDOM VARIABLES 152 4.1Joint Distribution and Densities152 4.2 Expectation Vectors and Covariance Matrices156 4.3 Properties of Covariance Matrices158 4.4 Simultaneous Diagonalizations of Two Covariance Matrices161 4.5 The Multidimensional Gaussian Law 170 4.6 Characteristic Functions of Random Vectors177 4.7 Summary 181 Problems 182 References185 5 ESTIMATION AND DECISION THEORY I 187 5.1Parameter Estimation188 5.2 Estimation of Vector Means and Covariance Matrices192 5.3 Maximum Likelihood Parameter Estimation195 5.4 Linear Estimation of Vector Parameters198 5.5 Optimum Properties of Least-Squares Estimators The Gauss-Markov Theorem 201 5.6 Estimation of Random Variables 205 5.7 Decision Theory: The Bayesian Approach 209 5.8 Summary 217 Problems 218 References 221 6 RANDOM SEQUENCES 222 6.1 Basic Concepts 222 6.2 Basic Principles of Discrete-Time Linear Systems 235 6.3 Random Sequences and Linear Systems 239 6.4 Convergence of Random Sequences 6.5 Laws of Large Numbers 253 6.6 Summary 257 Problems 257 References 262 7 RANDOM PROCESSES 263 7.1Definitions264 7.2 Some Important Random Processes 267 7.3 Linear Systems with Random Inputs 281 7.4 Classification of Random Processes 285 7.5 Summary 290 Problems 290 References 295 8 MEAN-SQUARE CALCULUS 296 8.1Continuity and Derivatives 296 8.2 Stochastic Integrals 308 8.3 Stochastic Differential Equations 311 8.4 Ergodicity 316 8.5 Karhunen-Loeve Expansion 323 8.6 Summary 327 App Integral Equations 328 Problems 332 References 336 9 STATIONARY PROCESSES AND SEQUENCES 337 9.1Power Spectral Density 338 9.2 More on White Noise 343 9.3 Stationary Process and Linear Systems 345 9.4 WSS Random Sequences 352 9.5 Vector Processes and State Equations 354 9.6 Representation of Bandlimited and Periodic Processes 360 9.7 Summary 367 App Residue Method for Inverse Fourier Transformation 368 Problems 374 References 379 10 ESTIMATION THEORY II 380 10.1 The Conditional Mean Revisited 380 10.2 Orthogonality and Linear Estimation 382 10.3 Innovations Sequences and Prediction 388 10.4 Kalman Predictor and Filter 393 10.5 Wiener Filters for Random Sequences 401 10.6 Linear Estimation for Random Processes 404 10.7 Summary 418 Problems 419 References 421 Index 423 nthdegree books