The purpose of this course is to introduce and to understand the significance of optimization theory in view of linear and nonlinear problems. The course also aims to gain the ability of researching using modern approaches in this field in both theoretically and practically.

Overview of optimization, and the required mathematical preliminary for optimization, unconstrained optimization, Local Minimizers, One-Dimensional Search Methods, Steepest Descent Method, Analysis of Gradient Methods, Newton's Method and Levenberg-Marquardt Modification, Newton's Method for Nonlinear Least-Squares Problem, The Conjugate Direction and the Conjugate Gradient Algorithms, The Conjugate Gradient Algorithm for Non-Quadratic Problems, Quasi-Newton Methods, Analysis of Least Squares and Recursive Least Squares Algorithm, Unconstrained Optimization and Neural Networks.