This book provides a framework for the design of competent optimization techniques by combining advanced evolutionary algorithms with state-of-the-art machine learning techniques. The primary focus of the book is on two algorithms that replace traditional variation operators of evolutionary algorithms, by learning and sampling Bayesian networks: the Bayesian optimization algorithm (BOA) and the hierarchical BOA (hBOA). They provide a scalable solution to a broad class of problems. The book provides an overview of evolutionary algorithms that use probabilistic models to guide their search, motivates and describes BOA and hBOA in a way accessible to a wide audience, and presents numerous results confirming that they are revolutionary approaches to black-box optimization.

At the 1997 Genetic Programming Conference at Stanford University, a selfdescribed “long red-haired guy” walked up to me and introduced himself. His name was Martin Pelikan, he came from Slovakia, and he wanted to visit my lab at Illinois. He was doing interesting work, and he seemed like a bright enough fellow, so we exchanged e-mail addresses and worked on overcoming the logistical problems of a visit. Doing so took longer than I expected, but on September 2, 1998 he arrived in Champaign and immediately got to work. The initial visit turned into a stint as a PhD student, the stint turned into a dissertation, and that dissertation has now been transformed into the book before you. It is a remarkable piece of work.

Simply stated Martin has shown how to take (a) population-based search, (b) Bayesian networks, (c) niching or clustering techniques and (d) compact representations and create a fairly general purpose optimizer of remarkable breadth and capability. His system is called the hierarchical Bayesian optimization algorithm (hBOA). I have used the term competent to characterize genetic and other search procedures that solve a large class of hard problems quickly, reliably, and accurately, and the book before you is a comprehensive description of the thinking that went into the design of a competent hBOA, its precursors, the analytical theory of hBOA operation, and empirical tests of hBOA on both bounding test functions and problems of interest in the real world; it is a thorough discussion of what in my view is the most competent solver to have emerged from these and related lines of thinking.