Abstract

Effects of Experience Bias When

Seeding With Prior Results

Seeding With Prior Results

Mitchell A. Potter

R. Paul Wiegand

H. Joseph Blumenthal

Donald A. Sofge

R. Paul Wiegand

H. Joseph Blumenthal

Donald A. Sofge

Seeding the population of an evolutionary algorithm with solutions from previous runs has proved to be useful when learning control strategies for agents operating in a complex, changing environment. It has generally been assumed that initializing a learning algorithm with previously learned solutions will be helpful if the new problem is similar to the old. We will show that this assumption sometimes does not hold for many reasonable similarity metrics. Using a more traditional machine learning perspective, we explain why seeding is sometimes not helpful by looking at the learning-experience bias produced by the previously evolved solutions. are sufficiently different, it can easily be the case that a bias toward prior behaviors will trap algorithms in new suboptima (Louis and Johnson 1997). Unfortunately, it is unclear what “sufficiently different” means for a given task; yet for those who employ methods like CEL and shaping, it is nevertheless important to understand when seeding is helpful, and when it can be harmful. One way to answer such a question is to define a variety of useful distance metrics and study how such measures affect the performance of seeding methods. Rather than having to rely on arbitrary notions of distance, though, we adopt a different perspective in which we concentrate