Generative Learning Structures and Processes for
Generalized Connectionist Networks
Department of Computer Science
Iowa State University
Computer Sciences Department
University of Wisconsin-Madison
Massively parallel networks of relatively simple computing elements offer an attractive and versatile framework for exploring a variety of learning structures and processes for intelligent systems. This paper briefly summarizes some popular learning structures and processes used in such networks. It outlines a range of potentially more powerful alternatives for pattern-directed inductive learning in such systems. It motivates and develops a class of new learning algorithms for massively parallel networks of simple computing elements. We call this class of learning processes generative for they offer a set of mechanisms for constructive and adaptive determination of the network architecture - the number of processing elements and the connectivity among them - as a function of experience. Generative learning algorithms attempt to overcome some of the limitations of some approaches to learning in networks that rely on modification of weights on the links within an otherwise fixed network topology e.g., rather slow learning and the need for an a-priori choice of a network architecture. Several alternative designs as well as a range of control structures and processes which can be used to regulate the form and content of internal representations learned by such networks are examined. Empirical results from the study of some generative learning algorithms are briefly summarized and several extensions and refinements of such algorithms, and directions for future research are outlined.
Pattern recognition, and learning to recognize patterns, are among the most central attributes of an intelligent entity. Learning, defined informally, is the process that enables a system to absorb information from its environment. Central to the more powerful types of learning is the ability to construct appropriate internal representations of the environment in which the learner operates. Learning must build the internal representations that perception and cognition utilize.
This research was partially supported by the Air Force Office of Scientific Research (grant AFOSR-89-0178), the National Science Foundation (grant CCR-8720060), the University of Wisconsin Graduate School, and the Iowa State University College of Liberal Arts and Sciences. A preliminary draft of this paper was published as technical report 91-2, Iowa State University Department of Computer Science. The current version appears in Information Sciences (Special issue on Artificial Intelligence and Neural Networks). The authors are grateful to the anonymous referees and Professor Kak for their suggestions on an earlier draft.