Modularity and hierarchy are fundamental notions in structured system design. By subdividing a large and unstructured problem into smaller and tractable chunks, design automation becomes possible. In this paper we discuss the use of modularity and hierarchy for functional specialization during the development of neural networks. We study the behavioral differences and requirements for back-propagation training of  feed-forward networks. Further we  illustrate that a deliberate mix of hierarchically imposed evaluation functions will improve network accuracy and learning speed.