COMP20411: Subsymbolic Processing and Neural Networks (2007-2008)
To introduce methods for extracting rules or learning from data, and provide the necessary mathematical background to enable the student to understand how the methods work and how to get the best performance from them. The course also includes heuristic problem-solving methods such as genetic algorithms. This course is pitched towards any student with a mathematical or scientific background who is interested in adaptive techniques for learning from data.
Upon completion of the course, the student should:
Evaluate whether a learning system is appropriate for a particular problem. (B)
Understand how to use data for learning, model selection, and testing. (B,D)
Understand generally the relationship between model complexity and model performance, and be able to use this to design a strategy to improve an existing system. (B,D)
Understand the advantages and disadvantages of the learning systems studied in the course, and decide which is appropriate for a particular application. (B)
Be able to apply neural networks to simple data sets and evaluate their performance. (A)
Make a naive Bayes classifier, and interprete the results as probabilities. (A,B,D)
Devise greedy algorithms and genetic algorithms for solving optimization problems. (A)
Assessment of Learning outcomesBy examination: Learning outcomes 1,2,3,4,5,7 by laboratory: Learning outcomes: 2,3,5,6,7
Contribution to Programme Learning OutcomesA1, A2, A5, B1, B3, D6 (especially probability and statistics)
Overview of approach. Symbolic versus subsymbolic methods, hand-built knowledge versus model extraction from data.
Introduction to learning theory and model evaluation 
The need to validate models learned from data. Techniques of performance estimation and validation. Generalization.
Neural Networks 
Perceptrons - learning linear discriminants, limitations of perceptrons. Multilayer perceptrons - learning, example applications.
Bayesian decisions and classification 
Basic probability theory and probabilistic modelling. Decisions and risk. Bayesian classification. The naive Bayes classifier.
Non-symbolic search techniques 
Greedy algorithms, hillclimbing, genetic algorithms. Application of genetic algorithms to reinforcement learning.