From: Chris Williams Subject: NIPS*96 post-conference workshop om Model Complexity Note the call for short presentations near the bottom of this message. NIPS*96 Post-conference Workshop MODEL COMPLEXITY Snowmass (Aspen), Colorado USA Friday Dec 6th, 1996 ORGANIZERS: Chris Williams (Aston University, UK, c.k.i.williams@aston.ac.uk) Joachim Utans (London Business School, UK, J.Utans@lbs.lon.ac.uk) OVERVIEW: One of the most important difficulties in using neural networks for real-world problems is the issue of model complexity, and how it affects the generalization performance. One approach states that model complexity should be tailored to the amount of training data available, e.g. by using architectures with small numbers of adaptable parameters, or by penalizing the fit of larger models (e.g. AIC, BIC, Structural Risk Minimization, GPE). Alternatively, computationally expensive numerical estimates of the generalization performance (cross-validation (CV), Bootstrap, and related methods) can be used to compare and select models (for example Moody and Utans, 1994). Methods based on regularization restrict model complexity by reducing the "effective" number of parameters (Moody 1992). On the other hand, Bayesian methods see no need to limit model complexity, as overfitting is obviated by marginalization, where predictions are made by averaging over the posterior weight distribution. As Neal (1995) has argued, there may be no reason to believe that neural network models for real-world problems should be limited to nets containing only a "small" number of hidden units. In the limit of an infinite number of hidden units neural networks become Gaussian processes, and hence are closely related to the splines approach (Wahba, 1990). Another important aspect of model building is the selection of a subset of relevant input variables to include in the model, for instance, in a regression context, the subset of independent variables, or lagged values for a time series problem. The aim of this workshop is to present the different ideas on these topics, and to provide guidance to those confronted with the problem of model complexity on real-world problems. SPEAKERS: Leo Breiman (University of California Berkeley) Federico Girosi (MIT) Trevor Hastie (Stanford) Michael Kearns (AT&T Laboratories Research) John Moody (Oregon Graduate Institute) Grace Wahba (University of Wisconsin at Madison) Hal White (University of California San Diego) Huaiyu Zhu (Santa Fe Institute) WORKSHOP FORMAT: Of the 6 hours scheduled, about 4 will be taken up with presentations from the speakers listed above. We also very keen to make sure that there is time for discussion of the points raised. However, we also want to provide an opportunity for others to make short presentations or raise questions; we are considering making available a limited number of mini-slots of approx. 5-10 minutes (2-3 overheads plus time for a short discusson) for presentations on relevant topics. Because the workshop is scheduled for one day only and depending on the number of proposals received we may schedule the short presentations to the extend beyond the regular morning session. CALL FOR PARTICIPATION: If you would like to make a 5-10 minute presentation please email the organizers by Thursday 12 December, giving your name, a title for your presentation and a short abstract. We will be finalizing the program in the following week. WEB PAGE: The workshop web page is located at http://www.ncrg.aston.ac.uk/nips96/ It includes abstracts for the invited talks.