From saymukh@anise.ee.cornell.edu Wed Nov 27 15:01:35 1996
Subject: Latest Workshop Program and some information

Dear Workshop participants,

I have enclosed the latest workshop program.  Please be advised that talks
are 20 minutes long, with about 5 more minutes for questions.  If any
questions are deemed worthy of extensive discussion, we shall deal with
them in the discussion sessions. 
..............

	    ONE-DAY WORKSHOP: MODELING ERROR SURFACES
                   Saturday, December 7, 1996
			Workshop Chairs
             Sayandev Mukherjee*  &  Terrence L. Fine
                School of Electrical Engineering 	     
              Cornell University, Ithaca, NY 14853         
                {saymukh,tlfine}@ee.cornell.edu                       

INTENDED AUDIENCE
(a) Those interested in algorithms for finding minima of (error)
    surfaces in high-dimensional spaces.  
(b) Those interested in effective statistical models for 
    high-dimensional surfaces.

SUMMARY 
The training error surface, given by the mean squared error
made by the neural network when trained upon a finite training set,
may have a number of minima growing exponentially in the size of the
training set, while the generalization error surface of the neural
network also may have multiple local minima.  Clearly, a successful
model for the generalization and/or training error surface would be of
great help in training and model selection.  However, all the
conventional models for surfaces (geostatistiques models, multivariate
splines, projection-pursuit models) are ineffective for the
high-dimensional spaces characteristic of neural networks.  Certain
known properties of the topology and minima of the error surface(s)
are presented and discussed in the light of their applicability to
developing more tractable models.

GOAL
We propose a one-day workshop so as to share our knowledge about the
local minima and topology of the error surface(s), and to increase our
awareness of the main stumbling blocks in the development of viable
models for the error surface.  Our hope is that at the end of the
workshop, the participants will have a better understanding of the
problems involved and be ready to tackle them.

PLANNED FORMAT AND TENTATIVE LIST OF SPEAKERS
Morning Session (3 hours):
1. Introduction and overview: Sayandev Mukherjee
2. Insights into local minima
(a) The XOR problem has no local minima: Len Hamey
(b) The training error surface may have exponentially many local
    minima: Manfred Warmuth 
(c) Local minima and their effect on the convergence properties of 
    backpropagation: Steve Lawrence
3. Group Discussion: Relation between error surface models and local minima
4. Statistical Models for the Error Surface: 
(a) Application of the Functional Central Limit Theorem and comparisons of 
    local minima of Gaussian random fields: Michael Turmon
Afternoon Session (3 hours):
(b) Toward more convex error surfaces: Grace Wahba
5. Group Discussion: Towards effective models for high-dimensional surfaces
6. Topological, algebraic and geometric investigations of the properties of the
   error surface
(a) Topology and geometry of single hidden layer network, least squares weight 
    solutions: Frans Coetzee or Virginia Stonick
(b) Critical points and local minima for error functions: Eduardo Sontag 
(c) On convexity and neural nets with a large number of hidden units: Andrew 
    Barron
7. Panel Discussion: Future directions (Speakers+Peter Bartlett)
8. Summary and round-up: Terrence Fine
___________________________________
*
 Contact; email address saymukh@ee.cornell.edu