span.GRAME {mso-gram-e:yes;}

Topics on Vision in Humans and Machines


CSE 291-B100, Spring 2005


David Kriegman

Kriegman {at}


Marian Stewart Bartlett

Marni {at}


Grading: S/U,  2 units only.


This graduate seminar will explore topics in vision such as stereo, segmentation, motion interpretation, object recognition, face identity and expression recognition.  In each class session, a topic will be considered from the perspectives of computer vision and human visual perception.  There will be assigned readings on both human and machine vision, and a pair of students will serve as discussants from the two perspectives.


Class structure: Every week, each student will submit two questions or comments on the readings to the Discussion Board. Students will also complete tutorials from the Cold Spring Harbor course on Computational Visual Neuroscience.


3/30       Organizational Meeting, Center Hall 223 9:30-11.


4/06       Statistics of Natural Images


4/13       Information Theory and Neural Representation


4/20       Luminance, Illumination, and Contrast Normalization


4/27       Motion


5/4          Color


5/11       Texture/Segmentation


5/18       Object Representation and Recognition


5/25       Face Processing


6/1         Active Exploration


Reading List



4/06       Statistics of Natural Images


Field (1987) Relations between the statistics of natural images and the response properties of cortical cells. J. Opt. Soc. Am.  (or more recent papers, e.g. ‘What is the goal of sensory coding’  Neural Computation). pdf


J. Huang and D. Mumford. Statistics of natural images and models. CVPR,  pages 541–7, 1999. pdf




Ron O. Dror, Thomas K. Leung, Edward H. Adelson, and Alan S. Willsky,

Statistics of Real-World Illumination  CVPR 2001 pdf


Ruderman & Bialek (1994). Statistics of natural images: scaling in the woods. Physical Review letters.  pdf


Field (1999). Wavelets, vision and the statistics of natural scenes. Phil. Trans. R. Soc. London A. 357, 2527-2542. pdf



4/13       Information Theory and Neural Representation


Simoncelli & Olshaussen (2001). Natural image statistics and Neural Representation. Ann Rev Neurosci. This was on the reading list for the cogsci200 on learning in vision. pdf


Linsker (1990). Perceptual neural organization – Some approaches based on network models and information theory.  Annual Review of Neurology 13, p. 257-281. pdf




Barlow (1989). Unsupervised learning. (Or his original 1961 paper ‘possible principles underlying the transformations of sensory messages.’ It comes highly recommended.) pdf


Olshaussen & Field (1996). Emergence of simple cell receptive field properties by learning a sparse code for natural images. Nature 381, p. 607-609.  pdf See also Vision Research 1997. pdf


Bell & Sejnowski (1996). The independent components of natural scenes are edge filters. Vision Research. pdf


Edelman & Intrator (2004). Unsupervised statistical learning in vision: Computational principles, biological evidence. Interesting but short. pdf


Shannon, C.E. (1948). A Mathematical Theory of Communication. Bell System Technocal Journal 27, 379-423, 623-656. pdf



4/20       Luminance, illumination, and contrast normalization


Wainwright, Schwartz & Simoncelli (2002). Natural image statistics and divisive normalization: Modeling nonlinearity and adaptation in cortical neurons. In Rao, Olshausen & Lewicki (eds). Probabilistic models of the brain: Perception and Neural Function. pdf


J. Ho, D. Kriegman, ``On the Efect of Illumination and Face Recognition” to appear in “Face Processing: Advanced Modeling and Methods,” R. Chellappa, W. Zhao, editors, 2005. pdf




Adelson (2000). Lightness perception and lightness illusions. In Gazzaniga (ed). The new cognitive neurosciences. pdf


Demos of Adelson’s lightness illusions.



4/27       Motion


Fleet, D. J., Black, M. J. and Nestares, O., (2002). Bayesian Inference of Visual Motion Boundaries, in Lakemeyer,  G. and Nebel, B. (Eds.) Exploring Artificial Intelligence in
the New Millennium, Morgan Kaufmann Pub. pdf

J. Xiao, S. Baker, I. Matthews, and T. Kanade (2004). Real-Time Combined 2D+3D Active Appearance Models. CVPR. pdf

Demos of the system in the Xiao paper and also demos of other AAM systems by the same group




Albright (1993) Cortical Processing of Visual Motion. In F.A. Miles and J. Wallman, Eds. Visual Motion and its Role in the Stabilization of Gaze. Elsavier Science Publishers, pg. 177-201. pdf


Adelson & Bergen (1985). Spatio-temporal energy models for the perception of motion. JOSA. pdf


Weiss, Simoncelli & Adelson (2002) Motion Illusions as optimal percepts. Nature Neuroscience. pdf


Black, M. J. and Anandan, P., (1996) Computer Vision and Image Understanding, CVIU, 63(1), pp. 75-104. pdf


Bruhn, Weikert and Schnorr (2005). Lucas/Kanade Meets Horn/Schunk: Combining Local and Global Optic Flow Methods.  IJCV 61(3):211-231, Feb 2005. pdf



5/4          Color


von der Twer, T. and Macleod. D.I.A. (2001). Optimal nonlinear codes for the perception of natural colors. Network, Computaiton inNeural Systems, 12 395-407. pdf


NOTE: We originally listed the following book chapter, but the Network paper is a little easier to follow, so Josh recommends reading that one instead. If you read the book chapter, then  Josh will be focusing on pgs 1-21 of the chapter, which covers the same material as the Network  paper. MacLeod, D.I.A. (2003). Colour Discrimination, Colour Constancy, and  Natural Scene Statistics (The Verriest Lecture). In: J.D. Mollon, J.  Pokorny, & K. Knoblauch (Eds.), Proceedings of the Thomas Young  Symposium of the International Colour Vision Society  London: Oxford  University Press. pdf  


Neitz, Carroll, Yamauchi, Neitz, & Williams (2002). Color Perception Is Mediated by a Plastic Neural Mechanism that Is Adjustable in Adults. Neuron, Vol. 35, 783. pdf




Brainard Kraft & Longere (2003). Color contstancy: developing empirical tests of computational models. In Maunsfeld & Heyer (Eds.) Colour Perception: Mind and the physical world. pdf


Delahunt & Brainard (2004). Does human color constancy incorporate the statistical regularity of natural daylight? Journal of vision 4 p. 57-81. pdf


Rhea Eskew. Chromatic Detection and  Discrimination, In Gegenfurtner and Sharpe (Eds), Color Vision from Genes to Perception. Cambridge University Press. p. 345-368.


Nathans, J., (1989) The genes for color vision. Sci Am, 1989. 260(2): p.
42-9. pdf



5/11       Texture / segmentation


Malik & Perona (1990) Preattentive texture discrimination with early vision mechanisms. JOSA pdf


Zhu, Wu, & Mumford (1997). Minimax Entropy Principle and its Application to Texture Modeling. Neural Computation 9(8), p. 1627-1660. pdf





Rosenholtz (2001). Search asymmetries? What search asymmetries? Perception & psychophysics. pdf


Rosenholtz (2000). Significantly different textures: a computational model of pre-attentive texture segmentation. ECCV. pdf


Farid & Adelson (2001). Synchrony does not promote grouping in temporally structured displays. Nature Neuroscience 4(9) p. 875-876. pdf


Heeger and Bergen (1995). Pyramid-Based Texture Analysis and Synthesis. Siggraph. P. 229-238. pdf


Fowlkes & Malik. Learning edge detection from human examples.


5/18       Object representation


Kersten, Mamassian & Yuille (2004). Object perception as Bayesian inference. Annual review of Psychology.  pdf (See also Kersetn & Yuille 2003, Bayesian models of object perception. Current Opinion in Neurobiology. ) pdf


Rothganger, F., Schmidt, C., Ponce, J. (2005). 3D Object Modeling Using Local Affine-Invariant Image Descriptors and Multi-View Spatial Constraints. International Journal of Computer Vision. pdf




Edelman & Duvdevani-barr (1997). A model of visual recognition and categorization. Phil Trans. Royal Soc. London 352 p. 1191-1202. pdf


2D vs 3D (Mike Tarr) and/or Biederman & Kolocsoi



5/25       Face Representation


Bartlett (submitted). Information maximization in face processing. Neurocomputing.  pdf


Webster, Werner, & Field (in press) . “Adaptation and the phenomenology of perception.”  In Clifford and Rhodes (Eds.) Fitting the mind to the world: adaptation and aftereffects in high level vision.  (Mike Webster has some cool work on face adaptation that speaks to the information maximization idea.) doc




Valentine (2000). Face Space models of face recognition. In Wenger & Townhend (Eds.) Computational, geometric, and process perspectives on facial cognition: Contexts and challenges. pdf



6/1         Active Exploration


Sprague, Ballard and Robinson (submitted). Modeling attention with Embodied Visual Behaviors.  ACM Transactions on Applied Perception. pdf 


Walker-Renninger, Coughlan, Verghese, & Malik (2005). An Information
Maximization Model of Eye Movements.  NIPS 17, 1121-1128. pdf

Please also read the introduction of Nelson (in press) Psych Review, for a review of probabilistic accounts of information acquisition (pgs 1-11 and 42-44, and the tables on 52,53,64). Nelson, J.D. (in press) Finding useful questions. On Bayesian diagnosticity, probability, impact, and information gain. Psychological Review. pdf




Walther, D. , Rutishauser, U., Koch, C., and Perona, P. (in  press). Selective visual attention enables learning and recognition of multiple objects in cluttered scenes. Computer Vision and Image Understanding. pdf


Denzler, J. & Brown, C.M. (2002). Information theoretic sensor data selection for active object recognition and state estimation. PAMI 24 p. 145-157. pdf


Yuille, Coughlan & Kersten (1998). Computational vision: Principles of perceptual inference. Book Chapter.  This is a good chapter introducing Bayesian models, as well as how they tie into information theory. Also covers active exploration. pdf


Lee & Yu (1999). An Information-Theoretic Framework for Understanding Saccadic Behaviors.  NIPS 12, 834-840. pdf

Najemnik, J. & Geisler, W. S. (2005, March 17).  Optimal eye movement strategies in visual search.  Nature, 434, 387-391.


Nelson & Cottrell (submitted). A probabilistic model of eye movements
in concept formation.



Related Tutorials


Cold Spring Harbor Computational Vision Tutorials by Eero Simoncelli, Paul Glimcher, EJ Cichilnisky, and others.


NIPS Computational Vision Tutorial by Dan Kertsen


ICA Tutorial by Hyvarinen & Oja


Linear Algebra tutorial by Eero Simoncelli

Linear Algebra in Parallel Distributed Processing by Mike Jordan.