Generative versus discriminative training of RBMs for classification of fMRI images

Tanya Schmah; Geoffrey E. Hinton; Richard S. Zemel; Steven L. Small; Stephen Strother

Generative versus discriminative training of RBMs for classification of fMRI images

Tanya Schmah, Geoffrey E. Hinton, Richard S. Zemel, Steven L. Small, Stephen Strother

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Neuroimaging datasets often have a very large number of voxels and a very small number of training cases, which means that overfitting of models for this data can become a very serious problem. Working with a set of fMRI images from a study on stroke recovery, we consider a classification task for which logistic regression performs poorly, even when L1- or L2- regularized. We show that much better discrimination can be achieved by fitting a generative model to each separate condition and then seeing which model is most likely to have generated the data. We compare discriminative training of exactly the same set of models, and we also consider convex blends of generative and discriminative training.

Original language	English (US)
Title of host publication	Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference
Publisher	Neural Information Processing Systems
Pages	1409-1416
Number of pages	8
ISBN (Print)	9781605609492
State	Published - 2009
Externally published	Yes
Event	22nd Annual Conference on Neural Information Processing Systems, NIPS 2008 - Vancouver, BC, Canada Duration: Dec 8 2008 → Dec 11 2008

Publication series

Name	Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference

Conference

Conference	22nd Annual Conference on Neural Information Processing Systems, NIPS 2008
Country/Territory	Canada
City	Vancouver, BC
Period	12/8/08 → 12/11/08

ASJC Scopus subject areas

Information Systems

Cite this

Schmah, T., Hinton, G. E., Zemel, R. S., Small, S. L., & Strother, S. (2009). Generative versus discriminative training of RBMs for classification of fMRI images. In Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference (pp. 1409-1416). (Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference). Neural Information Processing Systems.

Generative versus discriminative training of RBMs for classification of fMRI images. / Schmah, Tanya; Hinton, Geoffrey E.; Zemel, Richard S. et al.
Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference. Neural Information Processing Systems, 2009. p. 1409-1416 (Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Schmah, T, Hinton, GE, Zemel, RS, Small, SL & Strother, S 2009, Generative versus discriminative training of RBMs for classification of fMRI images. in Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference. Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference, Neural Information Processing Systems, pp. 1409-1416, 22nd Annual Conference on Neural Information Processing Systems, NIPS 2008, Vancouver, BC, Canada, 12/8/08.

Schmah T, Hinton GE, Zemel RS, Small SL, Strother S. Generative versus discriminative training of RBMs for classification of fMRI images. In Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference. Neural Information Processing Systems. 2009. p. 1409-1416. (Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference).

Schmah, Tanya ; Hinton, Geoffrey E. ; Zemel, Richard S. et al. / Generative versus discriminative training of RBMs for classification of fMRI images. Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference. Neural Information Processing Systems, 2009. pp. 1409-1416 (Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference).

@inproceedings{155772dda3ac43b392564a25c7db311e,

title = "Generative versus discriminative training of RBMs for classification of fMRI images",

abstract = "Neuroimaging datasets often have a very large number of voxels and a very small number of training cases, which means that overfitting of models for this data can become a very serious problem. Working with a set of fMRI images from a study on stroke recovery, we consider a classification task for which logistic regression performs poorly, even when L1- or L2- regularized. We show that much better discrimination can be achieved by fitting a generative model to each separate condition and then seeing which model is most likely to have generated the data. We compare discriminative training of exactly the same set of models, and we also consider convex blends of generative and discriminative training.",

author = "Tanya Schmah and Hinton, {Geoffrey E.} and Zemel, {Richard S.} and Small, {Steven L.} and Stephen Strother",

year = "2009",

language = "English (US)",

isbn = "9781605609492",

series = "Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference",

publisher = "Neural Information Processing Systems",

pages = "1409--1416",

booktitle = "Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference",

note = "22nd Annual Conference on Neural Information Processing Systems, NIPS 2008 ; Conference date: 08-12-2008 Through 11-12-2008",

}

TY - GEN

T1 - Generative versus discriminative training of RBMs for classification of fMRI images

AU - Schmah, Tanya

AU - Hinton, Geoffrey E.

AU - Zemel, Richard S.

AU - Small, Steven L.

AU - Strother, Stephen

PY - 2009

Y1 - 2009

N2 - Neuroimaging datasets often have a very large number of voxels and a very small number of training cases, which means that overfitting of models for this data can become a very serious problem. Working with a set of fMRI images from a study on stroke recovery, we consider a classification task for which logistic regression performs poorly, even when L1- or L2- regularized. We show that much better discrimination can be achieved by fitting a generative model to each separate condition and then seeing which model is most likely to have generated the data. We compare discriminative training of exactly the same set of models, and we also consider convex blends of generative and discriminative training.

AB - Neuroimaging datasets often have a very large number of voxels and a very small number of training cases, which means that overfitting of models for this data can become a very serious problem. Working with a set of fMRI images from a study on stroke recovery, we consider a classification task for which logistic regression performs poorly, even when L1- or L2- regularized. We show that much better discrimination can be achieved by fitting a generative model to each separate condition and then seeing which model is most likely to have generated the data. We compare discriminative training of exactly the same set of models, and we also consider convex blends of generative and discriminative training.

UR - http://www.scopus.com/inward/record.url?scp=78149327745&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=78149327745&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:78149327745

SN - 9781605609492

T3 - Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference

SP - 1409

EP - 1416

BT - Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference

PB - Neural Information Processing Systems

T2 - 22nd Annual Conference on Neural Information Processing Systems, NIPS 2008

Y2 - 8 December 2008 through 11 December 2008

ER -

Generative versus discriminative training of RBMs for classification of fMRI images

Abstract

Publication series

Conference

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this