Parsing With Compositional Vector Grammars
Richard Socher, John Bauer, Christopher D. Manning, Andrew Y. Ng
 |
Natural language parsing has typically been done with small sets of discrete categories such as NP and VP, but this representation does not capture the full syntactic nor semantic richness of linguistic phrases, and attempts to improve on this by lexicalizing phrases or splitting categories only partly address the problem at the cost of huge feature spaces and sparseness. Instead, we introduce a Compositional Vector Grammar (CVG), which combines PCFGs with a syntactically untied recursive neural network that learns syntactico-semantic, compositional vector representations. The CVG improves the PCFG of the Stanford Parser by 3.8% to obtain an F1 score of 90.4%. It is fast to train and implemented approximately as an efficient reranker it is about 20% faster than the current Stanford factored parser. The CVG learns a soft notion of head words and improves performance on the types of ambiguities that require semantic information such as PP attachments. |
Download Paper
- Download: SocherBauerManningNg_ACL2013.pdf
Code
- The new parser is shipped in Version 3.2 of the Stanford parser:
- Java code for training and testing the SU-RNN and full CVG model: http://nlp.stanford.edu/software/stanford-parser-full-2013-06-20.zip
- For instructions how to train the model on your own data or with different word vectors or other languages, see http://nlp.stanford.edu/software/parser-faq.shtml#rnn
- More information will follow soon.
Further Analysis and Visualization
- Here are three of the learned W matrices showing how the model learns a soft version of head words:
- VP-NP:
- DT-NP:
- ADJP-NP
- PP-NP
- QP-SBAR: Example for matrices that occur very rarely and hence stay close to the initialization
Bibtex
- Please cite the following paper when you use the code:@incollection{SocherEtAl2013:CVG,
title = {{Parsing With Compositional Vector Grammars}},
author = {Richard Socher and John Bauer and Christopher D. Manning and Andrew Y. Ng},
booktitle = {{ACL}},
year = {2013}
}
Comments Critique, Questions
Copy what you write before you post, then type in the password, post (nothing happens), then copy the text and re-post.