umls-senserelate-evaluation.pl - This program evaluates the umls-senserelate algorithm using the semeval scorer program
This program evaluates the senses assigned by the UMLS-SenseRelate algorithm using the semeval scorer program
Usage: umls-senserelate-evaluation.pl LOG_DIRECTORY
This is the directory outputted by the umls-targetword-senserelate.pl program which contains the key and answer files for each of the target words.
Prints out accuracy information to FILE
Sometimes of the senses in the key file do not map directly to the senses in the answer file. This will happen when using the NLM-WSD dataset which uses tags to indicate the sense of the ambiguous word and then those tags are mapped to concepts in the UMLS. The key file would contain the CUIs while the answer file contains the tags. Therefore, we need the sense (sometimes called choice) files to evaluate the mappings.
So this options takes the directory that contains the the sense file for each target word you are going to disambiguate or just the file itself.
The files for the target word contains the possible senses of the target word.
This may be temporary but right now this is who I have it because often times the possible senses change depending on the version of the UMLS that you are using. I felt this allowed the most flexibility with it.
The naming convention for this is a file called: <target word>.choices
The format for this file is:
<tag>|<target word name>|semantic type|CUI
This format is based on the choice files in the NLM-WSD dataset which we use for our experiments. If you are using the NLM-WSD dataset you can download these choice files from NLM's site. There are the 1999 tagset and the 2007 tagset available. You can find them here:
http://wsd.nlm.nih.gov/collaboration.shtml
Prints out accuracy information to STDOUT.
Displays the version information.
Displays the help information
Perl (version 5.8.5 or better) - http://www.perl.org
If you have any trouble installing and using UMLS-Similarity, please contact us via the users mailing list : umls-similarity@yahoogroups.com You can join this group by going to: http://tech.groups.yahoo.com/group/umls-similarity/ You may also contact us directly if you prefer : Bridget T. McInnes: bthomson at umn.edu Ted Pedersen : tpederse at d.umn.edu
Bridget T. McInnes, University of Minnesota
Copyright (c) 2010-2012,
Bridget T. McInnes, University of Minnesota Twin Cities bthomson at umn.edu Ted Pedersen, University of Minnesota Duluth tpederse at d.umn.edu Serguei Pakhomov, University of Minnesota Twin Cities pakh0002 at umn.edu Ying Liu, University of Minnesota Twin Cities liux0395 at umn.edu
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to:
The Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
To install UMLS::SenseRelate, copy and paste the appropriate command in to your terminal.
cpanm
cpanm UMLS::SenseRelate
CPAN shell
perl -MCPAN -e shell install UMLS::SenseRelate
For more information on module installation, please visit the detailed CPAN module installation guide.