Hi Arefeh
Attached.
If you look at the files produced in your tuning run, you should see the
following produced for each iteration:
run2.best100.out.gz
run2.dense
run2.extract.err
run2.extract.out
run2.features.dat
run2.init.opt
run2.mert.log
run2.mert.out
run2.mira.out
run2.moses.ini
run2.out
run2.scores.dat
run2.sparse-weights
run2.weights.txt
(run1 may be different)
In run2.best100.out.gz you should see the sparse feature values. In
run2.mert.out you should see the sparse feature weights.
cheers - Barry
On 20/08/16 12:47, arefeh kazemi wrote:
Hi Barry
Thanks,
I tried verbose and I can see my sparse feature and it's scores for
each sentence, but the weights.txt file is still empty.
(I use kbmira for tuning).
I think there is a problem with moses.ini file.
Could you please send an example .ini file with a sparse feature?
Thanks
Arefeh
On Thu, Aug 18, 2016 at 6:13 PM, Barry Haddow
<bhad...@staffmail.ed.ac.uk <mailto:bhad...@staffmail.ed.ac.uk>> wrote:
Hi Arefeh
The quickest way to see if Moses is using your feature is to put a
debug message in it to see if it gets called. You can also
increase the debug of Moses (try -v 2) to see if your feature's
scores appear in the breakdown.
To populate the weights file, you will need to run tuning (kbmira
or pro). If you just decode with an empty weights file, all the
weights will be set to 0 and Moses will not update the file,
cheers - Barry
On 18/08/16 14:15, arefeh kazemi wrote:
Hi Barry
Thanks.
I create an empty weights.txt file and write it's address in
moses.ini. Moses runs normally but weights file remains empty. It
seems moses doesn't use my feature.
Regards
Arefeh
On Wed, Aug 17, 2016 at 12:57 PM, Barry Haddow
<bhad...@staffmail.ed.ac.uk <mailto:bhad...@staffmail.ed.ac.uk>>
wrote:
Hi Arefeh
That seems OK. Tuning (with kbmira or pro) will create a
weights file for the sparse features, which you can add with:
[weight-file]
/path/to/sparse/weights
What goes wrong when you run moses?
cheers - Barry
On 17/08/16 07:50, arefeh kazemi wrote:
Hi
This is just a kindly reminder that I'm waiting for response.
Thanks
Arefe
---------- Forwarded message ----------
From: *arefeh kazemi* <akazem...@gmail.com
<mailto:akazem...@gmail.com>>
Date: Tue, Aug 9, 2016 at 7:28 PM
Subject: moses.ini file for sparse features
To: Moses-support <moses-support@mit.edu
<mailto:moses-support@mit.edu>>
Hi
I've implemented a sparse feature function in Moses Hiero
system but I don't know what are the parameters in moses.ini
file for a sparse feature.
for the dense version of my feature, I had these lines in my
ini file:
[features]
Aref num-features=4 ...
[weight]
Aref= 0.2 0.2 0.2 0.2
Now, What should I write for my sparse feature in .ini file?
I have removed the weights for Aref and also "num-features"
from ini file, but it doesn't work.
Regards
Arefeh Kazemi
--
Arefeh Kazemi
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu <mailto:Moses-support@mit.edu>
http://mailman.mit.edu/mailman/listinfo/moses-support
<http://mailman.mit.edu/mailman/listinfo/moses-support>
The University of Edinburgh is a charitable body, registered
in Scotland, with registration number SC005336.
--
Arefeh Kazemi
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
--
Arefeh Kazemi
# MERT optimized configuration
# decoder /home/bhaddow/moses.new/dist/977e8ea/bin/moses
# BLEU 0.283399 on dev
/home/bhaddow/data/wmt16/qt21-ro-en/newsdev2016.1.en-ro.true.en
# We were before running iteration 4
# finished Wed Apr 13 10:50:37 BST 2016
### MOSES CONFIG FILE ###
#########################
# input factors
[input-factors]
0
# mapping steps
[mapping]
0 T 0
[distortion-limit]
6
# additional settings
[feature]
TargetWordInsertionFeature name=TWI factor=0
path=/fs/magni0/bhaddow/experiments/wmt16/en-ro/model/sparse-features.24.ro.top50
SourceWordDeletionFeature name=SWD factor=0
path=/fs/magni0/bhaddow/experiments/wmt16/en-ro/model/sparse-features.24.en.top50
WordTranslationFeature name=WT input-factor=0 output-factor=0 simple=1
source-context=0 target-context=0
source-path=/fs/magni0/bhaddow/experiments/wmt16/en-ro/model/sparse-features.24.en.top50
target-path=/fs/magni0/bhaddow/experiments/wmt16/en-ro/model/sparse-features.24.ro.top50
PhraseLengthFeature name=PL
# feature functions
[feature]
UnknownWordPenalty
WordPenalty
PhrasePenalty
PhraseDictionaryCompact name=TranslationModel0 num-features=13
path=/fs/magni0/bhaddow/experiments/wmt16/en-ro/tuning/filtered.24/phrase-table.0-0.1.1
input-factor=0 output-factor=0
LexicalReordering name=LexicalReordering0 num-features=8
type=hier-mslr-bidirectional-fe-allff input-factor=0 output-factor=0
path=/fs/magni0/bhaddow/experiments/wmt16/en-ro/tuning/filtered.24/reordering-table.24.hier-mslr-bidirectional-fe
OpSequenceModel name=OpSequenceModel0 num-features=5
path=/fs/magni0/bhaddow/experiments/wmt16/en-ro/model/OSM.15/operationLM.bin
Distortion
KENLM lazyken=0 name=LM0 factor=0
path=/fs/magni0/bhaddow/experiments/wmt16/en-ro/lm/all.binlm.23 order=5
# dense weights for feature functions
[search-algorithm]
1
[weight-file]
/fs/magni0/bhaddow/experiments/wmt16/en-ro/tuning/tmp.24/run4.sparse-weights
[max-trans-opt-per-coverage]
100
[cube-pruning-pop-limit]
1000
[v]
0
[threads]
8
[s]
1000
[weight]
LexicalReordering0= 0.038456101061463 0.0374516770203483 0.0220959009915572
0.0498669293337116 0.0298206095205517 0.00767781286108161 0.00626529880981681
0.03338468028193
OpSequenceModel0= 0.0451261831356156 -0.030675703926822 0.137808631813838
-0.0386300809832872 -0.0640297216566551
Distortion0= -0.00474902057796337
LM0= 0.053400563002935
WordPenalty0= -0.0876941474639328
PhrasePenalty0= -0.0282173638737803
TranslationModel0= 0.0379918038878 0.00826205472382949 -0.000111040524446519
0.00616626928800785 0.0749186101634312 0.0473675555712462 -0.0378675754598192
-0.00663601507693318 0.011657406295361 0.00530866476105118 0.00664256844591075
-0.0243338163747455 0.0173861931121278
UnknownWordPenalty0= 1
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
