Hi Hasan
This is full of messages about incorrect alignments. It looks like
something went wrong at an earlier stage. I would start again from the
beginning with a small data set, in order to debug your setup,
cheers - Barry
On 02/11/16 15:32, Hasan Sait ARSLAN wrote:
Hi Barry,
I have rerun training starting from step 5, and I kept log file. I
have checked log file, and it doesn't show any bug about creating
phrase-table, but phrase-table is still not being created. I am
attaching log file, could you please check it, and show me where is
the problem?
Thanks,
log.gz
<https://drive.google.com/file/d/0BxvJK3H5ZKsnYzJiZmhjUWI0Qlk/view?usp=drive_web>
2016-11-02 14:28 GMT+02:00 Barry Haddow <bhad...@staffmail.ed.ac.uk
<mailto:bhad...@staffmail.ed.ac.uk>>:
Adding
-first-step 5 -last-step 5
will just run step 5 (phrase extraction)
On 02/11/16 12:01, Hasan Sait ARSLAN wrote:
For instance, could you show me an example?
Thanks,
2016-11-02 13:57 GMT+02:00 Barry Haddow
<bhad...@staffmail.ed.ac.uk <mailto:bhad...@staffmail.ed.ac.uk>>:
Hi Hasan
You can use the first-step and last-step arguments to run
steps manually.
cheers - Barry
On 02/11/16 11:46, Hasan Sait ARSLAN wrote:
Sorry,
as I see my model folder, I can say that Lexical translation
part also went well.
so, the only steps I have to run manually are'
* 5 Extract phrases
<http://www.statmt.org/moses/?n=FactoredTraining.ExtractPhrases>
* 6 Score phrases
<http://www.statmt.org/moses/?n=FactoredTraining.ScorePhrases>
* 7 Reordering model
<http://www.statmt.org/moses/?n=FactoredTraining.BuildReorderingModel>
* 8 Generation model
<http://www.statmt.org/moses/?n=FactoredTraining.BuildGenerationModel>
* 9 Configuration file
<http://www.statmt.org/moses/?n=FactoredTraining.CreateConfigurationFile>
2016-11-02 13:43 GMT+02:00 Hasan Sait ARSLAN
<hasan.sait.ars...@gmail.com
<mailto:hasan.sait.ars...@gmail.com>>:
Hi Barry,
Actually I know where I it stuck.
As I check my train folder, I can see the all the files
which show that
* 1 Prepare data
<http://www.statmt.org/moses/?n=FactoredTraining.PrepareData>
* 2 Run GIZA
<http://www.statmt.org/moses/?n=FactoredTraining.RunGIZA>
* 3 Align words
<http://www.statmt.org/moses/?n=FactoredTraining.AlignWords>
went well.
So, is there any way to run the remaining
* 4 Lexical translation
<http://www.statmt.org/moses/?n=FactoredTraining.GetLexicalTranslationTable>
* 5 Extract phrases
<http://www.statmt.org/moses/?n=FactoredTraining.ExtractPhrases>
* 6 Score phrases
<http://www.statmt.org/moses/?n=FactoredTraining.ScorePhrases>
* 7 Reordering model
<http://www.statmt.org/moses/?n=FactoredTraining.BuildReorderingModel>
* 8 Generation model
<http://www.statmt.org/moses/?n=FactoredTraining.BuildGenerationModel>
* 9 Configuration file
<http://www.statmt.org/moses/?n=FactoredTraining.CreateConfigurationFile>
steps manually?
2016-11-02 13:27 GMT+02:00 Barry Haddow
<bhad...@staffmail.ed.ac.uk
<mailto:bhad...@staffmail.ed.ac.uk>>:
Hi Hasan
It's not really possible to debug without a log
file, so I think that unfortunately you should start
again.
I suggest you start with a much smaller corpus to
try to debug your setup. If you use 100,000
sentences it should train within hours.
cheers - Barry
On 02/11/16 11:13, Hasan Sait ARSLAN wrote:
Hi Barry,
Unfortunately I didn't keep the log file. Is it
really a hopeless situation?
2016-11-02 13:10 GMT+02:00 Barry Haddow
<bhad...@staffmail.ed.ac.uk
<mailto:bhad...@staffmail.ed.ac.uk>>:
Hi Hasan
You should have run train_model.perl something
like this:
nohup nice /path/to/train_model.perl
ARGS_FOR_TRAINING &> log &
You just need the log file
cheers - Barry
On 02/11/16 11:00, Hasan Sait ARSLAN wrote:
Hi Barry,
I don't have any idea about where it is saved.
Cheers,
2016-11-02 12:55 GMT+02:00 Barry Haddow
<bhad...@staffmail.ed.ac.uk
<mailto:bhad...@staffmail.ed.ac.uk>>:
Hi Hasan
If your phrase table is empty, that would
explain why tuning didn't work. Something
went wrong earlier in the process. Could
you post your log file from train_model.perl,
cheers - Barry
On 02/11/16 10:45, Hasan Sait ARSLAN wrote:
Now currently I have a bigger problem. I
have trained my data for 5 days, and the
folder "train" is 39 G, but there is no
any phrases saved on phrase table. It is
annoying. What should I do now? I hope I
won't need to rerun everything from the
scratch
2016-11-02 12:26 GMT+02:00 Barry Haddow
<bhad...@staffmail.ed.ac.uk
<mailto:bhad...@staffmail.ed.ac.uk>>:
Hi Hasan
The error message should be written
into filterphrases.err, inside your
working directory,
cheers - Barry
On 02/11/16 10:02, Hasan Sait ARSLAN
wrote:
Hi Hieu,
I did in the way, you want. Plus, I
am sure the path and file names are
correctly spelt.
But still, I get the same error.
2016-11-01 21:41 GMT+02:00 Hasan
Sait ARSLAN
<hasan.sait.ars...@gmail.com
<mailto:hasan.sait.ars...@gmail.com>>:
Thank you Hieu, I will do it,
and give you feedback about it.
Kind Regards,
2016-11-01 21:37 GMT+02:00 Hieu
Hoang <hieuho...@gmail.com
<mailto:hieuho...@gmail.com>>:
the command you execute
looks ok. the problem is
likely to be with your using
tuning data.
You should look at your
tuning data, make sure
you've spelt the path and
filename correctly, use a
very small subset (eg. start
with 1 parallel sentence)
and increase the number of
sentences until you find the
problem
On 01/11/2016 12:30, Hasan
Sait ARSLAN wrote:
Hello,
I have trained dataset for
5 days, and for 2-3 days I
am dealing with the bug in
tuning, but still couldn't
solve the problem.
I use the following command
to run:
*/home/sait/mosesdecoder/scripts/training/mert-moses.pl
<http://mert-moses.pl>
/home/sait/Kairit/Task3/dataset_tur_en/dev.clean.tr
<http://dev.clean.tr>
/home/sait/Kairit/Task3/dataset_tur_en/dev.clean.en
/home/sait/mosesdecoder/bin/moses
/home/sait/Kairit/Task3/working_tr-en/train/model/moses.ini
--working-dir
/home/sait/Kairit/Task3/working_tr-en/mert
--decoder-flags "--threads 32"
*
Then I get the following error:
*Using SCRIPTS_ROOTDIR:
/home/sait/mosesdecoder/scripts
Assuming
--mertdir=/home/sait/mosesdecoder/bin
filtering the phrase
tables... T nov 1
19:17:39 EET 2016
exec:
/home/sait/mosesdecoder/scripts/training/filter-model-given-input.pl
<http://filter-model-given-input.pl>
./filtered
/home/sait/Kairit/Task3/working_tr-en/train/model/moses.ini
/home/sait/Kairit/Task3/dataset_tur_en/dev.clean.tr
<http://dev.clean.tr>
Executing:
/home/sait/mosesdecoder/scripts/training/filter-model-given-input.pl
<http://filter-model-given-input.pl>
./filtered
/home/sait/Kairit/Task3/working_tr-en/train/model/moses.ini
/home/sait/Kairit/Task3/dataset_tur_en/dev.clean.tr
<http://dev.clean.tr> >
filterphrases.out 2>
filterphrases.err
Exit code: 255
ERROR: Failed to run
'/home/sait/mosesdecoder/scripts/training/filter-model-given-input.pl
<http://filter-model-given-input.pl>
./filtered
/home/sait/Kairit/Task3/working_tr-en/train/model/moses.ini
/home/sait/Kairit/Task3/dataset_tur_en/dev.clean.tr
<http://dev.clean.tr>'. at
/home/sait/mosesdecoder/scripts/training/mert-moses.pl
<http://mert-moses.pl> line
1748.
*
I have searched over the
internet about the error,
but unfortunately I
couldn't find any
beneficial solution.
It is so annoying, and eats
my time, please help me to
solve my problem.
Thanks,
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
<mailto:Moses-support@mit.edu>
http://mailman.mit.edu/mailman/listinfo/moses-support
<http://mailman.mit.edu/mailman/listinfo/moses-support>
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
<mailto:Moses-support@mit.edu>
http://mailman.mit.edu/mailman/listinfo/moses-support
<http://mailman.mit.edu/mailman/listinfo/moses-support>
The University of Edinburgh is a
charitable body, registered in
Scotland, with registration number
SC005336.
The University of Edinburgh is a
charitable body, registered in
Scotland, with registration number SC005336.
The University of Edinburgh is a charitable
body, registered in
Scotland, with registration number SC005336.
The University of Edinburgh is a charitable body,
registered in
Scotland, with registration number SC005336.
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
