Install Train and Test Moses Machine Translation Toolkit

Following are few easy steps with my experience with installing moses toolkit

I am doing it for ubuntu 64 bit (linux) system you can make an analogy for other installations.

1) Download the binary moses from here http://www.statmt.org/moses/?n=Moses.Packages In my case I used this - http://www.statmt.org/~jie/linux/moses-2.1-1/moses_2.1-1_amd64.deb


2) Install that with following command in the relevant directory
 sudo dpkg -i http://www.statmt.org/~jie/linux/moses-2.1-1/moses_2.1-1_amd64.deb

This install moses and all other relevant tools like GIZA and Language Modeler (IRSTLM) in "/opt" directory. You can copy that to wherever you want or leave it as it is there.

3)  Now goto /moses folder in /opt and move all contents of /giza++-v1.0.7 to a new folder /tools in /moses. You can avoid this, but then need to make some changes in training script below, so don't avoid it for the sake of introduction.

4) Creating Language Model: Your environment is set up. Now create a language model using target language corpus using following shell script. You are in your workdirectory say - YF

/opt/moses/irstlm-5.70.04/bin/add-start-end.sh \
   < YF/monolingualFilepath.hin \
   > LMV1.fl

export IRSTLM=/opt/moses/irstlm-5.70.04; 

/opt/moses/irstlm-5.70.04/bin/build-lm.sh \
   -i LMV1.fl                  \
   -t ./tmp  -p -s improved-kneser-ney -o LMV2.fl

/opt/moses/irstlm-5.70.04/bin/compile-lm  \
   --text=yes \
   LMV2.fl.gz \
   MYLM.lm

/opt/moses/bin/build_binary \
   MYLM.lm \
   MyLm.lm.bin.hi

I you don't want to worry much about what is happening, name your monolingual input data file for LM creation as - monolingualFilepath.hin and put in the directory where this script will be run i.e. YF. And if you understand what's it. Please go ahead and do as as you want the files to be named.

5) The above step creates new files move all new files to a new folder and name it /lm. Your language model is ready of which you will be using the MyLM.lm.bin.hin version.

Check you built binarized LM for Hindi i.e. MyLM.lm.bin.hin with:

echo "यह हिंदी वाक्य है  क्या ?" | /opt/moses/bin/query MyLm.lm.bin.hi 

It gives some numbers per word and that means you are done with LM creation.

6) Training the English to Hindi SMT system: Keep your bi-lingual training corpus in a folder called /corpus inside /YF and name files traincorpus.hin and traincorpus.eng. And then run the following script to train a English to Hindi translation system.

/opt/moses/scripts/training/train-model.perl \
    -cores 4 \
    -root-dir /YF \
    -corpus YF/corpus/traincorpus -f eng -e hin \
    -alignment grow-diag-final-and \
    -reordering msd-bidirectional-fe \
    -lm 0:3:YF/lm/MyLm.lm.bin.hi:8 \
    -external-bin-dir /opt/moses/tools >& YF/trainingLog.out

 7) It will take some time depending on the training data size. Once it creates the moses.ini, we can say that the training is done. Now you can try the trained system by executing following command -

/opt/moses/bin/moses -f YF/model/moses.ini 

It will ask for the English sentence and will output the Hindi. 
And we are DONE!