kinow commented on code in PR #523: URL: https://github.com/apache/opennlp/pull/523#discussion_r1149277871
########## opennlp-dl/README.md: ########## @@ -4,44 +4,50 @@ This module provides OpenNLP interface implementations for ONNX models using the **Important**: This does not provide the ability to train models. Model training is done outside of OpenNLP. This code provides the ability to use ONNX models from OpenNLP. -To build with example models, download the models to the `/src/test/resources` directory. (These are the exported models described below.) +Models used in the tests are available in the opennlp evaluation test data. -``` - -export OPENNLP_DATA=/tmp/ -mkdir /tmp/dl-doccat /tmp/dl-namefinder +## NameFinderDL -# Document categorizer model -wget https://www.dropbox.com/s/n9uzs8r4xm9rhxb/model.onnx?dl=0 -O $OPENNLP_DATA/dl-doccat/model.onnx -wget https://www.dropbox.com/s/aw6yjc68jw0jts6/vocab.txt?dl=0 -O $OPENNLP_DATA/dl-doccat/vocab.txt +* Export a Huggingface NER model to ONNX, e.g.: -# Namefinder model -wget https://www.dropbox.com/s/zgogq65gs9tyfm1/model.onnx?dl=0 -O $OPENNLP_DATA/dl-namefinder/model.onnx -wget https://www.dropbox.com/s/3byt1jggly1dg98/vocab.txt?dl=0 -O $OPENNLP_DATA/dl-/namefinder/vocab.txt +``` +python -m transformers.onnx --model=dslim/bert-base-NER --feature token-classification exported ``` -## TokenNameFinder +## DocumentCategorizerDL -* Export a Huggingface NER model to ONNX, e.g.: +* Export a Huggingface classification (e.g. sentiment) model to ONNX, e.g.: ``` -python -m transformers.onnx --model=dslim/bert-base-NER --feature token-classification exported +python -m transformers.onnx --model=nlptown/bert-base-multilingual-uncased-sentiment --feature sequence-classification exported ``` -* Copy the exported model to `src/test/resources/namefinder/model.onnx`. -* Copy the model's [vocab.txt](https://huggingface.co/dslim/bert-base-NER/tree/main) to `src/test/resources/namefinder/vocab.txt`. +## SentenceVectors -Now you can run the tests in `NameFinderDLTest`. +* Convert a sentence vectors model to ONNX, e.g.: -## DocumentCategorizer - -* Export a Huggingface classification (e.g. sentiment) model to ONNX, e.g.: +Install dependencies: ``` -python -m transformers.onnx --model=nlptown/bert-base-multilingual-uncased-sentiment --feature sequence-classification exported +python3 -m pip install optimum onnx onnxruntime +``` + +Convert the model: + ``` Review Comment: Not sure if \`\`\`bash or \`\`\`python is supported on README files for GitHub, but shouldn't matter. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
