> > > > > > Option 2) > > > > > > Write c code in FFmpeg to convert tensorflow file format (format 1) > > directly > > > > > into memory representation (format 3), and so we controls everything > > > > > in > > > > > ffmpeg community. And the conversion can be extended to import more > > file > > > > > formats such as torch, darknet, etc. One example is that OpenCV uses > > this > > > > > method. > > > > > > > > > > > > The in memory representation (format 3) can still be current. > > > > > > > > > > > > > > > > Option 2 would be ideal, as it does not introduce any dependency for > > > > > using the native backend. > > > > > Yet I'm not sure how complex implementing the tf model reader can > be, > > > > > If I remember correctly the student said it was not trivial at the > > > > > time. > > > > > > > > yes, it is not easy, but I think it is worthy to do it. Here is a > > > > reference > > example > > > > for the complexity, see > > > > > > > https://github.com/opencv/opencv/blob/master/modules/dnn/src/tensorflow/ > > > > tf_importer.cpp. > > > > > > > > > > > > > > Is the tf model file stable? if not it will be a maintenance burden to > > > > > keep it working whenever tf releases a new version. This point makes > > > > > me think having control over our file format is good. > > > > > > > > imho, this issue is always there, no matter which method used, unless > > > > our > > > > format could be exported by tensorflow (it has little possibility). > > > > > > > > Whenever tf releases a new version with a new file format, we still have > to > > > > change the python script in phase 1 (convert tf file model to our > > > > format) > > which > > > > is even an external dependency at > > > > https://github.com/HighVoltageRocknRoll/sr, > > > > > > > > As from effort perspective, the current implementation is better since > > python > > > > script is simpler. But I think we are still worth implementing option 2 > > > > as > the > > > > ideal technical direction. > > > > > > I checked a bit more about https://github.com/HighVoltageRocknRoll/sr, it > is > > actually > > > not an converter (from tf model to native model), but hard code for given > > models. > > > And the native model is not exactly the same as tf model, it even changes > the > > behavior > > > of pad parameter of conv layer. > > > > > > If community is open to option 2, I'll try it. > > > > > Option 2 is fine for me. > > that's great, :)
looks that option 2 is a bit complex, TF model file is in protocol buffers (protobuf) format and not easy to parse it with simple c code. Since there is no official c support for protobuf, let's first image how the work can be done via official c++ support. 1. get protobuf compiler protoc, .h header files and .so library files (download or build from https://github.com/protocolbuffers/protobuf/tree/master/src). 2. get tensorflow model's .proto files from https://github.com/tensorflow/tensorflow/tree/master/tensorflow/core/framework. 3. generate .cc/.h files from .proto files (see step 2) via protoc (see step 1). 4. let the generated .cc/.h files be part of ffmpeg source tree, and build with protobuf header/library files. 5. at run time, the protobuf libraries are invoked. It means that the system should have installed protobuf dev package. furthermore, there is a compatible problem between the protobuf compiler, header files and library files. So, as a practice to fix it, the method is to make the protobuf source code be part of ffmpeg source tree. (it is a common practice, so we can many other projects contain the protobuf source code). I guess the above method is not acceptable in ffmpeg. I would be glad to continue if the community embrace this change. :) While the current implementation has external dependency, my new suggestion is: - add a python script under .../libavfilter/dnn/ (all other dnn source files will be also moved here later), so ffmpeg has the full control on it. - it is a script to convert tensorflow model file into native model file. (other formats such as caffe, torch can also be supported later if needed) thanks. _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".