[GitHub] aaronmarkham commented on issue #13647: Update lip reading example

aaronmarkham commented on issue #13647: Update lip reading example URL: https://github.com/apache/incubator-mxnet/pull/13647#issuecomment-461618966 I put the tar files in a separate bucket so you can pick how you want to download. To get the tar files: ``` aws s3 sync

[GitHub] aaronmarkham commented on issue #13647: Update lip reading example

aaronmarkham commented on issue #13647: Update lip reading example URL: https://github.com/apache/incubator-mxnet/pull/13647#issuecomment-461254738 @thomelane Ok, I'm creating the tar files now. The reason I didn't do that is that I kept getting disconnected and wanted to be able to resume

[GitHub] aaronmarkham commented on issue #13647: Update lip reading example

aaronmarkham commented on issue #13647: Update lip reading example URL: https://github.com/apache/incubator-mxnet/pull/13647#issuecomment-457411369 @seujung @soeque1 Let me know when you're ready for another review. This is

[GitHub] aaronmarkham commented on issue #13647: Update lip reading example

aaronmarkham commented on issue #13647: Update lip reading example URL: https://github.com/apache/incubator-mxnet/pull/13647#issuecomment-456561421 I see one [outstanding suggestion to put in the Python3 flag](https://github.com/apache/incubator-mxnet/pull/13647#discussion_r248661689)

[GitHub] aaronmarkham commented on issue #13647: Update lip reading example

aaronmarkham commented on issue #13647: Update lip reading example URL: https://github.com/apache/incubator-mxnet/pull/13647#issuecomment-453380303 I made it public read. You need AWS cli installed. 'sudo apt install awscli' You might need to still create an account with IAM and run aws

[GitHub] aaronmarkham commented on issue #13647: Update lip reading example

aaronmarkham commented on issue #13647: Update lip reading example URL: https://github.com/apache/incubator-mxnet/pull/13647#issuecomment-453211484 Putting this here, so when my comment in the review gets outdated, you still can see how to get the preprocessed data: The readme could

[GitHub] aaronmarkham commented on issue #13647: Update lip reading example

aaronmarkham commented on issue #13647: Update lip reading example URL: https://github.com/apache/incubator-mxnet/pull/13647#issuecomment-452475997 Multi-gpu training is working! Is there any advice you want to provide in the Readme about recommended maximum batch size or how to scale up

[GitHub] aaronmarkham commented on issue #13647: Update lip reading example

aaronmarkham commented on issue #13647: Update lip reading example URL: https://github.com/apache/incubator-mxnet/pull/13647#issuecomment-452405729 Thanks for adding multi-gpu - that's great! I'll test it out. Restarting CI due to a "OSError: [Errno 14] Bad address" error. Probably

[GitHub] aaronmarkham commented on issue #13647: Update lip reading example

aaronmarkham commented on issue #13647: Update lip reading example URL: https://github.com/apache/incubator-mxnet/pull/13647#issuecomment-452027678 The command line example seems to be working. It would be a nice bonus if this also used multi-gpu. Or is that an option that I missed?

[GitHub] aaronmarkham commented on issue #13647: Update lip reading example

aaronmarkham commented on issue #13647: Update lip reading example URL: https://github.com/apache/incubator-mxnet/pull/13647#issuecomment-451636856 I ran into another bug that is easily resolved. ``` mxnet.base.MXNetError: [17:39:13] src/io/local_filesys.cc:199: Check failed:

[GitHub] aaronmarkham commented on issue #13647: Update lip reading example

aaronmarkham commented on issue #13647: Update lip reading example URL: https://github.com/apache/incubator-mxnet/pull/13647#issuecomment-451452405 With regard to the data that is preprocessed - is this just the /data/align folder that is needed? So if I upload that to s3 that is all that

[GitHub] aaronmarkham commented on issue #13647: Update lip reading example

aaronmarkham commented on issue #13647: Update lip reading example URL: https://github.com/apache/incubator-mxnet/pull/13647#issuecomment-451425540 > * (3-2) Stop the preprocess. Then resume the process. If there is the processed image, it skips. This worked out! I finished

[GitHub] aaronmarkham commented on issue #13647: Update lip reading example

aaronmarkham commented on issue #13647: Update lip reading example URL: https://github.com/apache/incubator-mxnet/pull/13647#issuecomment-451068779 > When you did run 'preprocess_data.py', it actually did nothing because of bug. > I changed the paths with bug. it takes about 6 hours

[GitHub] aaronmarkham commented on issue #13647: Update lip reading example

aaronmarkham commented on issue #13647: Update lip reading example URL: https://github.com/apache/incubator-mxnet/pull/13647#issuecomment-450309197 I built the project on a GPU instance this time and was able to run main.py. However, I immediately get a dump of a lot of these errors:

[GitHub] aaronmarkham commented on issue #13647: Update lip reading example

aaronmarkham commented on issue #13647: Update lip reading example URL: https://github.com/apache/incubator-mxnet/pull/13647#issuecomment-450302412 Please add these to your prerequisites list: * scikit-image * scikit-video * dlib Also, I tried to run it without a GPU and