First weapon of choice when trying to convert document formats would be pandoc. Have a look at it, it can handle quite a lot of formats. Other than that, there are guides like this [1], but it looks to me that you should just stick to LaTeX, as it's the best solution for typesetting, especially equations. In theory, at least ePub does support equations, but it's questionable if they even come close to the capabilities of LaTeX equations. So it's not that unlikely that you'd just do what [1] explains in a far too complicated way (seriously, have LaTeX equations be run through a boat load of programs just to get a png? If ePub/mobi don't support svg, just create the PDF from LaTeX and convert e.g. on GUI with Inkscape or on CLI with ImageMagick).
But for a book, I kinda doubt any of these solutions would be feasible. The chance of lots of issues is just too big. So if you must create a ePub file, better transfer the text manually into a program designed for this job and insert the equations as images. Will be a lot of work, but chances are it won't be more work than jumping through a bunch of hoops and ending up having to redo all formatting either way afterwards. Best Richard [1]: https://minireference.com/blog/generating-epub-from-latex/