I always thought an interesting approach would be a handwriting -> LaTeX translator, and then a LaTeX -> SymPy translator (although I'm not sure if it would be LaTeX code, or an actual LaTeX document). Seemed like it would allow for a little more flexibility and would be made of smaller, more tractable subproblems.
-Gilbert On Wed, Mar 12, 2014 at 1:37 PM, Fedor Morozov <fmoroz...@gmail.com> wrote: > Hello, Matthew. > The problem is sure feasible, you can test a great implementation here: > http://webdemo.visionobjects.com > Unfortunately I haven't found any opensource formula recognition tools, so > maybe there is no code to adapt. > The are lots of articles on the topic even from the 80s. For example, > there is a recent survey > article<http://link.springer.com/article/10.1007/s10032-011-0174-4>about math > expressions retrieval. > The three main steps are as follows: symbol shape classification, formula > layout estimation and syntax analysis. > Most of the approaches try to parse printed formulas, in our case the task > is rather different - the formula is already detected and binarized, but > the shapes may significantly differ. > Robust shape estimation looks like the most challenging part, though it's > the one best known to me. > For the syntax analysis it could be useful to adapt some of sympy parsing > functionality. > Also if the project proves to be too difficult for one summer, even single > character recognition with smart interface can provide good usability. > > среда, 12 марта 2014 г., 23:03:39 UTC+4 пользователь Matthew написал: >> >> Handwritten input sounds neat. I use and find great value in >> http://detexify.kirelabs.org/classify.html >> >> Handwritten mathematical expression recognition is general and useful >> enough that I'd like to see it done outside of SymPy and then imported, >> this way other projects can also use it. >> >> Fedor, can you provide some background on this topic? What other >> attempts have there been? Are there other systems? What resources have >> you found? Is this even a feasible problem? What are the hard parts? >> >> -Matthew >> >> >> On Wed, Mar 12, 2014 at 11:46 AM, Fedor Morozov <fmor...@gmail.com>wrote: >> >>> Hello, David. >>> Unfortunately I have no experience with JavaScript yet, but since the >>> main challenge of the project is the backend part (formula recognition), I >>> can get enough JavaScript knowledge before the coding phase starts. >>> I also think that it would be great to develop Sympy Gamma mobile app >>> for the project and I have some skills for this (Java and Android >>> programming). >>> Now that the idea is confirmed to be useful I'll take a look into the >>> issues list, thank you. >>> >>> среда, 12 марта 2014 г., 20:38:33 UTC+4 пользователь David Li написал: >>> >>>> Hello, >>>> >>>> This sounds like a great idea for Gamma. How familiar are you with web >>>> development? In particular, do you have any experience with JavaScript? >>>> >>>> I would also encourage you to look at the application >>>> template<https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fsympy%2Fsympy%2Fwiki%2FGSoC-2014-Application-Template&sa=D&sntz=1&usg=AFQjCNHo2vVXxypTvp0G5lRmErpTdAsBRQ> >>>> and >>>> also perhaps contribute to SymPy if you haven't already (as it is a >>>> requirement). >>>> >>>> David >>>> >>>> On Tuesday, March 11, 2014 5:48:21 PM UTC-7, Fedor Morozov wrote: >>>>> >>>>> Hello. >>>>> My name is Fedor Morozov, I'm a third year CS student at Moscow State >>>>> University specializing in computer vision. >>>>> >>>>> Since touchscreen devices are highly popular nowadays, there are lots >>>>> of cases when it's easier to handwrite a formula then to enter it using a >>>>> keyboard. >>>>> State-of-the arts computer vision approaches achieve good results in >>>>> formula recognition, so it is possible to implement such a feature for >>>>> SymPy Gamma. >>>>> Along with NLP parsing this would provide a very powerful user input >>>>> capabilities. >>>>> There are already some similar 3rd-party applications for Wolfram >>>>> Alpha, you can see an example here <http://webdemo.visionobjects.com>. >>>>> A more challenging task would be formula recognition from photos, but >>>>> that's not necessary right now, I guess. >>>>> >>>>> So the proposed project is to implement formula recognition >>>>> capabilities for SymPy Gamma, it would also be great to make some >>>>> interface >>>>> for touchscreen input (web/mobile app?). >>>>> I've recently read a few articles on the topic, if you believe that >>>>> the project if useful, I'll come up with a full-fledged proposal and >>>>> algorithms review. >>>>> The easiest (conceptually) approach would be to select some of the >>>>> recent articles and code up the algorithm. >>>>> Than some improvements can be made, for example it would be cool to >>>>> use SymPy resources for formula correctness checking. >>>>> >>>>> As stated above, I specialize in computer vision, so I'm familiar with >>>>> basic algorithms in the area. >>>>> Now I'm researching feature descriptors, so I have some experience >>>>> with shape recognition (for example, shape context), a year ago I worked >>>>> on >>>>> a car plate recognition task. >>>>> Also last year I participated in GSoC with computational photography >>>>> project for opencv library. >>>>> >>>>> Thanks for your attention, hope to hear some comments soon. >>>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "sympy" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to sympy+un...@googlegroups.com. >>> To post to this group, send email to sy...@googlegroups.com. >>> >>> Visit this group at http://groups.google.com/group/sympy. >>> To view this discussion on the web visit https://groups.google.com/d/ >>> msgid/sympy/307c7fbd-50fd-4358-a767-5d6460bb5193%40googlegroups.com<https://groups.google.com/d/msgid/sympy/307c7fbd-50fd-4358-a767-5d6460bb5193%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > You received this message because you are subscribed to the Google Groups > "sympy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sympy+unsubscr...@googlegroups.com. > To post to this group, send email to sympy@googlegroups.com. > Visit this group at http://groups.google.com/group/sympy. > To view this discussion on the web visit > https://groups.google.com/d/msgid/sympy/371a8501-c8ac-477d-9dd8-e1e4f1a957b9%40googlegroups.com<https://groups.google.com/d/msgid/sympy/371a8501-c8ac-477d-9dd8-e1e4f1a957b9%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "sympy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sympy+unsubscr...@googlegroups.com. To post to this group, send email to sympy@googlegroups.com. Visit this group at http://groups.google.com/group/sympy. To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAGZqWdo5_q8VAU3xYcrtu%2BDT%2B66QKDJ3Fm38OiUh7FtUqd6H0Q%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
