If you don't have access to the ChatGPT beta, someone made a Streamlit app that does the same thing. You can give it a math problem and it uses ChatGPT to generate and run SymPy code to solve it https://mathgpt.streamlit.app/
Aaron Meurer On Thu, Mar 23, 2023 at 3:28 PM Aaron Meurer <asmeu...@gmail.com> wrote: > Actually I didn't notice it but there's literally an example of ChatGPT > generating SymPy code and executing on > https://openai.com/blog/chatgpt-plugins (scroll down to where it says > "code interpreter"). It's their main example of calling out to Python. > > Aaron Meurer > > On Thu, Mar 23, 2023 at 2:42 PM Aaron Meurer <asmeu...@gmail.com> wrote: > >> >> >> On Thu, Mar 23, 2023 at 12:24 PM S.Y. Lee <sylee...@gmail.com> wrote: >> >>> Wolfram had recently announced the collaboration of chatGPT and wolfram >>> alpha >>> ChatGPT Gets Its “Wolfram Superpowers”!—Stephen Wolfram Writings >>> <https://writings.stephenwolfram.com/2023/03/chatgpt-gets-its-wolfram-superpowers/> >> >> >> I wouldn't call this a "collaboration". OpenAI is adding a plugin system >> to ChatGPT and Wolfram is one of the first plugins >> https://openai.com/blog/chatgpt-plugins. >> >> If you scroll down on that page, there is also a tool that lets it >> execute Python code. I don't know if it has access to SymPy, but it likely >> does, since it seems to have access to other popular libraries like pandas >> and matplotlib. If anyone has access to ChatGPT Plus, could you check? >> >> >>> >>> They start to use chatgpt to generate the Wolfram code. >>> And it is likely to help the issues with correctness about math or >>> science facts >>> because once it translates to the wolfram functions, and the wolfram >>> function runs without error, the answer is correct. >>> >>> However, the argument I'd give is that making a combination of code with >>> *simplify, solve, integrate *are not still informative because even >>> though they do things logically correct, >>> they have problem that they can't inform people how to solve the >>> problems in details. >>> >>> So I'm thinking about an idea whether language models should really >>> generate a code that assembles the *rules* used in simplify, solve. >>> And then it can be more human readable (or can get more human readable >>> logic from it) >>> if it is like compose(solve_trig, simplify_cos, ...) >>> (in some pseudo sympy code) >>> >> >> A language model can do pretty much anything, so long as it's seen enough >> examples of it before. You can often just give it examples of what you >> want in the prompt and it will figure it out. >> >> ChatGPT (and other LLMs) are basically just huge pattern matching >> machines. That's all they are, which is why they break down whenever they >> have to do actual logic or reason about something they haven't seen before. >> So I suspect they could be used quite successfully for symbolic pattern >> matching, especially when that pattern matching is "fuzzy", like trying to >> find the best heuristic function to apply to an expression. >> >> Aaron Meurer >> >> >>> On Saturday, December 17, 2022 at 5:47:39 PM UTC+9 smi...@gmail.com >>> wrote: >>> >>>> In reviewing a PR related to units, I found ChatGPT to get correct the >>>> idea that a foot is bigger than an inch, but it said that a volt is bigger >>>> than a statvolt (see quoted GPT response [here]( >>>> https://github.com/sympy/sympy/pull/24325#issuecomment-1354343306)). >>>> >>>> /c >>>> >>>> On Thursday, December 15, 2022 at 1:58:33 PM UTC-6 Aaron Meurer wrote: >>>> >>>>> The trend with LLMs is much less structured. It doesn't use any >>>>> formalism. It just guesses the next character of the input based on >>>>> training on billions of examples. >>>>> >>>>> That's why I think that tools like SymPy that are more structured can >>>>> be useful. GPT can already write SymPy code pretty well, much better than >>>>> it can do the actual mathematics. It may be as simple as automatically >>>>> appending "and write SymPy code to verify this" to the end of a prompt >>>>> whenever it involves mathematics. This sort of approach has already been >>>>> proven to be able to solve university math problems (see >>>>> https://www.pnas.org/doi/pdf/10.1073/pnas.2123433119, where they >>>>> literally just take the input problem and prepend "use sympy" and the >>>>> neural network model does the rest). >>>>> >>>>> Aaron Meurer >>>>> >>>>> >>>>> >>>>> On Thu, Dec 15, 2022 at 2:21 AM S.Y. Lee <syle...@gmail.com> wrote: >>>>> >>>>>> > My hope is that tools like SymPy can be used as oracles for tools >>>>>> like GPT to help them verify their mathematics. >>>>>> >>>>>> In the most general context, "correct mathematics" can also be >>>>>> considered some "grammar". >>>>>> So there should be some grammar between Type-0 grammar to Type-1 >>>>>> grammar in Chomsky hierarchy >>>>>> <https://en.wikipedia.org/wiki/Chomsky_hierarchy>. >>>>>> In this context, a parser, or a parser with sympy oracle is the >>>>>> solution for such problem, >>>>>> such that any other ideas to solve such problem can be isomorphic to. >>>>>> >>>>>> However, building up such parser is off-direction for the researches >>>>>> of deep learning itself, >>>>>> because it would need a lot of efforts by experts, to interpret the >>>>>> sentence generated by GPT, >>>>>> and design a phrase structure grammar for it. >>>>>> >>>>>> I also thought about an idea that they can just tag arithmetics using SKI >>>>>> combinator calculus >>>>>> <https://en.wikipedia.org/wiki/SKI_combinator_calculus>. >>>>>> In this way, there is no wrong arithmetics for every random sequence >>>>>> of alphabets. >>>>>> >>>>>> However, I'm not sure that if this idea is already refuted by such >>>>>> contemporary researchers >>>>>> because it should be pretty much well-known. >>>>>> >>>>>> On Thursday, December 15, 2022 at 12:45:53 AM UTC+2 asme...@gmail.com >>>>>> wrote: >>>>>> >>>>>>> I've seen many examples on Twitter and elsewhere of chatgpt doing >>>>>>> terribly on actual mathematics. I instead asked it to "Find the extreme >>>>>>> values of the function x^3 + 5x^2 + 3x - 9". It got the first part with >>>>>>> the >>>>>>> derivative and the quadratic equation correct, but then did faulty >>>>>>> arithmetic when plugging the values back into the expression (the >>>>>>> correct >>>>>>> values are 0 and -256/27). It makes a sign error and also gets the >>>>>>> addition >>>>>>> wrong even with the wrong sign. >>>>>>> >>>>>>> My hope is that tools like SymPy can be used as oracles for tools >>>>>>> like GPT to help them verify their mathematics. Obviously in this case >>>>>>> you >>>>>>> don't need SymPy to just verify arithmetic, but I've also seen it do >>>>>>> just >>>>>>> as poorly on symbolic algebra problems. >>>>>>> >>>>>>> I've also pasted the full log below: >>>>>>> >>>>>>> [image: Screen Shot 2022-12-14 at 3.41.32 PM.png] >>>>>>> [image: Screen Shot 2022-12-14 at 3.41.42 PM.png] >>>>>>> >>>>>>> To find the extreme values of a function, we need to find the >>>>>>> critical points of the function, which are the points at which the >>>>>>> derivative of the function is equal to 0 or does not exist. >>>>>>> >>>>>>> The derivative of the function x^3 + 5x^2 + 3x - 9 is 3x^2 + 10x + >>>>>>> 3. To find the critical points, we need to solve the equation 3x^2 + >>>>>>> 10x + >>>>>>> 3 = 0. >>>>>>> >>>>>>> To solve this equation, we can use the quadratic formula: >>>>>>> >>>>>>> x = (-b ± √(b^2 - 4ac)) / 2a >>>>>>> >>>>>>> where a, b, and c are the coefficients of the quadratic equation >>>>>>> ax^2 + bx + c = 0. In our case, a = 3, b = 10, and c = 3, so the formula >>>>>>> becomes: >>>>>>> >>>>>>> x = (-10 ± √(10^2 - 4 * 3 * 3)) / 2 * 3 >>>>>>> >>>>>>> = (-10 ± √(100 - 36)) / 6 >>>>>>> >>>>>>> = (-10 ± √(64)) / 6 >>>>>>> >>>>>>> = (-10 ± 8) / 6 >>>>>>> >>>>>>> = (-18, -2) / 6 >>>>>>> >>>>>>> = (-3, -1/3) >>>>>>> >>>>>>> These are the critical points of the function x^3 + 5x^2 + 3x - 9. >>>>>>> >>>>>>> To find the extreme values of the function, we need to evaluate the >>>>>>> function at these critical points and at any points where the derivative >>>>>>> does not exist. In this case, the derivative exists for all values of >>>>>>> x, so >>>>>>> we only need to evaluate the function at the critical points. >>>>>>> >>>>>>> If we plug in the critical points into the function, we get: >>>>>>> >>>>>>> x^3 + 5x^2 + 3x - 9 = (-3)^3 + 5(-3)^2 + 3(-3) - 9 = -27 + 45 + 9 - >>>>>>> 9 = 8 >>>>>>> >>>>>>> and >>>>>>> >>>>>>> x^3 + 5x^2 + 3x - 9 = (-1/3)^3 + 5(-1/3)^2 + 3(-1/3) - 9 = -1/27 + >>>>>>> 5/9 + 1 - 9 = 5/9 - 1/27 - 8/9 = -64/27 >>>>>>> >>>>>>> The extreme values of the function x^3 + 5x^2 + 3x - 9 are 8 and >>>>>>> -64/27. >>>>>>> >>>>>>> On Wed, Dec 14, 2022 at 3:35 PM gu...@uwosh.edu <gu...@uwosh.edu> >>>>>>> wrote: >>>>>>> >>>>>>>> Just for some additional perspective. I have also tried this on >>>>>>>> some general chemistry word problems. In general, I see it getting >>>>>>>> basic >>>>>>>> one logical step processes correct (e.g. a single step dilution or >>>>>>>> grams -> >>>>>>>> moles). Things with multiple steps or requiring understanding the >>>>>>>> physical >>>>>>>> situation it does poorly on. That said, I think it does better than >>>>>>>> some of >>>>>>>> my weakest students. It does not seem to be able to use significant >>>>>>>> figures >>>>>>>> in computations (also a problem for my weaker students). >>>>>>>> >>>>>>>> It seems to be improving rapidly. If it can get to reliably >>>>>>>> differentiating between correct (workable) solutions and erroneous >>>>>>>> ones, it >>>>>>>> will be more useful to most people (including my students) than >>>>>>>> searches of >>>>>>>> the internet or a cheating sight such as Chegg. >>>>>>>> >>>>>>>> My two cents worth of opinion. >>>>>>>> >>>>>>>> Jonathan >>>>>>>> >>>>>>>> On Wednesday, December 14, 2022 at 4:28:05 PM UTC-6 Francesco >>>>>>>> Bonazzi wrote: >>>>>>>> >>>>>>>>> [image: chatgpt.sympy.matrix_diag.png] >>>>>>>>> >>>>>>>>> On Wednesday, December 14, 2022 at 11:26:37 p.m. UTC+1 Francesco >>>>>>>>> Bonazzi wrote: >>>>>>>>> >>>>>>>>>> Not everything is perfect... ChatGPT misses the *convert_to( ... >>>>>>>>>> ) *function in *sympy.physics.units*, furthermore, the given >>>>>>>>>> code does not work: >>>>>>>>>> >>>>>>>>>> [image: chatgpt.sympy.unit_conv.png] >>>>>>>>>> >>>>>>>>>> On Wednesday, December 14, 2022 at 11:24:29 p.m. UTC+1 Francesco >>>>>>>>>> Bonazzi wrote: >>>>>>>>>> >>>>>>>>>>> [image: chatgpt.sympy.logical_inference.png] >>>>>>>>>>> >>>>>>>>>>> On Wednesday, December 14, 2022 at 11:23:43 p.m. UTC+1 Francesco >>>>>>>>>>> Bonazzi wrote: >>>>>>>>>>> >>>>>>>>>>>> https://en.wikipedia.org/wiki/ChatGPT >>>>>>>>>>>> >>>>>>>>>>>> Some tested examples attached as pictures to this post. Quite >>>>>>>>>>>> impressive... >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "sympy" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to sympy+un...@googlegroups.com. >>>>>>>> >>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/sympy/6af62b19-1fb0-4681-9fd2-5e5fccfcb46fn%40googlegroups.com >>>>>>>> <https://groups.google.com/d/msgid/sympy/6af62b19-1fb0-4681-9fd2-5e5fccfcb46fn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "sympy" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to sympy+un...@googlegroups.com. >>>>>> >>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/sympy/74847ca3-124b-414d-aa36-01eb91096310n%40googlegroups.com >>>>>> <https://groups.google.com/d/msgid/sympy/74847ca3-124b-414d-aa36-01eb91096310n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "sympy" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to sympy+unsubscr...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/sympy/1fbd88f8-3513-4e02-a576-266352f3531fn%40googlegroups.com >>> <https://groups.google.com/d/msgid/sympy/1fbd88f8-3513-4e02-a576-266352f3531fn%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- You received this message because you are subscribed to the Google Groups "sympy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sympy+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAKgW%3D6L3qy-h94G8OrTqtjrqTu9woZQpXZxaZx7mjcD1L7q32g%40mail.gmail.com.
