Actually many companies have already take the intelligent anomaly detection algorithms into production use, such as: datadog, Azure, Tencent cloud, Ali Cloud,etc. For this" Every anomaly detection project seems to be abandoned after 6 months", it may be that AD project needs to be integrated with practical monitor architecture and the algorithms need to be continuously iterated. It's difficult for personally maintained projects to be continuously updated.
Best, Dong Shandong On Saturday, 26 June 2021 at 9:03:35 pm UTC+8 sup...@gmail.com wrote: > Is anyone using this kind of thing in production? I've seen lots of demos > and promises over the years. Every anomaly detection project seems to be > abandoned after 6 months. > > I'd like to avoid adding functionality that is "cool" but has no practical > application. > > For example: https://github.com/etsy/skyline > > On Fri, Jun 25, 2021 at 9:16 PM Levi Harrison <levisamue...@gmail.com> > wrote: > >> We could even use something like Yaegi <https://github.com/traefik/yaegi>, >> an interpreter, which would be slower than using compiled Go code, but >> makes the creation process a lot easier and more stable. This is even >> closer to having our own scripting language, but still is distinguished by >> the ability to simply drop this in with little extra work, and the >> adoption of Go as a language (Yaegi completely supports the Go >> specification). >> >> On Fri, Jun 25, 2021 at 7:02 AM Levi Harrison <levisamue...@gmail.com> >> wrote: >> >>> Yeah, no Windows support is less than desirable. Also after more >>> research, the library seems to be really fraught with issues. Hashicorp has >>> its >>> plugin library <https://github.com/hashicorp/go-plugin>, but I don't >>> think we can afford to take the performance hit of going over RPC. I did >>> find this library <https://github.com/pkujhd/goloader> though, which I >>> believe supports Windows and looks really promising. We would have to >>> subject it to further testing though because I don't think it's broadly >>> adopted. >>> >>> It is the case that a scripting engine as Björn suggested would probably >>> the cleanest way to allows users to define their own functions, but that >>> approach brings a lot more complexity so I think this is a good solution >>> for now, also having the added benefit of language that's already widely >>> known. >>> >>> On Fri, Jun 25, 2021 at 5:30 AM Julien Pivotto <roidel...@prometheus.io> >>> wrote: >>> >>>> On 24 Jun 14:53, Levi Harrison wrote: >>>> > Maybe we could utilize Golang's plugin library < >>>> https://pkg.go.dev/plugin>to >>>> > provide a way for users to write their own functions in compliance >>>> with the defined >>>> > format >>>> > < >>>> https://github.com/prometheus/prometheus/blob/7cb55d57328c60e4a69e741c4953b97e41bf0be3/promql/functions.go#L46> >>>> >>>> >>>> > and then dynamically load them into PromQL. >>>> >>>> That approach would ban windows users. I am not sure if that is an >>>> acceptable trade-off. >>>> >>>> > >>>> > On Thursday, June 24, 2021 at 5:14:51 PM UTC-4 bjo...@rabenste.in >>>> wrote: >>>> > >>>> > > On 24.06.21 00:05, 董善东 wrote: >>>> > > > hi,all >>>> > > > In the existing prometheus version, the anomaly detection still >>>> relies >>>> > > > fully on the rules setting. We find that it is inconvenient to >>>> set and >>>> > > hard >>>> > > > to maintain in practical use. >>>> > > > So I propose to add some statistical analysis functions to >>>> provide >>>> > > better >>>> > > > and stronger AD ability. >>>> > > >>>> > > Yeah, that's a frequent request. Unfortunately, there are so many >>>> > > statistical analysis functions that we can hardly just add them all. >>>> > > >>>> > > So far, the usual recommendation is to extract data from Prometheus >>>> > > via the HTTP API and feed it to a fully-fledged statistics tool. >>>> > > >>>> > > Obviously, that doesn't help you with alerts (which you probably >>>> want >>>> > > to keep within Prometheus). >>>> > > >>>> > > At the previous to last dev-summit (2021-05-27), we discussed the >>>> use >>>> > > case. >>>> > > >>>> > > Outcome was the following: >>>> > > * We want to explore supporting analytics use cases within PromQL >>>> behind >>>> > > a feature flag >>>> > > * We are open to wrapping other languages, e.g. R, Fortran, >>>> SciPython, >>>> > > given an accepted design doc >>>> > > >>>> > > See alse notes here: >>>> > > >>>> > > >>>> https://docs.google.com/document/d/11LC3wJcVk00l8w5P3oLQ-m3Y37iom6INAMEu2ZAGIIE/edit?ts=6036b8e0&pli=1#heading=h.sa2f6aem9wdt >>>> > > >>>> > > So I guess you could just implement the functions you like and put >>>> > > them into a PR, locked behind a feature flag. >>>> > > >>>> > > Personally, I'm still not sure if that's a sustainable >>>> > > approach. Perhaps integrating some scripting engine to allow >>>> > > user-defined functions might be better. But we'll see… >>>> > > >>>> > > -- >>>> > > Björn Rabenstein >>>> > > [PGP-ID] 0x851C3DA17D748D03 >>>> > > [email] bjo...@rabenste.in >>>> > > >>>> > >>>> > -- >>>> > You received this message because you are subscribed to the Google >>>> Groups "Prometheus Developers" group. >>>> > To unsubscribe from this group and stop receiving emails from it, >>>> send an email to prometheus-devel...@googlegroups.com. >>>> > To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/prometheus-developers/8fb5d7fc-30f8-4a68-aa27-05fb66b4c2e7n%40googlegroups.com >>>> . >>>> >>>> >>>> -- >>>> Julien Pivotto >>>> @roidelapluie >>>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "Prometheus Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to prometheus-devel...@googlegroups.com. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/prometheus-developers/CAJrMkW7XzsDdbz8yubnO1nZnOk_xxGrewKvWz4GZ10%2B7L5pwaA%40mail.gmail.com >> >> <https://groups.google.com/d/msgid/prometheus-developers/CAJrMkW7XzsDdbz8yubnO1nZnOk_xxGrewKvWz4GZ10%2B7L5pwaA%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/b69e13c9-5af0-4f28-a1ef-f99ab9698076n%40googlegroups.com.
