On Fri, Jun 12, 2015 at 9:07 AM, Michael Trinkala <[email protected]> wrote:
> I wouldn't blame that on Lua. The (heavily used) standard printf %g > specifier generates output like that and Influx has chosen not to work with > it. I would push back on Influx, if a row is typed as float is should > still be able to accept an integer. If they are inferring type and you > send a 3 first (defaulting the column to int) and then send a 2.7 I assume > sadness will also ensue. > > yes, the first type wins. but if we could check the type, we could be more specific with the printf specifier. many have already pushed hard on influx and lost. they just released 0.9 yesterday after months of delay, so i dont expect to make much headway on that argument. > BTW: the JIT/different JSON lib won't help you here unless you modify it > to not use the %g specifier when outputting doubles (also the JIT is > currently unsupported in the lua_sandbox (the module system has to be > patched)). > doh! ugh, i was uncertaing about luajit in the sandbox, but recalled seeing a luajit build option many moons ago and so thought this was a check in the plus column of going our own route. > > FYI: We do track type information in the protobuf so you can distinguish > an int from a double in the sandbox but obviously you would need a custom > outputter. > > > Trink > hmm, i'm totally stuck. this seems impossible to do in lua. having go parse json is going backwards since that's what they ran into on their side as a deal-breaker of a performance bottleneck. > > On Thu, Jun 11, 2015 at 3:21 PM, David Birdsong <[email protected]> > wrote: > >> >> >> On Thu, Jun 11, 2015 at 3:14 PM Mat Schaffer <[email protected]> wrote: >> >>> Can you maybe string.format the value to always have a fixed number of >>> decimal places when encoding for influx? >>> >> >> unfortunately not. both integers and floats are flowing through this >> encoder. >> >> i was hoping to open up this encoder plus the steps to set it up in case >> others might benefit, but we're probably going to have to lean on luajit >> and a different json module to make this work. the libraries are all >> internal, not necessarily secret, but for sure a barrier to sharing w/ >> others since build steps would be required and another repo of ours would >> have to be pilfered and made ready for public consumption. >> >> >>> >>> 2015年6月12日金曜日、David Birdsong<[email protected]>さんは書きました: >>> >>>> I set out to convert a stream of json objects that are plumbed to my >>>> shiny new influxdb 9.0 cluster and reached for my favorite tool: heka. >>>> >>>> Influxdb 9.0 RC32 and later want a new line protocol specific to >>>> Influx. I as thinking that I would keep the original json format around >>>> until the last hop before writing to influx as a useful data exposition >>>> format that other tools could read ie. heka and friends of heka. >>>> >>>> Given that the new line protocol needs string representations of >>>> numbers to infer types, lua is really screwing me here: >>>> >>>> > print(cjson.encode(6.0)) >>>> >>>> 6 >>>> >>>> >>>> Influx won't take a row that was originally typed as a float if it sees >>>> an integer next. Any suggestions? >>>> >>>> >>> >>> -- >>> >>> -Mat >>> >>> about.me/matschaffer >>> >>> >> _______________________________________________ >> Heka mailing list >> [email protected] >> https://mail.mozilla.org/listinfo/heka >> >> >
_______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
