Thanks Kostas. So if we're comfortable treating timestamps as longs (and doing conversions to human readable time at our application level), we can use CEP, ML lib etc. in addition to all basic Flink functions? That's great news?
To Matthias's point, *why then does the following not read "**Both timestamps and watermarks are specified as longs"?* Before I go headlong into developing on Flink, I just want to be sure I'm covered here. Again, thanks. (The Youtube videos from FlinkForward are also great, btw. Incredibly impressed with Data Artisans.) https://ci.apache.org/projects/flink/flink-docs-release-1.1/apis/streaming/ event_timestamps_watermarks.html#assigning-timestamps "Both timestamps and watermarks are specified as milliseconds since the Java epoch of 1970-01-01T00:00:00Z." On Mon, Dec 5, 2016 at 4:57 AM, Kostas Kloudas <k.klou...@data-artisans.com> wrote: > Hi Jeff, > > Actually in Flink timestamps are simple longs. > This means that you can assign anything you want as a timestamp, as long > as it fits in a long. > > Hope this helps and if not, we can discuss to see if we can find a > solution that > fits your needs together. > > Cheers, > Kostas > > On Dec 4, 2016, at 11:39 PM, jeff jacobson <jeff.harold.jacob...@gmail.com> > wrote: > > Wow. Really? Is there a way to do micros? A hack? A Jira story? Most > (all?) U.S. equity and European futures, options, and stock markets > timestamp in microseconds. This makes Flink unusable for a massive industry > vertical. To the extent lower-frequency time-series data is being used > (e.g. end of data prices), stream processing is kind of overkill. Love > everything I've read about Flink...there's got to be a way to make this > work, no? > > On Sun, Dec 4, 2016 at 5:27 PM, Matthias J. Sax <mj...@apache.org> wrote: > >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA512 >> >> Oh. My bad... Did not read your question carefully enough. >> >> Than the answer is no, it does not support microseconds (only >> milliseconds). >> >> - -Matthias >> >> >> On 12/4/16 2:22 PM, jeff jacobson wrote: >> > Sorry if I'm missing something. That link mentions milliseconds, >> > no? My question is whether or not I can specify microseconds where >> > 1000microseconds = 1millisecond. Thanks! >> > >> > On Sun, Dec 4, 2016 at 5:05 PM, Matthias J. Sax <mj...@apache.org >> > <mailto:mj...@apache.org>> wrote: >> > >> > Yes. It does. >> > >> > See: >> > https://ci.apache.org/projects/flink/flink-docs-release-1.1/apis/strea >> ming/event_timestamps_watermarks.html#assigning-timestamps >> <https://ci.apache.org/projects/flink/flink-docs-release-1.1/apis/streaming/event_timestamps_watermarks.html#assigning-timestamps> >> > >> > >> <https://ci.apache.org/projects/flink/flink-docs-release-1.1/apis/stream >> ing/event_timestamps_watermarks.html#assigning-timestamps >> <https://ci.apache.org/projects/flink/flink-docs-release-1.1/apis/streaming/event_timestamps_watermarks.html#assigning-timestamps> >> > >> > >> > "Both timestamps and watermarks are specified as millliseconds >> > since the Java epoch of 1970-01-01T00:00:00Z." >> > >> > >> > >> > -Matthias >> > >> > >> > On 12/04/2016 10:57 AM, jeff jacobson wrote: >> >> I've sourced stackoverflow, the docs, and the web but I can't >> >> figure out: does flink support microsecond timestamp resolution? >> >> Thanks! >> > >> > >> -----BEGIN PGP SIGNATURE----- >> Comment: GPGTools - https://gpgtools.org >> >> iQIYBAEBCgAGBQJYRJhUAAoJELz8Z8hxAGOiNKoP32ChGeNd7N8Zco2q6lsu+Hxd >> JZq62ey3wTrIUS+3oRlILwnu81cViQHtMMVBly3+YnqB85gNiaEUxEQTQCdKPl8G >> AqxoFIkMcrKGzwGXigKnCAoVIiyuPeNuhY1d1yv4rWrkt7qb0lCC02Xoq1C0hoS6 >> Stwk62GXmNRXPYpyjnSq/iAIMbjWaU+ZU0t4V3J8loroNuJ5QcUsJLfRXeo3/5ho >> f42L+IANyB5K7vnTxNZYyf5ShNVbTY9/iFaviluxrCNztqGTo7CxMpcyWyMS3wcF >> ycXcq/daB+guEJpW0sm4JtMPSsQ/kN99c/ig3t0HX1kDV7xrDDSF2qPvbYOWF38n >> omTr7RY3YRFi5LOKvBGa96Aw5UYjMddjcqozWId6xgdXfvz6RUeJCWa9RW8I6ptg >> 8TaJpM2WgDJMgMuzdl8dDv65l78DkLlNlNo53O66b/9Pt78P75KNjj8naD5kkj4C >> i9amwnUNNEnZucA2/1vhzr6cVSzrzBLL7juVj0VmABZo4itUZjjR0UkN7MB+ioWU >> trNhaXgE6EP/160n6D0/NUu02prm3jq8mK6gu9lZFWGbAeCUcch+CbvWSaiXAw3H >> BOieCsgZD1wfXQJ3wEmnqj/YP94uDlx1IjynskDevjk6OIyIysbBSIqgsUK6fvQ8 >> ztXO6ls7ARMOBmA= >> =/O+Q >> -----END PGP SIGNATURE----- >> > > >
