Just a side note: it hurts to hear that OPC UA sucks badly performance-wise. I have done tests with Milo OPC UA Client and SCADA OPC UA servers, and they perform pretty fine. 200 MQTT clients writing values via a Gateway with OPC UA Milo (Client) to two OPC UA servers, each OPC UA server handles 50000 values changes per second: https://www.linkedin.com/posts/andreas-vogler-361115122_graphql-mqtt-vertx-activity-6769029730797719552-ZkjG <https://www.linkedin.com/posts/andreas-vogler-361115122_graphql-mqtt-vertx-activity-6769029730797719552-ZkjG>
But I fully agree with MQTT - it’s simple - and I think that’s the reason why it is so popular… > On 10.03.2021, at 14:34, Christofer Dutz <[email protected]> wrote: > > Yes, > > OPC-UA has all of this ... and it's probably one of the reasons so many > implementations differ, and you have to again interpret results. It's also > probably a reason why OPC-UA sucks that badly performance-wise. > > Just saying that we should be careful what we choose as reference. In > contrast typical MQTT doesn't have all of this and it seems to be gaining > more and more traction, just because it doesn't have all of this overhead. > > Just my thoughts on this. > > Chris > > > > > -----Ursprüngliche Nachricht----- > Von: Łukasz Dywicki <[email protected]> > Gesendet: Mittwoch, 10. März 2021 12:58 > An: [email protected] > Betreff: Re: AW: Source Time of PLC Value? > > Matthias, > I do believe there are several points which are relevant and still not > possible to be implemented using available PLC4X API. Metadata might not be > most fortunate way, but it can enable or ease analytical and troubleshooting > scenarios. > > Two major points I see: > - Verification of device condition - ie. how does caller know response > latency? This is especially relevant with frequent polling. Currently caller > has no measures to know that and can not measure that in reliable way due to > async nature of API. > There is a way to implement backpressure technics with callback chains (we > can automatically delay execute call), but first we need to bring measures > for that. > - Caching of values - when we do cyclic subscriptions or emulation of these > through passive connections. There is no way to make caller code aware of > nature of data received. Unless he code everything on his end. > - Per operation timeout settings/automatic retry are another functionalities > which are relevant in case of querying and writing. We do not want every > driver to implement it, so making a metadata and/or decorator would unify > this layer between drivers. > > I speak of above from my own perspective cause I did implement an > experimental decorator API for PLC4X [1] to cope with CANopen arbitration > issues I found in devices we interface with. It allows to catch read/write > failures [2] and re-attempt entire operation. It is proven to work [3] > independently of driver. Sadly it has no measures to signal its execution > back to caller. In the end, traceability is pretty tough. > > Obviously OPC UA has everything sorted out, but device specific protocols > often do not. Even a limited set of metadata can give us a way to start > bridging the gap and making Ben's Milo/OPC UA server much closer to a real > thing. > > Best, > Łukasz > > [1] > https://github.com/ConnectorIO/connectorio-addons/tree/master/bundles/org.connectorio.plc4x.decorator/src/main/java/org/connectorio/plc4x/decorator > [2] > https://github.com/ConnectorIO/connectorio-addons/blob/master/bundles/org.connectorio.plc4x.decorator.retry/src/main/java/org/connectorio/plc4x/decorator/retry/ReadRetry.java > [3] > https://github.com/ConnectorIO/connectorio-addons/blob/master/bundles/org.connectorio.plc4x.decorator.retry/src/test/java/org/connectorio/plc4x/decorator/retry/RetryDecoratorReadTest.java > > > On 08.03.2021 23:15, Matthias Milan Strljic wrote: >> Hi all, >> >> i see there a point of how useful this meta information could be in >> some scenarios. But i would rather go the route of chris. Because I >> see there is more of a risk of an overcomplicated inhomogeneous api >> structure. I would not say that all API should have the same java >> structure on each platform but if you have for all the different >> platforms other featuresets you could also just use partially the code >> generation attempt to allow a separate meta info layer. >> >> And there would be the question if that is worth the additional feature? >> I mean I know we could add especially on the OPC UA side some meta >> information but is there more than the actual source timestamp? And >> if there is none, would it be not better to use it as the actual >> timestamp of the value? >> >> >> Einen schönen Abend ;) >> Greetings Matthias >> >> >> >> Am Mo., 8. März 2021 um 14:29 Uhr schrieb Christofer Dutz < >> [email protected]>: >> >>> Hi, >>> >>> I'm generally more concerned about users expecting us to deliver >>> feature that driver X has for driver Y too. >>> >>> I won't object, if you think it's worth doing. >>> >>> Perhaps if you could whip up an example in a feature branch? I think >>> perhaps I was still not understanding what you propose. >>> >>> Would that be ok? >>> >>> >>> Chris >>> >>> >>> -----Ursprüngliche Nachricht----- >>> Von: Lukas Ott <[email protected]> >>> Gesendet: Montag, 8. März 2021 14:20 >>> An: [email protected] >>> Betreff: Re: AW: Source Time of PLC Value? >>> >>> +1 to Lukasz >>> >>> Am Mo., 8. März 2021 um 14:09 Uhr schrieb Łukasz Dywicki < >>> [email protected]>: >>> >>>> Hold on for a second. >>>> >>>> Making options pushed over connection string is a long term recipe >>>> for disaster. Apache Camel is a prime example of what could happen >>>> if you starting with configuration with URIs and dynamic parameters. >>>> Most of components distinguish producer (writer) and consumer >>>> (receiver) options, most complicated components ended up with 40+ >>>> options settable via URI. Having a mix of PlcValues which are >>>> timestamped and >>> 'qualified' >>>> and not will complicate encoding logic (how we push above info to >>>> IEC encoder/handler or decorate it?) and how we keep custom types >>>> compatible with above? >>>> >>>> That is why I rather opt for extending API in a way which does not >>>> force existing drivers to do anything. When any of community >>>> members, regardless if its developer or user, will find it necessary >>>> to be ported in another driver, she or he have right to do it. It is >>>> not only yours duty to keep all drivers and languages the same. >>>> >>>> Speaking of which I am also not concerned about making all languages >>>> having the same functionality as languages tend to offer different >>>> options which might be hard to be ported. Lets stick to basics which >>>> are same (connection strings, field definitions, >>>> read/write/subscribe >>>> syntax) and let various runtime lead their own ways toward >>> implementation. >>>> >>>> Key point is simple - we won't be able to maintain all languages >>>> equally. We should not hold moving ie. go or java forward because we >>>> can't port new API functionality to C/Python/C#/whenever. If there >>>> is someone who *does* use above in production and *wish* to have new >>>> API parts then that person/organization duty is to do it. >>>> >>>> Best, >>>> Łukasz >>>> >>>> >>>> On 08.03.2021 13:17, Christofer Dutz wrote: >>>>> Hi, >>>>> >>>>> I must admit that I would be in favor of keeping it as simple as >>>> possible (max age in the connection string) and to implement more of >>>> the missing parts in plc4x (like the subscription simulation layer) >>>> and hereby getting the drivers we have a bit more aligned, than to >>>> implement more and more special API features, that only one or two >>>> drivers support. Cause this way we're running away form the >>>> project-promise of running a given program at any type of PLC with >>>> any >>> protocol. >>>>> >>>>> I mean ... at the current velocity the project is underway, I >>>>> wouldn't >>>> really want to add a new MetaData layer that probably in the end >>>> only >>>> 1-2 people will be implementing and which we must not only find out >>>> how to fill with life in the first place, but also port to all of >>>> our supported languages. >>>>> >>>>> If someone's willing to bring in some serious man/womenpower, I'm >>>>> fine >>>> with that ... or we add it to some wish-list for future extensions. >>>> But I wouldn't want to start something like this right now. >>>>> >>>>> Chris >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> -----Ursprüngliche Nachricht----- >>>>> Von: Łukasz Dywicki <[email protected]> >>>>> Gesendet: Montag, 8. März 2021 13:07 >>>>> An: [email protected] >>>>> Betreff: Re: Source Time of PLC Value? >>>>> >>>>> I have not mentioned metadata term in my answer, so credits for >>>>> mining >>>> it API go to you. :-) >>>>> >>>>> Now since you brought it I remember that JDBC has a "result set >>>> metadata". That might be thing we still miss. The result code is >>>> most important and available instantly, other information is >>>> supplemental and can be read additionally. >>>>> >>>>> >>>>> Best, >>>>> Łukasz >>>>> >>>>> On 08.03.2021 13:01, Ben Hutcheson wrote: >>>>>> +1 for Łukasz‘s metadata approach, it would avoid having to create >>>>>> duplicate PLcValue classes and give us a lot of flexibility in the >>>> future. >>>>>> >>>>>> On Mon, Mar 8, 2021 at 5:14 AM Christofer Dutz >>>>>> <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> sorry for being late to the party ... KNX is currently consuming >>>>>>> all my cycles. >>>>>>> Just wanted to add my thoughts to the discussion. >>>>>>> >>>>>>> Initially I thought about adding this sort of information to the >>>>>>> API, but then I thought that we have so little protocols >>>>>>> supporting this sort of concept that blowing up every PLCValue >>>>>>> with this time information just would waste CPU time and Memory. >>>>>>> >>>>>>> When it comes to simulating subscriptions by polling in the >>>>>>> background, I thought it would be a valid solution to provide a >>>>>>> default age a value can have in the cache and to make it possible >>>>>>> to override this in the connection string options. If a value is >>>>>>> too old, >>>> a return code of: >>>>>>> STALE_DATA could be returned. >>>>>>> >>>>>>> Also, we do have an option of adding an additional set of >>>>>>> PlcValue >>>> types. >>>>>>> If a driver supports this quality-of-service type of data, we >>>>>>> could return special versions of PlcValues, that have these >>> properties? >>>>>>> >>>>>>> This way we don't have to waste the CPU time and Memory for >>>>>>> protocols that don't support this concept (We could even >>>>>>> implement the methods on the default PlvValues to simply return >>>>>>> some constants to the PlcValues all have the same API. >>>>>>> >>>>>>> Chris >>>>>>> >>>>>>> >>>>>>> -----Ursprüngliche Nachricht----- >>>>>>> Von: Łukasz Dywicki <[email protected]> >>>>>>> Gesendet: Montag, 8. März 2021 10:43 >>>>>>> An: [email protected] >>>>>>> Betreff: Re: Source Time of PLC Value? >>>>>>> >>>>>>> Looking at issue we could utilize marker interfaces. >>>>>>> At least in Java we could define Qualified and Timestamped types. >>>>>>> >>>>>>> Yet looking at present state of API we are unlikely to make each >>>>>>> and every PlcValue in each and every variant (qualified and >>>>>>> timestamped, timestamped, qualified). It would force quite a lot >>>>>>> of >>> wrapping. >>>>>>> Safest option I see is to extend subscription/response API with >>>>>>> additional method to retrieve all markers. For most of drivers it >>>>>>> would return just an empty set. It leaves us also an open door >>>>>>> for future drivers (or devices) to ship additional piece of >>>>>>> metadata which >>>> does not fit into present API. >>>>>>> >>>>>>> Best, >>>>>>> Łukasz >>>>>>> >>>>>>> >>>>>>> On 08.03.2021 04:30, Otto Fowler wrote: >>>>>>>> So, maybe the extension is timestamp *and* timestamp source. >>>>>>>> >>>>>>>> uint64_t timestamp; >>>>>>>> typedef enum { >>>>>>>> /* timestamp was generated by the source device */ >>>>>>>> SOURCE, >>>>>>>> /* timestamp was generated by plc4x on receiving */ >>>>>>>> GENERATED, >>>>>>>> } TimestampQuality; >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> On Mar 5, 2021, at 18:04, Ben Hutcheson <[email protected]> >>>> wrote: >>>>>>>>> >>>>>>>>> We do have some error codes, but it would just need to be >>>>>>>>> extended a bit. I don't think we have one for BAD (we still >>>>>>>>> receive data but the source has marked it bad). >>>>>>>>> >>>>>>>>> Timestamps I think are definitely a good idea, especially if we >>>>>>>>> eventually DNP3. >>>>>>>>> >>>>>>>>> On Fri, Mar 5, 2021 at 5:39 PM Łukasz Dywicki >>>>>>>>> <[email protected]> >>>>>>> wrote: >>>>>>>>> >>>>>>>>>> You have response code for each individual field which allows >>>>>>>>>> you to determine state (OK, REMOTE_ERROR etc.). Same >>>>>>>>>> information should be available also for subscriptions. >>>>>>>>>> >>>>>>>>>> Best, >>>>>>>>>> Łukasz >>>>>>>>>> >>>>>>>>>> On 05.03.2021 21:04, Andreas Vogler wrote: >>>>>>>>>>> Ah, forgot: and is there a status available? >>>>>>>>>>> Valid/invalid/good/bad >>>>>>>>>> indicator of a plc value? >>>>>>>>>>> >>>>>>>>>>>> On 05.03.2021, at 21:02, Andreas Vogler >>>>>>>>>>>> <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> How can I get the source time of a PlcValue? >>>>>>>>>>>> >>>>>>>>>>>> a) from a subscription PlcSubscriptionEvent >>>>>>>>>>>> b) from a read request PlcReadResponse >>>>>>>>>>>> >>>>>>>>>>>> Best regards, >>>>>>>>>>>> Andreas >>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>> >>> >>
