Hi Christof, Yeah, that does not sound that good…
I typically deal with SCADA systems for that need - collect plc data, add some added value (metadata, structure, object models, ...) - and then publish this enriched data via …hm, maybe via OPC UA? … but sure, SCADA systems are not for free … or use PLC4X .. and maybe mapped.com <http://mapped.com/> to add enrich the data? :-) No glue what is behind the scenes of mapped.com <http://mapped.com/>… Andreas > On 10.03.2021, at 16:22, Christofer Dutz <[email protected]> wrote: > > Hi Andreas, > > Glad I didn't upset you too much ;-) > > And I guess I'm also a bit pissed about OPC-UA in general. At first, I had > such a hard time selling PLC4X as OPC-UA was promising so much, but without > proof, that almost nobody gave open-source and PLC4X a chance. So much money > was burnt in 2018 and 2019 with unsuccessful OPC-UA projects by the big > automation companies. > > Then end of 2019 some of the ones, that decided to go the OPC-UA path, > started contacting me again. But when we wanted to discuss terms, they > sometimes had the nerve to say something like: We wasted so much money on > OPC-UA and our budget is almost depleted ... we can only hire your company if > you greatly reduce the hourly rate. Guess what my answer was? ;-) > > Chris > > > > -----Ursprüngliche Nachricht----- > Von: Andreas Vogler <[email protected]> > Gesendet: Mittwoch, 10. März 2021 16:06 > An: [email protected] > Betreff: Re: Source Time of PLC Value? > > No problem, I think I understand what you mean… and I agree on the device > level OPC UA is maybe too much / to fat (or not well implemented :-)) > > > >> On 10.03.2021, at 15:50, Christofer Dutz <[email protected]> wrote: >> >> Hi Andreas, >> >> I didn't want to upset you. I should have been a bit more clear on this: >> >> In 2018, when I was travelling through the country proclaiming PLC4X, all I >> heard was: But OPC-UA can do all of this (Without having tested it) ... 2019 >> most of the people who tried it were already looking for alternatives. End >> of 2019 and beginning of 2020 was definitely the age of MQTT (Of course OPC >> UA now provided a MQTT-like communication form). >> >> I think OPC-UA is great for machine-to-machine communication and stuff like >> dashboards. But as soon as you want to do high volume data acquisition. >> Especially for use cases like machine-learning etc. the processing power of >> even modern PLCs can't handle the load. >> >> For example, at one of the biggest German car manufacturers, they were not >> able to get more than 200 datapoints every 2 seconds from a new S7 400 PLC >> via OPC-UA. These are probably the most powerful ones Siemens has to offer. >> If they requested 300, the PLC shut down the IO process due to overload. >> With PLC4X using the S7 protocol, we were able to get 2600 datapoints in >> 200ms from the same machines. >> >> On the client side on one of the standard VMs they were using, running the >> OPC-UA stack was able to collect data on 10 PLCs with 200 datapoints. Then >> the CPU power was 100% ... in our case we run PLC4X on the same VMs using S7 >> and there we could collect 200 PLCs with 2600 datapoints. So also on the >> client side OPC-UA is a very fat project, that requires a lot of CPU time. >> >> I hope this explains things a bit more. If you are just reading a few >> datapoints or using OPC-UA for Server-to-Server communication the servers >> will be able to cope with more. And if it doesn't you just add more nodes to >> the compute cluster. >> >> Chris >> >> >> >> >> >> -----Ursprüngliche Nachricht----- >> Von: Andreas Vogler <[email protected]> >> Gesendet: Mittwoch, 10. März 2021 15:38 >> An: [email protected] >> Betreff: Re: Source Time of PLC Value? >> Priorität: Niedrig >> >> Just a side note: it hurts to hear that OPC UA sucks badly >> performance-wise. I have done tests with Milo OPC UA Client and SCADA >> OPC UA servers, and they perform pretty fine. 200 MQTT clients writing >> values via a Gateway with OPC UA Milo (Client) to two OPC UA servers, >> each OPC UA server handles 50000 values changes per second: >> https://www.linkedin.com/posts/andreas-vogler-361115122_graphql-mqtt-v >> ertx-activity-6769029730797719552-ZkjG >> <https://www.linkedin.com/posts/andreas-vogler-361115122_graphql-mqtt- >> vertx-activity-6769029730797719552-ZkjG> >> >> But I fully agree with MQTT - it’s simple - and I think that’s the >> reason why it is so popular… >> >> >>> On 10.03.2021, at 14:34, Christofer Dutz <[email protected]> wrote: >>> >>> Yes, >>> >>> OPC-UA has all of this ... and it's probably one of the reasons so many >>> implementations differ, and you have to again interpret results. It's also >>> probably a reason why OPC-UA sucks that badly performance-wise. >>> >>> Just saying that we should be careful what we choose as reference. In >>> contrast typical MQTT doesn't have all of this and it seems to be gaining >>> more and more traction, just because it doesn't have all of this overhead. >>> >>> Just my thoughts on this. >>> >>> Chris >>> >>> >>> >>> >>> -----Ursprüngliche Nachricht----- >>> Von: Łukasz Dywicki <[email protected]> >>> Gesendet: Mittwoch, 10. März 2021 12:58 >>> An: [email protected] >>> Betreff: Re: AW: Source Time of PLC Value? >>> >>> Matthias, >>> I do believe there are several points which are relevant and still not >>> possible to be implemented using available PLC4X API. Metadata might not be >>> most fortunate way, but it can enable or ease analytical and >>> troubleshooting scenarios. >>> >>> Two major points I see: >>> - Verification of device condition - ie. how does caller know response >>> latency? This is especially relevant with frequent polling. Currently >>> caller has no measures to know that and can not measure that in reliable >>> way due to async nature of API. >>> There is a way to implement backpressure technics with callback chains (we >>> can automatically delay execute call), but first we need to bring measures >>> for that. >>> - Caching of values - when we do cyclic subscriptions or emulation of these >>> through passive connections. There is no way to make caller code aware of >>> nature of data received. Unless he code everything on his end. >>> - Per operation timeout settings/automatic retry are another >>> functionalities which are relevant in case of querying and writing. We do >>> not want every driver to implement it, so making a metadata and/or >>> decorator would unify this layer between drivers. >>> >>> I speak of above from my own perspective cause I did implement an >>> experimental decorator API for PLC4X [1] to cope with CANopen arbitration >>> issues I found in devices we interface with. It allows to catch read/write >>> failures [2] and re-attempt entire operation. It is proven to work [3] >>> independently of driver. Sadly it has no measures to signal its execution >>> back to caller. In the end, traceability is pretty tough. >>> >>> Obviously OPC UA has everything sorted out, but device specific protocols >>> often do not. Even a limited set of metadata can give us a way to start >>> bridging the gap and making Ben's Milo/OPC UA server much closer to a real >>> thing. >>> >>> Best, >>> Łukasz >>> >>> [1] >>> https://github.com/ConnectorIO/connectorio-addons/tree/master/bundles >>> / >>> org.connectorio.plc4x.decorator/src/main/java/org/connectorio/plc4x/d >>> e >>> corator >>> [2] >>> https://github.com/ConnectorIO/connectorio-addons/blob/master/bundles >>> / >>> org.connectorio.plc4x.decorator.retry/src/main/java/org/connectorio/p >>> l >>> c4x/decorator/retry/ReadRetry.java >>> [3] >>> https://github.com/ConnectorIO/connectorio-addons/blob/master/bundles >>> / >>> org.connectorio.plc4x.decorator.retry/src/test/java/org/connectorio/p >>> l c4x/decorator/retry/RetryDecoratorReadTest.java >>> >>> >>> On 08.03.2021 23:15, Matthias Milan Strljic wrote: >>>> Hi all, >>>> >>>> i see there a point of how useful this meta information could be in >>>> some scenarios. But i would rather go the route of chris. Because I >>>> see there is more of a risk of an overcomplicated inhomogeneous api >>>> structure. I would not say that all API should have the same java >>>> structure on each platform but if you have for all the different >>>> platforms other featuresets you could also just use partially the >>>> code generation attempt to allow a separate meta info layer. >>>> >>>> And there would be the question if that is worth the additional feature? >>>> I mean I know we could add especially on the OPC UA side some meta >>>> information but is there more than the actual source timestamp? And >>>> if there is none, would it be not better to use it as the actual >>>> timestamp of the value? >>>> >>>> >>>> Einen schönen Abend ;) >>>> Greetings Matthias >>>> >>>> >>>> >>>> Am Mo., 8. März 2021 um 14:29 Uhr schrieb Christofer Dutz < >>>> [email protected]>: >>>> >>>>> Hi, >>>>> >>>>> I'm generally more concerned about users expecting us to deliver >>>>> feature that driver X has for driver Y too. >>>>> >>>>> I won't object, if you think it's worth doing. >>>>> >>>>> Perhaps if you could whip up an example in a feature branch? I >>>>> think perhaps I was still not understanding what you propose. >>>>> >>>>> Would that be ok? >>>>> >>>>> >>>>> Chris >>>>> >>>>> >>>>> -----Ursprüngliche Nachricht----- >>>>> Von: Lukas Ott <[email protected]> >>>>> Gesendet: Montag, 8. März 2021 14:20 >>>>> An: [email protected] >>>>> Betreff: Re: AW: Source Time of PLC Value? >>>>> >>>>> +1 to Lukasz >>>>> >>>>> Am Mo., 8. März 2021 um 14:09 Uhr schrieb Łukasz Dywicki < >>>>> [email protected]>: >>>>> >>>>>> Hold on for a second. >>>>>> >>>>>> Making options pushed over connection string is a long term recipe >>>>>> for disaster. Apache Camel is a prime example of what could happen >>>>>> if you starting with configuration with URIs and dynamic parameters. >>>>>> Most of components distinguish producer (writer) and consumer >>>>>> (receiver) options, most complicated components ended up with 40+ >>>>>> options settable via URI. Having a mix of PlcValues which are >>>>>> timestamped and >>>>> 'qualified' >>>>>> and not will complicate encoding logic (how we push above info to >>>>>> IEC encoder/handler or decorate it?) and how we keep custom types >>>>>> compatible with above? >>>>>> >>>>>> That is why I rather opt for extending API in a way which does not >>>>>> force existing drivers to do anything. When any of community >>>>>> members, regardless if its developer or user, will find it >>>>>> necessary to be ported in another driver, she or he have right to >>>>>> do it. It is not only yours duty to keep all drivers and languages the >>>>>> same. >>>>>> >>>>>> Speaking of which I am also not concerned about making all >>>>>> languages having the same functionality as languages tend to offer >>>>>> different options which might be hard to be ported. Lets stick to >>>>>> basics which are same (connection strings, field definitions, >>>>>> read/write/subscribe >>>>>> syntax) and let various runtime lead their own ways toward >>>>> implementation. >>>>>> >>>>>> Key point is simple - we won't be able to maintain all languages >>>>>> equally. We should not hold moving ie. go or java forward because >>>>>> we can't port new API functionality to C/Python/C#/whenever. If >>>>>> there is someone who *does* use above in production and *wish* to >>>>>> have new API parts then that person/organization duty is to do it. >>>>>> >>>>>> Best, >>>>>> Łukasz >>>>>> >>>>>> >>>>>> On 08.03.2021 13:17, Christofer Dutz wrote: >>>>>>> Hi, >>>>>>> >>>>>>> I must admit that I would be in favor of keeping it as simple as >>>>>> possible (max age in the connection string) and to implement more >>>>>> of the missing parts in plc4x (like the subscription simulation >>>>>> layer) and hereby getting the drivers we have a bit more aligned, >>>>>> than to implement more and more special API features, that only >>>>>> one or two drivers support. Cause this way we're running away form >>>>>> the project-promise of running a given program at any type of PLC >>>>>> with any >>>>> protocol. >>>>>>> >>>>>>> I mean ... at the current velocity the project is underway, I >>>>>>> wouldn't >>>>>> really want to add a new MetaData layer that probably in the end >>>>>> only >>>>>> 1-2 people will be implementing and which we must not only find >>>>>> out how to fill with life in the first place, but also port to all >>>>>> of our supported languages. >>>>>>> >>>>>>> If someone's willing to bring in some serious man/womenpower, I'm >>>>>>> fine >>>>>> with that ... or we add it to some wish-list for future extensions. >>>>>> But I wouldn't want to start something like this right now. >>>>>>> >>>>>>> Chris >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -----Ursprüngliche Nachricht----- >>>>>>> Von: Łukasz Dywicki <[email protected]> >>>>>>> Gesendet: Montag, 8. März 2021 13:07 >>>>>>> An: [email protected] >>>>>>> Betreff: Re: Source Time of PLC Value? >>>>>>> >>>>>>> I have not mentioned metadata term in my answer, so credits for >>>>>>> mining >>>>>> it API go to you. :-) >>>>>>> >>>>>>> Now since you brought it I remember that JDBC has a "result set >>>>>> metadata". That might be thing we still miss. The result code is >>>>>> most important and available instantly, other information is >>>>>> supplemental and can be read additionally. >>>>>>> >>>>>>> >>>>>>> Best, >>>>>>> Łukasz >>>>>>> >>>>>>> On 08.03.2021 13:01, Ben Hutcheson wrote: >>>>>>>> +1 for Łukasz‘s metadata approach, it would avoid having to >>>>>>>> +create >>>>>>>> duplicate PLcValue classes and give us a lot of flexibility in >>>>>>>> the >>>>>> future. >>>>>>>> >>>>>>>> On Mon, Mar 8, 2021 at 5:14 AM Christofer Dutz >>>>>>>> <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> sorry for being late to the party ... KNX is currently >>>>>>>>> consuming all my cycles. >>>>>>>>> Just wanted to add my thoughts to the discussion. >>>>>>>>> >>>>>>>>> Initially I thought about adding this sort of information to >>>>>>>>> the API, but then I thought that we have so little protocols >>>>>>>>> supporting this sort of concept that blowing up every PLCValue >>>>>>>>> with this time information just would waste CPU time and Memory. >>>>>>>>> >>>>>>>>> When it comes to simulating subscriptions by polling in the >>>>>>>>> background, I thought it would be a valid solution to provide a >>>>>>>>> default age a value can have in the cache and to make it >>>>>>>>> possible to override this in the connection string options. If >>>>>>>>> a value is too old, >>>>>> a return code of: >>>>>>>>> STALE_DATA could be returned. >>>>>>>>> >>>>>>>>> Also, we do have an option of adding an additional set of >>>>>>>>> PlcValue >>>>>> types. >>>>>>>>> If a driver supports this quality-of-service type of data, we >>>>>>>>> could return special versions of PlcValues, that have these >>>>> properties? >>>>>>>>> >>>>>>>>> This way we don't have to waste the CPU time and Memory for >>>>>>>>> protocols that don't support this concept (We could even >>>>>>>>> implement the methods on the default PlvValues to simply return >>>>>>>>> some constants to the PlcValues all have the same API. >>>>>>>>> >>>>>>>>> Chris >>>>>>>>> >>>>>>>>> >>>>>>>>> -----Ursprüngliche Nachricht----- >>>>>>>>> Von: Łukasz Dywicki <[email protected]> >>>>>>>>> Gesendet: Montag, 8. März 2021 10:43 >>>>>>>>> An: [email protected] >>>>>>>>> Betreff: Re: Source Time of PLC Value? >>>>>>>>> >>>>>>>>> Looking at issue we could utilize marker interfaces. >>>>>>>>> At least in Java we could define Qualified and Timestamped types. >>>>>>>>> >>>>>>>>> Yet looking at present state of API we are unlikely to make >>>>>>>>> each and every PlcValue in each and every variant (qualified >>>>>>>>> and timestamped, timestamped, qualified). It would force quite >>>>>>>>> a lot of >>>>> wrapping. >>>>>>>>> Safest option I see is to extend subscription/response API with >>>>>>>>> additional method to retrieve all markers. For most of drivers >>>>>>>>> it would return just an empty set. It leaves us also an open >>>>>>>>> door for future drivers (or devices) to ship additional piece >>>>>>>>> of metadata which >>>>>> does not fit into present API. >>>>>>>>> >>>>>>>>> Best, >>>>>>>>> Łukasz >>>>>>>>> >>>>>>>>> >>>>>>>>> On 08.03.2021 04:30, Otto Fowler wrote: >>>>>>>>>> So, maybe the extension is timestamp *and* timestamp source. >>>>>>>>>> >>>>>>>>>> uint64_t timestamp; >>>>>>>>>> typedef enum { >>>>>>>>>> /* timestamp was generated by the source device */ >>>>>>>>>> SOURCE, >>>>>>>>>> /* timestamp was generated by plc4x on receiving */ >>>>>>>>>> GENERATED, >>>>>>>>>> } TimestampQuality; >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On Mar 5, 2021, at 18:04, Ben Hutcheson >>>>>>>>>>> <[email protected]> >>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> We do have some error codes, but it would just need to be >>>>>>>>>>> extended a bit. I don't think we have one for BAD (we still >>>>>>>>>>> receive data but the source has marked it bad). >>>>>>>>>>> >>>>>>>>>>> Timestamps I think are definitely a good idea, especially if >>>>>>>>>>> we eventually DNP3. >>>>>>>>>>> >>>>>>>>>>> On Fri, Mar 5, 2021 at 5:39 PM Łukasz Dywicki >>>>>>>>>>> <[email protected]> >>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> You have response code for each individual field which >>>>>>>>>>>> allows you to determine state (OK, REMOTE_ERROR etc.). Same >>>>>>>>>>>> information should be available also for subscriptions. >>>>>>>>>>>> >>>>>>>>>>>> Best, >>>>>>>>>>>> Łukasz >>>>>>>>>>>> >>>>>>>>>>>> On 05.03.2021 21:04, Andreas Vogler wrote: >>>>>>>>>>>>> Ah, forgot: and is there a status available? >>>>>>>>>>>>> Valid/invalid/good/bad >>>>>>>>>>>> indicator of a plc value? >>>>>>>>>>>>> >>>>>>>>>>>>>> On 05.03.2021, at 21:02, Andreas Vogler >>>>>>>>>>>>>> <[email protected]> >>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> How can I get the source time of a PlcValue? >>>>>>>>>>>>>> >>>>>>>>>>>>>> a) from a subscription PlcSubscriptionEvent >>>>>>>>>>>>>> b) from a read request PlcReadResponse >>>>>>>>>>>>>> >>>>>>>>>>>>>> Best regards, >>>>>>>>>>>>>> Andreas >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>> >>>>> >>>> >> >
