Lukas, for me the situation is more like this:
I have been working on PLC4X for 3,5 years full time now. If I sum up the paid gigs I got for the Java stuff in that time, I would probably be a beggar if codecentric hadn't been paying my bills. I am working on Go, because I am paid for it because the company, I work for, uses it (and not the Java version, because Java is too fat for most embedded edge gateways). I implemented the C version because it was a paid, by the EU funded project. I even started on PLC4Net because it looked like we would be needing that, but now we're using Go in the could too, but there are people just getting started working on this. I know I put a lot of additional work into PLC4X, way more than I'm paid for, but generally I implement what people want and prioritize on those willing to spend a few bucks. Right now, I know about way more companies that told me: Contact me when you have C, C#, ... than for Java. I never said that you shouldn't extend the API with Metadata as you propose, I just said that I personally would like the API a bit more streamlined and us to think of ways to implement it, so the functionalities and APIs aren't completely different - There were ways to implement things before Generics. You never came up with an alternative proposal, so my opinion didn't change. I'm definitely not going to try to come up with an alternative, as I don't see the point of the extension to start with. You also tend to say: This path leads to disaster, have a look at Camel. I have to admit, I like Camel :-) Also, there's a huge difference between changing an API and breaking an API. If later on we decide to add new features to the API, this doesn't necessarily break anything. And I never said I would block your proposal. I just pointed out some things that came to mind on it and I said that I'm probably not willing to invest my free time in something I don't see as important as you do. You say Java is most crowded ... well definitely there are more people involved in the project that could work on Java, but let's face it ... most aren't doing very much recently. The last few days I was having lots of fun with Sebastian who's really interested in the Go stuff. Chris -----Ursprüngliche Nachricht----- Von: Łukasz Dywicki <l...@code-house.org> Gesendet: Donnerstag, 11. März 2021 19:14 An: dev@plc4x.apache.org Betreff: Re: AW: Source Time of PLC Value? Hey Matthias and Chris, I agree with you about complexity and alignment. I am far from trying to cover each and every case. At the same time I am aware that making thing fine in first run is crucial to avoid major changes afterwards. That's also why I did throw pseudo-code upfront to brainstorm. Leaving it undiscussed could lead to complicated implementation and also usage. While I do appreciate work made to bring all (C/Python/C#?) languages under Apache PLC4X umbrella I also do see that sometimes they are being a baggage we carry on for no real use. When I say no use I mean that these are not officially adopted by anyone except Chris who made most of the work. Chris can't keep up all these alone. If theese are adopted then obviously users can and should take care of them given very limited resources we have. Above leads us to situation where Java which is simplest (or most crowded) part of our project can't move much forward. Java API can not extend its functionality because there are 2 or 3 other languages to carry on, with very few people who can do it. Any bigger step involving MSpec is a trouble forcing update of generators for few languages, holding us hostage of undisclosed use. Isn't it a time to make sure that C/C#/Python can still compile with 0.8 release? I feel it is a time to move forward with features we could ship in 0.9. And yes, effectively what I do propose is making unused languages/runtimes a lower priority. We never received external inquiry of these (at least here), never had external contribution and barely have docs for these. Despite of huge investment made they are not picking up to speed. If we focus on languages which been adopted and do not divide ourselves too much we can still attract people. When time comes we will help adopters who use different programming languages to close the gap. These are my 0.02€. Best, Łukasz On 11.03.2021 14:44, Matthias Milan Strljic wrote: > Hi Lukasz, > > I see your point there and I am not against "fatter" data structure either. > My problem is just that we should not make it complicated to use, but > mainly to implement. And if we add to go the route with additional > metadata, I would consider the need for a policy to homogenize certain > properties. I hope I wasn't a bit rude in my reply earlier and I'm not > against the idea of metadata either. Because when we start to make > deeper features optional on the one hand and different between > protocols or programming platforms on the other hand, the charm of > using PLC4X instead of a native driver gets increasingly lost. So I'm > actually +1 for the idea, but critical about the implementation. I > just love simplicity :D > > Greetings the simple boy Matthias > > > Am Mi., 10. März 2021 um 12:58 Uhr schrieb Łukasz Dywicki < > l...@code-house.org>: > >> Matthias, >> I do believe there are several points which are relevant and still >> not possible to be implemented using available PLC4X API. Metadata >> might not be most fortunate way, but it can enable or ease analytical >> and troubleshooting scenarios. >> >> Two major points I see: >> - Verification of device condition - ie. how does caller know >> response latency? This is especially relevant with frequent polling. >> Currently caller has no measures to know that and can not measure >> that in reliable way due to async nature of API. >> There is a way to implement backpressure technics with callback >> chains (we can automatically delay execute call), but first we need >> to bring measures for that. >> - Caching of values - when we do cyclic subscriptions or emulation of >> these through passive connections. There is no way to make caller >> code aware of nature of data received. Unless he code everything on his end. >> - Per operation timeout settings/automatic retry are another >> functionalities which are relevant in case of querying and writing. >> We do not want every driver to implement it, so making a metadata >> and/or decorator would unify this layer between drivers. >> >> I speak of above from my own perspective cause I did implement an >> experimental decorator API for PLC4X [1] to cope with CANopen >> arbitration issues I found in devices we interface with. It allows to >> catch read/write failures [2] and re-attempt entire operation. It is >> proven to work [3] independently of driver. Sadly it has no measures >> to signal its execution back to caller. In the end, traceability is >> pretty tough. >> >> Obviously OPC UA has everything sorted out, but device specific >> protocols often do not. Even a limited set of metadata can give us a >> way to start bridging the gap and making Ben's Milo/OPC UA server >> much closer to a real thing. >> >> Best, >> Łukasz >> >> [1] >> >> https://github.com/ConnectorIO/connectorio-addons/tree/master/bundles >> /org.connectorio.plc4x.decorator/src/main/java/org/connectorio/plc4x/ >> decorator >> [2] >> >> https://github.com/ConnectorIO/connectorio-addons/blob/master/bundles >> /org.connectorio.plc4x.decorator.retry/src/main/java/org/connectorio/ >> plc4x/decorator/retry/ReadRetry.java >> [3] >> >> https://github.com/ConnectorIO/connectorio-addons/blob/master/bundles >> /org.connectorio.plc4x.decorator.retry/src/test/java/org/connectorio/ >> plc4x/decorator/retry/RetryDecoratorReadTest.java >> >> >> On 08.03.2021 23:15, Matthias Milan Strljic wrote: >>> Hi all, >>> >>> i see there a point of how useful this meta information could be in >>> some scenarios. But i would rather go the route of chris. Because I >>> see there >> is >>> more of a risk of an overcomplicated inhomogeneous api structure. I >>> would not say that all API should have the same java structure on >>> each platform but if you have for all the different platforms other >>> featuresets you >> could >>> also just use partially the code generation attempt to allow a >>> separate meta info layer. >>> >>> And there would be the question if that is worth the additional feature? >>> I mean I know we could add especially on the OPC UA side some meta >>> information but is there more than the actual source timestamp? And >>> if there is none, would it be not better to use it as the actual >>> timestamp >> of >>> the value? >>> >>> >>> Einen schönen Abend ;) >>> Greetings Matthias >>> >>> >>> >>> Am Mo., 8. März 2021 um 14:29 Uhr schrieb Christofer Dutz < >>> christofer.d...@c-ware.de>: >>> >>>> Hi, >>>> >>>> I'm generally more concerned about users expecting us to deliver >>>> feature that driver X has for driver Y too. >>>> >>>> I won't object, if you think it's worth doing. >>>> >>>> Perhaps if you could whip up an example in a feature branch? I >>>> think perhaps I was still not understanding what you propose. >>>> >>>> Would that be ok? >>>> >>>> >>>> Chris >>>> >>>> >>>> -----Ursprüngliche Nachricht----- >>>> Von: Lukas Ott <ott.lukas...@gmail.com> >>>> Gesendet: Montag, 8. März 2021 14:20 >>>> An: dev@plc4x.apache.org >>>> Betreff: Re: AW: Source Time of PLC Value? >>>> >>>> +1 to Lukasz >>>> >>>> Am Mo., 8. März 2021 um 14:09 Uhr schrieb Łukasz Dywicki < >>>> l...@code-house.org>: >>>> >>>>> Hold on for a second. >>>>> >>>>> Making options pushed over connection string is a long term recipe >>>>> for disaster. Apache Camel is a prime example of what could happen >>>>> if you starting with configuration with URIs and dynamic >>>>> parameters. Most of components distinguish producer (writer) and >>>>> consumer (receiver) options, most complicated components ended up >>>>> with 40+ options settable via URI. Having a mix of PlcValues which >>>>> are timestamped and >>>> 'qualified' >>>>> and not will complicate encoding logic (how we push above info to >>>>> IEC encoder/handler or decorate it?) and how we keep custom types >>>>> compatible with above? >>>>> >>>>> That is why I rather opt for extending API in a way which does not >>>>> force existing drivers to do anything. When any of community >>>>> members, regardless if its developer or user, will find it >>>>> necessary to be ported in another driver, she or he have right to >>>>> do it. It is not only yours duty to keep all drivers and languages the >>>>> same. >>>>> >>>>> Speaking of which I am also not concerned about making all >>>>> languages having the same functionality as languages tend to offer >>>>> different options which might be hard to be ported. Lets stick to >>>>> basics which are same (connection strings, field definitions, >>>>> read/write/subscribe >>>>> syntax) and let various runtime lead their own ways toward >>>> implementation. >>>>> >>>>> Key point is simple - we won't be able to maintain all languages >>>>> equally. We should not hold moving ie. go or java forward because >>>>> we can't port new API functionality to C/Python/C#/whenever. If >>>>> there is someone who *does* use above in production and *wish* to >>>>> have new API parts then that person/organization duty is to do it. >>>>> >>>>> Best, >>>>> Łukasz >>>>> >>>>> >>>>> On 08.03.2021 13:17, Christofer Dutz wrote: >>>>>> Hi, >>>>>> >>>>>> I must admit that I would be in favor of keeping it as simple as >>>>> possible (max age in the connection string) and to implement more >>>>> of the missing parts in plc4x (like the subscription simulation >>>>> layer) and hereby getting the drivers we have a bit more aligned, >>>>> than to implement more and more special API features, that only >>>>> one or two drivers support. Cause this way we're running away form >>>>> the project-promise of running a given program at any type of PLC >>>>> with any >>>> protocol. >>>>>> >>>>>> I mean ... at the current velocity the project is underway, I >>>>>> wouldn't >>>>> really want to add a new MetaData layer that probably in the end >>>>> only >>>>> 1-2 people will be implementing and which we must not only find >>>>> out how to fill with life in the first place, but also port to all >>>>> of our supported languages. >>>>>> >>>>>> If someone's willing to bring in some serious man/womenpower, I'm >>>>>> fine >>>>> with that ... or we add it to some wish-list for future extensions. >>>>> But I wouldn't want to start something like this right now. >>>>>> >>>>>> Chris >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -----Ursprüngliche Nachricht----- >>>>>> Von: Łukasz Dywicki <l...@code-house.org> >>>>>> Gesendet: Montag, 8. März 2021 13:07 >>>>>> An: dev@plc4x.apache.org >>>>>> Betreff: Re: Source Time of PLC Value? >>>>>> >>>>>> I have not mentioned metadata term in my answer, so credits for >>>>>> mining >>>>> it API go to you. :-) >>>>>> >>>>>> Now since you brought it I remember that JDBC has a "result set >>>>> metadata". That might be thing we still miss. The result code is >>>>> most important and available instantly, other information is >>>>> supplemental and can be read additionally. >>>>>> >>>>>> >>>>>> Best, >>>>>> Łukasz >>>>>> >>>>>> On 08.03.2021 13:01, Ben Hutcheson wrote: >>>>>>> +1 for Łukasz‘s metadata approach, it would avoid having to >>>>>>> +create >>>>>>> duplicate PLcValue classes and give us a lot of flexibility in >>>>>>> the >>>>> future. >>>>>>> >>>>>>> On Mon, Mar 8, 2021 at 5:14 AM Christofer Dutz >>>>>>> <christofer.d...@c-ware.de> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi all, >>>>>>>> >>>>>>>> sorry for being late to the party ... KNX is currently >>>>>>>> consuming all my cycles. >>>>>>>> Just wanted to add my thoughts to the discussion. >>>>>>>> >>>>>>>> Initially I thought about adding this sort of information to >>>>>>>> the API, but then I thought that we have so little protocols >>>>>>>> supporting this sort of concept that blowing up every PLCValue >>>>>>>> with this time information just would waste CPU time and Memory. >>>>>>>> >>>>>>>> When it comes to simulating subscriptions by polling in the >>>>>>>> background, I thought it would be a valid solution to provide a >>>>>>>> default age a value can have in the cache and to make it >>>>>>>> possible to override this in the connection string options. If >>>>>>>> a value is too old, >>>>> a return code of: >>>>>>>> STALE_DATA could be returned. >>>>>>>> >>>>>>>> Also, we do have an option of adding an additional set of >>>>>>>> PlcValue >>>>> types. >>>>>>>> If a driver supports this quality-of-service type of data, we >>>>>>>> could return special versions of PlcValues, that have these >>>> properties? >>>>>>>> >>>>>>>> This way we don't have to waste the CPU time and Memory for >>>>>>>> protocols that don't support this concept (We could even >>>>>>>> implement the methods on the default PlvValues to simply return >>>>>>>> some constants to the PlcValues all have the same API. >>>>>>>> >>>>>>>> Chris >>>>>>>> >>>>>>>> >>>>>>>> -----Ursprüngliche Nachricht----- >>>>>>>> Von: Łukasz Dywicki <l...@code-house.org> >>>>>>>> Gesendet: Montag, 8. März 2021 10:43 >>>>>>>> An: dev@plc4x.apache.org >>>>>>>> Betreff: Re: Source Time of PLC Value? >>>>>>>> >>>>>>>> Looking at issue we could utilize marker interfaces. >>>>>>>> At least in Java we could define Qualified and Timestamped types. >>>>>>>> >>>>>>>> Yet looking at present state of API we are unlikely to make >>>>>>>> each and every PlcValue in each and every variant (qualified >>>>>>>> and timestamped, timestamped, qualified). It would force quite >>>>>>>> a lot of >>>> wrapping. >>>>>>>> Safest option I see is to extend subscription/response API with >>>>>>>> additional method to retrieve all markers. For most of drivers >>>>>>>> it would return just an empty set. It leaves us also an open >>>>>>>> door for future drivers (or devices) to ship additional piece >>>>>>>> of metadata which >>>>> does not fit into present API. >>>>>>>> >>>>>>>> Best, >>>>>>>> Łukasz >>>>>>>> >>>>>>>> >>>>>>>> On 08.03.2021 04:30, Otto Fowler wrote: >>>>>>>>> So, maybe the extension is timestamp *and* timestamp source. >>>>>>>>> >>>>>>>>> uint64_t timestamp; >>>>>>>>> typedef enum { >>>>>>>>> /* timestamp was generated by the source device */ >>>>>>>>> SOURCE, >>>>>>>>> /* timestamp was generated by plc4x on receiving */ >>>>>>>>> GENERATED, >>>>>>>>> } TimestampQuality; >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> On Mar 5, 2021, at 18:04, Ben Hutcheson >>>>>>>>>> <ben.hut...@gmail.com> >>>>> wrote: >>>>>>>>>> >>>>>>>>>> We do have some error codes, but it would just need to be >>>>>>>>>> extended a bit. I don't think we have one for BAD (we still >>>>>>>>>> receive data but the source has marked it bad). >>>>>>>>>> >>>>>>>>>> Timestamps I think are definitely a good idea, especially if >>>>>>>>>> we eventually DNP3. >>>>>>>>>> >>>>>>>>>> On Fri, Mar 5, 2021 at 5:39 PM Łukasz Dywicki >>>>>>>>>> <l...@code-house.org> >>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> You have response code for each individual field which >>>>>>>>>>> allows you to determine state (OK, REMOTE_ERROR etc.). Same >>>>>>>>>>> information should be available also for subscriptions. >>>>>>>>>>> >>>>>>>>>>> Best, >>>>>>>>>>> Łukasz >>>>>>>>>>> >>>>>>>>>>> On 05.03.2021 21:04, Andreas Vogler wrote: >>>>>>>>>>>> Ah, forgot: and is there a status available? >>>>>>>>>>>> Valid/invalid/good/bad >>>>>>>>>>> indicator of a plc value? >>>>>>>>>>>> >>>>>>>>>>>>> On 05.03.2021, at 21:02, Andreas Vogler >>>>>>>>>>>>> <andreas.vog...@me.com.INVALID> >>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> How can I get the source time of a PlcValue? >>>>>>>>>>>>> >>>>>>>>>>>>> a) from a subscription PlcSubscriptionEvent >>>>>>>>>>>>> b) from a read request PlcReadResponse >>>>>>>>>>>>> >>>>>>>>>>>>> Best regards, >>>>>>>>>>>>> Andreas >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>> >>> >> >
