Re: Supporting different layers of protocols depending on the used transport
Well I think there ist sort of protocol level logic and transport level logic.
Both usually don't really have much in common. For example for every serial
packet the serial transport logic waits for an ack message and if this doesn't
come in a given time Re sends the packet.
I wouldn't want to mix that with the normal protocol logic.
Chris
Von: Łukasz Dywicki
Gesendet: Donnerstag, 3. September 2020 00:42
An: dev@plc4x.apache.org
Betreff: Re: Supporting different layers of protocols depending on the used
transport
Maybe a bit of idea.. I think we have abstract fields (not sure if types). Can
we align both AmsData kinds somehow?
After all the role of transport is to ship payload and all we want do do is
unification of these at certain level in order to implement logic just once.
From high level point of view it can be seen as:
transport -> payload [-> mapping] -> protocol logic.
Obviously we miss currently mapping part, which might be relevant for future
cases also in CAN.
Cheers,
Łukasz Dywicki
--
Code-House
http://code-house.org
> On 2 Sep 2020, at 17:20, Christofer Dutz wrote:
>
> Hi ... bumping this thread back to life due to recent user-requests.
>
> __
>
> @Julian ... would you be able to provide me with some empty extension stub to
> implement something like in below email?
>
> Perhaps we should have a "withDefaultTransportProtocol" too ... as we do have
> a lot of transports (tcp, udp, serial, pcap, passive, can?, ...)
> I think usually we'll have one default transport and we wouldn't have to
> register handlers for every transport.
> So in the example below I would propose:
>
>@Override
>protected ProtocolStackConfigurer getStackConfigurer() {
>return SingleProtocolStackConfigurer.builder(AmsPacket.class,
> AmsPacketIO.class)
>.withProtocol(AdsProtocolLogic.class)
>.withDefaultTransportProtocol(AmsTCPPacket.class,
> AmsTCPPacketIO.class, AmsTcpTransportProtocol.class)
>.withTransportProtocol("serial", AmsSerialFrame.class,
> AmsSerialFrameIO.class, AmsSerialTransportProtocol.class)
>.littleEndian()
>.build();
>}
>
> This would only use the special AmsSerialTransportProtocol if the transport
> is "serial".
>
>
> Chris
>
>
>
> Am 12.08.20, 09:59 schrieb "Christofer Dutz" :
>
>After a little discussion with Julian we realized we need a little more:
>
>First of all using fixed classes to decide which branch to use makes it
> impossible to test using the embedded channel.
>Using a "transportFamily" property which each transport provides, makes
> this possible.
>
>Also do we have to use a different Encoder/Decoder for processing the
> packet depending on the used transport.
>So the AmsTcpTransportProtocol would be expected to consume AmsTCPPacket
> objects and produce AmsPacket objects.
>
>@Override
>protected ProtocolStackConfigurer getStackConfigurer() {
>return SingleProtocolStackConfigurer.builder(AmsPacket.class,
> AmsPacketIO.class)
>.withProtocol(AdsProtocolLogic.class)
>.withTransportProtocol("tcp", AmsTCPPacket.class,
> AmsTCPPacketIO.class, AmsTcpTransportProtocol.class)
>.withTransportProtocol("serial", AmsSerialFrame.class,
> AmsSerialFrameIO.class, AmsSerialTransportProtocol.class)
>.littleEndian()
>.build();
>}
>
>I bet this is gonna be some crazy generic stuff ...
>
>Chris
>
>Am 12.08.20, 09:04 schrieb "Christofer Dutz" :
>
>Hi all,
>
>taking this back to the list as I think it belongs here.
>
>While working on the ADS driver I noticed that we might have the need
> to pack a given protocol in different transport protocols, depending on the
> used transport.
>
>For ADS it has to either wrap the AMSPacket in a AmsTCPPacket if using
> TCP or in a AmsSerialFrame if it’s using serial. For serial also some ACK
> packets have to be created or processed.
>
>I wouldn’t want to mix that in to the driver itself as this should
> only worry about the AMSPacket logic itself.
>
>So I was thinking if it would be possible to do something like this:
>
>@Override
>protected ProtocolStackConfigurer getStackConfigurer() {
>return SingleProtocolStackConfigurer.builder(AmsPacket.class,
> AmsPacketIO.class)
>.withProtocol(AdsProtocolLogic.class)
>.withTransportProtocol(TcpTransport.class,
> AmsTcpTransportProtocol.class)
>.withTransportProtocol(SerialTransport.class,
> AmsSerialTransportProtocol.class)
>.littleEndian()
>.build();
>}
>
>Any thoughts?
>We probably have to extend the transports to return a “family” of
> transports so we can for example say that Raw-Socket and PCAP-Replay are
> “TCP” or “UDP” and for the EmbeddedTransport I would need to be able to
> configure
Re: Supporting different layers of protocols depending on the used transport
Maybe a bit of idea.. I think we have abstract fields (not sure if types). Can
we align both AmsData kinds somehow?
After all the role of transport is to ship payload and all we want do do is
unification of these at certain level in order to implement logic just once.
From high level point of view it can be seen as:
transport -> payload [-> mapping] -> protocol logic.
Obviously we miss currently mapping part, which might be relevant for future
cases also in CAN.
Cheers,
Łukasz Dywicki
--
Code-House
http://code-house.org
> On 2 Sep 2020, at 17:20, Christofer Dutz wrote:
>
> Hi ... bumping this thread back to life due to recent user-requests.
>
> __
>
> @Julian ... would you be able to provide me with some empty extension stub to
> implement something like in below email?
>
> Perhaps we should have a "withDefaultTransportProtocol" too ... as we do have
> a lot of transports (tcp, udp, serial, pcap, passive, can?, ...)
> I think usually we'll have one default transport and we wouldn't have to
> register handlers for every transport.
> So in the example below I would propose:
>
>@Override
>protected ProtocolStackConfigurer getStackConfigurer() {
>return SingleProtocolStackConfigurer.builder(AmsPacket.class,
> AmsPacketIO.class)
>.withProtocol(AdsProtocolLogic.class)
>.withDefaultTransportProtocol(AmsTCPPacket.class,
> AmsTCPPacketIO.class, AmsTcpTransportProtocol.class)
>.withTransportProtocol("serial", AmsSerialFrame.class,
> AmsSerialFrameIO.class, AmsSerialTransportProtocol.class)
>.littleEndian()
>.build();
>}
>
> This would only use the special AmsSerialTransportProtocol if the transport
> is "serial".
>
>
> Chris
>
>
>
> Am 12.08.20, 09:59 schrieb "Christofer Dutz" :
>
>After a little discussion with Julian we realized we need a little more:
>
>First of all using fixed classes to decide which branch to use makes it
> impossible to test using the embedded channel.
>Using a "transportFamily" property which each transport provides, makes
> this possible.
>
>Also do we have to use a different Encoder/Decoder for processing the
> packet depending on the used transport.
>So the AmsTcpTransportProtocol would be expected to consume AmsTCPPacket
> objects and produce AmsPacket objects.
>
>@Override
>protected ProtocolStackConfigurer getStackConfigurer() {
>return SingleProtocolStackConfigurer.builder(AmsPacket.class,
> AmsPacketIO.class)
>.withProtocol(AdsProtocolLogic.class)
>.withTransportProtocol("tcp", AmsTCPPacket.class,
> AmsTCPPacketIO.class, AmsTcpTransportProtocol.class)
>.withTransportProtocol("serial", AmsSerialFrame.class,
> AmsSerialFrameIO.class, AmsSerialTransportProtocol.class)
>.littleEndian()
>.build();
>}
>
>I bet this is gonna be some crazy generic stuff ...
>
>Chris
>
>Am 12.08.20, 09:04 schrieb "Christofer Dutz" :
>
>Hi all,
>
>taking this back to the list as I think it belongs here.
>
>While working on the ADS driver I noticed that we might have the need
> to pack a given protocol in different transport protocols, depending on the
> used transport.
>
>For ADS it has to either wrap the AMSPacket in a AmsTCPPacket if using
> TCP or in a AmsSerialFrame if it’s using serial. For serial also some ACK
> packets have to be created or processed.
>
>I wouldn’t want to mix that in to the driver itself as this should
> only worry about the AMSPacket logic itself.
>
>So I was thinking if it would be possible to do something like this:
>
>@Override
>protected ProtocolStackConfigurer getStackConfigurer() {
>return SingleProtocolStackConfigurer.builder(AmsPacket.class,
> AmsPacketIO.class)
>.withProtocol(AdsProtocolLogic.class)
>.withTransportProtocol(TcpTransport.class,
> AmsTcpTransportProtocol.class)
>.withTransportProtocol(SerialTransport.class,
> AmsSerialTransportProtocol.class)
>.littleEndian()
>.build();
>}
>
>Any thoughts?
>We probably have to extend the transports to return a “family” of
> transports so we can for example say that Raw-Socket and PCAP-Replay are
> “TCP” or “UDP” and for the EmbeddedTransport I would need to be able to
> configure what it should be.
>
>Chris
>
>
>
>
Re: Leaking nioEventLoopGroup threads
Hi Julian. I would like to submit a fix for this, but the relevant section
of the code is tricky for me to understand.
Basically, if the PooledPlcDriverManager is used with
DefaultNettyPlcConnections, when code calls the "close" method of the
PlcConnection, the "close" method of the defaultNettyPlcConnection is
intercepted and never called, and the nioEventLoopGroup threads are never
closed as your patch corrected. The pooled PlcConnection can go stale or
become invalidated by the PLC while it is waiting in the pool, so perhaps
the fix is to test if the connection isConnected when it is borrowed from
the pool, and then explicitly call the closed method before returning it if
the connection is not connected?
I would have thought the pooledPlcDriverManager would always test for and
return a connected PlcConnection, but I seem to remember another discussion
on this topic and the preference was expressed to not always call connect()
on the PlcConnection before returning.
>From the PooledPlcDriverManager class my potential fix is below (see
comment starting "//Call the close method..."
@Override
public PlcConnection getConnection(String url, PlcAuthentication
authentication) throws PlcConnectionException {
PoolKey poolKey = poolKeyFactory.getPoolKey(url, authentication);
if (LOGGER.isDebugEnabled()) {
if (authentication != noPlcAuthentication) {
LOGGER.debug("Try to borrow an object for url {} and authentication {}",
url, authentication);
} else {
LOGGER.debug("Try to borrow an object for url {}", url);
}
}
PlcConnection plcConnection;
try {
plcConnection = keyedObjectPool.borrowObject(poolKey); //Call the close
method if the connection has lost connection to close threads that might
remain open if (!plcConnection.isConnected()) { plcConnection.close(); }
} catch (Exception e) {
throw new PlcConnectionException(e);
}
// Used to invalidate a proxy
AtomicBoolean proxyInvalidated = new AtomicBoolean(false);
return (PlcConnection) Proxy.newProxyInstance(classLoader, new Class[]{
PlcConnection.class}, (proxy, method, args) -> {
if (proxyInvalidated.get()) {
throw new IllegalStateException("Proxy not valid anymore");
}
if ("close".equals(method.getName())) {
LOGGER.debug("close called on {}", plcConnection);
proxyInvalidated.set(true);
keyedObjectPool.returnObject(poolKey, plcConnection);
return null;
} else {
try {
return method.invoke(plcConnection, args);
} catch (InvocationTargetException e) {
if (e.getCause().getClass() == PlcConnectionException.class) {
keyedObjectPool.invalidateObject(poolKey, plcConnection);
proxyInvalidated.set(true);
}
throw e;
}
}
});
}
On Tue, Aug 25, 2020 at 10:30 AM Julian Feinauer <
j.feina...@pragmaticminds.de> wrote:
> Hi Adam,
>
>
>
> sorry form y late response and in facgt thanks for your feedback, happy to
> hear!
>
> I will merge my branch and close the issue, no worries.
>
>
>
> And you are exactly right, just log another issue fort he Pool and we hope
> that someone may find the time to have a look there : )
>
>
>
> Julian
>
>
>
> PS.: As you already digged into the code you can try to fix it yourself of
> course, I would be more than happy to get a PR from you : )
>
>
>
> *Von: *Adam Rossi
> *Datum: *Dienstag, 25. August 2020 um 15:45
> *An: *Julian Feinauer
> *Cc: *"dev@plc4x.apache.org"
> *Betreff: *Re: Leaking nioEventLoopGroup threads
>
>
>
> Juilan,
>
>
>
> My apologies - your fix did indeed work. The issue is that
> PooledPlcDriverManager does not seem to be calling the close method on the
> connection. Switching back to PlcDriverManager from PooledPlcDriverManager
> results in your new log comments showing up in the log, and more
> importantly no more leaks of the nioEventLoopGroup threads. I have tested
> the code in a loop for an hour or so and it is working perfectly.
>
>
>
> The PooledPlcDriverManager seems to intercept the close method on the
> plcConnection (lines 125 - 130):
>
>
>
> if ("close".equals(method.getName())) {
>
> LOGGER.debug("close called on {}", plcConnection);
>
> proxyInvalidated.set(true);
>
> keyedObjectPool.returnObject(poolKey, plcConnection);
>
> return null;
>
> } else {
>
>
>
> Which makes sense as it is trying to keep active connections pooled.
> However, when this connection is again retrieved from the pool it seems the
> plcConnection connects again and creates an additional nioEventLoopGroup
> thread, which is never closed.
>
>
>
> I am new to working with this project and Jira. It seems to me that I
> should close the issue I just created as your fix does indeed correct the
> original issue, and perhaps open another issue on PooledPlcDriverManager
> for this thread leak?
>
>
>
> Regards, Adam
>
>
>
> On Mon, Aug 24, 2020 at 4:39 AM Julian Feinauer <
> j.feina...@pragmaticminds.de> wrote:
>
> Hi,
>
> short feedback. I looked into the code and indeed it seems that we had a
> bug there which could lead tot he socket leak you described.
> I
Re: MODBUS over RTU configuration
Well the PLC4X folks have just recently started working on a new initiative:
PLC4PY (All PLC4X drivers in native Python implementations)
Unfortunately I'm consumed with my PLC4C ... if you want to join them ... they
would be happy about anyone helping.
But if you want to get rid of Python (which I can understand very well ;-) )
feel free to stick to PLC4J, which is obviously the most mature PLC4X ecosystem.
Chris
Am 02.09.20, 17:38 schrieb "Alessio Bernesco Làvore"
:
Yes, in fact right now we are using a python edge system (written by
ourselves) to connect, but i'd like to migrate it to java.
We mainly interact with OPC-UA or ModBus devices but sometimes the latter
are Serial 485 and so we need an RTU enabled driver.
I'll constantly keep an eye on the list and I'll contribute to the topic,
if I can give any help.
Greetings,
Alessio
On Wed, Sep 2, 2020 at 5:33 PM Christofer Dutz
wrote:
> Hi Alessio,
>
> thanks for volunteering to help with testing ... as Otto will agree ...
> testing stuff is sometimes a good thing to do ;-)
>
> Even if it's not highest priority for you, we should continue the
> discussion as this issue will come up sooner or later.
>
> Chris
>
>
> Am 02.09.20, 17:29 schrieb "Alessio Bernesco Làvore" <
> alessio.berne...@gmail.com>:
>
> Hello Chris,
> thank you!
>
> I'm currently using some exotic RTU devices, just for test, so mine is
> a
> low priority question.
> Of course i'm fully available to test any release against those
> devices.
>
> Greetings,
> Alessio
>
> On Wed, Sep 2, 2020 at 5:14 PM Christofer Dutz <
> christofer.d...@c-ware.de>
> wrote:
>
> > Hi Alessio,
> >
> > welcome on this super-cool community list of ours :-)
> >
> > Unfortunately I have a little "uncool" news ... while working on the
> new
> > drivers, it occurred to me that we currently are absolutely able to
> run one
> > protocol over multiple transports, however if the transports require
> > different handling on the lowest level, we currently haven't got
> that base
> > covered.
> >
> > We started discussing options on how to solve this problem, but the
> > discussion is ongoing ...
> >
> > Unfortunately I didn't update the documentation ... as I thought
> we'd be
> > able to implement this sooner.
> >
> > Possibly the 0.6.x version of PLC4X could work as starting with
> 0.7.0 we
> > have deleted all existing drivers and have replaced them with new,
> > generated ones and are still working on implementing all the
> features the
> > old ones had. Unfortunately you seem to be the first person to ask
> for
> > serial Modbus, so we hadn't prioritized that up too much.
> >
> > So for now .. please give the 0.6.1 a go (might have slightly
> different
> > syntax and connection string) and join in on the discussion here on
> the
> > list and possibly help us finish this ... as none of us have a
serial
> > Modbus device, it's sort of tricky to implement a driver for it.
> >
> > Chris
> >
> >
> >
> >
> > Am 02.09.20, 16:50 schrieb "Alessio Bernesco Làvore" <
> > alessio.berne...@gmail.com>:
> >
> > Hi everyone,
> >
> > i'm trying to complete some tests connecting a Modbus RTU device
> > (digital
> > input converter).
> >
> > The device is connected using a USB/485 converter, running with
> > pyModbus
> > i'm able to communicate with the device.
> >
> > I'm using this connection string:
> >
> > PlcConnection plcConnection = new
> >
> >
>
PlcDriverManager().getConnection("modbus:serial:///dev/ttyUSB0?unit-identifier=1");
> >
> > Then:
> > PlcReadRequest.Builder builder =
> plcConnection.readRequestBuilder();
> > builder.addItem("value-0", "holding-register:14");
> > PlcReadRequest readRequest = builder.build();
> > PlcReadResponse response = readRequest.execute().get();
> >
> > But i was unable to obtain a response, watching at the logs
> there are
> > plenty of:
> > io.netty.channel.nio.NioEventLoop: Migrated 1 channel(s)
> to
> > the new
> > Selector.
> > Selector.select() returned prematurely 512 times in a row;
> rebuilding
> > Selector
> >
> org.apache.plc4x.java.transport.serial.SerialPollingSelector@37f2b566.
> >
> > Another important
Re: MODBUS over RTU configuration
Yes, in fact right now we are using a python edge system (written by
ourselves) to connect, but i'd like to migrate it to java.
We mainly interact with OPC-UA or ModBus devices but sometimes the latter
are Serial 485 and so we need an RTU enabled driver.
I'll constantly keep an eye on the list and I'll contribute to the topic,
if I can give any help.
Greetings,
Alessio
On Wed, Sep 2, 2020 at 5:33 PM Christofer Dutz
wrote:
> Hi Alessio,
>
> thanks for volunteering to help with testing ... as Otto will agree ...
> testing stuff is sometimes a good thing to do ;-)
>
> Even if it's not highest priority for you, we should continue the
> discussion as this issue will come up sooner or later.
>
> Chris
>
>
> Am 02.09.20, 17:29 schrieb "Alessio Bernesco Làvore" <
> alessio.berne...@gmail.com>:
>
> Hello Chris,
> thank you!
>
> I'm currently using some exotic RTU devices, just for test, so mine is
> a
> low priority question.
> Of course i'm fully available to test any release against those
> devices.
>
> Greetings,
> Alessio
>
> On Wed, Sep 2, 2020 at 5:14 PM Christofer Dutz <
> christofer.d...@c-ware.de>
> wrote:
>
> > Hi Alessio,
> >
> > welcome on this super-cool community list of ours :-)
> >
> > Unfortunately I have a little "uncool" news ... while working on the
> new
> > drivers, it occurred to me that we currently are absolutely able to
> run one
> > protocol over multiple transports, however if the transports require
> > different handling on the lowest level, we currently haven't got
> that base
> > covered.
> >
> > We started discussing options on how to solve this problem, but the
> > discussion is ongoing ...
> >
> > Unfortunately I didn't update the documentation ... as I thought
> we'd be
> > able to implement this sooner.
> >
> > Possibly the 0.6.x version of PLC4X could work as starting with
> 0.7.0 we
> > have deleted all existing drivers and have replaced them with new,
> > generated ones and are still working on implementing all the
> features the
> > old ones had. Unfortunately you seem to be the first person to ask
> for
> > serial Modbus, so we hadn't prioritized that up too much.
> >
> > So for now .. please give the 0.6.1 a go (might have slightly
> different
> > syntax and connection string) and join in on the discussion here on
> the
> > list and possibly help us finish this ... as none of us have a serial
> > Modbus device, it's sort of tricky to implement a driver for it.
> >
> > Chris
> >
> >
> >
> >
> > Am 02.09.20, 16:50 schrieb "Alessio Bernesco Làvore" <
> > alessio.berne...@gmail.com>:
> >
> > Hi everyone,
> >
> > i'm trying to complete some tests connecting a Modbus RTU device
> > (digital
> > input converter).
> >
> > The device is connected using a USB/485 converter, running with
> > pyModbus
> > i'm able to communicate with the device.
> >
> > I'm using this connection string:
> >
> > PlcConnection plcConnection = new
> >
> >
> PlcDriverManager().getConnection("modbus:serial:///dev/ttyUSB0?unit-identifier=1");
> >
> > Then:
> > PlcReadRequest.Builder builder =
> plcConnection.readRequestBuilder();
> > builder.addItem("value-0", "holding-register:14");
> > PlcReadRequest readRequest = builder.build();
> > PlcReadResponse response = readRequest.execute().get();
> >
> > But i was unable to obtain a response, watching at the logs
> there are
> > plenty of:
> > io.netty.channel.nio.NioEventLoop: Migrated 1 channel(s)
> to
> > the new
> > Selector.
> > Selector.select() returned prematurely 512 times in a row;
> rebuilding
> > Selector
> >
> org.apache.plc4x.java.transport.serial.SerialPollingSelector@37f2b566.
> >
> > Another important part is that i'm unable to find where i can
> > configure the
> > serial communication parameters, like baud rate, stop bits, etc.
> >
> > Greetings,
> > Alessio
> >
> >
>
>
Re: MODBUS over RTU configuration
Hi Alessio,
thanks for volunteering to help with testing ... as Otto will agree ... testing
stuff is sometimes a good thing to do ;-)
Even if it's not highest priority for you, we should continue the discussion as
this issue will come up sooner or later.
Chris
Am 02.09.20, 17:29 schrieb "Alessio Bernesco Làvore"
:
Hello Chris,
thank you!
I'm currently using some exotic RTU devices, just for test, so mine is a
low priority question.
Of course i'm fully available to test any release against those devices.
Greetings,
Alessio
On Wed, Sep 2, 2020 at 5:14 PM Christofer Dutz
wrote:
> Hi Alessio,
>
> welcome on this super-cool community list of ours :-)
>
> Unfortunately I have a little "uncool" news ... while working on the new
> drivers, it occurred to me that we currently are absolutely able to run
one
> protocol over multiple transports, however if the transports require
> different handling on the lowest level, we currently haven't got that base
> covered.
>
> We started discussing options on how to solve this problem, but the
> discussion is ongoing ...
>
> Unfortunately I didn't update the documentation ... as I thought we'd be
> able to implement this sooner.
>
> Possibly the 0.6.x version of PLC4X could work as starting with 0.7.0 we
> have deleted all existing drivers and have replaced them with new,
> generated ones and are still working on implementing all the features the
> old ones had. Unfortunately you seem to be the first person to ask for
> serial Modbus, so we hadn't prioritized that up too much.
>
> So for now .. please give the 0.6.1 a go (might have slightly different
> syntax and connection string) and join in on the discussion here on the
> list and possibly help us finish this ... as none of us have a serial
> Modbus device, it's sort of tricky to implement a driver for it.
>
> Chris
>
>
>
>
> Am 02.09.20, 16:50 schrieb "Alessio Bernesco Làvore" <
> alessio.berne...@gmail.com>:
>
> Hi everyone,
>
> i'm trying to complete some tests connecting a Modbus RTU device
> (digital
> input converter).
>
> The device is connected using a USB/485 converter, running with
> pyModbus
> i'm able to communicate with the device.
>
> I'm using this connection string:
>
> PlcConnection plcConnection = new
>
>
PlcDriverManager().getConnection("modbus:serial:///dev/ttyUSB0?unit-identifier=1");
>
> Then:
> PlcReadRequest.Builder builder = plcConnection.readRequestBuilder();
> builder.addItem("value-0", "holding-register:14");
> PlcReadRequest readRequest = builder.build();
> PlcReadResponse response = readRequest.execute().get();
>
> But i was unable to obtain a response, watching at the logs there are
> plenty of:
> io.netty.channel.nio.NioEventLoop: Migrated 1 channel(s) to
> the new
> Selector.
> Selector.select() returned prematurely 512 times in a row; rebuilding
> Selector
> org.apache.plc4x.java.transport.serial.SerialPollingSelector@37f2b566.
>
> Another important part is that i'm unable to find where i can
> configure the
> serial communication parameters, like baud rate, stop bits, etc.
>
> Greetings,
> Alessio
>
>
Re: [jira] [Created] (PLC4X-214) [Modbus] Holding register addresses have an offset of 1 (Not reading the correct address)
Hi Ben, while I just updated the Modbus documentation I noticed your recent additions aren't documented anywhere. I could probably do it, but I would feel more comfortable if you could do so as you actually understand the stuff :-) Would be super awesome, if you could add the description of your format changes & additions in the file: src/site/asciidoc/users/protocols/modbus.adoc Chris Am 16.07.20, 11:48 schrieb "Ben Hutcheson" : Hi, *I think we have 6 separate memory areas. Do you have a mapping, That I could use? I mean which first digit represents which memory area?* I thought there were only 5 areas? 0x - Coils 1x - Inputs 3x - Input Registers 4x - Holding Registers 6x - Extended Registers I've seen the IEEE format for 32-bit floats, also another format that gets used is multiplying the float by 100 or 1000 and then dividing it by the same on the other end. e.g. 56.67 becomes 5667, Kind Regards Ben On Thu, Jul 16, 2020 at 4:10 AM Niclas Hedhman wrote: > For floats, I have only seen IEEE format. But can't rule out other. > > On Thu, Jul 16, 2020, 14:57 Christofer Dutz > wrote: > > > Guess it should be possible for plc4x to interpret INT as two shorts long > > as four... In that case it could probably also handle half precision > floats > > (16 bit), full floats and double, if the encoding is somewhat standard > > (which I assume it's not) > > > > Chris > > > > Von: Niclas Hedhman > > Gesendet: Donnerstag, 16. Juli 2020 08:33 > > An: dev@plc4x.apache.org > > Betreff: Re: [jira] [Created] (PLC4X-214) [Modbus] Holding register > > addresses have an offset of 1 (Not reading the correct address) > > > > To make things worse, there is equipment on the market with both 32-bit > > numbers as well IEEE floats. > > > > And many clients are incapable of doing something meaningful with > those... > > > > > > And then there is equipment that uses one register to indicate the > > magnitude of one or more other registers, say 1="divide by 1", 2="divide > by > > 10"... > > > > > > > > Niclas > > > > On Thu, Jul 16, 2020, 14:28 Christofer Dutz > > wrote: > > > > > Hi Ben and Otto, > > > > > > First off all, thank you Ben for that very detailed explanation. It > does > > > seem as if we should extend the parser to support the different numeric > > > variants. I don't see any problems in supporting both the hex-like one > as > > > well as the pure numeric one. > > > > > > I think we have 6 separate memory areas. Do you have a mapping, That I > > > could use? I mean which first digit represents which memory area? > > > > > > @otto Modbus doesn't allow floats. Just bits (coils) and shorts > > > (registers)... Haven't seen a somewhat standard way to encode anything > > else. > > > > > > Chris > > > > > > Von: Otto Fowler > > > Gesendet: Donnerstag, 16. Juli 2020 06:41 > > > An: dev@plc4x.apache.org > > > Betreff: Re: [jira] [Created] (PLC4X-214) [Modbus] Holding register > > > addresses have an offset of 1 (Not reading the correct address) > > > > > > Don’t forget embedded protocols are possible, > > > different devices format floats differently > > > some devices don’t want persistent connections > > > etc etc > > > > > > On July 15, 2020 at 20:48:39, Ben Hutcheson (ben.hut...@gmail.com) > > wrote: > > > > > > Hi, > > > > > > Answering some of the questions:- > > > *I guess what would be interesting, would be what address is going over > > the > > > wire for "30001" for example.* > > > The address that gets sent over the wire is the address starting from 0 > > i.e > > > 31 would be address 0. I didn't know that. > > > > > > *Also as a register is always a 16 bit value, the increments by two > sort > > of > > > puzzle me.* > > > The Modbus registers are numbered 31 thru to 365536 (or whatever > the > > > highest is for the device), they are 16 bit registers they don't > > increment > > > by 2. If you were mapping 32-bit data types to the registers then of > > course > > > you would increment by two but I don't know of any other situation or > > > device that increments the addresses by 2. > > > > > > > > > *Long time ago, back when the Modbus came to life it was simply a > > > memory area* > > > I do vaguely remember working on a device that didn't differentiate > > between > > > the memory areas and would just use the address when reading and > writing, > > > the leading digit would just specify data type. 01 would be a bit > in > > > 41. > > >
Re: Supporting different layers of protocols depending on the used transport
Hi ... bumping this thread back to life due to recent user-requests.
__
@Julian ... would you be able to provide me with some empty extension stub to
implement something like in below email?
Perhaps we should have a "withDefaultTransportProtocol" too ... as we do have a
lot of transports (tcp, udp, serial, pcap, passive, can?, ...)
I think usually we'll have one default transport and we wouldn't have to
register handlers for every transport.
So in the example below I would propose:
@Override
protected ProtocolStackConfigurer getStackConfigurer() {
return SingleProtocolStackConfigurer.builder(AmsPacket.class,
AmsPacketIO.class)
.withProtocol(AdsProtocolLogic.class)
.withDefaultTransportProtocol(AmsTCPPacket.class,
AmsTCPPacketIO.class, AmsTcpTransportProtocol.class)
.withTransportProtocol("serial", AmsSerialFrame.class,
AmsSerialFrameIO.class, AmsSerialTransportProtocol.class)
.littleEndian()
.build();
}
This would only use the special AmsSerialTransportProtocol if the transport is
"serial".
Chris
Am 12.08.20, 09:59 schrieb "Christofer Dutz" :
After a little discussion with Julian we realized we need a little more:
First of all using fixed classes to decide which branch to use makes it
impossible to test using the embedded channel.
Using a "transportFamily" property which each transport provides, makes
this possible.
Also do we have to use a different Encoder/Decoder for processing the
packet depending on the used transport.
So the AmsTcpTransportProtocol would be expected to consume AmsTCPPacket
objects and produce AmsPacket objects.
@Override
protected ProtocolStackConfigurer getStackConfigurer() {
return SingleProtocolStackConfigurer.builder(AmsPacket.class,
AmsPacketIO.class)
.withProtocol(AdsProtocolLogic.class)
.withTransportProtocol("tcp", AmsTCPPacket.class,
AmsTCPPacketIO.class, AmsTcpTransportProtocol.class)
.withTransportProtocol("serial", AmsSerialFrame.class,
AmsSerialFrameIO.class, AmsSerialTransportProtocol.class)
.littleEndian()
.build();
}
I bet this is gonna be some crazy generic stuff ...
Chris
Am 12.08.20, 09:04 schrieb "Christofer Dutz" :
Hi all,
taking this back to the list as I think it belongs here.
While working on the ADS driver I noticed that we might have the need
to pack a given protocol in different transport protocols, depending on the
used transport.
For ADS it has to either wrap the AMSPacket in a AmsTCPPacket if using
TCP or in a AmsSerialFrame if it’s using serial. For serial also some ACK
packets have to be created or processed.
I wouldn’t want to mix that in to the driver itself as this should only
worry about the AMSPacket logic itself.
So I was thinking if it would be possible to do something like this:
@Override
protected ProtocolStackConfigurer getStackConfigurer() {
return SingleProtocolStackConfigurer.builder(AmsPacket.class,
AmsPacketIO.class)
.withProtocol(AdsProtocolLogic.class)
.withTransportProtocol(TcpTransport.class,
AmsTcpTransportProtocol.class)
.withTransportProtocol(SerialTransport.class,
AmsSerialTransportProtocol.class)
.littleEndian()
.build();
}
Any thoughts?
We probably have to extend the transports to return a “family” of
transports so we can for example say that Raw-Socket and PCAP-Replay are “TCP”
or “UDP” and for the EmbeddedTransport I would need to be able to configure
what it should be.
Chris
Re: MODBUS over RTU configuration
Hello Chris,
thank you!
I'm currently using some exotic RTU devices, just for test, so mine is a
low priority question.
Of course i'm fully available to test any release against those devices.
Greetings,
Alessio
On Wed, Sep 2, 2020 at 5:14 PM Christofer Dutz
wrote:
> Hi Alessio,
>
> welcome on this super-cool community list of ours :-)
>
> Unfortunately I have a little "uncool" news ... while working on the new
> drivers, it occurred to me that we currently are absolutely able to run one
> protocol over multiple transports, however if the transports require
> different handling on the lowest level, we currently haven't got that base
> covered.
>
> We started discussing options on how to solve this problem, but the
> discussion is ongoing ...
>
> Unfortunately I didn't update the documentation ... as I thought we'd be
> able to implement this sooner.
>
> Possibly the 0.6.x version of PLC4X could work as starting with 0.7.0 we
> have deleted all existing drivers and have replaced them with new,
> generated ones and are still working on implementing all the features the
> old ones had. Unfortunately you seem to be the first person to ask for
> serial Modbus, so we hadn't prioritized that up too much.
>
> So for now .. please give the 0.6.1 a go (might have slightly different
> syntax and connection string) and join in on the discussion here on the
> list and possibly help us finish this ... as none of us have a serial
> Modbus device, it's sort of tricky to implement a driver for it.
>
> Chris
>
>
>
>
> Am 02.09.20, 16:50 schrieb "Alessio Bernesco Làvore" <
> alessio.berne...@gmail.com>:
>
> Hi everyone,
>
> i'm trying to complete some tests connecting a Modbus RTU device
> (digital
> input converter).
>
> The device is connected using a USB/485 converter, running with
> pyModbus
> i'm able to communicate with the device.
>
> I'm using this connection string:
>
> PlcConnection plcConnection = new
>
> PlcDriverManager().getConnection("modbus:serial:///dev/ttyUSB0?unit-identifier=1");
>
> Then:
> PlcReadRequest.Builder builder = plcConnection.readRequestBuilder();
> builder.addItem("value-0", "holding-register:14");
> PlcReadRequest readRequest = builder.build();
> PlcReadResponse response = readRequest.execute().get();
>
> But i was unable to obtain a response, watching at the logs there are
> plenty of:
> io.netty.channel.nio.NioEventLoop: Migrated 1 channel(s) to
> the new
> Selector.
> Selector.select() returned prematurely 512 times in a row; rebuilding
> Selector
> org.apache.plc4x.java.transport.serial.SerialPollingSelector@37f2b566.
>
> Another important part is that i'm unable to find where i can
> configure the
> serial communication parameters, like baud rate, stop bits, etc.
>
> Greetings,
> Alessio
>
>
Re: MODBUS over RTU configuration
Hi Alessio,
welcome on this super-cool community list of ours :-)
Unfortunately I have a little "uncool" news ... while working on the new
drivers, it occurred to me that we currently are absolutely able to run one
protocol over multiple transports, however if the transports require different
handling on the lowest level, we currently haven't got that base covered.
We started discussing options on how to solve this problem, but the discussion
is ongoing ...
Unfortunately I didn't update the documentation ... as I thought we'd be able
to implement this sooner.
Possibly the 0.6.x version of PLC4X could work as starting with 0.7.0 we have
deleted all existing drivers and have replaced them with new, generated ones
and are still working on implementing all the features the old ones had.
Unfortunately you seem to be the first person to ask for serial Modbus, so we
hadn't prioritized that up too much.
So for now .. please give the 0.6.1 a go (might have slightly different syntax
and connection string) and join in on the discussion here on the list and
possibly help us finish this ... as none of us have a serial Modbus device,
it's sort of tricky to implement a driver for it.
Chris
Am 02.09.20, 16:50 schrieb "Alessio Bernesco Làvore"
:
Hi everyone,
i'm trying to complete some tests connecting a Modbus RTU device (digital
input converter).
The device is connected using a USB/485 converter, running with pyModbus
i'm able to communicate with the device.
I'm using this connection string:
PlcConnection plcConnection = new
PlcDriverManager().getConnection("modbus:serial:///dev/ttyUSB0?unit-identifier=1");
Then:
PlcReadRequest.Builder builder = plcConnection.readRequestBuilder();
builder.addItem("value-0", "holding-register:14");
PlcReadRequest readRequest = builder.build();
PlcReadResponse response = readRequest.execute().get();
But i was unable to obtain a response, watching at the logs there are
plenty of:
io.netty.channel.nio.NioEventLoop: Migrated 1 channel(s) to the new
Selector.
Selector.select() returned prematurely 512 times in a row; rebuilding
Selector
org.apache.plc4x.java.transport.serial.SerialPollingSelector@37f2b566.
Another important part is that i'm unable to find where i can configure the
serial communication parameters, like baud rate, stop bits, etc.
Greetings,
Alessio
MODBUS over RTU configuration
Hi everyone,
i'm trying to complete some tests connecting a Modbus RTU device (digital
input converter).
The device is connected using a USB/485 converter, running with pyModbus
i'm able to communicate with the device.
I'm using this connection string:
PlcConnection plcConnection = new
PlcDriverManager().getConnection("modbus:serial:///dev/ttyUSB0?unit-identifier=1");
Then:
PlcReadRequest.Builder builder = plcConnection.readRequestBuilder();
builder.addItem("value-0", "holding-register:14");
PlcReadRequest readRequest = builder.build();
PlcReadResponse response = readRequest.execute().get();
But i was unable to obtain a response, watching at the logs there are
plenty of:
io.netty.channel.nio.NioEventLoop: Migrated 1 channel(s) to the new
Selector.
Selector.select() returned prematurely 512 times in a row; rebuilding
Selector
org.apache.plc4x.java.transport.serial.SerialPollingSelector@37f2b566.
Another important part is that i'm unable to find where i can configure the
serial communication parameters, like baud rate, stop bits, etc.
Greetings,
Alessio
Re: [MODBUS][DISCUSS] Improving handling of datatypes ...
Hi Julian, I agree ... if one driver would define an "INT" as 32bit integer and the others would treat it as 16bit ... that could be a problem. Perhaps having a statement of the project that we use the IEC 61131 types as a basis and if you want to use a given protocols different types, that you can prefix them .. Assuming a driver for the famous "HURZ" protocol would use 32bit INTs, then an "INT" could reference the 16bit version and a "HURZ_INT" could be the 32bit version? Chris Am 02.09.20, 15:26 schrieb "Julian Feinauer" : Hi, agree with your suggestion! Although we have to be careful to not mix it up with specific implementations of datatypes in some drivers. Julian Am 02.09.20, 15:21 schrieb "Christofer Dutz" : Hi all, today I was at a customer’s site and used the Modbus driver to get data. This generally worked fine. The thing however I found a little complicated, was that the PLC seemed to offer all values as 32Bit Floating point values. So in order to correctly read them, I had to read an array of two consecutive shorts and then manually convert them into a float. I bet we can do this better. So I thought … how about we use the same https://en.wikipedia.org/wiki/IEC_61131-3 datatypes we are already using in other drivers and for example if you write: holding-register:1:REAL it would automatically use modbus to read an array of two shorts and to internally convert these to one REAL/float. What do you think? I think we could probably do this in most drivers. Chris
Re: [MODBUS][DISCUSS] Improving handling of datatypes ...
Hi, agree with your suggestion! Although we have to be careful to not mix it up with specific implementations of datatypes in some drivers. Julian Am 02.09.20, 15:21 schrieb "Christofer Dutz" : Hi all, today I was at a customer’s site and used the Modbus driver to get data. This generally worked fine. The thing however I found a little complicated, was that the PLC seemed to offer all values as 32Bit Floating point values. So in order to correctly read them, I had to read an array of two consecutive shorts and then manually convert them into a float. I bet we can do this better. So I thought … how about we use the same https://en.wikipedia.org/wiki/IEC_61131-3 datatypes we are already using in other drivers and for example if you write: holding-register:1:REAL it would automatically use modbus to read an array of two shorts and to internally convert these to one REAL/float. What do you think? I think we could probably do this in most drivers. Chris
[MODBUS][DISCUSS] Improving handling of datatypes ...
Hi all, today I was at a customer’s site and used the Modbus driver to get data. This generally worked fine. The thing however I found a little complicated, was that the PLC seemed to offer all values as 32Bit Floating point values. So in order to correctly read them, I had to read an array of two consecutive shorts and then manually convert them into a float. I bet we can do this better. So I thought … how about we use the same https://en.wikipedia.org/wiki/IEC_61131-3 datatypes we are already using in other drivers and for example if you write: holding-register:1:REAL it would automatically use modbus to read an array of two shorts and to internally convert these to one REAL/float. What do you think? I think we could probably do this in most drivers. Chris
[DRAFT] September Board Report
Hi all … I just prepared our board report and saved it as a draft … please review and give feedback if you want me to add/remove/change things … Chris ## Description: The mission of the Apache PLC4X project is creating a set of libraries for communicating with industrial programmable logic controllers (PLCs) using a variety of protocols but with a shared API. ## Issues: None ## Membership Data: Apache PLC4X was founded 2019-04-17 (a year ago) There are currently 16 committers and 10 PMC members in this project. The Committer-to-PMC ratio is 8:5. Community changes, past quarter: - No new PMC members. Last addition was Lukas Ott on 2020-03-17. - No new committers. Last addition was Otto Fowler on 2020-04-30. ## Project Activity: Version 0.7.0 was released on 2020-05-25 and we are working hard on making the 0.8.0 ready for release. In the past few weeks we did quite a number of bug reports and feature requests which have mostly been addressed. Also in the last few weeks a lot of new folks have been showing, up some of which will probably be invited pretty to join the community soon, if they continue the current level of activity. Especially the new PLC4PY initiative, where a group of community members and new folks have started working on porting PLC4J to Python is currently seeding more activity in the community. Another notable initiative is one to bring support for CAN bus communication as well also the efforts to port PLC4J to C. ApacheCon@Home will be having 3 talks where PLC4X is a main topic or a sub-topic and we are looking forward to it. ## Community Health: - dev@plc4x.apache.org had a 5% decrease in traffic in the past quarter (577 emails compared to 602) - iss...@plc4x.apache.org had a 47% increase in traffic in the past quarter (270 emails compared to 183) - 39 issues opened in JIRA, past quarter (129% increase) - 26 issues closed in JIRA, past quarter (100% increase) - 260 commits in the past quarter (-30% decrease) - 13 code contributors in the past quarter (-7% decrease) - 19 PRs opened on GitHub, past quarter (-45% decrease) - 16 PRs closed on GitHub, past quarter (-52% decrease) - 224 Github Stars (up by 23) - 373 @ApachePLC4X Twitter account followers (up by 18)
