On 09/10/2008, Oghie Sheehy <[EMAIL PROTECTED]> wrote:
> > Yes.
>
> > I'm still considering how to fix up the read() method. Probably I'll
> > add a readBytes() method and maybe add an abstract class that
> > implements it.
>
> > The problem is that the sampler has to convert the string to bytes in
> > order to store it in the sample result. Wire protocols work in terms
> > of bytes, so this means multiple conversions, with the possibility of
> > data loss if some bytes are not both-ways convertible.
>
>
> Am with you now, had thought that strings were stored in the sample result
> but of course they're not.
>
> Like the idea of adding the readBytes() method and allowing the sampler to
> convert the bytes as appropriate because the responsibility of the client
> should be to handle the byte exchange alone.
>
> Rather than adding the abstract class could the byte[] implementation
> replace the string read() method in the sampler and drop the byte [] straight
> into the sampler result? This would affect existing testplans which are
> working because of the inherent data conversion and there may not be too many
> of these?
Yes, that's possible. In either case existing implementations would
need to be amended and rebuilt, so I suppose it does not matter which
is done.
It would be possible to extend the TCPClient interface to add a
readBytes() method, and have TCPSampler use instanceof to choose which
method to use. This would allow old implementations to continue, but
would be less efficient - and we want implementations to return bytes.
So I think it would be OK to break compatibility.
> Also think that a writeBytes() method is needed.
>
I see what you mean - the client implementation currently has to
convert the string into bytes, and the sampler has no control over the
encoding that is used.
I think it's unlikely that there are many 3rd party implementations
(if any) so perhaps the string method should be removed.
We need the input/output encodings to be specified on the sampler GUI
(they can default to the current setting, for backward compat.)
> > Sorry, should have explained in more detail: I was thinking that setup
> > parameters could be used to configure the protocol implementation.
> > This could be useful for minor variants on the protocol - e.g. 4byte
> > length prefix, and variable length strings.
>
>
> Yes this would make sense.
>
> ******
>
> Revisiting the length-prefixed binary extension based on the multiple
> conversion point raised above, apologies for repetition and stating the
> obvious here but just need to work through it...
Good idea.
> A client extension is required to handle length prefix regardless of whether
> byte[] or string is returned.
Yes.
> The client needs to know how many bytes contain the length and how to
> interpret these bytes. This would pollute the client/sampler relationship
> because the client needs to know about the structure of the byte stream.
I was suggesting that the sampler deals only in the data without prefixes.
The client needs to know how to add/remove the length prefix.
The prefix length is either fixed in the client, or the sampler can
pass in a parameter which tells it what particular protocol variation
is to be used.
The sampler is interested in the user data, the client looks after the
packaging.
> It seems reasonable that a service could use a 2 or 4-byte binary length
> while the message payload would contain character data so a client and
> sampler would not necessarily interpret bytes in the same way.
>
> It also seems reasonable that a length prefix should always specify the no.
> of bytes to read (as this is a wire-level field) rather than characters.
Yes.
> So the client should not be concerned about the character set of the data
> but only by the size and interpretation of the length prefix.
This will only be true if the input is bytes or a stream.
I now think the client show know the encoding (see below).
> The sampler needs to know how to represent the data in a human-readable form
> while not losing any information due to conversion. Therefore it needs to
> know whether the data is binary/character set.
I think that depends on how you look at the protocol. In your original
case, you would like JMeter to hande hex-encoded binary. You could
regard the conversion between hex and binary as part of the protocol,
and therefore handled by the client.
I'm not keen on putting too much processing in the sampler. The idea
is that JMeter users can develop their own implementations in order to
handle any special processing, and this can be provided in an add-on
jar. It's a lot harder for a user to make changes to the sampler.
I'm hoping that we can come up with more generic sampler which allows
client implementations to do what they need to do.
I'm now beginning to think that the client should still be passed the
string from the GUI, but that it also needs the encoding - which it
may ignore. This makes it more flexible for clients that need to
process the string before transmission.
> Lastly I think it would be justifiable and from a user-perspective
> preferable to hide the length prefix from sampler input and results.
Yes, see above.
> To achieve this I now think that the following changes should be made:
>
> Client:
> Either a single new client TCPLengthPrefixedClientImpl which takes length
> prefix length and length prefix format binary/character set as
> constructor/setter arguments. For this to be flexible these need to provided
> through the Sampler GUI.
> OR multiple new clients BinaryTCPLengthPrefixedClientImpl,
> DecimalTCPLengthPrefixedClientImpl etc. which take the length prefix length
> as a constructor.
>
> Client(s) would implement/override writeBytes(byte[]) method and generate
> the length prefix appropriately based on the size of the byte array.
Yes, that's what I was thinking.
> Client(s) would override readBytes() method and interpret the length bytes
> from the stream, these would not be passed back to the sampler.
Yes.
> The reason for adding/stripping length bytes is so that the sampler does not
> have to understand these.
Agreed.
> Sampler GUI:
> From above GUI needs to specify
> a) Length prefix length
> b) Length prefix format OR ClientImpl as above
No, I think these are part of the client implementation, see above.
> In addition the GUI needs to specify
> c) Data format
Again, I think this is best dealt with by the client implementation.
> Sampler:
> The sampler needs to convert the input text string into a byte[] for
> transmission and a
byte[] into a text string (e.g. to hex string for binary/to local
character set for character set). This then goes back into the
sampler result using the getBytes() method as before.
Not quite.
Originally I thought the sampler should convert the GUI data to bytes
using the specified encoding. I now think the sampler should pass in
the string and the encoding, as this is more flexible.
On return, it copies the bytes to the sample result, and sets the
appropriate encoding - there is no conversion at that point. The
response is only converted to a string when it is extracted from the
sample result, e.g. by the Tree View Listener.
The only issue I can see here is: how does the sampler know what
encoding to use for the response? This can be specified on the GUI (or
we can assume it is the same as the request encoding). If there is
likely to be a possibility that the encoding is defined in the
response message (for example, this is how HTTP responses work), then
I think it makes sense for the client to extract this information, and
pass it back to the sampler somehow.
We could add a getEncoding() method to the interface, but it might be
better to use a generic Object getParameter(String name) to the
interface, in case it is later found that there are other items that
need to be retrieved. The sampler would need to be updated, but
existing clients would not need to be updated.
In summary, I think the TCPClient interface should probably be amended
as follows:
void write(OutputStream os, String s, String encoding);
The
void write(OutputStream os, InputStream is);
method probably does not need the encoding.
void setParameter(String name, Object value);
Object getParameter(String name);
There would be an abstract class that provides a dummy implementation
for the get/set Parameter methods.
WDYT?
> ******
>
>
> -----Original Message-----
> From: sebb [mailto:[EMAIL PROTECTED]
>
> Sent: 08 October 2008 16:57
> To: JMeter Developers List
> Cc: Mike Cronin
> Subject: Re: TCP Sampler Extension to support length-prefixed binary
> data
>
>
> On 08/10/2008, Oghie Sheehy <[EMAIL PROTECTED]> wrote:
> > Thanks, would be happy to adopt the suggested approach and delegate the
> protocol-specific conversion to a new TCPClient implementation with selection
> from the TCPSampler GUI to support test plans using more than one.
> >
> > This would leave the TCPClient Interface read()/write() method signatures
> unchanged as strings would be passed in and out.
> >
>
> Yes.
>
> I'm still considering how to fix up the read() method. Probably I'll
> add a readBytes() method and maybe add an abstract class that
> implements it.
>
> The problem is that the sampler has to convert the string to bytes in
> order to store it in the sample result. Wire protocols work in terms
> of bytes, so this means multiple conversions, with the possibility of
> data loss if some bytes are not both-ways convertible.
>
> > Not sure whether setup parameters would be necessary given this approach
> as TCPClient Implementation would be specific to protocol/character set in
> use?
> >
>
> Sorry, should have explained in more detail: I was thinking that setup
> parameters could be used to configure the protocol implementation.
> This could be useful for minor variants on the protocol - e.g. 4byte
> length prefix, and variable length strings.
>
> >
> > -----Original Message-----
> > From: sebb [mailto:[EMAIL PROTECTED]
> > Sent: 08 October 2008 13:04
> > To: JMeter Developers List
> > Cc: Mike Cronin
> > Subject: Re: TCP Sampler Extension to support length-prefixed binary
> > data
> >
> >
> > On 08/10/2008, Oghie Sheehy <[EMAIL PROTECTED]> wrote:
> > > Hello,
> > >
> > > We have a requirement to test a TCP service which expects requests in
> length-prefixed binary form and responds with data structured in the same way.
> >
> > Is this a proprietary service, or are there several such services?
> >
> > > We have experimentally extended the TCP sampler code to support this
> by adding a "Length Prefixed Binary Data" option to the GUI.
> > >
> > > With this option checked:
> > >
> > > a) The TCPSampler sample() method treats the input request text as a
> hexadecimal string and converts it to a byte array.
> > > Conversely it reads a byte array and converts it to a hexadecimal
> string.
> > >
> > > b) The TCPClient has been extended to support the following new
> methods:
> > > void write(OutputStream os, byte[] b);
> > > byte [] readLengthPrefixed(InputStream is);
> > > The readLengthPrefixed method expects the first 2 bytes from the
> stream to contain the length (binary) of data to follow and reads this number
> of subsequent bytes returning a byte array containing both the length and
> data bytes.
> > >
> > > e.g.
> > > Request/Response String:
> "002d482d2d2d3e454e414320544350495020484541525420424541543c2d2d2d5443504449414c2056312e30312020"
> > > Input/Output Byte Array: [0x00,0x2d, 0x48, ......, 0x20]
> > >
> > > This is a simple but limited approach, it does not permit the
> transmission of non-length prefixed binary data nor does it support length
> prefixed character data. Finally as it is implemented the length prefix must
> be input as two byte binary (input as hexadecimal string) rather than being
> calculated or input as character decimal.
> > >
> > > There are options to make the solution more flexible but before
> spending time on these we would like to know whether the minimal approach
> taken above would be considered for inclusion in a future JMeter release?
> > >
> >
> > It sounds fairly straightforward, and would not affect existing test
> plans.
> >
> > I'm just wondering if it might be cleaner to add the functionality as
> > a new TCPClient implementation?
> >
> > At present the TCPSampler only supports a single implementation that
> > is defined via a property, but it could be enhanced to fetch the name
> > from the GUI to support test plans which require more than one
> > implementation.
> >
> > It might be necessary to extend the TCPClient interface to support
> > some kind of setup method to allow parameters to be passed in; these
> > would need to be added to the TCP Sample GUI.
> >
> > The advantage of this approach would be that there should be no need
> > to change the TCP Sampler further in order to accommodate new
> > protocols.
> >
> > I have just realised that the TCPClient interface read() method was
> > badly designed; it should have returned a byte array. At present both
> > the client and the sampler have to do String<>byte conversions - and
> > there is no provision for specifying the character set to be used.
> >
> > However, that does not affect your current requirements.
> >
> > > Thanks,
> > > Oghie.
> > >
> > >
> > >
> > >
> > >
> > >
> > > **********************************************************************
> > >
> > > E-mail disclaimer
> > > FEXCO Dynamic Currency Conversion Limited, registered in Ireland, No.
> 246289. Registered Office: FEXCO Centre, Iveragh Road, Killorglin, Co. Kerry.
> > >
> > > This message, including any attachments, is confidential. If you are
> not the named recipient, please contact the sender and delete the email from
> your system.
> > >
> > > **********************************************************************
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: [EMAIL PROTECTED]
> > > For additional commands, e-mail: [EMAIL PROTECTED]
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [EMAIL PROTECTED]
> > For additional commands, e-mail: [EMAIL PROTECTED]
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [EMAIL PROTECTED]
> > For additional commands, e-mail: [EMAIL PROTECTED]
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
