Re: Question regarding exchanging long characteristic values over BLE

[Andrzej Kaczmarek]

Hi Simon,

On Thu, Mar 8, 2018 at 9:18 AM, Simon Ratner  wrote:
> Old thread, but I just bumped into this myself so want to resurrect it.
>
> Current api makes it very difficult to implement a long characteristic that
> changes frequently (e.g. time- or sensor-based reading, or including a
> random component). In the case where mtu exchange fails or does not
> complete in time, the client may receive a mashup of two or more different
> values, if access_cb returns a current value each time. For example, a
> 32-byte value might end up as 22 bytes from sample 0 plus 10 bytes from
> sample 1 -- a combination that does not decode to a valid reading. One way
> around this is to lock in a stable sample for each connection, but that
> becomes harder to keep track of with many concurrent connections.

Reading long attributes is not atomic and there's not much stack can
do about it - it's just what spec says. Trying to make it look like
atomic operation in NimBLE would not be a good idea since there is
probably no good way to do this properly so instead we will end up
with complex code that breaks things.

I suggest to change approach here and instead of trying to read long
value just notify complete value in fragments. For example, add
 and  to each chunk and client should be
able to reassemble complete value easily without need to read
anything.

> I don't really have a solution yet, just complaining ;) Perhaps nimble
> holding on to the value for subsequent offset reads makes sense after all.
> I guess the difficulty there is knowing when to free it?

This is certainly one of biggest difficulties here. Also note that
attribute values are unique per-connection so this means we need to
store separate copy of value for each connection. This can easily
consume lots of memory and we don't really know if/when it can be
freed to make things work as expected.

I'd say if someone has good idea how such caching could be
implemented, it can be done as separate utility package so one can
reuse it in application easily. I'd certainly not want such thing to
be added to NimBLE directly since it's just not how ATT/GATT should
work according to spec.

> Cheers,
> simon

Best regards,
Andrzej


> On Tue, Jul 25, 2017 at 12:19 PM, Andrzej Kaczmarek <
> andrzej.kaczma...@codecoup.pl> wrote:
>
>> Hi,
>>
>>
>> On Tue, Jul 25, 2017 at 8:14 PM, Christopher Collins 
>> wrote:
>>
>> > On Tue, Jul 25, 2017 at 10:46:32AM -0700, Pritish Gandhi wrote:
>> > [...]
>> >
>> [...]
>>
>>
>> > > Is this true for notifications too? If I need to send notifications for
>> > > that long characteristic value, will my access callback be called
>> several
>> > > times based on the MTU of the connection?
>> >
>> > Unfortunately, Bluetooth restricts notifications to a single packet.  If
>> > the characteristic value is longer than the MTU allows, the notification
>> > gets truncated.  To get around this, the application needs to chunk the
>> > value into several notifications, and possibly use some protocol which
>> > indicates the total length, parts remaining, etc.
>> >
>>
>> Also client can just do long read in order to read remaining portion of
>> characteristic value
>>
>> - this is what ATT spec suggests.
>> It depends on actual use case, but this way client may be able to decide
>> whether it should read remaining portion of value or skip it, e.g. some
>> flags can be placed at the beginning of the characteristic value and they
>> will be always sent in notification.
>>
>> Best regards,
>> Andrzej
>>

Re: Question regarding exchanging long characteristic values over BLE

[Christopher Collins]

Hi Simon,

On Thu, Mar 08, 2018 at 12:18:41AM -0800, Simon Ratner wrote:
> Old thread, but I just bumped into this myself so want to resurrect it.
> 
> Current api makes it very difficult to implement a long characteristic that
> changes frequently (e.g. time- or sensor-based reading, or including a
> random component). In the case where mtu exchange fails or does not
> complete in time, the client may receive a mashup of two or more different
> values, if access_cb returns a current value each time. For example, a
> 32-byte value might end up as 22 bytes from sample 0 plus 10 bytes from
> sample 1 -- a combination that does not decode to a valid reading. One way
> around this is to lock in a stable sample for each connection, but that
> becomes harder to keep track of with many concurrent connections.

If you expect all clients to support a reasonable MTU, you might just
punt on the problem: if you can't fit the characteristic value in a
single response, send some sort of "characteristic not ready" response
instead.  You can determine the connection's MTU by calling
`ble_att_mtu()`.

This doesn't solve the general problem, of course.

> I don't really have a solution yet, just complaining ;) Perhaps nimble
> holding on to the value for subsequent offset reads makes sense after all.
> I guess the difficulty there is knowing when to free it?

That seems like a good solution.  Regarding how long to hold onto the
cached value, well, that's what syscfg is for :).

I thought of an alternative that won't actually work, but I'll share it
anyway: allow the application to associate a minimum MTU with each
characteristic.  If a peer attempts to read the characteristic over a
connection with an MTU that is too low, the stack initiates an MTU
exchange procedure, and only responds to the read request after the MTU
has increased.  Unfortunately, this doesn't work because changes in the
MTU don't apply to transactions that are already in progress.

Chris

> 
> Cheers,
> simon
> 
> 
> On Tue, Jul 25, 2017 at 12:19 PM, Andrzej Kaczmarek <
> andrzej.kaczma...@codecoup.pl> wrote:
> 
> > Hi,
> >
> >
> > On Tue, Jul 25, 2017 at 8:14 PM, Christopher Collins 
> > wrote:
> >
> > > On Tue, Jul 25, 2017 at 10:46:32AM -0700, Pritish Gandhi wrote:
> > > [...]
> > >
> > ​[...]​
> >
> >
> > > > Is this true for notifications too? If I need to send notifications for
> > > > that long characteristic value, will my access callback be called
> > several
> > > > times based on the MTU of the connection?
> > >
> > > Unfortunately, Bluetooth restricts notifications to a single packet.  If
> > > the characteristic value is longer than the MTU allows, the notification
> > > gets truncated.  To get around this, the application needs to chunk the
> > > value into several notifications, and possibly use some protocol which
> > > indicates the total length, parts remaining, etc.
> > >
> >
> > Also client can just do long read in order to read remaining portion of
> > characteristic value​
> >
> > ​- this is what ATT spec suggests.
> > It depends on actual use case, but this way client may be able to decide
> > whether it should read remaining portion of value or skip it, e.g. some
> > flags can be placed at the beginning of the characteristic value and they
> > will be always sent in notification.
> >
> > Best regards,
> > Andrzej
> >

Re: Question regarding exchanging long characteristic values over BLE

[Simon Ratner]

Old thread, but I just bumped into this myself so want to resurrect it.

Current api makes it very difficult to implement a long characteristic that
changes frequently (e.g. time- or sensor-based reading, or including a
random component). In the case where mtu exchange fails or does not
complete in time, the client may receive a mashup of two or more different
values, if access_cb returns a current value each time. For example, a
32-byte value might end up as 22 bytes from sample 0 plus 10 bytes from
sample 1 -- a combination that does not decode to a valid reading. One way
around this is to lock in a stable sample for each connection, but that
becomes harder to keep track of with many concurrent connections.

I don't really have a solution yet, just complaining ;) Perhaps nimble
holding on to the value for subsequent offset reads makes sense after all.
I guess the difficulty there is knowing when to free it?

Cheers,
simon


On Tue, Jul 25, 2017 at 12:19 PM, Andrzej Kaczmarek <
andrzej.kaczma...@codecoup.pl> wrote:

> Hi,
>
>
> On Tue, Jul 25, 2017 at 8:14 PM, Christopher Collins 
> wrote:
>
> > On Tue, Jul 25, 2017 at 10:46:32AM -0700, Pritish Gandhi wrote:
> > [...]
> >
> ​[...]​
>
>
> > > Is this true for notifications too? If I need to send notifications for
> > > that long characteristic value, will my access callback be called
> several
> > > times based on the MTU of the connection?
> >
> > Unfortunately, Bluetooth restricts notifications to a single packet.  If
> > the characteristic value is longer than the MTU allows, the notification
> > gets truncated.  To get around this, the application needs to chunk the
> > value into several notifications, and possibly use some protocol which
> > indicates the total length, parts remaining, etc.
> >
>
> Also client can just do long read in order to read remaining portion of
> characteristic value​
>
> ​- this is what ATT spec suggests.
> It depends on actual use case, but this way client may be able to decide
> whether it should read remaining portion of value or skip it, e.g. some
> flags can be placed at the beginning of the characteristic value and they
> will be always sent in notification.
>
> Best regards,
> Andrzej
>

Re: Question regarding exchanging long characteristic values over BLE

[Christopher Collins]

On Tue, Jul 25, 2017 at 10:46:32AM -0700, Pritish Gandhi wrote:
[...]
> Ah I see! I was assuming a call to access_cb() on a read to completely read
> the value of a characteristic so I was using that to trigger an event which
> would change the value of that characteristic again. I guess that
> assumption is incorrect and I should probably read the MTU of the channel
> and know for sure how many reads would be required before the entire value
> is read.

I do see how it would be useful to know when a characteristic has been
fully read.  I don't think there is any good way to know whether a read
is partial or full in the current stack.

Originally, the stack would handle partial reads by asking the
application for the specific portion of the characteristic being read.
If this were still the case, the application would know when the final
portion is being requested.  However, it was deemed that this interface
made the application deal with too many details, so we changed it to how
it is today.

It would be a fairly minor change to have the stack indicate:
* the offset being read, and  
* the maximum amount of data that the response might include.

The application could ignore this information except in cases like yours
where it needs to know when a characteristic has been fully read.

> Is this true for notifications too? If I need to send notifications for
> that long characteristic value, will my access callback be called several
> times based on the MTU of the connection?

Unfortunately, Bluetooth restricts notifications to a single packet.  If
the characteristic value is longer than the MTU allows, the notification
gets truncated.  To get around this, the application needs to chunk the
value into several notifications, and possibly use some protocol which
indicates the total length, parts remaining, etc.

> I guess I should handle the case that I cannot negotiate a high enough MTU
> with the peer even though there's a slim chance.

If you have no control over which devices will connect to your
application, then I'm afraid you're right.

Chris

Re: Question regarding exchanging long characteristic values over BLE

[Christopher Collins]

Hi Pritish,

On Mon, Jul 24, 2017 at 04:25:30PM -0700, Pritish Gandhi wrote:
> Hi All,
> So I'm trying to understand who is responsible for handling chunking the
> characteristic value in case the MTU of the connection is smaller than the
> characters ic value size.
> 
> So for example, if I have a characteristic value of size 100B. When an iOS
> App connects to my device and reads the characteristic value it gets the
> entire value in a single read.
> It seems like this may be the case because iOS automatically triggers an
> MTU exchange.
> 
> However, when the Android App tries to read the characteristic value,
> multiple reads are triggered (5 reads for 20B each). This causes the GATT
> server access_cb() handler to be called 5 times for the "same"
> characteristic read.
> 
> My questions are:
> a) Isn't the BLE stack supposed to handle chunking the characteristic value
> of 100B to 5 x 20B reads and supplying them to the Android App?
> 
> b) Does the Application have to save state for each read? check what the
> session MTU is and expect those many callbacks to occur before knowing that
> the entire characteristic value has been read?
> 
> c) Is an MTU exchange required for characteristic values > 20B to work
> correctly?
> 
> d) Is it the responsibility of the peripheral or client to initiate the MTU
> exchange?

The Read Long Characteristic Values GATT procedure consists of a
sequence of Read Blob Request ATT commands.  Each of these commands
specifies the offset to begin the read from.

The NimBLE host does not cache any attribute data after receiving a Read
Blob Request.  Instead, the stack asks the application for the full
attribute value for each read request that it receives.  The stack
extracts the requested portion of data from the full value and includes
it in its response to the peer.

So, your characteristic access callback will get called several times
when a peer performs a long read.  The correct behavior is for the
application to retrieve the full characteristic value each time.

I think this answers the above four questions, but please let me know if
something is missing or unclear.

> e) Is is possible for certain BLE stacks/phones to NOT support increasing
> the MTU?

Yes, it is possible, though unlikely.  The spec only requires devices to
support the default ATT MTU: 23.

Chris