NXC: MT-safe BT network?

[HaWe]

hi,
is already a multitasking-safe BT network available for NXC which does not block sending/receiving queues?
It should be able to
- initialize remote sensors (and remote sensor multiplexers)
- read remote sensors (and remote sensor multiplexers)
- control remote motors (and remote motor multiplexers) including related encoder counters
- send and receive any variable value calculated by any a remote procedure

[mightor]

I don't think so, but I am sure the community will be very grateful should you feel called upon to write one yourself.

- Xander

[HaWe]

as you mightor recall, I already tried it (for more than 3 years) - but I failed
(it was for my chess robot - (see also here) which worked only satisfactory with 1 remote NXT, and even this remote NXT did only work BT-single-tasked, not BT-multitasking-safe)

(actually my question was if there existed one and not if anybody thinks that there probably might be none) and that or if I should write my own one

[mcsummation]

You may recall that some of us tried to assist you with your Bluetooth problems. However, you adamantly refused our advice and thread-safe code.

[HaWe]

no, you're wrong,
my wish always was to have multi-threading-safe BT access, your advice was to avoid multi-threaded access.

On the other hand, e.g. the JHS BT commands pretended to be MT-safe, I tried them out, but they turned out not to be MT-safe as I demonstrated.
But more than this - not even all the single-tasked commands have been working correctly, they very often became corrupted or blocked or dropped - or have been sent / received several times redundantly by no reason of the programming code, as I also demonstrated ( I guess it's because of a BT mutex or timing issue, maybe the original fw BT speed is simply too slow for this).

On the other hand, I remember very well that you yourself have required MT-safe BT access for the beginning, and it's pretty Janus-headed to blame me that I'm stuck with this requirement.

But those are old stories, we shouldn't warm them up.

Anyway, my urgent need still is to have a stable and real multi-threading-safe BT access (sending + receiving BT messages from several different running independed threads), and this is the subject of the question of my TOP.
But I interpret your post so that you also don't know a solution in the sense of the TOP question

is already a multitasking-safe BT network available for NXC which does not block sending/receiving queues?

A way might be to internally queue-up all in- or outgoing MT-based BT messages by a superordinate task using sort of a FIFO message stack before physically sending/receiving, and after that sending /receiving them by working off the stack.
But it must be very quick though: I have to poll over all 20 sensors (touch, analog, i2c, 3 muxers) plus at least 4 encoder values on 3 remote NXTs in real-time (each and every sensor randomly 20 times each second), I have to control parallely 8 remote motors with max 100ms delay, and intermediately I have sometimes to transfer e.g. single numbers and up to 40x40 arrays of char from 1 NXT to an other one (numbers sometimes quickly, large arrays not time-sensitive).

[linusa]

no, you're wrong,
my wish always was to have multi-threading-safe BT access, your advice was to avoid multi-threaded access.

IIRC, the guidance by John and others was to avoid calling BT functions from across different threads at all times. Having done a lot of socket, serial and BT programming, this can generally be a good idea in certain software environments! The real question is, on a higher level: Is it possible to get fast and asynchronous BT high level functions? And yes, the implementation answer will contain threads, of course. But nobody said it has to be possible to talk to the BT subsystem from different threads!

A way might be to internally queue-up all in- or outgoing MT-based BT messages by a superordinate task using sort of a FIFO message stack before physically sending/receiving, and after that sending /receiving them by working off the stack.

And that is EXACTLY the way to do it! The BT functions are a bit fragile and sometimes quirky, yes. But with experience from experiments, you can use them reliably in a single threaded program. I know you did that! The real task now is to wrap all BT access in a single thread. Other threads can then carry out other things -- like polling sensors and motors.

The BT task has to be tight and simple: Just send and receive everything ASAP, in a round-robin fashion when talking to multiple NXTs. To keep the BT task utilized at full capacity, a data structure has to buffer incoming and outgoing packets, of course. You can use a queue for FIFO (or a priority queue for more fancy stuff), which I would recommend.

The BT task doesn't have to worry about forming packets or anything else. Just get data from the BT buffer to a user defined format, and vice versa.

Every external access to the incoming and outgoing queues is protected by mutexes!

A possible data structure for packets could be

Code: Select all

IncomingPacket:
  - Sender (NXT/Connection Nr.)
  - Data (Bytes/String), with length

OutgoingPacket:
  - Recipient (NXT/Connection Nr.)
  - Data (Bytes/String), with length

I know this is trivial, but this is all it takes. Even when it's not always possible to work easily with structs of arrays or arrays of structs, and without real pointers, you can still implement a small queue/ring buffer on your own! I'd personally "ignore" the mailbox system and not use it as part of this custom queue.

The queue push/pop operations to the queues need to be mutex protected.

The BT task then looks like this:

Code: Select all

repeat 
{
  while (!outgoingQueueIsEmpty())
  {
    sendBTPacket(  pushFromOutgoingQueue() )
  }
  
  repeat
  {
    IncomingPacket packet = receiveBTPacket();
    if( !packetIsEmpty(packet) )
    {
      pushToIncomingQueue(packet)
    }
  }
}

Then there are simple high level methods for sending and receiving: send( packet ), packetsAvailable(), packet = receive() (all usable outside the BT task, protected by mutexes). These functions can also return error codes when the queue is full, or drop packets, or block to wait out the bottleneck...

Use benchmarks how many packets per second (RX/TX) are possible to one NXT, two NXTs, three NXTs, and how much bytes/sec. Then you have limits on what is possible!

Other worker tasks can poll sensors and compile packets as fast as they can. Of course this is just the lowest level. The multiplexing and de-multiplexing can be done on top of this -- but it's not in the BT task and hence not a special problem.

But it must be very quick though: I have to poll over all 20 sensors (touch, analog, i2c, 3 muxers) plus at least 4 encoder values on 3 remote NXTs in real-time (each and every sensor randomly 20 times each second), I have to control parallely 8 remote motors with max 100ms delay, and intermediately I have sometimes to transfer e.g. single numbers and up to 40x40 arrays of char from 1 NXT to an other one (numbers sometimes quickly, large arrays not time-sensitive).

Well yeah, you can also add: "And I need 100kb/s and 8 threads and a golden rainbow". These are ambitious goals, and you know it. Saying "I need all of this" doesn't help. Find out what is possible with a tight small BT task, get experimentally verified throughput rates, and work with it!


Ford, could you tell me the "best" BT functions you used to send and receive BT packets, which worked stable in single task mode? I know there are multiple versions out there, but I forgot which are the "official" ones to use...

(it was for my chess robot - (see also here)

Btw, the second video is blocked :-/.

[HaWe]

making the BT commands "multi-threading-safe" was a try by JHS.
But yes, I indeed tried to write also such a single-threaded superordinate BT message stack program as you mentioned, but I failed as well with this appoach.

Always if I tried it the way

Code: Select all

...
wait until response number == x
send BTmessage(abcde)
...
wait until response number == y
send BTmessage(fghijk)
...

the whole program blocked and hung up (to stop only by removing the batteries)

So after all I have no well working BT version yet, but a golden rainbow (or better: a treasure chest full of gold) would be nice though.

In different approaches, over all all the other additional wait states (50ms...200ms) in between needed and waisted too much time, the BT connection is far too shaky and far too slow at all (too poor baud rate, too long idle or switching periods in between). So a faster BT communication protocol would be very much appreciated, sth like a "high(er) speed BT", maybe it's possible by a different firmware.

(Opposite to you and most other advanced users and experts round here, I am only a hobby programmer with no low level programming experience or an IT or computer science education at all ever.)

BTW the 2nd link you couldn't watch is only blocked in Germany courtesy of GEMA, but it's only the "music version" of the 1st one.

So over all, I interpret also your post as: "no, I don't know of any already available multi-threading safe BT network protocol"...(?)

[HaWe]

ps:
the code (receiving data from slaves) where there program hung (blocked) was:

Code: Select all

//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
task ReceiveSlaveData()    // NOS = Number of Slaves
//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
{
   string _msg;

   while(true)   {
     if (_NOS_>=1) {
       do {Wait(1);} while(ReceiveRemoteString(INBOX_11, true, _msg) != NO_ERR);
       ParseBTResponseMsg(0, _msg) ;
       Wait(20);
     }

     if (_NOS_>=2) {
       do {Wait(1);} while(ReceiveRemoteString(INBOX_21, true, _msg) != NO_ERR);
       ParseBTResponseMsg(1, _msg) ;
       Wait(20);
     }

     if (_NOS_==3) {
       do {Wait(1);} while(ReceiveRemoteString(INBOX_31, true, _msg) != NO_ERR);
       ParseBTResponseMsg(2, _msg) ;
       Wait(20);
      }

     Wait(20);
   }
}

it didn't block but the data got muddled-up, broken or dropped if I used this scheme:

Code: Select all

//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
task ReceiveSlaveData()    // NOS = Number of Slaves
//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
{
   string msg;

   while(true)

   {
     if (_NOS_>=1) {
       until(ReceiveRemoteString(INBOX_11, true, msg) == NO_ERR);
       ParseBTResponseMsg(0, msg) ;
     }

     if (_NOS_>=2) {
       until(ReceiveRemoteString(INBOX_21, true, msg) == NO_ERR);
       ParseBTResponseMsg(1, msg) ;
     }

     if (_NOS_==3) {
       until(ReceiveRemoteString(INBOX_31, true, msg) == NO_ERR);
       ParseBTResponseMsg(2, msg) ;
      }

     Wait(100);

   }
}

similar it was when sending to slaves:
SendRemoteString worked but led to multiple redundant messaging / receiving from the BT queue (e.g., started a slave's subroutine by the master 3 or 4 times immediately after the other although only was initiated once by the master and thus muddled up temporary variable values on the slave),
if I waited in between for an acknowledge byte to be returned from the slave (and checked it before sending again in case of failure) the whole program slowed down to 1-2minutes of waiting between 2 BT commands (like e.g., lift or lower the robot arm or open or close the claw) or it also hung up completely.

But all these examples have been used in my chess program - the BT net I need now is for a different project (a map-and-explore service robot with far more sensors).

edit: typo corrected

[mightor]

I think it is safe to assume nobody will write the library for you, so you have three choices:

• Write your own
• Pay someone money to write it for you
• Do without

- Xander

[HaWe]

my question was not about if anybody will do, my question was about if somebody already has or knows because the existing library is inadequate in terms of performance and reliability as I demonstrated and which meanwhile also should have been noticed by others. I meanwhile understood that you don't have or know.