Threading and transmission robustness: DBTManager, HCIHandler: Synchronize sendWithReply(..)

Overall experience w/ 'reply confusion', e.g.
- START_DISCOVERY-1 -> CMD_COMPLETE (STOP_DISCOVERY-2)
- STOP_DISCOVERY-2 -> CMD_COMPLETE (START_DISCOVERY-1)

Other command/reply confusion also occured, i.e. CONNECT/DISCOVERY, etc.

As the API and user space will operate multi threaded
and event based, the replies would need to be matched
to their respective commands.

Event though sending out commands is synchronized,
orderly replies received in the ringbuffer
would rely on the device/adapter to act synchronous.
This is not the case!

HCI replies may occur at a later time, i.e. CMD_STATUS being returned immediately
for a pending CMD_COMPLETED.
The BlueZ Manager communication aggregates HCI commands and hence acts similar.

For now we synchronize all sendWithReply(..)
so that all commands and their complete replies must be finished before the next.

To remove this enforced synchonization, we could replace the ringbuffer
with a list to find matching replies for the waiting commands.
In case the synchronization proves to be a performance burden (scaling),
we might chose such approach.

2 files changed, 2 insertions, 0 deletions

diff --git a/api/direct_bt/DBTManager.hpp b/api/direct_bt/DBTManager.hpp
index eeea4c0a..0a6acac4 100644
--- a/api/direct_bt/DBTManager.hpp
+++ b/api/direct_bt/DBTManager.hpp
@@ -88,6 +88,7 @@ namespace direct_bt {
             std::atomic<bool> mgmtReaderShallStop;
             std::mutex mtx_mgmtReaderInit;
             std::condition_variable cv_mgmtReaderInit;
+            std::recursive_mutex mtx_sendReply; // for sendWithReply
 
             /** One MgmtAdapterEventCallbackList per event type, allowing multiple callbacks to be invoked for each event */
             std::array<MgmtAdapterEventCallbackList, static_cast<uint16_t>(MgmtEvent::Opcode::MGMT_EVENT_TYPE_COUNT)> mgmtAdapterEventCallbackLists;
diff --git a/api/direct_bt/HCIHandler.hpp b/api/direct_bt/HCIHandler.hpp
index 432856b6..61c41e4f 100644
--- a/api/direct_bt/HCIHandler.hpp
+++ b/api/direct_bt/HCIHandler.hpp
@@ -100,6 +100,7 @@ namespace direct_bt {
             std::atomic<bool> hciReaderShallStop;
             std::mutex mtx_hciReaderInit;
             std::condition_variable cv_hciReaderInit;
+            std::recursive_mutex mtx_sendReply; // for sendWith*Reply, process*Command, ..
 
             void hciReaderThreadImpl();