AV53C1-USB-BLUETOOTH older linux kernel use

I need some suggestions from someone who knows some things about linux modules.

So briefly my situation: I have a pretty bare bones Linux system, with 4.19.102 kernel version, so by default, the dongle does not support it. It only has basic commands, no systemd or package manager, I need to install everything manually if I need to.
I patched the bluetooth module so that it will load up the correct RTL8761BUE firmware for the dongle, and it somewhat works. I only need inquiry scanning, nothing else, and that worked for me. But after some time, I got timeout errors for every command for the dongle, resulting a fully non-responsive dongle. The only way to solve is rebooting or reattaching it to the system, but I cannot reattach it manually unfortunately, I have to solve it just in software. And lately I got into an error that was not solved with just a reboot, so now I got some big concerns I haven’t had before.

What are the kernel specific features that are needed for the RTL8761BUE chipset? I know the documentation says 5.19 kernel at least, but now I have to make it work with an older kernel. I have tried pretty much all of the available methods I could find on the internet outside deeper patching of the kernel module, but none of them worked.

What I have installed:

  • Bluez 5.56
  • bluetooth, btusb ans btrtl modules from the 4.19.102 kernel (and patched to find the correct firmware version)
  • firmware for the dongle with teh date 2021.07.16

I only start a script, that has “hcitool inq” in loop, and read out the results using btmon in another terminal.

one example of what I got as a result:

I got a script running "hcitool inq" command in loop. After some time, the module get into a state where for any command, I only got a timeout error. So I cannot hciconfig down, up or reset the dongle for example

[    9.336661] Bluetooth: hci0: RTL: rtl: examining hci_ver=0a hci_rev=000b lmp_ver=0a lmp_subver=8761
[    9.338673] Bluetooth: hci0: RTL: rom_version status=0 version=1
[    9.338694] Bluetooth: hci0: RTL: rtl: loading rtl_bt/rtl8761bu_fw.bin
[    9.361036] Bluetooth: hci0: RTL: rtl: loading rtl_bt/rtl8761bu_config.bin
[    9.364907] Bluetooth: hci0: RTL: cfg_sz 6, total sz 20522
[   28.956534] Bluetooth: hci0: command 0x0401 tx timeout
[   45.926272] Bluetooth: hci0: command 0x0401 tx timeout
[   62.556275] Bluetooth: hci0: command 0x0401 tx timeout
[   79.196275] Bluetooth: hci0: command 0x0401 tx timeout
[   95.836250] Bluetooth: hci0: command 0x0401 tx timeout
[  112.486213] Bluetooth: hci0: command 0x0401 tx timeout

I also got this error in one of the cases, as far as I know, this is still a timeout error:

[   11.356428] Bluetooth: hci0: command 0x1001 tx timeout
[   19.686444] Bluetooth: hci0: RTL: HCI_OP_READ_LOCAL_VERSION failed (-110)

Does anyone have any suggestions how to make it stable enough just for inquiry scans? Or at least can someone give me some suggestions what to avoid so that I get the least errors and freezes? (unfortunately “not using it at all” is not an answer for me:D)

Hello @balint and thank you for your interest in our Community.

This setup isn’t officially supported since the kernel version is below the required 5.19 minimum for our AV53C1-USB-BLUETOOTH (as you have yourself confirmed from your post).

That being said, from the errors you shared, you might want to try:

  • Using btmgmt or bluetoothctl instead of hcitool for better stability.
  • Updating BlueZ (5.56) to a newer version, as recent releases (5.64+) include fixes for Realtek quirks.

If these steps don’t help, the recommended course of action is to ensure you’re running a compatible kernel for full functionality.

I hope this helps!

Emmanuel B.
StarTech.com Support

Hi Emmauel,

Thank you for your answer. I have just recently realized while checking the content of the btusb kernel module (even the latest one), that the kernel does not recognize the exact chipset:

The startech bluetooth dongle has the following values:
idProduct: 8761
idVendor: 2c0a

And even though there is a match for the product ID, but not for the vendor ID! Does that mean, that there is no firmware loading needed for the modul while initialising?

Hello @balint,

It is possible that if the vendor ID isn’t recognized by the kernel module, the driver may not trigger the firmware loading process automatically. Since the kernel you’re using isn’t officially supported by this adapter’s chipset, the behavior you’re seeing is likely related and we can’t guarantee stability in that environment.

The best steps you can try are:

  • Try btmgmt or bluetoothctl instead of hcitool.
  • Update BlueZ to the latest version, as newer releases include fixes for Realtek devices.

If these don’t resolve the issue, the recommended solution is to use a kernel version that meets the compatibility requirements for our AV53C1-USB-BLUETOOTH.

I hope this helps!

Emmanuel B.
StarTech.com Support