I had the same problem but I solved it. what happens is- every time you switch to bangla, the windows input language changes to "bengali-India" and when you press F12 again- it doesn't switch back to English. Also, avro keyboard toggle is for specific windows/apps, whereas when the input language changes, it changes everywhere. you can press shift+alt to toggle the input language, but it still is a hassle.
so, here is how I solved it: first uninstall avro completely. then while reinstalling it- uncheck "change input language with keyboard mode"(or something like that)- option- which makes windows change it's input language. then continue with the installation. oh, after you are done, don't forget to go back to settings and switch the language to Bengali(Bangladesh).