am I in control of that or do we have to wait until end of vote date?
On the conclusion of each voting proposal window of ~3 days (accepted/rejected), the staked votes will be tallied and checked if quorum has passed.
While you can vote on any number of goverance proposals; a voting timelock of ~3 days also applies on your staked BPT, so you will only be able to withdraw your stake, after the last proposal you voted on has already reached verdict.