Jenny! I have experienced the same scum, soap, mildew, random crud caused by bath toys! I, however, have a very quick and easy solution.
The first thing we do is divide the toys into three different sets. This way the toys stay "new" and it is exciting to get out a new batch. Also 2/3 of them are dry at all times.
I have to admit that after our move, we did have separate bathrooms. I know I am fortunate, but I LOVE it. However, when we did share we made sure he got his bath at night and we had showers in the morning. By morning the bath toys were dry(ish) and could be removed while we showered.
Third, and most important, I bought a GIANT colander. It lives in the tub with the bath toys.