I ordered a Toro, probably the problem goes away if I send the 480p through RGB.
I’m sorry, but I’m compelled to nitpick here. I think you mean if you send 480p through SCART.
Your Dreamcast is outputting RGB, whether you’re using a VGA box or cable; the difference is that the default output (with a VGA box or straight cable) is RGBHV, and the Toro lets you select between RGBHV (over the HD15 only, I believe) or RGBS (over both HD15 and SCART).
I’m not sure if the Toro processes the sync at all when in RGBHV mode (and I assume your existing VGA box simply passes the raw RGBHV through), but its sync combiner (for converting to RGBS) may help work around your sync issues.