Welp, reviving this. With the latest firmware the NES runs in Line5x somewhat reliably.
I have to start every game in 3x mode, but then I can switch the 5x mode and the image stays (only in 1600×1200 mode, not 1920×1200 or 1920×1080). However, certain games still drop out. Super Mario Bros. works perfectly, but Mega Man flickers. Both games display the same resolution/refresh data on the OSSC’s screen.
Is there something that should differ between the two? Maybe that’ll help me figure out if there’s anything I can change or if it’s just my TV. I have a feeling it’s not the TV since the refresh rate should be the same between 3x and 5x modes.