This was discussed on IRC, we're disabling the garbage collection scripts which we think are causing the filesystem to remount ro.

The delete part of koji-gc, the koji-prunesigs, and the koji-directory-cleanup have all been disabled in puppet.