Django doesn't have a built in way of doing this that I'm aware of but I can think of a couple of ways you could do something once, rather than having to manually modify lots of fieldsets.
The other approach would just involve python. Normally you just specify the fieldsets attribute in the admin as a tuple. But you could specify it as an imported function which takes the usual tuple as an argument. In your settings file you could specify a list of fieldnames you want to hide. You then need to write a function that returns a modified tuple, moving any fields that match one of your fieldnames into a new fieldset along with the collapse class.
fieldsets = hide_fields(
This might end up being interpreted as the following, assuming content is in the settings file as something you want to hide:
fieldsets = (