I've been using one of these on the SWD interface to program it.
Hookup details are here, and also in some posts on this forum.
You can go back and forth. Axoloti binaries live in a few different places. The one you mostly care about is "flash_main" (1MiB@0x08000000). By reading out the existing binary you can save it off, replace it with your program and then reprogram it when you want to go back to the standard code.
BTW - There's also a lump of code in "flash_system" (30KiB@0x1fff0000). It's probably a flash loader, although I haven't been using it or altering it.
Here are a copy of the axoloti binaries I pulled from my board.