Welcome. Looks like you've got a good skill set between you!
For track bed, many folk use 'crusher fines' or 'rock dust'. Same stuff used under flagstone paths. 4" well tamped down will provide a solid base. I found a 4" trenching shovel at Home Depot and used that to make the trench for the ballast.
Others use concrete, or run the track on blocks/bricks.
Finally, you may want to elevate it a little. A raised garden reduces back strain and cuts damage to the track from little feet. But that means a lot of walls and backfilling. Alternatively, a raised (3'-ish) wooden platform will make life easy though it is less like a 'garden' railway.
There are dozens of threads here on this topic. The easiest way to find them is to google "site:mylargescale.com track bed" or something similar.