Where do you get all that stuff? Great question. See the Flour Table for some hints.
I'll never forget the response I got when I called ADM Archer Daniels Midlin company. I pulled their name off the outside of one of my bags of flour and called them to see if I could buy flour from them.
"Sure, Do you want a train or truckload?"
It's not the easiest to find our unique g-f flours, but the internet is again our friend and the freezer is our storage bin.
Tapioca Flour/Starch I purchase at Asian stores. It is manufactured primarily in Indonesia and Brazil from a casava root and so it's fairly safe in whatever form. Also called casava flour. I have also purchased it in Utah in 50 lb bags from WINCO. It came in a Bob's Red Mill bag, but is lots cheaper than from their plant in Washington.
Sorghum Flour-- I purchase the grain from Twin Valley Mills in Nebraska and pick up when I'm driving from Utah to Tulsa annually. Since I've stopped driving and started flying, it is expensive $57.00 or so to buy a bucket of grain and have it shipped, but it still beats the three bucks a pound in the stores.
Corn Starch and Potato Starch (NOT POTATO FLOUR) I purchase in bulk, but don't buy more than you can use in a year, as these do have a shelf life when they no longer thicken.
I purchase and grind the 50 lb. bags of popcorn for corn meal and corn flour.
Bob's Red Mill is a great supplier, but have gotten more and more expensive with the market increase. If you can buy from them in bulk, that helps. Amazon also offers deals.
Buckwheat, Brown Rice, Millet, I purchase "food-grade" grain and grind my own. When a recipe calls for Teff I usually swap out for buckwheat. Quinoa and Amaranth I also buy in grain form from Walton feed and grind. I purchase many of these grains from a supplier in Tulsa who purchases bulk from Walton Feed in Montpelier, Idaho and have shipped to Tulsa. http://www.rainydayfoods.com/
I purchase from Honeyville Grains in Utah many grains as they have a $4.49 shipping anywhere in contiguous US. http://store.honeyvillegrain.com/
Auguson Farms used to be a favorite for angel food cake, but they have modified their pricing to reflect the market and now a cake is up to 7 bucks again. Ridiculous as it's only made with rice flour and corn starch.
Reality Bite: Buy a freezer just for grain. My husband bemoans the white double ziplocked bags of grain as the state of ours. "Honey, there is nothing in there I recognize." But back before GF, when the freezer was full of freezer burned meat, veggies, and obscure jars of forgotten strangeness, he couldn't tell then either.
Please add on your favorite bulk supplier in the comments as I am sure it will help many people. Thanks!