It’s not been widely known that the IRS has long permitted retirement accounts such as IRAs to hold gold and other precious metals. That’s because most retirement accounts are administered by bank and brokerage companies, who have no incentive to allow client to diversify into asset classes that are not within their expertise.