I just found this out. There is no penalty for early withdrawal on DIRECT contributions. You are correct that only the interest is penalized if you take that out early. There IS a 5 year hold (meaning if you take it out before 5 years you incur the 10% federal penalty) on backdoor ROTH IRA conversions. Each conversion has its own 5 year timeline as well. I confirmed this with Vanguard over the phone.