Crosby was born in Los Angeles, California, and grew up in Humboldt and Sonoma counties in Northern California, raised by a single mother from an upper-class family who owned a record store in Fortuna, California. His great-grandfather was John C. Crosby, a congressman from Massachusetts, and his ancestry on his mother's side is English and Native American. His father was a lounge guitarist whom he never met.