Yeah, today I woke up with this silly weird question, so which one was first? I went surfing the net and found out a bunch of articles about it, Ha! among other sources like yahoo, turns out many got the same troubling thought, makes me wonder about how are we conected, not the same yet so similar, but that is a whole other subject ,so in case you have the very same question or just want to know what to tell your kids if they ask here is the answer:

The citrus definitely got named first. The earliest recorded use of orange the fruit in English is from the 1300s and came to us from the Old French orenge, adapted from the Arabic nāranj, from the Persian nārang, from the Sanskrit nāranga ("orange tree"). The Sanskrit word's origin is unclear, but it might come from a Dravidian word meaning "fragrant."

The word's use as a color name doesn't crop up for another 200 years, in the early 1500s. English speakers probably didn't have a specific name for the color until the fruit was widely available in their markets and inspired one. Before then, linguists believe people generally referred to orange as "yellow-red," ġeolurēad in Old English.

Well I have learn something and hope you too 

Cheers :)