39

I was looking at the Noldor House of Finwë descendancy and on one side it shows Arwen, daughter of Elrond as we all know. But I didn't know this part: the other side breaks off into Aragorn as a descendant of Elros - Elrond's twin brother. Does that make them cousins?

44

The short answer is yes, they're directly related (first cousins) albeit separated by a very considerable age gap.

This handy family tree from The Encyclopaedia of Arda should be helpful to you:

Arwen Aragorn

Please note that the dotted line between Elros and Aragorn represents many thousands of years of lineage on Aragorn's side of the family.