Light speed question?

Ive come to accept the whole forward in time thing, even though it still doesnt make sense.. If you are moving that fast to something .5 lightyears away, and you go there and back, it should only take a year, not the lifespan of a civilization as I hear way too often. This whole thing makes it seem like the person moving that fast would be the center of everything.. Everyone else withers and dies while you go flying for 30 seconds? I know the whole "relative time thing", but travelling a lightyear at the speed of light should reasonably only take 1 year our time , the way people talk about this, its like you go so incredibly slow, but thats far from the case, how would take longer going somewhere the faster you go?