In this article, we study the so-called rectifying curves in an arbitrary dimensional Euclidean space. A curve is said to be a rectifying curve if, in all points of the curve, the orthogonal complement of its normal vector contains a fixed point. If this fixed point is chosen to be the origin, then this condition is equivalent to saying that the position vector of the curve in every point lies in the orthogonal complement of its normal vector. Here we characterize rectifying curves in the $n$-dimensional Euclidean space in different ways: using conditions on their curvatures, with an expression for the tangential component, the normal component, or the binormal components of their position vector, and by constructing them starting from an arclength parameterized curve on the unit hypersphere.