The $X$-coordinates of Pell equations and Padovan numbers

In this paper, we show that there is at most one value of the positive integer $X$ participating in the Pell equation $X^2-dY^2=k$, where $k\in\{\pm1,\pm4\}$, which is a Padovan number, with a few exceptions that we completely characterize.