When in doubt, don’t difference
In my experience, differencing data to achieve stationarity is a fraught exercise. It destroys a significant amount of information, and only makes sense when the data is a true random walk or close to it. If you have any doubt that the data is in fact a random walk, I would be extremely careful about differencing it.
This is especially true in the case of multivariate series in which there may be cointegration. Differencing such series destroys the long-run equilibrium / correction behavior in the data, leading to poor inference and inferences.
I would also be careful about differencing when you either don’t have much data or you know there’s likely a lot of measurement noise in the data. When you difference such data, you only amplify whatever noise is already present, to the point where the noise can overwhelm the signal. I don’t know a good rule of thumb, but I’d be wary of differencing when you have fewer than 100 datapoints.
When in doubt, use another method, like polynomial detrending or a more sophisticated method like the Hamilton filter or HP filter. Per @canovaFAQHowExtract, polynomial detrending is actually quite robust.
References
when cointegration may be present, simply getting rid of nonstationarity by differencing individual series so that they are all stationary throws away vast amounts of information and may distort inference. – Christopher Sims, Comments and discussion: Disentangling the channels of the 2007–2009 recession