Well, even gradient or diff does not actually differentiate data. They can give you an approximation to the derivative at each data point.
And, yes, you are correct to fear multiple sequential calls to gradient, as it is anapproximation at each step. The problem with numerical differentiation is it is a noise amplifiying process, sometimes called an ill-posed problem.
You don't tell us enough about your problem that I can be sure about the best approach. Is your data equally spaced in x? Is there noise in the process? This makes things more difficult.
There are several ideas you could use. I'd suggest that perhaps a Savitsky-Golay filter might be a good idea, if there is noise, and if the data is uniform in x. Or perhaps if there is no noise, then you might pass an interpolating spline through the points, then differentiate the spline twice, evaluating it at the points. Since you are looking for a second derivative, I'd probably suggest a quintic spline as a good idea. Or if there is noise in the data, then asmoothing or least squares spline canbe of value. Again, you would differentiate it at the end, and you would not incur the cost of recursive numerical differentiation.
If you truly want better advice, I would need to know more about the data, and your problem, as well as have you attach a copy of the data.
