I figured it out; the key is to not plot with categorical axis too early. I did something like
plot(1:34, 2*[1:34],'ko')
set(gca,'XTick',1:34)
hold on, plot([3.5 3.5],[-0.5 0.5],'k')
hold on, plot([6.5 6.5],[-0.5 0.5],'k')
hold on, plot([20.5 20.5],[-0.5 0.5],'k')
hold on, plot([30.5 30.5],[-0.5 0.5],'k')
set(gca,'XTickLabel',string(xNames))
xtickangle(45)