repeat fmincon get more time at each try!!!

I have 10000 minimization which are part of model fitting the different between optimization is just observation which are randomly permute. I use fmincon for optimization. My problem is why elapse time by fmincon grow slowly from 5 sec to about 19 sec???

OptimizerOptions = optimset('Display','off','MaxIter',100,'TolX',0.01,'Algorithm','interior-point');
Fitted=zeros(10000,m);
for i=1:10000
  OBJ=@x ModelError(X,Observation(i,:));
  InitialX = Low+(High-Low)rand(1,m);
  tic
     Fitted(i,:)=fmincon(OBJ,InitialX,[],[],[],[],Low,High,[],OptimizerOptions);
  toc
end

this code is the part of my code which the elapsed time changed. did any one have such problem? or any tips?