Should the kth nearest neighbour loss decrease as k increases?

I'm using ClassificationKNN.fit to train my classifier on some data. I've tried changing the number of neighbours to obtain the smallest loss, but as I increase the number of neighbours, the loss increases. I've tried different datasets and some of the example datasets, but every time it's the same.

I've following the commands on the 'Classification Using Nearest Neighbours' page:

load fisheriris
X = meas; 
Y = species; 
mdl = ClassificationKNN.fit(X,Y,'NumNeighbors',4); 
rloss = resubLoss(mdl)

Should I be looking at the cross validated loss instead? I've tried lots of different sized datasets and every time I get the best results with one neighbour when testing.

Many Thanks!

Answer 1

[Ilya]

Yes, you should be looking at the cross-validated loss.