
Hello Erkut,
The issue you're encountering is because “conv()” function in MATLAB is intended for numerical arrays, not symbolic expressions. And the variable “r” which holds the piecewise function is a symbolic expression.
As far as I know, MATLAB does not directly support convolution for symbolic expressions. So, you will have to perform the convolution manually following the convolution expression:

You may refer to below code for better understanding:
syms t tau;
a = 10;
b = 4;
r = piecewise(a < t < (a + b), (a + b), 0);
% Perform symbolic convolution
conv_result = int(r * subs(r, t, t - tau), tau, -inf, inf);
disp(conv_result);
Also attaching the documentations of symbolic substitute “subs()” and integration “int()” functions for reference: