When I run this code "FDx = Empty sys: 0 by 1" and FDy says the same thing. I think it because there is a Z component to M. I found that when the Z component is zero it gives a number for FDx and FDy. Does anyone know how to fix this?

R_AR = [0,7,5]
R_AB = [0,0,12]
F = [6,9,-1]

digits(4)
syms FDx FDy FDz;
M = vpa(cross(R_AB,[FDx, FDy, FDz])) +  vpa(cross(R_AR,F));
[FDx, FDy] = solve(M)

[Ramnarayan Krishnamurthy]

The equations you are trying to solve are inconsistent. So, there exists no solution that satisfy them. That is the reason FDx and FDy are empty.

If you want to compute FDx and FDy despite this, you could ignore the 'z' terms and then call 'solve'. But, again this system is inconsistent.