- Wirtinger derivatives: https://en.wikipedia.org/wiki/Wirtinger_derivatives
- Dlgradient: https://www.mathworks.com/help/deeplearning/ref/dlarray.dlgradient.html
How do I compute and visualize complex gradients? (ReLU example)
5 visualizzazioni (ultimi 30 giorni)
Mostra commenti meno recenti
How do I compute and visualize complex gradients?
In the real case I use:
x = -5:0.01:5;
ReLU = max(0,x);
dReLU = gradient(ReLU)./gradient(x);
% visualize:
a = figure;
plot(x,ReLU);
hold on;
plot(x,dReLU);
axis([-1 5 -1 5]);
xlabel('x');
ylabel('y');
legend({'y=ReLU(x)','y=ReLU''(x)'},'Location','northwest');
title('Real-Valued ReLU');
and for the complex case I want to use:
x = -5:0.2:5;
y = x;
[X,Y]=meshgrid(x,y);
z = X+1i*Y;
% Complex ReLU
f1 = figure;
complexReLU = max(0,real(z))+1i*(max(0,imag(z)));
complexReLU_abs = min(6.2, abs(complexReLU)); % 6.2 as upper limit just for nicer colors
sc = surfc(x,y,complexReLU_abs);
sc(2).LineStyle = 'none';
axis([-5 5 -5 5 0 6]);
xlabel('Real(z)');
ylabel('Imag(z)');
zlabel('|ReLU(z)|');
title('Magnitude of ReLU(z)');
colormap jet;
% Derivative of Complex ReLU
f3 = figure;
dcomplexReLU = gradient(complexReLU)./gradient(z);
dcomplexReLU_abs = abs(dcomplexReLU);
sc = surfc(x,y,dcomplexReLU_abs);
sc(2).LineStyle = 'none';
axis([-5 5 -5 5 0 6]);
xlabel('Real(z)');
ylabel('Imag(z)');
zlabel('|dReLU(z)|');
title('Magnitude of dReLU(z)');
colormap jet;
I receive a plot for the derivative of the complex ReLU, which is zero for real(z)<0 and imag(z)>0. But as you can see in the plot of the complex ReLU, this is cannot be true. How do I calculate complex gradients or is there a function that works in the complex as opposed to the gradient() function?
0 Commenti
Risposte (1)
Aditya
il 24 Gen 2024
Hi Julian,
I understand that you're looking to calculate the gradient of a complex function. This process is quite different from that for real-valued functions because standard gradient calculations don't apply to complex numbers. They overlook the Cauchy-Riemann equations, which are necessary to determine if a complex function is differentiable.
In the complex domain, a function is differentiable (holomorphic) at a point if it satisfies the Cauchy-Riemann equations there.
For the ReLU function, which is piecewise linear, the complex derivative does not exist everywhere because it is not holomorphic; it is not differentiable at the points where the piecewise linear sections meet (i.e., the nonlinearity at zero).
However, you can compute a sort of "generalized gradient" by considering the “Wirtinger derivatives”, which are used in complex optimization problems.
Here is how you can modify your code:
% Compute the Wirtinger derivatives
dcomplexReLU_dz = 0.5 * (X > 0) + 0.5 * (-1i) * (Y > 0);
dcomplexReLU_dz_star = 0.5 * (X > 0) + 0.5 * (1i) * (Y > 0);
% Absolute values for visualization
dcomplexReLU_dz_abs = abs(dcomplexReLU_dz);
dcomplexReLU_dz_star_abs = abs(dcomplexReLU_dz_star);
% Visualize the magnitude of the Wirtinger derivative with respect to z
f2 = figure;
sc = surfc(x, y, dcomplexReLU_dz_abs);
sc(2).LineStyle = 'none';
axis([-5 5 -5 5 0 1]);
xlabel('Real(z)');
ylabel('Imag(z)');
zlabel('|dReLU(z)/dz|');
title('Magnitude of dReLU(z)/dz');
colormap jet;
% Visualize the magnitude of the Wirtinger derivative with respect to z*
f3 = figure;
sc = surfc(x, y, dcomplexReLU_dz_star_abs);
sc(2).LineStyle = 'none';
axis([-5 5 -5 5 0 1]);
xlabel('Real(z)');
ylabel('Imag(z)');
zlabel('|dReLU(z)/dz*|');
title('Magnitude of dReLU(z)/dz*');
colormap jet;
Since MATLAB does not provide a built-in method to compute the “Wirtinger derivatives”, the manual implementation is necessary. However, for real-valued functions or functions that can be separated into real and imaginary parts treated as independent variables, you can use “dlgradient” to compute the gradients of each part separately.
For more information on “Wirtinger derivatives” and the “dlgradient” function, you can refer to following documentation link:
Hope this helps!
0 Commenti
Vedere anche
Categorie
Scopri di più su 2-D and 3-D Plots in Help Center e File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!