The multi-layer network below is being trained using backpropagation. The curren

Question

The multi-layer network below is being trained using backpropagation. The current input/output pair is x_p(k) = (1.0, 1.0, 1.0)^t and d_p(k) = (0.0, 0.5, 1.0)^t. The weights and node outputs are given in the table below (note that, for ease of making the table, I'm using the letter "w" to represent all weights since I've numbered the nodes sequentially). Assume the sigmoid activation function (logistic function) for each node with a = 1. Using a learning rate of 0.1 and a momentum term of 0.4, compute delta _6(k), delta _4(k) and w_64(k + 1) (assume that the previous weight change was 0.08). First, write out the backpropagation formula

Explanation / Answer

t is the target

units[l] is the number of units in layer l

n[l][i] is unit i in layer l
n[l][i].output is the output
n[l][i].delta is the delta
n[l][i].weight[j] is weight j
ek is the learning constant
adapt() {
int i,j,k,l;

for(l=layers-1;l>=0;l--)
    for(i=0;i<units[l];i++)
      if(l==layers-1)
        n[l][i].delta=
          ek*n[l][i].output*
          (1.0-n[l][i].output)*
          (t[i]-n[l][i].output);
      else {
        n[l][i].delta=0.0;
        for(k=0;k<units[l];k++)
          n[l][i].delta+=
            n[l+1][k].delta*
            n[l+1][k].weight[i];
        n[l][i].delta=n[l][i].delta*
          ek*n[l][i].output*
          (1.0-n[l][i].output);
       }
  

for(l=layers-1;l>=1;l--)
    for(i=0;i<units[l];i++)
      for(j=0;j<weights;j++)
        n[l][i].weight[j]+=
           n[l-1][j].output*
           n[l][i].delta;
    
  
   for(i=0;i<units[0];i++)
     for(j=0;j<weights;j++)
       n[0][i].weight[j]+=
         input[j]*n[0][i].delta;
}
When this algorithm is applied to the XOR we get the following output.

iteration no 0, inputs 0 1, target 1, output 0.477995
iteration no 20, inputs 0 0, target 1, output 0.447816
iteration no 40, inputs 1 0, target 0, output 0.450292
iteration no 60, inputs 0 0, target 1, output 0.549096
iteration no 80, inputs 1 0, target 0, output 0.460706
iteration no 100, inputs 0 0, target 1, output 0.507636
iteration no 120, inputs 0 1, target 1, output 0.571619
iteration no 140, inputs 1 0, target 0, output 0.451493
iteration no 160, inputs 0 1, target 1, output 0.570574
iteration no 180, inputs 0 0, target 1, output 0.575979
iteration no 200, inputs 0 1, target 1, output 0.744079
iteration no 220, inputs 1 0, target 0, output 0.233541
iteration no 240, inputs 0 1, target 1, output 0.755600
iteration no 260, inputs 1 1, target 0, output 0.185273
iteration no 280, inputs 0 1, target 1, output 0.788309
iteration no 300, inputs 1 1, target 0, output 0.167068
iteration no 320, inputs 1 0, target 0, output 0.123461
iteration no 340, inputs 1 1, target 0, output 0.132892
iteration no 360, inputs 1 1, target 0, output 0.133583
iteration no 380, inputs 1 1, target 0, output 0.116641
iteration no 400, inputs 1 0, target 0, output 0.088269
iteration no 420, inputs 0 0, target 1, output 0.861810
iteration no 440, inputs 1 1, target 0, output 0.102406
iteration no 460, inputs 1 0, target 0, output 0.080179
iteration no 480, inputs 1 0, target 0, output 0.075584
iteration no 500, inputs 0 0, target 1, output 0.884442
iteration no 520, inputs 0 0, target 1, output 0.892789
iteration no 540, inputs 0 1, target 1, output 0.923969
iteration no 560, inputs 1 0, target 0, output 0.064146
iteration no 580, inputs 1 1, target 0, output 0.071938
iteration no 600, inputs 1 1, target 0, output 0.075764
iteration no 620, inputs 1 1, target 0, output 0.074536
iteration no 640, inputs 1 1, target 0, output 0.069014
iteration no 660, inputs 1 1, target 0, output 0.066534
iteration no 680, inputs 0 0, target 1, output 0.918422
iteration no 700, inputs 0 0, target 1, output 0.924860
iteration no 720, inputs 1 1, target 0, output 0.065864
iteration no 740, inputs 1 0, target 0, output 0.052634
iteration no 760, inputs 0 0, target 1, output 0.927081
iteration no 780, inputs 1 0, target 0, output 0.050964
iteration no 800, inputs 0 1, target 1, output 0.948869
iteration no 820, inputs 1 0, target 0, output 0.049082
iteration no 840, inputs 1 0, target 0, output 0.048074
iteration no 860, inputs 1 1, target 0, output 0.057916
iteration no 880, inputs 1 1, target 0, output 0.056088
iteration no 900, inputs 0 1, target 1, output 0.954659
iteration no 920, inputs 1 1, target 0, output 0.057337
iteration no 940, inputs 0 0, target 1, output 0.944243
iteration no 960, inputs 1 0, target 0, output 0.045653
iteration no 980, inputs 0 0, target 1, output 0.946199

Related Questions

Navigate

• Browse (All)
• Browse T
• Subjects

---