Consider an d - nH - c network trained with n patterns for m epochs (a) What is
ID: 3706872 • Letter: C
Question
Consider an d - nH - c network trained with n patterns for m epochs (a) What is the space complexity in this problem? (Consider both the storage of network parameters as well as the storage of patterns, but not the program itself.) (b) Suppose the network is trained in stochastic mode. What is the time complexity? Because this is dominated by the number of multiply-accumulates, use this as a measure of the time complexity. (c) Suppose the network is trained in batch mode. What is the time complexity?Explanation / Answer
1a)
Space Complexity:
Total number of weights in the networks is (d*nh + nh*c). The amount of storage for n patterns is n*d.
1b)
In stochastic mode, we choose pattern randomly and compute w(t+1) = w(t) + delta(w(t)) until stop criteria is met. Each iteration involves computing delta(w(t)) and then adding it to w(t). Computation of delta(w(t)) takes [c*(2*nh+1)+ 2*(d+c+1)*nh] operations.
Therefore Time taken to compute w(t) + delta(wt) = [d*nh + nh*c ] + [c*(2*nh+1)+ 2*(d+c+1)*nh]
= 3* d*nh + 5* nh*c + c + 2* nh
For m epochs it would be m multiplied by above expression, so answer is m*(3* d*nh + 5* nh*c + c + 2* nh).
1c)
In Batch mode, number of iterations = n*m and thus time complexity is n* m*(3* d*nh + 5* nh*c + c + 2* nh)