necroforest wrote:basically, the weights of a neuron forms a line (or hyperplane) through your input space, and the activation function tells you which side of the line you're on (and how far from the line, if you're using a sigmoid function).
if you have a bunch of neurons organized in layers, the first layer chops up your input space into sections and "transforms" your input into a new space, then each layer transforms that space into another one, etc.
gabrielkfl wrote:I've even seen some examples that seem to work, I just can't understand HOW the hell they work. I mean, they're freaky - you start changing the weight values and suddently your network is able to tell apart a '4' and a '5', what the f*ck?
JPlus wrote:I think this boils down to the question "how is it possible that we can train a network to represent some function when starting out with a completely random weight pattern?". The answer is this: when we try an input on a network, we can see how far the output is from what it should have been. We adjust the weights slightly in a direction that would yield a better output. When we repeat this very often with many different examples, we will eventually have nudged the network enough to more or less do what we want.
gabrielkfl wrote:How is it possible for a neural network to identify a pattern based on numeric (weight) values?
samk wrote:gabrielkfl wrote:How is it possible for a neural network to identify a pattern based on numeric (weight) values?
If the weights are mostly zero, with some large positive and negative values thrown in, they effectively will encode a boolean logic circuit. Making them fractional allows learning by gradually adjusting them and tuning to the most typical input while responding in mostly the same way to small deviations.
cogman wrote:For the training, it is much akin to changing each attribute and having the NN see the differences. It mights see "It is brown" "It has a beak" "It isn't flying" "It doesn't have wings" and guess 0.78 that it is a bird. Your response will be to slap the system and say "No, its a platypus" To which it places a higher value on the wing node and flying node, and lower values on the beak and brown nodes. (based on how wrong it is. It should change the values based on how far off the final conclusions, but not all the way"
Jplus wrote:As cogman said, the reason for multiple layers is that you can abstract more. With a single layer network you can typically only answer yes/no questions (yes, it's a bird, no, it's not a platypus, yes, it might also be bat -- *SLAP*).
If you add a hidden layer, you can calculate continuous functions like z = x2 + y2 (and many, many other things). If you add another hidden layer, you can calculate pretty much anything. The only problem is that the training gets harder for every layer that you add.
Jplus wrote:For as far as I know there is no theory about how to figure out the ideal number of nodes in each hidden layer for a given task. Usually you'll just guess, then try whether your network will learn faster and better if you use more nodes, and if it doesn't, see how many nodes you can leave away without losing training speed and accuracy. The reason for the latter is that it reduces the risk of overfitting.
Jplus wrote:While perceptrons can only represent linearly separable functions, feed-forward networks with a hidden layer can represent any continuous function, and networks with two hidden layers can even represent any discontinuous function.
The proof is complex, but the main point is that the required number of hidden units grows exponentially with the number of inputs. For example, 2n/n hidden units are needed to encode all Boolean functions of n inputs.
Users browsing this forum: Bakstoola and 2 guests