Brain= ???? Bytes

For the discussion of the sciences. Physics problems, chemistry equations, biology weirdness, it all goes here.

Moderators: gmalivuk, Moderators General, Prelates

User avatar
Ddanndt
Posts: 60
Joined: Fri Oct 02, 2009 4:18 pm UTC
Location: Paris

Brain= ???? Bytes

Postby Ddanndt » Sun Jan 10, 2010 2:52 pm UTC

Hi everybody. I was revising for my exams and was just wondering how much information is contained in my poor little brain or if it was possible to measure someone's brain capacity like a hard-drive. Thx for your feedback...
God does not care about our mathematical difficulties — He integrates empirically.
—Albert Einstein

User avatar
gmalivuk
GNU Terry Pratchett
Posts: 26765
Joined: Wed Feb 28, 2007 6:02 pm UTC
Location: Here and There
Contact:

Re: Brain= ???? Bytes

Postby gmalivuk » Sun Jan 10, 2010 7:31 pm UTC

Glo wrote:My own opinion is: tends to infinity.

What do you even mean by that? Certainly the brain cannot store infinite information, being only finite in extent. And certainly there must even be some specific (and thus finite) upper bound. For instance, I don't think your brain can hold more bytes of information than the total number of atoms in your brain.

To answer the OP, though, I don't know that this is something we can give an exact (or even very approximate) answer to, since the way brains store information is so vastly different from the way computers use bits to store stuff.
Unless stated otherwise, I do not care whether a statement, by itself, constitutes a persuasive political argument. I care whether it's true.
---
If this post has math that doesn't work for you, use TeX the World for Firefox or Chrome

(he/him/his)

GeorgeH
Posts: 527
Joined: Mon Aug 17, 2009 6:36 am UTC

Re: Brain= ???? Bytes

Postby GeorgeH » Sun Jan 10, 2010 8:31 pm UTC

You're comparing apples to orangutans. For example, set your desktop wallpaper to a 1920x1200 photo of an elephant. Unless you are some kind of savant, it'd be impossible for you to memorize the RGB of every single pixel - so is your brain's capacity <2MB? The average person can distinguish between hundreds (and possibly thousands - if anyone knows the range please post it) of different faces, yet no computer in the world could accurately store hundreds of different faces in a data file even close to 2MB, so clearly the answer is no.

gmalivuk wrote: For instance, I don't think your brain can hold more bytes of information than the total number of atoms in your brain.

I agree with the point you're making, but I think your example is assuming too much about the specifics of how the brain works. If the brain somehow stores data in how atoms are arranged you could get an order of N! states, which is obviously much more capacity than an order of 2N for N bits/bytes. If you wanted to get really silly, 2 atoms could actually store an infinite amount of information - simply use an encoding scheme dependent upon their separation distance, and you're only bound by time and the size of your universe.

A better way of putting might be to say that your brain cells can only take on a finite number of configurations that in turn can only combine in a finite number of ways inside your head, so you're left with a finite number of distinct information states.

User avatar
Woofsie
Posts: 705
Joined: Fri Jun 20, 2008 1:11 pm UTC
Location: Ireland
Contact:

Re: Brain= ???? Bytes

Postby Woofsie » Sun Jan 10, 2010 9:17 pm UTC

Glo wrote:
gmalivuk wrote:
Glo wrote:My own opinion is: tends to infinity.

What do you even mean by that?


Hm, tends to infinity is not infinity, isn't it? I'm just pretty sure it's higher than estimation, I've provided in my post. And BTW your atom-number restriction itself obviously depicts tendation to infinity.


But there aren't infinitely many atoms in your brain, or even in the universe.

Are you using "tend to infinity" to just mean really big?

User avatar
gmalivuk
GNU Terry Pratchett
Posts: 26765
Joined: Wed Feb 28, 2007 6:02 pm UTC
Location: Here and There
Contact:

Re: Brain= ???? Bytes

Postby gmalivuk » Sun Jan 10, 2010 9:23 pm UTC

Yeah, "bigger than my estimate here" is not at all the same as "tends to infinity"...
Unless stated otherwise, I do not care whether a statement, by itself, constitutes a persuasive political argument. I care whether it's true.
---
If this post has math that doesn't work for you, use TeX the World for Firefox or Chrome

(he/him/his)

User avatar
Josephine
Posts: 2142
Joined: Wed Apr 08, 2009 5:53 am UTC

Re: Brain= ???? Bytes

Postby Josephine » Sun Jan 10, 2010 10:08 pm UTC

Glo wrote:
Woofsie wrote:But there aren't infinitely many atoms in your brain, or even in the universe.

Are you using "tend to infinity" to just mean really big?


I didn't write infinity. I didn't write there're infinetely many atoms in a brain. Dude, you are physicist (student), right? Why you put so profitless attention on math definitions?

For your question the answer is yes.


You're speaking to a very mathematically and logically oriented group. "tend to infinity" is so ambiguous that the only definition to be extrapolated from it is "infinite".
Belial wrote:Listen, what I'm saying is that he committed a felony with a zoo animal.

User avatar
poxic
Eloquently Prismatic
Posts: 4751
Joined: Sat Jun 07, 2008 3:28 am UTC
Location: Left coast of Canada

Re: Brain= ???? Bytes

Postby poxic » Sun Jan 10, 2010 10:26 pm UTC

My impression of how brains store things is this: neurons form messy, complex grids. When a particular firing sequence happens, we'll remember what that sequence is associated with.

Sorta like this. Here's an empty brain, somewhat simplified. We have a total of nine neurons, none of them firing.
Brain 1.jpg
Brain 1.jpg (2.38 KiB) Viewed 4494 times

If each neuron stored a single memory, we'd be limited to nine memories (number of neurons in the brain = number of memories). That's like a hard drive. The brain doesn't quite work that way, though. Below, our nine-neuron brain has three neurons firing, forming a circuit through the brain.
Brain 2.jpg
Brain 2.jpg (2.46 KiB) Viewed 4494 times

Let's say that our brain is remembering Mom. Below, it's remembering Dad.
Brain 3.jpg
Brain 3.jpg (2.47 KiB) Viewed 4494 times

And the last pic is our brain remembering the last year at school.
Brain 4.jpg
Brain 4.jpg (2.54 KiB) Viewed 4494 times

Or something like that. More than nine memories are possible, since there are more possible circuits. I don't know how to calculate the upper bound on the total memories available to a 9-neuron grid, but someone here probably can.
A man who is 'ill-adjusted' to the world is always on the verge of finding himself. One who is adjusted to the world never finds himself, but gets to be a cabinet minister.
- Hermann Hesse, novelist, poet, Nobel laureate (2 Jul 1877-1962)

User avatar
Josephine
Posts: 2142
Joined: Wed Apr 08, 2009 5:53 am UTC

Re: Brain= ???? Bytes

Postby Josephine » Sun Jan 10, 2010 10:43 pm UTC

Glo wrote:
poxic wrote:My impression of how brains store things is this: neurons form messy, complex grids. When a particular firing sequence happens, we'll remember what that sequence is associated with.


It's too simplified. Each memory is a set of all neurons in the network - a pattern. Associative remembering of the pattern - retrieving - will work because of synapses configurations allow retrieve the whole pattern from part of it (input). Learn something means to train synapses. The thing is that you can learn just aN patterns for N-neurons NN so that they remain stable under NN updating. Depending on an error you let for processing, constant a varies. For 1% error it's near 0.15, seems.


some coherency would be appreciated there.
Belial wrote:Listen, what I'm saying is that he committed a felony with a zoo animal.

User avatar
Link
Posts: 1404
Joined: Sat Mar 07, 2009 11:33 am UTC
Location: ᘝᓄᘈᖉᐣ
Contact:

Re: Brain= ???? Bytes

Postby Link » Sun Jan 10, 2010 10:51 pm UTC

poxic: if each neuron can have two states, "firing" and "not firing", then there are [imath]2^N[/imath] possibilities, where N is the number of neurons. I don't know a lot about neural networks, but I know enough to say that the simple bitwise approximation isn't sufficient. With memory, it's important to note that the actual stored part is pretty fuzzy; the brain actually fills in the blanks when you try to remember something.

User avatar
gmalivuk
GNU Terry Pratchett
Posts: 26765
Joined: Wed Feb 28, 2007 6:02 pm UTC
Location: Here and There
Contact:

Re: Brain= ???? Bytes

Postby gmalivuk » Sun Jan 10, 2010 11:15 pm UTC

Glo wrote:Dude, you are physicist (student), right? Why you put so profitless attention on math definitions?

Because real physicists use math?

If you just meant "bigger", you should say "bigger", not something ridiculous like "tends to infinity".

nbonaparte wrote:some coherency would be appreciated there.

That said, I don't think English is Glo's first language, so I think some slack should be given here...
Unless stated otherwise, I do not care whether a statement, by itself, constitutes a persuasive political argument. I care whether it's true.
---
If this post has math that doesn't work for you, use TeX the World for Firefox or Chrome

(he/him/his)

User avatar
joshz
Posts: 1466
Joined: Tue Nov 11, 2008 2:51 am UTC
Location: Pittsburgh, PA

Re: Brain= ???? Bytes

Postby joshz » Mon Jan 11, 2010 12:03 am UTC

Glo wrote:gmalivuk, I don't want to spend my time to explain you the difference between physics and mathematics. If you still don't know it, now it's definitely useless.
...um...you do know he never said that physics is the same as mathematics, right? Just that mathematics is really important for physics?
You, sir, name? wrote:If you have over 26 levels of nesting, you've got bigger problems ... than variable naming.
suffer-cait wrote:it might also be interesting to note here that i don't like 5 fingers. they feel too bulky.

User avatar
RogerMurdock
Posts: 158
Joined: Mon Jul 27, 2009 10:35 pm UTC

Re: Brain= ???? Bytes

Postby RogerMurdock » Mon Jan 11, 2010 12:20 am UTC

Glo you are acting like a child. You started this argument when you were called out for using the term "tending to infinity" incorrectly (which you did), and now you insist on acting damn near hostile after refusing to admit you were wrong. Cut it out, you're derailing this thread more than anyone else.

On topic, I've thought about this on numerous circumstances and I think it's a really interesting idea. "Information" is such a nebulous term though even if you could add up someone's memories and feelings of their entire life I'm not sure you could put an actual byte value on it. It would probably be huge though. Like really huge.

User avatar
gmalivuk
GNU Terry Pratchett
Posts: 26765
Joined: Wed Feb 28, 2007 6:02 pm UTC
Location: Here and There
Contact:

Re: Brain= ???? Bytes

Postby gmalivuk » Mon Jan 11, 2010 1:16 am UTC

Glo wrote:I don't want to spend my time to explain you the difference between physics and mathematics. If you still don't know it, now it's definitely useless.

If this is how you're going to continue conducting yourself in this thread, then you should stop posting here.

More specifically, the argument about "tending to infinity", and what Glo thinks it means, is now over.
Unless stated otherwise, I do not care whether a statement, by itself, constitutes a persuasive political argument. I care whether it's true.
---
If this post has math that doesn't work for you, use TeX the World for Firefox or Chrome

(he/him/his)

User avatar
poxic
Eloquently Prismatic
Posts: 4751
Joined: Sat Jun 07, 2008 3:28 am UTC
Location: Left coast of Canada

Re: Brain= ???? Bytes

Postby poxic » Mon Jan 11, 2010 1:25 am UTC

So the model of memory I proposed is lacking? What would be a more useful model (still simplified, preferably, since wetware is damnably complex stuff)?
A man who is 'ill-adjusted' to the world is always on the verge of finding himself. One who is adjusted to the world never finds himself, but gets to be a cabinet minister.
- Hermann Hesse, novelist, poet, Nobel laureate (2 Jul 1877-1962)

User avatar
Nath
Posts: 3148
Joined: Sat Sep 08, 2007 8:14 pm UTC

Re: Brain= ???? Bytes

Postby Nath » Mon Jan 11, 2010 2:46 am UTC

I'm somewhat familiar with artificial neural networks, which are greatly simplified versions of the squishy stuff in our heads. Here's how memory usually works in ANNs, in a nutshell.

You've got a bunch of neurons with inputs and outputs, each of which is a real number. The first layer of neurons takes its input directly from the observations, and its output becomes input for the second layer. You can have several layers. The output from the last layer is the output from the network as a whole.

Each input to a neuron has a weight. The neuron calculates its output as a simple function of its inputs and weights. So the first layer calculates its outputs as a function of the observations and the neuron weights. This trickles forward until the last layer, which calculates the final output(s). Now, if the output is what you wanted, hurray. If not, you adjust the weights so that the error in the output would be lowered if you made that same set of observations again. (How do you tell whether the output is what you want? Well, in the supervised setting that ANNs are usually used in, you know the true answer. In unsupervised settings, it can be tricky, but there are various criteria you could use.)

What this means is that the actual memories are stored in the weights of the neurons. (And the topology, but that tends to be relatively static). So the amount of data you can store is limited by the precision with which you can manipulate the weights. If you have an infinitely precise computer, a single neuron can store an infinite amount of data. Sadly, an infinitely precise computer probably doesn't exist. (You can make an arbitrarily precise computer, but the representation size of a single neuron would keep growing as you increase the precision, so that doesn't buy you anything.)

Also, since all the weights are adjusted when you make an incorrect prediction, your memories aren't really localized to any specific set of neurons. They are distributed over the entire set of neurons. However, if you have multiple possible outputs, some of the pathways will be much more informative about some outputs than others. So a certain amount of specialization happens implicitly.

This was a hurriedly written explanation, so it might be a bit unclear. In any case, ANNs are not real neurons, so this is a bit of a tangent. But thinking about this might give some interesting insights about how real neurons work, and how they store information. If human brains work anything like the above, then we can't really say precisely how much data we store without knowing more about the computational precision of the human brain.

User avatar
poxic
Eloquently Prismatic
Posts: 4751
Joined: Sat Jun 07, 2008 3:28 am UTC
Location: Left coast of Canada

Re: Brain= ???? Bytes

Postby poxic » Mon Jan 11, 2010 3:06 am UTC

Wow, I almost understood that. (My neurological education is limited to half-understood Scientific American articles from the last ten years.)

So it would be slightly more accurate to, say, use different shades of blue in my dopey little pictures to indicate the weight of a signal, and that the same firing pattern with a different mix of shades (weights) would represent different output? And to specify that the dopey little pictures represent patterns in the final layer, rather than the full cascade, I guess.

Or am I so far off that I'm not even wrong? :wink:
A man who is 'ill-adjusted' to the world is always on the verge of finding himself. One who is adjusted to the world never finds himself, but gets to be a cabinet minister.
- Hermann Hesse, novelist, poet, Nobel laureate (2 Jul 1877-1962)

User avatar
Nath
Posts: 3148
Joined: Sat Sep 08, 2007 8:14 pm UTC

Re: Brain= ???? Bytes

Postby Nath » Mon Jan 11, 2010 3:18 am UTC

That seems like a valid way to do it, though in practice one might use something simpler, such as making each possible output node correspond to a separate answer (for a classifier). The diagrams would no longer represent the entire memory of the network, though; they would merely tell you how the output is interpreted.

psyck0
Posts: 1651
Joined: Wed Mar 19, 2008 5:58 pm UTC
Contact:

Re: Brain= ???? Bytes

Postby psyck0 » Mon Jan 11, 2010 3:55 am UTC

I have some background in cognitive psychology. This is what I have learned in my schoolings about memory:

1) Each memory is represented by an activation pattern of every neuron in the brain (or a subset) represented in that activation. It is complex to describe which neurons are involved since some regions are clearly not involved with memory at all, and other regions are more involved with particular types of memories, so I'll just go with every neuron in the brain for simplicity of discussion.

2) Similar memories share similar activation patterns. This is how we are able to think and form associations so easily. It's easier to learn new things if they can be linked to many already known facts, because they share many similar activation patterns and so can be accessed more easily (oversimplifying, every 'similar enough' pattern is another chance to access it). Incidentally, this is why learning dates is so hard (and generally useless)- because they are only connected to one or two things, the things that happened on that day, and so are hard to integrate and hard to access later.

3) Memory is HOLOGRAPHIC. This is the kicker, and I don't fully understand it. The basic idea is that each memory is added to each other memory by adding the activation patterns using matrix algebra, with the net result of a single activation pattern that represents all of our memories. Because of this, our memory capacity is infinite. Additionally, because memory is holographic, memories are stored throughout the brain, not in one location. Just like a real hologram, removing a chunk of the brain only makes the memories more fuzzy, it doesn't remove any of them (taking out a chunk of a hologram makes the image fuzzy, but doesn't remove any part of the image).

Here is a paper discussion the issue, which unfortunately requires subscription.

Here is another transcript of an unpublished paper which is somewhat confusing, and disagrees that the capacity is infinite, although that is what I was taught. Perhaps my professor only meant that it was infinite in practice, since we have such a ridiculous number of neurons that could not possibly fill whatever theoretical limit we have.

User avatar
The EGE
not very good at pickup limes
Posts: 1081
Joined: Sat Dec 27, 2008 12:11 am UTC
Contact:

Re: Brain= ???? Bytes

Postby The EGE » Mon Jan 11, 2010 4:23 am UTC

Asssuming each neuron can hold one bit, then it's roughly 3TB. However, that's not neccessarily the case, so estimates vary between 1Tb and 1000TB. source

That, however, does not really answer the question of how much functional information the brain can store, which I can only describe as 'a lot'.
sillybear25 wrote:But it's NPH, so it's creepy in the best possible way.

Shivahn wrote:I'm in your abstractions, burning your notions of masculinity.

User avatar
gmalivuk
GNU Terry Pratchett
Posts: 26765
Joined: Wed Feb 28, 2007 6:02 pm UTC
Location: Here and There
Contact:

Re: Brain= ???? Bytes

Postby gmalivuk » Mon Jan 11, 2010 3:09 pm UTC

Right. Also keep in mind how different the processing is than a computer. I can recognize hundreds of different faces, but that's not because I actually remember each face with jpeg-like resolution. Rather there are some key patterns and facial structures that I associate with one person or another, and that's all I remember, which allows for a lot more *useful* information than might initially be suspected given brain size alone.
Unless stated otherwise, I do not care whether a statement, by itself, constitutes a persuasive political argument. I care whether it's true.
---
If this post has math that doesn't work for you, use TeX the World for Firefox or Chrome

(he/him/his)

User avatar
Diadem
Posts: 5654
Joined: Wed Jun 11, 2008 11:03 am UTC
Location: The Netherlands

Re: Brain= ???? Bytes

Postby Diadem » Mon Jan 11, 2010 4:43 pm UTC

psyck0 wrote:3) Memory is HOLOGRAPHIC. This is the kicker, and I don't fully understand it. The basic idea is that each memory is added to each other memory by adding the activation patterns using matrix algebra, with the net result of a single activation pattern that represents all of our memories. Because of this, our memory capacity is infinite. Additionally, because memory is holographic, memories are stored throughout the brain, not in one location. Just like a real hologram, removing a chunk of the brain only makes the memories more fuzzy, it doesn't remove any of them (taking out a chunk of a hologram makes the image fuzzy, but doesn't remove any part of the image).

I'm not quite sure holographic means what you think it means.

But our memory capacity certainly is not infinite. There's only a finite amount of information in the universe. Our brains must have a finite capacity. Physics requires it.
It's one of those irregular verbs, isn't it? I have an independent mind, you are an eccentric, he is round the twist
- Bernard Woolley in Yes, Prime Minister

User avatar
Xanthir
My HERO!!!
Posts: 5410
Joined: Tue Feb 20, 2007 12:49 am UTC
Location: The Googleplex
Contact:

Re: Brain= ???? Bytes

Postby Xanthir » Mon Jan 11, 2010 7:02 pm UTC

Diadem wrote:
psyck0 wrote:3) Memory is HOLOGRAPHIC. This is the kicker, and I don't fully understand it. The basic idea is that each memory is added to each other memory by adding the activation patterns using matrix algebra, with the net result of a single activation pattern that represents all of our memories. Because of this, our memory capacity is infinite. Additionally, because memory is holographic, memories are stored throughout the brain, not in one location. Just like a real hologram, removing a chunk of the brain only makes the memories more fuzzy, it doesn't remove any of them (taking out a chunk of a hologram makes the image fuzzy, but doesn't remove any part of the image).

I'm not quite sure holographic means what you think it means.

But our memory capacity certainly is not infinite. There's only a finite amount of information in the universe. Our brains must have a finite capacity. Physics requires it.

Assuming our brains had access to real numbers, the capacity is theoretically infinite.

You are correct, though, that the universe doesn't appear to allow anything access to real numbers, only integers.
(defun fibs (n &optional (a 1) (b 1)) (take n (unfold '+ a b)))

psyck0
Posts: 1651
Joined: Wed Mar 19, 2008 5:58 pm UTC
Contact:

Re: Brain= ???? Bytes

Postby psyck0 » Mon Jan 11, 2010 8:24 pm UTC

Diadem wrote:
psyck0 wrote:3) Memory is HOLOGRAPHIC. This is the kicker, and I don't fully understand it. The basic idea is that each memory is added to each other memory by adding the activation patterns using matrix algebra, with the net result of a single activation pattern that represents all of our memories. Because of this, our memory capacity is infinite. Additionally, because memory is holographic, memories are stored throughout the brain, not in one location. Just like a real hologram, removing a chunk of the brain only makes the memories more fuzzy, it doesn't remove any of them (taking out a chunk of a hologram makes the image fuzzy, but doesn't remove any part of the image).

I'm not quite sure holographic means what you think it means.

But our memory capacity certainly is not infinite. There's only a finite amount of information in the universe. Our brains must have a finite capacity. Physics requires it.

I certainly don't understand it, as I admitted in the paragraph you quoted. Later on, I went on to question the assertion that capacity is infinite as well.

Perhaps you could instead have been productive and given a better definition of holographic?

User avatar
Josephine
Posts: 2142
Joined: Wed Apr 08, 2009 5:53 am UTC

Re: Brain= ???? Bytes

Postby Josephine » Mon Jan 11, 2010 8:40 pm UTC

gmalivuk wrote:Right. Also keep in mind how different the processing is than a computer. I can recognize hundreds of different faces, but that's not because I actually remember each face with jpeg-like resolution. Rather there are some key patterns and facial structures that I associate with one person or another, and that's all I remember, which allows for a lot more *useful* information than might initially be suspected given brain size alone.
I think you've hit on the brain's data compression system. If the brain can fill in the blanks, you only need to remember a select few features of each face.
Belial wrote:Listen, what I'm saying is that he committed a felony with a zoo animal.

User avatar
kjsharke
Posts: 70
Joined: Fri Dec 07, 2007 3:53 pm UTC

Re: Brain= ???? Bytes

Postby kjsharke » Mon Jan 11, 2010 9:52 pm UTC

Nath wrote:I'm somewhat familiar with artificial neural networks, which are greatly simplified versions of the squishy stuff in our heads. Here's how memory usually works in ANNs, in a nutshell.
Spoiler:
You've got a bunch of neurons with inputs and outputs, each of which is a real number. The first layer of neurons takes its input directly from the observations, and its output becomes input for the second layer. You can have several layers. The output from the last layer is the output from the network as a whole.

Each input to a neuron has a weight. The neuron calculates its output as a simple function of its inputs and weights. So the first layer calculates its outputs as a function of the observations and the neuron weights. This trickles forward until the last layer, which calculates the final output(s). Now, if the output is what you wanted, hurray. If not, you adjust the weights so that the error in the output would be lowered if you made that same set of observations again. (How do you tell whether the output is what you want? Well, in the supervised setting that ANNs are usually used in, you know the true answer. In unsupervised settings, it can be tricky, but there are various criteria you could use.)

What this means is that the actual memories are stored in the weights of the neurons. (And the topology, but that tends to be relatively static). So the amount of data you can store is limited by the precision with which you can manipulate the weights. If you have an infinitely precise computer, a single neuron can store an infinite amount of data. Sadly, an infinitely precise computer probably doesn't exist. (You can make an arbitrarily precise computer, but the representation size of a single neuron would keep growing as you increase the precision, so that doesn't buy you anything.)

Also, since all the weights are adjusted when you make an incorrect prediction, your memories aren't really localized to any specific set of neurons. They are distributed over the entire set of neurons. However, if you have multiple possible outputs, some of the pathways will be much more informative about some outputs than others. So a certain amount of specialization happens implicitly.

This was a hurriedly written explanation, so it might be a bit unclear. In any case, ANNs are not real neurons, so this is a bit of a tangent. But thinking about this might give some interesting insights about how real neurons work, and how they store information. If human brains work anything like the above, then we can't really say precisely how much data we store without knowing more about the computational precision of the human brain.


I actually saw a presentation on "cortical simulations" at [url=SC09]http://sc09.supercomputing.org/[/url] -- they claim to have modeled something on the scale of a cat's brain, which required ~144 TB. That is memory to store the brain (dominated by synapses, not neurons), not memory that the brain can store. If I understand the scaling right, they'd need a few petabytes to model the human brain.

That said, I (knowing very little about this field) was underwhelmed by the presentation, considering it was a Gordon Bell prize winner... Apparently, I'm not the only one. So the memory required to model the brain may be lesser due to MPI buffers, replicated data, OS, etc; or much more because they made a lot of simplifying assumptions (which seems to be the case).

To answer the OP's question, my wild guess is that if you asked how much memory it would take to model the brain, I'd have to ask you how accurate you wanted the model to be. If you were using full QM for the whole thing, it would require a truly unphysical amount of memory. If you wanted to model each neuron as a simple on/off switch, you could probably run that on your computer today (but would it be worth doing?)

As for "how much our brain can remember"... that has to be less (assuming incompressible data), but I would say it would be hard to gauge because (as gmalivuk said) the brain remembers things in its own way -- faces aren't images of faces. I don't even think numbers are always numbers -- I had 50 digits of pi memorized once, but I think I was remembering the chant rather than the numbers themselves. Before CallerID, I didn't remember people's phone numbers so much as the patterns you made as you dialed them. my 2¢..

User avatar
Charlie!
Posts: 2035
Joined: Sat Jan 12, 2008 8:20 pm UTC

Re: Brain= ???? Bytes

Postby Charlie! » Mon Jan 11, 2010 11:14 pm UTC

They modeled a cat-brain-number-of-neurons worth of nodes, but they didn't come close to modeling an actual cat brain. Despite that, the media has occasionally made silly comments.
Some people tell me I laugh too much. To them I say, "ha ha ha!"

User avatar
Cobramaster
Posts: 201
Joined: Fri Nov 20, 2009 2:31 pm UTC
Location: Georgia Southern University Department of Chemistry, and Department of Biology.

Re: Brain= ???? Bytes

Postby Cobramaster » Tue Jan 12, 2010 1:15 am UTC

Well even though my model is completely wrong on how the brain stores information if you would go with equivalents in the digital world your brain's storage capacity if you remembered everything would require the storage needed for 70+years of 3-D Super HD video, complete surround sound, thousands of analytical readings both internal and external, plus a running commentary that is your thoughts. which if you think about that we really need to learn how the brain stores information because that is Peta-bytes in scale and in less than a three pound package since a good chunk of the brain does not store memory it just keeps the body running, plus it is relatively low on the power consumption scale.
SlyReaper wrote:Did you never notice the etymological link between "tyrannosaur" and "tyrant"? 1% of the dinosaurs had 99% of the prey. Occupy Pangaea.

User avatar
gmalivuk
GNU Terry Pratchett
Posts: 26765
Joined: Wed Feb 28, 2007 6:02 pm UTC
Location: Here and There
Contact:

Re: Brain= ???? Bytes

Postby gmalivuk » Tue Jan 12, 2010 3:13 am UTC

Cobramaster wrote:Super HD video

It's not actually all that high-res away from the fovea.
Unless stated otherwise, I do not care whether a statement, by itself, constitutes a persuasive political argument. I care whether it's true.
---
If this post has math that doesn't work for you, use TeX the World for Firefox or Chrome

(he/him/his)

User avatar
Xanthir
My HERO!!!
Posts: 5410
Joined: Tue Feb 20, 2007 12:49 am UTC
Location: The Googleplex
Contact:

Re: Brain= ???? Bytes

Postby Xanthir » Tue Jan 12, 2010 5:02 am UTC

Cobramaster wrote:if you remembered everything

Good thing we don't. We'd go crazy.
(defun fibs (n &optional (a 1) (b 1)) (take n (unfold '+ a b)))

User avatar
poxic
Eloquently Prismatic
Posts: 4751
Joined: Sat Jun 07, 2008 3:28 am UTC
Location: Left coast of Canada

Re: Brain= ???? Bytes

Postby poxic » Tue Jan 12, 2010 5:14 am UTC

There's some evidence that we do record everything but don't usually recall most of it. I just spent a couple of minutes searching for a reference to a documentary I once saw, but came up empty. It interviewed a young autistic woman who, when her mother gave her the past year's calendar, filled in every single day with what had happened, who had visited, the weather...

I suspect that autism can show us what lies underneath the normal human... overconsciousness? General purpose intelligence? Ordinary waking state? There seems to be much, much more that a human brain can do, but we don't usually access it consciously.

Edit: still haven't found the documentary I was thinking of, but I did find this article from 2003. Selectively shutting down (or overloading?) parts of the brain through transcranial magnetic stimulation led the author to develop savant-type abilities for short periods of time. The experimenter's theory is that it's our ability to conceptualise (imagine?) and contextualise that weakens our perceptions of all the data that surrounds us, and which we are constantly taking in regardless of where our attention is.
A man who is 'ill-adjusted' to the world is always on the verge of finding himself. One who is adjusted to the world never finds himself, but gets to be a cabinet minister.
- Hermann Hesse, novelist, poet, Nobel laureate (2 Jul 1877-1962)

User avatar
PM 2Ring
Posts: 3713
Joined: Mon Jan 26, 2009 3:19 pm UTC
Location: Sydney, Australia

Re: Brain= ???? Bytes

Postby PM 2Ring » Tue Jan 12, 2010 5:50 am UTC

Nath wrote:What this means is that the actual memories are stored in the weights of the neurons. (And the topology, but that tends to be relatively static). So the amount of data you can store is limited by the precision with which you can manipulate the weights. If you have an infinitely precise computer, a single neuron can store an infinite amount of data. Sadly, an infinitely precise computer probably doesn't exist.

Thanks for that summary, Nath.

I guess there's a tendency for people to think of neurones as simple on-off switches and ignore the subtleties of the weighted synaptic inputs, with all those funky neurotransmitters. IIRC, neurones can have many thousands of inputs, so representing a single neurone accurately would consume quite a few bytes. And let's not forget that in a real neural network new connections can form, too.

User avatar
gmalivuk
GNU Terry Pratchett
Posts: 26765
Joined: Wed Feb 28, 2007 6:02 pm UTC
Location: Here and There
Contact:

Re: Brain= ???? Bytes

Postby gmalivuk » Tue Jan 12, 2010 12:54 pm UTC

poxic wrote:There's some evidence that we do record everything but don't usually recall most of it. I just spent a couple of minutes searching for a reference to a documentary I once saw, but came up empty. It interviewed a young autistic woman who, when her mother gave her the past year's calendar, filled in every single day with what had happened, who had visited, the weather...

How is that evidence that we record everything and not, say, that this particular autistic woman recorded everything?
Unless stated otherwise, I do not care whether a statement, by itself, constitutes a persuasive political argument. I care whether it's true.
---
If this post has math that doesn't work for you, use TeX the World for Firefox or Chrome

(he/him/his)

psyck0
Posts: 1651
Joined: Wed Mar 19, 2008 5:58 pm UTC
Contact:

Re: Brain= ???? Bytes

Postby psyck0 » Tue Jan 12, 2010 2:36 pm UTC

poxic wrote:There's some evidence that we do record everything but don't usually recall most of it. I just spent a couple of minutes searching for a reference to a documentary I once saw, but came up empty. It interviewed a young autistic woman who, when her mother gave her the past year's calendar, filled in every single day with what had happened, who had visited, the weather...

I suspect that autism can show us what lies underneath the normal human... overconsciousness? General purpose intelligence? Ordinary waking state? There seems to be much, much more that a human brain can do, but we don't usually access it consciously.

Edit: still haven't found the documentary I was thinking of, but I did find this article from 2003. Selectively shutting down (or overloading?) parts of the brain through transcranial magnetic stimulation led the author to develop savant-type abilities for short periods of time. The experimenter's theory is that it's our ability to conceptualise (imagine?) and contextualise that weakens our perceptions of all the data that surrounds us, and which we are constantly taking in regardless of where our attention is.

I'm sorry to say this, but there have been many attempts to find an "unconscious" with "super-human powers" in psychology and all of them have failed. Savants aren't showing us something we do ourselves but that gets buried. They may have more perception of tiny details, but that is a detriment- it obscures the bigger picture, which we evolved to synthesize so we don't HAVE to rely on tiny details. They may be able to remember ridiculously long numbers or be really, really good at finding primes, but that seems to be simply because it's their single interest and they do it 10 hours a day in their heads. There isn't anything under our consciousness except the basic processes which keep us alive (homeostasis mechanisms) and processing centres. Having access to our processing centres would be pretty useless- the data is too raw for us to do anything with, and if anything, would overwhelm us. The only tool I think could be argued is partially under automatic control (I hesitate to use the word unconscious, because it doesn't exist as far as we can measure) is attention. You can focus your attention to some degree, but there are also many kinds of stimuli which will capture it automatically- sudden noises, bright colours, movement, etc. Perhaps it would be cool to have a bit more control over attention but I think you would quickly find yourself overburdened having to manually select everything you wanted to attend to, rather than allowing your goal-state to do that for you (having to think "look at the desk, no, not there, look at the dresser, not there, look at the floor, walk to the kitchen, look at the walls so you don't run into anything, look on the counter" instead of "look for my keys" and have attention automatically focus your vision on the places you are searching, and automatically process the stimuli you need to walk without running into anything).

RabidAltruism
Posts: 135
Joined: Fri Aug 29, 2008 9:25 pm UTC

Re: Brain= ???? Bytes

Postby RabidAltruism » Tue Jan 12, 2010 3:56 pm UTC

Nath's post corresponds well to how I understand ANN data processing, and may be the most appropriate way to address the OP's question.

That being said, someone mentioned (I think gmailvuk) that human information processing proceeds very differently from the way typical software handles problems - that we tend to store pretty low-resolution pictures, for example, and use various heuristic processing techniques to reproduce memories from those low-resolution pictures. With that in mind, I would submit that our "pragmatic" information-processing abilities depend not only on the total bytes of data we can store in various, alternative configurations of our synaptic efficiencies, but also on the efficiency of the functional processes by which we create new, stable equilibria in our synaptic configurations - and, hence, on both the rules by which we update our synaptic weights and on the activation functions our neural nets employ (the functions which relate inputs to a given neuron to that neuron's output - either a 1/0 or real number-esque firing rate, depending on what level of analysis you're considering). Here, I mean that what we would conventionally recognize as a 'memory' corresponds to a stable (in the dynamical systems sense of small disturbances to our inputs leading to a return to the equilibrium configuration of synaptic weights and firings) configuration of synaptic firings and weights, with some well-defined zone of inputs causing that configuration to dominate a particular area of cortex.

Presumably some weight-updating and activation functions allow for more and less efficient incorporation of new configurations into memory; some weight-updating functions will even destroy old memories (this is called catastrophic forgetting in the neural net literature, I believe), and, if they do so very freely, may be useless as far as information storage is concerned. Actually, thinking about it now, the weight-updating function used should be crucial to our estimate of the brain's storage capacity in the OP's sense, too, yes? I imagine weight-updating rules vary in their capacity to create statistical order among neuronal efficiencies on various sets of inputs; if there is in some sense a "typical" input(s), I suppose we'd want a weight-updating rule that creates orderly configurations of synaptic weights very well on that input and similar inputs.

I mean -- to pick an absurd example -- a "weight-updating function" that, upon specific patterns of firing, randomly scrambled the current configuration of synaptic weights would not yield anything we'd call stored information. Presumably also conventional, Hebbian-style weight-updating rules (and the commonly used sigmoid activation functions) have some limitations in their ability to create order in a finite network of weights and firing rates/activation states of finite precision, yes?
...she reminds you of the invisible form of the soul; she gives life to her own discoveries; she awakens the mind and purifies the intellect; she brings light to our intrinsic ideas; she abolishes oblivion and ignorance which are ours by birth...

User avatar
gmalivuk
GNU Terry Pratchett
Posts: 26765
Joined: Wed Feb 28, 2007 6:02 pm UTC
Location: Here and There
Contact:

Re: Brain= ???? Bytes

Postby gmalivuk » Tue Jan 12, 2010 4:14 pm UTC

RabidAltruism wrote:that we tend to store pretty low-resolution pictures, for example, and use various heuristic processing techniques to reproduce memories from those low-resolution pictures.

Of course to be fair, that is also roughly how facial recognition software works. It compares basic patterns in the current face to the basic patterns it has stored of thousands of other faces in order to try and "recognize" the current one.
Unless stated otherwise, I do not care whether a statement, by itself, constitutes a persuasive political argument. I care whether it's true.
---
If this post has math that doesn't work for you, use TeX the World for Firefox or Chrome

(he/him/his)

RabidAltruism
Posts: 135
Joined: Fri Aug 29, 2008 9:25 pm UTC

Re: Brain= ???? Bytes

Postby RabidAltruism » Tue Jan 12, 2010 4:26 pm UTC

Yes, absolutely - when I say the heuristic, statistical approach is unlike much of today's software, I have in mind a very brittle abstraction as my exemplar of "today's software." I think that's probably decreasingly accurate as time marches forward, with software that relies heavily on assessments of statistical properties of its input (e.g. facial recognition software) or programming which is biologically inspired (neural nets, genetic algorithms/programming) not at all fitting into my 'software is brittle' generalization.
...she reminds you of the invisible form of the soul; she gives life to her own discoveries; she awakens the mind and purifies the intellect; she brings light to our intrinsic ideas; she abolishes oblivion and ignorance which are ours by birth...

User avatar
Nintendon't
Posts: 61
Joined: Fri Apr 17, 2009 1:55 am UTC

Re: Brain= ???? Bytes

Postby Nintendon't » Tue Jan 12, 2010 5:39 pm UTC

From my understanding (at least what I learned in first year psychology) is that the brain is largely reconstructive, rather than an exact storage place for memories. Especially when considering false memories, the forgetting of memories, and such.

User avatar
poxic
Eloquently Prismatic
Posts: 4751
Joined: Sat Jun 07, 2008 3:28 am UTC
Location: Left coast of Canada

Re: Brain= ???? Bytes

Postby poxic » Tue Jan 12, 2010 5:52 pm UTC

Just to clarify: I don't think there are "superhuman abilities" hiding underneath ordinary consciousness, just that we do notice and record more than we're usually aware of. In the article I linked to, the author went from stick-figure drawings of a cat to detailed sketches with facial expressions (also of a cat) during a TMS session. Not because he suddenly developed artistic ability, but because he became better able to accurately recall the details of cats he'd seen.

Think of how you can be unable to recall someone's face, no matter how hard you try, until you remember that you saw them at a party last week. Suddenly you see a scene from your friend's house, where the TV was, where your drink was sitting, and the person you were trying to remember who happened to be standing by the kitchen. All that information was stored but most of it wasn't ... indexed, if you will. (You couldn't have answered the question "where did you put your drink down when so-and-so was over by the kitchen?" until you remembered that scene.)
A man who is 'ill-adjusted' to the world is always on the verge of finding himself. One who is adjusted to the world never finds himself, but gets to be a cabinet minister.
- Hermann Hesse, novelist, poet, Nobel laureate (2 Jul 1877-1962)

User avatar
gmalivuk
GNU Terry Pratchett
Posts: 26765
Joined: Wed Feb 28, 2007 6:02 pm UTC
Location: Here and There
Contact:

Re: Brain= ???? Bytes

Postby gmalivuk » Tue Jan 12, 2010 6:05 pm UTC

poxic wrote:Not because he suddenly developed artistic ability, but because he became better able to accurately recall the details of cats he'd seen.

I didn't read the article, but is there evidence presented that shows this is a matter of recalling information previously unavailable, and not, say, the ability to focus on transferring that recollection onto paper? Because I can already picture my friend Catherine quite accurately, but couldn't draw her very well because I have trouble translating that visual memory onto paper.

Something that would demonstrate the claim you're making is if a person honestly can't for the life of them recall what happened every day of some month last year or whatever, and then through TCM they are able to do so accurately and precisely.
Unless stated otherwise, I do not care whether a statement, by itself, constitutes a persuasive political argument. I care whether it's true.
---
If this post has math that doesn't work for you, use TeX the World for Firefox or Chrome

(he/him/his)

User avatar
Rockberry
Posts: 109
Joined: Tue Jan 12, 2010 9:24 am UTC
Location: UK

Re: Brain= ???? Bytes

Postby Rockberry » Tue Jan 12, 2010 7:35 pm UTC

According to Caprica (the TV Series) its 200 MB. I have no idea how they calculated that.


Return to “Science”

Who is online

Users browsing this forum: No registered users and 8 guests