My own avian learning experiment

[Tass]

So, I am the proud owner of a blue-and-gold macaw (huge parrot), and having handed in my PhD-thesis on friday I now also have time.

Macaws are very smart birds, and need activation and mental challenging to thrive. I have already trained her to put blocks into the appropriately shaped holes, during the sparse time I had to train her during my PhD. I am certain she could in principle learn just as much as the famous African gray Alex. When I train her I reward her with peanuts, which she is very fond of, while she gets her regular feed ad libitum. I figure there is no reason she shouldn't work for all her food as she would in nature.

I am therefore planning on building an automatic trainer to activate her all day whenever she feels like it. I have bought a Packard-Bell "Liberty Tab" tablet PC. I intend to set it up in her cage (suitably protected, but in a way so that she can use her tongue on the screen) with a program I am going to write, and connect it to an automatic feed dispenser.

What I would like your input on is two things: Good ideas on how to build the dispenser and which tablet output to use to control it (I was thinking of using the jackstick); and a brainstorm of things to try to teach her.

I am going to save all the data I collect on the successes and failures of the experiment. I am thinking of setting up a website were people can follow the project. Since the tablet includes a forward facing camera, I thought I might actually also experiment with letting people interact with her through the website, if I can handle the coding.

Ideas so far: First she must be taught simply that touching the screen means reward.
Then touching specific objects on the screen.
Then teaching her the names of colors and shapes etc.
Giving her control over the light in the room, see if she turns it off when she wants to sleep.
Letting people trough the internet practice the commands she knows (like saying "hi hi", makes her say "hi" back and lift a foot and wave) and reward her.

What do you think of it all? Any input is welcome, and I'll do my best to answer your questions and try to keep this thread updated on the progress of the project.

Note: This will not replace human interaction but merely supplement it. I'll still have her out of the cage and played with twice a day as I do now.

[Mr. Samsa]

I can't be much help in how to create your dispenser, but if it helps, in research labs it's often just called a "hopper" so that might help searching for info online. All it is is essentially a food tray that pops down when the program sends it a signal, so for someone who knows a bit about programming, I imagine it should be too hard to create a hopper connected to a usb cable to simply make it pop down. Labs tend to use a program called Med-PC and there are a few forums that discuss how the program operates - so whilst the code and program isn't freely available, it shouldn't be too hard to reverse engineer some of it (especially for simple tasks like opening a hopper).

Ideas so far: First she must be taught simply that touching the screen means reward.

This is the easiest part. All you need is to introduce an autoshaping procedure. For this you just need something like a red button displayed on your screen, whilst your hopper gives food at regular intervals (e.g. every minute). If you get the red button to "flash" as the hopper is released, then your bird will eventually pick up on what it's supposed to do - especially when pressing the button results in immediate access to the food (as opposed to having to wait the full minute to access it).

As for what else to teach him, I personally would look at how he responds to tasks which are traditionally something only humans can do - for example, set up a Monty Hall dilemma for him and see how well he does at it (as Herbranson does here with pigeons). It would also be interesting to see how well they perform with things like concept formation, like distinguishing between the styles of different painters. My interests are more towards comparative cognition and probably more geeky than the regular trickses that people want to teach their animals, so I'm not sure if those kinds of things would interest you.

Then teaching her the names of colors and shapes etc.

It would probably be good to look at how matching-to-sample tasks are done, as that's how you're going to teach the name of colours and shapes.

If you were able to create a food dispenser and a program to run these experiments though, and you're recording reliable data, then I'd recommend that you try to market it. Programs like Med-PC currently have a monopoly in this research area, like SPSS for stats, and it can be insanely expensive as only their equipment will work with their programs, so even if you sold it at an incredibly low price, you'd still probably make a fair bit of money from the sheer number of small universities that don't have the funding to create a full animal behavior lab but still want to do the research. Or, alternatively, you could release it for free.

Anyway, good luck. I'm definitely interested in seeing your progress so keep us updated, or send us a link to your website if you set it up.

[mfb]

If she can learn an order of symbols (numbers, colors, shapes, whatever - which is its own challenge first), it might be interesting to test this.

[Tass]

Thanks for your replies.

Rather than just giving her access to the food for a limited amount of time, I'd prefer i to simply dispense a small amount into her bowl. That way if the tab is some way from the bowl, I can also see if she learns to be patient and simply sit and earn a bunch of food before going to eat it later. I am thinking of making some sort of conveyor too lift food from the container and into her bowl.

Thanks for your training inputs. There is going to be a lot of things to experiment with when I get the basics running.

[mfb]

Hmm... do they have a concept of time?
It would be interesting to test a setup which forces her to wait for x seconds (after a button appears or whatever) in order to get a reward. Track the distribution of waiting time .

[Mr. Samsa]

Rather than just giving her access to the food for a limited amount of time, I'd prefer i to simply dispense a small amount into her bowl. That way if the tab is some way from the bowl, I can also see if she learns to be patient and simply sit and earn a bunch of food before going to eat it later. I am thinking of making some sort of conveyor too lift food from the container and into her bowl.

Not a great idea as it will result in a fair amount of confusion during the training and, at best, you'll get noisy data, and at worst, your bird won't learn what you're trying to teach it. To put it simply, animals learn through immediate consequences - so if they do X and then something good immediately follows, then they will be more likely to do X again in the future. By leaving food out for them to snack on whenever they wish, you risk the possibility that the otherwise random behavior that preceded them just ducking down to their food bowl will be reinforced (regardless of whatever task you are presenting them). Essentially, you'll be introducing an element of superstition learning within your experiment. And of course, having free access food available will make them less likely to pay attention to the task and less willing to participate (because why work for food when you have some leftover sitting in your bowl). In research labs, this possibility of superstitious responding is so serious that they usually have to implement specific procedures to curb it. For example, in a standard choice task where they are presented with two alternatives, they'll introduce a changeover delay, which just means that if they stop responding on one option and move to the other one, there is a period of time where no reinforcers can be accessed. This is done because sometimes the subjects will learn false associations, e.g. "If I press the left button 10 times and the right button 2 times, then I get the food", and this obviously results in some weird responding.

If you want to see how patient she can be, simply use a self-control task like this, where you use token (conditioned) reinforcers instead of primary reinforcers. You can test her patience by seeing how long she'll wait before 'cashing in' her tokens, and you won't have to worry about the confound of having freely accessible food available.

[Tass]

Wow, will it really be that much of a problem?

I was planning on having the bowl right beside the tablet in the beginning, moving it away was just a maybe for later. I am of course also going to make it play a "reward sound" so that she gets conditioned to associate that sound with reward. The idea was: She does something on the tab -> it goes 'pling' and a few seeds drop into her bowl next to her. Is that really so different from: She does something on the tab -> it goes 'pling' and the lid covering the seeds goes away for a few seconds?

Then once she had learned that doing well means pling, and pling means seeds in the bowl, I was thinking that maybe I could move it away and see if she will move back and forth to pick up the few seeds every time she earned them, or whether she will realize that she can stay and get some more plings before going to eat them.

In any case I am not going to let it fill up or let her have large amounts of leftovers, I that happens I will make the tasks harder, or let the tab turn off when she has earned too much.

Concerning the progress of the project. I have installed the app development software on my laptop, and have done some preliminary programming and gotten it to work on the tab. The hardware building is the limiting factor now; not just the feeder, but also the protection case for the electronics. I have let her play with the unprotected tab under close supervision, her tongue does work on the screen, as I was pretty certain it would, but it is nice to have it tested.

[Mr. Samsa]

Wow, will it really be that much of a problem?

It depends on how seriously you want to take it. Realistically, the way you've described it above, you probably won't encounter too many problems and with a little bit of luck any problems will be overcome by more training. If you were hoping to plot the results and make the data available, and especially if you were ever going to consider trying to publish your stuff (I know that's probably not your aim, but still worth highlighting this), then you will definitely get much cleaner data and much more consistent behavior if you account for the issues I described above.

I was planning on having the bowl right beside the tablet in the beginning, moving it away was just a maybe for later. I am of course also going to make it play a "reward sound" so that she gets conditioned to associate that sound with reward. The idea was: She does something on the tab -> it goes 'pling' and a few seeds drop into her bowl next to her. Is that really so different from: She does something on the tab -> it goes 'pling' and the lid covering the seeds goes away for a few seconds?

The "pling" is a good idea. For most lab set-ups the conditioning is done by the mechanics of the hopper dropping, which just makes a light 'thud' - but any sound will work, of course. The difference is essentially that you're setting up an open economy, rather than a closed one, which means that there is less incentive to work at the task you've given her. If it's only going to be a couple of seeds, and you won't let it build up, then obviously that quantity won't affect your data a great deal, but the question then just becomes: why allow it to possibly become an issue at all?

Then once she had learned that doing well means pling, and pling means seeds in the bowl, I was thinking that maybe I could move it away and see if she will move back and forth to pick up the few seeds every time she earned them, or whether she will realize that she can stay and get some more plings before going to eat them.

This would basically be a foraging experiment, and there are a number of equations which can be used to help you calculate the point at which she'll give up one "patch" for another. There's a good book by Stephens and Krebs called "Foraging Theory" that might be useful to you there. Unfortunately, the book itself can be quite expensive but the googlebooks link there is still pretty generous, and although the book is getting a little old now, the basics are still accurate enough for something like what you're doing (plus it's quite easy to read as well).

There are a number of experiments which have set up a foraging task using two alternatives (or more if you like) where there is no physical distance involved. The rewards are "lined up" on each alternative, so the left key might give 2 rewards in total, and the right key gives 8 rewards, and once all the rewards are taken from each alternative, the rewards get reset and they can start again. Then you just measure which option they prefer, how long they stay there, when they switch, etc.

In any case I am not going to let it fill up or let her have large amounts of leftovers, I that happens I will make the tasks harder, or let the tab turn off when she has earned too much.

Concerning the progress of the project. I have installed the app development software on my laptop, and have done some preliminary programming and gotten it to work on the tab. The hardware building is the limiting factor now; not just the feeder, but also the protection case for the electronics. I have let her play with the unprotected tab under close supervision, her tongue does work on the screen, as I was pretty certain it would, but it is nice to have it tested.

Good to hear it's coming along well. Have you planned on monitoring the amount of feed etc that your parrot receives? It can be important to weigh her every day (preferably anyway, but once a week would probably be fine) for two reasons: 1) to ensure she doesn't gain too much weight, and 2) to ensure that she doesn't lose too much weight. I mention the latter because, ideally, you'll want to limit her access to ad lib food so she's a bit hungry and she'll work for food (without become satiated halfway through the tasks and giving up). Normally experimental subjects are kept at around 85% of their free-feeding weight, which means you let them eat as much food as they like every day for weeks, weighing them consistently, and then working out generally what their 'maximum' weight it and calculating 85% of that. Then when you weigh her, you figure out whether she's under or over weight, and give her an amount of food which is appropriate for whether you need her to gain or lose weight (obviously she should get feed every day though, even if she's over the calculated 85% weight).

[PM 2Ring]

If you want to see how patient she can be, simply use a self-control task like this, where you use token (conditioned) reinforcers instead of primary reinforcers. You can test her patience by seeing how long she'll wait before 'cashing in' her tokens, and you won't have to worry about the confound of having freely accessible food available.

I've only quickly skimmed that PDF, so I might be misunderstanding the concept, but I expect that a relatively simple way to implement token reinforcers on the tablet PC would be via reward icons: each time that she successfully performs a task a reward icon appears on the screen. If she selects the reward icon, the food dispenser is activated. Unselected reward icons simply accumulate on the screen, maybe in some kind of progress bar arrangement. I assume it wouldn't be too hard to give her the initial training required for her to understand the reward icons. A minor variation: the reward icon images are always present on the screen, but they change colour to indicate that they represent an available food reward, eg the icons are green when dormant but change to a mango-orange colour when "ripe".

[Mr. Samsa]

I've only quickly skimmed that PDF, so I might be misunderstanding the concept, but I expect that a relatively simple way to implement token reinforcers on the tablet PC would be via reward icons: each time that she successfully performs a task a reward icon appears on the screen. If she selects the reward icon, the food dispenser is activated. Unselected reward icons simply accumulate on the screen, maybe in some kind of progress bar arrangement. I assume it wouldn't be too hard to give her the initial training required for her to understand the reward icons. A minor variation: the reward icon images are always present on the screen, but they change colour to indicate that they represent an available food reward, eg the icons are green when dormant but change to a mango-orange colour when "ripe".

Yeah definitely. The paper I linked to actually probably wasn't the best for a demonstration of token rewards, instead this paper might be a bit better. Essentially, you can use anything as a token reinforcer - in a standard token reward experiment with pigeons, you would just use a series of LEDs that light up and accumulate as your rewards increase. Whilst there's no evidence that animals can count (in a strict sense), they are good at distinguishing between one, few, and many, so the concept still works up until the point where there are too many tokens available to make any meaningful distinctions.

But yeah, as long as it's clear, visible, etc, then it'll work as a token reward. I'd probably avoid anything too complicated like changing colours on complex shapes or designs though.