Grid Computers.

A place to discuss the science of computers and programs, from algorithms to computability.

Formal proofs preferred.

Moderators: phlip, Prelates, Moderators General

Grid Computers.

Postby NewKat » Mon Apr 23, 2007 7:47 pm UTC

Just wondered what everybody thought...
NewKat
 
Posts: 7
Joined: Sun Apr 22, 2007 12:49 pm UTC

Postby iw » Tue Apr 24, 2007 1:52 am UTC

The most I've ever really heard about grid computing are some talks students gave when I worked at Argonne Labs. They consisted mostly of buzzwords, talking about how exciting grid computing is, and explaining what grid computing was without explaining what grid computing is ("grids are scalable, fault tolerant," and so on).

So I have no idea what it is. My impression of grid computing is cluster computing on steroids. To me it seems like one of those computing "revolutions" that no one can actually describe well. I'm reminded of XML and .NET.
iw
 
Posts: 150
Joined: Tue Jan 30, 2007 3:58 am UTC

Postby Ended » Tue Apr 24, 2007 1:57 am UTC

iw wrote:They consisted mostly of buzzwords


Like the first sentence of this. :?

My uni has what I think would be termed a 'grid computing' structure, where it's possible for people who need a lot of power to harness all the different terminals scattered over the uni site. It sounds pretty cool.

But yeah, I'm unsure if I understand the terminology correctly.
Generally I try to make myself do things I instinctively avoid, in case they are awesome.
-dubsola
Ended
 
Posts: 1459
Joined: Fri Apr 20, 2007 3:27 pm UTC
Location: The Tower of Flints. (Also known as: England.)

Postby NewKat » Wed Apr 25, 2007 4:42 pm UTC

Here is a quote from Catalyst;

"Some of today's large-scale scientific activities - modelling climate change, Earth observation, ... - involve handling millions of bytes of data very rapidly. such activities would not be possible without a new typw of computering. It is not possible for one single institution to store and analyse all this data, so scientists share computer storage and processing power around the world at hundreds of differnet locations. This is called grid computing. much of it is based on vast numbers of PCs linked to process data in parallel with other tasks. In the past, individual supercomputers might have been used but these could only deal with single, although very large, tasks."

Also, see GripPP for the UK's computing grid for handling particle physics data (http://www.gridpp.ac.uk)and CERN wich goes live this year (http://lcg.web.cern.ch/LCG/overview.html) and will produce 15petabytes of data a year. (1 petabyte = 10^15 bytes.)[/url]
NewKat
 
Posts: 7
Joined: Sun Apr 22, 2007 12:49 pm UTC

Postby QuantumTroll » Mon Jun 18, 2007 3:26 am UTC

I've heard people say that Grid Computing is the next step for the internet. I believe this is balderdash.

The basic idea behind Grids is that you can basically get as much computing power as you need without having to deal with individual supercomputing centers. Processing power becomes a shared resource among all the members of the Grid network. The word "grid" draws an analogy between power distribution and computation.

Unless everyone stops using desktops and laptops and moves to dumb terminals, Grids will be used for science and business only.

Not all science and business applications are appropriate for running on a Grid. What makes a program suitable? Generally, security must be a low priority, because putting data on the wire to god-knows-where obviously carries risk. For scheduling reasons, it helps if you know how long the program will run. Unless the data is already being distributed (like the CERN Grid and others), your input data and/or results might be in transit longer than the computation would take on a local machine.

All these gripes aside, I think Grid Computing is a tremendously exciting concept. Right now I work for IBM with free access to some BlueGene/L machines, and they're just sitting idle most of the time. It breaks my heart because I know there are researchers and grad students with projects on hold because of insufficient computer power. :cry: It would be totally swank if all this computer were hooked up so other people could use it when IBMers don't.
User avatar
QuantumTroll
 
Posts: 158
Joined: Fri Jun 08, 2007 12:00 am UTC

Postby aoanla » Thu Jun 21, 2007 1:12 pm UTC

QuantumTroll wrote:All these gripes aside, I think Grid Computing is a tremendously exciting concept. Right now I work for IBM with free access to some BlueGene/L machines, and they're just sitting idle most of the time. It breaks my heart because I know there are researchers and grad students with projects on hold because of insufficient computer power. :cry: It would be totally swank if all this computer were hooked up so other people could use it when IBMers don't.


Ahh, you should just stick Condor on them then... ¬¬

(I work in Grid Computing, and indeed look after one of the nodes for GridPP.)
aoanla
 
Posts: 108
Joined: Wed Jun 13, 2007 6:02 pm UTC

Postby QuantumTroll » Fri Jun 22, 2007 3:16 am UTC

aoanla wrote:
QuantumTroll wrote:All these gripes aside, I think Grid Computing is a tremendously exciting concept. Right now I work for IBM with free access to some BlueGene/L machines, and they're just sitting idle most of the time. It breaks my heart because I know there are researchers and grad students with projects on hold because of insufficient computer power. :cry: It would be totally swank if all this computer were hooked up so other people could use it when IBMers don't.


Ahh, you should just stick Condor on them then... ¬¬

(I work in Grid Computing, and indeed look after one of the nodes for GridPP.)


Heh, I just read your entry on the intro thread. I'm a Computer Scientist pretending to be a Physicist (but failing at the moment), and you're a Physicist turned Computer Scientist. Not surprising our paths have crossed...

Sticking Condor on BlueGene is an excellent idea. One of my coworkers was at a Condor convention of some sort at U Wisconsin just a few weeks ago. I think the goal is to eventually do what you said. Given the unusual nature of BG/L and BG/P, this may not be as easy as just slapping the code in position and typing "make"...

As a physicist looking after a node for GridPP, what does your job entail? Updating the software when new stuff comes out? Swapping out bad nodes? I'm interested in some of these practical aspects of Grids. I've got no idea what it's like working with one.
User avatar
QuantumTroll
 
Posts: 158
Joined: Fri Jun 08, 2007 12:00 am UTC

Postby zenten » Sat Jun 23, 2007 4:14 am UTC

Isn't Google running all their stuff on grid computers? Although I'm not sure that this is the correct term, since it's basically just a bunch of datacentre's located all over the place, with cheap commercially available computers running in banks within them.
zenten
 
Posts: 3798
Joined: Fri Jun 22, 2007 7:42 am UTC
Location: Ottawa, Canada

Postby aoanla » Sun Jun 24, 2007 2:35 pm UTC

QuantumTroll wrote:
aoanla wrote:Ahh, you should just stick Condor on them then... ¬¬

(I work in Grid Computing, and indeed look after one of the nodes for GridPP.)


Heh, I just read your entry on the intro thread. I'm a Computer Scientist pretending to be a Physicist (but failing at the moment), and you're a Physicist turned Computer Scientist. Not surprising our paths have crossed...


If life was a movie, this would be where we either turn out to be archnemeses or form a dynamic duo to fight crime.

Sticking Condor on BlueGene is an excellent idea. One of my coworkers was at a Condor convention of some sort at U Wisconsin just a few weeks ago. I think the goal is to eventually do what you said. Given the unusual nature of BG/L and BG/P, this may not be as easy as just slapping the code in position and typing "make"...

As a physicist looking after a node for GridPP, what does your job entail? Updating the software when new stuff comes out? Swapping out bad nodes? I'm interested in some of these practical aspects of Grids. I've got no idea what it's like working with one.


Updating the software (especially for the grid middleware itself, which is frequently updated, often in ways which unintentionally break stuff) is certainly a major part of it, as is dealing with all the usual sysadmin problems with things breaking and needing maintained. Here at Edinburgh, we're actually in the process of upgrading our presence, so there's also a certain amount of planning to be done (and political stuff which thankfully I have little to do with) as to what we upgrade to.
Perhaps the use of "node" was sloppy as well - Edinburgh, like any of the regional sites for GridPP/EGEE/WLHC (grid people like acronyms and initialisms) provides a number of systems to the grid; a number of "front-end nodes" which provide interfaces for job submission, storage and data-transfer and monitoring/accounting, and pools of compute nodes and storage nodes which actually do the computing and storage here. It's more important to keep the front-end nodes happy, as the compute nodes especially can afford to have a couple die without anyone really noticing elsewhere.
aoanla
 
Posts: 108
Joined: Wed Jun 13, 2007 6:02 pm UTC

Postby QuantumTroll » Mon Jun 25, 2007 9:36 pm UTC

aoanla wrote:If life was a movie, this would be where we either turn out to be archnemeses or form a dynamic duo to fight crime.

Heh. Foreshadowing?

Updating the software (especially for the grid middleware itself, which is frequently updated, often in ways which unintentionally break stuff) is certainly a major part of it, as is dealing with all the usual sysadmin problems with things breaking and needing maintained. Here at Edinburgh, we're actually in the process of upgrading our presence, so there's also a certain amount of planning to be done (and political stuff which thankfully I have little to do with) as to what we upgrade to.
Perhaps the use of "node" was sloppy as well - Edinburgh, like any of the regional sites for GridPP/EGEE/WLHC (grid people like acronyms and initialisms) provides a number of systems to the grid; a number of "front-end nodes" which provide interfaces for job submission, storage and data-transfer and monitoring/accounting, and pools of compute nodes and storage nodes which actually do the computing and storage here. It's more important to keep the front-end nodes happy, as the compute nodes especially can afford to have a couple die without anyone really noticing elsewhere.

Aww, it sounds awfully typical for computer work. Pretty much what I expected, but I was hoping for a VR-type interface to the machines, programs manifesting themselves as glowing cubes, which you feed to a virtual maw of computational power, with warning claxons and strobing orange lights announcing the failure of a node. Meh, some other time, perhaps...

I imagine you have some pretty nice connection speeds at work, though. :)
User avatar
QuantumTroll
 
Posts: 158
Joined: Fri Jun 08, 2007 12:00 am UTC

Postby wac » Wed Jul 11, 2007 1:59 am UTC

QuantumTroll wrote:Aww, it sounds awfully typical for computer work. Pretty much what I expected, but I was hoping for a VR-type interface to the machines, programs manifesting themselves as glowing cubes, which you feed to a virtual maw of computational power, with warning claxons and strobing orange lights announcing the failure of a node. Meh, some other time, perhaps...

I imagine you have some pretty nice connection speeds at work, though. :)


If node failure requires claxons and orange lights, odds are high you're probably doing something wrong. If you have an 11 year MTBF and ~8k machines you have to anticipate a couple component failures every day. And if you're thinking defensively, you have to presume that your single point of failure will be where things break. Fault-tolerance is everything.
User avatar
wac
 
Posts: 7
Joined: Mon Jan 15, 2007 7:42 am UTC
Location: near Seattle

Postby adlaiff6 » Thu Jul 12, 2007 8:23 am UTC

QuantumTroll wrote:Aww, it sounds awfully typical for computer work. Pretty much what I expected, but I was hoping for a VR-type interface to the machines, programs manifesting themselves as glowing cubes, which you feed to a virtual maw of computational power, with warning claxons and strobing orange lights announcing the failure of a node. Meh, some other time, perhaps...

And you can walk around the towers that make up the Gibson too!
3.14159265... wrote:What about quantization? we DO live in a integer world?

crp wrote:oh, i thought you meant the entire funtion was f(n) = (-1)^n
i's like girls u crazy
User avatar
adlaiff6
 
Posts: 274
Joined: Fri Nov 10, 2006 6:08 am UTC
Location: Wouldn't you rather know how fast I'm going?

Postby kaiden11 » Thu Aug 23, 2007 11:03 pm UTC

The way new chip architecture trends are going, I think "grid computing," from a programming perspective, is going to become incredibly important. Considering how parallel architectures are developed, marketed, and ultimately used as they are today, many programmers I've come into contact with have no concept of threading, parallelization, or even how their respective operating systems handle their various tasks. I think we're going to start seeing many more grid computing concepts make their way into the common programming vernacular, hopefully alongside tools or languages that will make the arduous and tedious task a little more friendly (and hopefully more foolproof).
User avatar
kaiden11
 
Posts: 4
Joined: Mon Aug 07, 2006 5:56 am UTC


Return to Computer Science

Who is online

Users browsing this forum: No registered users and 6 guests