0612: "Estimation"

This forum is for the individual discussion thread that goes with each new comic.

Moderators: Magistrates, Prelates, Moderators General

Re: "Estimation" Discussion

Postby Zorlin » Mon Jul 20, 2009 6:13 pm UTC

Cup of Dirt wrote:
vodka.cobra wrote:Sweet! The return of excellent XKCD comics!


Yeah, that was a rough weekend.

I tip my metaphorical hat to you, good sir.

Apocalypto the Bunny approves of your comment too.
Mysterious wizard rabbit of unknown proportions.

Meaux_Pas wrote:You're all mad.
Zorlin
 
Posts: 943
Joined: Sat May 30, 2009 2:31 am UTC

Re: "Estimation" Discussion

Postby philsov » Mon Jul 20, 2009 6:35 pm UTC

I just like the progress bars that go based off of number of files completed.

Sure, 99 of those files are a few kbs, they can be installed quickly.

But this last file... it's a gig or so. Enjoy the 99% for a while!
The time and seasons go on, but all the rhymes and reasons are wrong
I know I'll discover after its all said and done I should've been a nun.
User avatar
philsov
Not a fan of Diane Kruger
 
Posts: 1296
Joined: Sat Sep 20, 2008 7:58 pm UTC
Location: Texas

Re: "Estimation" Discussion

Postby FoolishOwl » Mon Jul 20, 2009 6:36 pm UTC

I've seen a lot of unstable time estimates in progress bars, but it's specifically the progress bar for Windows file copy operations that seems to have this sort of absurd fluctuation. I can understand fluctuating time estimates for downloading a file, in which the bandwidth may vary from moment to moment, there may be some compression that varies in effectiveness, etc. But all the information needed for a good estimate is available to the Windows file manager when copying files locally -- the actual file size, the speed of the drive, and so on -- so there's really no excuse for bad estimates.

It goes to show what happens when the source code is a secret.
FoolishOwl
 
Posts: 52
Joined: Mon Jun 29, 2009 8:36 pm UTC
Location: San Francisco, California

Re: "Estimation" Discussion

Postby Mr. Burke » Mon Jul 20, 2009 7:41 pm UTC

k3anan wrote:The algorithm that Microsoft uses for time estimates must really be off. Might be time to think of a new one?

That would be “Calculate speed of previous data, extrapolate for remaining data.”
Hard to improve without clearvoyance. Harddisks are not as deterministic as you'd like them. Specifically, copying many small files probably takes longer than copying one big one.
User avatar
Mr. Burke
 
Posts: 99
Joined: Wed Feb 04, 2009 4:56 pm UTC

Re: "Estimation" Discussion

Postby Clumpy » Mon Jul 20, 2009 7:55 pm UTC

This is the best XKCD in ages.
User avatar
Clumpy
 
Posts: 1883
Joined: Thu Jul 19, 2007 4:48 am UTC

Re: "Estimation" Discussion

Postby capncanuck » Mon Jul 20, 2009 8:38 pm UTC

AWA wrote:Gah, lost by a few seconds.

Anyway.

I generally experience this with torrents, but not much else.


Or you lost by a Plank time length.
capncanuck
 
Posts: 10
Joined: Sat Mar 07, 2009 3:37 pm UTC

Re: "Estimation" Discussion

Postby sircheeziusbreezius » Mon Jul 20, 2009 8:59 pm UTC

Strilanc wrote: A more proper estimate would be a range, like "between 2 and 5 hours". The estimates should become more accurate over time, too.


God also shouldn't be crazy, provided I'm not crazy for maybe believing in a god.
He who knows not and knows that he knows not is a student; teach him. He who knows not and knows not that he knows not is a fool; shun him. He who knows and knows that he knows is wise; follow him.
sircheeziusbreezius
 
Posts: 51
Joined: Thu Apr 09, 2009 2:03 pm UTC
Location: Lakeland, FL

Re: "Estimation" Discussion

Postby Ezbez » Mon Jul 20, 2009 9:50 pm UTC

My favorite is watching other people look at those progress bars. "Well now it says 5 minutes. It used to say 1 minute. The speed has been decreasing. I think it will be done in about 10 minutes."

Now if we just recurse and have someone watching them, making estimates based off their estimates, and someone watching him, etc.... we'd have either a supremely accurate or terribly inaccurate estimate. I'd guess that it would be the latter.
Ezbez
 
Posts: 58
Joined: Thu Jan 10, 2008 10:25 am UTC

Re: "Estimation" Discussion

Postby Bright Shadows » Tue Jul 21, 2009 1:06 am UTC

This comic is supremely awesome.

Y/N
Image
Bright Shadows
 
Posts: 645
Joined: Mon Dec 15, 2008 2:56 pm UTC

Re: "Estimation" Discussion

Postby scikidus » Tue Jul 21, 2009 1:32 am UTC

The number of applications which simply divide total file size by download speed to determine duration is shocking. Are rolling averages so hard?

Then again, are they that much more effective?
Happy hollandaise!

"The universe is a figment of its own imagination" -Douglas Adams
User avatar
scikidus
 
Posts: 793
Joined: Wed Nov 26, 2008 9:34 pm UTC
Location: New York, NY

Re: "Estimation" Discussion

Postby 10nitro » Tue Jul 21, 2009 2:43 am UTC

capncanuck wrote:
AWA wrote:Gah, lost by a few seconds.

Anyway.

I generally experience this with torrents, but not much else.


Or you lost by a Plank time length.

or you lost twenty dollars and your self respect. I did.
~ Luke Shumaker
FRC1024 Programmer
IT technician, GNU/Linux admin, comp. security guy
Eagle Scout
http://lukeshu.ath.cx
10nitro
 
Posts: 192
Joined: Mon Nov 03, 2008 8:46 pm UTC

Re: "Estimation" Discussion

Postby Faranya » Tue Jul 21, 2009 4:02 am UTC

I don't pay much attention to the time estimates. I just want a loading bar that accuratly reflects the current progress attained in relation to the total progress necessary for the job to be done.

I have had it up to here (image my hand is slightly above my eyebrows) with "loading" bars that simply cycle a small graphic through them over and over again while giving no real indication of the amount of work done. Seriously, can't the energy being used to show me a completely useless animated bar be instead use to get the job done faster?
Image
Faranya
 
Posts: 259
Joined: Tue Nov 18, 2008 3:10 am UTC

Re: "Estimation" Discussion

Postby arachnophilia » Tue Jul 21, 2009 5:27 am UTC

retro22 wrote:On my old Win95 machine with dial up I had made the mistake of trying to download a file that was far too large for my machine.... ...It MAY have told me that I had 399 years remaining <_<... Worst. Download. Ever.


that sounds about right for dialup.
User avatar
arachnophilia
 
Posts: 120
Joined: Thu Aug 23, 2007 1:18 am UTC
Location: god's waiting room

Re: "Estimation" Discussion

Postby Radical Pi » Tue Jul 21, 2009 5:44 am UTC

lol this is so true for me, and such perfect timing for this comic.

Today I was clearing off a hard drive for an old Win98 computer to dual-boot a lightweight linux, and the time remaining number kept climbing up higher and higher. I got a cell phone picture (horrible quality) at 2.5 million MINUTES. It peaked around 5 million minutes, then instantly dropped back down to 5 seconds remaining :P I think Microsoft was getting back at us for making fun of their estimation techniques
Radical Pi
 
Posts: 17
Joined: Tue Jul 21, 2009 1:32 am UTC

Re: "Estimation" Discussion

Postby 10nitro » Tue Jul 21, 2009 8:13 am UTC

Faranya wrote:I don't pay much attention to the time estimates. I just want a loading bar that accuratly reflects the current progress attained in relation to the total progress necessary for the job to be done.

I have had it up to here (image my hand is slightly above my eyebrows) with "loading" bars that simply cycle a small graphic through them over and over again while giving no real indication of the amount of work done. Seriously, can't the energy being used to show me a completely useless animated bar be instead use to get the job done faster?

The reason they show you an animation is to assure it hasn't frozen up. If the animation stops you have issues. This is why many programs have the animation side-by-side with the bar, with a slow-moving progress bar, you can't be sure if it's frozen or just slow. In Firefox the little circle of dots thing on the tab. In IE the earth/e in the upper right corner. In GNOME, the moving stripes on the load bar (with some themes, anyway). In the WinXP installer the little boxes in the bottom right. [edit]These boxes are sometimes useful, but sometimes they keep moving, and the mouse responding, even when installation freezes (it was stuck at "Setup will complete in approximately: 13 minutes / Installing devises" for >6 hours)[/edit]
Last edited by 10nitro on Tue Jul 21, 2009 2:49 pm UTC, edited 2 times in total.
~ Luke Shumaker
FRC1024 Programmer
IT technician, GNU/Linux admin, comp. security guy
Eagle Scout
http://lukeshu.ath.cx
10nitro
 
Posts: 192
Joined: Mon Nov 03, 2008 8:46 pm UTC

Re: "Estimation" Discussion

Postby owenja » Tue Jul 21, 2009 9:26 am UTC

Computationally estimating the time it takes to do something is surprisingly hard, and predictions that are both accurate and precise are literally impossible to make. As pointed out here, it's an instance of the halting problem.

Which is not to say that a decent progress bar is impossible, but as a rule it's a relatively small part of a program that would need a disproportionally large amount of effort to get right. Usually that time can be better spent on a different aspect of the program.
owenja
 
Posts: 2
Joined: Thu May 31, 2007 11:40 pm UTC

Re: "Estimation" Discussion

Postby anrieff » Tue Jul 21, 2009 10:59 am UTC

Radical Pi wrote:Today I was clearing off a hard drive for an old Win98 computer to dual-boot a lightweight linux, and the time remaining number kept climbing up higher and higher. I got a cell phone picture (horrible quality) at 2.5 million MINUTES. It peaked around 5 million minutes, then instantly dropped back down to 5 seconds remaining :P I think Microsoft was getting back at us for making fun of their estimation techniques


My guess: what you hit is a bug in Win98, which uses 32-bit integers to calculate file sizes (or at least, when concerned with remaining time estimation). This hit me one day when I tried to move a handful of movies between partitions on a Windows 98 SE. The progress bar was fluctuating wildly, and the actual estimation more often negative than positive, with astronomic numbers such as 3501 years appearing intermittently.
anrieff
 
Posts: 15
Joined: Thu Mar 19, 2009 8:34 pm UTC

Re: "Estimation" Discussion

Postby roderik » Tue Jul 21, 2009 1:16 pm UTC

seems like I've had it worse..
and I wasn't even looking at the time remaining.
the percentages it showed simply moved from 0% to 90% like they should, until they passed the 100% mark and went on to 300% then some time later dropped to -600% before finally finishing at around 900%.
it was horrible.
User avatar
roderik
 
Posts: 18
Joined: Fri Apr 10, 2009 4:16 pm UTC
Location: Below sealevel

Re: "Estimation" Discussion

Postby TheGuyWithTheHat » Tue Jul 21, 2009 1:16 pm UTC

Hilarious!
If English teachers teach English and Maths teachers teach Maths, then Head Teachers must teach...
Live in the UK? You must click here!

velocipirate wrote:but the question is, which cheese would a velociraptor choose?
User avatar
TheGuyWithTheHat
 
Posts: 322
Joined: Tue Jul 21, 2009 1:15 pm UTC
Location: 127.0.0.1

Multiple Linear Regression

Postby KiaserZohsay » Tue Jul 21, 2009 1:45 pm UTC

x = bytes copied so far
y = number of files copied so far
z = seconds elapsed

Data points are taken periodically during the process of copying multiple files (typically a few hundred).

Sx = Sum of all x data points
Sxx = Sum of the sqaures of all x data points
Etc.

Code: Select all
      Sx = Sx + CurrCompletedSize;
      Sxx = Sxx + 1.0 * CurrCompletedSize * CurrCompletedSize;
      Sxy = Sxy + 1.0 * CurrCompletedSize * CompletedCount;
      Sxz = Sxz + 1.0 * CurrCompletedSize * CompletedTime;
      Sy = Sy + CompletedCount;
      Syy = Syy + 1.0 * CompletedCount * CompletedCount;
      Syz = Syz + 1.0 * CompletedCount * CompletedTime;
      Sz = Sz + CompletedTime;
      N = N + 1.0;
      Det = Sxx*Syy*N - Sxx*Sy*Sy - Sxy*Sxy*N + Sxy*Sx*Sy + Sx*Sxy*Sy - Sx*Sx*Syy;
      if (Det > 0) then
        {
          A = (Sxz*Syy*N - Sxz*Sy*Sy + Syz*Sx*Sy - Syz*Sxy*N + Sz*Sxy*Sy - Sz*Sx*Syy) / Det;
          B = (Sxz*Sy*Sx - Sxz*Sxy*N + Syz*Sxx*N - Syz*Sx*Sx + Sz*Sx*Sxy -Sz*Sxx*Sy) / Det;
          C = (Sxz*Sxy*Sy - Sxz*Syy*Sx + Syz*Sxy*Sx - Syz*Sxx*Sy + Sz*Sxx*Syy - Sz*Sxy*Sxy) / Det;
          ProjectedTime = round( A*TotalSize/1024 + B*TotalCount + C );
        }


All of of this calculates constants A, B and C which are used to calculate projected total time for completion, which is used with elapsed time to estimate time remaining. On a typical copy run, the estimate fluctuates a little at the beginning, but then settles outs and counts down like a clock. My co-workers don't know what this code does, and my users don't even know it's there, but when my file copy dialog says you have 10 minutes left on 200 files, you better believe it will be done in ten minutes.

kz
KiaserZohsay
 
Posts: 5
Joined: Tue Jul 21, 2009 1:23 pm UTC

Re: "Estimation" Discussion

Postby 10nitro » Tue Jul 21, 2009 2:56 pm UTC

owenja wrote:Computationally estimating the time it takes to do something is surprisingly hard, and predictions that are both accurate and precise are literally impossible to make. As pointed out here, it's an instance of the halting problem.

To a mathematician this is relevant, but not so much in practical use. If a progress bar is inaccurate... someone fails (bytescopied/totalbytes... not hard). From there there are 2 basic ways of doing the estimate, looking at the current speed and what's left to do, or a rolling average.

Even without spending a lot of time on it, this will be way more accurate than Windows (pseudo code)(progress returns a % in 0-1):
Code: Select all
i0=timeSinceStart(bar)/progress(bar)
while ( progress(bar) != 1 ) {
  i1=i0;
  i0=timeSinceStart(bar)/progress(bar)
  estimate=((i0+i1)/2)-timeSinceStart(bar)
}

although the guy above me's code is probably better. My point is making a decent estimate isn't hard.
Last edited by 10nitro on Tue Jul 21, 2009 5:29 pm UTC, edited 1 time in total.
~ Luke Shumaker
FRC1024 Programmer
IT technician, GNU/Linux admin, comp. security guy
Eagle Scout
http://lukeshu.ath.cx
10nitro
 
Posts: 192
Joined: Mon Nov 03, 2008 8:46 pm UTC

Re: "Estimation" Discussion

Postby Ezbez » Tue Jul 21, 2009 5:04 pm UTC

10nitro: He stated that calculating time left not calculating amount done. Amount complete is usually quite accurate. (Though there are some counter examples, like Gametap which often tells me I am 101% of the way done and am downloading something like 7 billion gigabytes.)
Ezbez
 
Posts: 58
Joined: Thu Jan 10, 2008 10:25 am UTC

Re: "Estimation" Discussion

Postby 10nitro » Tue Jul 21, 2009 5:28 pm UTC

Ezbez wrote:10nitro: He stated that calculating time left not calculating amount done. Amount complete is usually quite accurate. (Though there are some counter examples, like Gametap which often tells me I am 101% of the way done and am downloading something like 7 billion gigabytes.)

...I know the code I gave estimates the time left... I'm counting on amount complete to be accurate.
In my code progress(bar) returns the percentage complete (0.00-1.00).
~ Luke Shumaker
FRC1024 Programmer
IT technician, GNU/Linux admin, comp. security guy
Eagle Scout
http://lukeshu.ath.cx
10nitro
 
Posts: 192
Joined: Mon Nov 03, 2008 8:46 pm UTC

Re: "Estimation" Discussion

Postby Wummi » Tue Jul 21, 2009 8:38 pm UTC

Image

30 seconds, actually.

and i just installed the copy app from page 1, and it is in fact awsome²

also, its 64bit compatible... wooohooo
Wummi
 
Posts: 14
Joined: Thu Jul 02, 2009 8:38 pm UTC

Re: "Estimation" Discussion

Postby nevetS » Thu Jul 23, 2009 3:19 pm UTC

First!
nevetS
 
Posts: 8
Joined: Sun May 13, 2007 10:01 pm UTC

Re: "Estimation" Discussion

Postby moimoimoi » Wed Jul 29, 2009 2:32 am UTC

PTP2009 wrote:As much as I'd love to bash Windows, it's not just them...

Image

This is a well-known problem with Stuffit, not with Mac.

owenja wrote:Computationally estimating the time it takes to do something is surprisingly hard, and predictions that are both accurate and precise are literally impossible to make.


That's why I love Mac's imprecise but accurate progress indicators. Its estimates are vague, but correct to the unit of time: e.g. "About 30 seconds".
moimoimoi
 
Posts: 9
Joined: Fri Jul 17, 2009 2:06 am UTC

Re: "Estimation" Discussion

Postby Tweedledum » Thu Jul 30, 2009 3:42 am UTC

Good comic. In almost every way.

Perhaps it has much to do with hardware. Despite estimates of 14+ hours in my copying time, It never took any of my computers over 9 minutes to do anything less than 14GB.

*Shrugs*
Learn. Invent. Experience.
User avatar
Tweedledum
 
Posts: 6
Joined: Thu Jul 30, 2009 3:07 am UTC
Location: A book

Re: "Estimation" Discussion

Postby Kimmeh » Thu Jul 30, 2009 7:48 pm UTC

Diadem wrote:
'; DROP DATABASE;-- wrote:Also I'm another who is highly against putting the punctuation inside the quotes. It makes no sense! There was no punctuation there in the original quote! D:

I agree. There should only be punctuation in the quote if the punctuation is part of the quote. For example if you quote a full sentence.

But what do you do if the quote is both a full sentence (so it should end with a dot) and the end of the sentence (so there should be a dot behind the quote). Adding both looks aweful.

Alice said: "Let's go to the beach.".

But leaving one out feels wrong, to me. They are both full sentences, and should both end with a dot. In my native language (Dutch) the rule is the leave out the first dot in this case. And that is probably the best solution. But sometimes leaving both might not look bad, so what to do then?

Alice said: "Let's go to the beach!", but I did not feel like going.

This quote is problematic.


Normally you'd put the period inside the quotes. Alice said, "Let's go to the beach."
User avatar
Kimmeh
 
Posts: 21
Joined: Sun Jun 21, 2009 11:13 pm UTC
Location: United States

Re: "Estimation" Discussion

Postby TheGuyWithTheHat » Thu Jul 30, 2009 7:50 pm UTC

Kimmeh wrote:
Diadem wrote:
'; DROP DATABASE;-- wrote:Also I'm another who is highly against putting the punctuation inside the quotes. It makes no sense! There was no punctuation there in the original quote! D:

I agree. There should only be punctuation in the quote if the punctuation is part of the quote. For example if you quote a full sentence.

But what do you do if the quote is both a full sentence (so it should end with a dot) and the end of the sentence (so there should be a dot behind the quote). Adding both looks aweful.

Alice said: "Let's go to the beach.".

But leaving one out feels wrong, to me. They are both full sentences, and should both end with a dot. In my native language (Dutch) the rule is the leave out the first dot in this case. And that is probably the best solution. But sometimes leaving both might not look bad, so what to do then?

Alice said: "Let's go to the beach!", but I did not feel like going.

This quote is problematic.


Normally you'd put the period inside the quotes. Alice said, "Let's go to the beach."


What I do is take the . out of the quote and put it outside, like "I ran away from a raptor".

When people put quotes, they don't usually put the whole sentence, so the way I do it seems logical to me.
If English teachers teach English and Maths teachers teach Maths, then Head Teachers must teach...
Live in the UK? You must click here!

velocipirate wrote:but the question is, which cheese would a velociraptor choose?
User avatar
TheGuyWithTheHat
 
Posts: 322
Joined: Tue Jul 21, 2009 1:15 pm UTC
Location: 127.0.0.1

Mention on MSDN blog

Postby gz3zbz » Thu Aug 25, 2011 3:42 am UTC

This comic (0612: Estimation) scored a comment on the Building Windows 8 blog at MSDN;

We’re anticipating that many of you are going to want to know what we’ve done to improve the accuracy of the estimated time remaining for a copy to complete. (This has been the source of some pretty funny jokes over the years).


http://blogs.msdn.com/b/b8/archive/2011/08/23/improving-our-file-management-basics-copy-move-rename-and-delete.aspx
gz3zbz
 
Posts: 1
Joined: Thu Aug 25, 2011 3:30 am UTC

Previous

Return to Individual XKCD Comic Threads

Who is online

Users browsing this forum: broarbape, sardia, Weeks, Whizbang and 30 guests