Using Programming in Everyday Life

A place to discuss the implementation and style of computer programs.

Moderators: phlip, Moderators General, Prelates

ThirstyMonkey
Posts: 15
Joined: Wed Aug 24, 2011 11:48 pm UTC

Using Programming in Everyday Life

Postby ThirstyMonkey » Wed Jul 18, 2012 9:45 pm UTC

After a year of taking Computer Science classes, I feel like I've learned a lot, but I have absolutely no idea how to practically apply it during everyday situations.

For example, today for my job (which is unfortunately not programming related), I had to take a bunch of Excel files in a folder and convert them all to PDFs. There were around 80 files and so for 20 minutes I had to open each one, go to File->SaveAs->PDF->Enter and close. Is there any way I could quickly have done this? What language would I use? How could I do similar iterative things with different sorts of files? And at what scale would it be simpler for me to just do it by hand?

In a similar vein, how do programmers use these their skills in their everyday life for things outside of work/fun projects?

Perhaps I'm simply not too creative, or perhaps I just haven't learned enough, but I can't imagine how to use the new skills I'm gathering.

User avatar
Xanthir
My HERO!!!
Posts: 5400
Joined: Tue Feb 20, 2007 12:49 am UTC
Location: The Googleplex
Contact:

Re: Using Programming in Everyday Life

Postby Xanthir » Wed Jul 18, 2012 10:25 pm UTC

That particular task is probably best done exactly as you did - manually. Spending 20 minutes doing something boring is crappy, but the amount of work necessary to automate it would likely be non-trivial. If you have to do that kind of thing a lot, look into automating it.

Programming is exactly as useful as you make it. For example, I and my wife cook a lot, but dealing with cookbooks and food magazines for recipes was really annoying, so I wrote a recipe app that gradually expanded into something that makes our grocery list for the week as well. This saves us literal hours per week, and helps us try things that looked interesting once, but aren't in one of our "favorite" cookbooks. For another example, I'd gotten annoyed at dealing with secure passwords, so I wrote a deterministic password generator so I only have to remember an easy phrase per site and a single secure password, but end up with unique secure passwords for every site I use. For a third example, I'd been experimenting with dice mechanics in D&D, and wanted to know how a new method I'd come up with compared to traditional rolling, so I wrote a simulator to give me decent data about the spread of values and help me twerk things. For a fourth example, I became interested in color recently, so I've been writing a lot of throwaway pages that let me play with different color spaces and color systems (using <canvas>, which *rocks* for doing simple graphics programs).

There are tons of ways to use programming all around you if you get used to looking for them, just like there's tons of places to use algebra and simple calculus if you know what you're looking for.
(defun fibs (n &optional (a 1) (b 1)) (take n (unfold '+ a b)))

Nem
Posts: 336
Joined: Fri Aug 14, 2009 12:19 pm UTC

Re: Using Programming in Everyday Life

Postby Nem » Wed Jul 18, 2012 10:53 pm UTC

ThirstyMonkey wrote:There were around 80 files and so for 20 minutes I had to open each one, go to File->SaveAs->PDF->Enter and close. Is there any way I could quickly have done this?


Yes. But you'd have to know how the files were structured. I believe PDF's spec is listed with the ISO these days. Adobe's website lists ISO 32,000.

Of course you could equally have recorded a macro. Which is effectively zero programming but would have saved you a fair bit of time. :wink:

Ben-oni
Posts: 278
Joined: Mon Sep 26, 2011 4:56 am UTC

Re: Using Programming in Everyday Life

Postby Ben-oni » Thu Jul 19, 2012 12:27 am UTC

I concur with everything Xanthir said. My uses are a bit different, of course. I generally always have an interpreter open that I can punch various questions into. Often math-related, but I tend to do a fair bit of data-manipulation as well. I rarely write whole programs for things as writing the code is as easy as asking the question. For simple probability questions, like Xanthir mentioned, it's easier to get the distributions from the computer than to work them out by hand.

The computer is a tool, and the more you know about what it can do, the more uses you'll find for it.

User avatar
Dason
Posts: 1311
Joined: Wed Dec 02, 2009 7:06 am UTC
Location: ~/

Re: Using Programming in Everyday Life

Postby Dason » Thu Jul 19, 2012 1:24 am UTC

Ben-oni wrote:For simple probability questions, like Xanthir mentioned, it's easier to get the distributions from the computer than to work them out by hand.

Eh - depends on what we're talking about and how comfortable you are with math.
double epsilon = -.0000001;

Ben-oni
Posts: 278
Joined: Mon Sep 26, 2011 4:56 am UTC

Re: Using Programming in Everyday Life

Postby Ben-oni » Thu Jul 19, 2012 4:24 am UTC

Dason wrote:
Ben-oni wrote:For simple probability questions, like Xanthir mentioned, it's easier to get the distributions from the computer than to work them out by hand.

Eh - depends on what we're talking about and how comfortable you are with math.

Oh, I'm pretty comfortable with the math but... how about an example? Roll four (six sided) dice. Reroll any ones, but only once. Then drop the lowest value and add the rest together. Do this six more times to get seven values, and take the highest six. Subtract 10 from each, divide by two, and floor the result. Add everything up. Now, what's the average? What's the variance? While it can be done by hand, it's so much easier to punch it into an interpreter and brute force an answer.

User avatar
sourmìlk
If I can't complain, can I at least express my fear?
Posts: 6393
Joined: Mon Dec 22, 2008 10:53 pm UTC
Location: permanently in the wrong
Contact:

Re: Using Programming in Everyday Life

Postby sourmìlk » Thu Jul 19, 2012 5:32 am UTC

You could use programming every day by programming every day. If you find something you want to make and then make it (whether or not it's very pragmatic), then you have an actual use for your programming ability.
Terry Pratchett wrote:The trouble with having an open mind, of course, is that people will insist on coming along and trying to put things in it.

User avatar
Dason
Posts: 1311
Joined: Wed Dec 02, 2009 7:06 am UTC
Location: ~/

Re: Using Programming in Everyday Life

Postby Dason » Thu Jul 19, 2012 6:13 am UTC

Ben-oni wrote:
Dason wrote:
Ben-oni wrote:For simple probability questions, like Xanthir mentioned, it's easier to get the distributions from the computer than to work them out by hand.

Eh - depends on what we're talking about and how comfortable you are with math.

Oh, I'm pretty comfortable with the math but... how about an example? Roll four (six sided) dice. Reroll any ones, but only once. Then drop the lowest value and add the rest together. Do this six more times to get seven values, and take the highest six. Subtract 10 from each, divide by two, and floor the result. Add everything up. Now, what's the average? What's the variance? While it can be done by hand, it's so much easier to punch it into an interpreter and brute force an answer.


Sure for something like that I would open up R and just code a simulation. You just said simple probability questions and like I said depending on what we're talking about and how comfortable you are with math sometimes it can be faster to do things directly.
double epsilon = -.0000001;

Ben-oni
Posts: 278
Joined: Mon Sep 26, 2011 4:56 am UTC

Re: Using Programming in Everyday Life

Postby Ben-oni » Thu Jul 19, 2012 7:39 am UTC

Dason wrote:Sure for something like that I would open up R and just code a simulation. You just said simple probability questions and like I said depending on what we're talking about and how comfortable you are with math sometimes it can be faster to do things directly.

Okay, I see the confusion now. For the very simple, I'll just answer off the top of my head or with half a piece of paper to scribble on. Then there are the simple problems that are kinda complicated to work out by hand, but which are simple to reduce to lots of arithmetic. And then there's the class of problems that I'll definitely use a computer for regardless, but which I expect to get paid to work on.

As for the example I offered, μ = 10.02042, σ² = 8.15534. I have only tried this, not proved it to be correct, so if someone wants to verify my numbers...?

User avatar
Dason
Posts: 1311
Joined: Wed Dec 02, 2009 7:06 am UTC
Location: ~/

Re: Using Programming in Everyday Life

Postby Dason » Thu Jul 19, 2012 8:05 am UTC

Ben-oni wrote:As for the example I offered, μ = 10.02042, σ² = 8.15534. I have only tried this, not proved it to be correct, so if someone wants to verify my numbers...?


I ran a simulation using 1000000 trials and ended up with a mean of 10.02039 and a variance of 8.169094 so it seems like our results are comparable.
double epsilon = -.0000001;

User avatar
freakish777
Posts: 354
Joined: Wed Jul 13, 2011 2:14 pm UTC

Re: Using Programming in Everyday Life

Postby freakish777 » Thu Jul 19, 2012 5:37 pm UTC

Xanthir wrote:That particular task is probably best done exactly as you did - manually. Spending 20 minutes doing something boring is crappy, but the amount of work necessary to automate it would likely be non-trivial. If you have to do that kind of thing a lot, look into automating it.


Correct. Until 80 becomes 8000. Then you want to look at C# and the Microsoft.Office.Interop.Excel namespace.

This is the type of thing where if you have to do it everyday for 30 minutes, you work on making an app on your lunch break (or at home). It should actually be somewhat trivial (more than 20 minutes for sure, not more than a day or two of work though).

User avatar
thoughtfully
Posts: 2253
Joined: Thu Nov 01, 2007 12:25 am UTC
Location: Minneapolis, MN
Contact:

Re: Using Programming in Everyday Life

Postby thoughtfully » Thu Jul 19, 2012 5:46 pm UTC

Or anything that speaks COM, or load it into OpenOffice/LibreOffice and use the Python/Java scripting there. So many choices!
Image
Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.
-- Antoine de Saint-Exupery

User avatar
Bubbles McCoy
Posts: 1106
Joined: Wed Jul 09, 2008 12:49 am UTC
Location: California

Re: Using Programming in Everyday Life

Postby Bubbles McCoy » Thu Jul 19, 2012 7:16 pm UTC

Another route is to use a scripting language like Python, Ruby, or Perl along with OLE. OLE libraries let you do a fair bit of internal manipulation of Office files, and I don't believe it's too much trouble to print a file to pdf using it. In my experience though it rarely works quite as well as you'd like, so you're going to have to spend a decent amount of time testing it. This kind of scripting is generally unsuitable for short if tedious tasks - you'd probably only want to bother automating something like this if it's a task that requires at least a couple of hours a month. My C# experience is pretty minimal so I can't really speak to which is better, so I guess I'll leave you with another "so many choices!" line.

User avatar
freakish777
Posts: 354
Joined: Wed Jul 13, 2011 2:14 pm UTC

Re: Using Programming in Everyday Life

Postby freakish777 » Thu Jul 19, 2012 10:08 pm UTC

For most "basic" stuff you want to do in C#, there's usually an example on CodeProject.com:

http://www.codeproject.com/Articles/175 ... Fs-the-Eas

Great Justice
Posts: 54
Joined: Sun Aug 15, 2010 5:28 am UTC

Re: Using Programming in Everyday Life

Postby Great Justice » Fri Jul 20, 2012 1:57 am UTC

You should automate the task anyway, because the only way you'll learn to use programming is to apply it. Whether your employer is lenient enough or you take some of your own time to do it, the cost:return begins high but is an investment which will improve only through practice.
Otherwise, your job will always be a string of ad-hoc tasks which will always be simpler to handle manually, since you don't have the knowledge of tools; and any job which is too big to perform by hand will be given to an existing experienced programmer, not you.

Spend the time pointing-and-clicking and have nothing to show for it but the PDFs themselves.

What happens when the spreadsheets get updated?
Or you have to do something very similar?
How do you know how long an implementation would take you?
... what if it took only 10 minutes to do so?

Your programming experience will often exponentiate the return on your time, as you gather the tools into your toolkit, and can better estimate your ability. Even spending 30+ min to program away 20 minutes of mind-numbing labor is of more value to your life.
It usually isn't Congress or the State that tries to abridge free expression or free speech, [...] actually, in the present situation, the main threat to expression comes from public opinion.
~Christopher Hitchens

Carnildo
Posts: 2023
Joined: Fri Jul 18, 2008 8:43 am UTC

Re: Using Programming in Everyday Life

Postby Carnildo » Fri Jul 20, 2012 4:49 am UTC

Great Justice wrote:You should automate the task anyway, because the only way you'll learn to use programming is to apply it. Whether your employer is lenient enough or you take some of your own time to do it, the cost:return begins high but is an investment which will improve only through practice.

That really depends on how long automating it would take. I recently had a similar task (take this folder hierarchy of documents, edit all of them in the same way) that took 40 hours of work to automate. If editing them by hand would have taken 20 minutes (rather than the estimated 50 hours), there's no way I would have gotten permission to automate it. (As it was, the real benefit of automating it was the negative error rate: the code didn't make mistakes of its own, and it found three mistakes in the source documents that manual editing would have missed.)

Ben-oni
Posts: 278
Joined: Mon Sep 26, 2011 4:56 am UTC

Re: Using Programming in Everyday Life

Postby Ben-oni » Fri Jul 20, 2012 5:26 am UTC

Carnildo wrote:
Great Justice wrote:You should automate the task anyway, because the only way you'll learn to use programming is to apply it. Whether your employer is lenient enough or you take some of your own time to do it, the cost:return begins high but is an investment which will improve only through practice.

That really depends on how long automating it would take. I recently had a similar task (take this folder hierarchy of documents, edit all of them in the same way) that took 40 hours of work to automate. If editing them by hand would have taken 20 minutes (rather than the estimated 50 hours), there's no way I would have gotten permission to automate it. (As it was, the real benefit of automating it was the negative error rate: the code didn't make mistakes of its own, and it found three mistakes in the source documents that manual editing would have missed.)

40 hours?! Explain yourself.

madd0ct0r
Posts: 44
Joined: Fri Jun 15, 2012 3:13 am UTC

Re: Using Programming in Everyday Life

Postby madd0ct0r » Fri Jul 20, 2012 6:10 am UTC

as a construction designer, being able to program was a damn useful skill on occasion.

Working on site, it was less useful, but I still ended up writing a few excel sheets to track various things or iterate solutions quickly.

Probability calcs and playing out game theory issues is a fun side of one of my hobby's, and I'll probably end up writing some phone apps for things I do regularly.

But the really hard, deep 'wired in ' and thinking in abstract maths?

Very very rare, but that's the same for everything.

User avatar
Laserdan
Posts: 162
Joined: Tue Dec 27, 2011 3:12 am UTC

Re: Using Programming in Everyday Life

Postby Laserdan » Fri Jul 20, 2012 10:07 am UTC

Imho an important skill for a developer is also to assess what's the right tool for the task. For example, given the information about Excel-> PDF conversion, if Excel doesn't do anything special but basically "print out" a PDF file, I'd simply install a PDF printer and print all 80. 5 minutes searching one and installing, and some minutes to print, but without a lot of the clicking.

When I started learning programing I resolved to automate every task I find tedious simply to get better at coding.

Examples: I had to collect specific (but quite verbose) system information from literally dozens of computers. I wrote an app in C# that collects all of those over network and packs them into packages named "Computername_OSVersion_DateTime.xxx". It was quick and dirty, but it worked. Later, other people wanted to use it too, so I expanded it with an understandable GUI.

Once I had to do certain tests with a engineering software that supports certain industrial touch panels, but not every test needed to be done on every panel. So I wrote an app that calculates an optimal distribution so that I do the same amount of tests, only that about every panel type got the same amount of testing. Also, don't even type the name "PICT-Tool", please. I found out this exists, is free and from Microsoft, and can do exactly that easier.

Another time, when WoW Cataclysm was just out, I wanted to have a tool to calculate reforging. Doesn't matter what it is right now, but it's basically the knappsack problem. My solution wasn't perfect (I certainly didn't exhaust even a large part of the possibilities), but it was much better than eyeballing it. This was at the time before tools for that were avaible (later, Adrianna of EJ.com made a much better tool for rogues, and it was visual - mine was a printout consisting of tables and lots of numbers and I decided to personally choose).
"Sobriety is a crutch for people who can't cope with drugs."

User avatar
Yakk
Poster with most posts but no title.
Posts: 11120
Joined: Sat Jan 27, 2007 7:27 pm UTC
Location: E pur si muove

Re: Using Programming in Everyday Life

Postby Yakk » Fri Jul 20, 2012 6:04 pm UTC

Yep -- the tendency to automate is good, in that it gets you better at automating.

If you have a problem that takes 20 minutes to do by hand, or 60-120 minutes to automate, doing it via automation means that next time you have a similar task it will be faster, which could include "wait, you did it wrong, try doing it with this slight twerk" in many cases.

And it is hard to get good at automating tasks without practice. And you are competing against all of the tools that help you automate.

In your case, the excel "open file, save as PDF" pattern could be easily automated using in-application scripting in excel. It could easily take more than 20 minutes because you'd have to learn the scripting language -- while you might pull it off in 10, the uncertainty is high because you don't know what you need to know yet. Yet I would still do that, even facing uncertainty, because it means I will after the fact know the basics of scripting in Excel, and who knows what that will be useful for?
One of the painful things about our time is that those who feel certainty are stupid, and those with any imagination and understanding are filled with doubt and indecision - BR

Last edited by JHVH on Fri Oct 23, 4004 BCE 6:17 pm, edited 6 times in total.

Carnildo
Posts: 2023
Joined: Fri Jul 18, 2008 8:43 am UTC

Re: Using Programming in Everyday Life

Postby Carnildo » Sat Jul 21, 2012 4:26 am UTC

Ben-oni wrote:
Carnildo wrote:
Great Justice wrote:You should automate the task anyway, because the only way you'll learn to use programming is to apply it. Whether your employer is lenient enough or you take some of your own time to do it, the cost:return begins high but is an investment which will improve only through practice.

That really depends on how long automating it would take. I recently had a similar task (take this folder hierarchy of documents, edit all of them in the same way) that took 40 hours of work to automate. If editing them by hand would have taken 20 minutes (rather than the estimated 50 hours), there's no way I would have gotten permission to automate it. (As it was, the real benefit of automating it was the negative error rate: the code didn't make mistakes of its own, and it found three mistakes in the source documents that manual editing would have missed.)

40 hours?! Explain yourself.

The job description changed twice during development, and the program to edit the documents wasn't even remotely scriptable, so I wound up automating it by simulating mouse clicks and keystrokes.

User avatar
Sc4Freak
Posts: 673
Joined: Thu Jul 12, 2007 4:50 am UTC
Location: Redmond, Washington

Re: Using Programming in Everyday Life

Postby Sc4Freak » Sat Jul 21, 2012 7:57 pm UTC

Ah, ever-changing requirements. The bane of every software engineer's existence.

User avatar
WanderingLinguist
Posts: 237
Joined: Tue May 22, 2012 5:14 pm UTC
Location: Seoul
Contact:

Re: Using Programming in Everyday Life

Postby WanderingLinguist » Sat Jul 21, 2012 11:33 pm UTC

The particular task the OP mentions could be easily automated, I think.

I'm a Mac user, so in my case, I'd just use Automator to record the action once, then edit the resulting code to make it repeat for every file in the folder.

I haven't used Windows in a long time, but as I recall under some rather-ancient version of Excel, you could record a macro of something, and the result of that macro would be available as code (VBA at the time, probably C# now, I'd guess). So I automated similar things by recording a macro for doing it once, then opening the code for the macro and changing the filename in the "open" operation to a variable. Stick the whole thing in a loop. Or you're too lazy to look up the API for getting a list of files in a folder, just open a console windows and type "dir *.xls", and copy and paste the results into the program directly. A quick search-and-replace can turn that into function calls to your open-and-save subroutine.

This kind of quick-and-dirty one-off approach is fine for this kind of task.

In school, we often learn about programming larger-scale applications. But when you talk about using programming in "everyday life" quite often a throw-away program you can write in 5 minutes for a given task is good enough.

Sure, it might take longer the first time you do it, but if you get used to it, being able to write throwaway programs to automate one-time (but repetitive and time-consuming) tasks can save a lot of time in the future.

To me the OP's problem seems like at worst (for someone with a bit of experience) a 5~10 minute task, which beats 30 minutes.

In cases like this, I'll do one or two by hand, time myself, and say "Okay, it's gonna take me 30 or so minutes to do this; it's worth spending 10 minutes to see if I can automate it". Worst case, I'll have wasted 10 minutes and the whole thing will have taken 40 minutes instead. But if I can automate it, then I've saved 20 minutes, and maybe I've learned a technique I can apply in the future that will save even more time.

It seems annoying to me that only experienced programmers seem to use programming for "everyday life" tasks, where most everyday life tasks are considerably less demanding and the ability to use programming for THOSE should be taught to more people. For an issue like the OP mentions, there's no need to spend lots of time building a foolproof solution, doing unit testing or QA, etc. That's what I think of when I think of "everyday programming".

It's like everyday math. Not everyone is going to use linear algebra in everyday life, but it's really useful to be able to figure in your head if someone gave you the wrong change at the supermarket, or figure out whether you're getting a good deal on a sale (vs. being tricked into paying more). Or in semi-advanced cases, calculate how much epoxy paint you need to paint the bottom of a curved swimming pool. We have everyday math, why not everyday programming? Enough to save people from the tedium of opening, saving and closing a whole bunch of Excel files.

ThirstyMonkey
Posts: 15
Joined: Wed Aug 24, 2011 11:48 pm UTC

Re: Using Programming in Everyday Life

Postby ThirstyMonkey » Mon Jul 23, 2012 9:52 pm UTC

Well, per the advice of everyone in this thread, I have set about trying to automate this task via an Excel macro. I have a macro that is completely functional in that it will convert every Excel file in a folder to a PDF. Now I am trying to set up a Windows Form Application so that this can whole process can be initiated from outside Excel. Basically the person will start this program (as an icon on the desktop) and then will just have to pick the folder where the Excel files are and the folder where they want the PDF versions to go. They'll then click 'OK' and it will all happen magically.

I'm looking for someone with some knowledge of C#/VBA/Excel macros here. I don't want the program to open the single Excel workbook where this macro is located. The workbook is completely empty except for this single macro; it's only purpose is to serve as a launching pad for this macro. But I don't want this program to be tied to a (seemingly) empty workbook. Does this macro even have to exist as a "macro?" Can it just be VBA code in a separate file that gets called when the user clicks "OK"? I'm uneasy on the exact differences between Excel macros and just plain VBA code.

User avatar
walkerm930
Posts: 69
Joined: Wed Apr 07, 2010 3:53 am UTC
Location: Canada: Ontario: Toronto

Re: Using Programming in Everyday Life

Postby walkerm930 » Tue Jul 24, 2012 4:53 am UTC

Another option for the mac is applescript. A quick google finds a couple options for using specific menu items. Also, just recently I couldn't be arsed to find a PDF viewer that had autoscrolling, so I wipped up a small script to do it for me. Unfortunately there doesn't seem to be anything useful for actually scrolling so it ends up being a bit hackish. (key code 125 is the down key)

Code: Select all

tell application "Preview" to activate
tell application "System Events"
   repeat
      delay 0.86
      key code 125
      if not frontmost of application "Preview" then
         exit repeat
      end if
   end repeat
end tell

I find that doing something helpful for real life has a good chance of being hackish to some degree.
In the gospel according to trig there are 3 primary rules: sin θ = x/h , cos θ = y/h and tan θ = x/y. These rules are not open to interpretation and are to be treated as law.


Return to “Coding”

Who is online

Users browsing this forum: No registered users and 15 guests