xkcd torrents

Think your art is better or your stick figures worse? Got a link to a site you want to share? Post it here!

Moderators: Jacque, Moderators General, Prelates

User avatar
Zorlin
Posts: 950
Joined: Sat May 30, 2009 2:31 am UTC
Location: Perth, Western Australia

xkcd torrents

Postby Zorlin » Mon Jun 08, 2009 7:43 am UTC

Hey guys!

I've uploaded some xkcd comics, prefixed and renamed with the comic numbers and titles, as a featured torrent on Mininova.
Grab a copy if you want an offline xkcd collection or to put it on portable devices - alt-texts are in alt-text.nfo :)
(side-note: I realize it's called the title text in the source. Everyone here calls it alt-text though and it's generally the most-used term so I'm sticking with it)

xkcd 001-050
xkcd 050-100
xkcd 101-150
xkcd 151-200
xkcd 201-250
xkcd 251-300
xkcd 301-350
xkcd 351-400
xkcd 401-450
xkcd 451-500
xkcd 501-550
xkcd 551-600

I'll be updating this thread as I post new xkcd-related torrents.
~ZORLiN
Last edited by Zorlin on Sun Sep 27, 2009 10:23 am UTC, edited 9 times in total.
Mysterious wizard rabbit of unknown proportions.

Check out #xkcd-hugs movie night! Watch movies with your fellow xkcdians.

Meaux_Pas wrote:You're all mad.

10nitro
Posts: 192
Joined: Mon Nov 03, 2008 8:46 pm UTC
Contact:

Re: XKCD torrents

Postby 10nitro » Tue Jul 21, 2009 3:33 am UTC

Code: Select all

#!/bin/sh
i=1
while [ "$?" = '0' ]; do
   if [ "$i" = '404' ]; then i=`expr $i + 1`; fi
   echo "getting xkcd $i"
   string=`wget -qO- xkcd.com/$i | grep '^<img'`
   
   echo $string | sed -e 's/.* title="//' -e 's/".*//' > $i.txt
   
   imgurl=`echo $string | sed -e 's/.* src="//'   -e 's/".*//'`
   imgout=$i.`echo $imgurl | sed 's/.*\.//'`
   i=`expr $i + 1`
   wget -qO- $imgurl > $imgout
done

it will grab all the comics until it reaches a 404 (yes, it knows to skip comic 404)
each comic will be in #.{jpg,png} and the alt text for that comic will be in #.txt
~ Luke Shumaker
FRC1024 Programmer
IT technician, GNU/Linux admin, comp. security guy
Eagle Scout
http://lukeshu.ath.cx

User avatar
Zorlin
Posts: 950
Joined: Sat May 30, 2009 2:31 am UTC
Location: Perth, Western Australia

Re: XKCD torrents

Postby Zorlin » Fri Jul 31, 2009 8:02 am UTC

Wow, that's pretty cool. I've been using Firefox extensions and manual labour. :roll:

Once I get back into a BAsh shell I'll post the rest. Thank you for the script.

Here's 051-100.
http://www.mininova.org/tor/2807064

Update:
Here's 101-150.
http://www.mininova.org/tor/2810605

Expect much more frequent releases - I couldn't get the BAsh script working (and it was not able to do part of what I'm doing anyways) but I created a hacky workaround to help assist creation of these packs.
Mysterious wizard rabbit of unknown proportions.

Check out #xkcd-hugs movie night! Watch movies with your fellow xkcdians.

Meaux_Pas wrote:You're all mad.

User avatar
Zorlin
Posts: 950
Joined: Sat May 30, 2009 2:31 am UTC
Location: Perth, Western Australia

Re: XKCD torrents

Postby Zorlin » Fri Jul 31, 2009 9:11 pm UTC

It's a double release today! :)

XKCD 151-200: http://www.mininova.org/tor/2812363
XKCD 201-250: http://www.mininova.org/tor/2812366
Mysterious wizard rabbit of unknown proportions.

Check out #xkcd-hugs movie night! Watch movies with your fellow xkcdians.

Meaux_Pas wrote:You're all mad.

10nitro
Posts: 192
Joined: Mon Nov 03, 2008 8:46 pm UTC
Contact:

Re: XKCD torrents

Postby 10nitro » Sun Aug 02, 2009 3:00 am UTC

This code will do mostly what you want.

Code: Select all

#!/bin/sh
i="${1-1}"
a="$i"
b=`expr $a + 49`
echo "grabbing xkcd $a-$b"
mkdir -p      "xkcd $a-$b"
while [ "$?" = '0' ]; do
   if [ "$i" = '404' ]; then i=`expr $i + 1`; fi
   echo " - grabbing xkcd $i"
   tmp=`tempfile`
   wget -qO- xkcd.com/$i | grep -e '^<img' -e '<h1>' > "$tmp"
   
   title=`sed -n -e 's/.*<h1>//' -e 's:</h1>.*::p' "$tmp"`
     img=`grep '^<img' "$tmp"`
   
   echo "$i - `echo "$img" | sed -e 's/.* title="//' -e 's/".*//'`" \
      >> "xkcd $a-$b/Alt-text.nfo"

   imgurl=`echo "$img" | sed -e 's/.* src="//' -e 's/".*//'`
   imgout="xkcd $a-$b/$i - $title.`echo $imgurl | sed 's/.*\.//'`"
   wget -qO- $imgurl > $imgout
   
   if [ "$i" = "$b" ]; then
      a=`expr $a + 50`
      b=`expr $b + 50`
      echo "grabbing xkcd $a-$b"
      mkdir -p      "xkcd $a-$b"
   fi
   i=`expr $i + 1`
done


comments on using it:
  • When you copy from the fora, it screws with indentation. This doesn't really matter, except `#!/bin/sh' MUST be all the way on the left.
  • make sure the script is executable (`chmod +x FILENAME')
  • put the file in the current directory, then run `./FILENAME'
  • to continue/start working at a certain comic number, run it as `./FILENAME COMIC'
  • HTML is not filtered out of the title. As far as I can tell, this only affects comic 472.
  • It puts `Alt-text.nfo' in the same directory as the images. (different from your torrents)
  • you still need to pack it all into a torrent
  • yes, it organises them into groups of 50
  • the disclaimer is not at the end of `Alt-text.nfo' is NOT added

Other comments:
When I compared our `Alt-text.nfo' files, I noticed 1)Randall puts 2 spaces after a period, you didn't. 2)you left out the text `with no context,' for comic 16. 3) in comic 39 you typed `its' rather than "it's"

Also, I noticed you type it as `XKCD', Randall has explicitly stated he likes it in all lowercase. (http://xkcd.com/about/)(I also recall him talking about why in an interview for wikinews)

Tsk, tsk, avoiding manual labour/repetitive tasks is why we invented computers.

Anyway, this is a really cool project, and I'll seed the torrents.
~ Luke Shumaker
FRC1024 Programmer
IT technician, GNU/Linux admin, comp. security guy
Eagle Scout
http://lukeshu.ath.cx

User avatar
Zorlin
Posts: 950
Joined: Sat May 30, 2009 2:31 am UTC
Location: Perth, Western Australia

Re: XKCD torrents

Postby Zorlin » Sun Aug 02, 2009 6:15 pm UTC

I actually created a SCAR script that automates it all nearly perfectly. It dies whenever it tries to copy characters Windows cannot handle from the title (because the save as... dialogue dies spectacularly) but this happens rarely enough.

As for 1-50, there's probably mistakes as they were done manually - the rest have been copy and paste (using a Firefox extension).

With regard to naming and errors; I'll edit the torrent names, as for the actual content of the torrents, short of remaking them there isn't much I can do. I'll talk to the mininova staff and request a removal of 1-50 so I can reupload it.
Mysterious wizard rabbit of unknown proportions.

Check out #xkcd-hugs movie night! Watch movies with your fellow xkcdians.

Meaux_Pas wrote:You're all mad.

Musturd
Posts: 3
Joined: Wed Nov 19, 2008 7:55 pm UTC

Re: xkcd torrents

Postby Musturd » Wed Sep 16, 2009 9:54 pm UTC

Please continue posting these torrents! I'm currently loading 1-250 on my new Zune HD

User avatar
Zorlin
Posts: 950
Joined: Sat May 30, 2009 2:31 am UTC
Location: Perth, Western Australia

Re: xkcd torrents

Postby Zorlin » Sun Sep 27, 2009 10:18 am UTC

Musturd: Glad you like them. You should check out some of the other stuff I post - there's some pretty awesome music and TV in there.

And... 300-600 are done. Original post has all the new links in it! Now I just have to wait for 650 to roll around (or start posting in packs of 10/25 or something). Thoughts?
Mysterious wizard rabbit of unknown proportions.

Check out #xkcd-hugs movie night! Watch movies with your fellow xkcdians.

Meaux_Pas wrote:You're all mad.

User avatar
Ran4
Posts: 131
Joined: Mon May 04, 2009 2:21 pm UTC

Re: xkcd torrents

Postby Ran4 » Mon Oct 19, 2009 7:20 pm UTC

Why upload them 50 at a time? That's just annoying, and hard to seed.

User avatar
Zorlin
Posts: 950
Joined: Sat May 30, 2009 2:31 am UTC
Location: Perth, Western Australia

Re: xkcd torrents

Postby Zorlin » Tue Oct 20, 2009 4:49 pm UTC

Ran4 wrote:Why upload them 50 at a time? That's just annoying, and hard to seed.

50 just seems like a good number. Higher than that and the archive isn't really up to date, lower than that and there's too many torrents.
Mysterious wizard rabbit of unknown proportions.

Check out #xkcd-hugs movie night! Watch movies with your fellow xkcdians.

Meaux_Pas wrote:You're all mad.

User avatar
bigglesworth
I feel like Biggles should have a title
Posts: 7461
Joined: Sat Apr 07, 2007 9:29 pm UTC
Location: Airstrip One

Re: xkcd torrents

Postby bigglesworth » Sun Feb 07, 2010 1:27 am UTC

Interesting idea.
Generation Y. I don't remember the First Gulf War, but do remember floppy disks.

User avatar
Zorlin
Posts: 950
Joined: Sat May 30, 2009 2:31 am UTC
Location: Perth, Western Australia

Re: xkcd torrents

Postby Zorlin » Sun Feb 21, 2010 8:52 am UTC

bigglesworth wrote:Interesting idea.
Boredom tends to lead to those. It also keeps me in the habit of releasing something every couple of weeks, even if it isn't non-xkcd related CC content.
Mysterious wizard rabbit of unknown proportions.

Check out #xkcd-hugs movie night! Watch movies with your fellow xkcdians.

Meaux_Pas wrote:You're all mad.

ErraticRoop
Posts: 19
Joined: Sun Feb 21, 2010 8:54 pm UTC
Location: CT, USA
Contact:

Re: xkcd torrents

Postby ErraticRoop » Wed Feb 24, 2010 12:54 am UTC

Wow that may be the best post on these forums! It'll keep us XKCD fans occupied reading the comics for a while! :D
Image

User avatar
hintss
Posts: 1294
Joined: Wed Nov 25, 2009 7:19 am UTC
Contact:

Re: xkcd torrents

Postby hintss » Wed Oct 20, 2010 1:03 am UTC

I'll seed them!

grabs .torrents, sends to henrey-desktop1.local

User avatar
Zorlin
Posts: 950
Joined: Sat May 30, 2009 2:31 am UTC
Location: Perth, Western Australia

Re: xkcd torrents

Postby Zorlin » Sat Nov 13, 2010 8:01 pm UTC

Just noticed these are 200 behind at this point. Anyone interested in 601-800? :wink:
Mysterious wizard rabbit of unknown proportions.

Check out #xkcd-hugs movie night! Watch movies with your fellow xkcdians.

Meaux_Pas wrote:You're all mad.

User avatar
hintss
Posts: 1294
Joined: Wed Nov 25, 2009 7:19 am UTC
Contact:

Re: xkcd torrents

Postby hintss » Tue Nov 16, 2010 12:42 am UTC

for downloading, I suggest neadownloader. it can just grab all images. but then, you'd need alt text

User avatar
Vaskafdt
Posts: 137
Joined: Fri Dec 25, 2009 8:56 am UTC
Location: Jerusalem

Re: xkcd torrents

Postby Vaskafdt » Sun Nov 21, 2010 2:03 am UTC

Finally something worth seeding more then the 1.5 ration i generally stick too. will start DL when I get home in a few hours.
My Art Blog: (Slightly NSFW)
Image

User avatar
Zorlin
Posts: 950
Joined: Sat May 30, 2009 2:31 am UTC
Location: Perth, Western Australia

Re: xkcd torrents

Postby Zorlin » Sun Nov 21, 2010 10:17 pm UTC

hintss wrote:for downloading, I suggest neadownloader. it can just grab all images. but then, you'd need alt text

Zorlin wrote:I actually created a SCAR script that automates it all nearly perfectly. It dies whenever it tries to copy characters Windows cannot handle from the title (because the save as... dialogue dies spectacularly) but this happens rarely enough.

As for 1-50, there's probably mistakes as they were done manually - the rest have been copy and paste (using a Firefox extension).

With regard to naming and errors; I'll edit the torrent names, as for the actual content of the torrents, short of remaking them there isn't much I can do. I'll talk to the mininova staff and request a removal of 1-50 so I can reupload it.
Mysterious wizard rabbit of unknown proportions.

Check out #xkcd-hugs movie night! Watch movies with your fellow xkcdians.

Meaux_Pas wrote:You're all mad.

User avatar
Vaskafdt
Posts: 137
Joined: Fri Dec 25, 2009 8:56 am UTC
Location: Jerusalem

Re: xkcd torrents

Postby Vaskafdt » Mon Nov 22, 2010 7:31 am UTC

Zorlin wrote:Just noticed these are 200 behind at this point. Anyone interested in 601-800? :wink:


I am, if the offer still stands :)
My Art Blog: (Slightly NSFW)
Image

User avatar
Zorlin
Posts: 950
Joined: Sat May 30, 2009 2:31 am UTC
Location: Perth, Western Australia

Re: xkcd torrents

Postby Zorlin » Tue Dec 06, 2011 10:33 pm UTC

New torrents coming as soon as xkcd hits #1000 ;)
Mysterious wizard rabbit of unknown proportions.

Check out #xkcd-hugs movie night! Watch movies with your fellow xkcdians.

Meaux_Pas wrote:You're all mad.


Return to “Your art and links”

Who is online

Users browsing this forum: No registered users and 7 guests