Thursday, 24. December 2009 15:49 - daniel - Other - 10 Comments
if [ "$1" = "" ]; then
echo "Usage: `basename $0` <4chan thread url>"
echo "4chan downloader"
echo "Downloading until canceled or 404'd"
LOC=$(echo "$1" | egrep -o '([0-9]*)$' | sed 's/\.html//g' )
echo "Downloading to $LOC"
if [ ! -d $LOC ]; then
while [ "1" = "1" ]; do
wget -O "$TMP" "$1"
if [ "$?" != "0" ]; then
rm $TMP $TMP2
egrep 'http://images.4chan.org/[a-z0-9]+/src/([0-9]*).(jpg|png|gif)' "$TMP" -o > "$TMP2"
#cat "$TMP2" | sed 's!/cb-nws!!g' > "$TMP"
wget -nc -i $TMP2
rm $TMP $TMP2
echo "Waiting 30 seconds befor next run"
Gosha - Monday, 28. December 2009 17:07
Gosha - Tuesday, 29. December 2009 7:09
I wanted to be able to save a whole thread, not just the images, so I edited your script: http://liten.besvikel.se/perma/chandl
Emil - Monday, 4. January 2010 20:21
Is there anyone that can make this script or Gosha's script work on a Mac with OS X 10.6? That would really be helpful :D
Jamy - Thursday, 7. January 2010 21:29
Pardon how do I use this? i am using win xp
Daniel - Thursday, 7. January 2010 21:54
You don't really use it on windows. Maybe if you get (e)grep, wget and bash for windows it might work.
sam2332 posted a autoit port of the old script for windows. You have to fix it for the new 4chan URLs, but it's easier to fix this than to get my script woring on windows.
Here is sam2332's script: http://dl.getdropbox.com/u/226498/script/4chan_img_downloader.au3
Daniel - Friday, 8. January 2010 1:26
@Gosha: If you want to download the whole thread (e.g. mirror it), you can use wget like this
wget -e robots=off -E -nd -nc -np -r -k -H -Dimages.4chan.org,thumbs.4chan.org [4chan url]
Ok, this are a shitload of options and it took me some time to look them up, so here is the explanation:
-e robots=off makes wget ignore robots.txt
-E renames html files to .html, so you don't get a file named 12345 but 12345.html instead
-nd no directory structure, downloads all files into the current folder
-nc no clobber, don't create a 12345.jpg.1 file if 12345.jpg already exists
-np no parent, don't go to parent directory, e.g. it will only download the thread but not the whole board
-r recursive, download all files that are referenced in the originally downloaded file
-k convert links in html files to point to the locally downloaded files
-H span hosts, don't just download from boards.4chan.org, but from every host
-Dimages.4chan.org,thumbs.4chan.org only download from these hosts (includes subdomains of thumbs.4chan.org, e.g. 0.thumbs.4chan.org)
Ok, that's it. pew.
Daniel - Friday, 8. January 2010 1:36
By the way, I thinks it's also easier to use this to just download the images. Use wget like this:
wget -e robots=off -E -nd -nc -np -r -H -Dimages.4chan.org -Rhtml [4chan url]
-Rhtml reject files with html extension
-k is no longer necessary because we delete the html file anyway.
This is cool, because there is no need for bash, grep, or anything else. You can just download wget for Windows and run this on Windows too.
Gosha - Tuesday, 17. August 2010 14:34
That's really neat. What's missing in the wget-only-version is that it only downloads it once, and not until the thread 404's. Also, the regex magic in the bash script makes it work on practically any wakaba board.
Redlegion - Tuesday, 21. December 2010 17:50
Hey, thanks for the awesome script. I updated it a bit myself to download five images concurrently. Pretty handy for 150 image threads where each image is roughly 2mb in size. Check it out here:
NOTE: I had to change a switch for mktemp to suit FreeBSD's version of mktemp. Might want to revert that if you aren't using FreeBSD.
James - Thursday, 20. January 2011 7:15
Awesome script, thanks.