Christian Long Tech notes and tidbits

What to do when wget won't work

Sometimes, you can't wget a file because of the way the server expects cookies to be handled.

wget http://example.com/frozzler-2.2.2.rpm


cUrl is better able to handle cookies.

Obtain cookie at home page,and save the cookie to cookies.txt

curl -c /path/to/temp/cookies.txt -o /dev/null http://example.com/home_page


Get a file, supplying the cookie saved in cookies.txt

curl -b /path/to/temp/cookies.txt -O http://example.com/frozzler-2.2.2.rpm


Or,all in one line, obtain cookie at home page, then get a file. This works by specifying the homepage as the first file to get, and directing its output to /dev/null. In addition, the -b option turns on cUrl's cookie parser. The cookies set during the visit to the first url (home page) are kept in memory, and are used to "authenticate" when requesting the file to be downloaded . Note that, in this case, the -o and -O options must follow (not preceed) the url to which they apply. You can have as many -o and -O options as you have urls. Note also that we indeed use "nada" (or any non-existent file) as the argument to the -b option. This starts the cUrl cookie parser without having to actually load any pre-existing cookies.

curl -b nada --url http://example.com/home_page -o /dev/null --url http://example.com/frozzler-2.2.2.rpm -O