Showing posts with label downloading. Show all posts
Showing posts with label downloading. Show all posts

Wednesday, September 12, 2007

Downloading in Python?

Currently, I'm using a rather ugly way of downloading files in my Python application by executing a "wget" process and extracting progress and speed information from wget's stderr output. Of course, this is ugly and it depends on many factors (did you know RedHat/Fedora's wget has a different output format?

Ideally, I'd like to replace the current code with something completely Python-ish (using urllib2?). The ideal solution would:
  • Be completely written in Python
  • Download URLs to specified local file names
  • Be threaded (or runnable in threads)
  • Report current speed (e.g. 40kb/s) and progress (40% done)
  • Cancelling in-progress downloads should be possible
  • Optionally provide the "estimated time left"
  • Handle URL redirections, HTTP authentication, etc..
Does anyone know of a (lightweight) library that does this or has anybody some code to share so that I don't have to start writing this thing from scratch? If not, some hints on how to best do these things would be very nice.