A sunday afternoon is always good for a surprising journey on the Internet. Today, I was looking into WSGI, after reading some Python-related websites. Combined with the fact that I was really upset about the new perli.net Codebase being slow and unstable, I decided to have a look at WSGI to fix my problems with multi-process mod_python and Python instances. I found the great tutorial "A Do-It-Yourself framework" by Ian Bicking.
I was quickly able to hack together a small WSGI-capable test application. Gladly, I have designed the perli.net Codebase so that the underlying (mod_python) is abstracted by a "Request" class I've written. This was a really good design decision, as I was able to easily adopt that class to work with WSGI instead of mod_python without having to look through the whole codebase. One the "Request" object (and some associated handling classes and functions) were updated to WSGI, I was able to start the test server (Paste's httpserver) and had the codebase running on WSGI! Great!
There were still some problems with SQLAlchemy, so after looking around on the web and asking on IRC, I did two things: Migrate from MyISAM to InnoDB (transaction-based SQL engine) and created a threading.Lock() object in the database access object and wrapped all database access calls between lock.acquire() and lock.release() calls. Now, SQLAlchemy is running much better and stable than before. Not only because of the changes I did, but also because the WSGI makes it possible to run one process with multiple threads (instead of multiple processes, where SQLAlchemy can't share objects between page requests).
To test my changes and to see if the server will stay stable with many concurrent requests, I installed Siege and pointed it at my local webserver. I was able to fix some miscellaneous bugs that cropped up while testing with Siege, and also had statistics on how fast a page would be generated. On my 2GHz (2GB RAM) MacBook, the first photo overview page loaded for about 6 to 15 seconds (HTML only), depending on load. This was way too long, so I thought about implementing some kind of cache.
I'm now caching certain pager data that can be cached and doesn't depend on users being logged in or not. This reduces the page load time to about 0.3 seconds (HTML only, again), so perli.net should be a bit faster on certain pages. A good space-time tradeoff, I think :)
Sunday, July 29, 2007
Saturday, July 28, 2007
Announcing python-jabberbot
I've just released the first version of python-jabberbot, a simple framework to easily write Jabber bots in Python. The framework utilizes the wonderful xmpppy library for the XMPP protocol and was inspired by the xmpppy library bot.py example, although my implementation is targeted at users wanting to simply implement some bot functionality in Python without much hassle.
To implement a bot with python-jabberbot, simply subclass the provided JabberBot class and add some methods to your class that start with "bot_", i.e. "bot_status". The method will receive three parameters: The "self" object, a "mess" object that is the message being received by the client and an "args" object which is a string and contains the parameters given to the jabber bot.
A basic example that returns some system information can be found on the website and a more advanced example showing how to do multi-threading and message broadcasting can be found here. This one has been Aanice afternoon time killer - I hope this is useful to somebody someday :)
To implement a bot with python-jabberbot, simply subclass the provided JabberBot class and add some methods to your class that start with "bot_", i.e. "bot_status". The method will receive three parameters: The "self" object, a "mess" object that is the message being received by the client and an "args" object which is a string and contains the parameters given to the jabber bot.
A basic example that returns some system information can be found on the website and a more advanced example showing how to do multi-threading and message broadcasting can be found here. This one has been Aanice afternoon time killer - I hope this is useful to somebody someday :)
Sunday, July 22, 2007
Yeah! UTF-8 and Unicode in Py3k!
From what I just read on GvR's Python 3000 Status Update, it seems like loads of problems that crop up when using non-ASCII characters in string literals will be finally gone in Python 3.0. While developing the new perli.net codebase, this has been a problem, as everything (database + code + HTML output) is UTF-8 encoded, and if you are not careful enough, Python 2.4 will bite you with an exception (string literals containing umlauts, etc..).
Having UTF-8 as the default source encodings will make things easier for code with non-ASCII string literals. That's one of the quirks I dislike about the current Python. I'm glad this problem is taken care of in Python 3.0.
Having UTF-8 as the default source encodings will make things easier for code with non-ASCII string literals. That's one of the quirks I dislike about the current Python. I'm glad this problem is taken care of in Python 3.0.
The broken Canon Digital IXUS 60
Yesterday evening, my Canon Digital IXUS 60 camera's lens got stuck, and so the camera stopped working altogether. Luckily, I was able to copy all pictures from the camera with a simple pnm-fetch, so at least the photos, videos and audio files (yeah, I tried that the first time yesterday ;) are safe. The error message (in German) was "Objektivfehler:Kamerarestart", an error which is famously known as the "E18 error".
Thanks to my sister, I can now try out her Casio Exilim EX-S770, which looks really slick and is thinner than the Digital IXUS 60. Now that I am very disappointed with the Canon brand cameras, I can try out how good the Casio is. I'll bring the IXUS back to the shop where I bought it in September 2006, and hopefully they will be able to fix it.
After some fiddling, I can get the IXUS to open the lens a bit and make photos, but they are badly unfocused, as you can see in this picture:
Thanks to my sister, I can now try out her Casio Exilim EX-S770, which looks really slick and is thinner than the Digital IXUS 60. Now that I am very disappointed with the Canon brand cameras, I can try out how good the Casio is. I'll bring the IXUS back to the shop where I bought it in September 2006, and hopefully they will be able to fix it.
After some fiddling, I can get the IXUS to open the lens a bit and make photos, but they are badly unfocused, as you can see in this picture:
Thursday, July 19, 2007
Python-ish Spambot Checker Code
I'm the maintainer and webmaster of PERLI.NET, and as such, I have to make sure publicly-available forms for user-submitted entries are spambot-proof. If I was way too Web2.0-ish, I'd be using reCAPTCHA, but as this would make my website rely on that service, and because I already had some fool-proof solution to that problem, I decided to re-implement my old "Enter this number:" checker, this time in Python:
This function takes a number in the range 0-99 and returns a string representation of it, as it would be spoken in German (the main audience of PERLI.NET is German-speaking). Now, I'm displaying the returned string (for example "einunddreißig") on the guestbook entry form and also include a hidden field that has the answer encoded (I'm using the simple method of multiplying the number by two, but you might chose a more sophisticated algorithm). I'm also displaying a text entry box where the user types in the number.
The guestbook entry is only saved when the number is keyed in correctly, if not, the entry is silently ignored. That should save you some guestbook-entry-deleting work and make your visitors happy! You can view the resulting guestbook entry form here.
def number2german( num):
"""Convert a number to a German string
Convert a number to a German string, as it would be
spoken aloud. Valid range: 0 - 99
"""
specials = {0:'null',1:'eins',11:'elf',12:u'zwölf',
16:'sechzehn',17:'siebzehn'}
tens = ['','zehn','undzwanzig',u'unddreißig','undvierzig',
u'undfünfzig','undsechzig','undsiebzig',
'undachtzig','undneunzig']
ones = ['null','ein','zwei','drei','vier',u'fünf',
'sechs','sieben','acht','neun']
if num in specials.keys():
return specials[num]
if num < 0 or num > 99:
return ''
if num % 10 == 0:
if tens[num/10][:3] == 'und':
return tens[int(num/10)][3:]
else:
return tens[int(num/10)]
return u'%s%s' % ( ones[num%10], tens[int(num/10)], )
This function takes a number in the range 0-99 and returns a string representation of it, as it would be spoken in German (the main audience of PERLI.NET is German-speaking). Now, I'm displaying the returned string (for example "einunddreißig") on the guestbook entry form and also include a hidden field that has the answer encoded (I'm using the simple method of multiplying the number by two, but you might chose a more sophisticated algorithm). I'm also displaying a text entry box where the user types in the number.
The guestbook entry is only saved when the number is keyed in correctly, if not, the entry is silently ignored. That should save you some guestbook-entry-deleting work and make your visitors happy! You can view the resulting guestbook entry form here.
Sunday, July 15, 2007
Exaile: Python-based GTK media player
I don't know on which planet I got linked to Exaile, a GTK audio player written in Python. Anyway, I installed it some days ago and downloaded some plug-ins (IM status, which works great with Pidgin; and Mini Mode, which makes Exaile about as high as the Gnome panel and only about one third of my screen wide).
Exaile is great, it has a slick, but simple interface, nice features (the music library is created by adding paths to my music collections, which is a no-nonsense way of managing the library ;). Bulk downloading album art and album cover search are great features. Last.fm integration is built-in, and DAAP support can be enabled by installing a plug-in.
There are still some annoyances: For example, tracks with no album set are displayed as "Trackname from by Artist", i.e. the album title is simply missing. But thanks to the great build/run architecture of Exaile that makes it possible to patch and test/debug directly in the SVN working directory, I was quickly able to come up with a patch, which will hopefully be included in Exaile soon.
Other than that, I can only recommend Exaile. It is in active development, and the problems and bugs that are currently present will hopefully be patched out in future versions :)
Exaile is great, it has a slick, but simple interface, nice features (the music library is created by adding paths to my music collections, which is a no-nonsense way of managing the library ;). Bulk downloading album art and album cover search are great features. Last.fm integration is built-in, and DAAP support can be enabled by installing a plug-in.
There are still some annoyances: For example, tracks with no album set are displayed as "Trackname from by Artist", i.e. the album title is simply missing. But thanks to the great build/run architecture of Exaile that makes it possible to patch and test/debug directly in the SVN working directory, I was quickly able to come up with a patch, which will hopefully be included in Exaile soon.
Other than that, I can only recommend Exaile. It is in active development, and the problems and bugs that are currently present will hopefully be patched out in future versions :)
Thursday, July 12, 2007
Trading services that serve me well
There's so much Web 2.0 going on recently that I really see something big coming our way. The biggest "me, me!" of the last two or so years is getting some internet service going, think of some funky name, leave out the last vowel, throw in tagging, rating and user accounts with some fancy API (well, here's what I really love about Web 2.0: JSON and XML and all the REST-based stuff are really easy to understand, easy to implement and standards-based - see my python-youtube client for a trivial example). There are lots of examples out in the wild.
Of course, some services are really neat - I've talked about Doodle before, and Last.fm is nice for finding related music and having an automatically generated "toplist" of songs, possibly spread over many machines, iPods and music player, because all can interface with Last.fm.
There are other services in my favourites list that are not really Web2.0-ish, but have been around for a long time. The fact that these services are still around gives you a clue about their usefulness:
GameTZ, the Game Trading Zone is a service where one can list games (+hardware), movies, books and music to form a "available" list and list things they want to have (the "wanted" list). The website makes it easy to find matches which can then result in an offer being sent. Offer history is kept on the website (some kind of dialog between the two traders) and after an agreement has been made, the offer can be transformed into a trade, which will then be pending until both sides of the trade mark it as "received". A trade is usually followed by rating the other side of the trade. Really cool, and I've already made good trades on this site. My profile can be found here if you want to have a look.
The next service I really like (although I haven't used it recently) is db.etree.org, the Trader's Database. It's basically something like GameTZ (only the listing part, not the offer/trade/rate part), but for lossless live concert recordings. Many artists allow their fans to swap live recordings of their concerts in lossless formats over the Internet, and the Trader's Database makes this very easy. With special features like AJAX-based search and metadata that are useful for trading live music, it's really nice and easy to find shows on the 'net.
All these services come without any API and have all their vowels in place (or at least there are no vowels to leave out). Still, they rock. I hope when all the Web2.0 hype is over, that these services will survive.
Of course, some services are really neat - I've talked about Doodle before, and Last.fm is nice for finding related music and having an automatically generated "toplist" of songs, possibly spread over many machines, iPods and music player, because all can interface with Last.fm.
There are other services in my favourites list that are not really Web2.0-ish, but have been around for a long time. The fact that these services are still around gives you a clue about their usefulness:
GameTZ, the Game Trading Zone is a service where one can list games (+hardware), movies, books and music to form a "available" list and list things they want to have (the "wanted" list). The website makes it easy to find matches which can then result in an offer being sent. Offer history is kept on the website (some kind of dialog between the two traders) and after an agreement has been made, the offer can be transformed into a trade, which will then be pending until both sides of the trade mark it as "received". A trade is usually followed by rating the other side of the trade. Really cool, and I've already made good trades on this site. My profile can be found here if you want to have a look.
The next service I really like (although I haven't used it recently) is db.etree.org, the Trader's Database. It's basically something like GameTZ (only the listing part, not the offer/trade/rate part), but for lossless live concert recordings. Many artists allow their fans to swap live recordings of their concerts in lossless formats over the Internet, and the Trader's Database makes this very easy. With special features like AJAX-based search and metadata that are useful for trading live music, it's really nice and easy to find shows on the 'net.
All these services come without any API and have all their vowels in place (or at least there are no vowels to leave out). Still, they rock. I hope when all the Web2.0 hype is over, that these services will survive.
Wednesday, July 11, 2007
Subscribing to podcasts in gPodder
Until today, gPodder users were presented the same dialog for adding and editing "channels" (i.e. podcast subscriptions). When adding a channel, only the URL was needed, so all other UI elements were simply hidden in the dialog. As adding a channel only involves entering one URL, and as our new, shiny channel navigator adds some free space in the lower left corner of the window, I decided to re-work the GUI, inspired by Tasks's main window. Here's the difference:
Of course, with this change, I also had the chance to revise the channel edit dialog a bit (remember, it's the same dialog, used for both adding and editing). Apart from stripping out unneeded code and re-factoring the code, I've tried to make the dialog a bit more pleasant to use. For one, I've removed the cancel button and changed the "OK" button into a "Close" button.
I hope you like the changes and it makes using gPodder a little bit more pleasant. Time to roll another release some time soon..
Of course, with this change, I also had the chance to revise the channel edit dialog a bit (remember, it's the same dialog, used for both adding and editing). Apart from stripping out unneeded code and re-factoring the code, I've tried to make the dialog a bit more pleasant to use. For one, I've removed the cancel button and changed the "OK" button into a "Close" button.
I hope you like the changes and it makes using gPodder a little bit more pleasant. Time to roll another release some time soon..
Monday, July 9, 2007
Visualize your last.fm history
Oh neat! While at rk-nightly, I've found a nice web service that graphs your listening habits on last.fm. It's called LastGraph, and you can see my current graph here. Sadly, I have only set up Rhythmbox with my last.fm account, so only songs that reside in my RB playlist get picked up by the service.
Lovely microformats
Today, I came across the microformats.org page. Not that I haven't heard about these microformats before, but today I decided to have a look at how to implement them. After reading about the hCard format, I implemented it on the user pages of the new perli.net codebase (still to be published). I also added hCard content to my about page. Another microformat I've implemented on the new perli.net codebase is hCalendar. This format lets you integrate calendar data into an event info page. The new perli.net event detail pages will offer hCalendar data, which will let you add these events directly into your calendaring application of choice.
Testing and using microformats is as simple as installing the Operator Add-on for Firefox. You can put it in the lower right corner of your Firefox window (in the status bar) and see it light up when there is microformat content on the webpage you're visiting.
Other microformats I've implemented where I thought it would be useful: rel-tag for links to tag pages and rel-home for links back to the homepage. I personally think that microformats are a GoodThing(tm), and am eager to see these formats gain more widespread use and implementation.
Testing and using microformats is as simple as installing the Operator Add-on for Firefox. You can put it in the lower right corner of your Firefox window (in the status bar) and see it light up when there is microformat content on the webpage you're visiting.
Other microformats I've implemented where I thought it would be useful: rel-tag for links to tag pages and rel-home for links back to the homepage. I personally think that microformats are a GoodThing(tm), and am eager to see these formats gain more widespread use and implementation.
Saturday, July 7, 2007
Software release cycles
Releasing software is fun. This weekend, two of my projects released a new version: wavbreaker 0.8.1 and Tennix 0.3.2. Both releases only contain a handful of bugfixes and only minor feature enhancements, and in both cases, the changes leading to the release have been available in the respective SCM repositories for a few weeks already.
In the free software world, releasing software often is important. Normally, distribution vendors only ship released versions of software, so while having all bugfixes and minor enhancements in a public Subversion repository is good for the interested developer, these updates won't reach the users of distribution packages.
Of course, releasing software is tedious work: One has to fix up README files, update version information on the project's website, submit release information to sites like Freshmeat or the FSF Directory. All this takes time, and when I'm releasing a new version of gPodder, I find myself working a whole evening while preparing everything that is needed for a release. Still, I try to release a new version of gPodder every month, if there are changes, no matter how small.
Depending on your project, you should start making up some "release plan". Commit yourself to doing one release per month and be prepared to do interim releases if you have a code rush on a project and push many changes into the codebase.
Don't fear that users of your project might get upset about a new version every four weeks. If a bug crops up in code you suppose has been released too early, you can always quickly roll another release.
Last but not least, doing a release improves publicity of your project, as it will be listed as "updated" on free software news sites (Freshmeat, etc..), users will see that the project is actively worked on and you'll have some content for the "news" section of your project's homepage.
In the free software world, releasing software often is important. Normally, distribution vendors only ship released versions of software, so while having all bugfixes and minor enhancements in a public Subversion repository is good for the interested developer, these updates won't reach the users of distribution packages.
Of course, releasing software is tedious work: One has to fix up README files, update version information on the project's website, submit release information to sites like Freshmeat or the FSF Directory. All this takes time, and when I'm releasing a new version of gPodder, I find myself working a whole evening while preparing everything that is needed for a release. Still, I try to release a new version of gPodder every month, if there are changes, no matter how small.
Depending on your project, you should start making up some "release plan". Commit yourself to doing one release per month and be prepared to do interim releases if you have a code rush on a project and push many changes into the codebase.
Don't fear that users of your project might get upset about a new version every four weeks. If a bug crops up in code you suppose has been released too early, you can always quickly roll another release.
Last but not least, doing a release improves publicity of your project, as it will be listed as "updated" on free software news sites (Freshmeat, etc..), users will see that the project is actively worked on and you'll have some content for the "news" section of your project's homepage.
Wednesday, July 4, 2007
gPodder gets new channel navigator
My PyGTK pet project, gPodder is currently receiving a bit of a face-lift for its main window. After Carlos Moffat has raised some concern about the usability when being subscribed to many channels and suggested a "sidebar".
So, I first added some unread episodes information to the channel drop down list, so one can easily see where new episodes are to be found. Two e-mails later, I thought I should improve this a bit more and added a gtk.TreeView and a gtk.HPaned onto the Podcasts tab, displaying the top gtk.ComboBox's contents (thanks to the reusability of Gtk's TreeModel structures) in the left pane.The next step was adding a bit more design to the whole UI, and while fixing many bugs that cropped up while testing the new interface (and cleaning up the source code, for example unifying the new episodes algorithm into a single function), I finally had a nice-looking interface that gives the user much more information about the subscribed channels and new episodes. By using the channel's iTunes cover image (if available) as icon, a user can easily recognize specific podcast channels if the list is huge. Adding a bit of the channel's description below its title also gives a better overview over one's channels. The number in the sidebar tells us how many new podcast episodes are available for the channel. Here's how it looks currently:
The code should be available in gPodder's Subversion repository in the upcoming days after I've cleaned up the commit a bit. Enjoy :)
Web-based meeting scheduler.
Benji has just informed me that there's a nice web-based service for coordinating meetings and the like: Doodle. Just enter the possible dates, send out a link to participants and they can select the dates they are available and everybody can instantly see the results. Neat!
Tuesday, July 3, 2007
Pingback/TrackBack for Git?
Wouldn't it be cool if Pingback/Trackback (nice comparison here) could be implemented for the distributed VCS Git? As every object in Git is verified by its hash, security would be built-in, and new commits on top of the origin's master branch have to be verified by the user, anyway.
I'd see the following benefits of having something like Pingback/Trackback in Git:
I realize this could probably only implemented with gitserve repositories (i.e. git://) and ssh+git repositores, but not with "dumb" (i.e. http://) repositories, but maybe it'd be possible to add some kind of CGI (think: gitweb) addition to allow for pingbacks. If a repository stores the URL of the gitweb installation of this repository and gitweb is extended to allow saving of ping/track-back repository URLs, we could have ping/track-back for all our Git repositories.
I'd see the following benefits of having something like Pingback/Trackback in Git:
- Automatic mirror discovery to download objects from
- Upstream author has a nice overview of available branches/forks
- Stale repositories could link to recently updated repositories
I realize this could probably only implemented with gitserve repositories (i.e. git://) and ssh+git repositores, but not with "dumb" (i.e. http://) repositories, but maybe it'd be possible to add some kind of CGI (think: gitweb) addition to allow for pingbacks. If a repository stores the URL of the gitweb installation of this repository and gitweb is extended to allow saving of ping/track-back repository URLs, we could have ping/track-back for all our Git repositories.
Monday, July 2, 2007
New Tennix! website
Last Wednesday, I prepared a new website for Tennix!, my SDL-based tennis game. Tennix! has initially been developed on Borland C for DOS in 2003 as a summer project at school. The original Tennix is still available on the net.
In the middle of May this year, I've started porting the old DOS-based game (with custom-written mouse interface and 256-color bitmap laoder) to SDL. This has been quite easy, given the fact that I didn't use drawing routines back then but simply blitted BMP files onto the screen to generate the UI and gameplay graphics. Loading and blitting BMP files are two of the few graphics manipulation methods available in stock SDL.
Later, I've enhanced some graphics (scaled to 640x480) and modifed the gameplay to be real tennis-like (instead of the ping pong from 2003). In the last few weeks, I've played around with the codebase and improved the code, so I thought the project would now deserve its own webpage. Thanks to Ryan from icculus.org for providing the hosting of the new Tennix! website.
Subscribe to:
Posts (Atom)