...making Linux just a little more fun!

<-- prev | next -->

PyCon 2006 Dallas

By Mike Orr (Sluggo)

My most distinct impression of Dallas was the birds. Two hundred on the electric wires. Another two hundred swoop in for the party. Others sit on nearby trees. Four times as many birds as I've ever seen in one place. All different kinds, all chirping loudly at each other. Now I know where birds go when they fly south for the winter: to Dallas. Especially to the intersection of Belt Line Road and the North Dallas Tollway. Around the intersection stand four or five steel-and-glass skyscrapers. Corporate execs watch the birds through their windows -- and get pooped on when they venture outside. But this isn't downtown; it's a suburb, Addison, with a shopping mall across the tollway. Behind CompUSA is the Marriott hotel where the Python conference took place.

PyCon was started three years ago as a low-budget developers' conference. It had always met at the George Washington University in Washington DC, but attendance has been growing by leaps each year and had reached the facility's capacity. After a nationwide search, the Python Software Foundation decided to sign a two-year contract with a Dallas hotel. That was partly because the Dallas organizers were so enthusiastic and hard-working, the hotel gave us an excellent deal, and we wanted to see how many people in the southern US would attend a conference if it were closer to home. There's no scientific count, but I did meet attendees from Texas, Utah, Arizona, Nevada, and Ohio, many of whom said they had never been to PyCon before. Overall attendance was 410, down from 450. Not bad considering the long move and the fact that many people won't go to Dallas because it's, well, Dallas. But the break-even point was in the high 300s so it didn't bust the bank. The layout had all the meeting rooms next to each other and there were sofas in the hallway, so it was easier to get to events and hold impromptu discussions than last year. The hotel staff was responsive; there were techs on-hand to deal with the sound system. The main problem was the flaky wi-fi, which will hopefully be improved next year (read "better be improved next year".) A hand goes out to Andrew Kuchling, who proved his ability not only in coding and documentation ("What's New in Python 2.x?") but also in conference organizing. This was his first year running PyCon, and he got things running remarkably smoothly.

There seemed to be more international attendees this year. I met people from the UK, Ireland, Germany, the Netherlands, Sweden, Japan, Argentina, and a German guy living in China. This is in spite of the fact that EuroPython and Python UK are now well established, and the main pycon.org site is now an umbrella covering both.

Here's the conference schedule. Several of the talks were audio- or video-recorded and will be available here.

Keynotes

Guido Note 1 delivered two keynotes. One was his usual "State of the Python Universe". The other was a look backward at Python's origins. The latter covered territory similar to this 2003 interview, which explains how Guido created Python to try out some language ideas and improve on ABC, a language he'd had both good and bad experiences with. He also explained why Python doesn't have type declarations: "Declarations exist to slow down the programmer." There you have it.

The state of the Python universe has three aspects: community activity, changes coming in Python 2.5, and changes coming in Python 3.0. There has been a lot of activity the past year:

An .egg is like a Java .jar file: a package that knows its version, what it depends on, and what optional services it can provide to other packages or take from other packages. This is similar to .rpm and .deb but is OS-neutral. It is expected that Linux package managers will eventually use eggs for Python packages. Eggs can be installed as directories or zip files. easy_install.py is a convenient command-line tool to download and install Python packages (eggs or tarballs) in one step. It will get the tarball from the Python Cheese Shop Note 2 (formerly known as the Python Package Index), or scrape the Cheese Shop webpage for the download URL. You can also provide the tarball directly, or another URL to scrape. Regardless of whether the original was an egg or a simple tarball, EasyInstall will install it as an egg, taking care of the *.pth magic needed for eggs.

Waitress: Well, there's egg and bacon; egg sausage and bacon; egg and spam; egg bacon sausage and spam; spam bacon sausage and spam; spam egg spam spam bacon and spam....
Wife: Have you got anything without spam?
Waitress: Well, there's spam egg sausage and spam, that's not got much spam in it.
Wife: Could you do the egg bacon spam and sausage without the spam then?
--Monty Python's Spam skit

Python 2.5 changes

The first alpha is expected May 6; the final by September 30. Python 2.4.3 will be released in April, followed by 2.4.4, the last of the 2.4 series. What's New in Python 2.5 is unfinished but explains the changes better than I can.

The most sweeping changes are the generator enhancements (PEP 342) and the new with keyword (PEP 343). These allow you to write coroutines and safe blocks. "Safe" means you can guarantee a file will be closed or a lock released without littering your code with try/finally stanzas. The methodology is difficult to understand unless you have a Computer Science degree, but the standard library will include helper functions like the following: 

#### OPENING/CLOSING A FILE ####
f = open("/etc/passwd", "r")
with f:
 for line in f: # Read the file line by line.
 words = line.strip().split(":")
 print words[0], words[1] # Print everybody's username and password.
 # The file is automatically closed when leaving this block, no matter
 # whether an exception occurs, or you return or break out, or you simply
 # fall off the bottom.

#### THREAD SYNCHRONIZATION ####
import thread
lock = thread.allocate_lock()
with lock:
 do_something_thread_unsafe()
 # The lock is automatically released when leaving this block.

#### DATABASE TRANSACTION ####
with transaction:
 c = connection.cursor()
 c.execute("UPDATE MyTable SET ...")
 # The transaction automatically commits when leaving this block, or
 # rolls back if an exception occurs.

#### REDIRECT STANDARD OUTPUT ####
f = open("log.txt", "w")
with stdout_redirected(f):
 print "Hello, log file!"
 # Stdout is automatically restored when leaving this block.

#### CURSES PROGRAMMING ####
with curses.wrapper2() as stdscr: # Switch the screen to CURSES mode.
 stdscr.addtext("Look ma, ASCII graphics!")
 # Text mode is automatically restored when leaving this block.

The same pattern works for blocking signals, pushing/popping the locale or decimal precision, etc.

Coroutines are generators that you can inject data into at runtime. Presumably the data will be used to calculate the next yield value. This not only models a series of request/response cycles, but it also promises to radically simplify asynchronous programming, making Twisted much more accessible. Twisted uses callbacks to avoid blocking, and that requires you to split your code into many more functions than normal. But with coroutines those "many functions" become a single generator.

Other changes for 2.5 include:

Guido resisted ctypes for a long time because it "provides new ways to make a Python program dump core" by exposing it to arbitrary C bugs. Skip Montanaro responded by listing several ways you can already make Python dump core (try these at home), and it was decided that ctypes wasn't any worse than those.

Python 3.0 changes

Now that Guido is employed at Google and can spend 50% of his paid time hacking Python, 3.0 doesn't have to wait until sometime after his 5-year-old son Orlijn graduates college. The planned changes for 3.0 are listed in PEP 3000. Guido highlighted the string/unicode issue: the str type will be Unicode, and a new bytes type will be for arbitrary byte arrays.

There's one late-breaking change, or rather a non-change: the unloved lambda will remain as-is forever. lambda creates anonymous functions from expressions. Attempts to abolish it were undone by use cases that required syntax that's arguably worse than lambda, and Ruby-style anonymous code blocks didn't fare any better. Programmers who overuse lambda should still be shot, however.

BitTorrent

The other keynotes were on Plone and BitTorrent. I missed the Plone talk, but for BitTorrent Steve Holden interviewed Bram Cohen, BitTorrent's creator. Bram talked about hacking while slacking (he wrote most of his code while intentionally unemployed and living on savings), his new job as the owner of Bittorrent Inc (not much time to code), why he chose Python for BitTorrent, why BitTorrent doesn't make you into a carrier of malware (you won't be uploading anything you didn't specifically request to download), Pascal, and many other topics.

I made a wonderful faux pas the first evening at dinner when I sat next to Bram, unaware he was going to speak. I asked what he did, and he said he invented BitTorrent. I didn't remember what that was and kept thinking of Linux's former version control system, whose name I couldn't remember. Yet the term "torrent file" kept crossing my brain, clearly a configuration file and not related to the Linux kernel. Finally I remembered, "BitTorrent is that distributed file download thing, right?" Bram said yes. So I asked the guys across the table, "What was the name of Linux's former version control system?" They said, "BitKeeper". Ah ha, no wonder I got them confused. I thought no more about it, then Bram ended up mentioning BitKeeper several times during his keynote, usually in terms of how bad it is. He talked about BitKeeper vs. git (Linux's new version control system), git vs. other things, and then about BitTorrent vs. Avalanche. Avalanche is a distributed file download system from Microsoft, which Bram called vaporware in his blog, stirring up a lot of controversy (including a newspaper article in Australia).

For those who think BitTorrent is all about illegally downloading copyrighted songs and movies, Bram points to SXSW, a music and film festival which will be using BitTorrent to distribute its performances. "The Problem with Publishing: More customers require more bandwidth. The BitTorrent Solution: Users cooperate in the distribution." Other articles point out that a BitTorrent client normally interacts with 30-50 peers, reducing the strain on the original server by thirtyfold.

Bram also warned people to download BitTorrent directly from www.bittorrent.com and not from some random Google search. Shady operators are selling scams that claim to be BitTorrent but contain spyware or viruses. The real BitTorrent is free to download, and is Open Source under a Jabber-like license. The company does accept voluntary donations, however, if you really want to give them money.

Session talks

The rest of PyCon was session talks, tutorials, lightning talks, Open Space, sprints, and informal "hallway discussions". The most interesting talks I saw or wanted to see were:

Python in Your Pocket: Python for Series 60
Nokia has released a version of Python for its S60 mobile phone which includes Bluetooth and mobile Internet. The slides showed it displaying stats from the GSM towers, weather reports, a stock market graph, an online dictionary, and a stupid guitar tuner. There are Python libraries to access the phone's battery usage, to dial a number, take a picture, manage your inbox and address book and calendar, and display arbitrary graphics and sound. You can write programs using an emulator and download the working programs to the phone via Bluetooth. (Note: the URL points to a slide show. Click anywhere to go to the next page. There appears to be no way to go backward.)
The State of Dabo
Dabo is an application development framework inspired by Visual Basic. It provides a high-level API for GUI design and SQL database viewers. The database viewer is something I've wanted for years: a simple way to make CRUD grids and data-entry forms similar to Microsoft Access. Only MySQL, PostgreSQL, and Firebird are supported so far, but others are on the way. The GUI API is so elegant that the wxPython developers themselves have started using it rather than their own API, which is a thin wrapper around the wxWidgets C++ library.
State-of-the-Art Python IDEs
A survey of several Python IDEs, based on research done by the Utah Python Users' Group.
Building Pluggable Software with Eggs
A tutorial on Python's new packaging format.
Docutils Developers Tutorial: Architecture, Extending, and Embedding
Docutils is the project name for ReStructured Text (ReST), a wiki-like markup format. Several talks mentioned using Docutils, and several Python projects have adopted it for documentation, data entry, etc. One talk explained how to make slides with it. Another speaker didn't make slides and had to download and learn Docutils the night before the talk, but he was able to get his slides done anyway.
What is Nabu?
ReST includes a construct called field lists that look like email headers; they represent virtual database records. ReST does not define the database or say what kind of record it is, only that it's a record. Nabu is the first application I've seen to fully exploit this potential. Nabu helps you extract relevant records from all documents into a central database, which can then be queried or used in an application. You'll have to decide beforehand which fields a "contact record" or "URL bookmark" should contain, and how you'll recognize one when you see it. Nabu's author Martin Blais keeps an editor open for every task he does, to record random notes, TODO plans, contacts, and URL bookmarks. The random notes are unstructured ReST sections; the contacts are field lists scattered throughout the document. He also has a file for each trip he's been on, containing random observations, restaurants to remember (or forget), etc. Nabu is also suitable for blog entries, book lists, calendar events, class notes, etc. One constraint is you must put "something else" between adjacent records so ReST knows where each record ends. Putting multiple records in a bullet list satisfies this constraint.
Web frameworks
"Python has more web application frameworks than keywords," it has been rightly observed. A few people go further and say, "You're not a real Python programmer until you've designed your own web framework." (Hint: subclass BaseHTTPServer.HTTPServer in the standard library; it's easy!) So it's not surprising that PyCon had three talks on TurboGears, two on Django, three on Zope (plus several BoFs), and a couple on new small frameworks. All the discussion last year about framework integration has made them more interoperable, but it has not cut the number of frameworks. If anything, they've multiplied as people write experimental frameworks to test design ideas. Supposedly there's a battle between TurboGears and Django for overall dominance, but the developers aren't competing, they just have different interests. Jacob Kaplan-Moss (Django developer) and I (TurboGears developer) ran the Lightning Talks together, and we both left the room alive. Some people work on multiple frameworks, hoping the holy grail will eventually emerge. Much of the work focuses on WSGI and Paste, which help tie diverse components using different frameworks together into a single application. Some work focuses on AJAX, which is what makes Gmail and Google Maps so responsive and is slowly spreading to other sites.

Lightning talks

Lightning talks are like movie shorts. If you don't like one, it's over in five minutes. They are done one after the other in hour-long sessions. Some people give lightning talks to introduce a brand-new project, others to focus on a specialized topic, and others to make the audience laugh. This year there were seventeen speakers for ten slots, so we added a second hour the next day. But in the same way that adding a new freeway lane encourages people to drive more, the number of excess speakers grew rather than shrank. We ended up with thirty speakers for twenty slots, and there would have been more if I hadn't closed the waiting list. The audience doesn't get bored and keeps coming back, so next year we'll try adding a third hour and see if we can find the saturation point. Some of the highlights were:

Testosterone
"A manly testing interface for Python." It's a CURSES-based front end to Unittest. It may work with py.test too since it just displays whatever the underlying routine prints.
rst.el
A ReStructured Text mode for Emacs.
Python in Argentina
Facundo Batista talked about Python use in Argentina and the PyAr users group. There are a hundred members throughout the country, and around fifteen that gather for a meeting in Buenos Aires.
Internet Censorship in China
Philipp von Weitershausen gave a quick overview of what it's like to use the Internet in China. There are 120 million users, with 20 million more added every year. Chinese sites respond much more slowly than international sites. Some 50,000 officials monitor what sites people visit. Wikipedia, the BBC, and VOA are blocked completely due to undesirable political content. Other sites go up and down, and the error keeps changing. Sometimes it's a timeout, other times an error message. Encrypted connections timeout. How do people feel about not having access to Wikipedia? Only 2% of Chinese Internet users know it exists, so most don't know what they're missing or think it's a big deal.
How to Replace Yourself with a Small Bot
Ben Collins-Sussman had too much time on his hands and wrote an IRC bot that masqueraded as himself. He put it on a channel and waited to see how long until people noticed. The bot had five or so canned remarks like "Did you check the FAQ?", which it recited at random whenever anybody addressed him. Hmm, maybe The Answer Gang here at the Gazette needs one of those. The experiment didn't succeed too well because the bot said wildly inappropriate things at the wrong time.

[ I don't understand what you mean by "didn't succeed", Mike - seems to me that wildly inappropriate responses to a question is perfectly normal IRC behavior... so how could anyone tell? -- Ben ]

There were other good talks too but since I was coordinating I couldn't devote as much attention to them as I would have liked.

Sprints

I attended the TurboGears sprint and worked on Docudo, a wiki-like beast for software documentation, which groups pages according to software version and arbitrary category and distinguishes between officially-blessed pages and user-contributed unofficial pages. We wrote up a spec and started an implementation based on the 20-Minute Wiki Tutorial. This version will store pages in Subversion as XHTML documents, with Subversion properties for the category and status, and use TinyMCE for editing. TinyMCE looks like a desktop editor complete with toolbars, but is implemented in Javascript. We've got parts of all these tasks done in a pre-alpha application that sort of works sometimes.

Other TurboGears fans worked on speeding up Kid templates, adding unittests, improving compatibility with WSGI middleware and Paste, using our configuration system to configure middleware, and replacing CherryPy with RhubarbTart and SQLObject with SQLAlchemy. Don't get a heart attack about the last two: they are just experimental now, won't be done till after TurboGears 1.0, and will require a smooth migration path for existing applications. We had a variety of skill levels in our sprint, and some with lesser skills mainly watched to learn some programming techniques.

There were ten people in the TurboGears sprint and fifty sprinters total. Other sprinters worked on Zope, Django, Docutils, the Python core, and a few other projects.

The sprint was valuable to me, even though I'm not fully committed to TurboGears, because I'm starting to write TurboGears applications at work: it was good to write an application with developers who know more than I do about it. That way they can say, "Don't do that, that's stupid, that's not the TG way." I would have liked to work on the RhubarbTart integration but I had to go with what's more practical for me in the short term. So sprinting is a two-way activity: it benefits the project, and it also benefits you. And it plants the seeds for future contributions you might make throughout the year.

Impressions of Dallas

Dallas was not the transportation wasteland I feared (ahem Oklahoma City, Raleigh, Charlotte...) but it did take 2-3 hours to get to PyCon without a car, and that includes taking a U-shaped path around most of Dallas. An airport shuttle van goes from the sprawling DFW campus to the south parking lot a mile away. From there another airport shuttle goes to the American Airlines headquarters, an apartment building (!), and finally the Dallas - Fort Worth commuter train. That's three miles or thirty minutes just to get out of the airport. The train goes hourly till 10:30pm, but not on Sundays. It cost $4.50 for an all-day rail/bus pass. The train pokes along at a leisurely pace, past flat green fields and worn-down industrial complexes, with a few housing developments scattered incongruously among the warehouses. Freight trains carried cylindrical cars labelled "Corn Syrup" and "Corn Sweetener". Good thing I wasn't near the cars with a hatchet; I feel about corn syrup the way some people feel about abortion. The train stopped in downtown Dallas at an open section of track euphemistically called "Union Station". I transferred to the light rail (blue line) going north. This train was speedy, going 55 mph underground and 40 mph above, with stops a mile apart. Not the slowpoke things you find in San Jose and Portland; this train means business. The companies along the way seem to be mostly chain stores. At Arapaho Station (pronounced like a rapper singing, "Ah RAP a ho!") in the suburb of Richardson, I transferred to bus 400, which goes twice an hour. A kind soul on the train helped me decide which bus to catch. The bus travels west half an hour along Belt Line Road, a six-lane expressway. It crosses other six-lane expressways every twenty blocks. Dallas has quite the automobile capacity. We're going through a black neighborhood now. The driver thinks the Marriott is a mile away and I should have gotten another bus, but the hotel map says it's at Belt Line Road and the North Dallas Tollway. When we approach the intersection with the birds, all is explained. The road the hotel is named after goes behind the hotel and curves, meeting the tollway. So the map was right.

[ A note from my own experience in Dallas, where I teach classes in the area described above: a shuttle from DFW is ~$20 for door-to-door service, and takes less than a half an hour. -- Ben ]

Around the hotel is mostly expense-account restaurants for the executive crowd. We didn't find a grocery store anywhere. So I learned to eat big at meals because there wouldn't be any food until the next one. There was a mall nearby and a drugstore, for all your non-food shopping.

The weather was... just like home (Seattle). Drizzle one day, heavy rain the next, clear the day after. Same sky but ten degrees warmer. Then the temperature almost doubled to 80 degrees (24C) for three days. In February! I would have been shocked but I've encountered that phenomenon in California a few times.

Saturday evening a local Pythoneer took two of us to downtown Dallas. Downtown has several blocks of old brick buildings converted to loft apartments and bars and art galleries, and a couple coffeehouses and a thrift shop. Another feature is the parking lot wavers. I swear, every parking lot had a person waving at the entrance trying to entice drivers. It's 10pm, you'd think the lot attendants would have gone home. Especially since there were plenty of metered spaces on the street for cheaper. There weren't many cars around: there were almost as many parking lots as cars! It was a bit like the Twilight Zone: all these venues and not many people. We went to Café Brazil, which is as Brazilian as the movie. In other words, not at all.

PyCon will be in Dallas next year around February-April, so come watch the birds. The following year it might be anywhere in the US.

Footnotes

1 Guido van Rossum, Python's founder.

2 The name "Cheese Shop" comes from a Monty Python skit. The Cheese Shop was formerly called the Python Package Index (PyPI), but was renamed because PyPI was confused with PyPy, a Python interpreter written in Python. They are both pronounced "pie-pie", and the attempt last year to get people to call PyPI "pippy" failed. Some people don't like the term "Cheese Shop" because it doesn't sell any cheese. But the shop in the skit didn't either.

Talkback: Discuss this article with The Answer Gang

picture Mike is a Contributing Editor at Linux Gazette. He has been a Linux enthusiast since 1991, a Debian user since 1995, and now Gentoo. His favorite tool for programming is Python. Non-computer interests include martial arts, wrestling, ska and oi! and ambient music, and the international language Esperanto. He's been known to listen to Dvorak, Schubert, Mendelssohn, and Khachaturian too.

Copyright © 2006, Mike Orr (Sluggo). Released under the Open Publication license unless otherwise noted in the body of the article. Linux Gazette is not produced, sponsored, or endorsed by its prior host, SSC, Inc.

Published in Issue 124 of Linux Gazette, March 2006

<-- prev | next -->
Tux