I’m working a piece of legacy software, and in performing a code review came across what appeared to be some extraneous parentheses. I’ve renamed the terms as generically as possible because my employer would prefer it that way, but we have a calculation that in its original form looks like this, where every term is a floating point value:

calculated_value = a * (b * (c * d + e * f)) + g

Since multiplication is associative, the outermost set of parentheses should be redundant. Thinking that would be the case, a coworker and I rewrote the calculation so:

calculated_value = a * b * (c * d + e * f) + g

This does not always return the same results, though. I have tested this on both 2.7.4 and 3.3.1, and for a certain set of values, these two forms of the calculation definitely return different results. Here is a small, complete program that produces different results between the two forms of the calculation on both 2.7.4 and 3.3.1:

# test.py
def with_parens(a, b, c, d, e, f, g):
    return (a * (b * (c * d + e * f)) + g)

def without_parens(a, b, c, d, e, f, g):
    return (a * b * (c * d + e * f) + g)

the_values = (1.1523070658790489, 1.7320508075688772, 0.14068856789641426, 0.5950026782638391, 0.028734293347820326, 21.523030539704976, 2.282302370324546)

result_with = with_parens(*the_values)
result_without = without_parens(*the_values)
print((result_with == result_without), result_with, result_without)

» python test.py
(False, 3.68370978406535, 3.6837097840653494)

» python3 test.py
False 3.68370978406535 3.6837097840653494

It is fairly obvious that there is a precision difference here. I am familiar with the ways that floating point math can be odd, but I would not expect the loss of associativity to be one of them. And of course, it isn’t. It just took me this long in writing up what was originally going to be a post on comp.lang.python to see it.

What is actually going on here is that Python respects order of operations (as well it should!), and floating point math is imprecise. In the first case, a * b * (<everything else>), the first thing that happens is a is multiplied by b and the result is then multiplied by everything else. In the second case, a * (b * <everything else>), b is multiplied by everything else and the result is multiplied by a at the end. Many times, this doesn’t matter, but sometimes there is a slight difference in the results because of the loss of precision when performing floating point operations.

Lesson learned (again): floating point math is hard, and will trick you. What happens here is functionally a loss of the associative property of multiplication. The two calculations are in a pure mathematical sense equivalent. But floating point math is not the same as pure math. Not even close.

A few weeks ago, I accidentally clobbered my Python 3 installation on OS X. I had gotten pretty much everything set up again—or so I thought. To my chagrin, as I got working on a GUI development project this afternoon, I ended up working on getting Python virtual environments to play nicely with OS X’s special Framework version of Python. (The fact that I’m using the homebrew version of Python complicates things even more.)

Moreover, this is not specific to wxPython: the same issues will crop up with any GUI development setup using Python. Since this was an inordinately painful process, and one I’ve gone through before, I resolved to write up what I did to get it working, so that I can avoid going through this painful process of exploration, Googling, and face-palming in the future.

Gladly, the solution is pretty simple, albeit a little ugly.

I already had virtualenvwrapper installed, but if you don’t, you can install it easily enough. (Note: use your primary system Python—the one you execute by typing python and nothing else. Things get confused and messy with virtualenv and virtualenvwrapper if you don’t.)

$ pip install virtualenvwrapper
$ source /usr/local/bin/virtualenvwrapper.sh

To install Python 3, I ran the following command, using Homebrew to get a Framework build of Python 3 on my system:

$ brew install python3 --framework

Note that the command to install a Python 2 framework is the same other than the name of the brew (python versus python3).

Once you have virtualenvwrapper and a Framework Python installed, you can go ahead and create a virtual environment to use:

$ mkvirtualenv -p /usr/local/bin/python3 my_virtual_env

Now things get a little funky. For reasons that are not yet entirely clear to me, the virtualenv tool (around which virtualenvwrapper and its corresponding mkvirtualenv call wrap) creates a non-Framework version of Python, even when you give it the path to a Framework Python.¹ This is, to say the least, frustrating. However, there is a workaround: simply copy the executable you want to use over the one created in the virtual environment.

You may want to make a backup of the original first—I did.

$ cp ~/.virtualenvs/my_virtual_env/python3.3 ~/.virtual_envs/my_virtual_env/python3.3-backup
$ cp /usr/local/bin/python3 ~/.virtualenvs/my_virtual_env/bin

That should do the trick. Obviously you’ll still need to install wxPython (or whatever else) in the correct site packages directory. In my case, that simply involved one more step:

$ cp -r ~/Downloads/wx/* ~/.virtualenvs/my_virtual_env/lib/python3.3/site-packages

Summary

To summarize, here is the full installation process on OS X:

1. Install virtualenvwrapper if you don’t already have it

   $ pip install virtualenvwrapper
   $ source /usr/local/bin/virtualenvwrapper.sh
   

2. Install a Framework version of Python(3)

   $ brew install python3 --framework
   

3. Create and then fix the virtual environment:

   $ mkvirtualenv -p /usr/local/bin/python3.3 my_virtual_env
   $ cp ~/.virtualenvs/my_virtual_env/python3.3 ~/.virtual_envs/my_virtual_env/python3.3-backup
   $ cp /usr/local/bin/python3 ~/.virtualenvs/my_virtual_env/bin
   

One of the new features Apple launched with iOS7 was iTunes Radio, available not only to iPhone and iPad users but also to anyone who uses iTunes on Windows or OS X. While the service got some attention back when it was announced alongside iOS7 early in the summer, it has received surprisingly little focus since then.

Surprisingly, I say, because I’ve been using it every day since it went public a week and a half ago, and it is without question the single best internet radio I’ve ever used. Not “tying with Pandora”, not “a little better than Rdio”, but head-and-shoulders above anything else in the field. Ten days and a few conversations with friends and acquaintances along, I’m convinced the service is a gamechanger for the music industry—and possibly a very good one.

It works

The number one reason I think iTunes Radio is going to change the game is because it just works. I have lots of friends who have had good luck with Pandora over the years, and a fair number of acquaintances who swear by Spotify for finding new music. Neither of them has ever worked for me at all. Part of this is my taste in music, which runs toward neoclassical and film scores—two areas in which all previous internet radio services’ libraries have simply been shallow (and here I’m being generous). though Another part is that the algorithms have simply failed me: even in more mainstream music, it’s always been a hit and miss experience, with the services often surfacing things that really didn’t match the other music at all.

iTunes, by contrast, has not only provided quite a bit of good listening among artists and music I already knew I liked, but has also provided me with new selections I hadn’t heard before, but loved. No exaggeration: I have discovered more new music that I want to buy and add to my library in the last ten days via iTunes Radio than in a year of Spotify use or a half decade on Pandora. Moreover, this experience has been repeated on stations ranging from contemporary classical to one built on Mumford & Sons. The stations, with the tuning inputs I have supplied, have consistently surfaced music that was actually what I was looking for, and introduced me to artists I had not yet heard but really enjoyed.

iTunes Radio just works in a way that other services haven’t for me. Friends and acquaintances with whom I have discussed it have had exactly the same experience. I am not sure what Apple is doing for their algorithm—one friend suggests they might be using groups that have toured together—but it is working, and working well. Add in the ability to tune a station to switch between “Hits”, “Variety”, and “Discovery”, and iTunes Radio is just fantastic.

So the first half of the equation is the consumer-facing side of things: as it stands, I think iTunes Radio is the best service of its kind out there.¹

Purchasability

The other half of the equation is the impact iTunes Radio may have on musicians. The last decade has been an interesting time for the music industry: innovation in means of distribution has been enormous, and the number of artists putting music into the public space has exploded. However, the result has been that the bottom has fallen out of the industry. Top performers still make boatloads of money; everyone else is scrambling to find their feet.

Pandora and Spotify pay artists notoriously low rates—a point that artists have made time and again, and which I will therefore not belabor. Airplay on terrestrial radio may be an order of magnitude harder to come by, but it pays at least that much better, too. In any case, it’s beyond dispute at this point that for all but the biggest acts, Spotify will be an irrelevant source of revenue.² To date, streaming services have been at best a way of getting ears on sounds, in the hope that those ears turn into purchasing—but they rarely do. Why buy, when you can listen endlessly?

As such, music producers—whether labels or individual artists—have always struggled with streaming options. They may provide exposure, but they rarely provide the income necessary to keep making more music. Pandora and others have offered options to purchase music one likes, but it has always been a little kludgy. Here, again, iTunes has it nailed.

Purchasing a song I like in iTunes Radio is one click. Adding it to a wish list to buy later, or showing the item in the iTunes Store, is two clicks. All the friction is gone. This is great for music consumers: it makes owning music we like trivial and immediate. This in turn means that it is also great for music producers: the chance that someone actually buys music they heard streaming goes up enormously when it is this easy.³ Purchasability—conversion, in marketing terms—has always been the missing element in streaming solutions, and the reasons artists have been so suspicious. Here, it’s front and center, simple and straightforward, and attractive. Like that artist? Buy her stuff, right now.

Conclusion: Gamechanger

So there you have it. iTunes Radio, although it has largely been overlooked by tech media in the last few weeks, is a gamechanger. In fact, it may be the biggest gamechanger in music we’ve seen in the last few years, because unlike the myriad other streaming services we’ve seen, it actually offers the promise of revenue to artists, while providing real value to consumers. I expect that the next few months will see iTunes Radio become increasingly popular as word of mouth spreads that it is just plain better than the alternatives. I also expect that as a result, people will starting buying more music again. That’s a net benefit to everyone. (And if somebody can come out and compete with Apple on both terms, more power to them: that, too, will benefit everyone.)

For nearly all of August, I have been helping a colleague work on a massive code refactoring project. The company I am consulting for relies on a decades-old (though quite robust) piece of engineering modeling software. The software was written in FORTRAN77 in the mid-to-late 1980s, as was common for such packages at the time. It is apparent that it was not written by competent software developers, however: the entire code-base smacks of fastest-way-to-get-it-to-work thinking. All of the code—all 4,000+ lines of it—came in a single file. Every variable in the program is global.¹

Yes, you read that right. Every variable in the program is global.

Since no one at the company is an expert in Fortran at this point—even a modern dialect like Fortran 95²—we have been translating the whole thing to Python in order to start re-working pieces of it. (We’re using Python because that’s the company’s standard; we’re reworking this piece of code for reasons long and uninteresting.) The formal translation step was completed in mid-July; since then we have been working on refactoring the project so that we can actually change things without breaking everything.

The trick, of course, is all those global variables. In order to be able to begin making any other changes, we have to eliminate the global state. Otherwise, a change in one module can (and often does) have unexpected effects elsewhere.

Our strategy has been simple: take things in small steps, albeit often very time-consuming small steps. In each of those steps we have aimed to make single, incremental change that makes things more explicit and safer. Often, this has simply meant passing and returning inordinate numbers of variables back and forth to functions, simply so that the behavior is explicit (not to say clear). At other times, this has meant doing very bad things that would be intolerable under any other circumstances, but were less bad than what we had before and allowed us to step forward toward our ultimate goals. The prime example—and one of which I’m fairly proud, though I hope never to have to use it again—is leveraging Python’s property decorators to allow us to begin using struct-like objects for passing around large amounts of data before all the global state involved in said structs was eliminated.

Basically, we created a stopgap wrapper that made struct access result in changing global state. That way, functions could receive the struct and act on it, with the global state invisible to them: from any function’s perspective, it is performing perfectly normal access/mutation operations on the class fields. Behind the scenes, we manipulated global state, until we were able to eliminate global dependencies for that variable entirely, at which point we replaced the quirky behavior with standard field initialization.

Here is the awful pattern my colleague and I used as part of that transition, followed by some explanation. (Do not attempt at home!)

class BasicallyAStruct():
    def __init__(self, some_field):
        global some_field_
        some_field_ = some_field

    @property
    def some_field(self):
        return some_field_

    @some_field.setter
    def some_field(self, value):
        global some_field_
        some_field = value

Here’s how it works. For starters, we renamed every global variable to end in an underscore (like some_field_) to make it easy to distinguish between global and local variables. Then, we created a class that is basically just a struct to hold data. (It can of course be expanded to be a full-up class with methods later if that makes sense.) In the constructor, we declare every global variable that the struct needs to reference, and then assign it the value passed to the constructor. Then we use Python’s property decorators to specify the access and mutation behavior: instead of storing the value to a class property like normal, calling the setter or getter³ actually returns or sets the value in the global variable. The result:

global some_field_  # set to some old value
my_struct = BasicallyAStruct(some_field_)  # create the struct

# Set
my_struct.some_field = new_value  # assign just like normal
print(some_field_)  # new_value

# Get
print(new_value == my_struct.some_field)  # True

Once we get to a point where we’ve completely eliminated the global state, we change the class definition to look like a normal class definition, completely removing the global declaration and the @property and @some_field.setter decorators:

class BasicallyAStruct():
    def __init__(self, some_field):
        self.some_field = some_field

From the functions using the struct, nothing has changed; they are already using standard class property access notation.

It has worked like a charm, and it demonstrates the power of decorators in a bit of an unusual situation. It is, of course, an awful use of decorators, to the extent that I would call it an abuse in general. If I ever found this in “final” code I would probably make a horrible noise in outrage; it’s a stupid thing to do. It is, however, less stupid than keeping everything global, and it made for a good intermediate solution that allowed us to minimize changes at each step along the way and therefore minimized the potential number of places anything could break as we refactored.⁴

I’m happy to say that almost all of those global variables are gone, and the classes are all looking pretty much like normal classes now. And the calling functions never even noticed.

I’m incredibly happy with how that came out—and I hope never to do anything like it again.

Or, Reflections Prompted by the One-Year Anniversary of App.net

Yesterday, App.net turned a year old. Most of my friends in “real life” have never even heard of the social network/platform,¹ so the anniversary rightly went by unnoticed and unheralded. For me, however, the day that App.net went live last year was the start of a sea change in the way I approach the software I use — a sea change that is still in progress in many ways. What changed? I decided I wanted to stop being a product and start being a customer.

It’s worth reading the original proposal for App.net: a social networking service, for which at least some users pay. This was incredibly audacious for a social networking backbone (though, as App.net founder Dalton Caldwell noted, not completely unheard of in internet services: GitHub, Dropbox, and a number of others offer the same model). Twitter was making it clear that they were following Google and Facebook’s models of selling customer info to advertisers, leaving developers and users of the ecosystem to the whims of whatever made the most sense for sales teams. Facebook’s monetization plans were becoming increasingly annoying (ads in my News Feed? No, that really isn’t why I’m here…). Google’s data-mining was well known but increasingly starting to bother me. All of these had a common thread: they were free to use, on the premise that we don’t mind being advertised to constantly and our data being sold to and analyzed by advertising companies. More and more, though, I did mind. The idea that I would pay for access to a social networking backbone would have seemed crazy at some point; no more.

The web was born and bred on free. In many ways, that has been a good thing. The democratizing effects of free access are well-documented, and people’s ability to publish and read such a wide array of information has been a great boon to the causes of education and liberty. On the other hand, those trends have engendered an expectation that everything digital ought to be free, and this expectation has had a great many deleterious effects as well. We have seen mass piracy of music and video, cutthroat rates (often consisting of “exposure”) for writing, and a flood of terrible content that always threatens to overwhelm the good content available. On balance, I think the web has been a good thing—but it has not been an unalloyed good, and in the last few years an increasing number of people have become dissatisfied with the status quo.

People deserve to be paid for their work, and at some point we have to figure out how to make that happen. Ads have proven insufficient to generate the revenue needed for sustainable business, but they have also incentivized content-providers to aim at generating the most hits, rather than providing the best content. This is not a new problem; the same issue drove a lot of the “yellow journalism” of the last century. The difference is that, coupled with the new, massive data and the disconnecting nature of the web (you don’t read a single “paper” anymore, you read a bunch of articles from various places all over the web), advertisizers have had both the means and the motive to pursue practices that are increasingly proven corrosive to privacy and, in the long run, contrary to the best interests of the public.

The way out—and the way that an increasing number of voices have embraced, including those behind App.net—is to go back to paying for things. (Shocking, I know!) With App.net, Dropbox, GitHub, and many other web services, users pay for access to premium functionality. App.net does not sell my data to anyone; they sell the engineering backbone as a service to me, as well as to developers who want to use that backbone to create their own services. Likewise, Dropbox doesn’t sell my data, they sell storage. GitHub doesn’t sell my data, they sell version control hosting.

I love this idea. So much so that I’m increasingly pulling out of ad-supported services where I’m the product instead of the customer, and moving toward things I pay for. It makes sure the service’s incentives are aligned with my best interests as a user, rather than orthogonally and possibly detrimentally to my desires. Not least, it makes sure the people building the service I’m using get paid. Those are good things. It costs more, to be sure, but that helps me think through what to use (and what to skip) more carefully, and that’s a good thing, too.²

Now, if we only someone could figure out a good way to do the same thing with music, writing, and video…

Oh, and App.net? It’s the single best social network I’ve ever used—the quality of conversations I’ve had there is far, far better than that I’ve had anywhere else. You should check it out.

We were at the doctor’s office for a checkup on our little girl late last week, and I have a Greek class coming up – a tough class for which I figured I’d spend some time studying and preparing while waiting for the doctor. Unfortunately, I just discovered yet another of the thousand little details Google doesn’t think matter. I can’t study Greek on my phone here because Google refuses to ship (or allow users to install) a typeface that includes the polytonic Greek character set on Android. The default system font, Roboto (a poor clone of Helvetica), has accents but no breathing marks. This means that at least half the words in classical and Koine Greek are simply missing letters on any web page, and for that matter any app that doesn’t embed its own fonts.

This is not a make or break issue with the phone, but as my primary tool for mobile work in situations like this, it is quite frustrating. What is more frustrating is Google’s absolute unresponsiveness about this. There are many ways the issue could be fixed, none of them particularly complex or time-consuming (unless the folks working on android have made some really silly decisions and coupled the font deeply in the OS, which I don’t believe for a second). No, by all appearances the explanation for the still-open ticket is quite simple: Google doesn’t care. Read on, intrepid explorer →

A good friend of mine has been making the transition to Mac from Windows recently, and the experience has highlighted a couple of things that Apple really ought to do different and better with OS X:

1. Turn the Dashboard off by default.
2. Provide information about app installation and uninstallation.

Read on, intrepid explorer →