This version of the site is now archived. See the next iteration at v4.chriskrycho.com.
Topic: “development”

Floating Point Math is Hard (i)

I’m working a piece of legacy software, and in performing a code review came across what appeared to be some extraneous parentheses. I’ve renamed the terms as generically as possible because my employer would prefer it that way, but we have a calculation that in its original form looks like this, where every term is a floating point value:

calculated_value = a * (b * (c * d + e * f)) + g

Since multiplication is associative, the outermost set of parentheses should be redundant. Thinking that would be the case, a coworker and I rewrote the calculation so:

calculated_value = a * b * (c * d + e * f) + g

This does not always return the same results, though. I have tested this on both 2.7.4 and 3.3.1, and for a certain set of values, these two forms of the calculation definitely return different results. Here is a small, complete program that produces different results between the two forms of the calculation on both 2.7.4 and 3.3.1:

# test.py
def with_parens(a, b, c, d, e, f, g):
    return (a * (b * (c * d + e * f)) + g)

def without_parens(a, b, c, d, e, f, g):
    return (a * b * (c * d + e * f) + g)

the_values = (1.1523070658790489, 1.7320508075688772, 0.14068856789641426, 0.5950026782638391, 0.028734293347820326, 21.523030539704976, 2.282302370324546)

result_with = with_parens(*the_values)
result_without = without_parens(*the_values)
print((result_with == result_without), result_with, result_without)
The results:
» python test.py
(False, 3.68370978406535, 3.6837097840653494)

» python3 test.py
False 3.68370978406535 3.6837097840653494

It is fairly obvious that there is a precision difference here. I am familiar with the ways that floating point math can be odd, but I would not expect the loss of associativity to be one of them. And of course, it isn’t. It just took me this long in writing up what was originally going to be a post on comp.lang.python to see it.

What is actually going on here is that Python respects order of operations (as well it should!), and floating point math is imprecise. In the first case, a * b * (<everything else>), the first thing that happens is a is multiplied by b and the result is then multiplied by everything else. In the second case, a * (b * <everything else>), b is multiplied by everything else and the result is multiplied by a at the end. Many times, this doesn’t matter, but sometimes there is a slight difference in the results because of the loss of precision when performing floating point operations.

Lesson learned (again): floating point math is hard, and will trick you. What happens here is functionally a loss of the associative property of multiplication. The two calculations are in a pure mathematical sense equivalent. But floating point math is not the same as pure math. Not even close.

Four more anti-patterns

Another set of absolutely lovely gems we found this week: an empty for, the almost-impossible if, continue just because, and source that doesn’t match the executable. Oh my!

The empty for

I’d seen this before, but we found it again while trying to diagnose an (unrelated) infinite loop bug in our source (more on that below): Read on, intrepid explorer →

Two (absurd) anti-patterns

A pair of anti-patterns I’ve run into recently in my software development work, both of which are absolutely awful, though in completely different (and quite distinct) ways. I thought I’d share The Empty If and Wash, Rinse, Repeat, just so the world can share a bit of my pain. Read on, intrepid explorer →

Don’t Sell (Out)

Today, it came to my attention that Avid, maker of various audio and video processing tools – tools you’ve heard of if you’re in those industries, high profile names like Pro Tools – has closed down the main office responsible for developing Sibelius and sourced the development to a team in the Ukraine.

Nothing against the Ukrainians, but shutting down the London development office responsible for nearly two decades and replacing it with a team a third its size does not bode well for future development.

Box art for Sibelius 7

This came out in as underhanded a way as possible, with the sorts of PR doublespeak and carefully prepared press releases I’ve come to expect from large corporations. Avid’s statements indicated that they were consolidating their engineering efforts in order to save money. Since Sibelius seems to be quite profitable, it’s clear that Avid is choosing to bleed the product to support its other goals. This move, in other words, reeks of bean-counting trumping any love of product or any real concern for the customers that have invested in Sibelius over the last several decades – invested more than money. Read on, intrepid explorer →

Good Programming in 3 Simple Rules

In the last few years, I have seen a little great code, some good code, a lot of mediocre code, and overwhelming amounts of bad code. (A shocking amount of my own code from previous years – especially when I was just starting – goes in the last two categories, alas.) The longer I have been at it and the more I have read (whether random articles on the web or the excellent Code Complete), the more I have concluded that good programming is simple. Incredibly hard, but simple. In fact, it is so simple, that you can sum it up in three short, easy to remember rules:

  1. Write code for people, not for computers.
  2. Don’t repeat yourself.
  3. Only do one thing at a time.

Read on, intrepid explorer →

JIRA, Confluence, Intranets and Windows Firewall

For my work with Quest Consultants, I've been working on setting up version control, issue tracking, and document control. After doing quite a bit of research, I settled on a few Atlassian products, integrating JIRA and Confluence with Subversion (you just can't beat TortoiseSVN for a Subversion client).

Installation goes off without a hitch, generally speaking, but I started to run into a singular, most annoying problem: I couldn't access the site on the intranet. Read on, intrepid explorer →

Starting to look more closely at the code I’m going to spend the next few months with. Oh, my…