1. When using the python2.7 gzip module with StringIO, it’s extremely important to call GzipFile.close() after you’ve finished writing to it. If you don’t, the archive will probably not be readable as the CRC record will not be appended, and you’ll get a bunch of IOErrors despite everything looking fine.

  2. for anyone using QtMultimedia QAudioInput with python’s wave module to write PCM data to a wave file: to convert between QAudioFormat’s sampleSize() number and wave’s sample width number, divide by 8, e.g:

    wave_file_to_write.setsampwidth(audio_format.sampleSize() / 8)

    QAudioFormat’s sampleRate() number works as it is.

  3. Any ideas why the Icelandic locale in +natsort doesn’t correctly sort Icelandic characters alphabetically (aábcdðeé etc)? I just implemented a rather awkward hacky way of sorting them using the alphabet and natsort.versorted, but would rather find a way to correct the root issue.

  4. Today’s lesson: be VERY careful with programmatic use of contribute_to_class(). It doesn’t overwrite existing fields of the same name, resulting in intriguing errors when the ORM tries to do a SELECT query containing the same columns hundreds of times over…

  5. Finally solved a long-standing problem getting Icelandic characters to work properly in files being downloaded onto Windows machines for use as SPSS syntax. Turns out the solution is to explicitly set the download charset to UTF-8, and to prepend an unnecessary BOM (yuk) to the beginning of the file as so (context: Django view):

    import codecs
    
    def export_spss(request):
        response = HttpResponse(export_spss(), status=200, mimetype="application/x-spss; charset=utf-8")
        response['Content-Disposition'] = 'attachment; filename=syntax.sps'
        response.content = codecs.BOM_UTF8 + response.content
        return response

    Why is a BOM, which should be completely unnecessary in a UTF-8 file (it has no variable byte order after all) apparently required by some Windows software in order to tell it that the file is UTF-8 encoded, despite Unicode mode being on? Sigh.

  6. The Heroku python client library is horribly out of date, and many simple things which should work don’t, throwing confusing errors. Here’s my version:

    # coding: utf-8
    
    import requests
    
    HEROKU_URL = 'https://api.heroku.com'
    
    
    class Client():
        def __init__(self, api_key, heroku_url=None):
            self.heroku_url = HEROKU_URL if heroku_url is None else heroku_url
            self.session = requests.Session()
            self.session.headers.update({'Accept': 'application/vnd.heroku+json; version=3'})
            self.session.auth = ('', api_key)
    
        def get(self, path):
            r = self.session.get('%s/%s' % (self.heroku_url.rstrip('/'), path.lstrip('/')))
            r.raise_for_status()
            return r.json()
    
        def post(self, path, data=None):
            r = self.session.post('%s/%s' % (self.heroku_url.rstrip('/'), path.lstrip('/')), data=data)
            r.raise_for_status()
            return r.json()
    

    Add similar methods for DELETE if you find you require it. I haven’t yet, as the idea of programatically being able to delete apps is much more worrying than the ability to create them.

  7. Today in “things I have typed instead of import”: imprto, improt, implore

    I actually quite like how implore pandas looks in my code.

  8. I’m getting too used to (puredata.info) — I just right-clicked a python class and expected a hypermedia “help” option with params, example usage etc.

    Jetbrains PyCharm does have an inline documentation feature, which (when invoked via a complex keyboard “shortcut”), produces this gem:

    Even when this feature does work, it shows code “documentation” in a monospace font, typically with no usage example or links to other relevant documentation, as is standard in puredata.

    Our tools are inadequate.

    Update: added puredata documentation for comparison:

    In case it’s not clear from the screenshot, that usage example is live code — it can be interacted with, changed, copied and pasted, played with, experimented with. We typically can’t do that with existing text-based code, let alone mere usage examples.

    More thoughts I want to add to this, but I will write them up as a full article.

  9. If you’re using pika as a client and are getting “ConnectionClosed” exceptions, test or look very carefully through the code being executed — it is probably not a pika or rabbitmq issue but some other exception.

    E.G. I just spent half an hour tearing apart pika connection handling code when it turns out the exceptions were caused by passing a dict rather than an instance of template.Context to template.render().

  10. Javascript has no real Set or Dictionary implementation, which for someone spoiled by python’s set and dicts is rather frustrating. However, in leiu of the poorly supported js Set type, plain old objects can be massaged into acting as both sets and dicts:

    
    // Python: d = dict()
    var d = {};
    
    // d['key'] = 'value'
    d['key'] = 'value';
    
    // d.get('nonexistent', 'fallback')
    d.hasOwnProperty('nonexistent') ? d['nonexistent'] : 'fallback';
    
    // d.keys()
    Object.keys(d);
    
    // s = set()
    var s = {};
    
    // s.add(1)
    s[1] = true;
    
    // 1 in s
    s.hasOwnProperty(1);
    
    // Accessing all values in set:
    Object.keys(s);
    

    Notes: the in operator can be used to test membership, but will incorrectly return true for __proto__, as well as all properties up the prototype chain, i.e. all properties and methods of Object. hasOwnProperty is much safer to use.

    Similarly, the use of the ternary operator for get-item-with-fallback could in theory be replaced with d['item'] || 'fallback', unless of course the value stored was falsey, in which case the or will incorrectly return a truthier fallback.

  11. Got lightblue+bluez set up on the raspberry pi, successfully paired and connected with Macbook. Not so much success communicating, sockets only worked one way, and the OS X doesn’t seem to be able to connect to services advertised on the pi despite being able to see that they exist. Also got started with USB programming in python with pyusb — is complex, but less so than I expected.

  12. When using to send content-disposition: attachment responses, you MUST explicitly set the response encoding (charset) otherwise windows will assume the response is in whatever-weird-encoding-windows-uses, rather than UTF-8 (you are using UTF-8, aren’t you?).

    CSV example code:

    response = HttpResponse(utf_8_encoded_csv_text, status=200, mimetype="text/csv; charset=utf-8")
    response['Content-Disposition'] = 'attachment; filename=data.csv'
    

    It seems that some weird old applications like SPSS need a BOM in there as well, even though UTF-8 doesn’t have a BOM. Add that like this

    response.content = '\xef\xbb\xbf' + response.content
    

    before sending the response

    return response
    
  13. Had many basic software development lessons hammered in by personal experience over the last couple of years: hierarchy bad. side effects bad. many moving parts bad. undue complexity bad. inconsistency bad. SQL databases fragile. always be reducing.

    It’s amazing just how seductive complex, unproductive tools can be. Successfully overcome+abandoned:

    • Codeigniter
    • Doctrine ORM
    • Bootstrap
    • Backbone, Ember, Angular
    • Symfony Security component

    PHP remains productive and speedy (with composer, delightful dependency management), python nice with some irritations. jQuery useful when absolutely necessary, plain with small libraries loaded via requirejs handle most progressive enhancement concisely. node.js nice for some things, preferring go’s approach to async programming but still not much everyday need for it.

    Avoiding middlemen: LESS, SASS, Coffeescript. Unnecessary for most of my work, and more moving parts is bad.

    Now bothering me is the frameworky nonsense accumulating in . Need to cleanse.