1. Sandeep Shetty: Very cool. What's the use-case though? Also is this coming of the doctrine based indexer that you had?

    @sandeepshetty thanks! I’ve wanted to plot tag usage over time for a while now to see if there are any interesting patterns. I’m not using doctrine any more, in fact I’m not even using a SQL database for indexing until I really need one — data stored in yaml files, indexed by a csv file in ~210 lines of code — see also waterpigs.co.uk/notes/4TQNY2

    When I post a note, adds one to the week counter for each tag, then I have an endpoint which makes that data into an SVG.

  2. New in : latest 3 articles and location of last checkin/location post on the homepage — hopefully some useful/interesting context.

    Next: mobile-focused homepage design.

  3. Chloe Weil:

    → An audio version of this article is available.

    I built Twitter. For the past few months I’ve loosely talked about my intent to publish micro-content (a tweet) to my site and sending a copy out to Twitter, as opposed to using Twitter directly. This publishing model goes by the acronym POSSE: Post to your Own Site and Syndicate Elsewhere.

    There are several notable reasons why people would choose to own their data. Personally, I felt liberated by the broader idea that services like Twitter, Facebook and whatever comes along in the future would be representations of me, rather than “me, elsewhere.” I hide in plain sight on other services by obfuscating my identity through a hilarious avatar, using a different poorly-considered, rarely-duplicated handle for each account, and avoiding amalgamating a community (“friends”, followees, connections, basically). I’m a serial late adopter because the idea that these things are me makes me uneasy. I still haven’t gotten to the root of this but I suppose it has to do with shame, as do all my issues.

    Implementing POSSE to Twitter also has the promise of solving the problem of neglecting my personal space on the web — this domain — which has always been my greatest hobby, and what brought me here in the first place. But what most compelled me is that I somehow find the need to build completely bespoke solutions to all my technical problems. For instance, I don’t use music subscription services because I have valuable personal metadata that is currently handled by iTunes, and the tool that I prototyped in Javascript to manage that data in the browser independent of the service, application or file is in development. I am very authentic yes.


    I spent the past three months avoiding actually building this. An autumn of procrastination included migrating my blog over to Publify, a Rails-based CMS that offers built-in POSSE to Twitter features, only to realize my incumbent CMS, Kirby, suited my needs better. I ended up abandoning my work.

    It only took a few moments to get POSSE to Twitter working. I found the action responsible for posting an item of content to my site in the code for Kirby’s admin panel and wrote a function that posted the content to Twitter using the API. But I also wanted to populate my existing 1,800 tweets into the CMS, in addition to being able to support what is modeled as an average of 1.6 items of micro-content per day. I read that this wouldn’t scale so well with Kirby, and, to my horror, it looked like I was going to have to go outside the file-based CMS and store my tweets in a database. The Kirby documentation itself recommended that databases are a better system to store large quantities of user-generated content, and fortunately, Kirby is built upon the Kirby Toolkit, which helped to subdue my nemesis, MySQL.

    I’m going to mention here that it’s important that I use tools built by people who share my values. Federic de Villamil, author of Publify, has written about data ownership and silos and Bastian Allgeier, author of Kirby, has written about and is working on a decentralized vision of the web. These are things that motivate me too. Because the authors care about the same things I do, their tools come with features that anticipate problems I’d need to solve, so I don’t waste time and hair struggling against software.

    What I ended up ultimately building, aided by a spec of sorts, is Web Design 101. It’s a form. It lives at a URL on my domain, and has a textarea that accepts 140 characters for the tweet, a text field for a URL if my tweet is an @reply (to enable conversation associations on twitter.com), and a toggle that determines whether I want the tweet to appear on my list of tweets on this domain. I would prefer to have my @replies hidden unless they’re really funny. The hidden tweets still get saved to the database and can be queried by their tweet ID, but they are excluded from the query that lists my tweets on my domain.

    When I submit the form, the API posts to my Twitter account, and the result of that request is posted to my database, which contains the full history of my tweets. The data submitted by Twitter is formatted similarly to the csv output of my archive of tweets, which is how I based my database schema. This is why I send the content to Twitter first and use the response to post to my database.

    This is the barest combination of elements hacked together in PHP, but it’s better than what I was doing before, which was posting to Twitter. And, I’m not going to proselytize, but how many of us have built a form that posts to a database, or requested that an API do a thing? That’s essentially all this is, and the details can be teased out in a couple dozen Google searches. Think about it.

    Ideally, I would like for my URLs to have the format /note/tweet_id or custom slugs rather than the query string be obvious, but, predictably, I’m having some .htaccess challenges and I’d rather kill myself. There is also some unfinished code that saves a unique hash along with the tweet so that in the future I can append a permashortcitation (new favorite word) to the end of my tweets, but again, .htaccess, and I really would rather die than learn regular expressions. In fact, “Never Learn RegEx” is on my bucket list, so I have no choice. Also, speak about destruction, but apparently I’m having timezone issues with the timestamp.

    I’m okay with all of this for now, because these are technical details (some of them glaring, yes), but I’m preoccupied by a shift in thinking. Ultimately, I would like to extend this model in order to write any length of content to my domain with the option to tweet 140 characters to Twitter, truly making Twitter an optional destination for anything, not just the micro-content originally intended for Twitter. This may help mend my mental dissociation between tweeting and blogging. But then I would be telling you all about how I built a CMS.

    @chloeweil great article and great work implementing ! Interested in your choice to use a database for performance reasons, was that prompted by actual experience or just the cited help thread? fwiw I’m having no performance problems storing >2000 notes in flat files with a CSV file index

  4. New in this version of :

    • Improved styling (still WIP, as always)
    • stream on homepage, currently just notes but will add other things too
    • content creation/editing UIs publicly viewable (take a peek!)
    • profile photo as the icon
    • complete code restructure, now using silex for HTTP routing
    • removed tonnes of nonsense framework code, replaced with small number of ≈200 line functional libraries. Clearer, easier to navigate and much more fun to work with
    • no more SQL databases — content is indexed using a custom built 209 LOC CSV index which is surprisingly speedy, and suits my needs perfectly
    • no more support for rendering content in many forms using content negotiation (HTML, JSON, ATOM etc.) — now only HTML+microformats2 representations of content are given
    • ATOM feed shimmed with microformats2 to ATOM converter
    • Pingbacks no longer natively accepted (though they are sent), using webmention.io to shim them into webmentions for easier handling

    The local maximum has been overcome, for now. There is still much to do.

  5. When people drop into your IRC channel to ask whether it’s possible to reply to something on your mailing list, that’s probably a good sign you should ditch your mailing list and use IRC + Wiki.

  6. Feedback Loops

    The larger the gap between one major feedback loop and the next largest of any given stage of an activity, the more assumptions must be made about that stage.

    Example: planing a hurdy gurdy top to thickness.

    Tasks, feedback loops in order of duration (timings are approximate from memory):

    • Every millimetre of planed wood, ≈150ms audible and physical feedback
    • Every ≈1cm of planed wood, ≈500ms visual feedback loop seeing the shavings protrude from the plane (or not, which is equally valuable)
    • Every ≈10cm of planed wood, ≈3s visual+physical+audible feedback loop of one complete shaving detaching
    • Every ≈20cm of planed wood, ≈5s visual feedback loop seeing freshly planed surface, erosion of pencil marks
    • Pause every ≈10 plane strokes, ≈20s physical feedback loop picking up the top and flexing
    • Pause every ≈20 plane strokes, ≈1 minute precision visual feedback from re-measuring the thickness of the top with a caliper

    The gaps between feedback loops become larger as the durations become larger, as do the assumptions which are made about the task at each level until the next feedback loop arrives. I suspect that experience level also affects both the value gained from each feedback loop, increasing the actor’s knowledge of the system and increasing the amount of time which can safely be left (i.e. the amount of assumption which is safe) before more feedback is required.

    Two observations: the existence of vastly longer feedback loops of experience accumulating which affect the shape of existing loops, and that the smallest feedback loops are broadcast by the environment (audible feedback, physical resistance) but longer ones require active participation (testing the system).