1. Sandeep Shetty: Very cool. What's the use-case though? Also is this coming of the doctrine based indexer that you had?

    @sandeepshetty thanks! I’ve wanted to plot tag usage over time for a while now to see if there are any interesting patterns. I’m not using doctrine any more, in fact I’m not even using a SQL database for indexing until I really need one — data stored in yaml files, indexed by a csv file in ~210 lines of code — see also waterpigs.co.uk/notes/4TQNY2

    When I post a note, adds one to the week counter for each tag, then I have an endpoint which makes that data into an SVG.

  2. Chloe Weil:

    → An audio version of this article is available.

    I built Twitter. For the past few months I’ve loosely talked about my intent to publish micro-content (a tweet) to my site and sending a copy out to Twitter, as opposed to using Twitter directly. This publishing model goes by the acronym POSSE: Post to your Own Site and Syndicate Elsewhere.

    There are several notable reasons why people would choose to own their data. Personally, I felt liberated by the broader idea that services like Twitter, Facebook and whatever comes along in the future would be representations of me, rather than “me, elsewhere.” I hide in plain sight on other services by obfuscating my identity through a hilarious avatar, using a different poorly-considered, rarely-duplicated handle for each account, and avoiding amalgamating a community (“friends”, followees, connections, basically). I’m a serial late adopter because the idea that these things are me makes me uneasy. I still haven’t gotten to the root of this but I suppose it has to do with shame, as do all my issues.

    Implementing POSSE to Twitter also has the promise of solving the problem of neglecting my personal space on the web — this domain — which has always been my greatest hobby, and what brought me here in the first place. But what most compelled me is that I somehow find the need to build completely bespoke solutions to all my technical problems. For instance, I don’t use music subscription services because I have valuable personal metadata that is currently handled by iTunes, and the tool that I prototyped in Javascript to manage that data in the browser independent of the service, application or file is in development. I am very authentic yes.


    I spent the past three months avoiding actually building this. An autumn of procrastination included migrating my blog over to Publify, a Rails-based CMS that offers built-in POSSE to Twitter features, only to realize my incumbent CMS, Kirby, suited my needs better. I ended up abandoning my work.

    It only took a few moments to get POSSE to Twitter working. I found the action responsible for posting an item of content to my site in the code for Kirby’s admin panel and wrote a function that posted the content to Twitter using the API. But I also wanted to populate my existing 1,800 tweets into the CMS, in addition to being able to support what is modeled as an average of 1.6 items of micro-content per day. I read that this wouldn’t scale so well with Kirby, and, to my horror, it looked like I was going to have to go outside the file-based CMS and store my tweets in a database. The Kirby documentation itself recommended that databases are a better system to store large quantities of user-generated content, and fortunately, Kirby is built upon the Kirby Toolkit, which helped to subdue my nemesis, MySQL.

    I’m going to mention here that it’s important that I use tools built by people who share my values. Federic de Villamil, author of Publify, has written about data ownership and silos and Bastian Allgeier, author of Kirby, has written about and is working on a decentralized vision of the web. These are things that motivate me too. Because the authors care about the same things I do, their tools come with features that anticipate problems I’d need to solve, so I don’t waste time and hair struggling against software.

    What I ended up ultimately building, aided by a spec of sorts, is Web Design 101. It’s a form. It lives at a URL on my domain, and has a textarea that accepts 140 characters for the tweet, a text field for a URL if my tweet is an @reply (to enable conversation associations on twitter.com), and a toggle that determines whether I want the tweet to appear on my list of tweets on this domain. I would prefer to have my @replies hidden unless they’re really funny. The hidden tweets still get saved to the database and can be queried by their tweet ID, but they are excluded from the query that lists my tweets on my domain.

    When I submit the form, the API posts to my Twitter account, and the result of that request is posted to my database, which contains the full history of my tweets. The data submitted by Twitter is formatted similarly to the csv output of my archive of tweets, which is how I based my database schema. This is why I send the content to Twitter first and use the response to post to my database.

    This is the barest combination of elements hacked together in PHP, but it’s better than what I was doing before, which was posting to Twitter. And, I’m not going to proselytize, but how many of us have built a form that posts to a database, or requested that an API do a thing? That’s essentially all this is, and the details can be teased out in a couple dozen Google searches. Think about it.

    Ideally, I would like for my URLs to have the format /note/tweet_id or custom slugs rather than the query string be obvious, but, predictably, I’m having some .htaccess challenges and I’d rather kill myself. There is also some unfinished code that saves a unique hash along with the tweet so that in the future I can append a permashortcitation (new favorite word) to the end of my tweets, but again, .htaccess, and I really would rather die than learn regular expressions. In fact, “Never Learn RegEx” is on my bucket list, so I have no choice. Also, speak about destruction, but apparently I’m having timezone issues with the timestamp.

    I’m okay with all of this for now, because these are technical details (some of them glaring, yes), but I’m preoccupied by a shift in thinking. Ultimately, I would like to extend this model in order to write any length of content to my domain with the option to tweet 140 characters to Twitter, truly making Twitter an optional destination for anything, not just the micro-content originally intended for Twitter. This may help mend my mental dissociation between tweeting and blogging. But then I would be telling you all about how I built a CMS.

    @chloeweil great article and great work implementing ! Interested in your choice to use a database for performance reasons, was that prompted by actual experience or just the cited help thread? fwiw I’m having no performance problems storing >2000 notes in flat files with a CSV file index

  3. Julien Genestoux: Barnaby Walters I'd love to chat about your #yahoopipes clone! I believe superfeedr can help/learn!

    @julien51 indeed, I have plans for a caching system using superfeedr, as I’ve quickly found that keeping the feedback loops as tiny as possible is vital for productive piping! What’s your preferred communication method? I hear jitsi does pretty good encrypted voice/video calls these days…

  4. Brennan Novak: Thank you @WillardFoxton for referring to my people (software devs) as "exceptionally dull weirdos." You sir, have a way with words and are clearly aware of what's going on in the world. Thank you for your valuable insights.

    Brennan Novak is that a reply-context I see there?