Hack on Noetic Nought

Restoring a broken Ubuntu upgrade

Wed, 20 Nov 2024 15:17:00 +0530

A couple of weeks ago, Shantanu and I were discussing Ubuntu upgrades – his upgrade was borked and he ended up re-installing his system. I was celebrating the fact that I never really had to throw away my OS installation entirely since I switched to Ubuntu after getting tired of Arch Linux in 2011-12.

And on cue, I had a really broken update on my dad’s laptop! It was really broken – no WiFi, no Ethernet connection, no GUI packages, etc.

The crash

I started the update but forgot about it, promptly. My dad was semi-supervising it, since he was using the laptop on-and-off. I came back to it a couple of days later, and I ran an autoremove, assuming everything else in the install went ok. Everything continued to work alright until some time later, we had to restart the X server because it had hung up. And it never came back. And on a full restart, no network – both Ethernet and Wifi.

`chroot` to the rescue!

Thankfully these are not the days when I had access to a single computer, and crashing it meant running to a friend and using their computer to download stuff. I quickly made a bootable USB on my laptop.

I boot my dad’s laptop using the live distro and chroot to the installed OS using the live distro. Live Ubuntu has Ethernet working, thankfully.

For apt-get install and dist-upgrade to work correctly with the chroot, I had to mount a bunch of volumes.

Due to some network manager changes in the update, the chroot system still wasn’t connecting to the internet. I probably should’ve looked at the resolv.conf file first, but ended up going the offline package install route…

Offline updates

I generate a list of missing packages to be downloaded using the --print-uris flag to apt-get on the chroot system; And use wget to download the packages on the “live” system; Then copying these packages to /var/cache/apt/archives/ gets the update to finish.

But more missing packages!

I rebooted into the installed OS, once apt-get dist-upgrade seems to have everything resolved. But, I still didn’t have a network! I had to chroot again into the system, and this time I figured out that there was an issue with resolv.conf being a broken soft-link which seemed to cause the broken network (with Ethernet).

For WiFi, I had to install proprietary drivers for the Broadcom wireless card (BCM43142 802.11b/g/n).

I had to also install the ubuntu-desktop and related packages that seem to have gone missing, and on reboot everything was up and running!

Outro

When I told Shantanu, I didn’t have to reinstall my Ubuntu from scratch doesn’t mean that all my updates were flawless, and things never broke. I have had such broken updates a few times – network updates leaving the system in a broken state after network connections died, Ubuntu upgrade being in a borked state after the xz vulnerability was discovered, etc.

Updates do fail. But, more often than not I’ve been able to restore things thanks to dpkg ’s robustness and awesomness! And in some extreme cases, reaching for chroot to save myself.

There’s probably nothing new to learn or discover in this post, but just some help forum links that might come in handy, in future. In this age of AI answers, it seemed nice to look through some old forum QAs and work things out using them.

Thanks to Shantanu and Kamal for reading drafts of this post.

Responsive Auto Export for Org Hugo

Wed, 13 Nov 2024 00:17:00 +0530

I use ox-hugo to write blog posts in org-mode and publish them using Hugo. I enjoy using org-mode for any writing that I do, including blog posts (when I am able to get myself to write them). After a long hiatus, I’ve been trying to get back to blogging, as this post might have given away. But, ox-hugo’s auto export seemed much slower than I remember it being.

Each time I hit save on my blog-posts.org file, Emacs gets busy for about 10ish seconds exporting the post to Hugo markdown. And this is annoying because I tend to hit save-buffer multiple times while writing in Emacs – thanks, muscle memory!

Enter Emacs profiler

I was on a flight and had some time to dig into this. If I was online, I probably would have looked through the README and/or the issue tracker, but I jumped in with the handy Emacs profiler.

The profiler-report showed that a big chunk of time was being spent in org-id-update-id-locations even before the actual export started. And then, during the export, a bulk of the time was spent in org-hugo--get-pre-processed-buffer. Once I knew the problem areas, I started poking around in the ox-hugo code to “fix” these issues.

Turn off Org ID location update

org-id-update-id-locations scans a bunch of org-mode files and stores a mapping of all the IDs of subtrees to their filenames. If you have a lot of org subtrees this can take a while, even if none of them actually have IDs. It turns out that I didn’t have any ID properties set, and this caused the update function to run before every export!

I simply added a new ID property on one of the subtrees in my blog-posts file to prevent the ID updation from running on every export! I’m not sure how a stale org-id-locations value affects cross links, but at this stage of writing I don’t care about the cross links. (spoiler: The next hack actually nullifies any impact a stale value might have had!)

Cutting down auto export time by 4-5 seconds is great! But, I’m still not happy to wait for 5-6 seconds in the middle of writing my posts. Let’s look at the other hotspot – the buffer preprocessing!

Turn off buffer preprocessing, maybe?

Currently, I don’t have any cross links between posts in my org-mode source. So, I can turn off this feature completely by setting org-hugo--preprocess-buffer to nil. Viola! Hitting save-buffer doesn’t freeze my Emacs any more. I can compose “100s of blog posts”™ in a flurry! ;)

But, if I’m going to have these “100s of blog posts”, wouldn’t it be better to have cross links? But, with preprocessing turned off when there are cross-links, the “auto-export and build” workflow breaks. The variable org-hugo--preprocess-buffer MUST be non-nil to produce posts with valid cross-links. If not, the exported markdown file processed by hugo ends up having broken cross-links, which crashes hugo serve and/or hugo build.

Unsetting and setting the org-hugo--preprocess-buffer variable for the writing vs publishing phase, respectively, isn’t an ergonomic workflow. It’s not an improvement over disabling and enabling the auto-export mode as needed. I want to enjoy auto export with Hugo’s live reload feature.

Moar workaround!

Looking through the code some more, I learnt org-hugo-link first uses a custom export handler, if one exists for a link’s protocol. I decided to piggy back on this functionality and made up hugo: protocol for cross-links.

The hugo link simply contains the EXPORT_FILE_NAME of the linked blog post i.e., name of the exported markdown file (without the .md extension) as the ‘path’ of the link. The custom protocol export handler can then generates a relref shortcode for Hugo to process in the exported markdown file.

This nicely works around the need to preprocess my entire blog-posts.org buffer to generate valid cross-links!

(org-link-set-parameters "hugo" :export #'pc/org-hugo-link-export-to-md "Export Hugo blog link to markdown file" )

(defun pc/org-hugo-link-export-to-md (path desc backend &optional info)
  "Export a link to a Hugo blog link in markdown format."
  (message (format "path: %s, desc: %s, backend: %s" path desc backend))
  (cond
   ((eq backend 'md)
    (if (equal org-export-current-backend 'hugo)
        (format "[%s]({{< relref \"%s\" >}})" desc path)
      (error "Cannot export Hugo link to non-Hugo backend")))
   (t (error "Cannot export Hugo link to non-Hugo backend"))))

Outro

I made a simple helper to make it easier to insert these cross-links with the hugo: protocol. It simply looks through all the headlines with an EXPORT_FILE_NAME property, and allows it to be inserted as a link with the hugo: protocol.

Now, cross-linking posts and publishing posts with cross-links are both a breeze. Stay tuned for the “100s of blog posts” ™ I’m going to write!

Thanks to Shantanu and Kamal for reading drafts of this post.

Made in 2022

Wed, 08 Mar 2023 23:35:00 +0530

Keeping with last year’s tradition, this post comes when we are almost 1/5th through this year.

2022 projects

Expense Tracker

I built a personal Expense Tracker over the month of October using SQLite and Streamlit, because I wasn’t comfortable sharing my financial data with a 3rd party. I’ve tried to keep it simple, so it’s not a pain to use. But, I’ve also tried to keep it extensible, for others to try and use.

The last few months have been quite busy personally, and I haven’t been able to use it as much as I would like to, but hopefully things will settle down soon. A couple of things that I’d like to improve:

The Sqlite file is stored locally, and the data is not synced between different computers.
The tagging/categorization of expenses is very manual, and a little bit more automation would be handy.

artful-dodger

A Next.js project for hosting an art/photo gallery with metadata stored in a Google Spreadsheet and images in a Google Drive (with an optional CDN, recommended). I built this for a friend, who is experimenting with creating “a more equitable model for curating, buying, and selling art”. The gallery is currently in “private beta”, and I can’t link to it. But, here’s a sample gallery here, whose configuration lives here.

ox-gist

I released my first Emacs MELPA package – ox-gist – an Orgmode backend to export and update sub-trees and buffers to GitHub gists. It was a great experience contributing to MELPA. I wrote a blog post about it too.

Termux scripts

I’ve started using Termux on my Android phone and have found the Text-to-Speech (TTS) API quite handy. I’ve a couple of scripts on my phone that lets me use TTS to read out contents on the clipboard or an article from my browser.

Fantasy League Site

With help from a bunch of volunteers collecting and validating data, Jonny and I built a website for the Fantasy League for Indian Ultimate State Championships. There was nothing technically challenging about the project, but it just involved a lot of data cleaning and co-ordination between the volunteers. The data was collected using pen and paper, instead of a digital app for collecting the stats. This made the data cleaning and validation significantly harder. I’d definitely push for digital data collection if people try to do something like this again.

Bookmarklets

A web-page with a bunch of Bookmarklets from my browser’s Bookmarks bar. I’ve had most of them around for a while, but putting them on a web page makes it easy to share and find myself on other browsers/computers.

Music

Unlike 2021, I’ve only been able to record a couple of songs. And this year (2023) hasn’t been any better, so far. I hope I’ll be able to make amends to this, soon, though.

Review of 2021 projects

Earworm

I didn’t make any updates to the package, not because it’s perfect but because I didn’t use it much in 2022 – thanks to how little I’ve been able to record.

While, writing this blog post I realized that workflow had a couple of annoyances. I improved the code to fix them.

I need to move an audio file back-and-forth between my phone and my laptop, one full round trip to add ID3 tags and cover image. This makes me put off uploading recorded files to the site. Also, the new media file needs to be named in a specific format. I’ve added a new CLI subcommand to automate all of this for me.
I always need the full collection of the music files on the computer, where I’m updating the site from. I can just scp the existing site onto my computer and merge the media directories.

Ukulele Tutorials Aggregator

The job of manually updating the site is a lot of work, and I haven’t had the interest to keep doing it.

RPi Server and Phone Backup

The physical setup was accidentally dismantled by folks I share my physical space with, and I haven’t been able to put back everything in place, yet. I’m hoping I’ll be able to get back to doing this at some point this year.

Outro

I’ve already built a couple of small things in 2023, and I hope I am able to find some interesting things to do this year.

Link to the 2021 post.

Moving from Heroku to Fly.io

Sun, 06 Nov 2022 12:19:00 +0530

A couple of months ago Heroku announced their plan to “Focus on Mission Critical” and shut down the free product plan. The ideas is to free up their developer time to focus on the important stuff instead of fighting abuse of the free plans. I’m surprised they invested in the free plans for so many years!

My Heroku Apps

I have a bunch of apps on the free (Hobby dev) plan, and only app on the Hobby Basic plan because it needed a bigger DB. All of these apps are small enough that I could easily host all of them on my shared Hetzner server, where this blog is hosted. But, I like using Heroku for services that I run for other people. It makes access control and deployment much easier, and lets me invite other people to help maintain apps easily, despite constraints like 10k rows in the Postgres DB and Dyno sleeping.

Over the years of using Heroku, I ended up accumulating a bunch of small apps on the service. Most of these apps aren’t actively being used any more and didn’t feel worth the effort of moving.

I decided to let most of the apps die, since they aren’t actively being used. Only a couple of apps are used regularly enough to be worth keeping alive – SOTG Calculator and RSVP app. I turned on maintenance mode on the RSVP app for a couple of days, as an experiment, and I had to bring back the services due to “popular” demand.

Why not pay Heroku?

I did consider just moving up to the Hobby basic plan and paying for running the app. I wasn’t keen on investing numerous hours to get these apps to run again. Akilesh convinced me that the amount to pay Heroku could be collected from the team and wouldn’t amount to much per person. I almost took this route.

But, it bugged me that the amount we’d be paying Heroku could fund a couple of turf practice sessions for the team. I could host the apps on my Hetzner server – we already pay enough for it. But, giving other people maintainer access to apps is important and this was not straightforward with Hetzner.

Also, the SOTG app couldn’t be funded this way.

Why Fly.io?

I ranted about this to my friends on Zulip and Rajesh offered to help me as a way to learn some devops stuff. I was more than happy to have someone to discuss this stuff with and get some help.

We decided to give Fly.io a try despite being skeptical about using yet another free service. Their free Hobby plan includes 3GB of persistent storage space (which neither of the apps currently use, but potentially could). They also built Turboku – a tool to move apps from Heroku. It seemed worth giving a try, and we could always fall back on Hetzner if it didn’t work out.

We had a couple of months before the Heroku shutdown date, when I ranted about this. Rajesh and I exchanged a few links and messages on how to potentially go about doing this, when we got down to it. But, he got busy with other things and now the deadline was only a month away. I decided to use the notes and do the migration myself. I was glad I had the notes to refer to. The initial “research” made the task seem less daunting.

Migrating a simple app

I started with the simpler app first – the SOTG app. I used the Turboku web page and it worked pretty seemlessly! It copied over the environment variables from Heroku and deployed the app.

But, using the Turboku web page meant that I didn’t have a fly.toml file to do any future deployments. I used the fly launch command to create a new app and edited the fly.toml file to point to the migrated app.

Setting up the custom domain was pretty simple too, using the CLI. I just had to run fly certs add sotg.indiaultimate.org after changing the DNS entries to point to sotg-calculator.fly.dev.

The whole thing felt quite simple and I was done in under an hour – reading the docs, installing the tool, trying out different things, getting it all working. Everything. Quite impressive!

Migrating a bigger app

Encouraged by this, I started with the RSVP app. I didn’t think it would be much harder than this, but I guess when there are a few more moving parts, it’s hard to tell what could pop-up.

I used the same approach for the migration – Turboku web page to migrate from Heroku and then ran the launch command to generate a fly.toml file that I edited to point to the deployed app.

The deployment seemed to go fine, but the app wouldn’t come up. I looked at the logs to find out that the app was getting OOM killed. Fly.io provides half the memory (256MB) that Heroku provided. But, I wasn’t expecting this tiny app to need much more.

I removed some default imports of large packages, but it didn’t really help that much. I did consider reducing the Gunicorn’s WEB_CONCURRENCY setting to 1, but didn’t want to take that route unless absolutely required. I looked around the forums a little, and found that using a Dockerfile instead of the buildpacks might improve the memory situation. And it did!

Next, I moved to setting up the custom domain and it worked from the Fly.io side seemlessly. But, the flask_dance library was generating http URLs in it’s OAuth redirect URLs. The app already uses FlaskSSLify, but that didn’t seem to be sufficient. After reading some Flask dance and Flask docs, I found that Werkzeug’s ProxyFix exists exactly for this!

After dealing with these two unexpected problems, I got to the problem that I was aware I had to tackle – Fly.io doesn’t have support for cron jobs. The RSVP app uses some scheduled jobs to periodically sync some metadata from Google Drive, Calendar, etc.

I took a hackish route to work around this. Using the schedule library, I wrote a “cron” script that would run the jobs on the desired schedule. I created a duplicate app to run the cron service, even though it could potentially be run as a different process on the same app. I didn’t want to risk any more OOM kills on the web app. But, it can be a little bit more work to keep the environment variables and other such things in sync between the two apps.

I was quite excited about Fly.io having a datacenter in Madras and was swayed by Fly.io’s marketing about running the app close to the users. I tried this but the performance was terrible since the Mongo DB was in North Virginia. I quickly switched back to using an East Coast region for Fly.io app too.

But, while writing this blog post, I realized that I could move the DB to India. I moved the DB to the AWS Mumbai datacenter, and the Fly.io app to the Madras datacenter.

All’s well that ends well

The app has been chugging along for a few days without any new hiccups. I’m hoping things stay this way for the weeks and months to come, if not years.

On the whole, I found the whole experience more beta-ish than deploying something to Heroku. But, I’m quite happy with how the whole thing turned out. I think things will get smoother with time – with my growing familarity with Fly.io and more people using their tools suggesting improvements.

I am wary of depending on yet another Hobby plan, but neither of these apps make any revenue and I’m not (yet) comfortable charging users for the small set of features they provide.

I’d like to hear stories of other people moving away from Heroku! Feel free to reach out!

GitHub Gists from Emacs Orgmode

Tue, 26 Apr 2022 12:37:00 +0530

TL;DR: I released my first Emacs MELPA package – ox-gist – An Orgmode backend to export and update sub-trees and buffers to GitHub gists. It was a great experience contributing to MELPA.

Motivation

I often share bits and pieces of my journal.org file with others, mostly for reading, some times for comments. The file contains different subtrees for each day with subtrees for different topics, meetings, articles, etc.

Orgmode lets you export subtrees to specific file names using the EXPORT_FILE_NAME property (and FILE_NAME buffer export option when exporting full buffers). It is easy to create these exported files on a publicly accessible webserver using the power of Emacs TRAMP mode. For instance, I often set the property to some thing like /ssh:muse-amuse.in:~/public_html/<filename> to publish stuff.

But, exporting to HTML is a distraction some times because I start fiddling with CSS and styling. Exporting to a simple org file doesn’t seem to cut it because there’s no styling at all, and making longer exports hard to read in a web-browser. Exporting to a GitHub Gist is a nice middle ground because GitHub renders Orgmode files reasonably well. Additionally, the comment functionality is pretty handy.

I could also directly use @defunkt’s excellent gist.el to share stuff as Gists. I’ve used it in the past, and it works well when I don’t care about having the contents of the Gist locally on my machine. It’s also quite easy to update existing/old Gists in this scenario, since there’s a single “source of truth” and gist.el makes it quite easy to update Gists.

But, this isn’t good enough when I want to have a “synced” copy of the text on my machine. Also, it doesn’t work very well when I want to share just a subtree from my notes file, and not the entire file. I’ve to export the subtree to a temporary Orgmode buffer and then create a Gist from that buffer.

Hackish wrapper around `gist.el`

A couple or so years ago, I wrote a wrapper around gist.el to let me post Orgmode subtrees as a Gist and also update them when I edited these subtrees locally. There was no support for exporting entire buffers since that wasn’t my use-case. This wrapper code was just a part of my .emacs and worked reasonably well for me.

A few weeks ago, I ended up procrastinating on writing and sharing notes for a discussion with a friend, and extracted this wrapper as a separate package – org2gist – to make it easy for other people to use.

MELPA submission and feedback

I submitted this as a package to MELPA while being skeptical about whether it would be accepted. I wasn’t sure if my thin wrapper around gist.el would meet MELPA’s criteria for a “Reasonably innovative package”:

MELPA provides a curated set of Emacs Lisp packages, not an exhaustive list of every single Emacs Lisp file ever created. By default, MELPA maintainers will reject packages that duplicate functionality provided by existing packages. Please try to improve existing packages instead of creating new ones when possible.

It was a pleasant surprise to get very helpful reviews from the MELPA maintainers. They not only thought the package was good enough to include in MELPA, but also looked at it’s code and gave great feedback — not just basic quality checks before accepting the package into MELPA.

I’m pretty sure mainting a package repository like MELPA is a lot of work, but as an end-user it mostly just worked for me and I never really gave it much thought. It was eye-opening to submit the package to MELPA and see first-hand the care and effort the maintainers put into what goes into the repository.

@riscy (a MELPA maintainer) suggested a different approach that I hadn’t thought of — turning the wrapper into an Orgmode Export backend.

Current version: `ox-gist`

It turned out to be a great idea which I ended up implementing. The user interface became much cleaner, since it used the Orgmode Export UI. And it also forced me to “complete” the feature-set by adding support for exporting entire Orgmode buffers to GitHub Gists, not just subtrees.

The package now adds an Orgmode export backend that lets you export a subtree or the whole buffer as a GitHub Gist, similar to how you’d export to HTML or LaTeX from Orgmode. The GitHub Gist ID is saved for each exported buffer or subtree and is to update a Gist when re-exporting an already exported subtree or buffer.

Outro

The package has now been added to MELPA as ox-gist. Feel free to give it a spin and give feedback directly to me or on the issue tracker!

I don’t know if anyone else would find the package useful and use it, though, MELPA tells me that 20-odd people atleast downloaded it. I’m happy this code may be to a handful of people.

Also, I now have a new found appreciation for MELPA and the work put into it by the maintainers. I’m glad I took the time out to submit this package to MELPA.

Thanks to Shantanu for reading drafts and helping me restructure this post.

Made in 2021

Thu, 24 Feb 2022 02:23:00 +0530

Some things I made in 2021:

Earworm: A static site generator to listen to audio (and video?) files in a directory. I use it to host a couple of simple websites for music files. It also happens to be the first package I uploaded to PyPI despite writing Python for many years.
Ukulele Tutorials Aggregator: Inspired by Ukealong.com, I set out to create an aggregator for Ukulele Tutorials for Indian songs. I’ve, with help from Kamal, populated about 300 songs on the site. I haven’t been able to keep up with maintaing it, after an initial bunch of work. I’ve wanted to automate more of the workflow, but haven’t yet.
RPi Server and Phone Backup: I’ve started to use a Raspberry Pi to backup data from my phone and laptop to a personal Hard disk. I’ve ansible scripts to reproduce the RPi’s setup, and some scripts to “sync” my phone’s data every few hours, when it’s on the same network. There’s a lot more that I need here, before I can ditch cloud backups of important things.
Music: I made covers of about 10 songs on the Ukulele, that I shared with some friends. I’ve ways to go before I’m comfortable sharing them on the web, publicly.

I’m glad to have had the energy, time and resources to be able to spend time working on some fun things despite everything that 2021 had in store for us. Let’s see what 2022 has in store!

PS: This post is quite late into 2022, but it’s an excuse to post something to a blog that’s not seen any updates since the middle of 2020.

Hat tip to Bryan Braun whose post inspired me to write this

Deploying Hugo Drafts, simplified

Sun, 31 May 2020 17:19:00 +0530

This is a follow-up post to my last post, Deploying Hugo Drafts. Kaushal Modi, the author of ox-hugo (a tool that lets you compose in org-mode and use Hugo to export), let me know about the build options in Hugo.

Have you looked at the new _build options: https://t.co/Nj8Uefis6Q. I haven't used them yet, but looks like they fit your use case.
— Kaushal (@kaushalmodi) May 30, 2020

The build options in Hugo let you configure building a post for different use cases like render a post but hide it in lists, or don’t render separate pages for posts and only show them in lists, etc.

To use these build options for my use case – being able to share draft post links with friends, without them appearing in the index page or RSS feed – I just need to add the following to my draft blog posts!

_build:
  render: true
  list: false

But, how do I do this with ox-hugo? I know ox-hugo doesn’t yet have this feature, for sure, because Kaushal mentioned that he hasn’t tried it out yet.

I checked to see if I could use the EXPORT_HUGO_CUSTOM_FRONT_MATTER property to get it to generate this extra metadata for each blog post. But that didn’t work. The property assumes all the arguments to it are top level front matter keys and values, which is good enough for “normal” use cases.

EDIT [2020-05-31 Sun]: I previously manually created the drafts directory with the _index.md, but Kaushal pointed me to more docs, again! :) There’s also an issue for adding an easier way to add these build options from ox-hugo.

I created a subtree like this in my org file, that creates the drafts/_index.md file with the required front matter and build options.

* Drafts
:PROPERTIES:
:EXPORT_FILE_NAME: _index
:EXPORT_HUGO_SECTION: drafts
:EXPORT_HUGO_CUSTOM_FRONT_MATTER: yaml
:END:

#+begin_src yaml :front_matter_extra t
  _build:
    render: false
    publishResources: false
    list: false
  cascade:
    _build:
      render: true
#+end_src

The build options for the index page turn off publishing and showing it in lists. With the cascade option, we can get the options to all the children – any files created in the drafts directory, but also override the render option to be true. In other words, the configuration above would translate to adding the following to every draft post:

title: A How-to on Hugo Drafts
_build:
  render: true
  publishResources: false
  list: false

Now, I just need to make sure that all my draft posts are created in this drafts directory. ox-hugo allows doing this by simply using the EXPORT_HUGO_SECTION property on my post subtree. I added this additional property to my helper function that I use to create hugo blog posts from any subtree.

* DRAFT A How-to on Hugo Drafts :blogging:hack:
:PROPERTIES:
:EXPORT_FILE_NAME: how-to-hugo-drafts
:EXPORT_HUGO_SECTION: drafts
:EXPORT_DATE: [2020-05-31 Sun 16:51]
:END:

I’m now all set to publish draft posts without requiring to use rsync and two hugo builds. :) Publishing a draft post would need me to remove the DRAFT todo keyword, like before, but additionally also remove the :EXPORT_HUGO_SECTION: property on the subtree. This should be quite quick to do, or pretty easy to put into an interactive function if required.

Deploying Hugo Drafts

Sat, 30 May 2020 19:07:00 +0530

Update [2020-05-31 Sun 17:23]: I published a simpler workflow for this, using Hugo’s build options that I wasn’t aware of, because they are relatively new.

A few friends and I run a weekly writing club similar to this. Once we have our drafts ready, we usually share them for reviews, etc.

Previously, I used to use Nikola, and it’s default behaviour is what worked well for this workflow – share the posts URL with reviewers, but the post shouldn’t appear in the feed, post list, etc.

If you set the status metadata field of a post to draft, it will not be shown in indexes and feeds. It will be compiled, and if you deploy it it will be made available, so use with care. If you wish your drafts to be not available in your deployed site, you can set DEPLOY_DRAFTS = False in your configuration.

I moved to using Hugo a few years ago. By default, it doesn’t publish the draft posts, but you could ask it to do so using the --buildDrafts option. But, that doesn’t do what I want – draft posts are treated just like any other post, and they appear in the feeds, post list, etc.

Given how fast hugo builds are, though, it is pretty easy to hack-up the behaviour I want.

Run a build with drafts, say to dev directory.

DEV_DIR="dev"
# Build drafts to dev dir
hugo --cleanDestinationDir -D -d "${DEV_DIR}" > /dev/null

Run a build without drafts, to public directory.

PUBLIC_DIR="public"
hugo --cleanDestinationDir -d "${PUBLIC_DIR}"

Copy the drafts from the dev build to the public build

rsync -ri --ignore-existing "${DEV_DIR}"/ "${PUBLIC_DIR}"

Deploy the public build

The code above is here as a single script. Here is the full deployment script for my blog.

My New Standing Desk

Thu, 27 Dec 2018 10:48:00 +0530

There is so much popular-science out there that describes sitting as the new smoking. But, there’s also some research that talk about standing all day not being good for the heart, etc. This article refers to a lot of research that present both sides of the coin.

I’m currently leaning towards switching positions regularly, rather than staying in a single posture for hours together. Standing or sitting – keep moving, keep taking breaks. But, standing makes it slightly easier to get more movement - just removing the need to get up from that chair, seems to help.

Some years ago, I built myself another make-shift standing desk, but didn’t end up using it - because it turned out to be not sufficiently big and comfortable to work on.

I have been looking around on the Internet to buy myself a standing desk. But, they are either super expensive, or not adjustable enough to ensure the right ergonomics. Given there’s so much being written against sitting, I’d have expected there to be enough demand for these desks to bring down their prices!

Last week, I built another desk, which feels much more sturdy than the previous one. It’s also much more easy to move around (within the house), and a breeze to assemble and disassemble. Again, I re-purposed some “furniture” lying around.

I’ve been using it only for a few days now. I’ll probably make another post with my experiences of using it. I plan to set it on the floor and use it as a squat desk, when I’m tired of standing. Let’s see how it goes!

Thanks Shantanu Choudhary for reading a draft of this post.

Software development practices for sports teams

Sun, 23 Sep 2018 11:31:00 +0530

Motivation

A few weeks ago, I came across a talk on Mob programming — a technique for working on a software project as a team, that was accidentally discovered by a team trying to figure out how to work well together. The speaker lets us in on how they stumbled on it, and how they fine-tuned it to make it more effective while being relaxed and enjoyable.

A few days before that, I shared a post on “getting feedback (as a software developer)” with my Ultimate Frisbee team-mates and it seemed to resonate with them. I’m often looking at things happening in my sports team from a software guy’s perspective, since I spend a lot of time being the software guy. But, I rarely, if ever, directly share these thoughts and ideas with my team-mates.

But, the mob programming talk inspired me to make a conscious effort to look for software development patterns that could be adopted by our (sports) team.

Most players on our team have been playing Ultimate for a while now (2-5 years). So, we have more problems due to miscommunication and everyone not being on the same page, than we have due to lack of skills or knowledge. When someone on the team does not understand the plan, the whole plan goes for a toss.

How do we make sure that everyone understands what the team is trying to do and is able to contribute their bit to the team’s plans and success? How do we make playing with each other as enjoyable as it can be?

“The object isn’t to make art, it’s to be in that wonderful state which makes art inevitable.” — Robert Henri

Some ideas for adoption

Here are some ideas that, I think, could be adopted by our team. Some of these are things that teams, including ours, do in some shape or form. But, identifying and naming would make it easier for people to talk about them and develop a common understanding of why and how to do them.

Ubiquitous Language

Eric Evans, in his book Domain Driven Design, talks about building a Ubiquitous language in the team. It grows as the team’s understanding of the problem grows, and helps the team communicate clearly throughout the development cycle.

Our team plays a lot of zone defense, and it usually works well for us But, it wasn’t working particularly well, during one of the tournaments last year. Luckily, we had Moby playing with us, and he asked us some probing questions around the objective of the play and the roles of each of the players. With just a short chat and a few small adjustments, the zone became way more effective. Essentially, we built a common understanding of the objective of the defense and brought everyone onto the same page.

Ultimate teams use a lot of lingo – a lot. New players just pick it up by osmosis, at practice, from the senior folks. This works most of the time, except when it doesn’t. Each person on the team has a slightly different understanding of what a term means, and the difference is sometimes big enough to cause a plan to fail due to miscommunication. Spending time building a common understanding would make the team communicate and operate more effectively. Practicing game scenarios is essentially a way of doing this, but it would be worth thinking about and discussing the terms and the language being used.

Off-field lingo should also be clarified using a similar process – for instance, what does “good sideline support” mean? What does giving or not giving “feedback on the field” mean?

Sprints

Agile development recommends continuous development of software. Often software teams use 2-4 week development cycles, called sprints. For each sprint, the team sets itself a goal, and the whole team prioritizes tasks and makes decisions in-line with this goal.

Sports teams would also benefit from sprints – short and focused practice cycles, where everyone on the team is aware of the team’s goal. This would make it much easier for players to help each other out. Also, players can align their individual goals with the team goal, making practices much more effective.

Retrospectives

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly. — The Agile Manifesto

Most Ultimate teams do some kind of post-practice retrospectives, though, they don’t call it that. These retrospectives are a discussion of the good, bad and the ugly. Usually, no conscious decisions are made about what practices are working well for the team, and what should be taken forward to upcoming ones.

Slightly longer term retrospectives would be quite useful too. If a team adopts the idea of sprints, each new sprint should start with a retrospective of the last one and planning for the next.

A retrospective’s participants are expected to follow the prime directive.

Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand. — Norm Kerth, Project Retrospectives: A Handbook for Team Review

Internalising this would be really helpful especially during post-tournament retrospectives, where emotions can run high – it would keep the discussion positive and focused on identifying problems and finding solutions, rather than spiraling into blame games, defensive arguments, etc.

Daily Stand-ups

Agile teams gather each day at the start of the work-day, to let each know about the progress of the work-items. The team tries to help each other get rid of the impediments to progress.

Ultimate teams could also benefit operating in this style – everyone shares with each other what they are working on, and the problems in their learning. This would make it easier for others to look out for things to help with. Team-mates would limit themselves to giving feedback to the thing that a player is working on.

Players setting themselves a small number of clear focus points would also force them to consciously think about and work on their game more effectively.

Face-to-face communication

The most efficient and effective method of conveying information to and within a development team is face-to-face conversation. — The Agile Manifesto

As our team has grown larger over the years, more and more communication has moved to the WhatsApp group. Keeping everyone informed via face-to-face communication, at this team size needs a lot of co-ordination and planning, and the time at practice is considered to be too precious to be spent talking and not practicing.

But, clearly WhatsApp doesn’t seem like the best medium to have discussions. Discussions rarely, if ever, happen on our groups. It often just seems to be information dissemination. Using a better communication tool could mitigate a few of these problems, but face-to-face discussions have other benefits. They bring about a greater sense of belonging to the team and increase buy-in into the team’s plans.

Unit tests vs. Integration tests

Practice games are helpful to gauge if a team is playing well together and if its plans are working. This seems analogous to integration tests — checking that we have functional software, end-to-end.

But, in the early stages of development, it’s common to write unit-tests – tests to ensure that each of the individual components works well in isolation. An analogous thing would be to have individual evaluations to help players identify weak areas, and to measure progress in those areas. It would also prevent players from slipping on important “indicators”.

Other ideas to think about

Some ideas that I find interesting, but don’t yet know if and how they can be adopted by sports teams are below.

Mob programming

In a mob programming session, everyone on the team is working on the same computer, in the same room, at the same time. This seems tremendously ineffective at the face of it, but turns out to be a good way to produce high quality software, quickly.

Hammock driven development

Bugs are incredibly expensive to be found and fixed in production software. They are cheaper to fix in the development phase. But, the design phase is the best place to fix them.

Rich Hickey recommends feeding a lot of information about the problem you are trying to solve to your waking mind, so that your background mind can feed off of it while you are asleep, and make useful connections.

Conclusion

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done. — The Agile manifesto

This sounds exactly like what players in our team need too. The captains and the mentors in the team should work towards providing the environment and the support everyone on the team needs, and trust them to do the job.

Thanks Meghana Iyer, Ravitheja Tetali, Shantanu Choudhary, Vandith PSR, Varun Rangarajan and Vivek Krishnaswamy for reading drafts of this post and giving helpful suggestions.

Where do the Children Play (in Bangalore)

Mon, 12 Feb 2018 13:11:00 +0530

A bunch of us have recently started playing Badminton regularly (on weekends), in Bangalore. We use Playo to book a court to play on. But, all of us live in different parts of the city, and it’s often difficult to find a convenient place to play. Often, we end up in bad courts because we don’t pay enough attention to the ratings, while focusing on trying to find a court in a convenient location.

To help make this search easier, I built a small web app that shows the venues from Playo on a map, and then allows you to specify the locations of players, and the radius (in km) that they are willing to travel. Hopefully, we end up finding a good court in the overlapping region. Clicking on the venue marker allows you to jump to the Playo booking page for that venue. The source is available here.

I usually go with leaflet when I want to do map related stuff. This is the first time I used Google APIs, and it was a pretty decent experience.
I also ended up using the Google URL shortener API to make it easier to share URLs. The entire state of the app is being saved in the URL fragment, so that URLs are shareable, but it makes them long and ugly. Short URLs seemed like a good workaround.
I got most of it working with vanilla javascript, until I started doing things where I felt it would be nice to have a light-weight two-way data and view binding library. I ended up trying knockoutjs for the first time, and enjoyed using it.
The data for Playo venues is updated everyday using a cron job that runs on Travis. So, the ratings of venues should be pretty up-to-date.
But, Playo allows players to rate venues overall, and not by sport. So, it may so happen that the venue is a decent one overall, and has good facilities for some other sport, but the badminton courts are still bad. But, some venues are badminton only, and in those cases the ratings are more useful.
It should be pretty easy to make the app fetch data for other things like restaurants/movie theaters, to make this app do more than just help find Playo venues.

To Moar Badminton!

Multiple remotes with nullmailer

Sat, 18 Nov 2017 04:50:00 +0530

UPDATE 2018-08-20: Moved away from nullmailer

I updated my OS and nullmailer possibly got updated, and this hack no longer works. I’ve switched to msmtp with a file based queue that I implemented myself. msmtp can be invoked with the --read-recipient-from argument, to choose the correct right mail server based on the from address.

This a reference for future-me, and possibly someone pulling off an all-nighter trying to get nullmailer to use the correct “remote”.

What is nullmailer and why use it?

Nullmailer is a simple mail transfer agent that can forward mail to a remote mail server (or a bunch of them).

I use Emacs to send email, and it can be configured to talk to a remote SMTP server to send email. But, this blocks Emacs until the email is sent and the connection closed. This is annoying, and having nullmailer installed locally basically lets Emacs delegate this job without blocking.

Why multiple remotes?

I have multiple email accounts, and I’d like to use the correct remote server for sending email based on the FROM address.

I expected nullmailer to have some configuration to be able to specify this. But, it turns out that nullmailer just forwards the email to all the configured remotes until one of them succeeds.

How do we, then, send email from the correct remote SMTP server?

Currently, I have two remotes - my personal domain (@muse-amuse.in) and GMail.

Having GMail as the first remote in nullmailer’s configuration wouldn’t let me send emails from my personal domain. GMail seems to agree to send the email coming from @muse-amuse.in, but overwrite the MAIL FROM address and change it to my GMail address.

So, @muse-amuse.in has to be the first remote. But, this server also seemed to accept and send emails with a @gmail.com FROM address. This was causing emails sent from my GMail ID to go into spam, as expected.

I had to reconfigure this mail server to reject relaying mails that didn’t belong to the correct domain names – i.e., reject relaying emails which had @gmail.com in the FROM address.

smtpd_sender_restrictions had to modified to have reject_sender_login_reject along with other values, and the smtpd_sender_login_maps had to be set to allow only the @muse-amuse.in domain. This serverfault answer explains this in more detail.

Emacs frame as a pop-up input

Thu, 14 Sep 2017 22:26:00 +0530

I wanted to try using a dialog box/pop-up window as a prompt to remind me to periodically make journal entries. I had the following requirements:

Simple, light-weight dialog box that allows text of arbitrary length
Ability to launch the dialog from the shell
Ability to have some placeholder or template text, each time the dialog is shown
Save the input text to a specific org-mode file
Write as little code of my own, as possible, to do this

I had initially thought about using a tool like zenity, or write a simple dialog box in Python using Qt, wx or even tk, and then yank the input text at the desired location. This probably wouldn’t have turned out to be too hard, but getting things to look and work exactly the way I wanted would have required more code than I was willing to write or maintain.

After avoiding doing this for a while, I finally realized that I could simply use Emacs with a new frame with the appropriate dimensions, and with the correct file/buffer open to the desired location. This would

eliminate the need for me to write the UI myself
eliminate the need to do text manipulation in code, to yank it at the right place, in the right form. By directly opening up the editor at the required location, the onus is on me (as a text inputting user) to put it in, the way I want it.
additionally provide me the comfort of being able to write with the full power of Emacs - keybindings and all that jazz.
let me leverage elisp to do essentially whatever I want with the buffer being displayed as the dialog box.

I ended up with a command that looks something like this

emacsclient -c -n\
            -F '((title . "Title") (left . (+ 550)) (top . (+ 400)) (width . 110) (height . 12))'\
            -e '(pc/open-journal-buffer)'

This worked pretty nicely, except for the fact that with gnome-shell, the pop-up frame doesn’t always appear raised. It often gets hidden in the Emacs windows group, and the whole idea of the pop-up acting as a reminder goes for a toss! But, thanks to this Ask Ubuntu post, I could fix this pretty easily.

emacsclient -c -n\
            -F '((title . "Title") (left . (+ 550)) (top . (+ 400)) (width . 110) (height . 12))'\
            -e '(progn (pc/open-journal-buffer) (raise-frame) (x-focus-frame (selected-frame)))'

Create a Public Jupyter Server, quickly!

Thu, 25 Aug 2016 00:00:00 +0530

I create public Jupyter notebooks once in a while, to collaborate with a friend, or to make it easier for myself to work with data on a remote machine.

Each time I need to look up the docs, and manually set-up a few things, before I can start using the notebook.

I just wrote a bash script that does the following, quickly -

Install Jupyter into a temporary virtualenv
Create certificate files
Start the server with https enabled and a password
Delete all temporary files, when the server is shutdown

You can get it here, if you’d like to use it.

Better styling bookmarklet

Tue, 07 Jun 2016 14:39:01 +0530

Some sites that I like a lot for their content, have zero styling on them. I find it pretty hard to read when the lines are long and I have to read all the way on my screen.

I had a simple bookmarklet until now that just changed the width of the page.

document.getElementsByTagName('body')[0].setAttribute('style', 'width: 600px')

Kamal shared with me a recommendation for 7 simple styling rules that will make pages easier to read. I liked them, and modified my bookmarklet to use these rules now.

Drag and drop the link below onto your bookmarks bar if you want to use it.

Better website

The code in a readable format is below.

// Create new stylesheet

(function() {
    // Create the <style> tag
    var style = document.createElement("style");

    // Add the <style> element to the page
    document.head.appendChild(style);

    // Add rules from bettermotherfuckingwebsite.com
    var sheet = style.sheet;
    sheet.insertRule("body{ margin:40px auto; max-width:650px; line-height:1.6; font-size:18px; color:#444; padding:0 10px; }");
    sheet.insertRule("h1,h2,h3{ line-height:1.2 }");

})();

Update [2016-06-07 Tue]: baali pointed me to Firefox’s Reader View which works well too.

Reading-time based scheduling

Mon, 06 Jun 2016 00:00:00 +0530

I had posted a link to an poem written on Medium on a Slack channel that I use with friends. A friend said that she liked the fact that the Slack article preview had the reading time from Medium in it. She could decide whether or not she wanted to read the poem or any other article at that moment.

This gave me the idea for a reading time extension for my browser, or my feed reader or my bookmarks – my reading list. The first version should be able to compute or extract the reading time for an article or a tab in my browser, and index them. I want to be able to specify the amount of time I will be able to spend reading, and be presented with something from my reading list. I think this would help with scheduling the reading of longer articles, and also to actually help me get through my reading list.

Reading time estimates that use heuristics based on word-count may not really work, and may do more harm than good. But, it may still be worth a try to see if it helps my reading habits in any way. A quick search pointed me to this extension, that can give the reading time for any page but doesn’t really do what I want.

Elfeed hook to fetch full content

Sat, 19 Dec 2015 13:42:29 +0530

I have started to use Pinboard’s unread tag as my to-read list. It has a bookmark-let that works pretty well for adding stuff into my “to-read” list. I then catch up on this list using elfeed and subscribing to the unread items' RSS feed. The work-flow is pretty nice for adding stuff into the list, and finding items on the list. But, when it comes to the actual reading part, the entries in the feed don’t have the actual content I want to read, and I end up opening the links in a browser.

Inspired by a comment from FiloSottile, I realized it should be pretty easy to setup a hook that fetches the actual content to make my reading work-flow smoother. I wrote a small script, using python-readability, to fetch the page content, given a URL. This script is then hooked onto elfeed-new-entry-hook, to fetch content of for new entries as they are fetched. All the old entries can be easily fixed with a single call to elfeed-apply-hooks-now.

(defun pc/get-url-content (url)
  "Fetches the content for a url."
  (shell-command-to-string (format "~/bin/get_article.py %s" url)))

(defun pc/get-entry-content (entry)
  "Fetches content for pinboard entries that are not tweets."
  (interactive
   (let ((entry elfeed-show-entry))
     (list entry)))

  (let ((url (elfeed-entry-link entry))
        (feed-id (elfeed-deref (elfeed-entry-feed-id entry)))
        (content (elfeed-deref (elfeed-entry-content entry))))
    (when (and (s-matches? "feeds.pinboard.in/" feed-id)
               (not (s-matches? "twitter.com/\\|pdf$\\|png$\\|jpg$" url))
               (string-equal "" content))
      (setq content (pc/get-url-content url))
      (setf (elfeed-entry-content entry) (elfeed-ref content)))))

(add-hook 'elfeed-new-entry-hook #'pc/get-entry-content)

Say Howdy with Emacs!

Thu, 28 May 2015 00:00:00 +0530

Staying in touch with people is something I’m not very good at. Since I am not on popular (among my friends/family) networks – FB and Whatsapp – I don’t even see random updates from people, to get some sense of being in touch.

I recently read some old posts by Sacha Chua and was inspired by how much code she had for contact management in her old blog posts. I was inspired by this post in particular to try and be more meticulous about how I stay in touch with people. Michael Fogleman blogged about his contact management work-flow using keepintouch. It seemed to do most of what I wanted, but I wanted this to be integrated with my org-contacts-db and I felt having native elisp code would make it easier to hook up email, chat, etc. to this.

I ended up writing a small utility called howdy to help me keep in touch with people. It currently has only a couple of features:

M-x howdy lets me update the last contacted timestamp for a contact.
Shows me contacts that I’m out of touch in the agenda, once I add the following snippet to an agenda file.
```
* Howdy
  %%(howdy-howdy)
```

I also have a few hooks to hook up jabber messages and email to update the db. I’ve added them to howdy-hooks.el in case anybody else wants to use them. They can also be used as examples to write other hooks. Feel free to contribute other hooks or suggest improvements. The library also ships with a modest test suite, that will hopefully make it easier for others to contribute.

I’m looking forward to experimenting with this over the next few weeks and improving it. Hopefully, it’ll help me keep in touch, better than I do now.

org-drill for making it stick!

Sat, 17 Jan 2015 00:00:00 +0530

Those who read the last few posts here, would know that I have been experimenting with org-drill (a spaced repetition extension to Org mode). I have been using the system (almost) religiously for the past 2 months, and I do find that it has helped a great deal! (in some respects). I have also spent a considerable amount of time trying to reduce the friction to put new stuff into the system, and am constantly on the look out for further improvements.

Using this system has definitely helped with retention, and I find that I can recall quite a few things I have read a few weeks ago, that I would normally have been unable to. Though, I can recall a lot of information, I have been having a feeling of “fragmentation”: the feeling of just retaining individual bits/fragments of information, while losing out on actually internalizing the knowledge; not seeing the big picture, etc.

Wozniak (the author of super-memo) warns against learning without understanding, and memorizing before actually learning stuff. I haven’t consciously added stuff into the system that I didn’t understand (when I added it), but, later it does feel like I have lost some connections or the understanding, and am only holding onto the fragments of information.

The problems as explained in (read: as interpreted by me from) Make it Stick appear to be:

The understanding (if any) at the time of adding stuff into the spaced-repetition system is untested. It may just be familiarity masquerading as understanding.
The lack of any spaced repetitions for the overall concept/understanding and actual repetitions only for individual bits doesn’t help retention of the understanding (even if there was any, in the first place).

To work around this, I’m going to try adding questions that test understanding, to the system. The Super-memo team strongly recommends keeping the drill items small and easy to answer. This may be helpful in keeping each drill session short, but I would really like to add conceptual questions to the system, and see how it goes. I hacked org-drill to allow me to type out answers, before looking at the “correct” ones. This is an adaptation of a system that a fellow Hacker Schooler uses, and shared. Also, hopefully forcing myself to type out the answer will help me get around the problem of sometimes saying “yeah I know that”, then looking at the answer only to reaffirm the feeling of familiarity, rather than actually testing myself. I’m still going to continue adding quick and short questions that test “bits of information”, though. But, hopefully the additional conceptual questions are going to tie things together and help fill in the gaps. Lets see how this goes!

For those interested, my hacks to org-drill below. The code is really a hack, and welcome any suggestions on cleaning up the code.

(advice-add 'org-drill-presentation-prompt :around 'pc/org-drill-presentation-prompt)

(defun pc/org-drill-presentation-prompt (old-fun &rest fmt-and-args)
  "A presentation prompt that allows capturing answers."

  (let ((cb (current-buffer))
        (heading (nth 4 (org-heading-components)))
        (entry-id (org-entry-get (point) "ID"))
        (input ""))
    (switch-to-buffer-other-window "*org-capture-drill-answer*")
    (org-mode)
    (insert "# Hit C-c C-c once you are done answering!\n")
    (org-insert-heading-respect-content)
    (insert (format "Answer: %s" heading))
    (org-entry-put (point) "QUESTION_ID" entry-id)
    (goto-char (point-max))
    (insert "  ")
    (org-time-stamp-inactive '(16))
    (insert "\n\n  ")
    (while (not (and input (equal input "")))
      (ignore-errors
        (execute-kbd-macro input))
      (setq input (read-key-sequence nil)))
    (switch-to-buffer-other-window cb)
    (apply old-fun fmt-and-args)))

(advice-add 'org-drill-reschedule :around 'pc/org-drill-reschedule)

(defun pc/org-drill-reschedule (old-fun)
  "Calls the original reschedule, but also archives the answer"
  (prog1 (funcall old-fun)
    (let ((cb (current-buffer)))
      (switch-to-buffer-other-window "*org-capture-drill-answer*")
      (pc/org-refile-to-datetree "drill.org_archive")
      (message (buffer-name))
      (switch-to-buffer-other-window cb)
      (kill-buffer "*org-capture-drill-answer*"))))

(require 'org-datetree)
(defun pc/org-refile-to-datetree (journal)
  "Refile an entry to journal file's date-tree"
  (interactive "fRefile to: ")
  (let* ((journal (expand-file-name journal org-directory))
         (date-string (or (org-entry-get (point) "TIMESTAMP_IA")
                          (org-entry-get (point) "TIMESTAMP")))
         (dct (decode-time (or (and date-string (org-time-string-to-time date-string))
                               (current-time))))
         (date (list (nth 4 dct) (nth 3 dct) (nth 5 dct))))
    (org-cut-subtree)
    (with-current-buffer (or (find-buffer-visiting journal)
                             (find-file-noselect journal))
      (org-mode)
      (save-excursion
        (org-datetree-file-entry-under (current-kill 0) date)
        (bookmark-set "org-refile-last-stored")))
    (message "Refiled to %s" journal)))

Jabber message queue

Wed, 10 Dec 2014 00:00:00 +0530

I’ve always wanted to be able to queue up messages to send to friends, until I go online the next time. I tried using email instead of chat a few times, or just ended up staying online with a busy status.

Finally, now that I have started using jabber-mode for chatting from within Emacs, I took out the time to write a “queuing system” for sending chat messages, similar to the mail queue for smtpmail. Instead of persisting sexps, though, I persist the messages in a JSON format and the queue is flushed every time I connect to jabber, in a jabber-post-connect-hook.

To make the interface as similar to the interface available when I am online, I hacked completion for the to ID using email addresses in my address book (mu4e~contact-list). I really like the fact that the chat buffer opens up, and I can type and send messages like I usually do. Hitting RET after typing a message queues it up, instead of trying to send it. Smooth!

The code is in my .emacs

erc-notifications when Emacs not in focus

Thu, 06 Nov 2014 00:00:00 +0530

I have been trying to get ERC working with notifications. Julien Danjou’s wonderful notifications module for ERC is great, but it is annoying to get notifications even when Emacs is in focus.

I had looked at circe-notifications, which has the feature but uses xdotool and xprop to do it. I was looking for something simpler, though… and it suddenly struck me that I have an auto-save hook in Emacs that is run when I focus out of it. I wondered if I could disable and enable notifications on focus, and it worked.

In case it is useful for somebody else -

(add-to-list 'erc-modules 'notifications)
(erc-notifications-mode)
(add-hook 'focus-out-hook 'erc-notifications-enable)
(add-hook 'focus-in-hook 'erc-notifications-disable)

I wonder if there are some corner cases where this doesn’t work, and that’s why the author of circe-notifications chose the tools that he did.

More input sources for org-drill

Tue, 04 Nov 2014 00:00:00 +0530

I’ve been trying to use org-drill regularly for the last few weeks. I don’t know how well it’s been going but I have been sticking to the routine religiously. I haven’t yet really tried out incremental reading, but in an attempt to make it as easy as possible, I wanted to have a pdf-reader integration, and some kind of integration with Kindle highlights. Browser integration is pretty straight-forward, thanks to some simple java-script.

I looked for a pdf-reader with some sort of plugin support, but I found nothing in Evince or Okular. I thought about pdfjs but it seemed slightly clunky to open pdfs in a browser, though I might shift to this if I don’t like what I finally ended up with. Good old xpdf seemed to be the only pdf reader that had some support for custom keybindings that allowed users to run external commands. With a little Python, I was able to setup a work-flow to capture snippets from xpdf, to add to org-drill. Custom key-bindings somehow don’t seem to work on xpdf bundled on Ubuntu. So, I ended up downloading and using the binary available on the xpdf site.

For Kindle highlights support, with minor updates to Thamer Mahmoud’s clip2org, I have a simple way of getting all the “new” clippings/highlights as org-drill headlines. I haven’t really started using this, and once I do, I may end-up automating even the merging of these items into the org-drill notes file. I’m looking forward to making better use of my Kindle, with this feature!

I don’t know if it would be useful to have more context information like section titles/chapter titles when capturing from html/pdf, but it seems like an interesting problem to try to solve.

Also, it might be easier(?) if I probably tried to have a DE level keybinding, and some code to get selection and file name of the currently active window/application.

Recurse Center, 2014-07-29

Tue, 29 Jul 2014 11:20:24 -0400

Hacking: learning by breaking stuff

Post lunch, I started pairing with Stephanie on the exercises in the Hacking book, and that turned out to be super fun!
We got through the chapter on exploitation, which focussed on stack overflows, heap overflows, overflows in function pointers, and string formatting exploits. It acted as more than a gentle refresher for C, and was super fun!
Running the examples on a 64bit system, added an additional amount of thinking required, and was good, IMO.
I hadn’t worked along with Stephanie on the material in the first two chapters. So, I went back home and tried to catch up, but couldn’t get all of it done.

Miscellaneous

I spent part of the morning wrapping up the install script for the raspberry project, and cleaning up the repository. It looks reasonable now, and I think I’m done with it until the end of HackerSchool.
I also tried to look for “easy” python issues that would involve me writing some C code, but didn’t really find anything other than documentation… I guess, I should pick a module and look for bugs in it, instead.
Also, while chatting with a friend, I had to explain that I don’t mention the last names of people or link to them, for privacy reasons. Hacker School is particularly careful about not divulging information about who are at Hacker School each batch, and I want to keep that tradition.

Let there be light, in skype!

Sat, 29 Mar 2014 00:00:00 +0530

At home, my video would always show a silhouette in G+ and Skype, and I always thought that this was because the lighting at home was not sufficient. At office, the video was decent. But, moving to sit right under the light also didn’t help much.

Struggling with a bunch of tools like v4lctl, guvcview didn’t help. But, during these struggles I noticed that cheese would show me bright and cheerful, while all the other programs showed my silhouette.

I set out to “fake” the output of cheese as a video device that skype and other programs could use. But, I didn’t have to go all the way. I ended up using v4l2loopback to create a loopback video device, and just using gst-launch to redirect video to that device, did the trick! Thanks gstreamer! Thanks v4l2loopback! :)

Here’s a convenient script to use it every time I need it.

Scraping Google Groups

Tue, 31 Dec 2013 00:00:00 +0530

I was playing around with a few ideas for creating a timeline for the mumbai ultimate group, and as a part of playing around with stuff, for that idea, I ended up wanting to scrape all the emails on our google group. After looking around a little bit, I failed to find anything that claims to be able to do this.

So I ended up writing my own hacky script to download all the emails sent on the group. Like I said, this is a hack and can be improved quite a bit, but I am not inclined to do anything about it, right now. Since, everything about google groups is ajaxy, this script uses selenium and does things on the page, that one would do by hand. It’s not something that I am proud of, but it does the job!

Hack on Noetic Nought

Restoring a broken Ubuntu upgrade

The crash

chroot to the rescue!

Offline updates

But more missing packages!

Outro

Responsive Auto Export for Org Hugo

Enter Emacs profiler

Turn off Org ID location update

Turn off buffer preprocessing, maybe?

Moar workaround!

Outro

Made in 2022

2022 projects

Review of 2021 projects

Outro

Moving from Heroku to Fly.io

My Heroku Apps

Why not pay Heroku?

Why Fly.io?

Migrating a simple app

Migrating a bigger app

All’s well that ends well

GitHub Gists from Emacs Orgmode

Motivation

Hackish wrapper around gist.el

MELPA submission and feedback

Current version: ox-gist

Outro

Made in 2021

Deploying Hugo Drafts, simplified

Deploying Hugo Drafts

My New Standing Desk

Software development practices for sports teams

Motivation

Some ideas for adoption

Ubiquitous Language

Sprints

Retrospectives

Daily Stand-ups

Face-to-face communication

Unit tests vs. Integration tests

Other ideas to think about

Mob programming

Hammock driven development

Conclusion

Where do the Children Play (in Bangalore)

Multiple remotes with nullmailer

What is nullmailer and why use it?

Why multiple remotes?

How do we, then, send email from the correct remote SMTP server?

Emacs frame as a pop-up input

Create a Public Jupyter Server, quickly!

Better styling bookmarklet

Reading-time based scheduling

Elfeed hook to fetch full content

Say Howdy with Emacs!

org-drill for making it stick!

Jabber message queue

erc-notifications when Emacs not in focus

More input sources for org-drill

Recurse Center, 2014-07-29

Hacking: learning by breaking stuff

Miscellaneous

Let there be light, in skype!

Scraping Google Groups

`chroot` to the rescue!

Hackish wrapper around `gist.el`

Current version: `ox-gist`