Tools on Noetic Nought

Made in 2022

Wed, 08 Mar 2023 23:35:00 +0530

Keeping with last year’s tradition, this post comes when we are almost 1/5th through this year.

2022 projects

I built a personal Expense Tracker over the month of October using SQLite and Streamlit, because I wasn’t comfortable sharing my financial data with a 3rd party. I’ve tried to keep it simple, so it’s not a pain to use. But, I’ve also tried to keep it extensible, for others to try and use.

The last few months have been quite busy personally, and I haven’t been able to use it as much as I would like to, but hopefully things will settle down soon. A couple of things that I’d like to improve:

The Sqlite file is stored locally, and the data is not synced between different computers.
The tagging/categorization of expenses is very manual, and a little bit more automation would be handy.

artful-dodger

A Next.js project for hosting an art/photo gallery with metadata stored in a Google Spreadsheet and images in a Google Drive (with an optional CDN, recommended). I built this for a friend, who is experimenting with creating “a more equitable model for curating, buying, and selling art”. The gallery is currently in “private beta”, and I can’t link to it. But, here’s a sample gallery here, whose configuration lives here.

ox-gist

I released my first Emacs MELPA package – ox-gist – an Orgmode backend to export and update sub-trees and buffers to GitHub gists. It was a great experience contributing to MELPA. I wrote a blog post about it too.

Termux scripts

I’ve started using Termux on my Android phone and have found the Text-to-Speech (TTS) API quite handy. I’ve a couple of scripts on my phone that lets me use TTS to read out contents on the clipboard or an article from my browser.

Fantasy League Site

With help from a bunch of volunteers collecting and validating data, Jonny and I built a website for the Fantasy League for Indian Ultimate State Championships. There was nothing technically challenging about the project, but it just involved a lot of data cleaning and co-ordination between the volunteers. The data was collected using pen and paper, instead of a digital app for collecting the stats. This made the data cleaning and validation significantly harder. I’d definitely push for digital data collection if people try to do something like this again.

Bookmarklets

A web-page with a bunch of Bookmarklets from my browser’s Bookmarks bar. I’ve had most of them around for a while, but putting them on a web page makes it easy to share and find myself on other browsers/computers.

Music

Unlike 2021, I’ve only been able to record a couple of songs. And this year (2023) hasn’t been any better, so far. I hope I’ll be able to make amends to this, soon, though.

Review of 2021 projects

Earworm

I didn’t make any updates to the package, not because it’s perfect but because I didn’t use it much in 2022 – thanks to how little I’ve been able to record.

While, writing this blog post I realized that workflow had a couple of annoyances. I improved the code to fix them.

I need to move an audio file back-and-forth between my phone and my laptop, one full round trip to add ID3 tags and cover image. This makes me put off uploading recorded files to the site. Also, the new media file needs to be named in a specific format. I’ve added a new CLI subcommand to automate all of this for me.
I always need the full collection of the music files on the computer, where I’m updating the site from. I can just scp the existing site onto my computer and merge the media directories.

Ukulele Tutorials Aggregator

The job of manually updating the site is a lot of work, and I haven’t had the interest to keep doing it.

RPi Server and Phone Backup

The physical setup was accidentally dismantled by folks I share my physical space with, and I haven’t been able to put back everything in place, yet. I’m hoping I’ll be able to get back to doing this at some point this year.

Outro

I’ve already built a couple of small things in 2023, and I hope I am able to find some interesting things to do this year.

Link to the 2021 post.

Made in 2021

Thu, 24 Feb 2022 02:23:00 +0530

Some things I made in 2021:

Earworm: A static site generator to listen to audio (and video?) files in a directory. I use it to host a couple of simple websites for music files. It also happens to be the first package I uploaded to PyPI despite writing Python for many years.
Ukulele Tutorials Aggregator: Inspired by Ukealong.com, I set out to create an aggregator for Ukulele Tutorials for Indian songs. I’ve, with help from Kamal, populated about 300 songs on the site. I haven’t been able to keep up with maintaing it, after an initial bunch of work. I’ve wanted to automate more of the workflow, but haven’t yet.
RPi Server and Phone Backup: I’ve started to use a Raspberry Pi to backup data from my phone and laptop to a personal Hard disk. I’ve ansible scripts to reproduce the RPi’s setup, and some scripts to “sync” my phone’s data every few hours, when it’s on the same network. There’s a lot more that I need here, before I can ditch cloud backups of important things.
Music: I made covers of about 10 songs on the Ukulele, that I shared with some friends. I’ve ways to go before I’m comfortable sharing them on the web, publicly.

I’m glad to have had the energy, time and resources to be able to spend time working on some fun things despite everything that 2021 had in store for us. Let’s see what 2022 has in store!

PS: This post is quite late into 2022, but it’s an excuse to post something to a blog that’s not seen any updates since the middle of 2020.

Hat tip to Bryan Braun whose post inspired me to write this

Instagram's user-friendliness

Wed, 24 Jun 2020 20:13:00 +0530

TL; DR: I’m a video content creator on Instagram, and don’t tell me Instagram is user friendly. Being able to preview posts before making them viewable for everyone will go a long way, though.

Introduction

I’ve been using Instagram quite regularly from around the beginning of this year, posting about one video a week on an average. Previously, I used it primarily as a content consumer for about an year, before getting off it for quite long since it started to turn into a big time-suck. In the current stint of using Instagram as a content-creator, I’ve had quite a few frustrating user experiences. On a platform that comes with a giant reputation of being user-friendly, this is a surprise.

A lot of my posts this year have been video content. While posting them, I ran into a bunch of things that didn’t “just work”. For instance, in the first 3 or 4 posts of the Humans of TIKS series, every post had some thing broken, leading to a poor viewing experience.

Some of my friends, though, found it surprising that I was frustrated with Instagram. “Isn’t it very user-friendly?”, I was asked by many. I think Instagram is just riding on their reputation from the past. The newer features, especially around video content, aren’t exactly “user-friendly”. Instagram does focus on giving the content consumer a nice (read: addictive) experience. But, as a creator, there are a bunch of hoops to jump. Content creators probably put up with it just for the reach Instagram gives.

In this post, I’ve listed down everything that I see as broken. I’ve also tried to suggest improvements to each broken feature that would make the experience more friendly. I start with the more common problem scenarios and progress towards the edge cases.

Caption formatting is needlessly hard

I’ve always had to fiddle with the caption of a post, after publishing it, if it had multiple paragraphs. There’s a magic algorithm that Instagram uses to decide if multiple line-breaks in the text actually mean separate paragraphs or not.

After some trial and error, I’ve figured out a few things about it. For instance, leaving a trailing space at the end of a paragraph causes paragraphs to be joined together. But, when the second “paragraph” starts with a non-text character, it doesn’t seem to matter what the previous paragraph ends with, they always are joined together as a single one.

I don’t want to find out these details by trial and error. Adding previews for captions would make this so much nicer. Some documentation for the “magic” would probably help too. Instagram could even look at WhatsApp, its messaging cousin, for some lessons on how to make writing text easier. Switching to a more common markup language would be a nice bonus.

Albums are restrictive

Posting multiple images in a single post (an album) is a pain. All images are cropped to the same orientation – square, portrait or landscape.

Until a few years ago, Instagram required every image to be square. If you want to keep things dead simple for your users, the square image restriction is as simple as it can get. At least it’s easy to understand and remember, even though I can’t post my pictures exactly the way I want.

The current same orientation restriction is worse on both fronts. I’m still left with not being able to post my pictures exactly the way I want. But, also the restriction is not easy to infer from the UI. I had to fiddle with the UI and my pictures for a while, to understand that there’s no way around this, and that Instagram is actually enforcing this restriction.

I don’t want to be thinking about Instagram’s restrictions while clicking my pictures. I want to take the best pictures I can, and upload them with the best possible orientation/crop.

Instagram may have these restrictions in place to ensure the best possible “viewing” experience. But, for the creators the user-friendly thing to do here would be to remove these restrictions. I think, even going back to the universal square image restriction would feel more user-friendly!

Public profiles, really?

I think, a public profile should be viewable by anybody without having to login, or having to use a specific kind of device. But, Instagram’s idea of “public” profiles is quite different.

If you are not logged into Instagram, and are viewing a public profile, you are prompted to login after scrolling down through a few posts on the profile. If you are on a desktop browser, you can’t open up any post from a public profile. You are prompted to login, when you try to view a post. Even on a mobile browser, you are prompted to login after opening 3 or 4 posts. Opening direct links (or opening posts in a new tab) works, though.

Instagram obviously wants to grow as much as it can, and it doesn’t really care about people not on the platform. As a content creator, though, I care about my audience, irrespective of whether they have an Instagram account or not. A user friendly platform would keep their users’ interests ahead of their growth goals and not do such shady stuff. Or at least, tell us the truth and stop calling it “public” profiles.

So many video types!

When I want to post a video on Instagram, I’ve to first decide what kind of a “post” to make. If I want to post a “short” video, it can go into a normal post/album if it is under a minute. If not, I’ll either need to split it up into smaller segments, or post it as an IGTV video. Stories can only be 15 seconds long, or they get chopped into multiple shorter stories.

IGTV supports longer videos – from 1 minute up to 60 minutes long. The videos need to be portrait videos with an aspect ratio of 4:5 or smaller, 9:16 is recommended, while normal posts prefer square (1:1) videos.

Just like with pictures, I don’t want to care about Instagram’s restrictions when capturing a video. I want to shoot my video, and let Instagram do what is required to share that video. I find it strange that there are hundreds of blog posts documenting how to upload an already recorded video to IGTV.

A user-friendly platform would let users upload the videos as they shot them, and prompt about adding some padding, or allowing users to crop and select a part of the video, to match the aspect ratio requirements.

9:16 IGTV videos are broken

The documented aspect ratio for an IGTV video is 9:16, but guess what! They aren’t displayed correctly in the mobile app.

They are displayed correctly in mobile and desktop browsers, though. But, given how Instagram nudges everyone to use the app, I would guess that 90-95% of viewers would be using the app, if not more.

After a lot of trial and error, I figured out that I need to upload my videos at a ratio of 9:21 or lower to be able to prevent the sides from being chopped off. Instagram’s UI or docs don’t mention this at all. Also, this workaround only works when uploading from a destkop browser. I haven’t yet figured out how to prevent mobile uploads from looking broken.

For a visual platform that is so bullish on everything looking perfect, this is horrendous! I don’t know where to place this - is this a bug in Instagram’s processing pipeline? Or is this a part of attempts to provide the best viewing experience? Just display my videos as advertised, please!

Video post-processing is magic

Any video I upload goes through a processing pipeline, that is undocumented, before it gets displayed to the users. This is possibly done to optimize for space, etc., but it has led to broken user experience for at least a couple of my videos.

On one occassion, it made the audio and video go out of sync. To be fair, the video did have some issues with the audio stream data missing for the first 3 seconds of the video, due to a bug in my video processing pipeline. The same video worked perfectly fine in MPlayer, mpv, VLC, and on YouTube. Noticing that Instagram’s post-processing made the audio and video go out of sync, after about 50 people watched it, is such a shame!

To avoid such problems, we decided to make a demo account and upload every video there, before posting it from the real account. Needing to do this seems super broken to me. I still did it, since I wanted everything to work just right. But, that wasn’t to be.

We had a video where we added some background music on the video, and it ended up having a 4-channel audio stream. It turns out the Instagram app on iOS won’t play such videos, and we only detected that after some people in our audience let us know. Instagram has this documented on their IGTV documentation under additional technical details. But, I only read that because things weren’t working. Again, our followers had a broken experience because of this.

It can be hard to “document” all the technical details of the post-processing pipeline. But, it should be fairly easy to detect things like 4-channel audio, and warning users about it!

Even simpler, but user-friendlier thing to do here would be to allow users to view the video after it has been uploaded and processed, before making it viewable by everyone. Currently, draft posts cannot be shared with friends. I’m not sure if they can even be found and viewed from multiple devices – it wasn’t intuitive to do this, when I tried. Just making it easy to preview posts before sharing it publicly would make things a lot lot more user-friendly. It feels quite backward to be using a private account for this to ensure no broken content is published.

This may sound too extreme, but just give me the exact ffmpeg command being run on the uploaded videos somewhere in the docs! I would then be able to verify that my videos are Instagram compliant, without having to upload a dozen videos and checking on multiple devices. With a lot of big brands using Instagram, I think there’d be other “expert” users who would appreciate these details.

Published content is uneditable

I would find the brokenness more tolerable if I could go back and change the uploaded videos. But, Instagram doesn’t let you edit published posts. It does seem like a reasonable restriction, since this content may have been referenced elsewhere. This seems to be Twitter’s take too, for instance.

For something as “heavy” as a video, though, it seems a little too restrictive. It may not be a great idea to allow changing content behind people’s backs. But, I’m not happy about my followers having to watch a broken post. Deleting a broken post and making a new one doesn’t really cut it – it feels spammy.

I’d like to see some kind of versioning added to the videos. Let user upload a newer version, and make it possible for users to see older versions too, if they chose to. Google Drive and Dropbox seem to do this, for instance.

Outro

Instagram started off by allowing users to only post square pictures. It was very easy to undestand and use, even if not very accomodating of different use cases. Everyone and their dog could use it!

A whole bunch of new features have been added in the last few years, trying to compete with the other platforms. Doing the simplest thing - posting a single picture - may still be easy to do, but trying to do anything more than that leads to a frustrating experience quite often. I think the reputation of being user-friendly is now just a vestige of the past.

People who use it regularly get used to its quirks and probably put up with the annoyances, because it gives them a way to reach their audiences with whom Instagram is popular. Instagram is “friendly” to the content consumer, and tries its best to keep the content extremely consumable to keep them coming back for more. The burden of figuring out and making the content fit the restrictions is left on the content-creators. This can often be frustrating.

Thanks to Kartik, Meghana, Nitin, Pratiksha, Ranjan, Ritesh, Shantanu and Vivek for reading drafts (or parts) of this post and helping by boucing off ideas.

The tools we build change us

Wed, 22 Apr 2020 12:54:00 +0530

A couple of weeks ago, I spent some time building a couple of “automation tools” for myself. It is still early days to talk about their impact, but I have already noticed some changes, and have some thoughts to share.

Thanks to the lock-down, I felt like I needed to do something to get more fresh-air and sunlight. I figured out a nice setup to work from my balcony, when I feel like it. It’s nice to sit in the balcony, since there’s not as much traffic, and there’s enough fresh-air and sunlight.

I usually work with dark themes in all my tools - code editor, terminal, OS, chat clients, etc. But, switching to the balcony during the day makes it harder to read text in dark mode. I have to switch all my tools to light mode. And then switch back, when I come back in. It takes me about a minute or so, each time to make this switch. In a few days, this started to seem like work! And I could see myself needing to push myself to step out, or step back in.

A few days later, I spent about half a day writing a tool that could switch modes for all of these programs with one command. If I was doing the switches manually each time, it would take 2-4 months for the switching time to add up to the 4-6 hours, I spent on getting everything working correctly. So, it’s not really a bad investment. But, what matters more is that I don’t feel a mental block to alternate between indoors and outdoors because of all the extra work that manual theme switching felt like. I am getting some fresh air and sunlight everyday. :)

The other tool I wrote recently was for generating a daily summary of what I worked on from my org-mode based clock entries. I add more detail to the bulleted-list summary that gets generated, and then post it on our team’s Zulip. I noticed a couple of changes because of this.

First, I clock things more diligently than I did before. If I forget to clock into something, I go back and add some manual entries, to maintain a proper log. Second, a couple of other people on my team also have started posting a summary of their day’s work, which is quite nice. This tool may have triggered a change in my teammates’ behavior, though I didn’t really intend it to.

We are creatures of habit, often going through the motions, without needing to think about what we are doing. This is evolution helping us save energy for the more critical tasks. But, our environments can have a big impact on our habits. Having tools around us, that are easy to use, can help us nudge ourselves in directions that we want to be nudged in.

The tools we build can change us; if we let them.

Thanks to Rajesh, Ritesh, Tejaa and Shantanu for reading drafts of this post.

Automation and Habits

Wed, 15 Aug 2018 18:44:00 +0530

We usually see automation as a thing we do to save time. But, there’s more to it than just saving time.

Several years ago, I built a tool called org2blog that let me write blog posts in my text-editor, and publish them with just a couple of key-strokes. I spent several man-weeks getting it to work well – adding lots of features including image uploads. The time I spent on developing and maintaining it dwarfs the time saved by automation.

But, in the three months following this, I published 30 blog posts – as many as I would’ve otherwise published during the whole year. org2blog wasn’t just saving time – it drastically reduced the friction to publish new posts. On some occasions where I would’ve previously avoided writing a new post, I ended up writing one because org2blog made it a breeze. I could focus on actually writing the post and not let my attention be hijacked by the repetitive and boring steps that were previously required to start writing a new post and publish it.

Automation abstracts away the details that we would’ve otherwise had to spend our attention on. This reduced mental overload encourages us to “do the right thing”, when we would’ve otherwise been lazy and taken the easier route of not doing something. No wonder, Douglas Adams confessed to happily spending a day on automating something that would’ve taken him ten seconds.

When extended to a team setting this would mean our (automation) tools nudging everyone on the team towards better habits and behavior. We must recognize this and treat our tools as an integral part of our team, and not just time-savers. The Zulip community is a good example of this. Tim, the lead developer of Zulip, pays particular attention to developer workflows in Zulip and is constantly on the lookout to automate tasks and/or eliminate unnecessary tasks, to make it easier for everyone to “do things right”.

The tools we build aren’t just saving us time, but are changing us back. Building better tools helps build better teams. Go forth and automate!

PS: This post is inspired from this talk by Jessica Kerr where, among other things, she talks about Automation and some reasons for doing it.

PPS: Jason Silva talks about humans as being a technology creating species, and how the tools we create change us, and how we change them to change us further. This seems a lot like the idea in this post, except at a much larger scale.

Thanks to Shantanu for reviewing this blog post.