Say Howdy with Emacs!

Staying in touch with people is something I'm not very good at. Since I am not on popular (among my friends/family) networks – FB and Whatsapp – I don't even see random updates from people, to get some sense of being in touch.

I recently read some old posts by Sacha Chua and was inspired by how much code she had for contact management in her old blog posts. I was inspired by this post in particular to try and be more meticulous about how I stay in touch with people. Michael Fogleman blogged about his contact management work-flow using keepintouch. It seemed to do most of what I wanted, but I wanted this to be integrated with my org-contacts-db and I felt having native elisp code would make it easier to hook up email, chat, etc. to this.

I ended up writing a small utility called howdy to help me keep in touch with people. It currently has only a couple of features:

  • M-x howdy lets me update the last contacted timestamp for a contact.
  • Shows me contacts that I'm out of touch in the agenda, once I add the following snippet to an agenda file.
    * Howdy
      %%(howdy-howdy)
    

I also have a few hooks to hook up jabber messages and email to update the db. I've added them to howdy-hooks.el in case anybody else wants to use them. They can also be used as examples to write other hooks. Feel free to contribute other hooks or suggest improvements. The library also ships with a modest test suite, that will hopefully make it easier for others to contribute.

I'm looking forward to experimenting with this over the next few weeks and improving it. Hopefully, it'll help me keep in touch, better than I do now.

WiFi Channels & my router's random invisibility

To update some data plan changes, I had to restart my router today. I did so from the (web) admin interface, from my dad's laptop. Guess what! The router goes invisible (to the laptop), after the restart! Other devices were still happily connected.

There were a few occasions in the past when this laptop had failed to find our home router. The solution that worked for my dad before was to restart the router; once, or may be a couple of times. But, this time it didn't work.

I had a feeling this was something to do with "Channels", but I had no idea what was going on.

I logged onto the router and started looking at the Wireless settings in the hope of finding some clue, and there it was! Wireless Channel: Auto Scan

auto-scan.jpg

It was anybody's guess that some channels that the router uses weren't supported by my dad's laptop. The restarts sometimes helped because the router would scan for channels that it thinks would work well, and end-up using something that the laptop's wifi card supported.

The router only had options of channels from 1 to 13. To see what channels a wifi card supports, you can run the iwlist freq command.

iwlist freq
eth1      26 channels in total; available frequencies :
	  Channel 01 : 2.412 GHz
	  Channel 02 : 2.417 GHz
	  Channel 03 : 2.422 GHz
	  Channel 04 : 2.427 GHz
	  Channel 05 : 2.432 GHz
	  Channel 06 : 2.437 GHz
	  Channel 07 : 2.442 GHz
	  Channel 08 : 2.447 GHz
	  Channel 09 : 2.452 GHz
	  Channel 10 : 2.457 GHz
	  Channel 11 : 2.462 GHz
	  Channel 12 : 2.467 GHz
	  Channel 13 : 2.472 GHz
	  Channel 14 : 2.484 GHz
	  Channel 36 : 5.18 GHz
	  Channel 38 : 5.19 GHz
	  Channel 40 : 5.2 GHz
	  Channel 42 : 5.21 GHz
	  Channel 44 : 5.22 GHz
	  Channel 46 : 5.23 GHz
	  Channel 48 : 5.24 GHz
	  Channel 149 : 5.745 GHz
	  Channel 153 : 5.765 GHz
	  Channel 157 : 5.785 GHz
	  Channel 161 : 5.805 GHz
	  Channel 165 : 5.825 GHz

So, the laptop's wifi card support all the channels that the router could use. iwlist freq also gives the current Channel if the device is already connected to a router. I ran this on my laptop and found that the router was using Channel 13. Why the hell wouldn't this work?! I don't know!

I decided to set a channel manually, but how do I pick a channel? The idea of "Auto Scan" is to find a channel that has relatively low interference. How do we do this manually? sudo iwlist scan scans for all the WiFi access points nearby and gives detailed information about them, which includes the Channel.

It looked like most of the routers nearby used channels 1 and 6, with a few 3s and 4s. Assuming, most of the other routers aren't doing this "Auto Scan" business, I set my router to always use Channel 9, and the laptop seems to be happily connected, now!

Playing music using mpsyt from Emacs

I've started using the wonderful mpsyt to play any music from youtube, since I'm not really interested in the video. But, since I use emacs for chat/IRC, I end up getting youtube links into emacs and opening them opens them up in my browser. I ended up writing some elisp to play the songs from within an instance of mpsyt running inside an emacs buffer.

(defun pc/short-url-at-point ()
  "Gets the short url at point.

This function is required only because
`thing-at-point-url-at-point' ignores urls (without a scheme)
that don't start with www."
  (let ((bounds (thing-at-point-bounds-of-url-at-point t)))
    (when (and bounds (< (car bounds) (cdr bounds)))
      (buffer-substring-no-properties (car bounds) (cdr bounds)))))

(defun pc/mpsyt-url (url)
  (let ((buffer (current-buffer))
	(mpsyt-proc-name "*mpsyt*"))

    ;; Start a new term with *mpsyt* if there isn't one
    (unless (get-process mpsyt-proc-name)
      (when (get-buffer mpsyt-proc-name)
	(kill-buffer (get-buffer mpsyt-proc-name)))
      (ansi-term "mpsyt" "mpsyt"))

    ;; Play given url in mpsyt
    (let ((mpsyt-proc (get-process mpsyt-proc-name)))
      ;; If something is already playing, stop it and play this...
      (term-send-string mpsyt-proc "\\n\\n\\n")
      ;; We wait for a bit, since looking for the prompt seems to fail, sometimes?
      (sleep-for 1)
      (term-send-string mpsyt-proc "\\n")

      ;; Actually send the command to playurl
      (term-simple-send (get-process mpsyt-proc-name)
			(format "playurl %s" url)))

    (switch-to-buffer buffer)))

(defun pc/mpsyt-url-at-point ()
  "Play the URL at point using mpsyt."
  (interactive)
  (let ((url (or (url-get-url-at-point) (pc/short-url-at-point))))
    (if (not url)
      (message "No URL found")
	(message (format "Playing %s with mpsyt" url))
      (pc/mpsyt-url url))))

The current version of mpsyt crashes when run from inside emacs due to a bug in the code to get the terminal size, which should be fixed once this patch is merged.

I would've expected thing-at-point-url-at-point to be able to find urls even when they don't have a schema, but it tries to guess the schema from urls and fails to work when the url starts with youtube.com instead of www.youtube.com.

I started off using the command-line interface of mpsyt by running it using shell-command or start-process. But, it seemed useful to have a buffer of mpsyt to switch to – easier to search for new music, repeating songs, etc. Not all tasks/actions are achievable through mpsyt's command line args.

I ended up writing more code than I thought I would have to1. But, I'm pretty happy with how this all works, right now.

Footnotes:

1

- Isn't it true, more often than not?

org-drill for making it stick!

Those who read the last few posts here, would know that I have been experimenting with org-drill (a spaced repetition extension to Org mode). I have been using the system (almost) religiously for the past 2 months, and I do find that it has helped a great deal! (in some respects). I have also spent a considerable amount of time trying to reduce the friction to put new stuff into the system, and am constantly on the look out for further improvements.

Using this system has definitely helped with retention, and I find that I can recall quite a few things I have read a few weeks ago, that I would normally have been unable to. Though, I can recall a lot of information, I have been having a feeling of "fragmentation": the feeling of just retaining individual bits/fragments of information, while losing out on actually internalizing the knowledge; not seeing the big picture, etc.

Wozniak (the author of super-memo) warns against learning without understanding, and memorizing before actually learning stuff. I haven't consciously added stuff into the system that I didn't understand (when I added it), but, later it does feel like I have lost some connections or the understanding, and am only holding onto the fragments of information.

The problems as explained in (read: as interpreted by me from) Make it Stick appear to be:

  1. The understanding (if any) at the time of adding stuff into the spaced-repetition system is untested. It may just be familiarity masquerading as understanding.
  2. The lack of any spaced repetitions for the overall concept/understanding and actual repetitions only for individual bits doesn't help retention of the understanding (even if there was any, in the first place).

To work around this, I'm going to try adding questions that test understanding, to the system. The Super-memo team strongly recommends keeping the drill items small and easy to answer. This may be helpful in keeping each drill session short, but I would really like to add conceptual questions to the system, and see how it goes. I hacked org-drill to allow me to type out answers, before looking at the "correct" ones. This is an adaptation of a system that a fellow Hacker Schooler uses, and shared. Also, hopefully forcing myself to type out the answer will help me get around the problem of sometimes saying "yeah I know that", then looking at the answer only to reaffirm the feeling of familiarity, rather than actually testing myself. I'm still going to continue adding quick and short questions that test "bits of information", though. But, hopefully the additional conceptual questions are going to tie things together and help fill in the gaps. Lets see how this goes!

For those interested, my hacks to org-drill below. The code is really a hack, and welcome any suggestions on cleaning up the code.

(advice-add 'org-drill-presentation-prompt :around 'pc/org-drill-presentation-prompt)

(defun pc/org-drill-presentation-prompt (old-fun &rest fmt-and-args)
  "A presentation prompt that allows capturing answers."

  (let ((cb (current-buffer))
	(heading (nth 4 (org-heading-components)))
	(entry-id (org-entry-get (point) "ID"))
	(input ""))
    (switch-to-buffer-other-window "*org-capture-drill-answer*")
    (org-mode)
    (insert "# Hit C-c C-c once you are done answering!\\n")
    (org-insert-heading-respect-content)
    (insert (format "Answer: %s" heading))
    (org-entry-put (point) "QUESTION_ID" entry-id)
    (goto-char (point-max))
    (insert "  ")
    (org-time-stamp-inactive '(16))
    (insert "\\n\\n  ")
    (while (not (and input (equal input "")))
      (ignore-errors
	(execute-kbd-macro input))
      (setq input (read-key-sequence nil)))
    (switch-to-buffer-other-window cb)
    (apply old-fun fmt-and-args)))

(advice-add 'org-drill-reschedule :around 'pc/org-drill-reschedule)

(defun pc/org-drill-reschedule (old-fun)
  "Calls the original reschedule, but also archives the answer"
  (prog1 (funcall old-fun)
    (let ((cb (current-buffer)))
      (switch-to-buffer-other-window "*org-capture-drill-answer*")
      (pc/org-refile-to-datetree "drill.org_archive")
      (message (buffer-name))
      (switch-to-buffer-other-window cb)
      (kill-buffer "*org-capture-drill-answer*"))))

(require 'org-datetree)
(defun pc/org-refile-to-datetree (journal)
  "Refile an entry to journal file's date-tree"
  (interactive "fRefile to: ")
  (let* ((journal (expand-file-name journal org-directory))
	 (date-string (or (org-entry-get (point) "TIMESTAMP_IA")
			  (org-entry-get (point) "TIMESTAMP")))
	 (dct (decode-time (or (and date-string (org-time-string-to-time date-string))
			       (current-time))))
	 (date (list (nth 4 dct) (nth 3 dct) (nth 5 dct))))
    (org-cut-subtree)
    (with-current-buffer (or (find-buffer-visiting journal)
			     (find-file-noselect journal))
      (org-mode)
      (save-excursion
	(org-datetree-file-entry-under (current-kill 0) date)
	(bookmark-set "org-refile-last-stored")))
    (message "Refiled to %s" journal)))

How I learnt to use Emacs' profiler

I learnt to use Emacs' profiler yesterday, after many hours of yak-shaving, trying to get Memacs working. Memacs is a memory extension system for Emacs written by Karl Voit, that I have been meaning to try out for a long time now. Seeing lots of review posts at the turn of the year and watching Karl's recent Emacs Chat with Sacha Chua pushed me to try and finally set it up.

I started writing a module to create a Memacs file – an org archive file – from my browser history. It was pretty easy to write, and I had it spitting out a huge file with 22k entries after about a couple of hours of work. Then I excitedly pulled up my agenda, and turned on the option to view archived entries, only to be super-disappointed. It turned out to be extremely slow! Actually, the agenda never came up with the 22k entries file that I had. At least not in 5 or so minutes, before I got impatient. The performance was unacceptable even when I reduced it to 5k entries.

I was pretty sure it wasn't that slow for Karl in his demo and tweeted to him, asking for a workaround. Meanwhile, I looked at his dot-emacs, but wasn't able to dig out what was needed to speed up things. He confirmed that his performance was way better than what I was getting.

First, I ruled out the possibility of it being because of the SSD, since clearly my CPU usage was peaking, and the task was CPU bound and not I/O. Next, I tried using the same file on a different machine (with a different version of Emacs and org-mode), and it worked blazingly fast. So, it was either the version of Emacs or org-mode that I was using.

I should have stopped, thought clearly, and started experimenting with org version, but hindsight is 20-20. I tried Ubuntu's pre-built Emacs and agendas were fast! I suspected my Emacs build, since I recently started building Emacs from git. I built two or three other versions of Emacs, and wasted a lot of time, before realizing that I wasn't using the org-mode source bundled inside Emacs for the tests, and there were two "independent" variables.

Finally, I began bisecting org-mode's source and found that all hell broke loose with an inconspicuous change around release 8.2.6. It turns out that org-overview was broken before this, and collapsing all the trees in a newly opened org-buffer (default option) wasn't working. Once this bug was fixed, opening huge org files would slow down by a great deal, in turn causing agenda generation to be unbearably slow.

All I had to do was add a #+STARTUP: showeverything to the top of the file. This speeded up things by about 50 times! It turns out, I later found out, that all of this is documented on Worg. I did try a few search engine queries, but sadly none of them brought this up. Adding the following to my config, speeded up agenda generation by about 150-200 times!

(setq org-agenda-inhibit-startup t) ;; ~50x speedup
(setq org-agenda-use-tag-inheritance nil) ;; 3-4x speedup

In the course of all this debugging, I learnt how to use Emacs' profiler. The profile reports along with git bisect, eventually helped me figure out what the problem was.

To profile the CPU usage, all you have to do is add a call like

(profiler-start 'cpu)  ;; or M-x profiler-start

at the place where you wish to start it. Emacs will then start collecting information about where time is being spent, by sampling every sampling-interval seconds (default 10^6 nanoseconds = 1 milli second).

You can view the information being collected, at any point of time using

(profiler-report) ;; or M-x profiler-report

The report is a nice, interactive tree with the percentage of time spent in each call. You can stop profiling by calling (profiler-stop). If you have more than one report, you can compare them by hitting = in one of the report buffers. I'm definitely going to use this for other things! (like speeding up my startup?)

Now that I have Memacs working with reasonably fast agenda views, I'm looking forward to collecting as much personal information as I can! Thanks Karl for writing Memacs. I am going to be a pretty heavy user, I think! There seem to be a few rough edges, though, and I hope to help smoothen them out a little bit, over the next few weeks.

Jabber message queue

I've always wanted to be able to queue up messages to send to friends, until I go online the next time. I tried using email instead of chat a few times, or just ended up staying online with a busy status.

Finally, now that I have started using jabber-mode for chatting from within Emacs, I took out the time to write a "queuing system" for sending chat messages, similar to the mail queue for smtpmail. Instead of persisting sexps, though, I persist the messages in a JSON format and the queue is flushed every time I connect to jabber, in a jabber-post-connect-hook.

To make the interface as similar to the interface available when I am online, I hacked completion for the to ID using email addresses in my address book (mu4e~contact-list). I really like the fact that the chat buffer opens up, and I can type and send messages like I usually do. Hitting RET after typing a message queues it up, instead of trying to send it. Smooth!

The code is in my .emacs

Using tmux to "screen share"

I wanted to pair with a friend of mine, exploring Magit mode in Emacs. There are a couple of projects to make it easy to use tmux(-like) to simplify this.

  • pairing lets you share a screen and pair, on a common server where both the users have access to. But I wanted to use my machine, instead of the server.
  • tmate gets around the restriction of having a common server where both users have access, but needs a custom install of tmux, and the use of a 3rd party service.

I worked around this, using an ssh reverse tunnels. Here are the steps, for anyone who'd like to reproduce.

  1. Add your partner(-in-crime)'s ssh key to authorized keys on a server that has a public IP (example.com, let's say).
  2. Add your server's key to authorized keys on your local machine.
  3. Create a reverse tunnel from your machine to the server.
    ssh -fNR 19999:localhost:22 server_user@example.com
    

    -R 19999 essentially forwards port 19999 on example.com to localhost's 22. -fN is to say no terminal, send connection to background.

  4. Your partner first ssh's to example.com and then ssh's to your laptop.

    On your partner's machine

    ssh server_user@example.com
    

    On the server, your partner does

    ssh your_username@localhost -p 19999
    

    Your partner is on your machine, and can run rm -rf! But, hopefully they'll only run tmux attach. Obviously, do this only with people you trust!

  5. Start a tmux session locally.
  6. Profit!

erc-notifications when Emacs not in focus

I have been trying to get ERC working with notifications. Julien Danjou's wonderful notifications module for ERC is great, but it is annoying to get notifications even when Emacs is in focus.

I had looked at circe-notifications, which has the feature but uses xdotool and xprop to do it. I was looking for something simpler, though… and it suddenly struck me that I have an auto-save hook in Emacs that is run when I focus out of it. I wondered if I could disable and enable notifications on focus, and it worked.

In case it is useful for somebody else -

(add-to-list 'erc-modules 'notifications)
(erc-notifications-mode)
(add-hook 'focus-out-hook 'erc-notifications-enable)
(add-hook 'focus-in-hook 'erc-notifications-disable)

I wonder if there are some corner cases where this doesn't work, and that's why the author of circe-notifications chose the tools that he did.

More input sources for org-drill

I've been trying to use org-drill regularly for the last few weeks. I don't know how well it's been going but I have been sticking to the routine religiously. I haven't yet really tried out incremental reading, but in an attempt to make it as easy as possible, I wanted to have a pdf-reader integration, and some kind of integration with Kindle highlights. Browser integration is pretty straight-forward, thanks to some simple java-script.

I looked for a pdf-reader with some sort of plugin support, but I found nothing in Evince or Okular. I thought about pdfjs but it seemed slightly clunky to open pdfs in a browser, though I might shift to this if I don't like what I finally ended up with. Good old xpdf seemed to be the only pdf reader that had some support for custom keybindings that allowed users to run external commands. With a little Python, I was able to setup a work-flow to capture snippets from xpdf, to add to org-drill. Custom key-bindings somehow don't seem to work on xpdf bundled on Ubuntu. So, I ended up downloading and using the binary available on the xpdf site.

For Kindle highlights support, with minor updates to Thamer Mahmoud's clip2org, I have a simple way of getting all the "new" clippings/highlights as org-drill headlines. I haven't really started using this, and once I do, I may end-up automating even the merging of these items into the org-drill notes file. I'm looking forward to making better use of my Kindle, with this feature!

I don't know if it would be useful to have more context information like section titles/chapter titles when capturing from html/pdf, but it seems like an interesting problem to try to solve.

Also, it might be easier(?) if I probably tried to have a DE level keybinding, and some code to get selection and file name of the currently active window/application.

Bookmarks [2014-11-01]

  • 50 Ways To Love Someone | Thought Catalog

    Go for walks together as much as you’re able.

  • Foundations of Data Science

    These notes are a first draft of a book being written by Hopcroft and Kannan and in many places are incomplete. However, the notes are in good enough shape to prepare lectures for a modern theoretical course in computer science. Please do not put solutions to exercises online as it is important for students to work out solutions for themselves rather than copy them from the internet.

  • Asyncio

    Curated content from around the net to get started with asyncio

  • s16h/py-must-watch · GitHub

    Must-watch videos about Python.

  • josephmisiti/awesome-machine-learning · GitHub

    A curated list of awesome machine learning frameworks, libraries and software (by language).

  • About - Mastering Emacs

    The blog will cover all facets of Emacs and will be suitable for beginners and – I hope – experts alike.