Create a Public Jupyter Server, quickly!

I create public Jupyter notebooks once in a while, to collaborate with a friend, or to make it easier for myself to work with data on a remote machine.

Each time I need to look up the docs, and manually set-up a few things, before I can start using the notebook.

I just wrote a bash script that does the following, quickly -

  • Install Jupyter into a temporary virtualenv
  • Create certificate files
  • Start the server with https enabled and a password
  • Delete all temporary files, when the server is shutdown

You can get it here, if you'd like to use it.

Python reload and module dict

I was trying to play around with Nikola's code today and learnt about a documented weirdness of Python's reload.

# Work in a temporary directory
import os
import tempfile
os.chdir(tempfile.mkdtemp())

# Module content, original and updated
ORIGINAL = "# PLUGINS = []"
UPDATED = "PLUGINS = ['rss']"

def create_conf_file(content):
    """Create a conf.py module with given content."""
    with open('conf.py', 'w') as f:
	f.write(content)
create_conf_file(ORIGINAL)
import conf
# PLUGINS is not defined in the module, originally.
print(conf.PLUGINS)
AttributeError: module 'conf' has no attribute 'PLUGINS'
create_conf_file(UPDATED)
import conf
print(conf.PLUGINS)
AttributeError: module 'conf' has no attribute 'PLUGINS'

conf doesn't yet have a PLUGINS attribute, since the new module isn't imported until we reload.

import importlib
importlib.reload(conf)
print(conf.PLUGINS)
['rss']
# We write back the original file. PLUGINS should be empty!
create_conf_file(ORIGINAL)
import conf
importlib.reload(conf)
print(conf.PLUGINS, "<---Whaaaat!")
['rss'] <---Whaaaat!

The behavior is well documented, along with a reasoning of why it is the way it is, but you can trip over it if you don't know. I hit a bug and was wondering if there was a race condition somewhere, until I read the docs for reload. On reload, the module dict is updated, instead of creating a new dict. Any values not redefined in the new code for the module remain unchanged.

Playing music using mpsyt from Emacs

I've started using the wonderful mpsyt to play any music from youtube, since I'm not really interested in the video. But, since I use emacs for chat/IRC, I end up getting youtube links into emacs and opening them opens them up in my browser. I ended up writing some elisp to play the songs from within an instance of mpsyt running inside an emacs buffer.

(defun pc/short-url-at-point ()
  "Gets the short url at point.

This function is required only because
`thing-at-point-url-at-point' ignores urls (without a scheme)
that don't start with www."
  (let ((bounds (thing-at-point-bounds-of-url-at-point t)))
    (when (and bounds (< (car bounds) (cdr bounds)))
      (buffer-substring-no-properties (car bounds) (cdr bounds)))))

(defun pc/mpsyt-url (url)
  (let ((buffer (current-buffer))
	(mpsyt-proc-name "*mpsyt*"))

    ;; Start a new term with *mpsyt* if there isn't one
    (unless (get-process mpsyt-proc-name)
      (when (get-buffer mpsyt-proc-name)
	(kill-buffer (get-buffer mpsyt-proc-name)))
      (ansi-term "mpsyt" "mpsyt"))

    ;; Play given url in mpsyt
    (let ((mpsyt-proc (get-process mpsyt-proc-name)))
      ;; If something is already playing, stop it and play this...
      (term-send-string mpsyt-proc "\\n\\n\\n")
      ;; We wait for a bit, since looking for the prompt seems to fail, sometimes?
      (sleep-for 1)
      (term-send-string mpsyt-proc "\\n")

      ;; Actually send the command to playurl
      (term-simple-send (get-process mpsyt-proc-name)
			(format "playurl %s" url)))

    (switch-to-buffer buffer)))

(defun pc/mpsyt-url-at-point ()
  "Play the URL at point using mpsyt."
  (interactive)
  (let ((url (or (url-get-url-at-point) (pc/short-url-at-point))))
    (if (not url)
      (message "No URL found")
	(message (format "Playing %s with mpsyt" url))
      (pc/mpsyt-url url))))

The current version of mpsyt crashes when run from inside emacs due to a bug in the code to get the terminal size, which should be fixed once this patch is merged.

I would've expected thing-at-point-url-at-point to be able to find urls even when they don't have a schema, but it tries to guess the schema from urls and fails to work when the url starts with youtube.com instead of www.youtube.com.

I started off using the command-line interface of mpsyt by running it using shell-command or start-process. But, it seemed useful to have a buffer of mpsyt to switch to – easier to search for new music, repeating songs, etc. Not all tasks/actions are achievable through mpsyt's command line args.

I ended up writing more code than I thought I would have to1. But, I'm pretty happy with how this all works, right now.

Footnotes:

1

- Isn't it true, more often than not?

Recurse Center, 2014-08-27

  • I spent the whole of yesterday just working on the LED-bot.
  • Spent some time cleaning up meta-stuff about the project, making it a real package, adding a setup.py, added LICENSE & AUTHOR files, etc.
  • I found it really frustrating that there's no good/standard way to remove duplication of information between pip's requirements.txt and install_requires
  • We got the bot running on the Beagle Bone, but it turned out to be too slow. After some playing around with trying to get rid of Python for-loops, I was able to use PIL's image.tobytes and some image cropping to get the data to be sent to the LEDs, and it sped up the display quite a bit.

Recurse Center, 2014-08-26

  • I spent most of the day refactoring the code for the LED Bot, and am happy with the way it looks right now.
  • My talk at Hack and Tell went OK. I was distracted by live_reveal not displaying properly on the smaller resolution, and not being able to mirror screens!
  • There were a bunch of interesting talks presented by the others. http://comparea.org, Taxis and Rainbows were the most interesting ones for me.

Recurse Center, 2014-08-20

  • I finished going through the third of Dave Beazley's tutorials on Generators and Co-routines.
  • Also refactored api-diff to use astroid, instead of the ast module. Kinda happy with what I have, for now. May be more, later…

Recurse Center, 2014-08-21

  • I added an additional view to xtab, to see all the pages that were killed, and reopen them, if you so choose.
  • Also, cleaned up the api-diff project and renamed it to what-changed.
  • Started thinking about building the GitHub organization dashboard, that I thought would be interesting to have, since the 1st or 2nd week of Hacker School.
  • A bunch of awesome thursday presentations, and job fair and awesome food by Chef Warren!

Recurse Center, 2014-08-19

  • I spent most of the day working through David Beazly's tutorials on generators and coroutines, and it was mind bending.
  • I learnt about send and throw methods on generators! I hadn't heard of them before, and was totally surprised to come across them.
  • I spent a little time late in the day getting class diffs working, and refactoring the api-diff tool to remove duplication of code between classes and modules. But, not very happy with the way I did it.

Recurse Center, 2014-08-18

  • I refactored the API-diff code, and am pretty happy with what I have right now, and need to add support for diffing Classes. I'm considering using Atstroid instead of Python's ast module, since some of the work that I want to do, has already been done for me.
  • With the white-boarding group, we worked on a set of simple exercises for strings and arrays.
  • José Valim's presentation on Elixir was interesting and got me reading up a stuff about concurrency, and related stuff. Thanks!

Recurse Center, 2014-08-17

Friday

  • Updated my HS profile page with projects that I worked on, and a short bio.
  • Joined the group discussing algorithmic complexity
  • Submitted a pull request to blaggregator to fix issues with broken profile images.

Saturday & Sunday

I almost didn't write any code during the weekend.