Best Practices for Scientific Computing

Mon, 24 Dec 2012 00:00:00 +0530

Shantanu and I gave a short talk titled “Software Carpentry for Scientists” for the graduate students of Chemical Engineering department, IISc, this Friday. We gave a short introduction to Git, TDD, Numpy/Scipy, etc and mentioned a few things from Greg Wilson et al’s paper.

I promised to revert to them with links to a few resources. I figured it would be more beneficial, if I just put it in a publicly available place.

A summary of the paper by Greg Wilson et. al., is below.

Useful resources

Software Carpentry

Paper by Greg Wilson et. al.
Software Carpentry

Git & version control

TDD

SciPy

http://scipy-lectures.github.com

Python

http://docs.python.org/tutorial

GUI tools in Python

Summary of paper by Greg Wilson et. al.

Write programs for people, not computers
- a program should not require its readers to hold more than a handful of facts in memory at once.
- names should be consistent, distinctive and meaningful.
- code style and formatting should be consistent.
- all aspects of software development should be broken down into tasks roughly an hour long
Automate repetitive tasks
- rely on the computer to repeat tasks
- save recent commands in a file for re-use
- use a build to automate scientific work-flows
Use the computer to record history
- software tools should be used to track computational work automatically.
Make incremental changes
- work in small steps with frequent feedback and course correction
Use version control
- use a version control system
- everything that has been created manually should be put in version control
Don’t repeat yourself (or others)
- every piece of data must have a single authoritative representation in the system
- code should be modularized rather than copied and pasted
- re-use code instead of rewriting it
Plan for mistakes
- add assertions to programs to check their operation
- use an off-the-shelf unit testing library
- use all available oracles when testing programs
- turn bugs into test cases
- use a symbolic debugger
Optimize software only after it works correctly
- use a profiler to identify bottlenecks
- write code in the highest-level language possible
Document design and purpose, not mechanics
- document interfaces and reasons, not implementations
- refactor code instead of explaining how it works
- embed the documentation for a piece of software in that software
Collaborate
- use pre-merge code reviews
- use pair programming when bringing someone new up to speed and when tackling particularly tricky problems
- use an issue tracking tool

Awe[ful] @ Science !!?!??!

Thu, 21 Dec 2006 00:00:00 +0530

I was at the B.M Birla Science Musuem yesterday… it was calm with no one in there.. the only people i could see were the sweepers and cleaners.. the place was dull and lifeless… I dint like it.. but slowly.. as the afternoon progressed.. people started coming in and by 5 in the evening.. there were quite a few in. The museum seemed to have sprung to life suddenly… there was a lot of action all around with people enjoying in ( and trying to understand) the wonderful world of science….

A lot of kids had come along with their parents… to dive into the world of science… they were enjoying it… the parents also turned into kids and were having great fun… it was a great sight… the parents tried their best to demistify the happenings… those with a science background did it from thier knowledge.. the others patiently went through the explanations…

The kids were thoroughly enjoying it… I don’t know if they would later on in life remember what they learnt now… but definitely a love for science and physics will be ignited in their hearts, it will fill their consciousness… the spirit of curiosity will have multiplied manifold… even the elders..connected with the child somewhere deep within themselves… they too enjoyed the wonders of the world of science…[and ofcourse the company and thrill in the eyes of their kids..] I too enjoyed the experience… the experiments and the thrill of the others brought a smile to my face too…. but it set me thinking…

If science can inspire so much awe.. what’s happening at our engineering colleges? When people have the chance to learn a whole lot of science and put it to practical use… why isn’t it that they are excited about it? Why is it that people just endup mugging a few formulae just before exams and get over with it? Where’s the problem? Where is that curiosity, all of us had as a child, lost?

I can recollect, in the last sem while doing experiments in the MT lab there wasn’t even a tenth of the awe or joy… that i could see and feel at the museum… why?? Is it that the child within us.. [the future engineers] dead? { If that’s true.. then the country is in deep trouble..} But I don’t think that’s true… I enjoyed my visit to the museum totally…and I am sure every other ’to be engineer’ would have too.. then where’s the trouble?

I don’t know exactly… but I wonder if its because everything is equated to marks and grades… all that matters to an engineering student [and the world around him/her..] is his/her CG or %age… Is this what the country needs? Were the ’temples of Learning’- the IIT’s and a whole lot of other technical institutions set-up for this very pupose? Of what use are those engineers to the country, who aren’t in awe of science and engineering, whose spirit of curiosity has been overpowered and suppressed by the present education system?

Isn’t it time we have a closer look at our education system… why are we still sticking to the ‘British sytem’ introduced during a colonial rule.. to break the back bone of our nation… to supress our ability to think rationally.. to produce third rate clerks to work for them at lower wages? Why is the emphasis on marks and grades growing exponentially? Shouldn’t we start working to cut this growth at its root? Is this sytem good enough to produce “real” engineers, doctors, lawyers, journos, CA’s, architects, designers, writers, artists etc etc..

Where have all the gurukuls of the ancient past and the Santiniketans of the recent past gone? Isn’t the country in desparate need of more of these ’temples of learning’??

The highest education is that which does not merely give us information but makes our life in harmony with all existence. – Rabindranath Tagore

Science is a wonderful thing if one does not have to earn one’s living at it –Albert Einstein

Science on Noetic Nought