The old UNIX tools are still on top!

I'm sorry. In some respects, I am probably an old fart for saying this, but programs that I use regularly, I need to be efficient. No, I don't need a lot of bells and whistles, and yes, I would not mind reading a manual or two to get into it. It seems that his way of thinking is not too popular right now: even programmers are using bloated tools and programming languages that almost seem to be desinged in order to minimize efficiency and maximize memory usage*

An excellent example of this is the Office Suite. How come we now need quite potent 3D capable computers to do word processing? Why do I need to see values being updated when I process numbers? How come starting the tool I should be using the most takes forever on my mac when more complicated tools do not? Just very odd this, very odd.

In my work, the frustration of having tools that are full of bling and empty of punch have brought me back to the old tools. And, I am now soo happy again :-) By the old style tools, I mean the Emacs editor, and the production packages that have been developed for it : ESS (Emacs Speaks Statistics), AUCTeX, RefTex and Sweave (for R). Just a fantastic combination!

Using these tools, you can easilly create an (in my humble opinion) unbeatable work flow :

1) Please do your stastistics in R. Easy and powerful language that is for statistics what Matlab was (is?) for engineering and signal processing. Lots of manuals around, and an archive of posts to the mailing list where you could get the answer to most of your questions without even posing them :-).
2) Paste your R code into code block in a Sweave document in Emacs. Add any informative text concerning the outcome of your code arounde the block, and Presto! You are on your way into writing your article. You'll get lots of helpful menues for both the code blocks and (LaTeX) documentation parts form AUCTeX and the RefTeX packages, depending on what you decide to edit in the document, and you even get a nice reference management system (BibTeX) included at no cost. RefTex gives you really smooth (although not overly graphical) and efficient management of within-document references and citations in different styles. Works with Mendeley too, by the way, which is something Word is having a difficult time managing....
3) Start ESS in another Emacs window. Now you will have the ability to do test runs of a code block or just a highlighted region of the code directly in an R session.
4) Run the Sweave document through the R Sweave command (just use the menu options) and you'll get a LaTeX document with the results of the R code as tables, figures, text portions or just numbers. Find an error inte the data set? Just Sweave and LaTeX it again, and the report will be updated without you having to fiddle with image placement.
5) Keep the source file under revision control (Git, Subversion, CVS... whatever you choose). If you then decide make major code adjustments, you can easilly to back and forth inbetween the versions to make sure that everything still work ok. If someone asks you why a figure on page 4 now don't show exactly the same data as it did in the version three weeks ago, you can tell him. If you suspect you made an error in your transformation of the data, you can just go to the code and see what you did. Whenever that was. Try doing that in point and click usage of SPSS ;-)
6) Save your ESS/R session as a transcript, at least one for each day. If you do, you can easilly re-run the entire day's transactions within R and reproduce the results. This is a quite handy feature to have when you just deleated a data set you started creating this morning, with no backup in between... You laugh? Well, I can tell you, it happened to me, and I was then very happy about having a transcript file ;-)

Yes, you need to be publishing in a journal that has a LaTeX style file, but most of the large publishing houses for journals provide one. And, seriously, if you don't have one and you need to submit your work in Word format, just produce a PDF version of your paper and copy-paste it into the Word template. It will still save you loads on frustration and anger management funds to get away from Word. Just trust me, take the plunge...


*Just my opinion. If you don't agree with me, please show me where I am wrong.