Five tips for five years

I saw some wonderful tweets the other day where the author gave their former naive self some time-travelling advice from their present self about how best to get started with programming. I did intend to keep the link but sadly my filing system has failed and it’s fallen through the cracks. I’m grateful to the author, in any case, for inspiring this post, written on the fifth anniversary of my own initial foray into the world of programming. I will give myself five pieces of advice for five years, also because five is a nice round-ish number.

1. It may sound obvious, but it’s worth saying and it’s probably the most important of the points in this post- write code. Write code to solve problems you face in your work. Write code to find out why the code that you’re writing doesn’t work. Write code to make sure that the code that you’re writing gives you the right answer. Write code for fun. Write code to see if you can do something with code instead of by hand. Write code to write “Hey, look what I did” blog posts. Write code to produce minimal examples so you can get help on forums. Writing code is an art (we’ll come back to the points in this post that caution against churning our code that “just works”), and, like all arts, you need to log the hours. So log the hours.

2. Following on from this, and as espoused by the pre-eminent stats guru, Hadley Wickham, never be ashamed of your code. The code I wrote five years ago is laughably bad. At a glance it’s quite obvious I had no idea what I was doing. Many of the functions are the wrong functions. It’s by turns ridiculously verbose and frustratingly taciturn. It’s badly commented and baffling to me, never mind about any other poor individual who might have to maintain or change it. But you learn by experience and coming back to all this dreadful code taught me WHY it’s important to write legible, well commented code and WHY it’s important to refactor code and WHY it’s important to use the paradigms in the language you’re writing in (functions, objects…).

3. At the same time as you’re writing code that solves problems, having fun, and learning how to make code that works, listen to the experts. You won’t understand it all yet but you need to file it away for later. There are many brilliant people on the internet sharing their knowledge with anybody who cares to listen and you need to listen to each and explore for yourself what they’re saying. In my time I’ve run down many dead-ends (or, I should say, things that are dead-ends for me now, with my level of skill) but have learned a great deal when I crashed into the wall at the end. Emacs is the best text editor in the world? Great! Wham! No, Vim is the best editor. Great! Wham! You must use MVC for PHP/ MySQL applications? I’ll use CakePHP! Wham!

CakePHP was probably the brick wall that I hit the hardest. I didn’t understand it at all. Now I’m starting to understand the principles, and the why, but haven’t quite got to the how.

4. While it’s good to just roam the internet, writing terrible code, reading advice from experts, and generally just having fun, there is definitely value in doing a paid course where you get help with your own code from an expert. I’ve taken two courses now, one with the Open University, Introduction to object oriented programming with Java, and one with O’Reilly, Web security with PHP and MySQL. Having a real person read, assess, and improve my actual own real code not only taught me a lot but also gave me discipline to write code that could be understood and easily criticised by somebody other than me. This is also a good way to dip your toes into different waters and start to understand the strengths and weaknesses of different languages and approaches. I have learnt Java and Python (on https://www.coursera.org/) and although I haven’t yet used them for anything serious it’s helped me to understand the situations in which you would use them for something serious and why.

5. This only applies to certain types of programmer, but it certainly applied to me. Run your own server. They’re quite cheap to rent (or you could use an old computer in your own house) and you can do fun things with them like run WordPress (I self host this blog) and Shiny Server. Running a server will cause you a lot of headaches but will teach you a tremendous amount about your operating system as well as giving you an insight into the world of running applications and servers in real world applications. This only applies to people who want to write applications for the web, of course. Some programmers will spend their whole lives blissfully ignorant about such matters.

6. One extra one for fun. Use Linux. It’s fun. You can “take the back off” quite easily which makes it good for learning and the community is great. You’ll probably break and replace your operating system many times getting your graphics/ sound/ network card working and that’s more grist to the mill learning how your operating system works.

Don’t apologise- it’s just feedback

I’m cross posting this to my team’s blog which can be found here to bring together my two worlds of programming and Linux-geekery with the Involvement and experience in UK health services which I use programming and Linux-geekery for in my job. Here it is, a story from a meal out in a pub and the lessons the NHS might learn from it.

I witnessed a rather amusing event after a meal in a pub last week, but thinking about it the next day I thought perhaps there are some lessons in it.

A man approaches the bar after eating his meal, wishing to pay, and asks “Do you have a feedback form?”. It’s only a village pub, not the sort of place you’d really expect to have a feedback form. They look a little bit baffled and suggest he could email. Unperturbed, he rattles off three complaints, counting each off on his fingers as he goes.

They apologise, and he responds “You don’t have to apologise, it’s just feedback. But I don’t think I’ll be coming back”. And with that he walks off, presumably never to be seen again.

The incident stuck with me for a couple of reasons. Firstly, he clearly felt quite irate. His complaints, to be honest, were pretty valid and I basically agree with most of what he said. I had a pleasant evening myself so will be going back, but he was essentially spot on with his assessment of the experience. Despite being irate, however, he clearly had no desire for an apology, since he dismissed the offer of one.

Secondly, because he felt irate, the whole experience was quite uncomfortable for everyone. You could see the bar staff (all junior in age and seniority, the manager being inside the kitchen somewhere) were quite taken aback at his manner and didn’t know quite how to respond.

Thirdly, although he clearly wanted to feed back, and didn’t even seem necessarily to want to vent his spleen in person (given that he first asked for a feedback form), he wasn’t really interested in improving the venue for his own benefit (as regular customers might) since he vowed never to return.

So what can the NHS learn from all this? I think there are a few lessons.

Firstly, although feedback forms are often criticised for being impersonal, it’s clear that in this situation it would have helped the staff by removing them from this confrontational situation. The individual in question clearly didn’t want an apology, or compensation, or even to see improvements on their return, so feeding their very accurate impressions of the venue back on a form would have spared the staff the awkwardness of meeting this challenge face to face.

Secondly, it’s a good reminder that everybody has their own idiosyncratic relationship with feedback. Some people just want to have a rant at somebody and get it off their chest. Some people want to constructively engage and keep using a service and watch it improve. Some people just want an apology. Some people (as in this case) just have a good assessment to share and feel natural about sharing it on a form or face to face, and have no interest in apologies, or improvements, or anything else.

Thirdly, it just goes to show that good feedback can be found anywhere. It was quite a difficult situation and proved impossible to resolve face to face, with the poor old staff just looking uncomfortable, and the man disappeared into the night before they’d even replied at all. Nonetheless, he was spot on and I certainly hope that they do make all the changes which he suggested ready for my next visit.

Downloading Lucida Console on Ubuntu

Well, it’s been a horribly long time since I blogged anything. There’s another small chaos generator in the house and he and his older brother are keeping me pretty busy.

Anyway, this is a quick post to solve a problem which appears to have no Google Juice at all- downloading Lucida Console for Linux. If you want Lucida fonts generally there’s some stuff here, but at least when I tried it there was no Lucida Console. There are various bits of outdated and broken links on the subject, and I’m pretty sure one of the sites I was visiting tried to give me a Windows virus, so be careful out there, nonetheless I have found and installed it successfully.

The download that worked and didn’t try to infect my computer is here. I installed it following the instructions here (basically just pop the file in /usr/share/fonts/).

And is how you install Lucida Console on Ubuntu. Hopefully this will work its way up Google’s list and save somebody some head scratching (some of the links on the first page were from 2008, needless to say the links and instructions in them did not work).

Primum non nocere- gathering feedback data in the NHS

One of the bits of the internet where I hang out is alive with debate around this comment piece in The Guardian. I would advise you to click through to read it because I can’t give a good summary of how hard-hitting it is but, in summary, it describes a couple who had exemplary service at an A + E department when they suffered a miscarriage but were then very distressed when they received a follow up text the next day asking them to rate their experience of A + E using the NHS’s Friends and Family Test (how likely are you to recommend this service to others: Extremely likely… etc.).

The author imagines an individual involved in the gathering, analysing, and scrutiny of data gathered in this way as “…a faceless man in a suit pointing to a pie chart at a meeting of senior managers and telling his colleagues that customer satisfaction is at a record high”… and speculates that “He’ll probably get a pay rise for improving results”.

It’s fair to say that this piece had quite an emotional effect on me. My wife and I suffered a miscarriage before the birth of our first child and I can only describe it as like a hammer blow to the stomach. I would have gladly chosen the hammer blow instead because hammers to the stomach don’t multiply the pain they inflict on you and deal it to your spouse. And they certainly don’t inflict that same pain on you day after day and through each subsequent negative pregnancy test. So as I read my heart went completely to the couple and what they had been through.

I then found myself torn apart as the piece demolished the methods and purpose of gathering feedback in the NHS since I have dedicated myself to this very thing, teaching myself programming and markup languages, learning the principles of database design and really being consumed in thought pretty much round the clock as to how we can better gather, analyse, and present feedback data in order to improve services.

I don’t wish in any way, therefore, to argue with or criticise the piece. The texts should not have been sent and the pain they caused is very real. However, I would like to highlight the existence of many strands of work relating to patient feedback. Text surveys may well have their place for particular individuals at particular times (I see a hepatologist yearly, for example, I would respond very well to a follow up text after each appointment).

There are many dedicated people working with patient feedback data, not just faceless bureaucrats; I work with many people who, with me, truly believe in the power of stories (and, yes, tickbox data) to change services. Good and bad services can improve when they listen to their patients and, although the service this couple received was very good, feedback data can be important in improving poor services (or even setting off alarm bells in very poor services).

For those who are not convinced, please visit my Trust’s dedicated feedback website. We share all of our feedback transparently with the world over the internet and also post all the projects and changes that have come about because of patient feedback. The Friends and Family Test is one small part of the work that we do collecting data. Many in the health service feel it is a poor question or badly calibrated, and I would hate its shadow to be cast over the work that so many individuals at my Trust do, individuals who are passionate about improving services and the lives of the people who use them.

And if you’re still not convinced, tell me why and we’ll do it better.

I did it! I edited some open sourced code!

I do try to promote the benefits of open source wherever I go, and one of these many benefits is, of course, the ability of endusers or intermediate parties to modify the source code to make changes to the software. A couple of times I have been dismissed by members of the closed source contingent who mock the idea that individuals who work outside of the IT world (e.g. in healthcare, my own area) would sit down and modify or even read reams of source code.

I think this is a rather silly argument, but that needn’t detain us here. I am writing today merely to say that I have now finally, with my own two hands, modified some source code because the original program didn’t do what I want.

Admittedly, the program in question is part of an R package and the function I modified is really quite small and written in R (my strongest language by far) but still, the point stands. A statistic I wanted was being output in a very silly and annoying way. If I were using STATA or SPSS I could probably have fixed the problem but I would have had to capture the output as text and then munge it by hand, which would have taken absolutely ages. Instead I just fired up the source, fiddled with a few lines, reran the function, and I was finished.

It’s proof of concept for me, at least. May it be the first of many changes, and one day hopefully I can actually make myself useful and make a pull request.

All hail the mighty Cron- setting up cron jobs and a mail server to collect their output logs

At a certain point, everyone with a penchant for databases and Linux is going to want to make use of cron, whether it’s to back up their data, run maintenance on certain fields of a database, or something else (in my case, do a huge pull of MySQL tables and format ready to drop nicely into a Shiny application).

Cron is, quite simply, a little pixie who lives on your server and executes scripts at times you determine: every day at midnight, every Wednesday morning, every 6 hours on weekdays, you name it, cron does it.

The internet is, as ever, generous with its advice, but I’ve read many web pages on my journey to getting the whole thing set up correctly so I thought I would pull some of the links and the advice into one post to help out other newbies such as myself.

My cron job is an R script, and this page gives a nice intro to running R scripts over cron, and also includes this link which gives loads of examples of customising when you want the scripts to run.

This should get you nicely on your way, but the foible of cron I had problems with is that it sends output and error messages by email, not by writing into a log. So when your script doesn’t work (which mine didn’t) it’s hard to know why. The first thing to know is that you can at least make sure the process is actually running, because there is a log of events, it just doesn’t have any output in it. As I discovered here, you can run:


grep CRON /var/log/syslog

This will give you a list of cron activity, most likely with “(CRON) info (No MTA installed, discarding output)” in it (on Ubuntu, anyway). What this means is there is no mail server and so the output (with those lovely error messages) is just trashed.

“I’m not setting up a whole mail server just to read some error messages”, I thought to myself, however, as time wore on and it became apparent that I couldn’t debug the script without seeing the output I thought I would have a go. It’s actually very easy to set up a simple mail server for this purpose. There are various bits and pieces of advice here but in essence you just need to run:


sudo apt-get install postfix

In the configuration screen select “Local only”. In my case, that’s all I needed to do. Now to read your mail just


sudo apt-get install mailutils

And then run


mail

That’s it! I could now read the output and I found the error message, which came about because the working directory differed between running the script straight off the server myself and from a cron job. I don’t really know why, but I don’t care at this point, it works, that will do for today.

Querying Google analytics data with R

This post is in response to a question on Twitter, but it may be of interest to others. I produced a Shiny app for querying Google Analytics data for my book. The data on the live version which lives here is automatically pulled in via the API each time the application is run. That code cannot live on my GitHub because it needs the username and password for the API. So I’m reproducing it here. It’s very simple and is based on the wonderful R package rga which can be found here. Obviously the XXXXXXXs have got real things in the real version, this is the redacted version.

It’s pretty self explanatory really with the documentation for the package, so I won’t labour it by explaining any of the code. The only thing perhaps of note is the where=”ga.rga” instruction in the rga.open() call, this allows you to store a file called ga.rga which stores the authentication credentials of your application and means you don’t need to keep entering the code that Google gives you the first time you run it.

I’m happy to get questions as comments to this blog post or on my Twitter @chrisbeeley.


library(rga)

rga.open(instance = "ga", where="ga.rga", 
         client.id = "XXXXXX.apps.googleusercontent.com", 
         client.secret = "XXXXXX")

analytics = ga$getData(XXXXXX, batch = TRUE,
                       start.date = "2013-01-01",
                       metrics = "ga:visitors, ga:visits, ga:bounces, ga:timeOnSite",
                       dimensions = "ga:dateHour, ga:networkDomain",
                       sort = "", filters = "", segment = "")

analytics$Date = as.Date(paste0(substr(analytics$dateHour, 1, 4), "/",
                                substr(analytics$dateHour, 5, 6), "/", substr(analytics$dateHour, 7, 8)))

analytics$Hour = as.numeric(substr(analytics$dateHour, 9, 10))

# change domain

analytics$Domain = factor(recode(analytics$networkDomain, '"nhs.uk" = "NHS"; else = "Other"'))