Thursday, July 9, 2015

When work and pastimes collide

Someone asked me to write a few paragraphs about what I like about my work and coding in general, so here's what I sent. It quite nicely summarises the transition from physics based problem solving, and other, further reaching topics.

I'm Aidan and I use computers all the time for work and as a hobby! I work on the CMS experiment at CERN and it's my job to analyse the data that we collect. It's a very exciting job because we're using some of the world's largest data sets and the world's most powerful computing systems, and we're the first people to see the results. It's not a simple job, because the task is so large, and there are many opportunities for mistakes and failures, of both humans and computers. The final results we show the public are usually just a graph, or s single number, or even a diagram of a single physics event, but there are hundreds of tiny steps between getting the data and showing those results. That's what I do, and there's something very satisfying about making that possible.

Some of the computing resources available at CERN. (CERN Computing)

It's all about problem solving and being creative in how to approach these problems. At CERN we have access to so many different resources, and we are given the freedom to use them as we see fit. We're faced with some of the most challenging and promising data in the world, we're given some of the most impressive tools to analyse them, and we can play about with whatever methods we like to get the answers we need. For me it's all about working out how to solve the problems and be ready for the when the data arrive. They arrived a few weeks ago and for the past two years I've been slowly building up the software needed to analyse them and to understand the data sets we use. As it happens I was one of the first people to analyse the most recent data, perhaps the first in the world, and seeing the data, using the software I had written myself, is one of the best feelings in the world. It's not something you "do", it's something you create, and becomes your pride and joy. For the past couple of weeks I've been staying up very late and working from home, just to get a few extra hours to analyse the data, and to be the first to see it, because this is a once in a lifetime opportunity. If you want a project that you can make your own, that you can build up from almost nothing into something beautiful and elegant, then data analysis might be perfect for you!

I also develop code in my spare time. When I was a child we didn't have the internet, and all we had at home was a BBC micro computer with 32 Kb of memory, and a handbook to a computer language called BBC BASIC. In those days the floppy disks really were floppy, they were about 13 cm wide, and had to be stored in protective paper sleeves. It was with those tools that I learned how to code and how write simple programs. I knew that this is what I wanted to do with my life (or at least a part of it.) Eventually we got a family computer and a slow dial-up internet connection. Over the years I've learned so many different computing languages, and I keep learning new languages and skills today. Some of my coding projects have been ongoing for over a decade, so it really is a gift that keeps on giving. I think that as well being a lot of fun, coding helps develop a lot problem solving skills, and helps clarify the way we think. It's one thing to ask someone to make a cup of tea, but if you have a tell a computer how to do it then you suddenly realise how precise you have to be! But once you've worked out how to tell a computer how to make one cup of tea, you can tell it to make thousands of cups of tea, you can tell it to tell other computers how to make cups of tea, you can change things around a bit to get a cup of coffee instead. Suddenly all these possibilities open up and you realise that you can do so much more than you could before.

Computing is all about empowerment and extending your reach to do things which are beyond even most intelligent and tireless of people. If there's a problem you want to solve, then the only limits are your own ingenuity and the limits of your hardware. One time I lost my phone and I needed to make an alarm clock, so I spent about 20 minutes making a web page and now I've got an alarm clock I can use any time. Problem solved! I've always enjoyed maths and science and this has motivated a lot of my coding projects. Sometimes someone will tell me about some theorem or a fractal or a pattern, and for the next few days all I want to do is find out more, to analyse it, to find out the answers, and with coding that's possible. If, like me, you have an insatiable urge to discover things and understand things, then learning how to code is one of the best things you can do with your time. If you want to answer a question you can go online, get a little help, find the data you need, and get your answer. Some of the data I've analysed have been general election results, LGBT rights worldwide, stock prices, and how much money I spend on train tickets to visit family and friends (which told me I could afford to visit family and friends more often!)

LGBT rights in Europe, a temporal intersection of politics, social change, and geography, that just begs to be analysed.

Solving problems is one thing, but making games is another. I love to make games, and sharing them with my friends. If solving problems and analysing data are fun and interesting, then that's nothing compared to interacting with other people. Suddenly you have to combine computing with psychology, and that's fascinating. Even more than that you have to keep up with the latest technology, which puts you at the cutting edge. There have been many times where I've looked online for the answer to a problem and found that nobody has posted an answer, so it's possible that I'm the first person to come across it! I love to push technology to its limits and see how far I can push software before it breaks. As soon a a new technology opens up I'm all over it and seeing what can be done, what fun I can have. One of my current projects involves writing educational software about data collection at the LHC. It's something I've wanted to do for a long time, and it's all my own work. As far as I can tell there's nobody else in the world doing anything even similar to what I'm doing, and now I'm getting funding from a UK research council to develop this further and take it into schools. If you have an idea that want to pursue, and the patience to follow it through, then there are so many opportunities that open up for you.

It's not just games though, you can make some beautiful art with software. There's a mathematical fractal known as the Apollonian gasket which is where you pack a space with circles, so I learned how to make this fractal, then decided to use it to fill a photo. Suddenly I had a really cool image that I could print off and put on my wall. Even better than that, I could scale it up to any size without losing any image quality!

Apollonian gasket art.

There is something very satisfying about making something of your own, especially when you show it to a friend and they have a hard time believing that you made such a thing. That's what I do on my website as well, I make an interesting game or a tool and for some reason it doesn't occur to some people that writing your own software is possible. Of course it is, because real people have done it and none of us is particularly special in our talents. It takes a little patience, some practice, and a drive to create something new. As time goes on we find more and more resources at our disposal, more and more experts to help us, better hardware to use, and all this makes it easier than ever to start developing software. There's an excellent community out there as well, other hobbyists who want to share the joy of coding with the latest newcomers. Some of the most fascinating people I've ever met have been other coders, people who freely share their own projects, help strangers out, write games for fun, and write software for the greater good. If you want to see some examples of open and honest altruism then speak to coders online and see what they have made and freely shared. They've made something that has cost them nothing but their time, patience, experience and ingenuity, then shared it with the world. Some things are frivolous, some are crowd-sourced projects that help people out, some lead to revolutionary new techniques or ideas.

It's an exciting time to be a hobbyist or a developer, and I've found that there's no firm boundary between my hobby ends and my work begins. The skills I learn in my spare time I apply to my work, and the problems I solve in my work inspire new projects. I now find myself developing educational software, inspired by my work and initially a hobby, now occupying more of my time and something that will feature prominently on my CV. It's like the saying goes, if you find a job you love you'll never have to work a day in your life!

Wednesday, July 8, 2015

New data!

Recently the LHC has started Run 2 and this week there are high intensity beams delivering a lot of data. This is a very exciting time to be a physicist and what I have been waiting for the past two years. In fact, my contract as been extended to make the best use of this new data. No amount of working with simulation or theory can compare to working with real data. This job has a lot of advantages and disadvantages, and right now the job is extremely motivating and enjoyable.

What the first 2015 data looked like at CMS.

Something I've wanted to do with new data is automatically analyse them as they arrive, and now I can do that for the first time. (I had tried when I worked on ATLAS, but due to time and technology constraints it was not possible.) Now that we have a globally distributed computing system where any computer can access any file at any time I can stream the latest data each night and run the analysis before I even wake up. So that's what I'm doing right now and it's one of the most fun projects I've ever developed! I've spent the past couple of years preparing and tweaking the software so that I'd have a push-button system in place by now, and it's worked very well so far. There are going to be many teething problems, of course, but they're minor compared to the labour that will be saved. My first large test job is currently running, analysing at its peak 300 events per second (100 events per second, once I/O is taken into account.) Following on from a previous post about how my personal computing resources are not sufficient to do all the work I need to do, I've contacted IT support and been given a generous allowance of disk space, as well being able to run my CPU and I/O intensive jobs overnight. This is going to make a huge difference, and has allowed me to support my students with the simulations and datasets that I analyse. Seeing all this come together is a wonderful experience.

This is how excited physicists get when we first see beams in the machine! I was a very proud Shift Leader that day.

On the one hand this is, of course, very rewarding, but on the other hand a little disappointing that this isn't already being done. There seems to be a culture in particle physics that real work is about manually submitting jobs and analysing data, and there are politics associated with the notion that people should take turns in carrying the burden. We have the technology and expertise to run an analysis automatically, so why not do that? That's what we would see in the private sector. We have access to some of the largest datasets the world has ever seen and some of the most powerful computing resources ever created, it would be a crime not to exploit those as much as possible. I get the impression that I'm the only person doing this kind of work, and if that's the case then it's certainly time to move on to a different workplace. But first, I have data to analyse! I love data.