Nigel Babu

Blog

  • Better Problem Definition

    I’m a core developer on CKAN at Open Knowledge, the most widely used data catalog software. Early this year, we released version 2.2 of CKAN with a complete overhaul of the filestore. Amusingly, right after that, we started getting more and more complaints about data loss from the old filestore from on the ckan-dev list. One of the many folks, helped narrow it down to a particular file called persisted_state.json.

    This file is created by a library called ofs. Every time a new file is added to the filestore, OFS does the following:

    • Read the persisted_state.json file.
    • Convert the JSON to a Python dict.
    • Add an element to this dict with the metadata of the new file.
    • Convert the dict back to JSON.
    • Write this new JSON to persisted_state.json file.

    This causes concurrency problems when things were added to the filestore at high frequency and eventually lead to data loss. Oh joy.

    Technically, this wasn’t a bug in CKAN’s codebase. We already solved the core problem at this point by switching to a new filestore which did not use ofs. We couldn’t abandon our users though and I volunteered to find a fix. I read through ofs code and I thought of solving the problem there. After an hour or two of reading up on concurrency and documentation on the python, I still didn’t have a working solution. Eventually, I asked myself what I was looking to solve.

    My original problem: “OFS is not thread-safe, causing data loss”. I then realized, that’s not what I wanted to solve. A better problem to solve was: “OFS is not thread-safe, causing data loss. Our users need their data.”. So, I wrote a script that would re-generate the persisted_state.json file with just enough metadata to start working. It isn’t a complete fix, but it was a productive fix. The script was “dramatically” called ofs-hero.

    Lesson Learnt: Defining the problem properly helps you solve it better.

  • Fossmeet 2014

    On January 24, I got an email from the Speakers Team at FOSSMeet asking me if I would be able to propose a talk. Fast forward to February 13th, I was on a bus heading to Kozhikode. It’s been quite a while since I’ve talked a conference and this would be the first I’d talk about open data. Despite being a Malayali, I’d actually not met a lot of members of the FOSS community from Kerala. As usual, I enjoyed meeting and talking to people about their work and what they do (yes, I’m still an introvert :D).

    On the first day, I attended Praveen’s talk (fine, not talk, a discussion) about privacy. Rather fierce arguments broke about privacy, specifically whether the government should invade it to save lives. I’m fairly certain it got a lot of people thinking about privacy. It’s hard to think about things like privacy unless you can contextualize it for yourself and that’s exactly what happened.

    After lunch, I sat in the session about Anoop’s workshop about contributing to Open Source. It was meant to give people an idea of the tools you should know. I only sat in it for an hour or so and they were learning git at that time. A while later, I stepped out, primarily because I was starting to get sleepy.

    I got back to the main auditorium, just in time to learn that a student passed away on the campus, in the grounds we could see from the auditorium. A wall fell down and he was stuck under it. All of attendees were asked to be in one of the lecture halls while the organizers talked to the faculty and figure out what next. The organizers decided to cancel all the entertainment activities that were planned and the hack night. The remaining sessions were held as an informal discussion rather than actual talks. Later that night, the events of next day were also canceled.

    My talk had a few people and we had good conversation about open data and thanks to Nirbheek, we had people glance at The Ballot. I couldn’t give the talk I planned, but I’m grateful for the discussions we had. Later that day, the students were leading a protest in front of the director’s house and the rest of the event was formally canceled.

    FOSSMeet seems to be a wonderful place to get more students aware of free and open source and kickstarting contributions. The organizers had done a good job, but were just unlucky with the turn of events. Now that I’ve attended FOSSMeet once, I’m planning on attending the next editions for sure.

  • Goodbye Ubuntu

    On March 8th, my Ubuntu membersip will expire. I’ve been getting email notifications for a few days and I’ve decided not to renew my membership. Ubuntu introduced me to open source. Thank you for the great operating system and the sense of community that I’ve had for a few years. I’ve made a lot of friends and I’ve had a lot of mentors who’ve helped me become a better person.

    With hggdh at UDS-0

    I won’t disappear entirely – I will still be in a few IRC channels and help in whatever little way I can.

    Thank you everyone for the spectacular few years.

  • Cambridge and London – Winter 2014

    I’m sitting in the lobby of Clink 261, a hostel near King’s Cross, hearing so many languages that my brain hurts trying to identify them (Nope, I can’t). To complicate matters, the reduced sunshine is confusing. The week has consisted of meetings, plotting (to take over the world, of course), drinks, ranting, and generally fun things. It’s been a tiring but spectacular week with my team at the OKF. As a remote employee, I enjoy meeting my colleagues in person and spending time together.

    As always, my trips are more about people than places. In Cambridge, once I was properly awake, I met a colleague for lunch at Cambridge Blue, where we had deer pie for lunch. My Twitter followers seemed a bit mad that I ate Bambi. From there, we went to his friends’ and had tea with them, while enjoying watching their kids play and chatting about books, trains, and comics. Later in the evening, I caught up with an old roommate and his family.

    I walked everywhere in Cambridge except when I had luggage or if I was, cough, otherwise unable to walk. Our hotel was a 40-minute walk away from the venue of our meetings. I took a taxi on a few mornings, but when I could, I walked. The cold manages to wake me up, and in a few minutes, I’m nice and warm (weather hacking?!). There are other advantages too, like a lovely pastry shop on way 😉

    I spent 3 days with my team plotting to take over the world with CKAN and then 2 days with the rest of the team. An extremely produtive time during the day and a good time chatting with everyone afterward, especially people from other teams. We had a “Cheese Off”, which France won, closely followed by Brazil. The close of the summit involved a puppy and cake!

    I’m in London now, taking a day off for myself. I might head out and grab some lunch or take a walk. No plans for today and I’m leaving it that way.

  • Quick Tip: Ansible Debugging

    Today I learned something about Ansible debugging from benno on #ansible. Occasionally, commands can get stuck, especially if it’s waiting for input. You can’t fix this until you recognize what’s going on and see the prompt. In other words, you want to see the stdout and stderr on the target machine. Here’s what you do:

    • Run ansible with -vvv.
    • Login to the remote host where the command is being executed.
    • Find the ansible process executing the command and kill them.
    • The stdout and stderr should be printed to the console where ansible was running.