Blog

Git Tips You Probably Didn’t Know
I’ve been using git for quite a while now and some of it’s features continue to amaze me. Here’s a few things I learned recently.

Finding only your changes to master.

When you’ve made a change against a master that’s moving often, you find that simply doing git diff master doesn’t give the right diff. It shows you the difference between your branch and current master. That’s not what you want in most cases. The master branch would have changed and now the command also shows those changes. GitHub does the right thing and the right command in this case is the following:
```
git diff master... 
```
Copying your changes in one branch into another

Recently, our designer Sam Smith started working on making CKAN more responsive. His Branch was based off master and I wanted to make a new branch based off release-v2.2 with his changes on top of it. My instinct was to make a patch.
```
git diff master...responsive > ../responsive.patch 
```
Then, apply the patch onto a different branch. This would surely work, but I’m using a version control system! It should be smart about this. The good folks in #git pointed me in the right direction.

First find the last commit from the responsive branch that you don’t want to copy.
```
git checkout responsive git log master..responsive 
```
The last commit in that log is what you want to copy. In this case, it’s 7587c6e8fe49c809ef7357b6f88496bd06ac93b9, so now you want to do git log 7587c6e8fe49c809ef7357b6f88496bd06ac93b9^. The first commit is the one you don’t want to keep.
```
git checkout responsive git checkout -b reponsive-2.2 git rebase 7587c6e8fe49c809ef7357b6f88496bd06ac93b9 --onto release-2.2 
```
Thus, responsive-2.2 is a new branch with responsive changes based on top of release-v2.2!
2014-04-26
Dharamshala Getaway

We ran with our luggage. None of the buses at ISBT were to Dharamshala. Just as we were about to give up hope, a guy said, “It’s just started from the platform and is near the gate. Run and you can catch it.” More running and I got there just as the driver was about the close the luggage doors, just in time to get our luggage in and get in. I still can’t believe we actually caught the bus. This vacation definitely started on an exciting note.

I woke up to scary, but beautiful mountains all around us. The fear was definitely ecause of how our driver drove the bus like a bike, squeezing into tight spots (you have a dirty mind…) band (seemingly) barely using the brakes. After we got to Dharamshala, the bus driver seemed extremely careful, because our taxi ride to Naddi, where we booked a cottage, was scarier. We did that trip several times during our stay and every single time I was convinced, we’d meet with an accident on a hairpin turn. The drivers in these mountains seem to have missed that lesson about honking on a blind curve, or at least slowing down a bit.

Staying in Naddi had its downsides and upsides. We had a spectacular view and the only noise we could hear were the sparrows at the doors and the kids playing cricket in the small ground behind us. It was also a bit far from everything and we would be charged Rs. 200 to McLeodganj or Rs. 300 to Dharamshala. After we caught up with sleep on the first day, we walked around and saw the sights in Naddi. We thought about going to a waterfall nearby, but we were told it probably wasn’t worth it. We walked down to the Dal Lake instead. Now, it’s only 2 km away, but in retrospect, I’m not sure it was a good idea. We learned that we’re both not in good shape and walking downhill is actually harder. Dal lake is beautiful and quiet. We sat there for some time ~~resting our feet and catching our breath~~ enjoying the view.

The climb up and walk back to our rooms thoroughly exhausted us. The next morning, we went down to Dharamshala. After lunch, our plans were to visit the Infinity Hackerspace. It was quite the bus ride and then a “20 minute walk” which took us 40 minutes. The view from there is just spectacular. It looks as though the mountains are just another walk away (of course, it’s not). I spent the afternoon chatting to sva and planemad about their life there, internet woes, and renewable energy. We started heading back after an hour or two long stay there. This vacation definitely included more exercise than I get with my daily life (I think life is trying to tell me something :P).

The next day was our last day in Dharamshala and it was reserved for McLeodganj. We had the most physical strain on this day because we took the wrong turn to the temple and ended up going down quite a bit and climbing up an even stepper slope. But hey, we finally got there! The Tibetan museum in the complex was eye-opening to the problems of the Tibetan people and their conflicts with China. We seem to never get a clear picture of this conflict. On the way back, we did some shopping and got something to eat.

While we were eating lunch, the sky outside was clear and it was a slightly chilly day. Right after we finished lunch, visibility outside dropped to less than 50m and suddenly the whole placed had a magical feel. Just as suddenly as it came, the clouds vanished and it was back to normal. I’ll remember that moment though, when the restaurant and the square seemed empty for 5 minutes. I couldn’t even see across the road.

Overall, this was a thoroughly enjoyable vacation. It taught me that I’m completely out of shape physically and I’ll need to do something about it 🙂

2014-04-17
Better Problem Definition
I’m a core developer on CKAN at Open Knowledge, the most widely used data catalog software. Early this year, we released version 2.2 of CKAN with a complete overhaul of the filestore. Amusingly, right after that, we started getting more and more complaints about data loss from the old filestore from on the ckan-dev list. One of the many folks, helped narrow it down to a particular file called persisted_state.json.

This file is created by a library called ofs. Every time a new file is added to the filestore, OFS does the following:
- Read the persisted_state.json file.
- Convert the JSON to a Python dict.
- Add an element to this dict with the metadata of the new file.
- Convert the dict back to JSON.
- Write this new JSON to persisted_state.json file.
This causes concurrency problems when things were added to the filestore at high frequency and eventually lead to data loss. Oh joy.

Technically, this wasn’t a bug in CKAN’s codebase. We already solved the core problem at this point by switching to a new filestore which did not use ofs. We couldn’t abandon our users though and I volunteered to find a fix. I read through ofs code and I thought of solving the problem there. After an hour or two of reading up on concurrency and documentation on the python, I still didn’t have a working solution. Eventually, I asked myself what I was looking to solve.

My original problem: “OFS is not thread-safe, causing data loss”. I then realized, that’s not what I wanted to solve. A better problem to solve was: “OFS is not thread-safe, causing data loss. Our users need their data.”. So, I wrote a script that would re-generate the persisted_state.json file with just enough metadata to start working. It isn’t a complete fix, but it was a productive fix. The script was “dramatically” called ofs-hero.

Lesson Learnt: Defining the problem properly helps you solve it better.
2014-04-08
Fossmeet 2014

On January 24, I got an email from the Speakers Team at FOSSMeet asking me if I would be able to propose a talk. Fast forward to February 13th, I was on a bus heading to Kozhikode. It’s been quite a while since I’ve talked a conference and this would be the first I’d talk about open data. Despite being a Malayali, I’d actually not met a lot of members of the FOSS community from Kerala. As usual, I enjoyed meeting and talking to people about their work and what they do (yes, I’m still an introvert :D).

On the first day, I attended Praveen’s talk (fine, not talk, a discussion) about privacy. Rather fierce arguments broke about privacy, specifically whether the government should invade it to save lives. I’m fairly certain it got a lot of people thinking about privacy. It’s hard to think about things like privacy unless you can contextualize it for yourself and that’s exactly what happened.

After lunch, I sat in the session about Anoop’s workshop about contributing to Open Source. It was meant to give people an idea of the tools you should know. I only sat in it for an hour or so and they were learning git at that time. A while later, I stepped out, primarily because I was starting to get sleepy.

I got back to the main auditorium, just in time to learn that a student passed away on the campus, in the grounds we could see from the auditorium. A wall fell down and he was stuck under it. All of attendees were asked to be in one of the lecture halls while the organizers talked to the faculty and figure out what next. The organizers decided to cancel all the entertainment activities that were planned and the hack night. The remaining sessions were held as an informal discussion rather than actual talks. Later that night, the events of next day were also canceled.

My talk had a few people and we had good conversation about open data and thanks to Nirbheek, we had people glance at The Ballot. I couldn’t give the talk I planned, but I’m grateful for the discussions we had. Later that day, the students were leading a protest in front of the director’s house and the rest of the event was formally canceled.

FOSSMeet seems to be a wonderful place to get more students aware of free and open source and kickstarting contributions. The organizers had done a good job, but were just unlucky with the turn of events. Now that I’ve attended FOSSMeet once, I’m planning on attending the next editions for sure.

2014-03-11
Goodbye Ubuntu

On March 8th, my Ubuntu membersip will expire. I’ve been getting email notifications for a few days and I’ve decided not to renew my membership. Ubuntu introduced me to open source. Thank you for the great operating system and the sense of community that I’ve had for a few years. I’ve made a lot of friends and I’ve had a lot of mentors who’ve helped me become a better person.

I won’t disappear entirely – I will still be in a few IRC channels and help in whatever little way I can.

Thank you everyone for the spectacular few years.

2014-03-05