_{Source: Polyglot YYC}

This weekend I went to Polyglot YYC 2019. It’s a gathering of tech people from around Calgary. The event was open to all possible topics, hence the name Polyglot. It was the first unconference style meetup I went to and this post is a summary of my day.

About the attendees, I don’t have the official count, but I imagine there were close to a hundred people. Quite a few people represented the sponsor companies. The sponsors advertised for hiring new employees, mostly software engineers. On the other hand, I met a few people who joined this venue to talk to prospective employers. Some of the attendees were enrolled in a coding bootcamp to switch careers. I was quite happy to see the community praising their enthusiasm towards the bootcampers. Moreover, I met a few regular tech meetup people after a long time. I used to go to all tech meetups I could find in town before having kids, and I felt great to be among the self-motivated crowd after a long break.

About the event, I was fascinated by how the unconference took its shape. At check-in time, everyone got a couple of forms to write down topics of interest, either as a host or a participant. Everyone could vote for five such topics. I didn’t prepare beforehand. So, I put up a topic that I’m presently curios about, “Writing for Developers”. A handful of people voted for me, but it didn’t make the cut. I found some ideas better than mine and was happy that those got voted to the top. In hindsight, I should’ve proposed the topic of “Why Are You Not Innovating?”. I did an internal presentation on this topic at Cisco and it was generally praised by my colleagues.

My other observation is, between technical and soft-skill related topics, I liked the soft-skill ones better. For example, I found the topics on hiring, choosing a technical vs. managerial career path to be more interesting than the topics such as GraphQL and ReactJS. A few years ago, I’d just choose the technical topics without thinking twice. This is also a pattern in my recent blog posts or reading list.

In the hiring session, I saw a positive attitude towards hiring remotes and treating them as equals. This is a major mindset shift among the community.

In the individual contributor vs. management career path session, the attendee list included both kinds as well as people that had transitioned in either direction. The one take home message I got from this session was, when confused with career choice, individual contributors and managers should take the step to switch roles. And if things don’t work out, it’s totally possible to revert later.

I’m looking forward to the 2020 edition of this event and may even prepare to come up with a good idea for the unconference. If you can, please join us for the next round.

Showing a Screenshot of Microsoft Word for Mac and Apple Pages

This is what you get by default when you open a blank document on both editors. I love that Apple Pages puts a deep focus on the content. If you haven’t used it, I’d recommend trying it out. I know Microsoft Word has a known face, so you’re likely used to all the distractions that sorrounds the content. But, if you can, give Apple Pages a try.

_{Source: Henry Burrows}

Whenever faced with a production issue, I find exceptions to be an extremely useful information source. A careful look at an exeption has often led to quick discovery of the source of a trouble. On the flip side, I have also faced a lot of chaotic debugging sessions because of poor exception handling. Here, I present the common anti-patterns that I recommend fixing while reviewing pull-requests. Most programmers are already familiar with the mechanics of exception handling. Yet, I see these anti-patterns everyday.

I primarily see these anti-patterns to be control-flow or logging related as shown below:

Control-flow Anti-patterns

Unhandled. When an exception is unhandled, if often results in a clueless user experience for the end user as well as the developer.

1
2
3

def notify
  post.email! #May fail due to configuration, network, or authentication
end

Catch-all. With catch-all errors, it’s often difficult to quickly detect the original problem. For the same reason, the end users don’t get specific and actionable error messages.

1
2
3
4
5

def create
  post.save! #May fail due to database issues
rescue => error
  # handle
end

If-else Exceptions. Exceptions mean something unexpected took place. If-else is used for logical known code paths. For example, when accepting an API request, invalid input data is often a known logical path. Using exceptions for it will trigger false alarms.

1
2
3
4
5
6

def create
  post = Post.new(params)
  post.save!
rescue ValidationError => error
  log_exception(error)
end

Wrapped Exception. A new exception is raised hiding the original exception. In such cases, if the exception is handled by the caller, critical context information is lost since the orignal stacktrace is no longer available.

1
2
3
4
5

def create
  post.save! #May fail due to database issues
rescue SaveError => error
  raise CustomSaveError.new('Failed to save the post')
end

Useless Custom Exception. Introducing a new exception type when a pre-defined exception suits just fine.

1
2
3
4
5
6

def create(text:)
  if text == nil
    #Could just use pre-defined ArgumentError
    raise EmptyTextException.new("Text can't be empty")   end
  #...
end

Leaky Handler. Handling an error without cleaning system resources such as file handles, open network connections, can cause cascading system outage.

1
2
3
4
5
6
7
8

def create
  #Will leak this file handle if read succeeds, but write fails
  file = File.open('/some/new.txt', 'w')
  file.write('some text')
rescue FileNotFoundError, FileSaveError => error
  log.warn('...')
  raise error
end

Logging Anti-Patterns

Silent Handler. Makes it very difficult to debug problems.

1
2
3
4

def create
  post.save! #May fail due to database issues
rescue
end

Debug-only Handler. Similar to silent handler since most production apps run in non-debug log level.

1
2
3
4
5

def create
  post.save! #May fail due to database issues
rescue SaveError => error
  log.debug "failed to save post #{error.message} #{error.backtrace.join}"
end

Custom Message-only Handler. Some exception handlers only log a custom message leaving the details of the exceptions. As a result, critical information is lost that can be very useful for debugging.

1
2
3
4
5

def create
  post.save! #May fail due to database issues
rescue SaveError
  log.warn "failed to save post"
end

Message-only Handler. Without Stacktrace, it gets very difficult to trace the root of a problem since often times exception handlers wrap a few lines of code.

1
2
3
4
5
6
7

def create
  email = User.find(params[:id]).email
  post = Post.find(params[:id])
  comment = post.comments.create!(name: user.name)
rescue NotFoundError => error # Could happen in line 2 or 4
  log.warn "failed to save post #{error.message}"
end

Sneaky Handler. Some exception handlers return nil or a value. The caller can’t distinguish between a successful vs. exception case and fails in subsequent steps.

1
2
3
4
5
6

def create
  post.save! #May fail due to database issues
rescue SaveError => error
  log.warn "failed to save post #{error.message} #{error.backtrace.join}"
  return null
end

There are times when you intentionally have to use some of these anti-patterns. But those are rare. It’s critical for the developers to think about the information that’d help in swiftly debugging a production problem. As such, developers must avoid the noise and provide all context information for errors to help diagnose potential system problems.

Happy coding.

Rainer Maria Rilke

All creators take a deep interest in the creations of others. All filmmakers watch a lot of movies, all good writers are also the most prolific readers, all artists can talk at length about the smallest pieces of art they have seen.

We, software developers, if we want to claim ourselves as the artists of this craft, we must be prolific readers of code. There’s been no better time as today. We have immediate access to millions of lines of carefully written code out there in the internet. Just like artists of any craft, I’ve had so much fun spending time with my fellow developers that read code for the pure joy of learning something new.

Just had so much fun reading this ruby code today from the Ruby on Rails project:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

class Array
  # Wraps the array in an +ArrayInquirer+ object, which gives a friendlier way
  # to check its string-like contents.
  #
  #   pets = [:cat, :dog].inquiry
  #
  #   pets.cat?     # => true
  #   pets.ferret?  # => false
  #
  #   pets.any?(:cat, :ferret)  # => true
  #   pets.any?(:ferret, :alligator)  # => false
  def inquiry
    ActiveSupport::ArrayInquirer.new(self)
  end
end

The beautify of this rich API is an art. I love it. You may have different opinions. But I hope you find your love of art in code. There’s plenty of art in code out there for everyone to enjoy.

After being a people leader at work for the past six years, I’m now going through a phase of introspection. Essentially, I’m trying to understand my own philosophy about people leadership so that I can clearly communicate it to the people that I support.

My most important realization is, people leadership is all about developing people. What I mean by this is, for everyone I’m supporting, I must carefully build a plan that provides them with the opportunities to stay motivated so that they can thrive. With this goal, once I wrote down my understanding of what motivates each of the people I support, it was quite eye opening to see the differences among people. Going through this process, I also realized how unprepared I was in terms of providing them with a clear career path to achieve their best.

If you’re interested, I’ve shared a template of the people development document here.

1
2
3
4
5
6

Name: ___
Date:___
Current role: ___
Motivation: ___
Upcoming opportunity in the next three months:___
Upcoming opportunity in the next year:___

I consider this to be a living document as people often develop new interests and the opportunities at work change with time. But keeping a clear log of each individual’s career is a great way to establish and manage expectations. You can build this with the the individuals directly and update it when you meet for one-on-one feedbacks. As a lead, when you collaboratively build this, you empower them and build a trusting relationship as you both see how the motivations align with the work.

From the past 13 years of my time in the industry, working for 5 different companies, personally I’ve always felt a little under-informed about how my leaders planned a career development for me. Through my introspection of being a people leader, I realized I didn’t honestly appreciate the need for such clarity among the people I supported. So, I wanted to change it. And found the written document to be a simple yet surprisingly powerful tool to fill this void.

Now, if you’re a people leader, I’d recommend doing this exercise with your people. You’ll be pleasantly surprised by the outcome.

If you aren’t a people leader, you can write it down for yourself and ask your leader to collaborate on it. This way, when you have a one-on-one, you both will have the same reference document to focus on.

Recently, we had a production outage for a few minutes due a database migration on one of our Ruby on Rails apps. The deployment went fine through a few stages, but the problem only showed up at the last and the largest stage. This is exactly what happened during the deploy process.

1. New code was deployed. Restart was pending, so the server was still running old code.
2. Migrations ran.
3. One migration removed a column that was used in the old code, but no longer used in the new code.
4. The next migration was a data migration that inserted one row / user to a table. This was a very slow migration, taking 5+ minutes.
5. The old code failed because it tried to use a column in the database that was no longer there. To make things worse, the column was referenced at all page loads within the app.
6. The long running migration didn’t finish because it ran into a timeout.
7. The servers weren’t restarted because the migration had failed. So, the new code wasn’t served at all.
8. There was no automatic database rollback to restore the system into a good state with the old code.

The team was able to resolve the issues within the next 5 minutes, but it was the worst system outage we’ve seen in years. For anyone dealing with a large Ruby on Rails app, you can use the following safeguards to avoid such problems:

1. Do not remove a column from the database while the current code is still using it. Do it at a later release.
2. When a deployment fails at the migration step, ensure you have a rollback policy so that the system can be automatically restored to a known good state.
3. Consider data migrations to be a performance problem and always test the migrations with relaistic load before production release.
4. If possible, run your data migrations seprately from schema migrations so that you don’t incur deployment delays for optional new data.

Src: Claudia Dea

Most problems are solved inefficiently. Innovation follows when a problem is researched first.

I spent a fair amount of time in grad school. First, sixteen months on my MSc, and then another forty eight months on my PhD program. Also, I worked as a pro software engineer for 20 hours/week during my MSc days and then full-time during the PhD days. It was quite an undertaking, but I survived and now claiming my bragging rights :-)

This post is about a general problem-solving approach that I internalized as the most valuable lesson from my grad school days. I hope you find it useful. I’ll try and explain the concept using a personal story. Here you go.

Recently, I was discussing plans for the near future with my wife, Shahana. She’s an engineer. In the past, Shahana worked in the telecom industry as a network planner for a few years. Then, she decided to take an extended maternity leave to stay with our two little kids, Shopoth and Shera. Now, honestly, she’s had enough and seriously contemplating a return to work. Except, she’s unsure about what career to choose that she’d really enjoy.

Given this vague problem of finding a new career, she felt lost for a while. As a true supportive husband, I decided to be her man and gave her an earful!

Silly jokes apart, we decided to set the context first by asking and answering the following questions:

1. Why is she looking for a new career?
2. Why not remain in the old career?
3. What new career options exist?
4. Where are the opportunities?
5. Who knows more about those?
6. How does she get a job in the preferred area?

After spending a few days of research, she developed a very clear understanding of the problem and the initial vagueness of the situation was mostly wiped clean resulting in a few clear constraints. From a job portal, Shahana then listed available job requirements from a number of open positions matching her criteria. After consolidation of the requirements, the gap was clear. It was obvious that she’d need to develop essential new skills to qualify for the jobs. As she acquires the new skills, she already knows the evaluation. If this doesn’t work, she can repeat the same process to either improve her chances or choose an alternate career path entirely.

This was a direct application of the problem-solving framework that I learned from grad school. Here’s a nerdy version of the framework:

1
2
3
4
5
6

  Given a problem
    1. Ask the Wh questions -> list constraints
    2. Research -> list existing solutions
    3. Is the problem already solved? -> If yes, stop
    4. Innovate, fill in the gaps.
    5. Evaluate

In real life, most problems are deceptive. We think we already know the best solution based on our past experience and go straight to #4 leaving all other parts out. Even if going straight to #4 is actually a wise move, it only becomes an innovation with a tiny impact, as it doesn’t become available at step 3 for others to leverage.

A large number of problems can be solved at step 3, by leveraging a solution or a mix of multiple solutions that already exist. Even when problems get past step 3, it’s highly likely that the gap is smaller than the extent of the whole problem. So, an innovation can focus only on solving the gap, which offers speed and intelligent reuse.

This post is so prosaic already. But if you’re still with me, I’ll make it even more textbook like by giving you a couple of practice problems. Go ahead and apply my 5-step problem solving approach:

1. Jane is a musician, a prodigy so to speak. She’s been invited to talk about music as a therapy. She’s not good at speaking, but this is a big opportunity. What does she do?
2. Mike is a young startup CEO. He’s very good at business and technology. He needs a lawyer to take care of the legal matters. How does he hire the best lawyer possible?

Past few months at work we’re trying to revamp our CI infra. This has been a migration from a Jenkins cluster to GoCD. This post is a result of my frustrations with the state of CI in the age of Git/GitHub.

With Pull Requests, or a form of pre-merge review process, it’s common to leverage a few automated checks alongside our human peers to remove potentially breaking changes from being merged into the repo. These changes typically require a CI server and often multiple CI jobs/pipelines to check test run / coverage results, linting, or any other policy that can be automatically enforced.

This is where you need to integrate your GitHub/what have you with your GoCD or the likes. And it’s a time consuming task for any complicated project. You essentially, setup pipelines that are somewhat duplicate to be executed pre and post merge. The more I think about, the more it feels like the whole CI process should be a headless one, and deeply embedded within the code repo. This would save unneccessary hassles such as setting up user accounts / notifications across multiple systems, navigating between web interfaces to see important logs when things fail etc.

I like the direction GitLab is taking. They have made CI part of GitLab UI so you can see the code and its build status in the same place. I’d like other CI environments to do the same, or at least be a little app/plugin that you can add on top of GitHub - to blend within the GitHub experience such that it doesn’t feel like a foreign object.

Finally, I defended my PhD thesis yesterday. Family, friends, and colleagues are all sending their wishes, and it feels great to have crossed this milestone.

To be honest, deep inside, I carry a sad and guilty feeling, which at times overshadows the feeling of acomplishment. This phenomenon is hard to write about, but I wanted to give it a try anyway.

So, it was 2012. Life was good. I got promoted to Senior Consultant at ThoughtWorks and Shahana got a job almost immediately following the completion of her grad school. We spent an amazing vacation to end the year 2011, and the summer of 2012 was full of weekend trips to all the beautiful touristy places. That summer we also found that we were gonna be first time parents. It was an exciting time.

Along came fall. It started getting cold and wintery in Calgary. Both of us would be home by 5:30 PM, and would look for things to do to keep busy till bed time, which is about midnight. With slippery and often icy sidewalk, going for walks oudoors would mean a serious risk for the soon-to-be mom. So, we mostly stayed indoors. We were naive, and didn’t quite learn to enjoy the serenity of the “nothing to do” time. So, in one of such evenings, out of sheer boredom, I decided to shoot an email to my then ex-supervisor asking if he could meet to discuss potentially a part-time PhD admission for me.

He wrote back and we meet next week. He was quick to accept that request and also ensured that he’d be able to pay my tuition and expences as a scholarship, if required, for the full four years. I discussed this with Shahana and we decided to take this opportunity. Officially, the program would start in Fall 2013, about 5 months after the due date of our first born.

This is going to be a long read. But I’m writing it while it’s fresh in my memory from this overwhelmingly emotional time that I’m going through since the exam yesterday. This write up is an attempt to dump the different incidents that are showing up like storms in my mind at this time. I think it’s written for myself, more than anything. But I appologize if you’re reading this, since it’s a long format story, the format that most people don’t like to read online.

Back to the story, even though the program officially started in Fall 2013, I actually started working on my research right after we had our meeting. The first sub-project was to run a literature review on the area of interest for the PhD. This would give me a head-start in terms of positioning my research a whole year before enrolling officially. And I thought, I’d also get a feeling about enrolling into a PhD that’d help to “fail fast”, essentially I could call it a “quit” if I didn’t like the work before enrolling. Also, I switeched my job to join SourceFire, now part of Cisco, because the traveling projects with ThoughtWorks weren’t working for me anymore. I wanted to be with my wife, and international travels were screwing up with my Canadian immigration and citizenship requirements.

Life was still quite easy till the April of 2013. Then, we had our first born and it changed upside down. I think like most other first time parents, we had little idea about how much work it is to raise a little child on your own without substantial family support. To make things a little more challenging, the deadline for the first paper I wrote was about two weeks after Shopoth was born. In that two week, we barely got any sleep. But Shahana was adament, and let me take the time away to complete the paper on time. It felt good to be able to submit the paper. We both felt that it was doable.

Fast forward about a month. So, Shopoth is about 7 weeks old. My mother had left after helping us with the first few weeks of Shopoth’s life. We , the three of us, went for a walk for the first time around the beautiful tree-laden neighborhood of the varsity community in Calgary. I recall precisely, it was a beautiful early summer day. We just crossed a playground and turned left on a corner. My phone beeped signaling an email notification about the paper I submitted. It had this:

We regret to inform you that your submission titled …has not been accepted…

I felt sweaty, and wanted to keep it to myself. I could only do so for a few hours, and then shared this with Shahana. She was still recovering from the c-section, and was under a lot of stress due to the sudden changes in her life. I remember, she was saying only positive things about it and tried to encourage me to carry on.

As it happens, I moved on and decided to get into the PhD program anyway. During this year, I also got quite a bit engaged with things at the new job and started taking more responsibilities from a leadership perspective. To complete the course requirements, I’d have to complete at least three graduate courses, and transfer the credits from one of the courses I did while doing my MBA before coming to Canada.

Most people eventually come across a time when they’d need to provide names of colleagues as references while applying for a new job. The referees are often involved in the last stage of the hiring process, when the employer is generally satisfied that the candidate is a potential hire. At this stage, the employer expects to find a few things:

1. Verification of facts. The employer wants to confirm the validity of the information such as the role, responsibilities, and achievements provided by the candidate about their past experience.
2. Qualitative feedback. The employer wants to understand the “cultural fitness” aspect of the candidate by asking questions related to the individual’s interaction and collaboration skills.

While these are valid goals, I personally find the practice of using references to achieve these goals to be a rather flawed one. Here’s why:

1. References aren’t replacements for interviews. As an employer, the responsibility is on the employer to be smart enough to verify that the skills and experiences as advertised by a candidate’s resume reflect the reality. If the employer can’t do that with a reasonable amount of confidence, they’re likely to hire wrong candidates anyway.
2. Likely outdated. If a candidate is currently employed, due to the secret nature of most job searches, they are unlikely to use one of their coworkers as the referee. As a result, the longer a candidate is employed by the most recent employer, the more outdated the references tend to get.
3. Likely tampered. If a candidate wants to use a referee, they’d generally contact ahead of time with the referee. Even if they don’t contact, the candidates are likely to mention names of people that are almost certain to say good things about them. For any qualitative feedback, this setup limits the possibility of getting any negative traits of a candidate.
4. Open to interpretation. Reference checks are often done over phone. Also, the person calling to check the reference often hasn’t met the candidate during the interview steps. This creates a double-blindness of sort. So, anything said during the reference check process suffers from possibilities of information gap.

In my opinion, if reference checks are used, they should only be used to verify factual information. That too, should be done keeping in mind that the information may be outdated, or intentionally biased.