My first end-to-end programming project — time of celebration and humility
Yesterday, I released the first draft of a project that I was working on for the last few months. It is for one of my clients which is an UN organization headquartered in Geneva — it is something that I conceived during my last visit to Geneva in February.
My client had a large manual of financial policies and procedures as a Word/PDF document that was updated every year. We thought of extracting the contents and making it available in the Finance helpdesk portal so that can easily access any rule. This would provide a number of advantages as the latest version of the policies is always available which were now being sent by email bursts. When the agent respond to their clients, they can easily link the relevant section of the document. We can analyze how many times different policies are viewed, linked in the responses to queries, etc — there were a number of potential benefits. So, it was an exciting idea to automate the entire process as manually creating these pages would be a very time-consuming task.
At the time I conceived the idea, I was not even sure whether it was possible technologically to implement idea. So, I checked with an experienced programmer to evaluate technical feasibility. Python had a package for extracting Word document and we did some quick together and it all seemed possible and easy. Initially, my intention was to outsource the work to the programmer but he was not available for a number of months. Given that I was learning Python, I thought of this as a way to challenge myself to apply my learning.
As in any real-life situation, the devil is in the details. As I extracted the data and compared it to the contents of the original documents, we found a number of issues — the python package was dropping hyperlinks and the associated text and scores of other issues. I did spend quite a bit of time researching, trying out various strategies, and even trying out 2 other packages in python. But the new packages seemed to solve the old issues, but created other new issues. In the end, it looked very bleak — I was coming to the realization that there was no way to do this unless I was ready to directly read from the Word doc without using any package and deal with all the complex parsing logic, etc. Given that I was quite new to python, this was daunting to say the least.
But mammoth came with its own interesting twist — mammoth would take the Word document and spit out a HTML page. Now, my job was to figure out how to parse the HTML document and to split that into different folders and pages and upload them through the API to the helpdesk system.
Again, I faced this challenge. Here I was — just a beginner who has reached his first plateau of comfort in a new land and I had to write a parser to parse HTML documents. Honestly I did not even know where to start.
To add to the pressure, I had a back injury which put me in bed for nearly 2 weeks, increasing the pressure to deliver in even shorter time than what I had. When I came back to work, I could barely sit in the chair for 30 minutes continuously without getting back-pain. So, I had to take frequent breaks and work much lesser hours. The client was eager to get this project rolled out and the expectations were high. So, this increased the stress on me even more. Last week during the weekly progress call, I committed to show them something next week.
What happened in this last one week is nothing short of amazing — lot of coincidences that led me to calmly make progress and reach the goal post despite all heavy odds against it. I would get stuck and keep working calmly doing what I could do using small incremental steps despite self-doubt and enormous pressure as I was feeling that I was no way going to meet the deadline. And then something would open up either in the form of some links I clicked as part of my research, or some random idea would pop into my head and I would try it out. Bit by bit, I made progress. As I look back, I am totally in awe of the divine guiding force that kept guiding me to make small incremental choices to move forward. It might look like genius, but if not for the constant support of these signs, i would have never even made beyond the first few steps. Since I believe in God, I see it as God’s hand. But even people who do not yet believe in God often see such signs in a lot of their endeavours.
Finally yesterday, I delivered the first draft release and the client was very happy. For me, it is a feeling of immense satisfaction as this is my first end-to-end programming project conceived and delivered by me using what I have learnt in the last 9 months. It has been 16 years since I have left my job as a programmer to purse MBA and then pursuing a number of other careers. But in all these years, I have never lost my love for programming. So, this project gave me the opportunity to do nearly full-time what I would love to do, even I don’t get a penny out of it. I am so grateful for this journey and the joy of living doing the things I love and serving my clients. Al Hamdu Lillah — all praise and immense thanks to the Almighty, All-wise and All-merciful.