Tuesday 27 May 2014

The Siemens corporate research blunder[The end of machine learning in my life]

Days were going smooth at Informatica. Life was on track. After working for more than 3 years with the team, rarely I got something that I found difficult. When you're mind is not so busy it wanders a lot. My mind did too.
I started feeling what I was doing in this reputed product development company was not challenging enough. The guy in me was not satisfied. He wanted more...a lot more. Appraisal came and I got a salary hike of 18% and was promoted to Lead Software Engineer-6th year of my career-a good enough time considering Informatica was my third company. Strange I was not happy with my appraisal. 18% was not good enough for me. I felt underpaid and under learned considering 6 years of experience. That machine learning worm bit me again. I started reading articles about how cool data science is and how cool the data scientist are. I created my profile in LinkedIN-tweeked it a little to look good for a data science role.
Anyways I even applied to few jobs but nobody came back. Strange enough I was not even good at basic statistics and I dreamt of being a cool data scientist.

I changed gears-"Big data" is hot these days..I should try getting some serious research level work in Big data. After all I was the one who converted those data transformations in Informatica to map reduce. I have the experience with hadoop and that is what big data is all about. May be I can read a little "mahout". May be I can learn some "NLP" or "Data mining"-they looked more approachable to me than a hard core data scientist.
So I started applying to those "Big Data" jobs and I did recieve a few calls. Unfortunately I could not clear interviews from amazon, inMobi but I did clear a couple of startups who were willing to offer me a job with no hike in my salary. I could not summon up enough courage to join them. After all I had a family to support-what if they shut down?

Days passed and I developed a new fantasy-I want to work in Informatica, US. It's been 3 years and I was eligible for applying for relocation. I approached my manager and let him know. He convinced me a little but I was adamant. I reached out to all big guys including the hiring manager in US who had positions. He demoralised me and asked me to work in my current team for few more years. I was angry and sad and became very resentful of Informatica- a company that gave me a lot.

Days passed and one day I got a call from [24]7 iLabs for technical lead position.I thought of blackmailing Informatica if I get the offer.
I had a good interview with 247 and they said they will offer me what I expected-near 20% hike. I got a call from HR and he said my domain is little different from theirs and they might consider me for SMTS. I was ok since it was some data science + engineering work I was willing to explore and the company is known to use predictive analytics/Big data. If Informatica sends me to US I am good else I will join 247

I received their offer with almost no hike in my current salary(except that they made everything as fixed + 1 lakh joining bonus). Anyways I decided I would join them if they offered me work in the area of so called "Big Data". I discussed the work with architect who told me that they were hiring me for distributed infrastructure platform teams where I would be working mostly on non functional requirements like sacalbility, reliability, performace, high availability etc. That did not make sense to me and I rejected the offer.

Time passed and then one day I applied to Siemens corporate research. They had an outstanding job description related to research in the area of big data technologies - apache spark, hadoop, yarn, cassandra etc etc.Also they said 'Phd desirable' for the position. I immediately applied for the job without expecting any call from them.One fine day I got a call from the manager at Siemens research who said he wants to talk to me. I said ok and he started the phone interview without informing me prior. He asked difficult computer science questions. He asked me floating point representation and about 2s complement, 3s complement , statistic questions -something that I don't think ppl asked these days in interviews. I was impressed. However I could not impress him much. Finally he said he would discuss and let me know. I did not expected him to revert.

A few days later I got a call from Siemens research. They asked me to come and give a presentation on one of the projects that I did in Informatica. I went to their office- gave a confident presentation on the best project I did in Informatica. I had a couple of tech interviews after that that mainly concentrated on theoretical DBMS and some basic programming. Overall the interview was easy. I prayed to GOD in desperation that this is the ultimate opportunity that must be given to me.

Days passed anxiously. After a few days I got the offer from Siemens and I was happy.Again it was not a great offer monetarily-exactly same as [24]7. But I was happy that now I can pull off my last weapon-threatening the manager that I would leave if I am not given opportunity for US. I was happy that even if I don't get the opportunity atleast I would be a researcher working on some cutting edge technologies.

I went to my manager and told him that I have decided to leave. He was shocked and honestly I was too. INFA was a second home to me. My life was good here. He tried convincing me a lot but I was adamant. They informed the manager in US about my decision to quit if not given opportunity for US. Ultimately nothing happened and I left Informatica.

I joined siemens-the first 3 days were induction. The gut feeling was not good at the first day. This was a pretty big company-too much crowd and looked like a university with big canteen and free breakfast and dinner. I joined my research team which was busy preparing for a conference about Big Data in coming days. There was a lot of enthusiasm. Few asked me as to why I joined Siemens Research from Informatica. I felt awkward answering that. I soon realised the place was not the way I perceived it to be. It was a team with no clear focus on things-they had big data and cloud computing as their research area. However the research was just 20% of their portfolio and rest belonged to consultancy projects that they used to do for their own development team. I soon realised that they were struggling to convince the higher management about value addition of using distributed computing and big data for their projects. People were not buying their ideas. I soon realised that the company policies were stringent that for getting a necessary software installed you need to go through approval cycles.I realised that the company is not inherently into software. They are an industrial automation and healthcare equipment company to which software is just a part of operation. They don't sell softwares. Coming from a good product company I felt I was in some kind of TCS or infosys. I was frustated and felt cheated. Even their hiring standards were not upto the mark. They mostly hired M.Techs and few PHDs from a local college called - IIIT Bangalore. A couple of PHDs already left the company and a couple of them stayed with no idea what to do.

In desparation I contacted Informatica to take me back. I went to informatica to dicuss with a manager of another team. He asked me if I was still on US thing to which I honestly relied as Yes. Nevertheless they decided not to take me back and I  felt so stupid. Now the company I worked for 3.7 years and where I had high respect for my work denied to take me back.

2 months passed in all this. I felt so stupid...I soon realised that If I stay in this place for 1 year..nobody is going to take me and the only option that I would have is to either stay in company or be some kind of freelance trainer...honestly this so called research unit of siemens was the worst career decision I took..one fine day while I was thinking of my position-it suddenly struck to me that I still have an offer from [24]7.

I called them in desperation and they told me that they still would love to take me. I resigned from siemens and after 15 days of notice period relieved.

It;s been 4 months here at [24]7 at platform team. My team work on large scale distributed infrastructure for their chat software. I am happy with what I am doing and would be doing here. Beside me sits the Data Science team who use machine learning for building models for predictive analytics. I don't see them as a different league anymore.

I am content with what I am doing. This marks the end of machine learning in my life.