The Hands On Engineering Manager Paradox

The current economy appears to be breeding a crop of job descriptions looking for people that will program, create systems architecture, speak the business language of upper management, hire the engineering staff, manage the staff, put together an engineering budget, setup the servers, etc. To put this politely, not going to happen. Or at least not for very long.

I certainly have been in situations were I was doing all of these things. However, realistically not all of them very well. Sure, in a very early stage start up you might need to ask someone to initiate multidisciplinary tasks. However, no one is going to be good at all of these tasks. Setting an expectation that someone should be able to handle them all could have negative long term implications for the individual being asked to perform these tasks and for the organization as a whole.

There are fundamental differences between the skill sets to manage an organization and to build systems. There are a few people on the planet that are versatile enough to flip back and forth between these modes of operation. However, even for these people, the work load gets to a point where there needs to be focus to establish any real progress on any of the tasks.

I understand why a business owner would want to have all of these tasks encapsulated in a single individual. Budgetary constraints, investor pressure to control staffing, the institution of a lean and mean startup culture to name a few.

In some cases the business owners that are looking for these interdisciplinary individuals do not come from a technical background and do not fully understand the focused concentration level required to create a system that scales, is reliable and can be understood by the rank and file technical staff. The last thing you want this person to do is to be spending time managing the day to day operations of your engineering group when they are deeply involved in getting your system designed, built and up and running. Conversely, if an individual is beavering away programming, configuring servers, installing hardware, etc. and not paying attention to day to day engineering organization management the entire development process could come to a screeching stop.

The practical consequences of proceeding along with this model are as follows:

Someone Else Is Managing The Engineering Group - If you are asking a programmer or systems architect to manage your engineering group you will wind up having another person in your organization stepping in and managing the day to day operations. This could be one of the co-founders, product or project manager or an executive staff member.

Your Early Design And Systems Have Issues - We all remember the Friendster debacle. This system could never adjust to the popularity of the service. It fundamentally lacked an architecture or any real supported systems structure. Was the engineering managers asked to do everything resulting in a lack of focus on a reliable sustainable systems architecture?

Staff Churn and Organizational Dysfunction - What a bad way to start a company. You will be facing enough challenges trying to get your business up and running the last thing you need is a bunch of frustrated engineers wanting to quit because nothing is working right and they are being mismanaged.

The stress of a struggling system and a disorganized engineering group stresses out other parts of the organization. This can lead to some serious confrontations between various individuals and groups.

So how do you avoid the downside of a single engineering manager simultaneously executing managerial and technical tasks and still stay within the budget?

Engineering Team Management Oversight - In the early stage of the business or organization establish someone with management skills to oversee the engineering group. This could be someone on the executive team or it could be an outside consultant. Have this individual work with the engineers to staff and organize the group, deal with personnel issues and act as a liaison for the engineering team to make sure their concerns and issues are voiced. Conversely, this individual will communicate management directives to the engineers in way that they can translate into concrete action.

Engineering Process - Establish a development process that is well undertood by all members of the organization. Make sure that process matches up with the business goals of the company. In a start up or small company realize that just about everyone in the organization is part of the development process

Architecture and Platform - Clearly state the business objective of the company and obtain a satisfactory systems architecture plan to achieve the objectives. Get outside advice if necessary and entertain a number of options. Once you establish your platform and architecture they are very hard to change.

Short and Long Term Objectives - You may be in a mad rush to get something up a running. This is understandable just make sure you do not fool yourself into believing that a rushed system is extensible and sustainable. It will most likely have to be replaced or significantly modified to run your business. Do not blame your engineers for getting something up and running quickly and then castigate them later when they tell you the system will have to be changed to accommodate your growth.

Engineering Organization Staff Planning - Be prepared monetarily and organizational for what it takes to run a real company. In the early stage of your company you may have a few programmers and consultants getting you going. This will be short lived if you are moderately successful. Have a good understanding of what it will take to effectively run an engineering group. Anticipate the need for an engineering manager, architect, DBA, programmers, QA, product management and IT support.

Crisis Management - Watch for signs that the engineering group is in a perpetual state of crisis management. If nothing appears to be going right or you appear to be exerting a tremendous amount of time and energy to get tasks completed you are in crisis management. Take the time to determine what organizational or technical issues are causing the problem. Address this issue immediately before you develop a perpetual crisis management culture.

Focus and Prioritization - Of all the tasks and items you want the hands-on engineering manager to perform determine the most important priority for that individual and clear the deck for them. Get the high priority items complete. Do not let the manager become distracted. This is especially important for technical and architectural tasks. Ping ponging back and forth between tasks is exhilarating but very dangerous if you are building a team and a system.

In conclusion, the current investment market places pressure on engineering organizations to consolidate functions and tasks into single individuals. This is understandable and a reality. However, do not fool yourself into believing that managing an engineering organization is not a full time responsibility requiring multidisciplinary skills not often found in a single individual. Understand the implications and compensate for the challenges an organization will face if the engineering managers is expected to be multidimensional. Prepare for the future realizing that eventually the organization has to build an engineering organization to be successful.

When The Agile Development Process Does Not Address The Business Model

I recently built a development team and development process around a very aggressive business model. The model could be described as a “marketing” centric model as opposed to a “product” model. In the midst of the organization and infrastructure building exercise I explored various development processes including XP, RUP and Agile. During the later stages of the organization’s growth I focused specifically on Agile to address some of the quality, speed to market and communication issues the organization was experiencing. After a lot of study and some trial and error it became obvious that Agile development was not the right fit. Much of this misalignment resulted from the forces being placed on the development organization by the business model and operational expectations associated with this model.

Short Cycle Development – The organization had an implicit expectation that changes to the production system were required on a daily basis. This translated into several pushes to production in a 24 hour period. Often product changes were pushed on Fridays and over the weekend.

Detailed Specification Requirements – The business owners wanted to know specifically what each feature would look like and were not interested in releasing “iterative” versions to incrementally add functionality to the application. This is directly correlated with the short cycle expectations and releasing product on a daily basis. With short cycles the product was difficult to break down into even smaller iterative bytes.

Quality Control - Quality was enforced by a QA department that worked from a detailed specification. Quality checking and an understanding of the change came after the initial development cycle using the specification as a guideline. Thus, the "waterfall" process at the backend of the process.

Aversion To Infrastructure Investments - The notion of investing in building a unit testing infrastructure was not a high priority. It was important to get product out the door so any investment in development resources not focused on direct product releases was discouraged. Also, the unit test infrastructure would have been best built at the initial outset of establishing the applications architecture. Coming back retroactively and adding this to the platform would have been very difficult.

Organizational Structure – This organization was a startup and leveraged individuals over a number of different projects for the purpose of preserving capital making it difficult to form true product teams where QA, PM and Development collaborated in defining, developing and testing the application.

The Scrum – It was hard to form a real scrum for a single release or product. The scrums turned out to be product update meetings on all outstanding projects. These were good meetings but not true “scrum” sessions. They were also hard to pull together because of the waterfall culture of the company. People had a tendency to meet and strategize within their organizations rather then cross organizationally.

In the end the development process evolved organically from the culture. It turned out to be a hybrid of RUP using Use Cases to cover usage models. Waterfall process was used to move the products through the cycle. The PM’s would define and build requirements; Development would build and submit to QA for testing and eventual release.

Was this the optimum process for this organization? Not really, releases were not always released on time. Some of this was due to the extreme need for speed leading to mistakes in specifying the features. Quality remained an issue because of the pressure to release and to the general lack of understanding of the impact of a change on the larger system. On the other hand rapid a short cycle release culture was institutionalized allowing for a rate of change to the production system that was relatively very impressive. Relative meaning, in relation to other development processes that dictate a more scheduled and less frequent release cycle.

In retrospect, one of the biggest mistakes made was to not build a unit test infrastructure early on in the architecture and design of the system. This would have allowed for a higher quality release to production while still maintaining a short development cycle culture.