It is a very macho and cool thing to brag about a few engineers getting together with the latest web technology such as Python and Ruby on Rails and cranking out a prototype web site and platform to get a business up and running on the internet. The theory is who cares what we develop now we will toss it anyway if and when the business takes off. Who knows if this business is even going to make it. Why waste our time thinking about a robust platform now. Let’s get something up a running.
There is some merit and truth to this strategy but it has some very real downsides if a business evolves from this first “prototype”. What happens if there is real merit to the business model? Without any consideration for an architecture that is scalable and extensible the development team and the company can find themselves in real trouble right when you need the business to grow and address the demands of the marketplace. Look at some of the popular social networks. They are terribly unstable and at times do not work at all. These are companies that actually have deep pockets and big engineering staffs. Imagine a company that is not as well funded and dealing with similar issues. This situation can become a real nightmare and can potential undermine the business. This is called changing the “engine and wheels on the car” when you are moving at light speed. Thrust me you do not want to find yourself in this situation if at all possible.
To avoid this problem during the early startup phase of a company the engineering staff should put some thought into what if this business is successful scenario. What can we do now assure that the business will be successful without causing too much of a slow down on getting the early version up and running.
The following are some areas to be considered.
Development Platform: Just because a programming environment is easy to use and the cool web 2.0 platform dejour does not necessarily mean it is going to help you when you have more than three developers and 4 hits a day to your site.
If you are remotely successful you will need to employee a caching strategy to decrease hits to the DB, tune and configure the web servers for memory management purposes and integrate with a number of web services API’s, integrate with a build and deploy environment.
Make sure your platform will give you this flexibility. Pick a platform that has proven to be robust, has a large development community supporting it, has a large pool of developers in your area that are using it, supports object oriented principles and has proven to be scalable.
Development Methodology: Set ground rules for coding standards, naming conventions, methods/class definition and usage, database development strategy and responsibility, common library creation, source code tree structure, source code management, etc. Think about the future. For example, in my most recent assignment we started out with a monolithic code base that required the entire code base to be committed for a change to any part of the code base. This eventually resulted in grid lock when multiple projects were being worked on with different lifecycle and delivery dat requirements. Break up you application into discrete sections that will allow for various development efforts to move independently.
The Database: This may be the most important decision you make!!!! It is all the rage to go the open source route, MySQL, Postgres, etc. These platforms have their merit but don’t pick them just because they do not have a licensing fee. Web applications are characteristically two tier DB centric applications. It is all about the data. In choosing a DB find out what they are good at and what they are not. Do they support stored procedures and triggers? Do they support clustering? If so, does it work and is anyone using it. How do they handle high transaction volume? How well do they replicate and backup. Does back-up activity have a significant impact on the production database? Are you going to protect your data? If so how? Do you need to warehouse your data for reporting reasons. What is you redundancy strategy? If the main DB goes belly-up who are you going to call? Will you be prepared. Does the provider have a good support staff.
Security: What is the security strategy for your website, data, passwords, etc. The last thing you want is being in the middle of a big ramp-up in traffic and have your site hacked. This is devastating from a PR and practical perspective. Establish coding standards that emphasis security. Use a security scanning service that scans your site daily to determine if there are any vulnerabilities? Determine what data needs to be encrypted. Who will have access to what in your system? Lock the environment down as best you can.
Reporting: This is an area that is frequently not taken seriously by developers but is a big deal from a business perspective. Find out what the business owners need to determine the health and welfare of the business. Make sure that your data structures, tags, events page views, naming conventions, are structured is such a way that the business owners can get what they need out of the reporting system. If you are contemplating using an off the shelf web tracking tool make sure it is going to give you what you really need. In many cases they only check page views and do not help you determine how people are navigating though your site. In most cases the data accumulated from these applications are hard to combine with customer data you are accumulating in your DB. If you are going to use a sophisticated product like Omniture make sure you understand what comes out of the box and what requires “consulting” expertise. The consulting can break the bank.
Hosting Center: Outside of having to change DB’s in mid stream this is the second biggest risk. You do not want to have to move from one hosting center to another. Have a good plan for what hosting will look like in a successful scenario. Can you architect your application in such a way that will allow you to leverage virtualization of servers and the use of services supplied by existing application operators such as Amazon and Salesforce.com. It would be great if you could take full advantage of another outfits experience and expertise. Do you need hands on system engineering resources to manage your servers? If so, will you have enough of these resources when the company is firing on all cylinders? Leverage what is going on in new hosting models to decrease your staffing needs and to decrease your overall hosting costs. Try to speculate what your bandwidth needs will be. Different hosting models have a different pricing structure and bandwidth is one of the costs that are played with in a hosting contract.
These are some of the obvious things you should engage in early in the development process to make everyone’s life easier. Make sure you properly sell this investment in infrastructure and architecture and development time to the co-founders and to the board. They will resist any investment that does not have immediate benefit. However, they are usually savvy business people and will get it if you explain the risk management aspect of the investment. In the end as a developer your life will be a lot better and you will be able to concentrate on the things you most like to do, which is coding, if you think about these items. More importantly the business will be able to grow rapidly and without significant engineering redesigns.
Showing posts with label Development Process. Show all posts
Showing posts with label Development Process. Show all posts
Saturday, September 20, 2008
Tuesday, September 16, 2008
Motivating Software Engineers
Motivating engineers is a touchy feely subject that engineers usually do not engage in. It is very important from an organizational perspective but by the very nature of the engineering discipline it is not a topic that gets the attention it deserves. This is a very important subject because most companies base their projections and business model on the underlying technology that supports the business. When engineers are motivated productivity increases, problems are solved more efficiently, turnover over is reduced, staffing becomes easier and ideas are generated that have a positive impact on the products a company produces. Motivating engineers is not the exclusive domain of the engineering manager. It is the responsibility of executives, managers, other departments and individuals within the organization to take into consideration the engineering perspective when establishing corporate culture, company events and business goals.
Clarity Of Purpose: Engineers like clarify and specifics when it comes to task assignments. The very nature of software engineering requires everything to be boiled down to a 0 and 1. This creates a culture of specificity that translates upward into goal setting. This clarity is not limited to product specifications. The engineers will want to know why they are engaging in a project, what is the end game and what is the organization trying to achieve from a business perspective. This will translate to buy-in and communication within the group and assure that the engineers understand the overriding goals of the project.
Focus: Software engineering is a tedious occupation that requires constant attention to detail rarely found in other occupations. It requires intense concentration during a development cycle to get a product developed properly. If the focus of the project is shifting, the goals keep changing or the definition of the project is in a constant state of flux then engineers will become frustrated. Commit to a project, stay the course, release the project and measure its results. Certainly change is a reality for all organization. However, if an organization is constantly changing product focus eventually this will lead to doubt about the decision making capabilities of the management team. Changing a products focus in mid-stream is especially demoralizing to an engineering team. Planning, design and coding will be wasted. No one wants to see an effort wasted on a consistent basis.
Early Stage Input To Process: Frequently engineers are brought in late to the development process. Goals are already set, business objectives are stated, some product requirements are established and assumptions are made about the time it will take to deliver a product. The engineers are then expected to react to these details and assumptions and provide technical details on how and when they are going to deliver. Engineers will see this as a disregard for their input. This lack of involvement results in a “why are we doing this” or “they really do not know what they are doing” attitude. Bring the team into the process early to understand goals, objectives, provide input and to get collective buy-in to the product.
Feedback: Both positive and negative feedback is welcome in a development group. Of course, you always have to be careful with negative feedback. When negative feedback is given it should be delivered in such a way that identifies an area that the engineer has control over and could have done a better job. Positive feedback should be given on a regular basis to make sure people understand what they are doing well and to reinforce the proper behavior.
Transparency: Keep the engineering team abreast of business activity, issues the company is facing, new strategic direction and activity in other departments. Engineers are bright people and naturally analytical. In the absence of information they will analyze and make assumptions. With only limited information they may make the wrong assumptions and translated them to the rest of the engineering organization. The engineers want to feel that they are a vital part of the company. If the management team is making decisions that impact the engineering organization they should make the engineers aware of these decision.
Game Play and Geek Culture: The use of the word geek is not meant to be derogatory. Engineers have their own way of communicating and having fun. In one of my assignments the engineering team had regular times during the week where we would play online multiplayer games such as World of Warcraft. The CEO would also participate in game play. This was a great way to bring the team together and to blow off steam. It was one of the better events that we had in the group. Warcraft was a game that developers could relate too. It was special to them and helped bind the group in conversation and healthy competition. In a more recent assignment we played online poker. Poker is not a developers game per say but it is a multiplayer game and does have the benefit of engaging other departments in a group effort.
The Power of Food – This may sound strange but the power of food and communal feasts is a way to keep engineers engaged and happy. Engineers spend hours and hours sitting in front of a computer monitor. Sometimes it helps to get them up and engaged with a larger community in a non confrontational way. A weekly or monthly food feast with various departments is a great way to facilitate communication and to collaborate in a casual setting. In addition, just having a regular snack and food stash is a great way to allow engineers to have a way of getting nourishment during the day. Engineers are often sequestered for hours and in some cases days working on a project. Not having to worry about how they are going to sustain themselves is a nice to have. It also demonstrates that the organization understands their situation and is supportive.
Engineers Love Challenges: Engineers are very competitive. This may seem strange because most of them may not engage in regular competitive sporting events. However, this does not mean that they are not competitive. They are intellectually competitive and want to be challenged in that way. They like to participate in team competitions. The concept of scrums or collaborative development is a manifestation of a team competition where a group works together to achieve goals. Engineers also like to solve complex problems and be recognized for these achievements. Managers at all levels of the organization should appreciate when a business problem requires a significant degree of intellectual horsepower to solve. This effort should be acknowledged.
In conclusion, motivating engineers is important for the health and welfare of organizations dependent on a technology platform and or product for the success of the organizations. Engineers are different culturally and intellectually. They require special attention to maintain their enthusiasm and commitment to an organization. Engineers are competitive and can also be social in their own way. The challenge for any organization is torealize these attributes and to explicitly address them to assure the success of an organization.
Clarity Of Purpose: Engineers like clarify and specifics when it comes to task assignments. The very nature of software engineering requires everything to be boiled down to a 0 and 1. This creates a culture of specificity that translates upward into goal setting. This clarity is not limited to product specifications. The engineers will want to know why they are engaging in a project, what is the end game and what is the organization trying to achieve from a business perspective. This will translate to buy-in and communication within the group and assure that the engineers understand the overriding goals of the project.
Focus: Software engineering is a tedious occupation that requires constant attention to detail rarely found in other occupations. It requires intense concentration during a development cycle to get a product developed properly. If the focus of the project is shifting, the goals keep changing or the definition of the project is in a constant state of flux then engineers will become frustrated. Commit to a project, stay the course, release the project and measure its results. Certainly change is a reality for all organization. However, if an organization is constantly changing product focus eventually this will lead to doubt about the decision making capabilities of the management team. Changing a products focus in mid-stream is especially demoralizing to an engineering team. Planning, design and coding will be wasted. No one wants to see an effort wasted on a consistent basis.
Early Stage Input To Process: Frequently engineers are brought in late to the development process. Goals are already set, business objectives are stated, some product requirements are established and assumptions are made about the time it will take to deliver a product. The engineers are then expected to react to these details and assumptions and provide technical details on how and when they are going to deliver. Engineers will see this as a disregard for their input. This lack of involvement results in a “why are we doing this” or “they really do not know what they are doing” attitude. Bring the team into the process early to understand goals, objectives, provide input and to get collective buy-in to the product.
Feedback: Both positive and negative feedback is welcome in a development group. Of course, you always have to be careful with negative feedback. When negative feedback is given it should be delivered in such a way that identifies an area that the engineer has control over and could have done a better job. Positive feedback should be given on a regular basis to make sure people understand what they are doing well and to reinforce the proper behavior.
Transparency: Keep the engineering team abreast of business activity, issues the company is facing, new strategic direction and activity in other departments. Engineers are bright people and naturally analytical. In the absence of information they will analyze and make assumptions. With only limited information they may make the wrong assumptions and translated them to the rest of the engineering organization. The engineers want to feel that they are a vital part of the company. If the management team is making decisions that impact the engineering organization they should make the engineers aware of these decision.
Game Play and Geek Culture: The use of the word geek is not meant to be derogatory. Engineers have their own way of communicating and having fun. In one of my assignments the engineering team had regular times during the week where we would play online multiplayer games such as World of Warcraft. The CEO would also participate in game play. This was a great way to bring the team together and to blow off steam. It was one of the better events that we had in the group. Warcraft was a game that developers could relate too. It was special to them and helped bind the group in conversation and healthy competition. In a more recent assignment we played online poker. Poker is not a developers game per say but it is a multiplayer game and does have the benefit of engaging other departments in a group effort.
The Power of Food – This may sound strange but the power of food and communal feasts is a way to keep engineers engaged and happy. Engineers spend hours and hours sitting in front of a computer monitor. Sometimes it helps to get them up and engaged with a larger community in a non confrontational way. A weekly or monthly food feast with various departments is a great way to facilitate communication and to collaborate in a casual setting. In addition, just having a regular snack and food stash is a great way to allow engineers to have a way of getting nourishment during the day. Engineers are often sequestered for hours and in some cases days working on a project. Not having to worry about how they are going to sustain themselves is a nice to have. It also demonstrates that the organization understands their situation and is supportive.
Engineers Love Challenges: Engineers are very competitive. This may seem strange because most of them may not engage in regular competitive sporting events. However, this does not mean that they are not competitive. They are intellectually competitive and want to be challenged in that way. They like to participate in team competitions. The concept of scrums or collaborative development is a manifestation of a team competition where a group works together to achieve goals. Engineers also like to solve complex problems and be recognized for these achievements. Managers at all levels of the organization should appreciate when a business problem requires a significant degree of intellectual horsepower to solve. This effort should be acknowledged.
In conclusion, motivating engineers is important for the health and welfare of organizations dependent on a technology platform and or product for the success of the organizations. Engineers are different culturally and intellectually. They require special attention to maintain their enthusiasm and commitment to an organization. Engineers are competitive and can also be social in their own way. The challenge for any organization is torealize these attributes and to explicitly address them to assure the success of an organization.
Wednesday, September 10, 2008
When The Agile Development Process Does Not Address The Business Model
I recently built a development team and development process around a very aggressive business model. The model could be described as a “marketing” centric model as opposed to a “product” model. In the midst of the organization and infrastructure building exercise I explored various development processes including XP, RUP and Agile. During the later stages of the organization’s growth I focused specifically on Agile to address some of the quality, speed to market and communication issues the organization was experiencing. After a lot of study and some trial and error it became obvious that Agile development was not the right fit. Much of this misalignment resulted from the forces being placed on the development organization by the business model and operational expectations associated with this model.
Short Cycle Development – The organization had an implicit expectation that changes to the production system were required on a daily basis. This translated into several pushes to production in a 24 hour period. Often product changes were pushed on Fridays and over the weekend.
Detailed Specification Requirements – The business owners wanted to know specifically what each feature would look like and were not interested in releasing “iterative” versions to incrementally add functionality to the application. This is directly correlated with the short cycle expectations and releasing product on a daily basis. With short cycles the product was difficult to break down into even smaller iterative bytes.
Quality Control - Quality was enforced by a QA department that worked from a detailed specification. Quality checking and an understanding of the change came after the initial development cycle using the specification as a guideline. Thus, the "waterfall" process at the backend of the process.
Aversion To Infrastructure Investments - The notion of investing in building a unit testing infrastructure was not a high priority. It was important to get product out the door so any investment in development resources not focused on direct product releases was discouraged. Also, the unit test infrastructure would have been best built at the initial outset of establishing the applications architecture. Coming back retroactively and adding this to the platform would have been very difficult.
Organizational Structure – This organization was a startup and leveraged individuals over a number of different projects for the purpose of preserving capital making it difficult to form true product teams where QA, PM and Development collaborated in defining, developing and testing the application.
The Scrum – It was hard to form a real scrum for a single release or product. The scrums turned out to be product update meetings on all outstanding projects. These were good meetings but not true “scrum” sessions. They were also hard to pull together because of the waterfall culture of the company. People had a tendency to meet and strategize within their organizations rather then cross organizationally.
In the end the development process evolved organically from the culture. It turned out to be a hybrid of RUP using Use Cases to cover usage models. Waterfall process was used to move the products through the cycle. The PM’s would define and build requirements; Development would build and submit to QA for testing and eventual release.
Was this the optimum process for this organization? Not really, releases were not always released on time. Some of this was due to the extreme need for speed leading to mistakes in specifying the features. Quality remained an issue because of the pressure to release and to the general lack of understanding of the impact of a change on the larger system. On the other hand rapid a short cycle release culture was institutionalized allowing for a rate of change to the production system that was relatively very impressive. Relative meaning, in relation to other development processes that dictate a more scheduled and less frequent release cycle.
In retrospect, one of the biggest mistakes made was to not build a unit test infrastructure early on in the architecture and design of the system. This would have allowed for a higher quality release to production while still maintaining a short development cycle culture.
Short Cycle Development – The organization had an implicit expectation that changes to the production system were required on a daily basis. This translated into several pushes to production in a 24 hour period. Often product changes were pushed on Fridays and over the weekend.
Detailed Specification Requirements – The business owners wanted to know specifically what each feature would look like and were not interested in releasing “iterative” versions to incrementally add functionality to the application. This is directly correlated with the short cycle expectations and releasing product on a daily basis. With short cycles the product was difficult to break down into even smaller iterative bytes.
Quality Control - Quality was enforced by a QA department that worked from a detailed specification. Quality checking and an understanding of the change came after the initial development cycle using the specification as a guideline. Thus, the "waterfall" process at the backend of the process.
Aversion To Infrastructure Investments - The notion of investing in building a unit testing infrastructure was not a high priority. It was important to get product out the door so any investment in development resources not focused on direct product releases was discouraged. Also, the unit test infrastructure would have been best built at the initial outset of establishing the applications architecture. Coming back retroactively and adding this to the platform would have been very difficult.
Organizational Structure – This organization was a startup and leveraged individuals over a number of different projects for the purpose of preserving capital making it difficult to form true product teams where QA, PM and Development collaborated in defining, developing and testing the application.
The Scrum – It was hard to form a real scrum for a single release or product. The scrums turned out to be product update meetings on all outstanding projects. These were good meetings but not true “scrum” sessions. They were also hard to pull together because of the waterfall culture of the company. People had a tendency to meet and strategize within their organizations rather then cross organizationally.
In the end the development process evolved organically from the culture. It turned out to be a hybrid of RUP using Use Cases to cover usage models. Waterfall process was used to move the products through the cycle. The PM’s would define and build requirements; Development would build and submit to QA for testing and eventual release.
Was this the optimum process for this organization? Not really, releases were not always released on time. Some of this was due to the extreme need for speed leading to mistakes in specifying the features. Quality remained an issue because of the pressure to release and to the general lack of understanding of the impact of a change on the larger system. On the other hand rapid a short cycle release culture was institutionalized allowing for a rate of change to the production system that was relatively very impressive. Relative meaning, in relation to other development processes that dictate a more scheduled and less frequent release cycle.
In retrospect, one of the biggest mistakes made was to not build a unit test infrastructure early on in the architecture and design of the system. This would have allowed for a higher quality release to production while still maintaining a short development cycle culture.
Subscribe to:
Posts (Atom)