"Systems Development": a New Discipline for a New Education

Do university programs adequately prepare software developers for today’s world of work? Now that Agile software development is being adopted at an unprecedented rate, developer skillsets particularly need broadening. In this article Dr. Dave West discusses a nascent discipline, “Systems Development”, which emphasizes humanity, craft, design, creativity, innovation, and emergence - in contrast with current university computing disciplines, infused with a value system centered on formalism, control, engineering, and determinism. In his view, Systems Development should be called “Reality Construction” ⁱ because it is based on the fundamental premise that everything a Systems Developer does is necessarily integrated with, and necessarily changes (reconfigures), an existing complex system - the human, organizational, socio-political, and cultural system we call reality. West speaks from experience when he proposes a better educational experience that replaces the sterility of today’s classrooms and labs with the workshop, or “bottega” – the approach he is currently using in his program at College of Santa Fe.

Our profession remains broken.

Yes, great strides have been made to improve process, quality, and professionalism and, in some cases we are able to deliver fantastically complicated reliable and usable software. Nevertheless, as an industry we still struggle to achieve a 30-40 percent “success” rate and an even lower “success within time and budget” rate.

Numerous factors contribute to the general malaise: false metaphors (engineering); ambition (we already solved the easy problems); pace of change (technology and the world); and blurring of boundaries between technical and human systems.

A major contributor is the widening gap between what we learn in school (and how we learn it) and what is required of our professional selves.

It is almost certainly past time that we defined a discipline that better fits what we actually do and create an alternative mechanism for educating individuals in that discipline.

Limning the Territory

The ACM-AIS-IEEE Computing Curricula 2005 Overview Report ⁱⁱ outlines a complicated overlapping set of related disciplines that include: Electrical Engineering, Computer Engineering, Computer Science, Software Engineering, Information Technology, and Information Systems.

Of the six, the first three (EE, CE, and CS) focus on the machine and the theory behind the machine. (Part of CS concerns itself with the virtual machine via operating systems and language compilers).

Software Engineering and Information Systems approximate our profession while Information Technology focuses on the infrastructure especially networks, security, and administration.

Nascent in the ACM document is the beginning of a definition of another discipline – Systems Development (SD).

Systems Development incorporates skill sets from several of the ACM classifications: programming from computer science; software, process, and tools from software engineering; and an applications focus from information systems.

To this mixture, SD adds: an extended knowledge of systems (General, composite and complex), a deeper understanding of the history and philosophy of computing and science, significant understanding of business and management, and multiple topics in philosophy, mathematics, anthropology, art, humanities and more.

All six of the ACM computing disciplines are infused with a value system centered on formalism, control, engineering, and determinism.

In contrast, SD emphasizes humanity, craft, design, creativity, innovation, and emergence.

Systems Development should be called “Reality Construction”^iv because it is based on the fundamental premise that everything a Systems Developer does is necessarily integrated with, and necessarily changes (reconfigures), an existing complex system - the human, organizational, socio-political, and cultural system we call reality.

SD delivers altered states of reality.

Mastery

The ideal architect should be a man of letters, a skillful draftsman, a mathematician, familiar with historical studies, a diligent student of philosophy, acquainted with music; not ignorant of medicine, learned in the responses of jurisconsults, familiar with astronomy, and astronomical calculations

Vitruvius 25 B.C.

Vitruvius’ architect had to be a polymath – had to have a significant degree of competence in all fields of extant knowledge with significant skills in specific areas.

The same is true of a professional (Master) Systems Developer. (With the caveat that no one, today, can be conversant with ALL extant knowledge.)

The path to mastery passes through several stages:

1. Rote understanding of concepts and vocabulary.
2. Applying the knowledge under the direction of others.
3. Applying the knowledge when working alone.
4. Applying the knowledge in a novel context.
5. Ability to mentor others.
6. Ability to teach others (groups as well as one-on-one).
7. Ability to make an original contribution or extension to a given area of knowledge.

A professional should be able to exhibit stage 4 abilities for the entire body of knowledge and stages 5-7 for a significant subset.

Excepting stage one, mastery is always based on one’s ability to actively do something, not simply and passively know something. This bias is an echo of Giambatisa Vico (1668-1744) who asserted, “We are sure only of that which we build.”

All Extant Knowledge

The two previous sections suggested that the domain of knowledge, and the degree of mastery, for System Developers is rather expansive.

While that knowledge domain is not literally, “all extant knowledge,” it is significantly greater – at least double – the breadth of knowledge expected of university graduates with a Masters degree in any of the ACM curricular categories.

I cannot detail the contents of the knowledge domain in this space. [In part because it is still being defined and refined in preparation for opening the first SD Bottega in late spring 2009.] Instead I would like to move the discussion to modularization and learning environment.

It would be impossible to deliver the scope of content envisaged for Systems Developers if it were modularized in accordance with the typical credit hour / course / semester / academic year tradition. Doubling content would mean doubling the time required to complete a degree from 6 years to 12 – something that would never happen.

Instead of credit-hours and courses, we propose dividing the knowledge domain into much finer grained discrete knowledge units. When we did this at New Mexico Highlands (see below) we used the default term competency to represent a knowledge unit.

The metaphor of a mosaic is useful to illustrate knowledge units (competencies) and the relationships among them. The mosaic represents the entire domain of expected knowledge – each discrete tile represents an individual competency. It is important to note that each “tile” represents more than knowledge – it also includes aspects of practice (application of the knowledge), skill, and experience.

As is true with any picture, the mosaic exhibits themes – patterned relationships among subsets of the tiles comprising the mosaic.

Each tile has an associated color, indicating the discipline or field most associated with the knowledge represented by that tile. Units of philosophy, anthropology, literature, mathematics, history, science, etc. - are distributed across the entire mosaic, showing at-a-glance the relationships of knowledge across disciplines and achieving a degree of interdisciplinary integration impossible in a course-based program.

The mosaic provides an immediate gestalt view of the entire domain to be mastered. Themes ^v provide focus without sacrificing relationships to elements outside of that focus. Colored tiles show contributions from all disciplines – in context, with cross-disciplinary relationships obvious. Students not only know that they have to know Z, they know why, and they know how to apply Z as they learn it.

How big is this mosaic – how many tiles?

At New Mexico Highlands (experience report below) we defined a little over 500 competencies for the SD “major.” That number did not include liberal arts and other required courses.

A reasonable estimate, based on both the Highlands experience and ongoing efforts to define the set of competencies for SD, yields a mosaic with 1,000 – 1,500 tiles.

Although few would deny the possibility of that many tiles (as educators we are constantly battling the need to add more subject matter to our curriculum); most would raise the purely pragmatic objection – you cannot put that much breadth and depth into a curriculum and expect anyone to master it in the 6 years typically allotted to completing a Bachelors plus Masters degree program.

This objection, however, is not grounded in any understanding of how much or how fast individuals can learn. Instead, it is based on assumptions of how much learning you can pack into a 16-week course that meets for 3 50-minute periods per week.

How fast individuals and teams can learn a given body of knowledge is not unlike the problem of how fast an individual or team can complete a software project. Agile development suggests the mosaic (a collection of tiles) be considered akin to a ‘project backlog.’ Given a time-box of arbitrary duration, each student selects a set of competencies from the product backlog to mastered in that time interval – a ‘sprint backlog.’ At Highlands we call the sprint backlogs, individual learning plans (ILP) and used a time-box interval of 6 weeks.

Because all but the first stage of mastery requires ‘doing,’ it is essential to tie the ILP to actual project work, preferably real-world project work.

Each student determines their individual “learning velocity” relatively quickly. It is then a simple matter to plot how long it will take to complete their education.

Learning velocity is not simply a matter of individual difference. The proper environment can make a huge difference.

Bottega (the classroom)

Imagine replacing the sterility of today’s classrooms and labs with the workshop – a bottega ^vi - where Leonardo worked and studied.

• A “storefront” where goods and services are produced and delivered to paying customers.
• A workshop simultaneously engaged in the craft, in building the tools and discovering the techniques that advance and support the craft, and teaching that craft to apprentices.
• A place noisy with multiple projects and activities; walls and benches covered with works in progress.
• A place filled with the tools of the craft.
• With room for lounging, and eating facilities as well.
• An intellectual center that is a “must visit” for masters, scientists, and thinkers visiting the area. A center overseen by local masters and journeymen.
• A fountain of innovation and creativity!
• An environment and atmosphere that is very self-consciously multi- and inter-disciplinary; that mixes theory and practice almost without differentiation.
• A place full of music, conversation, laughter, and even loud argument.
• A place to share food and drink, perhaps sleep, and even love ^vii.

A bottega consciously mimics the ideal workspace advocated for agile software development: open, lots of whiteboards and communication artifacts, movement, collaboration, and socialization.

To be maximally effective, students need to spend large blocks of time in the bottega; ideally most their time.

The bottega operates as a “one room schoolhouse” and everyone, regardless of degree of mastery, is in the same place and exposed to everything in that environment. If something occurring in one part of the room sounds interesting or relevant, the student is free to roll their chair across the floor and participate.

In this kind of environment a lot of knowledge is absorbed non-consciously, almost by osmosis. When the time comes that an individual needs to consciously acquire and apply a bit of knowledge, that process is accelerated by virtue of the reservoir of non-conscious “background learning” they have already completed.

Personal responsibility, deadlines, and understanding and satisfying customers are all aspects of being a master. Commercial product development provides the educational and experiential opportunity necessary for students to achieve those aspects of mastery.

Can this idealized environment be created, maintained, and can it work?

Experience Report – New Mexico Highlands University

In August, 2004, 34 students, freshmen through graduate students, entered a unique program in Software Development at New Mexico Highlands University.

The degree program had no courses. Students were expected to master 500+ competencies in their “major” plus the usual general and liberal arts course requirements in order to graduate with a Bachelor of Arts degree.

The program was apprenticeship-based: everyone in the program was able to earn money working on real-world software delivered to paying customers with real-world timelines.

Fifty percent of the students were female (extraordinary for a computing degree) and 60+ percent were minority, and almost 80% were economically disadvantaged.

Two full-time faculty, Pam Rostal and Dave West, designed and operated the program. Internationally known master developers (e.g. Linda Rising and Ron Jeffries) visited the program, working with students (pair programming) and contributing their expertise via hands-on mentoring and conducting other types of learning sessions.

All development work and instruction took place in a “one-room-schoolhouse.”

Our studio environment included: island work areas, wall-to-wall whiteboards, hard floors and wheeled chairs. Couches, a refrigerator, and a microwave provided some nominal but much used amenities.

Significant time was spent fostering “culture and community.” Ceremonies were used to initiate people into the community and advancements in apprenticeship levels. Pot-luck meals (sometimes catered meals) were incorporated into our iteration retrospectives and ceremonies. Colored shirts denoting apprenticeship level, and the ubiquitous presence of the program’s logo, assisted in creating a sense of identity and belonging.

X bar times 10 alluding to the goal of graduating professionals ten times better than average. Software is our medium, Craft (Mastery) is our goal, People are our focus, Systems is our perspective, and Agility is our process.

Our freshman retention rate was 100% between first and second semesters at a University that averaged less than 50% for the same interval. Forty additional students – mostly freshmen – had committed to enter the program in the fall of 2005.

Seven students in the inaugural class were offered full-time employment by the end of the first year, 2 were accepted to graduate school, and one was able to concurrently fulfill requirements for a Masters degree in computer science.

Eight students co-authored papers and delivered presentations at refereed conferences, OOPSLA in Los Angeles and Agile in Denver.

Lessons learned from the first (and only ^viii) year the program was in place, include:

• Students learned far more, in shorter time. Based on this experience we believe that the average entering freshman will be able to complete the entire SD curriculum (twice as much learning as the average 6 year education) in less than five years.
• The depth of learning and understanding exceeded expectations.
• Students rapidly evolved a process of self-directed learning.
• Students selected, installed, and configured their own tools.
• The “strong sense of community,” “human values (software by humans for humans),” and the “open collaborative learning environment” were the factors women students consistently mentioned as the primary reasons for entering and staying with the program.
• Student’s learned teamwork and project management skills concurrently with technical skills.
• Students managed all aspects of all projects, not faculty.
• Computer Science faculty that had opposed the new program became active participants the second semester.
• Students averaged 30 hours a week in the studio.
• The “noisy” environment was not a distraction – it was an asset.
• The quality of student work was comparable to professional work – according to our real-world paying customers.
• Even the most junior apprentices made valuable contributions to their projects. They did not inhibit or slow down the work in any way.
• Both Pam and Dave averaged 70+ hours a week in the studio. (This time commitment would be unsustainable in the long run.)

When the program was cancelled, students organized a letter writing campaign, arranged a meeting with New Mexico Governor Richardson’s staff, and solicited endorsements from some of the nationally prominent individuals they had met at conferences. This was to no avail, but it reinforced the conviction that the program should continue, and efforts were initiated to replicate the effort at another college.

A more detailed report on West’s NMHU program was published on InfoQ in 2006: Experience Report: Agile Development Apprenticeship at NMHU.

Moving Forward – Transcendence Corporation, The College of Santa Fe, and the Santa Fe Complex

Transcendence Corporation was formed during the Highlands experience – to be the contracting agent for the apprenticeship program. (The University was forbidden, by state law, from entering into that kind of work contract.)

The College of Santa Fe is a small (less than 2,000 students) liberal arts college with very strong programs in the fine and performing arts.

The Santa Fe Complex offers a bottega environment, expertise in math, science, visualization, agent-based modeling, applied complexity, and the “intersection of art and science.”

A first approximation of the program described above was established at the College of Santa Fe in the fall, 2007.

In the fall of 2008 the existing program will be redefined as a five-year competency-based Master of Science program.

During the fall 2008 and spring 2009, the three organizations will work cooperatively to:

• Convene a workshop of interested educators and professionals to finalize the full “curriculum mosaic” discussed above.
• Offer bottega style targeted workshops for students at the College and advanced placement high school students.
• Recruit an inaugural cohort of 50 students including at least 10-15 high school juniors and seniors and 10 “transfers” from the College.
• Solicit projects (we offer off-shore rates and, based on the Highlands experience, higher quality); internships, and externships.
• Establish sponsorship relations with organizations – especially national and international consulting companies – that cannot recruit suitable graduates and have been forced to create their own in-house boot camps and training programs.
• Open the first bottega in late spring / early summer of 2009.

Anyone interested in working with us in this effort, (especially the curriculum workshop, contracting for project work, or sponsorship relations), is expressly invited to contact the author.

About the Author

Dave West has been a professional software developer for too many years to count, in roles ranging from programmer to Director of IT. For the past fifteen years he has split his time between consulting and academia. Currently Professor of Business, Innovation, and Technology at the College of Santa Fe: formerly at New Mexico Highlands University and the University of St. Thomas. His own education was an eclectic mix that included Asian Philosophy (undergraduate at Macalester College), Computer Science and Cultural Anthropology (graduate degrees at University of Wisconsin-Madison) and a Ph.D. in Cognitive Anthropology (also at UW-M). He is the author of Object Thinking (MS Press Professional, 2002) and is nearing completion of a second book on “reinventing software development.” Any expertise he possesses is focused in the areas of objects, agility, design, innovation, and change.

Notes and References