I had an opportunity to interview Ivar Jacobson as he was preparing to come to Australia to deliver a Keynote address for the SoftEd's Software Development Conference (2009). Ivar is truly an icon in the IT industry having pioneered use-case techniques and been a key contributor to UML and RUP.

Personally, I have had mixed experiences implementing process improvements and here was an opportunity to discuss the trends in developing software solutions. The world had changed from the early methodology bookcases, and even RUP is sounding a little passé. So what, I wondered, comes next? What works? Here’s how our interview went, with my questions in bold:

What is the relevance and role of development methodologies today when there is so much packaged software and outsourcing/off-shoring is commonplace?

Most companies have a lot of legacy and now we have the cloud, with Software as a Service in the cloud. So we have software that is globally developed but still have to integrate it. But this software was not designed to be integrated – so we still need to do some kind of development even if you don’t do a lot of coding.

So we developed the technique of System of Systems, where systems can be a package or an existing system or something in the cloud or whatever. We still need to identify interfaces, and understand how they integrate - some kind of requirements, maybe using use-cases at the higher-level.

Basically the way to deal with that is top-down and bottom-up. Bottom-up meaning you have to start with what you have, but if you only start with what you have, you will never get to what is in the best interests of the business.

So you also need to go top-down which means you look at the business processes and determine what kind of software you need. So you have to balance top-down and bottom-up approaches and reach a compromise but some cases you still have to develop something new.

So it is not the same problem as you have when you develop a singular application, it is not the same as Greenfield development. But much of development is not Greenfield anymore; mostly it is starting from something so it is instead minor changes, but changes that are enough to make the organization successful.

I have been speaking about this for ten years, so it is not really something new. When it comes to outsourcing, it is a similar problem but other ingredients and I have written an article that goes into that in more detail.

But even with the growing realization that there is little "Greenfields Development" today.... do you think we have changed our approach. Have our methodologies kept up with the times?

The approach of System of Systems I talked about has many different names today. It talked about as EA, it’s talked about as SOA and in some spaces it is called PL methodologies. People use many other names for the same thing. But in reality it is seldom that companies have successfully implemented them.

There is a lot of talk of EA but there is a lot of implementation pain around the world. The same thing with SOA, there are a lot of problems with it.

Basically the problem is that they both try and do too much. They make it too big and try to solve too big a problem. EA fail for two main reasons, there are a lot of reasons but these are two. They create a lot of paperwork rather than focusing on execution, things that can run and be valued. Two, to implement EA you typically have to work across different layers of the EA, layers like the data model, the applications, and another layer underneath all of that, like security that is cross-cutting (or shared) pieces. If you develop these layers separately then it takes a lot of effort to integrate that. So the only way to really make sure they are integrated is to make sure that the completed software works across over all layers.

So basically in a simple world what I would do would be to layout an architectural roadmap, which is not a big task even for a huge system. An architectural roadmap where the enterprise systems, the sub-systems, the reused components, the high-level components that you need ....everything then I would take a project, and most of the projects are customer facing, meaning that you do something that is valued by the customer. If you do that, and you select the right project, then you will exercise everything in the whole architecture, the key components, even at the lower level components. So you develop the EA not by doing a lot of upfront work and then going down to implement it in a waterfall manner.

Instead you have mini-projects that will exercise all layers, and use components at all layers and deliver value to the users -very quickly, because we don’t have time to wait to deliver value.

I also have some other issues with EA, but not just EA, but also new systems. The way we buy and sell, which relates to outsourcing..... If you build a house it probably needs to have some requirements, to have a specification. Then you go on to build the house. But that might work there but it does not work in our industry because you really cannot say when the requirements are done. No user has the ability to say confidently this is what you have to build -because, the only way to know for sure is to be using the product. That means the only way to work is to work iteratively. Working with small projects and from every project you learn and then comes a time when you say now I have enough I can actually give you the requirement.

Listening to you describe the how projects need to exercise the different layers of the enterprise architecture reminded me of how use-cases were designed originally to exercise an object-oriented system. Is it that IT solutions are so sophisticated today that you now talk of projects as in some ways a higher-order abstraction of a traditional use-case, with the enterprise now replacing the system? Can you elaborate on that?

An EA is not simply architecture; it is a lot of things. When we go to deliver it we have to deliver both the architecture and the executables underlying it. In a presentation a few years ago I went through all the different interpretations of EA - as we say in Swedish: "A cute child has many names." Or the other way around, there one name with many meanings, in this respect. So the use cases have to be written clearly so they can be coded and tested, so when you have a project on the architectural roadmap you get something that is really useful. You really can deliver something useful and you start with use-cases and go through the different layer and identify new components, new frameworks, old frameworks, whatever, all the way down to code. So you deliver the EA through a series of projects. You grow your architecture project by project.

That is a good entré into Agile..... You are known to have some strong views on Agile. Can you tell us what concerns you about Agile Projects?

I am not negative to Agile; on the contrary, we all need to be agile. Agility should be in everything we do. The only concern I have now is that everything is agile is everything is agile now. Basically the meaning of agile has disappeared, the true meaning of it. The only thing I am negative to is that now the meaning of agile has been so removed it has come to mean everything good in software. What I am negative to ... is that software development is a fashion industry. We are creating fashions faster in the software industry that in the clothing industry.

Ten, fifteen years ago we were all running after objects, then it was components, then UML then it was RUP then, two years or so ago it was Extreme Programming - the latest silver-bullet - and now it is SCRUM. None of these have everything but they have something. And we need a bit of all of them, in some balance. This is my biggest problem. Universities have not done a great job in helping us to get a good theory for software development. We don't know something basic; we don't know what it is to develop software in the more general sense. So that means that as soon as something new comes we change everything, we change terminology, everything. We love something new, and whatever we had in the past was wrong. We don’t tolerate people who used the older methods ......

When I see the CV of someone and I see something like - I have a PhD in Computer Science and I am a Scrum-master. One take about 7 years to get and it takes a couple of days to become a Scrum-master and then you have to $2,500 - and you do that with sixty other people. That is a wonderful business, but in a few days how much more can you know that you did not know before.

So what should we do as an industry?

My deep concern is that the software industry does not know enough of what it does. We have tried to identify the process kernel, those core services that every methodology has. The key concepts - for example, we always have an implementation stage, we always have a backlog because it can’t all be done. So we have identified twenty-plus things that we always have in development projects. But they can look different dependent upon the way you want to do things. So if you use for instance iterative development, you will have different ways of doing these things that are in common - compared to if you use user stories or if you use use-cases driven development. We have different ways of doing basically the same the thing ... but that is Ok, at least we can know what is it that we need to do. Whether we document it, or not, it is not a completely different dimension and we can add a little bit at the end of the documentation for different kinds of products or projects, dependent upon what is needed. But what is interesting is that we always do a number of things.

I would like to see more research in the reuse of software engineering practices. In that way it would be harder to come up with new hype, because we can relate it to something we understand, so it does not become just another silver bullet. It will happen eventually.

The problem we have is that few researchers have really developed software, and universities have to teach what the industry needs. The industry has to have it but then we don’t teach software in a way that is lasting. There are universities that teach industry courses over five years, and that's fine, but the problem is that the students only learn one way, just one way of doing software. They have no real fundamental knowledge of what software engineering is. So if something new comes along they cannot defend what they know, so they throw out the baby with the bath water. Instead of expanding what we have, using what they have and improving it - and building on it. I have a blog entry (http://ivarjacobson.wordpress.com/2009/03/) on this subject.

Do you want to make any concluding remarks? Any last words for our readers?

There is just one topic, one thing I'd like to repeat. We need to get some feeling for what software engineering is really all about and to understand the different practices in a formal way. So these practices can be easily compared, so we can integrate them and compose them and so on to make them work. This is what I see missing and it is the problem we are tackling ourselves at Ivar Jacobson International. We are doing it in a pragmatic way, but it should be formalized.