These are the slides with comments of my conference talk at MedConf 2012 in Lucerne:
Software development without using open source software (OSS) is unthinkable in today’s world. This holds also for medical technical projects.
This presentation is split into three parts. First we take a look at some myths and facts about open source software. Then at how to integrate an OSS into your project so it can be validated against regulatory requirements. And finally, we’ll see what is important when selecting open source software.
During my career as a software engineer, I got used to use open source software a lot.
But when I started in my very first medical technical project, I was confronted with strong opinions about open source software.
I was told that they don’t meet the quality needs. It was unthinkable to use some sloppy written code of someone who has nothing better to do in his spare time than coding.
And that it would be insane to rely on a piece of software for which there was no commercial support. Overall, using open source software would just be far too risky.
Okay, I said, but let’s take a look at some facts.
First, we need to take a look at what kinds of software we actually have in our project.
There is the code we wrote ourselves. Then there is the code we bought – typically as closed source. And there is open source code.
There are even some mixtures of these kinds. For example there are commercial libraries, which offer you the source code, too. Or there are open source projects that offer commercial support. But for simplicity, let’s use these three types of source code.
Now, let’s take a look at the myth that the quality of open source software is not as good as the quality of our own code or of commercial libraries.
The question here is: What does actually influence quality?
The main driver for quality is …
… feedback.
The more feedback a projects gets about defects and its usability the better its quality gets.
But this is a big problem for the code we wrote ourselves. There is much less feedback than commercial or open source projects get. Simply due to the fact, that they are used in more projects, by more developers and users.
Domain know-how of the developers is another big influence on the quality. Only when the developers know what they are building, quality can get really high.
Obviously, our own team cannot know everything. So there are aspects in our software where we lack domain know-how. Or does your team know how to write a scheduling component for example?
That means, that the quality of commercial or open source software in special areas is much higher than when built by ourselves. Commercial and open source projects focus on a single topic and therefore gain domain know-how on this single topic.
Transparency helps to assess quality. When you can look at the code, or see how many defects are in the defect tracking system, you can decide for yourself whether a library or product is good enough for you.
Most commercial products don’t give you access to the source code or the defect tracking system. Therefore, they are a black box and hard to assess.
In open source software, as in your own code, you have full access. You can run some quality metrics against the code and check how many open defects lie around in the defect tracking system and how long it normally takes the team to fix a defect.
Finally, quality is a result of motivated teams and innovative solutions.
People working in open source projects are likely highly motivated. They spend their spare time!
The people involved in your team or from the company you buy software from may be a bit less motivated.
The coverity scan (http://scan.coverity.com/) of the year 2011 states that “Open source quality for active projects in Coverity Scan is better than the software industry average.“
To sum up, if you are looking for quality in your software, try not to implement it yourself.
Another myth is that you don’t get any guarantees with open source software.
So what happens if you need help with a defect? You probably don’t get a support contract.
It’s like with your own code. There, you have to support yourself.
With commercial products, you can rely on the support. But far too often, support is slow, and you’ll have to wait for the next release for your problem to be fixed.
With open source software you get the best of both worlds. Either you can take measures on your own and fix the defect yourself, or rely on the project team and the community to fix the problem for you.
Although you don’t have a contract in hand, open source software gives you flexibility to act on your own, or wait for the project to take care of defects. The latter is in most cases the cheaper variant.
An analysis of the Food and Drug Administration shows that from the recalled medical devices between 1992 and 1998, 7.7% were caused by software failures.
79% of these software related recalls were caused by software defects that were introduced when changes were made to the software after its initial production and distribution.
Therefore, if you are looking for quality, externalize maintenance and profit from the feedback of everyone else using the software.
When using open source software in your projects, there are other things you need to consider, too.
The license of the open source software you want to use can have a big influence on how you have to distribute your software.
If you for example use a GPL license, all derived work has to be published under GPL, too. Therefore, avoid GPL open source software if you want to build commercial software.
On the other side, projects licensed under Apache 2.0 require you only to include the license and information from where you have the code, somewhere in the installation.
Unfortunately, there are a lot of different licenses. You may need a lawyer to make sure that you don’t run into any problems with some of these.
Another aspect you have to be aware of is copy right and patent violation.
In your own code, it’s up to you to make sure that the developers don’t violate any patents or copy rights.
This also applies to most open source projects. But there are foundations that take care of these questions, like the Eclipse and Apache Foundation.
And if you use open source software that is used by a wide variety of other companies, it’s unlikely that someone will succeed with a patent lawsuit.
Regarding risk management, writing your own code, buying a commercial product and using open source software have all their advantages and disadvantages.
Regarding legal aspects, commercial products are best for you (they would have a problem and not you). Open source software has a bit less risk than writing your own code because of the foundations we have seen before helping to mitigate these risks. And you are probably not alone.
The risk of bad quality is highest for your own code. Mainly because of the lack of feedback compared to commercial and open source software. Commercial products and open source software with bad quality likely won’t survive very long.
There is always the risk that the product you use is no longer maintained. This is more likely in open source projects. But at least, you have the source code and could continue using it. If a commercial product is dead, your only option is to leave it. There is also an end of life in your own code. If know-how about your code is sparsely distributed in your team, a team member leaving could have the same effect.
Open source software could get commercial. However this happens quite rarely and you could still continue to use the latest open version of the product. Or of course, buy the new version. Obviously, the risk of getting commercial has already occurred for commercial products.
Implementing your own code is for sure the most expensive variant if there are commercial or open source products available that solve the same problem. Therefore, the financial risk is highest for your own code. The financial risk is lowest for open source software because you get the software for free. However, it still costs to integrate and learn an open source software.
The risk that the solution you built or bought is only a short-living solution is highest for your own code. That’s because of less feedback compared to commercial or open source software. A short-lived solution forces you to invest effort soon because it does not support future development.
Finally, there is the risk that the roadmap or release cycle of the product you chose does not match your needs in the future. Open source software gives you however the possibility to act on your own and adapt the source code yourself. This can be very important regarding your own deployment cycles.
Overall, open source software is a strong choice regarding risks. Especially, if financial aspects have a heavy weight for you.
Looking back at my first medical technical project, once we convinced the developers and project leaders that open source software can save us a lot of money and trouble, the guys from the regulatory department stepped in.
They said that they don’t want any open source software, how should this software be validated?
The most important point is that you take the source code of the open source project and add it to the source control system where your own code lives.
This is the only way you will know in the future, which version of the open source software was used in which of your releases. And maybe the open source project dies and you cannot download the sources anymore.
Furthermore, you will have to maintain your software probably for a long time. This can be a problem with commercial products. Maybe the product does not support the latest operating system anymore. With open source software, you can adapt the code yourself as a last resort.
Regarding validation, it can be a great advantage that you can fix defects yourself. You could do this in a way so that only a minimum of code is changed and therefore minimizing validation effort needed.
It’s very unlikely that you get support for validation from the open source project.
To validate the open source software, you should treat it just like your own code. Do whatever you do with your own code to get it validated.
The advantage of open source software is that it is not a black box to you.
You can review the code and if the open source software contains unit or acceptance tests, you can use them, too.
Additionally, you could even include the source code when submitting your project for validation.
When using open source software, make sure that you use an …
… up-to-date version.
Documentation and help from the community is normally only available for up-to-date versions.
And you can only profit from externalized maintenance if you update regularly.
Good open source projects provide you with tutorials how to upgrade from one version to the next. Therefore you shouldn’t be behind more than one version.
Looking back at my first medical technical project, it took some time to get support from all involved parties. But finally, open source software could be used in great variety.
But there is still one problem to solve:
How to find the right open source software.
First of all, the open source software should solve a problem of yours. But that is not enough.
You should check whether the project has a road map so that you can decide whether the project runs in the right direction. Take a look at news, releases, commit and the number of committers to see whether the project is alive. Check whether there is an issue tracker and how many defects are in there. And whether these defects get fixed in a reasonable amount of time.
Technically, you should check whether you can build the sources so you could fix defects yourself. Check whether there are unit and acceptance tests and whether the code style suits you.
And very important, check if there are any external dependencies and if they meet your requirements.
In order to get you up to speed quickly, take a look at the…
… documentation. Are there a wiki, web page and samples. Are they up-to-date? Do you find success stories of other people using this software?
Is there a community that supports the OSS. Is there a forum? Are questions answered? In a reasonable time frame?
It takes quite a lot of time to get a picture of an open source software. But it will save you lots of problems when done thoroughly.
To sum everything up, I’d like you to think about …
… whether open source software could help you in your projects to get better quality, less maintenance or to finish your project earlier before beginning to write your own code.
And if you use open source software …
… please provide feedback.
