Open source software guideline
Introduction
This guideline provides information and advice for Queensland Government departments to consider when using and releasing open source software (OSS). These guidelines are non-mandatory and are for information only. While some information communicates other mandatory obligations, which may be relevant in the context of existing policy or legislation, departments are strongly recommended to further investigate these obligations in light of their own business requirements. The information within these guidelines does not constitute legal advice, and departments are encouraged to obtain the necessary legal advice prior to using and/or releasing Government-owned OSS.
OSS products are becoming increasingly available, with a range of mature supported products that are suitable for use by agencies to achieve business outcomes.
By making Queensland Government in house software available as OSS (free of charge where possible) we support the:
- cultivation of trust with Queenslander's and our partners
- leveraging of our existing investments within government and industry.
OSS can give agencies better control over information technology. Several public institutions have transitioned to free and OSS. OSS can ensure citizen data is handled in a trustworthy manner since proprietary software may not allow total control over the employed functions of applications.
Purpose
This document guides Queensland Government agencies intending both use and/or release Queensland Government-owned software as open source software.
The Queensland Government endorses the use of appropriately licensed OSS products within the Queensland Government and the release of open source software where copyright resides with the State of Queensland.
Agencies should also consider OSS in the context of the Leveraged: Share before buy before build principle located in the QGEA foundation principles document.
Audience
This document is primarily intended for:
- Solution architects
- Software designers
- Software developers
- System analysts
- Technical development managers
- Software licence managers
- Software procurement officers
- Intellectual property coordinators
- Information security managers
- Commercialisation managers.
Background
Digital trends and client expectations for transparency have provided opportunities for organisations (including governments) to implement open before closed practices in their business. This is evidenced by an increasing number of open data repositories and digital dashboards. Another area for consideration is the release of government-owned source code as OSS.
The popularity and adoption of OSS products cant be ignored. Gartner reported ninety-five percent of mainstream IT organizations leverage nontrivial open-source software assets within their mission-critical IT portfolios, whether they know it or not (for example, indirectly through commercial proprietary software using OSS libraries).
In 2016, the USA Government published a federal source code policy. This policy mandates at least 20% of custom source code developed by USA federal agencies must be released as OSS. In addition, the policy requires all source code be shared between agencies.
Benefits
The expected benefits of Queensland Government supporting the use of OSS products and open source software development across government, and in industry-supplied open source software applications, include:
- reusing existing developed software components to build new solutions (avoid reinventing the wheel each time)
- improving quality and function of code through best practice in issues management, collaborations among skilled developers, faster debugging and code forking
- broadening the base for sourcing ongoing support
- broadening usage within and across industries and wider range users
- increasing ease of interoperability from wider use
- increasing sharing of common and potentially, higher quality code
- increasing agility and faster open source software development times, including the ability to modify and create tailor-made specialist application
- increasing competition with proprietary products, potentially providing economic and business benefits
- making it easier for all companies to bid for government contracts
- reducing the high cost of proprietary software packages
- facilitating sharing through the development of a modern, agile, efficient, and resilient digital platform, that can grow as business user needs change.
Risks
Potential risks may include:
exposure of security weaknesses and other vulnerabilities where appropriate support processes are not in place or commercial support has not been obtained
- loss of commercialisation opportunities if using OSS components with licenses that prevent re-packaging for commercial use
- expectation for government to provide formal and ongoing support to external users of the code.
While this document considers the different risks of releasing open source software, agencies need to determine the risks in the context of their own business environment including documenting such risks and seeking approval from the appropriate authority.
Use of OSS
There are many advantages to using open source, from agility, flexibility, speed and cost-effectiveness to name a few. All acquisition and management of open source software products must be in accordance with the Queensland Procurement Policy requirements, the Procurement and disposal of ICT products and services policy and the Software asset management policy.
All open source software use, modification and distribution are to be in accordance with its licence conditions.
While there are a number of business models for OSS, the two different types of use this guideline will cover is:
- commercially supported this is probably the most common way government utilises open source software, as support is provided by a commercial entity, and risk is monitored and governed. Examples include use of OSS products and platforms for Database, Workflow, Storage, Identity, Integration, AI & Machine Learning frameworks.
- self-supported as the name suggests, support activities (and subsequently all risks) need to be actively managed by the agency.
Regardless of the type of use, it is strongly recommended that prior to the use of OSS products, a risk assessment be undertaken and signed off by an accountable officer.
Risks/considerations
There are a few risks that come with open source software and if overlooked, could be costly to fix further down the track. The following table looks at the risks across both the commercially supported and self-supported approaches
Commercially supported | Self-supported | |
|---|---|---|
Security Management | The ownership of security risks for the deployment of the software and hosting environment is not something that can be transferred to a commercial vendor providing only the software product for your own installation and associated software updates & support. As such it is important that security governance and a risk management process be put in place to ensure that the business risks are understood It is also the responsibility of the agency to understand the level of protection that a solution needs, particularly when solutions handle sensitive information and will require a greater level of controls. It is therefore important that agencies have a clear understanding of the security approach of a vendor, and ensure security expectations are incorporated into the contract. This includes regular, identification and resolution of vulnerabilities, regular security monitoring and provide security alerts. Similarly, agencies should ensure that any controls are implemented to the satisfaction of the agency. Where solutions will store personal information, it is vital to ensure the software is secure to ensure obligations under the Information Privacy Act 2009 are met. | The advantage of OSS can also be its vulnerability. Open source projects make their code available to anybody. This includes all varieties of hackers, white hat, grey hat and black hat. This has the advantage that the open source community can flag potential exploits they find in the code and give open source project managers time to fix the issues before publicly revealing information on the vulnerabilities. However, eventually these open source software vulnerabilities are made publicly available on vulnerability databases for anyone to view. Malicious hackers can target organisations that are slow to patch the applications. This was the approach taken by black hat hackers when they used publicly available exploits to attack Equifaxin 2017 when the credit reporting agency exposed the personal details of 145 million people. The attack was successful because the black hat hackers noticed Equifax used an older version of the open source Apache Struts framework and used that information to their advantage. Dealing with this risk from the organization perspective means recognising that open source exploits are made vulnerable and that hackers stand to gain a lot from attempting to breach services that use vulnerable components. Update as quickly as possible or pay the consequences Open source software offers the advantage of being able to view the source code, information flows, have more control over data, undertake security testing, and adapt solution to ensure it meets agency privacy obligations. Where solutions will store personal information, it is vital to ensure the software is secure to ensure obligations under the Information Privacy Act 2009 are met. |
Licence /Contract Management | A contract will generally cover the supply of product and services, and this will include any OSS products. As such, not only will contract management be important but also license management, as contracts can also include not only access to code repositories but also the need to adhere to any licence terms and conditions or end user service agreements of software inclusions. | All open source software come with a licence, and the terms and condition of the licence must be adhered to. Check the terms carefully as the terms and conditions can vary (there are over 200 different types of licences), with some licences being more permissive than others. Licence management can be even more complicated where multiple open source components have been used in a single product, and each component could have a different licence. As such its vital to understand the license terms and conditions for all your open source components. |
Quality Management | Purchasing support from a vendor means that they will the quality management this includes providing certificated releases which are maintained, tested and supported. Often there is a community edition, and the re-packaged commercially supported version which takes enhancements for the community and tests them, packages them into releases. For example, KeyCloak is the community edition which nightly updates, where as Red Hat Single Sign-on is the supported edition with a specific product release candance. However it is important that when the commercial vendor has released any changes, that the agency will apply such updates as soon as possible following their release. | We want our open source products to be of high quality and be stable. Measuring quality is difficult due to the collaborative nature of OSS, and no agreed standards. The open source community is made up of people with different skill levels, different involvement levels, and different time commitments. However, there are a number of aspects you can check before choosing an OSS product: number of commitments (level of its activity), high number of comments and shares, how many bugs are fixed, number and severity of open bugs and a consistent posting schedule. Abandoned projects |
Component Management | Component management and unsupported applications are not an issue when going with a commercially supported product which packages and integrates components together as one as that is what you are paying them to do! | As mentioned before, open source can be complicated where multiple open source components have been used in a single product, and each component could have a different licence terms, and even different dependencies. How will you keep abreast of updates in the open source community? Do you know if different areas in your department are using different versions of the same OSS product? Has a particular product in the Open source community stopped being supported (e.g. infrequently updated)? Agencies may find automated open source management tools useful to enable efficient, effective and continuous tracking of open source usages and cross-referencing matching with the most current data about open source components, vulnerabilities/bug detection, risks, time tracking, notifications, licences, potential licence violations, fixes and updates. |
Installation, training and ongoing support | With a commercially supported product, agencies can tap into the knowledge and training provided by the vendor, not only during installation, but also ongoing technical support from experts, access to documentation, and tools to assist with the deployment and maintenance of your solution. | Installation, training and ongoing support is all up to you. It is therefore important that the necessary skills, time and resources are available to ensure that you not only correctly install the software, but also that all users understand how to use and maintain the software, and that the agency can troubleshoot issues as they arise. |
Release of OSS
We encourage agencies to add to the community by releasing developed in-house as open source following appropriate due diligence.
Copyright in any OSS products must be owned by the State of Queensland prior to release. Any Queensland Government initiated open source software should be released under the most appropriate licensing approved by the Open Source Initiative.
It is strongly recommended that prior to the release of OSS products, a risk assessment be undertaken and signed off by an accountable officer.
There are two main models when releasing OSS:
- A once-off release of the software or IP to the community for ongoing development by the community. This might grant a license to the community to continue development and re-use the IP. This might be used by Government to archive IP from a closed project to donate the IP to the community for the community to make use and advance the agenda. It might be used by Government to release a sub-component of a larger system that is reusable and has wider value to allow it to be utilised and enhanced by a wider community, and for Govt to also receive a stream of enhanced to the sub-component which can be incorporated into the wider system.
- A once-off release of the software or IP to the community, with Govt continuing to develop along with the community in an open source model. Both communities develop on an open and transparent model, and share the support burden of the software and can utilise the software for their own purposes.
OSS release stages
The following sections show the various stages of releasing government software as OSS:
Pre-release is the stage where the business considers the viability of releasing a specific piece of Queensland Government software as OSS. It concludes with the business decision to either release or not to release as OSS, and the type of release (continuous development or once-off)
On-approval is the stage where the agency prepares the code for release including de-identification and removing any vulnerabilities. The agency would also consult on the release plan with central government agencies such as the Queensland Government Chief Customer and Digital Group (QGCDG) (via qgea@qld.gov.au) if the open source software is to be released under the banner of Queensland Government. The agency would also implement any future resourcing requirements as a result of OSS release.
Release is the stage where the code is released to the public as OSS. This includes relevant open source repository creations (e.g. GitHub), code uploads and annotations and comments associated with the code repository.
Post-Release is to ensure appropriate periodic monitoring by the authoring agency of the code repository to provide any support, and applicability and suitability of licences. This includes processes where the agency intends to continue developing their OSS product openly with the community. Reviews also consider any internal and external changes to government policies and priorities that may lead to continued retention, modification, or even, retraction of the OSS code.
Pre-release: Testing the viability of releasing government software as OSS
The checklist below provides some considerations to test the viability of releasing a piece of government software as OSS. Note: even if a piece of government open source software is deemed a less viable candidate by the agency, the business decision-maker could continue to pursue OSS release on the proviso that they have chosen to either mitigate or accept the potential dis-benefits.
More viable candidate for OSS release | Less viable candidate for OSS release | |
|---|---|---|
Source code ownership | Source code intellectual property (IP) wholly owned by the Queensland Government or Terms and conditions, and existing licences allow for OSS release. Licenses for any sub open source components used allow for repackaging and the intended use. | Source code IP not wholly owned by Queensland Government. Source code is an extension to proprietary software (customisation). Terms, conditions and existing licences do not immediately support OSS release. May need to re-negotiate, extend or request for appropriate owner to re-licence the code to make it a more viable candidate for OSS release. |
Preparation cost | Financial and time effort for making the code fit for release is relatively low. Time effort includes: de-identifying private information on individuals and business entities removing security vulnerabilities including comments in code on security weaknesses and API tokens. Migrating the history of the software project such as outstanding & completed tasks, issues, pull requests. Making available design documentation, installation instructions and testing and deployment documentation and processes so the community can download, install and contribute enhancements. Publishing community contribution guidelines, code of conduct and coding standards | Financial and time effort to make code fit for release is high or highly complex. |
Support | The business may choose to not support the open source software and make this explicit as part of OSS release. The business may choose to support the open source software at whatever levels they deem appropriate and make this explicit as part of OSS release or outsource support. | The business would support the software if it were released as OSS, but are not ready or choose not to provide these levels of support. |
Competitive products | No commercial providers offering equivalent open source software such that their business would be undercut by the availability of an open source alternative. | Potential commercial providers offering equivalent open source software such that their business would be undercut by the availability of an open source alternative. This would not necessarily prevent the release of Queensland Government software as open source but would be a trigger to anticipate and prepare for a potential political impact. |
Commercialisation opportunities [1] | Previous attempts at commercialisation have not gained traction or have been unsuccessful. Highly attractive to release as OSS (based on business considerations). Highly achievable to release as OSS (based on effort and cost) No concerted effort at commercialisation. There has been talk but no concerted action in the past x years (x as determined by the business) | No discussion has been made to explore commercialisation opportunities. Highly attractive and highly achievable to commercialise. |
OSS licences
In addition to the above, the agency should consider the which OSS licence would be the most appropriate fit for the government-owned open source software, including what support needs to be put in place as a result of the chosen licence.
The spectrum of permissive to restrictive licenses
For example, a more permissive licence such as the BSD 3-Clause requires low reciprocity from users. Users need only retain copyright notice, not use the copyright holder nor names of contributors to endorse or promote derived provides, and if these conditions are met, they are allowed to use the source code in any form, with or without modification, and can be redistributed using other licences. Is the agency supportive of code use with or without modification, for its intended or non-intended use?
On the other hand, more restrictive licences such as some variations of GNU GPL if the user links their own code to GPL-licenced code, the whole product needs to be GPL-licenced. In this instance, if the agency's purpose for OSS release is community sharing without financial implications to users, this would be a more appropriate licence.
See Open Source Initiative for the list of Open Source Initiatives (OSI) approved licences.
As of the date of publication of this guideline, the OSI-most popular licences are:
- Apache License 2.0
- BSD 3-Clause "New" or "Revised" license
- BSD 2-Clause "Simplified" or "FreeBSD" license
- GNU General Public License (GPL)
- GNU Library or "Lesser" General Public License (LGPL)
- MIT license
- Mozilla Public License 2.0
- Common Development and Distribution License
- Eclipse Public License.
Other open source licences may also be considered appropriate. Due diligence and understanding of these licences are necessary.
In practice
Other Queensland Government agencies have chosen LGPL as one of the more permissive licences as it allows the code to be linked to other open source or proprietary code, and any derived products can be distributed with no open source or commercialisation restrictions.
Release options
Another important element to consider during the pre-release stage is whether the product will be released as a once-off or whether the government will commit to the continual development and support. Factors to consider when making the decision is whether the agencies has software-engineering documentation and processes including continuous integration systems, unit testing and version control. As such, sufficient resources are needed to support continued development and engagement with the open source community including: fixing bugs, assessing and responding to feedback. Also agencies may choose to release smaller, more manageable components, rather than an entire open source product. No matter which option you choose it is important to manage expectations and clearly communicate with users your release approach.
Conclusion of pre-release
Some agencies may choose to produce a business case to document the considerations above. Other agencies may choose to communicate with their business decision-makers via email or minutes of meetings. Regardless of the medium, pre-release considerations need to be appropriately documented.
This stage concludes when the appropriate governance body or business owner provides documented business approval for releasing a specific Queensland Government software for OSS release.
On approval: Preparing the source code for release
This stage is about developing a release plan. This describes the list of steps to be undertaken in preparation for, and to release the source code.
A typical release plan would include the following activities (in no particular order):
- de-identify private information in the code and in code comments
- remove security tokens and keys including API keys
- remove all descriptive information in both literals, constants, variables and comments in code that directly contain or point to closed business knowledge, security vulnerabilities and other weaknesses. This could include ICT footprints in the code (e.g. server names, IP addresses, protocols, hidden API) that could be used to attack/exploit government applications.
- decide on releasing source code as a code repository within the umbrella of the Queensland Government GitHub account. While the Queensland government encourages use of code repository within the umbrella of the Queensland Government GitHub account, in rare situations the use of other platforms may result in better project integration, wider OSS community exposure and faster code testing. Alternative platforms may include, but not limited to GitLab, BitBucket and SourceForge.
- create an account on the relevant open source repository platform (e.g. GitHub) this might include the creation of a perpetual email account e.g. opensourcerelease@departmentname.qld.gov.au to ensure that the account is not tied to particular employee which could cause problems of continuity with employee attrition and employees on leave
- prepare the source code according to the needs of the code repository platform e.g. zip format
- if the decision is to continuously develop the source code, then a support plan should be developed, along with assigning roles and responsibilities (such as a code steward) to managing the ongoing support needs.
- consider any intellectual property implications prior to release, as part of due diligence, and obtain legal advice as necessary. For further information refer to the Queensland Public Sector Intellectual Property Principles.
Your agency may wish to consult the QGCDG via qgea@qld.gov.au as an added sanity check on the release plan.
Release
At this stage, the code is released to the public as OSS. This includes repository creations, code uploads, and annotations and comments associated with the code repository.
The objective of this stage is to implement tasks as per the release plan developed from On approval stage and adapt as necessary.
The level of support as determined in the pre-release stage comes into play, ensuring clear communication to users on whether OSS product will be actively supported or not. Where it will be actively supported, a code stewardship approach will be useful to support adopt and reuse the code. The responsibilities of a code steward include providing a central contact point for issues, support technical community engagement, undertake code reviews, resolve any bugs, and ensure overall code health.
Post-release
Post-release is to ensure appropriate periodic monitoring of the code repository to provide support, and applicability and suitability of licences. Reviews also consider any internal and external changes to government policies and priorities that may lead to continued retention, modification, or even, retraction of the OSS code.
Questions that could be asked include:
- Should the OSS continue to be made available as OSS? Have the internal or external circumstances changed that a review of the initial decision is required?
- Is the current support model working? If not, what changes are required?
- Is the current licence still valid?
In most situations, the status quo of continuation is applied. In extenuating circumstances where the environment has drastically changed, the business owner of the OSS may wish to begin the Pre-release stage and its subsequent stages.
Departments should also have clear governance processes on how to manage OSS contributions, including how and when you will respond to contributors, monitoring the forum your OSS product has been published to and ultimately, the decision on whether or not to accept a contribution.
Participating in OSS projects & forums
Contributions are the essence of participating in the open source community, whether it be a finished software product, or the development of a smaller open source component. Participating in OSS projects and forums allows for the free exchange of ideas to drive innovation, creativity and advances in technology. Having a community of developers from different backgrounds, expertise, experiences and countries can make a great difference and will support the creation of new and versatile ideas. It will also pick up errors in code that a traditional solo or closed team approach may miss.
Like any person contributing to a public forum and collaborative software project, there should always be the highest priority on patience and civility when contributing and responding to posts. Posting can be a scary thing, especially the first time, so be sure to be welcoming and empowering. Departmental policies on Use of ICT resources, facilities and devices can also provide clarity when using forums and be clear when responses are representing departmental views versus personal opinions. It is recommended that when contributing on OSS forums where the agency is using the software, it is done so through official social media accounts, and comments are representative of the agency in an authorised environment. Further information can also be located in the Principles for the use of social media networks and emerging technologies and the Personal use of social media guideline.
The following looks at contributing on OSS forums from both a use and release perspective:
Contributing to OSS projects
Agencies should consider contributing to OSS projects that they use, because this can help the open source product stay current and in turn, reduce the number of exploitable vulnerabilities and improve security. Be aware that when contributing, particularly to commercial software, those companies may require you to sign up to a Contributor License Agreement (CLA) which sets the ground rules regarding where the intellectual property ownership lies as part of contributing submission to an open source product. This includes factors such as:
- disclosing where you are making submissions in the course of work from an employer, or as an individual.
- whether you are submissions material that contains third party intellectual property (and that you have the necessary permissions to do so).
- What rights the OSS project owner has over all contributions made
Release
When releasing open source to an open forum and sharing platforms, many of the pre-release elements have been discussed in the section 3 above, such as choosing which open source licence you want to use, where you are new to releasing, it may be more manageable to start by release smaller, more manageable components. Other elements you may want to consider when releasing OSS to a forum include:
- Consider establishing a Contributor License Agreement, which will ensure the intellectual property implications are considered fully, both when received at the time, and the ability to use contributions later down the track
- Clean up/sanitise any code to ensure it is readable, and doesn't contain any unnecessary history, internal references, and sensitive information such as passwords and credentials
- Ensure you have done sufficient testing prior to release
- Ensure you adequate describe the code people wont contribute if they don't know what the code will be used for
- Set you expectations for contributors Will this be a closed project? Will you accept contributions only after thorough testing? Are you happy to receive any and all contributions?
- Use support tools that guide people to what sort of contributions are most sought after for example finding hidden bugs may be your number one priority.
- How will you get the word out that you have released open source perhaps some advertising may be helpful?
- At what stage will you make it open from the beginning? Or towards the end of the OSS project?
- Is the code being released with data? Ensure that the data is suitable for release (dummy data is always the best bet).
- Release documentation which outlines how others can install and configure your software, and also documentation for how others can contribute and enhance your software e.g. outline coding standards, requirements for automated tests, and how to build and test the software.
- Evaluating your release success will the open source still be available 12 months down the track? What are the lessons learned from the release that use for your next release?
Further reading
- Open Source Initiative FAQs
- Open Source Initiative Licenses & Standards
- Open Source Initiative list of all approved licences
- USA Federal Government Open Source projects
- UK Cabinet Office, Open Source Software Options for Government
[1] Please note: In line with the QGEA Information access and use policy, departments must provide access to government information, free of charge to the maximum extent possible. As such commercialisation of OSS is discouraged