Lesson 2.1 What is Shared and Open Data
Learning outcomes
At the end of this lesson, learners will (be able to):
Acquire knowledge about shared and open data
Recognize the value of open data in agriculture
Define challenges of moving to an open data landscape
Understand the benefits of publishing and using open data
1. Introduction
This lesson aims to introduce you the principles of sharing data, what makes data open and the benefits of opening data. While the focus throughout the course is on shareable, structured data, this lesson makes a particular link between shared and open data on a spectrum.
In Lesson 1.1, it was explained that the availability of more data at global and farmer level helps to enhance extension, trade and financial services. These services can increase income and yield. The use of ICT makes it possible to forecast the future much better than ever or to answer seemingly complicated questions much more quickly based on data. Such questions might be: Where does our food come from? Can we manage risks in our farm and take control measures against droughts or pests? Are we able to predict problems such as floods or low yields? Can we make informed decisions on what to grow, what treatment to apply, when to plant, treat or harvest? Technologies today allow us to build services to answer these questions, but data only offers these opportunities when it is usable.
The notion of open data has been around for some years. Considerable amounts of data today are generated by the public sector, e.g. soil surveys, cultivar registrations, pesticide residues, health care, defence industries, infrastructure, public education, and telecommunications. See the categories of datasets presented in Lesson 1.1, including individual datasets accessible on public portals. In 2009, various governments, including the USA, UK and Canada, launched open government initiatives to open up their public information.
In addition to public data, for which there is a general demand for openness, private sector data is also more and more important for decision making: while it is not always feasible to make this data completely open, many of the principles of open data (access, reuse, interoperability) apply also to the sharing of private sector data even if under different access conditions (see the data spectrum in the next chapter).
Open access to research and sharing of data are vital resources for food security and nutrition, driven by farmers, researchers, extension experts, policy makers, governments, international agencies and other private-sector and civil-society stakeholders participating in ‘innovation systems’ and along value chains. Lack of institutional, national and international policies and openness of data limit the effectiveness of agricultural and nutritional data from research and innovation. Making open data and data exchange in the value chain work for agriculture requires a shared agenda to increase the supply, quality, and interoperability of data, alongside action to build capacity for the use of data by all stakeholders[1].
From mobile technology used by health workers to open data released by government ministries, data is becoming ever more valuable, as agricultural business development and global food policy decisions are being made based upon it. But the agriculture sector is also home to severe resource inequality. The largest agricultural companies make billions of dollars per year, in contrast to subsistence farmers growing just enough to feed themselves, or smallholder farmers who grow enough to sell on a year-by-year basis[2].
The scarcity of available data prevents us from identifying and learning from real progress at the global and national levels. It also hides inequalities within countries, making it more difficult for governments to know about them or for others to hold governments fully accountable[3]. National averages are not enough to see who is being left behind, as nutritional levels can vary even within households. Beyond just collecting data, it should be used actively to make better choices and inform and advocate decision-making from the household level all the way up to policy level.
2. Notion of shared and open data
To make the data open, the important thing about data is how it is licensed. For data to be considered open, it must be:
accessible, which usually means published on the web
available in a machine-readable format
with a licence that permits anyone to access, use and share it – commercially and non-commercially.
Many individuals and organisations collect a broad range of different types of data in order to perform their tasks. Government is particularly significant in this respect, both because of the quantity and centrality of the data it collects, but also because most of that government data is public data by law, and therefore could be made open and available for others to use[4].
The open data movement has been advocated strongly by governments to allow others to benefit from their data and their desire to be transparent, but research institutions and the private sector also generate data which they are willing to share as a common good[5].
Open data is data that can be freely used, reused (modified) and redistributed (shared) by anyone[6]. The Open Data Handbook emphasizes the importance of the definition of open and highlights key features about open data:
Availability and Access: The data must be available as a whole, and at no more than a reasonable reproduction cost, preferably by downloading over the internet. The data must also be available in a convenient and modifiable form. Managing data can be costly in terms of time and resources needed. An example of costing for data management can be seen at UK Data Service[7].
Reuse and Redistribution: The data must be provided under terms that permit reuse and redistribution including intermixing with other datasets.
Universal Participation: Everyone must be able to use, reuse and redistribute – there should be no discrimination against fields of endeavour or against persons or groups. For example, ‘non-commercial’ restrictions that would prevent ‘commercial’ use, or restrictions of use for certain purposes (e.g. only in education), are not allowed[8].
3. Open data principles
The Open Definition makes precise the meaning of ‘open’ with respect to knowledge, promoting a robust common in which anyone may participate, and interoperability is maximised. Knowledge is open if anyone is free to access, use, modify and share it – subject at most to measures that preserve provenance and openness[9].
Open data must comply with an open licence or a status. It must be in a public domain or under an open licence. Without a licence, the data cannot be reused.
It must be accessible and downloadable via the internet. Any additional information necessary for licence compliance must also accompany the work, such as an attribution to say that people who use the data must credit whoever is publishing it, or a share-alike requirement to say that people who mix the data with other data have to also release the results as open data.
Open data must be in a machine-readable form which is processable by a computer and where the individual elements of the work can be easily accessed and modified. It must also be in an open format which places no restrictions, monetary or otherwise, upon its use and can be fully processed with at least one free/libre/open-source software tool.
The licence used for the open data should be compatible with other open licences. It should permit free use, redistribution, creation of derivatives, and compilation of the licensed work. It must allow any part of the work to be freely used, distributed, or modified separately from any other part of the work or from any collection of works in which it was originally distributed. The licence must not discriminate against any person or group.
The Open Data Charter, which is a collaboration between over 70 governments, agrees on six principles for how governments should be publishing information. Each of them is explained below briefly. On their site, the Charter also provides detailed action items to achieve each of these principles[10].
Open by Default: Free access to and use of government data (data held by national, regional, local, and city governments, international governmental bodies, and other types of institutions in the wider public sector) brings a significant value to society and the economy, and the government data should, therefore, be open by default.
Resources, standards, and policies for the creation, use, exchange, and harmonisation of open data should be globally developed, adopted and promoted so long as citizens are confident that open data will not compromise their right to privacy.
Timely and Comprehensive: Data may require time, human and technical resources to be released and published. It is important to identify which data to prioritize for release by consulting with the data users. The data must be comprehensive, accurate, and of high quality.
Accessible and Usable: Opening up data enables stakeholders to make informed decisions. The data should be easily discoverable and accessible, and made available without any barriers.
Comparable and Interoperable: The data should be published in structured and standardised formats to support interoperability, traceability and reuse. It should also be easy to compare within and between sectors, across geographic locations, and over time in order to be the most effective and useful.
For Improved Governance and Citizen Engagement: Open data strengthens governance and provides a transparent and accountable foundation to improve decision-making and how land markets operate. It enables civic participation and better-informed engagement between governments and citizens.
For Inclusive Development and Innovation: Openness stimulates creativity and innovation. Open data by its nature offers an equitable resource for all people regardless of where they come from or who they are and provides a less digitally divided environment to access and use the data.
4. Benefits for shared and open data
The benefits of open data are diverse and range from improved efficiency of public administrations, economic growth in the private sector to wider social welfare and citizen empowerment.
Performance can be enhanced by open data and contribute to improving the efficiency of public services in health and nutrition. Greater efficiency in processes and delivery of public services can be achieved thanks to cross-sector sharing of data, which can for example provide an overview of unnecessary spending. Resources can be better targeted thanks to local-level, disaggregated data, showing which areas and populations have the greatest needs.
The economy can benefit from easier access to information, content and knowledge, in turn contributing to the development of innovative services and the creation of new business models.
Social welfare can be improved as society benefits from information that is more transparent and accessible. Open Data enhances collaboration, participation and social innovation[11].
GODAN’s report, ‘How can we improve agriculture, food and nutrition with open data?’, specifies three ways that open data can help solve practical problems in the agriculture and nutrition sectors.
Driving organisational and sector change through transparency. Transparency around targets, subsidy distribution and pricing, for example, creates incentives which affect the behaviour of producers, regulators and consumers. By requiring companies, government departments and other organisations to publish key datasets – performance data, spend data or supply-chain data, for example – governments, regulators and companies can monitor, analyse and respond to trends in that sector. More importantly, publishing this data across a sector can ultimately transform how products and services are delivered. We can refer here to the same example as given above, Abolobi Fisher from Open Water, of using data about fisher practice for small-scale fisheries industry to make informed decisions.
Providing farmers with more accurate, accessible, timely information – from large agriculture groups to the individual smallholders – will help to ensure food commodity markets function well in future. Progress will be driven largely by providing better access to accurate, timely information for individual smallholder farmers, businesses and policy makers alike. Open data can and should be part of the solution. Open data promotes transparency across the sector to accelerate progress, identify areas for improvement and help create new insights[13].
5. Open data acts as change agent
Open data acts as a change agent. Implementing an open data initiative often involves cultural and institutional change. Opening data goes far beyond putting data on a website under an open licence. Applying the technology is relatively easy when compared with bringing about a cultural change, which can be much harder[14]. It requires consulting with potential data users internally within an institution as well as the external stakeholders.
The same is true for private-sector data sharing projects that may not adopt fully open data approaches but still need to change their attitude towards their data and engage with other actors for its reuse.
However, this difficulty of adopting a change does not stop the amount of data which is increasingly becoming openly available. There are still challenges related to data management, licensing, interoperability and exploitation. There is a need to evolve policies, practices and ethics around closed, shared, and open data.
The challenges involved in opening agricultural data are best addressed at the level of a particular problem in a specific field, where standards can be identified or developed, and data released as part of solving a problem. This is especially true when advocates can point to a clear theory of change. GODAN addresses this issue with care and sets out five strategic steps[16] for pursuing solution-focused open data initiatives for agriculture and nutrition:
Engage with the growing open data community, including key problem owners and experts at GODAN, to identify the challenges that open data can help solve.
Build open data strategies and projects with a focus on finding solutions to land tenure, agriculture and nutrition problems.
Develop the infrastructure, assets and capacities for open data in relevant organisations and networks.
Use open data and support users of relevant data.
Learn through ongoing evaluation, reflection and sharing to ensure we can all continue to improve our practice.
Footnotes
Last updated
Was this helpful?
