Information silos vs. open data in development organizations

Very good article, good point: it is not that developing organisations are keeping their data secret, it is just that (to them?) it appears (!) to be still very hard to interconnect those databases. As I am currently assembling such data myself and contacted some organisations, there is one aspect, I would like to draw your attention on: some organisation might not be allowed to appropriately give away data due to contract reasons.
a.t.m. people seem to be doomed to keep reinventing the wheel for some time.

Hi Christian, thanks for another interesting blog post!

I think one aspect of the problem is that “formal organisations” have a tendency to work on their own mission (and internal operation and survival in competition with other organisations). That leads to “data hugging disorder”, and opening up the silos is a matter of calculating “business value” of “giving things away”.

“2.0″ approaches bypass “formal organisation” as a necessary component to implement a (partial) mission shared by a group of people, and “sharing” becomes crucial as a driver to getting things done, not an exchange of value.

It’s also more common to assemble a diverse set of tools and websites (each good for part of the work) to contain the data and stories, so sharing and openness is also part of the choice of technology and partnerships. Information has to flow as easily as possible.

I think we’ll see more interest in opening up the silos once the established organisations (working more at meso and macro levels) want the data/information from the 2.0 projects (usually more micro level) to flow *into* their silos, to ease the burden of demonstrating their accountability for impacts on meso and macro levels.

Once that happens, the conversation about getting data *out* of their silos will become easier too: those who manage to become effective in nurturing an ecosystem of “organisations” and “2.0 initiatives” will be better able to compete for donor money.

There’s already a shift in some of the bigger organisations to show more “leadership and vision” rather than “implementation power”, so I’m hopeful

Rolf.

@Rolf Kleef: good point. Seen this way, it might be regarded as positive to have so many microprojects growing into 2.0 – as it leads to loosing track of ‘the big picture’, which might eventually lead to what you described as ‘established organisations want the data/information from the 2.0 projects’.

@ckreutz: “nationmaster” for example is compiling data from a lot of different sources. There are ways of automating queries – the important part would be opening up databases – via API, as you suggested.

Hi Christian,

Thanks for yet another good thought piece. I agree with you that the lack of open data is a big problem at this moment.

I have been working in a development organization for last 8 years; of these, I spent over 7 years working on making sharing/opening information. The idea behind our efforts was not to ask people to start using one big database or same database or even the same internal schema (because in most cases all of these were well in place) but to at least open up the content of the database using agreed standards and protocols. We worked with many of the big partners (or information owners) to define standards (metadata schemas) for exchanging information and looked at the best possible technology for exchange (first XML, now also RSS or RDF).

I must say we have been able to succeed to some extent. One good example is AgriFeeds (http://www.agrifeeds.org) which brings together news and events information in the areas of agriculture, forestry, fisheries, nutrition, and etc from organizations working in these areas. Even though these are just what we would call very simple information types (when compared to statistical or geospatial data), it has been a very long uphill walk. We worked often one-on-one with data providers to “validate” their outputs so that they met the minimum requirements (even of basic RSS!).

You have highlighted some of the problems already – proprietary databases/systems and their lack of appropriate output formats, lack of focus on the bigger picture, etc. One problem that I think we have been facing time and time again is to actually show the information owners (in our case often national/regional repositories) the benefits of sharing/opening up their content. They do not see how sharing across will increase visibility of their information. The silo culture is prevalent (what we have is good enough for us!) and the degree of effort required to open up the content still overrides their understanding of ROI. We have looked at grey literature, learning resources, project information etc. with mixed results. The bottom-line for us has been that we need to do lot of awareness raising through good concrete examples followed up with capacity building/development in “opening up data” (basically training on standards and protocols) and providing them with easy to use tools (data converters such as XSLT,etc.).

I know that other parts of my organization has been working with big standard setting bodies such as SDMX and ISO to set up exchange standards for our own content. But, sometimes, that is not always enough.

I am not sure this is 100% relevant but you may wish to take a look at some of the activities in the area of opening agriculture (in its broadest sense) “information” at:
* Agricultural Information Management Standards – http://www.fao.org/aims/
* Example of information aggregation portal – http://www.gfis.net/
* Coherence in Information for Agricultural Research for Development – http://www.ciard.net/
* Statistical Data and Metadata Exchange – http://sdmx.org/
* Geonetworks (uses ISO 19115) – http://www.fao.org/geonetwork/srv/en/main.home

RT @ckreutz Information silos vs. open data in development organizations http://tr.im/qh36. Interesting post proposing M&E 2.0!

RT Information silos vs. open data in development organizations http://alturl.com/omr5

RT arjencito RT @ckreutz Information silos vs. open data in development organizations http://tr.im/qh36. Interesting post proposing M&E 2.0!

We see similar dynamics in healthcare “silos”… RT @bart Information silos vs. open data in development organizations: http://ff.im/-4F9YT

#ict4d | Information silos vs. open data in development organizations (courtesy Christian Kreutz): http://tinyurl.com/n6ghtj @ckreutz

Hi Christian,

I think the biggest issue here is that most organizations have ‘closed’ or ‘private’ as the ‘default’ setting for all their data and often also publications. We need to change this so ‘open’ (and re-usable) is the default and closed becomes the exception.

However, there iss lots of good work going on to make data and datasets more open – here’s an example from the cgiar: http://csi.cgiar.org

there’s also people working on aid data – http://www.aidinfo.org/

here’s a sector example: http://www.conservationcommons.org/

in agricultural research, Gauri mentioned CIARD – whose principles surely apply as much to data as to information that is the main focus of that group.

see more on data on the iaald blog – http://iaald.blogspot.com/search/label/data

in the cgiar, a ‘triple A’ effort is seeking to bring together efforts to make research outputs – data and info – available, accessible and applicable. See http://iaald.blogspot.com/search/label/aaa or http://ictkm.wordpress.com/tag/aaa/

cheers

Peter
http://www.iaald.org

@Christian @Simon – I am traveling presently but will be back at home in Upstate New York on Monday. I am at GMT -5. What would be a good time to chat? I sleep very little, so I can easily accommodate your schedules.

Cheers,
Mark Oppenneer

My e-mail is mark ‘at’ ethnosproject.org. I am logged in as ethnosproject on Skype. How should we go about setting up a time? We can ‘meet’ together – or just dial me on Skype when you wish (I’ll remain logged in all day). Cheers, Mark Oppenneer.

Not surprised by closed development data, but good points: RT @ckreutz Information silos vs. open data in development orgs http://tr.im/qh36