¶ 1 Leave a comment on paragraph 1 0 If part of the intent of gathering together eight or more years of writing was to understand whether there was a central point then what have I learnt through the exercise? It seems to me that the three pieces in the following section best represent the various attempts at synthesis I have attempted over the years. Each could be seen as developing a specific thread. “From Scarce Narratives to Abundant Fragments” most represents the the thread of technical opportunities and “Who do You Get to Say I Am” focusses on the growth of my interest in non-scientific research approaches and models and consequent understanding of the importance of context and communities. “The Road Less Travelled” represents the most recent attempt to bring together my ideas on research assessment and how it drives incentives.
¶ 2 Leave a comment on paragraph 2 0 “From Scarce Narratives…” was one of my earliest attempts at integrating my emerging ideas about the need for human readable narratives with the graph and web based view of research objects that would support computational reading. In that sense it emerges out of the pieces in the section on Networks. It was originally written as a white paper for a meeting in Dagstuhl that would later lead to the formation of the Future of Research Communication and E-Scholarship organisation (FORCE11). Ironically for a piece focussed on the question of how to communicate and publish research it has never been “published” either in a formal scholarly setting or on my blog, sitting instead for five years as a public Google Doc that a few people have read from time to time.
¶ 3 Leave a comment on paragraph 3 0 It attempts to chart a path in which we can have the best of both worlds. Individual objects can be captured and managed, the relationships between them described in ways that allows both the automated aggregation and display of the small scale stories. But the larger scale stories still require human intervention to build a compelling narrative. The question of resourcing is at the centre of this, and by extension the issues of resourcing peer review. In the end this remains my solution that original false dichotomy of Web 2.0 vs semantic web. It combines the idea of networks of research objects with the need for story telling. Ultimately the root of my journey into humanities perspectives might also be found here as well. It took time for me to understand and value the idea of narrative, but it has remained central to my thinking ever since.
¶ 4 Leave a comment on paragraph 4 0 Two of the previous sections in particular drew out a strand about the importance of narratives; my growing awareness of different perspectives, different ways of thinking and ultimately different ways of knowing. In the pieces filed under “A Personal Digression” in particular I sought to juxtapose these different worlds of my own experience to see how they might illuminate each other. “Who do You Get to Say I am?” draws together some of those threads, of expertise, narrative, as well as power and exclusion. Ultimately it is a first shot at a new personal narrative as I move from my origins in the sciences towards the approaches and thinking of social sciences and humanities. It is a deliberate attempt to blend language and approaches from different disciplines to illustrate how they are all ultimately the same. The question once again, is of what story I tell myself about who I am, and how readily I can persuade others to share that. Once again it comes back around to the narrative.
¶ 5 Leave a comment on paragraph 5 0 The final essay is my most recent attempt to bring many of these themes together. Implicit in the 2009 paper on Article Level Metrics written with Shirley Wu is the idea that many metrics can support different kinds of assessment. As the idea of metrics driven assessment has become more mainstream it has provoked a very strong and negative reaction within the academy. The objection is that simple numbers cannot reflect the richness of modern scholarship. Yet researchers continue to reach for simple, and inappropriate, numbers such as Impact Factors when faced with difficult decisions.
¶ 6 Leave a comment on paragraph 6 0 The problem of course is in the illusion that the numbers are somehow objective. That they can tell us what we should do. They can’t. They can help us to understand what is happening, help us build models that support decision making. But they can never tell us what matters. In “The Road Less Travelled” written from a talk given at the Rio de Janeiro Conference on Open Science in 2014 I attempted to connect those two apparently disparate world views. That we must make the decisions on what we want from our investment in research based on subjective human and community values, but that we have more data then ever before on how information flows and what approaches work in maximising those flows.
¶ 7 Leave a comment on paragraph 7 0 The challenge is no less than reconfiguring the way that we think about setting the agenda for the research enterprise. What started as a need to undermine a single metric has expanded to a need to completely re-think our institutions. Yet of course alongside there has been the growing understanding of just how hard that is. But, just as was the case with the rant against Impact Factors, the seeds of change may lie in showing those who care how their own basic standards are being ignored. As more data is collected and we have a greater view over what is really happening we have the opportunity to expose those inconsistencies and problems. Solutions will not come rapidly but, as with Impact Factors, exposing the dissonance can at least move us in the right direction.
¶ 8 Leave a comment on paragraph 8 0 What is perhaps more interesting than how each of these pieces represents the current endpoint of a particular thread is how all three pieces intertwine. The idea of narrative, of the need to be able to tell our own stories, runs through all three, as do ideas of networks and how they change what we can observe and how we can choose to act. Over time the importance of acknowledging power and its imbalances, of the challenges of consultation and community engagement emerge, always in tension with the concept of expertise. Many of the other pieces in the collection illustrate the development of those individual ideas better but they don’t for those most part bring them together. Those other texts can provide clearer snapshots of the process. But it is in the three pieces in this section where those ideas most clearly come together. They represent the most complete view of my thinking up to now.
¶ 9 Leave a comment on paragraph 9 0 In a sense this can be seen as a metaphor for the central idea of this collection. The connections across the whole collection are loose, but local clusters are more tightly connected revealing the themes that have emerged and developed over time. Any research process, or scholarship more generally, generates objects, some of which are shared, some of which are not. Some of them are shared widely and some with a small, select, group. In moving from a print to an online world we have the opportunity to share many more of those objects and to represent those objects in different ways. We’re starting to make progress on the former, but not so much on the latter. Because we can choose to share more and more flexibly we can tell our story in multiple ways. We can make multiple collections of those shared objects. However as we start to share more, and the expectation of a more complete record increases, we are starting to see how messy the context behind each set of stories, each collection of objects, really is.
¶ 10 Leave a comment on paragraph 10 0 Because those sharing events are online many more of them can be tracked and a much richer picture is emerging of how knowledge flows from one place to another. But once again the data is messy and incomplete. We can tell stories by threading this data together, just as we can thread research objects together to tell our research stories, but in both cases we haven’t reached the promised point where insight emerges directly. We are limited to seeing the threads, we are not yet operating over the network as a whole. Even where the promise of “big data”, mining or integration is starting to be realised we seem to be restricted to looking for things we expect to find.
¶ 11 Leave a comment on paragraph 11 0 The Large Hadron Collider detects far more events than it can record. The experiments are set up so that only specific events that might be of interest are recorded at all, let alone analysed. This is done for very sensible technical reasons, there isn’t enough hard disk space on earth to record all the events, but it exposes an interesting bias. As we increase the data at our fingertips we are increasingly only looking for what we expect, only thinking to look for those things that fit with the stories we tell ourselves. That is why taking a critical and questioning approach to our framings, our stories and the context in which we collect that data is so important.
¶ 12 Leave a comment on paragraph 12 0 It is not that those stories or filters or contexts are inherently bad. Indeed they are necessary for our limited human capacities to deal with the information that we have. But that we need to examine these and to understand how they effect what we choose to do. We need to think about our values and how they inform what we do, and whether what we do is actually congruent with the values that we profess. We need to examine how our institutions shape our behaviour and how they could be better designed to reflect shared values, but first we need to discuss what those values are. We need first and above all to learn how to tell our own story, to ourselves, and then to the wider world. Only then can we think about how our institutions and incentives should be shaped.