65. Hugeness

This week's episode is on one of my favorite topics. How do we shape the humanitarian narrative using evidence? “Specifically, how do we use quantitative information, numbers, statistics to make decisions? My guest is Brendan Lawson, who's an academic and has written a book called The Life of a Number, as well as an article he's co-authoring with friend of the pod, Joel Glassman, called Ten Things We Know About Humanitarian Numbers. It's a wonderful, long and geeky conversation, and I will not make it longer with a lengthy introduction, but I have to mention one slightly embarrassing thing about this conversation. When I was editing the episode, I realized that rather than discussing ten things we know about humanitarian numbers, we actually only discussed nine. We simply forgot one of them. I considered going back to Brendan to record the missing piece, but instead I made a virtue out of the situation and encouraged you to read the article so that you can get all ten facts about humanitarian numbers. So, if you really want to know what number ten is, you will have to do a bit of research yourself and go read the article. It's a quick and interesting read, and you can find a link to it in the show notes. I will not tell you what it is we forgot, but I can say as much as it is rule number five. Don't forget to provide us with feedback either on social media or on info@trumanitarian.org. It would also be nice if you could share the show with colleagues, share it with friends that you think might be interested. Most importantly, as always, enjoy the conversation.

Brendan Lawson, welcome to Trumanitarian. You are a lecturer at Loughborough University, where you specialize in media and communication, and you've just published a book called The Life of a Number, which I really enjoyed reading it. It's a very interesting piece of work you've done.

Hello, thank you for having me.

You also happened to hang out with a friend of the part called Joel Glassman. Joel was on humanitarian on episode eight, where he spoke about a book he had written around Sphere Standards and the role that quantification plays in humanitarian action.

And the the episode is called Needology, and I have to admit that you and Joe are some of the least boring academics I know in this field. I I find the stuff you do very interesting. I think the role: understanding what happens when we turn humanitarian situations into statistics and numbers, how that influences decision-making and humanitarian outcomes is important.

But I've spoken a lot. I may have misrepresented what it is you do. So please tell us what do you do?

Yeah, you capture it pretty well. So I, like you said, I'm rooted very much in media and communication slash journalism studies. That's my discipline that I did my PhD in. But my work, especially during my PhD and afterwards, in the period afterwards as well, really focused on the way that humanitarian crises were reported by journalists using data. And this kind of led me into looking at Joel's book, as you mentioned, Joel Glassman's Quantification of Humanitarian Needs. And what I found really interesting in that book was kind of this, a lot of the theoretical underpinning that I'd been using to understand how numbers are produced, then how they relate to things like power is something that Joel elaborated on, but also introduced really sort of interesting aspects around kind of the importance of materiality or machines in order to actually conduct these measurements that then turn into data. And there's a section in his book where he refers to media and communication discussions that were particularly relevant to my work. And it was something that I hadn't really noticed in a lot of the sort of humanitarian rights, or humanitarian crisis literature was kind of a sustained engagement with media and communications as a field beyond kind of simplistic engagements with it. So, I spoke to Joel, contacted him, had a chat with him and kind of set the ball rolling really for this special issue that came out in the Journal of Humanitarian Affairs. We had an interesting workshop at IHSA and then materialized into this special issue. But we wanted to focus on multidisciplinary approaches to humanitarian data. Joel coming from historical background, but also having this sort of element of anthropology in there as well. Me, coming from media and communications background, but also a bit of kind of media sociology. And we discipline wise maybe came from quite different areas but ended up having discussions that kind of became more than the sum of their parts. And that materialized both into the author's contributions to the special issue, but also to kind of our introductory piece, which we might talk about today. The Ten Things We Know About Humanitarian Numbers. Deliberately provocative and a bit satirical to put a number on the amount of things that we know about numbers. But I think it kind of captures the different ways in of thinking and where we kind of met in the middle. And that very much kind of reflects my book, The Life of a Number as well, which focuses more on the UK and how numbers circulated in the UK became meaningful during the pandemic. But a lot of the theoretical basis to this kind of chimes in with the sort of ten things we know about humanitarian numbers.

So that is really interesting. Now, coming from the media and communication side of things, how do you see the humanitarian world as a case? What characterizes the narratives that sort of underpin humanitarian action as compared to some of the other cases you work with?

Yeah, this is something I find really interesting, especially, well, in my work, but also when I come to review papers as well, when you're looking at something like journalism studies, media and communication studies, if you're looking just at the news media, it kind of sets itself up into different genres, whether it's celebrity reporting, whether it's economics, politics, whether it's international development, human rights, humanitarianism. And this was kind of a really important conceptual work that has to be done around what's the distinctive nature of reporting on humanitarian crises compared to conflicts, which are interlinked, but are not always the same. Or how does it differ from something that's very different, such as UK politics?

“In general, I found that journalists that I spoke to were more likely to report not from the site of humanitarian crisis. And they were reporting from some sort of distance, either in some base in London or in New York. And that's where the peculiar power of data, when you either don't have access to these spaces or don't get sent to these spaces, data then plays a peculiar power to be able to summarize and rationalize and sort of objectify a particular setting. So it almost became more important to have that as a source of knowledge. But then what that knowledge could also do is kind of rank these kinds of crises in some sort of loose order. They didn't really make any particular sense. It was kind of an order that was deeply colonial, deeply sort of xenophobic, but would mean that there would be some sort of scale and order at least at a performative level. I have to admit that for me, 95% of the reporting from crisis and in particular sudden onset crisis is cookie cutter journalism. They changed the place, the date and the number, and then they tell exactly the same story as they did last time. And maybe you sprinkle a bit on top of a humanitarian saying, oh, you know, this is the worst crisis I've seen in the 25 years I've been in this work.

It's incredibly simplistic. And it's interesting to think about what do numbers do in that situation? How do we rank different crises and what is a big number and what is a small number?

Yeah, and I think to add to that as well, the inability of either having access to corresponding data or different data sets or having direct personal experience means that the verification process is almost non-existent in an article I've published in a sort of more media and comms journal, then it refers to this idea of outsourcing verification to the sources themselves. So instead of checking what these sources are saying, it's actually about constructing, well, what sources are trustworthy and what are not because we can't check the numbers that are coming to us.

Yeah, yeah, and that's the tricky part of kind of when numbers are more closely associated with evidence than pretty much any other type of knowledge, they can give an appearance of kind of scientific rigor or evidence, but often are wildly inaccurate. And yeah, this idea of sort of being precisely wrong happens quite a lot.

I would like to take us back to your book. There are two central concepts in there that I found really interesting. Firstly, you talk of quantitative realism. What is that and why is it important? And then this business of data bounds, what is it?

Yeah, like you said, these two concepts are sort of fundamental to the book itself and they operate in relation to each other. I can explain them separately first, and then I think kind of explaining how they intersect I think makes the most sense. I'll probably end up overlapping anyway through the explanations. But quantitative realism, it's really this idea that you can use numbers to gain an understanding of some sort of objective quality of the world around us. So, the realism of numbers is that they reveal a reality, but not just that, that numbers are a better way to reveal this reality, this sort of hidden reality than the alternatives, i.e. qualitative sort of testimony or any sort of qualitative data. And what's kind of fundamental to this is obviously a long history of kind of science and the emergence of statistics in kind of Western society and cementing this idea that we can understand the world through numbers and data bounds is a concept that needs this quantitative realism to function and dominate. Data bounds is something that I came up with that is probably more of the original of the two in terms of concept. And I use the term bounds here within data bounds both as a verb and as a noun. So, the verb is that data sort of actively bounds a phenomenon by representing it. So, through the sort of consistent representation of say the economy using GDP, then it consistently bounds and actively bounds a phenomenon. So, it's sort of this active process, but it's also the noun as well, which is that it sets a boundary within which conversations can happen using numbers. And when you're outside those bounds, the conversation seems absurd. So again, to go back to the economy example, if you talk about the economy without mentioning sort of these big macroeconomic indicators, then at a level of policy, at a level of kind of national scoping, your conversation almost seems absurd if it doesn't include this data to talk about that phenomenon. So, in some ways, the data bounds are kind of like notions, kind of Foucauldian notions of discourse, where there's kind of nothing outside discourses, he sort of famously put, or hegemony, with sort of Gramscian hegemony, where you can't see beyond the horizon of something. So, it sets the scope within which things can be talked about. And this is particularly relevant for highly kind of quantified phenomena, such as the economy and becomes less relevant for some areas which aren't highly quantified.

Great. So if I understand it correctly, quantitative realism is basically saying it's good to have a thermometer and 20° is warmer than 17° and it is great to know that it is 20° today.

Yeah. And also, that it's better to know it's 20° than to know it's warm enough. It reveals something more fundamental, more objective, more certain than a qualitative term or category.

And then data bounds is that we know that numbers take on a broader meaning than just the number itself and the data bound is that implicit story that the number carries with it.

Yeah, it's...I would say it's kind of the space within which you can think react, engage with, talk about a particular topic or phenomena. And how data sort of actively bounds that space within the quantitative at the expense often of qualitative engagements with that topic.

It would be great to explore these two concepts with some concrete examples from your book. But before we do that, I'd like to just touch on another thing: You pick COVID-19 as a case and one of the things you write is that of course, you, just like the rest of us, were directly affected by the crisis, you write about losing family member and not being able to go through the normal grieving rituals, and that of course somehow all of this shapes your perception of what COVID is. What do you think it brings to your study that you were personally in there and not just a spectator? What's the upside and what's the down?

Everything had shut down. And this project was born out of a sort of academic interest to understand this really peculiar moment that we found ourselves in, but also as a kind of coping mechanism to understand what was happening, to try and maybe take some control in a situation that felt like you'd relinquished all control to either public health experts or politicians. So, the process of gathering individual cases and developing this sort of list of numbers that were particularly important in this context was deeply rooted in my personal experiences. It would either be things that were particularly pertinent that day because restrictions were planning to be lifted depending on case numbers. And then it's kind of, well, what's the significance of these case numbers? Or it would be things that I'd see in graffiti and posters that would be along the side of the street that would refer to case numbers not being accurate and it somehow linking into a conspiracy theory. More than any other piece of research I did, it linked into my personal experience that structured what I thought was important and not important. The downside of this is, yes, I was affected by the pandemic, but in no way compared to other people in UK society who lived in maybe more cramped housing conditions, had certain comorbidities...You hear stories of some people who lost all four of their grandparents within a year, for example. Those quite visceral, painful, deeply structural issues didn't affect me the same way. So, my understanding of structural inequality had to be somewhat abstract and somewhat academic rather than something that I could experience myself. And also meant that I focused on the UK and quite deliberately focused on the UK because I felt like in this sort of incredibly strange time, sticking to a context where I had a lot of tacit understanding that I could lean on to untangle the complexity of what was happening was really important for me. But it did mean pushing against, editor at points saying, well, what about more international comparisons? But I thought the UK kind of presented a case study of extremes anyway, where we'd often be the last country to lock down when we're being told to or be the first to roll out the vaccine and be very proud of it. It's often positioned by both critics and plaudits as exceptionalism.

So I thought it was particularly interesting to focus on.

When I think of COVID, it is like the mother of all black swans, if you want. The level of uncertainty and ambiguity that we had was just incredible. And one of the things we struggled with at ACAPS was to know where to focus and what to talk about. For me, my immediate thoughts went to Zimbabwe, a country where I have lived for some years. I thought economic collapse, HIV prevalence around 12%... And then COVID on top of that, it's going to be a disaster. But initially, not much happened. But then suddenly, a number of middle-income countries in Latin America were hit extremely hard. And that thought had just never crossed my mind. So, for me, COVID and the way I reacted to it has also led to a deep reflection of how guided we are by our biases, our exposure, our history and how careful we must be when we deal with very, very ambiguous and uncertain issues such as COVID. And the reason I think it's so important is that in the future, we will see more and more of this. COVID was not a spectator sport as we talked about.

And that's going to be true in the future as well. More and more of us will be directly affected by crisis, also in parts of the world that normally are quite well protected. So, in a sense, COVID was also sort of a trial run for how we manage this sort of crisis.

Yeah, and I think this, again, coming back to kind of media and communication discipline, often a lot of my colleagues are looking at things that they're directly involved in, whether it's sort of local journalism or whether it's kind of cultural studies that they engage with– the idea of positionality is often central because it's something that they're living through both during their research, but also outside of their research. Often what I'd find interesting speaking to people in the sort of humanitarian sector is there would be a reflection on positionality, but it would often be maybe more in relation to kind of identity and kind of post-colonial thought around how your position as a researcher may be coming from the global north, then engaging with people in the global south, how that relationship would work. But that sort of direct experience of the thing that you're researching as part of both your life, but also your family's life and your extended family's life and the school you went to didn't come in the same way because of the nature of the research. Whereas like you said, I think with future crises that sort of demand us to put ourselves and our own experiences sort of front and center, then this type of reflection becomes not just like an academic exercise or signaling that you are thinking about these things, but absolutely fundamental to the research that you do.

Now let's have a look at some of the examples you use in your book.

You speak of the relationship between protecting people from COVID and then protecting the economy, and then the two different ideas there is around the relationship there. One is that it is a tradeoff, zero-sum game where protecting people means hurting the economy and then the other idea is that it is possible to somehow possible to steer through and navigate these two concerns and protect people and economic interests at the same time. Tell us a bit more.

accurate, but it's the policy that seems to dictate how that data bound is formed. And that's where I really think the idea of data bound kind of comes into its own when it sort of normalizes through data, how we can talk, think, engage and practice a particular phenomenon, in this case, COVID.

It is very interesting that you can tell different stories basically with the same data.

That was the distinction with the... short-term national data was used for trade off, so seeing, well cases have gone up. OK, we need to go down. We need to introduce restrictions, whereas it's the long-term international comparisons that underpin protect both to say well, actually you know this works in comparison with other countries.

The word that keeps popping into my mind as we speak about this is calibration. How do you calibrate your understanding of what's going on in a situation where you experience a very rapid than dramatic change, such as COVID or sudden onset disasters? I think it's really hard for us to calibrate the new reality and compare it to what we know, the muscle memory we have, if you like. And that's why I really like the concept of hugeness in your book. It's a wonderful word, hugeness. So maybe let's tell us what hugeness is and what is the role it plays when it comes to quantitative realism and data bounds?

Yeah, this was basically my first chapter. This was almost like my first love with the book, this chapter which centres on a politician or a set of politicians quite early in the pandemic, talking about 1 billion items of PPE, personal protective equipment, and, as some context essentially used iPosition as a rhetorical device to deflect away from some failings around PPE that in the inquiry that's happening now has kind of been laid to bear around either corruption in contracts being given out or a lack of preparedness, essentially, for this particular type of pandemic. So you have this number, 1 billion, which was actually, as a slight aside, 500 million individual surgical gloves were included in this total, right? So not pairs of gloves, which would be 250 million, but 500 million individual pairs of gloves. You'd also have things like bottles of bleach and waste bags counted as well. So, it was this creative accounting to reach a huge number. And that's kind of important, I think, as context, because there's some drive there, clearly. You know, people who are doing those calculations know that they need to arrive at as big a number as possible. I think it's not too much of a stretch to feel like that is the case when you're counting individual surgical gloves that come as a pair.

Absolutely, but let's face it, if you want to get people to make big decisions, you need big numbers.

Yeah, and this is the argument that I put forward is to say that this notion of Hugeness has become an important element of kind of public discourse, and we deal with these kind of huge numbers in there, let's say billions almost every day, whether it's funding for trying to improve local towns and cities, whether it's out of aid that's pledged to a certain area. But what I argue is that these big numbers, say, 1 billion and bigger, but you probably, you know, count millions in there as well. This dual nature comes to quantitative realism, that's both mathematical but also abstract as well. Billig, who I refer to in the book as well, I think calls them magical numbers. And it's this kind of notion that they do have mathematical syntax. You can write down, as I have in my notes here, one and then nine zeros following on.

Exactly. And I think of it as calibration again, and really what the combination of quantitative realism and hugeness does, as far as I can see, is that on one side, it makes you accept that this is just huge, and we need to do something. Oh my God, we need a billion or whatever... And then accountability goes out the window.

And I think that's the important thing is to say that if hugeness applies to one billion or two billion, then where does accountability for that kind of notion of whether it is one billion or two billion come in? If they both represent hugeness, then how can we understand that accountability? And what I would say as well that I find interesting because I kind of like the sort of history and philosophy of science stuff or history and philosophy of data is that the Greeks and the Romans both had terms that essentially meant countless, but they would almost position them as part of their numerical scale. So, I think it was the Greeks who had ‘mermex’, which I think is supposed to reflect a countless number of ants. And that later transferred into the Latin, I think, for ‘myriad’ or into the term for myriad, which means sort of, you know, lots of, but these things weren't just kind of siloed as separate ways to understand quantity. They were almost like brought into the scale, almost like you'd see a scale now that would say one million, two billion, three billion, countless. And it almost has this kind of feeling of, yeah, exponential growth or hugeness. And I think it's something that, whilst mathematical language has given us so much, we have lost something in the way that we engage with numbers.

As you explain it like that, what I'm thinking is that when you're out there in the field responding to something huge, then you feel it's a drop in the ocean. It never seems solvable what you're dealing with. It gives meaning that you move forward and that you do something and that you help where you can. But there is a sense that, my God, we're dealing with a million orphans here. Maybe we can help a few thousand, but how do we pick those among the million? And what is... So that hugeness, you know, is both disempowering, but it also gives you a license to at least...

Trumanitarian Transcript Template.docx 13/23

Yeah, I I think it it, it also does link in just to the kind of notion of modernity and this idea that we engage with, you know, millions of other citizens in our country and then kind of hundreds of millions of other people in the continent that we're within. And we're sort of set within these huge networks that operate of huge quantities. And, you know, it kind of came hand in glove with when we could talk about these quantities in such large, precise numbers, that we're sort of forced to think on a scale that intuitively makes very little sense.

So we have this rational myth around quantitative realism. Numbers are important, and it's important that 20° is 20° and not just warm enough, as we spoke about earlier. And then there are the data bounds that in a very invisible way define and constrains the way we speak about numbers.

And I think that in day-to-day life that is pretty useful because we live in a fairly stable environment. But what's interesting for me is that that when you then have a crisis, normality goes out the window and a data bound that may work very well in peacetime may not work very well in war. And to sort of unpack this difference, I think this article that you and Joel Glassman recently published in the Journal of Humanitarian Affairs is very interesting.

It really helps unpack the difference and show the meaning that numbers have in crisis. The article is called “Ten Things We Know About Humanitarian Numbers”, and I would like to go through those 10 things one-by-one. So, the first one is numbers look like fact, but quite often they're not.

Yeah, so this is very much linked to that notion of quantitative realism. The impression that numbers give us, that they have this sort of almost firm anchoring quality where we can anchor our decisions, our behavior, our policy to numbers because they are facts. And this often belies the more common kind of process of understanding numbers that, actually, is a process. You go through a production and the way to understand numbers is that there's a series of decisions that are made for why you need that number in the first place, what data to collect, how to analyze it, how to frame the analysis, how to communicate the analysis. And that was kind of the reason for putting that one first is because it kind of sets up the whole sort of approach that we have, which is to say, it's not that numbers are not important, it's just that often they look more to be of a fact than they actually are. And that seems to be a consistency across how people view numbers.

Trumanitarian Transcript Template.docx 14/23

Yeah, and I would argue that that is not just true in the humanitarian world, but across the board. I think we get much more specific when we come to number two, which reads, crisis break knowledge, numbers are unreliable when we most need them. I think this is very precise.

Yeah, and I would argue that that is not just true in the humanitarian world, but across the board. I think we get much more specific when we come to number two, which reads, crisis break knowledge, numbers are unreliable when we most need them. I think this is very precise. Yeah, and I think this is where it comes down to this idea of data being something that emerges from an infrastructure and how that infrastructure, including labor, buildings, networks, internet connections and ability to travel. When you have something that's a crisis, then you have this kind of moment where the previous rules didn't apply and the previous data that could be collected can't be collected either because the crisis throws something that we've not dealt with before, like COVID, which we refer to in the special issue of kind of understanding, how do we conceptualize it? How do we calibrate what this crisis is? But also, the crisis throws up real problems in how you collect that data. So, it's sort of this double whammy that links into a later point around what you're now looking for, what has become important because of the crisis, but also how is the crisis then affected, how that can be collected?”

And for us practically speaking, we know that we have a very limited time window to get the money. In two weeks, nobody will care about this. And so there needs to be a compelling story out there while the cameras are on, or we will not be able to fundraise.

Number three is also very interesting. Numbers might expose social injustice. Sometimes they hide it.

Yeah, I think this is the classic mantra with numbers is that they can sort of reveal this reality. And if you use them to reveal injustices, then they can sort of really lay bare the injustice that needs to be addressed. But coming back to the idea of how numbers can frame particular topics, Joel, who took the lead on this section, I thought, you know, sort of makes a really interesting comparison here to say, if you talk about just people who are in need of humanitarian aid, you position their need as sort of almost divorced from the context within which it's set. Whereas if you talk about how a certain amount of, small amount of people own a certain amount of money or assets and a large amount as a much larger amount, then you create this sort of relational dynamic where you can point to the cause of the injustice being inequality, rather than just referring to poverty as this sort of a-contextual kind of social injustice, but you're not really left with a way to actually engage with it, other than thinking, okay, well, they need help, rather than thinking structurally, well, how has it emerged that some people need help and some people don't need help? And I think that's what really comes through the way that data is framed, and that's where it becomes particularly important to think of that kind of positioning through lines of communication.

Number four, information travels fast, numbers travel faster. I have to say that for academics, you guys are very good at soundbites. I really like this, it's very cool.

Yeah, I feel like there was sort of, I think me and Joel would sort of chuckle a bit when we were writing these, and often I'd write sort of overly wordy, kind of headings or quite boring headings, and Joel will say, come on, Brendan, make it more exciting, we need to capture people's imagination, so I'm glad it works. And yeah, we were sort of thinking about this notion of numbers as a language, which I think is essentially the best way to understand numbers as rooted in a mathematical language. And then thinking about, well, how does that language differ compared to other languages when we set it within a broader sort of media ecosystem? And we sort of then kind of conceptualized it as this notion of having tiers, so you'd have certain languages that may be known in a particular area of the world, but outside of that, there's not really an ability to understand that language. So that language often travels slower. Then maybe you have something like the English language that is more widely known and it travels quicker again.

But when it comes to numbers, there seems to be this sort of ability for numbers to travel almost the quickest of any type of information and almost when you think about it like that, you think the number travels so quick and then almost lagging behind is this kind of other language that provides context to that number, whether it's in English or whether it's French or German or whatever language that you'd be speaking in that context almost takes a little bit of time to catch up conceptually to then mean that these numbers can almost have this kind of runaway sense of meaning. They can be spotted, they'll travel to the point, to the point the quickest, they'll be spotted the most easily and then they'll almost circulate through this system in a way that's devoid from that other language that's trying to contextualize it.

Exactly. And I think disconnecting time between different information spaces can have some negative effects. Right now, we are all looking at Israel and Gaza and look at the role that numbers play there, how many people were taken hostage, how many people are killed on both sides.

I think Ukraine is another example where we are almost every day looking at numbers, how many people got killed, how many kilometers did they take today. And really, this hyperfocus distorts the nature of our understanding. And this hyperfocus distorts our understanding of what's really going on.

It really is precisely wrong when you look at the big picture. It sometimes reminds me of the Tour de France, actually, and the way that some TV stations cover that event. The focus is on who is in the lead right now and how many seconds do they have down to the peloton. And really does not tell you anything about the race's underlying dynamic or who will win. They now have a one minute and 52 second lead, but we all know that they will be caught in 30 kilometers. But that very rapid numbers game, I find very unhelpful and again, tending to be precisely wrong.

Yeah, and I guess to take your analogy further is to say from, especially what you were saying earlier, that it's kind of like an inverted Tour de France because it's like the race is only meaningful for 30 seconds. Who can break away and be ahead in the exact right moment. And it doesn't matter whether they get caught or not, because in that short period of time, they've gained a lot of attention and maybe attracted more funding. So then they rejoin the pack and then there's another event that means that just breaking away is the important part. It's not the overall race necessarily important in terms of how to critically engage with this stuff.

Right. Number six: all data are local.

Yes. So, this comes from the Loukissas statement to say, ‘all data are local’. And I found it a really good way to kind of understand data not as this kind of ethereal global kind of set of knowledge that exists in some sort of cloud system that allows us to kind of, you know, yeah, from locate things at a local level, but almost have this idea that data is this global thing that is unmoored from local contexts. And what I think is important about that, all data are local, is how much data we see as global is locally produced. So, GDP is a good example, I think, often for people would imagine that it's sort of this external body that goes into each individual country and assesses their books and says, okay, well, you know, we can reliably say that you have a higher GDP than this other country. But the reality of it is that there's each individual country doing their national accounting, which can rely on different definitions of what counts as GDP. There's sort of three main definitions, but some countries do one, some countries do two, some countries do all three, but pick one out of them, do all three, and then average them. And the idea here is to really emphasize that if that happens at a national level, then national level data will have these sorts of local variations. So, it's to set it within this local context. It links into this idea of institutions and the role institutions have in producing data.

Yeah, and I guess that's number seven. Machines and formulas do not exist outside society and politics. So I guess the implication is that we have to understand how a number was produced and where it was produced.

Yeah, exactly. I think it's to really emphasise this notion that data that is produced is linked to kind of what I outlined in the book in terms of the life of a number, to think in terms of needs of a number. So why is that number needed. How is the data collected? How is it analyzed? How is it communicated? How does it gain meaning in society and to think of sort of algorithms AI?And to think of algorithms, AI, data, within that framework of society and politics encasing that data. But I guess with data bounds, saying that data encases society and politics as well. So, this sort of relationship between the two.

Would it be right to say that this really speaks to issues around comparability of, let's say, for example, Venezuela and Burundi. If we have a severity measure that says how big and how bad those two crises are, it's hard to compare them because they were produced in different contexts. But I guess at the same time, if you take for granted that, but I guess at the same time, if they are produced with the same bias every time, it is possible to compare Venezuela over time or Burundi over time because the bias, in a sense, is constant so you can see differences. Would you say that's right?

Yeah, and I think this is the kind of, the thing that we wanted to make sure we didn't fall into a trap of this relativist trap with data where people say, well, you know, if you can only, you have to set all data production within local context. There's no way you can compare even between one county to another because there's different local context. And I think that relativism is like quite good for a kind of theoretical exercise or teaching students to kind of really break down their assumptions about what data is. But I think it has to be built up from that point to say, well, data is still a form of knowledge that we want to use. So how can we use it in a way that's the most useful and the most accurate? And like you said, if someone consistently miscounts or consistently uses a certain method that isn't approved by– you know– isn't internationally recognized, you can know something about changes within that data. It's just to say that you can't compare it directly without making a lot of adjustments to other sets of data.

Brendan, I begin to get the feeling that you are actually a quantitative realist yourself.

Yeah. Yeah, it's funny, isn't it? It's like, it's quite hard to...

I think quantitative realism has such a power over how we process the world that it's very, very difficult to step outside the notion that when you see three apples, that that notion of three exists independently to humans. You know, it's kind of, that's the alluring nature of it. And I think often what I see in a lot of the literature around critical data studies is that people say, well, you know, we can't know anything with data, so let's sort of disregard it, go back to a previous era where we don't use data. And it seems kind of absurdist to me. For me, the critique is to develop a way to understand and critique and engage with numbers that's meaningful and relies on, like engages with our experience instead of kind of throwing the baby out with bathwater with data kind of doesn't seem like there's much point in it.

And I guess if we take #8, it speaks very well to that. It says there are things set in numbers that would not be taken seriously if said otherwise.

Yeah, I really like this one. I– again, I can't take credit for this one. This is what this one's Joel's. But it's really good. It emerged out of a conversation that we had about how countries are ranked. And often whenever you see a ranking of countries that has something to say positively about whether it's standard of living, or most desirable place to live or happiest place on Earth, then, Not always, but generally you see sort of–

Denmark.

Yeah, exactly. I can see a smile on your face. It must be true, but you see, you see Norway, Denmark, Sweden, you know, Scandinavian countries come out top, which again is to say, well, there must be something there that that works. But what works is set within how you judged that ranking, right? So, it works within that framework. And for me might seem very logical, for you might seem very logical to someone from the Caribbean might seem slightly absurdist when they go through how this is counted. But it's less kind of the accuracy of that. And it's more the ability that you can say that using mathematical language, and you couldn't say it in other terms.

So that I couldn't say, Scandinavia is better than the Caribbean without coming under some severe flack, at some conference or some talk at university. But I could say that they rank above the Caribbean, the Scandinavian countries rank above the Caribbean countries. And it'd be seen as almost like I was deferring this kind of claim to science or to data, which can make that claim and it can make that assertion because it's built on evidence. But the evidence framework is as subjective as my claim that Scandinavian countries are better than Caribbean ones. And that's the thing that I find really fascinating what you can claim with numbers and what's acceptable.

So number nine says, sometimes we do not need more data, we just need to know the data that exists. Why is that in there?

“Yeah, so this was, it's in there because there's this real push often within the humanitarian sector, but again, kind of reflects a kind of broader push towards this need that as long as we have more data, we'll somehow understand the topic better if we have more data. And this maybe has like a more alluring or common sense approach when we put it in the social sciences often because people will be like, oh yeah, well, we spent loads of time thinking qualitatively about this and experiences and meaning, but we need some data to actually understand whether these things exist. But what I found was, and I found this quote slash like article by Brenner, who was a Nobel Prize winner in the sciences, I think biology, and they basically say, look, we keep on getting more and more data, but we have no idea what it's saying. And we need a model, we need a theory to understand this data. And I think often within the sciences, but also social sciences, humanitarian sector, when data is quite easy to gather through certain technological devices and when it's needed in order for say, funders to give money to a particular project, or through accountability to make sure that you've actually been doing things right, there's this push to just keep on producing data, but there's less time to say, well, what does this data actually mean? And often for me, I think it's a harder mental task to come up with these sorts of theoretical frameworks to analyze the data. So often it's something that's not taken up as much, but I think a biologist saying it really emphasizes that, okay, this is something that actually applies across a lot of different sectors, because you'd expect the biologist to say, well, we've got the data, so we can just make a claim off it.

And I have to say that I really agree with that. Sometimes data collection that we do is almost ritualistic. It has a lot to do with the street credibility of being able to say, oh, you know, we went out there, we collected the data, and now we can document how bad it is. And the way in ACAPS we would express this, number nine is make sense, not data. Very cool. Let's see what number ten says. There are things that cannot be expressed in numbers, but matter nevertheless.

Yeah, this was one. Yeah, I took more of the lead on two. It kind of links in maybe to point eight to say what numbers can claim, but this really gets at the limits of quantification. And we refer here to the estimated number of Rohingya refugees who were killed through the retrospective mortality surveys. And to say these numbers are incredibly important to understand the severity of a particular context. But what quantification does in its sort of operating logic is look for things that are easy to categorize and then they become quantified and then it becomes more common sense to then quantify them again. And there's nobody saying that you shouldn't be counting death as this biological transition from a living human person to somebody who's no longer living. That is essential to capture that. But it's to say, well, what is not captured when we only focus on that?

So what's not captured is the family and community rituals that would go alongside that, the trauma that you would have that would be different if it was somebody who was a distant cousin who you hadn't met compared to a close family member. And it's to say that numbers, however much they try to, maybe through a like hurt scale of your level of trauma you experienced from this event, they try to access these kind of human experiences but are fundamentally flawed and don't give us the essence, the experience. They miss something. It's poetry and data. It's kind of these two polar opposites of human expression and data is fundamentally ill-equipped to capture these ambiguities and these emotions in the way that other types of expression can.

Brendan, I think these are 10 extremely useful things to remember when you're making decisions in the humanitarian sector and you have something that looks like an evidence base but may not be that solid. I think we really need to deconstruct the role that numbers play in humanitarian decision-making, but at the same time, I also have the feeling that this is very complex. So what would be your advice to all of us who are somehow engaged in using numbers in our daily work life, making decisions, allocating resources? What's best practice here and how do we navigate this very complex issue?

Yeah, for me, I'd say from my research, there's a number of different things I feel like I can share. I think this notion of quantitative realism, I think is important to come back to because as I put in the conclusion of the book, then quantitative realism needs to be both respected as a technical exercise. We can't lose sight of the fact that numbers are produced and need good institutions, they need experts. We do have disciplinary rules on how to do statistics and they're not rooted in purely social constructionism for the sake of it. They're rooted in good practices to try and understand something about the world. But there is a risk of getting drunk on those numbers to the point where you can't see other realities that are expressed by non-quantitative forms of knowledge. So it's, for me, this constant balance between paradoxes and yes, we need some certainty in terms of making policy decisions, but to see numbers as better defined by paradoxes rather than by absolutes. They're both ideological and scientific. They're both emotive and rational, certain and uncertain. But I'd say kind of that as kind of a fundamental level, I guess, to engage with it. There's the notion of data bounds that I think is quite important to say, well, what can I say in this space? What can I not say? And am I not saying the other thing because it doesn't have an evidence base or because people would see it as sort of absurdist that you would talk about a topic in this way? And is there a way I can address that? And then I'd say a bit like what you were saying, really these 10 points almost could map over to a kind of like 10 point checklist that's introduced to a process where you're dealing with data to say, okay, have I thought through these things? They're not definitive, you know, kind of yes, no situations, but say, okay, well, did I actually, am I actually forgetting that during crises, we have to rapidly respond to something, I'm not sure if I can catch them, but is the infrastructure there to give me enough certainty in this data? Am I saying something here that I would be uncomfortable saying without data? So then is it okay for me to say it? I think having that almost as a checklist can work quite well with the fundamental understanding of this quantitative realism, data bounds tied into that. I also don't think it'd be particularly onerous. I think it's just almost a reminder of things that people are already doing, what can sometimes forget to do when the pressure's on.

Brendan, thank you so much for coming on Trumanitarian. Thank you for sharing your insights into how data is used and what we as humanitarians must be aware of when we jump around with all these huge numbers. I really appreciate your work. I think it's a very thoughtful piece of research you've done. And I think it's important that we as practitioners, as a community of practice, do not just throw numbers around, but really reflect on how we can be, or how we can contribute towards principled humanitarian outcomes in the way we use numbers and inform humanitarian decision making.

Yeah, thank you, Lars. I've really enjoyed the conversation as well. And thanks for having me on. And I really hope that some of the conversation resonates with your listeners. And if they have any sort of follow-up questions or want to reach out, positive or negative, you know, my door is always open.

Open. Yes. We'll make sure to include both links to the research you mentioned as well as your e-mail in the show notes.

Thank you, Lars Peter.