Zusammenfassung

Chan Junis in Gaza, 2024. – CC-BY-SA 4.0: Jefunky Software that automatically evaluates large amounts of data in real time is changing the way military targets are selected. An analysis of the Lavender system shows that it violates international humanitarian law. We speak with Taylor Kate Woodcock and Rainer Rehak about the consequences of „AI“ in armed conflicts.

Im Detail

Software that automatically evaluates large amounts of data in real time is changing the way military targets are selected. An analysis of the Lavender system shows that it violates international humanitarian law. We speak with Taylor Kate Woodcock and Rainer Rehak about the consequences of „AI“ in armed conflicts.

Chan Junis in Gaza, 2024. – CC-BY-SA 4.0: Jefunky This software is vital in a violent machinery and searches for potential bombing targets in Gaza and Lebanon: The Lavender system is designed to mark all suspected operatives in the military wings of Hamas and Palestinian Islamic Jihad. It detects people, vehicles and other objects in real time.

Israel’s military has called it a „game changer“ in yielding targets, as decision-making becomes faster and more automated.

The new Gaza documentary „Naza“ describes the secret methods used by the Israeli army, as told by soldiers involved. The film by Juval Abraham and Rachel Szor, which received the special jury prize at the Venice International Film Festival this weekend, aims to examine Israeli war crimes and uncover systematic offenses.

Such systems are trained using known datasets of individuals in order to identify behavioral patterns. When they then encounters new individuals in the war zone who exhibit similar behavioral patterns, those individuals are flagged as potential targets.

In this way, Lavender marked tens of thousands of people who were classified as members of military adversaries. This system was used together with another software-based decision support system called „The Gospel“, which suggested buildings and structures as targets.

The civilian damage and human suffering caused by such a system raise ethical and legal questions. We speak with scientists Taylor Kate Woodcock and Rainer Rehak about their research paper „Automating civilian harm: On Israel’s use of the AI-enabled targeting system Lavender in Gaza and International Humanitarian Law“, published in June.

They explain why the use of such software leads to systematic violations of international law.

Taylor Kate Woodcock is a PhD researcher in public international law at the Asser Institute at the University of Amsterdam. Rainer Rehak is a computer scientist at the Weizenbaum Institute for the Networked Society in Berlin.

Dieses Interview gibt es auch auf Deutsch.

An „AI-enabled“ data processing system

netzpolitik.org: Dear Taylor, dear Rainer, you recently wrote about the targeting system „Lavender“ and the International Humanitarian Law. Please describe to a layperson what the „Lavender“ system is and what it is supposed to achieve for the Israeli military?

Taylor Kate Woodcock. Taylor Kate Woodcock: The Israeli military describes the Lavender system as an advanced system to track down and target militants within Gaza. It is an „AI-enabled“ data processing system used by the Israeli military to calculate targets for later attacks.

The system itself is therefore not an attack system, but part of an earlier stage in what experts call the „kill-chain“. The results of Lavender are then fed into coordination and timing systems and eventually end up in the attack schedules of fighter jets, bombers, artillery, drones or ground troops, which then carry out the actual attacks.

Lavender works by producing a risk score between 1 and 100 for each person in Gaza signifying the probability of membership in or closeness to militant groups, such as the militant arm of Hamas (Al-Qassam Brigades) or the Palestinian Islamic Jihad. Since everyone in Gaza gets this score, a threshold can be set in the system and everyone above this score becomes a target for military strikes.

netzpolitik.org: How is the Lavender system’s scoring work in detail?

Rainer Rehak.

Rainer Rehak: Control resulting from the Israeli occupation of Gaza allows Israel to have vast amounts of data on virtually anything within the Gaza Strip, from satellite imagery, cell phone locations, video surveillance footage, geodata, communication data and metadata, building information, movement data, payment information, drone data, and even information on family relations and social structures, as we know from independent journalistic projects from the US and Israel.

All this heterogeneous data is connected and combined to serve as the data foundation for a method called positive-unlabeled learning (PUL) by an „AI“ system. This means that data from a few actually confirmed cases of militants is used to statistically analyse the rest of the data and produce some kind of statistical similarity across all data relative to the known cases.

This statistical „likeness“ is then transformed into a scale of 1 to 100 points for each person in Gaza. Once you have such a number, a threshold can be set to produce targets.

Around 1,000 potential targets every day

netzpolitik.org: In your paper, you wrote about another „AI“ system, „The Gospel“. What is the „The Gospel“ system for?

Rainer Rehak: There is much less information on the Gospel system, but generally speaking, it is a system used for classifying buildings in Gaza according to their use, from civilian private or commercial use to military use, the data mentioned before is used accordingly.

netzpolitik.org: What sources and data did you rely on for your paper?

Taylor Kate Woodcock: We relied upon several sources of investigative journalism that use anonymous sources from within the IDF and US contractors and access to classified documents. In particular, we rely on the analysis of Yuval Abraham, as well as the work of Michal Biesecker, Sam Mednick and Garace Burke, journalists with the Associated Press.

The challenge in researching the use of AI in military operations is the secrecy around how these systems are really used in practice and the outcomes that result.

We do find that the findings of these independent media outlets are also supported by the technical plausibility of the systems described, the fact that they are based on anonymous sources with real practical experience, and the fact that the descriptions also provide an explanation for the terrible images of harm in Gaza that we see repeatedly reported.

netzpolitik.org: Gospel and Lavender focus on high volumes of targets. What does a high volume of targets practically mean? How many targets does the system find per day, for example?

Taylor Kate Woodcock: Recent estimates suggest that around 1,000 potential targets were identified every day during the first two years of the armed conflicts in Gaza and Lebanon. Between 7 October 2023 and the end of 2025, a total of 850,000 were detected in real time, according to Israel’s largest arms supplier, Elbit Systems.

Such numbers suggest that the Israel Defense Forces (IDF) had targeted up to or over half the entire population and infrastructure in Gaza.

Rainer Rehak: The daily numbers vary because the Israeli military also chooses the threshold according to the availability of means to attack, such as bombers and ammunition. On days when all initial targets had been hit but there were still fighter jets ready, they simply lowered the threshold until the desired number of new targets was calculated.

Or in other words: The system was frequently reconfigured to match the military means available. In terms of international law this shows that the targets calculated never really had a material basis for actually justifying the strikes. All of Gaza was basically ordered into one big kill list and then attacked from top to bottom with everything available.

Gegen die unheilvolle Verflechtung von Informatik und Militär

Altering military targeting

netzpolitik.org: How is human supervision ensured?

Rainer Rehak: According to the reports each target was evaluated by a human supervisor, but they only spent a maximum of 20 seconds per target, often only checking whether the target was male. Here we can see what the whistleblowers meant by saying the emphasis is on quantity and not on quality, which is noteworthy given the fact that such decisions mean the difference between life and death.

Taylor Kate Woodcock: A general risk with the use of AI systems is that human operators overtrust them and do not scrutinise their outputs, a phenomenon called „automation bias“.

Risks of automation bias mean that even when human supervision is integrated into targeting using AI systems – albeit within extremely compressed timeframes – the output can inform targeting decisions without real scrutiny or verification.

The scale of high-volume targets, the speed at which these are produced by AI systems, and the risk of automation bias significantly alter targeting by reducing the time available for human verification of targets. However, targeting is subject to legal obligations under international law, which requires that the lawfulness of targets is verified and measures are taken to prevent civilian harm.

Looking at the time spent on each target, such obligations were clearly not met, as whistleblowers confirmed, describing their role as merely „being a stamp of approval“.

netzpolitik.org: How are civilians treated using Lavender?

Rainer Rehak: We have to note, that the whole statistical approach does not really allow for such a distinction between civilians and lawful targets, because it is neither clear how exactly the score is calculated nor how the threshold is decided. One manual internal test showed that the system itself had an error rate of around 10%.

In addition to target generation, the system also explicitly allows for configuring acceptable civilian deaths, sometimes coldly called collateral damage. With assumed junior operatives (lower score) the accepted number was set to 15–20, and with higher ones the number was reportedly „hundreds“.

So, for 100 targets the system was explicitly configured to cause at least 1,500 to 10.000 civilian deaths, which is likely to be the minimum given that the Gaza Strip is as densely populated as Manhattan in New York.

The timing was often chosen to hit targets not during the day, but at night, when they are at home with their families, causing even more civilian harm. This specific timing system is called „Where’s daddy?“.

Taylor Kate Woodcock: We analysed this conduct in detail in our paper and came to the conclusion, that it systematically and gravely violates international law.

„Civilian harm is pre-programmed into the system“

netzpolitik.org: How does the underlying technical infrastructure look like, and who provides it?

Rainer Rehak: The whole data on Gaza is estimated to be around 13.6 petabytes (13,600 terabytes) which is not easy to handle, especially if you want to train statistical models, let alone Large Language Models which is also being done. The digital infrastructure is provided by Google and Amazon under the project name Project Nimbus costing 1.2 billion US dollars.

The data integration of the heterogeneous data sources is done by Palantir. Furthermore Microsoft, Cisco, Dell and Red Hat/IBM provide additional IT services around the kill-chain. This shows the huge role that the private sector are playing in the integration and use of AI systems in armed conflicts.

netzpolitik.org: You write in your paper that the IDF’s systems cause mass civilian harm and constitute systematic violations of international humanitarian law. What does systematic mean here?

Taylor Kate Woodcock: In international humanitarian law there are three core principles regarding the legal conduct of armed conflict:

Clearly distinguishing between lawful targets (e.g. combatants, militants, or military objects and buildings) and civilians, only attacking the former is lawful.

Taking constant care and feasible precautions to avoid or minimise harm to civilians.

Avoiding attacks where the expected incidental harm to civilians is excessive in comparison to the anticipated military advantage of the attack.

In analysing the system we found that civilian harm was not the exception in one-off situations caused by mistakes, but that the whole design, setup and configuration of the systems necessarily produce the amount of civilian harm (people and infrastructure) we eventually see reported in the media and as admitted by the military itself.

Conclusively, one could say that civilian harm is pre-programmed into the system, and looking at the numbers could even be seen as a core function of it.

Such a system violates all three principles gravely and in a systematic fashion. Targeting processes designed in this way show reckless disregard for the selection of lawful targets and the protection of civilians, suggesting that this harm is automated and indiscriminate.

We can therefore not say that the AI system „malfunctions“, but that levels of harm are in fact anticipated through the deliberate configuration and use of these systems.

This repetition for thousands of targets throughout the conflict and explicit choices around system configuration and use suggest that this harm is systematic, as well as having a structural impact on targeting processes that do not adequately ensure the protections required by international law.

netzpolitik.org: If the violations of humanitarian law are „pre-programmed“ and are not a side effect but a „core function“ of the system, does that mean that no legal use of such a system is possible? Or could it be adapted?

Taylor Kate Woodcock: The Lavender system and the whole kill-chain are apparently designed for „mass assassinations“ (Yuval Abraham) using bombs in densely populated areas.

If one takes civilian harm and international law seriously the system in its current form, including the statistical score, the arbitrary threshold, the configured civilian harm, and other properties described at length in the paper, cannot be legally used.

And if we hypothetically think of adapting the system, we might, for example, require the civilian harm configuration to be set to zero, but then the system would likely not produce zero valid targets, since Gaza is an urban area. We think, an adaptation of this system into legality is virtually impossible.

Rainer Rehak: Given this evaluation we find it especially disturbing that the Lavender system has supposedly been co-developed with Israeli universities, as Maya Wind of the University of California has described it in her book „Towers of Ivory and Steel“.

netzpolitik.org: Are there more armed conflicts with illegal use of AI systems for targeting?

Taylor Kate Woodcock: Unfortunately, these intuitions seem to be confirmed in other instances of the use of AI systems for targeting in various current conflicts, including in Iran. There, we also see that the use of AI systems reshapes targeting processes in a way that puts structural strain on the protections required by international law.

We see this for example in reporting about the United States’ bombing of the Minab School in Iran, which shows how the use of AI for targeting can exacerbate challenges with outdated intelligence information, human verification, and the protection of civilians.

How violations of international humanitarian law can be tried

netzpolitik.org: What follows from your analysis of the Lavender system?

Rainer Rehak: In our understanding, the systems in question should be immediately put out of operation. This demand also applies to their use in Iran or Lebanon by Israel, and potentially to other actors such as the United States. Second, we definitely need more transparency regarding such systems especially when they involve probabilistic components.

Third, the export of such systems must be prohibited, since this is a common practice for Israel, as we know from the works of journalist Antony Loewenstein (The Palestine Laboratory). Fourth, we have to look beyond the clean language of „precision strikes“ and „targeted killings“ and pay attention to the material conditions and effects on the ground.

Not doing this would result in falling for a „moral red herring“ and focusing on the wrong things, because with Israel having one of the most technically sophisticated armies in the world, the use of Lavender and Co. with all its implications should be considered deliberate.

Considering the fact that Gaza has been flattened in what a UN Commission of Inquiry and many relevant international organisations call a genocide, Lavender is only a small mosaic, but possibly symptomatic of the warfare in Gaza at large.

netzpolitik.org: Your paper ends with the question of whether such systematic violations of international humanitarian law should be considered war crimes, which should be examined by courts. Who could bring such violations before a court?

Taylor Kate Woodcock: Grave violations of international humanitarian law can be tried as war crimes, either by international courts and tribunals, such as the International Criminal Court in The Hague, or by individual countries, exercising something called „universal jurisdiction“, allowing them to prosecute crimes that are so significant they constitute offenses against all of humankind.

At present, the International Criminal Court has issued an arrest warrant for the Israeli Prime Minister Benjamin Netanyahu for allegations of the war crimes of starvation as a method of warfare and of intentionally directing attacks against the civilian population; and the crimes against humanity of murder, persecution, and other inhumane acts from at least 8 October 2023 until at least 20 May 2024.

We believe that the use of AI in Gaza should be a significant component of this case, evidencing systematic and grave violations of international law.

In addition to individual criminal responsibility, international law also has frameworks for the responsibilities of countries, exercised through the International Court of Justice, also in The Hague. In 2023 South Africa filed a case at the ICJ accusing Israel of violating the Genocide Convention.

Other international bodies such as the UN have also made statements on the genocide, which are not legal processes but nonetheless demonstrate the international scrutiny on Israel for its conduct in Gaza. An example is the finding of a UN Commission of Inquiry that Israel committed genocide in Gaza.

Aside from these processes that condemn unlawful conduct after the fact, international law also plays an important preventative function. A huge range of countries are now seeking to integrate AI into their military capabilities, and should take international law seriously when doing so.

Though the use of systems such as Lavender by the IDF in Gaza is a very specific context, nonetheless other states and supervisory bodies should look to this to see why much more caution and transparency around these systems is needed to respect their obligations under international law.

Rainer Rehak: Finally, this conflict will probably not be resolved by better and more advanced weapon systems, but only by political action, by acknowledging realities and by addressing the core issues like the illegal occupation of Palestine and fundamental rights for all people. If we do not put our focus and means there, we will probably only see a change in the horrific war technologies used.

International law was born out of the lessons from World War II and we should take its principles seriously, regardless of the actor.

netzpolitik.org: Thank you very much for your time!

Die Arbeit von netzpolitik.org finanziert sich zu fast 100% aus den Spenden unserer Leser:innen.

Werde Teil dieser einzigartigen Community und unterstütze auch Du unseren gemeinwohlorientierten, werbe- und trackingfreien Journalismus jetzt mit einer Spende.