Voice samples can be analysed in order to identify unknown persons in tapped telephone conversations, audio chats and video files. If the technology were applied to internet nodes, then it would be of particular interest to intelligence services.
The Speaker Identification Integrated Project (SIIP) on the use of speech biometrics by the police, co-financed by the European Union (EU), has successfully passed its final test. This was announced by the international police organisation Interpol in a press release. SIIP’s objective is to identify and locate “criminals and terrorists” through the analysis of their voices.
A total of 19 authorities, companies and institutes are involved in SIIP, including the Italian Ministry of Defence, the University of Groningen and the companies Nuance and Airbus. The police organisation Interpol, of which 190 states are members, is the intended end-user of the project. Other interested parties include the Italian Carabinieri, the German Federal Criminal Police Office (BKA), the Portuguese Criminal Police and the British Metropolitan Police. The project is scheduled to be terminated until April 2018. Following tests in the field, the participants are now in the process of drafting their final report.
Analysis of VoIP, SatCom and the internet
Speaker identification using SIIP is based on software that determines characteristics such as gender, age, language and regional origin from a speech sample and compares these with a database of available samples. One of the advantages of this method is that speaker comparisons are also possible in foreign languages. The final field trial was conducted last week at Interpol’s headquarters in Lyon. The Metropolitan Police and the Portuguese Criminal Police provided files containing voice samples for the test. These were then compared with audio recordings of tapped conversations and with internet content.
SIIP can be used to automate the analysis of VoIP telephone calls, enabling subjects to be placed under surveillance, even when they use different identities. The project description lists Skype, Viber, Tango, ooVoo and Google Talk as popular VoIP providers. Conventional satellite phone calls can also be processed by SIIP to establish in tapped conversations whether one and the same person is using multiple different SIM cards to make phone calls. This method can also be used to search audio and video data on the openly accessible Internet. This is intended to allow investigators to track voice samples of suspects or defendants in other audio files or on the internet. In SIIP, integrated data mining programmes are used for this purpose. In a video shown during a conference, this is likened to “finding a fish in the ocean”.
Using speaker identification, those involved in the SIIP project intend to improve the legal watertightness of evidence in court. In investigations, authorities could use SIIP to identify networks with other suspects (“mapping/tracing the suspect”). Police officers are able to network and exchange information via an SIIP Info Sharing Center at Interpol for the purpose of exchanging information at the international level.
Research for intelligence services?
SIIP is suspected of being used for intelligence research. Internet nodes could thus be monitored for specific voice samples in order to be informed in real time when a person under surveillance makes VoIP calls, takes part in unencrypted chats or can be heard in videos. The Israeli company Verint Systems coordinates the project and is said to have close links to the intelligence services Mossad and the NSA. One of the partners involved in SIIP is the Austrian company Sail Labs, which received half a million euros from the EU for its research work. According to a report by the German newspaper DIE ZEIT, Sail Labs was, at least initally, a front company of the Federal Intelligence Service (BND) that had emerged at the turn of the millennium from an earlier outfit established by the BND agent Christoph Klonowski. According to other media reports, the BND founded or took over further front companies at that time.
Klonowski, who at that time operated under the cover identity “Stephan Bodenkamp”, took part in an EU project promoting speech recognition himself and even oversaw it. The European Commission had a demo version of a platform developed for 2.1 million euros that went by the name SENSUS. According to research conducted by Christiane Schulzki-Haddouti, Klonowski worked at the BND as director of “machine translation and artificial intelligence” during that period. The European Commission was even aware of his membership of the BND. The camouflage authority itself had initiated financing by the Commission. Sail Labs may still be using patents for applications that were developed for the BND.
“Acoustic comparison” since the 1980s
In the past decade, the German Federal Office of Criminal Investigation (BKA) developed an automated “speaker recognition system” (SPES). From 2007 to 2010, the BKA’s Forensic Science Institute brought the experiences made to bear at EU level. Under the title “Correlation between phonetic-acoustic-auditory and automatic approaches in forensic speaker identification”, the project spearheaded by the BKA sought to compare automatic analysis with the “acoustic-phonetic” method also applied by many European police authorities.
This “acoustic-phonetic” method is based on manual measurements of pitch and resonance frequencies, and has been used at the BKA since the 1980s. “Acoustic trace material” is compared by the investigators, a process that is also supported by software. The phonetics researcher Hermann Künzel, who introduced the method at the BKA in the 1980s, referred to this process as “acoustic comparison”. Künzel claims to have identified RAF member Peter-Jürgen Boock as one of the kidnappers of Hanns Martin Schleyer by using this method. Interviews which the RAF had conducted and recorded with Schleyer were used as voice samples. This voice identification had allegedly helped get to the bottom of the kidnapping of Jan Philipp Reemtsma, the extortion of food company Nestlé and the identity of department store blackmailer “Dagobert”.
Dubious association for enhanced speech biometrics at BAMF
Now retired, Hermann Künzel joined the association DITS. center e. V., which was established one year ago. As a network of computer scientists, signal engineers, former military and intelligence personnel, the association seeks to promote language technology for the police and for customs and immigration authorities.
DITS.center e. V. is supported, among other things, by funds from “pilot customers”. Klaus Ehrenfried Schmidt, who led the SENSUS project for Europol at that time, and (at least for a temporary period) Christoph Klonowski, who is apparently the former BND agent of SENSUS, are involved in this association. DITS.center e. V. probably selected the Federal Office for Migration and Refugees (BAMF) as one of its “pilot customers”. Applications advertised by the association include “language, dialect recognition, speaker identification and speaker verification”.
BAMF has now introduced automated dialect recognition. As confirmed by a spokeswoman at BAMF on request, Klonowski from DITS. center e. V. approached the authority last year and suggested “collaborative efforts in the field of voice biometrics”. BAMF was also in contact with Künzel.
Image: SIIP graphics.