DeepLI™ - Lawful Interception Software

---

• Overview
• Product Info
• Product Description -- Lawful Interception Features
• Site Models
• Telecom Mode
• Analytics Mode
• Audio Quality
• Stream Alignment
• High Capacity
• Test Regimen
• Application and Diagnostic Table
• API Interface
• Packet Logging
• mediaMin and mediaTest Demos
•    OpenLI ⁴ support ◳
• Published Source Code
• Related Applications
• How To Get a Quote
• Related Items
• Tech Support

Overview

DeepLI™ (Deep Lawful Interception) is a new generation of Lawful Interception software with emphasis on site model flexibility, extreme high capacity, superior audio quality, and deep learning. Site models include on-premise, co-located, and SaaS (cloud LI). Addition of deep learning support is ongoing, including speech recognition and diarization.

DeepLI is currently deployed in Tier 1 LEAs and carriers ¹. Here are key reasons why:

• Extreme high capacity and reliability on standard Linux servers. For on-premise implementations, maximum session capacity exceeds 500 concurrent sessions on standard servers ²
• Highest audio quality -- innovative algorithms for inter-stream alignment, stream gap concealment, packet loss mitigation, audio frame loss concealment, and RTP stream merging (multiple endpoint merging)
• Robust -- continuous stress testing applied in Signalogic labs, with a minimum of 600 hrs prior to a new release
• Ready-to-use -- full featured reference applications for basic applications, and minimum API interface callable from C/C++ for high capacity, application-specific use cases
• Ultra secure -- developed and tested entirely in the US, in labs based in north Texas and the Bay Area, with no outsourcing. Zero dependencies on open source except for codecs and Linux. No third-party libraries. Protected source codes are not stored on AWS, Azure or any other untrustworthy outside data center. Cyber and security reviews of source code can be arranged ³
• Test regimen -- due to many customer relationships over many years, Signalogic maintains a database of 100s of RTP media pcaps, covering an enormous range of test cases, including all types of codecs, multiple streams, child streams (RFC8108), with/without packet arrival times, early media, call-waiting gaps, DTMF, etc

DeepLI is based on shared library modules within SigSRF software (Github page) ◳, including packet media, streaming, voice codecs, algorithm, diagnostic, inference, and others. SigSRF provides the high capacity / high reliability foundation for DeepLI. SigSRF software functionality includes:

• Telecom Mode or Analytics Mode operation
• Complete packet flow including pcap or UDP I/O, packet and multiple SSRC handling (RFC8108), jitter buffer, transcoding, voice algorithms, variable ptime, and numerous RFCs
• Signal processing including real-time RTP stream merging, stream deduplication, insertion of user-defined signal processing, FLC (frame loss concealment), AGC, echo cancellation, and more
• Extreme concurrent session capacity on x86 Linux servers
• OpenLI ⁴ support ◳

Demo versions of the mediaMin and mediaTest reference applications run on x86 Linux servers. Customized, application-specific trial / eval versions are also available. Online demos include codecs for EVS, AMR-NB, AMR-WB, and G711. Customized trial / eval versions can include additional codecs. For more information on the EVS codec, see the HPC EVS Codec product page ◳.

¹ "Tier 1" means carriers approximately the size of T-Mobile (around 45B annual), and for LEAs means FBI level national agencies
² For example, each session with two (2) EVS wideband streams and one (1) AMR-NB stream on an HP DL380 running Ubuntu Linux
³ Depending on business case discussions conducted under NDA
⁴ OpenLI ◳ is an open source project maintained by the University of Waikato ◳

Product Info

Product Description - DeepLI Features

Site Models	On-Premise, Co-Located, SaaS (cloud LI)
Operating Modes	Telecom, Analytics. Both modes support packet send rates slower or faster than packet ptime, and Analytics mode includes an option for pcaps without arrival timestamps
Applications	mediaMin: ready-to-use LEA application, with output options including UDP/RTP, wav file, and pcap   mediaTest: test and measurement application for codecs, audio quality, stress tests, custom packet and jitter buffer settings, and more   All applications are reference applications, including best practice techniques and methods, full C/C++ source code and header files, Makefiles, and detailed, extensive notes and comments   All applications provide a well documented command line interface, including numerous quick start and functional test examples. See section 2, Command Line Usage, and section 6, Functional Testing, in SigSRF documentation ◳. See also "APIs" in this table
APIs	Two levels of API are available: (i) fine-grained control at the packet/media thread level, and (ii) simplified, minimal interface based on session setup and packet queue push/pull. The mediaMin application is based on the minimal API. Section 3.1, Minimum API Applications, in SigSRF documentation ◳ contains a source code example showing a packet push/pull loop implemented in 10 lines of C code
I/O	I/O includes UDP/RTP, pcap, and waveform file. IPv4 and IPv6 supported. wav, .au, compressed bitstream, and other file formats supported
Capacity	50 sessions per core 1
Stream Processing	Stream group setup and management, real-time RTP stream merging and deduplication, user-defined signal processing. RTP stream merging output options include real-time UDP packet flow, unified mono wav file and pcap, and per stream pcap and either mono or N-channel wav file. Stream deduplication removes reverb when the same source appears on multiple streams
Session Management	Static and dynamic session creation from (i) session configuration files, (ii) SDP info, (iii) IRI info, or some combination. Dynamic session creation includes an RTP audo-detect option. Live session control and modification
Codecs	EVS, AMR-NB, AMR-WB, AMR-WB+, G711, G729, G726
Jitter Buffer	Advanced options include dynamic re-ordering depth, packet loss mitigation, burst/gap compensation, SID and media packet repair, dynamic stream creation (RFC8108), on-demand flush, RFC7198 (temporal), live stats. Inter-stream packet rate mismatch and fluctuation compensated up to 20%
Multithreading	Multistream, multisession, multichannel, fully re-entrant, XDAIS compliant. See also "Capacity" in this table
DTX, SID, CNG	DTX can be applied and controlled on per session basis, or automated depending on codec type. VAD available for older codecs without inherent DTX support
PLC and FLC 2	Includes (i) codec decoder output and (ii) stream merging audio output
Voice/Packet Handling	DTMF, echo cancellation, sampling rate conversion, variable ptime, tone generation, and other audio and RTP processing
Stress and Functional Test	Documentation includes dozens of functional and stress test command lines and reference pcaps, along with test validation information
Logging	Event and packet logging. Packet logging includes SSRC group collation, analysis of dropped, missing, and duplicated packets, and timestamp matching
Energy Saver	Packet/media thread inactivity detection and throttle down, with user-specified settings

¹ Xeon E5-2660 v0 @2.2 GHz, two (2) EVS wideband streams and one (1) AMR-NB stream per session, transcoded to G711. See section 5.0, High Capacity Operation, in SigSRF Documentation ◳ for more information
² Packet Loss Concealment and Frame Loss Concealment

 

Site Models

Site models supported by DeepLI include On-Premise, Co-Located, and SaaS (cloud LI). The diagrams below show platform architecture and data flow for these models.

Notes about the above diagams:

1. SaaS models are flexible, depending on data privacy, session capacity, and other application-specific requirements
2. High capacity figures quoted here and in SigSRF documentation ◳ require private servers, typically deployed in an On-Premise model

 

Telecom Mode

DeepLI™ telecom mode supports direct handling of bidirectional IP/UDP/RTP traffic. This mode is sometimes also referred to as "clocked" mode, as a wall clock reference is required for correct jitter buffer operation. Examples of telecom mode applications include network midpoints such as SBC (Session Border Controller) and media gateway, and endpoints such as handsets and softphones. Typically telecom applications have hard requirements for real-time performance and latency. A telecom mode data flow diagram is shown below.

DeepLI™ Lawful Interception telecom mode flow diagram

 

Analytics Mode

DeepLI™ analytics mode supports indirect handling of IP/UDP/RTP traffic, where traffic is encapsulated or "one step removed", having been captured, copied, or relayed from direct traffic for additional processing. This mode is sometimes also referred to as data driven or "clockless" mode, the latter description referring to jitter buffer packet processing either wholly or partially without a wall clock reference. In general, analytics mode applications operate after real-time traffic has already occurred, and process uni-directional traffic, although depending on the application it may be necessary to reproduce or emulate original real-time behavior and handle output endpoints. In addition to Lawful Interception, examples of analytics mode applications include web IT data analytics such as speaker identification and automatic speech recognition (ASR). An analytics mode data flow diagram is shown below.

DeepLI™ Lawful Interception analytics mode flow diagram

 

Audio Quality

DeepLI™ places utmost emphasis on audio quality. A number of areas in the software contain packet, media, and signal processing algorithms designed to achieve the highest possible audio quality. These include:

• media and SID packet repair. Packet loss gaps up to 5 ptimes for media and 3 ptimes for SID are repaired. Techniques include PLC and backward/forward looking estimation based on surrounding context
• dynamic jitter buffer, including late packet holdoff, resync, and burst flushing
• interstream gap handling, including pastdue data for stream group processing (e.g. stream merging)
• stream group output frame loss concealment (FLC)
• optional stream deduplication algorithm, to remove reverb between similar streams

In addition to detailed packet analysis and logging, DeepLI also support audio quality metrics and visualization. Below are some examples.

Wireshark capture showing 1 sec timing markers added to stream group output

Because Wireshark will show time axis variations reflecting packet delta interval irregularity and timestamp gaps, any variation in spacing between timing markers indicates a potential problem. This makes the timing marker method a useful visualization tool for quickly finding timing issues, either in SigSRF software or application software.

Wireshark capture showing output buffer boundary timing markers added to stream group output

In the output buffer boundary marker display above, the dynamic nature of the stream group gap handling algorithm can be seen in action, as it dynamically adjusts buffer sizes depending on incoming audio frame rate variation and underrun (gaps).

Audio Quality Testing

Audio quality testing is part subjective and part quantitative. Subjective testing takes a lot of time and several human listeners to agree that yes, quality is acceptable. In the extreme case, a panel of listeners is convened over several days and a Mean Opinion Score (MOS) is determined. This complexity makes quantitative testing methods desirable, although difficult. DeepLI incorporates the following quantitative audio quality test methods:

• injecting visual markers in media output; the above displays show two examples
• end-of-call run-time stats summary and event logging. Run-time stats summaries can be use to keep functional test case records, for later comparison to see if any stats have changed due to application or system modifiecations, or comparisons between systems
• per stream and stream group wav output, for subsequent examination by media analysis tools such as Audacity, Matlab, Hypersignal (see examples below), etc
• alarms and event logging for packet cutoff and thread preemption, to aid forensic analysis of audio quality disruption events such as network outages, Linux system issues, etc

Below is a run-time stats summary example.

DeepLI end-of-call run-time stats summary example, showing dynamic session creation of 3 EVS streams and one AMR-WB stream, child stream creation (RFC8108), and stream group output

As shown in the above example, run-time packet stats include packet loss, timing, and out-of-order, stream group underrun and overrun, jitter buffer stats, and more.

Below are examples of stream group wav output after sliding FFT spectrographic analysis using Hypersignal software, along with the corresponding time domain waveform displayed in Wireshark. These examples show clean, high quality output, with no wideband noise vertical lines that would indicate gaps, glitches, or other audio artifacts.

Frequency domain sliding FFT spectrographic analysis of stream group wav output (using Hypersignal heatmap option)

In the spectrographic analysis display below, early media ring tones can be identified by their stable frequency content over 100 msec to 1 sec, unlike speech which is typically "quasi-stationary" for no more than 20 msec.

Frequency domain sliding FFT spectrographic analysis of stream group wav output (using Hypersignal heatmap option). This example includes early media ring tones

Stream Alignment

Applications that merge RTP streams from multiple endpoints, such as conversational ASR and diarization, remote work groups, and lawful interception, often require extremely accurate stream alignment. Merged output must assemble in real-time, maintaining exactly regular sampling intervals. Endpoints with different latencies, packet send rates, and other variations make this task challenging. DeepLI™ handles this with a series of algorithms developed in US based labs due to multiple customer engagements in telecom, analytics, and embedded system applications.

Below is an overlay of Wireshark screen caps showing an example of multiple streams merged into a combined, real-time audio output.

DeepLI™ example of multiple stream alignment and real-time merging

 

High Capacity

DeepLI™ was architected from inception for high capacity, multicore, concurrent thread operation. Below is an htop screen capture showing 504 concurrent sessions running on a 32-core Xeon server.

DeepLI™ htop display showing high capacity packet/media processing, 504 concurrent sessions

The above htop display shows 12 mediaMin application threads and 10 packet/media threads running concurrently. Application threads are hyperthreaded (highlighted in yellow) and packet/media threads are not hyperthreaded (highlighted in red); i.e. one thread per physical x86 core. The area highlighted in orange indicates unused logical cores to allow maximum packet/media thread performance. Each packet/media thread is handling 48 to 51 sessions.

For more information, see section 5.0, High Capacity Operation, in SigSRF Documentation ◳.

For low capacity applications, license pricing scales depending on application needs. There is no requirement to pay for high capacity operation if not needed. But -- it's always there, available to be turned on should it be needed.

 

Test Regimen

Functional Tests

Due to many customer relationships over many years, Signalogic maintains a database of 100s of RTP media pcaps, covering an enormous range of test cases, including:

• all types of codecs
• packet loss -- intermittent and sustained up to 10%
• sustained high and low send rates
• multiple streams
• multiple child streams, both SSRC creation and resume (RFC8108)
• with/without packet arrival times
• early media
• call on-hold and call waiting gaps
• sequence number and timestamp wraps
• DTMF RTP events
• long duration (multiple hours)

Stress Tests

Signalogic continuously runs stress tests on separate servers dedicated to 24/7 testing. These include repeating, random combinations of functional tests, high capacity tests, "extreme conditions" such as 1 msec packet push rates, packet gaps, removals, and other impairments, artificial Linux thread stalls, and others.

Capacity Tests

The SigSRF documentation includes a number of mediaTest and mediaMin command lines for capacity tests, including multiple application threads, multiple packet/media threads, and combinations. See the High Capacity section, above.

 

Application and Diagnostic Table

DeepLI™ application and diagnostic functionality is shown in the table below. General functional areas are shown at left, then further subdivided. Diagnostic functionality includes test and measurement capabilities provided by the mediaTest application. As noted above, both mediaMin and mediaTest are reference applications based on SigSRF shared library modules (SigSRF Github page ◳ ).

 

    Notes

•  CC = Communication Content (aka ipMMCC)
•  CNG = Comfort Noise Generation
•  DTX = Discontinuouis Transmission
•  FLC = Frame Loss Concealment
•  IRI = Interception Related Information (aka ipMMIRI)
•  PLC = Packet Loss Concealment
•  SDP = Session Description Protocol
•  SID = Silence Insertion Descriptor
•  VAD = Voice Activity Detection

 

API Interface

DeepLI™ provides two levels of API:

• Fine-grained control at the packet/media thread level
• Simplified, minimal interface based on session setup and packet queue push/pull

The mediaMin application is based on the minimal API. API header files are availble for inspection in the online SDK demos or upon request. Below is a mediaMin source code excerpt showing a packet push/pull loop implemented in 10 lines of C code.

 

 

 

Note that packet output queues are separated into jitter buffer, transcoded, and stream group. For more information on the above source code excerpt, and packet push/pull queues, see SigSRF documentation ◳ section 3.1, Minimum API Applications.

 

Packet Logging

Upon application request, DeepLI generates a detailed packet log, including input packet history, jitter buffer output packet history, input packet statistics, and comparison / analysis of input vs. output. Below is an example packet log, edited for brevity.

Ingress Packet info for SSRC = 0x6506e546, first seq num = 8, last seq num = 13023 ...

Seq num 8              timestamp = 11200, pkt len = 6 SID
Seq num 9              timestamp = 13120, pkt len = 33
Seq num 10             timestamp = 13440, pkt len = 33
Seq num 11             timestamp = 13760, pkt len = 33
Seq num 12             timestamp = 14080, pkt len = 33
Seq num 13             timestamp = 14400, pkt len = 33
Seq num 14             timestamp = 14720, pkt len = 33
Seq num 15             timestamp = 15040, pkt len = 33
   :
   :
Seq num 85             timestamp = 176000, pkt len = 33
Seq num 86             timestamp = 176320, pkt len = 33
Seq num 88 ooo 87      timestamp = 176960, pkt len = 33
Seq num 87 ooo 88      timestamp = 176640, pkt len = 33
Seq num 89             timestamp = 177280, pkt len = 33
Seq num 90             timestamp = 177600, pkt len = 33
   :
   :
   :
[ additional input streams ]

Jitter Buffer Packet info for SSRC = 0x6506e546, first seq num = 8, last seq num = 51340 ...

Seq num 8              timestamp = 11200, pkt len = 6 SID
Seq num 9              timestamp = 11520, pkt len = 6 SID CNG-R
Seq num 10             timestamp = 11840, pkt len = 6 SID CNG-R
Seq num 11             timestamp = 12160, pkt len = 6 SID CNG-R
Seq num 12             timestamp = 12480, pkt len = 6 SID CNG-R
Seq num 13             timestamp = 12800, pkt len = 6 SID CNG-R
Seq num 14             timestamp = 13120, pkt len = 33
Seq num 15             timestamp = 13440, pkt len = 33
   :
   :
   :
[ additional output streams ]


** Packet Stats and Analysis **

Stream groups found = 1, group indexes = 0

Stream group 0, 3 streams

  Stream 0, channel 0, SSRC = 0x13883145, 156 input pkts, 156 output pkts

    Input packets = 156, ooo packets = 0, SID packets = 0, seq numbers = 0..155, missing seq numbers = 0, max consec missing seq numbers = 0
    Input packet loss = 0.000%
    Input ooo = 0.000%, max ooo = 0

    Output packets = 156, ooo packets = 0, seq numbers = 0..155, missing seq numbers = 0, max consec missing seq numbers = 0, SID packets = 0, SID R packets = 0, repaired SID packets = 0, repaired media packets = 0
    Output packet loss = 0.000%
    Output ooo = 0.000%, max ooo = 0

    Packets dropped by jitter buffer = 0
    Packets duplicated by jitter buffer = 0
    Timestamp mismatches = 0

  Stream 1, channel 2, SSRC = 0x6506e546, 12912 input pkts, 51333 output pkts

    Input packets = 12912, ooo packets = 1346, SID packets = 5652, seq numbers = 8..13023, missing seq numbers = 104, max consec missing seq numbers = 1
    Input packet loss = 0.805%
    Input ooo = 5.212%, max ooo = 2

    Output packets = 51333, ooo packets = 0, seq numbers = 8..51340, missing seq numbers = 0, max consec missing seq numbers = 0, SID packets = 5702, SID R packets = 38320, repaired SID packets = 53, repaired media packets = 51
    Output packet loss = 0.000%
    Output ooo = 0.000%, max ooo = 0

    Packets dropped by jitter buffer = 0
    Packets duplicated by jitter buffer = 0
    Timestamp mismatches = 0

  Stream 2, channel 4, SSRC = 0xb6ef05cc, 50862 input pkts, 51325 output pkts

    Input packets = 50862, ooo packets = 8876, SID packets = 0, seq numbers = 65..51389, missing seq numbers = 463, max consec missing seq numbers = 2
    Input packet loss = 0.910%
    Input ooo = 8.726%, max ooo = 2

    Output packets = 51325, ooo packets = 0, seq numbers = 65..51389, missing seq numbers = 0, max consec missing seq numbers = 0, SID packets = 0, SID R packets = 0, repaired SID packets = 0, repaired media packets = 463
    Output packet loss = 0.000%
    Output ooo = 0.000%, max ooo = 0

    Packets dropped by jitter buffer = 0
    Packets duplicated by jitter buffer = 0
    Timestamp mismatches = 0

Note in the above packet log example, out-of-order input packets are marked as "ooo".

mediaMin and mediaTest Demos

The SigSRF SDK download page (Github) ◳ contains free, limited demo versions of the mediaMin and mediaTest reference applications, including source code, along with several mediaMin and mediaTest command line examples ◳. In addition to Lawful Interception functionality and OpenLI support ◳, the demos provide test, measurement, diagnostic, and event / packet logging examples. Demo limitations include number of concurrent sessions, number of packets processed, and functionality. To remove or relax functionality limits, customized trial versions are available upon request and subsequent to business case discussion. All C/C++ source code for both applications is published.

Published Source Code

Published C/C++ source code includes:

• SigSRF header files
• all source and Makefiles for the mediaMin and mediaTest applications¹
• packet/media thread source demonstrating full use of the SigSRF API
• signal processing source for stream group output
• stream deduplication
• insertion of user-defined signal processing

Published source code is stored on Github and is available in online demo packages.

¹ As noted in API Interface above, the mediaMin application relies only on the "minimal API" interface.

Related Applications

Other available software related to DeepLI:

• SigSRF Software Github page ◳
• SigMRF Software ◳
• NFV Transcoding ◳

How to Get a Quote

Enter the fields shown above, including:

• CPU Type
• License Type
• Product Configuration

Then click on "Get Quote / Info" to receive pricing via e-mail.

Related Items

1) Codec Wav Samples, before and after encode/decode ◳

2) mediaMin and mediaTest demo download (Github page) ◳

mediaTest high capacity media session transcoding screen capture, showing EVS codec performance benchmarking in diagnostic mode

3) Surveillance Video Suspect Detection and Tracking ◳

Vidilant suspect detection software screen capture of surveillance video analysis

4) ffmpeg Acceleration ◳

FFmpeg screen capture with c66x acceleration enabled

Tech Support

Signalogic's engineering staff designs, develops, maintains, operates, and tests software and hardware in the company's in-house lab, using servers from HP, Dell, Supermicro, Artesyn, Advantech, and others, and Linux installations including CentOS, Ubuntu, Red Hat, Wind River, Gentoo, and more. Customers can submit technical questions via e-mail, phone, or Skype chat.

Signalogic engineers are experts in server and embedded system development. Contract development is available, for example product based quotations with NRE modifications to customer specs. Signalogic is a member of third-party programs for HP, Dell, Intel, and Texas Instruments. A high level of expert, sustained, and detailed tech support is a distinct advantage of working with Signalogic.