Invention for The use of data mining or machine learning to improve target detection and classification as a deterrent against unmanned aerial systems

Invented by Dwaine A. Parker, Damon E. Stern, Lawrence S. Pierce, Xidrone Systems Inc

The Xidrone Systems Inc invention works as follows

A system to provide integrated detection and deterrence of an unmanned vehicle, including but not restricted to aerial or similar technology unmanned vehicles, using a tracking element an identification element an interdiction element and a deterrent element. The elements contain sensors which observe in real-time quantifiable information about the object of concern to create an assessment on the risk or threat the object poses to a protected zone of interest. This assessment is based upon the application of machine-learning and data mining protocols and techniques to the data set related to the object. Third parties can access this data via APIs and interface protocols. The deterrent component, using machine learning or artificially intelligence algorithms, selects actions from a menu of deterrent options. These actions are not limited to the actions included in this embodiment but also include those available from third-parties such as specialized “hunter drones”. Although designed to be autonomous, a Human In The Loop can override automated system solutions.

Background for The use of data mining or machine learning to improve target detection and classification as a deterrent against unmanned aerial systems

Commercially, “Small Unmanned Aerial Systems” (sUAS), weighing under 20 kg or 55 lbs, commonly referred as “drones”, are available to the public. Some drones are not airborne, like those that travel under or on the water. ?Drone? As shown in FIG. The unmanned aircraft, ship, or spacecraft described in 1A can be any of these. Unmanned aircraft, ships, or spacecraft may be controlled by remote control, onboard computers and/or human operators, and can operate autonomously, or semi-autonomously. UAV, UAS and sUAS are also included in the ‘definition’ section. See also UAV, UAS, sUAS and RPA in the?definition?

Even if drones are operated with no malicious intent, there can be significant risks to the commercial and general aviation industry, government, private, and public interests. Unmanned systems are also used to intentionally violate privacy in personal, commercial and educational activities, as well as entertainment, sports, and government interests.

Unintended consequences of commercially available, off-the shelf (OTS), unmanned aerial systems include the ability to be used for terrorist or criminal activities. There is a genuine need for an integrated system and method of detecting, tracking, identifying/classifying and deterring the approach of a commercial unmanned aerial system or other drone towards a location where personal, public, commercial, educational, athletic, entertainment, governmental and military activities occur and where a commercial unmanned aerial system or other drone could potentially be used for invading privacy, or carrying out terrorist and criminal activities within a civilian or other environment.

FIG. The system shown in FIG. 1 is non-limiting and can be used to detect, track, classify, and interdict a UAS, such as a UAS 44. Once the system has been applied to a UAS, or another drone, it is called the target 44 because the UAS, or other drone becomes the target to be sensed, tracked, and in some embodiments possible interdiction.

In the example shown, different sensors are used using different technologies to detect the target. In one non-limiting embodiment of the sensors, they include a commercial radar 43, an optical, infrared, and/or laser distance finder 16, an omnidirectional receiving antenna (RF), 14 and a directional receiving/direction finding radio frequency antenna 12, This equipment can be mounted separately, such as on mobile trailers or the same structure or fixture.

The outputs from these sensors 43, 16 and 14 are analyzed by a sensor fusion process (described below) in order to detect the target 44 and determine a threat level. In certain non-limiting embodiments a human may be in the loop. In some scenarios, the system can be completely automated by using algorithms that apply machine learning protocols and data mining protocols. The system will deploy interdiction 10 if the threat level is higher than a threshold. This can be used to disable or destroy target 44. The system is therefore a vital tool in protecting critical infrastructure, such as airports and bridges, powerlines, factories, nuclear/power stations, shipping facilities, stadiums, military bases, large public venues etc. from the threat of target 44.

The example non-limiting embodiments are a fully integrated detection and interdiction system that leverages multiple sensor systems, including but not restricted to: Radars, Electro-Optical and Infra-Red imagers (EO/IR), Laser Range Findings (LRF), or Light Detection and Rangings (LIDAR), to derive the necessary location and spectral properties of a commercial drone or unmanned aerial vehicle (UAV). Unique analytical processes and algorithms utilize the data collected by a sensor suite or from outside sources to identify, categorize and specify the waveform, pulse size, and frequency that will be generated by an RF contra-measure. This allows for the exploiting of inherent vulnerabilities in the onboard electronics of an object such as a sUAS (designated as 44) shown in FIG. A small Unmanned Aerial System can be less than 20 kg (55 lbs) and have a low profile. The narrow beam RF countermeasure can be programmed to send RF signals that are customized to the target detected. This will disrupt and overwhelm the unmanned systems navigation and control systems of the subject, resulting in the airborne unmanned system crashing, landing or returning to the launch location based upon the subject’s internal processes.

Air defense is traditionally the domain of the military and not the police or private security. The advent of small, affordable, and capable sUAS weighing less that 20 kg (55 pounds) creates a requirement to detect and deter unauthorized use. Small drones have different flight characteristics, detection signatures and profiles. They are unlikely to be detected or deterred using conventional radar, electronic countermeasures or kinetic systems, without risking significant collateral damage. The existing UAS countermeasures are designed to detect and destroy larger aircraft, such as the Iranian Ababil 3 and Chinese Sky-09P or the Russian ZALA 42108. These large and midsized unmanned aerial vehicles will not be used in a domestic environment. A traditional radar/air defence system designed to scan in the skies is a very effective way to detect and track midsized and large military drones. The military countermeasures to combat UAS/UAVs that target friendly positions include lethal offensive systems equipped with deadly and destructive weapons such as artillery, lasers, electromagnetic beams and bullets. The military countermeasures also include powerful RF systems that are designed to jam, disrupt or spoof SATNAV (GPS), (satellite positioning/global navigation system) signals. This approach is prone to causing collateral damage and negative navigation effects for all GPS receivers in the vicinity. The non-limiting embodiments resolves negative or collateral effects through a tailored and precise application of specific RF emissions needed to remotely control these small commercial drones and/or disrupt them without jamming SATNAV (GPS), which many civilian vehicles are now commonly using for land, sea, and air navigation.

Military UAS Countermeasures are designed for and used on a battlefield or in hostile environments. In many cases, using a military solution in a commercial or civilian environment is not suitable or permitted due to government regulations and inherent liabilities. The use of SATNAV/GPS spoofing and jamming can disrupt civilian and military activities, such as cell phones and aviation navigation. This makes it illegal to do outside of a combat zone or military operation. The military’s focus on force protection and counter-battery missions, using battlefield radars that aren’t designed or capable of detecting targets moving slowly at low angles or heights over buildings, trees or just above the horizon is another issue. Countermeasures that only rely on GPS and radar jamming/spoofing or aircraft detection systems do not offer a viable solution for commercially available sUAS. Cost, weight, size and power consumption are all disadvantages of a full military UAS countermeasure. The gap between military and civilian operations requires the development of an integrated multi-sensor countersUAS system. The demand for a successful counter sUAS system that detects, tracks, identifies/classifies and deters against commercial drones without causing collateral damage or interference in a civilian environment is growing exponentially and is directly applicable to other unmanned systems employing similar or different technology.

The exemplary non-limiting implementations herein alleviate the problems noted with the military counter UAS systems and provide a novel, efficient and effective integrated detection, tracking, identifying/classifying and countermeasure solution against small unmanned aerial systems (sUAS) operating in a commercial or civilian environment. These examples can employ deterrence techniques and countermeasures that are better suited for civilian environments. However, they may also benefit from the full range of countermeasures and interdiction methods used by military systems. The implementations described herein also offer increased security, privacy and protection against threats involving small unmanned aircraft systems/vehicles (sUAS/UAV), or other similar manned system?such as ultralight aircraft. They are applicable to government, military, commercial and private concerns. Counter Unmanned Aerial Systems of Systems are provided in the example embodiments. They can be used to detect, classify/identify, track, and interdict or deter small unmanned aerial systems or vehicles.

The example non-limiting system disclosed herein provides an integrated solution that protects from ground level to several thousand feet above the ground surface and laterally several miles. It comprises components using, in part: existing technologies for a different use; multiplexing of hardware components designed specifically for this application; sophisticated built-in algorithms to calculate the exact X (Longitude), Y (Latitude) and Z (Altitude) coordinates for the subject sUAS. Machine learning and data mining can be used to improve the accuracy and reliability of assessment and deterrent systems. This capability allows operation to be completely autonomous or supervised by a system operator (also known as a “Human-in-the-loop” or HiL) who can provide real-time data and imagery to aid in picture compilation and threats assessment, as well as a visual display and the location of the suspected sUAS.

Operating a sUAS counter system in a commercial or civilian environment will often require precision countermeasures, to ensure minimal or zero collateral damage to the area surrounding the sUAS. The non-limiting embodiments have unique capabilities in how they integrate, or “fuse”/correlate, multiple sensors to detect and track sUAS/UAVs, as well as identify/classify them. The system also uses multiple independent sources for acquiring the X-Y-Z coordinates needed to align and direct the automatic antenna alignment system, and to provide the initial tracking/targeting data to the EO/IR (16) system and countermeasure system (10). In some embodiments, three independent sources include the radar system (43) the DF system (14), and the combined EO/IR/LRF system (16). Combining the independent X, Y, and Z coordinate data of each system will provide a precise 8-digit (or more) GPS geo-location/tracking grid coordinate to use in the geo-location, mapping, aiming and tracking systems.

The example non-limiting techniqe herein uses radar in the X band frequency range as one sensor to detect, track and categorize a commercial sUAS/UAV. Radar is used in a unique way to distinguish between the suspect sUAS, and the low-altitude, highly cluttered environment. The K and Ka band are usually optimized for targets at higher altitudes and speeds. The existing art treats the technical challenge of sUAS as any other aerial target, while the non-limiting examples described approach this challenge like it was a ground target. This non-limiting embodiment’s fundamentally different approach provides a unique and novel solution for the detection airborne sUAS systems or signature systems. Due to their high frequency, range and power, as well as other optimized characteristics of radars, most applications of aircraft detection are susceptible to distortions when viewing objects on the ground. This is classified by some as “Clutter”.

The example technology is non-limiting and uses a Laser Range Finder and/or Light detection & Ranging (LIDAR), coupled with a Electrical Optic and Infra-Red camera system (EO/IR), as one or multiple sensors, to detect, track and identify/classify a sUAS/UAV. The radar system (43) provides the initial target information (location) to the EO/IR and LRF systems (16). The X-Y-Z coordinates from the radar are used to align the EO/IR and LRF cameras towards the suspected sUAS. The LRF, a multi-purpose, comprehensive targeting system, incorporates the same tracking, fire-control, and precision-guided munitions capabilities as advanced military weaponry, including fighter aircraft and armored vehicles. The LRF, in conjunction with the EO/IR cameras, provides the digital display for the system operator. It shows the field-of-view and displays the suspected sUAS targets along with important pieces of information including range-to target, target velocity, target deterrent angle and wind speed and direction. The use of fire control technology allows for extreme accuracy at long ranges, even with fast-moving sUAS targets. It also allows non-limiting embodiments to perform precision countermeasures in a controlled civilian environment. Fire control systems work as computers to guide the countermeasures. The LRF will produce an X-Y-Z data point once the sUAS has been tagged. This is then sent to the countermeasures system (14) which automatically aligns the destructive or nondestructive deterrent element (10) (RF, Laser Impulse, Munitions etc.). It is an advantage to current military systems in that it provides the steps necessary for greater safety when operating within a commercial or civilian environment. During the time that the suspect sUAS is being viewed through the EO/IR camera system, the images and heat signatures are compared using machine learning and data mining techniques to the original selected image as well as images resident in internal and external databases with known sUAS images and heat signatures for possible type identification/classification and threat assessment. Video/photo analysis is used to determine if a sUAS has a payload and the type.

The non-limiting example technology used here utilizes components of LRF to link the tracking EO/IR optical with the fire trigger. Targeting technology allows a user to designate a specific sUAS by aligning the reticle and pressing the button. The tracking optic knows exactly what to do when the user tags the target. The trigger and optic work together to release the countermeasure you have selected. The tracking system guides the trigger of the fire control to release the countermeasure precisely at the moment required to impact your target while causing minimal collateral damage in areas around the target sUAS. The link between the tracking optical and the fire-control trigger is made up of dozens microprocessors, electronic, electrooptic and electromechanical components. The image is compared with the selected image when the system triggers the fire control mechanism. Again, algorithms and protocols are used based on data mining and machine learning techniques. The tracking optic will interrupt the triggering signal if the images are not perfectly matched with the designated point or tag. This prevents the transmission of the tailored interdiction signal. When the images have been aligned, the interrupt will be released. This allows the interdiction transmission to take place. The fire control trigger is activated as soon as the system crosses the designated point. This countermeasure is perfectly targeted. This automated fire-control system virtually eliminates any human error due to misaiming, timing, system movement, vibrating or other environmental factors.

One non-limiting example of technology uses Radio Frequency and Direction Finding technology to detect and track, classify and identify a commercial sUAS/UAV. The system scans the area in 360-degrees using a highly sensitive RF receiver to detect any RF signals commonly used between the operator and sUAS. Signal processor filters can be used to eliminate signatures not found in the commercially available population of sUAS. The observed signal characteristics are compared with a database or library of modulation characteristics, such as frequency, pulse width, and duration, to identify commercially available sUAS. Machine learning can be used to train a deep neural network in order to identify a wide range of targets, and distinguish between them and other moving objects like birds. The azimuth of a suspect sUAS will be passed on to other sensor systems when an observed signal is statistically similar or matches an expected sUAS signature. The high gain is directed at that azimuth to further refine the azimuth of the suspect sUAS. This sensor element allows for the non-limiting embodiments to operate passively as required.

The example non-limiting techniqe herein uses a deterrent to deter or suppress an unmanned system, and if it is operated in a suitable environment, destroy the unmanned system. The systems’ detect feature can also be used to track and locate low-flying airborne threats, which are not UAVs or sUASs. This is especially useful for systems that travel on land and water. The non-limiting embodiments’ fused set sensor phenomenologies can detect any future technology, even if it is not detected by conventional air defense systems. The non-limiting embodiments (s) RF/DF systems are designed to accept the state data generated from non-organic sensors, and will integrate this “told-in” data into the target identification process and algorithms.” The RF/DF system is designed to accept state data generated by non-organic sensors and will incorporate this?told-in?

As stated above, this example non-limiting tech is designed to accept, but not require, subject sUAS locations, classifications, or other states data generated by nonorganic sensors. This mechanism is a combination software and hardware that can be used to integrate these components. It provides a solution for uninvited and invasive commercial sUAS/UAV activities regarding privacy, security and terrorist threats. The non-limiting embodiments are connected via secure wired and wireless internal control networks. They can be used to remotely operate the system or add additional sensors or third-party sources, or provide, transmit or supply target data derived from embodiments to third-party systems using defined protocols and APIs.

An example of a non-limiting system (102) for providing integrated multi-sensor detection against small commercial unmanned aircraft systems/vehicles (44), and Unmanned Landborne or Seaborne Systems (ULSS), includes a detecting (103), 104, 105), a tracker (103, 104,105), an identification (103, 104, 105) element and an interdiction (102 ). Each element has sensors that collect data in real-time about the sUAS/ULSS. Real-time data, for example, can be used to control a system function called the automatic antenna align assembly (or ‘Slew-to Cue’). The real-time data, in conjunction with stored data from the system database, or a cloud-based database can be used to produce real-time information and forecasts about the sUAS/ULSS. Third-party data, whether real-time data or data stored in a database, can be supplied to the system to automate the process and determine the best course of action to take against the sUAS/ULSS based upon all available data. The data that is generated, derived, or supplied can include, but not be limited to, information that can detect or alert, track, identify or classify a sUAS/ULSS if it enters sensitive airspace, or a defined area. The system operator can override this course of action if necessary. The system records all data input by the user. This data can then be used to perform Data Mining (DM), which is a process that allows the data to be mined for use in future encounters with sUAS/ULSS activities.

The systems of this non-limiting example technology provide an integrated multisensor system which can be deployed either as a “permanent installation” or as a mobile system on land, sea, or air platforms. As a mobile system, it can be deployed on a land, sea or air platform.

The systems of this non-limiting example technology may be strategically deployed in order to monitor airspace around a protected area such as a place, person, or event. They offer 360-degree azimuth cover extending out from the receiving antennae to a distance of approximately 2 kilometers (6,560 feet), and within the boundaries of the system up to an elevation of about 1,5 kilometers (4920 feet). The distances listed above are averages and can increase with the incorporation of future technologies and optional embodiments. The detection boundary(s) is/are a designated area. The system map can be used to identify, outline and overlay a protected area. This can then be viewed and monitored remotely by the HiL/system operator.

Click here to view the patent on Google Patents.