Tactical infrastructure such as fencing, roads, and lights are important to securing a nation’s border. However it alone is not enough to stop the unlawful movement of individuals and contraband into a country.
“Technology will be the primary driver of all land, maritime, and air domain awareness – this may become only more apparent as [U.S. Customs and Border Protection (CBP)] faces future threats,” according to testimony from CBP officials with a Senate hearing on homeland security in 2015.
And machine vision’s fingerprints are common over that technology. “The details extracted from fixed and mobile surveillance systems, ground sensors, imaging systems, as well as other advanced technologies enhances situational awareness and enables CBP to detect, identify, monitor, and appropriately reply to threats inside the nation’s border regions,” the testimony states.
In the U.S.-Mexico border within the state of Arizona, as an example, Top Machine Vision Inspection System Manufacturer persistently detect and track so-called “items of interest.” Designed to withstand its harsh desert surroundings, IFT is equipped with radar, commercial off-the-shelf daylight cameras and thermal imaging sensors, and microwave transmitters that send data to border agents on the Nogales station for analysis and decision-making.
On all 3 fronts of land, maritime, and aerial surveillance, machine vision companies are providing imaging systems – and, more often, research into the generated data – that meet government agencies’ objectives of flexibility, cost effectiveness, as well as simple deployment in border security applications.
Managing Diverse Conditions – The perennial trouble with vision systems found in border surveillance applications is handling the diversity of the outdoor environment using its fluctuating lighting and climate conditions, as well as varied terrain. Despite the challenges, “you will find places in which you can implement controls to enhance upon the intelligence of the system,” says Dr. Rex Lee, president and CEO of Pyramid Imaging (Tampa, Florida). He points to customers who monitor trains across the southern border of the U.S. for illegal passengers.
“Those trains will need to go under a trellis, which can be built with the proper sensors and lighting to help inspect the trains,” Dr. Lee says. Government agencies tasked with border security use infrared cameras to detect targets during the night and then in other low-light conditions, but thermal imaging has its limits, too. “Infrared cameras work really well once you can use them in high-contrast conditions,” Dr. Lee says. “However, if you’re attempting to pick up a human at 98.6°F on a desert floor that is 100°F, the desert is emitting radiation at nearly the same portion of the spectrum. So customers count on other parts from the spectrum such as shortwave infrared (SWIR) to attempt to catch the difference.”
Infrared imaging works well in monitoring motorized watercraft because the boat’s engine has a thermal signature. “What’s nice about water is the fact it’s relatively uniform and it’s very easy to ‘wash out’ that background and see anomalies,” Dr. Lee says.
But the problem is that the oceans present a huge amount of area to pay for. Says Dr. Lee, “To find out everything is actually a compromise between having a lot of systems monitoring the water or systems that are loaded with the sky, in which case you have the problem of seeing something really tiny in a huge overall view.”
CMOS Surpasses CCD – One key change in imaging systems found in border surveillance applications is definitely the shift from CCD to CMOS sensors as the latter is surpassing the product quality and gratification from the former. To support this change, 2 yrs ago Adimec Advanced Image Systems bv (Eindhoven, the Netherlands) integrated the newest generation of CMOS image sensors – that offer significant improvements in image quality and sensitivity – into its TMX number of rugged commercial off-the-shelf cameras for high-end security applications. TMX cameras keep a maximum frame rate of 60 fps or 30 fps for RGB color images at full HD resolution.
Furthermore, CMOS image sensors are emerging as a replacement for electron-multiplying CCDs (EMCCDs), says Leon van Rooijen, Business Line Director Global Security at Adimec. Thanks to their superior performance over CCDs in low-light conditions, EMCCDs often are deployed in applications like harbor or coastal surveillance.
But EMCCDs have distinct disadvantages. As an example, an EMCCD must be cooled in order to deliver the best performance. “That is quite some challenge within the sense of integrating power consumption and in addition the fact that you need to provide high voltage to the sensors,” van Rooijen says. “And if you wish to have systems operating for any long duration without maintenance, an EMCCD is not the very best solution.”
To solve these challenges, Adimec is concentrating on image processing “to get the most out of the most recent generation CMOS ahead even closer to the performance global security customers are utilized to with EMCCD without all the downsides from the cost, integration, and reliability,” van Rooijen says.
Adimec is also tackling the task of mitigating the turbulence that takes place with border surveillance systems over very long ranges, particularly as systems which were using analog video are taking steps toward higher resolution imaging to cover the larger areas.
“When imaging at long range, you have atmospheric turbulence by the heat rising from your ground, and on sea level, rising or evaporated water creates problems regarding the haze,” van Rooijen says. “We are going to show turbulence mitigation in the low-latency hardware baked into our platform and definately will work with system integrators to optimize it for land and sea applications because they possess the biggest issues with turbulence.”
More Than Pictures – Like machine vision systems deployed in industrial applications, border security systems generate lots of data that will require analysis. “The surveillance industry traditionally has been a little slower to include analytics,” says Dr. Lee of Pyramid Imaging. “We percieve significant opportunity there and have been dealing with a lot of our customers so that analytics are definitely more automated with regards to precisely what is being detected as well as analyze that intrusion, and then have the ability to require a proper response.”
Some companies have developed software that identifies anomalies in persistent monitoring. For instance, when a passenger at the airport suddenly abandons a suitcase, the application will detect that the object is unattended nefqnm everything around it will continue to move.
Even with robust vision-based surveillance capabilities at all points of entry, U.S. border patrol and homeland security must contend with a much bigger threat. “America does a very good job checking people arriving, but we all do a really poor job knowing if they ever leave,” Dr. Lee says. “We know the best way to solve that problem using technology, but that can cause their own problems.
“The right place to do this reaches the Automated Vision Inspection Machines in the TSA line, that you can have a mechanism to record everybody,” Dr. Lee continues. “But that will be expensive because you should do this at each airport in america. Monitoring and recording slows things down, and TSA is under plenty of pressure to speed things up.” Another surveillance option that government departments have discussed takes noncontact fingerprints at TSA each and every time someone flies. “Most of the American public won’t tolerate that,” Dr. Lee says. “They are likely to debate that fingerprinting is just too much government oversight, and will result in a lot of pressure and pushback.”