Komatsu keeps ML balls rolling, signs Nvidia to boost site safety

Construction and mining equipment vendor Komatsu is continuing its AI and Machine-Learning initiative with a partnership with Nvidia, which will use Nvidia’s GPU and AI tech to analyze construction and job sites – to improve worker safety and operational efficiency. The deal follows a partnership with Cloudera, which Komatsu signed to boost its customer support offerings, through data analytics and predictive maintenance.

Nvidia’s Jetson AI Platform will be the heart of the new system, which will use GPUs installed in heavy machinery to visualize and analyze these job sites – ingesting video feeds and sensor data to ‘serve as the brain’ to improve safety and productivity. For Komatsu, which does a lot of its business in its domestic Japanese market, improving the construction industry’s safety record is of great importance.

According to Japan’s Federation of Construction Contractors, some 300 deaths and 15,000 injuries occurred in the Japanese construction industry last year – which is around ten-times the deaths that the UK’s industry suffered, but apparently around four-times fewer injuries. Compounding the safety record is Japan’s severe labor shortage, due to its aging population. Around a third of its 3.4m skilled workers are expected to leave in the next decade – a big problem, but also a big opportunity for the likes of Komatsu to address using automated offerings.

With Cloudera, Komatsu is hoping to build a machine-learning analytics platform, to help better push its equipment to global mining customers. Based on Microsoft’s Azure cloud, and using the Cloudera Enterprise system, Komatsu will be able to provide customers with monitoring services and predictive maintenance, to boost efficiency. Komatsu says the system has doubled the longwall mining production hours – a huge improvement.

The Nvidia deal is part of Komatsu’s ongoing SmartConstruction initiative, which it began in 2015. The main thrust of that project is collecting data related to the job sites and workers, and so far, some 4,000 sites in Japan have been connected. There are plans to expand the project abroad, but the new Nvidia partnership means that the richness of data that can be collected and processed should rise significantly.

The Nvidia hardware at the heart of the new deal is Jetson, a credit-card sized board that houses the Tegra SoC. The first iteration of Jetson was launched in April 2014, called the TK1 and housing the Kepler-architecture-based K1 SoC. It was succeeded by the TX1 in November 2015, and later the TX2 in March 2017 – which uses the Tegra X2 SoC that is built around a 6-core CPU (a quad-core ARM Cortex-A47 CPU and a dual-core Nvidia Denver), and a 256-core Pascal GPU, with 8GB of RAM – in a 7.5W package. In thousand-unit volumes, the TX2 costs $400.

Nvidia envisions the Jetson family being used to power edge-devices like drones from SkyCatch and CCTV cameras – which will create 3D models of the construction sites. To improve productivity, the system will track the interactions between people and objects (including Komatsu’s equipment), to ensure that machinery is used in the most efficient manner.

The maps that SkyCatch’s drones create will be used in conjunction with OPTiM’s management software tool that will identify both individuals and machinery, using video pulled from the site surveillance cameras. Both of these companies are Komatsu partners and are in Nvidia’s Inception AI-focused startup program.

As for Komatsu’s machinery, Nvidia says that Jetson will be used in stereo camera systems housed in the driving cabs of its construction vehicles, for real-time driver instruction – to alert them to a risk such as hidden workers or nearby machinery, and risks they may have forgotten about, such as power lines or pipes.

In this current iteration, Komatsu isn’t going as far as handing over control of the vehicles to automated systems powered by Nvidia, but that evolution seems inevitable. Nvidia is hotly pursuing the conventional car market, but industrial equipment is another very viable market.

For future projects, Komatsu and Nvidia are planning high-resolution rendering and virtual simulations of construction and mining sites, which will serve as the basis of automated machinery. Freeing up drivers and operators to carry out other tasks on construction sites would be very beneficial in markets like Japan, which are short on labor anyway – but from a safety perspective, removing human workers from the risk-zone could open up new opportunities in environments that might be considered too hazardous with current equipment and techniques.

“Artificial Intelligence is sweeping across industries, and its next frontier is autonomous intelligent machines,” said Nvidia CEO Jensen Huang, at the GTC Japan conference, where the deal was announced. “Future machines will perceive their surroundings and be continuously alert, helping operators work more efficiently and safely. The construction and mining industries will benefit greatly from these advances.”