Posted by Zeeshan Zia, PhD
Machine Vision as a technology has been around for at least 40 years. These solutions have traditionally enabled automated camera based inspection of parts to detect part damage, identify missing components, and uncover process issues.
More recently, new startups and big technology companies alike have also entered this space, mostly focusing on ease-of-deployment and leveraging the technical advances from the field of “deep learning” to improve the accuracy of defect detection.
While these single image-based solutions find broad applicability on fully automated production lines as well as on “end-of-the-line” quality stations, they have significant drawbacks when it comes to manual sub-assembly, packaging, high-mix assembly, and main assembly work.
In this blog, we highlight three key shortcomings, and propose an alternative way of automating “in-process” quality control by live “video analysis” which solves several of these challenges.
1. Quality cannot be assessed by just looking at the final product
In practice, the vast majority of manual builds involve some parts that get layered over during the sub-assembly process. This means that correct assembly cannot be assessed merely by looking at the final product.
Let’s look at a few real-world examples:
In the following process video, a valve is assembled where the key is a tiny part that gets hidden under gears (around 00:23). Meanwhile, a missing key is precisely the biggest quality challenge on this line. Now it may be possible to force the operator to have a traditional machine vision system “sign off” on each valve build, by putting the shaft with just the key under a camera, and click on a screen. However, that solution requires significant investment and modification to the process, while causing the throughput to go down.
Here’s another example of assembling a planetary gear system. Note how layer upon layer of “carriers” are placed and gears and grease are installed on these carriers. Again, it is obviously not possible to inspect this assembly by looking at the final image.
Finally, here’s an example of an electronic gadget assembly. Note how layers upon layers are assembled and can no longer be inspected just by looking at the final product. It maybe possible to setup a specialized system designed on top of traditional part inspection solutions, where the worker clicks on a screen to inform the system that certain steps have been completed, and the system inspects the quality of just those tasks. However, such systems inevitably reduce throughput by as much as 2-3x, and expect the worker to change how they behave.
The only scalable solution to the problem of quality control in manual assembly is live video analysis. A machine vision solution must be able to observe the entire process of assembly, not just the final product, and do so without requiring user interaction.
2. Issues Caught Too late
Even when an “end-of-the-line” visual inspection solution is sufficient to stop a defective product from leaving the factory, the product may need to be sent to the re-work line or be scrapped altogether.
Traditional machine vision solutions do not improve first time yields. They may be able to reduce product returns, however the costs associated with defects do not go down.
Again, the solution is to perform such visual inspection “in-process” via live video analysis and alert the operator as soon as an assembly mistake is detected to help them fix the mistake right there and then.
3. Lack of insights on avoiding defects
Yet another issue with capturing defects via image-based final product inspection is the lack of insights on why those defects happened in the first place, and how to avoid them.
If an operation is going to install cameras on a process, why not capture the entire cycle and capture all relevant statistics alongside a detailed video record. Video analytics solutions empower quality engineers to perform detailed root cause analysis as every cycle is stored in video and automatically annotated, as well as provide them with analytics to improve the process.
Live Video Analytics in action
Technological advances over the past five years in deep learning for video analysis, reduction in cloud storage prices, and rapid improvement in computational resources available on ordinary desktop/laptop computers have enabled real-time processing of video data at the edge.
We leave you with an example of video analytics running live on the electronic device assembly process we mentioned earlier. Note, how such a solution is able to track every individual step of the process as well as measure the amount of time being spent on these steps. If any mistake gets made, the system can be setup to provide real-time alerts to the worker. Talk to us, if you want a live demo of our video-based in-process quality control solution.
Huy takes our product roadmap and figures out what’s achievable with today’s AI advances, and what we need to invent to solve our customers’ pain points.
Huy led teams of scientists and engineers at NEC and invented technologies for driver assistance systems that are part of 500,000+ Honda cars. He is an inventor on more than 20 US patents, and his research papers are cited by every lab from MIT, Stanford, and CMU to Apple, Google, and Baidu.
Michael works closely with our marquis customers to understand their needs, and figures out the best solution that immediately adds dollars to their bottom line.
Michael has almost 20 years of enterprise sales experience. He has held sales leadership roles at pioneering AI and computer vision startups.
Andrey takes inventions and converts them into innovations. He leads our product engineering team to turn our scientific prototypes into robust customer-facing products.
Andrey co-founded and successfully exited a startup to immigrate to the US. He spent his next eight years building the core data science infrastructure for Amazon’s supply chain planning team, and the CV/ML infrastructure for Microsoft’s HoloLens organization.
Zeeshan focuses on delighting our customers by understanding their needs and defining product and strategy to meet and exceed these needs. He works closely with our sales leadership to grow our customer base.
Zeeshan was a scientific leader at Microsoft’s HoloLens division, where he invented AI technologies to assist frontline workers across industries. Earlier, he shipped IoT platforms and visual defect detection software for manufacturers at Siemens and MVTec GmbH.
