How It Works

Pathfinder provides a “zero code” programming environment that allows non-technical personnel to set up a new process. The entire setup process takes only a couple hours of user’s time. Also, once set up, we make it incredibly straight-forward to modify a process. In the following, we describe the four key steps in this process.

Setup

Capture

Label

Deploy

Modify

The first step in setting up a new process is to define its Standard Operating Procedure (SOP). The SOP can either be entered on a step-by-step basis as ordinary text. Alternatively, it can be fetched directly from the Manufacturing Execution System (MES) if we have an integration in place with Pathfinder. Users can specify groups of steps that must strictly be performed in order, and they can also specify which steps may be performed out of order. Further, users are able to assign certain steps as “Optional”, meaning that the system won’t issue a real-time alert if those steps are not detected; however when they do appear their timing statistics will appear in the Analytics page. Users can also assign IoT signals to steps e.g. barcode or Light Tower behavior upon step detection. Users can also specify content for Projected Work Instructions if a projector is being used, and they can specify alternative language description of the steps.

Specify

Capture

The second step is capturing videos of cycles of the assembly process. Customers mount a camera and connect it to a Windows computer on the workstation, and install and run Pathfinder Client software. In turn, the client can be controlled from the Pathfinder web portal. The user can use the portal to start recording training data from the workstation camera. Users can also schedule recording sessions to start at a specific date and time e.g. if the next shift is expected to start later, or the product variation in question will only be built on certain days and shifts of the week. We also allow users to upload pre-recorded videos directly from the portal using the “update” functionality. Pathfinder client enforces privacy settings on camera stream e.g. facial blurring or fixed region pixelation, before either locally storing or uploading the data into the cloud.

Label

Once cycle videos are captured, the user is expected to label one or more cycles of the process. This labeling is also performed using the Pathfinder web portal, and is in the form of time series labels, i.e. the user provides associations between step names and start-stop times for the steps. The labeling process usually takes about ten minutes of the user’s time. We have recently added an alternative approach to providing labels at the level of static scene regions and moveable tools and parts. Pathfinder then takes these handful of labels and creates a much larger training set using a number of AI tricks. The advantage of generating labels for a lot of variations in performing the task is that the trained models then become able to capture a correspondingly large set of ways of performing the activity.

Deploy

Once training labels are provided, the user asks the system to train a machine learning model corresponding to the training dataset. The machine learning (AI) model is a computational models of the appearance of various steps, that can detect the performance of these steps. This “training” request is made from the portal as well. Training of machine learning models is performed in our compute clusters in the AWS cloud environment. Once the appropriate model is trained by the system, the user receives a notification that the model is ready. They can then deploy the model also from the portal at the click of a button. The user can choose whether they want to enable privacy features e.g. facial blurring or fixed region pixelation, as well as specify whether they want to turn on “Trace” video recording upon deployment. The model takes two to three minutes to download, and then the user is able to observe it tracking the steps of a live process on the workstation in real-time.

Modify

Pathfinder allows conveniently modifying processes for which data has already been collected. It is possible to add, delete, rename, change ordering as well as merge step definitions with a few clicks. Pathfinder also enables steps to be marked as “optional” which means that operators aren’t alerted if those steps are missed. Pathfinder also provides the option to explicitly define the end-states of steps by specifying visual appearance of the unit upon successful step completion. This allows verifying that a step (or group of steps) achieved their desired goal.