Invented by Yuta Namiki, Fanuc Corp
The Fanuc Corp invention works as follows
The machine learning device creates machine learning training data by using an image that was captured on an inspection target with a symbol that indicates a fault. It also includes a creation unit which creates the data from the inputted image. In this case, the creation unit creates data that consists of the inputted image and a label which retains OK, which means that the image does not have a flaw, or creates data that consists of the image and a label which retains the value NG, which means that there
Background for Machine Learning Device, Inspection Device and Machine Learning Method
Field of Invention
The present invention is a device for machine learning and a method of machine learning. It relates, in particular, to a machine-learning device and a method of machine-learning that can generate training data efficiently for machine-learning. It also relates to an Inspection Device made using the machine learning device and method.
Related Art
In the past, it was known that a system could be used to acquire an image of an inspecting target using a visual camera, and then perform inspection by image processing. When performing an inspection of an image target, there is a way that uses machine learning techniques to distinguish between OK (good products) and NG (so called inferior products in which defects were found).
In order for this machine learning to be performed, we need training data which consist of an image of the target inspection and a label indicating whether the target inspection captured in the image is OK, or NG. When preparing multiple groups of training data sets, the image of the target inspection and the label with the value?OK? If you prepare several groups of sets of training data for the image of the inspection target and label with a value?OK? It is possible to perform machine-learning using these groups of training data sets. The machine learning device can then distinguish between OK or NG on new images of the target inspection based on contents generated by machine-learning.
Before performing machine learning, a person will often determine OK or NG, and then use the result of that determination as the training data. A person must also correct the machine’s misjudgments, as it may still make mistakes after machine learning.
In such a situation, it is important to send the correction or determination made by a human to an electronic device for machine learning. People can label an image of the target inspection obtained by the sensor with OK or NG. The group consisting of the image and label with a value of “OK” is a result of this. The group of the image of the inspection target and label having a value of?OK? It is easy to obtain. “Patent Documents 1 and 2, which are documents related to the technology, can be provided.
Patent Document 1 discloses a technology for performing learning to determine the position in which a surface inspection device can capture an inspection surface. This learning involves the association of the capturing sequence with the capturing location. Patent Document 1 is not machine learning.
Patent Document 2 discloses a technology that allows an inspector to visually inspect the coating on a vehicle’s body and attach marks to defects. In a subsequent step, the place where a marking has been attached is detected and a polishing process is carried out using this location as a defect site. “In this Patent Document 2, correctional processing is carried out on the places where marks are attached.
Patent Document 1: Japanese Unexamined Patent Application Publication No. 2009-14357 2009-14357
Patent Document 2: Japanese Unexamined Patent Application Publication No. H05-196444 H05-196444
However in the case where a person makes the judgement of OK or Ng, it is often deemed OK or Ng by viewing the actual workpiece (inspection target). The person who has viewed the actual inspection target is then required to view an image, and label it OK or NG. In this article, two examples are given. A person can identify a defect in an image that captures the target by viewing the image. (B) The target is divided into several sites and multiple images are taken. A person searches for the images that contain a defect and labels them with NG.
In the case (A), after finding a defect in the actual inspection target, a person searches for the image obtained by the visual sensor and looks at the location that corresponds to the defect. The person then assigns the NG symbol along with indicating the defect site. The work of finding a defect on an actual inspection target and searching for the site defect on the image is cumbersome.
In case (B), a person first finds a defect in the inspection target and then searches for an image that corresponds to the defect, among the multitude of images. The person assigns the NG tag to this image. The work of finding a defect on the inspection target and searching for an image that includes this defect becomes cumbersome. The workload also increases when there are many images.
The current situation is not yet known. “It would be great if this defect could be used by someone viewing the inspection target in its original form and then using the data as training for machine learning.
The present invention aims to provide a machine-learning device and a machine-learning method that can create training data more efficiently for machine learning.
In order for the inventors to achieve the object above, they have investigated thoroughly the following principles. An inspector first marks a particular symbol on the defect portion of an actual inspection target. A similar method is used to capture an image and obtain an image of the target where the symbol is marked. A specific symbol can be found in the image acquired, and then the area marked with the symbol becomes a defect. It is possible to label the image as NG when recording a defect.
According to this principle, it’s possible to obtain training data for machine learning. The training data includes at least an image (called a ‘training image’) for machine learning. The label will retain the value NG or Ok. The symbol on the inspection target can be removed to allow machine learning.
In addition, present inventors have analysed the following processing, such as for the mechanism of removal symbol. In removal method 1, the symbol used to mark the inspection target is a symbol created using a particular shape or color. If there is a particular shape or color symbol in an image, this can be removed by image processing and used as the training image. It is also possible to configure the machine learning so that it does not use a symbol or a color of an image.
As removal method 2, an inspection target is also marked with fluorescent paint. It is possible, for example, to use fluorescent paint that is transparent and colorless under visible light but fluoresces when exposed to UV light. After marking the symbol with fluorescent paint on the inspection target, the inspector can obtain an image to be used for learning/inspection by capturing the image under visible light illumination. A defect image can also be obtained by taking a picture under UV light illumination. The symbol will appear in the photo.
The present invention uses the following configuration. The present invention relates in a first aspect to a machine-learning device (for instance, the machine-learning device 40 described below) for creating machine learning training data, where the machine-learning device includes: an input unit that captures an image of an inspection target (for instance, the inspection targets 12 described below) which has a symbol indicating a fault marked on it; and a unit that creates the data based upon the inputted image. In this case, the unit retains a label with a