You can access the Japanese version from here.
QMモジュールを使った倉庫内のスマートな品質検査アプリ
Abstract
By combining SAP Build Apps, Node.js on SAP BTP Kyma, SAP S/4HANA, and Amazon Textract, you can replace the data input tasks for quality management with optical character recognition technology, thereby reducing the working hours of quality control personnel in the warehouse.
Use case
A Japanese overhead crane manufacturing company conducts quality inspections before shipping their overhead cranes. This inspection involves a total of 72 inspection items, with results indicated using abbreviations like A or B, represented as Accepted or Correction, for instance. The company has implemented SAP S/4HANA EWM for warehouse management and wanted to integrate the quality inspection process during outbound delivery into this system. However, they were filling out inspection reports on paper, which was time-consuming as they had to manually input the results for dozens of inspection items. To streamline this process, they developed an application utilizing optical character recognition technology to automatically analyze the inspection item results. This application then communicates with the SAP S/4HANA quality management module via API to transmit the inspection result and usage decision generated from the analysis results.
Architecture
- SAP Build Apps: It is a low-code/no-code service that comes with a rich set of components and functions by default, making it easy to develop applications with minimal effort. It includes UI-based deployment features, automatic type definitions based on objects received via API communication, and code completion features, making it a user-friendly service even for a person without a technical background.
- Kyma: It is an orchestration application based on a microservices architecture platform, providing functionalities such as API management between different services. In this project, middleware development using Node.js was carried out on the Kyma platform. For more detailed architecture information, please refer to the following link.
Architecting Out-of-the-Box Node.js Middleware Development for SAP S/4HANA
- SAP S/4HANA: The Extended Warehouse Management (EWM) and Quality Management (QM) modules are used. With embedded EWM and QM modules, product quality is ensured while delivering to customers and other warehouses. The setup for quality management and the execution process can be replicated on SAP S/4HANA. You can find details on how to do this in the following two links. Additionally, these articles outline procedures for quality management during inbound processes, but you can perform quality management during outbound processes by changing the inspection type.
Customizing: Quality Inspection in EWM under No Inspection Rule
Execution: Quality Inspection in EWM under No Inspection Rule
- Amazon Textract: It allows for optical character recognition (OCR) and includes functionality for recognizing text in table formats as well.
Solution architecture
The fundamental flow of Quality Management in embedded EWM
Before delving into the application's description, let's take a look at the fundamental flow of Quality Management in embedded EWM.
The fundamental flow of Quality Management in embedded EWM
Inspection plan
Inspection plan describes the whole procedures and specifications for inspecting a product. This plan includes inspection procedures, inspection characteristics, and other relevant data. Multiple inspection operations can be set up within the inspection plan, with several inspection characteristics associated with each operation. In the example of the inspection paper provided, the items in the second column correspond to inspection operations, such as "ロードチェーン(Load Chain)" and "走行レール(Rail)." On the other hand, the inspection items in the table represent inspection characteristics, including "キズ、変形は無いか(Check for scratches and deformations)" or "取付ボルトの緩み、脱落の有無(Presence of loose or missing mounting bolts)".
Inspection plan structure
Inspection report paper
Inspection lot
Inspection lot is used to record, process, and manage information related to quality inspections. This describes inspection process data, inspection results, evaluation costs, and usage decision, among other details. By assigning the previously mentioned inspection plan to a specific inspection lot, it becomes possible to conduct quality inspections of the product.
Inspection result
Inspection result refers to the actual outcomes of a quality inspection. This includes test values and assessments obtained during the inspection. It indicates whether the product meets quality standards and whether any defects were identified.
Usage decision
Usage decision is made based on the inspection results, and subsequent follow-up action is initiated. As the follow-up action, if the product is "Accepted", the outbound delivery process proceeds, while if it is "Rejected", additional processes, such as further inspection or analysis, are carried out under shipment is stopped. There is also an inspection score for usage decision, and this can be adjusted for scoring method and pass criteria within SAP S/4HANA.
In the case of the overhead crane, an "A" rating (meaning Accepted) for all inspection items is required, which corresponds to an inspection score of 100 points. If there is an issue with any item, shipment is halted and a repair request is initiated for that specific item.
Inspection result and usage decision
Since this application is intended for use by quality inspectors in a warehouse, it doesn't involve the generation of inspection plans or inspection lots. Instead, it processes inspection results and usage decisions based on the information from the inspection lots.
The app main processes and implementations
Inspection list page - In progress
On this page, you will find a list of items subject to quality inspection. You can use the provided query parameters to narrow down your search based on date ranges or inspection completion status.
| Technical service name | API_INSPECTIONLOT_SRV |
| HTTP Method | GET |
| Endpoint and query parameters | /A_InspectionLot?$filter=InspectionLotStartDate ge datetime'2023-09-01T00:00:00' and InspectionLotStartDate le datetime'2023-09-30T23:59:59' and to_InspectionLotWithStatus/InspLotStatusInspCompleted eq null and Plant eq '1710' |
Inspection list page - In progress
You can make use of the "Date field" component which is installable from the SAP Build Apps Marketplace to determine the date range.
Date field component in marketplace
Inspection list page - calendar
Scan papers page
A photo of the inspection paper related to the selected inspection lot on the quality inspection list page is taken, and it is analyzed using Amazon Textract.
Scan papers page
Scan result page - fetch scan result
You need to pass 'ocr' object to SAP Build Apps, including the analysis results, Japanese and English sentences, and inspection characteristics. First, you can read the analysis results from Amazon Textract response.
{
Blocks: [
{
BlockType: "WORD",
Confidence: 99.12993621826172,
Text: "B",
TextType: "HANDWRITING",
Geometry: {
//...
},
Id: "0cafe990-c84d-4113-a413-6905c0ba0ca2",
},
{
BlockType: "CELL",
Confidence: 87.060546875,
RowIndex: 3,
ColumnIndex: 5,
RowSpan: 1,
ColumnSpan: 1,
Geometry: {
// ...
},
Id: "c310d6ad-9390-4b3f-a38f-7759daa009a8",
Relationships: [
{
Type: "CHILD",
Ids: ["0cafe990-c84d-4113-a413-6905c0ba0ca2"],
},
],
},
// ...
],
};
Using the BlockType, RowIndex, ColumnIndex, and the Ids of Relationships, an array of objects that include the inspection index and inspection results is created.
[
{
index: "1",
inspectionResult: "A",
},
{
index: "2",
inspectionResult: "A",
},
// ...
];
In the next step, you will write Japanese sentences for the inspection items and their corresponding English translations in Node.js. This is necessary due to the lack of analysis functionality for Japanese data presented in tables within Amazon Textract.
Finally, regarding the inspection characteristics, you obtain a list of inspection characteristics for a specific inspection lot. The characteristics in the response contains the inspection operation and characteristic IDs, and whether they are qualitative or quantitative.
| Technical service name | API_INSPECTIONLOT_SRV |
| HTTP Method | GET |
| Endpoint and query parameter | /A_InspectionCharacteristic?$filter=InspectionLot eq '17000000953' |
Here is the extracted list of inspection characteristics:
[
{
InspectionLot: "17000000953",
InspPlanOperationInternalID: "1",
InspSpecIsQuantitative: false,
InspectionCharacteristic: "10",
SelectedCodeSet: "QUAL-CRN",
},
{
InspectionLot: "17000000953",
InspPlanOperationInternalID: "1",
InspSpecIsQuantitative: false,
InspectionCharacteristic: "20",
SelectedCodeSet: "QUAL-CRN",
},
// ...
];
Since all 72 items are qualitative characteristics, it is necessary to fetch the specific qualitative criteria from SAP S/4HANA. You will use the SelectedCodeSet for this purpose later.
Next, you have to connect each of these inspection items with the results obtained from Amazon Textract. Lastly, the ocr data created in this way will be sent to SAP Build Apps.
const ocr = [
{
jaID: "押釦スイッチ",
enID: "Button",
isShow: true,
characteristicList: [
{
index: "1",
inspectionOperationJA: "押釦スイッチ",
inspectionOperationEN: "Button",
inspectionCharacteristicJA: "ケース、 方向表示の損傷",
inspectionCharacteristicEN:
"Damage to the case and direction indicators",
inspectionResult: "A",
inspectionResultText: "Accepted",
lotID: "17000000953",
operationID: "1",
characteristicID: "10",
catalogName: "QUAL-CRN",
chacteristicAttributeID: "",
resultCategory: "",
validValuesNumber: 1,
},
{
index: "2",
inspectionOperationJA: "押釦スイッチ",
inspectionOperationEN: "Button",
inspectionCharacteristicJA: "スイッチの作動状態、ゴムカバーの亀裂",
inspectionCharacteristicEN:
"Operational status of switches and cracks in rubber covers",
inspectionResult: "A",
inspectionResultText: "Accepted",
lotID: "17000000953",
operationID: "1",
characteristicID: "20",
catalogName: "QUAL-CRN",
chacteristicAttributeID: "",
resultCategory: "",
validValuesNumber: 1,
},
// ...
],
},
{
jaID: "ロードチェーン",
enID: "Load chain",
isShow: true,
characteristicList: [
{
index: "5",
inspectionOperationJA: "ロードチェーン",
inspectionOperationEN: "Load chain",
inspectionCharacteristicJA: "キズ、変形は無いか",
inspectionCharacteristicEN: "Check for scratches and deformations",
inspectionResult: "D",
inspectionResultText: "Defective",
lotID: "17000000953",
operationID: "2",
characteristicID: "10",
catalogName: "QUAL-CRN",
chacteristicAttributeID: "",
resultCategory: "",
validValuesNumber: 1,
},
// ...
],
},
// ...
];
In SAP Build Apps, similar to mobile app development frameworks like React Native or Flutter, you can create components that hold information for each element in an array using iterative processes. With this, you can create components for operationList and for each characteristicList.
Repeated with ocr (operation list)
Repeated with characteristic list
And then, you can assign values such as the index of each element appropriately to the content of the component.
Assign each index of characteristic list to Content
The result page created through this process is shown in the image below.
Scan result page
Scan result page - post inspection result and usage decision to SAP S/4HANA
To send inspection result and usage decision to SAP S/4HANA, you will first obtain catalogs (lists of options) for each characteristic using the SelectedCodeSet. Considering to performance, you use cache for these requests.
| Technical service name | API_CHARCATTRIBUTECATALOG_SRV |
| HTTP Method | GET |
| Endpoint and query parameter | /A_CharcAttribSeldSetCode?$filter=CharacteristicAttributeCodeGrp eq 'QUAL-CRN' |
| Technical service name | API_CHARCATTRIBUTECATALOG_SRV |
| HTTP Method | GET |
| Endpoint and query parameter | /A_CharcAttribSeldSetCodeT?$filter=CharacteristicAttributeCodeGrp eq 'QUAL-CRN' |
You will combine the API responses obtained from these two GET requests.
[
{
CharacteristicAttributeCodeGrp: "QUAL-CRN",
CharacteristicAttributeCode: "1",
CharcAttributeValuation: "A",
CharacteristicAttributeCodeTxt: "Accepted",
},
{
CharacteristicAttributeCodeGrp: "QUAL-CRN",
CharacteristicAttributeCode: "2",
CharcAttributeValuation: "R",
CharacteristicAttributeCodeTxt: "Repair",
},
{
CharacteristicAttributeCodeGrp: "QUAL-CRN",
CharacteristicAttributeCode: "3",
CharcAttributeValuation: "R",
CharacteristicAttributeCodeTxt: "Observation",
},
{
CharacteristicAttributeCodeGrp: "QUAL-CRN",
CharacteristicAttributeCode: "4",
CharcAttributeValuation: "R",
CharacteristicAttributeCodeTxt: "Defective",
},
];
You will use the inspectionResultText to search the catalog for the corresponding item related to the inspectionResult of the OCR object (which can be A, B, C, or D).
const findCharacteristicAttribute = (inspectionResultText) => {
const foundCharacteristic = catalog.find(
(characteristic) =>
characteristic.CharacteristicAttributeCodeTxt === inspectionResultText
);
if (foundCharacteristic) {
return {
CharacteristicAttributeCatalog:
foundCharacteristic.CharacteristicAttributeCode,
CharacteristicAttributeCode:
foundCharacteristic.CharacteristicAttributeCode,
InspectionValuationResult: foundCharacteristic.CharcAttributeValuation,
};
} else {
// Error handling
}
};You will send the objects obtained through findCharacteristicAttribute, along with the ID information, to SAP S/4HANA for all 72 inspection characteristics.
| Technical service name | API_INSPECTIONLOT_SRV |
| HTTP Method | POST |
| Endpoint | /A_InspectionResult |
// Headers
{
"Content-Type": "application/json",
"x-csrf-token": "ZAaslFH-gRaItcYMUQ_SWw==\r",
Cookie: "sap-usercontext=...",
}
// Body
{
InspectionLot: "17000000953",
InspPlanOperationInternalID: "1",
InspectionCharacteristic: "10",
CharacteristicAttributeCatalog: "1",
CharacteristicAttributeCodeGrp: "QUAL-CRN",
CharacteristicAttributeCode: "1",
InspResultValidValuesNumber: "1",
InspectionResultText: "",
InspectionResultHasLongText: "",
InspectionValuationResult: "A",
InspectionResultStatus: "5",
};To ensure that the expected update state for usage decision matches between the Node.js client-side and the server-side, you will use the value obtained as "ChangedDateTime" from the GET request of the inspection lot. This value will be used as an ETag.
| Technical service name | API_INSPECTIONLOT_SRV |
| HTTP Method | GET |
| Endpoint | /A_InspectionLot('17000000953') |
Since the follow-up action for the usage decision code "A1" in the usage decision group "UD01" is configured as "Move inspected products to the staging area" in SAP S/4HANA, you do not need to pass information regarding the follow-up action in the request.
| Technical service name | API_INSPECTIONLOT_SRV |
| HTTP Method | POST |
| Endpoint | /A_InspLotUsageDecision |
// Headers
{
"Content-Type": "application/json",
"x-csrf-token": "ZAaslFH-gRaItcYMUQ_SWw==\r\n",
Cookie: "sap-usercontext=...",
}
// Body
{
d: {
InspectionLot: "17000000953",
InspLotUsageDecisionLevel: "L",
InspectionLotQualityScore: "100",
InspLotUsageDecisionCatalog: "3",
SelectedCodeSetPlant: "1710",
InspLotUsgeDcsnSelectedSet: "UD01",
InspLotUsageDecisionCodeGroup: "UD01",
InspectionLotUsageDecisionCode: "A1",
ChangedDateTime: "/Date(1563885880000+0000)/",
},
};By following these HTTP interactions, you will complete the quality inspection process.
Scan result page - complete inspection
Inspection list page - Completed
On the inspection list page, it has been confirmed that the status of inspection lot 17000000953 has changed from "In progress" to "Completed."
Inspection list page - Completed
Conclusion
In this project, the data entry process for quality management has been streamlined by combining SAP Build Apps, Node.js on SAP BTP Kyma, SAP S/4HANA, and Amazon Textract. Depending on the requirements, you may also consider adding the following features:
- Inspection Result Confirmation Functionality: For inspection items where input errors cannot be tolerated, you will add a feature to display the analyzed inspection results in a modal window. This allows users to confirm that there are no mistakes, ensuring the accuracy of quality data.
- Improved Analysis Accuracy through Photos: In cases where the accuracy of analysis results is not high, you will incorporate a feature to capture two photos for a single inspection sheet to compare if both of them have the same results. It's important to note that analysis accuracy depends significantly on factors such as training data, the characters being analyzed, character count, and language.
- Alternative to Paper Inspection Sheets: In quality management processes where wearing gloves is not necessary, you will create inspection reports directly within the mobile app instead of using paper inspection sheets. This eliminates processes involving paper and streamlines the workflow.
Reference
- SAP Managed Tags:
2 Comments
