Pallet Recognition for Forklift: A Literature Review

Abstract

Forklift trucks are essential tools for manual pallet handling. Storage heights at high rack storage areas often reach up to 12 meters in which bulky Cargo can hide the view of forklift operator. Pallet recognition is a fundamental issue for Industries & warehouses, Particularly, a pallet recognition approach is presented to recognize pallets in the warehouses, based on calculating the degree of similarity at each location of the palette. Once a pallet has been recognized, it is matched that data with stored data. Transporting and handling pallets in a storage area is the essential function of a forklift truck. The view of the forklift truck driver is limited during his work by the lift pole, the fork, and the roof of the forklift truck. If a pallet is loaded the view is additionally hindered by its load. When the storage rack is not at a ground level or near to it the pallet handling becomes more difficult because of the distance between the forklift truck driver and the storage rack. To support the forklift truck driver during his work, there is need to advance the forklifts with different technologies and systems which provide diversity in Working and safety, security for forklift operator and other workers. This paper shows various results from research on different pallet recognition and forklift operation systems.

Introduction

I. INTRODUCTION

Nowadays, Technology is identified as the most important area of manufacturing process & decision making, followed by quality assurance and control. Due to technology industrial works have become easier. One of the Application which is used in industry or in storehouse which is Forklift. A forklift is a combination of hydraulics, a pulley system and other things to lift heavy materials across different distances. Forklifts are commonly used in warehouses, construction sites and other places that need to transport lots of heavy materials. Many times, material is situated in boxes and its fixed-on pallets. Forklifts are best used in unloading pallets of blocks/bricks, steel joists, and construction equipment and materials, especially in hauling them from the delivery truck and carrying them to the site. Warehouses – Forklift trucks are commonly used in warehouse operations. In this paper, we are focused on forklift and Pallet recognition systems. In storage warehouse forklifts are used for everyday common warehouse functions, which includes keeping stock orders, lifting the required equipment and sometimes workers, moving the materials as per the need, stacking materials, and much more. Up to 5,000 lbs. weight can be handled by forklifts. The models like load on average, and some heavy-duty models can handle up to 20,000 lbs. payloads. Many times, material is situated in boxes and its fix on pallets. The best use of Forklifts is for blocks/bricks pallet uploading, also steel joists, all kinds of construction equipment and materials, transporting them from the delivery truck and on to the site. If a forklift driver who retired 25 years ago came back to work in a warehouse in today’s date, he or she would be astonished by how much forklifts have changed. Similarly in the automobile industry, vehicle machine design, fuel efficiency, and embarked technology have all evolved rapidly in just the past few years. One of the areas that has seen several biggest enhancements is safety and security. Manufacturers, assemblers of forklifts and accessories have dedicated a great amount of effort to develop cutting-edge products that help machine minders use forklifts more safely. This study, as per the best of our knowledge, is the first study that aims to review the various changes in forklift and pallet recognition systems in industry in different fields.

The flow of this rest of the paper is formulated as follows. The literature selection methodology is presented in Section II. Changes in forklift and pallet recognition systems are reviewed in section III, further, conclusion is in Section IV and acknowledgement is described in Section V.

II. LITERATURE SELECTION METHODOLOGY

A methodology for selection of currently available research work was decided and performed in order to study the forklift and pallet recognition systems literature for this literature review. This following section provides the description of the literature selection procedure.

A. Keywords Filtering

The methodology of selection of related articles started with a search from Google Scholar database with at the one of the following keywords as the article title: (1) forklift, (2) forklift for pallet recognition, (3) warehouse management using forklift (4) forklift Raspberry Pi. This search resulted in 135 research papers, journals and articles.

B. Publishers Filtering

This methodology totally focused on researched articles published in the following publishers: (1) IEEE, (2) MDPI, (3) Elsevier, (4) Springer, (5) Taylor & Francis Group, (6) IOP publishing, (7) International Journal of Artificial Intelligence,

(8) Indian Journal of Science and Technology, (9) The International Journal of Engineering and Science, (10) Journal of Robotics and Mechatronics, (11) Journal of Robotics and Control, (12) Logistics Journal. This search resulted in a reduction in the number of papers from 135 to 62.

C. Abstracts Filtering

Considering the 62 articles from the previous filtering, an abstract reading was performed in order to further filter only the most relevant articles that specifically study the various changes in forklift and pallet recognition systems in industry in different fields. Thus, 32 articles were selected from the forklift literature.

III. CHANGES IN FORKLIFT AND PALLET RECOGNITION SYSTEMS

In this paper, thirty-two articles based on various changes in forklift and pallet recognition systems are reviewed.

In [1], the authors Investigated how Augmented Reality (AR) could help forklift Masters in performing their pallet racking and picking up tasks by Overlaying virtual guidelines over the real time camera feed. To investigate this and test this, they designed and created a prototype system based on a toy forklift and conducted a study with it. they used camera (Microsoft LifeCam studio) attached to a side of the fork and Opti track motion tracking system with tracking markers attached on the video from the camera and the tracking information is fed into the model software developed with the Unity 3D game engine the fork frame. For AR visualization they used the Direct Linear Transform camera calibration method and the results showed that AR cues helped the participants to perform tasks with a higher success rate and provided better usability.

In [2], Authors created a system consisting of an RFID- equipped forklift that is able to recognize the loaded pallets and self- localization was addressed for a tissue paper factory warehouse. They used sensor fusion algorithm for forklift self- localization. After a Various numbers of investigation system was evaluated. After experiments analysis indicated a certain drop in the performance of on the board kinematic sensors and the UWB system with an increase in the forklift speed. The achieved localization performance together with a limited computational burden makes the proposed system a potential candidate to be tested for collision avoidance systems.

In [3],authors have Developed a vision-based intelligent forklift Automatic Guided Vehicle also known as (AGV) to perform various tasks in factories and warehouses for transportation, quality and efficiency improvement as well as reducing the manpower also human interference in service and manufacturing industries .this is Based on a vision-based adaptive localization algorithm with the global pose correction and with help of visual servo motion controller, the automatic driving function of the AGV can be used. Keeping most of structural body the same they upgraded the forklift AGV with Control box and gyroscope, Lens hood and camera and Passive wheels and encoders. A landmark-based SLAM method is used to build up surrounding environmental map and to correct any localization error by matching the map. The various Experiments were conducted in a factory environment to validate the performance of the proposed vision-based intelligent forklift AGV.

In [4], the authors have proposed a study to assess the fungal contamination and levels of mycotoxin in the filters from the air conditioning system of forklift cabinets, which acts as an indicator to assess level of exposure to the occupied drivers working in a waste sorting facility. The waste management industry being an important employer, keeping a note that exposure of waste to the handling workers of micro-organisms is considered a serious health problem. Apart from fungal contamination, it is important to scrutinize the co-existence of mycotoxins in this setting Forklifts created with air conditioner and closed cabinet generally used in waste industry to transport the waste and other products to be discarded within the facilities, which possibly increases the risk of exposure and cause under certain health conditions.

In [5], For navigating automatic forklift, authors have proposed a method for locating of the forklift. To improve image processing and robust object identification efficiency they have used color image which is transformed from RGB space into HSV and YUV space.

Further, they found the mid- point of the pallets, and calculated the position of pallets which are relative to forklift by camera space model which builds the relative relationship between real world space and image space. Also, in order to improve system speed, they have used the Kalman filter process to decrease the processing data. results from the experiment indicated that this method has satisfactory performance inaccuracy and efficiency improvisation of localization. The complete computer visual system can identify and locate pallet in industrial environment, which is simulated to achieve a complete linear imaging model by monocular stereo vision system calibration, which then improves efficiency of system with the Kalman filter for its state estimation. The method needs additional signal, such as floor- embedded wires, ultrasonic sensors, or laser beacons which can accurately locate the forklift’s position.

In [6], the authors have provided video analyses and in- clinometry objective measures of physical workloads. The study aim was to measure and observe back, arm and head postures along with movements of forklift truck operators also called (FLTOs) during a normal working day, to analyze the differences between types of forklift trucks and to report workload and health. They took Twenty-five male FLTOs in a high-level warehouse were randomly. And collected data which comprised of video analyses of postures and movements, technical measurements and a questionnaire to measuring health, pain and workload for the forklift operations. This revealed that on average basis, the FLTOs rotated their head in total, 232 times/h for more than 45°. The Video analysis revealed that FLTOs periodically drive the fork-lift truck sideways with the head rotated in the direction of travel, in which the head gets highly rotated and extended. observations and in-clinometry during the working day have the potential tobe a valuable part of risk assessment promoting occupational safety and health.

In [7], the authors have concentrated on fault diagnosis strategies. Forklifts, being one of the most widely used vehicles for transportation in short distance of cargo, also its safety is very important. Many electronic components are being used in modern forklift development in large scale, there is also need of the fault diagnosis, and fault re construction and fault tolerant control (FTC) technology are certain important aspects to ensure safety. formulating sensor fault detection and reconstruction of forklift, a sliding mode observer (SMO) with adaptive regulation law is Used to detect disturbance. After Experiments on electric forklift, it can be seen that the sliding mode observer (SMO) can effectively reconstruct the sensor fault value, and also realize estimation of robust faults as compared to fault tolerant control (FTC) strategy.

In [8], authors have described a self-determining pallet handling system for forklift, which automatically can unload and convey the pallets for harvesting outdoor veg- eatables and vegetation. In Japan, there is aging depopulation of work force are very serious issues. Hence, there is need for development of autonomous agricultural robots which can save labor and manpower. When forks are inserted into a narrow pallet hole, approximate pallet posture calculated estimation and perfect control of a forks and the forklifts are needed. The proposed system can detect the pallet by use of deep learning- based detection of object from an image acquired. Based on the results of defect object and measurement taken with horizontal 3D light detection and ranging (LiDAR), the system it accurately estimates a distance and also vertical and horizontal divergence between the pallet and the forklift in the background field. The forklift is being controlled by sliding mode control (SMC) which is tough to disturb. Further, the vertical LiDAR pallet is scanned for adjusting the height of the fork precisely. Various experimental results indicate the applicability of the proposed system in real time agricultural environment.

In [9], Research provided by authors gives valuable information regarding work associated endangerments with the use of smart glasses on forklift trucks with reference to workload. A monitor used as a digital information system has the distinctive advantage over the smart glasses that it can be used by spectacle users. Additionally, this study has exhibited that the acceptance of this device is higher than head-mounted display systems in probability. Besides, it is also not clear whether the workload associated with the study is physical strain. The applicability of smart glasses in this work case may cause problems which could become chronic in case being exposed for a longer period of time. Also, it is quite possibility that using smart glasses can increase the risks of accidents which are caused by distraction. Hence It is concluded that Before using Smart glasses on forklift trucks, companies should consider and assess thoroughly whether an additional workload is acceptable.

In [10], Authors provide research. The research reported in this paper Intent's to reduce warehouse accidents which causes human injuries and loss and damage to pallet and goods.by using the trending camera technology in combination with computer vision Technique. They have used A time-of-flight camera, which provides 2D and 3D data. With 2D data humans are detected in driving path by the use of support vector machine. Differentiating between humans and different factors like storage facility, grants a warning which is two level system to enable. The aim of this Proposed system is to approve warnings to be more sensitively configured when humans are in sight of path. calculation of possible collisions and segmentation of the 2D image is done with 3D image. The results presented in this paper emphasize how the with a support vector machine, humans can be detected can be and optimized.

In [11], authors have designed a human like dual forklift mechanism for the purpose of applications with container handling in limited space for example decks, warehouses, temporary small docks and ports. especially, a RGBD vision system with a turn lock and hydraulic lifting structure and a four-wheel steering structure, are designed. These parts enable the dual forklifts with the ability of detection and the location, navigation and omnidirectional movement, lifting and locking the containers. The proposed dual-forklift handling system is smaller in size, flexible to move, easy to operate and impressively improves the level of forklifts automatically.

In [12] Authors have developed a machine learning vision- based method which identifies to realize the timely detachment of shovel teeth(fork) of forklift. The method known as double threshold binarization method which is based on HSV color space is proposed. In binary image the region with impurities is removed by the use of fusion feature of the shovel tooth, which further extracts the bucket region. The determination of whether the tooth is missing by the principle of similarity between the area of the tooth area and the area template is done in scrap tooth missing detection phase. The results from simulation algorithm using shovel image in MATLAB indicate that the algorithm maintains high accuracy in regards of recognition and provides an impressive solution for forklift shovel detection.

In [13], the authors Focus at the problems of insufficient timeliness and lacking monitoring accuracy of forklift working condition observation, a multiple sensor monitoring online method for forklift is proposed In this paper, the structure of forklift monitoring platform is developed and main control algorithm of forklift working condition data monitoring is prepared. The forklift monitoring is acquired by multi-sensor. The results from experiments show that the method can acquire live monitoring of the working state information of the forklift and can assist the management personnel to extensively manage the forklift.

In [14], Authors designed an iBeacon enabled indoor positioning solution for management system of warehouses in a real-life industrial case. Indoor positioning techniques which estimate object locations like eye recognition, trilateration and fingerprinting methods are implemented for application in numerous warehouses. In this work, they combined the iBeacon implementation with the current warehouse working process. They set up beacon devices as emitters and made indoor position presumption with Receive Signal Strength Indication (RSSI) of beacons obtained at base stations. This minimizes the alteration of the time working procedure. The aim is to provide cost-effective indoor positioning for warehouse management systems. Through this solution, picking time of order of finished product in warehouses and raw material warehouse has been shortened from 17.2 minutes to 3.1 minutes with correction in estimation rate of 91.2% and 6.2 minutes to 1.9 minutes with 96.8% approximately. Due to the use of business intelligence (BI) the entire process has been reengineered.

In [15] paper Authors describe a real industrial problem that is location and tracking of forklift trucks which requires perfect positioning, and they presented a study on the potentiality of solving this challenge using UWB ultra-wide band technology. They have developed a very realistic simulation tool, which is based on the Gazebo physics simulator and has multiple sensors which track forklifts in an indoor environment. The set of measurements are captured in a real scenario and measurements are from a set of real sensors. This simulator is used together with a physical model of the forklift and location algorithm to obtain estimations of position in different situations with different interferences and obstacles. There is an additional inertial sensor and optical sensor were modeled in to testing its effect on supporting the location based on Ultra-Wide Band. To build this simulator they used machine learning techniques to generate various models which can provide simulated ultra- wide band values which are based on the distance between the anchor and each tag and the obstacles around and between them. The performance of this simulator has Been tested by creating a 3D model of the place where the measurements were originally taken and comparing the simulated results with those obtained from the measurement from the software.

In [16] Authors describe for warehouse management how to exploit “smart spaghetti” which is a spaghetti chart that is automatically generated by smart tracking devices. That conceives impressive improvements in the work and layout organization of a warehouse, also reduces the risk of collision that can happen between forklifts. and provides solutions for operator’s safety. The methodology that is involved automatically here is mapping of the spaghetti charts that entails searching for critical areas where the risk of collision is high and identifying the interventions that to be carried out to avoid misses. “Smart spaghetti” Is a valuable decision support tool which identity's improvements in the system with the changes in the layout or changes in performance in the activities. This application of the proposed technique can be essential in warehouses.

In [17], Authors have proposed control architecture of an autonomous forklift. The combination between the motion control that is developed in theories and behavior-based control architecture is developed to Create an accident preventive motion, high precision, motion for autonomous vehicles, fast movement, and failure recovery action. A (FSM) A finite state machine which regulates the planning and control algorithms is presented. This control structure is a combination of feedback controls, multiple sensors and a decision making in a module for automation of a forklift.

The productiveness of this architecture Is verified by results from experiments of autonomous forklift transporting a pallet from an initial location to a desired location.

In [18], paper authors have proposed and created a highly developed design of Autonomous Simultaneous Localization and Mapping which is also known as (SLAM) based Forklift Robot. The idea behind this work is to create a design for an object lifting forklift robot which works in a known as well as an unknown environment like storage warehouses for lifting and shifting boxes, pallets from one place to another. The forklift robot, being a SLAM based, has the capability to allow the controller of the robot to make decisions regarding movements of the robot on its own. with none of human interventions and work with the objects for detection, decision and lifting. For object detection, a machine learning approach called Optical Character Recognition (OCR) is used in which K- nearest principle to detect the user define object (box) is used. This SLAM based forklift robot can be used in hazardous, dangerous situations and in those difficult places where it is not possible for the human to work.

In [19], Authors have proposed an approach towards an autonomous forklift which is Toyota Industries Corporation forklift that can load and unload the pallets from a truck on an inclined ground. The RGB-D camera, which is embedded on the front of forklift, detects the pallet by process of semantic segmentation, depth and calculates the orientation and position of the pallet while the forklift inserts the forks into the slot of the pallet. A load detector is also embedded to stop the forklift if the pallet hits the sensor. effectiveness of the approach is verified with an experiment in which forklift is used outdoor environment like fields with the real agricultural task.

In [20], Authors have described the development of robotic forklift, which is expected to operate along with human handler, for handling pallet materials within busy, existing semi-outdoor structured storage facilities and warehouses. This robot operates in minimally prepared, structured environments, in which the forklift handles the transportable pallets cargo using local sensing and transporting it while keeping interaction with other moving vehicles. The robot operates in close Presence to people, including its designated human supervisor, also other workers who may block or cross its path, and also forklift operators who climb inside the robot and operate it manually. This is made possible due to a interaction mechanisms that provide safe, effective operation around people. They have used radio frequency identification (RFID) antennas on forklift trucks to detect and identify goods and stock them in shelves automatically. also have developed an interface based on stylus gestures and commands which can be made on a handheld tablet computer. The ultimate goal is to reduce the supervisor’s burden. This helps to increase the efficiency of warehousing processes.

In [21], Authors have proposed an autonomous forklift system for stacking multiple pallets using multiple RGB-D cameras. The RGB-D cameras are mounted on the forklift to localize the pallets edges and feet till the end of the stacking operation under a constrained sensor position. The system accurately extracts the pallet feet and edges using a region growing-based algorithm and it calculates the position and orientation of the pallets. The deviation of the position and orientation between the pallet feet and edges is minimized through a path following control. The forklift is then controlled using the path following control to converge the errors between two pallets. They have demonstrated the proposed system by conducting an experiment under an assumed real setting. The experimental results show that the proposed system has successfully accomplished the multiple pallets stacking.

In [22], Authors have provided a new approach for detection and localization of euro pallets. euro pallets are the standard pallets which are used for transportation in Europe. this proposed system can detect pallets, which are Aligned up to 90° in relation to the sensor plane. The pipeline detection is based on geometrical attributes of the wooden blocks of the pallets. For recording depth images, which are transformed into point clouds Kinect v2 camera is used Then point clouds are processed with the help of the open-source Point Cloud Library. Experiments showed that we can detect pallets under both static and dynamic conditions. Hence, a Kinect v2 camera is mounted on the front of a forklift. They showed that it is possible to detect and localize pallets with up to 15 frames Per second, while driving the forklift system performs the detection during driving a forklift, point cloud must be processed within a set time limit. The detection and localization results in the pallets’ position and orientation in relation to the camera coordinate system. The received data can be forwarded to top-level systems, like supply chain system systems. The results show that the localization of pallets is possible in the scenario considered.

In [23], Authors have proposed the system which tackles the problem of integrating Visual Serving Control (VSC) into the functionalities of an Articulated-Frame-Steering (AFS) hydraulic forklift. The controller is efficient in sorting down top-level messages into many pieces’ commands for the various software elements of the vehicle. controllers are self-dependent and not necessarily flexible. The requisitioned. Flexibility is acquired by controlling interactions of software elements by a state machine. It also provides consistent, coordination of the modules for a successful pallet picking process. Robot Operating System (ROS) is used for high-level control systems for mapping, path planning, obstacle avoidance, and marker detection. This proposed architecture is tested on a real machine. Videos showing the test runs are also available on YouTube.

In [24], Authors have proposed a smart forklift mechanism for Automatic Guided Vehicle pallet forklifts which utilizes a stepper motor and ultrasonic distance sensor. This smart forklift mechanism is equipped with raspberry pi model B as the main microcontroller and is combined with an ultrasonic distance sensor. The result of the ultrasonic distance sensor is that the error is zero percent so the precision of the height can be fully controlled. Step Revolution (SPR) method makes the stepper motor move smoothly and the number of rotations can be controlled as per Requirements. main components are motor stepper, drive motor stepper, proximity sensor based on ultrasonic and raspberry pi. the support component is DC motor driver as wheel mover, coupler connector, and lead screw as a rotary shaft. AGV will scan patterns with a camera and then it will move to another pattern. After it reaches the next pattern the forklift mechanism will do its job.

In [25], Authors have proposed a system which will minimize accidents in the beverage industry. Generally, alert for moving forklift is just the horn, so the operators must be cautious while passing in the aisles. for minimizing human errors which cause forklift accidents, and to speed up the moving, picking and delivery activities with forklifts, a flexible and accurate alert system traffic LED is used. The traffic LEDs of the forklift are designed to be flexible which can move along with the forklift. So, the LED alerts forklift masters in quick manner since they work automatically whenever there are other forklifts moving around. wireless technology is utilized in This traffic LEDs in the form of WI-FI signal. when there is presence of a transmitter towards the receiver there will be Changes in WI-FI signal strength. the signal is transformed into the radius of the certain object. When the forklift is approaching at a certain distance the radius will activate the “proceed” LED on one of the forklifts and the “stop” LED at another forklift. Hence, the forklift traffic runs smoothly without any danger.

In [26], Authors have proposed presented and discussed a possible application of object detection with CNNs to the problem of detecting, localizing, and tracking pallets using 2D laser rangefinder data only. which is achieved by transforming 2D rangefinder data into bitmap like images where CNNs can look for possible candidate pallets. This paper presents a novel architecture allowing a robot to detect, localize, and track (possibly multiple) pallets using machine learning techniques based on an on-board 2D laser rangefinder only. The architecture is consisting of two stages in the first stage is a pallet detector which employs a Faster Region-based Convolutional Neural Network known as (Faster R-CNN); in the second stage there is a Kalman filter which localizes, and tracks detected pallets. when Pallet candidates which are detected by two CNNs which are in cascade they passed down to a Kalman filter based tracker, which permits having an evaluation of pallet placement at any time, even when they are not clearly visible, also serving the system for filtering false positives. The proposed architecture presents detection and classification results. We conclude that our approach is a viable solution to correctly detect, localize and track pallets reliably, while attaining reasonable performance for real-world applications.

In [27], Authors have drawn comparison between a camera- based approach and a sensor-based approach of detecting the occupancy status of the load handling device of industrial trucks, related work using sensors and cameras in the logistics sector is examined, the sensor-based approach uses an ultrasonic distance sensor together with a Raspberry Pi single board computer. After data is collected it is transmitted through a Raspberry Pi single board computer, a (WLAN) stick and the Wi-Fi connection in the warehouse to the central computer where it is logged in the files. The camera-based approach uses machine learning techniques and image processing algorithms. Both approaches solution has been tested in a real-world environment. central warehouse of a German distributor of electronic goods has been selected as test site. After assessment it has resulted that approach has revealed that the sensor-based approach is superior the ultrasonic distance sensor connected to Raspberry Pi is the better alternative when only the detection rate at the time is being considered. The endeavor of setting up and configuring the sensor is less if compared to camera-based solution. Also, in camera-based approach, the fork detection was not reliable enough for industrial use.

In [28], Authors have carried out experiments with real-world datasets for pallet position prediction. To fork safely pallets requires accurate relative angles and distances between AGVs and pallets. They developed a deep learning model with multi- task learning to predict the angles and distances simultaneously. Further, the performance of the proposed multi-task model and single-task models in five deep learning architectures were compared. It is observed that the multi-task model outperformed the other models in the difficult tasks and maintained its performance in the easy tasks. correspondingly, the proposed model outperformed other models in the pallet recognition issue, in both tasks which required high performance.

In [29], Authors have proposed a method for automatic pallet detection which is based on a monocular vision system. They proposed a traditional approach which undertakes recognition of both the pallet front side and its two pockets into. a dataset was collected in a warehouse for training and evaluation. That dataset contained different pallets configurations images, which are either on racks or on the ground and three convolutional neural networks were compared with arbitrary orientation on a novel dataset which was acquired in a warehouse. Experimental results indicated that Faster R-CNN and SSD performance is better than YOLOv4.

In [30], Authors have a new pallet detection system which uses a Direction Weighted Overlapping (DWO) ratio and Adaptive Structure Feature (ASF) for forklifts to pick up a pallet is proposed, which uses a monocular vision system on the forklift. Combining both ratio methods for pallet detection, the proposed method removes most of the non-stationary (dynamic) background and increases the processing efficiency. An Ad boost scheme based on used in algorithm to detect pallets. It can detect the pallet in a dark environment. calculating the DWO ratio between the detected pallets and tracking records, it avoids erroneous candidates during object tracking by calculating the DWO ratio between the detected pallets and tracking records. many small/medium scale enterprises, startups can afford it. This system pallet detection is accurate and erroneous candidates are ignored in industrial environment. The proposed system accurately finds the pallet structure to recognize the pallet, and the proposed direction weighted overlapping ratio is used for the detection of moving objects. This work improves pallet detection and solves the problem and provides effective design. the pallet detection rate by 95 %. with this algorithm that is proposed in this system.

In [31], Authors have proposed a method of automatic pallet handling based on the line structured light sensor, this includes pallet identification, development of line structured light sensor, pallet localization and visual vehicle docking. two aspects are

included in this method. First, based on embedded image processing board design of the line structured light sensor that includes a FPGA and a DSP, they used light stripe center detection based on Hessian matrix decomposition in by embedded visual processing board. Besides, they also identified and localized the pallet by using the geometry structure of model match method, and for driving the vehicle used position based visual serving method to approach the pallet. This method is suitable for identifying and localizing less complicated scenarios.

In [32], Authors have proposed new collision-warning system for forklifts in this project to capture information about the environment. This system uses a time-of-flight camera. to predict impending collisions Computer-vision algorithms use this data, whereby collisions with humans and objects are distinguished. This collision warning system delivers 2D and 3D data and can overcome warnings in warehouse situations. objects are identified by clustering as well as information about the movement of objects in forklift’s path is found using 3D data. 2D data is used by Machine-learning algorithms used to detect people in the path. to Distinguishing people and non- human objects. to establish a two-level warning system. This system has already been evaluated and included academic settings.

TABLE I

Summary of Systems and Technologies Used in Forklift Based Applications

IV. ACKNOWLEDGEMENT

We are grateful to Atharva College of Engineering for giving us the opportunity to do major project work and to create a literature review under the Department of Electronics and Telecommunication Engineering. We feel privileged to express our deepest sense of gratitude and sincere thanks to our project guide Professor Prajakta Pawar for her continuous support and guidance throughout this literature review related project work.

Conclusion

In this paper, a literature review of thirty-two papers about forklift operations and new pallet detection was presented. The focus was on pallet recognition using forklift. Many systems have been developed for seamless forklift operations and pallet detection, and more technologies and systems are currently working on making upgrades to these systems with cutting edge technologies. With this literature study various systems using sensors, signals, algorithms, cameras, technologies, were understood. These systems were mostly developed for indoor and outdoor warehouse management, also known as smear warehousing.

References

[1] Bhuvaneswari Sarupuri; Gun A. Lee; Mark Billinghurst, “An Augmented Reality Guide for Assisting Forklift Operation, 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct),” 19-23 September 2016. [2] Andrea Motroni, Alice Buffi, Paolo Nepa, \" Forklift Tracking: Industry 4.0 Implementation in Large-Scale Warehouses through UWB Sensor Fusion,AdvancedSensors and Sensing Technologies for Indoor Localization\" Appl. Sci. 2021, 11(22), 10607. [3] Luyang Li; Yun-Hui Liu; Mu Fang; Zhizeng Zheng; Hengbo Tang, \" Vision-Based Intelligent Forklift Automatic Guided Vehicle (AGV),2015 IEEE International Conference on Automation Science and Engineering (CASE)\",08 October 2015. [4] Carla Viegas, Tiago Faria, Ana Cebola de Oliveira, Liliana Aranha Caetano, Elisabete Carolino, Anita Quintal- Gomes, Magdalena Twaru?ek, Robert Kosicki, Ewelina Soszczy?ska & Susana Viegas,\" A new approach to assess occupational exposure to airborne fungal contamination and mycotoxins of forklift drivers in waste sorting facilities, Mycotoxin Research volume 33, pages285–295\"(2017). [5] Gang Chen; Rui Peng; Zhicheng Wang; Weidong Zhao,\" Pallet recognition and localization method for vision guided forklift, 2012 8th International Conference on Wireless Communications, Networking and Mobile Computing”, September 2012. [6] Bo Rolander, Mikael Forsman, Bijar Ghafouri , Farhad Abtahi , Charlotte Wåhlin,\" Measurements and observations of movements at work for warehouse forklift truck operators ,INTERNATIONAL JOURNAL OF OCCUPATIONAL SAFETY AND ERGONOMICS (JOSE)\"2022, VOL. 28, NO. 3, 1840–1848. [7] Zhilu Zhang, Benxian Xiao,\" Sensor Fault Diagnosis and Fault Tolerant Control for Forklift Based on Sliding Mode Theory IEEE Access (Volume:8)\"84858 84866, April 2020. [8] Ryosuke Iinuma, Yusuke Kojima, Hiroyuki Onoyama, Takanori Fukao, Shingo Hattori, and Yasunori Nonogaki, \"Pallet Handling System with an Autonomous Forklift for Outdoor Fields, Journal of Robotics and Mechatronics Vol.32 No.5, \"2020 [9] Benno Gross, Michael Bretschneider-Hagems, Andreas Stefan, and Jörg Rissler, \" Monitors vs. Smart Glasses: A Study on Cognitive Workload of Digital Information Systems on Forklift Trucks Springer International, part of Springer Nature 2018, 569–578. [10] Armin Lang and Willibald A. Günthner, \"Evaluation of the Usage of Support Vector Machines for People Detection for a Collision Warning System on a Forklift\",\"Springer International Publishing AG 2017\"322–337, [11] Yuxiang Yang, Yaohui Pan, Xiaojing Zhu, Mingyu Gao, Jing Zhang, and Dacheng Tao, \" A human-like dual- forklift collaborative mechanism for container handling, IEEE Transactions on IndustrialElectronics\" 0278-0046 (c) 2020 IEEE [12] Ze Li, Yi Wei, Yupu Zhu, Lan Wang,\"Research on Forklift Detection of Forklift Blade Based on Machine Vision,2019 IEEE 3rd Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC 2019)\" [13] Zhang Zhiqiang, Tian Jinjin, Zhan Peng, \"On line monitoring method of forklift truck working condition based on multi sensor, IOP Publishing, Journal of Physics:Conference Series,\"1635 (2020) [14] Zhiheng Zhao, Ji Fang, George Q Huang, Mengdi Zhang, \"iBeacon Enabled Indoor Positioning for Warehouse Management,2016 4th International Symposium on Computational and Business Intelligence\" ©2016 IEEE [15] Valentín Barral, Pedro Suárez-Casal, Carlos J. Escudero and José A. García-Naya \"Multi-Sensor Accurate Forklift Location and Tracking Simulation in Industrial Indoor Environments, mdpi, Electronics 2019, 8, 1152\" [16] Alessandra Cantini, Filippo De Carlo and Mario Tucci, \"Towards Forklift Safety in a Warehouse: An Approach Based on the Automatic Analysis of Resource Flows. [17] Augie Widyotriatmo, Anugrah K. Pamosoajiand Keum- Shik Hong, \"Control Architecture of an Autonomous Material Handling Vehicle, International Journal of Artificial Intelligence, 2013, Volume 10, Number S13 Copyright © 2013 by IJAI (CESER Publications)\" [18] Muhammad Tayyab, Syed Noman Mahmood, Syed Ashar Alam, Saqib Akhter and Tahir Qadri, \"Autonomous SLAM based Forklift Robot,Indian Journal of Science and Technology,Vol,10(45),DOI: 10.17485/ December 2017\" [19] Ryosuke Iinuma, Yusuke Hori, Hiroyuki Onoyama,Yukihiro Kubo, Takanori Fukao4, \"Pallet Detection and Estimation for Fork Insertion with RGB-D Camera, Proceedings of 2021 IEEE International Conference on Mechatronics and Automation\" [20] Aashishkumar L Sharnangat, M. S. Tufail, \"Design of Wireless Operated (On RFID) Forklift in Warehouse, The International Journal of Engineering and Science (IJES) Volume 2, Issue 4, Pages 64-67 2013\" [21] Ryosuke Iinuma, Yusuke HoHiroyuki Onoyama, Yukihiro Kubo, and Takanori Fukao, \"Robotic Forklift for Stacking Multiple Pallets with RGB-D Cameras, Journal of Robotics and Mechatronics Vol.33 No.6, 2021\" [22] Benjamin Molter, Johannes Fottner, \"Real-time Pallet Localization with 3D Camera Technology for Forklifts in Logistic Environments,2018 IEEE Internat\"ional Conference on Service Operations and Logistics, and Informatics (SOLI)\" [23] Mohammad M. Aref, Reza Ghabcheloo, Antti Kolu & Jouni Mattila, \"Vision-Guided Autonomous Forklift ,RAAD 2016:Advances in Robot Design and Intelligent Control pp 338–346\" [24] Florentinus Budi Setiawan, Phoa Marcellino Siva, Leonardus Heru Pratomo, Slamet Riyadi, \"Design and Implementation of Smart Forklift for Automatic Guided Vehicle Using Raspberry Pi 4, Journal of Robotics and Control (JRC) Volume 2, Issue 6, November 2021\" [25] A.Z. Arfianto, R. Susanto, M.B. Rahmat,Hariyanto Soeroso,Faris Nofandi,Valian Yoga Pudya Ardhana,Ari Wibawa Budi Santosa,Ardiansyah, \"Unmanned Vehicle Using Received Signal Strength Indicator (RSSI) in Instant Beverage Industry,2019 International Conference on Advanced Mechatronics, Intelligent Manufacture and Industrial Automation (ICAMIMIA)\" [26] Ihab S. Mohamed, Alessio Capitanelli, Fulvio Mastrogiovanni, Stefano Rovetta & Renato Zaccaria,\"Detection, localization and tracking of pallets using machine learning techniques and 2D range data,Neural Computing and Applications volume 32, pages8811–8828 (2020)\" [27] Ça?da? Özgür,Cyril Alias ,Bernd Noch,\"Comparing sensor- based and camera-based approaches to recognizing the occupancystatus of the load handling device of forklift trucks,2016 LogisticsJournal: Proceedings – ISSN 2192-9084\" [28] Chunghyup Mok, Insung Baek, Yoon Sang Cho, Younghoon Kim, Seoung Bum Kim, \"Pallet Recognition with Multi- Task Learning for Automated Guided Vehicles mdpi journal Appl. Sci. 2021,11,11808.” [29] Michela Zaccaria,Riccardo Monica, Jacopo Aleott, \"A Comparison of Deep Learning Models for Pallet Detection in Industrial Warehouses,978-1-7281-9080-8/20/©2020 IEEE\" [30] Jia-Liang Syu, Hsin-Ting Li,Jen-Shiun Chiang, Chih-Hsien Hsia Po-Han, Chi-Fang Hsieh,Shih-An, \"A computer vision assisted system for autonomous forklift vehicles in real factory environment, Springer Science + Business Media New York 2016\" [31] Shijun Wang, Aixue Ye, Hao Guo, Jiaojiao Gu, Xiaonan Wang and Kui Yuan,\"Autonomous Pallet Localization and Picking for Industrial Forklifts Based on the Line Structured Light, Proceedings of 2016 IEEE International Conference on Mechatronics and Automation\" [32] Armin Lang, \"Evaluation of an Intelligent Collision Warning System for Forklift Truck Drivers in Industry,2018: Digital Human Modeling. Applications in Health, Safety, Ergonomics, and Risk Management pp 610–622

Copyright

Copyright © 2023 Prajakta Pawar, Rutuja Sande, Tejal Kondalkar, Sandesh Benere. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.