OCR

Tensorflow short overview

Optical Character Recognition

• computer vision & ML 기술을 사용해, 이미지에 포함된 글자를 인식
• text detection 과 text recognition 의 두 단계로 나눌 수 있음
• text detection: 글자를 bounding box 로 인식
• text recognition: bounding box 안의 글자 인식

OpenCV text detection/recognition example:

https://github.com/opencv/opencv/blob/master/samples/dnn/text_detection.cpp

OCR TFLite community project by community contributors:

https://github.com/tulasiram58827/ocr_tflite

OpenCV text detection:

https://www.pyimagesearch.com/2018/08/20/opencv-text-detection-east-text-detector/

• Why is natural scene text detection so challenging?
  : image/ sensor noise, viewing angles, blurring, lighting conditions, resolution, non-paper. objects(reflective), non-planar objects, unknown objects
  
• The EAST deep learning text detector
  : An Efficient and Accurate Scene Text Detector

Deep Learning based Text Detection Using OpenCV:

https://learnopencv.com/deep-learning-based-text-detection-using-opencv-c-python/

1. load the OpenCV model into memory

   • cv2.dnn.readNet()

2. prepare the input image

   • blobFromImage() : mean subtraction, scaling, optional channel swapping

3. forward pass the blob through the network

   • network output을 (1) Text-box geometry, (2) confidence score of the detected box, 2가지로 설정 가능

   outputLayers = []
    outputLayers.append("feature_fusion/Conv_7/Sigmoid")
    outputLayers.append("feature_fusion/concat_3")

   • 네트워크에 입력 이미지 전달

   net.setInput(blob)
    output = net.forward(outputLaoyers)
    scores = output[0]
    geometry = output[1]

4. process the output

   • layer output (geometry & score) 이 여러개 일 수 있기 때문에 그 중에서 text box에 해당하는 값을 선택해야 함

    [boxes, confidences] = decode(scores, geometry, confThreshold)

   • Non-Maxmum Supperession 으로 text box candidate 을 선택

   indices = cv.dnn.NMSBoxesRotated(boxes, confidences, confThreshold, nmsThreshold)